代码改变世界

Hadoop 2.6.0编译on mac

2015-05-28 12:35  追风的蓝宝  阅读(867)  评论(0)  编辑  收藏  举报

花了一个晚上的时间弄了下hadoop的编译环境，碰到些错误，这里保存下。

需要编译Hadoop,不但需要安装Maven，还需要安装protobuf

安装Maven

  • 下载：apache-maven-3.0.5-bin.tar.gz
  • 解压：tar -zxvf apache-maven-3.0.5-bin.tar.gz
  • 设置环境变量，打开/etc/profile文件，添加
    • export MAVEN_HOME=/opt/apache-maven-3.0.5
    • export PATH=$PATH:$MAVEN_HOME/bin
  • 执行命令使之生效：source /etc/profile或者./etc/profile
  • 验证：mvn -v 
安装protobuf
  • 解压：tar -zxvf protobuf-2.5.0.tar.gz 
  • 进入安装目录，进行配置，执行命令，./configure
  • 安装命令：make & make check & make install
  • 验证：protoc --version

编译Hadoop

  • 配置MAVEN国内镜像配置，在${MAVEN_HOME}/conf/settings.xml内设置。
1 <mirror>
2     <id>CN</id>
3     <name>OSChina Central</name>                                                                                                 
4     <url>http://maven.oschina.net/content/groups/public/</url>
5     <mirrorOf>central</mirrorOf>
6 </mirror>
  • 将Hadoop Project 导入到Eclipse
 1  
 2 
 3 Importing projects to eclipse
 4 
 5 When you import the project to eclipse, install hadoop-maven-plugins at first.
 6 
 7   $ cd hadoop-maven-plugins
 8 
 9   $ mvn install
10 
11 Then, generate eclipse project files.
12 
13   $ mvn eclipse:eclipse -DskipTests
14 
15 At last, import to eclipse by specifying the root directory of the project via
16 
17 [File] > [Import] > [Existing Projects into Workspace].
  • 出现问问题：
Exception in thread "main" java.lang.AssertionError: Missing tools.jar at: /Library/Java/JavaVirtualMachines/jdk1.7.0_60.jdk/Contents/Home/Classes/classes.jar. Expression: file.exists()
    at org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:395)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.assertFailed(ScriptBytecodeAdapter.java:683)
    at org.codehaus.mojo.jspc.CompilationMojoSupport.findToolsJar(CompilationMojoSupport.groovy:371)
    at org.codehaus.mojo.jspc.CompilationMojoSupport.this$4$findToolsJar(CompilationMojoSupport.groovy)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnCurrentN(ScriptBytecodeAdapter.java:78)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnCurrent0(ScriptBytecodeAdapter.java:112)
    at org.codehaus.mojo.jspc.CompilationMojoSupport.execute(CompilationMojoSupport.groovy:318)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.MojoExecutor.executeForkedExecutions(MojoExecutor.java:364)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:198)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:355)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:216)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:160)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

解决方法：http://www.tuicool.com/articles/7N7BruB

this is as the compiler needs a file named tools.jar but localted at classes.jar under dir Classes but found nothing.so u can do workaround:

cd `java_home`/
mkdir Classes
cd Classes
ln -s ../lib/tools.jar classes.jar

but u should know,in macbook,the command 'ln' is ONLY effect when u locate in the target dir,otherwise u will see:

cd `java_home`
mkdir Classes
ln -s lib/tools.jar Classes/classes.jar
# or:ln -s lib/tools.jar /absolute/path/to/Classes/classes.jar
cp Classes/classes.jar /tmp
Error:no such file or dir found