Mac OS X 10.10编译Hadoop 2.6.0笔记
原本通过brew 安装了Hadoop 2.6.0,但是使用过程中报一个错误(错误描述),查了资料说是因为native库的问题,看了一下,发现通过brew安装的版本根本就没有native目录,所以下载了源码自己编译。结果陆续出了不少问题,记录如下:
1.错误: 程序包com.sun.javadoc不存在
这个错误尝试了很多方法,包括在pom.xml中指定tools.jar文件等等
最后通过’export JAVA_HOME=/usr/libexec/java_home -v 1.6’命令切换JDK版本为1.6(OS X上安装了最新的1.8.x版本)才解决
2.[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.6.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
查了资料说是缺少protobuf库,通过brew安装之后,结果编译时提示需要的是2.5.0版本,但是brew安装的是2.6.1版,所以只好卸载了,然后手动下载2.5.0版(https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz),然后自行编译安装
$ sudo ./configure
$ sudo make
$ sudo make check
$ sudo make install
$ sudo ldconfig
$ protoc --version
3.[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "/usr/local/Cellar/hadoop/hadoop-2.6.0-src/hadoop-common-project/hadoop-common/target/native"): error=2, No such file or directory
通过brew install cmake安装cmake
4.[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 1
再往前翻,看到
main:
[exec] -- Configuring incomplete, errors occurred!
[exec] See also "/usr/local/Cellar/hadoop/hadoop-2.6.0-src/hadoop-common-project/hadoop-comCMake Error at /usr/local/Cellar/cmake/3.1.0/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
[exec] Could NOT mon/target/native/CMakeFiles/CMakeOutput.log".
[exec] find JNI (missing: JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY
[exec] JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH)
[exec] Call Stack (most recent call first):
[exec] /usr/local/Cellar/cmake/3.1.0/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:374 (_FPHSA_FAILURE_MESSAGE)
[exec] /usr/local/Cellar/cmake/3.1.0/share/cmake/Modules/FindJNI.cmake:286 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
[exec] JNIFlags.cmake:123 (find_package)
[exec] CMakeLists.txt:24 (include)
查了之后发现主要是因为cmake没有找到jni.h文件,再一看,在$JAVA_HOME目录下根本没有include目录,这个mac自带的jdk1.6好像有问题,于是下载了JDK 1.7,并重新export JAVA_HOME
5.提示tools.jar错误
在$JAVA_HOME下mkdir classes, 然后ln -s ../lib/tools.jar classes.jar