编译hadoop
uos@uos-PC:~/wss/hadoop-rel-release-2.8.5$ cat BUILDING.txt
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | Requirements: * Windows System * JDK 1.7+ * Maven 3.0 or later * Findbugs 1.3.9 ( if running findbugs) * ProtocolBuffer 2.5.0 * CMake 2.6 or newer * Windows SDK 7.1 or Visual Studio 2010 Professional * Windows SDK 8.1 ( if building CPU rate control for the container executor) * zlib headers ( if building native code bindings for zlib) * Internet connection for first build (to fetch all Maven and Hadoop dependencies) * Unix command -line tools from GnuWin32: sh, mkdir , rm , cp , tar , gzip . These tools must be present on your PATH. * Python ( for generation of docs using 'mvn site' ) |
sudo apt install findbugs
sudo apt install openjdk-8-jdk maven
mvn -v
1 2 3 4 5 6 | uos@uos-PC:~ /wss/hadoop-rel-release-2 .8.5$ mvn - v Apache Maven 3.6.0 Maven home: /usr/share/maven Java version: 1.8.0, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-8-openjdk-mips64el/jre Default locale: zh_CN, platform encoding: UTF-8 OS name: "linux" , version: "4.19.0-loongson-3-server" , arch: "mips64el" , family: "unix" |
uos@uos-PC:~/wss/hadoop-rel-release-2.8.5$ java -version
1 2 3 | openjdk version "1.8.0" OpenJDK Runtime Environment (Loongson 8.1.3-mips64r2-uos) (build 1.8.0-1.8.0.212-2deepin-b00) OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode) |
sudo vi /etc/profile
1 2 3 | export JAVA_HOME= /usr/lib/jvm/java-8-openjdk-mips64el export CLASSPATH=.:$JAVA_HOME /lib/dt .jar:$JAVA_HOME /lib/tools .jar export PATH=$JAVA_HOME /bin :$PATH |
source /etc/profile
编译:
mvn clean package -DskipTests -Pdist,native
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time : 07:52 min [INFO] Finished at: 2022-12-20T18:05:41+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.8.5:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http: //cwiki .apache.org /confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hadoop-common |
uos@uos-PC:~/wss/hadoop-rel-release-2.8.5$ cat BUILDING.txt
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | Build instructions for Hadoop ---------------------------------------------------------------------------------- Requirements: * Unix System * JDK 1.7+ * Maven 3.0 or later * Findbugs 1.3.9 ( if running findbugs) * ProtocolBuffer 2.5.0 * CMake 2.6 or newer ( if compiling native code), must be 3.0 or newer on Mac * Zlib devel ( if compiling native code) * openssl devel ( if compiling native hadoop-pipes and to get the best HDFS encryption performance) * Linux FUSE (Filesystem in Userspace) version 2.6 or above ( if compiling fuse_dfs) * Internet connection for first build (to fetch all Maven and Hadoop dependencies) * python ( for releasedocs) |
只有configure.ac,没有configure 运行autoconf命令来生成configure,但是报以下错误:
1 2 3 4 5 6 7 8 | uos@uos-PC:~ /wss/protobuf/protobuf-2 .5.0$ autoconf configure.ac:17: error: possibly undefined macro: AM_MAINTAINER_MODE If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. configure.ac:32: error: possibly undefined macro: AM_INIT_AUTOMAKE configure.ac:49: error: possibly undefined macro: AM_CONDITIONAL configure.ac:75: error: possibly undefined macro: AC_PROG_LIBTOOL configure.ac:140: error: possibly undefined macro: AC_CXX_STL_HASH |
运行这个命令解决以上的错误:autoreconf --install
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | src /Makefile .am:314: warning: source file 'google/protobuf/compiler/cpp/cpp_plugin_unittest.cc' is in a subdirectory, src /Makefile .am:314: but option 'subdir-objects' is disabled src /Makefile .am:314: warning: source file 'google/protobuf/compiler/java/java_plugin_unittest.cc' is in a subdirectory, src /Makefile .am:314: but option 'subdir-objects' is disabled src /Makefile .am:314: warning: source file 'google/protobuf/compiler/java/java_doc_comment_unittest.cc' is in a subdirectory, src /Makefile .am:314: but option 'subdir-objects' is disabled src /Makefile .am:314: warning: source file 'google/protobuf/compiler/python/python_plugin_unittest.cc' is in a subdirectory, src /Makefile .am:314: but option 'subdir-objects' is disabled src /Makefile .am:206: warning: source file 'google/protobuf/compiler/main.cc' is in a subdirectory, src /Makefile .am:206: but option 'subdir-objects' is disabled src /Makefile .am:380: warning: source file 'google/protobuf/compiler/mock_code_generator.cc' is in a subdirectory, src /Makefile .am:380: but option 'subdir-objects' is disabled src /Makefile .am:380: warning: source file 'google/protobuf/testing/file.cc' is in a subdirectory, src /Makefile .am:380: but option 'subdir-objects' is disabled src /Makefile .am:380: warning: source file 'google/protobuf/compiler/test_plugin.cc' is in a subdirectory, src /Makefile .am:380: but option 'subdir-objects' is disabled src /Makefile .am:391: warning: source file 'google/protobuf/testing/zcgunzip.cc' is in a subdirectory, src /Makefile .am:391: but option 'subdir-objects' is disabled src /Makefile .am:388: warning: source file 'google/protobuf/testing/zcgzip.cc' is in a subdirectory, src /Makefile .am:388: but option 'subdir-objects' is disabled src /Makefile .am: installing './depcomp' parallel-tests: installing './test-driver' |
./configure
1 2 3 4 5 6 7 8 9 10 | checking for RCC... no checking for xlC_r... no checking for xlC... no checking whether we are using the GNU C++ compiler... no checking whether g++ accepts -g... no checking dependency style of g++... none checking how to run the C++ preprocessor... /lib/cpp configure: error: in ` /home/uos/wss/protobuf/protobuf-2 .5.0': configure: error: C++ preprocessor "/lib/cpp" fails sanity check See `config.log' for more details |
apt-get install build-essential
./configure
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | checking for the pthreads library -llthread... no checking whether pthreads work with -pthread... yes checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE checking if more special flags are required for pthreads... no checking whether to check for GCC pthread /shared inconsistencies... yes checking whether -pthread is sufficient with -shared... yes checking whether what we have so far is sufficient with -nostdlib... no checking whether -lpthread saves the day... yes checking the location of hash_map... <unordered_map> checking that generated files are newer than configure... done configure: creating . /config .status config.status: creating Makefile config.status: creating src /Makefile config.status: creating protobuf.pc config.status: creating protobuf-lite.pc config.status: creating config.h config.status: executing depfiles commands config.status: executing libtool commands |
make
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | Making all in src make [2]: 进入目录“ /home/uos/wss/protobuf/protobuf-2 .5.0 /src ” /bin/bash .. /libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite- strings -Woverloaded-virtual -Wno-sign-compare -O2 -g -DNDEBUG -MT common.lo -MD -MP -MF .deps /common .Tpo -c -o common.lo ` test -f 'google/protobuf/stubs/common.cc' || echo './' `google /protobuf/stubs/common .cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -pthread -Wall -Wwrite- strings -Woverloaded-virtual -Wno-sign-compare -O2 -g -DNDEBUG -MT common.lo -MD -MP -MF .deps /common .Tpo -c google /protobuf/stubs/common .cc -fPIC -DPIC -o .libs /common .o In file included from google /protobuf/stubs/common .cc:34: . /google/protobuf/stubs/once .h: In function ‘void google::protobuf::GoogleOnceInit(google::protobuf::ProtobufOnceType*, void (*)())’: . /google/protobuf/stubs/once .h:125:30: error: cannot convert ‘google::protobuf::ProtobufOnceType*’ {aka ‘long int*’} to ‘const volatile Atomic32*’ {aka ‘const volatile int*’} if (internal::Acquire_Load(once) != ONCE_STATE_DONE) { ^~~~ In file included from . /google/protobuf/stubs/atomicops .h:184, from . /google/protobuf/stubs/once .h:81, from google /protobuf/stubs/common .cc:34: . /google/protobuf/stubs/atomicops_internals_mips_gcc .h:170:55: note: initializing argument 1 of ‘google::protobuf::internal::Atomic32 google::protobuf::internal::Acquire_Load(const volatile Atomic32*)’ inline Atomic32 Acquire_Load(volatile const Atomic32* ptr) { ~~~~~~~~~~~~~~~~~~~~~~~~~^~~ In file included from google /protobuf/stubs/common .cc:34: . /google/protobuf/stubs/once .h: In function ‘void google::protobuf::GoogleOnceInit(google::protobuf::ProtobufOnceType*, void (*)(Arg*), Arg*)’: . /google/protobuf/stubs/once .h:134:30: error: cannot convert ‘google::protobuf::ProtobufOnceType*’ {aka ‘long int*’} to ‘const volatile Atomic32*’ {aka ‘const volatile int*’} if (internal::Acquire_Load(once) != ONCE_STATE_DONE) { ^~~~ In file included from . /google/protobuf/stubs/atomicops .h:184, from . /google/protobuf/stubs/once .h:81, from google /protobuf/stubs/common .cc:34: . /google/protobuf/stubs/atomicops_internals_mips_gcc .h:170:55: note: initializing argument 1 of ‘google::protobuf::internal::Atomic32 google::protobuf::internal::Acquire_Load(const volatile Atomic32*)’ inline Atomic32 Acquire_Load(volatile const Atomic32* ptr) { ~~~~~~~~~~~~~~~~~~~~~~~~~^~~ make [2]: *** [Makefile:1619:common.lo] 错误 1 make [2]: 离开目录“ /home/uos/wss/protobuf/protobuf-2 .5.0 /src ” make [1]: *** [Makefile:669:all-recursive] 错误 1 make [1]: 离开目录“ /home/uos/wss/protobuf/protobuf-2 .5.0” make : *** [Makefile:576:all] 错误 2 |
为int指定了一个别名:ProtobufOnceType* ; 为int 指定别名Atomic32*
1 2 3 4 5 6 7 8 | uos@uos-PC:~ /wss/protobuf/protobuf-2 .5.0$ grep -r "ProtobufOnceType" src /google/protobuf/stubs/once .cc:void GoogleOnceInitImpl(ProtobufOnceType* once, Closure* closure) { src /google/protobuf/stubs/once .h: // * A type called ProtobufOnceType. src /google/protobuf/stubs/once .h: // ProtobufOnceType. This is the only legal way to declare such a variable. src /google/protobuf/stubs/once .h: // * A function GoogleOnceInit(ProtobufOnceType* once, void (*init_func)()). src /google/protobuf/stubs/once .h: // This function , when invoked multiple times given the same ProtobufOnceType src /google/protobuf/stubs/once .h: // * This implementation guarantees that ProtobufOnceType is a POD (i.e. no src /google/protobuf/stubs/once .h:typedef bool ProtobufOnceType; |
typedef bool ProtobufOnceType;
这行代码定义了ProtobufOnceType的类型为bool型。
vim src/google/protobuf/stubs/common.cc
vim src/google/protobuf/stubs/once.h
vim src/google/protobuf/stubs/atomicops.h :不支持MIPS64位的,添加宏定义,使其支持MIPS64。
vim src/google/protobuf/stubs/platform_macros.h
uos@uos-PC:~/wss/protobuf/protobuf-2.5.0$ vim m4/acx_check_suncc.m4
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· AI与.NET技术实操系列:基于图像分类模型对图像进行分类
· go语言实现终端里的倒计时
· 如何编写易于单元测试的代码
· 10年+ .NET Coder 心语,封装的思维:从隐藏、稳定开始理解其本质意义
· .NET Core 中如何实现缓存的预热?
· 分享一个免费、快速、无限量使用的满血 DeepSeek R1 模型,支持深度思考和联网搜索!
· 25岁的心里话
· 基于 Docker 搭建 FRP 内网穿透开源项目(很简单哒)
· ollama系列01:轻松3步本地部署deepseek,普通电脑可用
· 按钮权限的设计及实现