Hadoop学习之路（三）Hadoop-2.7.5在CentOS-6.7上的编译
下载Hadoop源码
1、登录官网
2、确定你要安装的软件的版本
一个选取原则： 不新不旧的稳定版本
几个标准：
1）一般来说，刚刚发布的大版本都是有很多问题
2）应该选择某个大版本中的最后一个小版本
阅读编译文档
1、准备一个hadoop源码包，我选择的hadoop的版本是：hadoop-2.7.5-src.tar.gz，在hadoop-2.7.5-src.tar.gz的源码包根目录下有一个文档叫做BUINDING.txt，这其中说明了编译hadoop所需要的一些编译环境相关的东西。不同的hadoop版本的要求都不一样。对应的版本参照BUINDING.txt。
Requirements: * Unix System * JDK 1.7+ * Maven 3.0 or later * Findbugs 1.3.9 (if running findbugs) * ProtocolBuffer 2.5.0 * CMake 2.6 or newer (if compiling native code), must be 3.0 or newer on Mac * Zlib devel (if compiling native code) * openssl devel ( if compiling native hadoop-pipes and to get the best HDFS encryption performance ) * Linux FUSE (Filesystem in Userspace) version 2.6 or above ( if compiling fuse_dfs ) * Internet connection for first build (to fetch all Maven and Hadoop dependencies)
软件安装
对应以上需求，我们准备好所要求版本的这些软件。
JDK的安装
选择版本：jdk1.8.0_73
安装依赖包
根据编译指导文件BUILDING.txt，安装相关依赖程序包
[root@master soft]# yum -y install gcc-c++ build-essential autoconf automake libtool cmake zlib1g-dev pkg-config libssl-devua svn openssl-devel ncurses-devel
1 [root@master soft]# yum -y install gcc-c++ build-essential autoconf automake libtool cmake zlib1g-dev pkg-config libssl-devua svn openssl-devel ncurses-devel 2 已加载插件：fastestmirror, refresh-packagekit, security 3 设置安装进程 4 Loading mirror speeds from cached hostfile 5 * base: mirrors.163.com 6 * extras: mirrors.163.com 7 * updates: mirrors.cqu.edu.cn 8 base | 3.7 kB 00:00 9 extras | 3.4 kB 00:00 10 updates | 3.4 kB 00:00 11 No package build-essential available. 12 No package zlib1g-dev available. 13 No package pkg-config available. 14 No package libssl-devua available. 15 解决依赖关系 16 --> 执行事务检查 17 ---> Package autoconf.noarch 0:2.63-5.1.el6 will be 安装 18 ---> Package automake.noarch 0:1.11.1-4.el6 will be 安装 19 ---> Package cmake.x86_64 0:2.8.12.2-4.el6 will be 安装 20 ---> Package gcc-c++.x86_64 0:4.4.7-18.el6 will be 安装 21 --> 处理依赖关系 libstdc++-devel = 4.4.7-18.el6，它被软件包 gcc-c++-4.4.7-18.el6.x86_64 需要 22 --> 处理依赖关系 libstdc++ = 4.4.7-18.el6，它被软件包 gcc-c++-4.4.7-18.el6.x86_64 需要 23 --> 处理依赖关系 gcc = 4.4.7-18.el6，它被软件包 gcc-c++-4.4.7-18.el6.x86_64 需要 24 --> 处理依赖关系 libmpfr.so.1()(64bit)，它被软件包 gcc-c++-4.4.7-18.el6.x86_64 需要 25 ---> Package libtool.x86_64 0:2.2.6-15.5.el6 will be 安装 26 ---> Package ncurses-devel.x86_64 0:5.7-4.20090207.el6 will be 安装 27 ---> Package openssl-devel.x86_64 0:1.0.1e-57.el6 will be 安装 28 --> 处理依赖关系 openssl = 1.0.1e-57.el6，它被软件包 openssl-devel-1.0.1e-57.el6.x86_64 需要 29 --> 处理依赖关系 zlib-devel，它被软件包 openssl-devel-1.0.1e-57.el6.x86_64 需要 30 --> 处理依赖关系 krb5-devel，它被软件包 openssl-devel-1.0.1e-57.el6.x86_64 需要 31 ---> Package subversion.x86_64 0:1.6.11-15.el6_7 will be 安装 32 --> 处理依赖关系 perl(URI) >= 1.17，它被软件包 subversion-1.6.11-15.el6_7.x86_64 需要 33 --> 执行事务检查 34 ---> Package gcc.x86_64 0:4.4.7-18.el6 will be 安装 35 --> 处理依赖关系 libgomp = 4.4.7-18.el6，它被软件包 gcc-4.4.7-18.el6.x86_64 需要 36 --> 处理依赖关系 cpp = 4.4.7-18.el6，它被软件包 gcc-4.4.7-18.el6.x86_64 需要 37 --> 处理依赖关系 libgcc >= 4.4.7-18.el6，它被软件包 gcc-4.4.7-18.el6.x86_64 需要 38 --> 处理依赖关系 cloog-ppl >= 0.15，它被软件包 gcc-4.4.7-18.el6.x86_64 需要 39 ---> Package krb5-devel.x86_64 0:1.10.3-65.el6 will be 安装 40 --> 处理依赖关系 libkadm5(x86-64) = 1.10.3-65.el6，它被软件包 krb5-devel-1.10.3-65.el6.x86_64 需要 41 --> 处理依赖关系 krb5-libs = 1.10.3-65.el6，它被软件包 krb5-devel-1.10.3-65.el6.x86_64 需要 42 --> 处理依赖关系 libselinux-devel，它被软件包 krb5-devel-1.10.3-65.el6.x86_64 需要 43 --> 处理依赖关系 libcom_err-devel，它被软件包 krb5-devel-1.10.3-65.el6.x86_64 需要 44 --> 处理依赖关系 keyutils-libs-devel，它被软件包 krb5-devel-1.10.3-65.el6.x86_64 需要 45 ---> Package libstdc++.x86_64 0:4.4.7-16.el6 will be 升级 46 ---> Package libstdc++.x86_64 0:4.4.7-18.el6 will be an update 47 ---> Package libstdc++-devel.x86_64 0:4.4.7-18.el6 will be 安装 48 ---> Package mpfr.x86_64 0:2.4.1-6.el6 will be 安装 49 ---> Package openssl.x86_64 0:1.0.1e-42.el6 will be 升级 50 ---> Package openssl.x86_64 0:1.0.1e-57.el6 will be an update 51 ---> Package perl-URI.noarch 0:1.40-2.el6 will be 安装 52 ---> Package zlib-devel.x86_64 0:1.2.3-29.el6 will be 安装 53 --> 执行事务检查 54 ---> Package cloog-ppl.x86_64 0:0.15.7-1.2.el6 will be 安装 55 --> 处理依赖关系 libppl_c.so.2()(64bit)，它被软件包 cloog-ppl-0.15.7-1.2.el6.x86_64 需要 56 --> 处理依赖关系 libppl.so.7()(64bit)，它被软件包 cloog-ppl-0.15.7-1.2.el6.x86_64 需要 57 ---> Package cpp.x86_64 0:4.4.7-18.el6 will be 安装 58 ---> Package keyutils-libs-devel.x86_64 0:1.4-5.el6 will be 安装 59 ---> Package krb5-libs.x86_64 0:1.10.3-42.el6 will be 升级 60 --> 处理依赖关系 krb5-libs = 1.10.3-42.el6，它被软件包 krb5-workstation-1.10.3-42.el6.x86_64 需要 61 ---> Package krb5-libs.x86_64 0:1.10.3-65.el6 will be an update 62 ---> Package libcom_err-devel.x86_64 0:1.41.12-23.el6 will be 安装 63 --> 处理依赖关系 libcom_err = 1.41.12-23.el6，它被软件包 libcom_err-devel-1.41.12-23.el6.x86_64 需要 64 ---> Package libgcc.x86_64 0:4.4.7-16.el6 will be 升级 65 ---> Package libgcc.x86_64 0:4.4.7-18.el6 will be an update 66 ---> Package libgomp.x86_64 0:4.4.7-16.el6 will be 升级 67 ---> Package libgomp.x86_64 0:4.4.7-18.el6 will be an update 68 ---> Package libkadm5.x86_64 0:1.10.3-65.el6 will be 安装 69 ---> Package libselinux-devel.x86_64 0:2.0.94-7.el6 will be 安装 70 --> 处理依赖关系 libselinux = 2.0.94-7.el6，它被软件包 libselinux-devel-2.0.94-7.el6.x86_64 需要 71 --> 处理依赖关系 libsepol-devel >= 2.0.32-1，它被软件包 libselinux-devel-2.0.94-7.el6.x86_64 需要 72 --> 处理依赖关系 pkgconfig(libsepol)，它被软件包 libselinux-devel-2.0.94-7.el6.x86_64 需要 73 --> 执行事务检查 74 ---> Package krb5-workstation.x86_64 0:1.10.3-42.el6 will be 升级 75 ---> Package krb5-workstation.x86_64 0:1.10.3-65.el6 will be an update 76 ---> Package libcom_err.x86_64 0:1.41.12-22.el6 will be 升级 77 --> 处理依赖关系 libcom_err = 1.41.12-22.el6，它被软件包 libss-1.41.12-22.el6.x86_64 需要 78 --> 处理依赖关系 libcom_err = 1.41.12-22.el6，它被软件包 e2fsprogs-libs-1.41.12-22.el6.x86_64 需要 79 --> 处理依赖关系 libcom_err = 1.41.12-22.el6，它被软件包 e2fsprogs-1.41.12-22.el6.x86_64 需要 80 ---> Package libcom_err.x86_64 0:1.41.12-23.el6 will be an update 81 ---> Package libselinux.x86_64 0:2.0.94-5.8.el6 will be 升级 82 --> 处理依赖关系 libselinux = 2.0.94-5.8.el6，它被软件包 libselinux-utils-2.0.94-5.8.el6.x86_64 需要 83 --> 处理依赖关系 libselinux = 2.0.94-5.8.el6，它被软件包 libselinux-python-2.0.94-5.8.el6.x86_64 需要 84 ---> Package libselinux.x86_64 0:2.0.94-7.el6 will be an update 85 ---> Package libsepol-devel.x86_64 0:2.0.41-4.el6 will be 安装 86 ---> Package ppl.x86_64 0:0.10.2-11.el6 will be 安装 87 --> 执行事务检查 88 ---> Package e2fsprogs.x86_64 0:1.41.12-22.el6 will be 升级 89 ---> Package e2fsprogs.x86_64 0:1.41.12-23.el6 will be an update 90 ---> Package e2fsprogs-libs.x86_64 0:1.41.12-22.el6 will be 升级 91 ---> Package e2fsprogs-libs.x86_64 0:1.41.12-23.el6 will be an update 92 ---> Package libselinux-python.x86_64 0:2.0.94-5.8.el6 will be 升级 93 ---> Package libselinux-python.x86_64 0:2.0.94-7.el6 will be an update 94 ---> Package libselinux-utils.x86_64 0:2.0.94-5.8.el6 will be 升级 95 ---> Package libselinux-utils.x86_64 0:2.0.94-7.el6 will be an update 96 ---> Package libss.x86_64 0:1.41.12-22.el6 will be 升级 97 ---> Package libss.x86_64 0:1.41.12-23.el6 will be an update 98 --> 完成依赖关系计算 99 100 依赖关系解决 101 102 ================================================================================================================ 103 软件包 架构 版本 仓库 大小 104 ================================================================================================================ 105 正在安装: 106 autoconf noarch 2.63-5.1.el6 base 781 k 107 automake noarch 1.11.1-4.el6 base 550 k 108 cmake x86_64 2.8.12.2-4.el6 base 8.0 M 109 gcc-c++ x86_64 4.4.7-18.el6 base 4.7 M 110 libtool x86_64 2.2.6-15.5.el6 base 564 k 111 ncurses-devel x86_64 5.7-4.20090207.el6 base 641 k 112 openssl-devel x86_64 1.0.1e-57.el6 base 1.2 M 113 subversion x86_64 1.6.11-15.el6_7 base 2.3 M 114 为依赖而安装: 115 cloog-ppl x86_64 0.15.7-1.2.el6 base 93 k 116 cpp x86_64 4.4.7-18.el6 base 3.7 M 117 gcc x86_64 4.4.7-18.el6 base 10 M 118 keyutils-libs-devel x86_64 1.4-5.el6 base 29 k 119 krb5-devel x86_64 1.10.3-65.el6 base 504 k 120 libcom_err-devel x86_64 1.41.12-23.el6 base 33 k 121 libkadm5 x86_64 1.10.3-65.el6 base 143 k 122 libselinux-devel x86_64 2.0.94-7.el6 base 137 k 123 libsepol-devel x86_64 2.0.41-4.el6 base 64 k 124 libstdc++-devel x86_64 4.4.7-18.el6 base 1.6 M 125 mpfr x86_64 2.4.1-6.el6 base 157 k 126 perl-URI noarch 1.40-2.el6 base 117 k 127 ppl x86_64 0.10.2-11.el6 base 1.3 M 128 zlib-devel x86_64 1.2.3-29.el6 base 44 k 129 为依赖而更新: 130 e2fsprogs x86_64 1.41.12-23.el6 base 554 k 131 e2fsprogs-libs x86_64 1.41.12-23.el6 base 121 k 132 krb5-libs x86_64 1.10.3-65.el6 base 675 k 133 krb5-workstation x86_64 1.10.3-65.el6 base 814 k 134 libcom_err x86_64 1.41.12-23.el6 base 38 k 135 libgcc x86_64 4.4.7-18.el6 base 103 k 136 libgomp x86_64 4.4.7-18.el6 base 134 k 137 libselinux x86_64 2.0.94-7.el6 base 109 k 138 libselinux-python x86_64 2.0.94-7.el6 base 203 k 139 libselinux-utils x86_64 2.0.94-7.el6 base 82 k 140 libss x86_64 1.41.12-23.el6 base 42 k 141 libstdc++ x86_64 4.4.7-18.el6 base 295 k 142 openssl x86_64 1.0.1e-57.el6 base 1.5 M 143 144 事务概要 145 ================================================================================================================ 146 Install 22 Package(s) 147 Upgrade 13 Package(s) 148 149 总下载量：41 M 150 下载软件包： 151 (1/35): autoconf-2.63-5.1.el6.noarch.rpm | 781 kB 00:00 152 (2/35): automake-1.11.1-4.el6.noarch.rpm | 550 kB 00:00 153 (3/35): cloog-ppl-0.15.7-1.2.el6.x86_64.rpm | 93 kB 00:00 154 (4/35): cmake-2.8.12.2-4.el6.x86_64.rpm | 8.0 MB 00:00 155 (5/35): cpp-4.4.7-18.el6.x86_64.rpm | 3.7 MB 00:00 156 (6/35): e2fsprogs-1.41.12-23.el6.x86_64.rpm | 554 kB 00:00 157 (7/35): e2fsprogs-libs-1.41.12-23.el6.x86_64.rpm | 121 kB 00:00 158 (8/35): gcc-4.4.7-18.el6.x86_64.rpm | 10 MB 00:01 159 (9/35): gcc-c++-4.4.7-18.el6.x86_64.rpm | 4.7 MB 00:00 160 (10/35): keyutils-libs-devel-1.4-5.el6.x86_64.rpm | 29 kB 00:00 161 (11/35): krb5-devel-1.10.3-65.el6.x86_64.rpm | 504 kB 00:00 162 (12/35): krb5-libs-1.10.3-65.el6.x86_64.rpm | 675 kB 00:00 163 (13/35): krb5-workstation-1.10.3-65.el6.x86_64.rpm | 814 kB 00:00 164 (14/35): libcom_err-1.41.12-23.el6.x86_64.rpm | 38 kB 00:00 165 (15/35): libcom_err-devel-1.41.12-23.el6.x86_64.rpm | 33 kB 00:00 166 (16/35): libgcc-4.4.7-18.el6.x86_64.rpm | 103 kB 00:00 167 (17/35): libgomp-4.4.7-18.el6.x86_64.rpm | 134 kB 00:00 168 (18/35): libkadm5-1.10.3-65.el6.x86_64.rpm | 143 kB 00:00 169 (19/35): libselinux-2.0.94-7.el6.x86_64.rpm | 109 kB 00:00 170 (20/35): libselinux-devel-2.0.94-7.el6.x86_64.rpm | 137 kB 00:00 171 (21/35): libselinux-python-2.0.94-7.el6.x86_64.rpm | 203 kB 00:00 172 (22/35): libselinux-utils-2.0.94-7.el6.x86_64.rpm | 82 kB 00:00 173 (23/35): libsepol-devel-2.0.41-4.el6.x86_64.rpm | 64 kB 00:00 174 (24/35): libss-1.41.12-23.el6.x86_64.rpm | 42 kB 00:00 175 (25/35): libstdc++-4.4.7-18.el6.x86_64.rpm | 295 kB 00:00 176 (26/35): libstdc++-devel-4.4.7-18.el6.x86_64.rpm | 1.6 MB 00:01 177 (27/35): libtool-2.2.6-15.5.el6.x86_64.rpm | 564 kB 00:00 178 (28/35): mpfr-2.4.1-6.el6.x86_64.rpm | 157 kB 00:00 179 (29/35): ncurses-devel-5.7-4.20090207.el6.x86_64.rpm | 641 kB 00:00 180 (30/35): openssl-1.0.1e-57.el6.x86_64.rpm | 1.5 MB 00:00 181 (31/35): openssl-devel-1.0.1e-57.el6.x86_64.rpm | 1.2 MB 00:00 182 (32/35): perl-URI-1.40-2.el6.noarch.rpm | 117 kB 00:00 183 (33/35): ppl-0.10.2-11.el6.x86_64.rpm | 1.3 MB 00:00 184 (34/35): subversion-1.6.11-15.el6_7.x86_64.rpm | 2.3 MB 00:00 185 (35/35): zlib-devel-1.2.3-29.el6.x86_64.rpm | 44 kB 00:00 186 ---------------------------------------------------------------------------------------------------------------- 187 总计 4.1 MB/s | 41 MB 00:09 188 warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY 189 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 190 Importing GPG key 0xC105B9DE: 191 Userid : CentOS-6 Key (CentOS 6 Official Signing Key) <centos-6-key@centos.org> 192 Package: centos-release-6-7.el6.centos.12.3.x86_64 (@anaconda-CentOS-201508042137.x86_64/6.7) 193 From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 194 运行 rpm_check_debug 195 执行事务测试 196 事务测试成功 197 执行事务 198 正在升级 : libcom_err-1.41.12-23.el6.x86_64 1/48 199 正在升级 : libselinux-2.0.94-7.el6.x86_64 2/48 200 正在升级 : krb5-libs-1.10.3-65.el6.x86_64 3/48 201 正在升级 : libgcc-4.4.7-18.el6.x86_64 4/48 202 正在升级 : libstdc++-4.4.7-18.el6.x86_64 5/48 203 正在安装 : libkadm5-1.10.3-65.el6.x86_64 6/48 204 正在升级 : libss-1.41.12-23.el6.x86_64 7/48 205 正在安装 : autoconf-2.63-5.1.el6.noarch 8/48 206 正在安装 : mpfr-2.4.1-6.el6.x86_64 9/48 207 正在安装 : cpp-4.4.7-18.el6.x86_64 10/48 208 正在安装 : automake-1.11.1-4.el6.noarch 11/48 209 正在安装 : libstdc++-devel-4.4.7-18.el6.x86_64 12/48 210 正在安装 : ppl-0.10.2-11.el6.x86_64 13/48 211 正在安装 : cloog-ppl-0.15.7-1.2.el6.x86_64 14/48 212 正在升级 : openssl-1.0.1e-57.el6.x86_64 15/48 213 正在升级 : e2fsprogs-libs-1.41.12-23.el6.x86_64 16/48 214 正在安装 : libcom_err-devel-1.41.12-23.el6.x86_64 17/48 215 正在升级 : libgomp-4.4.7-18.el6.x86_64 18/48 216 正在安装 : gcc-4.4.7-18.el6.x86_64 19/48 217 正在安装 : libsepol-devel-2.0.41-4.el6.x86_64 20/48 218 正在安装 : libselinux-devel-2.0.94-7.el6.x86_64 21/48 219 正在安装 : perl-URI-1.40-2.el6.noarch 22/48 220 正在安装 : zlib-devel-1.2.3-29.el6.x86_64 23/48 221 正在安装 : keyutils-libs-devel-1.4-5.el6.x86_64 24/48 222 正在安装 : krb5-devel-1.10.3-65.el6.x86_64 25/48 223 正在安装 : openssl-devel-1.0.1e-57.el6.x86_64 26/48 224 正在安装 : subversion-1.6.11-15.el6_7.x86_64 27/48 225 正在安装 : gcc-c++-4.4.7-18.el6.x86_64 28/48 226 正在安装 : libtool-2.2.6-15.5.el6.x86_64 29/48 227 正在升级 : e2fsprogs-1.41.12-23.el6.x86_64 30/48 228 正在升级 : krb5-workstation-1.10.3-65.el6.x86_64 31/48 229 正在安装 : cmake-2.8.12.2-4.el6.x86_64 32/48 230 正在升级 : libselinux-python-2.0.94-7.el6.x86_64 33/48 231 正在升级 : libselinux-utils-2.0.94-7.el6.x86_64 34/48 232 正在安装 : ncurses-devel-5.7-4.20090207.el6.x86_64 35/48 233 清理 : krb5-workstation-1.10.3-42.el6.x86_64 36/48 234 清理 : e2fsprogs-1.41.12-22.el6.x86_64 37/48 235 清理 : openssl-1.0.1e-42.el6.x86_64 38/48 236 清理 : krb5-libs-1.10.3-42.el6.x86_64 39/48 237 清理 : e2fsprogs-libs-1.41.12-22.el6.x86_64 40/48 238 清理 : libss-1.41.12-22.el6.x86_64 41/48 239 清理 : libselinux-utils-2.0.94-5.8.el6.x86_64 42/48 240 清理 : libselinux-python-2.0.94-5.8.el6.x86_64 43/48 241 清理 : libstdc++-4.4.7-16.el6.x86_64 44/48 242 清理 : libgcc-4.4.7-16.el6.x86_64 45/48 243 清理 : libselinux-2.0.94-5.8.el6.x86_64 46/48 244 清理 : libcom_err-1.41.12-22.el6.x86_64 47/48 245 清理 : libgomp-4.4.7-16.el6.x86_64 48/48 246 Verifying : libstdc++-devel-4.4.7-18.el6.x86_64 1/48 247 Verifying : e2fsprogs-1.41.12-23.el6.x86_64 2/48 248 Verifying : krb5-devel-1.10.3-65.el6.x86_64 3/48 249 Verifying : libcom_err-1.41.12-23.el6.x86_64 4/48 250 Verifying : cmake-2.8.12.2-4.el6.x86_64 5/48 251 Verifying : subversion-1.6.11-15.el6_7.x86_64 6/48 252 Verifying : krb5-libs-1.10.3-65.el6.x86_64 7/48 253 Verifying : e2fsprogs-libs-1.41.12-23.el6.x86_64 8/48 254 Verifying : krb5-workstation-1.10.3-65.el6.x86_64 9/48 255 Verifying : gcc-c++-4.4.7-18.el6.x86_64 10/48 256 Verifying : automake-1.11.1-4.el6.noarch 11/48 257 Verifying : keyutils-libs-devel-1.4-5.el6.x86_64 12/48 258 Verifying : cpp-4.4.7-18.el6.x86_64 13/48 259 Verifying : libselinux-2.0.94-7.el6.x86_64 14/48 260 Verifying : mpfr-2.4.1-6.el6.x86_64 15/48 261 Verifying : autoconf-2.63-5.1.el6.noarch 16/48 262 Verifying : zlib-devel-1.2.3-29.el6.x86_64 17/48 263 Verifying : gcc-4.4.7-18.el6.x86_64 18/48 264 Verifying : libstdc++-4.4.7-18.el6.x86_64 19/48 265 Verifying : ncurses-devel-5.7-4.20090207.el6.x86_64 20/48 266 Verifying : openssl-devel-1.0.1e-57.el6.x86_64 21/48 267 Verifying : perl-URI-1.40-2.el6.noarch 22/48 268 Verifying : libgcc-4.4.7-18.el6.x86_64 23/48 269 Verifying : ppl-0.10.2-11.el6.x86_64 24/48 270 Verifying : libselinux-python-2.0.94-7.el6.x86_64 25/48 271 Verifying : libss-1.41.12-23.el6.x86_64 26/48 272 Verifying : libsepol-devel-2.0.41-4.el6.x86_64 27/48 273 Verifying : libselinux-utils-2.0.94-7.el6.x86_64 28/48 274 Verifying : libgomp-4.4.7-18.el6.x86_64 29/48 275 Verifying : cloog-ppl-0.15.7-1.2.el6.x86_64 30/48 276 Verifying : openssl-1.0.1e-57.el6.x86_64 31/48 277 Verifying : libkadm5-1.10.3-65.el6.x86_64 32/48 278 Verifying : libcom_err-devel-1.41.12-23.el6.x86_64 33/48 279 Verifying : libselinux-devel-2.0.94-7.el6.x86_64 34/48 280 Verifying : libtool-2.2.6-15.5.el6.x86_64 35/48 281 Verifying : libgomp-4.4.7-16.el6.x86_64 36/48 282 Verifying : libss-1.41.12-22.el6.x86_64 37/48 283 Verifying : libstdc++-4.4.7-16.el6.x86_64 38/48 284 Verifying : openssl-1.0.1e-42.el6.x86_64 39/48 285 Verifying : libgcc-4.4.7-16.el6.x86_64 40/48 286 Verifying : libselinux-utils-2.0.94-5.8.el6.x86_64 41/48 287 Verifying : e2fsprogs-1.41.12-22.el6.x86_64 42/48 288 Verifying : krb5-libs-1.10.3-42.el6.x86_64 43/48 289 Verifying : krb5-workstation-1.10.3-42.el6.x86_64 44/48 290 Verifying : libselinux-2.0.94-5.8.el6.x86_64 45/48 291 Verifying : libcom_err-1.41.12-22.el6.x86_64 46/48 292 Verifying : libselinux-python-2.0.94-5.8.el6.x86_64 47/48 293 Verifying : e2fsprogs-libs-1.41.12-22.el6.x86_64 48/48 294 295 已安装: 296 autoconf.noarch 0:2.63-5.1.el6 automake.noarch 0:1.11.1-4.el6 297 cmake.x86_64 0:2.8.12.2-4.el6 gcc-c++.x86_64 0:4.4.7-18.el6 298 libtool.x86_64 0:2.2.6-15.5.el6 ncurses-devel.x86_64 0:5.7-4.20090207.el6 299 openssl-devel.x86_64 0:1.0.1e-57.el6 subversion.x86_64 0:1.6.11-15.el6_7 300 301 作为依赖被安装: 302 cloog-ppl.x86_64 0:0.15.7-1.2.el6 cpp.x86_64 0:4.4.7-18.el6 303 gcc.x86_64 0:4.4.7-18.el6 keyutils-libs-devel.x86_64 0:1.4-5.el6 304 krb5-devel.x86_64 0:1.10.3-65.el6 libcom_err-devel.x86_64 0:1.41.12-23.el6 305 libkadm5.x86_64 0:1.10.3-65.el6 libselinux-devel.x86_64 0:2.0.94-7.el6 306 libsepol-devel.x86_64 0:2.0.41-4.el6 libstdc++-devel.x86_64 0:4.4.7-18.el6 307 mpfr.x86_64 0:2.4.1-6.el6 perl-URI.noarch 0:1.40-2.el6 308 ppl.x86_64 0:0.10.2-11.el6 zlib-devel.x86_64 0:1.2.3-29.el6 309 310 作为依赖被升级: 311 e2fsprogs.x86_64 0:1.41.12-23.el6 e2fsprogs-libs.x86_64 0:1.41.12-23.el6 312 krb5-libs.x86_64 0:1.10.3-65.el6 krb5-workstation.x86_64 0:1.10.3-65.el6 313 libcom_err.x86_64 0:1.41.12-23.el6 libgcc.x86_64 0:4.4.7-18.el6 314 libgomp.x86_64 0:4.4.7-18.el6 libselinux.x86_64 0:2.0.94-7.el6 315 libselinux-python.x86_64 0:2.0.94-7.el6 libselinux-utils.x86_64 0:2.0.94-7.el6 316 libss.x86_64 0:1.41.12-23.el6 libstdc++.x86_64 0:4.4.7-18.el6 317 openssl.x86_64 0:1.0.1e-57.el6 318 319 完毕！ 320 [root@master soft]#
安装Maven
编译要求：Maven 3.0 or later
安装软件：apache-maven-3.0.5-bin.tar.gz
[root@hadoop1 soft]# ls apache-maven-3.3.9-bin.tar.gz [root@hadoop1 soft]# chmod 755 apache-maven-3.3.9-bin.tar.gz [root@hadoop1 soft]# tar -zxvf apache-maven-3.3.9-bin.tar.gz
。。。
[root@hadoop1 soft]# mv apache-maven-3.3.9 /opt/
[root@hadoop1 soft]# vi /etc/profile
配置mvn的环境变量
export M2_HOME=/opt/apache-maven-3.3.
export PATH=$PATH:$M2_HOME/bin
测试
[root@hadoop1 soft]# mvn -v Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00) Maven home: /opt/apache-maven-3.3.9 Java version: 1.8.0_73, vendor: Oracle Corporation Java home: /usr/local/jdk1.8.0_73/jre Default locale: zh_CN, platform encoding: UTF-8 OS name: "linux", version: "2.6.32-573.el6.x86_64", arch: "amd64", family: "unix" [root@hadoop1 soft]#
安装Findbugs
编译要求：Findbugs 1.3.9
安装软件：findbugs-3.0.1.tar.gz
[root@hadoop1 soft]# ls findbugs-3.0.1.tar.gz [root@hadoop1 soft]# chmod 755 findbugs-3.0.1.tar.gz [root@hadoop1 soft]# tar -zxvf findbugs-3.0.1.tar.gz -C /opt
配置Findbugs环境变量
[root@hadoop1 soft]# vi /etc/profile
export FINDBUGS_HOME=/opt/findbugs-3.0.1
export PATH=$PATH:$FINDBUGS_HOME/bin
测试
[root@hadoop1 soft]# findbugs -version 3.0.1 [root@hadoop1 soft]#
安装ProtocolBuffer
编译要求：ProtocolBuffer 2.5.0
安装软件：protobuf-2.5.0.tar.gz，不建议用其它版本
[root@hadoop1 soft]# ls protobuf-2.5.0.tar.gz [root@hadoop1 soft]# chmod 755 protobuf-2.5.0.tar.gz [root@hadoop1 soft]# tar -zxvf protobuf-2.5.0.tar.gz -C /opt
[root@hadoop1 soft]# cd /opt/protobuf-2.5.0/
[root@hadoop1 protobuf-2.5.0]# ./configure
[root@hadoop1 protobuf-2.5.0]# make
[root@hadoop1 protobuf-2.5.0]# make install
测试
[root@hadoop1 protobuf-2.5.0]# protoc --version libprotoc 2.5.0 [root@hadoop1 protobuf-2.5.0]#
修改maven的配置文件，添加maven的下载源
[root@hadoop1 protobuf-2.5.0]# cd /opt/apache-maven-3.3.9/conf/
[root@hadoop1 conf]# vi settings.xml
在mirrors中添加alimaven的下载源
<mirrors>
<!-- mirror
| Specifies a repository mirror site to use instead of a given repository. The repository that
| this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
| for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
|
<mirror>
<id>alimaven</id>
<mirrorOf>central</mirrorOf>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
</mirror>
<mirror>
<id>mirrorId</id>
<mirrorOf>repositoryId</mirrorOf>
<name>Human Readable Name for this Mirror.</name>
<url>http://my.repository.com/repo/path</url>
</mirror>
-->
</mirrors>
安装Ant
[root@hadoop1 soft]# ls apache-ant-1.9.4-bin.tar.gz hadoop-2.7.5-src [root@hadoop1 soft]# tar -zxvf apache-ant-1.9.4-bin.tar.gz -C /opt/
配置环境变量
[root@hadoop1 apache-ant-1.9.4]# vi /etc/profile
#Ant export ANT_HOME=/opt/apache-ant-1.9.4 export PATH=$PATH:$ANT_HOME/bin
检测
[root@hadoop1 apache-ant-1.9.4]# ant -version Apache Ant(TM) version 1.9.4 compiled on April 29 2014 [root@hadoop1 apache-ant-1.9.4]#
安装Snappy
解压
[root@hadoop1 soft]# ls hadoop-2.7.5-src snappy-1.1.1.tar.gz [root@hadoop1 soft]# tar -zxvf snappy-1.1.1.tar.gz -C /opt/
安装
[root@hadoop1 soft]# cd /opt/snappy-1.1.1/ [root@hadoop1 snappy-1.1.1]# ./configure [root@hadoop1 snappy-1.1.1]# make [root@hadoop1 snappy-1.1.1]# make install
查看snappy文件库
[root@hadoop1 snappy-1.1.1]# ls -lh /usr/local/lib | grep snappy -rw-r--r-- 1 root root 228K 3月 3 09:51 libsnappy.a -rwxr-xr-x 1 root root 953 3月 3 09:51 libsnappy.la lrwxrwxrwx 1 root root 18 3月 3 09:51 libsnappy.so -> libsnappy.so.1.2.0 lrwxrwxrwx 1 root root 18 3月 3 09:51 libsnappy.so.1 -> libsnappy.so.1.2.0 -rwxr-xr-x 1 root root 145K 3月 3 09:51 libsnappy.so.1.2.0 [root@hadoop1 snappy-1.1.1]#
开始编译hadoop
[root@hadoop1 soft]# ls
[root@hadoop1 soft]# tar -zxvf hadoop-2.7.5-src.tar.gz
在编译之前防止java.lang.OutOfMemoryError:Java heap space堆栈问题，在centos系统中执行命令
[root@hadoop1 snappy-1.1.1]# export MAVEN_OPTS="-Xms256m -Xmx512m"
进入源码包下，执行命令进行编译
[root@hadoop1 snappy-1.1.1]# cd /soft/hadoop-2.7.5-src/ [root@hadoop1 hadoop-2.7.5-src]# mvn package -Pdist,native,docs -DskipTests -Dtar
如果中途编译失败，并且不要文档的话，请使用这个命令：
[root@master ~]# mvn clear package -Pdist,native -DskipTests -Dtar -Dsnappy.lib=/usr/local/lib -Dbundle.snappy -Drequire.openssl
编译成功之后，hadoop-2.7.5.tar.gz位于/soft/hadoop-2.7.5-src/hadoop-dist/target目录下，这是编译后文件夹的状态
至此，大功告成！！！
