Hadoop-CDH5.7.0 for CentOS7
一、需求
系统
CentOS 7 最小化安装
JDK环境
JDK版本：1.8.0_91 jdk-8u91-linux-x64.rpm
下载地址：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
CM包
CM版本：5.7.0 cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz
下载地址：http://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz
CDH包
CDH版本：5.7.0
CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 manifest.json
下载地址
http://archive.cloudera.com/cdh5/parcels/5.7/CDH-5.7.1-1.cdh5.7.1.p0.11-el7.parcel
http://archive.cloudera.com/cdh5/parcels/5.7/CDH-5.7.1-1.cdh5.7.1.p0.11-el7.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.7/manifest.json
java-mysql连接jar包
下载地址：http://central.maven.org/maven2/mysql/mysql-connector-java/6.0.2/mysql-connector-java-6.0.2.jar
数据库
类型：MariaDB
版本：5.5.47-MariaDB MariaDB Server
节点
| 主机名 | IP地址 | 说明 |
| bigdata01.tfpay.com | 172.17.28.1 | master datanode namenode |
| bigdata02.tfpay.com | 172.17.28.2 | datanode |
| bigdata03.tfpay.com | 172.17.28.3 | datanode |
| bigdata04.tfpay.com | 172.17.28.4 | datanode |
节点的数量可以根据自己的需求变更，也可以在CDH安装完成后，在管理界面自己拓展添加
二、底层环境准备
注：准备安装前需要把相关的包文件下载下来，并上传到服务器上
1、安装JDK
所有节点
1）安装系统原有java
# yum -y install java # java -version openjdk version "1.8.0_91" OpenJDK Runtime Environment (build 1.8.0_91-b14) OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode) # rpm -qa|grep java javapackages-tools-3.4.1-11.el7.noarch python-javapackages-3.4.1-11.el7.noarch java-1.8.0-openjdk-headless-1.8.0.91-0.b14.el7_2.x86_64 java-1.8.0-openjdk-1.8.0.91-0.b14.el7_2.x86_64 tzdata-java-2016d-1.el7.noarch
2）卸载OpenJDK
# rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.91-0.b14.el7_2.x86_64
# rpm -e --nodeps java-1.8.0-openjdk-1.8.0.91-0.b14.el7_2.x86_64
3）安装下载的java
# rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.91-0.b14.el7_2.x86_64 # rpm -e --nodeps java-1.8.0-openjdk-1.8.0.91-0.b14.el7_2.x86_64
# rpm -ivh jdk-8u91-linux-x64.rpm Preparing... ################################# [100%] Updating / installing... 1:jdk1.8.0_91-2000:1.8.0_91-fcs ################################# [100%] Unpacking JAR files... tools.jar... plugin.jar... javaws.jar... deploy.jar... rt.jar... jsse.jar... charsets.jar... localedata.jar... jfxrt.jar... # java -version java version "1.8.0_91" Java(TM) SE Runtime Environment (build 1.8.0_91-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode) # javac -version javac 1.8.0_91
2、更改主机名和hosts
所有节点
注意每个主机名对应的IP，按实际情况更改。
# echo "bigdata0*.tfpay.com" > /etc/hostname # echo "172.17.28.1 bigdata01.tfpay.com bigdata01" >> /etc/hosts # echo "172.17.28.2 bigdata02.tfpay.com bigdata02" >> /etc/hosts # echo "172.17.28.3 bigdata03.tfpay.com bigdata03" >> /etc/hosts # echo "172.17.28.4 bigdata04.tfpay.com bigdata04" >> /etc/hosts
3、关闭防火墙
所有节点
# systemctl stop firewalld # systemctl disable firewalld Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. # systemctl status firewalld ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Jun 20 16:05:43 bigdata05.tfpay.com systemd[1]: Starting firewalld - dynamic firewall daemon... Jun 20 16:05:45 bigdata05.tfpay.com systemd[1]: Started firewalld - dynamic firewall daemon. Jun 20 16:17:51 bigdata05.tfpay.com systemd[1]: Stopping firewalld - dynamic firewall daemon... Jun 20 16:17:52 bigdata05.tfpay.com systemd[1]: Stopped firewalld - dynamic firewall daemon. # sed -i "s/SELINUX=enforcing/SELINUX=disabled/" /etc/selinux/config # reboot
重启服务器使hostname和selinux的配置生效
# getenforce Disabled # sestatus -v SELinux status: disabled
4、安装NTP服务
所有节点
# yum -y install ntp
更改配置文件/etc/ntp.conf
在master节点namenode节点
注释掉原有的server指向把时间服务器的域名地址指向一个靠谱可连的时间服务器，我选择的是ntp.api.bz.
# vi /etc/ntp.conf …… #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst server ntp.api.gz iburst ......
在其余的节点上，把时间服务器的server指向master namenode节点
# vi /etc/ntp.conf …… #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst server 172.17.28.1 iburst ......
所有节点
# systemctl start ntpd
# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
5、SSH无密码登陆
# systemctl start ntpd
# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
配置master namenode节点无密码登陆到其他节点
在master namenode节点
# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): 回车 Enter passphrase (empty for no passphrase): 回车 Enter same passphrase again: 回车 Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: bc:14:e2:59:42:1f:21:80:eb:49:d8:3b:7c:8d:b7:f9 root@bigdata05.tfpay.com The key's randomart image is: +--[ RSA 2048]----+ | ...o o. | | . . o . | | o . o + | |. + . * . | | + o oo S | | * o o. . | | o . o. | | o | | .E | +-----------------+ # ssh-copy-id 172.17.28.2 # ssh-copy-id 172.17.28.3 # ssh-copy-id 172.17.28.4
6、安装数据库
master namenode节点
# yum -y install mariadb-server # vi /etc/my.cnf [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock symbolic-links=0 character-set-server=utf8 \\添加的行 [mysqld_safe] log-error=/var/log/mariadb/mariadb.log pid-file=/var/run/mariadb/mariadb.pid !includedir /etc/my.cnf.d # systemctl start mariadb # systemctl enable mariadb ln -s '/usr/lib/systemd/system/mariadb.service' '/etc/systemd/system/multi-user.target.wants/mariadb.service' # mysql_secure_installation /usr/bin/mysql_secure_installation: line 379: find_mysql_client: command not found NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY! In order to log into MariaDB to secure it, we'll need the current password for the root user. If you've just installed MariaDB, and you haven't set the root password yet, the password will be blank, so you should just press enter here. Enter current password for root (enter for none): OK, successfully used password, moving on... Setting the root password ensures that nobody can log into the MariaDB root user without the proper authorisation. Set root password? [Y/n] y \\设置数据库root密码 New password: \\设置密码 Re-enter new password: \\再次输入密码 Password updated successfully! Reloading privilege tables.. ... Success! By default, a MariaDB installation has an anonymous user, allowing anyone to log into MariaDB without having to have a user account created for them. This is intended only for testing, and to make the installation go a bit smoother. You should remove them before moving into a production environment. Remove anonymous users? [Y/n] y \\移除匿名用户 ... Success! Normally, root should only be allowed to connect from 'localhost'. This ensures that someone cannot guess at the root password from the network. Disallow root login remotely? [Y/n] y \\禁止root用户远程登录 ... Success! By default, MariaDB comes with a database named 'test' that anyone can access. This is also intended only for testing, and should be removed before moving into a production environment. Remove test database and access to it? [Y/n] y \\删除测试数据库 - Dropping test database... ... Success! - Removing privileges on test database... ... Success! Reloading the privilege tables will ensure that all changes made so far will take effect immediately. Reload privilege tables now? [Y/n] // ... Success! Cleaning up... All done! If you've completed all of the above steps, your MariaDB installation should now be secure. Thanks for using MariaDB!
数据库只是安装了一下，并没有做一些详细的配置。如果有需求可以另行配置。
7、其他安装与配置
所有节点
# yum -y install psmisc MySQL-python at bc bind-libs bind-utils cups-client cups-libs cyrus-sasl-gssapi cyrus-sasl-plain ed fuse fuse-libs httpd httpd-tools keyutils-libs-devel krb5-devel libcom_err-devel libselinux-devel libsepol-devel libverto-devel mailcap noarch mailx mod_ssl openssl-devel pcre-devel postgresql-libs python-psycopg2 redhat-lsb-core redhat-lsb-submod-security x86_64 spax time zlib-devel # chmod +x /etc/rc.d/rc.local # echo "echo 0 > /proc/sys/vm/swappiness" >>/etc/rc.d/rc.local # echo "echo never > /sys/kernel/mm/transparent_hugepage/defrag" >>/etc/rc.d/rc.local # echo 0 > /proc/sys/vm/swappiness # echo never > /sys/kernel/mm/transparent_hugepage/defrag # yum -y install rpcbind # systemctl start rpcbind # echo "systemctl start rpcbind" >> /etc/rc.d/rc.local
三、开始安装CDH
1、传宝、解包
把下载好的一下相应包文件上传到master namenode节点内
mysql-java连接包在所有节点上都要上传
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # tar zxvf cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz -C /opt/ # ls /opt/ cloudera cm-5.7.0
2、创建数据库
# mysql -uroot -p Enter password: \\输入数据库密码 Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 10 Server version: 5.5.47-MariaDB MariaDB Server Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create database monitor DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> grant all on *.* to root@"%" Identified by "1234.com"; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> exit # /opt/cm-5.7.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -p1234.com --scm-host localhost scm scm scm JAVA_HOME=/usr/java/jdk1.8.0_91 Verifying that we can write to /opt/cm-5.7.0/etc/cloudera-scm-server Creating SCM configuration file in /opt/cm-5.7.0/etc/cloudera-scm-server Executing: /usr/java/jdk1.8.0_91/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.7.0/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.7.0/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. 2016-06-17 13:32:20,562 [main] INFO com.cloudera.enterprise.dbutil.DbCommandExecutor - Successfully connected to database. All done, your SCM database is configured correctly!
3、创建用户
所有节点
#useradd --system --home=/opt/cm-5.7.0/run/cloudera-scm-server --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
4、制作本地源
master datanote节点执行
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # cp CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 manifest.json /opt/cloudera/parcel-repo/ # mv CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha
5、拷贝jar包
所有节点
# ls
CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json
CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar
# cp mysql-connector-java-6.0.2.jar /usr/share/java/mysql-connector-java.jar
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # cp mysql-connector-java-6.0.2.jar /usr/share/java/mysql-connector-java.jar
master namenode节点
# cp mysql-connector-java-6.0.2.jar /opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hive/lib/ # cp mysql-connector-java-6.0.2.jar /opt/cm-5.7.0/share/cmf/lib/
6、更改配置
# sed -i "s/server_host=localhost/server_host=172.17.28.1/" /opt/cm-5.7.0/etc/cloudera-scm-agent/config.ini # mkdir /opt/cm-5.7.0/run/cloudera-scm-agent
7、启动服务
master namenode节点
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-server start
除了master namenode节点
# scp -r 172.17.28.1:/opt/cm-5.7.0 /opt
所有节点
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-agent start
8、浏览页面开始安装
1）登陆
打开浏览器，访问maser namenode节点的7180端口
输入用户名和密码admin、admin
2）接受协议
3）选择版本
本文档选择Cloudera Express免费开源版本
4）指定主机
5）选择CDH版本
注：选择在制作本地源时选择的版本，如果选择别的版本的话会去官网下载那样安装速度会很慢
6）检查主机正确性
确保全部验证通过
7）选择安装的服务
本文选择的是安装所有服务
8）角色分配
这里应该根据自己的实际需求和服务器的硬件参数自己选择
9）数据库设置
10）群集设置
可以根据自己的实际情况更改设置，本文采用默认配置。这些配置可以在集群安装完成以后后续更改
11）开始安装
无错误信息全部通过
12）安装完成
第一次安装完成后，会出现一些配置的警告信息。这些可以根据提示信息更改。
四、错误总结
1、pstree: command not found
错误展示
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-server start /opt/cm-5.7.0/etc/init.d/cloudera-scm-server: line 109: pstree: command not found Starting cloudera-scm-server: [FAILED] # /opt/cm-5.7.0/etc/init.d/cloudera-scm-agent start /opt/cm-5.7.0/etc/init.d/cloudera-scm-agent: line 108: pstree: command not found Starting cloudera-scm-agent: [FAILED]
原因
因为系统是最小化安装，默认没有安装
解决方法
# yum -y install psmisc
2、cloudera-scm-agent start 启动失败
错误展示
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-agent start Starting cloudera-scm-agent: [FAILED]
查询日志
、
# cd /opt/cm-5.7.0/log/cloudera-scm-agent/ # ls cloudera-scm-agent.out # cat cloudera-scm-agent.out [22/Jun/2016 11:02:09 +0000] 3695 MainThread agent INFO SCM Agent Version: 5.7.0 Unable to create the pidfile.
原因
经过查找后发现/opt/cm-5.7.0/run目录下没有cloudera-scm-agent文件夹
# cd /opt/cm-5.7.0/run/ # ls cloudera-scm-server
解决方法
# mkdir /opt/cm-5.7.0/run/cloudera-scm-agent
3、install: invalid user ‘cloudera-scm’
错误展示
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-server start install: invalid user ‘cloudera-scm’
原因
没有创建运行server的用户
解决方法
#useradd --system --home=/opt/cm-5.7.0/run/cloudera-scm-server --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
4、检查主机正确性警告
警告
解决方法
# echo 0 > /proc/sys/vm/swappiness # echo never > /sys/kernel/mm/transparent_hugepage/defrag # echo "echo 0 > /proc/sys/vm/swappiness" >>/etc/rc.d/rc.local # echo "echo never > /sys/kernel/mm/transparent_hugepage/defrag" >>/etc/rc.d/rc.local
5、启动Hive失败
错误展示
原因
根据提示发现是没有mysql-java连接的驱动jar包
解决方法
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # cp mysql-connector-java-6.0.2.jar /opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hive/lib/
6、启动NFS Gateway失败
错误展示
原因
解决方法
# yum -y install rpcbind # systemctl start rpcbind # echo "systemctl start rpcbind" >> /etc/rc.d/rc.local # chmod +x /etc/rc.d/rc.local
7、启动Oozie,impala失败
错误展示
原因
没有mysql-java连接jar包
解决方法
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # cp mysql-connector-java-6.0.2.jar /usr/share/java/mysql-connector-java.jar
