安装hadoop集群-cm 5.14.4
环境 要求：
centos 7
java 1.8
cm 5.14.4
mysql 5.7
4核16G
工具要求：
mysql-connector-java-5.1.39.jar
1、绑定 hosts
10.31.210.15 test-algo-hadoop-01
10.31.210.16 test-algo-hadoop-02
10.31.210.17 test-algo-hadoop-03
2、受信ssh key
test-algo-hadoop-01 自动登录 其它 机器
test-algo-hadoop-02 自动 登录 其它 的机器
test-algo-hadoop-01 和test-algo-hadoop-02互登录
3、修改内核并增加到rc.local文件
# 内存参数调整 #sysctl -w vm.swappiness=10 或者 编辑vim /etc/sysctl.conf，
在最后加入： #vm.swappiness = 10
编辑启动项vim /etc/rc.local,最后加入：
#echo never > /sys/kernel/mm/transparent_hugepage/defrag
#echo never > /sys/kernel/mm/transparent_hugepage/enabled
关闭防火墙
#systemctl disable firewalld
#systemctl stop firewalld
设置时间同步
ntpd
4、下载软件包和工具
mysql-connector-java-5.1.39.jar
CDH-5.14.4-1.cdh5.14.4.p0.3-el7.parcel
CDH-5.14.4-1.cdh5.14.4.p0.3-el7.parcel.sha
cloudera-manager-centos7-cm5.14.4_x86_64.tar.gz
manifest.json
解压 ：
#cp /data/soft/cloudera-manager-centos7-cm5.14.4_x86_64.tar.gz /opt/ && cd /opt
#tar zcvf cloudera-manager-centos7-cm5.14.4_x86_64.tar.gz
#cp /data/soft/CDH-5.14.4-1.cdh5.14.4.p0.3-el7.parcel* /opt/cloudera/parcel-repo/
5、安装jdk 1.8 && mysql （略）
6、配置mysql帐号
grant all on *.* to 'hadoop'@'10.31.%' identified by 'xxxx' with grant option;
flush privileges;
创建数据库
create database amon DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database rman DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database metastore DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database sentry DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database nav DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database navms DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database monitor DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
安装java（所有机器） && mysql jar库
#cp /data/soft/mysql-connector-java-5.1.39.jar /opt/cm-5.14.4/share/cmf/cloudera-navigator-server/jars/
pw
#cp /data/soft/mysql-connector-java-5.1.39.jar /opt/cm-5.14.4/share/cmf/lib
#mkdir -p /usr/share/java && cp /data/soft/mysql-connector-java-5.1.39.jar /usr/share/java/mysql-connector-java.jar
#初始化mysql
/opt/cm-5.14.4/share/cmf/schema/scm_prepare_database.sh mysql cm -htest-algo-hadoop-01 -uhadoop -p --scm-host "10.31.%" scm
yum安装：
yum install libxslt-devel libxslt-python httpd mod_ssl cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain chkconfig bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs redhat-lsb httpd httpd-tools unzip ntp
cm agent 配置
#vim /opt/cm-5.14.4/etc/cloudera-scm-agent/config.ini
server_host=test-algo-hadoop-01
#scp到其它的服务器
#scp -P 57301 -r /opt/cm-5.14.4 test-algo-hadoop-02:/opt/
#scp -P 57301 -r /opt/cm-5.14.4 test-algo-hadoop-03:/opt/
在所有节点创建cloudera-scm用户
#useradd --system --home=/opt/cm-5.14.4/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
启动cm server 和agent:
/opt/cm-5.14.4/etc/init.d/cloudera-scm-server restart
/opt/cm-5.14.4/etc/init.d/cloudera-scm-agent restart
访问 ：http://服务器IP:7180
安装cloudera 时报错
Error, CM server guid updated, expected 25cf17b3-391a-4368-848a-07118d6f11fb, received b3d4a47a-1476-4ea4-b236-24426b1b8540
rm /opt/cm-5.14.4/lib/cloudera-scm-agent/cm_guid -f
/opt/cm-5.14.4/etc/init.d/cloudera-scm-agent restart
重新生成guid
