CDH搭建Hadoop集群_Centos7

CDH搭建Hadoop集群(Centos7)

一、说明 

节点(CentOS7.5) Server  || Agent CPU

node11 Server  || Agent 6G

node12 Agent 2G

node13 Agent 2G

 

#下载mysql-connector-java

http://dev.mysql.com/downloads/connector/j/

 

二、系统环境搭建

1、网络配置(所有节点)

1．设置主机名和IP

#修改主机名

hostnamectl set-hostname node11

hostnamectl set-hostname node12

hostnamectl set-hostname node13

2．配置/etc/hosts  IP和主机名的映射

#所有节点执行

cat >> /etc/hosts <<EOF 192.168.92.23 node11 192.168.92.24 node12 192.168.92.25 node13

EOF

 

2、SSH免密码登录

#一直回车直即可

ssh-keygen -t rsa

ssh-copy-id -i node11

ssh-copy-id -i node12

ssh-copy-id -i node13

 

ssh root@node11 date

ssh root@node12 date

ssh root@node13 date

 

3、关闭防火墙

systemctl status firewalld

systemctl stop firewalld

systemctl disable firewalld

#查看防火墙状态

firewall-cmd --state

systemctl status firewalld

 

4、关闭SELINUX

#查看selinux状态

sestatus -v

vi /etc/selinux/config

SELINUX=disabled

 

5、安装JDK

#jdk-8u271这个版本太高

mkdir -p /software

rpm -qa | grep java

#卸载自带open jdk

rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.141-2.6.10.1.el7_3.x86_64

rpm -e --nodeps java-1.7.0-openjdk-1.7.0.141-2.6.10.1.el7_3.x86_64

rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.131-3.b12.el7_3.x86_64

rpm -e --nodeps java-1.8.0-openjdk-1.8.0.131-3.b12.el7_3.x86_64

 

cd /software

scp jdk-8u191-linux-x64-rpm.zip root@node12:/software/

scp jdk-8u191-linux-x64-rpm.zip root@node13:/software/

unzip jdk-8u191-linux-x64-rpm.zip

rpm -ivh  jdk-8u191-linux-x64.rpm

ln -s /usr/java/jdk1.8.0_191-amd64/ /usr/java/jdk1.8.0_191

 

cat >> /etc/profile <<EOF

export JAVA_HOME=/usr/java/jdk1.8.0_191

export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/sbin

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

EOF

 

source /etc/profile

java -version

 

6、设置NTP

yum -y install ntp

#注释掉所有server *.*.*的指向，新添加一条可连接的ntp服务器

#在其他节点上把ntp指向master服务器地址即可(/etc/ntp.conf下) server node11 iburst

vi /etc/ntp.conf

server 192.168.92.23 iburst

 

systemctl start ntpd

systemctl enable ntpd

systemctl status ntpd

 

7、安装配置MySql

#主节点安装

#卸载mariadb

rpm -qa|grep mariadb

rpm -e --nodeps mariadb-libs-5.5.52-1.el7.x86_64

rpm -qa|grep mariadb

 

#上传MySQL包到：/software

ll

mysql-5.7.24-linux-glibc2.12-x86_64.tar.gz

#mysql-connector-java-5.1.39.tar.gz

 

3．安装MySQL

groupadd mysql

useradd -r -g mysql -s /sbin/nologin mysql

mv mysql-5.7.18-linux-glibc2.5-x86_64.tar.gz /usr/local/

cd /usr/local/

tar zxvf mysql-5.7.18-linux-glibc2.5-x86_64.tar.gz -C /usr/local/

ln -s mysql-5.7.18-linux-glibc2.5-x86_64/ mysql

cd mysql

mkdir mysql-files

ll

chown -R mysql.mysql .

ll

#op).e)osp8dF

bin/mysqld --initialize --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data

cp -rf /etc/my.cnf /etc/my.cnf.old

 

cat >> /etc/my.cnf <<EOF

[mysqld]

basedir=/usr/local/mysql

datadir=/usr/local/mysql/data

EOF

 

#启动mysql skip-grant-tables

bin/mysqld_safe --user=mysql &

cp support-files/mysql.server /etc/init.d/mysql

chmod a+x /etc/init.d/mysql

chkconfig --add mysql

chkconfig mysql on

service mysql status

 

#设置root的密码

#设置mysql环境变量

echo "export PATH=$PATH:/usr/local/mysql/bin" >> /etc/profile

source /etc/profile

 

mysql -uroot -p

#use mysql;

#update user set authentication_string=PASSWORD('123456') where User='root';

#以上修改密码不成功，可以set

SET PASSWORD = PASSWORD('123456');

#或者

#alter user root@'localhost' identified by '123456';

flush privileges;

 

#重新初始化

exit

killall mysqld

rm -rf /usr/local/mysql/data/

chown -R mysql.mysql /usr/local/mysql

#Z,hBoE:fh20>

bin/mysqld --initialize --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data

bin/mysql_ssl_rsa_setup

service mysql start

mysql -uroot -p

SET PASSWORD = PASSWORD('123456');

flush privileges;

exit

 

三、Cloudera Manager Server&Agent安装

CM安装包下载地址:   https://archive.cloudera.com/cm5/cm/5/

CDH安装包下载地址：  http://archive.cloudera.com/cdh5/parcels/5.15.0/

1、安装Cloudera Manager Server&Agent

#三台机器都要安装

cd /software/

scp cloudera-manager-centos7-cm5.15.0_x86_64.tar.gz root@node12:/software/

scp cloudera-manager-centos7-cm5.15.0_x86_64.tar.gz root@node13:/software/

tar xvzf cloudera-manager-centos7-cm5.15.0_x86_64.tar.gz  -C /opt

 

2、配置CM Agent

1.cm的安装包的server和agent都是同一个包，启动的时候启动命令指定为server还是agent；

2.cm有一个工作目录，需要我们来创建；创建一个用户；修改agent的配置，给它指定server的位置；那个端口7182是server和agent通信的端口；

3.server_host配置为server的名字,三台机器

vi /opt/cm-5.15.0/etc/cloudera-scm-agent/config.ini

 

3、创建用户cloudera-scm（所有节点）

useradd --system --home=/opt/cm-5.15.0/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

 

4、创建Parcel目录

创建parcel目录，这个目录是server和agent用来接收和发送数据的目录，server端的parcel-repo这个目录会把所有的安装文件全部下载到此目录，而agent也需要安装包，parcels就是用来存储指定的安装包的，当然需要有权限能操作这些目录；

#Server节点

mkdir -p /opt/cloudera/parcel-repo

chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo

 

#Agent节点

mkdir -p /opt/cloudera/parcels

chown cloudera-scm:cloudera-scm /opt/cloudera/parcels

 

#把CDH安装包移到cloudera/parcel-repo下面,并修改其中后缀为sha1的文件为sha,如下

mv CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel /opt/cloudera/parcel-repo

mv CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel.sha1 /opt/cloudera/parcel-repo

cd /opt/cloudera/parcel-repo

mv CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel.sha1 CDH-5.15.0-1.cdh5.15.0.p0.21-el7.parcel.sha

 

5、配置CM Server的数据库

cd /software

tar zxvf mysql-connector-java-5.1.39.tar.gz

cd mysql-connector-java-5.1.39

cp mysql-connector-java-5.1.39-bin.jar mysql-connector-java.jar

cp mysql-connector-java.jar /opt/cm-5.15.0/share/cmf/lib/

 

mysql -u root -p123456 use mysql; grant all on *.* to 'root'@'%' identified by '123456' with grant option; flush privileges;

exit

 

#初始化cm数据库:

cd /opt/cm-5.15.0/share/cmf/schema/ #格式：数据库类型、数据库、数据库服务器、用户名、密码、cm server服务器 ./scm_prepare_database.sh mysql cm -h node11 -uroot -p123456 --scm-host node11 scm scm scm

 

JAVA_HOME=/usr/java/jdk1.8.0_191

Verifying that we can write to /opt/cm-5.15.0/etc/cloudera-scm-server

log4j:ERROR Could not find value for key log4j.appender.A

log4j:ERROR Could not instantiate appender named "A".

Creating SCM configuration file in /opt/cm-5.15.0/etc/cloudera-scm-server

Executing: /usr/java/jdk1.8.0_271/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cm-5.15.0/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.15.0/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.

log4j:ERROR Could not find value for key log4j.appender.A

log4j:ERROR Could not instantiate appender named "A".

[2021-01-10 21:12:06,901] INFO 0[main] - com.cloudera.enterprise.dbutil.DbCommandExecutor.testDbConnection(DbCommandExecutor.java) - Successfully connected to database.

All done, your SCM database is configured correctly!

 

#创建其他组件需要的数据库

mysql -uroot -p123456

create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

exit

 

6、启动CM Manager&Agent服务

#先确保启动了mysql

service mysql status

#node11:执行

/opt/cm-5.15.0/etc/init.d/cloudera-scm-server start

#node11,12,13: 执行

/opt/cm-5.15.0/etc/init.d/cloudera-scm-agent start 

#service服务启动需要几分钟，取决于服务器的性能，此时我们可以使用

netstat -apn|grep 7180

#查询端口被占用后，就可以用浏览器打开:用户名与密码分别为admin/admin

http://192.168.92.23:7180/

 

7、Server&Agent开机自启动服务

#配置server主节点node11

cp /opt/cm-5.15.0/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server

chkconfig --add cloudera-scm-server

chkconfig cloudera-scm-server on

 

vi /etc/init.d/cloudera-scm-server

#CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default} 修改为如下

CMF_DEFAULTS=${CMF_DEFAULTS:-/opt/cm-5.15.0/etc/default}

 

chkconfig --list cloudera-scm-server

service cloudera-scm-server start & systemctl start cloudera-scm-server.service

service cloudera-scm-server status

systemctl status cloudera-scm-server.service

 

#配置agent所有节点,node11,node12,node13

cp /opt/cm-5.15.0/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent

chkconfig --add cloudera-scm-agent

chkconfig cloudera-scm-agent on

 

vi /etc/init.d/cloudera-scm-agent

 

#CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default} 修改为

CMF_DEFAULTS=${CMF_DEFAULTS:-/opt/cm-5.15.0/etc/default}

 

#添加：export JAVA_HOME=/usr/local/java/jdk1.8

service cloudera-scm-agent start & systemctl start cloudera-scm-agent.service

service cloudera-scm-agent status

systemctl status cloudera-scm-agent.service

 

四、CDH5安装 

 下一步,选择免费版本

继续,下一步,直到如下界面,可以在当前管理的主机列表中看到对应的节点。选择要安装的节点，这里我选择全部节点，继续。

选择自己下载的CDH版本,点击继续

接下来是安装Parcel,服务器检查，

#更改交换空间:  

echo 10 > /proc/sys/vm/swappiness

#如下禁用此设置，然后将同一命令添加到 /etc/rc.local 等初始化脚本中，以便在系统重启时予以设置。以下主机将受到影响: 

echo never > /sys/kernel/mm/transparent_hugepage/defrag

echo never > /sys/kernel/mm/transparent_hugepage/enabled

 

cat >> /etc/rc.local <<EOF

echo never > /sys/kernel/mm/transparent_hugepage/defrag

echo never > /sys/kernel/mm/transparent_hugepage/enabled

EOF

 

cat /etc/rc.local

 

重新运行检测,

全部完成,点击完成,这里选择自定义安装,不要选择：Hue,Isilon,Kafka,Kudu,spark,sqoop

服务配置，一般情况下保持默认就可以了，也可以自己手动根据节点情况进行分配，点击继续

测试如下数据库连接

#这里启动hive报错找不到mysql连接驱动包,拷贝一份  mysql-connector-java.jar   到  /opt/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hive/lib/下面

#主节点执行

cp /opt/cm-5.15.0/share/cmf/lib/mysql-connector-java.jar /opt/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hive/lib/

点击继续,默认集群设置,直到安装成功. 

 

----------------------------------------------------------------------------------------------

1.CM+CDH安装大数据的过程中出现：主机运行状态不良情况

集群重启后出现此情况，重启即可，如下为参考

rm -f /opt/cloudera-manager/cm-5.14.0/lib/cloudera-scm-agent/cm_guid find / -name cm_guid sudo service cloudera-scm-agent restart