docker安装cdh6.3.2集群(联网版)
一、前言
本次采用的在线安装方式,cdh为6.3.2版本,系统为centos7.4, docker节点可以为任意多个,下文将以3个docker容器为示例进行展示。此方法也可用在docker swarm上,docker容器能够互连,网络互通即可。
二、基础系统镜像制作
说明:此步在安装完docker后进行操作,步骤位置放的不对!!!
因为dockerhub上提供的centos系统为精简系统,使用时会缺少一些组件,制作一个centos7的基础镜像,Dockerfile为:
FROM docker.io/ansible/centos7-ansible RUN yum -y install openssh-server RUN yum -y install bind-utils RUN yum -y install which
RUN yum -y install sudo
在Dockerfile同级目录执行:
docker build -t centos7-cdh .
生成要用的基础centos7的镜像。
三、宿主机初始化
以下在docker的宿主机上进行操作。
3.1配置yum源
yum install -y wget \ && mkdir -p /etc/yum.repos.d/repo_bak \ && mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/repo_bak/ \ && wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo \ && wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo \ && yum clean all \ && yum makecache \ && yum update –y
3.2安装docker-ce
yum install -y yum-utils device-mapper-persistent-data lvm2 \ && yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo\ && yum makecache fast \ && yum -y install docker-ce \ && yum clean all \ && docker version
3.3配置docker
data-root:docker根目录路径
insecure-registries:docker私有仓库地址
registry-mirrors:公共镜像仓库加速地址
mkdir -p /etc/docker/ && \ ( cat <<EOF { "data-root":"/home/docker", "registry-mirrors": ["https://1e7waog4.mirror.aliyuncs.com"] } EOF ) >>/etc/docker/daemon.json \ && systemctl start docker \ && systemctl enable docker \ && cat /etc/docker/daemon.json \ && docker version
安装docker命令补全工具
yum install -y bash-completion \ && source /usr/share/bash-completion/completions/docker \ && source /usr/share/bash-completion/bash_completion \ && yum clean all
3.4安装基本工具
yum install -y vim wget ntp net-tools \
&& yum clean all
3.5关闭防火墙
systemctl stop firewalld \ && systemctl disable firewalld \ && systemctl status firewalld
3.6设置SELinux模式
setenforce 0 \ || sed -i 's/enforcing/disabled/g' /etc/selinux/config \ && sed -i 's/permissive/disabled/g' /etc/selinux/config \ && getenforce
3.7配置时间同步
vim /etc/ntp.conf
将时钟服务器更改为如下4个
server 0.cn.pool.ntp.org server 1.cn.pool.ntp.org server 2.cn.pool.ntp.org server 3.cn.pool.ntp.org
注意:若为离线安装,则要将服务器地址改为内网时间服务器地址。
启动设置ntp服务
systemctl start ntpd \ && systemctl enable ntpd \ && ntpdate -u 0.cn.pool.ntp.org \ && hwclock --systohc \ && date
3.8 创建自定义网络
docker network create --subnet=172.10.0.0/16 hadoop_net \ && docker network ls
3.9启动容器
docker run -d \ --add-host cm.hadoop:172.10.0.2 \ --net hadoop_net \ --ip 172.10.0.2 \ -h cm.hadoop \ -p 10022:22 \ -p 7180:7180 \ --restart always \ --name cm.hadoop \ --privileged \ centos7-cdh \ /usr/sbin/init \ && docker ps
四、容器安装ClouderaManager
4.1初始化环境
进入第三步创建的容器中,并将root的登录密码改为root
docker exec –it {dockerId} /bin/bash
su root
passwd
root
root
安装基础环境
yum install -y kde-l10n-Chinese telnet reinstall glibc-common vim wget ntp net-tools \
&& yum clean all
4.2配置中文环境变量
( cat <<EOF export LC_ALL=zh_CN.utf8 export LANG=zh_CN.utf8 export LANGUAGE=zh_CN.utf8 EOF ) >> ~/.bashrc \ && localedef -c -f UTF-8 -i zh_CN zh_CN.utf8 \ && source ~/.bashrc \ && echo $LANG
4.3设置NTP时间同步服务
vim /etc/ntp.conf
更改为以下四个时钟服务器
server 0.cn.pool.ntp.org server 1.cn.pool.ntp.org server 2.cn.pool.ntp.org server 3.cn.pool.ntp.org
启动ntp服务
systemctl start ntpd && \ systemctl enable ntpd && \ ntpdate -u 0.cn.pool.ntp.org && \ hwclock --systohc && \ date
4.4准备MySQL安装包(采用docker独立安装跳过此步)
mkdir -p /root/hadoop_CHD/mysql \ && wget -O /root/hadoop_CHD/mysql/mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar \ https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar \ && ls /root/hadoop_CHD/mysql
说明:
1、通过wget下载mysql安装十分慢,可以使用window系统的电脑讯雷等下载工具下载后,将安装包放至宿主机,再使用docker cp mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar {容器ID}:/root/hadoop_CHD/mysql命令将安装包放至容器中,前提是容器的/root/hadoop_CHD/mysql目录必须事先创建。
2、可以将mysql部署在一个独立的容器中,若采用此方法则跳此步,单独部署的方法在此不再阐述。
4.5准备MySQL JDBC驱动
mkdir -p /root/hadoop_CHD/mysql-jdbc \ && wget -O /root/hadoop_CHD/mysql-jdbc/mysql-connector-java-5.1.48.tar.gz \ https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.48.tar.gz \ && ls /root/hadoop_CHD/mysql-jdbc
说明:
1、通过wget下载安装十分慢,可以使用window系统的电脑讯雷等下载工具下载后,将安装包放至宿主机,再使用docker cp mysql-connector-java-5.1.48.tar.gz {容器ID}:/root/hadoop_CHD/mysql-jdbc命令将安装包放至容器中,前提是容器的/root/hadoop_CHD/mysql-jdbc目录必须事先创建。
4.6准备Cloudera-Manager安装包
mkdir -p /root/hadoop_CHD/cloudera-repos \ && wget -O /root/hadoop_CHD/cloudera-repos/allkeys.asc \ https://archive.cloudera.com/cm6/6.3.0/allkeys.asc \ && wget -O /root/hadoop_CHD/cloudera-repos/cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm \ https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm \ && wget -O /root/hadoop_CHD/cloudera-repos/cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm \ https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm \ && wget -O /root/hadoop_CHD/cloudera-repos/cloudera-manager-server-6.3.0-1281944.el7.x86_64.rpm \ https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-6.3.0-1281944.el7.x86_64.rpm \ && wget -O /root/hadoop_CHD/cloudera-repos/cloudera-manager-server-db-2-6.3.0-1281944.el7.x86_64.rpm \ https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-db-2-6.3.0-1281944.el7.x86_64.rpm \ && wget -O /root/hadoop_CHD/cloudera-repos/enterprise-debuginfo-6.3.0-1281944.el7.x86_64.rpm \ https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/enterprise-debuginfo-6.3.0-1281944.el7.x86_64.rpm \ && wget -O /root/hadoop_CHD/cloudera-repos/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm \ https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm \ && ll /root/hadoop_CHD/cloudera-repos
说明:
1、通过wget下载mysql安装十分慢,可以使用window系统的电脑讯雷等下载工具下载后,将安装包放至宿主机,再使用docker cp 下载的rpm {容器ID}:/root/hadoop_CHD/cloudera-repos命令将安装包放至容器中,前提是容器的/root/hadoop_CHD/cloudera-repos目录必须事先创建。
4.7准备Parcel包
mkdir -p /root/hadoop_CHD/parcel \ && wget -O /root/hadoop_CHD/parcel/ CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel \ https://archive.cloudera.com/cdh6/6.3.2/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel \ && wget -O /root/hadoop_CHD/parcel/manifest.json \ https://archive.cloudera.com/cdh6/6.3.2/parcels/manifest.json \ && ll /root/hadoop_CHD/parcel
说明:
1、通过wget下载mysql安装十分慢,可以使用window系统的电脑讯雷等下载工具下载后,将安装包放至宿主机,再使用docker cp 下载的parcel {容器ID}:/root/hadoop_CHD/parcel命令将安装包放至容器中,前提是容器的/root/hadoop_CHD/parcel目录必须事先创建。
2、随着时间推移官网可能会不定期删除测试不稳定的版本,可能导致上述链接无可用的下载资源,可以进入https://archive.cloudera.com/cdh6/找到想要的版本进行下载。
4.8 搭建本地yum源
yum -y install httpd createrepo \ && systemctl start httpd \ && systemctl enable httpd \ && cd /root/hadoop_CHD/cloudera-repos/ && createrepo . \ && mv /root/hadoop_CHD/cloudera-repos /var/www/html/ \ && yum clean all \ && ll /var/www/html/cloudera-repos
4.9安装jdk
cd /var/www/html/cloudera-repos/;rpm -ivh oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
4.10 安装配置MySQL数据库(采用docker独立安装跳过此步)
cd /root/hadoop_CHD/mysql/;tar -xvf mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar \ && yum install -y libaio numactl \ && rpm -ivh mysql-community-common-5.7.27-1.el7.x86_64.rpm \ && rpm -ivh mysql-community-libs-5.7.27-1.el7.x86_64.rpm \ && rpm -ivh mysql-community-client-5.7.27-1.el7.x86_64.rpm \ && rpm -ivh mysql-community-server-5.7.27-1.el7.x86_64.rpm \ && rpm -ivh mysql-community-libs-compat-5.7.27-1.el7.x86_64.rpm \ && echo character-set-server=utf8 >> /etc/my.cnf \ && rm -rf /root/hadoop_CHD/mysql/ \ && yum clean all \ && rpm -qa |grep mysql
4.11数据库授权
( cat <<EOF set password for root@localhost = password('123456Aa.'); grant all privileges on *.* to 'root'@'%' identified by '123456Aa.'; flush privileges; CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456Aa.'; GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456Aa.'; GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456Aa.'; GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456Aa.'; GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '123456Aa.'; GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456Aa.'; GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '123456Aa.'; GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '123456Aa.'; GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456Aa.'; SHOW DATABASES; EOF ) >> /root/c.sql
获取MySQL初始密码
systemctl start mysqld && grep password /var/log/mysqld.log | sed 's/.*\(............\)$/\1/'
执行SQL脚本
mysql -uroot –p
输入查询出的默认密码,然后执行:
source /root/c.sql
4.12配置mysql jdbc驱动
mkdir -p /usr/share/java/ \ && cd /root/hadoop_CHD/mysql-jdbc/;tar -zxvf mysql-connector-java-5.1.48.tar.gz \ && cp /root/hadoop_CHD/mysql-jdbc/mysql-connector-java-5.1.48/mysql-connector-java-5.1.48-bin.jar /usr/share/java/mysql-connector-java.jar \ && rm -rf /root/hadoop_CHD/mysql-jdbc/ \ && ls /usr/share/java/
4.13安装Cloudera Manager
( cat <<EOF [cloudera-manager] name=Cloudera Manager 6.3.0 baseurl=http://172.10.0.2/cloudera-repos/ gpgcheck=0 enabled=1 EOF ) >> /etc/yum.repos.d/cloudera-manager.repo \ && yum clean all \ && yum makecache \ && yum install -y cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server \ && yum clean all \ && rpm -qa | grep cloudera-manager
4.14配置parcel库
cd /opt/cloudera/parcel-repo/;mv /root/hadoop_CHD/parcel/* ./ \ && sha1sum CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel | awk '{ print $1 }' > CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha \ && rm -rf /root/hadoop_CHD/parcel/ \ && chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/* \ && ll /opt/cloudera/parcel-repo/
4.15初始化scm库
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm 123456Aa.
4.16启动cloudera-server服务
systemctl start cloudera-scm-server \ && sleep 2 \ && tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log | grep "INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server"
五、配置CDH的worker节点
以下为worker容器的准备方式,若为多个时,重复执行以下步骤,创建多个worker节点。
5.1创建多个worker容器
创建2个work容器,
Worker-1:
docker run -d \ --add-host cm.hadoop:172.10.0.2 \ --add-host cdh01.hadoop:172.10.0.3 \ --net hadoop_net \ --ip 172.10.0.3 \ -h cdh01.hadoop \ -p 20022:22 \ --restart always \ --name cdh01.hadoop \ --privileged \ centos7-cdh \ /usr/sbin/init \ && docker ps
说明:
1、上述黄色部分,因创建不同的容器配置而不同,根据实际情况填写。
Worker-2:
docker run -d \ --add-host cm.hadoop:172.10.0.2 \ --add-host cdh02.hadoop:172.10.0.4 \ --net hadoop_net \ --ip 172.10.0.4 \ -h cdh02.hadoop \ -p 30022:22 \ --restart always \ --name cdh02.hadoop \ --privileged \ centos7-cdh \ /usr/sbin/init \ && docker ps
5.2安装基本工具
上一步创建的所有容器均执行,修改root的登录密码改为root
docker exec –it {dockerId} /bin/bash
su root
passwd
root
root
然后执行:
yum install -y kde-l10n-Chinese telnet reinstall glibc-common vim wget ntp net-tools \
&& yum clean all
5.3配置中文环境变量
( cat <<EOF export LC_ALL=zh_CN.utf8 export LANG=zh_CN.utf8 export LANGUAGE=zh_CN.utf8 EOF ) >> ~/.bashrc \ && localedef -c -f UTF-8 -i zh_CN zh_CN.utf8 \ && source ~/.bashrc \ && echo $LANG
5.4配置NTP时间同步服务
vim /etc/ntp.conf
更改为以下四个时钟服务器
server 0.cn.pool.ntp.org server 1.cn.pool.ntp.org server 2.cn.pool.ntp.org server 3.cn.pool.ntp.org
启动ntp服务
systemctl start ntpd && \ systemctl enable ntpd && \ ntpdate -u 0.cn.pool.ntp.org && \ hwclock --systohc && \ date
5.5配置MySQL JDBC
mkdir -p /usr/share/java/ \ && wget -O /usr/share/java/mysql-connector-java-5.1.48.tar.gz \ https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.48.tar.gz \ && cd /usr/share/java/;tar -zxvf mysql-connector-java-5.1.48.tar.gz \ && cp /usr/share/java/mysql-connector-java-5.1.48/mysql-connector-java-5.1.48-bin.jar /usr/share/java/mysql-connector-java.jar \ && rm -rf mysql-connector-java-5.1.48 mysql-connector-java-5.1.48.tar.gz \ && ls /usr/share/java/
说明:
1、通过wget下载安装十分慢,可以使用window系统的电脑讯雷等下载工具下载后,将安装包放至宿主机,再使用docker cp mysql-connector-java-5.1.48.tar.gz {容器ID}:/usr/share/java/命令将安装包放至容器中,前提是容器的/usr/share/java/目录必须事先创建。
5.6修改CM主机的host文件
echo "172.10.0.3 cdh01.hadoop cdh01" >> /etc/hosts echo "172.10.0.4 cdh02.hadoop cdh02" >> /etc/hosts
注意:不是在worker节点上操作,而是在3.9创建的cloud-manager节点上操作,有多少个work容器,执行多少次,内容按真实情况填写即可。
六、CM管理平台创建CDH集群
6.1登陆CM管理平台
http://IP:7180/cmf/login 账号密码:admin/admin
6.2欢迎页
此面一直点击“继续”。
6.3集群安装
欢迎页
创建集群
添加节点
红色部分为: 172.10.0.[2-4]
选择存储
自定义存储库:http://172.10.0.2/cloudera-repos
Jdk安装
SSH凭据,密码为容器root用户的登录密码,此处为root。
安装代理
安装大数据组件
集群状态检查
6.4集群设置
红色为添加3个节点
Datanode-> /dfs/datanode
Namenode-> /dfs/namenode
HDFS检查点-> /dfs/checkpoint
NodeManager 本地目录-> /dfs/nodemanager