docker搭建hadoop2.8.5

1.在宿主机上下载好安装包:hadoop-2.8.5.tar.gz  ; jdk1.8.0_131.tar.gz共两个包

2.在Docker仓库中拉去镜像: 

               docker pull centos

3.创建容器: 

              docker run -i -t -d --name centos_hdp centos:centos

4.从宿主机拷贝hadoop-2.8.5.tar.gz  、 jdk1.8.0_131.tar.gz两个包到容器中:

             docker cp jdk-8u131-linux-x64.tar.gz  centos_hdp:/usr/local

             docker cp hadoop-2.8.5.tar.gz centos_hdp:/opt/

5.进入后台运行的容器中:

           docker exec -it centos_hdp bash

           tar -zxvf /usr/local/jdk1.8.0_131.tar.gz -C /usr/local/

           tar -zxvf /opt/hadoop-2.8.5.tar.gz -C /opt/

           vim /etc/proflie

         source /etc/profile

                 vim  /opt/hadoop-2.8.5/etc/hadoop/core-site.xml

  

              vim  /opt/hadoop-2.8.5/etc/hadoop/hdfs-site.xml

            vim /opt/hadoop-2.8.5/etc/hadoop/mapred-site.xml

         vim /opt/hadoop-2.8.5/etc/hadoop/yarn-site.xml

          vim /opt/hadoop-2.8.5/etc/hadoop/hadoop-env.sh

          cat /opt/hadoop-2.8.5/etc/hadoop/slave

6.退出容器,把当前容器打包成镜像:

                docker commit -a "mayunzhen" -m "hadoop base images" centos_hdp centos_hdp:2.8.5

7.导出镜像成jar包

               docker save centos_hdp:2.8.5 -o centos_hdp.jar 

8.centos_hdp.jar镜像包传到其他宿主机上

               scp centos_hdp.jar root@192.168.130.166:/

               scp centos_hdp.jar root@192.168.130.167:/

               scp centos_hdp.jar root@192.168.130.168:/

9.分别在三台(192.168.130.166,192.168.130.167,192.168.130.168)宿主机上执行导入镜像

              docker load -i centos_hdp.jar

10.在每个宿主机上,配置docker weave ,让不同宿主机上的容器能通信

         详情见链接:https://www.cnblogs.com/kevingrace/p/6859173.html

11.运行hadoop容器命令:

       (192.168.130.166主机上)docker run  -itd  -h hadoop-master --name hadoop-master  --net=hadoop -v /etc/localtime:/etc/localtime:ro  -p 50070:50070 -p 8088:8088 -p 9000:9000 iammayunzhen/hadoop:2.8.52

                                                    weave attach 192.168.1.11/24 hadoop-master

       (192.168.130.167主机上)docker run  -itd  -h hadoop-slave1 --name hadoop-slave1 --net=hadoop -v /etc/localtime:/etc/localtime:ro  iammayunzhen/hadoop:2.8.52

                                                    weave attach 192.168.1.12/24 hadoop-slave1

       (192.168.130.168主机上)docker run  -itd  -h hadoop-slave2 --name hadoop-slave2 --net=hadoop  -v /etc/localtime:/etc/localtime:ro  iammayunzhen/hadoop:2.8.52

                                                    weave attach 192.168.1.13/24 hadoop-slave2

        测试三个容器IP(192.168.1.1,192.168.1.2,192.168.1.3)互相能通信:

12.分别在三个容器(192.168.1.1,192.168.1.2,192.168.1.3)上配置/etc/hosts

 

13.分别在三个容器(192.168.1.1,192.168.1.2,192.168.1.3)上配置无密码登录

        详情:https://www.cnblogs.com/shuochen/p/10441455.html

14.master容器中执行启动hadoop

         source /etc/profile

        start-all.sh

15.查看集群状态:

 

 

 

 

 

 

制作好的docker hadoop:2.8.5镜像已经上传到https://hub.docker.com/repository/docker/iammayunzhen/centos_hdp(或者可 docker pull  iammayunzhen/centos_hdp:2.8.5进行下载),有需要,可供大家下载,参考。

Refereces:

docker hadoop镜像制作:https://www.jianshu.com/p/bf76dfedef2f

docker容器时间和宿主机同步:https://www.cnblogs.com/kevingrace/p/5570597.html 

多宿主机容器之间通信:https://www.cnblogs.com/kevingrace/p/6859173.html

docker容器中IP无法ssh登录:http://blog.chinaunix.net/uid-26168435-id-5732463.html

安装netstat命令:yum install net-tools

查看端口情况:netstat -tulnp |grep 22

安装passwd命令:yum install -y passwd

多容器IP无密码登录:https://www.cnblogs.com/shuochen/p/10441455.html 

 yum安装ssh客户端 : yum -y install openssh-clients     https://www.cnblogs.com/nulige/articles/9324564.html

posted @ 2019-12-18 17:49  马云12314  阅读(534)  评论(0编辑  收藏  举报