1) 拉下来 ES集群 spark集群 两套快速部署环境， 并只用docker跑起来，并保存到私库。
2）弄清楚怎么样打包 linux镜像（或者说制作）。
3）试着改一下，让它们跑在集群里面。
4) 弄清楚
Dockerfile 怎么制作镜像
docker-compose 里面的启动项 及 与 mesos里面怎么对应起来。
5）写一个spack程序
及在ES环境里造少量数据查一下。
ES 环境docker :参考贴子来跑的，https://cloud.tencent.com/developer/article/1098820
记录下主要步骤：
1）拉镜像到私库
2）内网虚拟机环境，禁用其它服务没有就跳过，ip修改，免密，等。
systemctl disable mesos-slave systemctl stop mesos-slave systemctl disable zookeeper systemctl stop zookeeper systemctl disable mesos-master systemctl stop mesos-master systemctl disable marathon systemctl stop marathon systemctl enable docker systemctl start docker systemctl daemon-reload systemctl restart docker
3）从私库拉镜象，不需要每个全拉下来，拉对应的。
docker pull 192.168.1.153:31809/kafka.new.es docker pull 192.168.1.153:31809/zookeeper.new.es docker pull 192.168.1.153:31809/elastic/elasticsearch:5.6.8.new.es docker pull 192.168.1.153:31809/elastic/kibana:5.6.8.new.es docker pull 192.168.1.153:31809/elastic/logstash:5.6.8.new.es docker pull 192.168.1.153:31809/elastic/elasticsearch:5.6.8.new.es
4）
写脚本主要有 cp.test.sh 和 docker.run.sh
cp.test.sh 主要是测试 脚本流程 及 处理免密登陆第一次时需要输入yes问题
4.1
将 cp.test.sh 放在任意一台 ，将自己发送给所有节点，在每台这样操作，输入yes.
再发一次确保没漏
4.2
执行 docker.run.sh 启动容器
docker.run.sh主要做和三件事
1）将自己发给其它节点
2）处理自己 相关docker 的启动
3）通过ssh 向其它节点发送命令 在其它节点执行 docker.run.sh(当然这时根据参数是不会再次shh其它节点)
总之就是 docker.run.sh 将会停止和删除所有节点中原来的所有容器，并且启动该启动的容器。
另外附上相关操作脚本。
ip有修改
-------------------公共库--------------------------------------- docker pull jenkins docker tag jenkins 192.168.1.153:31809/jenkins:latest docker push 192.168.1.153:31809/jenkins:latest docker rmi jenkins docker rmi 192.168.1.153:31809/jenkins docker pull mysql docker tag mysql 192.168.1.153:31809/mysql:latest docker push 192.168.1.153:31809/mysql:latest docker rmi mysql docker rmi 192.168.1.153:31809/mysql docker pull tomcat docker tag tomcat 192.168.1.153:31809/tomcat:latest docker push 192.168.1.153:31809/tomcat:latest docker rmi tomcat docker rmi 192.168.1.153:31809/tomcat docker pull maven docker tag maven 192.168.1.153:31809/maven:latest docker push 192.168.1.153:31809/maven:latest docker rmi maven docker rmi 192.168.1.153:31809/maven -------------------公共库--------------------------------------- -----------------快速部署ES集群.txt-------------------------- https://cloud.tencent.com/developer/article/1098820 docker pull zookeeper docker tag zookeeper 192.168.1.153:31809/zookeeper.new.es docker push 192.168.1.153:31809/zookeeper.new.es docker rmi zookeeper docker rmi 192.168.1.153:31809/zookeeper.new.es docker pull wurstmeister/kafka docker tag wurstmeister/kafka 192.168.1.153:31809/kafka.new.es docker push 192.168.1.153:31809/kafka.new.es docker rmi wurstmeister/kafka docker rmi 192.168.1.153:31809/kafka.new.es docker pull docker.elastic.co/elasticsearch/elasticsearch:5.6.8 docker tag docker.elastic.co/elasticsearch/elasticsearch:5.6.8 192.168.1.153:31809/elastic/elasticsearch:5.6.8.new.es docker push 192.168.1.153:31809/elastic/elasticsearch:5.6.8.new.es docker rmi docker.elastic.co/elasticsearch/elasticsearch:5.6.8 docker rmi 192.168.1.153:31809/elastic/elasticsearch:5.6.8.new.es docker pull docker.elastic.co/kibana/kibana:5.6.8 docker tag docker.elastic.co/kibana/kibana:5.6.8 192.168.1.153:31809/elastic/kibana:5.6.8.new.es docker push 192.168.1.153:31809/elastic/kibana:5.6.8.new.es docker rmi docker.elastic.co/kibana/kibana:5.6.8 docker rmi 192.168.1..153:31809/elastic/kibana:5.6.8.new.es docker pull docker.elastic.co/logstash/logstash:5.6.8 docker tag docker.elastic.co/logstash/logstash:5.6.8 192.168.1.153:31809/elastic/logstash:5.6.8.new.es docker push 192.168.1.153:31809/elastic/logstash:5.6.8.new.es docker rmi docker.elastic.co/logstash/logstash:5.6.8 docker rmi 192.168.1.153:31809/elastic/logstash:5.6.8.new.es -----------------快速部署ES集群.txt------------------------------ -----------------spark:2.2镜像-------------------------- https://www.cnblogs.com/hongdada/p/9475406.html docker pull singularities/spark docker tag singularities/spark 192.168.1.153:31809/singularities/spark docker push 192.168.1.153:31809/singularities/spark docker rmi singularities/spark docker rmi 192.168.1.153:31809/singularities/spark ----------------- spark:2.2镜像------------------------------
-----------------centos7 镜像--------------------------
https://www.jianshu.com/p/4801bb7ab9e0
docker tag 7 192.168.1.153:31809/centos7
docker push 192.168.1.153:31809/centos7
docker rmi 7
docker rmi 192.168.1.153:31809/centos7
----------------- centos7 镜像------------------------------
-----------------singularities/hadoop 2.8 镜像--------------------------
https://www.jianshu.com/p/4801bb7ab9e0
docker tag singularities/hadoop:2.8 192.168.1.153:31809/singularities/hadoop.2.8
docker push 192.168.1.153:31809/singularities/hadoop.2.8
docker rmi singularities/hadoop:2.8
docker rmi 192.168.1.153:31809/singularities/hadoop.2.8
----------------- singularities/hadoop 2.8 镜像 ------------------------------
--docker images --查看库 curl -X GET http://192.168.1.153:31809/v2/_catalog {"repositories":["nginx"]} --查看库 curl -X GET http://192.168.1.153:31809/v2/_catalog {"name":"nginx","tags":["latest"]}
免密登陆测试脚本
cp.test.sh
flag=$1 getip() { ifconfig|grep 192|awk '{print $2}' } ip=`getip` echo "salf IP:" $ip cpToOtherVM() { if [[ "${flag}" == "y" ]]; then if [[ "${ip}" != "$1" ]]; then scp -r /z-hl-c53cc450-62bf-4b65-b7f2-432e2aae9c62-v5.json $1:/ fi fi } execOtherVmShell() { if [[ "${flag}" == "y" ]]; then if [[ "${ip}" != "$1" ]]; then ssh root@$1 "sh /cp.test.sh" fi fi } echo "copy to" cpToOtherVM "192.168.1.100" cpToOtherVM "192.168.1.101" cpToOtherVM "192.168.1.102" sleep 1 cpToOtherVM "192.168.1.110" cpToOtherVM "192.168.1.111" cpToOtherVM "192.168.1.112" sleep 1 cpToOtherVM "192.168.1.120" cpToOtherVM "192.168.1.121" cpToOtherVM "192.168.1.122" cpToOtherVM "192.168.1.123" sleep 3 echo "exec other" execOtherVmShell "192.168.1.100" execOtherVmShell "192.168.1.101" execOtherVmShell "192.168.1.102" execOtherVmShell "192.168.1.110" execOtherVmShell "192.168.1.111" execOtherVmShell "192.168.1.112" execOtherVmShell "192.168.1.120" execOtherVmShell "192.168.1.121" execOtherVmShell "192.168.1.122" execOtherVmShell "192.168.1.123" echo "exec salf action"
运行容器脚本
docker.run.sh
[root@docker-master3 /]# cat docker.run.sh flag=$1 getip() { ifconfig|grep 192|awk '{print $2}' } ip=`getip` echo "salf IP:" $ip cpToOtherVM() { if [[ "${flag}" == "y" ]]; then if [[ "${ip}" != "$1" ]]; then scp -r /etc/sysctl.conf $1:/etc/sysctl.conf scp -r /docker.run.sh $1:/docker.run.sh fi fi } execOtherVmShell() { if [[ "${flag}" == "y" ]]; then if [[ "${ip}" != "$1" ]]; then ssh root@$1 "docker ps -a |grep 192.168.1. |awk -F ' ' '{print $1}'| xargs -i docker kill {}" echo "stop all docker" sleep 2 ssh root@$1 "docker ps -a |grep 192.168.1. |awk -F ' ' '{print $1}'| xargs -i docker rm {}" echo "rm all docker" sleep 5 ssh root@$1 "sh /docker.run.sh" fi fi } echo "copy to" cpToOtherVM "192.168.1.100" cpToOtherVM "192.168.1.101" cpToOtherVM "192.168.1.102" sleep 1 cpToOtherVM "192.168.1.110" cpToOtherVM "192.168.1.111" cpToOtherVM "192.168.1.112" sleep 1 cpToOtherVM "192.168.1.120" cpToOtherVM "192.168.1.121" cpToOtherVM "192.168.1.122" cpToOtherVM "192.168.1.123" sleep 1 echo "exec salf action" docker ps -a |grep 192.168.1. |awk -F ' ' '{print $1}'| xargs -i docker kill {} sleep 2 docker ps -a |grep 192.168.1. |awk -F ' ' '{print $1}'| xargs -i docker rm {} sleep 3 function runZookeeper() { echo "exec runZookeeper" $1 $2 # 启动 docker run --name zookeeper \ --net=host \ --restart always \ -v /data/zookeeper:/data/zookeeper \ -e ZOO_PORT=2181 \ -e ZOO_DATA_DIR=/data/zookeeper/data \ -e ZOO_DATA_LOG_DIR=/data/zookeeper/logs \ -e ZOO_MY_ID=$2 \ -e ZOO_SERVERS="server.1=192.168.1.100:2888:3888 server.2=192.168.1.101:2888:3888 server.3=192.168.1.102:2888:3888" \ -d 192.168.1.153:31809/zookeeper.new.es sleep 2 } function runKafka() { echo "exec runKafka" $1 $2 # 机器有11块盘，这里都用起来 mkdir -p /data{1..11}/kafka # 启动 docker run --name kafka \ --net=host \ --volume /data1:/data1 \ --volume /data2:/data2 \ --volume /data3:/data3 \ --volume /data4:/data4 \ --volume /data5:/data5 \ --volume /data6:/data6 \ --volume /data7:/data7 \ --volume /data8:/data8 \ --volume /data9:/data9 \ --volume /data10:/data10 \ --volume /data11:/data11 \ -e KAFKA_BROKER_ID=$2 \ -e KAFKA_PORT=9092 \ -e KAFKA_HEAP_OPTS="-Xms8g -Xmx8g" \ -e KAFKA_HOST_NAME=$1 \ -e KAFKA_ADVERTISED_HOST_NAME=$1 \ -e KAFKA_LOG_DIRS=/data1/kafka,/data2/kafka,/data3/kafka,/data4/kafka,/data5/kafka,/data6/kafka,/data7/kafka,/data8/kafka,/data9/kafka,/data10/kafka,/data11/kafka \ -e KAFKA_ZOOKEEPER_CONNECT="192.168.1.100:2181,192.168.1.101:2181,192.168.1.102:2181" \ -d 192.168.1.153:31809/kafka.new.es sleep 2 } function runMaster() { echo "exec runMaster" $1 $2 #!/bin/bash # 删除已退出的同名容器 #docker ps -a | grep es_master |egrep "Exited|Created" | awk '{print $1}'|xargs -i% docker rm -f % 2>/dev/null # 启动 docker run --name es_master \ -d --net=host \ --restart=always \ --privileged=true \ --ulimit nofile=655350 \ --ulimit memlock=-1 \ --memory=1G \ --memory-swap=-1 \ --cpus=0.5 \ --volume /data:/data \ --volume /etc/localtime:/etc/localtime \ -e TERM=dumb \ -e ES_JAVA_OPTS="-Xms8g -Xmx8g" \ -e cluster.name="iyunwei" \ -e node.name="MASTER-"$2 \ -e node.master=true \ -e node.data=false \ -e node.ingest=false \ -e node.attr.rack="0402-K03" \ -e discovery.zen.ping.unicast.hosts="192.168.1.110:9301,192.168.1.111:9301,192.168.1.112:9301,192.168.1.110:9300,192.168.1.112:9300,192.168.1.113:9300,192.168.1.120:9300,192.168.1.121:9300,192.168.1.122:9300,192.168.1.123:9300" \ -e discovery.zen.minimum_master_nodes=2 \ -e gateway.recover_after_nodes=5 \ -e network.host=0.0.0.0 \ -e transport.tcp.port=9301 \ -e http.port=9201 \ -e path.data="/data/iyunwei/master" \ -e path.logs=/data/elastic/logs \ -e bootstrap.memory_lock=true \ -e bootstrap.system_call_filter=false \ -e indices.fielddata.cache.size="25%" \ 192.168.1.153:31809/elastic/elasticsearch:5.6.8.new.es sleep 2 } function runClient() { echo "exec runClient" $1 $2 #!/bin/bash #docker ps -a | grep es_client |egrep "Exited|Created" | awk '{print $1}'|xargs -i% docker rm -f % 2>/dev/null docker run --name es_client \ -d --net=host \ --restart=always \ --privileged=true \ --ulimit nofile=655350 \ --ulimit memlock=-1 \ --memory=1G \ --memory-swap=-1 \ --cpus=0.5 \ --volume /data:/data \ --volume /etc/localtime:/etc/localtime \ -e TERM=dumb \ -e ES_JAVA_OPTS="-Xms31g -Xmx31g" \ -e cluster.name="iyunwei" \ -e node.name="CLIENT-"$2 \ -e node.master=false \ -e node.data=false \ -e node.attr.rack="0402-K03" \ -e discovery.zen.ping.unicast.hosts="192.168.1.110:9301,192.168.1.111:9301,192.168.1.112:9301,192.168.1.110:9300,192.168.1.112:9300,192.168.1.113:9300,192.168.1.120:9300,192.168.1.121:9300,192.168.1.122:9300,192.168.1.123:9300" \ -e discovery.zen.minimum_master_nodes=2 \ -e gateway.recover_after_nodes=2 \ -e network.host=0.0.0.0 \ -e transport.tcp.port=9300 \ -e http.port=9200 \ -e path.data="/data/iyunwei/client" \ -e path.logs=/data/elastic/logs \ -e bootstrap.memory_lock=true \ -e bootstrap.system_call_filter=false \ -e indices.fielddata.cache.size="25%" \ 192.168.1.153:31809/elastic/elasticsearch:5.6.8.new.es sleep 2 } function runDATA () { echo "exec runDATA" $1 $2 #!/bin/bash #docker ps -a | grep es_data |egrep "Exited|Created" | awk '{print $1}'|xargs -i% docker rm -f % 2>/dev/null docker run --name es_data \ -d --net=host \ --restart=always \ --privileged \ --ulimit nofile=655350 \ --ulimit memlock=-1 \ --volume /data:/data \ --volume /data1:/data1 \ --volume /data2:/data2 \ --volume /data3:/data3 \ --volume /data4:/data4 \ --volume /data5:/data5 \ --volume /data6:/data6 \ --volume /data7:/data7 \ --volume /data8:/data8 \ --volume /data9:/data9 \ --volume /data10:/data10 \ --volume /data11:/data11 \ --volume /etc/localtime:/etc/localtime \ --ulimit memlock=-1 \ -e TERM=dumb \ -e ES_JAVA_OPTS="-Xms31g -Xmx31g" \ -e cluster.name="iyunwei" \ -e node.name="DATA-"$2 \ -e node.master=false \ -e node.data=true \ -e node.ingest=false \ -e node.attr.rack="0402-Q06" \ -e discovery.zen.ping.unicast.hosts="192.168.1.110:9301,192.168.1.111:9301,192.168.1.112:9301,192.168.1.110:9300,192.168.1.112:9300,192.168.1.113:9300,192.168.1.120:9300,192.168.1.121:9300,192.168.1.122:9300,192.168.1.123:9300" \ -e discovery.zen.minimum_master_nodes=2 \ -e gateway.recover_after_nodes=2 \ -e network.host=0.0.0.0 \ -e http.port=9200 \ -e path.data="/data1/iyunwei/data,/data2/iyunwei/data,/data3/iyunwei/data,/data4/iyunwei/data,/data5/iyunwei/data,/data6/iyunwei/data,/data7/iyunwei/data,/data8/iyunwei/data,/data9/iyunwei/data,/data10/iyunwei/data,/data11/iyunwei/data,/data12/iyunwei/data" \ -e path.logs=/data/elastic/logs \ -e bootstrap.memory_lock=true \ -e bootstrap.system_call_filter=false \ -e indices.fielddata.cache.size="25%" \ 192.168.1.153:31809/elastic/elasticsearch:5.6.8.new.es sleep 2 } function runKibana() { echo "exec runKibana" $1 $2 #!/bin/bash #docker ps -a | grep kibana | egrep "Exited|Create" | awk '{print $1}'|xargs -i% docker rm -f % 2>/dev/null docker run --name kibana \ --restart=always \ -d --net=host \ -v /data:/data \ -v /etc/localtime:/etc/localtime \ --privileged \ -e TERM=dumb \ -e SERVER_HOST=0.0.0.0 \ -e SERVER_PORT=5601 \ -e SERVER_NAME=Kibana-$2 \ -e ELASTICSEARCH_URL=http://localhost:9200 \ -e ELASTICSEARCH_USERNAME=elastic \ -e ELASTICSEARCH_PASSWORD=changeme \ -e XPACK_MONITORING_UI_CONTAINER_ELASTICSEARCH_ENABLED=false \ -e LOG_FILE=/data/elastic/logs/kibana.log \ 192.168.1.153:31809/elastic/kibana:5.6.8.new.es sleep 2 } function runLogstash() { echo "exec runLogstash" $1 $2 #!/bin/bash #docker ps -a | grep logstash |egrep "Exited|Created" | awk '{print $1}'|xargs -i% docker rm -f % 2>/dev/null docker run --name logstash \ -d --net=host \ --restart=always \ --privileged \ --ulimit nofile=655350 \ --ulimit memlock=-1 \ -e ES_JAVA_OPTS="-Xms16g -Xmx16g" \ -e TERM=dumb \ --volume /etc/localtime:/etc/localtime \ --volume /data/elastic/config:/usr/share/logstash/config \ --volume /data/elastic/config/pipeline:/usr/share/logstash/pipeline \ --volume /data/elastic/logs:/usr/share/logstash/logs \ 192.168.1.153:31809/elastic/logstash:5.6.8.new.es sleep 2 } function cfgkafka() { if [[ "${ip}" = "$1" ]]; then echo "exec cfgkafka" $1 $2 mkdir -p /data/zookeeper runZookeeper $1 $2 runKafka $1 $2 fi } function cfgMaster() { if [[ "${ip}" = "$1" ]]; then echo "exec cfgMaster" $1 $2 mkdir -p /data/iyunwei/master chown -R 1000:1000 /data/iyunwei mkdir -p /data/iyunwei/client chown -R 1000:1000 /data/iyunwei runMaster $1 $2 runClient $1 $2 runKibana $1 $2 runLogstash $1 $2 fi } function cfgDATA() { if [[ "${ip}" = "$1" ]]; then echo "exec cfgDATA" $1 $2 mkdir -p /data{1..12}/iyunwei/data chown -R 1000:1000 /data{1..12}/iyunwei runDATA $1 $2 fi } index=0 for kafkaClusterIP in "192.168.1.100" "192.168.1.101" "192.168.1.102" do index=$(($index+1)) echo "cfgkafka" $kafkaClusterIP $index cfgkafka $kafkaClusterIP $index done #Master MasterIndex=0 for MasterIP in in "192.168.1.110" "192.168.1.111" "192.168.1.112" do MasterIndex=$(($MasterIndex+1)) echo "cfgMaster" $MasterIP $MasterIndex cfgMaster $MasterIP $MasterIndex done #DATA DATAIndex=0 for DATAIP in "192.168.1.120" "192.168.1.121" "192.168.1.122" "192.168.1.123" do DATAIndex=$(($DATAIndex+1)) echo "cfgDATA" $DATAIP $DATAIndex cfgDATA $DATAIP $DATAIndex done sleep 3 echo "exec other vm action" execOtherVmShell "192.168.1.100" execOtherVmShell "192.168.1.101" execOtherVmShell "192.168.1.102" execOtherVmShell "192.168.1.110" execOtherVmShell "192.168.1.111" execOtherVmShell "192.168.1.112" execOtherVmShell "192.168.1.120" execOtherVmShell "192.168.1.121" execOtherVmShell "192.168.1.122" execOtherVmShell "192.168.1.123" curl -XPUT http://192.168.1.111:9200/_license?acknowledge=true -d @z-hl-c53cc450-62bf-4b65-b7f2-432e2aae9c62-v5.json -uelastic:changeme
内核参数 cat /etc/sysctl.conf
vm.max_map_count = 655360
vm.swappiness = 1
ip:规划与原贴不同
为保证通信方便，不组建复杂网络环境，全部放在同一网段。
192.168.1.100-102 相同为kafka
192.168.1.110-112 对应 192.168.2.100-102
192.168.1.120-123 对应 192.168.3.100-103
现在，容器布署成功
kafka镜像配置文件有权限问题
es-master主面kibana 可访问 但登不进云
http://192.168.1.110:5601/login?next=%2F#?_g=()
应该是没有注册的问题
curl 命令注册时端口不能访问，需要看一下
容器 ps -a 全部者在
容器 ps kafka没有运行
因为ip和原镜像的ip不一样，有可能要进入容器里面去拿配置文件，修改对应ip
拿出来后还要重新打镜象才行，目前不了解要哪些配置要改
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.1.151 docker-slave1 docker-slave1.com 192.168.1.152 docker-slave2 docker-slave2.com 192.168.1.153 docker-slave3 docker-slave3.com 192.168.1.161 docker-master1 docker-master1.com 192.168.1.162 docker-master2 docker-master3.com 192.168.1.163 docker-master3 docker-master3.com 192.168.1.110 es-master1 es-master1.com 192.168.1.111 es-master2 es-master2.com 192.168.1.112 es-master3 es-master3.com 192.168.1.110 es-kafka1 es-kafka1.com 192.168.1.101 es-kafka2 es-kafka2.com 192.168.1.102 es-kafka3 es-kafka3.com 192.168.1.120 es-data1 es-data1.com 192.168.1.121 es-data2 es-data2.com 192.168.1.122 es-data3 es-data3.com 192.168.1.124 es-data4 es-data4.com
