docker(17)：docker跨宿主机通信之overlay
docker(17)：docker跨宿主机通信之overlay
https://www.cnblogs.com/CloudMan6/p/7270551.html
为支持容器跨主机通信，Docker 提供了 overlay driver，使用户可以创建基于 VxLAN 的 overlay 网络。VxLAN 可将二层数据封装到 UDP 进行传输，VxLAN 提供与 VLAN 相同的以太网二层服务，但是拥有更强的扩展性和灵活性
Docerk overlay 网络需要一个 key-value 数据库用于保存网络状态信息，包括 Network、Endpoint、IP 等。Consul、Etcd 和 ZooKeeper 都是 Docker 支持的 key-vlaue 软件，我们这里使用 Consul。
1 准备
docker01 192.168.0.136
docker02 192.168.0.137
我们节省机器在 192.168.0.136 上部署支持的组件，比如 Consul
修改136配置，重启docker
1.1 开启consul
[root@docker-136 ~]# cat /etc/docker/daemon.json { "registry-mirrors": ["https://6ek3bw21.mirror.aliyuncs.com"], "insecure-registries": ["192.168.0.136:5000"], "live-restore":true } [root@docker-136 ~]# systemctl restart docker
docker run -d -p 8500:8500 -h consul --name consul progrium/consul -server –bootstrap
容器启动后，可以通过 http://192.168.0.136:8500 访问 Consul。
1.2 修改主机配置文件
接下来修改 192.168.0.136和192.168.0.137 的 docker daemon 的配置文件/etc/docker/daemon.json
[root@docker-137 ~]# cat /etc/docker/daemon.json { "registry-mirrors": ["https://6ek3bw21.mirror.aliyuncs.com"], "insecure-registries": ["192.168.0.136:5000"], "cluster-store": "consul://192.168.0.136:8500", "cluster-advertise": "192.168.0.137:2376", "hosts":["tcp://0.0.0.0:2376","unix:///var/run/docker.sock"], "live-restore":true } [root@docker-137 ~]# netstat -lntp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6807/sshd tcp 0 0 0.0.0.0:10050 0.0.0.0:* LISTEN 15890/zabbix_agentd tcp6 0 0 :::2376 :::* LISTEN 21099/dockerd tcp6 0 0 :::22 :::* LISTEN 6807/sshd tcp6 0 0 :::10050 :::* LISTEN 15890/zabbix_agentd [root@docker-137 ~]# [root@docker-136 ~]# cat /etc/docker/daemon.json { "registry-mirrors": ["https://6ek3bw21.mirror.aliyuncs.com"], "insecure-registries": ["192.168.0.136:5000"], "cluster-store": "consul://192.168.0.136:8500", "cluster-advertise": "192.168.0.136:2376", "hosts":["tcp://0.0.0.0:2376","unix:///var/run/docker.sock"], "live-restore":true } [root@docker-136 ~]# [root@docker-136 ~]# vim /usr/lib/systemd/system/docker.service [root@docker-136 ~]# systemctl daemon-reload [root@docker-136 ~]# systemctl restart docker [root@docker-136 ~]#
可能会报错解决方案如下
vim /usr/lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd systemctl daemon-reload systemctl restart docker
如果是重启是Consul那台宿主机，请重启 docker restart consul
1.3 consul的页面上查看自动注册情况
2 创建网络启动容器
2.1 创建overlay网络
docker network create -d overlay ol-1
这是全局创建的，在任何一台宿主机创建一次即可。
[root@docker-136 ~]# docker network create -d overlay ol-1 0d896e2f87294e623e20276f00f6d68a0ab54225f52318ca8dc8beeaa83c5f83 [root@docker-136 ~]# docker network ls NETWORK ID NAME DRIVER SCOPE ad706917f112 bridge bridge local 41e6b3ae3e5d host host local 1b816abc98ca macvlan_2 macvlan local 906b0f4bce31 none null local 0d896e2f8729 ol-1 overlay global [root@docker-136 ~]#
这是c/s架构，创建完成后，我们就能远程连接查看另外宿主机的docker镜像和容器了，不需要账户密码。
[root@docker-136 ~]# docker -H 192.168.0.137 images Cannot connect to the Docker daemon at tcp://192.168.0.137:2375. Is the docker daemon running? [root@docker-136 ~]# docker -H 192.168.0.137:2376 images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.0.136:5000/centos-7-ssh-nginx v1 93af120d6d0b 6 days ago 413MB centos latest 9f38484d220f 5 months ago 202MB [root@docker-136 ~]# docker -H 192.168.0.137:2376 ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES bd45fabb6692 192.168.0.136:5000/centos-7-ssh-nginx:v1 "/bin/bash" 5 hours ago Exited (130) 4 hours ago macvlan237 [root@docker-136 ~]#
2.2 启动容器测试
136容器测试
[root@docker-136 ~]# docker run -it --network ol-1 --name web-1 192.168.0.136:5000/centos-7-ssh-nginx:v1 /bin/sh sh-4.2# sh-4.2# sh-4.2# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 10.0.0.2 netmask 255.255.255.0 broadcast 10.0.0.255 ether 02:42:0a:00:00:02 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.18.0.2 netmask 255.255.0.0 broadcast 172.18.255.255 ether 02:42:ac:12:00:02 txqueuelen 0 (Ethernet) RX packets 8 bytes 656 (656.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 sh-4.2# sh-4.2# sh-4.2# sh-4.2# cat /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 10.0.0.2 6ee8323663bd sh-4.2# sh-4.2# sh-4.2# sh-4.2# ping www.baidu.com PING www.a.shifen.com (14.215.177.39) 56(84) bytes of data. 64 bytes from 14.215.177.39 (14.215.177.39): icmp_seq=1 ttl=53 time=38.9 ms 64 bytes from 14.215.177.39 (14.215.177.39): icmp_seq=2 ttl=53 time=37.7 ms 64 bytes from 14.215.177.39 (14.215.177.39): icmp_seq=3 ttl=53 time=37.2 ms ^C --- www.a.shifen.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 37.299/37.985/38.937/0.712 ms sh-4.2# ping 192.168.0.136 PING 192.168.0.136 (192.168.0.136) 56(84) bytes of data. 64 bytes from 192.168.0.136: icmp_seq=1 ttl=64 time=0.067 ms 64 bytes from 192.168.0.136: icmp_seq=2 ttl=64 time=0.054 ms ^C --- 192.168.0.136 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 999ms rtt min/avg/max/mdev = 0.054/0.060/0.067/0.010 ms sh-4.2# ping 192.168.0.137 PING 192.168.0.137 (192.168.0.137) 56(84) bytes of data. 64 bytes from 192.168.0.137: icmp_seq=1 ttl=63 time=1.03 ms 64 bytes from 192.168.0.137: icmp_seq=2 ttl=63 time=560 ms ^C --- 192.168.0.137 ping statistics --- 3 packets transmitted, 2 received, 33% packet loss, time 2002ms rtt min/avg/max/mdev = 1.037/280.851/560.665/279.814 ms sh-4.2# sh-4.2# sh-4.2# ping web-2 PING web-2 (10.0.0.3) 56(84) bytes of data. 64 bytes from web-2.ol-1 (10.0.0.3): icmp_seq=1 ttl=64 time=37.3 ms 64 bytes from web-2.ol-1 (10.0.0.3): icmp_seq=2 ttl=64 time=21.2 ms ^C --- web-2 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1003ms rtt min/avg/max/mdev = 21.293/29.300/37.307/8.007 ms sh-4.2# cat /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 10.0.0.2 6ee8323663bd sh-4.2#
137容器测试
[root@docker-137 ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 33072e1bee88 192.168.0.136:5000/centos-7-ssh-nginx:v1 "/bin/bash" 51 seconds ago Up 46 seconds 80/tcp web-3 5108d66f37c8 192.168.0.136:5000/centos-7-ssh-nginx:v1 "/bin/sh" 9 minutes ago Up 5 minutes 80/tcp web-2 bd45fabb6692 192.168.0.136:5000/centos-7-ssh-nginx:v1 "/bin/bash" 5 hours ago Exited (130) 4 hours ago macvlan237 [root@docker-137 ~]# [root@docker-137 ~]# docker inspect web-2 …… "Networks": { "ol-1": { "IPAMConfig": null, "Links": null, "Aliases": [ "5108d66f37c8" ], "NetworkID": "0d896e2f87294e623e20276f00f6d68a0ab54225f52318ca8dc8beeaa83c5f83", "EndpointID": "79cec2a2e05d60eae837f892d74c87204a55a38f131d26a61d772eb1b270d642", "Gateway": "", "IPAddress": "10.0.0.3", "IPPrefixLen": 24, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:0a:00:00:03", "DriverOpts": null } [root@docker-137 ~]# docker run -it --network ol-1 --name web-3 192.168.0.136:5000/centos-7-ssh-nginx:v1 /bin/bash [root@33072e1bee88 /]# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 10.0.0.4 netmask 255.255.255.0 broadcast 10.0.0.255 ether 02:42:0a:00:00:04 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.18.0.3 netmask 255.255.0.0 broadcast 172.18.255.255 ether 02:42:ac:12:00:03 txqueuelen 0 (Ethernet) RX packets 6 bytes 516 (516.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@33072e1bee88 /]# ping web-1 PING web-1 (10.0.0.2) 56(84) bytes of data. 64 bytes from web-1.ol-1 (10.0.0.2): icmp_seq=1 ttl=64 time=17.3 ms 64 bytes from web-1.ol-1 (10.0.0.2): icmp_seq=2 ttl=64 time=0.340 ms ^C --- web-1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1002ms rtt min/avg/max/mdev = 0.340/8.826/17.313/8.487 ms [root@33072e1bee88 /]# ping web-2 PING web-2 (10.0.0.3) 56(84) bytes of data. 64 bytes from web-2.ol-1 (10.0.0.3): icmp_seq=1 ttl=64 time=0.179 ms 64 bytes from web-2.ol-1 (10.0.0.3): icmp_seq=2 ttl=64 time=0.101 ms ^C --- web-2 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 0.101/0.140/0.179/0.039 ms [root@33072e1bee88 /]# ping www.baidu.com PING www.a.shifen.com (14.215.177.39) 56(84) bytes of data. ^C64 bytes from 14.215.177.39: icmp_seq=1 ttl=53 time=37.8 ms --- www.a.shifen.com ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 37.892/37.892/37.892/0.000 ms [root@33072e1bee88 /]# ping 192.168.0.136 PING 192.168.0.136 (192.168.0.136) 56(84) bytes of data. 64 bytes from 192.168.0.136: icmp_seq=1 ttl=63 time=1.13 ms 64 bytes from 192.168.0.136: icmp_seq=2 ttl=63 time=0.529 ms ^C --- 192.168.0.136 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1002ms rtt min/avg/max/mdev = 0.529/0.829/1.130/0.301 ms [root@33072e1bee88 /]# ping 192.168.0.137 PING 192.168.0.137 (192.168.0.137) 56(84) bytes of data. 64 bytes from 192.168.0.137: icmp_seq=1 ttl=64 time=0.106 ms 64 bytes from 192.168.0.137: icmp_seq=2 ttl=64 time=0.116 ms 64 bytes from 192.168.0.137: icmp_seq=3 ttl=64 time=0.115 ms ^C --- 192.168.0.137 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.106/0.112/0.116/0.009 ms [root@33072e1bee88 /]#
