cali1e4a9cee8dc这是什么东西？
//我们查下k8s node节点，发现很多类似 cali7c620a7a67b 这样的类似网络设备的东西。
//这些是什么呢？
//k8s集群节点ht10，node网络情况。
[root@ht10 calico]# ifconfig
cali1e4a9cee8dc: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 495759 bytes 283173556 (270.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 495759 bytes 283173556 (270.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 //这个就是对应 [root@ht10 calico]# ip a 命令结果里面的网卡编号77 cali23eb546861c: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 cali72e0f3b6f9b: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 33001 bytes 1980958 (1.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 11 bytes 1421 (1.3 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 caliaa64c1a7f09: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 calica7f2079b57: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255 ether 02:42:83:9e:06:86 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.129.51.204 netmask 255.255.255.0 broadcast 10.129.51.255 ether 06:34:a0:00:0b:81 txqueuelen 1000 (Ethernet) RX packets 215955 bytes 65222322 (62.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 110081 bytes 77325192 (73.7 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens224: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.129.53.204 netmask 255.255.255.0 broadcast 10.129.53.255 ether 06:0f:2e:00:0d:03 txqueuelen 1000 (Ethernet) RX packets 33001 bytes 1980958 (1.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 11 bytes 1421 (1.3 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 495759 bytes 283173556 (270.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 495759 bytes 283173556 (270.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
没有安装成k8s节点的普通机器上采样应该就是,去掉calico,docker，那么ifconfig结果如下
[root@普通机器 ~]# ifconfig eth1 Link encap:Ethernet HWaddr 00:50:56:A2:78:01 inet addr:10.129.55.19 Bcast:10.129.55.255 Mask:255.255.255.0 inet6 addr: fe80::250:56ff:fea2:7801/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2197601980 errors:0 dropped:0 overruns:0 frame:0 TX packets:1940531676 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:848519208193 (790.2 GiB) TX bytes:815714904879 (759.6 GiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:46657 errors:0 dropped:0 overruns:0 frame:0 TX packets:46657 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2622944 (2.5 MiB) TX bytes:2622944 (2.5 MiB) tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.8.0.1 P-t-P:10.8.0.2 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:52063234 errors:0 dropped:0 overruns:0 frame:0 TX packets:56440262 errors:0 dropped:747 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:3556025519 (3.3 GiB) TX bytes:14467954568 (13.4 GiB)
//这个tun0可有可无。
我们分析如下：
1、centos7的网卡命名已经变成了 ens* 类似的， 不再是centos6的 eth* 这样的方式
我们的服务器基本都是eth0,eth1,ens* 这种形式的。 当然有的机器可能有网卡信息，但是对应配置文件没有情况也非常常见。
2、ens192和ens224是两个网卡，即该机器有两个实际网卡
3、tunl0，百度一下可以看到是ipip隧道, 就是calico利用tunl0来进行通讯，同时安装calico的时候分配ip.
[root@ht22 k8snode]# modinfo ipip
filename: /lib/modules/3.10.0-1160.45.1.el7.x86_64/kernel/net/ipv4/ipip.ko.xz
alias: netdev-tunl0
alias: rtnl-link-ipip
license: GPL
retpoline: Y
rhelversion: 7.9
srcversion: 8032CC3EDB2F63D42025A07
depends: ip_tunnel,tunnel4
intree: Y
vermagic: 3.10.0-1160.45.1.el7.x86_64 SMP mod_unload modversions
signer: CentOS Linux kernel signing key
sig_key: F5:EF:E0:5F:FF:E7:34:59:FB:55:F6:1B:3A:BD:2F:18:1F:8E:51:A0
sig_hashalgo: sha256
parm: log_ecn_error:Log packets received with corrupted ECN (bool)
4、lo 本地回环接口（网络接口，注意区别loop 叫回环设备），这个是针对网络通讯而言
5、docker0,在docker安装之后,就会产生一个docker0的虚拟网桥
ip a 方式查看网卡编号
[root@ht10 calico]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 06:34:a0:00:0b:81 brd ff:ff:ff:ff:ff:ff
inet 10.129.51.204/24 brd 10.129.51.255 scope global ens192
valid_lft forever preferred_lft forever
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 06:0f:2e:00:0d:03 brd ff:ff:ff:ff:ff:ff
inet 10.129.53.204/24 brd 10.129.53.255 scope global noprefixroute ens224
valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:83:9e:06:86 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
71: cali23eb546861c@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 3
75: tunl0@NONE: <NOARP> mtu 1480 qdisc noqueue state DOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
76: calica7f2079b57@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0
77: cali1e4a9cee8dc@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 1
78: caliaa64c1a7f09@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 2
79: cali72e0f3b6f9b@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 4
注意： link-netnsid 为跨网络接口设置对等网络标识
单独看下这个设备，对于这个设备来说，这其实是一个veth peer：
[root@ht5 calico]# ip -d link show dev cali7c620a7a67b
119: cali7c620a7a67b@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 3 promiscuity 0
veth addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
我们看到最后一行有veth字样，代表这是一个veth设备，对这个@if4来说，这表示 link’s peer 端对端端口的索引。尽管这个属性似乎对任何接口都可用，但它只适用于几种接口类型:veth、macvlan、vlan(子接口)，表示和另一个接口的关系。
我们下面做个实验，看下pod对应的宿主机的网卡是什么，宿主机上创建容器(pod里的)对应网卡是什么？
pod是通过master或dashborad等创建的。
1、master上执行操作
[root@master-38 ~]# kubectl get pods -n cc -o wide NAME READY STATUS RESTARTS AGE IP NODE mdp-cc-85fd974f8c-j2q8v 1/1 Running 0 3h 172.17.76.28 ht10 //没有加命名空间，所以出错 [root@master-38 ~]# kubectl exec mdp-cc-85fd974f8c-j2q8v -it -- /bin/bash Error from server (NotFound): pods " p-cc-85fd974f8c-j2q8v" not found //加上命名空间，进入pod [root@master-38 ~]# kubectl exec mdp-cc-85fd974f8c-j2q8v -it -n cc -- /bin/bash root@mdp-cc-85fd974f8c-j2q8v:/# ip a //进入pod里面 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.0 4: eth0@if77: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 0e:62:0d:13:27:03 brd ff:ff:ff:ff:ff:ff inet 172.17.76.28/32 scope global eth0 valid_lft forever preferred_lft forever //注意77就是宿主机网卡设备的编号
2、现在我们登录宿主机ht10这台机器。
[root@ht10 net.d]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 06:34:a0:00:0b:81 brd ff:ff:ff:ff:ff:ff inet 10.129.51.204/24 brd 10.129.51.255 scope global ens192 valid_lft forever preferred_lft forever 3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 06:0f:2e:00:0d:03 brd ff:ff:ff:ff:ff:ff inet 10.129.53.204/24 brd 10.129.53.255 scope global noprefixroute ens224 valid_lft forever preferred_lft forever 4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:83:9e:06:86 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever 71: cali23eb546861c@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 3 75: tunl0@NONE: <NOARP> mtu 1480 qdisc noqueue state DOWN group default qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.0 76: calica7f2079b57@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0 77: cali1e4a9cee8dc@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 1 //pod这个就是对应的网络设备 link-netnsid 1：所在网络命名空间的id为1，也就是对端在netnsid为1的网络命名空间里 78: caliaa64c1a7f09@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 2 79: cali72e0f3b6f9b@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 4 //执行ip a 或者ip addr 命令之后，经过前面有编号的，就不是很乱了，cali7c620a7a67b
//link-netnsid 为跨网络接口设置对等网络标识
3、docker容器和网卡的对应关系:
[root@ht10 net.d]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 22da71747c54 registry.aaa.com.cn/cc/test/mdp-cc "java -jar /mdp-cc…" 18 minutes ago Up 18 minutes
容器 id 是 22da71747c54，进入容器
[root@ht10 net.d]# docker exec -it 22da71747c54 /bin/bash
root@mdp-cc-85fd974f8c-j2q8v:/# cat /sys/class/net/eth0/iflink //容器内输入命令查看 77 这里我们看到是77。
