18.flannel的配置

Kubernetes网络通信： 
    (1) 容器间通信：同一个Pod内的多个容器间的通信, lo 
    (2) Pod通信：Pod IP <--> Pod IP 
    (3) Pod与Service通信：PodIP <--> ClusterIP 
    (4) Service与集群外部客户端的通信； 

CNI:  
    flannel 
    calico 
    canel 
    kube-router 
    ... 
     
    解决方案： 
        虚拟网桥 
        多路复用：MacVLAN 
        硬件交换：SR-IOV 
[root@master ~]# cat /etc/cni/net.d/10-flannel.conflist
{
  "name": "cbr0",
  "plugins": [
    {
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}

    flannel： 
        支持多种后端： 
            VxLAN 
                (1) vxlan 
                (2) Directrouting 
            host-gw: Host Gateway   #不推荐，只能在二层网络中，不支持跨网络，如果有成千上万的Pod，容易产生广播风暴
            UDP:  性能差
             
        flannel的配置参数： 
            Network：flannel使用的CIDR格式的网络地址，用于为Pod配置网络功能； 
                10.244.0.0/16 -> 
                    master: 10.244.0.0/24 
                    node01: 10.244.1.0/24 
                    ... 
                    node255: 10.244.255.0./24 
                     
                10.0.0.0/8 
                    10.0.0.0/24 
                    ... 
                    10.255.255.0/24 
                     
            SubnetLen：把Network切分子网供各节点使用时，使用多长的掩码进行切分，默认为24位； 
             
            SubnetMin：10.244.10.0/24 
             
            SubnetMax: 10.244.100.0/24 
             
            Backend：vxlan, host-gw, udp 
                vxlan:  

 flannel Pod间跨主机通信原理抓包[root@master ~]# ip rdefault via 172.20.0.1 dev ens3 proto dhcp metric 100

10.244.0.0/24 dev cni0 proto kernel scope link src 10.244.0.1 
10.244.1.0/24 via 10.244.1.0 dev flannel.1 onlink 
10.244.2.0/24 via 10.244.2.0 dev flannel.1 onlink 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 
172.20.0.0/16 dev ens3 proto kernel scope link src 172.20.0.91 metric 100 
从路由可以看出 Pod  cni0 -----> flannel.1 -----> 物理网卡  -----> 对端
[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:62:b2:ca brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.91/16 brd 172.20.255.255 scope global noprefixroute dynamic ens3
       valid_lft 2412sec preferred_lft 2412sec
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:40:09:09:b4 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default 
    link/ether 26:e1:60:30:27:c1 brd ff:ff:ff:ff:ff:ff
    inet 10.244.0.0/32 scope global flannel.1
       valid_lft forever preferred_lft forever
5: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    link/ether 0a:58:0a:f4:00:01 brd ff:ff:ff:ff:ff:ff
    inet 10.244.0.1/24 scope global cni0
       valid_lft forever preferred_lft forever
6: veth0f580b07@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
    link/ether a2:98:01:9b:b5:dc brd ff:ff:ff:ff:ff:ff link-netnsid 0
7: vethb8510761@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
    link/ether 1a:00:6b:77:33:fc brd ff:ff:ff:ff:ff:ff link-netnsid 1
113703: vethfc114a8b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
    link/ether fa:ec:f9:ee:42:93 brd ff:ff:ff:ff:ff:ff link-netnsid 3
30029: veth023640f@if30028: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether d2:8a:82:e6:8f:40 brd ff:ff:ff:ff:ff:ff link-netnsid 5
13217: br-973161700d44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:54:2e:ec:14 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-973161700d44
       valid_lft forever preferred_lft forever
13227: vethfdee34f@if13226: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 7e:3e:6a:e2:44:f0 brd ff:ff:ff:ff:ff:ff link-netnsid 2
13229: veth49aa01b@if13228: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-973161700d44 state UP group default 
    link/ether b2:27:38:6e:4d:da brd ff:ff:ff:ff:ff:ff link-netnsid 4

cni0 提供Pod网络共享的地址来源
flannel.1 是对数据包进行 vxlan封装
[root@master ~]# brctl show cni0
bridge name    bridge id        STP enabled    interfaces
cni0        8000.0a580af40001    no        veth0f580b07
                            vethb8510761
                            vethfc114a8b
[root@master ~]# brctl show flannel.1
bridge name    bridge id        STP enabled    interfaces
flannel.1        can't get info Operation not supported
抓包

15:13:26.796845 IP 172.20.0.93.52954 > 172.20.0.76.otv: OTV, flags [I] (0x08), overlay 0, instance 1
IP 10.244.1.175 > 10.244.2.223: ICMP echo request, id 3072, seq 221, length 64