一、需求场景：同一个子网（subnet）下，子网内的虚机可以访问外网、虚机之间可以互通
-
Kubernetes >= 1.16
-
Docker >= 1.12.6
-
OS: CentOS 7/8, Ubuntu 16.04/18.04
-
Other Linux distributions with geneve, openvswitch and ip_tables module installed. You can use commands modinfo geneve, modinfo openvswitch and modinfo ip_tables to verify
-
Kernel boot with ipv6.disable=0
-
Kube-proxy MUST be ready so that Kube-OVN can connect to apiserver by service address
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: kube-ovn-att1
namespace: default
spec:
# value：attach的name.namespace,如果type=kube-ovn，需要再加上“.ovn”
config: '{
"cniVersion": "0.3.0",
"type": "kube-ovn",
"server_socket": "/run/openvswitch/kube-ovn-daemon.sock",
"provider": "kube-ovn-att1.default.ovn"
}'
说明：
使用type=kube-ovn，不能使用type=macvlan。因为type=macvlan，虚机之间网络无法互通
官方说明：https://link.zhihu.com/?target=https%3A//kubevirt.io/user-guide/virtual_machines/interfaces_and_networks/
kubectl apply -f att.yaml
kubectl get http://networkattachmentdefinition.k8s.cni.cncf.io -A
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
name: kubeovn-subnet
spec:
protocol: IPv4
provider: kube-ovn-att1.default.ovn
cidrBlock: 172.55.0.0/16
gateway: 172.55.0.1
excludeIps:
- 172.55.0.1..172.55.0.10
说明：
a. provider要和步骤1中attach定义的provider保持一致。虚机为附加网卡分配ip时，就是根据provider关联subnet的。
b. 可用ip域、gateway、排除的ip段均根据实际情况，自行设定
kubectl apply -f kube-ovn-subnet.yaml
kubectl get subnet
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: vm.fedora.use.kubeovnatt.1
annotations:
k8s.v1.cni.cncf.io/networks: kube-ovn-att1
spec:
running: true
template:
metadata:
labels:
kubevirt.io/size: small
kubevirt.io/domain: vm.danny
spec:
domain:
devices:
disks:
- name: containerdisk
disk:
bus: virtio
- name: cloudinitdisk
disk:
bus: virtio
interfaces:
- name: default
bridge: {}
- name: eth1
bridge: {}
resources:
requests:
memory: 1024M
networks:
- name: default
pod: {}
- name: eth1
multus:
networkName: default/kube-ovn-att1
volumes:
- name: containerdisk
containerDisk:
image: kubevirt/fedora-cloud-registry-disk-demo
- name: cloudinitdisk
cloudInitNoCloud:
userData: |
#!/bin/bash
echo "fedora" | passwd fedora --stdin
dhclient eth1
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: vm.fedora.use.kubeovnatt.2
annotations:
k8s.v1.cni.cncf.io/networks: kube-ovn-att1
spec:
running: true
template:
metadata:
labels:
kubevirt.io/size: small
kubevirt.io/domain: vm.danny
spec:
domain:
devices:
disks:
- name: containerdisk
disk:
bus: virtio
- name: cloudinitdisk
disk:
bus: virtio
interfaces:
- name: default
bridge: {}
- name: eth1
bridge: {}
resources:
requests:
memory: 1024M
networks:
- name: default
pod: {}
- name: eth1
multus:
networkName: default/kube-ovn-att1
volumes:
- name: containerdisk
containerDisk:
image: kubevirt/fedora-cloud-registry-disk-demo
- name: cloudinitdisk
cloudInitNoCloud:
userData: |
#!/bin/bash
echo "fedora" | passwd fedora --stdin
dhclient eth1
说明：
fedora：
dhclient eth1 # 虚机启动时，配置并开启附属网卡。否则附属网卡无法完成ip配置。不同操作系统，配置不一样。需要去对应的官网查看
kubectl apply -f use.att.fedora.vm-kubeovnAtt1.yaml
kubectl apply -f use.att.fedora.vm-kubeovnAtt2.yaml
kubectl get vm
kubectl get pod
virtctl console vm.fedora.use.kubeovnatt.1
virtctl console vm.fedora.use.kubeovnatt.2
说明：virtctl需要提前安装
账号密码在定义虚机的yaml中指定（fedora:fedora）
证明attach已经成功分配给虚机
证明attach已经成功分配给虚机
ping 百度能成功返回数据包，证明已通外网
ping 百度能成功返回数据包，证明已通外网
两台虚机通过attach网段相互ping对方都能成功收到数据包，证明虚机之间的attach网络互通
两台虚机通过attach网段相互ping对方都能成功收到数据包，证明虚机之间的attach网络互通
安装过程中可能遇到的问题:
问题一：
error adding container to network "macvlan": DelegateAdd: cannot set "macvlan" interface name to "net1": validateIfName: interface name net1 already exists
解决方案：/etc/cni/net.d/00-multus.conf 自动维护有bug，删除让其自动重建【目前是手工删除。最好通过定时器删除】
参考：https://github.com/k8snetworkplumbingwg/multus-cni/issues/1089
问题二：
附加网卡ping不通同网段机器（attach的type=macvlan）
https://github.com/kubevirt/kubevirt/issues/5483
解决方案：macvlan CNI就是无法互通，attach使用kube-ovn即可
