kube-ovn 1.12基于bfd探测网关可用性

安装kube-ovn 1.12

docker pull kubeovn/kube-ovn:v1.12.2

git clone https://github.com/kubeovn/kube-ovn.git
cd kube-ovn
git checkout release-1.12
make kind-install

k get node -owide

增加启动参数

kube-ovn-controller增加启动参数

k edit deploy kube-ovn-controller -n kube-system
- --external-gateway-vlanid=100
- --external-gateway-switch=external

kube-ovn-cni增加启动参数

k edit ds kube-ovn-cni -n kube-system
- --external-gateway-switch=external

确认pod已重建

k get pod -n kube-system | grep kube-ovn

underlay.yaml

在kube-ovn中，underlay网络可以是带vlan的虚拟网络，也可以是带vlan的物理网络。
在kind安装的k8s集群中，docker容器作为k8s节点，kube-ovn underlay网关节点中ovnext0网卡需要arping物理网关ip成功后，该oeip才能ready，kube-ovn-controller才会创建bfd。
但是，docker虚拟容器网关无法响应带vlan的arp请求，所以使用logicalGateway作为网关，不再使用k8s节点ip作为underlay网络，即使用带vlan的虚拟网络作为underlay网络。

apiVersion: kubeovn.io/v1
kind: ProviderNetwork
metadata:
  name: external
spec:
  defaultInterface: eth0
---
apiVersion: kubeovn.io/v1
kind: Vlan
metadata:
  name: vlan
spec:
  id: 100
  provider: external
---
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
  name: external
spec:
  protocol: IPv4
  cidrBlock: 10.30.0.0/16
  gateway: 10.30.0.1
  vlan: vlan
  logicalGateway: true

ovn-external-gw-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: ovn-external-gw-config
  namespace: kube-system
data:
  enable-external-gw: "true"
  external-gw-nodes: "kube-ovn-worker,kube-ovn-worker2"
  type: "centralized"
  external-gw-nic: "eth0"
  external-gw-addr: "10.30.0.1/16"

vpc-subnet.yaml

kind: Vpc
apiVersion: kubeovn.io/v1
metadata:
  name: vpc1
spec:
  namespaces:
  - default
  enableExternal: true
  enableBfd: true
---
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
  name: subnet1
spec:
  cidrBlock: 10.31.0.0/24
  default: false
  disableGatewayCheck: false
  disableInterConnection: true
  enableEcmp: true
  gatewayNode: ""
  gatewayType: distributed
  natOutgoing: false
  private: false
  protocol: IPv4
  provider: ovn
  vpc: vpc1
  namespaces:
  - default

oeip.yaml

lsp是在ovn基于bfd的ecmp静态路由场景中，网关节点上提供一个ovs internal port作为ecmp路由的下一跳。

kind: OvnEip
apiVersion: kubeovn.io/v1
metadata:
  name: kube-ovn-worker
spec:
  externalSubnet: external
  type: lsp
---
kind: OvnEip
apiVersion: kubeovn.io/v1
metadata:
  name: kube-ovn-worker2
spec:
  externalSubnet: external
  type: lsp

bfd探测

k ko nbctl show vpc1

k ko nbctl lr-route-list vpc1

k get oeip

k ko nbctl list bfd

bfd探测的是ovn0网关节点上ovnext ns中ovnext0网卡，ip地址是lsp ip。

ip netns exec ovnext tcpdump -i ovnext0 -ne

ip netns exec ovnext ip a s ovnext0

tcpdump -i br-external -ne | grep -E "BFD"

根据上面geneve报文分析网关探测方式。

参考资料

https://kubeovn.github.io/docs/v1.12.x/advance/ovn-l3-ha-based-ecmp-with-bfd/