OpenShift和F5的集成手册
OpenShift和F5的集成步骤，记录如下，如实际操作中有变更会再度编辑修改。
1.整体架构
使用BIG-IP作为Openshift的Router，能实现以下功能：
- 为Services创建BIG-IP本地流量规则
- 提供HTTP/HTTPS路由
- 为Route资源添加BIG-IP健康检查
本文步骤是通过F5替换OpenShift中的Router,整体的架构如下
2.安装步骤
为BIG-IP设备建立节点
- 在OCP集群中建立一个f5-kctlr-openshift-hostsubnet.yaml文件，内容如下
apiVersion: v1 kind: HostSubnet metadata: name: f5-bigip-01 annotations: pod.network.openshift.io/fixed-vnid-host: "0" pod.network.openshift.io/assign-subnet: "true" # provide a name for the BIG-IP device's host Node host: f5-bigip-node-01 # Provide an IP address to serve as the BIG-IP VTEP in the OpenShift SDN hostIP: 172.16.1.28
Host为F5的主机名，hostIP为F5的地址
- 建立Host Subnet
oc create -f f5-kctlr-openshift-hostsubnet.yaml hostsubnet "f5-bigip-01" created
验证一下
oc get hostsubnet NAME HOST HOST IP SUBNET f5-big-ip f5-bigip-node 172.16.1.28 10.129.2.0/14
BIG-IP系统设置
需要以管理员或资源管理员身份登录BIG-IP系统
建立VXLAN tunnel
- 建立vxlan profile
create /net tunnels vxlan ose-vxlan flooding-type multipoint
- 建立vxlan tunnel
设置local-address为HostSubnet’s hostip
设置key=0允许设备访问所有的openshift的项目和子网
create /net tunnels tunnel openshift_vxlan key 0 profile ose-vxlan local-address 172.16.1.28
- 生成一个Self IP在VXLAN中
Self IP的地址范围必须落在集群的子网掩码中，可以通过oc get clusternetwork来查看集群的子网掩码
A self IP address is an IP address on the BIG-IPsystem that you associate with a VLAN, to access hosts in that VLAN. By virtue of its netmask, a self IPaddress represents an address space , that is, a range of IP addresses spanning the hosts in the VLAN, rather than a single host address.
Self IP are used for each device. Each BIG-IP has a self-IP in a VLAN which is the IP defined on the interface.
Floating IP are for a cluster. They are VIPs, and this is the IP shared by your cluster members.
create /net self 10.129.2.3/14 allow-service none vlan openshift_vxlan
- 在VXLAN中建立一个浮动的self IP
使用Openshift SDN分配给BIG-IP HostSubnet的子网地址
create /net self 10.129.2.4/14 allow-service none traffic-group traffic-group-1 vlan openshift_vxlan
- 验证BIG IP的对象建立
show /net tunnels tunnel openshift_vxlan show /net running-config self 10.129.2.3/14 show /net running-config self 10.129.2.4/14
- 建立一个partition
create auth partition OpenShift
- 在partition下建立一个virtual server
部署BIG-IP Controller
- 建立service account
oc create serviceaccount bigip-ctlr [-n kube-system] serviceaccount "bigip-ctlr" created
- 建立Cluster role和 Cluster Role binding
# For use in OpenShift clusters apiVersion: v1 kind: ClusterRole metadata: annotations: authorization.openshift.io/system-only: "true" name: system:bigip-ctlr rules: - apiGroups: ["", "extensions"] resources: ["nodes", "services", "endpoints", "namespaces", "ingresses", "routes" ] verbs: ["get", "list", "watch"] - apiGroups: ["", "extensions"] resources: ["configmaps", "events", "ingresses/status"] verbs: ["get", "list", "watch", "update", "create", "patch" ] - apiGroups: ["", "extensions"] resources: ["secrets"] resourceNames: ["<secret-containing-bigip-login>"] verbs: ["get", "list", "watch"] --- apiVersion: v1 kind: ClusterRoleBinding metadata: name: bigip-ctlr-role userNames: - system:serviceaccount:kube-system:bigip-ctlr subjects: - kind: ServiceAccount name: bigip-ctlr roleRef: name: system:bigip-ctlr
oc create -f f5-kctlr-openshift-clusterrole.yaml [-n kube-system] clusterrole "system:bigip-ctlr" created clusterrolebinding "bigip-ctlr-role" created
- 建立Deployment
- --bigip-url 为设备的IP
- --bigip-partition为之前F5下创建的Partition,Openshift
- --route-vserver-addr 为F5对外提供服务的IP
- openshift-sdn-name指向tunnel的名字
apiVersion: extensions/v1beta1 kind: Deployment metadata: name: k8s-bigip-ctlr spec: replicas: 1 template: metadata: name: k8s-bigip-ctlr labels: app: k8s-bigip-ctlr spec: # Name of the Service Account bound to a Cluster Role with the required # permissions serviceAccountName: bigip-ctlr containers: - name: k8s-bigip-ctlr image: "f5networks/k8s-bigip-ctlr" env: - name: BIGIP_USERNAME valueFrom: secretKeyRef: # Replace with the name of the Secret containing your login # credentials name: bigip-login key: username - name: BIGIP_PASSWORD valueFrom: secretKeyRef: # Replace with the name of the Secret containing your login # credentials name: bigip-login key: password command: ["/app/bin/k8s-bigip-ctlr"] args: [ # See the k8s-bigip-ctlr documentation for information about # all config options # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest "--bigip-username=$(BIGIP_USERNAME)", "--bigip-password=$(BIGIP_PASSWORD)", "--bigip-url=10.10.10.10", "--bigip-partition=openshift", "--pool-member-type=cluster", "--openshift-sdn-name=/Common/openshift_vxlan", "--manage-routes=true", "--route-vserver-addr=1.2.3.4", "--route-label="App1" ] imagePullSecrets: - name: f5-docker-images - name: bigip-login
oc create -f f5-k8s-bigip-ctlr_openshift-sdn.yaml [-n kube-system] deployment "k8s-bigip-ctlr" created
- 验证部署成功
oc get pods NAME READY STATUS RESTARTS AGE k8s-bigip-ctlr-1962020886-s31l4 1/1 Running 0 1m
验证
- 建立route
创建项目，创建应用，然后创建Route
apiVersion: v1 kind: Route metadata: labels: name: myService name: myService-route-unsecured annotations: # See the k8s-bigip-ctlr documentation for information about # all Route Annotations # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest/#supported-route-annotations virtual-server.f5.com/balance: least-connections-node spec: host: mysite.example.com path: "/myApp" port: targetPort: 80 to: kind: Service name: myService
在本地hosts文件中添加mysite.example.com到F5的virtual ip,然后浏览器访问
- 创建https route
apiVersion: route.openshift.io/v1 kind: Route metadata: annotations: virtual-server.f5.com/balance: least-connections-node labels: app: f5-test name: f5-test-2 spec: host: f5-tes-2t.example.com tls: insecureEdgeTerminationPolicy: Allow termination: edge to: kind: Service name: f5-test
浏览器访问https://f5-test.example.com，即能看到应用页面。
- Openshit上创建Service后，F5会自动创建新的Pool，Pool里的资源即为Service下的Pod
- 请求到达F5后，F5根据请求的域名，找到对应的Pool，请求直接到达Pod。不会请求集群里的Route。
详细参考
https://clouddocs.f5.com/containers/v2/openshift/kctlr-use-bigip-openshift.html
