Cilium Gateway API 特性（转载）

一、环境信息

主机	IP
ubuntu	10.0.0.234

软件	版本
docker	26.1.4
helm	v3.15.0-rc.2
kind	0.18.0
kubernetes	1.23.4
ubuntu os	Ubuntu 22.04.6 LTS
kernel	5.15.0-106

二、`Cilium` `Gateway API` 流程图

Cilium 现在提供完全一致的 Gateway API 实现。 Gateway API 是 Kubernetes 集群中南北负载均衡和流量路由的新标准，并且是 Ingress API 的长期继承者。 Gateway API 代表了流量管理的未来。

GatewaAPI 从设计之初就是为了解决 Ingress API 的局限性

Gateway API 的创建是源于 Ingress API 存在一些局限性：首先，它不提供用户需要定义的高级负载均衡功能。它本身仅支持简单的基于路径的 HTTP 流量请求路由
其次，用户管理变得不切实际：产品供应商通过注释来解决 Ingress API 中功能缺乏的问题。但是注释虽然非常强大，但最终会导致一个 Ingress 与另一个 Ingress 之间出现不一致
第三，由于 Ingress API 是单一 API 资源，因此它操作受限：不适合具有共享负载均衡基础设施的多团队集群

在 Cilium 1.13 中，Cilium Gateway API 通过了所有 Gateway API 一致性测试 (v0.5.1)[https://isovalent.com/blog/post/cilium-release-113/]

参考官方链接

三、`Cilium` `Gateway API` 模式环境搭建

安装须知

配置 Cilium 时，必须启用 NodePort，使用 nodePort.enabled=true 或使用 kubeProxyReplacement 将 kube-proxy 替换为部分或严格。
配置 Cilium 时，必须使用 --enable-l7-proxy 标志（默认已启用）启用 L7 代理。
必须预先安装以下来自 Gateway API v0.5.1 的 CRD。(安装步骤)(https://docs.cilium.io/en/v1.13/network/servicemesh/gateway-api/gateway-api/#prerequisites)
与 Ingress 类似，Gateway API 控制器会创建一个 LoadBalancer 类型的服务，因此您的环境需要支持该服务。

`kind` 配置文件信息

root@KinD:~# cat install.sh

#!/bin/bash
date
set -v

# 1.prep noCNI env
cat <<EOF | kind create cluster --name=cilium-gatewayapi-http --image=kindest/node:v1.23.4 --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  # kind 默认使用 rancher cni，cni 我们需要自己创建
  disableDefaultCNI: true
  # 此处使用 cilium 代替 kube-proxy 功能
  kubeProxyMode: "none"
nodes:
  - role: control-plane
  - role: worker
  - role: worker

containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor.evescn.com"]
    endpoint = ["https://harbor.evescn.com"]
EOF

# 2.install necessary tools
for i in $(docker ps -a --format "table {{.Names}}" | grep cilium) 
do
    echo $i
    docker cp /usr/bin/ping $i:/usr/bin/ping
    docker exec -it $i bash -c "sed -i -e 's/jp.archive.ubuntu.com\|archive.ubuntu.com\|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list"
    docker exec -it $i bash -c "apt-get -y update >/dev/null && apt-get -y install net-tools tcpdump lrzsz bridge-utils >/dev/null 2>&1"
done

安装 k8s 集群

root@KinD:~# ./install.sh

Creating cluster "cilium-gatewayapi-http" ...
 ✓ Ensuring node image (kindest/node:v1.23.4) 🖼
 ✓ Preparing nodes 📦 📦 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing StorageClass 💾 
 ✓ Joining worker nodes 🚜 
Set kubectl context to "kind-cilium-gatewayapi-http"
You can now use your cluster with:

kubectl cluster-info --context kind-cilium-gatewayapi-http

Thanks for using kind! 😊

安装 gateway api

root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.5.1/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.5.1/config/crd/standard/gateway.networking.k8s.io_gateways.yaml
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.5.1/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.5.1/config/crd/experimental/gateway.networking.k8s.io_referencegrants.yaml
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created

安装 cilium 服务

root@KinD:~# cat cilium.sh
#/bin/bash
set -v
controller_node_ip=`kubectl get node -o wide --no-headers | grep -E "control-plane|bpf1" | awk -F " " '{print $6}'`

helm repo add cilium https://helm.cilium.io > /dev/null 2>&1
helm repo update > /dev/null 2>&1

helm install cilium cilium/cilium \
  --set k8sServiceHost=$controller_node_ip \
  --set k8sServicePort=6443 \
  --version 1.13.0-rc5 \
  --namespace kube-system \
  --set debug.enabled=true \
  --set debug.verbose=datapath \
  --set monitorAggregation=none \
  --set ipam.mode=cluster-pool \
  --set cluster.name=cilium-gatewayapi-http \
  --set kubeProxyReplacement=strict \
  --set bpf.masquerade=true \
  --set gatewayAPI.enabled=true

root@KinD:~# bash cilium.sh

--set 参数解释

--set kubeProxyReplacement=strict
- 含义: 启用 kube-proxy 替代功能，并以严格模式运行。
- 用途: Cilium 将完全替代 kube-proxy 实现服务负载均衡，提供更高效的流量转发和网络策略管理。
--set bpf.masquerade
- 含义: 启用 eBPF 功能。
- 用途: 使用 eBPF 实现数据路由，提供更高效和灵活的网络地址转换功能。
--set gatewayAPI.enable=true:
- 启用 Gateway API 支持，Cilium 将支持并管理 Gateway API 资源。

查看安装的服务

root@KinD:~# kubectl get pods -A
NAMESPACE            NAME                                                           READY   STATUS    RESTARTS   AGE
kube-system          cilium-55g65                                                   1/1     Running   0          103s
kube-system          cilium-bb4s7                                                   1/1     Running   0          103s
kube-system          cilium-operator-58959f76d6-46b62                               1/1     Running   0          103s
kube-system          cilium-operator-58959f76d6-92kwx                               1/1     Running   0          103s
kube-system          cilium-r5v9v                                                   1/1     Running   0          103s
kube-system          coredns-64897985d-l5rqq                                        1/1     Running   0          131m
kube-system          coredns-64897985d-qx7bl                                        1/1     Running   0          131m
kube-system          etcd-cilium-gatewayapi-http-control-plane                      1/1     Running   0          132m
kube-system          kube-apiserver-cilium-gatewayapi-http-control-plane            1/1     Running   0          132m
kube-system          kube-controller-manager-cilium-gatewayapi-http-control-plane   1/1     Running   0          132m
kube-system          kube-scheduler-cilium-gatewayapi-http-control-plane            1/1     Running   0          132m
local-path-storage   local-path-provisioner-5ddd94ff66-7t65r                        1/1     Running   0          131m

安装 `metallb` 服务，提供 `LoadBanlencer` 功能

安装 metallb 服务

root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/metallb/metallb/v0.13.10/config/manifests/metallb-native.yaml

namespace/metallb-system created
customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
serviceaccount/controller created
serviceaccount/speaker created
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/controller created
rolebinding.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
configmap/metallb-excludel2 created
secret/webhook-server-cert created
service/webhook-service created
deployment.apps/controller created
daemonset.apps/speaker created
validatingwebhookconfiguration.admissionregistration.k8s.io/metallb-webhook-configuration created

创建 metalLb layer2 的 IPAddressPool

root@KinD:~# cat metallb-l2-ip-config.yaml
---
# metallb 分配给 loadbanlencer 的 ip 地址段定义
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: ippool
  namespace: metallb-system
spec:
  addresses:
  - 172.18.0.200-172.18.0.210

---
# 创建 L2Advertisement 进行 IPAddressPool 地址段
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: ippool
  namespace: metallb-system
spec:
  ipAddressPools:
  - ippool


root@KinD:~# kubectl apply -f metallb-l2-ip-config.yaml
ipaddresspool.metallb.io/ippool unchanged
l2advertisement.metallb.io/ippool created

root@KinD:~# kubectl get ipaddresspool -n metallb-system
NAME     AUTO ASSIGN   AVOID BUGGY IPS   ADDRESSES
ippool   true          false             ["172.18.0.200-172.18.0.210"]
root@KinD:~# kubectl get l2advertisement -n metallb-system
NAME     IPADDRESSPOOLS   IPADDRESSPOOL SELECTORS   INTERFACES
ippool   ["ippool"]

`k8s` 集群安装 `Pod` 和 `Service`

root@KinD:~# cat cni.yaml
---
apiVersion: apps/v1
kind: DaemonSet
#kind: Deployment
metadata:
  labels:
    app: cni
  name: cni
spec:
  #replicas: 1
  selector:
    matchLabels:
      app: cni
  template:
    metadata:
      labels:
        app: cni
    spec:
      containers:
      - image: harbor.dayuan1997.com/devops/nettool:0.9
        name: nettoolbox
        securityContext:
          privileged: true

---
apiVersion: v1
kind: Service
metadata:
  name: cni
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  type: ClusterIP
  selector:
    app: cni

root@KinD:~# kubectl apply -f cni.yaml
daemonset.apps/cni created
service/serversvc created

查看安装服务信息

root@KinD:~# kubectl get pods -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP          NODE                             NOMINATED NODE   READINESS GATES
cni-9c8h5   1/1     Running   0          7s    10.0.2.2    cilium-gatewayapi-http-worker    <none>           <none>
cni-j9bzr   1/1     Running   0          7s    10.0.1.69   cilium-gatewayapi-http-worker2   <none>           <none>

root@KinD:~# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
cni          ClusterIP   10.96.145.109   <none>        80/TCP    20s
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   134m

四、测试 `Gateway` `HTTP` 特性

配置 `Gateway` `HTTP`

root@KinD:~# cat gateway.yamnl
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: basic-gateway
spec:
  gatewayClassName: cilium
  listeners:
  - protocol: HTTP
    port: 80
    name: web-gateway
    allowedRoutes:
      namespaces:
        from: Same
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: http-app-cni
spec:
  parentRefs:
  - name: basic-gateway
    namespace: default
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: cni
      port: 80


root@KinD:~# kubectl apply -f gateway.yaml
gateway.gateway.networking.k8s.io/basic-gateway created
httproute.gateway.networking.k8s.io/http-app-cni created

root@KinD:~# kubectl get svc
NAME                           TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
cilium-gateway-basic-gateway   LoadBalancer   10.96.30.37     172.18.0.200   80:30594/TCP   21s
cni                            ClusterIP      10.96.145.109   <none>         80/TCP         4m9s
kubernetes                     ClusterIP      10.96.0.1       <none>         443/TCP        138m

root@KinD:~# kubectl get gateway
NAME            CLASS    ADDRESS        READY   AGE
basic-gateway   cilium   172.18.0.200   True    37s

root@KinD:~# kubectl get HTTPRoute
NAME           HOSTNAMES   AGE
http-app-cni               54s

创建 Gateway 后 basic-gateway 在定义的地址池中自动获取了一个 IP 地址，并且自动创建了一个 cilium-gateway-basic-gateway svc 服务，服务的 EXTERNAL-IP 为 basic-gateway 从地址池中自动获取的 IP 地址

测试 `Gateway` `HTTP`

root@KinD:~# GATEWAY=$(kubectl get gateway basic-gateway -o jsonpath='{.status.addresses[0].value}')
root@KinD:~# curl -v http://"$GATEWAY"/
*   Trying 172.18.0.200:80...
* Connected to 172.18.0.200 (172.18.0.200) port 80 (#0)
> GET / HTTP/1.1
> Host: 172.18.0.200
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: envoy
< date: Tue, 16 Jul 2024 06:14:57 GMT
< content-type: text/html
< content-length: 46
< last-modified: Tue, 16 Jul 2024 06:09:16 GMT
< etag: "66960e8c-2e"
< accept-ranges: bytes
< x-envoy-upstream-service-time: 0
< 
PodName: cni-j9bzr | PodIP: eth0 10.0.1.69/32
* Connection #0 to host 172.18.0.200 left intact

在宿主机 172.18.0.1 主机上访问业务正常访问，使用 ip 地址为 172.18.0.200

五、测试 `Gateway` `HTTPS` 特性

创建 tls 证书，此处使用了一个测试证书

root@KinD:~# kubectl  create secret tls demo-cert --cert=tls.crt --key=tls.key
secret/demo-cert created

配置 `Gateway` `HTTPS`

root@KinD:~# cat gateway-https.yamnl
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: tls-gateway
spec:
  gatewayClassName: cilium
  listeners:
  - name: https
    protocol: HTTPS
    port: 443
    hostname: "cni.evescn.com"
    tls:
      certificateRefs:
      - kind: Secret
        name: test-cert
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: https-app-cni
spec:
  parentRefs:
  - name: tls-gateway
  hostnames:
  - "cni.evescn.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: cni
      port: 80


root@KinD:~# kubectl apply -f gateway-https.yaml
gateway.gateway.networking.k8s.io/tls-gateway created
httproute.gateway.networking.k8s.io/https-app-cni created

root@KinD:~#  kubectl get svc
NAME                           TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)         AGE
cilium-gateway-basic-gateway   LoadBalancer   10.96.30.37     172.18.0.200   80:30594/TCP    7m21s
cilium-gateway-tls-gateway     LoadBalancer   10.96.5.61      172.18.0.201   443:30197/TCP   19s
cni                            ClusterIP      10.96.145.109   <none>         80/TCP          11m
kubernetes                     ClusterIP      10.96.0.1       <none>         443/TCP         145m

root@KinD:~# kubectl get gateway
NAME            CLASS    ADDRESS        READY   AGE
basic-gateway   cilium   172.18.0.200   True    7m47s
tls-gateway     cilium   172.18.0.201   True    45s

root@KinD:~# kubectl get HTTPRoute
NAME            HOSTNAMES                AGE
http-app-cni                             7m56s
https-app-cni   ["cni.evescn.com"]   54s

创建 gateway 后 tls-gateway 在定义的地址池中自动获取了一个 IP 地址，并且自动创建了一个 cilium-gateway-tls-gateway svc 服务，服务的 EXTERNAL-IP 为 tls-gateway 从地址池中自动获取的 IP 地址

测试 `Gateway` `HTTPS`

/etc/hosts 新增 dns 解析

root@KinD:~# cat /etc/hosts
......
172.18.0.201 cni.evescn.com

测试 Gateway HTTPS

root@KinD:~# curl --cacert /root/evescn.com.pem -v https://cni.evescn.com/
*   Trying 172.18.0.201:443...
* Connected to cni.evescn.com (172.18.0.201) port 443 (#0)
> GET / HTTP/1.1
> Host: cni.evescn.com
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: envoy
< date: Tue, 16 Jul 2024 06:25:27 GMT
< content-type: text/html
< content-length: 46
< last-modified: Tue, 16 Jul 2024 06:09:16 GMT
< etag: "66960e8c-2e"
< accept-ranges: bytes
< x-envoy-upstream-service-time: 0
< 
PodName: cni-j9bzr | PodIP: eth0 10.0.1.69/32
* Connection #0 to host cni.evescn.com left intact

在宿主机 172.18.0.1 主机上访问业务正常访问，使用 ip 地址为 172.18.0.201

六、转载博客

https://bbs.huaweicloud.com/blogs/418684

posted @ 2024-07-16 14:30 evescn 阅读(79) 评论(0) 编辑收藏举报

刷新页面返回顶部

evescn

Cilium Gateway API 特性（转载）

Cilium Gateway API 特性（转载）

一、环境信息

二、Cilium Gateway API 流程图

三、Cilium Gateway API 模式环境搭建

安装须知

kind 配置文件信息

安装 metallb 服务，提供 LoadBanlencer 功能

k8s 集群安装 Pod 和 Service

四、测试 Gateway HTTP 特性

配置 Gateway HTTP

测试 Gateway HTTP

五、测试 Gateway HTTPS 特性

创建 tls 证书，此处使用了一个测试证书

配置 Gateway HTTPS

测试 Gateway HTTPS

六、转载博客

公告

二、`Cilium` `Gateway API` 流程图

三、`Cilium` `Gateway API` 模式环境搭建

`kind` 配置文件信息

安装 `metallb` 服务，提供 `LoadBanlencer` 功能

`k8s` 集群安装 `Pod` 和 `Service`

四、测试 `Gateway` `HTTP` 特性

配置 `Gateway` `HTTP`

测试 `Gateway` `HTTP`

五、测试 `Gateway` `HTTPS` 特性

配置 `Gateway` `HTTPS`

测试 `Gateway` `HTTPS`