为Nginx服务添加ServiceEntry和workloadEntry对象，治理目标为网格外部服务的出向流量
测试在网格内部访问网格外部服务
我是开了三台云主机每台都安装下docker和docker-compose
第一台
[root@VM-0-12-centos ~]# cat Deploy-Nginx/docker-compose.yml
version: '3.3'
services:
nginx2001:
image: nginx:1.20-alpine
volumes:
- ./html/nginx2001:/usr/share/nginx/html/
networks:
envoymesh:
ipv4_address: 172.31.201.11
aliases:
- nginx
expose:
- "80"
ports:
- "10.0.0.12:8091:80"
networks:
envoymesh:
driver: bridge
ipam:
config:
- subnet: 172.31.201.0/24
[root@VM-0-12-centos ~]# cat Deploy-Nginx/html/nginx2001/index.html
<title>nginx.magedu.com</title>
Nginx 2001 ~~第二台
[root@VM-0-6-centos ~]# cat Deploy-Nginx/docker-compose.yml
version: '3.3'
services:
nginx2002:
image: nginx:1.20-alpine
volumes:
- ./html/nginx2002:/usr/share/nginx/html/
networks:
envoymesh:
ipv4_address: 172.31.201.12
aliases:
- nginx
expose:
- "80"
ports:
- "10.0.0.6:8091:80"
networks:
envoymesh:
driver: bridge
ipam:
config:
- subnet: 172.31.201.0/24
[root@VM-0-6-centos ~]# cat Deploy-Nginx/html/nginx2002/index.html
<title>nginx.magedu.com</title>
Nginx 2002 ~~第三台
[root@VM-0-14-centos ~]# cat Deploy-Nginx/docker-compose.yml
version: '3.3'
services:
nginx2101:
image: nginx:1.21-alpine
volumes:
- ./html/nginx2101:/usr/share/nginx/html/
networks:
envoymesh:
ipv4_address: 172.31.201.13
aliases:
- nginx
- canary
expose:
- "80"
ports:
- "10.0.0.14:8091:80"
networks:
envoymesh:
driver: bridge
ipam:
config:
- subnet: 172.31.201.0/24
[root@VM-0-14-centos ~]# cat Deploy-Nginx/html/nginx2101/index.html
<title>nginx.magedu.com</title>
Nginx 2101 ~~三台都执行下docker-compose up，启动之后curl访问下有没有问题
docker-compose up -d进入sleep使用curl访问外部服务，使用while循环访问
[root@k8s-master ~]# kubectl exec -it sleep-698cfc4445-dvx5n -- /bin/sh
/ $ curl 1.13.251.235:8091
<title>nginx.magedu.com</title>
Nginx 2101 ~~
/ $ curl 175.27.156.219:8091
<title>nginx.magedu.com</title>
Nginx 2002 ~~
/ $ curl 1.13.169.203:8091
<title>nginx.magedu.com</title>
Nginx 2001 ~~
while true; do curl curl 1.13.251.235:8091; sleep 1; done打开kiali，可以看到流量
将外部的服务引入到网格内部
[root@k8s-master 01-Service-Entry]# kubectl apply -f 01-serviceentry-nginx.yaml
serviceentry.networking.istio.io/nginx-external created
[root@k8s-master 01-Service-Entry]# cat 01-serviceentry-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: nginx-external
spec:
hosts:
- nginx.magedu.com
addresses:
- "1.13.169.203"
- "175.27.156.219"
- "1.13.251.235"
ports:
- number: 8091
name: http
protocol: HTTP
location: MESH_EXTERNAL
resolution: STATIC
endpoints:
- address: "1.13.169.203"
ports:
http: 8091
- address: "175.27.156.219"
ports:
http: 8091
- address: "1.13.251.235"
ports:
http: 8091看下有没有生成listener
[root@k8s-master 01-Service-Entry]# istioctl pc listeners sleep-698cfc4445-dvx5n
[root@k8s-master 01-Service-Entry]# istioctl pc cluster sleep-698cfc4445-dvx5n
[root@k8s-master 01-Service-Entry]# istioctl pc endpoint sleep-698cfc4445-dvx5n
再次使用sleep访问外部nginx查看协议没有变成http
创建一个pod加一个解析
[root@k8s-master ~]# kubectl run client-$RANDOM --image ikubernetes/admin-box:v1.2 --restart=Never -it --command -- /bin/bash
root@client-25390 /# vi /etc/hosts
1.13.169.203 nginx.magedu.com
root@client-25390 /# while true; do curl nginx.magedu.com:8091; sleep 1 ;done可以看到三台都可以访问到
这时我们就可以使用destinationrule和virtualservices
[root@k8s-master 01-Service-Entry]# cat 02-destinationrule-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: nginx-external
spec:
host: nginx.magedu.com
trafficPolicy:
loadBalancer:
consistentHash:
httpHeaderName: X-User
connectionPool:
tcp:
maxConnections: 10000
connectTimeout: 10ms
tcpKeepalive:
time: 7200s
interval: 75s
http:
http2MaxRequests: 1000
maxRequestsPerConnection: 10
outlierDetection:
maxEjectionPercent: 50
consecutive5xxErrors: 5
interval: 2m
baseEjectionTime: 1m
minHealthPercent: 40
[root@k8s-master 01-Service-Entry]# cat 03-virtualservice-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: nginx-external
spec:
hosts:
- nginx.magedu.com
http:
- name: falut-injection
match:
- headers:
X-Testing:
exact: "true"
route:
- destination:
host: nginx.magedu.com
fault:
delay:
percentage:
value: 5
fixedDelay: 2s
abort:
percentage:
value: 5
httpStatus: 555
- name: nginx-external
route:
- destination:
host: nginx.magedu.com[root@k8s-master 02-Workload-Entry]# kubectl apply -f 01-workloadentry-nginx.yaml
[root@k8s-master 02-Workload-Entry]# kubectl apply -f 02-serviceentry-nginx.yaml [root@k8s-master 02-Workload-Entry]# cat 01-workloadentry-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
name: workload-nginx2001
labels:
version: v1.20
spec:
address: "1.13.169.203"
ports:
http: 8091
labels:
app: nginx
version: v1.20
instance-id: Nginx2001
---
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
name: workload-nginx2002
labels:
version: v1.20
spec:
address: "175.27.156.219"
ports:
http: 8091
labels:
app: nginx
version: v1.20
instance-id: Nginx2002
---
[root@k8s-master 02-Workload-Entry]# cat 02-serviceentry-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: nginx-external
spec:
hosts:
- nginx.magedu.com
ports:
- number: 80
name: http
protocol: HTTP
targetPort: 8091
location: MESH_EXTERNAL
resolution: STATIC
workloadSelector:
labels:
app: nginx[root@k8s-master 02-Workload-Entry]# kubectl get workloadentry
NAME AGE ADDRESS
workload-nginx2001 3m31s 172.29.1.201
workload-nginx2002 3m31s 172.29.1.202
[root@k8s-master 02-Workload-Entry]# kubectl get workloadentry --show-labels
NAME AGE ADDRESS LABELS
workload-nginx2001 4m15s 172.29.1.201 version=v1.20
workload-nginx2002 4m15s 172.29.1.202 version=v1.20将网格外部的端点引入到网格内部
也可以单独配置destinationrule和virtualservice
[root@k8s-master 02-Workload-Entry]# cat 03-destinationrule-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: nginx-external
spec:
host: nginx.magedu.com
trafficPolicy:
loadBalancer:
simple: RANDOM
connectionPool:
tcp:
maxConnections: 10000
connectTimeout: 10ms
tcpKeepalive:
time: 7200s
interval: 75s
http:
http2MaxRequests: 1000
maxRequestsPerConnection: 10
outlierDetection:
maxEjectionPercent: 50
consecutive5xxErrors: 5
interval: 2m
baseEjectionTime: 1m
minHealthPercent: 40
[root@k8s-master 02-Workload-Entry]# cat 04-virtualservice-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: nginx-external
spec:
hosts:
- nginx.magedu.com
http:
- name: falut-injection
route:
- destination:
host: nginx.magedu.com
fault:
delay:
percentage:
value: 5
fixedDelay: 2s
abort:
percentage:
value: 5
httpStatus: 555将1.21版本也定义成workloadentry，访问的时候就可以三台都访问到。
[root@k8s-master 03-WorkloadEntry-Subsets]# kubectl apply -f 01-workloadentry-nginx.yaml
[root@k8s-master 03-WorkloadEntry-Subsets]# cat 01-workloadentry-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
name: workload-nginx2001
spec:
address: "172.29.1.201"
ports:
http: 8091
labels:
app: nginx
version: "v1.20"
instance-id: Nginx2001
---
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
name: workload-nginx2002
spec:
address: "172.29.1.202"
ports:
http: 8091
labels:
app: nginx
version: "v1.20"
instance-id: Nginx2002
---
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
name: workload-nginx2101
spec:
address: "172.29.1.203"
ports:
http: 8091
labels:
app: nginx
version: "v1.21"
instance-id: Nginx2101
---[root@k8s-master 03-WorkloadEntry-Subsets]# cat 02-serviceentry-nginx.yaml
---
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: nginx
spec:
hosts:
- nginx.magedu.com
ports:
- number: 80
name: http
protocol: HTTP
location: MESH_EXTERNAL
resolution: STATIC
workloadSelector:
labels:
app: nginx
---
子集划分
[root@k8s-master 03-WorkloadEntry-Subsets]# kubectl apply -f 03-destinationrule-subsets.yaml
[root@k8s-master 03-WorkloadEntry-Subsets]# cat 03-destinationrule-subsets.yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: nginx-external
spec:
host: nginx.magedu.com
trafficPolicy:
loadBalancer:
simple: RANDOM
connectionPool:
tcp:
maxConnections: 10000
connectTimeout: 10ms
tcpKeepalive:
time: 7200s
interval: 75s
http:
http2MaxRequests: 1000
maxRequestsPerConnection: 10
outlierDetection:
maxEjectionPercent: 50
consecutive5xxErrors: 5
interval: 2m
baseEjectionTime: 1m
minHealthPercent: 40
subsets:
- name: v20
labels:
version: "v1.20"
- name: v21
labels:
version: "v1.21"
---根据权重对子集进行流量划分
[root@k8s-master 03-WorkloadEntry-Subsets]# kubectl apply -f 04-virtualservice-wegit-based-routing.yaml
[root@k8s-master 03-WorkloadEntry-Subsets]# cat 04-virtualservice-wegit-based-routing.yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: nginx-external
spec:
hosts:
- nginx.magedu.com
http:
- name: default
route:
- destination:
host: nginx.magedu.com
subset: v21
weight: 5
- destination:
host: nginx.magedu.com
subset: v20
weight: 95故障注入
[root@k8s-master 03-WorkloadEntry-Subsets]# cat 05-virtualservice-headers-based-routing.yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: nginx-external
spec:
hosts:
- nginx.magedu.com
http:
- name: falut-injection
match:
- headers:
X-Canary:
exact: "true"
route:
- destination:
host: nginx.magedu.com
subset: v21
fault:
delay:
percentage:
value: 5
fixedDelay: 2s
- name: default
route:
- destination:
host: nginx.magedu.com
subset: v20
fault:
abort:
percentage:
value: 5
httpStatus: 555出口流量治理
[root@k8s-master 04-Egress-Gateway]# cat 01-workloadentry-nginx.yaml
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
name: workload-nginx2001
spec:
address: "172.29.1.201"
ports:
http: 8091
labels:
app: nginx
version: "v1.20"
instance-id: Nginx2001
---
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
name: workload-nginx2002
spec:
address: "172.29.1.202"
ports:
http: 8091
labels:
app: nginx
version: "v1.20"
instance-id: Nginx2002
---
apiVersion: networking.istio.io/v1beta1
kind: WorkloadEntry
metadata:
name: workload-nginx2101
spec:
address: "172.29.1.203"
ports:
http: 8091
labels:
app: nginx
version: "v1.21"
instance-id: Nginx2101
---
[root@k8s-master 04-Egress-Gateway]# cat 02-serviceentry-nginx.yaml
---
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: nginx
spec:
hosts:
- nginx.magedu.com
ports:
- number: 80
name: http
protocol: HTTP
location: MESH_EXTERNAL
resolution: STATIC
workloadSelector:
labels:
app: nginx
---
[root@k8s-master 04-Egress-Gateway]# cat 03-destinationrule-subsets.yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: nginx-external
spec:
host: nginx.magedu.com
trafficPolicy:
loadBalancer:
simple: RANDOM
connectionPool:
tcp:
maxConnections: 10000
connectTimeout: 10ms
tcpKeepalive:
time: 7200s
interval: 75s
http:
http2MaxRequests: 1000
maxRequestsPerConnection: 10
outlierDetection:
maxEjectionPercent: 50
consecutive5xxErrors: 5
interval: 2m
baseEjectionTime: 1m
minHealthPercent: 40
subsets:
- name: v20
labels:
version: "v1.20"
- name: v21
labels:
version: "v1.21"
---
[root@k8s-master 04-Egress-Gateway]# cat 04-gateway-egress.yaml
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: egress
namespace: istio-system
spec:
selector:
app: istio-egressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
[root@k8s-master 04-Egress-Gateway]# cat 05-virtualservice-wegit-based-routing.yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: nginx-external
spec:
hosts:
- nginx.magedu.com
gateways:
- istio-system/egress
- mesh
http:
- match:
- gateways:
- mesh
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
- match:
- gateways:
- istio-system/egress
route:
- destination:
host: nginx.magedu.com
subset: v21
weight: 5
- destination:
host: nginx.magedu.com
subset: v20
weight: 95
[root@k8s-master 04-Egress-Gateway]# cat README.md
# ServiceEntry and Egress Gateway
ServiceEntry、WorkloadEntry和Egress Gateway使用示例。
### 前提条件
需要基于相应的docker-compose定义，于Kubernetes集群外部的主机上启动几个外部nginx实例；
#### 注意
- 各实例监听的地址需要事先进行配置
- 如果有必要，可根据实际情况修改相应的IP地址
### 测试命令
于启动的专用测试客户端Pod的交互式的接口中运行如下命令
```bash
v20=0; v21=0; while true; do if curl -s nginx.magedu.com | grep "^Nginx 20.*" &> /dev/null; then let v20++; else let v21++; fi; echo ${v20}:${v21}; sleep 0.0$RANDOM; done
```
[root@k8s-master 04-Egress-Gateway]# 
