istio: 取消注入sidecar
1. 问题
Consul注入istio以后,功能异常了,后台管理页面都打不开,无法选择leader:
2023-04-19T09:33:27.886Z [ERROR] agent: Coordinate update error: error="No cluster leader"
2023-04-19T09:33:46.598Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"
2023-04-19T09:33:57.469Z [ERROR] agent: Coordinate update error: error="No cluster leader"
2023-04-19T09:34:16.298Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"
2023-04-19T09:34:27.653Z [ERROR] agent: Coordinate update error: error="No cluster leader"
2023-04-19T09:34:39.173Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"
2023-04-19T09:34:59.006Z [ERROR] agent: Coordinate update error: error="No cluster leader"
2023-04-19T09:35:03.442Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader”
先不研究如何让consul兼容istio的问题,先考虑如何让consul不注入istio,
2. 取消注入sidecar
看了下官网,取消sidecar注入的配置很简单,只要在对应的deployment/daemonset/statefulset配置sidecar.istio.io/inject:"false"即可
apiVersion: apps/v1
spec:
template:
metadata:
annotations:
sidecar.istio.io/inject: “false"
3. consul取消注入sidecar实例
3.1. 注入以后的部署情况:
qiteck@server:~/program/docker_service/consul/cicd/deploy/k8s/nodeport$ sudo kubectl get pods --all-namespaces -o wide |grep consul
default consul-client-hrlxb 2/2 Running 0 84s 10.244.1.42 server <none> <none>
default consul-server-0 2/2 Running 0 85s 10.244.1.41 server <none> <none>
default consul-server-1 2/2 Running 0 66s 10.244.1.43 server <none> <none>
default consul-server-2 2/2 Running 0 52s 10.244.1.44 server <none> <none>
可以看到每个pod部署2个container
3.2. 取消注入配置
consul-client
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: consul-client
labels:
name: consul-client
spec:
selector:
matchLabels:
name: consul-client
template:
metadata:
labels:
name: consul-client
annotations:
sidecar.istio.io/inject: "false"
spec:
containers:
- name: consul
image: consul:1.10.3
imagePullPolicy: IfNotPresent
args:
- "agent"
- "-ui"
- "-data-dir=/consul/data"
- "-bind=0.0.0.0"
- "-client=0.0.0.0"
- "-advertise=$(POD_IP)"
- "-retry-join=consul-server-0.consul-server.$(NAMESPACE).svc.cluster.local"
- "-retry-join=consul-server-1.consul-server.$(NAMESPACE).svc.cluster.local"
- "-retry-join=consul-server-2.consul-server.$(NAMESPACE).svc.cluster.local"
- "-domain=consul" ## 默认情况下,Consul响应“consul”中的DNS查询。域。该标志可用于更改该域。该域中的所有查询都假定由Consul处理,不会递归解决。
- "-disable-host-node-id"
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
lifecycle:
postStart:
exec:
command:
- /bin/sh
- -c
- consul reload
preStop:
exec:
command:
- /bin/sh
- -c
- consul leave
volumeMounts:
- name: consul-data-dir
mountPath: /consul/data
ports:
- containerPort: 8500
name: http
- containerPort: 8400
name: rpc
- containerPort: 8443
name: https
- containerPort: 8301
name: serf-lan
- containerPort: 8301
name: serf-lan-8301
protocol: UDP
- containerPort: 8302
name: serf-wan
- containerPort: 8302
name: serf-wan-udp
protocol: UDP
- name: dns-tcp
containerPort: 8600
- name: dns-udp
containerPort: 8600
protocol: UDP
- containerPort: 8300
name: server
volumes:
- name: consul-data-dir
hostPath:
path: /Users/xuan/desktop/consul/data
type: DirectoryOrCreate
consul-server
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: consul-server
labels:
name: consul-server
spec:
serviceName: consul-server
selector:
matchLabels:
name: consul-server
replicas: 3
template:
metadata:
labels:
name: consul-server
annotations:
sidecar.istio.io/inject: "false"
spec:
terminationGracePeriodSeconds: 10
containers:
- name: consul
image: consul:1.10.3
imagePullPolicy: IfNotPresent
args:
- "agent"
- "-server"
- "-bootstrap-expect=3"
- "-ui"
- "-data-dir=/consul/data"
- "-bind=0.0.0.0"
- "-client=0.0.0.0"
- "-advertise=$(POD_IP)"
- "-retry-join=consul-server-0.consul-server.$(NAMESPACE).svc.cluster.local"
- "-retry-join=consul-server-1.consul-server.$(NAMESPACE).svc.cluster.local"
- "-retry-join=consul-server-2.consul-server.$(NAMESPACE).svc.cluster.local"
- "-domain=consul" ## 默认情况下,Consul响应“consul”中的DNS查询。域。该标志可用于更改该域。该域中的所有查询都假定由Consul处理,不会递归解决。
- "-disable-host-node-id"
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- containerPort: 8500
name: http
- containerPort: 8400
name: rpc
- containerPort: 8443
name: https-port
- containerPort: 8301
name: serf-lan
- containerPort: 8301
name: serf-lan-8301
protocol: UDP
- containerPort: 8302
name: serf-wan
- containerPort: 8302
name: serf-wan-udp
protocol: UDP
- name: dns-tcp
containerPort: 8600
protocol: TCP
- name: dns-udp
containerPort: 8600
protocol: UDP
- containerPort: 8300
name: server
3.3.取消注入以后的部署情况:
qiteck@server:~/program/docker_service/consul/cicd/deploy/k8s/nodeport$ sudo kubectl get pods --all-namespaces -o wide |grep consul
default consul-client-44ksc 1/1 Running 2 (60s ago) 2m 10.244.1.45 server <none> <none>
default consul-server-0 1/1 Running 0 114s 10.244.1.46 server <none> <none>
default consul-server-1 1/1 Running 0 110s 10.244.1.47 server <none> <none>
default consul-server-2 1/1 Running 0 107s 10.244.1.48 server <none> <none>
可以看到每个pod部署1个container了。
然后功能也正常了。