Envoy 手动TLS场景
1、环境
K8S:
CA和NFS：
主机名 IP OS OpenSSL版本 NFS版本
ha01 10.0.8.131 Ubuntu 20.04.3 LTS 1.1.1f v4
2、场景1：front-envoy向客户端提供TLS，后端HTTP，（front-tls）
2.1、结构拓扑
2.2、操作步骤
2.2.1、front-envoy制作证书
openssl genrsa -out ca.key 2048 #生成CA的私钥
openssl req -new -key ca.key -out ca.csr #生成CA的证书签署请求
openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt #生成CA的自签证书
openssl genrsa -out front-envoy.key 2048 #生成front-envoy私钥
openssl req -new -key front-envoy.key -out front-envoy.csr #生成front-envoy的证书签署请求
openssl x509 -req -days 365 -in front-envoy.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out front-envoy.crt #使用CA的证书和私钥签发front-envoy证书
openssl x509 -noout -modulus -in front-envoy.crt | openssl md5 #检查签发的证书和私钥是否匹配
openssl rsa -noout -modulus -in front-envoy.key | openssl md5 #检查签发的证书和私钥是否匹配
然后将front-envoy.crt和front-envoy.key 放入nfs 共front-envoy使用
2.2.2、创建新的namespace
kind: Namespace
apiVersion: v1
metadata:
name: envoy
namespace: envoy
2.2.3、创建pv和pvc
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs
labels:
app: envoy
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain # 回收策略
nfs:
path: /data/k8s-nfs
server: 10.0.8.131
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: envoy # pvc 名字
namespace: envoy
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
selector:
matchLabels:
app: envoy # 指定 pv 的标签
2.2.4、配置configmap为front-envoy和sidecar envoy准备配置文件
kind: ConfigMap
apiVersion: v1
metadata:
name: envoy
namespace: envoy
data:
front-envoy-config: |
admin:
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 443 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: webservice
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: local_cluster }
http_filters:
- name: envoy.filters.http.router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/etc/envoy/certs/front-envoy.crt"
private_key:
filename: "/etc/envoy/certs/front-envoy.key"
clusters:
- name: local_cluster
connect_timeout: 0.25s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: { address: webserver-0, port_value: 8080 }
- endpoint:
address:
socket_address: { address: webserver-1, port_value: 8080 }
sidecar-envoy-config: |
admin:
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 8080 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: webservice
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: local_cluster }
http_filters:
- name: envoy.filters.http.router
clusters:
- name: local_cluster
connect_timeout: 0.25s
type: STATIC
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: { address: 127.0.0.1, port_value: 80 }
主要配置：
2.2.5、创建front-envoy service和pod
kind: Service
apiVersion: v1
metadata:
name: front-envoy
namespace: envoy
spec:
type: ClusterIP
selector:
app: front-envoy
ports:
- name: https
port: 443
targetPort: 443
protocol: TCP
- name: http1
port: 8080
targetPort: 443
protocol: TCP
- name: http2
port: 80
targetPort: 443
protocol: TCP
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: front-envoy
namespace: envoy
spec:
replicas: 1
selector:
matchLabels:
app: front-envoy
template:
metadata:
name: front-envoy
namespace: envoy
labels:
app: front-envoy
spec:
containers:
- name: envoy
image: envoyproxy/envoy-alpine:v1.20-latest
imagePullPolicy: IfNotPresent
ports:
- name: admin
containerPort: 9901
hostPort: 9901
protocol: TCP
- name: https
containerPort: 443
hostPort: 30443
protocol: TCP
env:
- name: ENVOY_UID
value: "0"
volumeMounts:
- name: http-front-envoy
mountPath: /etc/envoy/envoy.yaml
subPath: envoy.yaml
- name: certs
mountPath: /etc/envoy/certs
subPath: certs
volumes:
- name: http-front-envoy
configMap:
name: envoy
items:
- key: front-envoy-config
path: envoy.yaml
- name: certs
persistentVolumeClaim:
claimName: envoy
readOnly: false
2.2.6、创建sidecar service和服务pod
kind: Service
apiVersion: v1
metadata:
name: webserver-0
namespace: envoy
spec:
selector:
app: sidecar-0
ports:
- name: sidecar
port: 8080
targetPort: 8080
protocol: TCP
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: webserver-0
namespace: envoy
spec:
replicas: 1
selector:
matchLabels:
app: sidecar-0
template:
metadata:
name: webserver-0
namespace: envoy
labels:
app: sidecar-0
spec:
containers:
- name: sidecar0
image: envoyproxy/envoy-alpine:v1.20-latest
imagePullPolicy: IfNotPresent
ports:
- name: admin
containerPort: 9901
hostPort: 9901
protocol: TCP
env:
- name: ENVOY_UID
value: "0"
volumeMounts:
- name: sidecar-envoy
mountPath: /etc/envoy/
- name: webserver01
image: ikubernetes/demoapp:v1.0
env:
- name: HOST
value: "127.0.0.1"
volumes:
- name: sidecar-envoy
configMap:
name: envoy
items:
- key: sidecar-envoy-config
path: envoy.yaml
---
kind: Service
apiVersion: v1
metadata:
name: webserver-1
namespace: envoy
spec:
selector:
app: sidecar-01
ports:
- name: sidecar
port: 8080
targetPort: 8080
protocol: TCP
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: webserver-1
namespace: envoy
spec:
replicas: 1
selector:
matchLabels:
app: sidecar-01
template:
metadata:
name: webserver-1
namespace: envoy
labels:
app: sidecar-01
spec:
containers:
- name: sidecar01
image: envoyproxy/envoy-alpine:v1.20-latest
imagePullPolicy: IfNotPresent
ports:
- name: admin
containerPort: 9901
hostPort: 9901
protocol: TCP
env:
- name: ENVOY_UID
value: "0"
volumeMounts:
- name: sidecar-envoy
mountPath: /etc/envoy/
- name: webserver01
image: ikubernetes/demoapp:v1.0
env:
- name: HOST
value: "127.0.0.1"
volumes:
- name: sidecar-envoy
configMap:
name: envoy
items:
- key: sidecar-envoy-config
path: envoy.yaml
2.2.7、验证
在pod demov10-59d6cd7449-rtxxw中测试
3、场景2：front-envoy和客户端之间TLS，front-envoy和后端envoy也是tls（https-https）
需要CA签发front-envoy的证书和后端envoy的证书
前后端envoy配置如下：
kind: ConfigMap
apiVersion: v1
metadata:
name: envoy
namespace: envoy
data:
front-envoy-config: |
admin:
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 443 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: webservice
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: local_cluster }
http_filters:
- name: envoy.filters.http.router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/etc/envoy/certs/front-envoy.crt"
private_key:
filename: "/etc/envoy/certs/front-envoy.key"
clusters:
- name: local_cluster
connect_timeout: 0.25s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: { address: webserver-0.envoy, port_value: 443 }
- endpoint:
address:
socket_address: { address: webserver-1.envoy, port_value: 443 }
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
sidecar-envoy-config: |
admin:
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 443 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: webservice
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: local_cluster }
http_filters:
- name: envoy.filters.http.router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/etc/envoy/certs/webserver.crt"
private_key:
filename: "/etc/envoy/certs/webserver.key"
clusters:
- name: local_cluster
connect_timeout: 0.25s
type: STATIC
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: { address: 127.0.0.1, port_value: 80 }
在前端envoy上护短cluster中增加配置
后端增加配置，当然和场景1一样申请签发证书，然后挂载进容器
场景3、 https-passthrough，front-envoy 监听在https端口，但是不做处理直接传给后端envoy处理
主要配置文件：
kind: ConfigMap
apiVersion: v1
metadata:
name: envoy
namespace: envoy
data:
front-envoy-config: |
admin:
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 443 }
filter_chains:
- filters:
- name: envoy.filters.network.tcp_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
cluster: local_cluster
stat_prefix: https_passthrough
clusters:
- name: local_cluster
connect_timeout: 0.25s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: { address: webserver-0.envoy, port_value: 443 }
- endpoint:
address:
socket_address: { address: webserver-1.envoy, port_value: 443 }
sidecar-envoy-config: |
admin :
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 443 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: webservice
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: local_cluster }
http_filters:
- name: envoy.filters.http.router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/etc/envoy/certs/webserver.crt"
private_key:
filename: "/etc/envoy/certs/webserver.key"
clusters:
- name: local_cluster
connect_timeout: 0.25s
type: STATIC
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: { address: 127.0.0.1, port_value: 80 }
front-envoy配置：
后端envoy提供正常https服务即可
