Envoy TLS基础

Envoy Mesh中TLS常用场景
TLS配置位置说明
TLS环境配置实例
TLS环境TCP Proxy模式配置实例
参考文档

回到顶部

Envoy Mesh中TLS常用场景

Front Proxy面向下游客户端提供https服务，但Front Proxy、Mesh内部的各服务间依然使用http协议；
- https (下游) → http (上游)
Front Proxy面向下游客户端提供https服务，而且Front Proxy、Mesh内部的各服务间也使用https协议；
- https (下游) → https (上游)
- 但是内部各Service间的通信也有如下两种情形
  - 仅客户端验证服务端证书
  - 客户端与服务端之间互相验证彼此的证书（mTLS）
- 注意：对于容器化的动态环境来说，证书预配和管理将成为显著难题
Front Proxy直接以TCP Proxy的代理模式，在下游客户端与上游服务端之间透传tls协议；
- https-passthrough
- 集群内部的东西向流量同样工作于https协议模型

回到顶部

TLS配置位置说明

Listener

Listener面向DownStream（下游客户端）通信，与下游建立tls通信时通常配置在Listener上；

与下游建立tls通信时Listener作为server端，需要提供证书和私钥；

Cluster

Cluster面向上游通信，与上游建立tls通信时通常配置在Cluster上；

与上游建立tls通信时Cluster作为客户端；

单向tls通信时；
- 需要ca证书，验证上游服务器证书；
双向tls通信时；
- 需要ca证书，验证上游服务器证书；
- 需要client端证书和私钥；

回到顶部

TLS环境配置实例

环境说明

envoy：Front Proxy,地址为172.31.7.2，监听于443端口
webserver_http-01：第一个后端服务 http
webserver_http-01-sidecar：第一个后端服务的Sidecar Proxy,地址为172.31.7.11，监听于80端口
webserver_http-02：第二个后端服务 http
webserver_http-02-sidecar：第二个后端服务的Sidecar Proxy,地址为172.31.7.12, 监听于80端口
webserver_tls-01：第三个后端服务 tls
webserver_tls-01-sidecar：第三个后端服务的Sidecar Proxy,地址为172.31.7.13，监听于443端口
webserver_tls-02：第四个后端服务 tls
webserver_tls-02-sidecar：第四个后端服务的Sidecar Proxy,地址为172.31.7.14, 监听于443端口
webserver_mtls-01：第五个后端服务 mtls
webserver_mtls-01-sidecar：第五个后端服务的Sidecar Proxy,地址为172.31.7.15，监听于443端口
webserver_mtls-02：第六个后端服务 mtls
webserver_mtls-02-sidecar：第六个后端服务的Sidecar Proxy,地址为172.31.7.16, 监听于443端口

拓扑结构

创建证书目录

# mkdir -pv certs/{CA,front-envoy}
# touch certs/index.txt
# touch certs/serial
# echo "1000" >  certs/serial
# echo "unique_subject = no" > certs/index.txt.attr  # Sign multiple certs for the same CN

准备openssl.conf文件

查看代码

  # environment variable values
CERT_DIR=certs

[ ca ]
# `man ca`
default_ca = CA_default

[ CA_default ]
# Directory and file locations.
dir               = ${ENV::CERT_DIR}
certs             = $dir
crl_dir           = $dir/crl
new_certs_dir     = $dir
database          = $dir/index.txt
serial            = $dir/serial
# certificate revocation lists.
crlnumber         = $dir/crlnumber
crl               = $dir/crl/intermediate-ca.crl
crl_extensions    = crl_ext
default_crl_days  = 30
default_md        = sha256

name_opt          = ca_default
cert_opt          = ca_default
default_days      = 375
preserve          = no
policy            = policy_loose

[ policy_loose ]
# Allow the CA to sign a range of certificates.
countryName             = optional
stateOrProvinceName     = optional
localityName            = optional
organizationName        = optional
organizationalUnitName  = optional
commonName              = supplied
emailAddress            = optional

[ req ]
# `man req`
default_bits        = 4096
distinguished_name  = req_distinguished_name
string_mask         = utf8only
default_md          = sha256

[ req_distinguished_name ]
countryName                    = Country Name (2 letter code)
stateOrProvinceName            = State or Province Name
localityName                   = Locality Name
0.organizationName             = Organization Name
organizationalUnitName         = Organizational Unit Name
commonName                     = Common Name

# Certificate extensions (`man x509v3_config`)

[ v3_ca ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ client_cert ]
basicConstraints = CA:FALSE
nsCertType = client
nsComment = "OpenSSL Generated Client Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth

[ server_cert ]
basicConstraints = CA:FALSE
nsCertType = server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth

创建证书

创建ca证书

# openssl genrsa -out certs/CA/ca.key 4096
# openssl req -config openssl.conf -new -x509 -days 3650 -sha256 -key certs/CA/ca.key -extensions v3_ca -out certs/CA/ca.crt -subj /CN=envoy-ca

创建front-proxy server证书

# openssl genrsa -out certs/front-envoy/server.key 2048
# openssl req -config openssl.conf -new -sha256 -key certs/front-envoy/server.key -out certs/front-envoy/server.csr -subj /CN=front-proxy
# openssl ca -batch -config openssl.conf -extensions server_cert -days 3650 -notext -md sha256 -in certs/front-envoy/server.csr -out certs/front-envoy/server.crt -cert certs/CA/ca.crt -keyfile certs/CA/ca.key

创建front-proxy client证书

# openssl genrsa -out certs/front-envoy/client.key 2048
# openssl req -config openssl.conf -new -sha256 -key certs/front-envoy/client.key -out certs/front-envoy/client.csr -subj /CN=front-proxy
# openssl ca -batch -config openssl.conf -extensions client_cert -days 3650 -notext -md sha256 -in certs/front-envoy/client.csr -out certs/front-envoy/client.crt -cert certs/CA/ca.crt -keyfile certs/CA/ca.key

修改证书权限

# docker run -it  --rm envoyproxy/envoy:v1.23-latest id envoy
uid=101(envoy) gid=101(envoy) groups=101(envoy)
# chown -R 101.101 certs/

front-envoy.yaml

查看代码

node:
  id: front-envoy
  cluster: front-envoy

admin:
  profile_path: /tmp/envoy.prof
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
       address: 0.0.0.0
       port_value: 9901

layered_runtime:
  layers:
  - name: admin
    admin_layer: {}

static_resources:
  secrets:
  - name: server_cert
    tls_certificate:
      certificate_chain:
        filename: "/etc/envoy/certs/server.crt"
      private_key:
        filename: "/etc/envoy/certs/server.key"
  - name: client_cert
    tls_certificate:
      certificate_chain:
        filename: "/etc/envoy/certs/client.crt"
      private_key:
        filename: "/etc/envoy/certs/client.key"
  - name: validation_context
    validation_context:
      trusted_ca:
        filename: "/etc/envoy/ca/ca.crt"

  listeners:
  - name: listener_http
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 80
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: auto
          stat_prefix: ingress_http
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog 
              path: "/dev/stdout"           
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                redirect:
                  https_redirect: true
                  port_redirect: 443
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  - name: listener_https
    address:
      socket_address: { address: 0.0.0.0, port_value: 443 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_https
          codec_type: AUTO
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog 
              path: "/dev/stdout"           
          route_config:
            name: https_route
            virtual_hosts:
            - name: https_route
              domains: ["*"]
              routes:
              - match:
                  prefix: "/http/"
                route:
                  prefix_rewrite: "/"
                  cluster: http
              - match:
                  prefix: "/tls/"
                route:
                  prefix_rewrite: "/"
                  cluster: tls
              - match:
                  prefix: "/mtls/"
                route:
                  prefix_rewrite: "/"
                  cluster: mtls
          http_filters:
          - name: envoy.filters.http.router
            typed_config: 
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
      transport_socket:  # DownstreamTlsContext
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            tls_certificate_sds_secret_configs:
            - name: server_cert

  clusters:
  - name: http
    connect_timeout: 0.25s
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: http
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: http
                port_value: 80

  - name: tls
    connect_timeout: 0.25s
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: tls
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: tls
                port_value: 443
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        common_tls_context:
          validation_context_sds_secret_config:
            name: validation_context

  - name: mtls
    connect_timeout: 0.25s
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: mtls
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: mtls
                port_value: 443
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        common_tls_context:
          tls_certificate_sds_secret_configs:
          - name: client_cert
          validation_context_sds_secret_config:
            name: validation_context

envoy-sidecar-proxy_http.yaml

查看代码

admin:
  profile_path: /tmp/envoy.prof
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
       address: 0.0.0.0
       port_value: 9901

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 80 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service 
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route: { cluster: local_cluster }
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  clusters:
  - name: local_cluster
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: local_cluster
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address: { address: 127.0.0.1, port_value: 8080 }

envoy-sidecar-proxy_tls.yaml

查看代码

admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 9901

static_resources:
  listeners:
  - name: listener_http
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 80
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: auto
          stat_prefix: ingress_http
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog 
              path: "/dev/stdout"          
          route_config:
            name: local_route
            virtual_hosts:
            - name: service
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: local_service
          http_filters:
          - name: envoy.filters.http.router
            typed_config: 
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router 

  - name: listener_https
    address:
      socket_address: { address: 0.0.0.0, port_value: 443 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_https
          codec_type: AUTO
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog 
              path: "/dev/stdout"          
          route_config:
            name: https_route
            virtual_hosts:
            - name: https_route
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: local_service
          http_filters:
          - name: envoy.filters.http.router
            typed_config: 
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router 
      transport_socket:  
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            tls_certificates:    
              certificate_chain:
                filename: "/etc/envoy/certs/server.crt"
              private_key:
                filename: "/etc/envoy/certs/server.key" 

  clusters:
  - name: local_service
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: round_robin
    load_assignment:
      cluster_name: local_service
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 8080

envoy-sidecar-proxy_mtls.yaml

查看代码

admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 9901

static_resources:
  listeners:
  - name: listener_http
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 80
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: auto
          stat_prefix: ingress_http
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog 
              path: "/dev/stdout"          
          route_config:
            name: local_route
            virtual_hosts:
            - name: service
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/"
                redirect:
                  https_redirect: true
                  port_redirect: 443
          http_filters:
          - name: envoy.filters.http.router
            typed_config: 
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  - name: listener_https
    address:
      socket_address: { address: 0.0.0.0, port_value: 443 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_https
          codec_type: AUTO
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog 
              path: "/dev/stdout"          
          route_config:
            name: https_route
            virtual_hosts:
            - name: https_route
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: local_service
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
      transport_socket:  # DownstreamTlsContext
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            tls_certificates:    # 基于DataSource，直接给出证书和私钥文件
              certificate_chain:
                filename: "/etc/envoy/certs/server.crt"
              private_key:
                filename: "/etc/envoy/certs/server.key" 
          require_client_certificate: true   # 强制验证客户端证书

  clusters:
  - name: local_service
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: round_robin
    load_assignment:
      cluster_name: local_service
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 8080

docker-compose.yaml

查看代码

version: '3.3'

services:
  envoy:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./front-envoy.yaml:/etc/envoy/envoy.yaml
    - ./certs/front-envoy:/etc/envoy/certs/
    - ./certs/CA/:/etc/envoy/ca/
    networks:
      envoymesh:
        ipv4_address: 172.31.7.2
        aliases:
        - front-proxy
    expose:
      - "80"
      - "443"
      - "9901"
    ports:
      - "8080:80"
      - "8443:443"
      - "9901:9901"
    depends_on:
    - webserver_http-01-sidecar
    - webserver_http-02-sidecar
    - webserver_tls-01-sidecar
    - webserver_tls-02-sidecar
    - webserver_mtls-01-sidecar
    - webserver_mtls-02-sidecar
    
  webserver_http-01-sidecar:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./envoy-sidecar-proxy_http.yaml:/etc/envoy/envoy.yaml
    hostname: webserver_http-01
    networks:
      envoymesh:
        ipv4_address: 172.31.7.11
        aliases:
        - webserver_http-01-sidecar
        - front-proxy
        - http

  webserver_http-01:
    image: ikubernetes/demoapp:v1.0
    environment:
      - PORT=8080
      - HOST=127.0.0.1
    network_mode: "service:webserver_http-01-sidecar"
    depends_on:
    - webserver_http-01-sidecar

  webserver_http-02-sidecar:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./envoy-sidecar-proxy_http.yaml:/etc/envoy/envoy.yaml
    hostname: webserver_http-02
    networks:
      envoymesh:
        ipv4_address: 172.31.7.12
        aliases:
        - webserver_http-02-sidecar
        - front-proxy
        - http

  webserver_http-02:
    image: ikubernetes/demoapp:v1.0
    environment:
      - PORT=8080
      - HOST=127.0.0.1
    network_mode: "service:webserver_http-02-sidecar"
    depends_on:
    - webserver_http-02-sidecar

  webserver_tls-01-sidecar:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./envoy-sidecar-proxy_tls.yaml:/etc/envoy/envoy.yaml
    - ./certs/front-envoy/:/etc/envoy/certs/
    hostname: webserver_tls-01
    networks:
      envoymesh:
        ipv4_address: 172.31.7.13
        aliases:
        - webserver_tls-03-sidecar
        - tls

  webserver_tls-01:
    image: ikubernetes/demoapp:v1.0
    environment:
      - PORT=8080
      - HOST=0.0.0.0
    network_mode: "service:webserver_tls-01-sidecar"
    depends_on:
    - webserver_tls-01-sidecar

  webserver_tls-02-sidecar:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./envoy-sidecar-proxy_tls.yaml:/etc/envoy/envoy.yaml
    - ./certs/front-envoy/:/etc/envoy/certs/
    hostname: webserver_tls-02
    networks:
      envoymesh:
        ipv4_address: 172.31.7.14
        aliases:
        - webserver_tls-02-sidecar
        - tls

  webserver_tls-02:
    image: ikubernetes/demoapp:v1.0
    environment:
      - PORT=8080
      - HOST=0.0.0.0
    network_mode: "service:webserver_tls-02-sidecar"
    depends_on:
    - webserver_tls-02-sidecar

  webserver_mtls-01-sidecar:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./envoy-sidecar-proxy_mtls.yaml:/etc/envoy/envoy.yaml
    - ./certs/front-envoy/:/etc/envoy/certs/
    hostname: webserver_mtls-01
    networks:
      envoymesh:
        ipv4_address: 172.31.7.15
        aliases:
        - webserver_mtls-01-sidecar
        - mtls

  webserver_mtls-01:
    image: ikubernetes/demoapp:v1.0
    environment:
      - PORT=8080
      - HOST=0.0.0.0
    network_mode: "service:webserver_mtls-01-sidecar"
    depends_on:
    - webserver_mtls-01-sidecar

  webserver_mtls-02-sidecar:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./envoy-sidecar-proxy_mtls.yaml:/etc/envoy/envoy.yaml
    - ./certs/front-envoy/:/etc/envoy/certs/
    hostname: webserver_mtls-02
    networks:
      envoymesh:
        ipv4_address: 172.31.7.16
        aliases:
        - webserver_mtls-02-sidecar
        - mtls

  webserver_mtls-02:
    image: ikubernetes/demoapp:v1.0
    environment:
      - PORT=8080
      - HOST=0.0.0.0
    network_mode: "service:webserver_mtls-02-sidecar"
    depends_on:
    - webserver_mtls-02-sidecar

networks:
  envoymesh:
    driver: bridge
    ipam:
      config:
        - subnet: 172.31.7.0/24

运行并测试

启动服务

docker-compose up

查看证书

查看代码

 # curl 172.31.7.2:9901/certs
{
 "certificates": [
  {
   "ca_cert": [
    {
     "path": "/etc/envoy/ca/ca.crt",
     "serial_number": "51d6bb31f33491a0c776fb6534c4c2f7794aaee0",
     "subject_alt_names": [],
     "days_until_expiration": "3649",
     "valid_from": "2022-08-20T03:10:25Z",
     "expiration_time": "2032-08-17T03:10:25Z"
    }
   ],
   "cert_chain": [
    {
     "path": "/etc/envoy/certs/client.crt",
     "serial_number": "1001",
     "subject_alt_names": [],
     "days_until_expiration": "3649",
     "valid_from": "2022-08-20T03:17:16Z",
     "expiration_time": "2032-08-17T03:17:16Z"
    }
   ]
  },
  {
   "ca_cert": [],
   "cert_chain": [
    {
     "path": "/etc/envoy/certs/server.crt",
     "serial_number": "1000",
     "subject_alt_names": [],
     "days_until_expiration": "3649",
     "valid_from": "2022-08-20T03:16:15Z",
     "expiration_time": "2032-08-17T03:16:15Z"
    }
   ]
  },
  {
   "ca_cert": [
    {
     "path": "/etc/envoy/ca/ca.crt",
     "serial_number": "51d6bb31f33491a0c776fb6534c4c2f7794aaee0",
     "subject_alt_names": [],
     "days_until_expiration": "3649",
     "valid_from": "2022-08-20T03:10:25Z",
     "expiration_time": "2032-08-17T03:10:25Z"
    }
   ],
   "cert_chain": []
  }
 ]
}

查看front-envoy启动的Listener

# curl 172.31.7.2:9901/listeners
listener_http::0.0.0.0:80
listener_https::0.0.0.0:443

测试访问服务

我们可在curl命令上为其提供私有CA证书文件，或者使用“-k”选项忽略提示的风险，从而访问https服务。

http请求自动跳转至https服务上

# curl -I 172.31.7.2
HTTP/1.1 301 Moved Permanently
location: https://172.31.7.2:443/
date: Sat, 20 Aug 2022 08:14:44 GMT
server: envoy
transfer-encoding: chunked

访问http

# curl -k https://172.31.7.2/http/
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: webserver_http-01, ServerIP: 172.31.7.11!
# curl -k https://172.31.7.2/http/
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: webserver_http-02, ServerIP: 172.31.7.12!

访问tls

# curl -k https://172.31.7.2/tls/
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: webserver_tls-01, ServerIP: 172.31.7.13!
# curl -k https://172.31.7.2/tls/
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: webserver_tls-02, ServerIP: 172.31.7.14!

访问mtls

# curl -k https://172.31.7.2/mtls/
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: webserver_mtls-01, ServerIP: 172.31.7.15!
# curl -k https://172.31.7.2/mtls/
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: webserver_mtls-02, ServerIP: 172.31.7.16!

查看访问日志

tls-wgs-2-envoy-1                                   | [2022-08-20T09:24:43.445Z] "GET /http/ HTTP/1.1" 200 - 0 103 2 2 "-" "curl/7.68.0" "bc0b5abb-2be1-4aaf-b146-b223ce330bbf" "172.31.7.2" "172.31.7.11:80"
tls-wgs-2-envoy-1                                   | [2022-08-20T09:24:45.006Z] "GET /http/ HTTP/1.1" 200 - 0 103 2 1 "-" "curl/7.68.0" "04b259d7-0e8a-45ff-aeea-023b6f8c0bf8" "172.31.7.2" "172.31.7.12:80"
tls-wgs-2-envoy-1                                   | [2022-08-20T09:24:48.928Z] "GET /tls/ HTTP/1.1" 200 - 0 102 5 4 "-" "curl/7.68.0" "b71bd17d-9fb6-42ff-bd66-166a7366215d" "172.31.7.2" "172.31.7.14:443"
tls-wgs-2-envoy-1                                   | [2022-08-20T09:24:51.585Z] "GET /tls/ HTTP/1.1" 200 - 0 102 3 2 "-" "curl/7.68.0" "076b3ec7-5a83-49b8-a663-e1bcaa560153" "172.31.7.2" "172.31.7.13:443"
tls-wgs-2-webserver_tls-01-sidecar-1                | [2022-08-20T09:24:51.586Z] "GET /tls/ HTTP/1.1" 200 - 0 102 2 1 "-" "curl/7.68.0" "076b3ec7-5a83-49b8-a663-e1bcaa560153" "172.31.7.2" "127.0.0.1:8080"
tls-wgs-2-webserver_mtls-01-sidecar-1               | [2022-08-20T09:24:54.662Z] "GET /mtls/ HTTP/1.1" 200 - 0 103 3 2 "-" "curl/7.68.0" "8532ac7a-aaba-4e47-a5d0-63d444a7a6e8" "172.31.7.2" "127.0.0.1:8080"
tls-wgs-2-webserver_mtls-01-sidecar-1               | [2022-08-20T09:24:55.801Z] "GET /mtls/ HTTP/1.1" 200 - 0 103 1 1 "-" "curl/7.68.0" "af41c174-c7a0-4227-8340-f161af0af4e1" "172.31.7.2" "127.0.0.1:8080"
tls-wgs-2-webserver_mtls-02-sidecar-1               | [2022-08-20T09:24:56.981Z] "GET /mtls/ HTTP/1.1" 200 - 0 103 2 2 "-" "curl/7.68.0" "49e4cbfa-f13f-4085-8866-b03a29b611d9" "172.31.7.2" "127.0.0.1:8080"
tls-wgs-2-envoy-1                                   | [2022-08-20T09:24:54.659Z] "GET /mtls/ HTTP/1.1" 200 - 0 103 7 6 "-" "curl/7.68.0" "8532ac7a-aaba-4e47-a5d0-63d444a7a6e8" "172.31.7.2" "172.31.7.15:443"
tls-wgs-2-envoy-1                                   | [2022-08-20T09:24:55.800Z] "GET /mtls/ HTTP/1.1" 200 - 0 103 2 2 "-" "curl/7.68.0" "af41c174-c7a0-4227-8340-f161af0af4e1" "172.31.7.2" "172.31.7.15:443"
tls-wgs-2-envoy-1                                   | [2022-08-20T09:24:56.981Z] "GET /mtls/ HTTP/1.1" 200 - 0 103 3 3 "-" "curl/7.68.0" "49e4cbfa-f13f-4085-8866-b03a29b611d9" "172.31.7.2" "172.31.7.16:443"

openssl s_client命令

查看代码

# openssl s_client -connect 172.31.7.2:443
# 如下命令结果显示，tls传话已然能正常建立，但curl命令无法任何服务端证书的CA，除非我们给命令指定相应的私有CA的证书，以便于验证服务端证书
CONNECTED(00000003)
Can't use SSL_get_servername
depth=0 CN = front-proxy
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = front-proxy
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:CN = front-proxy
   i:CN = envoy-ca
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEkDCCAnigAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwEzERMA8GA1UEAwwIZW52
b3ktY2EwHhcNMjIwODIwMDMxNjE1WhcNMzIwODE3MDMxNjE1WjAWMRQwEgYDVQQD
DAtmcm9udC1wcm94eTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMcj
jvCXA7P9S02uI8rZSVqbOegtz39ssPQ9bGkbtZjjBUGysqYs1VKdtR00Gy+eJ55F
1LwGA3fs8nBFNIo2eSAEMkr8tOQFELCbj7kJbMKFqCKBO2i61T4cP++uUM8QzlYw
rQNaLdZcdPPk6uZffd5Gt5N2DoakMt5O3OFV+EGQgbSb9LBdju3zQCkfwZNpewhs
RpMNzdT4m4pbM1dBnhq04sOHwsjwgApeLeTxWrV6DGCBkx6TXOPadnxrI+I7bChZ
65EO4WKHADqlTvlHtMfU+qB6qCHIRxP/k1dYA95PINZ9dB0gqqesah8x5f3soC2V
mQgw/cO8xvbbj4kx7JECAwEAAaOB6jCB5zAJBgNVHRMEAjAAMBEGCWCGSAGG+EIB
AQQEAwIGQDAzBglghkgBhvhCAQ0EJhYkT3BlblNTTCBHZW5lcmF0ZWQgU2VydmVy
IENlcnRpZmljYXRlMB0GA1UdDgQWBBRsX1uAWFyCxOOKG+tuSOH+W7ssyjBOBgNV
HSMERzBFgBRDVByyc+OZZC6ivLr2ujfpTFgT2KEXpBUwEzERMA8GA1UEAwwIZW52
b3ktY2GCFFHWuzHzNJGgx3b7ZTTEwvd5Sq7gMA4GA1UdDwEB/wQEAwIFoDATBgNV
HSUEDDAKBggrBgEFBQcDATANBgkqhkiG9w0BAQsFAAOCAgEAE5OiviTeeelTRYSZ
w6ml0xiPEm9VGfGvgV/hhrVimlRoLV/rjz52z+ZO2hpvgdEBlaRIheKz38EieDgo
QGZAUIgMZA2F2iVnhtPX3R5lGVClzfPq1pvZb30WH5ukMY7MGY82bfoUgzuCz9jS
h5sNrsGgI/qI9yAp/memgPCmEhInHhnpsq/RHJuF+HnX9F8GvG3xy3TOAZl1Fz05
WqlXustuMy9y0fb+wAwmSGX82bWwbgsRlmSy3ZfnVAG5DaZ+/gvljg/rxR9n2Xuq
59SbTqjlmskN7SohpmAmgF6wClz7AgpaKkXN9+wJ9t9fn3IUUC3PD9ETSuB7FAlT
stGiKz1ZdGcSnCkFaqJT3XIfGeT5W4Ujj6oKzNHr9rZ43g5IRHIscEx010nVatX9
Zr+zNoVivhhthuiA7CJvkvHJIMFCeadRmugVu2O23XhAde4vcIXymeXFtrGSBUW3
GUtByhEpOG0KiInHqHTr+DaZuAiXcfb3MjaPqyJ6Onth5hHDhK2VWGXrsWlFjJ6P
10mZzwjX8rNXx78oMWdU4frZNBarSEoVbeBvR+SEt19vVwhXZZOED5EUb8jBsLpl
t48Natd11R05RT57EWCl+1QPrvBCO3R64sj3KmWl1PPzpyUtlP1yov1pdbfXt+aQ
L785fuyO0FHXha61ldCmAhnMYAE=
-----END CERTIFICATE-----
subject=CN = front-proxy

issuer=CN = envoy-ca

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1662 bytes and written 363 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 21 (unable to verify the first certificate)
---

回到顶部

TLS环境TCP Proxy模式配置实例

环境说明

五个Service:

envoy：Front Proxy,地址为172.31.9.2，监听于8443端口
webserver01：第一个后端服务
webserver01-sidecar：第一个后端服务的Sidecar Proxy,地址为172.31.9.11，监听于443端口
webserver02：第二个后端服务
webserver02-sidecar：第二个后端服务的Sidecar Proxy,地址为172.31.9.12, 监听于443端口

front-envoy.yaml

查看代码

 admin:
  profile_path: /tmp/envoy.prof
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
       address: 0.0.0.0
       port_value: 9901

static_resources:
  listeners:
  - name: listener_http
    address:
      socket_address: { address: 0.0.0.0, port_value: 8443 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.tcp_proxy
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
          cluster: web_cluster_01
          stat_prefix: https_passthrough

  clusters:
  - name: web_cluster_01
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: web_cluster_01
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address: { address: 172.31.9.11, port_value: 443 }
        - endpoint:
            address:
              socket_address: { address: 172.31.9.12, port_value: 443 }

envoy-sidecar-proxy_https-passthrough.yaml

查看代码

admin:
  profile_path: /tmp/envoy.prof
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
       address: 0.0.0.0
       port_value: 9901

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 443 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service 
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route: { cluster: local_cluster }
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            tls_certificates:
            - certificate_chain:
                filename: "/etc/envoy/certs/server.crt"
              private_key:
                filename: "/etc/envoy/certs/server.key"

  clusters:
  - name: local_cluster
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: local_cluster
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address: { address: 127.0.0.1, port_value: 8080 }

docker-compose.yaml

查看代码

version: '3.3'

services:
  envoy:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./front-envoy.yaml:/etc/envoy/envoy.yaml
    networks:
      envoymesh:
        ipv4_address: 172.31.9.2
        aliases:
        - front-proxy
    depends_on:
    - webserver01-sidecar
    - webserver02-sidecar

  webserver01-sidecar:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./envoy-sidecar-proxy_https-passthrough.yaml:/etc/envoy/envoy.yaml
    - ./certs/front-envoy/:/etc/envoy/certs/
    hostname: webserver01
    networks:
      envoymesh:
        ipv4_address: 172.31.9.11
        aliases:
        - webserver01-sidecar

  webserver01:
    image: ikubernetes/demoapp:v1.0
    environment:
      - PORT=8080
      - HOST=127.0.0.1
    network_mode: "service:webserver01-sidecar"
    depends_on:
    - webserver01-sidecar

  webserver02-sidecar:
    image: envoyproxy/envoy:v1.23-latest
    volumes:
    - ./envoy-sidecar-proxy_https-passthrough.yaml:/etc/envoy/envoy.yaml
    - ./certs/front-envoy/:/etc/envoy/certs/
    hostname: webserver02
    networks:
      envoymesh:
        ipv4_address: 172.31.9.12
        aliases:
        - webserver02-sidecar

  webserver02:
    image: ikubernetes/demoapp:v1.0
    environment:
      - PORT=8080
      - HOST=127.0.0.1
    network_mode: "service:webserver02-sidecar"
    depends_on:
    - webserver02-sidecar

networks:
  envoymesh:
    driver: bridge
    ipam:
      config:
        - subnet: 172.31.9.0/24

运行和测试

创建

docker-compose up

测试服务

# curl -k  https://172.31.9.2:8443/
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: webserver02, ServerIP: 172.31.9.12!
# curl -k  https://172.31.9.2:8443/
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: webserver01, ServerIP: 172.31.9.11!

停止后清理

docker-compose down

回到顶部

参考文档

https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/transport_socket/transport_socket

https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/base.proto#envoy-v3-api-msg-config-core-v3-transportsocket

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/transport_sockets/tls/v3/tls.proto#extension-envoy-transport-sockets-tls

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/transport_sockets/tls/v3/tls.proto#envoy-v3-api-msg-extensions-transport-sockets-tls-v3-downstreamtlscontext

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/transport_sockets/tls/v3/tls.proto#envoy-v3-api-msg-extensions-transport-sockets-tls-v3-upstreamtlscontext

posted @ 2022-08-20 17:47 小吉猫阅读(651) 评论(0) 编辑收藏举报

刷新页面返回顶部

登录后才能查看或发表评论，立即登录或者逛逛博客园首页

相关博文：

· Envoy TLS

· Envoy HTTP路由

· Envoy 手动TLS场景

· 数据平面Envoy部分三：Envoy静态配置

· 数据平面Envoy部分五：Envoy集群管理

阅读排行：
· winform 绘制太阳，地球，月球运作规律
· AI与.NET技术实操系列（五）：向量存储与相似性搜索在 .NET 中的实现
· 超详细：普通电脑也行Windows部署deepseek R1训练数据并当服务器共享给他人
· 上周热点回顾（3.3-3.9）
· AI 智能体引爆开源社区「GitHub 热点速览」

公告

昵称：小吉猫
园龄： 4年3个月
粉丝： 19
关注： 0

+加关注

2025年3月

日

一

二

三

四

五

六

小吉猫

Envoy TLS基础

Envoy Mesh中TLS常用场景

TLS配置位置说明

Listener

Cluster

TLS环境配置实例

环境说明

拓扑结构

创建证书目录

准备openssl.conf文件

创建证书

创建ca证书

创建front-proxy server证书

创建front-proxy client证书

修改证书权限

front-envoy.yaml

envoy-sidecar-proxy_http.yaml

envoy-sidecar-proxy_tls.yaml

envoy-sidecar-proxy_mtls.yaml

docker-compose.yaml

运行并测试

启动服务

查看证书

查看front-envoy启动的Listener

测试访问服务

http请求自动跳转至https服务上

访问http

访问tls

访问mtls

查看访问日志

openssl s_client命令

TLS环境TCP Proxy模式配置实例

环境说明

front-envoy.yaml

envoy-sidecar-proxy_https-passthrough.yaml

docker-compose.yaml

运行和测试

创建

测试服务

停止后清理

参考文档

公告

搜索

常用链接

随笔分类

随笔档案

阅读排行榜

评论排行榜

推荐排行榜

最新评论

目录导航