EFK收集Envoy的访问日志
## 使用EFK收集Envoy的访问日志
### 环境说明
##### Envoy Mesh使用的网络: 172.31.76.0/24
##### 7个Service:
- front-envoy：Front Proxy,地址为172.31.76.10
- 3个后端服务，仅是用于提供测试用的上游服务器
- service_blue
- service_red
- service_green
- 三个日志服务
- elasticsearch，地址为172.31.76.15，绑定宿主机的9200端口
- kibana，地址为172.31.76.16，绑定宿主机的5601端口
- filebeat
##### 特殊要求
目录logs/envoy/下的日志文件front-envoy-access.log的属主需要修改为envoy容器中运行envoy进程的用户envoy，其UID和GID默认分别为100和101，否则，front-envoy进程将日志写入到该文件时，将显示为“Permission Denied.”
```
chown 100.101 logs/envoy/front-envoy-access.log
启动
docker-compose up# cat docker-compose.yaml
# Author: MageEdu <mage@magedu.com>
version: '3.3'
services:
front-envoy:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
container_name: front-envoy
volumes:
- ./front-envoy.yaml:/etc/envoy/envoy.yaml
- ./logs/envoy:/logs/envoy
networks:
envoymesh:
ipv4_address: 172.31.76.10
aliases:
- front-envoy
expose:
# Expose ports 80 (for general traffic) and 9901 (for the admin server)
- "80"
- "9901"
service_blue:
image: ikubernetes/servicemesh-app:latest
networks:
envoymesh:
aliases:
- colored
- blue
environment:
- SERVICE_NAME=blue
expose:
- "80"
service_green:
image: ikubernetes/servicemesh-app:latest
networks:
envoymesh:
aliases:
- colored
- green
environment:
- SERVICE_NAME=green
expose:
- "80"
service_red:
image: ikubernetes/servicemesh-app:latest
networks:
envoymesh:
aliases:
- colored
- red
environment:
- SERVICE_NAME=red
expose:
- "80"
elasticsearch:
image: "docker.elastic.co/elasticsearch/elasticsearch:7.14.2"
environment:
- "ES_JAVA_OPTS=-Xms1g -Xmx1g"
- "discovery.type=single-node"
- "cluster.name=myes"
- "node.name=myes01"
ulimits:
memlock:
soft: -1
hard: -1
networks:
envoymesh:
ipv4_address: 172.31.76.15
aliases:
- es
- myes01
ports:
- "9200:9200"
volumes:
- elasticsearch_data:/usr/share/elasticsearch/data
kibana:
image: "docker.elastic.co/kibana/kibana:7.14.2"
environment:
ELASTICSEARCH_URL: http://myes01:9200
ELASTICSEARCH_HOSTS: '["http://myes01:9200"]'
networks:
envoymesh:
ipv4_address: 172.31.76.16
aliases:
- kibana
- kib
ports:
- "5601:5601"
filebeat:
image: "docker.elastic.co/beats/filebeat:7.14.2"
networks:
envoymesh:
ipv4_address: 172.31.76.17
aliases:
- filebeat
- fb
user: root
command: ["--strict.perms=false"]
volumes:
- ./filebeat/filebeat.yaml:/usr/share/filebeat/filebeat.yml
#- /var/lib/docker:/var/lib/docker:ro
#- /var/run/docker.sock:/var/run/docker.sock
- ./logs/envoy:/logs/envoy:ro
volumes:
elasticsearch_data:
networks:
envoymesh:
driver: bridge
ipam:
config:
- subnet: 172.31.76.0/24
# cat front-envoy.yaml
node:
id: front-envoy
cluster: mycluster
admin:
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
layered_runtime:
layers:
- name: admin
admin_layer: {}
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 80
name: listener_http
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
access_log:
- name: envoy.access_loggers.file
typed_config:
"@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: "/logs/envoy/front-envoy-access.log"
log_format:
json_format: {"start": "[%START_TIME%] ", "method": "%REQ(:METHOD)%", "url": "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%", "protocol": "%PROTOCOL%", "status": "%RESPONSE_CODE%", "respflags": "%RESPONSE_FLAGS%", "bytes-received": "%BYTES_RECEIVED%", "bytes-sent": "%BYTES_SENT%", "duration": "%DURATION%", "upstream-service-time": "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%", "x-forwarded-for": "%REQ(X-FORWARDED-FOR)%", "user-agent": "%REQ(USER-AGENT)%", "request-id": "%REQ(X-REQUEST-ID)%", "authority": "%REQ(:AUTHORITY)%", "upstream-host": "%UPSTREAM_HOST%", "remote-ip": "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"}
#text_format: "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\" \"%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%\"\n"
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: vh_001
domains: ["*"]
routes:
- match:
prefix: "/"
route:
cluster: mycluster
http_filters:
- name: envoy.filters.http.router
clusters:
- name: mycluster
connect_timeout: 0.25s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: mycluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: colored
port_value: 80
# cat filebeat/filebeat.yaml
filebeat.inputs:
- type: log
paths:
- '/logs/envoy/*access.log'
processors:
- decode_json_fields:
fields: ["message"]
target: ""
overwrite_keys: true
output.elasticsearch:
hosts: ["http://elasticsearch:9200"]
indices:
- index: "filebeat-%{+yyyy.MM.dd}"
# - index: "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}"
#output.console:
# enabled: true
# codec.json:
# pretty: true
# #escape_html: false
logging.json: true
logging.metrics.enabled: false
# cat logs/envoy/front-envoy-access.log 如果docker-compose日志汇总报accesslog-with-efk-elasticsearch-1 exited with code 137，扩大虚拟机内存。
访问es测试
curl 172.31.76.15:9200
curl 172.31.76.15:9200/_cat/indices
文本日志
先使用类似如下命令向Front-Envoy发起请求，以便持续生成访问日志；
while true; do curl 172.31.76.10/service/colors; sleep 0.$RANDOM; done登录Kibana添加index
kibana可以正常看到日志
