ingress 日志切割
背景
ingress本质上也是个nginx，多个项目日志全部往access.log中输入就显得杂乱无章可读性不高，不利于日志阅读关键信息获取，排错就异常痛苦。
so 闲话少叙开干！
1、部署ingress
我这里ingress是用helm部署的
[root@master ~]# helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION myingress default 1 2024-05-28 15:40:52.042004305 +0800 CST deployed nginx-ingress-controller-9.3.18 1.4.0 [root@master ~]#
还不知道咋部署的或者也想简单快速搭建起ingress测试环境，可以参考我前面写的博客，
2、部署测试应用
apache
[root@master nginx-ingress-controller]# cat apache.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: web01
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: httpd01
template:
metadata:
labels:
app: httpd01
spec:
containers:
- name: httpd
image: httpd:latest
---
apiVersion: v1
kind: Service
metadata:
name: httpd-svc
namespace: default
spec:
selector:
app: httpd01
ports:
- protocol: TCP
port: 80
targetPort: 80
tomcat
[root@master nginx-ingress-controller]# cat tomcat.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: web02
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: tomcat01
template:
metadata:
labels:
app: tomcat01
spec:
containers:
- name: tomcat
image: tomcat:8.5.45
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-svc
namespace: default
spec:
selector:
app: tomcat01
ports:
- protocol: TCP
port: 8080
targetPort: 8080
检查一下应用情况
[root@master nginx-ingress-controller]# kubectl get pod,svc NAME READY STATUS RESTARTS AGE pod/myingress-nginx-ingress-controller-796ddc4cc7-d7pqq 1/1 Running 0 52m pod/web01-c94b5464f-xj848 1/1 Running 0 29h pod/web02-6d96d48778-ntl5w 1/1 Running 0 29h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/httpd-svc ClusterIP 10.68.62.49 <none> 80/TCP 29h service/kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 91d service/myingress-nginx-ingress-controller NodePort 10.68.133.2 <none> 80:32080/TCP,443:32013/TCP 52m service/tomcat-svc ClusterIP 10.68.78.237 <none> 8080/TCP 29h [root@master nginx-ingress-controller]#
3、测试应用访问情况
分别创建apache和tomcat的ingress
[root@master nginx-ingress-controller]# cat ingress-httpd.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: httpd-ingress
namespace: default
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: www.httpd.com
http:
paths:
- backend:
service:
name: httpd-svc
port:
number: 80
path: /
pathType: Prefix
[root@master nginx-ingress-controller]# cat ingress-tomcat.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: tomcat-ingress
namespace: default
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: www.tomcat.com
http:
paths:
- backend:
service:
name: tomcat-svc
port:
number: 8080
path: /tomcat
pathType: Prefix
添加本地host访问验证
[root@master nginx-ingress-controller]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.0.223 www.httpd.com 192.168.0.223 www.tomcat.com # 访问apache [root@master nginx-ingress-controller]# curl www.httpd.com:32080 <html><body><h1>It works!</h1></body></html> # 访问tomcat [root@master nginx-ingress-controller]# curl -I www.tomcat.com:32080/tomcat HTTP/1.1 200 Date: Tue, 28 May 2024 08:38:00 GMT Content-Type: text/html;charset=UTF-8 Connection: keep-alive
查看access日志
I have no name!@myingress-nginx-ingress-controller-796ddc4cc7-d7pqq:/var/log/nginx$ more access.log 172.20.196.128 - - [28/May/2024:07:42:37 +0000] "HEAD / HTTP/1.1" 200 0 "-" "curl/7.29.0" 84 0.001 [default-httpd-svc-80] [] 172.20.196.134:80 0 0.001 200 d6c33f2841728141dab47129e3b3f0f0 172.20.196.128 - - [28/May/2024:07:42:43 +0000] "GET / HTTP/1.1" 200 45 "-" "curl/7.29.0" 83 0.001 [default-httpd-svc-80] [] 172.20.196.134:80 45 0.001 200 aa51f172de9a0cdb699347d624c7608f 172.20.196.128 - - [28/May/2024:07:43:08 +0000] "GET /tomcat HTTP/1.1" 200 11211 "-" "curl/7.29.0" 90 0.004 [default-tomcat-svc-8080] [] 172.20.196.135:8080 11204 0.005 200 689d4315f3767be1ed9b2d1bae2b7361 172.20.196.128 - - [28/May/2024:08:38:00 +0000] "HEAD /tomcat HTTP/1.1" 200 0 "-" "curl/7.29.0" 91 0.003 [default-tomcat-svc-8080] [] 172.20.196.135:8080 0 0.003 200 3eb1c665887e3a2351324b1c4fcfecc7
可以看到两个应用日志都在一个文件中
4、设置日志格式
修改valus.yaml文件，配置日志格式（这一步不是必要的，但是既然日志做切割了，为什么不让格式更好看一些呢）
[root@master nginx-ingress-controller]# vim values.yaml
...
## ref: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/ ## config: log-format: | log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"';
更新helm应用
[root@master nginx-ingress-controller]# helm upgrade myingress -f values.yaml --namespace default .
修改apache应用日志的输出文件位置
[root@master ~]# kubectl edit ingress httpd-ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/server-snippet: | # 添加这两行
access_log /var/log/nginx/httpd_access.log;
nginx.ingress.kubernetes.io/use-regex: "true"
name: httpd-ingress
namespace: default
spec:
ingressClassName: nginx
rules:
- host: www.httpd.com
http:
paths:
- backend:
service:
name: httpd-svc
port:
number: 80
path: /
pathType: Prefix
修改好之后在去看nginx controller pod 日志文件路径
I have no name!@myingress-nginx-ingress-controller-796ddc4cc7-d7pqq:/var/log/nginx$ ls access.log error.log httpd_access.log
再访问几次看看内容
I have no name!@myingress-nginx-ingress-controller-796ddc4cc7-d7pqq:/var/log/nginx$ more httpd_access.log 172.20.196.128 - - [28/May/2024:07:52:43 +0000] "GET / HTTP/1.1" 200 45 "-" "curl/7.29.0" 172.20.196.128 - - [28/May/2024:07:52:45 +0000] "GET / HTTP/1.1" 200 45 "-" "curl/7.29.0"
到这里日志按项目切割就完成了
5、按日期切割
其实这里我主要纠结点在于ingress是在k8s中的，而不是二进制的，是不是不太好操作呢。 其实是自己瞎自己O(∩_∩)O
步骤跟二进制的nginx日志按日期切割一样
第一步：重命名旧的access.log 第二部：执行 kill -USR1 `cat /tmp/nginx/nginx.pid` 让ingress生成新的access.log
具体日志实现
#!/bin/bash
set -eu
log_dir=/var/log/nginx #源日志目录
back_logs_path="$log_dir/backup/$(date -d 'yesterday' +'%F')" #备份日志目录
pod_name=$(kubectl get pod -n test| grep controller | awk '{print $1}')
# 检查是否找到 pod
if [ -z "$pod_name" ]; then
echo "No pod found matching 'controller'"
exit 1
fi
# 创建备份目录
kubectl exec "$pod_name" -n test -- sh -c "mkdir -p ${back_logs_path}"
# 获取ingress进程号
ng_pid=$(kubectl exec "$pod_name" -n test -- cat /tmp/nginx/nginx.pid)
# 检查ng_pid是否为空
if [ -z "$ng_pid" ]; then
echo "Failed to get nginx PID."
exit 1
fi
kubectl exec "$pod_name" -n test -- sh -c "find $log_dir -maxdepth 1 -name '*access*' | xargs -I {} mv {} {}.$(date -d 'yesterday' +'%F')"
kubectl exec "$pod_name" -n test -- sh -c "find $log_dir -maxdepth 1 -name '*access*' | xargs -I {} mv {} $back_logs_path"
kubectl exec "$pod_name" -n test -- sh -c "kill -USR1 $ng_pid"
辛苦你看到最后，奖励一张美女图片
