ingress 日志切割

背景

ingress本质上也是个nginx，多个项目日志全部往access.log中输入就显得杂乱无章可读性不高，不利于日志阅读关键信息获取，排错就异常痛苦。

so 闲话少叙开干！

1、部署ingress

我这里ingress是用helm部署的

[root@master ~]# helm list
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
myingress       default         1               2024-05-28 15:40:52.042004305 +0800 CST deployed        nginx-ingress-controller-9.3.18 1.4.0      
[root@master ~]#

　还不知道咋部署的或者也想简单快速搭建起ingress测试环境，可以参考我前面写的博客，

2、部署测试应用

apache

[root@master nginx-ingress-controller]# cat apache.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  name: web01
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpd01
  template:
    metadata:
      labels:
        app: httpd01
    spec:
      containers:
      - name: httpd
        image: httpd:latest
---
apiVersion: v1
kind: Service
metadata:
  name: httpd-svc
  namespace: default
spec:
  selector:
    app: httpd01
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

　tomcat

[root@master nginx-ingress-controller]# cat tomcat.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  name: web02
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tomcat01
  template:
    metadata:
      labels:
        app: tomcat01
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5.45
---
apiVersion: v1
kind: Service
metadata:
  name: tomcat-svc
  namespace: default
spec:
  selector:
    app: tomcat01
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080

　　检查一下应用情况

[root@master nginx-ingress-controller]# kubectl get pod,svc 
NAME                                                      READY   STATUS    RESTARTS   AGE
pod/myingress-nginx-ingress-controller-796ddc4cc7-d7pqq   1/1     Running   0          52m
pod/web01-c94b5464f-xj848                                 1/1     Running   0          29h
pod/web02-6d96d48778-ntl5w                                1/1     Running   0          29h
 
NAME                                         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
service/httpd-svc                            ClusterIP   10.68.62.49    <none>        80/TCP                       29h
service/kubernetes                           ClusterIP   10.68.0.1      <none>        443/TCP                      91d
service/myingress-nginx-ingress-controller   NodePort    10.68.133.2    <none>        80:32080/TCP,443:32013/TCP   52m
service/tomcat-svc                           ClusterIP   10.68.78.237   <none>        8080/TCP                     29h
[root@master nginx-ingress-controller]# 

3、测试应用访问情况　

分别创建apache和tomcat的ingress

[root@master nginx-ingress-controller]# cat ingress-httpd.yaml 
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpd-ingress
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
    - host: www.httpd.com
      http:
        paths:
          - backend:
              service:
                name: httpd-svc
                port:
                  number: 80
            path: /
            pathType: Prefix

[root@master nginx-ingress-controller]# cat ingress-tomcat.yaml 
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tomcat-ingress
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
    - host: www.tomcat.com
      http:
        paths:
          - backend:
              service:
                name: tomcat-svc
                port:
                  number: 8080
            path: /tomcat
            pathType: Prefix

　　添加本地host访问验证

[root@master nginx-ingress-controller]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
 
192.168.0.223 www.httpd.com
192.168.0.223 www.tomcat.com
 
# 访问apache
[root@master nginx-ingress-controller]# curl  www.httpd.com:32080
<html><body><h1>It works!</h1></body></html>
# 访问tomcat
[root@master nginx-ingress-controller]# curl -I  www.tomcat.com:32080/tomcat
HTTP/1.1 200 
Date: Tue, 28 May 2024 08:38:00 GMT
Content-Type: text/html;charset=UTF-8
Connection: keep-alive

查看access日志

I have no name!@myingress-nginx-ingress-controller-796ddc4cc7-d7pqq:/var/log/nginx$ more access.log 
20.196.128 - - [28/May/2024:07:42:37 +0000] "HEAD / HTTP/1.1" 200 0 "-" "curl/7.29.0" 84 0.001 [default-httpd-svc-80] [] 172.20.196.134:80 0 0.001 200 d6c33f2841728141dab47129e3b3f0f0
20.196.128 - - [28/May/2024:07:42:43 +0000] "GET / HTTP/1.1" 200 45 "-" "curl/7.29.0" 83 0.001 [default-httpd-svc-80] [] 172.20.196.134:80 45 0.001 200 aa51f172de9a0cdb699347d624c7608f
20.196.128 - - [28/May/2024:07:43:08 +0000] "GET /tomcat HTTP/1.1" 200 11211 "-" "curl/7.29.0" 90 0.004 [default-tomcat-svc-8080] [] 172.20.196.135:8080 11204 0.005 200 689d4315f3767be1ed9b2d1bae2b7361
20.196.128 - - [28/May/2024:08:38:00 +0000] "HEAD /tomcat HTTP/1.1" 200 0 "-" "curl/7.29.0" 91 0.003 [default-tomcat-svc-8080] [] 172.20.196.135:8080 0 0.003 200 3eb1c665887e3a2351324b1c4fcfecc7

可以看到两个应用日志都在一个文件中

4、设置日志格式

修改valus.yaml文件，配置日志格式（这一步不是必要的，但是既然日志做切割了，为什么不让格式更好看一些呢）

[root@master nginx-ingress-controller]# vim values.yaml<br>...<br>## ref: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/
##
config:
  log-format: |
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

更新helm应用

1	`[root@master nginx-ingress-controller]# helm upgrade myingress -f values.yaml --namespace default .`

　　修改apache应用日志的输出文件位置

[root@master ~]# kubectl edit ingress httpd-ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/server-snippet: |       # 添加这两行
      access_log /var/log/nginx/httpd_access.log;
    nginx.ingress.kubernetes.io/use-regex: "true"
  name: httpd-ingress
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: www.httpd.com
    http:
      paths:
      - backend:
          service:
            name: httpd-svc
            port:
              number: 80
        path: /
        pathType: Prefix

　修改好之后在去看nginx controller pod 日志文件路径

1 2	`I have no name!@myingress-nginx-ingress-controller-796ddc4cc7-d7pqq:/var/log/nginx$` `ls` `access.log error.log httpd_access.log`

　再访问几次看看内容

I have no name!@myingress-nginx-ingress-controller-796ddc4cc7-d7pqq:/var/log/nginx$ more httpd_access.log 
172.20.196.128 - - [28/May/2024:07:52:43 +0000] "GET / HTTP/1.1" 200 45 "-" "curl/7.29.0"
172.20.196.128 - - [28/May/2024:07:52:45 +0000] "GET / HTTP/1.1" 200 45 "-" "curl/7.29.0"

　到这里日志按项目切割就完成了

5、按日期切割

其实这里我主要纠结点在于ingress是在k8s中的，而不是二进制的，是不是不太好操作呢。其实是自己瞎自己O(∩_∩)O

步骤跟二进制的nginx日志按日期切割一样

1 2	`第一步：重命名旧的access.log` `第二部：执行` `kill` -USR1 ``cat` `/tmp/nginx/nginx`.pid` 让ingress生成新的access.log

　具体日志实现

#!/bin/bash
 
set -eu
 
log_dir=/var/log/nginx     #源日志目录
back_logs_path="$log_dir/backup/$(date -d 'yesterday' +'%F')"     #备份日志目录
pod_name=$(kubectl get pod -n test| grep controller | awk '{print $1}')
 
# 检查是否找到 pod
if [ -z "$pod_name" ]; then
  echo "No pod found matching 'controller'"
  exit 1
fi
 
# 创建备份目录
kubectl exec "$pod_name" -n test -- sh -c "mkdir -p ${back_logs_path}"
 
# 获取ingress进程号
ng_pid=$(kubectl exec "$pod_name" -n test -- cat /tmp/nginx/nginx.pid)
 
 
# 检查ng_pid是否为空
if [ -z "$ng_pid" ]; then
  echo "Failed to get nginx PID."
  exit 1
fi
 
kubectl exec "$pod_name" -n test -- sh -c "find $log_dir -maxdepth 1 -name '*access*' | xargs -I {} mv {} {}.$(date -d 'yesterday' +'%F')"
 
kubectl exec "$pod_name" -n test  -- sh -c "find $log_dir -maxdepth 1 -name '*access*' | xargs -I {}  mv {} $back_logs_path"
 
kubectl exec "$pod_name" -n test  -- sh -c "kill -USR1 $ng_pid"