Nginx高级配置-自定义json格式日志
Nginx高级配置-自定义json格式日志
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
在大数据运维工作中，我们经常会使用flume，filebeat相关日志收集工具取收集日志，但这些日志在收集前都日志基本上都是json格式的，通过flume收集日志到hdfs集群，开发人员就直接使用java，scala语言取处理日志，有的时候会使用到spark,fink等框架去处理日志。因此nginx配置为json格式还是非常有必要的。
访问日志是记录客户端即用户的具体请求内容信息，全局配置模块中的error_log是记录nginx服务器运行时的日志保存路径和记录日志的level，因此有着本质的区别，而且Nginx的错误日志一般只有一个，但是访问日志可以在不同server中定义多个，定义一个日志需要使用access_log指定日志的保存路径，使用log_format指定日志的格式，格式中定义要保存的具体日志内容。
一.自定义默认格式日志
如果是要保留日志的源格式，只是添加相应的日志内容，则配置如下。
关于nginx日志使用的变量名称含义，博主推荐阅读:
https://www.cnblogs.com/yinzhengjie/p/12046613.html
1>.编写主配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/nginx/conf/nginx.conf
worker_processes 4;
worker_cpu_affinity 00000001 00000010 00000100 00001000;
events {
worker_connections 100000;
use epoll;
accept_mutex on;
multi_accept on;
}
http {
include mime.types;
default_type text/html;
charset utf-8;
log_format my_default_format '$remote_addr - $remote_user [$time_local] "$request"' '$status $body_bytes_sent "$http_referer"' '"$http_user_agent"' '"$http_x_forwarded_
for"' '$server_name:$server_port';
access_log logs/access.log my_default_format;
include /yinzhengjie/softwares/nginx/conf.d/*.conf;
}
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# nginx -t
nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful
[root@node101.yinzhengjie.org.cn ~]#
2>.编写子配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/nginx/conf.d/share.conf
server {
listen 80;
server_name node101.yinzhengjie.org.cn;
location / {
root /yinzhengjie/data/web/nginx/static;
index index.html;
}
location /nginx_status {
stub_status;
allow 172.30.1.108;
deny all;
}
location /main {
index index.html;
default_type text/html;
set $name jason;
set $nginx_name $server_name;
echo "姓名: $name";
echo "************";
echo "Nginx服务器名称: $nginx_name";
}
}
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# nginx -t
nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful
[root@node101.yinzhengjie.org.cn ~]#
3>.重新加载nginx的配置文件
[root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep
root 9297 1 0 Dec17 ? 00:00:00 nginx: master process nginx
nginx 11823 9297 0 12:50 ? 00:00:00 nginx: worker process
nginx 11824 9297 0 12:50 ? 00:00:00 nginx: worker process
nginx 11825 9297 0 12:50 ? 00:00:00 nginx: worker process
nginx 11826 9297 0 12:50 ? 00:00:00 nginx: worker process
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# nginx -s reload
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep
root 9297 1 0 Dec17 ? 00:00:00 nginx: master process nginx
nginx 11890 9297 1 12:57 ? 00:00:00 nginx: worker process
nginx 11891 9297 1 12:57 ? 00:00:00 nginx: worker process
nginx 11892 9297 1 12:57 ? 00:00:00 nginx: worker process
nginx 11893 9297 1 12:57 ? 00:00:00 nginx: worker process
[root@node101.yinzhengjie.org.cn ~]#
4>.浏览器访问"http://node101.yinzhengjie.org.cn/main"并查看日志格式,如下图所示。
二.自定义json格式日志
Nginx的默认访问日志记录内容相对比较单一，默认的格式也不方便后期做日志统计分析，生产环境中通常将nginx日志转换为json日志，然后配合使用ELK做日志收集-统计-分析。
1>.编辑主配置文件
[root@node101.yinzhengjie.org.cn ~]# cat -n /yinzhengjie/softwares/nginx/conf/nginx.conf
1 worker_processes 4;
2 worker_cpu_affinity 00000001 00000010 00000100 00001000;
3
4 events {
5 worker_connections 100000;
6 use epoll;
7 accept_mutex on;
8 multi_accept on;
9 }
10
11 http {
12 include mime.types;
13
14 default_type text/html;
15
16 charset utf-8;
17
18 log_format my_access_json '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"clientip":"$remote_addr",' '"size":$body_bytes_sent,' '"responsetime":$request_time,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"uri":"$uri",' '"domain":"$host",' '"xff":"$http_x_forwarded_for",' '"referer":"$http_referer",' '"tcp_xff":"$proxy_protocol_addr",' '"http_user_agent":"$http_user_agent",' '"status":"$status"}';
19
20
21 access_log logs/access_json.log my_access_json;
22
23 include /yinzhengjie/softwares/nginx/conf.d/*.conf;
24 }
25
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# nginx -t
nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful
[root@node101.yinzhengjie.org.cn ~]#
2>.编辑子配置文件
[root@node101.yinzhengjie.org.cn ~]# cat -n /yinzhengjie/softwares/nginx/conf.d/share.conf
1 server {
2 listen 80;
3 server_name node101.yinzhengjie.org.cn;
4
5 location / {
6 root /yinzhengjie/data/web/nginx/static;
7 index index.html;
8 }
9
10 location /nginx_status {
11 stub_status;
12 allow 172.30.1.108;
13 deny all;
14 }
15
16 location /main {
17 index index.html;
18 default_type text/html;
19 set $name jason;
20 set $nginx_name $server_name;
21 echo "姓名: $name";
22 echo "************";
23 echo "Nginx服务器名称: $nginx_name";
24 }
25
26 }
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# nginx -t
nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
3>.重新加载nginx配置文件
[root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep
root 9297 1 0 Dec17 ? 00:00:00 nginx: master process nginx
nginx 11890 9297 0 12:57 ? 00:00:00 nginx: worker process
nginx 11891 9297 0 12:57 ? 00:00:00 nginx: worker process
nginx 11892 9297 0 12:57 ? 00:00:00 nginx: worker process
nginx 11893 9297 0 12:57 ? 00:00:00 nginx: worker process
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# nginx -s reload
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep
root 9297 1 0 Dec17 ? 00:00:00 nginx: master process nginx
nginx 11946 9297 1 13:25 ? 00:00:00 nginx: worker process
nginx 11947 9297 1 13:25 ? 00:00:00 nginx: worker process
nginx 11948 9297 0 13:25 ? 00:00:00 nginx: worker process
nginx 11949 9297 1 13:25 ? 00:00:00 nginx: worker process
[root@node101.yinzhengjie.org.cn ~]#
4>.浏览器访问"http://node101.yinzhengjie.org.cn/main"并查看日志格式,如下图所示。
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。
欢迎交流学习技术交流，个人微信: "JasonYin2020"（添加时请备注来源及意图备注）
作者: 尹正杰, 博客: https://www.cnblogs.com/yinzhengjie/p/12047186.html
