Nginx核心知识100讲学习笔记（陶辉）详解HTTP模块（接受请求模块|正则表达式|冲突合并）

一、冲突的配置指令以谁为准？

1、配置块的嵌套

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

main http {         upstream { … }         split_clients {…}         map {…}         geo {…}         server {                 if () {…}                 location {                         limit_except {…}                 }                 location {                         location {                         }                 }         }         server {         } }

2、指令的Context

?

1 2 3 4 5 6 7 8 9

Syntax:  log_format name [escape=default|json|none] string ...; Default:  log_format combined "..."; Context:  http   Syntax: access_log path [format [buffer=size] [gzip[=level]] [flush=time] [if=condition]]; access_log off; Default:  access_log logs/access.log combined; Context:  http, server, location, if in location, limit_excep

3、值指令：存储配置项的值（可以合并）

?

1 2 3

root access_log gzip

4、动作类指令：指定行为（不可以合并）

?

1 2	rewrite proxy_pass

生效阶段

?

1 2 3

server_rewrite 阶段 rewrite 阶段 content 阶段

5、存储值的指令集成规则：向上覆盖

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14

server {     listen 8080;     root /home/geek/nginx/html;     access_log logs/geek.access.log main;     location /test {         root /home/geek/nginx/test;         access_log logs/access.test.log main;     }     location /dlib {         alias dlib/;     }     location / {     } }

子配置不存在时，直接使用父配置块；子配置存在时，直接覆盖父配置块

二、HTTP模块合并配置的实现

1、指令在哪个块下生效？

就是11个阶段的那个阶段

2、指令允许出现在那些块下？

?

1 2 3

{ ngx_string("valid_referers"),       NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_1MORE, ......

3、在server块内生效、从http向server合并指令：

?

1	• char *(*merge_srv_conf)(ngx_conf_t *cf, void *prev, void *conf);

任何一个nginx模块中都有一个结构体叫：ngx_module_t 这个结构体提供的所有配置指令都在

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14

ngx_module_t  ngx_http_referer_module = {     NGX_MODULE_V1,     &ngx_http_referer_module_ctx,          /* module context */     ngx_http_referer_commands,             /* module directives */     NGX_HTTP_MODULE,                       /* module type */     NULL,                                  /* init master */     NULL,                                  /* init module */     NULL,                                  /* init process */     NULL,                                  /* init thread */     NULL,                                  /* exit thread */     NULL,                                  /* exit process */     NULL,                                  /* exit master */     NGX_MODULE_V1_PADDING };

这个结构体提供的所有配置指令都在都在ngx_command_t的数组中每一个元素就是一条指令

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

static ngx_command_t  ngx_http_referer_commands[] = {       { ngx_string("valid_referers"),       NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_1MORE,       ngx_http_valid_referers,       NGX_HTTP_LOC_CONF_OFFSET,       0,       NULL },       { ngx_string("referer_hash_max_size"),       NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,       ngx_conf_set_num_slot,       NGX_HTTP_LOC_CONF_OFFSET,       offsetof(ngx_http_referer_conf_t, referer_hash_max_size),       NULL },       { ngx_string("referer_hash_bucket_size"),       NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,       ngx_conf_set_num_slot,       NGX_HTTP_LOC_CONF_OFFSET,       offsetof(ngx_http_referer_conf_t, referer_hash_bucket_size),       NULL },         ngx_null_command };

定义了8个可回调方法

?

1 2 3 4 5 6 7 8 9 10 11 12 13

static ngx_http_module_t  ngx_http_referer_module_ctx = {     ngx_http_referer_add_variables,        /* preconfiguration */     NULL,                                  /* postconfiguration */       NULL,                                  /* create main configuration */     NULL,                                  /* init main configuration */       NULL,                                  /* create server configuration */     NULL,                                  /* merge server configuration */       ngx_http_referer_create_conf,          /* create location configuration */     ngx_http_referer_merge_conf            /* merge location configuration */ };

4、配置缓存在内存

?

1	• char *(*merge_loc_conf)(ngx_conf_t *cf, void *prev, void *conf);

重点看合并函数

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80

static char * ngx_http_referer_merge_conf(ngx_conf_t *cf, void *parent, void *child) {     ngx_http_referer_conf_t *prev = parent;     ngx_http_referer_conf_t *conf = child;       ngx_uint_t                 n;     ngx_hash_init_t            hash;     ngx_http_server_name_t    *sn;     ngx_http_core_srv_conf_t  *cscf;       if (conf->keys == NULL) {         conf->hash = prev->hash;   #if (NGX_PCRE)         ngx_conf_merge_ptr_value(conf->regex, prev->regex, NULL);         ngx_conf_merge_ptr_value(conf->server_name_regex,                                  prev->server_name_regex, NULL); #endif         ngx_conf_merge_value(conf->no_referer, prev->no_referer, 0);         ngx_conf_merge_value(conf->blocked_referer, prev->blocked_referer, 0);         ngx_conf_merge_uint_value(conf->referer_hash_max_size,                                   prev->referer_hash_max_size, 2048);         ngx_conf_merge_uint_value(conf->referer_hash_bucket_size,                                   prev->referer_hash_bucket_size, 64);           return NGX_CONF_OK;     }       if (conf->server_names == 1) {         cscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_core_module);           sn = cscf->server_names.elts;         for (n = 0; n < cscf->server_names.nelts; n++) {   #if (NGX_PCRE)             if (sn[n].regex) {                   if (ngx_http_add_regex_server_name(cf, conf, sn[n].regex)                     != NGX_OK)                 {                     return NGX_CONF_ERROR;                 }                   continue;             } #endif               if (ngx_http_add_referer(cf, conf->keys, &sn[n].name, NULL)                 != NGX_OK)             {                 return NGX_CONF_ERROR;             }         }     }       if ((conf->no_referer == 1 || conf->blocked_referer == 1)         && conf->keys->keys.nelts == 0         && conf->keys->dns_wc_head.nelts == 0         && conf->keys->dns_wc_tail.nelts == 0)     {         ngx_log_error(NGX_LOG_EMERG, cf->log, 0,                       "the \"none\" or \"blocked\" referers are specified "                       "in the \"valid_referers\" directive "                       "without any valid referer");         return NGX_CONF_ERROR;     }       ngx_conf_merge_uint_value(conf->referer_hash_max_size,                               prev->referer_hash_max_size, 2048);     ngx_conf_merge_uint_value(conf->referer_hash_bucket_size,                               prev->referer_hash_bucket_size, 64);     conf->referer_hash_bucket_size = ngx_align(conf->referer_hash_bucket_size,                                                ngx_cacheline_size);       hash.key = ngx_hash_key_lc;     hash.max_size = conf->referer_hash_max_size;     hash.bucket_size = conf->referer_hash_bucket_size;     hash.name = "referer_hash";     hash.pool = cf->pool;

三、Listen 指令

语法解析

?

1 2 3 4 5 6 7 8 9

Syntax:<br> listen address[:port] [default_server] [ssl] [http2 | spdy] [proxy_protocol] [setfib=number] [fastopen=number] [backlog=number] [rcvbuf=size] [sndbuf=size] [accept_filter=filter] [deferred] [bind] [ipv6only=on|off] [reuseport] [so_keepalive=on|off|[keepidle]:[keepintvl]:[keepcnt]]; listen port [default_server] [ssl] [http2 | spdy] [proxy_protocol] [setfib=number] [fastopen=number] [backlog=number] [rcvbuf=size] [sndbuf=size] [accept_filter=filter] [deferred] [bind] [ipv6only=on|off] [reuseport] [so_keepalive=on|off|[keepidle]:[keepintvl]:[keepcnt]]; listen unix:path [default_server] [ssl] [http2 | spdy] [proxy_protocol] [backlog=number] [rcvbuf=size] [sndbuf=size] [accept_filter=filter] [deferred] [bind] [so_keepalive=on|off|[keepidle]:[keepintvl]:[keepcnt]];<br> Default:  listen *:80 | *:8000; Context:  server

示例

?

1 2 3 4 5 6 7 8

listen unix:/var/run/nginx.sock; listen 127.0.0.1:8000; listen 127.0.0.1; listen 8000; listen *:8000; listen localhost:8000 bind; listen [::]:8000 ipv6only=on; listen [::1];

四、接收请求事件模块

1、图解过程

 

 

 2、过程中的疑难点

两种内存池

?

1 2	分配连接内存池：connection_pool_size:512 请求内存池

如果60s没收到请求就超时：client_header_timeout: 60s

把用户的data读取到我的用户态、这样就要分配内存：client_header_buffer_size:1k；这个很大并不合适、如果超过1k会有什么变化？

五、接收请求http模块

1、图解接收过程

 2、过程中的疑难点

1、请求内存池会分配多大？

基本是连接内存池的8倍、因为请求的上下文存在业务、如果分配的太小的话会不停的扩充会影响性能

2、分配大内存：

解决了什么问题：解决有一些url太长了、最大会是多大？
并不是分配32k、先分配一个8k，把刚刚的内容拷贝到我们的第一部分、剩下的7k再去接收剩下的url如果还没有接收完、按照以上迭代、最多分配32k

?

1	large_client_header_buffers:4 8k

3、什么叫表示URL?

nginx会有很多变量、这些变量它并不是复制一份、而是c语言的指针指向我们接收的请求行、所以nginx的性能才如此强大

这里的large_client_header_buffers:4 8k和解析请求是复用的也就是解析请求用了16k 那么head就只剩下16k

 3、过大的请求头部

?

1 2 3 4 5 6 7

Syntax:  client_header_buffer_size size; Default:  client_header_buffer_size 1k; Context:  http, server   Syntax:  large_client_header_buffers number size; Default:  large_client_header_buffers 4 8k; Context:  http, serve

六、正则表达式

1、元字符

2、重复

3、正则表达式【\和（）】

\：取消原字符的特殊含义

?

1 2	原始url：/admin/website/article/35/change/uploads/party/5.jpg 转换后的：/static/uploads/party/5.jpg

（）分组与取值

?

1 2 3

匹配原始url的正则表达式： /^\/admin\/website\/article\/(\d+)\/change\/uploads\/(\w+)\/(\w+)\.(png|jpg|gif|jpeg|bmp)$/ rewrite^/admin/website/solution/(\d+)/change/uploads/(.*)\.(png|jpg|gif|jpeg|bmp)$/static/uploads/$2/$3.$4 last;

4、正则表达式的验证pcre2test

安装

?

1 2 3 4 5 6

wget file:///C:/Users/lah/Downloads/pcre2-10.35.tar.gz tar xf pcre2-10.35.tar.gz cd pcre2-10.35 ./configure --enable-utf8 make && make install echo $?

测试验证：

?

1 2 3 4 5 6 7 8 9 10

[root@ceph-client ~]# pcre2test PCRE2 version 10.35 2020-05-09 re> /^\/admin\/website\/article\/(\d+)\/change\/uploads\/(\w+)\/(\w+)\.(png|jpg|gif|jpeg|b mp)$/ > data> /admin/website/article/35/change/uploads/party/5.jpg 0: /admin/website/article/35/change/uploads/party/5.jpg 1: 35 2: party 3: 5 4: jpg

注意事项：

1、在nginx中使用不需要添加/但是在pcre2test中需要加

2、不能带点、所以这里的.我用了转义符号

?

1	\.(png|jpg|gif|jpeg|bmp)$/