w10 http基本原理 & Nginx部署
一、http 基本原理
1 用户访问网站的原理 1.0 DNS 全称Domain Name System/Server,它在一个网站运行中起到了至关重要的作用,它的主要作用是负责把网站域名解析为对应的IP地址。 www.baidu.com ==== www.baidu.com. 13台根服务器 根域名 . 顶级域名 .com .org 二级域名 baidu.com. baidu.cn. 三级域名 www.lewenedu.com bbs.lewenedu.com 1.1 dns本地缓存的 Windows是有的,Linux 没有,只有hosts [E:\~]$ ipconfig /displaydns 清空用 ipconfig /flushdns 剩下就只有 \etc\hosts 1.2 DNS记录的类型 A记录 www.lewen.org. 118.26.204.147 域名对应ip记录 CNAME记录 www.lewenedu.com www.lewen.org 别名记录 找 a 给你 b
1.3 dig命令
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@web01 ~]# yum install bind-utils -y [root@oldboy ~]# dig +trace www.etiantian.org ; <<>> DiG 9.3.6-P1-RedHat-9.3.6-20.P1.el5_8.2 <<>> +trace www.etiantian.org ;; global options: printcmd . 25991 IN NS m.root-servers.<wiz_tmp_plugin_tag id="wizKMHighlighterSpan_t_t" style="cursor: pointer; border-bottom-width: 1px; border-bottom-color: rgb(0, 0, 204); border-bottom-style: dashed; background-color: rgb(255, 255, 0);">net</wiz_tmp_plugin_tag>. . 25991 IN NS b.root-servers.net. . 25991 IN NS g.root-servers.net. . 25991 IN NS k.root-servers.net. . 25991 IN NS f.root-servers.net. . 25991 IN NS l.root-servers.net. . 25991 IN NS d.root-servers.net. . 25991 IN NS c.root-servers.net. . 25991 IN NS i.root-servers.net. . 25991 IN NS h.root-servers.net. . 25991 IN NS j.root-servers.net. . 25991 IN NS e.root-servers.net. . 25991 IN NS a.root-servers.net. ;; Received 436 bytes from 192.168.120.13#53(192.168.120.13) in 1 ms # a-m一共13个root DNS服务器, m.root-servers.net根DNS负责了此次的解析 org. 172800 IN NS b0.org.afilias-nst.org. org. 172800 IN NS d0.org.afilias-nst.org. org. 172800 IN NS b2.org.afilias-nst.org. org. 172800 IN NS c0.org.afilias-nst.info. org. 172800 IN NS a0.org.afilias-nst.info. org. 172800 IN NS a2.org.afilias-nst.info. ;; Received 437 bytes from 202.12.27.33#53(m.root-servers.net) in 136 ms #解析org的一共6个DNS服务器, .org的b0.org.afilias-nst.org. DNS负责了此次的解析 etiantian.org. 86400 IN NS dns.bizcn.com. etiantian.org. 86400 IN NS dns.cnmsn.net. ;; Received 89 bytes from 199.19.54.1#53(b0.org.afilias-nst.org) in 331 ms #解析etiantian.org的一共2个DNS服务器,授权DNS dns.bizcn.com负责了最终的A记录解析 www.etiantian.org. 600 IN A 118.26.204.147 etiantian.org. 600 IN NS dns.cnmsn.net. etiantian.org. 600 IN NS dns.bizcn.com. ;; Received 169 bytes from 218.93.205.110#53(dns.bizcn.com) in 39 ms</span>
2 HTTP协议
全称HyperText Transfer Protocol,中文名为超文本传输协议,是互联网中最常用的一种网络协议。 HTTP的重要应用之一是WWW服务。
2.1 http协议原理
a.基于socket使用TCP协议进行数据传输; b.短连接,一次请求和响应之后断开连接; c.无状态,由于短连接的特性所引发的现象。 数据格式:
请求:
请求头:
请求体:
"GET /index http1.1.\r\nhost:www.baidu.com\r\ncontent-type:json\r\n\r\n"
"POST /index http1.1.\r\nhost:www.baidu.com\r\ncontent-type:json\r\n\r\nname=alex&age=18"
响应:
响应头:
响应体:
"Location:xxxx\r\n\r\n"
URL 统一资源定位符号 www.lewenedu.com www.lewenedu.com/image/lewen.jpg URI:/image/lewen.jpg
URI 统一资源标识符号 范围大些 www.lewenedu.com/lewen.avi URI:/lewen.avi
2.2 http响应报文与实战检查
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@web01 ~]# curl -v www.baidu.com * About to connect() to www.baidu.com port 80 (#0) * Trying 111.13.100.92... connected * Connected to www.baidu.com (111.13.100.92) port 80 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 > Host: www.baidu.com > Accept: */* > < HTTP/1.1 200 OK < Accept-Ranges: bytes < Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform < Connection: Keep-Alive < Content-Length: 2381 < Content-Type: text/html < Date: Tue, 21 Aug 2018 10:30:47 GMT < Etag: "588604c8-94d" < Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT < Pragma: no-cache < Server: bfe/1.0.8.18 < Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/ < <!DOCTYPE html> <!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always name=referrer><link rel=stylesheet type=text/css href=http://s1.bdstatic.com/r/www/cache/bdorz/baidu.min.css><title>百度一下,你就知道</title></head> <body link=#0000cc> <div id=wrapper> <div id=head> <div class=head_wrapper> <div class=s_form> <div class=s_form_wrapper> <div id=lg> <img hidefocus=true src=//www.baidu.com/img/bd_logo1.png width=270 height=129> </div> <form id=form name=f action=//www.baidu.com/s class=fm> <input type=hidden name=bdorz_come value=1> <input type=hidden name=ie value=utf-8> <input type=hidden name=f value=8> <input type=hidden name=rsv_bp value=1> <input type=hidden name=rsv_idx value=1> <input type=hidden name=tn value=baidu><span class="bg s_ipt_wr"><input id=kw name=wd class=s_ipt value maxlength=255 autocomplete=off autofocus></span><span class="bg s_btn_wr"><input type=submit id=su value=百度一下 class="bg s_btn"></span> </form> </div> </div> <div id=u1> <a href=http://news.baidu.com name=tj_trnews class=mnav>新闻</a> <a href=http://www.hao123.com name=tj_trhao123 class=mnav>hao123</a> <a href=http://map.baidu.com name=tj_trmap class=mnav>地图</a> <a href=http://v.baidu.com name=tj_trvideo class=mnav>视频</a> <a href=http://tieba.baidu.com name=tj_trtieba class=mnav>贴吧</a> <noscript> <a href=http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2f%3fbdorz_come%3d1 name=tj_login class=lb>登录</a> </noscript> <script>document.write('<a href="http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u='+ encodeURIComponent(window.location.href+ (window.location.search === "" ? "?" : "&")+ "bdorz_come=1")+ '" name="tj_login" class="lb">登录</a>');</script> <a href=//www.baidu.com/more/ name=tj_briicon class=bri style="display: block;">更多产品</a> </div> </div> </div> <div id=ftCon> <div id=ftConw> <p id=lh> <a href=http://home.baidu.com>关于百度</a> <a href=http://ir.baidu.com>About Baidu</a> </p> <p id=cp>©2017 Baidu <a href=http://www.baidu.com/duty/>使用百度前必读</a> <a href=http://jianyi.baidu.com/ class=cp-feedback>意见反馈</a> 京ICP证030173号 <img src=//www.baidu.com/img/gs.gif> </p> </div> </div> </div> </body> </html> * Connection #0 to host www.baidu.com left intact * Closing connection #0
2.3 http状态码讲解
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
Nginx 403 forbidden多种原因及故障模拟重现 http://lewen.blog.51cto.com/2561410/1633952 apache服务Forbidden 403问题精彩总结 http://lewen.blog.51cto.com/2561410/581383 生产环境常见HTTP状态码的博客文章见 http://lewen.blog.51cto.com/2561410/716294
网站流量术语 2.4 衡量网站访问量 PV page view 每个页面的访问量 UV 独立的用户数量(浏览器名称+型号+系统版本+其他) IP 独立的ip数量。 假设一个客户机用户通过ADSL等直接拨号上网,但是上网的时候偶尔掉线,一共重新拔号了3次(相近时间重新拨号IP相同的几率是极少的),然后每次都继续打开老男孩的博客地址,请问此时,网站独立IP数是多少? 答案:3 blog.lewen.com 并发:在同一段时间内容同时最多能承受多少用户 通过 http://alexa.chinaz.com/ 找 2.5 如何统计PV UV IP 通过日志分析 使用第三方的分析工具。 https://w3techs.com/technologies/overview/web_server/all lnmp === lamp linux nginx mysql php
二、nginx特点及安装部署
Nginx HTTP服务器的特色及优点:
支持高并发:能支持几万并发连接(特别是静态小文件业务环境)。
资源消耗少:在3万并发连接下,开启10个Nginx线程消耗不到200MB内存。
可以做HTTP反向代理及加速缓存,即负载均衡功能,内置对RS节点服务器健康检查功能,这相当于专业的haproxy软件或lvs的功能。
具备squid等专业缓存软件等的缓存功能。
支持异步网络IO事件模型epoll (Linux 2.6+)。
开始安装Nginx
http://nginx.org/ http://nginx.org/download/nginx-1.12.2.tar.gz
mkdir -p /home/lewen/tools cd /home/lewen/tools wget http://nginx.org/download/nginx-1.12.2.tar.gz
1 编译安装Nginx pcre perl语言兼容的正则表达式库 yum install pcre pcre-devel openssl-devel -y ./configure make make install tar xf nginx-1.12.2.tar.gz cd nginx-1.12.2 1.添加nginx 虚拟用户 useradd -s /sbin/nologin -M www 2.配置 ./configure --user=www --group=www --prefix=/application/nginx-1.12.2 --with-http_stub_status_module --with-http_ssl_module [root@web01 nginx-1.12.2]# echo $? 0 3.编译 make [root@web01 nginx-1.12.2]# echo $? 0 4.编译安装 make install [root@web01 nginx-1.12.2]# echo $? 0 5.创建链接 ln -s /application/nginx-1.12.2 /application/nginx ./configure --help --prefix=PATH # 设置安装路径。 --user=USER # 进程用户权限。 --group=GROUP # 进程用户组权限。 --with-http_stub_status_module # 激活状态信息。 --with-http_ssl_module # ssl模块
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
安装错误: 在安装环节中,如果遇到如下错误: ------------------------------------------------------------------- ./configure: error: SSL modules require the OpenSSL library. You can either do not enable the modules, or install the OpenSSL library into the system, or build the OpenSSL library statically from the source with nginx by using --with-openssl=<path> option. ------------------------------------------------------------------- error: SSL modules require the OpenSSL library. yum install openssl-devel -y ###nginx: [emerg] getpwnam("www") failed [root@web01 nginx-1.12.2]# /application/nginx/sbin/nginx -t nginx: the configuration file /application/nginx-1.12.2/conf/nginx.conf syntax is ok nginx: [emerg] getpwnam("www") failed nginx: configuration file /application/nginx-1.12.2/conf/nginx.conf test failed
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
1)启动并检查安装结果 [root@web01 nginx-1.12.2]# /application/nginx/sbin/nginx -t nginx: the configuration file /application/nginx-1.12.2/conf/nginx.conf syntax is ok nginx: configuration file /application/nginx-1.12.2/conf/nginx.conf test is successful 2)启动Nginx服务。 [root@web01 nginx-1.12.2]# /application/nginx/sbin/nginx 3)查看Nginx服务对应的端口是否成功启动。 [root@web01 nginx-1.12.2]# ps -ef |grep nginx root 10117 1 0 01:16 ? 00:00:00 nginx: master process /application/nginx/sbin/nginx www 10118 10117 0 01:16 ? 00:00:00 nginx: worker process root 10120 4801 0 01:16 pts/0 00:00:00 grep nginx [root@web01 nginx-1.12.2]# ss -lntup |grep nginx tcp LISTEN 0 511 *:80 *:* users:(("nginx",10117,6),("nginx",10118,6))
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
4)检查Nginx启动的实际效果。 web排错三部曲下面介绍客户端排查的思路。 第一步,在客户端上ping 服务器端IP,命令如下。 ping 10.0.0.8 排除物理线路问题影响 第二步,在客户端上telnet服务器端IP、端口,命令如下: telnet 10.0.0.8 80 排除防火墙等得影响 nmap -p80 10.0.0.8 第三步,在客户端使用wget命令检测,如下: curl -I 10.0.0.8 模拟用户访问,排除http服务自身问题,根据输出在排错 提示:以上三步是客户端访问网站异常排查的重要三部曲。 [root@web01 nginx-1.12.2]# curl -I 10.0.0.8 HTTP/1.1 200 OK Server: nginx/1.12.2 Date: Mon, 12 Feb 2018 17:28:01 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Mon, 12 Feb 2018 17:09:33 GMT Connection: keep-alive ETag: "5a81ca4d-264" Accept-Ranges: bytes
[root@www ~]# tree /application/nginx/ /application/nginx/ |-- client_body_temp |-- conf #这是nginx的所有的配置文件的目录,极其重要。 | |-- nginx.conf #这是nginx默认的主配置文件 | |-- nginx.conf.default |-- fastcgi_temp #fastcgi临时数据目录 |-- html #这是编译安装时nginx的默认站点目录,相当于网站的根目录,前面已说明,类似apache的默认站点htdocs目录。 | |-- 50x.html # 错误页面优雅替代显示文件,例如:出现502错误时会调用此页面。# error_page 500 502 503 504 /50x.html; | `-- index.html #默认的首页文件 首页文件名字是在nginx.conf中事先定义好的 |—logs #这是nginx默认的日志路径,包括错误日志及访问日志。 | |-- access.log #这是nginx的默认访问日志文件,使用tail -f access.log可以时时观看网站用户访问情况信息。 | |-- error.log #这是nginx的错误日志文件,如果nginx出现启动故障等问题,一定要看看这个错误日志。 | `-- nginx.pid #nginx的pid文件, nginx进程启动后,会把所有进程的ID号写到此文件。 |—sbin #这是nginx命令的目录,如nginx的启动命令nginx。 | `—nginx #nginx的启动命令nginx
去出注释和空行 egrep -v '#|^$' conf/nginx.conf.default >conf/nginx.conf
配置文件详细
Nginx虚拟主机配置实战
1 虚拟主机类型
基于域名的虚拟主机
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@web01 nginx]# cat conf/nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; server { listen 80; server_name www.lewen.org; location / { root html/www; index index.html index.htm; } } }
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
mkdir -p /application/nginx/html/www [root@web01 nginx]# echo 'web01 www.lewen.org'>/application/nginx/html/www/index.html [root@web01 nginx]# cat /application/nginx/html/www/index.html web01 www.lewen.org /application/nginx/sbin/nginx -s reload [root@web01 nginx]# echo '10.0.0.8 www.lewen.org' >>/etc/hosts [root@web01 nginx]# ping www.lewen.org PING www.lewen.org (10.0.0.8) 56(84) bytes of data. 64 bytes from www.lewen.org (10.0.0.8): icmp_seq=1 ttl=64 time=0.019 ms --- www.lewen.org ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 892ms rtt min/avg/max/mdev = 0.019/0.019/0.019/0.000 ms
多个虚拟主机配置
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@web01 nginx]# cat conf/nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; server { listen 80; server_name www.lewen.org; location / { root html/www; index index.html index.htm; } } server { listen 80; server_name bbs.lewen.org; location / { root html/bbs; index index.html index.htm; } } server { listen 80; server_name blog.lewen.org; location / { root html/blog; index index.html index.htm; } } }
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@web01 nginx]# mkdir -p /application/nginx/html/{www,bbs,blog} [root@web01 nginx]# for name in www bbs blog ;do echo "web01 $name.lewen.org">/application/nginx/html/$name/index.html ;done [root@web01 nginx]# for name in www bbs blog ;do cat /application/nginx/html/$name/index.html ;done web01 www.lewen.org web01 bbs.lewen.org web01 blog.lewen.org [root@web01 nginx]# tail -1 /etc/hosts 10.0.0.8 www.lewen.org bbs.lewen.org blog.lewen.org
2 基于端口的虚拟主机
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@web01 conf]# cat nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; server { listen 80; server_name www.lewen.org; location / { root html/www; index index.html index.htm; } } server { listen 81; server_name bbs.lewen.org; location / { root html/bbs; index index.html index.htm; } } server { listen 82; server_name blog.lewen.org; location / { root html/blog; index index.html index.htm; } } } [root@web01 conf]# /application/nginx/sbin/nginx -s reload [root@web01 conf]# ss -lntup |grep nginx tcp LISTEN 0 511 *:80 *:* tcp LISTEN 0 511 *:81 *:* tcp LISTEN 0 511 *:82 *:*
3 规范优化Nginx配置文件
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
[root@web01 conf]# pwd [root@web01 conf]# tree extra/ ├ bbs.conf ├ blog.conf └ www.conf 0 directories, 3 files [root@web01 conf]# cat extra/www.conf server { listen 80; server_name www.lewen.org; location / { root html/www; index index.html index.htm; } } [root@web01 conf]# cat extra/bbs.conf server { listen 80; server_name bbs.lewen.org; location / { root html/bbs; index index.html index.htm; } } [root@web01 conf]# cat extra/blog.conf server { listen 80; server_name blog.lewen.org; location / { root html/blog; index index.html index.htm; } } [root@web01 conf]# cat nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; include extra/*; } [root@web01 conf]# /application/nginx/sbin/nginx -t nginx: the configuration file /application/nginx-1.12.2/conf/nginx.conf syntax is ok nginx: configuration file /application/nginx-1.12.2/conf/nginx.conf test is successful [root@web01 conf]# /application/nginx/sbin/nginx -s reload
4 虚拟主机别名配置
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
jd.com www.jd.com lewenedu.com www.lewenedu.com lewen.org ==== www.lewen.org server { listen 80; server_name www.lewen.org lewen.org; location / { root html/www; index index.html index.htm; } } [root@web01 conf]# /application/nginx/sbin/nginx -t nginx: the configuration file /application/nginx-1.12.2/conf/nginx.conf syntax is ok nginx: configuration file /application/nginx-1.12.2/conf/nginx.conf test is successful [root@web01 conf]# /application/nginx/sbin/nginx -s reload
5 Nginx状态信息功能
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
cat >>/application/nginx/conf/extra/status.conf<<EOF ##status.conf server{ listen 80; server_name status.etiantian.org; location / { stub_status on; access_log off; } } EOF
添加hosts解析
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
Active connections: 6 server accepts handled requests 48 48 87 Reading: 0 Writing: 1 Waiting: 5 其中,第一个server表示Nginx启动到现在共处理了48个连接; 第二个accepts表示Nginx启动到现在共成功创建48次握手; 请求丢失数=(握手数-连接数),可以看出,本次状态显示没有丢失请求。 第三个handled requests,表示总共处理了87次请求; Reading为Nginx 读取到客户端的 Header (请求头)信息数。 Writing为Nginx 返回给客户端的 Header (响应头)信息数。 Waiting为Nginx 已经处理完正在等候下一次请求指令的驻留连接。
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
location / { stub_status on; access_log off; allow 10.0.0.0/24; #<==设置允许和禁止的IP段访问 deny all; #<==设置允许和禁止的IP段访问 }
6 nginx 日志配置
http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
Nginx日志格式中默认的参数配置如下: log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; Nginx记录日志的默认参数配置如下: access_log logs/access.log main; access_log logs/access_www.log main; access_log logs/access_bbs.log main; access_log logs/access_blog.log main; for name in www bbs blog ; do sed "/server_name/a access_log logs/access_$name.log main;" extra/$name.conf ; done
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
报错 [root@web01 conf]# nginx -t nginx: [emerg] unknown log format "basic" in /application/nginx-1.14.0//conf/extra/bbs.conf:4 nginx: configuration file /application/nginx-1.14.0//conf/nginx.conf test failed 错误配置 worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; include extra/*; log_format basic '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; }
注意加载顺序
正确配置 worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; log_format basic '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; include extra/*; # 分发配置文件到extra目录 }
nginx官网
http://nginx.org/en/docs/http/request_processing.html
http://nginx.org/en/docs/control.html
Nginx访问日志轮询切割
cd /application/nginx/logs/ for name in www bbs blog ;do mv access_$name.log access_$name-$(date +%F).log ;done /application/nginx/sbin/nginx -s reload
后续:
使用logrotate+定时任务对nginx的访问日志进行轮询切割。