29.nginx负载均衡深入透彻

1.负载均衡的应用

　
　　
　　
负载均衡与反向代理：

　　

2.负载均衡环境搭建
　　

HOST　　NAME 　　IP　　说明
lb01 　　 10.0.0.5 　 Nginx   主负载均衡器
lb02 　　 10.0.0.6 　 Nginx   辅负载均衡器
web01 　10.0.0.8      web01服务器
web02     10.0.0.7      web02服务器

06-期中架构-web02-10.0.0.7
D:\vmware_centos\06-期中架构-web02-10.0.0.7
[root@oldboyedu-mu ~]# sed -i 's#210$#7#g' /etc/sysconfig/network-scripts/ifcfg-eth*
[root@oldboyedu-mu ~]# hostname web02
[root@oldboyedu-mu ~]# hostname
web02
[root@oldboyedu-mu ~]# vim /etc/sysconfig/network
[root@oldboyedu-mu ~]# cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=web02
[root@oldboyedu-mu ~]# /etc/init.d/network restart
class01_期中架构-web02-10.0.0.7
[root@web02 ~]# ifconfig |grep "inet addr"
inet addr:10.0.0.7 Bcast:10.0.0.255 Mask:255.255.255.0
inet addr:172.16.1.7 Bcast:172.16.1.255 Mask:255.255.255.0
inet addr:127.0.0.1 Mask:255.0.0.0

07-期中架构-lb01-10.0.0.5
D:\vmware_centos\07-期中架构-lb01-10.0.0.5
暂时修改：
[root@lb01 opt]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
#DNS1=223.5.5.5
DNS1=114.114.114.114
或者：
#DNS1=223.5.5.5
DNS1=114.114.115.115

08-期中架构-lb02-10.0.0.6
D:\vmware_centos\08-期中架构-lb02-10.0.0.6

查看别人编译安装好的编译参数：
[root@web02 nginx-1.12.2]# /application/nginx/sbin/nginx -V
nginx version: nginx/1.12.2
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC)
built with OpenSSL 1.0.1e-fips 11 Feb 2013
TLS SNI support enabled
configure arguments: --user=www --group=www --prefix=/application/nginx-1.12.2 --with-http_stub_status_module --with-http_ssl_module

配置用于测试的web服务：
[root@web01 nginx]# cp conf/nginx.conf{,.bak.before}
[root@web01 nginx]# vim conf/nginx.conf
[root@web02 nginx]# cp conf/nginx.conf{,.bak.before}
[root@web02 nginx]# vim conf/nginx.conf
#web01 web02 nginx.conf 分别部署
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
server {
listen 80;
#注意：该虚拟主机优先被访问！
server_name bbs.etiantian.org;
location / {
root html/bbs;
index index.html index.htm;
}
access_log logs/access_bbs.log main;
}
server {
listen 80;
server_name www.etiantian.org;
location / {
root html/www;
index index.html index.htm;
}
access_log logs/access_www.log main;
}
}

[root@web01 ~]# /application/nginx/sbin/nginx -t
[root@web01 ~]# /application/nginx/sbin/nginx -s reload

[root@web02 ~]# /application/nginx/sbin/nginx -t
[root@web02 ~]# /application/nginx/sbin/nginx -s reload

[root@web01 ~]# tree /application/nginx/html/ -L 1
/application/nginx/html/
├── 50x.html
├── bbs
├── blog
├── index.html
└── www
[root@web01 ~]# tree /application/nginx/html/ -Ld 1
/application/nginx/html/
├── bbs
├── blog
└── www
[root@web01 ~]# for name in www bbs blog;do echo "`hostname`:$name.etiantian.org" >/application/nginx/html/$name/oldboy.html; done
[root@web01 ~]# for name in www bbs blog;do cat /application/nginx/html/$name/oldboy.html; done
web01:www.etiantian.org
web01:bbs.etiantian.org
web01:blog.etiantian.org

[root@web02 ~]# mkdir -p /application/nginx/html/{www,bbs,blog}
[root@web02 ~]# tree /application/nginx/html/ -Ld 1
/application/nginx/html/
├── bbs
├── blog
└── www
[root@web02 ~]# for name in www bbs blog;do echo "`hostname`:$name.etiantian.org" >/application/nginx/html/$name/oldboy.html; done
[root@web02 ~]# for name in www bbs blog;do cat /application/nginx/html/$name/oldboy.html; done
web02:www.etiantian.org
web02:bbs.etiantian.org
web02:blog.etiantian.org

web01 web02环境准备完成
[root@lb01 ~]# curl 10.0.0.8/oldboy.html
web01:bbs.etiantian.org
[root@lb01 ~]# curl 10.0.0.7/oldboy.html
web02:bbs.etiantian.org

[root@lb01 ~]# curl -H Host:www.etiantian.org 10.0.0.8/oldboy.html
web01:www.etiantian.org
[root@lb01 ~]# curl -H Host:www.etiantian.org 10.0.0.7/oldboy.html
web02:www.etiantian.org
有问题！

3.nginx负载均衡搭建
[root@lb01 nginx]# cp conf/nginx.conf{,.bak}
[root@lb01 nginx]# vim conf/nginx.conf
#lb01 部署
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
#定义（创建）web服务器池，包含7/8两个节点；weight表权重。
upstream server_pools {
server 10.0.0.7:80 weight=1;
server 10.0.0.8:80 weight=1;
}
#定义代理的负载均衡域名虚拟主机。
server {
listen 80;
#访问网站bbs.etiantian.org，就将请求通过proxy_pass转给后边web服务器池里的节点。
server_name bbs.etiantian.org;
location / {
proxy_pass http://server_pools;
}
}
[root@lb01 nginx]# /application/nginx/sbin/nginx -t
[root@lb01 nginx]# /application/nginx/sbin/nginx -s reload
[root@lb01 nginx]# ss -lntup|grep 80
tcp LISTEN 0 511 *:80 *:* users:(("nginx",7573,6),("nginx",13282,6))

说明：现在所有的网站（这里是bbs.etiantian.org）解析到了主机为lb01:10.0.0.5的负载均衡上而不是后边的web主机上。

本地\etc\hosts文件解析：
#10.0.0.8 www.etiantian.org blog.etiantian.org bbs.etiantian.org status.etiantian.org
10.0.0.5 www.etiantian.org blog.etiantian.org bbs.etiantian.org status.etiantian.org

测试：
输入：http://bbs.etiantian.org/oldboy.html
显示： web01:bbs.etiantian.org
刷新网页显示：web02:bbs.etiantian.org
两页面内容来回循环！

默认日志：
[root@lb01 nginx]# tail -f ./logs/access.log
10.0.0.253 - - [01/Mar/2018:20:45:34 +0800] "GET /oldboy.html HTTP/1.1" 200 24 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
10.0.0.253 - - [01/Mar/2018:20:45:35 +0800] "GET /oldboy.html HTTP/1.1" 200 24 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
说明：用户IP；访问的URI；大小；

ctrl+F5强制刷新！
至此负载均衡的简单功能完成！

4.nginx负载均衡-配置多个虚拟主机

server_name bbs.etiantian.org;
实际中可能有多个网站、多个虚拟机主机！

[root@lb01 nginx]# vim conf/nginx.conf
[root@lb01 nginx]# cat conf/nginx.conf
#lb01部署
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
#服务器web池
upstream server_pools {
server 10.0.0.7:80 weight=1;
server 10.0.0.8:80 weight=1;
}
server {
listen 80;
server_name bbs.etiantian.org;
location / {
proxy_pass http://server_pools;
}
}
#添加虚拟主机www.etiantian.org
server {
listen 80;
server_name www.etiantian.org;
location / {
proxy_pass http://server_pools;
}
}
}
[root@lb01 nginx]# /application/nginx/sbin/nginx -t
[root@lb01 nginx]# /application/nginx/sbin/nginx -s reload

测试：
输入：http://bbs.etiantian.org/oldboy.html和http://www.etiantian.org/oldboy.html
都显示： web01:bbs.etiantian.org
刷新网页显示：web02:bbs.etiantian.org
两页面内容来回循环，这不符合要求！

验证故障：
wireshark进行抓包观察。

解决办法：
[root@lb01 nginx]# cat conf/nginx.conf
#lb01部署
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
#服务器web池
upstream server_pools {
server 10.0.0.7:80 weight=1;
server 10.0.0.8:80 weight=1;
}
server {
listen 80;
server_name bbs.etiantian.org;
location / {
proxy_pass http://server_pools;
#添加；要求负载均衡机(10.0.0.5)向后web服务器(10.0.0.8)请求时网址要保持不变！
proxy_set_header Host $host;
}
}
server {
listen 80;
server_name www.etiantian.org;
location / {
proxy_pass http://server_pools;
#添加；
proxy_set_header Host $host;
}
}
}
说明：参数proxy_set_header修改反向代理，向后面发出请求的时候的 请求头的信息 以和用户的请求头的一样。
[root@lb01 nginx]# /application/nginx/sbin/nginx -t
[root@lb01 nginx]# /application/nginx/sbin/nginx -s reload

测试：
输入：http://www.etiantian.org/oldboy.html
显示： web01:www.etiantian.org
刷新网页显示：web02:www.etiantian.org
两页面内容来回循环，符合要求！

5.nginx负载均衡-后端节点服务器记录用户真实IP地址
　　
这里采用了nat模式，这里10.0.0.253相当于是客户IP，vmnet8是windows和linux在nat模式下沟通的桥梁！

查看担任nginx负载均衡的lb01(10.0.0.5)日志可以得到用户IP：
[root@lb01 nginx]# tail -f ./logs/access.log
10.0.0.253 - - [01/Mar/2018:20:45:34 +0800] "GET /oldboy.html HTTP/1.1" 200 24 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

查看web01(10.0.0.8)日志可以得到nginx负载均衡的lb01的IP：
[root@web01 ~]# tail -f /application/nginx/logs/access_www.log
10.0.0.5 - - [01/Mar/2018:21:16:32 +0800] "GET /oldboy.html HTTP/1.0" 200 24 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)" "-"

[root@lb01 nginx]# vim conf/nginx.conf
[root@lb01 nginx]# cat conf/nginx.conf
#lb01部署
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
#服务器web池
upstream server_pools {
server 10.0.0.7:80 weight=1;
server 10.0.0.8:80 weight=1;
}
server {
listen 80;
server_name bbs.etiantian.org;
location / {
proxy_pass http://server_pools;
proxy_set_header Host $host;
#添加；将$remote_addr包含的内容放到X-Forwarded-For(专门存放真实的用户IP地址)中。
proxy_set_header X-Forwarded-For $remote_addr;
}
}
server {
listen 80;
server_name www.etiantian.org;
location / {
proxy_pass http://server_pools;
proxy_set_header Host $host;
#添加；
proxy_set_header X-Forwarded-For $remote_addr;
}
}
}
[root@lb01 nginx]# /application/nginx/sbin/nginx -t
[root@lb01 nginx]# /application/nginx/sbin/nginx -s reload

测试：后端节点服务器记录用户真实IP地址（查看web01(10.0.0.8)日志可以得到用户IP）
[root@web01 ~]# tail -f /application/nginx/logs/access_www.log
10.0.0.5 - - [01/Mar/2018:21:48:51 +0800] "GET /oldboy.html HTTP/1.0" 200 24 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)" "10.0.0.253"
10.0.0.5 - - [01/Mar/2018:21:48:52 +0800] "GET /oldboy.html HTTP/1.0" 200 24 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)" "10.0.0.253"