nginx负载均衡

nginx负载均衡

目录

nginx负载均衡介绍

nginx应用场景之一就是负载均衡。在访问量较多的时候，可以通过负载均衡，将多个请求分摊到多台服务器上，相当于把一台服务器需要承担的负载量交给多台服务器处理，进而提高系统的吞吐率；另外如果其中某一台服务器挂掉，其他服务器还可以正常提供服务，以此来提高系统的可伸缩性与可靠性。

下图为负载均衡示例图，当用户请求发送后，首先发送到负载均衡服务器，而后由负载均衡服务器根据配置规则将请求转发到不同的web服务器上。

反向代理与负载均衡

nginx通常被用作后端服务器的反向代理，这样就可以很方便的实现动静分离以及负载均衡，从而大大提高服务器的处理能力。

nginx实现动静分离，其实就是在反向代理的时候，如果是静态资源，就直接从nginx发布的路径去读取，而不需要从后台服务器获取了。

但是要注意，这种情况下需要保证后端跟前端的程序保持一致，可以使用Rsync做服务端自动同步或者使用NFS、MFS分布式共享存储。

Http Proxy模块，功能很多，最常用的是proxy_pass和proxy_cache

如果要使用proxy_cache，需要集成第三方的ngx_cache_purge模块，用来清除指定的URL缓存。这个集成需要在安装nginx的时候去做，如：

./configure --add-module=../ngx_cache_purge-1.0 ......

nginx通过upstream模块来实现简单的负载均衡，upstream需要定义在http段内

在upstream段内，定义一个服务器列表，默认的方式是轮询，如果要确定同一个访问者发出的请求总是由同一个后端服务器来处理，可以设置ip_hash，如：

upstream idfsoft.com {
  ip_hash;
  server 127.0.0.1:9080 weight=5;
  server 127.0.0.1:8080 weight=5;
  server 127.0.0.1:1111;
}

注意：这个方法本质还是轮询，而且由于客户端的ip可能是不断变化的，比如动态ip，代理，FQ等，因此ip_hash并不能完全保证同一个客户端总是由同一个服务器来处理。

定义好upstream后，需要在server段内添加如下内容：

server {
  location / {
    proxy_pass http://idfsoft.com;
  }
}

nginx负载均衡配置

环境说明

系统 IP 角色 服务
centos8 192.168.222.250 Nginx负载均衡器 nginx
centos8 192.168.222.137 Web1服务器 apache
centos8 192.168.222.138 Web2服务器 nginx

nginx负载均衡器使用源码的方式安装nginx，另外两台Web服务器使用yum的方式分别安装nginx与apache服务

nginx源码安装可以看我的博客nginx，里面有nginx详细的源码安装

修改Web服务器的默认主页
Web1：

[root@Web1 ~]# yum -y install httpd   //下载服务
[root@Web1 ~]# systemctl stop firewalld.service  //关闭防火墙
[root@Web1 ~]# vim /etc/selinux/config 
SELINUX=disabled
[root@Web1 ~]# setenforce 0
[root@Web1 ~]# systemctl disable --now firewalld.service 
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@Web1 ~]# cd /var/www/html/
[root@Web1 html]# ls
[root@Web1 html]# echo "apache" > index.html  //编辑内容到网站里面
[root@Web1 html]# cat index.html 
apache
[root@Web1 html]# systemctl enable --now httpd
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service.
[root@Web1 html]# ss -antl
State      Recv-Q     Send-Q          Local Address:Port           Peer Address:Port     Process     
LISTEN     0          128                   0.0.0.0:22                  0.0.0.0:*                    
LISTEN     0          128                      [::]:22                     [::]:*                    
LISTEN     0          128                         *:80                        *:*

访问:

Web2:

[root@Web2 ~]# yum -y install nginx  //下载服务
[root@Web2 ~]# systemctl stop firewalld.service //关闭防火墙 
[root@Web2 ~]# vim /etc/selinux/config 
SELINUX=disabled
[root@Web2 ~]# setenforce 0
[root@Web2 ~]# systemctl disable --now firewalld.service 
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@Web2 ~]# cd /usr/share/nginx/html/
[root@Web2 html]# ls
404.html  50x.html  index.html  nginx-logo.png  poweredby.png
[root@Web2 html]# echo "nginx" > index.html  //编辑内容到网站里面
[root@Web2 html]# cat index.html 
nginx
[root@Web2 html]# systemctl enable --now nginx.service 
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
[root@Web2 html]# ss -antl
State      Recv-Q     Send-Q          Local Address:Port           Peer Address:Port     Process     
LISTEN     0          128                   0.0.0.0:111                 0.0.0.0:*                    
LISTEN     0          128                   0.0.0.0:80                  0.0.0.0:*                    
LISTEN     0          32              192.168.122.1:53                  0.0.0.0:*                    
LISTEN     0          128                   0.0.0.0:22                  0.0.0.0:*                    
LISTEN     0          128                      [::]:111                    [::]:*                    
LISTEN     0          128                      [::]:80                     [::]:*                    
LISTEN     0          128                      [::]:22                     [::]:*

访问:

开启nginx负载均衡和反向代理

[root@nginx ~]# vim /usr/local/nginx/conf/nginx.conf
...

upstream webserver {              //http字段内添加
    server 192.168.222.137;
    server 192.168.222.138;
}
...

 location / {               //server字段里面修改
            root   html;
             proxy_pass http://webserver;
        }

[root@nginx ~]# systemctl reload nginx.service 
//重新加载配置

测试:
在浏览器输入nginx负载均衡器的IP地址


编辑nginx负载均衡器的nginx配置文件

[root@nginx ~]# vim /usr/local/nginx/conf/nginx.conf
 upstream webserver {      //在http字段内修改
    server 192.168.222.137 weight=3;
    server 192.168.222.138;
}
[root@nginx ~]# systemctl reload nginx.service 
//重新加载配置
[root@nginx ~]# curl 192.168.222.250
apache
[root@nginx ~]# curl 192.168.222.250
apache
[root@nginx ~]# curl 192.168.222.250
apache
[root@nginx ~]# curl 192.168.222.250
nginx
[root@nginx ~]# curl 192.168.222.250
apache
[root@nginx ~]# curl 192.168.222.250
apache
[root@nginx ~]# curl 192.168.222.250
apache
[root@nginx ~]# curl 192.168.222.250
nginx
//可以观察到每访问三次apache就会访问一次nginx，意思就是配置要连续访问3次，才会进行下一次轮查询，当集群中有配置较低，较老的服务器可以进行使用，来减轻这些服务器的压力。
[root@nginx ~]# vim /usr/local/nginx/conf/nginx.conf
 upstream webserver {    //http字段里面进行修改
     ip_hash; 
    server 192.168.222.137 weight=3;
    server 192.168.222.138;
}
[root@nginx ~]# systemctl reload nginx.service 
//重新加载配置
[root@nginx ~]# curl 192.168.222.250
nginx
[root@nginx ~]# curl 192.168.222.250
nginx
[root@nginx ~]# curl 192.168.222.250
nginx
[root@nginx ~]# curl 192.168.222.250
nginx
[root@nginx ~]# curl 192.168.222.250
nginx
[root@nginx ~]# curl 192.168.222.250
nginx
[root@nginx ~]# curl 192.168.222.250
nginx
//可以看见访问到的全部是nginx,因为ip_hash配置，这条配置可以让客户端访问到服务器端，以后就一直是此服务器来进行响应客户端，所以才会一直访问到nginx，当然前面已经说过，这个方式的本质还是轮询，并不能保证一个客户端总是由同一个服务器来进行响应

Keepalived高可用nginx负载均衡器

实验环境

系统 角色 服务 IP
centos8 nginx负载均衡器,master nginx,keepalived 192.168.222.250
centos8 nginx负载均衡器,backup nginx,keepalived 192.168.222.139
centos8 Web1服务器 apache 192.168.222.137
centos8 Web2服务器 nginx 192.168.222.138

nginx源码安装可以看我的博客nginx，里面有nginx详细的源码安装
VIP为：192.168.222.133

修改Web服务器的默认主页

Web1：

[root@Web1 ~]# yum -y install httpd   //下载服务
[root@Web1 ~]# systemctl stop firewalld.service  //关闭防火墙
[root@Web1 ~]# vim /etc/selinux/config 
SELINUX=disabled
[root@Web1 ~]# setenforce 0
[root@Web1 ~]# systemctl disable --now firewalld.service 
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@Web1 ~]# cd /var/www/html/
[root@Web1 html]# ls
[root@Web1 html]# echo "apache" > index.html  //编辑内容到网站里面
[root@Web1 html]# cat index.html 
apache
[root@Web1 html]# systemctl enable --now httpd
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service.
[root@Web1 html]# ss -antl
State      Recv-Q     Send-Q          Local Address:Port           Peer Address:Port     Process     
LISTEN     0          128                   0.0.0.0:22                  0.0.0.0:*                    
LISTEN     0          128                      [::]:22                     [::]:*                    
LISTEN     0          128                         *:80                        *:*

访问:

Web2:

[root@Web2 ~]# yum -y install nginx  //下载服务
[root@Web2 ~]# systemctl stop firewalld.service //关闭防火墙 
[root@Web2 ~]# vim /etc/selinux/config 
SELINUX=disabled
[root@Web2 ~]# setenforce 0
[root@Web2 ~]# systemctl disable --now firewalld.service 
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@Web2 ~]# cd /usr/share/nginx/html/
[root@Web2 html]# ls
404.html  50x.html  index.html  nginx-logo.png  poweredby.png
[root@Web2 html]# echo "nginx" > index.html  //编辑内容到网站里面
[root@Web2 html]# cat index.html 
nginx
[root@Web2 html]# systemctl enable --now nginx.service 
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
[root@Web2 html]# ss -antl
State      Recv-Q     Send-Q          Local Address:Port           Peer Address:Port     Process     
LISTEN     0          128                   0.0.0.0:111                 0.0.0.0:*                    
LISTEN     0          128                   0.0.0.0:80                  0.0.0.0:*                    
LISTEN     0          32              192.168.122.1:53                  0.0.0.0:*                    
LISTEN     0          128                   0.0.0.0:22                  0.0.0.0:*                    
LISTEN     0          128                      [::]:111                    [::]:*                    
LISTEN     0          128                      [::]:80                     [::]:*                    
LISTEN     0          128                      [::]:22                     [::]:*

访问:

开启nginx负载均衡和反向代理

Keepalived高可用的主节点的nginx是需要设置开机自启的
master:

[root@master ~]# systemctl status nginx.service 
● nginx.service - nginx server daemon
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2022-10-18 21:27:54 CST; 1h 1min ago
  Process: 46768 ExecStart=/usr/local/nginx/sbin/nginx (code=exited, status=0/SUCCESS)
 Main PID: 46769 (nginx)
    Tasks: 2 (limit: 12221)
   Memory: 2.6M
   CGroup: /system.slice/nginx.service
           ├─46769 nginx: master process /usr/local/nginx/sbin/nginx
           └─46770 nginx: worker process

Oct 18 21:27:54 nginx systemd[1]: Starting nginx server daemon...
Oct 18 21:27:54 nginx systemd[1]: Started nginx server daemon.
[root@master ~]# vim /usr/local/nginx/conf/nginx.conf
...

upstream webserver {              //http字段内添加
    server 192.168.222.137;
    server 192.168.222.138;
}
...

 location / {               //server字段里面修改
            root   html;
            proxy_pass http://webserver;
        }

[root@master ~]# systemctl reload nginx.service 
//重新加载配置

测试:
在浏览器输入nginx负载均衡器的IP地址

backup：
Keepalived高可用的备用节点的nginx是不设置开机自启的，如果开启的话，后面访问VIP的时候可能会访问不到，可以在需要测试的时候进行开启

[root@backup ~]# systemctl status nginx.service 
● nginx.service - nginx server daemon
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2022-10-18 22:25:31 CST; 1s ago
  Process: 73641 ExecStart=/usr/local/nginx/sbin/nginx (code=exited, status=0/SUCCESS)
 Main PID: 73642 (nginx)
    Tasks: 2 (limit: 12221)
   Memory: 2.7M
   CGroup: /system.slice/nginx.service
           ├─73642 nginx: master process /usr/local/nginx/sbin/nginx
           └─73643 nginx: worker process

Oct 18 22:25:31 backup systemd[1]: Starting nginx server daemon...
Oct 18 22:25:31 backup systemd[1]: Started nginx server daemon.
[root@backup ~]# vim /usr/local/nginx/conf/nginx.conf
...

upstream webserver {              //http字段内添加
    server 192.168.222.137;
    server 192.168.222.138;
}
...

 location / {               //server字段里面修改
            root   html;
            proxy_pass http://webserver;
        }
[root@backup ~]# systemctl reload nginx.service 
//重新加载一下配置

访问:
在浏览器输入nginx负载均衡器的IP地址

安装Keepalived

master:

[root@master ~]# dnf list all |grep keepalived  //查找系统中是否存在其安装包
Failed to set locale, defaulting to C.UTF-8
keepalived.x86_64                                      2.1.5-6.el8                                            AppStream 
[root@master ~]# dnf -y install keepalived

backup：

[root@backup ~]# dnf list all |grep keepalived //查找系统中是否存在其安装包
Failed to set locale, defaulting to C.UTF-8
Module yaml error: Unexpected key in data: static_context [line 9 col 3]
Module yaml error: Unexpected key in data: static_context [line 9 col 3]
Module yaml error: Unexpected key in data: static_context [line 9 col 3]
Module yaml error: Unexpected key in data: static_context [line 9 col 3]
Module yaml error: Unexpected key in data: static_context [line 9 col 3]
Module yaml error: Unexpected key in data: static_context [line 9 col 3]
Module yaml error: Unexpected key in data: static_context [line 9 col 3]
Module yaml error: Unexpected key in data: static_context [line 9 col 3]
keepalived.x86_64                                                 2.1.5-6.el8                                            AppStream   
[root@backup ~]# dnf -y install keepalived

配置Keepalived

master

[root@master ~]# cd /etc/keepalived/
[root@master keepalived]# ls
keepalived.conf
[root@master keepalived]# mv keepalived.conf{,-bak}  //备份一下配置文件
[root@master keepalived]# ls
keepalived.conf-bak
[root@master keepalived]# vim keepalived.conf  //编辑一个新配置文件
[root@master keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01
}

vrrp_instance VI_1 {        //这里主备节点需要一致
    state BACKUP
    interface ens33      //网卡
    virtual_router_id 51
    priority 100     //这里比备节点的高
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass tushanbu   //密码(可以随机生成)
    }
    virtual_ipaddress {
        192.168.222.133    //高可用虚拟IP（VIP）地址
    }
}

virtual_server 192.168.222.133 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.222.250 80 {  //主节点ip
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.222.139 80 {   //备节点ip
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@master keepalived]# systemctl enable --now keepalived.service 
Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /usr/lib/systemd/system/keepalived.service.

backup:

[root@backup ~]# cd /etc/keepalived/
[root@backup keepalived]# ls
keepalived.conf
[root@backup keepalived]# mv keepalived.conf{,-bak} //备份一下配置文件
[root@backup keepalived]# ls
keepalived.conf-bak
[root@backup keepalived]# vim keepalived.conf //编辑新的配置文件
[root@backup keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb02    
}

vrrp_instance VI_1 {       //这里主备节点需要一致
    state BACKUP
    interface ens33      //网卡
    virtual_router_id 51
    priority 90     //这里比主节点的小
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass tushanbu   //密码(可以随机生成)
    }
    virtual_ipaddress {
        192.168.222.133    //高可用虚拟IP（VIP）地址
    }
}

virtual_server 192.168.222.133 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.222.250 80 {   //主节点ip
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.222.137 80 {   //备节点ip
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@backup keepalived]# systemctl enable --now keepalived.service 
Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /usr/lib/systemd/system/keepalived.service.
[root@backup keepalived]# systemctl start nginx
//此时测试的时候可以开启nginx

查看VIP
master:

[root@master keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:05:f4:28 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.250/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.222.133/32 scope global ens33
       valid_lft forever preferred_lft forever

backup:

[root@backup keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:31:af:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.139/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe31:aff9/64 scope link 
       valid_lft forever preferred_lft forever

//VIP在master主机上面因为在Keepalived配置文件里我们设置master的优先级要比backup高一些，所以VIP在这里很正常

访问:

master:

[root@master keepalived]# curl 192.168.222.133
apache
[root@master keepalived]# curl 192.168.222.133
nginx

此是关闭master上面的nginx和keepalived的

[root@master keepalived]# systemctl stop nginx.service 
[root@master keepalived]# systemctl stop keepalived.service 
[root@master keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:05:f4:28 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.250/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
//此时master上面没有VIP

backup:

[root@backup keepalived]# systemctl enable --now keepalived
[root@backup keepalived]# systemctl start nginx.service 
[root@backup keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:31:af:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.139/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.222.133/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe31:aff9/64 scope link 
       valid_lft forever preferred_lft forever
//此时backup上面出现VIP，备节点变成了主节点

[root@backup keepalived]# curl 192.168.222.133
apache
[root@backup keepalived]# curl 192.168.222.133
nginx

访问:

可以看到，其中一个nginx负载均衡器挂掉了，也不会影响正常访问，这就是nginx负载均衡的高可用的配置

重启master上面的nginx和keepalived

[root@master keepalived]# systemctl enable --now keepalived
[root@master keepalived]# systemctl enable --now nginx
[root@master keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:05:f4:28 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.250/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.222.133/32 scope global ens33
       valid_lft forever preferred_lft forever
//可以发现VIP出现在master节点上面

编写脚本监控Keepalived和nginx的状态

master:

[root@master keepalived]# cd
[root@master ~]# mkdir /scripts
[root@master ~]# cd /scripts/
[root@master scripts]# vim check_nginx.sh
[root@master scripts]# cat check_nginx.sh
#!/bin/bash
nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
    if [ $nginx_status -lt 1 ];then
            systemctl stop keepalived
    fi
[root@master scripts]# chmod +x check_nginx.sh 
[root@master scripts]# ll
total 4
-rwxr-xr-x. 1 root root 151 Oct 19 00:32 check_nginx.sh
[root@master scripts]# vim notify.sh
[root@master scripts]# cat notify.sh 
#!/bin/bash
case "$1" in
    master)
        nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
        if [ $nginx_status -lt 1 ];then
            systemctl start nginx
        fi
    ;;
    backup)
        nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
        if [ $nginx_status -gt 0 ];then
            systemctl stop nginx
        fi
    ;;
    *)
         echo "Usage:$0 master|backup VIP"
    ;;
esac

[root@master scripts]# chmod +x notify.sh 
[root@master scripts]# ll
total 8
-rwxr-xr-x. 1 root root 151 Oct 19 00:32 check_nginx.sh
-rwxr-xr-x. 1 root root 399 Oct 19 00:35 notify.sh

backup:
可以先提前创建好存放脚本的目录

[root@backup keepalived]# cd
[root@backup ~]# mkdir  /scripts
[root@backup ~]# cd /scripts/

从主节点上面将脚本到备节点提前创建好的存放目录里面

[root@master scripts]# scp notify.sh 192.168.222.139:/scripts/
root@192.168.222.139's password: 
notify.sh                                                          100%  399   216.0KB/s   00:00    

[root@backup scripts]# ls
notify.sh
[root@backup scripts]# cat notify.sh 
#!/bin/bash
case "$1" in
    master)
        nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
        if [ $nginx_status -lt 1 ];then
            systemctl start nginx
        fi
    ;;
    backup)
        nginx_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bnginx\b'|wc -l)
        if [ $nginx_status -gt 0 ];then
            systemctl stop nginx
        fi
    ;;
    *)
         echo "Usage:$0 master|backup VIP"
    ;;
esac

配置keepalived加入监控脚本的配置

master:

[root@master scripts]# cd
[root@master ~]# vim /etc/keepalived/keepalived.conf
[root@master ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
 
global_defs {
   router_id lb01
}
 
vrrp_script nginx_check{
    script "/scripts/check_nginx.sh"
    interval 5
    weight -20
}
  
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
! Configuration File for keepalived
 
global_defs {
   router_id lb01
}
 
vrrp_script nginx_check{                                //添加
    script "/scripts/check_nginx.sh"                    //添加
    interval 1                                          //添加
    weight -20                                          //添加
}                                                       //添加
 
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass tushanbu
    }
    virtual_ipaddress {
        192.168.222.133
    }
     track_script {                     //添加
        nginx_check                     //添加
    }                                   //添加
    notify_master "/scripts/notify.sh master"  //添加
}
virtual_server 192.168.222.133 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP
 
    real_server 192.168.222.250 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
 
    real_server 192.168.222.139 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@master ~]# systemctl restart keepalived.service

backup:
backup无需检测nginx是否正常，当升级为MASTER时启动nginx，当降级为BACKUP时关闭

[root@backup scripts]# cd
[root@backup ~]# vim /etc/keepalived/keepalived.conf
[root@backup ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
 
global_defs {
   router_id lb02
}
 
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass tushanbu
    }
    virtual_ipaddress {
        192.168.222.133
    }
    notify_master "/scripts/notify.sh master"           //添加
    notify_backup "/scripts/notify.sh backup"           //添加
}
virtual_server 192.168.222.133 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP
 
    real_server 192.168.222.250 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
 
    real_server 192.168.222.139 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@backup ~]# systemctl restart keepalived.service

测试
正常状态运行查看状态

[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:f6:83:57 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.250/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.222.133/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fef6:8357/64 scope link 
       valid_lft forever preferred_lft forever
[root@master]# curl 192.168.222.133
apache
[root@master]# curl 192.168.222.133
nginx
//此时VIP在主节点上面

关闭master的nginx

[root@master ~]# systemctl stop nginx.service 
[root@master ~]# ss -antl
State      Recv-Q     Send-Q          Local Address:Port           Peer Address:Port     Process     
LISTEN     0          128                   0.0.0.0:22                  0.0.0.0:*                    
LISTEN     0          128                      [::]:22                     [::]:*                    
[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:05:f4:28 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.250/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
//没有VIP

backup:

[root@backup ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:31:af:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.139/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.222.133/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe31:aff9/64 scope link 
       valid_lft forever preferred_lft forever
[root@backup ~]# curl 192.168.222.133
apache
[root@backup ~]# curl 192.168.222.133
nginx
//备节点变成主机节点

重新开启master的nginx

[root@master ~]# systemctl restart keepalived.service 
[root@master ~]# systemctl restart nginx.service 
[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:f6:83:57 brd ff:ff:ff:ff:ff:ff
    inet 192.168.222.250/24 brd 192.168.222.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.222.133/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fef6:8357/64 scope link 
       valid_lft forever preferred_lft forever
[root@master]# curl 192.168.222.133
apache
[root@master]# curl 192.168.222.133
nginx
//此时VIP重新回到master上面
posted @ 2022-10-19 01:27  涂山小布  阅读(174)  评论(0)  编辑  收藏  举报