nginx+keepalived高可用

1.环境两台Centos服务器

IP1:192.168.1.78

IP2:192.168.1.79

VIP：192.168.1.88

2.安装keeplived软件和nginx

每台服务器上都要安装keeplived和nginx

• nginx安装省略
• keepalived安装

tar -xvf keepalived-1.3.5.tar.gz

cd keepalived-1.3.5

./configure

make && make install

cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d

cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig

mkdir /etc/keepalived

cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

cp /usr/local/sbin/keepalived /usr/sbin

• 配置keepalived

主节点，修改keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     #acassen@firewall.loc
     #failover@firewall.loc
     #sysadmin@firewall.loc
   }
   #notification_email_from Alexandre.Cassen@firewall.loc
   #smtp_server 192.168.200.1
   #smtp_connect_timeout 30
   router_id master-node
   #vrrp_skip_check_adv_addr
   #vrrp_strict
   #vrrp_garp_interval 0
   #vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    mcast_src_ip 192.168.1.78
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.88

    }
}

从节点，修改keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     #acassen@firewall.loc
     #failover@firewall.loc
     #sysadmin@firewall.loc
   }
   #notification_email_from Alexandre.Cassen@firewall.loc
   #smtp_server 192.168.200.1
   #smtp_connect_timeout 30
   router_id master-node
   #vrrp_skip_check_adv_addr
   #vrrp_strict
   #vrrp_garp_interval 0
   #vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
     mcast_src_ip 192.168.1.79
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.88

    }
}

2.测试

一、测试vip绑定、漂移

启动两个节点上的keepalived服务，使用ip a命令查看vip在哪个节点上。

Master：

~
[root@hrsw etc]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:6c:b0:3a brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.78/24 brd 192.168.1.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.1.88/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::5b7e:1a3d:618d:1e93/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:6c:b0:44 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.40/24 brd 192.168.1.255 scope global dynamic ens37
       valid_lft 540sec preferred_lft 540sec
    inet6 fe80::9f21:50a8:7aa9:759b/64 scope link 
       valid_lft forever preferred_lft forever

Backup:

[root@hrsw sysconfig]# ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:b0:0f:fd brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.79/24 brd 192.168.1.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::5e88:7cdb:644:41e/64 scope link 
       valid_lft forever preferred_lft forever
    inet6 fe80::5b7e:1a3d:618d:1e93/64 scope link tentative dadfailed 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:b0:0f:07 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.41/24 brd 192.168.1.255 scope global dynamic ens37
       valid_lft 539sec preferred_lft 539sec
    inet6 fe80::2b81:1f13:5102:80ef/64 scope link 
       valid_lft forever preferred_lft forever

可以看出VIP在master节点，现在将Master节点的keepalived服务停掉，或直接把Master节点服务器关机，再次查看

Master:

[root@hrsw etc]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:6c:b0:3a brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.78/24 brd 192.168.1.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.1.88/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::5b7e:1a3d:618d:1e93/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:6c:b0:44 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.40/24 brd 192.168.1.255 scope global dynamic ens37
       valid_lft 366sec preferred_lft 366sec
    inet6 fe80::9f21:50a8:7aa9:759b/64 scope link 
       valid_lft forever preferred_lft forever

Backup:

[root@hrsw sysconfig]# ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:b0:0f:fd brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.79/24 brd 192.168.1.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.1.88/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::5e88:7cdb:644:41e/64 scope link 
       valid_lft forever preferred_lft forever
    inet6 fe80::5b7e:1a3d:618d:1e93/64 scope link tentative dadfailed 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:b0:0f:07 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.41/24 brd 192.168.1.255 scope global dynamic ens37
       valid_lft 380sec preferred_lft 380sec
    inet6 fe80::2b81:1f13:5102:80ef/64 scope link 
       valid_lft forever preferred_lft forever

可以看出VIP已经漂移到Backup节点上。查看Backup日志，显示切换到了MASTER STATE

Jun  8 09:56:24 hrsw Keepalived_vrrp[2427]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun  8 09:56:25 hrsw Keepalived_vrrp[2427]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun  8 09:56:25 hrsw Keepalived_vrrp[2427]: VRRP_Instance(VI_1) setting protocol VIPs.

再次启动Master节点上的keepalived服务，查看Backup日志，显示切换回了BACKUP STATE

Jun  8 09:54:56 hrsw dhclient[680]: bound to 192.168.1.41 -- renewal in 249 seconds.
Jun  8 09:54:56 hrsw dbus[615]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Jun  8 09:54:56 hrsw dbus-daemon: dbus[615]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Jun  8 09:54:56 hrsw systemd: Started Network Manager Script Dispatcher Service.
Jun  8 09:54:56 hrsw nm-dispatcher: req:1 'dhcp4-change' [ens37]: new request (3 scripts)
Jun  8 09:54:56 hrsw nm-dispatcher: req:1 'dhcp4-change' [ens37]: start running ordered scripts...
Jun  8 09:55:44 hrsw Keepalived_vrrp[2427]: VRRP_Instance(VI_1) Received advert with higher priority 100, ours 99
Jun  8 09:55:44 hrsw Keepalived_vrrp[2427]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jun  8 09:55:44 hrsw Keepalived_vrrp[2427]: VRRP_Instance(VI_1) removing protocol VIPs.

二、测试访问

两个节点keepalived和nginx都正常情况下：

关闭Masterkeepalived服务

开启Masterkeepalived服务，关闭nginx服务

发现这个时候就访问不到了，原因是Master keepalived服务是正常的，所以访问1.88的时候还是去访问了Master上的nginx，但是现在nginx已经停止了，所以访问不到了。

我们要达到的效果是nginx停了，VIP也要随之切换。怎么办呢？让keepalived监控nginx的健康状况，如果nginx异常，就停止该keepalived服务。

修改keepalived.conf,增加如下代码

vrrp_script chk_nginx {
    script "/etc/keepalived/check_nginx.sh"
    interval 2
    weight 2
}

track_script {
    chk_nginx
}

#!/bin/bash
counter=`ps -ef|grep nginx|grep -v "grep"|wc -l`
if [ $counter -eq 0 ]; then
    /mnt/Disk/nginx2/sbin/nginx
    sleep 2
    counter=`ps -ef|grep nginx|grep -v "grep"|wc -l`
    if [ $counter -eq 0 ]; then
        pkill -9 keepalived
    fi
fi