keepalived的简单使用

原理简述

本篇主要学习keepalived配合nginx实现nginx的高可用, 也就是需要keepalived检测到nginx宕机时停用keepalived, 备用keepalived会自动接收过来.

简单的原理(如下图), 主备服务器会配置相同的vip(虚拟ip), 谁的优先级高谁来接收vip的请求, 然后nginx和keepalived部署在同一个服务器上面, keeplived控制机器接收到vip的请求, 交给了nginx来处理请求. nginx的功能主要是负责负载均衡, nginx的安装配置在此不再赘述, 可以参考这个: ngix安装与使用

keepalived功能有很多, 此篇只是最简单的配合ngxin实现高可用的demo.

安装

安装常用的的指令包: yum install -y curl gcc openssl-devel libnl3-devel net-snmp-devel
安装: yum install -y keepalived
启动: systemctl start keepalived
重启: systemctl restart keepalived
关闭: systemctl stop keepalived
开机自启: systemctl enable keepalived

修改配置文件: vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

# 定义虚拟路由, 必须叫VI_1
vrrp_instance VI_1 {
    state MASTER #设置为主服务器, 备份服务器设置为BACKUP
    interface enp0s3 #监控的网络接口(ifconfig或者ip addr指令找出网卡)
    priority 100 #(优先级, 主机大一点, 备份机小一点)
    virtual_router_id 99 #同一个vrrp_instance下routerId必须是一致的

    authentication {
        auth_type PASS #vrrp认证方式主备必须一致
        auth_pass 12345 #密码
    }

    virtual_ipaddress {
        192.168.0.99 #虚拟ip, 主从一致, 可配置多个
    }
}

另外一台机相同方法, 相同配置(state改成BACKUP, priority调整调一下, 此例中是80)

vrrp 的主从并不是通过stat配置的MASTER和BACKUP决定的, 是通过优先级决定的

默认日志位置: /var/log/message位置修改参考: keepalived 配置日志方法

参考1: Linux下Keepalived安装与配置

参考2: Keepalived原理介绍和配置实践

参考3: keepalived介绍、安装及配置详解

参考4: https://codor.lanzoue.com/b012qnsvc 密码:1i77

参考5: keepalive安装部署踩坑参考

检查是否脑裂

以下enp0s3是网卡名称, 可通过ip a或者ifconfig查看

使用tcpdump -i enp0s3 -nn host 224.0.0.18
或者
tcpdump -i enp0s3 | grep VRRP进行查看, 默认的广播通道为224.0.0.18 (我把时间删除了, 内容是我改的)

192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20
192.168.0.117 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 80, authtype simple, intvl 1s, length 20
192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20
192.168.0.117 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 80, authtype simple, intvl 1s, length 20
192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20
192.168.0.117 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 80, authtype simple, intvl 1s, length 20

如果结果如上, 说明出现了脑裂(主备都向外宣誓我是老大),

出现这种情况的原因是防火墙或者iptables拦截了vrrp请求, 进行放行即可.

防火墙(推荐):
```
firewall-cmd --add-rich-rule='rule protocol value="vrrp" accept' --permanent
firewall-cmd --reload
```
iptables:
```
iptables -A INPUT -s 192.168.1.0/24 -d 224.0.0.18 -j ACCEPT
iptables -A INPUT -s 192.168.1.0/24 -p vrrp -j ACCEPT
```
不存在可以进行安装, yum install iptables-services
- 参考1: Keepalived高可用切换过程
- 参考2: centos7 keepalived VRRP协议 firewalld配置
- 参考3: keepalived的脑裂问题与解决
- 参考4: 使用keepalived时iptables需要开放的协议
- 参考5: linux iptables防火墙中的工作常用命令

最后附上正常运行结果, 即只有100优先级(优先级大的那个)的机器来广播自己是老大

09:26:55.782258 IP 192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20
09:26:56.782910 IP 192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20
09:26:57.783787 IP 192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20
09:26:58.784709 IP 192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20
09:26:59.784792 IP 192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20
09:27:00.785171 IP 192.168.0.111 > 224.0.0.18: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20

测试ip漂移

ip漂移: 就是主备切换过程成, vip漂到真实ip上的过程. 也称为主备切换.

测试过程就是停用master机器上面的keepalived或者关机master机器, 查看backup机器是否正常接过来, 一般1s左右可以切换过去. 当出现脑裂情况的时候切换过程也能实现, 只是很慢大约7s左右. 具体原因未深究.

漂移过程可以通过抓包实现, 也可以通过两给ngxin转发到不同tomcat中的项目或网页, 或者修改ngxin的默认网页进行测试都可.

至此位置简单使用就完成了, 下面介绍几个功能配置

VRRP脚本

签到keepalived的配置文件夹: cd /etc/keepalived/

创建一个脚本文件: vim nginx_check.sh

#!/bin/bash
count=`ps -C nginx --no-header |wc -l`
if [ $count -eq 0 ];then
		killall keepalived
fi

赋予执行权限: chmode +x nginx_check.sh
引入脚本: vim keepalived.conf

与vrrp_instance同级, 其中
- chk_nginx: 脚本名称, 自定义的
- script: 脚本位置
- interval: 执行间隔
- weight: 权重, 如果是负数, 当执行失败时候会影响vrrp_instance中的优先级priority, 因为主备切换是通过优先级的高低的进行切换的, 所以也可以通过这个优先级来进行主动控制主备切换. 而脚本中的内容可以很灵活地实现很多功能. 此个demo中只是简单实现检测到ngxin关闭后自动关闭keepalived, 也可以实现检测启动后进行开启, 然后延迟2s后查看是否启动成功, 未成功再进行关闭keepalived或者降低优先级(配合右键通知).
```
vrrp_script chk_nginx {
	script "/etc/keepalived/nginx_check.sh"
	interval 2
	#weight -30
}
```
配置到vrrp_instance中, 与authentication和virtual_ipaddress同级
```
track_script {
	chk_nginx
}
```

修改后的配置文件

! Configuration File for keepalived

vrrp_script chk_nginx {
    script "/etc/keepalived/nginx_check.sh"
    interval 2
    #weight -30
}

vrrp_instance VI_1 {
    state MASTER
    interface enp0s3
    priority 100
    advert_int 1
    virtual_router_id 99
    authentication {
        auth_type PASS
        auth_pass 221531
    }

    track_script {
        chk_nginx
    }

    virtual_ipaddress {
        192.168.0.99
    }
}

测试

正常启动时候, 手动关闭nginx, 查看keepalived的状态.
参考:
- 参考1: keepalived之vrrp_script详解
- 参考2: Keepalived 的主备切换怎么做

邮件配置

邮件功能是linux上面的mail指令.

安装mail: yum -y install mailx

编辑配置文件(设置发送人信息): vim /etc/mail.rc, 在末尾处添加

set from=xxx@163.com
set smtp=smtp.163.com
set smtp-auth-user=xxx@qq.com
set smtp-auth-password=KJFHTOSXZQPNFAIU  #邮箱需要开启POP3/SMTP服务并设置密钥
set smtp-auth=login
set ssl-verify=ignore

测试mail功能: echo test mail | mail -s testa 收件人id@qq.com
- -s 后面是主题的意思
- echo test maill 中的test mail 是邮件正文.
- 最后跟着收件人

配置到keepalived中, 方法1

创建脚本 vim mail_send.sh(记得赋予执行权限)

可以使用./mail_send.sh master进行测试

#!/bin/bash
contact='收件人邮箱@qq.com'
notify() {
  	  mailsubject="$(hostname) to be $1, vip  转移"
  	  mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
  	  echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
  	  master)
  			  notify master
  			  ;;
  	  backup)
  			  notify backup
  			  ;;
  	  fault)
  			  notify fault
  			  ;;
  	  *)
  			  echo "Usage: $(basename $0) {master|backup|fault}"
  			  exit 1
  			  ;;
esac

修改配置文件: vim keepalived.conf

vrrp_instance下与authentication同级处

notify_master "/etc/keepalived/mail_send.sh master"
notify_backup "/etc/keepalived/mail_send.sh backup"
notify_fault "/etc/keepalived/mail_send.sh fault"

整体配置文件

! Configuration File for keepalived

vrrp_script chk_nginx {
    script "/etc/keepalived/nginx_check.sh"
    interval 2
    #weight -30
}

vrrp_instance VI_1 {
    state MASTER
    interface enp0s3
    priority 100
    advert_int 1
    virtual_router_id 99
    # 当进入master/backup/fault状态时触发脚本, 可携带参数
    notify_master "/etc/keepalived/mail_send.sh master"
    notify_backup "/etc/keepalived/mail_send.sh backup"
    notify_fault "/etc/keepalived/mail_send.sh fault"
    authentication {
        auth_type PASS
        auth_pass 221531
    }

    track_script {
        chk_nginx
    }

    virtual_ipaddress {
        192.168.0.99
    }
}

配置到keepalived中, 方法2

脚本内容, 下面这个是漂移到master时, 另外创建backup和fault

#!/bin/bash
contacts='收件人邮箱1, 收件人邮箱2'
ip a > ipa_temp.txt

echo "$(date +'%F %T'): Keepalived instance I became MASTER on $(hostname).    --- from master" | mail -s "Master Keepalived notification" -a ipa_temp.txt "$contacts"

修改配置文件: vim keepalived.conf

vrrp_instance下与authentication同级处, 后面的root是执行人和所在组

notify_master /etc/keepalived/mail_send_master.sh root root
notify_backup /etc/keepalived/mail_send_backup.sh root root
notify_fault /etc/keepalived/mail_send_fault.sh root root

测试状态转移时有没有邮箱接收到即可, 通过重启, 停用
脚本调试
如果遇到脚本配置好了, 但是脚本并没有执行,可通过一下步骤调试.
- 单独执行脚本查看是否成功(检查脚本可执行性)
- 检查keepalived的执行状态: systemctl status keepalived
- 检查keepalived日志, 默认位置/var/log/messages
- 脚本中添加调试日志, 如: echo "notify_master script called at $(date)" >> /var/log/keepalived-notify.log
- 通过命令检查配置文件是否正确:sudo keepalived -t -f /etc/keepalived/keepalived.conf
- 如果之前成功过后来不执行了, 可使用故障转移工具，例如 Pacemaker 或 Corosync
- 可能是版本问题, 可尝试升级最新版本
参考:
- 参考1: keepalived 日志设置、邮箱设置和通知
- 参考2: keepalived邮件通知
- 参考3: Keepalived故障切换时的邮件通知
- 参考4: mail指令同时发送给多个用户

学习链接

posted @ 2023-04-05 17:27 Codorld 阅读(398) 评论(2) 编辑收藏举报

刷新页面返回顶部

Codorld

Don't BB. Show me the code.