LVS DR模式实战案例

概论

LVS介绍以及DR模式介绍可以看我的这篇博客：https://www.cnblogs.com/yanshicheng/p/15760843.html
本次实验用到五台机器，在 DR 模型中所有主机都需要配置 VIP。并且RS要做ARP相应通告级别。
解决地址冲突三种方式：

在前端网关做静态地址绑定。
在各 RS 使用 arptables
在各 RS 修改内核参数，来限制通告级别，主要为下面的参数

限制响应级别：arp_ignore
	0：默认值，表示可使用本地任意接口上配置的任意地址进行响应
	1：仅在请求的目标IP配置在本地主机的接收到请求报文的接口上时，才给予响应
限制通告级别：arp_announce
	0：默认值，把本机所有接口的所有信息向每个接口的网络进行通告
	1：尽量避免将接口信息向非直接连接网络进行通告
	2：必须避免将接口信息向非本网络进行通告

配置要点：

所有RS服务器要限制apr通告级别
DR 服务器采用双IP桥接网络，一个时VIP，一个时DIP。
每个 RS 服务器都需要配置 VIP。
每个 RS 服务器可以出外网。
客户端访问为VIP，生产环境VIP应该是公网IP。

DR模式单网段配置示例

拓扑图

客户端配置

# ip 地址
3: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1c:42:45:3b:df brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.100/24 brd 192.168.10.255 scope global enp0s6
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:1:21c:42ff:fe45:3bdf/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 2591986sec preferred_lft 604786sec
    inet6 fe80::21c:42ff:fe45:3bdf/64 scope link 
       valid_lft forever preferred_lft forever
# 默认网关为 路由器入口测地址
root@ops102:~# ip r
default via 192.168.10.201 dev enp0s6 proto static

路由器配置

开启内核参数： net.ipv4.ip_forward = 1
网口enp0s7(入口侧)IP：192.168.10.201
网口enp0s7(内侧口)IP：10.10.0.100
1. 同时也是各个RS的网关

LVS配置

配置IP地址及网关

# 主机IP
3: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1c:42:4d:88:f8 brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.104/24 brd 10.10.0.255 scope global enp0s6
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:1:21c:42ff:fe4d:88f8/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 2591706sec preferred_lft 604506sec
    inet6 fe80::21c:42ff:fe4d:88f8/64 scope link 
       valid_lft forever preferred_lft forever
# 默认网关指向路由器内侧口
root@ops104:~# ip r
default via 10.10.0.100 dev enp0s6 proto static

配置VIP
DR及RS都一样

root@ops104:~# ifconfig lo:1 10.0.0.200/32

配置LVS策略

root@ops104:~# ipvsadm -A -t 10.10.0.200:80 -s wlc
root@ops104:~# ipvsadm -a -t 10.10.0.200:80 -r 10.10.0.105 -g
root@ops104:~# ipvsadm -a -t 10.10.0.200:80 -r 10.10.0.106 -g
root@ops104:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.10.0.200:80 wlc
  -> 10.10.0.105:80               Route   1      0          0         
  -> 10.10.0.106:80               Route   1      0          0

RS 配置

所有 RS 配置只有IP不一样其余一样。

IP配置

3: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1c:42:ef:da:f7 brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.105/24 brd 10.10.0.255 scope global enp0s6
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:1:21c:42ff:feef:daf7/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 2591942sec preferred_lft 604742sec
    inet6 fe80::21c:42ff:feef:daf7/64 scope link 
       valid_lft forever preferred_lft forever
# 默认网关为路由器内侧口
root@ops105:~# ip r
default via 10.10.0.100 dev enp0s6 proto static

VIP配置

root@ops105:~#  ifconfig lo:1 10.0.0.100/32

ARP限制

root@ops105:~#     echo -e "net.ipv4.conf.all.arp_ignore = 1 \n
> net.ipv4.conf.lo.arp_ignore = 1 \n
> net.ipv4.conf.lo.arp_announce = 1 \n
> net.ipv4.conf.all.arp_announce = 1
> " >> /etc/sysctl.conf

root@ops105:~#  sysctl -p

客户端访问测试

客户端访问 VIP

root@ops102:~# for i in $(seq 10); do curl 10.10.0.200; done
this is WEB1 10.10.0.105 page
this is WEB2 10.10.0.106 page
this is WEB1 10.10.0.105 page
this is WEB2 10.10.0.106 page
this is WEB1 10.10.0.105 page
this is WEB2 10.10.0.106 page
this is WEB1 10.10.0.105 page
this is WEB2 10.10.0.106 page
this is WEB1 10.10.0.105 page
this is WEB2 10.10.0.106 page

DR模式多网段配置示例

多网段其实和单网段配置类似。

拓扑图

客户端

客户端保持不变

路由器

路由器增加网卡并且增加IP 10.211.55.103

root@ops103:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1c:42:1a:f1:29 brd ff:ff:ff:ff:ff:ff
    inet 10.211.55.103/24 brd 10.211.55.255 scope global enp0s5
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:0:21c:42ff:fe1a:f129/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 2591953sec preferred_lft 604753sec
    inet6 fe80::21c:42ff:fe1a:f129/64 scope link 
       valid_lft forever preferred_lft forever
3: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1c:42:ed:cb:04 brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.100/24 brd 10.10.0.255 scope global enp0s6
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:1:21c:42ff:feed:cb04/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 2591953sec preferred_lft 604753sec
    inet6 fe80::21c:42ff:feed:cb04/64 scope link 
       valid_lft forever preferred_lft forever
4: enp0s7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1c:42:47:45:a9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.201/24 brd 192.168.10.255 scope global enp0s7
       valid_lft forever preferred_lft forever
    inet6 fdb2:2c26:f4e4:1:21c:42ff:fe47:45a9/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 2591953sec preferred_lft 604753sec
    inet6 fe80::21c:42ff:fe47:45a9/64 scope link 
       valid_lft forever preferred_lft forever

DR

lvs服务器配置如下

ifconfig lo:1 10.211.55.200/32

root@ops104:~# ipvsadm -A -t 10.211.55.200:80  -s wlc
root@ops104:~# ipvsadm -a -t 10.211.55.200:80  -r 10.10.0.105 -g
root@ops104:~# ipvsadm -a -t 10.211.55.200:80  -r 10.10.0.106 -g
root@ops104:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.211.55.200:80 wlc
  -> 10.10.0.105:80               Route   1      0          0         
  -> 10.10.0.106:80               Route   1      0          0

RS

RS修改VIP即可：ifconfig lo:1 10.211.55.200/32
网关及IP保持不变

测试

初始化脚本

DR

#!/bin/bash
#Author:YanShicheng
#Date:2021-11-24
vip='172.16.0.100'
iface='lo:1'
mask='255.255.255.255'
rs_addr_list=("10.10.0.102:80 1" "10.10.0.103:80 2")
port='80'
scheduler='wrr'
type='-g'


case $1 in
    start)
        ifconfig $iface $vip netmask $mask #broadcast $vip up
        iptables -F
        ipvsadm -C
        ipvsadm -A -t ${vip}:${port} -s $scheduler

        for line in "${rs_addr_list[@]}"; do
            addr=`echo ${line} | awk '{print $1}' | sed s/[[:space:]]//g`
            weight=`echo ${line} | awk '{print $NF}'`
            if [ -n $weight ];then
                ipvsadm -a -t ${vip}:${port} -r ${addr} $type -w $weightip vsipvs
                echo "ipvsadm -a -t ${vip}:${port} -r ${addr} $type -w $weight"
            else
                ipvsadm -a -t ${vip}:${port} -r ${addr} $type
                echo "ipvsadm -a -t ${vip}:${port} -r ${addr} $type "
            fi
        done
        echo "The VS Server is Ready!"
        ;;
    stop)
        ipvsadm -C
        ifconfig $iface down
        echo "The VS Server is Canceled!"
        ;;
    *)
        echo "Usage: $(basename $0) start|stop"
        exit 1
        ;;
esac

RS

#!/bin/bash
#Author:YanShicheng
#Date:2020-11-24
vip=172.16.0.100
mask='255.255.255.255'
dev=lo:1

case $1 in
    start)
    echo -e "net.ipv4.conf.all.arp_ignore = 1 \n
net.ipv4.conf.lo.arp_ignore = 1 \n
net.ipv4.conf.lo.arp_announce = 1 \n
net.ipv4.conf.all.arp_announce = 1
" >> /etc/sysctl.conf
        sysctl -p > /dev/null

        ifconfig $dev $vip netmask $mask 
        echo "The RS Server is Ready!"
        ;;
    stop)
        ifconfig $dev down
        sed -i '/^net\.ipv4\.conf\.all\.arp_ignore/d' /etc/sysctl.conf
        sed -i '/^net\.ipv4\.conf\.lo\.arp_ignore/d' /etc/sysctl.conf
        sed -i '/^net\.ipv4\.conf\.lo\.arp_announce/d' /etc/sysctl.conf
        sed -i '/^net\.ipv4\.conf\.all\.arp_announce/d' /etc/sysctl.conf
        sysctl -p > /dev/null
        echo "The RS Server is Canceled!"
        ;;
    *) 
        echo "Usage: $(basename $0) start|stop"
        exit 1
        ;;
esac

posted @ 2022-01-10 18:00 闫世成阅读(134) 评论(0) 编辑收藏举报

刷新页面返回顶部

闫世成

专注自动化运维技术