Haproxy04-Haproxy的基本使用
1、HAProxy简介
- HAProxy是使用C语言编写的开源软件,提供双机热备、高可用、负载均衡、虚拟主机、基于TCP(四层)和HTTP(七层)的应用代理、图形界面查看信息等功能。
- HAProxy配置简单、维护方便,而且可以对后端服务器进行健康状态检测(相当于keepalived健康状态检测)。
- 当其代理的后端服务器出现故障时,Haproxy会自动的将其摘除。
- 当故障的后端服务器恢复后,Haproxy会自动将其加入进来,对外提供服务。
- HAProxy特别适用于高负载、访问量大,且需要会话保持以及七层代理的业务应用。HAProxy运行在普通服务器上,就可以支持数以万计的并发连接。并且它的运行模式使得它可以很简单、安全的整合到各种已有的网站架构中。
- Haproxy从1.3版本起,引入了frontend、backend的功能。
- frontend(acl规则匹配)可以让运维管理人员根据任意HTTP请求头做规则匹配,然后把请求定向到相应的backend(事先定义好的server pools,等待前端把请求转过来的服务器组)。
- 通过frontend和backend可以很容易的实现haproxy的各种7层代理功能。
- HAProxy支持两种代理模式:
- (1)基于4层的tcp应用代理(例如,可用于邮件服务、内部协议通信服务器、MySQL、HTTPS服务等)。
- (2)基于7层的http代理。在4层tcp代理模式下,HAProxy仅在客户端和服务器之间进行流量转发。但是在7层http代理模式下,HAProxy会分析应用层协议,能通过允许、拒绝、交换、增加、修改或者删除请求头(request)或者响应头(response)中的内容来控制。
- HAProxy支持两种处理模式:
- 单进程处理模式:所有客户端连接全部都由同一个服务进程来处理,目标就是等待连接,来一个分配一个,主要消耗cpu。
- 多线程处理模式:多线程模式消耗内存,会限制并发而且多线程需要进程间通信,也会消耗相当多的cpu资源。
- Haproxy软件的最大优势是可以对请求头进行操作以及sesson会话功能。(七层)
2、健康状态检查
- 是否对一个后端服务器进行健康状态检查:
- 如果后端服务器没有设置check,就不进行健康状态检查,并且始终认为后端服务器是可用的。
- 如果后端服务器设置了check,就进行健康状态检查。
- 如果设置了check,但没有配置检查方法,仅进行简单的tcp检查
- 如果设置了check,并配置检查方法,就按照指定的检查方法进行检查。
- 如果要深入探究健康状态检测可以查看下面参数:
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
global段参数: max-spread-checks max-spread-checks spread-checks 代理参数矩阵 -- keyword -------------------------- defaults - frontend - listen -- backend - http-check comment X - X X http-check connect X - X X http-check disable-on-404 X - X X http-check expect X - X X http-check send X - X X http-check send-state X - X X http-check set-var X - X X http-check unset-var X - X X tcp-check comment X - X X tcp-check connect X - X X tcp-check expect X - X X tcp-check send X - X X tcp-check send-lf X - X X option ldap-check X - X X option external-check X - X X option mysql-check X - X X option pgsql-check X - X X option redis-check X - X X option spop-check - - - X option tcp-check X - X X option ssl-hello-chk X - X X option smtpchk X - X X option smtpchk <hello> <domain> option httpchk X - X X option httpchk <uri> option httpchk <method> <uri> option httpchk <method> <uri> <version> option log-health-checks (*) X - X X timeout check X - X X
2.1、基于端口的健康状态检测(TCP)
- 在进行健康状态检查时,进检查端口是否存在。
1、创建配置文件
]# cat > /apps/haproxy/haproxy.cfg << EOF global daemon pidfile /apps/haproxy/haproxy.pid chroot /apps/haproxy/empty user haproxy group haproxy log 127.0.0.1 local3 info maxconn 4096 hard-stop-after 5m default-path config zero-warning node node_name1 description 'global_hh1' defaults mode http log global option httpslog option dontlognull option forwardfor option httpclose option http-server-close option redispatch retries 3 timeout check 2000 timeout http-request 10s timeout queue 15s timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 2m timeout tunnel 4h maxconn 2048 listen stats bind :18080 stats enable stats uri /haproxystatus stats auth admin1:admin1 stats auth admin2:admin2 stats admin if TRUE stats realm Haproxy\ status stats refresh 10s stats hide-version frontend frontend_hh1 description "frontend frontend_hh1" bind :80 default_backend server_hh1 backend server_hh1 balance roundrobin #check开启健康状态检查,inter检查的时间间隔,rise几次成功的检查就认为后端服务器可用,fall几次失败的检测就认为后端服务器不可用。 server httpA 192.168.1.101:80 check inter 1500 rise 3 fall 2 weight 1 server httpB 192.168.1.102:80 check inter 1500 rise 3 fall 2 weight 1 EOF
3、启动httpd服务
//启动两个后端服务器的httpd服务 ]# systemctl start httpd.service
2、停止一个httpd服务
//登录HAProxy的状态页面观察变化(观察结果:相应的后端服务器停止了) ]# systemctl stop httpd.service
3、使用ns开启80端口
//登录HAProxy的状态页面观察变化(观察结果:相应的后端服务器启动了) ]# nc -l 80
2.2、基于后端主机文件的健康状态检测
1、创建配置文件
]# cat > /apps/haproxy/haproxy.cfg << EOF global daemon pidfile /apps/haproxy/haproxy.pid chroot /apps/haproxy/empty user haproxy group haproxy log 127.0.0.1 local3 info maxconn 4096 hard-stop-after 5m default-path config zero-warning node node_name1 description 'global_hh1' defaults mode http log global option httpslog option dontlognull option forwardfor option httpclose option http-server-close option redispatch retries 3 timeout check 2000 timeout http-request 10s timeout queue 15s timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 2m timeout tunnel 4h maxconn 2048 listen stats bind :18080 stats enable stats uri /haproxystatus stats auth admin1:admin1 stats auth admin2:admin2 stats admin if TRUE stats realm Haproxy\ status stats refresh 10s stats hide-version frontend frontend_hh1 description "frontend frontend_hh1" bind :80 default_backend server_hh1 backend server_hh1 balance roundrobin #通过url进行健康状态检查(IP:PORT/hh.html) option httpchk GET /hh.html HTTP/1.0 #check开启健康状态检查,inter检查的时间间隔,rise几次成功的检查就认为后端服务器可用,fall几次失败的检测就认为后端服务器不可用。 server httpA 192.168.1.101:80 check inter 1500 rise 3 fall 2 weight 1 server httpB 192.168.1.102:80 check inter 1500 rise 3 fall 2 weight 1 EOF
2、提供测试页
//在192.168.1.101上执行 ]# cat > /var/www/html/hh.html << EOF <h1>192.168.1.101, check1</h1> EOF //在192.168.1.102上执行 ]# cat > /var/www/html/hh.html << EOF <h1>192.168.1.102, check2</h1> EOF
3、启动httpd服务
//启动两个后端服务器的httpd服务 ]# systemctl start httpd.service
4、删除测试文件
//登录HAProxy的状态页面观察变化(观察结果:相应的后端服务器停止了) ]# rm -f /var/www/html/hh.html
3、实现会话粘性
3.1、基于源IP实现会话绑定
1、创建配置文件
]# cat > /apps/haproxy/haproxy.cfg << EOF global daemon pidfile /apps/haproxy/haproxy.pid chroot /apps/haproxy/empty user haproxy group haproxy log 127.0.0.1 local3 info maxconn 4096 hard-stop-after 5m default-path config zero-warning node node_name1 description 'global_hh1' defaults mode http log global option httpslog option dontlognull option forwardfor option httpclose option http-server-close option redispatch retries 3 timeout check 2000 timeout http-request 10s timeout queue 15s timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 2m timeout tunnel 4h maxconn 2048 listen stats bind :18080 stats enable stats uri /haproxystatus stats auth admin1:admin1 stats auth admin2:admin2 stats admin if TRUE stats realm Haproxy\ status stats refresh 10s stats hide-version frontend frontend_hh1 description "frontend frontend_hh1" bind :80 default_backend server_hh1 backend server_hh1 #通过负载平衡算法实现基于源IP的会话绑定 balance source server httpA 192.168.1.101:80 check inter 1500 rise 3 fall 2 weight 1 server httpB 192.168.1.102:80 check inter 1500 rise 3 fall 2 weight 1 EOF
2、在不同的机器上访问测试页
]# for i in {1..10}; do curl http://10.1.1.12/index.html; done
3.2、基于cookie实现会话绑定
1、创建配置文件
]# cat > /apps/haproxy/haproxy.cfg << EOF global daemon pidfile /apps/haproxy/haproxy.pid chroot /apps/haproxy/empty user haproxy group haproxy log 127.0.0.1 local3 info maxconn 4096 hard-stop-after 5m default-path config zero-warning node node_name1 description 'global_hh1' defaults mode http log global option httpslog option dontlognull option forwardfor option httpclose option http-server-close option redispatch retries 3 timeout check 2000 timeout http-request 10s timeout queue 15s timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 2m timeout tunnel 4h maxconn 2048 listen stats bind :18080 stats enable stats uri /haproxystatus stats auth admin1:admin1 stats auth admin2:admin2 stats admin if TRUE stats realm Haproxy\ status stats refresh 10s stats hide-version frontend frontend_hh1 description "frontend frontend_hh1" bind :80 default_backend server_hh1 backend server_hh1 balance source #在后端启用基于cookie的会话粘性 cookie SERVERID insert indirect nocache #为后端服务器的cookie设置一个值<value> server httpA 192.168.1.101:80 check inter 1500 rise 3 fall 2 weight 1 cookie s101 #为后端服务器的cookie设置一个值<value> server httpB 192.168.1.102:80 check inter 1500 rise 3 fall 2 weight 1 cookie s102 EOF
2、在haproxy服务器上抓包分析
]# sudo tcpdump -i ens33 -s 0 -v -nn -l port ! 22 | grep index -A 3 #第一次请求的请求头,没有header头Cookie GET /index.html HTTP/1.1 Host: 10.1.1.12 Connection: keep-alive Pragma: no-cache Cache-Control: no-cache Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Mobile Safari/537.36 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9 Accept-Encoding: gzip, deflate Accept-Language: zh-CN,zh;q=0.9 #第一次请求的响应头,haproxy的header头中有set-cookie HTTP/1.1 200 OK date: Mon, 25 Jul 2022 08:22:09 GMT server: Apache/2.4.6 (CentOS) last-modified: Sun, 24 Jul 2022 07:09:29 GMT etag: "1e-5e487bf955216" accept-ranges: bytes content-length: 30 content-type: text/html; charset=UTF-8 set-cookie: SERVERID=s102; path=/ cache-control: private #第二次请求的请求头,有header头Cookie GET /index.html HTTP/1.1 Host: 10.1.1.12 Connection: keep-alive Pragma: no-cache Cache-Control: no-cache Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Mobile Safari/537.36 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9 Accept-Encoding: gzip, deflate Accept-Language: zh-CN,zh;q=0.9 Cookie: SERVERID=s102 #第二次请求的响应头,haproxy返回header头中没有set-cookie HTTP/1.1 200 OK date: Mon, 25 Jul 2022 08:23:29 GMT server: Apache/2.4.6 (CentOS) last-modified: Sun, 24 Jul 2022 07:09:29 GMT etag: "1e-5e487bf955216" accept-ranges: bytes content-length: 30 content-type: text/html; charset=UTF-8 <h1>192.168.1.102, test2</h1>
4、不同的域名(header头)使用不同的后端
1、创建配置文件
]# cat > /apps/haproxy/haproxy.cfg << EOF global daemon pidfile /apps/haproxy/haproxy.pid chroot /apps/haproxy/empty user haproxy group haproxy log 127.0.0.1 local3 info maxconn 4096 hard-stop-after 5m #default-path config zero-warning node node_name1 description 'global_hh1' defaults mode http log global option httpslog option dontlognull option forwardfor option httpclose option http-server-close option redispatch retries 3 timeout check 2000 timeout http-request 10s timeout queue 15s timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 2m timeout tunnel 4h maxconn 2048 listen stats bind :18080 stats enable stats uri /haproxystatus stats auth admin1:admin1 stats auth admin2:admin2 stats admin if TRUE stats realm Haproxy\ status stats refresh 10s stats hide-version frontend frontend_hh1 description "frontend frontend_hh1" bind :80 #过滤域名是www.hh1.com的请求 acl hh1 hdr(host) -i www.hh1.com #访问www.hh1.com的请求被代理到 use_backend server_hh1 if hh1 #过滤域名是www.hh2.com的请求 acl hh2 hdr(host) -i www.hh2.com #访问www.hh2.com的请求被代理到 use_backend server_hh2 if hh2 backend server_hh1 balance roundrobin server httpA 192.168.1.101:80 check inter 1500 rise 3 fall 2 backend server_hh2 balance roundrobin server httpB 192.168.1.102:80 check inter 1500 rise 3 fall 2 EOF
2、测试
]# for i in {1..20}; do curl -H "HOST:www.hh1.com" http://10.1.1.12:80/; done ]# for i in {1..20}; do curl -H "HOST:www.hh2.com" http://10.1.1.12:80/; done
5、不同的url使用不同的后端
1、创建配置文件
]# cat > /apps/haproxy/haproxy.cfg << EOF global daemon pidfile /apps/haproxy/haproxy.pid chroot /apps/haproxy/empty user haproxy group haproxy log 127.0.0.1 local3 info maxconn 4096 hard-stop-after 5m #default-path config zero-warning node node_name1 description 'global_hh1' defaults mode http log global option httpslog option dontlognull option forwardfor option httpclose option http-server-close option redispatch retries 3 timeout check 2000 timeout http-request 10s timeout queue 15s timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 2m timeout tunnel 4h maxconn 2048 listen stats bind :18080 stats enable stats uri /haproxystatus stats auth admin1:admin1 stats auth admin2:admin2 stats admin if TRUE stats realm Haproxy\ status stats refresh 10s stats hide-version frontend frontend_hh1 description "frontend frontend_hh1" bind :80 #过滤uri以/static1/开始的请求 acl hh1 path_beg -i /static1/ #访问/static1/的请求被代理到 use_backend server_hh1 if hh1 #过滤uri以/static2/开始的请求 acl hh2 path_beg -i /static2/ #访问/static2/的请求被代理到 use_backend server_hh2 if hh2 backend server_hh1 balance roundrobin server httpA 192.168.1.101:80 check inter 1500 rise 3 fall 2 backend server_hh2 balance roundrobin server httpB 192.168.1.102:80 check inter 1500 rise 3 fall 2 EOF
2、提供测试页
//在192.168.1.101上执行 ]# mkdir /var/www/html/static1/ ]# echo '<h1>192.168.1.101, static1</h1>' > /var/www/html/static1/hh.html //在192.168.1.102上执行 ]# mkdir /var/www/html/static2/ ]# echo '<h1>192.168.1.102, static2</h1>' > /var/www/html/static2/hh.html
3、测试
]# for i in {1..20}; do curl http://10.1.1.12/static1/hh.html; done ]# for i in {1..20}; do curl http://10.1.1.12/static2/hh.html; done
6、错误页面(有问题)
//haproxy本机提供错误页面 errorfile //sorry server来提供错误页面 errorloc和errorloc302 errorloc303
问题1:errorfile
- 使用haproxy 2.5.6和2.6.2版本,不能正常启动haproxy。
]# /apps/haproxy/sbin/haproxy -c -f /apps/haproxy/haproxy.cfg [NOTICE] (10990) : haproxy version is 2.6.2-16a3646 [NOTICE] (10990) : path to executable is /apps/haproxy/sbin/haproxy [ALERT] (10990) : config : parsing [/apps/haproxy/haproxy.cfg:52] : errorfile : '/var/www/html/errorfiles/404.http': unabled to parse headers (error offset: -2) [ALERT] (10990) : config : Error(s) found in configuration file : /apps/haproxy/haproxy.cfg [ALERT] (10990) : config : Fatal errors found in configuration.
问题2:errorloc
- 使用haproxy 2.5.6和2.6.2版本,可以正常启动haproxy,但跳转不到指定的错误页面。
frontend frontend_hh1 #响应码404使用指定的错误页面 errorfile 404 /var/www/html/errorfiles/404.http #errorloc 404 http://www.baidu.com #errorloc303 404 http://www.jingdong.com
7、使用Keepalived+HAProxy实现负载均衡RabbitMQ
- 参见:https://www.cnblogs.com/maiblogs/p/16482831.html
# #