openstack controller ha测试环境搭建记录（三）——配置haproxy
haproxy.cfg请备份再编辑：
# vi /etc/haproxy/haproxy.cfg
global
chroot /var/lib/haproxy
daemon
group haproxy
maxconn 4000
pidfile /var/run/haproxy.pid
user haproxy
defaults
log global
maxconn 4000
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout check 10s
listen galera_cluster
bind 10.0.0.10:3306
balance source
option httpchk
server controller2 10.0.0.12:3306 check port 9200 inter 2000 rise 2 fall 5
server controller3 10.0.0.13:3306 check port 9200 inter 2000 rise 2 fall 5
每个节点都要编辑haproxy.cfg。
此处只是测试haproxy，为每个节点的3306端口做负责平衡。3306是mariadb的监听端口，此处未安装，将在下一篇文章中安装。
由于需要将集群资源绑定到VIP，需要修改各节点的内核参数：
# echo 'net.ipv4.ip_nonlocal_bind = 1'>>/etc/sysctl.conf
# sysctl -p
在集群中增加haproxy服务资源：
# crm configure primitive haproxy lsb:haproxy op monitor interval="30s"
“lsb:haproxy”表示haproxy服务。
ERROR: lsb:haproxy: got no meta-data, does this RA exist?
ERROR: lsb:haproxy: got no meta-data, does this RA exist?
ERROR: lsb:haproxy: no such resource agent
Do you still want to commit (y/n)? n
似乎crm命令不识别“haproxy”服务。查看crm目前能识别哪些服务：
# crm ra list lsb
netconsole network
netconsole和network位于/etc/rc.d/init.d目录中，是centos7默认情况下仅有的服务脚本，推测在此目录创建haproxy的服务脚本即可（每个节点均要）：
# vi /etc/rc.d/init.d/haproxy
内容如下：
#!/bin/bash
case "$1" in
start)
systemctl start haproxy.service
;;
stop)
systemctl stop haproxy.service
;;
status)
systemctl status -l haproxy.service
;;
restart)
systemctl restart haproxy.service
;;
*)
echo '$1 = start|stop|status|restart'
;;
esac
记得授予可执行权限：
chmod 755 /etc/rc.d/init.d/haproxy
再次确认crm命令是否能识别“haproxy”：
# crm ra list lsb
haproxy netconsole network
已经有了haproxy，“service haproxy status”命令也能用了，请再次尝试创建haproxy服务资源。
查看资源状态：
# crm_mon
Last updated: Tue Dec 8 11:28:35 2015
Last change: Tue Dec 8 11:28:28 2015
Stack: corosync
Current DC: controller2 (167772172) - partition with quorum
Version: 1.1.12-a14efad
2 Nodes configured
2 Resources configured
Online: [ controller2 controller3 ]
myvip (ocf::heartbeat:IPaddr2): Started controller2
haproxy (lsb:haproxy): Started controller3
目前haproxy资源在节点controller3上，在controller3上查看haproxy服务状态，已是active：
# systemctl status -l haproxy.service
定义HAProxy和VIP必须在同一节点上运行：
# crm configure colocation haproxy-with-vip INFINITY: haproxy myvip
定义先接管VIP之后才启动HAProxy：
# crm configure order haproxy-after-vip mandatory: myvip haproxy
--------------------------------------------------------------------------------------------
把HAProxy实例部署在openstack的控制节点上，已经成为一种共识。实例的数量最好是奇数，如3、5等。
官网完整的haproxy.cfg实例内容如下：
global
chroot /var/lib/haproxy
daemon
group haproxy
maxconn 4000
pidfile /var/run/haproxy.pid
user haproxy
defaults
log global
maxconn 4000
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout check 10s
listen dashboard_cluster
bind <Virtual IP>:443
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:443 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:443 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:443 check inter 2000 rise 2 fall 5
listen galera_cluster
bind <Virtual IP>:3306
balance source
option httpchk
server controller1 10.0.0.4:3306 check port 9200 inter 2000 rise 2 fall 5
server controller2 10.0.0.5:3306 backup check port 9200 inter 2000 rise 2 fall 5
server controller3 10.0.0.6:3306 backup check port 9200 inter 2000 rise 2 fall 5
listen glance_api_cluster
bind <Virtual IP>:9292
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:9292 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:9292 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:9292 check inter 2000 rise 2 fall 5
listen glance_registry_cluster
bind <Virtual IP>:9191
balance source
option tcpka
option tcplog
server controller1 10.0.0.1:9191 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:9191 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:9191 check inter 2000 rise 2 fall 5
listen keystone_admin_cluster
bind <Virtual IP>:35357
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:35357 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:35357 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:35357 check inter 2000 rise 2 fall 5
listen keystone_public_internal_cluster
bind <Virtual IP>:5000
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:5000 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:5000 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:5000 check inter 2000 rise 2 fall 5
listen nova_ec2_api_cluster
bind <Virtual IP>:8773
balance source
option tcpka
option tcplog
server controller1 10.0.0.1:8773 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8773 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8773 check inter 2000 rise 2 fall 5
listen nova_compute_api_cluster
bind <Virtual IP>:8774
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:8774 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8774 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8774 check inter 2000 rise 2 fall 5
listen nova_metadata_api_cluster
bind <Virtual IP>:8775
balance source
option tcpka
option tcplog
server controller1 10.0.0.1:8775 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8775 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8775 check inter 2000 rise 2 fall 5
listen cinder_api_cluster
bind <Virtual IP>:8776
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:8776 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8776 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8776 check inter 2000 rise 2 fall 5
listen ceilometer_api_cluster
bind <Virtual IP>:8777
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:8777 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8777 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8777 check inter 2000 rise 2 fall 5
listen spice_cluster
bind <Virtual IP>:6080
balance source
option tcpka
option tcplog
server controller1 10.0.0.1:6080 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:6080 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:6080 check inter 2000 rise 2 fall 5
listen neutron_api_cluster
bind <Virtual IP>:9696
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:9696 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:9696 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:9696 check inter 2000 rise 2 fall 5
listen swift_proxy_cluster
bind <Virtual IP>:8080
balance source
option tcplog
option tcpka
server controller1 10.0.0.1:8080 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8080 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8080 check inter 2000 rise 2 fall 5
