rabbitmq+ keepalived+haproxy高可用集群详细命令
公司要用rabbitmq研究了两周，特把
rabbitmq 高可用的研究成果备下
后续会更新封装的类库
安装erlang
wget http://www.gelou.me/yum/erlang-18.3-1.el6.x86_64.rpm
yum install erlang-18.3-1.el6.x86_64.rpm
安装rabbitmq rpm包：
wget http://www.gelou.me/yum/rabbitmq-server-3.6.1-1.noarch.rpm
rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
yum install rabbitmq-server-3.6.1-1.noarch.rpm
启动rabbitmq，并验证启动情况
rabbitmq-server --detached &ps aux |grep rabbitm
以服务的方式启动
service rabbitmq-server start
开启系列防火墙端口
/sbin/iptables -I INPUT -p tcp --dport 5672 -j ACCEPT
/sbin/iptables -I INPUT -p tcp --dport 4369 -j ACCEPT
/sbin/iptables -I INPUT -p tcp --dport 25672 -j ACCEPT
/sbin/iptables -I INPUT -p tcp --dport 15672 -j ACCEPT
iptables -I INPUT -p tcp --dport 8888 -j ACCEPT
iptables -I INPUT -p tcp --dport 5670 -j ACCEPT
iptables -I INPUT -p tcp --dport 5670 -j ACCEPT
/etc/rc.d/init.d/iptables save
/etc/init.d/iptables restart
/etc/init.d/iptables status
设置开机启动
rpm -aq |grep chkconfig
export PATH=/sbin:$PATH
chkconfig
chkconfig rabbitmq-server on
安装web管理
rabbitmq-plugins enable rabbitmq_management
访问地址
http://172.20.16.57:15672/#/
重复安装第二台等
集群命令重复安装第二台等
vim /etc/hosts
192.168.43.74 TD-YichehuiLinu
192.168.87.5 YichehuiLinux
重启
reboot
chmod 700 /var/lib/rabbitmq/.erlang.cookie
echo -n "AZVOCZYZZBVFLBPTBXU" > /var/lib/rabbitmq/.erlang.cookie
chmod 400 /var/lib/rabbitmq/.erlang.cookie
ps -ef | grep ^rabbitmq | cut -c 9-16 | xargs kill -9
rabbitmq-server --detached &ps aux |grep rabbitm
机器全启动后分别执行
rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit@各机器名
rabbitmqctl start_app
rabbitmqctl cluster_status
设置policy，以ha.开头的队列将会被镜像到集群其他所有节点,一个节点挂掉然后重启后会自动同步队列消息
必须将整个RabbitMQ集群的状态设置为镜像模式， 节点都执行
rabbitmqctl set_policy ha-all-queue "^ha\." '{"ha-mode":"all","ha-sync-mode":"automatic"}'
备用命令
rabbitmqctl stop_app
rabbitmqctl change_cluster_node_type disc
或
rabbitmqctl change_cluster_node_type ram
rabbitmqctl start_app
添加用户名admin,密码admin
rabbitmqctl add_user zhang 1234
设置admin用户tags为管理员
rabbitmqctl set_user_tags zhang administrator
设置admin用户配置、写、读的权限
rabbitmqctl set_permissions -p / zhang ".*" ".*" ".*"
haproxy搭建
1、安装haproxy
yum install haproxy
2、安装rsyslog
a. 检查rsyslog是否已安装
rpm -q rsyslog
b. 添加haproxy的log配置
cd /etc/rsyslog.d
vim haproxy.conf
=== 文件内容
$ModLoad imudp
$UDPServerRun 514
local2.* /var/log/haproxy.log
local3.* /var/log/haproxy.log
local10.* /var/log/haproxy.log
==========
c. 编辑/etc/sysconfig/rsyslog
vim /etc/sysconfig/rsyslog
添加内容：
SYSLOGD_OPTIONS="-c 2 -r -m 0"
d. 创建日志文件,并授权
cd /var/log
touch haproxy.log
chmod a+w haproxy.log
e. 重启rsyslog服务用法：
/etc/init.d/rsyslog {start|stop|restart|condrestart|try-restart|reload|force-reload|status}
/etc/init.d/rsyslog restart
配置所有机器
4、配置haproxy
> /etc/haproxy/haproxy.cfg
vim /etc/haproxy/haproxy.cfg
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2 notice
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode tcp
option tcplog
option dontlognull
option http-server-close
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
###haproxy statistics monitor by laijingli 20160222
listen statics 0.0.0.0:8888
mode http
log 127.0.0.1 local0 debug
transparent
stats refresh 60s
stats uri / haproxy-stats
stats realm Haproxy \ statistic
stats auth zhang:1234
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
listen rabbitmq_cluster
bind 0.0.0.0:5670
mode tcp
balance roundrobin
server ych51 172.20.16.51:5672 check inter 5000 rise 2 fall 2
server ych50 172.20.16.50:5672 check inter 5000 rise 2 fall 2
server TD-YichehuiLinu 192.168.43.74:5672 check inter 5000 rise 2 fall 2
server YichehuiLinux 192.168.87.5:5672 check inter 5000 rise 2 fall 2
server YichehuiLinux 192.168.87.5:9200 check inter 5000 rise 2 fall 2
5、 启动haproxy
/etc/init.d/haproxy restart
6、 备用命令停止haproxy
killall haproxy
http://192.168.87.5:8888/
http://192.168.43.74:15672/#/
三、安装和设置keepalived
1、安装keepalived
yum install keepalived
chkconfig --add keepalived
chkconfig --level 35 keepalived on
2、修改配置文件
配置keepalived
> /etc/keepalived/keepalived.conf
vim /etc/keepalived/keepalived.conf
内容：
=====================================================================
! Configuration File for keepalived
# 全局配置，demo中配置了邮件信息，此处因为调试中，暂时没有添加
global_defs {
}
# 集群资源监控，组合track_script进行
vrrp_script check_haproxy {
script "killall -0 haproxy"
interval 2
}
vrrp_instance HAPROXY_HA {
# 设置当前主机为主节点，如果是备用节点，则设置为BACKUP
# 备用节点时，设置为：
# state BACKUP
state MASTER
# 指定HA监测网络接口，可以用ifconfig查看来决定设置哪一个
interface eth1
# 虚拟路由标识，同一个VRRP实例要使用同一个标识，主备机
virtual_router_id 80
# 因为当前环境中VRRP组播有问题，改为使用单播发送VRRP报文
# 这个地方需要关注，之前未做此设置，结果主备节点互相不能发现，因此主备节点都升级成了MASTER，并且绑定了VIP
# 备用节点时，内容为：
# unicast_src_ip 192.168.200.199
# unicast_peer {
# 192.168.200.200
# }
# 设置优先级，确保主节点的优先级高过备用节点
# 备用节点时，设置为：
priority 80
# priority 100
# 用于设定主备节点间同步检查时间间隔
advert_int 2
# 设置高可用集群中不抢占功能，在主机down后，从机接管，当主机重新恢复后，设置此功能，备机将继续提供服务，从而避免因切换导致的隐患
nopreempt
# 设置主备节点间的通信验证类型及密码，同一个VRRP实例中需一致
authentication {
auth_type PASS
auth_pass 1234
}
# 当keepalived切换状态到MASTER时，执行脚本
notify_master "/etc/keepalived/master.sh"
# 当keepalived切换状态到BACKUP时，执行脚本
notify_backup "/etc/keepalived/backup.sh"
# 当keepalived切换状态到FAULT时，执行脚本
notify_fault "/etc/keepalived/fault.sh"
# 当keepalived切换状态到STOP时，执行脚本
notify_fault "/etc/keepalived/stop.sh"
# 集群资源监控，组合vrrp_script进行
track_script {
check_haproxy
}
# 设置虚拟IP地址，当keepalived状态切换为MASTER时，此IP会自动添加到系统中
# 当状态切换到BACKUP时，此IP会自动从系统中删除
# 可以通过命令ip add查看切换后的状态
virtual_ipaddress {
192.168.43.201
}
}
====================================================================
配置全部
3、启动服务
因为是为了实现haproxy的高可用，启动时需要顺序启动：
(1) 启动haproxy：
haproxy -f /etc/haproxy/haproxy.cfg
(2) 启动keeepalived：先启动master节点，后启动BACKUP节点
/etc/init.d/keepalived restart
4、停止keeepalived服务
/etc/init.d/keepalived stop
大哥广告 阁楼
