Saltstack之multi-master
一、实验环境：
1、salt版本：
[root@master master]# salt --versions-report Salt: 2015.5.10 Python: 2.7.5 (default, Nov 6 2016, 00:28:07) Jinja2: 2.7.2 M2Crypto: 0.21.1 msgpack-python: 0.4.8 msgpack-pure: Not Installed pycrypto: 2.6.1 libnacl: Not Installed PyYAML: 3.10 ioflo: Not Installed PyZMQ: 14.3.1 RAET: Not Installed ZMQ: 3.2.5 Mako: Not Installed Tornado: Not Installed timelib: Not Installed dateutil: Not Installed
2、系统版本：
[root@master master]# cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core)
二、Salt搭建主备master
1、安装新的master server
2、copy master keys到新的master对应的目录(master.pem和master.pub)
3、启动新的master进程
4、配置minions配置文件
5、Restart minions
6、在新的master上accept keys
7、测试两个salt-master对salt-minion的test.ping
[root@master master]# salt -L 192.168.163.13 test.ping 192.168.163.13: True [root@standby minions]# salt -L 192.168.163.13 test.ping 192.168.163.13: True
备注：
配置multi-master主要的问题是：每一个master使用相同的private key. Private key在master第一次启动时自动生成的(注意：配置multi-master时，一定要在启动新master前将老master的private key copy到对应目录)。
修改minion配置文件：
master
-saltmaster1.example.com
-saltmaster2.example.com
三、遇到的问题：
1、在minion端进行salt-call测试的时候(前提条件将主master stop)，发现：
[root@standby salt]# salt-call test.ping [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 1 of 4) [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 2 of 4) [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 3 of 4) [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 4 of 4) [WARNING ] Attempted to authenticate with master 192.168.199.39 and failed [WARNING ] Master ip address changed from 192.168.199.39 to 192.168.163.13 local: True ----以上测试结果是将minion中的auth_tries修改为4，默认值为7. 将值改成3次，并关闭主备master的测试结果： [root@standby minion]# salt-call test.ping [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 1 of 3) [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 2 of 3) [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 3 of 3) [WARNING ] Attempted to authenticate with master 192.168.199.39 and failed [WARNING ] Master ip address changed from 192.168.199.39 to 192.168.163.13 [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 1 of 3) [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 2 of 3) [INFO ] SaltReqTimeoutError: after 60 seconds. (Try 3 of 3) [WARNING ] Attempted to authenticate with master 192.168.163.13 and failed [ERROR ] An un-handled exception was caught by salt's global exception handler: AttributeError: 'SMinion' object has no attribute 'functions' Traceback (most recent call last): File "/usr/bin/salt-call", line 11, in <module> salt_call() File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 227, in salt_call client.run() File "/usr/lib/python2.7/site-packages/salt/cli/call.py", line 71, in run caller.run() File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 236, in run ret = self.call() File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 107, in call if fun not in self.minion.functions: AttributeError: 'SMinion' object has no attribute 'functions' Traceback (most recent call last): File "/usr/bin/salt-call", line 11, in <module> salt_call() File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 227, in salt_call client.run() File "/usr/lib/python2.7/site-packages/salt/cli/call.py", line 71, in run caller.run() File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 236, in run ret = self.call() File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 107, in call if fun not in self.minion.functions: AttributeError: 'SMinion' object has no attribute 'functions'
2、master数据共享问题：
masters之间并不会共享信息，public keys需要在每台master上accept，文件共享需要手工完成，或者使用类似git工具确保file_roots目录文件一致。
具体需要同步的目录有：
Minion keys:
- /etc/salt/pki/master/minions
- /etc/salt/pki/master/minions_pre
- /etc/salt/pki/master/minions_rejected
备注：直接共享/etc/salt/master目录是强烈反对的。允许外部访问master.pem key将带来严重的安全风险。
4、minion keys可以参考使用：
方案一：
*/10 * * * * rsync -av --progress --delete --timeout=30 root@192.168.199.39:/etc/salt/pki/master/minions/ /etc/salt/pki/master/minions/
方案二：修改salt-key的源代码：
当主master有accept的时候同步给备master，在配置文件中配置备机IP，只有两边同步成功了才算成功；
删除minion的时候只用通过salt-key -d的方式删除，或者配合rsync的方式，防止通过rm的方式删除minion。
5、file_roots和pillar_roots等文件可以放在git上。
需要提及的是，本实验的salt版本需要修改minion.py文件：
因为minion注册的时候会先往IP小的机器注册，而无法按你指定的IP顺序注册。
修改代码如下：
从minion.py代码中查看得到for master in set(self.opts['master']):中
class MultiMinion(MinionBase): ''' Create a multi minion interface, this creates as many minions as are defined in the master option and binds each minion object to a respective master. ''' # timeout for one of the minions to auth with a master MINION_CONNECT_TIMEOUT = 5 def __init__(self, opts): super(MultiMinion, self).__init__(opts) def minions(self): ''' Return a dict of minion generators bound to the tune_in method dict of master -> minion_mapping, the mapping contains: opts: options used to create the minion last: last auth attempt time auth_wait: time to wait for next auth attempt minion: minion object generator: generator function (non-blocking tune_in) ''' if not isinstance(self.opts['master'], list): log.error( 'Attempting to start a multimaster system with one master') sys.exit(salt.defaults.exitcodes.EX_GENERIC) ret = {} #在这里对master进行了一个排序 for master in self.opts['master']: # for master in set(self.opts['master']): s_opts = copy.deepcopy(self.opts) s_opts['master'] = master s_opts['multimaster'] = True ret[master] = {'opts': s_opts, 'last': time.time(), 'auth_wait': s_opts['acceptance_wait_time']} try: minion = Minion( s_opts, self.MINION_CONNECT_TIMEOUT, False, 'salt.loader.{0}'.format(master)) ret[master]['minion'] = minion ret[master]['generator'] = minion.tune_in_no_block() except SaltClientError as exc: log.error('Error while bringing up minion for multi-master. Is master at {0} responding?'.format(master)) return ret
