Kingbase V8R6 集群运维系列 -- 在线删除standby节点
案例说明：
KingbaseES V8R6读写分离集群支持在线扩容和缩容，在业务高峰期间，可以通过在线扩容提升集群的横向扩展能力；当业务低谷期间，可以通过在线缩容，节省运营成本。本案例通过在线删除集群standby节点，详细描述集群在线缩容的过程。
案例环境：
操作系统：
[root@node1 ~]# cat /etc/centos-release
CentOS Linux release 7.2.1511 (Core)
数据库适用版本：
KingbaseES V8R6
集群架构：
一、查看当前集群节点状态信息
1、集群节点信息
[kingbase@node3 bin]$ ./repmgr cluster show
ID | Name | Role | Status | Upstream | Location | Priority | Timeline | Connection string
----+---------+---------+-----------+----------+----------+----------+----
1 | node248 | primary | * running | | default | 100 | 5 | host=192.168.7.248 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2 | node249 | standby | running | node248 | default | 100 | 5 | host=192.168.7.249 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
3 | node243 | standby | running | node248 | default | 100 | 5 | host=192.168.7.243 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2、主备流复制信息
test=# select * from sys_stat_replication;
pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start | backend_xmin | sta
te | sent_lsn | write_lsn | flush_lsn | replay_lsn | write_lag | flush_lag | replay_lag | sync_priority | sync_state | reply_time
------+----------+---------+------------------+---------------+-----------
3403 | 16384 | esrep | node249 | 192.168.7.249 | | 14247 | 2021-03-01 12:03:37.651419+08 | | strea
ming | 1/F9005BA0 | 1/F9005BA0 | 1/F9005BA0 | 1/F9005BA0 | | | | 1 | quorum | 2021-03-01 12:31:56.432
910+08
3419 | 16384 | esrep | node243 | 192.168.7.243 | | 34939 | 2021-03-01 12:03:43.876552+08 | | strea
ming | 1/F9005BA0 | 1/F9005BA0 | 1/F9005BA0 | 1/F9005BA0 | | | | 1 | quorum | 2021-03-01 12:30:42.663
663+08
(2 rows)
3、复制槽信息
test=# select * from sys_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
---------------+--------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
repmgr_slot_2 | | physical | | | f | t | 3403 | | | 1/F9005BA0 |
repmgr_slot_3 | | physical | | | f | t | 3419 | | | 1/F9005BA0 |
(2 rows)
二、将standby节点unregister
1、unregister standby node
[kingbase@node3 bin]$ ./repmgr standby unregister;
INFO: connecting to local standby
INFO: connecting to primary database
NOTICE: unregistering node 3
INFO: try to drop slot "repmgr_slot_3" of node 3 on primary node
WARNING: replication slot "repmgr_slot_3" is still active on node 3
INFO: standby unregistration complete
2、查看集群节点信息
[kingbase@node3 bin]$ ./repmgr cluster show
ID | Name | Role | Status | Upstream | Location | Priority | Timeline | Connection string
----+---------+---------+-----------+----------+----------+----------+----
1 | node248 | primary | * running | | default | 100 | 5 | host=192.168.7.248 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2 | node249 | standby | running | node248 | default | 100 | 5 | host=192.168.7.249 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
=== 从以上获知，集群节点信息已经没有unregister的节点信息===
三、停止本地集群服务删除复制槽
1、停止本地集群服务
[kingbase@node3 bin]$ ./sys_monitor.sh stoplocal
Service process "node_export" was killed at process 4928
Service process "postgres_ex" was killed at process 4929
2021-03-01 12:33:52 begin to stop repmgrd on "[localhost]".
2021-03-01 12:33:54 repmgrd on "[localhost]" stop success.
2021-03-01 12:33:54 begin to stop DB on "[localhost]".
waiting for server to shut down.... done
server stopped
2021-03-01 12:33:55 DB on "[localhost]" stop success.
关闭cron计划任务：
[kingbase@node2 bin]$ cat /etc/cron.d/KINGBASECRON
#*/1 * * * * kingbase . /etc/profile;/home/kingbase/cluster/R6HA/KHA/kingbase/bin/kbha -A daemon -f /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../etc/repmgr.conf >> /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../kbha.log 2>&1
#*/1 * * * * kingbase /home/kingbase/cluster/kha/db/bin/network_rewind.sh
停止hamgr进程：
[kingbase@node2 bin]$ ps -ef |grep repmgr
kingbase 7536 1 0 12:20 ? 00:01:12 /home/kingbase/cluster/R6HA/KHA/kingbase/bin/repmgrd -d -v -f /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../etc/repmgr.conf
kingbase 7955 1 0 12:21 ? 00:00:01 /home/kingbase/cluster/R6HA/KHA/kingbase/bin/kbha -A
daemon -f /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../etc/repmgr.conf
[kingbase@node2 bin]$ kill -2 7536 7955
停止export进程：
[kingbase@node2 bin]$ ps -ef |grep kingbase
kingbase 8190 1 0 12:21 ? 00:00:00 /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../share/node_exporter
kingbase 8191 1 0 12:21 ? 00:00:00 /home/kingbase/cluster/R6HA/KHA/kingbase/bin/../share/postgres_exporter
......
[kingbase@node2 bin]$ kill -2 8190
[kingbase@node2 bin]$ kill -2 8191
2、查看复制槽信息
Tips:
复制槽状态active=‘f’时，才能被删除。
test=# select * from sys_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
---------------+--------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
repmgr_slot_2 | | physical | | | f | t | 3403 | | | 1/F9007590 |
repmgr_slot_3 | | physical | | | f | f | | | | 1/F9007590 |
(2 rows)
3、删除复制槽
test=# select sys_drop_replication_slot('repmgr_slot_3');
sys_drop_replication_slot
---------------------------
(1 row)
test=# select * from sys_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
---------------+--------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------
repmgr_slot_2 | | physical | | | f | t | 3403 | | | 1/F9007590 |
(1 row)
4、查看流复制信息
test=# select * from sys_stat_replication;
pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start | backend_xmin | sta
te | sent_lsn | write_lsn | flush_lsn | replay_lsn | write_lag | flush_lag | replay_lag | sync_priority | sync_state | reply_time
------+----------+---------+------------------+---------------+-----------------+-------------+-------------------------------+--------------+------
3403 | 16384 | esrep | node249 | 192.168.7.249 | | 14247 | 2021-03-01 12:03:37.651419+08 | | strea
ming | 1/F9007590 | 1/F9007590 | 1/F9007590 | 1/F9007590 | | | | 1 | quorum | 2021-03-01 12:35:38.819940+08
(1 row)
四、重新启动集群服务测试
[kingbase@node1 bin]$ ./sys_monitor.sh restart
2021-03-01 12:37:43 Ready to stop all DB ...
......
2021-03-01 12:38:15 repmgrd on "[192.168.7.249]" start success.
ID | Name | Role | Status | Upstream | repmgrd | PID | Paused? | Upstream last seen
----+---------+---------+-----------+----------+---------+-------+---------+--------------------
1 | node248 | primary | * running | | running | 14373 | no | n/a
2 | node249 | standby | running | node248 | running | 12219 | no | n/a
2021-03-01 12:38:23 Done.
=== 重新启动集群服务，启动后集群节点状态正常，standby节点删除成功===****
KINGBASE研究院
