KingbaseES R6 集群通过备库clone在线添加新节点
案例说明：
KingbaseES R6集群可以通过图形化方式在线添加新节点，但是在添加新节点clone环节时，是从主库copy数据到新的节点，这样在生产环境，如果数据量大，将会对主库的网络I/O造成压力。可以通过‘repmgr standby clone’指定从已有的备库进行克隆，减轻主库压力。
数据库版本：
test=# select version();
version
----------------------------------------------------------------------------------------------------------------------
KingbaseES V008R006C005B0023 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
(1 row)
集群架构：
[kingbase@node1 bin]$ cat /etc/hosts
......
192.168.7.248 node1 #主库节点
192.168.7.249 node2 #新增备库节点
192.168.7.243 node3 #备库节点
[kingbase@node1 bin]$ ./repmgr cluster show
ID | Name | Role | Status | Upstream | Location | Priority | Timeline | Connection string
----+---------+---------+-----------+----------+----------+----------+----------+---------------
1 | node243 | standby | running | node248 | default | 100 | 12 | host=192.168.7.243 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2 | node248 | primary | * running | | default | 100 | 12 | host=192.168.7.248 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
一、新节点准备工作
=新节点的系统配置的准备，可以参考集群部署的要求。建立和集群其他节点相同的集群管理路径，并从其他节点上传相关的目录和文件（除了data目录）到新节点=
[kingbase@node2 R6C5R]$ pwd
/home/kingbase/cluster/R6C5/R6C5R
[kingbase@node2 R6C5R]$ ls -lh
total 145M
-rw-rw-r--. 1 kingbase kingbase 62 Mar 22 13:24 control.so
-rwxr-xr-x. 1 kingbase root 145M Mar 22 13:20 db.zip
drwxrwxr-x. 9 kingbase kingbase 99 Mar 22 13:25 kingbase
-rwxrwxrwx. 1 kingbase kingbase 3.4K Mar 22 13:24 license.dat
[kingbase@node2 R6C5R]$ cd kingbase/
[kingbase@node2 kingbase]$ ls -lh
total 32K
drwxrwxr-x. 2 kingbase kingbase 6 Mar 22 13:24 archive
drwxr-xr-x. 2 kingbase kingbase 4.0K Mar 22 13:24 bin
-rw-------. 1 kingbase kingbase 8.0K Mar 22 16:25 copy_file
drwxrwxr-x. 2 kingbase kingbase 101 Mar 22 13:25 etc
drwxrwxr-x. 5 kingbase kingbase 8.0K Nov 5 17:20 lib
drwxr-xr-x. 2 kingbase kingbase 37 Mar 22 13:25 log
drwxrwxr-x. 8 kingbase kingbase 4.0K Mar 22 13:25 share
二、在新节点执行在线clone
1、首先在主库上建立新备库的replication slot
test=# select sys_create_physical_replication_slot('repmgr_slot_3');
sys_create_physical_replication_slot
--------------------------------------
(repmgr_slot_3,)
(1 row)
test=# select * from sys_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn
---------------+--------+-----------+--------+----------+-----------+--------+------------+-------+--------------+-------------+---------------------
repmgr_slot_1 | | physical | | | f | t | 8229 | 68604 | | 0/44004000 |
repmgr_slot_3 | | physical | | | f | f | | | | 0/44003100 |
(2 rows)
2、在新节点执行clone预演
[kingbase@node2 bin]$ ./repmgr standby clone -h 192.168.7.243 -U esrep -d esrep --dry-run
NOTICE: destination directory "/home/kingbase/cluster/R6C5/R6C5R/kingbase/data" provided
INFO: connecting to source node
DETAIL: connection string is: host=192.168.7.243 user=esrep dbname=esrep
DETAIL: current installation size is 108 MB
INFO: "repmgr" extension is installed in database "esrep"
INFO: parameter "max_replication_slots" set to 32
INFO: parameter "max_wal_senders" set to 32
NOTICE: checking for available walsenders on the source node (2 required)
INFO: sufficient walsenders available on the source node
DETAIL: 2 required, 32 available
NOTICE: checking replication connections can be made to the source server (2 required)
INFO: required number of replication connections could be made to the source server
DETAIL: 2 replication connections required
NOTICE: standby will attach to upstream node 2
HINT: consider using the -c/--fast-checkpoint option
INFO: all prerequisites for "standby clone" are met
3、在新节点执行clone
[kingbase@node2 bin]$ ./repmgr standby clone -h 192.168.7.243 -U esrep -d esrep
# 注：-h ，指定已有的备库的ip地址。
NOTICE: destination directory "/home/kingbase/cluster/R6C5/R6C5R/kingbase/data" provided
INFO: connecting to source node
DETAIL: connection string is: host=192.168.7.243 user=esrep dbname=esrep
DETAIL: current installation size is 108 MB
NOTICE: checking for available walsenders on the source node (2 required)
NOTICE: checking replication connections can be made to the source server (2 required)
INFO: creating directory "/home/kingbase/cluster/R6C5/R6C5R/kingbase/data"...
NOTICE: starting backup (using sys_basebackup)...
HINT: this may take some time; consider using the -c/--fast-checkpoint option
INFO: executing:
/home/kingbase/cluster/R6C5/R6C5R/kingbase/bin/sys_basebackup -l "repmgr base backup" -D /home/kingbase/cluster/R6C5/R6C5R/kingbase/data -h 192.168.7.243 -p 54321 -U esrep -X stream -S repmgr_slot_3
NOTICE: standby clone (using sys_basebackup) complete
NOTICE: you can now start your Kingbase server
HINT: for example: sys_ctl -D /home/kingbase/cluster/R6C5/R6C5R/kingbase/data start
HINT: after starting the server, you need to register this standby with "repmgr standby register"
4、启动新节点数据库服务
[kingbase@node2 bin]$ ./sys_ctl -D /home/kingbase/cluster/R6C5/R6C5R/kingbase/data start
waiting for server to start....2022-03-22 12:14:15.746 CST [3450] LOG: sysaudit extension initialized
.......
server started
5、注册新备库到集群
[kingbase@node2 bin]$ ./repmgr standby register --force
INFO: connecting to local node "node249" (ID: 3)
INFO: connecting to primary database
WARNING: --upstream-node-id not supplied, assuming upstream node is primary (node ID 2)
INFO: standby registration complete
NOTICE: standby node "node249" (ID: 3) successfully registered
6、查看集群节点状态
[kingbase@node2 bin]$ ./repmgr cluster show
ID | Name | Role | Status | Upstream | Location | Priority | Timeline | Connection string
----+---------+---------+-----------+----------+----------+----------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------
1 | node243 | standby | running | node248 | default | 100 | 12 | host=192.168.7.243 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2 | node248 | primary | * running | | default | 100 | 12 | host=192.168.7.248 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
3 | node249 | standby | running | node248 | default | 100 | 12 | host=192.168.7.249 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
=由以上获知，新的备库节点添加成功。=
三、备库clone错误案例
=在从指定备库clone前，如果未创建replication slot 出现以下错误=
错误日志：
[kingbase@node2 bin]$ ./repmgr standby clone -h 192.168.7.243 -U esrep -d esrep
NOTICE: destination directory "/home/kingbase/cluster/R6C5/R6C5R/kingbase/data" provided
INFO: connecting to source node
DETAIL: connection string is: host=192.168.7.243 user=esrep dbname=esrep
DETAIL: current installation size is 108 MB
NOTICE: checking for available walsenders on the source node (2 required)
NOTICE: checking replication connections can be made to the source server (2 required)
INFO: creating directory "/home/kingbase/cluster/R6C5/R6C5R/kingbase/data"...
NOTICE: starting backup (using sys_basebackup)...
HINT: this may take some time; consider using the -c/--fast-checkpoint option
INFO: executing:
/home/kingbase/cluster/R6C5/R6C5R/kingbase/bin/sys_basebackup -l "repmgr base backup" -D /home/kingbase/cluster/R6C5/R6C5R/kingbase/data -h 192.168.7.243 -p 54321 -U esrep -X stream -S repmgr_slot_3
NOTICE: creating replication slot "repmgr_slot_3" on upstream node 2
ERROR: drop_replication_slot(): unable to drop replication slot "repmgr_slot_3"
DETAIL:
ERROR: replication slot "repmgr_slot_3" does not exist
DETAIL: query text is:
SELECT pg_catalog.pg_drop_replication_slot('repmgr_slot_3')
ERROR: unable to delete replication slot "repmgr_slot_3" on source node
NOTICE: standby clone (using sys_basebackup) complete
NOTICE: you can now start your Kingbase server
HINT: for example: sys_ctl -D /home/kingbase/cluster/R6C5/R6C5R/kingbase/data start
HINT: after starting the server, you need to register this standby with "repmgr standby register"
KINGBASE研究院
