KingbaseES V8R6集群运维案例之---同一主机节点部署多个集群
案例说明：
在同一主机环境，由于生产需要，需要部署两个集群；本案例详细描述了两个集群的部署过程。
注意：同一主机部署多个集群需要先部署securecmdd服务，节点之间通过securecmdd服务通讯。多个集群共享主机的securecmdd服务在节点之间通讯。
适用版本：
KingbaseES V8R6
一、检测主机节点securecmdd服务
1、查看securecmdd服务及端口
[kingbase@node101 bin]$ ps -ef |grep securecmd
root 15486 1 0 14:34 ? 00:00:00 sys_securecmdd: /home/kingbase/cluster/securecmdd/bin/sys_securecmdd -f /etc/.kes/securecmdd_config [listener] 0 of 128-256 startups
[kingbase@node101 bin]$ netstat -antlp|grep 8890
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 0.0.0.0:8890 0.0.0.0:* LISTEN -
tcp6 0 0 :::8890 :::* LISTEN -
2、测试securecmd连接（所有节点）
# 本机
[kingbase@node101 bin]$ ./sys_securecmd kingbase@127.0.0.1 'whoami'
kingbase
[kingbase@node101 bin]$ ./sys_securecmd root@127.0.0.1 'whoami'
root
# 远程
[kingbase@node101 bin]$ ./sys_securecmd kingbase@192.168.1.102 'whoami'
kingbase
[kingbase@node101 bin]$ ./sys_securecmd root@192.168.1.102 'whoami'
root
二、第一个集群部署
1、集群部署（参考官方文档，可以采用部署工具或脚本部署）
https://help.kingbase.com.cn/v8/install-updata/k-deploy/index.html
2、查看第一个集群节点状态
[kingbase@node101 bin]$ ./repmgr cluster show
ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string
----+-------+---------+-----------+----------+----------+----------+----------+---------+---------------------------------------------------------------------------------------------------------------------------------------------------
1 | node1 | primary | * running | | default | 100 | 6 | | host=192.168.1.101 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2 | node2 | standby | running | node1 | default | 100 | 6 | 0 bytes | host=192.168.1.102 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
3、第一个集群配置
[kingbase@node101 bin]$ cat ../etc/repmgr.conf
use_scmd=on
ha_running_mode='DG'
node_id=1
node_name='node1'
conninfo='host=192.168.1.101 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3'
connection_check_type='mix'
data_directory='/data/kingbase/hac7/data'
#data_directory='/home/kingbase/cluster/R6HA/ha7/kingbase/data'
log_file='/home/kingbase/cluster/R6HA/ha7/kingbase/kingbase/log/hamgr.log'
kbha_log_file='/home/kingbase/cluster/R6HA/ha7/kingbase/kingbase/log/kbha.log'
sys_bindir='/home/kingbase/cluster/R6HA/ha7/kingbase/kingbase/bin'
scmd_options='-q -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ServerAliveInterval=2 -o ServerAliveCountMax=5 -p 8890'
trusted_servers='192.168.1.1'
running_under_failure_trusted_servers='on'
repmgrd_pid_file='/home/kingbase/cluster/R6HA/ha7/kingbase/kingbase/etc/hamgrd.pid'
kbha_pid_file='/home/kingbase/cluster/R6HA/ha7/kingbase/kingbase/etc/kbha.pid'
......
三、第二个集群部署（脚本部署）
1、脚本部署所需文件
[kingbase@node102 r6_install]$ ls -l
total 234236
-rw-rw-r-- 1 kingbase kingbase 237558136 Apr 7 14:07 db.zip
-rw-rw-r-- 1 kingbase kingbase 12397 Apr 19 16:09 install.conf
-rw-r--r-- 1 kingbase kingbase 3454 Apr 7 14:08 license.dat
-rw-rw-r-- 1 kingbase kingbase 2114981 Apr 19 15:43 securecmdd.zip
-rw-rw-r-- 1 kingbase kingbase 3902 Apr 7 14:07 trust_cluster.sh
-rw-rw-r-- 1 kingbase kingbase 152215 Apr 7 14:07 V8R6_cluster_install.sh
2、创建集群部署目录（两个集群分别在不同目录下）
[kingbase@node102 r6_install]$ mkdir -p /home/kingbase/cluster/R6HA/hac7/kingbase/
3、配置部署配置文件
[kingbase@node102 r6_install]$ cat install.conf|grep -v ^$|grep -v ^#
[install]
on_bmj=0
all_ip=(192.168.1.102 192.168.1.101)
witness_ip=""
production_ip=()
local_disaster_recovery_ip=()
remote_disaster_recovery_ip=()
install_dir="/home/kingbase/cluster/R6HA/hac7"
zip_package="/home/kingbase/r6_install/db.zip"
license_file=(license.dat)
db_user="system" # the user name of database
db_port="54325" # the port of database, defaults is 54321
db_mode="oracle" # database mode: pg, oracle
db_auth="scram-sha-256" # database authority: scram-sha-256, md5, default is scram-sha-256
db_case_sensitive="no" # database case sensitive settings: yes, no. default is yes - case sensitive; no - case insensitive (NOTE. cannot set to 'no' when db_mode="pg").
db_checksums="yes" # the checksum for data: yes, no. default is yes - a checksum is calculated for each data block to prevent corruption; no - nothing to do.
archive_mode="on" # enables archiving; off, on, or always
db_encoding="" # Cararcter set encoding to use in the new database.Specify a tring constant,or an integer encoding number, default value provided by locale command.
db_collate="" # Collation order(LC_COLLATE) to use in the new database,This affects the sort order applied to strings, default value provided by locale command.
db_ctype="" # Character classification(LC_CTYPE) to use int the new database. This affects the categorization of characters, default value provided by locale command.
other_db_init_options="" # addional initdb options,such as "--scenario-tuning"
trusted_servers="192.168.1.1"
running_under_failure_trusted_servers='on'
data_directory="/home/kingbase/cluster/R6HA/hac7/kingbase/data"
waldir=''
virtual_ip=""
net_device=(enp0s3 enp0s3)
net_device_ip=(192.168.1.102 192.168.1.101)
ipaddr_path="/sbin"
arping_path=""
ping_path="/bin"
super_user="root"
execute_user="kingbase"
deploy_by_sshd=0 # choose whether to use sshd when deploy, 0 means not to use (deploy by sys_securecmdd), 1 means to use (deploy by sshd), default value is 1; when on_bmj=1, it will auto set to no(deploy_by_sshd=0)
use_scmd=1 # Is the cluster running on sys_securecmdd or sshd? 1 means yes (on sys_securecmdd), 0 means no (on sshd), default value is 1; when on_bmj=1, it will auto set to yes(use_scmd=1)
reconnect_attempts="10" # the number of retries in the event of an error
reconnect_interval="6" # retry interval
recovery="standby" # the way of cluster recovery: standby/automatic/manual
ssh_port="22" # the port of ssh, default is 22
scmd_port="8890" # the port of sys_securecmdd, default is 8890
auto_cluster_recovery_level='1'
use_check_disk='off'
synchronous=''
sync_in_same_location=0
failover_need_server_alive='off'
4、拷贝db.zip文件到集群安装目录下并解压（所有节点）
1）拷贝db.zip到所有节点
[kingbase@node102 r6_install]$ cp db.zip /home/kingbase/cluster/R6HA/hac7/kingbase
[kingbase@node102 r6_install]$ scp db.zip node101:/home/kingbase/cluster/R6HA/hac7/kingbase
db.zip 100% 227MB 88.2MB/s 00:02
2）解压db.zip
[kingbase@node101 kingbase]$ unzip db.zip
[kingbase@node101 kingbase]$ ls -lh
total 227M
drwxr-xr-x 2 kingbase kingbase 4.0K Oct 29 14:57 bin
-rw-rw-r-- 1 kingbase kingbase 227M Apr 19 16:07 db.zip
drwxrwxr-x 5 kingbase kingbase 8.0K Oct 29 14:57 lib
drwxrwxr-x 7 kingbase kingbase 4.0K Oct 29 14:57 share
5、拷贝license.dat文件到集群bin目录下
[kingbase@node102 r6_install]$ cp license.dat /home/kingbase/cluster/R6HA/hac7/kingbase/bin/
[kingbase@node102 r6_install]$ scp license.dat node101:/home/kingbase/cluster/R6HA/hac7/kingbase/bin/
license.dat 100% 3454 3.3MB/s 00:00
6、执行脚本部署
[kingbase@node102 r6_install]$ sh V8R6_cluster_install.sh
........
2023-04-19 16:14:25 repmgrd on "[192.168.1.101]" start success.
ID | Name | Role | Status | Upstream | repmgrd | PID | Paused? | Upstream last seen
----+-------+---------+-----------+----------+---------+-------+---------+--------------------
1 | node1 | primary | * running | | running | 32636 | no | n/a
2 | node2 | standby | running | node1 | running | 4895 | no | 1 second(s) ago
[2023-04-19 16:14:26] [NOTICE] redirecting logging output to "/home/kingbase/cluster/R6HA/hac7/kingbase/log/kbha.log"
[2023-04-19 16:14:28] [NOTICE] redirecting logging output to "/home/kingbase/cluster/R6HA/hac7/kingbase/log/kbha.log"
2023-04-19 16:14:28 Done.
[INSTALL] start up the whole cluster ... OK
---如上所示，第二个集群部署完成。
7、第二个集群配置
[kingbase@node102 bin]$ cat ../etc/repmgr.conf
use_scmd=on
ha_running_mode='DG'
node_id=1
node_name='node1'
conninfo='host=192.168.1.102 user=esrep dbname=esrep port=54325 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3'
connection_check_type='mix'
data_directory='/home/kingbase/cluster/R6HA/hac7/kingbase/data'
log_file='/home/kingbase/cluster/R6HA/hac7/kingbase/log/hamgr.log'
kbha_log_file='/home/kingbase/cluster/R6HA/hac7/kingbase/log/kbha.log'
sys_bindir='/home/kingbase/cluster/R6HA/hac7/kingbase/bin'
scmd_options='-q -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ServerAliveInterval=2 -o ServerAliveCountMax=5 -p 8890'
trusted_servers='192.168.1.1'
running_under_failure_trusted_servers='on'
repmgrd_pid_file='/home/kingbase/cluster/R6HA/hac7/kingbase/etc/hamgrd.pid'
kbha_pid_file='/home/kingbase/cluster/R6HA/hac7/kingbase/etc/kbha.pid'
......
四、集群验证
1、第一个集群节点状态
[kingbase@node102 bin]$ pwd
/home/kingbase/cluster/R6HA/hac7/kingbase/bin
[kingbase@node102 bin]$ ./repmgr cluster show
ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string
----+-------+---------+-----------+----------+----------+----------+----------+---------+---------------------------------------------------------------------------------------------------------------------------------------------------
1 | node1 | primary | * running | | default | 100 | 1 | | host=192.168.1.102 user=esrep dbname=esrep port=54325 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2 | node2 | standby | running | node1 | default | 100 | 1 | 0 bytes | host=192.168.1.101 user=esrep dbname=esrep port=54325 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2、第二个集群节点状态
[kingbase@node101 bin]$ pwd
/home/kingbase/cluster/R6HA/ha7/kingbase/kingbase/bin
[kingbase@node101 bin]$ ./repmgr cluster show
ID | Name | Role | Status | Upstream | Location | Priority | Timeline | LSN_Lag | Connection string
----+-------+---------+-----------+----------+----------+----------+----------+---------+---------------------------------------------------------------------------------------------------------------------------------------------------
1 | node1 | primary | * running | | default | 100 | 6 | | host=192.168.1.101 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
2 | node2 | standby | running | node1 | default | 100 | 6 | 0 bytes | host=192.168.1.102 user=esrep dbname=esrep port=54321 connect_timeout=10 keepalives=1 keepalives_idle=10 keepalives_interval=1 keepalives_count=3
五、总结
在通过脚本部署第二个集群时，需要先创建好集群安装目录，并将db.zip文件拷贝到集群安装目录下，解压；再将license.dat文件也要拷贝到集群安装目录下，然后执行脚本部署。对于需要vip的集群环境，不同的集群配置不同的vip地址。多个集群的数据库服务同时启动，需要配置不同的数据库服务端口。
KINGBASE研究院
分类:
高可用 R6
【推荐】国内首个AI IDE，深度理解中文开发场景，立即下载体验Trae
【推荐】编程新体验，更懂你的AI，立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包，你的智能百科全书，全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell：AI 加持，快人一步
· 全程不用写代码，我用AI程序员写了一个飞机大战
· MongoDB 8.0这个新功能碉堡了，比商业数据库还牛
· 记一次.NET内存居高不下排查解决与启示
· 白话解读 Dapr 1.15：你的「微服务管家」又秀新绝活了
· DeepSeek 开源周回顾「GitHub 热点速览」