使用obd通过配置文件部署多节点集群
1 环境介绍
- 操作系统：CentOS 7.6
- 可用内存不低于8G
- 每个虚拟机4个CPU
| 主机IP | 角色 |
| 192.168.1.71 | observer,obproxy,obclient,obd |
| 192.168.1.72 | observer |
| 192.168.1.73 | observer |
注释：
- observer：oceanbase数据库
- obproxy：oceanbase反向代理
- obclient：oceanbase客户端
- obd：oceanbase deploy
2 准备工作
2.1 创建用户
[root]# useradd admin
[root]# echo "Mypasswd@2023" | passwd admin --stdin2.2 主机之间配置ssh互信
[root]# cat>>/etc/hosts<EOF
192.168.1.71 host1
192.168.1.72 host2
192.168.1.73 host3
EOF
[root]# echo "admin ALL=(ALL)NOPASSWD:ALL" >>/etc/sudoers
[admin]# ssh-keygen
[admin]# ssh-copy-id host1
[admin]# ssh-copy-id host2
[admin]# ssh-copy-id host32.3 修改内核参数
[root]# vim /etc/sysctl.conf
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.ip_local_port_range = 3500 65535
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_slow_start_after_idle=0
vm.swappiness = 0
vm.min_free_kbytes = 2097152
vm.max_map_count=655360
fs.aio-max-nr=1048576
[root]# sysctl -p2.4 修改会话变量
只修改当前会话
[root]# ulimit -c unlimited修改全局会话变量，已经登录的会话需要重连
[root]# vi /etc/security/limits.conf
* soft nofile 655360
* hard nofile 655360
* soft nproc 655360
* hard nproc 655360
* soft core unlimited
* hard core unlimited
* soft stack unlimited
* hard stack unlimited查看会话变量
[root]# ulimit -a2.5 关闭防火墙
[root]# systemctl stop firewalld
[root]# systemctl disable firewalld2.6 关闭selinux
[root]# vi /etc/selinux/config
SELINUX=disabled
[root]# setenforce 02.7配置时钟同步
安装chrony
[root]# yum install -y chrony配置与阿里时钟同步
[root]# vi /etc/chrony.conf
server ntp.aliyun.com minpoll 4 maxpoll 10 iburst
allow 192.168.1.0/16
local stratum 10重启chronyd
[root]# systemctl restart chronyd查看时间同步活动
[root]# chronyc activity查看时间服务器
[root]# chronyc sources查看同步状态
[root]# chronyc sources -v校准时间服务器：
[root]# chronyc tracking3 部署集群
3.1 安装obd
在192.168.1.71上安装obd（ob-deploy）
[admin]# sudo rpm -ivh ob-deploy-2.0.0-10.el7.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:ob-deploy-2.0.0-10.el7 ################################# [100%]
Installation of obd finished successfully
Please source /etc/profile.d/obd.sh to enable it
[admin]# source /etc/profile.d/obd.sh3.2 编辑OBD配置文件
参考配置文件（observer、obproxy）：https://github.com/oceanbase/obdeploy/blob/master/example/mini-distributed-with-obproxy-example.yaml
修改配置文件（3副本observer，1个obproxy）
[admin] vi mini-distributed-with-obproxy-example.yaml
oceanbase-ce:
servers:
- name: server1
ip: 192.168.1.71
- name: server2
ip: 192.168.1.72
- name: server3
ip: 192.168.1.73
global:
devname: ens33
cluster_id: 1
memory_limit: 6G
system_memory: 1G
datafile_size: 20G
log_disk_size: 15G
cpu_count: 4
production_mode: false
enable_syslog_wf: false
enable_syslog_recycle: true
max_syslog_file_count: 4
appname: obcluster
root_password: OceanBase010
proxyro_password: OceanBase010
server1:
mysql_port: 2881
rpc_port: 2882
home_path: /home/admin/observer
zone: zone1
server2:
mysql_port: 2881
rpc_port: 2882
home_path: /home/admin/observer
zone: zone2
server3:
mysql_port: 2881
rpc_port: 2882
home_path: /home/admin/observer
zone: zone3
obproxy-ce:
depends:
- oceanbase-ce
servers:
- 192.168.1.71
global:
listen_port: 2883
prometheus_listen_port: 2884
home_path: /home/admin/obproxy
enable_cluster_checkout: false
skip_proxy_sys_private_check: true
enable_strict_kernel_release: false3.3 部署集群
部署命令：obd cluster deploy [集群名] -c [集群配置文件]
[admin]# obd cluster deploy obce-3zones -c mini-distributed-with-obproxy-example.yaml
+--------------------------------------------------------------------------------------------+
| Packages |
+--------------+---------+------------------------+------------------------------------------+
| Repository | Version | Release | Md5 |
+--------------+---------+------------------------+------------------------------------------+
| oceanbase-ce | 4.1.0.0 | 100000202023040520.el7 | d598937b1cfb1df85e2c2231acf024e4994db533 |
| obproxy-ce | 4.1.0.0 | 7.el7 | 2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe |
+--------------+---------+------------------------+------------------------------------------+
Repository integrity check ok
Parameter check ok
Open ssh connection ok
Cluster status check ok
Initializes observer work home ok
Initializes obproxy work home ok
Remote oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 repository install ok
Remote oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 repository lib check !!
Remote obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe repository install ok
Remote obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe repository lib check ok
Try to get lib-repository
Download oceanbase-ce-libs-4.1.0.0-100000202023040520.el7.x86_64.rpm (155.18 K): 99% [######################################################################################################################################### ] ETA: 0:00:00 291.68 kB/s
Package oceanbase-ce-libs-4.1.0.0-100000202023040520.el7 is available.
install oceanbase-ce-libs-4.1.0.0 for local ok
Remote oceanbase-ce-libs-4.1.0.0-100000202023040520.el7-5d24535db655b4dce6fc62aedc4d0e867225792a repository install ok
Remote oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 repository lib check ok
obce-3zones deployed
Trace ID: 5b7350f8-df50-11ed-8eca-000c29501e19
If you want to view detailed obd logs, please run: obd display-trace 5b7350f8-df50-11ed-8eca-000c29501e19部署过程中涉及到：初始化相关目录、拷贝安装包到对应主机上、安装组件包。
查看集群状态，此时还没启动observer和初始化集群，status值为“deployed”
[admin]# obd cluster list
+----------------------------------------------------------------------+
| Cluster List |
+-------------+--------------------------------------+-----------------+
| Name | Configuration Path | Status (Cached) |
+-------------+--------------------------------------+-----------------+
| obce-3zones | /home/admin/.obd/cluster/obce-3zones | deployed |
+-------------+--------------------------------------+-----------------+
Trace ID: d31bf24a-df50-11ed-8f2d-000c29501e19
If you want to view detailed obd logs, please run: obd display-trace d31bf24a-df50-11ed-8f2d-000c29501e19查看集群组件信息，此时还看不到集群组件信息
[admin]# obd cluster display obce-3zones
Deploy "obce-3zones" is deployed
See https://www.oceanbase.com/product/ob-deployer/error-codes .
Trace ID: f214dd10-df50-11ed-b05c-000c29501e19
If you want to view detailed obd logs, please run: obd display-trace f214dd10-df50-11ed-b05c-000c29501e193. 4 启动和初始化集群
集群部署做了如下工作：初始化目录、安装组件软件。
启动obce-3zones集群
[admin]# obd cluster start obce-3zones
Get local repositories ok
Search plugins ok
Open ssh connection ok
Load cluster param plugin ok
Check before start observer ok
[WARN] OBD-1012: (192.168.1.71) clog and data use the same disk (/data)
[WARN] OBD-1012: (192.168.1.72) clog and data use the same disk (/data)
[WARN] OBD-1012: (192.168.1.73) clog and data use the same disk (/data)
Check before start obproxy ok
Start observer ok
observer program health check ok
Connect to observer ok
Initialize oceanbase-ce ok
Start obproxy ok
obproxy program health check ok
Connect to obproxy ok
Initialize obproxy-ce ok
Wait for observer init ok
+------------------------------------------------+
| observer |
+--------------+---------+------+-------+--------+
| ip | version | port | zone | status |
+--------------+---------+------+-------+--------+
| 192.168.1.71 | 4.1.0.0 | 2881 | zone1 | ACTIVE |
| 192.168.1.72 | 4.1.0.0 | 2881 | zone2 | ACTIVE |
| 192.168.1.73 | 4.1.0.0 | 2881 | zone3 | ACTIVE |
+--------------+---------+------+-------+--------+
obclient -h192.168.1.71 -P2881 -uroot -Doceanbase -A
+------------------------------------------------+
| obproxy |
+--------------+------+-----------------+--------+
| ip | port | prometheus_port | status |
+--------------+------+-----------------+--------+
| 192.168.1.71 | 2883 | 2884 | active |
+--------------+------+-----------------+--------+
obclient -h192.168.1.71 -P2883 -uroot -Doceanbase -A
obce-3zones running
Trace ID: d3d7c64a-df5b-11ed-ba1b-000c29501e19
If you want to view detailed obd logs, please run: obd display-trace d3d7c64a-df5b-11ed-ba1b-000c29501e19
查看集群状态，此时status等于“running”
[admin]# obd cluster list
+----------------------------------------------------------------------+
| Cluster List |
+-------------+--------------------------------------+-----------------+
| Name | Configuration Path | Status (Cached) |
+-------------+--------------------------------------+-----------------+
| obce-3zones | /home/admin/.obd/cluster/obce-3zones | running |
+-------------+--------------------------------------+-----------------+
Trace ID: 7840993c-df5c-11ed-a63a-000c29501e19
If you want to view detailed obd logs, please run: obd display-trace 7840993c-df5c-11ed-a63a-000c29501e19
查看集群组件信息
[admin]# obd cluster display obce-3zones
Get local repositories and plugins ok
Open ssh connection ok
Cluster status check ok
Connect to observer ok
Wait for observer init ok
+------------------------------------------------+
| observer |
+--------------+---------+------+-------+--------+
| ip | version | port | zone | status |
+--------------+---------+------+-------+--------+
| 192.168.1.71 | 4.1.0.0 | 2881 | zone1 | ACTIVE |
| 192.168.1.72 | 4.1.0.0 | 2881 | zone2 | ACTIVE |
| 192.168.1.73 | 4.1.0.0 | 2881 | zone3 | ACTIVE |
+--------------+---------+------+-------+--------+
obclient -h192.168.1.71 -P2881 -uroot -Doceanbase -A
Connect to obproxy ok
+------------------------------------------------+
| obproxy |
+--------------+------+-----------------+--------+
| ip | port | prometheus_port | status |
+--------------+------+-----------------+--------+
| 192.168.1.71 | 2883 | 2884 | active |
+--------------+------+-----------------+--------+
obclient -h192.168.1.71 -P2883 -uroot -Doceanbase -A
Trace ID: 969d6e0a-df5c-11ed-aefc-000c29501e19
If you want to view detailed obd logs, please run: obd display-trace 969d6e0a-df5c-11ed-aefc-000c29501e19
安装obclient
[admin]# sudo rpm -ivh obclient-2.2.2-1.el7.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:obclient-2.2.2-1.el7 ################################# [100%]
连接oceanbase，默认密码为空
[admin]# obclient -h192.168.1.71 -P2881 -uroot -Doceanbase -A -p
Enter password:
Welcome to the OceanBase. Commands end with ; or \g.
Your OceanBase connection id is 3221524400
Server version: OceanBase_CE 4.1.0.0 (r100000202023040520-0765e69043c31bf86e83b5d618db0530cf31b707) (Built Apr 5 2023 20:26:14)
Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
obclient [oceanbase]>
查看各zone状态
obclient [oceanbase]> select zone,svr_ip,inner_port,svr_port,status from __all_server;
+-------+--------------+------------+----------+--------+
| zone | svr_ip | inner_port | svr_port | status |
+-------+--------------+------------+----------+--------+
| zone1 | 192.168.1.71 | 2881 | 2882 | ACTIVE |
| zone2 | 192.168.1.72 | 2881 | 2882 | ACTIVE |
| zone3 | 192.168.1.73 | 2881 | 2882 | ACTIVE |
+-------+--------------+------------+----------+--------+
参考：《oceanbase社区版入门与实战.pdf》
