Oceanbase-01-安装初体验-v3.1.2

安装开始

环境准备

服务器规划

角色	机器	用途
OBD、OBCLIENT、OBPROXY	10.51.xx.65	中控机、部署机ob代理、obclient
OBSERVER	10.51.xx.66	zone1
	10.51.xx.67	zone2
	10.51.xx.68	zone3

在高可用里面应该考虑将 obproxy部署多个节点，obproxy应该是一个无状态的服务，多节点保持高可用冗余。

在测试安装时其实用最少3台机器也可以部署。OBD、OBCLIENT与OBserver放在一起，OBPROXY可以部署三个节点，我这里测试环境就没有这个设计了。

服务器准备

[root@dbdriver ~]# cat /etc/hosts
10.51.xx.65   dbdriver.bj.bob.test    dbdriver
10.51.xx.66   db01.bj.bob.test        db01
10.51.xx.67   db02.bj.bob.test        db02
10.51.xx.68   db03.bj.bob.test        db03

关闭透明大页

vi /usr/lib/tuned/throughput-performance/tuned.conf
[vm]
transparent_hugepages=never

或者写入到引导启动中
grubby --info /boot/vmlinuz-3.10.0-957.el7.x86_64
grubby --args="transparent_hugepage=never" --update-kernel /boot/vmlinuz-3.10.0-957.el7.x86_64
grubby --info /boot/vmlinuz-3.10.0-957.el7.x86_64

配置中控机ssh互信

[root@dbdriver ~]# ssh-keygen -t rsa
[root@dbdriver ~]# cd .ssh
[root@dbdriver .ssh]# cat id_rsa.pub > authorized_keys
[root@db01 ~]# mkdir .ssh
[root@db01 ~]# cat /tmp/id_rsa.pub >> ~/.ssh/authorized_keys
[root@db02 ~]# mkdir .ssh
[root@db02 ~]# cat /tmp/id_rsa.pub >> ~/.ssh/authorized_keys
[root@db03 ~]# mkdir .ssh
[root@db03 ~]# cat /tmp/id_rsa.pub >> ~/.ssh/authorized_keys

编辑远程命令

[root@dbdriver ~]# vi r_scp
#/usr/bin/bash
hosts=(
"db01"
"db02"
"db03"
"dbdriver"
)
for host in "${hosts[@]}"
do
echo "begin to scp " $@ " on " $host
scp -r $1 $host:$2
done

[root@dbdriver ~]# vi r_commad
#/usr/bin/bash
hosts=(
"db01"
"db02"
"db03"
"dbdriver"
)
for host in "${hosts[@]}"
do
echo "begin to run " $@ " on " $host
ssh $host $@
done

[root@dbdriver ~]# chmod +x r_scp
[root@dbdriver ~]# chmod +x r_commad

命令测试
[root@dbdriver ~]# ./r_commad date
begin to run  date  on  db01
Sat May  7 10:41:51 CST 2022
begin to run  date  on  db02
Sat May  7 10:41:51 CST 2022
begin to run  date  on  db03
Sat May  7 10:41:51 CST 2022
begin to run  date  on  dbdriver
Sat May  7 18:42:01 CST 2022

配置时钟同步

[root@dbdriver ~]# vi /etc/chrony.conf

server 0.rhel.pool.ntp.org iburst
server 1.rhel.pool.ntp.org iburst
server 2.rhel.pool.ntp.org iburst
server 3.rhel.pool.ntp.org iburst
server 10.51.xx.4 iburst  -->加入ntp服务器

[root@dbdriver ~]# systemctl start chronyd
[root@dbdriver ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; disabled; vendor preset: enabled)
   Active: active (running) since Mon 2022-05-09 16:13:35 CST; 4s ago
     Docs: man:chronyd(8)
           man:chrony.conf(5)
  Process: 150672 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS)
  Process: 150657 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 150667 (chronyd)
    Tasks: 1
   CGroup: /system.slice/chronyd.service
           └─150667 /usr/sbin/chronyd
		   
[root@dbdriver ~]# chronyc sources -v
210 Number of sources = 1

  .-- Source mode  '^' = server, '=' = peer, '#' = local clock.
 / .- Source state '*' = current synced, '+' = combined , '-' = not combined,
| /   '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
||                                                 .- xxxx [ yyyy ] +/- zzzz
||      Reachability register (octal) -.           |  xxxx = adjusted offset,
||      Log2(Polling interval) --.      |          |  yyyy = measured offset,
||                                \     |          |  zzzz = estimated error.
||                                 |    |           \

MS Name/IP address         Stratum Poll Reach LastRx Last sample

^* 10.51.xx.4                    2   6     7     1   -476ns[+28810s] +/-   12ms

依次配置所有服务器
[root@dbdriver ~]# ./r_scp /etc/chrony.conf  /etc
[root@dbdriver ~]# ./r_commad systemctl start chronyd
[root@dbdriver ~]# ./r_commad systemctl status chronyd
[root@dbdriver ~]# ./r_commad chronyc sources -v

规划磁盘

对于磁盘规划Oceanbase建议：
如果企业级应用至少挂三块磁盘，如果做了raid，可以做逻辑卷。
如果个人测试可以挂载一块磁盘，本次安装使用挂载一块盘。

数据盘

数据盘的配置参数为 data_dir, ${data_dir}/sstable 将自动创建
数据盘的大小由 datafile_disk_percentage参数决定,datafile_disk_percentage的默认值为 95.
data_dir 创建后无法调整大小。您可以通过加减机器为 OceanBase 数据库进行扩容和缩容。目前，OceanBase 数据库不支持单机的磁盘级扩容和缩容。

• 疑问一：仅能通过加减机器实现扩容？不能通过加减磁盘扩容吗？后面测试一下。

日志盘

事务日志盘的配置参数为 redo_dir 。建议您将事务日志盘的大小设置为 OceanBase 数据库内存的 3 倍到 4 倍及以上。事务日志盘包含多个固定大小的文件。这些文件位于安装目录 ${redo_dir}/{clog,ilog,slog} 下。您可以根据您的需要自动创建和清除事务日志。事务日志达到磁盘总量的 80% 时，将触发自动清除。但是，只有事务日志对应的内存数据已经合并融至基线数据中时，事务日志才能被删除。在相同数据量的情况下，事务日志的大小约为内存数据大小的三倍。因此事务日志盘所需空间上限与两次合并后的数据总量成正比。经验公式：事务日志文件大小 = 增量数据内存上限的 3 到 4 倍。

安装盘

OceanBase 数据库安装盘的配置参数为 home_path 。建议您为 OceanBase 数据库安装盘预留至少 200 G 空间，以保存 7 天及以上的日志。OceanBase 数据库的 RPM 包安装目录位于 {home_path} 下。其中，基线数据文件和事务日志文件会通过软链接分别指向独立的数据盘和事务日志盘。OceanBase 数据库的运行日志位于 ${home_path}/log 下。运行日志会不断增长，并且 OceanBase 数据库无法自动删除运行日志。因此您需要定时删除运行日志。

官方建议区分开，使用不同的磁盘，我这里测试就创建了一个父目录。

[root@dbdriver ~]# ./r_commad mkdir -p /oceanbase
[root@dbdriver ~]# mkfs.xfs -f /dev/nvme0n1
[root@dbdriver ~]# ./r_commad mkfs.xfs -f /dev/nvme0n1
[root@dbdriver ~]# ./r_commad mount /dev/nvme0n1 /oceanbase

配置 limits.conf

[root@dbdriver ~]# vi /etc/security/limits.conf
/etc/security/limits.conf
root soft nofile 655350
root hard nofile 655350
* soft nofile 655350
* hard nofile 655350
* soft stack 20480
* hard stack 20480
* soft nproc 655360
* hard nproc 655360
* soft core unlimited
* hard core unlimited
* 
[root@dbdriver ~]# ./r_scp /etc/security/limits.conf  /etc/security/

配置 limits.conf

[root@dbdriver ~]# vi /etc/sysctl.conf
# for oceanbase
## 修改内核异步 I/O 限制
fs.aio-max-nr=1048576
## 网络优化
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.ip_local_port_range = 3500 65535
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_slow_start_after_idle=0
vm.swappiness = 0
vm.min_free_kbytes = 2097152
# 此处为 OceanBase 数据库的 data 目录
kernel.core_pattern = /oceanbase/data/core-%e-%p-%t

拷贝到其他节点
[root@dbdriver ~]# ./r_scp /etc/sysctl.conf  /etc/

生效内核参数
[root@dbdriver ~]# ./r_commad sysctl -p

关闭防火墙和 SELinux

管理防火强
[root@dbdriver ~]# ./r_commad systemctl disable firewalld
[root@dbdriver ~]# ./r_commad systemctl stop firewalld
[root@dbdriver ~]# ./r_commad systemctl status firewalld

管理selinux
[root@dbdriver ~]# vi /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
#SELINUX=enforcing
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

拷贝到其他节点
[root@dbdriver ~]# ./r_scp /etc/selinux/config  /etc/selinux/

临时关闭
[root@dbdriver ~]# ./r_commad setenforce 0

创建OS用户

创建用户
./r_commad  useradd -U obdba -d /home/obdba -s /bin/bash
passwd 

配置sudo
[root@dbdriver ~]# vi /etc/sudoers
## Allow root to run any commands anywhere
root    ALL=(ALL)       ALL
admin   ALL=(ALL) NOPASSWD: ALL   #加入位置

[root@dbdriver ~]# ./r_scp /etc/sudoers /etc/

修改相关目录权限
[root@dbdriver ~]# ./r_commad mkdir -p /oceanbase/data
[root@dbdriver ~]# ./r_commad mkdir -p /oceanbase/redo
[root@dbdriver ~]# ./r_commad mkdir -p /oceanbase/home
[root@dbdriver ~]# ./r_commad chown -R obdba:obdba /oceanbase/data
[root@dbdriver ~]# ./r_commad chown -R obdba:obdba /oceanbase/redo
[root@dbdriver ~]# ./r_commad chown -R obdba:obdba /oceanbase/home

创建完obdba用户可以为obdba用户配置互信
参考上面root配置户型

安装数据库

使用一台机器部署 ODP。
使用三台机器部署 OceanBase 集群

下载所有安装介质

[root@dbdriver soft]# ls -l
total 138228
-rw-r--r--. 1 root root   658620 May 10 19:20 libobclient-2.0.0-2.el7.x86_64.rpm
-rw-r--r--. 1 root root 41916564 May 10 19:09 obclient-2.0.0-2.el7.x86_64.rpm
-rw-r--r--. 1 root root 41916564 May 10 19:08 ob-deploy-1.2.0-15.el7.x86_64.rpm
-rw-r--r--. 1 root root  8179432 May 10 19:09 obproxy-3.2.0-1.el7.x86_64.rpm
-rw-r--r--. 1 root root 48708456 May 10 19:09 oceanbase-ce-3.1.2-10000392021123010.el7.x86_64.rpm
-rw-r--r--. 1 root root   158948 May 10 19:09 oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm

在中控机器上安装ODP

[root@dbdriver soft]# yum install ob-deploy-1.2.0-15.el7.x86_64.rpm

在每台目标机器上安装 OBLibs

[root@dbdriver ~]# cd /oceanbase/soft/
[root@dbdriver soft]# yum install -y oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm
[root@db01 soft]# yum install -y oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm
[root@db02 soft]# yum install -y oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm
[root@db03 soft]# yum install -y oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm

在中控机器上安装 OBClient

[root@dbdriver soft]# yum install -y yum install libobclient-2.0.0-2.el7.x86_64.rpm
[root@dbdriver soft]# yum install -y libobclient-2.0.0-2.el7.x86_64.rpm

在中控机器上配置数据库源及安装配置数据

安装 OceanBase 数据库和 ODP之前，为了数据安全，建议您切换到非 root 用户，我这里使用了obdba用户

[obdba@dbdriver soft]$ obd mirror disable remote
Disable remote ok

[obdba@dbdriver soft]$ obd mirror clone *.rpm
name: libobclient
version: 2.0.0
release:2.el7
arch: x86_64
md5: f73cae67e2ff5be0682ac2803aba33a7ed26430e
add libobclient-2.0.0-2.el7.x86_64.rpm to local mirror
name: obclient
version: 2.0.0
release:2.el7
arch: x86_64
md5: 1d2c3ee31f40b9d2fbf97f653f549d896b7e7060
add obclient-2.0.0-2.el7.x86_64.rpm to local mirror
mirror obclient-2.0.0-2.el7.x86_64.rpm existed. Do you want to overwrite? [y/n]: y
name: obclient
version: 2.0.0
release:2.el7
arch: x86_64
md5: 1d2c3ee31f40b9d2fbf97f653f549d896b7e7060
add ob-deploy-1.2.0-15.el7.x86_64.rpm to local mirror
name: obproxy
version: 3.2.0
release:1.el7
arch: x86_64
md5: 8d5c6978f988935dc3da1dbec208914668dcf3b2
add obproxy-3.2.0-1.el7.x86_64.rpm to local mirror
name: obproxy-ce
version: 3.2.3
release:2.el7
arch: x86_64
md5: bdd299bda2bdf71fd0fd3f155b6a2e39dffd2be1
add obproxy-ce-3.2.3-2.el7.x86_64.rpm to local mirror
name: oceanbase-ce
version: 3.1.2
release:10000392021123010.el7
arch: x86_64
md5: 7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed
add oceanbase-ce-3.1.2-10000392021123010.el7.x86_64.rpm to local mirror
name: oceanbase-ce-libs
version: 3.1.2
release:10000392021123010.el7
arch: x86_64
md5: 94fff0ab31de053051dba66039e3185fa390cad5
add oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm to local mirror
[obdba@dbdriver soft]$  obd mirror list
+------------------------------------------------------------------+
|                      Mirror Repository List                      |
+----------------------------+--------+---------+------------------+
| SectionName                | Type   | Enabled | Update Time      |
+----------------------------+--------+---------+------------------+
| oceanbase.community.stable | remote | False   | 1970-01-01 08:00 |
| oceanbase.development-kit  | remote | False   | 1970-01-01 08:00 |
| local                      | local  | -       | 2022-05-13 14:19 |
+----------------------------+--------+---------+------------------+

编辑部署文件：

尤其是其中几处密码，必须设置，而且有几处必须一样，我干脆全都设置一样的了，否则使用proxy登录不上。

在观察日志发现proxy实际上会通过proxyro用户链接observer。

user:
   username: obdba
   password: obdba
oceanbase-ce:
  servers:
    - name: server1
      ip: 10.51.xx.66
    - name: server2
      ip: 10.51.xx.67
    - name: server3
      ip: 10.51.xx.68
  global:
    home_path: /oceanbase/home
    data_dir: /oceanbase/data
    redo_dir: /oceanbase/redo
    devname: enp59s0f1
    memory_limit: 64G
    appname: obtest
    root_password: welcome1
    proxyro_password: welcome1
  server1:
    zone: zone1
  server2:
    zone: zone2
  server3:
    zone: zone3
obproxy-ce:
  depends:
    - oceanbase-ce
  servers:
    - 10.51.xx.65
  global:
    home_path: /oceanbase/obproxy
    skip_proxy_sys_private_check: true
    enable_strict_kernel_release: false
    cluster_name: obtest
    obproxy_sys_password: welcome1
    observer_sys_password: welcome1

开始部署集群

[obdba@dbdriver soft]$ obd cluster autodeploy obtest -c oceanbase-temp.yaml
oceanbase-ce-3.1.2 already installed.
obproxy-ce-3.2.3 already installed.
Cluster param config check ok
Open ssh connection ok
Generate observer configuration ok
Generate obproxy configuration ok
oceanbase-ce-3.1.2 already installed.
obproxy-ce-3.2.3 already installed.
+-------------------------------------------------------------------------------------------+
|                                          Packages                                         |
+--------------+---------+-----------------------+------------------------------------------+
| Repository   | Version | Release               | Md5                                      |
+--------------+---------+-----------------------+------------------------------------------+
| oceanbase-ce | 3.1.2   | 10000392021123010.el7 | 7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed |
| obproxy-ce   | 3.2.3   | 2.el7                 | bdd299bda2bdf71fd0fd3f155b6a2e39dffd2be1 |
+--------------+---------+-----------------------+------------------------------------------+
Repository integrity check ok
Parameter check ok
Open ssh connection ok
Remote oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed repository install ok
Remote oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed repository lib check ok
Remote obproxy-ce-3.2.3-bdd299bda2bdf71fd0fd3f155b6a2e39dffd2be1 repository install ok
Remote obproxy-ce-3.2.3-bdd299bda2bdf71fd0fd3f155b6a2e39dffd2be1 repository lib check ok
Cluster status check ok
Initializes observer work home ok
Initializes obproxy work home ok
obtest deployed
Get local repositories and plugins ok
Open ssh connection ok
Load cluster param plugin ok
Check before start observer ok
[WARN] (10.51.xx.66) clog and data use the same disk (/oceanbase)
[WARN] (10.51.xx.67) clog and data use the same disk (/oceanbase)
[WARN] (10.51.xx.68) clog and data use the same disk (/oceanbase)  --> 这里说最好log和data不要放在一个磁盘，其实也是wal的机制要求把日志放到其他盘，不要一损俱损。

Check before start obproxy ok
Start observer ok
observer program health check ok
Connect to observer ok
Initialize cluster
Cluster bootstrap ok
Wait for observer init ok
+------------------------------------------------+
|                    observer                    |
+--------------+---------+------+-------+--------+
| ip           | version | port | zone  | status |
+--------------+---------+------+-------+--------+
| 10.51.xx.66 | 3.1.2   | 2881 | zone1 | active |
| 10.51.xx.67 | 3.1.2   | 2881 | zone2 | active |
| 10.51.xx.68 | 3.1.2   | 2881 | zone3 | active |
+--------------+---------+------+-------+--------+

Start obproxy ok
obproxy program health check ok
Connect to obproxy ok
Initialize cluster
+------------------------------------------------+
|                    obproxy                     |
+--------------+------+-----------------+--------+
| ip           | port | prometheus_port | status |
+--------------+------+-----------------+--------+
| 10.51.xx.65 | 2883 | 2884            | active |
+--------------+------+-----------------+--------+
obtest running

查看集群

[obdba@dbdriver soft]$ obd cluster list
+------------------------------------------------------------+
|                        Cluster List                        |
+--------+---------------------------------+-----------------+
| Name   | Configuration Path              | Status (Cached) |
+--------+---------------------------------+-----------------+
| obtest | /home/obdba/.obd/cluster/obtest | running         |
+--------+---------------------------------+-----------------+


[obdba@dbdriver soft]$ obd cluster display obtest
Get local repositories and plugins ok
Open ssh connection ok
Cluster status check ok
Connect to observer ok
Wait for observer init ok
+------------------------------------------------+
|                    observer                    |
+--------------+---------+------+-------+--------+
| ip           | version | port | zone  | status |
+--------------+---------+------+-------+--------+
| 10.51.xx.66 | 3.1.2   | 2881 | zone1 | active |
| 10.51.xx.67 | 3.1.2   | 2881 | zone2 | active |
| 10.51.xx.68 | 3.1.2   | 2881 | zone3 | active |
+--------------+---------+------+-------+--------+

Connect to obproxy ok
+------------------------------------------------+
|                    obproxy                     |
+--------------+------+-----------------+--------+
| ip           | port | prometheus_port | status |
+--------------+------+-----------------+--------+
| 10.51.xx.65 | 2883 | 2884            | active |
+--------------+------+-----------------+--------+

首次登陆

直接登录某一实例

[obdba@dbdriver soft]$ obclient -h10.51.xx.66 -P2881 -uroot@sys -p -c -A oceanbase
Enter password:
Welcome to the OceanBase.  Commands end with ; or \g.
Your MySQL connection id is 3221503614
Server version: 5.7.25 OceanBase 3.1.2 (r10000392021123010-d4ace121deae5b81d8f0b40afbc4c02705b7fc1d) (Built Dec 30 2021 02:47:29)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

通过代理登录

[obdba@dbdriver soft]$ obclient -h10.51.xx.65 -P2883 -uroot@sys#obtest -p -c -A oceanbase
Enter password:
Welcome to the OceanBase.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.25 OceanBase 3.1.2 (r10000392021123010-d4ace121deae5b81d8f0b40afbc4c02705b7fc1d) (Built Dec 30 2021 02:47:29)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [oceanbase]>
MySQL [oceanbase]> exit
Bye

安装出问题卸载

[obdba@dbdriver soft]$ obd cluster destroy obtest
Get local repositories and plugins ok
Open ssh connection ok
Stop observer ok
Stop obproxy ok
obtest stopped
Get local repositories and plugins ok
Open ssh connection ok
Cluster status check ok
observer work dir cleaning ok
obproxy work dir cleaning ok
obtest destroyed