基于Patroni的Citus高可用环境部署

合集 - Patroni-Citus(12)

1.patroni中的citus2024-12-13

2.基于Patroni的Citus高可用环境部署2024-12-13

3.Patroni处理流程图2024-12-134.Patroni在Zookeeper中保存的信息2024-12-135.源码安装patroni+postgresql+citus2024-12-136.分布式Citus集群与Patroni2024-12-137.Citus的restart详解2024-12-138.Patroni中对pg的重启、停止和启动详解2024-12-139.Patroni中对主备切换、故障转移和命令行的流程整理2024-12-1310.Patroni 流程整理目录-配置文件2024-12-1311.Patroni 流程整理目录-初始化2024-12-2412.Patroni 流程整理-主循环2024-12-24

收起

基于Patroni的Citus高可用环境部署

参考文献：https://www.cnblogs.com/VicLiu/p/14333979.html

1. 实验环境

主要软件

• CentOS 7.8
• PostgreSQL 12
• Citus 10.4
• patroni 1.6.5
• etcd 3.3.25

主机名	别名	地址
node1	test41	192.168.198.164
node2	test44	192.168.198.172
node3	test42	192.168.198.165
node4	test45	192.168.198.173
vip1	test43	192.168.198.166
vip2	test46	192.168.198.174

机器和VIP资源

• Citus CN
  • node1：192.168.198.164
  • node2：192.168.198.172
• Citus Worker
  • node3：192.168.198.165
  • node4：192.168.198.173
• etcd
  • node4：192.168.198.173
• VIP（Citus CN )
  • 读写VIP：192.168.198.166
  • 只读VIP：192.168.198.174

2. 环境准备

2.1 时间同步

所有节点设置时钟同步

yum install -y ntpdate
ntpdate time.windows.com && hwclock -w

2.2 关闭防火墙

如果使用防火墙需要开放postgres，etcd和patroni的端口。

• postgres:5432
• patroni:8008
• etcd:2379/2380

更简单的做法是将防火墙关闭

setenforce 0
sed -i.bak "s/SELINUX=enforcing/SELINUX=permissive/g" /etc/selinux/config
systemctl disable firewalld.service
systemctl stop firewalld.service
iptables -F

或者：

# 查询防火墙状态
systemctl status firewalld
# 停止防火墙服务	
systemctl stop firewalld
# 禁用防火墙
systemctl disable firewalld

2.3 etcd部署

因为本文的主题不是etcd的高可用，所以只在node4上部署单节点的etcd用于实验。生产环境至少需要3台独立的机器，也可以和数据库部署在一起。etcd的部署步骤如下

安装需要的包

yum install -y gcc python-devel epel-release

安装etcd

yum install -y etcd

编辑etcd配置文件/etc/etcd/etcd.conf, 参考配置如下

ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.198.173:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://192.168.198.173:2379"
ETCD_NAME="etcd0"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.198.173:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.198.173:2379"
ETCD_INITIAL_CLUSTER="etcd0=http://192.168.198.173:2380"
ETCD_INITIAL_CLUSTER_TOKEN="cluster1"
ETCD_INITIAL_CLUSTER_STATE="new"

启动etcd

systemctl start etcd

设置etcd自启动

systemctl enable etcd

2.4 PostgreSQL + Citus + Patroni HA部署

在需要运行PostgreSQL的实例上安装相关软件

安装PostgreSQL 12和Citus

yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
# 如果无法拉取
wget https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm --no-check-certificate
yum install -y pgdg-redhat-repo-latest.noarch.rpm


yum install -y postgresql12-server postgresql12-contrib
yum install -y citus_12
# 如果无法安装 ssl出现问题 可以加上 --setopt=sslverify=false
yum --setopt=sslverify=false install -y postgresql12-server postgresql12-contrib
yum --setopt=sslverify=false install -y citus_12
yum --setopt=sslverify=false --skip-broken install -y citus_12

安装Patroni

yum install -y gcc epel-release
yum install -y python-pip python-psycopg2 python-devel

pip install --upgrade pip
pip install --upgrade setuptools
pip install patroni[etcd]


pip install --upgrade pip -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip install --upgrade setuptools -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip install patroni[etcd] -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com

创建PostgreSQL数据目录

mkdir -p /pgsql/data
chown postgres:postgres -R /pgsql
chmod -R 700 /pgsql/data

创建Partoni的service配置文件/etc/systemd/system/patroni.service

[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL
After=syslog.target network.target
 
[Service]
Type=simple
User=postgres
Group=postgres
#StandardOutput=syslog
ExecStart=/usr/bin/patroni /etc/patroni.yml
ExecReload=/bin/kill -s HUP $MAINPID
KillMode=process
TimeoutSec=30
Restart=no
 
[Install]
WantedBy=multi-user.target

创建Patroni配置文件/etc/patroni.yml,以下是node1的配置示例

scope: cn
namespace: /service/
name: pg4

restapi:
  listen: 0.0.0.0:8008
  connect_address: 192.168.198.173:8008

etcd:
  host: 192.168.198.173:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    master_start_timeout: 300
    synchronous_mode: false
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        listen_addresses: "0.0.0.0"
        port: 5432
        wal_level: logical
        hot_standby: "on"
        wal_keep_segments: 1000
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"
        max_connections: "100"
        max_prepared_transactions: "100"
        shared_preload_libraries: "citus"
        citus.node_conninfo: "sslmode=prefer"
        citus.replication_model: streaming
        citus.task_assignment_policy: round-robin

  initdb:
  - encoding: UTF8
  - locale: C
  - lc-ctype: zh_CN.UTF-8
  - data-checksums

  pg_hba:
  - host replication repl 0.0.0.0/0 md5
  - host all all 0.0.0.0/0 md5

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 192.168.198.173:5432
  data_dir: /pgsql/data
  bin_dir: /usr/pgsql-12/bin

  authentication:
    replication:
      username: repl
      password: "123456"
    superuser:
      username: postgres
      password: "123456"

  basebackup:
    max-rate: 100M
    checkpoint: fast

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false

其他PG节点的patroni.yml需要相应修改下面4个参数

• scope
  • node1，node2设置为cn
  • node3，node4设置为wk1
• name
  • node1~node4分别设置pg1~pg4
• restapi.connect_address
  • 根据各自节点IP设置
• postgresql.connect_address
  • 根据各自节点IP设置

启动Patroni

在所有节点上启动Patroni。

systemctl start patroni

同一个cluster中，第一次启动的Patroni实例会作为leader运行，并初始创建PostgreSQL实例和用户。后续节点初次启动时从leader节点克隆数据

查看cn集群状态