5.ClickHouse系列之配置副本集

1. 编写clickhouse-replication.yml文件

该代码已上传至gitee,可克隆下来

# 副本集部署示例
version: '3'
services:
  zoo1:
    image: zookeeper
    restart: always
    hostname: zoo1
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181
    networks:
      - ckNet

  zoo2:
    image: zookeeper
    restart: always
    hostname: zoo2
    ports:
      - 2182:2181
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181
    networks:
      - ckNet
  zoo3:
    image: zookeeper
    restart: always
    hostname: zoo3
    ports:
      - 2183:2181
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181
    networks:
      - ckNet
  ck1:
    image: clickhouse/clickhouse-server
    container_name: ck1
    ulimits:
      nofile:
        soft: "262144"
        hard: "262144"
    volumes:
      - ./config.d:/etc/clickhouse-server/config.d
      - ./config.xml:/etc/clickhouse-server/config.xml
      - ./logs/:/var/log/clickhouse-server
    ports:
      - 18123:8123 # http接口用
      - 19000:9000 # 本地客户端用
    depends_on:
      - zoo1
    networks:
      - ckNet
  ck2:
    image: clickhouse/clickhouse-server
    container_name: ck2
    ulimits:
      nofile:
        soft: "262144"
        hard: "262144"
    volumes:
      - ./config.d:/etc/clickhouse-server/config.d
      - ./config.xml:/etc/clickhouse-server/config.xml
    ports:
      - 18124:8123 # http接口用
      - 19001:9000 # 本地客户端用
    depends_on:
      - zoo1
    networks:
      - ckNet

networks:
  ckNet:
    driver: bridge

其中config.d目录下ck_zk.xml配置如下

<?xml version="1.0" encoding="utf-8" ?>
<yandex>
    <zookeeper-servers>
        <node index="1">
            <host>zoo1</host>
            <port>2181</port>
        </node>
        <node index="2">
            <host>zoo2</host>
            <port>2181</port>
        </node>
        <node index="3">
            <host>zoo3</host>
            <port>2181</port>
        </node>
    </zookeeper-servers>
</yandex>

对于config.xml主要新增以下三行

    <!-- 副本集配置新增该两行 -->
    <zookeeper incl="zookeeper-servers" optional="true"></zookeeper>
    <include_from>/etc/clickhouse-server/config.d/ck_zk.xml</include_from>

    <!-- 远程访问，windows可能不支持，可以打开下面 -->
    <!--     <listen_host>::</listen_host>-->

    <!-- Same for hosts without support for IPv6: -->
     <listen_host>0.0.0.0</listen_host>  

2.创建表

datagrip工具连接127.0.0.1:18123与127.0.0.1:18124

分别创建表house,其中/clickhouse/table/01/house的01代表分片，rep_001为副本名称

CREATE TABLE house (
    id String,
    city String,
    region String,
    name String,
    price Float32,
    publish_date DateTime
) ENGINE=ReplicatedMergeTree('/clickhouse/table/01/house', 'rep_001') PARTITION BY toYYYYMMDD(publish_date) PRIMARY KEY(id) ORDER BY (id, city, region, name)
SETTINGS index_granularity=8192

CREATE TABLE house (
    id String,
    city String,
    region String,
    name String,
    price Float32,
    publish_date DateTime
) ENGINE=ReplicatedMergeTree('/clickhouse/table/01/house', 'rep_002') PARTITION BY toYYYYMMDD(publish_date) PRIMARY KEY(id) ORDER BY (id, city, region, name)
SETTINGS index_granularity=8192

此时，副本集的表已创建完成，通过prettyZoo工具可以看到副本rep_001与rep_002

3. 验证副本集

在副本rep_001表house中插入INSERT INTO house(name, price) VALUES ('场中小区', '59680');,
然后在副本rep_002中查询，可以看到数据已插入，反过来插入查询也一样结果

4. 副本写入流程图

了解了怎么具体部署副本集，咱们现在了解下副本写入的流程

欢迎关注公众号算法小生或沈健的技术博客