使用docker-compose创建主从模式 Redis集群

使用docker-compose创建主从模式 Redis集群

原文链接
版本信息：5.0.6

一、目录结构如下

image.png

  • 创建一个redis_master_slave文件夹，用于存放redis集群的配置文件
  • 这里我对每个节点又单独创建了master、slave1、slave2文件夹，也可以不创建

1.1 docker-compose.yaml内容

1.1.1 master节点的docker-compose.yaml

version: '3'

services:
  redis:
    image: redis:latest
    container_name: redis
    restart: always
    ports:
      - 6379:6379
    networks:
      mynetwork:
        ipv4_address: 172.19.0.10
    volumes:
      - ./redis.conf:/usr/local/etc/redis/redis.conf:rw
      - ./data:/data:rw
    command:
      /bin/bash -c "redis-server /usr/local/etc/redis/redis.conf "
networks:
  mynetwork:
    external: true

slave1节点、slave2节点的不同点在于 IP和端口映射不一样。

1.1.2 slave1节点的docker-compose.yaml

version: '3'

services:
  redis:
    image: redis:latest
    container_name: redis_slave1
    restart: always
    ports:
      - 6380:6379
    networks:
      mynetwork:
        ipv4_address: 172.19.0.11
    volumes:
      - ./redis.conf:/usr/local/etc/redis/redis.conf:rw
      - ./data:/data:rw
    command:
      /bin/bash -c "redis-server /usr/local/etc/redis/redis.conf "
networks:
  mynetwork:
    external: true

1.1.3 slave2节点的docker-compose.yaml

version: '3'

services:
  redis:
    image: redis:latest
    container_name: redis_slave2
    restart: always
    ports:
      - 6381:6379
    networks:
      mynetwork:
        ipv4_address: 172.19.0.12
    volumes:
      - ./redis.conf:/usr/local/etc/redis/redis.conf:rw
      - ./data:/data:rw
    command:
      /bin/bash -c "redis-server /usr/local/etc/redis/redis.conf "
networks:
  mynetwork:
    external: true

1.2 redis.conf内容

1.2.1 master节点的redis.conf

bind 0.0.0.0
protected-mode no
port 6379
timeout 0
save 900 1 # 900s内至少一次写操作则执行bgsave进行RDB持久化
save 300 10
save 60 10000
rdbcompression yes
dbfilename dump.rdb
dir /data
appendonly yes
appendfsync everysec
requirepass 12345678

1.2.2 slave1节点的redis.conf

bind 0.0.0.0
protected-mode no
port 6379
timeout 0
save 900 1 # 900s内至少一次写操作则执行bgsave进行RDB持久化
save 300 10
save 60 10000
rdbcompression yes
dbfilename dump.rdb
dir /data
appendonly yes
appendfsync everysec
requirepass 12345678

slaveof 172.19.0.10 6379
masterauth 12345678

1.2.3 slave2节点的redis.conf

bind 0.0.0.0
protected-mode no
port 6379
timeout 0
save 900 1 # 900s内至少一次写操作则执行bgsave进行RDB持久化
save 300 10
save 60 10000
rdbcompression yes
dbfilename dump.rdb
dir /data
appendonly yes
appendfsync everysec
requirepass 12345678

slaveof 172.19.0.10 6379
masterauth 12345678

二、启动节点服务

依次启动各个节点服务，当slave节点启动的时候，会自动连接master节点，同时master节点也会显示连接日志。

2.1 master节点日志

Creating redis ... done
Attaching to redis
redis    | 1:C 06 Sep 2020 01:05:44.001 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis    | 1:C 06 Sep 2020 01:05:44.001 # Redis version=5.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis    | 1:C 06 Sep 2020 01:05:44.001 # Configuration loaded
redis    | 1:M 06 Sep 2020 01:05:44.005 * Running mode=standalone, port=6379.
redis    | 1:M 06 Sep 2020 01:05:44.005 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis    | 1:M 06 Sep 2020 01:05:44.005 # Server initialized
redis    | 1:M 06 Sep 2020 01:05:44.005 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis    | 1:M 06 Sep 2020 01:05:44.008 * Ready to accept connections
redis    | 1:M 06 Sep 2020 01:06:30.505 * Replica 172.19.0.11:6380 asks for synchronization
redis    | 1:M 06 Sep 2020 01:06:30.505 * Full resync requested by replica 172.19.0.11:6380
redis    | 1:M 06 Sep 2020 01:06:30.505 * Starting BGSAVE for SYNC with target: disk
redis    | 1:M 06 Sep 2020 01:06:30.506 * Background saving started by pid 10
redis    | 10:C 06 Sep 2020 01:06:30.521 * DB saved on disk
redis    | 10:C 06 Sep 2020 01:06:30.527 * RDB: 0 MB of memory used by copy-on-write
redis    | 1:M 06 Sep 2020 01:06:30.582 * Background saving terminated with success
redis    | 1:M 06 Sep 2020 01:06:30.589 * Synchronization with replica 172.19.0.11:6380 succeeded
redis    | 1:M 06 Sep 2020 01:07:53.029 * Replica 172.19.0.12:6381 asks for synchronization
redis    | 1:M 06 Sep 2020 01:07:53.029 * Full resync requested by replica 172.19.0.12:6381
redis    | 1:M 06 Sep 2020 01:07:53.029 * Starting BGSAVE for SYNC with target: disk
redis    | 1:M 06 Sep 2020 01:07:53.030 * Background saving started by pid 11
redis    | 11:C 06 Sep 2020 01:07:53.040 * DB saved on disk
redis    | 11:C 06 Sep 2020 01:07:53.041 * RDB: 0 MB of memory used by copy-on-write
redis    | 1:M 06 Sep 2020 01:07:53.057 * Background saving terminated with success
redis    | 1:M 06 Sep 2020 01:07:53.063 * Synchronization with replica 172.19.0.12:6381 succeeded
  • master节点启动后，等待接收连接
  • 收到副本 172.19.0.11:6380 的同步请求
  • 和172.19.0.11:6380进行同步
  • 收到副本 172.19.0.12:6381 的同步请求
  • 和172.19.0.12:6381进行同步

2.2 slave1节点日志

Creating redis_slave1 ... done
Attaching to redis_slave1
redis_slave1 | 1:C 06 Sep 2020 01:18:21.869 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_slave1 | 1:C 06 Sep 2020 01:18:21.869 # Redis version=5.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis_slave1 | 1:C 06 Sep 2020 01:18:21.869 # Configuration loaded
redis_slave1 | 1:S 06 Sep 2020 01:18:21.872 * Running mode=standalone, port=6379.
redis_slave1 | 1:S 06 Sep 2020 01:18:21.872 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_slave1 | 1:S 06 Sep 2020 01:18:21.873 # Server initialized
redis_slave1 | 1:S 06 Sep 2020 01:18:21.873 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_slave1 | 1:S 06 Sep 2020 01:18:21.878 * Reading RDB preamble from AOF file...
redis_slave1 | 1:S 06 Sep 2020 01:18:21.879 * Reading the remaining AOF tail...
redis_slave1 | 1:S 06 Sep 2020 01:18:21.881 * DB loaded from append only file: 0.009 seconds
redis_slave1 | 1:S 06 Sep 2020 01:18:21.881 * Ready to accept connections
redis_slave1 | 1:S 06 Sep 2020 01:18:21.881 * Connecting to MASTER 172.19.0.10:6379
redis_slave1 | 1:S 06 Sep 2020 01:18:21.882 * MASTER <-> REPLICA sync started
redis_slave1 | 1:S 06 Sep 2020 01:18:21.882 * Non blocking connect for SYNC fired the event.
redis_slave1 | 1:S 06 Sep 2020 01:18:21.882 * Master replied to PING, replication can continue...
redis_slave1 | 1:S 06 Sep 2020 01:18:21.884 * Partial resynchronization not possible (no cached master)
redis_slave1 | 1:S 06 Sep 2020 01:18:21.886 * Full resync from master: 83cf8d4614610468aa53c43c553ca46a26c1e3be:924
redis_slave1 | 1:S 06 Sep 2020 01:18:21.976 * MASTER <-> REPLICA sync: receiving 176 bytes from master
redis_slave1 | 1:S 06 Sep 2020 01:18:21.981 * MASTER <-> REPLICA sync: Flushing old data
redis_slave1 | 1:S 06 Sep 2020 01:18:21.982 * MASTER <-> REPLICA sync: Loading DB in memory
redis_slave1 | 1:S 06 Sep 2020 01:18:21.986 * MASTER <-> REPLICA sync: Finished with success
redis_slave1 | 1:S 06 Sep 2020 01:18:21.988 * Background append only file rewriting started by pid 10
redis_slave1 | 1:S 06 Sep 2020 01:18:22.020 * AOF rewrite child asks to stop sending diffs.
redis_slave1 | 10:C 06 Sep 2020 01:18:22.020 * Parent agreed to stop sending diffs. Finalizing AOF...
redis_slave1 | 10:C 06 Sep 2020 01:18:22.020 * Concatenating 0.00 MB of AOF diff received from parent.
redis_slave1 | 10:C 06 Sep 2020 01:18:22.023 * SYNC append only file rewrite performed
redis_slave1 | 10:C 06 Sep 2020 01:18:22.024 * AOF rewrite: 0 MB of memory used by copy-on-write
redis_slave1 | 1:S 06 Sep 2020 01:18:22.088 * Background AOF rewrite terminated with success
redis_slave1 | 1:S 06 Sep 2020 01:18:22.097 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
redis_slave1 | 1:S 06 Sep 2020 01:18:22.109 * Background AOF rewrite finished successfully

2.3 slave2节点日志

Creating redis_slave2 ... done
Attaching to redis_slave2
redis_slave2 | 1:C 06 Sep 2020 01:18:51.630 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_slave2 | 1:C 06 Sep 2020 01:18:51.630 # Redis version=5.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis_slave2 | 1:C 06 Sep 2020 01:18:51.630 # Configuration loaded
redis_slave2 | 1:S 06 Sep 2020 01:18:51.636 * Running mode=standalone, port=6379.
redis_slave2 | 1:S 06 Sep 2020 01:18:51.637 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_slave2 | 1:S 06 Sep 2020 01:18:51.637 # Server initialized
redis_slave2 | 1:S 06 Sep 2020 01:18:51.638 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_slave2 | 1:S 06 Sep 2020 01:18:51.651 * Reading RDB preamble from AOF file...
redis_slave2 | 1:S 06 Sep 2020 01:18:51.653 * Reading the remaining AOF tail...
redis_slave2 | 1:S 06 Sep 2020 01:18:51.657 * DB loaded from append only file: 0.019 seconds
redis_slave2 | 1:S 06 Sep 2020 01:18:51.657 * Ready to accept connections
redis_slave2 | 1:S 06 Sep 2020 01:18:51.657 * Connecting to MASTER 172.19.0.10:6379
redis_slave2 | 1:S 06 Sep 2020 01:18:51.658 * MASTER <-> REPLICA sync started
redis_slave2 | 1:S 06 Sep 2020 01:18:51.658 * Non blocking connect for SYNC fired the event.
redis_slave2 | 1:S 06 Sep 2020 01:18:51.659 * Master replied to PING, replication can continue...
redis_slave2 | 1:S 06 Sep 2020 01:18:51.661 * Partial resynchronization not possible (no cached master)
redis_slave2 | 1:S 06 Sep 2020 01:18:51.664 * Full resync from master: 83cf8d4614610468aa53c43c553ca46a26c1e3be:966
redis_slave2 | 1:S 06 Sep 2020 01:18:51.717 * MASTER <-> REPLICA sync: receiving 176 bytes from master
redis_slave2 | 1:S 06 Sep 2020 01:18:51.730 * MASTER <-> REPLICA sync: Flushing old data
redis_slave2 | 1:S 06 Sep 2020 01:18:51.731 * MASTER <-> REPLICA sync: Loading DB in memory
redis_slave2 | 1:S 06 Sep 2020 01:18:51.739 * MASTER <-> REPLICA sync: Finished with success
redis_slave2 | 1:S 06 Sep 2020 01:18:51.747 * Background append only file rewriting started by pid 11
redis_slave2 | 1:S 06 Sep 2020 01:18:51.782 * AOF rewrite child asks to stop sending diffs.
redis_slave2 | 11:C 06 Sep 2020 01:18:51.782 * Parent agreed to stop sending diffs. Finalizing AOF...
redis_slave2 | 11:C 06 Sep 2020 01:18:51.782 * Concatenating 0.00 MB of AOF diff received from parent.
redis_slave2 | 11:C 06 Sep 2020 01:18:51.786 * SYNC append only file rewrite performed
redis_slave2 | 11:C 06 Sep 2020 01:18:51.786 * AOF rewrite: 0 MB of memory used by copy-on-write
redis_slave2 | 1:S 06 Sep 2020 01:18:51.859 * Background AOF rewrite terminated with success
redis_slave2 | 1:S 06 Sep 2020 01:18:51.861 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
redis_slave2 | 1:S 06 Sep 2020 01:18:51.864 * Background AOF rewrite finished successfully

三、测试

这里以RDM工具作为redis连接的客户端，分别连接redis的三个实例。
image.png
在master上输入 set name zhangsan，在slave1中通过 get name 可以查看
image.png

四、原理

主从的原理是比较简单的，一主多从，主数据库可以读也可以写，从数据库仅读。

但是，主从模式一般实现读写分离，主数据库仅写，减轻主数据库的压力。原理图如下：
image.png
下面是工作机制：
image.png

  1. 当slave启动后会向master发送SYNC命令，master节点收到从数据库的命令后通过bgsave保存快照（rdb持久化），并且从此时起执行的命令会被缓存起来。
  2. master会将保存的快照发送给slave，并且继续缓存期间的写命令。
  3. slave收到主数据库发送过来的快照就会加载到自己的数据中。
  4. 最后master将缓存的命令同步给slave，slave收到命令后执行一遍，这样master与slave数据就保持一致了。

五、优缺点

5.1 优点

  1. 之所以运用主从复制，是因为主从一定程度上解决了单机版并发量大，导致亲求延迟或者redis宕机服务停止的问题
  2. 从数据库分担主数据库的读压力，若是主数据库只是写模式，那么实现读写分离，主数据库就没有读压力了
  3. 另一方面解决了单机版单点故障的问题，若是主数据库挂了，那么从数据库可以随时顶上来

综上来说，主从模式一定程度上提高了系统的可用性和性能，是实现哨兵和集群的基础。主从同步是以异步方式进行同步，期间redis仍然可以响应客户端提交的查询和更新的请求。

5.2缺点

主从模式好是好，但是也有自己的缺点。

  1. 比如数据的一致性问题，假如主数据库写操作完成，那么他的数据会被复制到从数据库，若是还没有及时复制到从数据库，读请求又来了，此时读取的数据就不是最新的数据。
  2. 若是主从同步的过程网络出故障了，导致主从同步失败，也会出现数据一致性的问题
  3. 主从模式不具备自动容错和恢复的功能，一旦主数据库挂掉，从节点晋升为主数据库的过程需要人为操作，维护的成本就会升高，并且主节点的写能力，存储能力都会受到限制。

到此结束。

posted @ 2020-09-10 20:44  在线打工者  阅读(1415)  评论(0)  编辑  收藏  举报