redis集群搭建

 
需求
搭建一个redis集群，支持故障恢复,。
Redis内置的一个集群需要至少3个主节点，所以搭建规模为3主3个从的集群用于测试。节点信息列表：
 
role  ip port bus port remark
master1 192.168.201.209 7000 17000 主节点1
master2 192.168.201.209 7001 17001 主节点2
master3 192.168.201.209 7002 17002 主节点3
replica1 192.168.201.209 7003 17003 从节点1
replica1 192.168.201.209 7004 17004 从节点2
replica1 192.168.201.209 7005 17005 从节点3
sentinel1 192.168.201.209 6000 - 哨兵1
sentinel2 192.168.201.209 6001 - 哨兵2
sentinel3 192.168.201.209 6002 - 哨兵3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
环境：
  • 镜像redis:alpine，redis的版本为5.0.3
  • macOS 10.13.4
  • Docker 18.03.1-ce
 
 
 
docker中用到的配置文件命名
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
➜  config pwd
/Users/cymin/redis/config
➜  config tree .
.
├── m1.conf
├── m2.conf
├── m3.conf
├── r1.conf
├── r2.conf
├── r3.conf
├── s1.conf
├── s2.conf
└── s3.conf
  
0 directories, 9 files

　　

Note that:
  • Masters are called M1, M2, M3, ..., Mn.
  • Slaves are called R1, R2, R3, ..., Rn (R stands for replica).
  • Sentinels are called S1, S2, S3, ..., Sn.
 
 
3个redis master实例配置
?
1
2
3
4
5
6
7
8
9
➜  config cat m1.conf
cluster-announce-ip 192.168.201.209
cluster-announce-port 7000
cluster-announce-bus-port 17000
  
cluster-enabled yes
cluster-config-file "/tmp/nodes.conf"
cluster-node-timeout 5000
appendonly yes

　　

剩余的master端口依次+1, cluster-announce-bus-port的配置+10000.
说明：
  • cluster-announce-ip 告诉redis实例外部映射地址
  • cluster-announce-port 告诉redis实例外部映射端口
  • cluster-announce-bus-port 开启集群总线端口,值=cluster-announce-port +10000
  • masterauth | requirepass 密码，master和replica需要配置为一样的密码
  • 其他都是基本配置,相关文档：https://redis.io/topics/cluster-tutorial
 
 
 
3个redis replica(slave)实例配置
?
1
2
3
4
5
6
7
8
9
➜  config cat r1.conf
cluster-announce-ip 192.168.201.209
cluster-announce-port 7003
cluster-announce-bus-port 17003
  
cluster-enabled yes
cluster-config-file "/tmp/nodes.conf"
cluster-node-timeout 5000
appendonly yes

　　

剩余的cluster-announce-port端口依次+1，cluster-announce-bus-port的配置+10000。
sentinel实例之间的通讯端口默认是26379.
 
 
 
3个哨兵实例配置
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
➜  config cat s1.conf
sentinel announce-ip 192.168.201.209
sentinel announce-port 6000
  
sentinel monitor master1 192.168.201.209 7000 2
sentinel down-after-milliseconds master1 60000
sentinel failover-timeout master1 180000
sentinel parallel-syncs master1 1
  
sentinel monitor master2 192.168.201.209 7001 2
sentinel down-after-milliseconds master2 60000
sentinel failover-timeout master2 180000
sentinel parallel-syncs master2 1
  
sentinel monitor master3 192.168.201.209 7002 2
sentinel down-after-milliseconds master3 60000
sentinel failover-timeout master3 180000
sentinel parallel-syncs master3 1　
剩余的announce-port配置依次加1.
说明：每个哨兵只需要监控master节点即可，配置项说明见文档：https://redis.io/topics/sentinel
 
 
 
启动redis实例和哨兵
run.sh
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 3个master
docker run --name master1 -tid --network service -v $HOME/redis/config:/opt/config -p 7000:6379 -p 17000:16379  redis:alpine redis-server /opt/config/m1.conf
docker run --name master2 -tid --network service -v $HOME/redis/config:/opt/config -p 7001:6379 -p 17001:16379  redis:alpine redis-server /opt/config/m2.conf
docker run --name master3 -tid --network service -v $HOME/redis/config:/opt/config -p 7002:6379 -p 17002:16379  redis:alpine redis-server /opt/config/m3.conf
  
# 3个replica
docker run --name replica1 -tid --network service -v $HOME/redis/config:/opt/config -p 7003:6379 -p 17003:16379  redis:alpine redis-server /opt/config/r1.conf
docker run --name replica2 -tid --network service -v $HOME/redis/config:/opt/config -p 7004:6379 -p 17004:16379  redis:alpine redis-server /opt/config/r2.conf
docker run --name replica3 -tid --network service -v $HOME/redis/config:/opt/config -p 7005:6379 -p 17005:16379  redis:alpine redis-server /opt/config/r3.conf
  
# 3个sentinel
docker run --name sentinel1 -tid --network service -v $HOME/redis/config:/opt/config -p 6000:26379 redis:alpine redis-sentinel /opt/config/s1.conf
docker run --name sentinel2 -tid --network service -v $HOME/redis/config:/opt/config -p 6001:26379 redis:alpine redis-sentinel /opt/config/s2.conf
docker run --name sentinel3 -tid --network service -v $HOME/redis/config:/opt/config -p 6002:26379 redis:alpine redis-sentinel /opt/config/s3.conf

　　

 
 
创建集群
?
1
2
docker exec -it master1 sh
/data # redis-cli --cluster create 192.168.201.209:7000 192.168.201.209:7001 192.168.201.209:7002 192.168.201.209:7003 192.168.201.209:7004 192.168.201.209:7005 --cluster-replicas 1
最后收到"[OK] All 16384 slots covered.”的日志时，集群搭建成功。
 
 
 
连接集群中某个redis实例
?
1
redis-cli -c -h 192.168.201.209 -p 7000

　　

查看集群状态
?
1
redis-cli cluster nodes
 
 
查看某个节点的主从状态
?
1
redis-cli -h 192.168.201.209 -p 7000 info Replication

 

 
 
查看某个哨兵的状态
?
1
redis-cli -h 192.168.201.209 -p 6000 info Sentinel

　　

 
故障自动恢复测试
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 集群节点状态
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547716358000 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547716357525 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547716357000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 slave 7d0d00d7710fc416ac2c3f44ea86b306b498038e 0 1547716357014 6 connected
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547716358535 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 master - 0 1547716356511 1 connected 0-5460
  
# 停止master1
docker stop master1
  
# 集群节点状态
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547716754542 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547716754542 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547716753000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 master - 0 1547716754000 7 connected 0-5460
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547716754946 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 master,fail - 1547716421213 1547716420405 1 disconnected
  
# 重新启动master1
docker restart master1
  
# 集群节点状态
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547716837000 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547716837000 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547716836000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 master - 0 1547716838053 7 connected 0-5460
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547716837138 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 slave e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 0 1547716838153 7 connected

观察当id为7d0d00d7710fc416ac2c3f44ea86b306b498038e的master节点挂掉的时候，其下id为e9f1294a466d3c2e4c3b40d59c8237c346c7fed7的slave的节点自动升级为master节点，当之前的master又重新启动的时候自动变成slave节点。

 
 
增加一个节点
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 增加一个redis实例，replica4端口映射到本地为7006
docker run --name replica4 -tid --network service -v $HOME/redis/config:/opt/config -p 7006:6379 -p 17006:16379  redis:alpine redis-server /opt/config/r4.conf
  
# 给id为4de60b987e84de81a6fce10d68f4b1ba0d2ac28a增加一个slave
redis-cli --cluster add-node 192.168.201.209:7006 192.168.201.209:7002 --cluster-slave --cluster-master-id 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a
  
# 查看集群节点状态
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547718761000 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547718761820 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547718761000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 master - 0 1547718761000 7 connected 0-5460
435a5e171bd4ffc02e862d174609fa0d07045925 192.168.201.209:7006@17006 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547718761312 3 connected
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547718760000 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 slave e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 0 1547718761517 7 connected

　　

 
 
 
删除一个节点
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
/data # redis-cli --cluster del-node 192.168.201.209:7000 435a5e171bd4ffc02e862d174609fa0d07045925
>>> Removing node 435a5e171bd4ffc02e862d174609fa0d07045925 from cluster 192.168.201.209:7000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
 
# 查看集群节点状态
/data # redis-cli cluster nodes
b22123911ffed94e6827a94c55891a4488dbbf5c 192.168.201.209:7003@17003 slave 507f3ec4b24d8d380b4da0c107214dae5b4c08b5 0 1547719609601 4 connected
da63ff7af84ba911d905a8a2e0494520e1eaa6c9 192.168.201.209:7004@17004 slave 4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 0 1547719610512 5 connected
507f3ec4b24d8d380b4da0c107214dae5b4c08b5 192.168.201.209:7001@17001 myself,master - 0 1547719611000 2 connected 5461-10922
e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 192.168.201.209:7005@17005 master - 0 1547719611626 7 connected 0-5460
4de60b987e84de81a6fce10d68f4b1ba0d2ac28a 192.168.201.209:7002@17002 master - 0 1547719610512 3 connected 10923-16383
7d0d00d7710fc416ac2c3f44ea86b306b498038e 192.168.201.209:7000@17000 slave e9f1294a466d3c2e4c3b40d59c8237c346c7fed7 0 1547719610613 7 connected

　　

 
 
数据同步测试
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
/data # redis-cli -c -h 192.168.201.209 -p 7000
192.168.201.209:7000> keys *
(empty list or set)
192.168.201.209:7000> set name cymin
-> Redirected to slot [5798] located at 192.168.201.209:7001
OK
192.168.201.209:7001>
/data # redis-cli -c -h 192.168.201.209 -p 7001
192.168.201.209:7001> keys *
1) "name"
192.168.201.209:7001>
/data # redis-cli -c -h 192.168.201.209 -p 7002
192.168.201.209:7002> keys *
(empty list or set)
192.168.201.209:7002> get name
-> Redirected to slot [5798] located at 192.168.201.209:7001
“cymin”
192.168.201.209:7001>
192.168.201.209:7001> get afawfgawfgw
-> Redirected to slot [15081] located at 192.168.201.209:7002
(nil)
192.168.201.209:7002> get feawgwegewgew
-> Redirected to slot [7926] located at 192.168.201.209:7001
(nil)
192.168.201.209:7001>

观察数据读写均在master节点，如果在replica节点中读写key，则会自动重定向到对应的master节点中。

 
 
集群密码设置
进入各个(6个)实例进行依次设置:
?
1
2
3
redis-cli -c -h 192.168.201.209 -p 7000
127.0.0.1:7000> config set masterauth passwd123
127.0.0.1:7000> config set requirepass passwd123
 
# 重新进入实例，写入配置
?
1
2
redis-cli -a abc123 -c -h 192.168.201.209 -p 7000
127.0.0.1:7000> config rewrite

 

 
 
 
java连接集群事例
Jar:
?
1
compile group: 'redis.clients', name: 'jedis', version: '3.0.1'

　　 

使用jedis在代码中连接redis集群，因为是集群，可以自动发现，也同样只要一个地址就行了
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
  
import java.util.HashSet;
import java.util.Set;
  
public class RedisTest {
    public static void main(String[] args) {
        testWithPassword();
    }
  
    public static void test() {
        Set<HostAndPort> jedisClusterNodes = new HashSet<>();
        jedisClusterNodes.add(new HostAndPort("192.168.201.208", 7000));
        JedisCluster jc = new JedisCluster(jedisClusterNodes);
        String value = jc.get("name");
        System.out.println(value);
    }
  
  
    public static void testWithPassword() {
        Set<HostAndPort> jedisClusterNodes = new HashSet<>();
        jedisClusterNodes.add(new HostAndPort("192.168.201.208", 7000));
        GenericObjectPoolConfig gopc = new GenericObjectPoolConfig();
        gopc.setMaxTotal(32);
        gopc.setMaxIdle(4);
        JedisCluster jc = new JedisCluster(jedisClusterNodes, 10000, 10000, 3,"abc123", gopc);
        String value = jc.get("name");
        System.out.println(value);
    }
}

　　

 
 
 
 
The end. 
posted @   cymin  阅读(1825)  评论(0)  编辑  收藏  举报
编辑推荐：
· 从 HTTP 原因短语缺失研究 HTTP/2 和 HTTP/3 的设计差异
· AI与.NET技术实操系列：向量存储与相似性搜索在 .NET 中的实现
· 基于Microsoft.Extensions.AI核心库实现RAG应用
· Linux系列：如何用heaptrack跟踪.NET程序的非托管内存泄露
· 开发者必知的日志记录最佳实践
阅读排行：
· winform 绘制太阳，地球，月球 运作规律
· 超详细：普通电脑也行Windows部署deepseek R1训练数据并当服务器共享给他人
· 上周热点回顾（3.3-3.9）
· AI 智能体引爆开源社区「GitHub 热点速览」
· 写一个简单的SQL生成工具
点击右上角即可分享
微信分享提示