(四)Redis主从复制和sentinel
主从复制
数据副本
扩展读性能
- 一个master可以有多个slave
- 一个slave只能有一个master
- 数据流只能从mater流向slave
slaveof命令式复制:
redis-8380> slaveof 127.0.0.1:6379
配置复制:
slaveof ip port
# 具体看版本
slaveof/replicaof ip port
slave-read-only yes
1、主从复制demo
拷贝配置文件:
mkdir config
cp redis.conf config/
cd config
cp redis.conf redis-6379.conf # (主)
cp redis-6379.conf redis-6380.conf # (从)
配置从节点:
# 具体看版本
slaveof/replicaof ip port
slave-read-only yes
启动服务:
redis-server redis-6379.conf
redis-server redis-6380.conf
测试:
- 主节点
[root@localhost ~]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=6380,state=online,offset=2322,lag=0
master_replid:998f21abddaadcf5cb8b6595659b76ded9b2c603
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:2322
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:2322
127.0.0.1:6379> set kkk fffffffff
OK
127.0.0.1:6379> get kkk
"fffffffff"
- 从节点
[root@localhost config]# redis-cli -p 6380
127.0.0.1:6380> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:2364
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:998f21abddaadcf5cb8b6595659b76ded9b2c603
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:2364
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:2364
127.0.0.1:6380> get kkk
"fffffffff"
127.0.0.1:6380> set kkk
(error) ERR wrong number of arguments for 'set' command
Redis Sentinel
1、服务端高可用
主从复制高可用的问题
1、手动故障转移
2、写能力和存储能力受到限制
Redis Sentinel
- 客户端从redis sentinel获取redis信息
- Redis Sentinel管理redis集群
- Redis Sentinel本身是多节点
安装配置:
1、配置开启主从节点
主节点:默认配置
从节点:replicaof 127.0.0.1 7000 # 主节点的ip port
2、配置开启sentinel监控主节点
port ${port}
dir ./
logfile ${port}.log
sentinel monitor mymaster 127.0.0.1 7000 # 监控的主节点 name ip port
sentinel down-after-milliseconds mymaster 30000 # 30秒认定为down
sentinel parallel-syncs mymaster 1 # 并发还是串行
sentinel failover-timeout mymaster 180000 # 故障转移时间
3、实际应用多机器
[root@localhost config]# redis-server redis-7000.conf
[root@localhost config]# redis-server redis-7001.conf
[root@localhost config]# redis-server redis-7002.conf
[root@localhost config]# ps -ef | grep 700
root 29701 1 0 12:53 ? 00:00:00 redis-server *:7000
root 29708 1 0 12:53 ? 00:00:00 redis-server *:7001
root 29714 1 0 12:53 ? 00:00:00 redis-server *:7002
root 29719 29651 0 12:53 pts/0 00:00:00 grep --color=auto 700
4、启动redid-sentinel
redis-sentinel sentinel-26379.conf
port 26381
daemonize yes
pidfile /var/run/redis-sentinel.pid
logfile "/usr/local/redis-5.0.4/logs/sentinel-26381.log"
dir /tmp
sentinel monitor mymaster 192.168.0.109 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
5、启动后发现配置文件改变
# sentinel找到了监控的主节点的从节点,并写入了配置文件
protected-mode no
sentinel leader-epoch mymaster 0
sentinel known-replica mymaster 192.168.0.109 7002
sentinel known-replica mymaster 192.168.0.109 7001
sentinel current-epoch 0
6、启动多个redis-sentinel
sed "s/26379/26380/g" sentinel-26379.conf > sentinel-26380.conf
sed "s/26379/26381/g" sentinel-26379.conf > sentinel-26381.conf
redis-sentinel sentinel-26380.conf
redis-sentinel sentinel-26381.conf
ps -ef | grep 263 | grep -v col
root 29763 1 0 03:50 ? 00:02:19 redis-sentinel *:26379 [sentinel]
root 30868 1 0 17:02 ? 00:00:00 redis-sentinel *:26380 [sentinel]
root 30873 1 0 17:02 ? 00:00:00 redis-sentinel *:26381 [sentinel]
2、客户端高可用
需要的信息:
1、Sentinel地址集合
2、masterName
注意:Sentinel不是代理模式
jedis:
JedisSentinelPool sentinelPool = new JedisSentinelPool(masterName, sentinelSet, poolConfig, timeout);
Jedis jedis = null;
try {
jedis = redisSentinelPool.getResource();
callback(jedis);
} catch (Exception e) {
log.error(e.getMessage(), e);
} finally {
if (jedis != null) {
jedis.close();
}
}
example:
@Test
public void testRedisSentinel() throws InterruptedException {
String masterName = "mymaster";
Set<String> sentinelSet = new HashSet<>();
sentinelSet.add("192.168.0.109:26379");
sentinelSet.add("192.168.0.109:26380");
sentinelSet.add("192.168.0.109:26381");
JedisSentinelPool pool = new JedisSentinelPool(masterName, sentinelSet);
int count = 0;
while (true) {
try (Jedis resource = pool.getResource()) {
HostAndPort currentHostMaster = pool.getCurrentHostMaster();
String host = currentHostMaster.getHost();
int port = currentHostMaster.getPort();
resource.set("host" + count, host);
resource.set("port" + count, String.valueOf(port));
List<String> valueList = resource.mget("host" + count, "port" + count);
log.info("host" + count + " {}, port" + count + " {};", valueList.get(0), valueList.get(1));
} catch (Exception e) {
e.printStackTrace();
}
Thread.sleep(1000);
count++;
if (count == 10000) {
break;
}
log.info("count = {}", count);
}
}
3、故障转移模拟
启动java程序:
[INFO ] 2019-03-29 17:09:47,130 method:com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:49)
host9 192.168.0.109, port9 6379;
[INFO ] 2019-03-29 17:09:48,130 method:com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:58)
count = 10
[INFO ] 2019-03-29 17:09:48,132 method:com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:49)
host10 192.168.0.109, port10 6379;
# 6379 shutdown后
count = 11redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream.
at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
at redis.clients.jedis.util.RedisInputStream.readByte(RedisInputStream.java:43)
at redis.clients.jedis.Protocol.process(Protocol.java:154)
at redis.clients.jedis.Protocol.read(Protocol.java:219)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:309)
at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:236)
at redis.clients.jedis.Jedis.set(Jedis.java:149)
at com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:69)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:48)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:292)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
[INFO ] 2019-03-29 17:09:50,141 method:com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:58)
count = 12
at redis.clients.jedis.util.Pool.getResource(Pool.java:59)
at redis.clients.jedis.JedisSentinelPool.getResource(JedisSentinelPool.java:213)
at com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:69)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:48)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:292)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Failed connecting to host 192.168.0.109:6379
at redis.clients.jedis.Connection.connect(Connection.java:204)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:100)
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1862)
at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:117)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:888)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:432)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:361)
at redis.clients.jedis.util.Pool.getResource(Pool.java:50)
... 24 more
# 新选举master后
[INFO ] 2019-03-29 17:10:19,301 method:com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:49)
host41 192.168.0.109, port41 6381;
[INFO ] 2019-03-29 17:10:20,301 method:com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:58)
count = 42
[INFO ] 2019-03-29 17:10:20,303 method:com.zuizui.connect.redis.JedisTest.testRedisSentinel(JedisTest.java:49)
host42 192.168.0.109, port42 6381;
4、三个定时任务
每10秒:每个sentinel对master和slave执行info
- 发现slave节点
- 确定主从关系
每2秒:每个sentinel通过master节点的channel交换信息(pub/sub)
- 通过_sentinel_:hello 频道交互
- 交互对节点和自身信息
每1秒:每个sentinel对其他sentinel和redis执行ping
- 心跳检查
5、主观下线和客观下线
sentinel monitor <masterName> <ip> <port> <quorunm>
quorunm:配置几个sentinel节点主观认定redis下线时,认定为客观下线
6、领导者选举
- 只有一个sentinel节点完成故障转移
通过sentinel is master-down-by-addr命令都希望成为leader
- 每个做主观下线的sentinel节点向其他sentinel节点发送命令,要求将它设置为领导者
- 收到命令的sentinel节点如果没有同意通过其他Sentinel节点发送的命令,则同意该请求,否则拒绝
- 如果该Sentinel节点发现自己的票数已经超过Sentinel集合半数且超过quorum,将成为领导者
7、节点运维
节点下线:
1、主节点下线:手动故障转移 sentinel failover <masterName>
2、从节点下线:清理一些数据文件,但是要考虑读写分离的情况
3、sentinel下线:清理一些数据文件
节点上线:
1、主节点:sentinel failover进行替换
2、从节点:配置slaveof即可
3、sentinel节点:配置启动