linux搭建redis分布式伪集群

因为服务器有限,所以我准备在一台服务器上搭建redis分布式集群, 其实多台服务器部署集群也是类似的方式

redis集群准备三主节点三从节点, 7001-7006端口

1.安装redis

下载redis

wget https://download.redis.io/releases/redis-6.2.6.tar.gz

解压redis

tar -zxvf redis-6.2.6.tar.gz

进行安装redis

[root@hecs-406672 /]# cd /opt/redis-6.2.6
[root@hecs-406672 redis-6.2.6]# make install

出现了src目录就表示基本安装完成了

Hint: It's a good idea to run 'make test' ;)

    INSTALL redis-server
    INSTALL redis-benchmark
    INSTALL redis-cli
make[1]: Leaving directory '/opt/redis-6.2.6/src'
[root@hecs-406672 redis-6.2.6]# 

2. 配置redis集群文件

创建redis-cluster并复制redis.conf

[root@hecs-406672 opt]# mkdir redis-cluster
[root@hecs-406672 opt]# cd redis-cluster/
[root@hecs-406672 redis-cluster]# cp ../redis-6.2.6/redis.conf redis.conf
[root@hecs-406672 redis-cluster]# ls
redis.conf
[root@hecs-406672 redis-cluster]# 

编辑redis.conf文件

# 绑定访问ip信息
bind 0.0.0.0

# # 修改对应的端口
port 7001

#修改为后台启动
daemonize yes

#关闭保护模式
protected-mode no

#启动AOF文件
appendonly yes

#指定数据文件存放位置，必须要指定不同的目录位置，不然会丢失数据
dir /usr/local/redis-cluster/redis-7001

# 日志文件名
logfile "redis-7001.log"

# .pid文件初始化
pidfile /var/run/redis_7001.pid

# 启动集群模式
cluster-enabled yes

# 节点离线的超时时间
cluster-node-timeout 15000
  
# 注释打开并修改node节点 
cluster-config-file nodes-7001.conf

#如果要设置密码需要增加如下配置：
#设置redis访问密码
requirepass 123456

#设置集群节点间访问密码，跟上面一致
masterauth 123456

将redis-cluster下的redis.conf复制6份并修改其中7001的字样对应自己的名称

# 绑定访问ip信息
bind 0.0.0.0

# # 修改对应的端口
port 7002

#修改为后台启动
daemonize yes

#关闭保护模式
protected-mode no

#启动AOF文件
appendonly yes

#指定数据文件存放位置，必须要指定不同的目录位置，不然会丢失数据
dir /usr/local/redis-cluster/redis-7002

# 日志文件名
logfile "redis-7002.log"

# .pid文件初始化
pidfile /var/run/redis_7002.pid

# 启动集群模式
cluster-enabled yes

# 节点离线的超时时间
cluster-node-timeout 15000
  
# 注释打开并修改node节点 
cluster-config-file nodes-7002.conf

#如果要设置密码需要增加如下配置：
#设置redis访问密码
requirepass 123456

#设置集群节点间访问密码，跟上面一致
masterauth 123456

redis-cluster下的配置

[root@hecs-406672 redis-cluster]# ls
redis-7001.conf  redis-7002.conf  redis-7003.conf  redis-7004.conf  redis-7005.conf  redis-7006.conf  redis.conf
[root@hecs-406672 redis-cluster]# 

创建redis数据存储目录

mkdir /usr/local/redis-cluster/redis-7001 -p
mkdir /usr/local/redis-cluster/redis-7002 -p
mkdir /usr/local/redis-cluster/redis-7003 -p
mkdir /usr/local/redis-cluster/redis-7004 -p
mkdir /usr/local/redis-cluster/redis-7005 -p
mkdir /usr/local/redis-cluster/redis-7006 -p

然后依次启动redis,切换到之前安装的redis的src目录下

[root@hecs-406672 src]# cd /opt/redis-6.2.6/src/
[root@hecs-406672 src]# ./redis-server /opt/redis-cluster/redis-7001.conf 

  [root@hecs-406672 src]# ./redis-server /opt/redis-cluster/redis-7002.conf
  [root@hecs-406672 src]# ./redis-server /opt/redis-cluster/redis-7003.conf
  [root@hecs-406672 src]# ./redis-server /opt/redis-cluster/redis-7004.conf
  [root@hecs-406672 src]# ./redis-server /opt/redis-cluster/redis-7005.conf
  [root@hecs-406672 src]# ./redis-server /opt/redis-cluster/redis-7006.conf

监控redis状态

[root@hecs-406672 src]# ps -ef|grep redis
root       10367       1  0 20:24 ?        00:00:00 ./redis-server 0.0.0.0:7001 [cluster]
root       10375       1  0 20:25 ?        00:00:00 ./redis-server 0.0.0.0:7002 [cluster]
root       10381       1  0 20:25 ?        00:00:00 ./redis-server 0.0.0.0:7003 [cluster]
root       10387       1  0 20:25 ?        00:00:00 ./redis-server 0.0.0.0:7004 [cluster]
root       10393       1  0 20:25 ?        00:00:00 ./redis-server 0.0.0.0:7005 [cluster]
root       10399       1  0 20:25 ?        00:00:00 ./redis-server 0.0.0.0:7006 [cluster]
root       10406    5832  0 20:27 pts/0    00:00:00 grep --color=auto redis
[root@hecs-406672 src]# 

3. 集群搭建

上面仅仅是搭建了集群，6个节点之间还没有关系，下面通过 节点握手让集群各节点之间，发生关系

如果你的服务器有防火墙或者安全组的话需要开放,不然它们相互之间访问可能会出现问题,不止是7001-7006,还有17001-17006对应的守护进程

[root@hecs-406672 src]# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:17001           0.0.0.0:*               LISTEN      10367/./redis-serve 
tcp        0      0 0.0.0.0:17002           0.0.0.0:*               LISTEN      10375/./redis-serve 
tcp        0      0 0.0.0.0:17003           0.0.0.0:*               LISTEN      10381/./redis-serve 
tcp        0      0 0.0.0.0:17004           0.0.0.0:*               LISTEN      10387/./redis-serve 
tcp        0      0 0.0.0.0:17005           0.0.0.0:*               LISTEN      10393/./redis-serve 
tcp        0      0 0.0.0.0:17006           0.0.0.0:*               LISTEN      10399/./redis-serve 
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1535/sshd           
tcp        0      0 0.0.0.0:7001            0.0.0.0:*               LISTEN      10367/./redis-serve 
tcp        0      0 0.0.0.0:7002            0.0.0.0:*               LISTEN      10375/./redis-serve 
tcp        0      0 0.0.0.0:7003            0.0.0.0:*               LISTEN      10381/./redis-serve 
tcp        0      0 0.0.0.0:7004            0.0.0.0:*               LISTEN      10387/./redis-serve 
tcp        0      0 0.0.0.0:7005            0.0.0.0:*               LISTEN      10393/./redis-serve 
tcp        0      0 0.0.0.0:7006            0.0.0.0:*               LISTEN      10399/./redis-serve 
tcp6       0      0 :::22                   :::*                    LISTEN      1535/sshd           
udp        0      0 127.0.0.1:323           0.0.0.0:*                           763/chronyd         
udp6       0      0 ::1:323                 :::*                                763/chronyd  

集群搭建命令

　　cluster-replicas 1 主从搭配比例，1表示一主一从，2表示一主2从，要使用可以被客户端访问到的ip
登录腾讯云服务器的7001节点以下命令（默认按照顺序自动分配主从，一般顺序是一主一从按照你的命令排列顺序)

redis-cli -a 123456 --cluster create --cluster-replicas 1 127.0.0.1:7001 127.0.0.1:7004 127.0.0.1:7002 127.0.0.1:7005 127.0.0.1:7003 127.0.0.1:7006

[root@hecs-406672 src]# redis-cli -a 123456 --cluster create --cluster-replicas 1 123.60.95.50:7001 123.60.95.50:7004 123.60.95.50:7002 123.60.95.50:7005 123.60.95.50:7003 123.60.95.50:7006
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 123.60.95.50:7003 to 123.60.95.50:7001
Adding replica 123.60.95.50:7006 to 123.60.95.50:7004
Adding replica 123.60.95.50:7005 to 123.60.95.50:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 1b69afc78814de9def0daa19a927ddfecbeb97f8 123.60.95.50:7001
slots:[0-5460] (5461 slots) master
M: 92f1b003e113ea2ab1c7c034c42215d3339f8e40 123.60.95.50:7004
slots:[5461-10922] (5462 slots) master
M: 99869909090eb198ac45ac3455af1e14a5aa097b 123.60.95.50:7002
slots:[10923-16383] (5461 slots) master
S: e2688b2778f2961aa75f2b7f30a66fdb343b4799 123.60.95.50:7005
replicates 99869909090eb198ac45ac3455af1e14a5aa097b
S: 7144359a82d368f51024363bdcafc1ef1a9d27b3 123.60.95.50:7003
replicates 1b69afc78814de9def0daa19a927ddfecbeb97f8
S: bc6d019904433f0b9b8dcea5ee49e5793f59febb 123.60.95.50:7006
replicates 92f1b003e113ea2ab1c7c034c42215d3339f8e40
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 123.60.95.50:7001)
M: 1b69afc78814de9def0daa19a927ddfecbeb97f8 123.60.95.50:7001
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: e2688b2778f2961aa75f2b7f30a66fdb343b4799 123.60.95.50:7005
slots: (0 slots) slave
replicates 99869909090eb198ac45ac3455af1e14a5aa097b
M: 99869909090eb198ac45ac3455af1e14a5aa097b 123.60.95.50:7002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: bc6d019904433f0b9b8dcea5ee49e5793f59febb 123.60.95.50:7006
slots: (0 slots) slave
replicates 92f1b003e113ea2ab1c7c034c42215d3339f8e40
M: 92f1b003e113ea2ab1c7c034c42215d3339f8e40 123.60.95.50:7004
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 7144359a82d368f51024363bdcafc1ef1a9d27b3 123.60.95.50:7003
slots: (0 slots) slave
replicates 1b69afc78814de9def0daa19a927ddfecbeb97f8
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@hecs-406672 src]#

查看集群信息

#登录进redis集群
redis-cli -h 123.60.95.50  -p 7001 -a 123456
#查看集群信息
cluster info
#查看集群节点
cluster nodes

 

[root@hecs-406672 src]# redis-cli -h 123.60.95.50  -p 7001 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
123.60.95.50:7001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:157
cluster_stats_messages_pong_sent:181
cluster_stats_messages_sent:338
cluster_stats_messages_ping_received:176
cluster_stats_messages_pong_received:157
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:338
123.60.95.50:7001> cluster nodes
e2688b2778f2961aa75f2b7f30a66fdb343b4799 123.60.95.50:7005@17005 slave 99869909090eb198ac45ac3455af1e14a5aa097b 0 1670503260000 3 connected
99869909090eb198ac45ac3455af1e14a5aa097b 123.60.95.50:7002@17002 master - 0 1670503257404 3 connected 10923-16383
bc6d019904433f0b9b8dcea5ee49e5793f59febb 123.60.95.50:7006@17006 slave 92f1b003e113ea2ab1c7c034c42215d3339f8e40 0 1670503260415 2 connected
92f1b003e113ea2ab1c7c034c42215d3339f8e40 123.60.95.50:7004@17004 master - 0 1670503259000 2 connected 5461-10922
7144359a82d368f51024363bdcafc1ef1a9d27b3 123.60.95.50:7003@17003 slave 1b69afc78814de9def0daa19a927ddfecbeb97f8 0 1670503259000 1 connected
1b69afc78814de9def0daa19a927ddfecbeb97f8 192.168.0.160:7001@17001 myself,master - 0 1670503258000 1 connected 0-5460
123.60.95.50:7001> 

 集群数据分配校验

换一种命令登录redis集群

[root@hecs-406672 src]# ./redis-cli -p 7001 -a 123456 -c
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7001> 

注意：一定要加上-c，不然节点之间是无法自动跳转的,且数据也无法均匀分配到各个节点

测试在集群中设置key-value

127.0.0.1:7001> set str1 1
OK
127.0.0.1:7001> set str2 2
-> Redirected to slot [9547] located at 123.60.95.50:7004
OK
123.60.95.50:7004> 

可以看到key为str2的数据被分配到7004, 可以通过登录redis工具或直接操作命令的方式查看数据是否存在主节点及对应的从节点上

至此redis分布式集群搭建告一段落了