redis

一、hightlights

1、performance description

   2、data type

  3、persistence

(1)When snapshot, it will block client request;

(2)“aof” is not used to master-slave synchronization.

二、installation

1、Decompression redisà” redis-3.2.10.tar.gz”

2、“tar zxvf redis-3.2.10.tar.gz”

3、“make”

4、“make PREFIX=/usr/common/redis/ install”

三、configuration

1、environment variables

1 export JAVA_HOME=/usr/local/java/jdk1.8.0_261
2 export CLASSPATH=.:$JAVA_HOME/lib.tool.jar:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar
3 export NODE_HOME=/usr/local/node
4 export REDIS_HOME=/usr/local/redis
5 export PATH=$JAVA_HOME/bin:$NODE_HOME/bin:$REDIS_HOME/bin:$PATH

2、redis.conf

(1)” cp redis.conf /usr/local/redis/conf/”

(2)bind ip address(for remote login)

(3)protected mode no(for remote login)

(4)“requirepass password”(for remote login)

(5)“rename-command command newname”

3、setup auto startup

(1)startup script(utils/redis_init_script)

 1 #!/bin/sh
 2 # chkconfig:   2345 90 10
 3 # Simple Redis init.d script conceived to work on Linux systems
 4 # as it does use of the /proc filesystem.
 5  
 6 #redis服务器监听的端口
 7 REDISPORT=6379
 8  
 9 #服务端所处位置
10 EXEC=/usr/local/bin/redis-server
11  
12 #客户端位置
13 CLIEXEC=/usr/local/bin/redis-cli
14  
15 #redis的PID文件位置，需要修改
16 PIDFILE=/var/run/redis_${REDISPORT}.pid
17  
18 #redis的配置文件位置，需将${REDISPORT}修改为文件名
19 CONF="/etc/redis/${REDISPORT}.conf"
20  
21 case "$1" in
22     start)
23         if [ -f $PIDFILE ]
24         then
25                 echo "$PIDFILE exists, process is already running or crashed"
26         else
27                 echo "Starting Redis server..."
28                 $EXEC $CONF
29         fi
30         ;;
31     stop)
32         if [ ! -f $PIDFILE ]
33         then
34                 echo "$PIDFILE does not exist, process is not running"
35         else
36                 PID=$(cat $PIDFILE)
37                 echo "Stopping ..."
38                 $CLIEXEC -p $REDISPORT shutdown
39                 while [ -x /proc/${PID} ]
40                 do
41                     echo "Waiting for Redis to shutdown ..."
42                     sleep 1
43                 done
44                 echo "Redis stopped"
45         fi
46         ;;
47     *)
48         echo "Please use start or stop as first argument"
49         ;;
50 esac

(2)setup

 1 mkdir /etc/redis
 2 cp redis.conf /etc/redis/6379.conf
 3 # 将启动脚本复制到/etc/init.d目录下，本例将启动脚本命名为redisd（通常都以d结尾表示是# 后台自启动服务）
 4 cp redis_init_script /etc/init.d/redisd
 5 # 设置为开机自启动服务器
 6 chkconfig redisd on
 7 # 打开服务
 8 service redisd start
 9 # 关闭服务
10 service redisd stop

4、warning

(1)“WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.”

1 “vim /etc/sysctl.conf”
2 “net.core.somaxconn= 1024”
3 “sysctl -p”

(2)” WARNING overcommit_memory is set to 0! Background save may fail under low memory condition.”

1 “vim /etc/sysctl.conf”prevent save failure because memory is not enough
2 “vm.overcommit_memory=1”
3 “sysctl –p”

四、operations

1、start-->"redis-server /usr/common/redis/conf/redis.conf &"("&" daemonize)

2、client

(1)login-->"redis-cli –h ip –p port -a"

(2)“auth password” or “redis-cli –a password”

 3、basic software description

(1)“redis-server”àthe daemon boot program of server

(2)“redis-cli”àcommand line operation toolàtelnet

(3)Redis Desktop Manageràgraphical interfaces

(4)“redis-benchmark”àperformance testing tool,test the write and read capacity on your system.

(5)“redis-check-aof”àcheck update log appendonly.aof,

(6)“redis-check-dump”àcheck rdb files in the local dump.

4、basic operation

(1)basic

1 “set key value”
2 “get key”
3 “del key”
4 “nil”-->NULL
5 “exists key”
6 “keys *”-->get all

(2)dump

1 There are 16 default databases in the redis
2 Identifier-->0-15
3 “select id”-->switch between dumps

(3)telnet

1 “telnet ip port”
2 “set key value”
3 “get key”

 (4)NC

1 “echo “set key value”|nc ip port”
2 “echo “get key”|nc ip port”

(5)help

1 “redis-server - -help”
2 “redis-cli - -help”
3 “help command”
4 “help @group”view all commands in the group
5 “help ”+Tabergodic

(6)String

 1 “set key value”
 2 “get key”
 3 “set counter number”
 4 “incr counter”
 5 “incrby counter number”
 6 “decr counter”
 7 “decrby counter number”
 8 “getset key value”first get, then set
 9 “mset key1 value1 key2 value2 …”
10 “mget key1 key2 …”
11 “append key value”
12 “strlen key”
13 “substr key number1 number2”

(7)List

1 “lpush key value1 value2 …”
2 “llen key”
3 “lrange key number1 number2”
4 “lpop key”
5 “lrem key count value”
6 “linsert key”

(8)Set

1 “sadd key value1 value2 …”
2 “smembers key”
3 “srem key value1 value2 …”
4 “spop key”
5 “del key”

(9)Sorted Set

1 “zadd key score1 value1 score2 value2 …”
2 “zrange key”
3 “zscore key value”
4 “zrangebyscore key number1 number2”
5 “zrevrange key number1 number2”
6 “number:0 -1”all
7 “zremrangebyscore key number1 number2”

(10)Hash

1 “hset key field value”
2 “hvals key”
3 “hgetall key”

5、multiple instances

(1)“mkdir -p 6380/data”

(2)“mkdir -p 6381/data”

(3)“cp conf/redis.conf 6380/data/”

(4)“cp conf/redis.conf 6381/data/”

(5)“vim 6380/data/redis.conf”

1 “protected-mode no”
2 “port 6380”
3 “dir /usr/local/redis/6380/data/”
4 “requirepass liuhonglei545366”
5 “appendonly yes”
6 “pidfile /usr/local/redis/6380/data/redis_6380.pid”
7 “bind 127.0.0.1 192.168.1.101”

(6)“vim 6381/data/redis.conf”

(7)“vim /etc/sysconfig/iptables”à6380,6381

(8)“service iptables restart”

(9)“redis-server /usr/common/redis/6380/data/redis.conf &”

(10)“redis-server /usr/common/redis/6381/data/redis.conf &”

(11)“netstat -lntup|grep 63”

(12)“redis-cli -h 192.168.182.131 -p 6380 -a liuhonglei545366”

(13)“redis-cli -p 6381 -a liuhonglei545366”

(14)“ps -ef|grep redis”

(15)Modify parameters online

1 “config set key value”
2 “config get key”
3 “config get *”

五、Master-Slave replication

1、description

(1)Master-slave synchronization characteristics

(2)One master, more slaves

(3)More slaves can connect master and themselves

(4)Master-slave non-block, master can handle request of client continually when replicate data

(5)Improve the scalability of system

  2、Principle of synchronization

Redis主从同步原理-SYNC

和MySQL主从复制的原因一样，Redis虽然读取写入的速度都特别快，但是也会产生读压力特别大的情况。为了分担读压力，Redis支持主从复制，Redis的主从结构可以采用一主多从或者级联结构，下图为级联结构。

 Redis主从复制可以根据是否是全量分为全量同步和增量同步。

1、全量同步

　　Redis全量复制一般发生在Slave初始化阶段，这时Slave需要将Master上的所有数据都复制一份。具体步骤如下： 
　　1）从服务器连接主服务器，发送SYNC命令； 
　　2）主服务器接收到SYNC命名后，开始执行BGSAVE命令生成RDB文件并使用缓冲区记录此后执行的所有写命令； 
　　3）主服务器BGSAVE执行完后，向所有从服务器发送快照文件，并在发送期间继续记录被执行的写命令； 
　　4）从服务器收到快照文件后丢弃所有旧数据，载入收到的快照； 
　　5）主服务器快照发送完毕后开始向从服务器发送缓冲区中的写命令； 
　　6）从服务器完成对快照的载入，开始接收命令请求，并执行来自主服务器缓冲区的写命令；

 　　

完成上面几个步骤后就完成了从服务器数据初始化的所有操作，从服务器此时可以接收来自用户的读请求。

2、增量同步

　　Redis增量复制是指Slave初始化后开始正常工作时主服务器发生的写操作同步到从服务器的过程。 

增量复制的过程主要是主服务器每执行一个写命令就会向从服务器发送相同的写命令，从服务器接收并执行收到的写命令。

3、Redis主从同步策略

　　主从刚刚连接的时候，进行全量同步；全同步结束后，进行增量同步。当然，如果有需要，slave 在任何时候都可以发起全量同步。redis 策略是，无论如何，首先会尝试进行增量同步，如不成功，要求从机进行全量同步。

4、其他

Redis 2.8以后提供了PSYNC优化了断线重连的效率 

http://blog.csdn.net/sk199048/article/details/77922589

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

3、procedure

(1)“slaveof 192.168.182.131 6379”

(2)“asterauth liuhonglei545366”

(3)“monitor”

(4)“slave-serve-stale-data yes”

(5)“slave-read-only yes”

(6)“repl-ping-slave-period 10”

(7)“repl-timeout 60”

(8)“repl-backlog-size 1mb”

(9)“repl-backlog-ttl 3600”

(10)“slave-priority 100”

4、info

  1 # Server
  2 redis_version:3.2.10
  3 redis_git_sha1:00000000
  4 redis_git_dirty:0
  5 redis_build_id:ba3a18d88825a41a
  6 redis_mode:standalone
  7 os:Linux 2.6.32-431.el6.i686 i686
  8 arch_bits:32
  9 multiplexing_api:epoll
 10 gcc_version:4.4.7
 11 process_id:1765
 12 run_id:6e77d7fb718f13eada317553db87f90d38dcb43d
 13 tcp_port:6381
 14 uptime_in_seconds:3276
 15 uptime_in_days:0
 16 hz:10
 17 lru_clock:461154
 18 executable:/root/redis-server
 19 config_file:/usr/common/redis/6381/data/redis.conf
 20 
 21 # Clients
 22 connected_clients:2
 23 client_longest_output_list:0
 24 client_biggest_input_buf:0
 25 blocked_clients:0
 26 
 27 # Memory
 28 used_memory:667296
 29 used_memory_human:651.66K
 30 used_memory_rss:2396160
 31 used_memory_rss_human:2.29M
 32 used_memory_peak:715768
 33 used_memory_peak_human:698.99K
 34 total_system_memory:523042816
 35 total_system_memory_human:498.81M
 36 used_memory_lua:25600
 37 used_memory_lua_human:25.00K
 38 maxmemory:3221225472
 39 maxmemory_human:3.00G
 40 maxmemory_policy:noeviction
 41 mem_fragmentation_ratio:3.59
 42 mem_allocator:jemalloc-4.0.3
 43 
 44 # Persistence
 45 loading:0
 46 rdb_changes_since_last_save:0
 47 rdb_bgsave_in_progress:0
 48 rdb_last_save_time:1510409120
 49 rdb_last_bgsave_status:ok
 50 rdb_last_bgsave_time_sec:0
 51 rdb_current_bgsave_time_sec:-1
 52 aof_enabled:1
 53 aof_rewrite_in_progress:0
 54 aof_rewrite_scheduled:0
 55 aof_last_rewrite_time_sec:0
 56 aof_current_rewrite_time_sec:-1
 57 aof_last_bgrewrite_status:ok
 58 aof_last_write_status:ok
 59 aof_current_size:204
 60 aof_base_size:82
 61 aof_pending_rewrite:0
 62 aof_buffer_length:0
 63 aof_rewrite_buffer_length:0
 64 aof_pending_bio_fsync:0
 65 aof_delayed_fsync:0
 66 
 67 # Stats
 68 total_connections_received:1
 69 total_commands_processed:323
 70 instantaneous_ops_per_sec:0
 71 total_net_input_bytes:4737
 72 total_net_output_bytes:126607
 73 instantaneous_input_kbps:0.00
 74 instantaneous_output_kbps:0.04
 75 rejected_connections:0
 76 sync_full:0
 77 sync_partial_ok:0
 78 sync_partial_err:0
 79 expired_keys:0
 80 evicted_keys:0
 81 keyspace_hits:0
 82 keyspace_misses:0
 83 pubsub_channels:0
 84 pubsub_patterns:0
 85 latest_fork_usec:303
 86 migrate_cached_sockets:0
 87 
 88 # Replication
 89 role:slave
 90 master_host:192.168.182.131
 91 master_port:6379
 92 master_link_status:up
 93 master_last_io_seconds_ago:10
 94 master_sync_in_progress:0
 95 slave_repl_offset:4603
 96 slave_priority:100
 97 slave_read_only:1
 98 connected_slaves:0
 99 master_repl_offset:0
100 repl_backlog_active:0
101 repl_backlog_size:1048576
102 repl_backlog_first_byte_offset:0
103 repl_backlog_histlen:0
104 
105 # CPU
106 used_cpu_sys:55.38
107 used_cpu_user:1.87
108 used_cpu_sys_children:0.02
109 used_cpu_user_children:0.01
110 
111 # Cluster
112 cluster_enabled:0
113 
114 # Keyspace
115 db0:keys=2,expires=0,avg_ttl=0
116 db1:keys=1,expires=0,avg_ttl=0

5、DNS

1 “vim /etc/hosts”
2 “192.168.182.131 www.xxx.com”
3 “vim /usr/local/redis/6380/data/redis.conf”
4 “slaveof www.xxx.com 6379”

六、Advanced features

1、Publish/subscribe

(1)“subscribe channel”

(2)“publish channel message”

(3)Batch publish/subscribe

(3.1)“psubscribe channel*”

(3.2)“publish channel1”

(3.3)“publish channel2”

2、data expire

(1)“ttl key”

(2)“exists key”

(3)“del key”

(4)“expire key seconds”

(5)“expireat key timestamp”(Timestamp-->” date +%s -d "2017-11-11 23:48:00")

(6)“set key value [EX seconds] [PX miliseconds]”

3、Transaction

(1)Simple transaction

(1.1)“setnx key value”-->If key exists, do not execute. If not, set the value.

(1.2)“multi”-->”set key value1”-->”set key value2” …-->”exec”

4、persistence

(1)Snapshottingàdump.rdb

(1.1)“save”-->save snapshot in the primary thread, block all requests of client, not recommended

(1.2)Warning: every time of snapshot persistence will write all datum in the memory into disk holonomic once, not replicate changed data only, when io often, it serious impacts on performance

(2)Append-only file-->aof

1 “appendonly.aof”
2 “appendonly yes”
3 “appendfsync always”
4 “appendfsync everysec”
5 “appendfsync no”
6 “bgrewriteaof”
7 “auto-aof-rewrite-percentage 100”
8 “auto-aof-rewrite-min-size 64mb”

七、summarize

 1、Memory Optimize

(1)Hash

(1.1)“hash-max-zipmap-entries 64”

(1.2)“hash-max-zipmap-value 512”

(2)List

(2.1)“list-max-ziplist-value 64”

(2.2)“list-max-ziplist-entries 512”

3、Cluster

 (1)details

(1)所有的redis节点彼此互联(PING-PONG机制),内部使用二进制协议优化传输速度和带宽.

(2)节点的fail是通过集群中超过半数的节点检测失效时才生效.

(3)客户端与redis节点直连,不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可

(4)redis-cluster把所有的物理节点映射到[0-16383]slot上,cluster 负责维护node<->slot<->value

Redis 集群中内置了 16384 个哈希槽，当需要在 Redis 集群中放置一个 key-value 时，redis 先对 key 使用 crc16 算法算出一个结果，然后把结果对 16384 求余数，这样每个 key 都会对应一个编号在 0-16383 之间的哈希槽，redis 会根据节点数量大致均等的将哈希槽映射到不同的节点

 (2)building-->there should be at least three nodes in the cluster, so needs 6 servers

(3)redis.conf

1 Cp redis.conf 6379/
2 Vim 6379/redis.conf
3 “daemonize yes”
4 “pidfile /var/run/redis_6379.pid”
5 “#save 900 1 #save 300 10 #save 60 10000”
6 “maxmemory 100m”

4、Master slave replication(read write separation)

 (1)Advantage

(1.1)Avert single point of failure

(1.2)Build read write separation,  meet the need of read more and write less

(1.3)redis.conf

1 Cp 6379 6380 –r
2 Vim ./6380/redis.conf
3 “:%s/6379/6380/g”
4 Vim ./6381/redis.conf
5 “:%s/6379/6381/g”
6 Slaveof <masterip> <masterport>

(1.4)redis-cli-->"Slaveof <masterip> <masterport>"

(1.5)view information-->"INFO replication"

......

 

posted @ 2020-08-01 16:11  遥~  阅读(154)  评论(0)  编辑  收藏  举报