随笔- 290 文章- 0 评论- 433 阅读- 334万

Redis碎碎念

1. 添加slave时的主从日志

slave的日志

64798:S 12 Jan 13:24:22.483 * SLAVE OF 127.0.0.1:6379 enabled (user request from 'id=2 addr=127.0.0.1:51366 fd=6 name= age=864 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=slaveof')
64798:S 12 Jan 13:24:23.421 * Connecting to MASTER 127.0.0.1:6379
64798:S 12 Jan 13:24:23.422 * MASTER <-> SLAVE sync started
64798:S 12 Jan 13:24:23.422 * Non blocking connect for SYNC fired the event.
64798:S 12 Jan 13:24:23.422 * Master replied to PING, replication can continue...
64798:S 12 Jan 13:24:23.423 * Partial resynchronization not possible (no cached master)
64798:S 12 Jan 13:24:23.426 * Full resync from master: b6c0d819dc23ff2148ece7042b54cd3ede547dc9:71
64798:S 12 Jan 13:24:24.724 * MASTER <-> SLAVE sync: receiving 74144808 bytes from master
64798:S 12 Jan 13:24:25.059 * MASTER <-> SLAVE sync: Flushing old data
64798:S 12 Jan 13:24:25.870 * MASTER <-> SLAVE sync: Loading DB in memory
64798:S 12 Jan 13:24:27.562 * MASTER <-> SLAVE sync: Finished with success

master的日志

64796:M 12 Jan 13:24:23.423 * Slave 127.0.0.1:6380 asks for synchronization
64796:M 12 Jan 13:24:23.423 * Full resync requested by slave 127.0.0.1:6380
64796:M 12 Jan 13:24:23.423 * Starting BGSAVE for SYNC with target: disk
64796:M 12 Jan 13:24:23.425 * Background saving started by pid 65161
65161:C 12 Jan 13:24:24.642 * DB saved on disk
65161:C 12 Jan 13:24:24.645 * RDB: 0 MB of memory used by copy-on-write
64796:M 12 Jan 13:24:24.723 * Background saving terminated with success
64796:M 12 Jan 13:24:25.064 * Synchronization with slave 127.0.0.1:6380 succeeded

2. 安装redis-migrate-tool依赖包

yum install autoconf automake libtool bzip2

3. 编译过程中报错

zmalloc.h:50:10: fatal error: jemalloc/jemalloc.h: No such file or directory
 #include <jemalloc/jemalloc.h>
          ^~~~~~~~~~~~~~~~~~~~~

解决方法：

make distclean

4. redis python 客户端支持的选项

https://github.com/redis/redis-py/blob/master/redis/client.py

5. Python redis 客户端 scan_iter 的实现原理

https://github.com/redis/redis-py/blob/1cbb97ecd7a0bf87aec332a257b589919385d90d/redis/cluster.py#L452

    def scan_iter(
        self,
        match: Optional[PatternT] = None,
        count: Optional[int] = None,
        _type: Optional[str] = None,
        **kwargs,
    ) -> Iterator:

        # Do the first query with cursor=0 for all nodes
        # 第一次 scan 获取的 cursors 的结果如下：{'10.0.0.198:6379': 1, '10.0.0.198:6380': 0, '10.0.0.198:6381': 6}，其中，1，0，6分别是三个节点游标的位置。
        # data 是返回的数据
        cursors, data = self.scan(match=match, count=count, _type=_type, **kwargs)
        yield from data
        # 过滤掉 cursor 等于 0 的节点
        cursors = {name: cursor for name, cursor in cursors.items() if cursor != 0}
        if cursors:
            # Get nodes by name
            # nodes的内容如下：{'10.0.0.198:6379': [host=10.0.0.198,port=6379,name=10.0.0.198:6379,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=10.0.0.198,port=6379,db=0>>>], '10.0.0.198:6380': [host=10.0.0.198,port=6380,name=10.0.0.198:6380,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=10.0.0.198,port=6380,db=0>>>], '10.0.0.198:6381': [host=10.0.0.198,port=6381,name=10.0.0.198:6381,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=10.0.0.198,port=6381,db=0>>>]}
            nodes = {name: self.get_node(node_name=name) for name in cursors.keys()}

            # Iterate over each node till its cursor is 0
            kwargs.pop("target_nodes", None)
            while cursors:
                # 迭代所有 cursor 不等于 0 的节点，所以，从这里可以看出，python客户端不是一个节点迭代完才迭代另外一个节点，而是几个节点同时进行的。
                for name, cursor in cursors.items():
                    cur, data = self.scan(
                        cursor=cursor,
                        match=match,
                        count=count,
                        _type=_type,
                        target_nodes=nodes[name],
                        **kwargs,
                    )
                    yield from data
                    # 把节点对应的游标存储在 cursors 中，继续迭代
                    cursors[name] = cur[name]

                cursors = {
                    name: cursor for name, cursor in cursors.items() if cursor != 0
                }