redis设计与实现 note

redis设计与实现

https://github.com/CyC2018/CS-Notes/blob/master/docs/notes/Redis.md

1. 数据结构

跳跃表

与红黑树等平衡树相比，跳跃表具有以下优点：

1. 插入速度非常快速，因为不需要进行旋转等操作来维护平衡性；
2. 更容易实现；
3. 支持无锁操作。

哈希表

有两个hash表指针，每次添加的时候，如果正在rehash，则进行n步rehash操作 [https://github.com/linyiqun/Redis-Code/blob/master/struct/dict.c]

/* Performs N steps of incremental rehashing. Returns 1 if there are still
 * keys to move from the old to the new hash table, otherwise 0 is returned.
 * Note that a rehashing step consists in moving a bucket (that may have more
 * than one key as we use chaining) from the old to the new hash table. */
/* hash重定位，主要从旧的表映射到新表中
 * 如果返回1说明旧的表中还存在key迁移到新表中，0代表没有 */
int dictRehash(dict *d, int n) {
    if (!dictIsRehashing(d)) return 0;
	
	/* 根据参数分n步多次循环操作 */
    while(n--) {
        dictEntry *de, *nextde;

        /* Check if we already rehashed the whole table... */
        if (d->ht[0].used == 0) {
            zfree(d->ht[0].table);
            d->ht[0] = d->ht[1];
            _dictReset(&d->ht[1]);
            d->rehashidx = -1;
            return 0;
        }

        /* Note that rehashidx can't overflow as we are sure there are more
         * elements because ht[0].used != 0 */
        assert(d->ht[0].size > (unsigned long)d->rehashidx);
        while(d->ht[0].table[d->rehashidx] == NULL) d->rehashidx++;
        de = d->ht[0].table[d->rehashidx];
        /* Move all the keys in this bucket from the old to the new hash HT */
        /* 移动的关键操作 */
        while(de) {
            unsigned int h;

            nextde = de->next;
            /* Get the index in the new hash table */
            h = dictHashKey(d, de->key) & d->ht[1].sizemask;
            de->next = d->ht[1].table[h];
            d->ht[1].table[h] = de;
            d->ht[0].used--;
            d->ht[1].used++;
            de = nextde;
        }
        d->ht[0].table[d->rehashidx] = NULL;
        d->rehashidx++;
    }
    return 1;
}

/* Low level add. This function adds the entry but instead of setting
 * a value returns the dictEntry structure to the user, that will make
 * sure to fill the value field as he wishes.
 *
 * This function is also directly exposed to user API to be called
 * mainly in order to store non-pointers inside the hash value, example:
 *
 * entry = dictAddRaw(dict,mykey);
 * if (entry != NULL) dictSetSignedIntegerVal(entry,1000);
 *
 * Return values:
 *
 * If key already exists NULL is returned.
 * If key was added, the hash entry is returned to be manipulated by the caller.
 */
/* 添加一个指定key值的Entry */
dictEntry *dictAddRaw(dict *d, void *key)
{
    int index;
    dictEntry *entry;
    dictht *ht;

    if (dictIsRehashing(d)) _dictRehashStep(d);

    /* Get the index of the new element, or -1 if
     * the element already exists. */
    /* 如果指定的key已经存在，则直接返回NULL说明添加失败 */
    if ((index = _dictKeyIndex(d, key)) == -1)
        return NULL;

    /* Allocate the memory and store the new entry */
    ht = dictIsRehashing(d) ? &d->ht[1] : &d->ht[0];
    entry = zmalloc(sizeof(*entry));
    entry->next = ht->table[index];
    ht->table[index] = entry;
    ht->used++;

    /* Set the hash entry fields. */
    dictSetKey(d, entry, key);
    return entry;
}

2. 过期键删除策略

惰性删除 + 定期删除

typedef struct redisDb {
    dict *dict;                 /* The keyspace for this DB */
    dict *expires;              /* Timeout of keys with a timeout set */
    dict *blocking_keys;        /* Keys with clients waiting for data (BLPOP) */
    dict *ready_keys;           /* Blocked keys that received a PUSH */
    dict *watched_keys;         /* WATCHED keys for MULTI/EXEC CAS */
    int id;
    long long avg_ttl;          /* Average TTL, just for stats */
} redisDb;

惰性删除只在碰到过期键时进行删除操作，定期删除每隔一段时间主动查找并删除过期键，redisDB结构内的expires字典保存了所有键的过期时间，称为过期字典

3. 持久化

RDB
将某个时间点的所有数据都存放到硬盘上。
可以将快照复制到其它服务器从而创建具有相同数据的服务器副本。
如果系统发生故障，将会丢失最后一次创建快照之后的数据。
如果数据量很大，保存快照的时间会很长。

AOF
将写命令添加到 AOF文件（Append Only File）末尾。
使用 AOF 持久化需要设置同步选项，从而确保写命令什么时候会同步到磁盘文件上。
Redis提供了 AOF 重写特性，能够减小 AOF 文件大小。
使用子进程而非子线程可以避免锁。
同时追加命令至AOF缓冲区，AOF重写缓冲区。

分布式锁

消息队列

redis应用场景

http://www.scienjus.com/redis-use-case/