rocksdb wiki文档阅读笔记

 

由于是英文文档，不做笔记过一阵就忘了，现在把关键点记录到这，开发的时候使用。

具体wiki地址：https://github.com/facebook/rocksdb/wiki

1）Column Faimilies 意味着可以将写入的key进行逻辑分组存储，不影响读取，若未指定则使用default；

2）MemTable 在内存中存储的数据，当达到一定的条件时会flush到文件；

• memtable_factory: The factory object of memtable. By specifying factory object user can change the underlying implementation of memtable, and provide implementation specific options.
• write_buffer_size: Size of a single memtable.
• db_write_buffer_size: Total size of memtables across column families. This can be used to manage the total memory used by memtables.
• write_buffer_manager: Instead of specifying a total size of memtables, user can provide their own write buffer manager to control the overall memtable memory usage. Overrides db_write_buffer_size.
• max_write_buffer_number: The maximum number of memtables build up in memory, before they flush to SST files.

3）Write Ahead Log 每次写操作都会记录这个操作，主要用于故障、重启等时恢复MemTable的状态（和redis机制有点像）；

DBOptions::wal_dir

DBOptions::wal_dir sets the directory where RocksDB stores write-ahead log files, which allows WALs to be stored in a separate directory from the actual data.

DBOptions::WAL_ttl_seconds, DBOptions::WAL_size_limit_MB

These two fields affect how quickly archived WALs will be deleted. Nonzero values indicate the time and disk space threshold to trigger archived WAL deletion. See options.h for detailed explanation.

DBOptions::max_total_wal_size

In order to limit the size of WALs, RocksDB uses DBOptions::max_total_wal_size as the trigger of column family flush. Once WALs exceed this size, RocksDB will start forcing the flush of column families to allow deletion of some oldest WALs. This config can be useful when column families are updated at non-uniform frequencies. If there's no size limit, users may need to keep really old WALs when the infrequently-updated column families hasn't flushed for a while.

DBOptions::avoid_flush_during_recovery

This config is self explanatory.

DBOptions::manual_wal_flush

DBOptions::manual_wal_flush determines whether WAL flush will be automatic after every write or purely manual (user must invoke FlushWAL to trigger a WAL flush).

DBOptions::wal_filter

Through DBOptions::wal_filter, users can provide a filter object to be invoked while processing WALs during recovery. Note: Not supported in ROCKSDB_LITE mode

WriteOptions::disableWAL

WriteOptions::disableWAL is useful when users rely on other logging or don't care about data loss.

 

4）Cache 相当于在MemTable之上又做了一层cache（有LRUCache和ClockCache两类），其实索引和过滤相关也可以cache，具体可以看文档，可以更好的控制内存使用；

#include "rocksdb/cache.h"
rocksdb::BlockBasedTableOptions table_options;
table_options.block_cache = rocksdb::NewLRUCache(100 * 1048576); // 100MB uncompressed cache

rocksdb::Options options;
options.table_factory.reset(rocksdb::NewBlockBasedTableFactory(table_options));
ocksdb::DB* db;
rocksdb::DB::Open(options, name, &db);
... use the db ...
delete db

 

5）支持前缀查询，默认即使这样的行为，无需其它特定设置：

https://github.com/facebook/rocksdb/wiki/Prefix-Seek-API-Changes

 

6）TTL机制 可对插入数据设置过期时间，但是使用有限制；

必须要是用下面的API:

static Status DBWithTTL::Open(const Options& options, const std::string& name, StackableDB** dbptr, int32_t ttl = 0, bool read_only = false);



7）讲解了两阶段提交的实现：

https://github.com/facebook/rocksdb/wiki/Two-Phase-Commit-Implementation