levelDB, TokuDB, BDB等kv存储引擎性能对比——wiredtree, wiredLSM，LMDB读写很强啊

在：http://www.lmdb.tech/bench/inmem/

2. Small Data Set

Using the laptop we generate a database with 20 million records. The records have 16 byte keys and 100 byte values so the resulting database should be about 2.2GB in size. After the data is loaded a "readwhilewriting" test is run using 4 reader threads and one writer. All of the threads operate on randomly selected records in the database. The writer performs updates to existing records; no records are added or deleted so the DB size should not change much during the test.

The tests in this section and in Section 3 are all run on a tmpfs, just like the RocksDB report. I.e., all of the data is stored only in RAM. Additional tests using an SSD follow in Section 4.

The pertinent results are tabulated here and expanded on in the following sections.

Engine	Load Time			Overhead	Load Size	Writes/Sec	Reads/Sec	Run Time			Final Size	CPU%	Process Size
	Wall	User	Sys		KB			Wall	User	Sys	KB		KB
LevelDB	00:34.70	00:44.72	00:06.70	1.4818443804	2246004	10232	26678	00:49:58.73	01:31:48.62	00:52:50.95	3452388	289%	2138508
Basho	00:40.41	01:24.39	00:17.82	2.5293244246	2368768	10232	68418	00:19:32.94	01:14:10.04	00:01:19.19	2612436	386%	6775376
BerkeleyDB	02:12.61	01:58.92	00:13.57	0.9990950909	5844376	9565	86202	00:15:28.44	00:42:07.97	00:17:27.49	5839912	385%	3040716
Hyper	00:38.78	00:49.88	00:06.43	1.4520371325	2246448	10208	138393	00:09:38.39	00:35:06.12	00:02:06.18	2292632	385%	2700088
LMDB	00:10.55	00:08.15	00:02.37	0.9971563981	2516192	10224	1449709	00:00:55.46	00:03:37.63	00:00:01.67	2547968	395%	2550408
RocksDB	00:21.54	00:34.70	00:05.99	1.8890436397	2256032	10233	91544	00:14:37.74	00:54:06.84	00:02:38.04	3181764	387%	6713852
TokuDB	01:45.12	01:41.58	00:47.37	1.4169520548	2726168	9881	109682	00:12:12.91	00:37:41.45	00:07:10.03	3920784	367%	5429056
WiredLSM	01:10.93	02:35.55	00:18.62	2.4555195263	2492440	10230	179617	00:07:26.24	00:28:55.85	00:00:07.76	2948988	390%	3205396
WiredBtree	00:17.79	00:15.68	00:02.09	0.9988757729	2381876	10021	752078	00:01:53.46	00:06:36.98	00:00:14.78	4752568	362%	3415468

3. Larger Data Set

These tests use 100 million records and are run on the 16 core server. Aside from the data set size things are much the same. Here are the tabular results:

Engine	Load Time			Overhead	Load Size	Writes/Sec	Reads/Sec	Run Time			Final Size	CPU%	Process Size
	Wall	User	Sys		KB			Wall	User	Sys	KB		KB
LevelDB	03:06.75	04:41.26	00:42.87	1.7356358768	11273396	9184	7594	01:00:02.00	01:22:11.46	01:52:10.46	13734168	323%	3284192
Basho	04:22.96	11:09.24	02:18.93	3.0733571646	11449492	10211	80135	01:00:23.00	14:32:23.67	00:11:49.40	13841220	1464%	19257796
BerkeleyDB	14:59.45	13:34.30	01:25.15	1	28381956	3378	55066	01:00:02.00	03:02:00.69	12:42:39.63	28387880	1573%	14756768
Hyper	03:43.61	05:41.14	00:39.02	1.7001028577	11280092	10231	11673	01:00:04.00	01:59:42.09	01:53:24.27	15149416	387%	6332460
LMDB	01:04.15	00:52.31	00:11.82	0.9996882307	12605332	10230	2486800	00:11:14.14	02:47:58.57	00:00:10.06	12627692	1598%	12605788
RocksDB	02:28.66	03:59.92	00:30.97	1.8222117584	11289688	10232	129397	01:00:22.00	12:08:05.94	02:51:58.54	12777708	1490%	18599544
TokuDB	07:44.10	09:17.31	02:54.82	1.5775263952	12665136	4601	70208	01:00:15.00	03:02:37.44	11:21:45.00	15328956	1434%	23315964
WiredLSM	07:10.50	19:25.80	02:31.10	3.0590011614	12254620	10194	278415	01:00:05.00	15:51:04.17	00:02:09.76	16016296	1586%	17723992
WiredBtree	02:07.49	01:49.52	00:17.97	1	11932620	10145	1320939	00:20:58.10	05:06:13.60	00:05:14.87	23865368	1560%	20743232

 

看这个pdf里有对kv存储的架构和底层原理的详细介绍：

https://daim.idi.ntnu.no/masteroppgaver/008/8885/masteroppgave.pdf