levelDB, TokuDB, BDB等kv存储引擎性能对比——wiredtree, wiredLSM，LMDB读写很强啊
在：http://www.lmdb.tech/bench/inmem/
2. Small Data Set
Using the laptop we generate a database with 20 million records. The records have 16 byte keys and 100 byte values so the resulting database should be about 2.2GB in size. After the data is loaded a "readwhilewriting" test is run using 4 reader threads and one writer. All of the threads operate on randomly selected records in the database. The writer performs updates to existing records; no records are added or deleted so the DB size should not change much during the test.
The tests in this section and in Section 3 are all run on a tmpfs, just like the RocksDB report. I.e., all of the data is stored only in RAM. Additional tests using an SSD follow in Section 4.
The pertinent results are tabulated here and expanded on in the following sections.
| Engine | Load Time | Overhead | Load Size | Writes/Sec | Reads/Sec | Run Time | Final Size | CPU% | Process Size | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Wall | User | Sys | KB | Wall | User | Sys | KB | KB | |||||
| LevelDB | 00:34.70 | 00:44.72 | 00:06.70 | 1.4818443804 | 2246004 | 10232 | 26678 | 00:49:58.73 | 01:31:48.62 | 00:52:50.95 | 3452388 | 289% | 2138508 |
| Basho | 00:40.41 | 01:24.39 | 00:17.82 | 2.5293244246 | 2368768 | 10232 | 68418 | 00:19:32.94 | 01:14:10.04 | 00:01:19.19 | 2612436 | 386% | 6775376 |
| BerkeleyDB | 02:12.61 | 01:58.92 | 00:13.57 | 0.9990950909 | 5844376 | 9565 | 86202 | 00:15:28.44 | 00:42:07.97 | 00:17:27.49 | 5839912 | 385% | 3040716 |
| Hyper | 00:38.78 | 00:49.88 | 00:06.43 | 1.4520371325 | 2246448 | 10208 | 138393 | 00:09:38.39 | 00:35:06.12 | 00:02:06.18 | 2292632 | 385% | 2700088 |
| LMDB | 00:10.55 | 00:08.15 | 00:02.37 | 0.9971563981 | 2516192 | 10224 | 1449709 | 00:00:55.46 | 00:03:37.63 | 00:00:01.67 | 2547968 | 395% | 2550408 |
| RocksDB | 00:21.54 | 00:34.70 | 00:05.99 | 1.8890436397 | 2256032 | 10233 | 91544 | 00:14:37.74 | 00:54:06.84 | 00:02:38.04 | 3181764 | 387% | 6713852 |
| TokuDB | 01:45.12 | 01:41.58 | 00:47.37 | 1.4169520548 | 2726168 | 9881 | 109682 | 00:12:12.91 | 00:37:41.45 | 00:07:10.03 | 3920784 | 367% | 5429056 |
| WiredLSM | 01:10.93 | 02:35.55 | 00:18.62 | 2.4555195263 | 2492440 | 10230 | 179617 | 00:07:26.24 | 00:28:55.85 | 00:00:07.76 | 2948988 | 390% | 3205396 |
| WiredBtree | 00:17.79 | 00:15.68 | 00:02.09 | 0.9988757729 | 2381876 | 10021 | 752078 | 00:01:53.46 | 00:06:36.98 | 00:00:14.78 | 4752568 | 362% | 3415468 |
3. Larger Data Set
These tests use 100 million records and are run on the 16 core server. Aside from the data set size things are much the same. Here are the tabular results:
| Engine | Load Time | Overhead | Load Size | Writes/Sec | Reads/Sec | Run Time | Final Size | CPU% | Process Size | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Wall | User | Sys | KB | Wall | User | Sys | KB | KB | |||||
| LevelDB | 03:06.75 | 04:41.26 | 00:42.87 | 1.7356358768 | 11273396 | 9184 | 7594 | 01:00:02.00 | 01:22:11.46 | 01:52:10.46 | 13734168 | 323% | 3284192 |
| Basho | 04:22.96 | 11:09.24 | 02:18.93 | 3.0733571646 | 11449492 | 10211 | 80135 | 01:00:23.00 | 14:32:23.67 | 00:11:49.40 | 13841220 | 1464% | 19257796 |
| BerkeleyDB | 14:59.45 | 13:34.30 | 01:25.15 | 1 | 28381956 | 3378 | 55066 | 01:00:02.00 | 03:02:00.69 | 12:42:39.63 | 28387880 | 1573% | 14756768 |
| Hyper | 03:43.61 | 05:41.14 | 00:39.02 | 1.7001028577 | 11280092 | 10231 | 11673 | 01:00:04.00 | 01:59:42.09 | 01:53:24.27 | 15149416 | 387% | 6332460 |
| LMDB | 01:04.15 | 00:52.31 | 00:11.82 | 0.9996882307 | 12605332 | 10230 | 2486800 | 00:11:14.14 | 02:47:58.57 | 00:00:10.06 | 12627692 | 1598% | 12605788 |
| RocksDB | 02:28.66 | 03:59.92 | 00:30.97 | 1.8222117584 | 11289688 | 10232 | 129397 | 01:00:22.00 | 12:08:05.94 | 02:51:58.54 | 12777708 | 1490% | 18599544 |
| TokuDB | 07:44.10 | 09:17.31 | 02:54.82 | 1.5775263952 | 12665136 | 4601 | 70208 | 01:00:15.00 | 03:02:37.44 | 11:21:45.00 | 15328956 | 1434% | 23315964 |
| WiredLSM | 07:10.50 | 19:25.80 | 02:31.10 | 3.0590011614 | 12254620 | 10194 | 278415 | 01:00:05.00 | 15:51:04.17 | 00:02:09.76 | 16016296 | 1586% | 17723992 |
| WiredBtree | 02:07.49 | 01:49.52 | 00:17.97 | 1 | 11932620 | 10145 | 1320939 | 00:20:58.10 | 05:06:13.60 | 00:05:14.87 | 23865368 | 1560% | 20743232 |
看这个pdf里有对kv存储的架构和底层原理的详细介绍：
https://daim.idi.ntnu.no/masteroppgaver/008/8885/masteroppgave.pdf
