MongoDB 影响性能的 5 个配置选项及其设置方法

 

MongoDB 性能概述

MongoDB 性能由多个因素组成：操作系统设置、数据库配置设置、数据库内部设置、内存设置和应用程序设置。这里将重点介绍与性能有关的 MongoDB 数据库配置选项以及如何设置这些选项

 

MongoDB 配置选项

MongoDB 使用 YAML 文件格式的配置文件。 linux 环境下，默认的配置文件是/etc/mongod.conf，具体取决于操作系统。

下面介绍以下有助于 MongoDB 环境性能的五个配置选项！

 

(1) storage.wiredTiger.engineConfig.cacheSizeGB

storage:
   wiredTiger:
       engineConfig:
           cacheSizeGB: <value>

自 MongoDB 3.0 开始，MongoDB 一直使用 WiredTiger 作为默认存储引擎，因此将从 WiredTiger 的角度来检查 MongoDB 的内存性能。默认情况下，MongoDB 将保留 (50%*RAM-1)GB 用于 WiredTiger 缓存或 256 MB（以较大者为准）。

例如，如果系统内存为 16 GB，则 WiredTiger 缓存大小为 7.5 GB。

( 0.5 * (16-1) )

缓存的大小对确保 WiredTiger 的性能非常重要。对于是否应该改变默认值，一个好的经验法则是，缓存的大小应足以容纳整个应用程序的工作集。

以运行以下命令来验证 MongoDB 认为的内存限制：

> db.hostInfo().system.memLimitMB 
NumberLong(32077)

那我们如何才能知道是应该增加还是减少cache的大小呢？通过分析cache使用的统计信息即可：

db.serverStatus().wiredTiger.cache
{
"application threads page read from disk to cache count" : 9,
"application threads page read from disk to cache time (usecs)" : 17555,
"application threads page write from cache to disk count" : 1820,
"application threads page write from cache to disk time (usecs)" : 1052322,
"bytes allocated for updates" : 20043,
"bytes belonging to page images in the cache" : 46742,
"bytes belonging to the history store table in the cache" : 173,
"bytes currently in the cache" : 73044,
"bytes dirty in the cache cumulative" : 38638327,
"bytes not belonging to page images in the cache" : 26302,
"bytes read into cache" : 43280,
"bytes written from cache" : 20517382,
"cache overflow score" : 0,
"checkpoint blocked page eviction" : 0,
"eviction calls to get a page" : 5973,
"eviction calls to get a page found queue empty" : 4973,
"eviction calls to get a page found queue empty after locking" : 20,
"eviction currently operating in aggressive mode" : 0,
"eviction empty score" : 0,
"eviction passes of a file" : 0,
"eviction server candidate queue empty when topping up" : 0,
"eviction server candidate queue not empty when topping up" : 0,
"eviction server evicting pages" : 0,
"eviction server slept, because we did not make progress with eviction" : 735,
"eviction server unable to reach eviction goal" : 0,
"eviction server waiting for a leaf page" : 2,
"eviction state" : 64,
"eviction walk target pages histogram - 0-9" : 0,
"eviction walk target pages histogram - 10-31" : 0,
"eviction walk target pages histogram - 128 and higher" : 0,
"eviction walk target pages histogram - 32-63" : 0,
"eviction walk target pages histogram - 64-128" : 0,
"eviction walk target strategy both clean and dirty pages" : 0,
"eviction walk target strategy only clean pages" : 0,
"eviction walk target strategy only dirty pages" : 0,
"eviction walks abandoned" : 0,
"eviction walks gave up because they restarted their walk twice" : 0,
"eviction walks gave up because they saw too many pages and found no candidates" : 0,
"eviction walks gave up because they saw too many pages and found too few candidates" : 0,
"eviction walks reached end of tree" : 0,
"eviction walks started from root of tree" : 0,
"eviction walks started from saved location in tree" : 0,
"eviction worker thread active" : 4,
"eviction worker thread created" : 0,
"eviction worker thread evicting pages" : 902,
"eviction worker thread removed" : 0,
"eviction worker thread stable number" : 0,
"files with active eviction walks" : 0,
"files with new eviction walks started" : 0,
"force re-tuning of eviction workers once in a while" : 0,
"forced eviction - history store pages failed to evict while session has history store cursor open" : 0,
"forced eviction - history store pages selected while session has history store cursor open" : 0,
"forced eviction - history store pages successfully evicted while session has history store cursor open" : 0,
"forced eviction - pages evicted that were clean count" : 0,
"forced eviction - pages evicted that were clean time (usecs)" : 0,
"forced eviction - pages evicted that were dirty count" : 0,
"forced eviction - pages evicted that were dirty time (usecs)" : 0,
"forced eviction - pages selected because of too many deleted items count" : 0,
"forced eviction - pages selected count" : 0,
"forced eviction - pages selected unable to be evicted count" : 0,
"forced eviction - pages selected unable to be evicted time" : 0,
"forced eviction - session returned rollback error while force evicting due to being oldest" : 0,
"hazard pointer blocked page eviction" : 0,
"hazard pointer check calls" : 902,
"hazard pointer check entries walked" : 25,
"hazard pointer maximum array length" : 1,
"history store key truncation calls that returned restart" : 0,
"history store key truncation due to mixed timestamps" : 0,
"history store key truncation due to the key being removed from the data page" : 0,
"history store score" : 0,
"history store table insert calls" : 0,
"history store table insert calls that returned restart" : 0,
"history store table max on-disk size" : 0,
"history store table on-disk size" : 0,
"history store table out-of-order resolved updates that lose their durable timestamp" : 0,
"history store table out-of-order updates that were fixed up by moving existing records" : 0,
"history store table out-of-order updates that were fixed up during insertion" : 0,
"history store table reads" : 0,
"history store table reads missed" : 0,
"history store table reads requiring squashed modifies" : 0,
"history store table remove calls due to key truncation" : 0,
"history store table writes requiring squashed modifies" : 0,
"in-memory page passed criteria to be split" : 0,
"in-memory page splits" : 0,
"internal pages evicted" : 0,
"internal pages queued for eviction" : 0,
"internal pages seen by eviction walk" : 0,
"internal pages seen by eviction walk that are already queued" : 0,
"internal pages split during eviction" : 0,
"leaf pages split during eviction" : 0,
"maximum bytes configured" : 8053063680,
"maximum page size at eviction" : 376,
"modified pages evicted" : 902,
"modified pages evicted by application threads" : 0,
"operations timed out waiting for space in cache" : 0,
"overflow pages read into cache" : 0,
"page split during eviction deepened the tree" : 0,
"page written requiring history store records" : 0,
"pages currently held in the cache" : 24,
"pages evicted by application threads" : 0,
"pages queued for eviction" : 0,
"pages queued for eviction post lru sorting" : 0,
"pages queued for urgent eviction" : 902,
"pages queued for urgent eviction during walk" : 0,
"pages read into cache" : 20,
"pages read into cache after truncate" : 902,
"pages read into cache after truncate in prepare state" : 0,
"pages requested from the cache" : 33134,
"pages seen by eviction walk" : 0,
"pages seen by eviction walk that are already queued" : 0,
"pages selected for eviction unable to be evicted" : 0,
"pages selected for eviction unable to be evicted as the parent page has overflow items" : 0,
"pages selected for eviction unable to be evicted because of active children on an internal page" : 0,
"pages selected for eviction unable to be evicted because of failure in reconciliation" : 0,
"pages walked for eviction" : 0,
"pages written from cache" : 1822,
"pages written requiring in-memory restoration" : 0,
"percentage overhead" : 8,
"tracked bytes belonging to internal pages in the cache" : 5136,
"tracked bytes belonging to leaf pages in the cache" : 67908,
"tracked dirty bytes in the cache" : 493,
"tracked dirty pages in the cache" : 1,
"unmodified pages evicted" : 0
}

可以重点关注以下字段：

• wiredTiger.cache.maximum bytes configured: cache的最大字节数

• wiredTiger.cache.bytes currently in the cache：当前在cache中的数据量大小，该值不能超过 "maximum bytes configured" 值

• wiredTiger.cache.tracked dirty bytes in the cache：当前在cache中的脏数据量大小，该值不能超过"bytes currently in the cache"的值

通过观察上述值，我们可以确定是否需要增加实例的 WiredTiger 缓存大小。此外，对于读负载高的应用，还应该关注"wiredTiger.cache.pages read into cache"的值。如果该值持续偏高，增加缓存大小可能会提高整体读取性能。

(2) storage.wiredTiger.engineConfig.directoryForIndexes

storage:
   wiredTiger:
       engineConfig:
           directoryForIndexes: <true or false>

将此值设为 true 会在 storage.dbPath 目录中创建两个目录，一个名为 collection，用于存放集合数据文件，另一个名为 index，用于存放索引数据文件。这样就可以根据需要为集合和索引创建独立的存储卷，从而将磁盘 I/O 量分散到每个卷上，但对于大多数现代存储选项，只需将磁盘条带化到两个卷上（RAID 0）就能获得相同的性能优势。这有助于将索引 I/O 从基于集合的 I/O 中分离出来，并减少基于存储的延迟，不过基于索引的 I/O 由于规模较小，成本不会太高。

(3) storage.wiredTiger.collectionConfig.blockCompressor

storage:
   wiredTiger:
       collectionConfig:
           blockCompressor: <value>

该选项设置所有集合数据的压缩选项。 该参数的可能值为none、snappy（默认）、zlib 和 zstd。 压缩对性能有何帮助？ WiredTiger 缓存一般会以未压缩的方式存储更改，但一些非常大的文档除外。现在，我们需要将未压缩的数据写入磁盘。

 

压缩类型：

·Snappy 压缩相当简单明了，它收集最多 32KB 的数据，然后压缩数据，如果压缩成功，就将数据块写入(四舍五入)到最接近的 4KB大小。

·Zlib 压缩的工作方式略有不同；它会收集更多数据，并压缩到足以填满磁盘上 32KB 的数据块。这需要更多的 CPU 资源，但通常能获得更好的压缩率（与 snappy 和 zlib 之间的固有差异无关）。

·Zstd 是一种较新的压缩算法，由 Facebook 开发，比 zlib 有了改进（压缩率更高、CPU 占用更少、性能更快）。

 

选择哪种压缩算法在很大程度上取决于你的工作负载。对于大多数写入量大的工作负载，snappy 压缩会比 zlib 和 zstd 性能更好，但需要更多磁盘空间。对于读取重的工作负载，zstd 通常是最佳选择，因为它的解压缩率更高。

(4) storage.directoryPerDB

storage:
   directoryPerDB: <true or false>

与上述配置文件选项（storage.wiredTiger.engineConfig.directoryForIndexes）类似，将此值设为 true 会在 storage.dbPath 中为 MongoDB 实例中的每个数据库创建一个单独的目录。这样，就可以根据需要为每个数据库创建单独的存储卷，从而将磁盘 I/O 量分散到每个卷上。

当多个数据库都需要大量 I/O 时，这将有所帮助。此外，如果将此参数与 storage.wiredTiger.engineConfig.directorForIndexes 结合使用，目录结构将如下所示：

-Database_name
    -Collection
    -Index

(5) net.compression.compressors

net:
   compression:
       compressors: <value>

该选项允许你压缩 mongos、mongod 甚至 mongo shell 之间的网络流量。目前有三种可用的压缩方式：snappy、zlib 和 zstd。自 MongoDB 3.6 开始，压缩已默认启用。在 MongoDB 3.6 和 4.0 中，snappy 是默认设置。

自 MongoDB 4.2 起，默认依次为 snappy、zstd 和 zlib 压缩。还需要注意的是，网络对话的每一端都必须至少有一个支持互用的压缩器才能进行压缩。

例如，如果 shell 使用 zlib 压缩，但 mongod 只接受 snappy 压缩，那么两者之间就不会发生压缩。如果两者都接受 zstd 压缩，那么它们之间就会使用 zstd 压缩。

设置压缩后，由于跨网络传输的数据大小减小，有时甚至会大幅减小，因此对减少复制延迟和整体网络延迟非常有帮助。在云环境中，设置该配置选项还能降低数据传输成本。