代码改变世界

mongodb报错Sort exceeded memory limit of 104857600 bytes

  abce  阅读(850)  评论(0编辑  收藏  举报

mongodb运行过程中,遇到错误信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
2023-07-14T09:29:33.853 ERR Failed to QueryBsPoolUnivStat error="(QueryExceededMemoryLimitNoDiskUseAllowed) Executor error during find command :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting."
2023-07-14T09:29:33.853 ERR Failed to query LoadAllStockpoolUnivStatHistory  perf history error="(QueryExceededMemoryLimitNoDiskUseAllowed) Executor error during find command :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting."
panic: Load  all LoadAllStockpoolUnivStatHistory: (QueryExceededMemoryLimitNoDiskUseAllowed) Executor error during find command :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting.
 
goroutine 1 [running]:
abc/srv/abc/stock/data.(*DataClient).LoadMongoCache(0xc000042600)
        /test/srv/abc/stock/data/data.go:455 +0x9a5
abc/srv/abc/stock.NewDataServer(0xc000042600, {0xc0008b02e0, 0x4})
        /test/srv/abc/stock/provider.go:64 +0x2f4
main.setupRouter({0x7f6690308800, 0xc0003dd520}, 0x32?, {0xc0008b02e0, 0x4})
        /test/cmd/abce/main.go:83 +0x265
main.main()
        /test/cmd/abce/main.go:65 +0x685


mongodb的版本:5.0.13

报错产生的原因:
因为查询排序数据量太大,导致报错。
详细内容可以参看官方文档。地址如下:

1
https://www.mongodb.com/docs/manual/reference/operator/aggregation/sort/?_ga=2.90473562.1126669746.1689308968-1238152925.1673317434#sort-operator-and-memory

 

具体说明:

1
2
Starting in MongoDB 6.0, pipeline stages that require more than 100 megabytes of memory to execute write temporary files to disk by default. In earlier verisons of MongoDB, you must pass { allowDiskUse: true } to individual find and aggregate commands to enable this behavior.
Individual find and aggregate commands may override the allowDiskUseByDefault parameter by either: Using { allowDiskUse: true } to allow writing temporary files out to disk when allowDiskUseByDefault is set to false Using { allowDiskUse: false } to prohibit writing temporary files out to disk when allowDiskUseByDefault is set to true

 

解决方案:
根据文档,可以在查询中添加{ allowDiskUse: true }

不过本次没有修改代码并添加{ allowDiskUse: true },因为发现集合中缺少索引,创建索引后,问题得到解决。

网上有人说可以修改参数:internalQueryExecMaxBlockingSortBytes。比如,执行以下命令:

1
db.adminCommand({setParameter: 1, internalQueryExecMaxBlockingSortBytes: 335544320})

但是,我查了一下,没有这个参数:

1
2
3
4
5
> use admin
switched to db admin
> db.runCommand( { getParameter : 1, "internalQueryExecMaxBlockingSortBytes" : 1 } );
{ "ok" : 0, "errmsg" : "no option found to get" }
>


其它
获取所有配置参数的值:

1
db.runCommand( { getParameter : '*' } )
相关博文:
阅读排行:
· DeepSeek 开源周回顾「GitHub 热点速览」
· 物流快递公司核心技术能力-地址解析分单基础技术分享
· .NET 10首个预览版发布:重大改进与新特性概览!
· AI与.NET技术实操系列(二):开始使用ML.NET
· .NET10 - 预览版1新功能体验(一)
历史上的今天:
2022-07-14 找出没有主键或唯一性约束的表
2022-07-14 【MySQL】MySQL错误日志中的Multi-Threaded Slave Statistics
2019-07-14 crontab 使用日期时间命名重定向文件
2015-07-14 11G新特性 -- archival(long-term)backups
2015-07-14 11G新特性 -- RMAN Multisection Backups
2015-07-14 RHEL7 -- 修改主机名
2015-07-14 RHEL7 -- NetworkManager
点击右上角即可分享
微信分享提示