关于mongo的$push和$addToSet的内存占用报错

在使用mongo的聚合管道的时候会发现如果涉及的文档数过多，出现如下报错

com.mongodb.MongoCommandException: Command failed with error 146 (ExceededMemoryLimit): '$push used too much memory and cannot spill to disk. Memory limit: 104857600 bytes' on server 192.168.2.234:27017. The full response is {"ok": 0.0, "errmsg": "$push used too much memory and cannot spill to disk. Memory limit: 104857600 bytes", "code": 146, "codeName": "ExceededMemoryLimit"}
  at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:175)
  at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:302)
  at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:258)
  at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
  at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:500)
  at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:71)
  at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:224)
  at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:202)
  at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:118)
  at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:110)
  at com.mongodb.internal.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:343)
  at com.mongodb.internal.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:334)
  at com.mongodb.internal.operation.CommandOperationHelper.executeCommandWithConnection(CommandOperationHelper.java:220)
  at com.mongodb.internal.operation.CommandOperationHelper$5.call(CommandOperationHelper.java:206)
  at com.mongodb.internal.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:462)
  at com.mongodb.internal.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:203)
  at com.mongodb.internal.operation.AggregateOperationImpl.execute(AggregateOperationImpl.java:189)
  at com.mongodb.internal.operation.AggregateOperation.execute(AggregateOperation.java:296)
  at com.mongodb.internal.operation.AggregateOperation.execute(AggregateOperation.java:41)
  at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:190)
  at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:135)
  at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:92)
  at com.mongodb.client.internal.MongoIterableImpl.forEach(MongoIterableImpl.java:121)
  at com.mongodb.client.internal.MongoIterableImpl.into(MongoIterableImpl.java:130)

 

经过网上查询，源赖式，佐田，mongo把这两个聚合操作里面的内存限制在了100M,具体不太清楚为啥限制，默认是100M吧，但是在之后mongodb版本里面可以进行设置，通过mongo shell如下操作即可，命令行的话，通过setParameter

> newLimit = 100 * 1024 * 1024;

> db.adminCommand({setParameter: 1, internalQueryMaxAddToSetBytes: newLimit})

不过，不太建议改这个东西，没有意义，因为mongo的文档大小有限制，为16M，就算你在聚合管道的某个阶段给$push操作数组的内存

大小放开，但是，最后如果输出的文档包含此过大数组，还是会报错

com.mongodb.MongoCommandException: Command failed with error 10334 (BSONObjectTooLarge): 'BSONObj size: 23537822 (0x167289E) is invalid. Size must be between 0 and 16793600(16MB) First element:

后面太长了，就不贴了，所以一般改了也没啥用吧，除非确实是中间结果需要处理。

参考链接：

https://stackoverflow.com/questions/63498668/how-to-set-internalquerymaxaddtosetbytes-in-mongo-db-version-3-6-9