解决 pymongo 在linux 的子进程中操作父进程的链接报错,MongoClient opened before fork. Create MongoClient only after forking.

 

linux上不能在子进程中操作全局变量client的链接,否则报错。需要说明的是win的多进程不是fork实现的,所以子进程操作client没事


封装1个get_col的函数就行了。判断pid。

 1 import os
 2 from multiprocessing import Process
 3 
 4 import pymongo.collection
 5 from auto_run_on_remote import  run_current_script_on_remote
 6 from pymongo import MongoClient
 7 
 8 run_current_script_on_remote()
 9 
10 cleint = MongoClient(host='127.0.0.1')
11 
12 
13 pid__col_map = {}
14 
15 def get_col(db:str,col:str,mongo_connect_url='mongodb://127.0.0.1') ->pymongo.collection.Collection:
16     pid = os.getpid()
17     key = (pid,mongo_connect_url,db,col)
18     if key not in pid__col_map:
19         pid__col_map[key] = MongoClient(mongo_connect_url).get_database(db).get_collection(col)
20     return pid__col_map[key]
21 
22 
23 def col3() ->pymongo.collection.Collection:
24     return get_col('testdb', 'col3')
25 
26 def f():
27     # cleint.get_database('testdb').get_collection('testcol').insert_one({"a":1})   # 如果在子进程中运行这个函数,这个在linux + pymongo 4.xx报错,在3.xx会警告。
28     # get_col('testdb','testcol2').insert_one({"a":1})
29     col3().insert_one({"b": 2})
30 
31 
32 if __name__ == '__main__':
33     [Process(target=f).start() for i in range(3)]

 

posted @ 2022-02-15 14:04  北风之神0509  阅读(172)  评论(0编辑  收藏  举报