Python多进程
import os print('Process (%s) start...' % os.getpid()) # Only works on Unix/Linux/Mac: pid = os.fork() if pid == 0: print('I am child process (%s) and my parent is %s.' % (os.getpid(), os.getppid())) else: print('I (%s) just created a child process (%s).' % (os.getpid(), pid))
Unix/Linux/Mac操作系统提供fork()系统调用
Process (876) start... I (876) just created a child process (877). I am child process (877) and my parent is 876.
Windows下 multiprocessing
from multiprocessing import Process import os #子进程执行的代码 def run_proc(name): print('Run child process %s (%s)...' %(name,os.getpid())) if __name__ == '__main__': print('Parent process %s.' % os.getpid()) p = Process(target=run_proc, args=('test',)) #创建子进程Process实例 print('Child process will start.') p.start() #start()方法启动 p.join() #等待子进程结束后继续往下进行 通常用于进程间的同步 print('Child process end.')
Parent process 928. Child process will start. Run child process test (929)... Child process end.
Pool 进程池
from multiprocessing import Pool import os, time, random def long_time_task(name): print('Run task %s (%s)...' % (name, os.getpid())) start = time.time() time.sleep(random.random() * 3) end = time.time() print('Task %s runs %0.2f seconds.' % (name, (end - start))) if __name__ == '__main__': print('Parent process %s.' % os.getpid()) p = Pool(4)
# Pool的默认大小为为CPU的核数 for i in range(5): p.apply_async(long_time_task, args=(i,)) print('Waiting for all subprocesses done...') p.close() p.join()
# join()会等待所有子进程执行完毕 调用join()前必须执行close()
# 调用close()后就不能添加新的Process
print('All subprocesses done.')
Parent process 1784. Waiting for all subprocesses done... Run task 0 (8128)... Run task 1 (9688)... Run task 2 (7180)... Run task 3 (6308)... Task 2 runs 0.23 seconds. Run task 4 (7180)... Task 3 runs 0.62 seconds. Task 1 runs 1.44 seconds. Task 0 runs 2.14 seconds. Task 4 runs 2.32 seconds. All subprocesses done.
0
1
2
3
4
Pool
Pool
Process
之间肯定是需要通信的,操作系统提供了很多机制来实现进程间的通信。Python的multiprocessing
模块包装了底层的机制,提供了Queue
、Pipes
等多种方式来交换数据。
以Queue
为例,在父进程中创建两个子进程,一个往Queue
里写数据,一个从Queue
里读数据:
from multiprocessing import Process, Queue import os, time, random # 写数据进程执行的代码: def write(q): print('Process to write: %s' % os.getpid()) for value in ['A', 'B', 'C']: print('Put %s to queue...' % value) q.put(value) time.sleep(random.random()) # 读数据进程执行的代码: def read(q): print('Process to read: %s' % os.getpid()) while True: value = q.get(True) print('Get %s from queue.' % value) if __name__=='__main__': # 父进程创建Queue,并传给各个子进程: q = Queue() pw = Process(target=write, args=(q,)) pr = Process(target=read, args=(q,)) # 启动子进程pw,写入: pw.start() # 启动子进程pr,读取: pr.start() # 等待pw结束: pw.join() # pr进程里是死循环,无法等待其结束,只能强行终止: pr.terminate()
Process to write: 50563 Put A to queue... Process to read: 50564 Get A from queue. Put B to queue... Get B from queue. Put C to queue... Get C from queue.
参考 https://www.liaoxuefeng.com/wiki/0014316089557264a6b348958f449949df42a6d3a2e542c000/001431927781401bb47ccf187b24c3b955157bb12c5882d000sdsd
分类:
Tech / Python
· 没有源码,如何修改代码逻辑?
· 一个奇形怪状的面试题:Bean中的CHM要不要加volatile?
· [.NET]调用本地 Deepseek 模型
· 一个费力不讨好的项目,让我损失了近一半的绩效!
· .NET Core 托管堆内存泄露/CPU异常的常见思路
· DeepSeek “源神”启动!「GitHub 热点速览」
· 微软正式发布.NET 10 Preview 1:开启下一代开发框架新篇章
· C# 集成 DeepSeek 模型实现 AI 私有化(本地部署与 API 调用教程)
· DeepSeek R1 简明指南:架构、训练、本地部署及硬件要求
· NetPad:一个.NET开源、跨平台的C#编辑器