Python多进程
Published on 2019-01-11 22:42 in 分类: Tech / Python with zgxme
分类: Tech / Python

Python多进程

    复制代码
    import os
    
    print('Process (%s) start...' % os.getpid())
    # Only works on Unix/Linux/Mac:
    pid = os.fork()
    if pid == 0:
        print('I am child process (%s) and my parent is %s.' % (os.getpid(), os.getppid()))
    else:
        print('I (%s) just created a child process (%s).' % (os.getpid(), pid))
    复制代码

    Unix/Linux/Mac操作系统提供fork()系统调用

    Process (876) start...
    I (876) just created a child process (877).
    I am child process (877) and my parent is 876.

    Windows下 multiprocessing 

    复制代码
    from multiprocessing import Process
    import os
    #子进程执行的代码
    def run_proc(name):
        print('Run child process %s (%s)...' %(name,os.getpid()))
    
    
    if __name__ == '__main__':
        print('Parent process %s.' % os.getpid())
        p = Process(target=run_proc, args=('test',)) #创建子进程Process实例
        print('Child process will start.')
        p.start() #start()方法启动
        p.join() #等待子进程结束后继续往下进行 通常用于进程间的同步
        print('Child process end.')
    复制代码
    Parent process 928.
    Child process will start.
    Run child process test (929)...
    Child process end.

     

    Pool 进程池

    复制代码
    from multiprocessing import Pool
    import os, time, random
    
    
    def long_time_task(name):
        print('Run task %s (%s)...' % (name, os.getpid()))
        start = time.time()
        time.sleep(random.random() * 3)
        end = time.time()
        print('Task %s runs %0.2f seconds.' % (name, (end - start)))
    
    
    if __name__ == '__main__':
        print('Parent process %s.' % os.getpid())
        p = Pool(4)
    # Pool的默认大小为为CPU的核数
    for i in range(5): p.apply_async(long_time_task, args=(i,)) print('Waiting for all subprocesses done...') p.close() p.join()
    # join()会等待所有子进程执行完毕 调用join()前必须执行close()
    # 调用close()后就不能添加新的Process
      
    print('All subprocesses done.')
    复制代码

     

    复制代码
    Parent process 1784.
    Waiting for all subprocesses done...
    Run task 0 (8128)...
    Run task 1 (9688)...
    Run task 2 (7180)...
    Run task 3 (6308)...
    Task 2 runs 0.23 seconds.
    Run task 4 (7180)...
    Task 3 runs 0.62 seconds.
    Task 1 runs 1.44 seconds.
    Task 0 runs 2.14 seconds.
    Task 4 runs 2.32 seconds.
    All subprocesses done.

    复制代码
    0
    1
    2
    3
    4
    Pool
    Pool
    

    Process之间肯定是需要通信的,操作系统提供了很多机制来实现进程间的通信。Python的multiprocessing模块包装了底层的机制,提供了QueuePipes等多种方式来交换数据。

    
    

    Queue为例,在父进程中创建两个子进程,一个往Queue里写数据,一个从Queue里读数据:

     
    复制代码
    from multiprocessing import Process, Queue
    import os, time, random
    
    # 写数据进程执行的代码:
    def write(q):
        print('Process to write: %s' % os.getpid())
        for value in ['A', 'B', 'C']:
            print('Put %s to queue...' % value)
            q.put(value)
            time.sleep(random.random())
    
    # 读数据进程执行的代码:
    def read(q):
        print('Process to read: %s' % os.getpid())
        while True:
            value = q.get(True)
            print('Get %s from queue.' % value)
    
    if __name__=='__main__':
        # 父进程创建Queue,并传给各个子进程:
        q = Queue()
        pw = Process(target=write, args=(q,))
        pr = Process(target=read, args=(q,))
        # 启动子进程pw,写入:
        pw.start()
        # 启动子进程pr,读取:
        pr.start()
        # 等待pw结束:
        pw.join()
        # pr进程里是死循环,无法等待其结束,只能强行终止:
        pr.terminate()
    复制代码

     

    
    
    Process to write: 50563
    Put A to queue...
    Process to read: 50564
    Get A from queue.
    Put B to queue...
    Get B from queue.
    Put C to queue...
    Get C from queue.
    
    

     

     

     

    参考 https://www.liaoxuefeng.com/wiki/0014316089557264a6b348958f449949df42a6d3a2e542c000/001431927781401bb47ccf187b24c3b955157bb12c5882d000sdsd

    posted @   zgxme  阅读(182)  评论(0编辑  收藏  举报
    编辑推荐:
    · 没有源码,如何修改代码逻辑?
    · 一个奇形怪状的面试题:Bean中的CHM要不要加volatile?
    · [.NET]调用本地 Deepseek 模型
    · 一个费力不讨好的项目,让我损失了近一半的绩效!
    · .NET Core 托管堆内存泄露/CPU异常的常见思路
    阅读排行:
    · DeepSeek “源神”启动!「GitHub 热点速览」
    · 微软正式发布.NET 10 Preview 1:开启下一代开发框架新篇章
    · C# 集成 DeepSeek 模型实现 AI 私有化(本地部署与 API 调用教程)
    · DeepSeek R1 简明指南:架构、训练、本地部署及硬件要求
    · NetPad:一个.NET开源、跨平台的C#编辑器
    点击右上角即可分享
    微信分享提示