multiprocessing多进程模块

1 基本理解

　　python不支持真多线程，无法在单进程中调用多核cpu。对于cpu密集型任务，可以使用多进程。python会调用OS原生多进程，运行在多核上，以此提高运行速度。

 

2 基本实现

import  multiprocessing
import  time

def test(n):
    print('this is the num %s process' %n)
    time.sleep(100)

if __name__ == '__main__':
    for i in range(10):
        mp = multiprocessing.Process(target=test,args=(i,))
        mp.start()
        #mp.join()
#和threading模块使用方式基本一致
#调用10个进程，系统就会同步运行10个进程
#如果在主循环中使用join方法，就会等待子进程运行结束后再往下走。

　　

3 父子进程间关系

import  multiprocessing
import  time,os

def test(n):
    print('this is the num %s process,pid is %s parent pid is %s' %(n,os.getpid(),os.getppid()))
    time.sleep(100)

if __name__ == '__main__':
    print('the pid is %s, the parent pid is %s' %(os.getpid(),os.getppid()))
    for i in range(10):
        mp = multiprocessing.Process(target=test,args=(i,))
        mp.start()

　　

[root@yhzk01 scripts]# python3 mp.py 
the pid is 128430, the parent pid is 102584
this is the num 0 process,pid is 128431 parent pid is 128430
this is the num 1 process,pid is 128432 parent pid is 128430
this is the num 2 process,pid is 128433 parent pid is 128430
this is the num 8 process,pid is 128439 parent pid is 128430
this is the num 5 process,pid is 128436 parent pid is 128430
this is the num 6 process,pid is 128437 parent pid is 128430
this is the num 3 process,pid is 128434 parent pid is 128430
this is the num 7 process,pid is 128438 parent pid is 128430
this is the num 4 process,pid is 128435 parent pid is 128430
this is the num 9 process,pid is 128440 parent pid is 128430

[root@yhzk01 ~]# ps aux |grep mp.py
root     128430  0.2  0.8 141552  8148 pts/0    S+   15:26   0:00 python3 mp.py
root     128431  0.0  0.6 141552  6416 pts/0    S+   15:26   0:00 python3 mp.py
root     128432  0.0  0.6 141552  6412 pts/0    S+   15:26   0:00 python3 mp.py
root     128433  0.0  0.6 141552  6412 pts/0    S+   15:26   0:00 python3 mp.py
root     128434  0.0  0.6 141552  6412 pts/0    S+   15:26   0:00 python3 mp.py
root     128435  0.0  0.6 141552  6416 pts/0    S+   15:26   0:00 python3 mp.py
root     128436  0.0  0.6 141552  6416 pts/0    S+   15:26   0:00 python3 mp.py
root     128437  0.0  0.6 141552  6420 pts/0    S+   15:26   0:00 python3 mp.py
root     128438  0.0  0.6 141552  6420 pts/0    S+   15:26   0:00 python3 mp.py
root     128439  0.0  0.6 141552  6420 pts/0    S+   15:26   0:00 python3 mp.py
root     128440  0.0  0.6 141552  6420 pts/0    S+   15:26   0:00 python3 mp.py
root     128444  0.0  0.0 112660   984 pts/1    S+   15:26   0:00 grep --color=auto mp.py

[root@yhzk01 ~]# ps aux |grep 102584
root     102584  0.0  0.2 116308  2640 pts/0    Ss   May30   0:00 -bash
root     128442  0.0  0.0 112660   976 pts/1    R+   15:26   0:00 grep --color=auto 102584

#可见，bash进程启动子进程mp.py。mp.py再启动多个子进程。

　　

4 class Queue()

　　线程共享内存数据，进程的内存是独立的。

　　在queue模块中，有class Queue，但这个队列只是在线程间通信的。

　　在mulitprocessing模块中，也有class Queue，可以提供父进程与子进程或者子进程之间的数据交互。