并发编程之线程池,进程池

一.为什么引入池的概念,什么是池

  受限于硬件的发展,硬件跟不上软件的发展

  在保证计算机硬件安全的情况下,最大限度的利用了计算机

  池其实是降低了程序的运行效率,但是保证了计算机硬件的安全

我们再使用进程和线程时,不可能无限制的去开进程或线程。因此我们需要用到进程池,线程池来解决这一问题。

二.Python模块-----concurrent.futures 模块(并发未来)

1.concurrent模块是用来创建并行的任务,提供了更高级别的接口

2.模块导入进程池 与 线程池

from concurrent.futures import ProcessPoolExecutor  进程池

from concurrent.futures import ThreadPoolExecutor 线程池

3.p = ProcessPoolExecutor(max_works)  对于进程池  如果不写max_works 默认是CPU数目

   p= ThreadPoolExecutor(max_works) 对于线程池  如果不写max_works 默认是CPU数目*5

4.p.submit(task,i)是异步提交  task 是函数  i 是 task所需的参数

 obj = p.submit(task,i)  返回的是一个对象obj

 obj.result()是函数返回的结果

 p.shutdown()  关闭池子  相当于 close  join

三.线程池,进程池 

基于concurrent模块的线程池与进程池

 1 from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor
 2 import time
 3 
 4 def task(n):
 5     print(n)
 6     time.sleep(2)
 7     return n+1
 8 
 9 def call_back(n):
10     print('结果是:',n.result())
11 
12 if __name__ == '__main__':
13     start = time.time()
14     pool = ThreadPoolExecutor(5)
15     #pool = ProcessPoolExecutor(5)
16     for i in range(7):
17         res = pool.submit(task,i).result()
18         print(res)
19 
20     pool.shutdown()
21 
22     print(time.time()-start)
23 #14.00873589515686
同步执行
from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor
import time
 
 
def task(n):
     print(n)
     time.sleep(2)
     return n+1
 
def call_back(n):
     print('结果是:',n.result())
 
if __name__ == '__main__':
     start = time.time()
     pool = ThreadPoolExecutor(5)
    #pool = ProcessPoolExecutor(5)
     for i in range(7):
         res = pool.submit(task,i)
         p_list.append(res)
 
     pool.shutdown()
 
     for p in p_list:
          print('>>>:',p.result())
   
 
     print(time.time()-start)
  # 4.002536296844482
异步执行

 使用回调函数实现异步

from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor
import time


def task(n):
    print(n)
    time.sleep(2)
    return n+1

def call_back(n):
    print('结果是:',n.result())

if __name__ == '__main__':
    start = time.time()
    pool = ThreadPoolExecutor(5)
    #pool = ProcessPoolExecutor(5)
    for i in range(7):
        res = pool.submit(task,i).add_done_callback(call_back)  #异步
    pool.shutdown()

    print(time.time()-start)
    # 4.002536296844482
回调函数实现异步

 

posted @ 2019-08-16 10:43  s686编程传  阅读(182)  评论(0编辑  收藏  举报