并发编程：异步+回调机制

一：进程

1、导入模块

   2.定义访问下载函数

 

   3.解析下载的内容

  4.在进程池里放任务，并把任务提交给get函数

 

 完整代码：

 1 from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor
 2 import requests
 3 import os
 4 import time
 5 import random
 6 
 7 def get(url):
 8     print('%s GET %s' %(os.getpid(),url))   
 9     response=requests.get(url)               
10     if response.status_code == 200:          
11         return response.text                
12 
13 def pasrse(obj):     
14     res=obj.result()  
15     print('%s 解析结果为：%s' %(os.getpid(),len(res)))
16 
17 if __name__ == '__main__':
18     urls=[
19         'https://www.baidu.com',
20         'https://www.baidu.com',
21         'https://www.baidu.com',
22         'https://www.baidu.com',
23         'https://www.baidu.com',
24         'https://www.baidu.com',
25         'https://www.baidu.com',
26         'https://www.baidu.com',
27         'https://www.python.org',
28     ]
29 
30     pool=ProcessPoolExecutor(4)     
31     for url in urls:                
32         obj=pool.submit(get,url)
33         obj.add_done_callback(pasrse)  
34     print('主进程',os.getpid())
View Code
运行结果：

主进程 5588
6976 GET https://www.baidu.com
17844 GET https://www.baidu.com
16832 GET https://www.baidu.com
8956 GET https://www.baidu.com
17844 GET https://www.baidu.com
5588 解析结果为：2443
8956 GET https://www.baidu.com
5588 解析结果为：2443
16832 GET https://www.baidu.com
5588 解析结果为：2443
6976 GET https://www.baidu.com
5588 解析结果为：2443
17844 GET https://www.python.org
5588 解析结果为：2443
5588 解析结果为：2443
5588 解析结果为：2443
5588 解析结果为：2443
5588 解析结果为：48887

Process finished with exit code 0

 

二：线程

过程与进程相似，但线程没有主次之分，哪个线程闲着了就由哪个线程干回调的活

from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor
from threading import current_thread
import requests
import os
import time
import random

def get(url):
    print('%s GET %s' %(current_thread().name,url))
    response=requests.get(url)
    time.sleep(random.randint(1,3))

    if response.status_code == 200:
        return response.text

def pasrse(obj):
    res=obj.result()
    print('%s 解析结果为：%s' %(current_thread().name,len(res)))

if __name__ == '__main__':
    urls=[
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.baidu.com',
        'https://www.python.org',
    ]

    pool=ThreadPoolExecutor(4)
    for url in urls:
        obj=pool.submit(get,url)
        obj.add_done_callback(pasrse)
View Code

运行结果：

ThreadPoolExecutor-0_0 GET https://www.baidu.com
ThreadPoolExecutor-0_1 GET https://www.baidu.com
ThreadPoolExecutor-0_2 GET https://www.baidu.com
ThreadPoolExecutor-0_3 GET https://www.baidu.com
ThreadPoolExecutor-0_0 解析结果为：2443
ThreadPoolExecutor-0_0 GET https://www.baidu.com
ThreadPoolExecutor-0_2 解析结果为：2443
ThreadPoolExecutor-0_2 GET https://www.baidu.com
ThreadPoolExecutor-0_1 解析结果为：2443
ThreadPoolExecutor-0_1 GET https://www.baidu.com
ThreadPoolExecutor-0_0 解析结果为：2443
ThreadPoolExecutor-0_0 GET https://www.baidu.com
ThreadPoolExecutor-0_3 解析结果为：2443
ThreadPoolExecutor-0_3 GET https://www.python.org
ThreadPoolExecutor-0_2 解析结果为：2443
ThreadPoolExecutor-0_0 解析结果为：2443
ThreadPoolExecutor-0_1 解析结果为：2443
ThreadPoolExecutor-0_3 解析结果为：48887

Process finished with exit code 0

 

posted @ 2019-10-31 21:49  zh_小猿  阅读(198)  评论(0)  编辑  收藏  举报