aysncio执行tasks的方法
1. asyncio
import time import asyncio # func called as one task async def worker_1(time_sleep): print('worker_1 had been called') await asyncio.sleep(time_sleep) print('worker_1 had awaited done!') async def main(): print('main begin !') tasks = [asyncio.create_task(worker_1(t)) for t in range(2, 9)] # task2 = asyncio.create_task(worker_2()) # await task1 # await task2 await asyncio.gather(*tasks) print('main done done res') if __name__ == '__main__': start_time = time.perf_counter() asyncio.run(main()) end_time =time.perf_counter() print('asyncio.run(main()): runtime: {}'.format(end_time - start_time))
2. 显示爬虫时间最长的tasks
要等所有任务都结束才行,用for task in tasks: await task 即可
3. await asyncio.gather(*tasks)
async def main(urls):
tasks = [asyncio.create_task(crawl_page(url)) for url in urls]
await asyncio.gather(*tasks)
用一个例子来演示会更加清晰