python小工具：用python操作HP的Quality Center （二）----- 用异步方式提高速度

上接第一篇 http://www.cnblogs.com/sdet/p/6874631.html

 

在python中,很简单地能把http请求通过异步的方式发送，以下代码在python 3.6.0上运行通过，

import asyncio
import requests

async def main():
    loop = asyncio.get_event_loop()
    future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
    future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
    response1 = await future1
    response2 = await future2
    print(response1.text)
    print(response2.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

按这个思路，把我上次的python 上传QC的工具移到python 3.6上，做简单改写后，上传文件速度变得飞快。

原来传1次10个文件，我用时485秒，Time: 485.037000 seconds

现在仅用时75秒，日志如下：

[2017-06-02 12:45:21,714] post http://xxxx.qc.com:80/qcbin/authentication-point/authenticate
[2017-06-02 12:45:22,401] headers = {'Authorization': 'Basic xxxxxxxxxx'}
[2017-06-02 12:45:22,401] 200 OK
[2017-06-02 12:45:22,401] --------------------
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894980/attachments
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894983/attachments
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894982/attachments
[2017-06-02 12:45:22,416] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894981/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894986/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894987/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894984/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894985/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894988/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894979/attachments
[2017-06-02 12:45:51,835] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:51,835] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:51,835] 200 OK
[2017-06-02 12:45:51,835] --------------------
[2017-06-02 12:45:59,504] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:59,504] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:59,504] 200 OK
[2017-06-02 12:45:59,504] --------------------
[2017-06-02 12:45:59,520] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:59,520] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:59,520] 200 OK
[2017-06-02 12:45:59,520] --------------------
[2017-06-02 12:46:03,244] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:03,244] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:03,244] 200 OK
[2017-06-02 12:46:03,244] --------------------
[2017-06-02 12:46:05,059] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:05,059] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:05,059] 200 OK
[2017-06-02 12:46:05,059] --------------------
[2017-06-02 12:46:08,068] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:08,068] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:08,068] 200 OK
[2017-06-02 12:46:08,068] --------------------
[2017-06-02 12:46:10,090] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:10,090] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:10,090] 200 OK
[2017-06-02 12:46:10,090] --------------------
[2017-06-02 12:46:13,878] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:13,878] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:13,878] 200 OK
[2017-06-02 12:46:13,878] --------------------
[2017-06-02 12:46:14,299] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:14,299] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:14,299] 200 OK
[2017-06-02 12:46:14,299] --------------------
[2017-06-02 12:46:36,965] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:36,965] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:36,965] 200 OK
[2017-06-02 12:46:36,965] --------------------
TIME:  75.2970232963562

 从日志上看，10个上传请求，在1秒钟内从本地发起。并在30秒后开始陆续收到QC服务器的响应。

 

改写的思路：

 

原来的写法:

把10个post请求组装好，在一个for循环中一个一个依次发送。

 

现在的写法：

先把10个post请求组装好，交给asyncio的event loop，然后await这些请求。

 response1 = await future1
 response2 = await future2

 这两句执行的时候，python使用异步的方式，不会阻塞住程序等第一个响应回来，

 而是会直接往下走，把所有请求都发出去。再等服务器一个一个回应。

 

 于是这个小工具的代码逻辑就从：

  执行一段可能是计算也可能是IO的逻辑------》如果是IO的话就等IO完成------》执行一段可能是计算也可能是IO的逻辑------》这样继续循环下去

 变成了：

  执行一段计算--------》异步批量执行一些IO---------》等这一批IO完成----》执行下一段计算--------》异步批量执行一些IO--------》这样继续循环下去

 

 小结完毕，继续学习。。。