python requests下载文件慢

使用python的三方库 requests 在下载文件时，很慢。

源代码：直接根据返回内容写文件，文件并不是很大，但是写起来很慢

     response = requests.post('https://xxxx', data=json.dumps(payload), headers=headers,
                      cookies=cookies, proxies=proxies)
     if response.content:
         with open(target_path, 'wb') as f:
              f.write(response.content)
         return response.text

 

调整后代码：

 1 url="https://xx.gin.csv"
 2 response = requests.get(url, stream = True)
 3 
 4 text_file = open("data.txt","wb")
 5 for chunk in response.iter_content(chunk_size=1024):
 6     text_file.write(chunk)
 7 writing one chunk at a time to text file
 8 
 9 text_file.close()

 

 with requests.post('https://xx.com/download', data=json.dumps(payload), headers=headers,
                       cookies=cookies, proxies=proxies) as r:
        r.raise_for_status()
        with open(target_path, 'wb') as f:
            for chunk in r.iter_content(chunk_size=2048):
                # If you have chunk encoded response uncomment if
                # and set chunk_size parameter to None.
                # if chunk:
                f.write(chunk)
        delete_yhhd_file(cookies, file_code)  # 删除文件请求

 

posted @ 2022-03-07 16:10  normalpers  阅读(1329)  评论(0)  编辑  收藏  举报