python通过http下载文件的方法&解决MemoryError问题

1、通过requests.get方法

?

1 2 3 4

r = requests.get("http://200.20.3.20:8080/job/Compile/job/aaa/496/artifact/bbb.iso")   with open(os.path.join(os.path.dirname(os.path.abspath("__file__")),"bbb.iso"),"wb") as f:     f.write(r.content)

 

2、urllib2方法

?

1 2 3 4 5 6 7

import urllib2 print "downloading with urllib2" url = '"http://200.21.1.22:8080/job/Compile/job/aaa/496/artifact/bbb.iso"' f = urllib2.urlopen(url) data = f.read() with open(os.path.join(os.path.dirname(os.path.abspath("__file__")),"bbb.iso"),"wb") as f:     f.write(data)

 

3、下载大文件

很多时候，我们下载大文件python报内存错误，你打开任务管理器，会很明显的看到python程序的内存在不停的增大，最终会导致程序奔溃

 

self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
MemoryError

 

我们可以采用下面的代码来解决大文件下载的问题

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

import requests     r = requests.get(     url="http://10.242.255.110/K%3A/SSL/Feature_bridge_mode/alpha/20210809/SDP2.1.7.15_B_Build20210809.run",     stream=True )       f = open("bbb123","wb")   for chunk in r.iter_content(chunk_size=1024):     if chunk:         f.write(chunk)