day_1:Requests
常用的发出请求的方法
r = requests.get('https://www.baidu.com') r = requests.post('https://www.baidu.com')
.....
URL传参使用参数param:httpbin.org/get?key1=val1&key2=val2
param = {"key1":"val1","key2":"val2"} r = requests.get('https://httpbin.org/get', params=param)
响应内容
r = requests.get('https://www.baidu.com') print(r.encoding) # 文本编码 r.encoding = 'utf-8' print(r.text) # 文本内容 print(r.content) # 二进制文本 print(r.json()) # json文本
套接字响应内容
url = 'https://www.baidu.com' r = requests.get(url=url, stream=True) # stream=True 必须设置 print(r.raw) # <urllib3.response.HTTPResponse object at 0x10eb15e10> print(r.iter_content(100)) # <generator object iter_slices at 0x10eafdeb8> # r.raw原始的字节流 - 它不会转换响应内容 # r.iter_content 将自动解码gzip和deflate 传输编码
自定义标题
url = 'https://api.github.com/some/endpoint' headers = {'user-agent': 'my-app/0.0.1'} r = requests.get(url, headers=headers)
POST请求(处理表单请求)
payload = {'key1': 'value1', 'key2': 'value2'} r = requests.post("https://httpbin.org/post", data=payload) print(r.text)
响应状态代码r.status_code和r.raise_for_status()(当status_code=200时r
aise_for_status()为None 否则报错
)
url = 'https://www.baidu.com' try: r = requests.get(url=url) print(r.status_code) r.raise_for_status() except: pass
服务器响应头和cookeis:r.headers、r.cookeis
url = 'https://www.baidu.com' try: r = requests.get(url=url) print(r.status_code) r.raise_for_status() r.headers r.cookies except: pass
重定向和历史
url = 'https://www.baidu.com' try: r = requests.get(url=url, allow_redirects=False) # 禁止重定向 r.raise_for_status() r.history except: pass
设置超时时间(秒)
url = 'https://www.baidu.com' try: r = requests.get(url=url,time=5) r.raise_for_status() r.history except: pass
会话Session(保持cookies等登录状态)
s = requests.Session() s.auth = ('user', 'pass') s.headers.update({'x-test': 'true'}) # both 'x-test' and 'x-test2' are sent s.get('https://httpbin.org/headers', headers={'x-test2': 'true'})
SSL证书验证
url = 'https://www.baidu.com' try: r = requests.get(url=url, verify=False) # 忽略ssl证书验证 r.raise_for_status() except: pass
代理
proxies = { 'http': 'http://10.10.1.10:3128', 'https': 'http://10.10.1.10:1080', } proxies = {'http': 'http://user:pass@10.10.1.10:3128/'} # 带用户密码 requests.get('http://example.org', proxies=proxies)