爬虫二之Requests
requests
实例引入
import requests
response = requests.get('https://www.baidu.com')
response.status_code
response.text
response.cookies
请求方式
post()
put()
delete()
head()
options()
请求
基本get请求
带参数get请求
data = {'name':'germey', 'age':'22}
response = request.get('http://httpbin.org/get', params=data)
print(respones.text)
解析json
response.json()
获取二进制数据
response.content
response=request.get('https://github.com/favicon.ico')
f = open('favicon.ico', 'wb')
f.write(response.content)
f.close()
添加headers
headers={
'User-Agent':'Mozilla/5.0 (Macintosh; intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36'
}
response = request.get('https://www.zhihu.com/explore'. headers=headers_
POST请求
data = {}
headers = {}
response = request.post('http://httpbin.org/post', data=data, headers=headers)
response属性
status_code
headers
cookies
url
history
高级操作
文件上传
files = {'file':open('favicon.ico','rb')}
response = request.get('http://httpbin.org/post', files=files)
获取cookie
for key,value in response.cookies.items():
print(key + '=' + value)
会话维持
requests.get('http://httpbin.org/cookies/set/number/123456789)
response = requests.get('http://httpbin.org/cookies')
上述方法无法得到想要的cookie
s = requests.Session()
s.get(...)
response = s.get(...)
证书验证
暂时不看。如果发生情况则添加参数 verify=False
代理设置
proxies={}
response = requests.get(' ', proxies=proxies)
超时设置
from requests.exceptions import ReadTimeout
try:
#some codes
except ReadTimeout:
print('Timeout')
认证设置
request.get(...,auth={'user','123'})
