Python requests

Python requests

Requests

常用的 HTTP 请求有 GET, POST, PUT, DELETE
还有 OPTION 等

使用 requests 发送请求

import requests

requests.get(url, params=None, **kwargs)
requests.post(url, data=None, json=None, **kwargs)
requests.put(url, data=None, **kwargs)
requests.delete(url, **kwargs)
requests.head(url, **kwargs)
requests.options(url, **kwargs)

添加参数

添加参数只需要在发送请求时传入参数字典即可

发送 GET 请求如果需要添加一般参数还可以使用 URL 拼接的方法：
示例：

import requests
requests.get('https://httpbin.org/get?name=doubtful&blog=cnblogs.com/dbf-')

添加一般参数：

import requests
data = {
    'name': 'doubtful',
    'blog': 'cnblogs.com/dbf-'
}
requests.get('https://httpbin.org/get', params=data)
requests.post('https://httpbin.org/get', data=data)

添加请求头：

import requests
headers = {
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/'
                  '76.0.3809.100 Safari/537.36'
}
requests.get('https://httpbin.org/get', headers=headers)
requests.post('https://httpbin.org/get', headers=headers)

发送文件

import requests
files = {'file': open('1.jpg', 'rb')}
response = requests.post('https://httpbin.org/post', files=files)

会话维持

如果直接使用多次 requests 每一次请求都是独立请求，无法维持会话，需要使用 requests.Session 来维持会话
未使用 Session ：

import resquests
resquests.get('https://httpbin.org/cookies/set/user/doubtful')
response = resquests.get('https://httpbin.org/cookies')
print(response.text)

执行结果：

{
  "cookies": {}
}

使用 Session ：

import requests
s = requests.Session()
s.get('https://httpbin.org/cookies/set/user/doubtful')
response = s.get('https://httpbin.org/cookies')
print(response.text)

执行结果：

{
  "cookies": {
    "user": "doubtful"
  }
}

证书验证

当对 HTTPS 网站发送请求时会先对网站证书进行检查
如果网站证书有问题会报错，可以通过设置 verify 参数使程序不终止，但依然会有警告

import requests
response = requests.get('https://httpbin.org/get', verify=False)

可以通过调用 urllib3.disable_warning() 不显示警告

import requests
from requests import urllib3
urllib3.disable_warning()
response = requests.get('https://httpbin.org/get', verify=False)

指定证书

import requests
response = requests.get('https://httpbin.org/get', cert=('/path/server.crt', '/path/key'))

代理设置

HTTP 代理

import requests
proxies = {
	'http': 'http://127.0.0.1:9999/',
	'https': 'https://127.0.0.1:9999/'
}
response = requests.get('https://httpbin.org/get', proxies=proxies)

需要登录的代理：

import requests
proxies = {
	'http': 'user:password@http://127.0.0.1:9999/'
}
response = requests.get('https://httpbin.org/get', proxies=proxies)

SOCKS 代理

使用 SOCKS 代理需要先安装

pip install requests[socks]

import requests
proxies = {
	'http': 'socks5://127.0.0.1:9999/',
	'https': 'socks5://127.0.0.1:9999/'
}
response = requests.get('https://httpbin.org/get', proxies=proxies)

设置超时

设置超时时间之后，如果超时就会报错

import requests
requests.get('https://httpbin.org/get', timeout=1)

异常处理

如果不希望程序中断可以捕捉到超时错误，使程序不中断

import requests
from requests.exceptions import Timeout
try:
	requests.get('https://httpbin.org/get', timeout=1)
except Timeout:
	print('timeout')

认证设置

import requests
from requests.auth import HTTPBasicAuth
response = r.get('https://httpbin.org/get', auth=HTTPBasicAuth('user', '123'))

import requests
response = r.get('https://httpbin.org/get', auth=('user', '123'))

Response

Response 的方法：

Response.text		# 获取响应内容（Unicode）
Response.json()		# 返回 JSON 格式的响应内容	等于 json.loads(Response.text)
Response.content	# 返回 bytes 类型的响应内容

Response 的属性：

Response.status_code	# 响应的状态码
Response.headers		# 响应头
Response.cookies		# 服务器传回的 Cookies
Response.url 			# 最终响应的 URL
Response.history		# 历史 Requests 得到 Response 对象的列表