笔记-pyrhon-lib-requests
笔记-pyrhon-lib-requests
1. 简介
Requests is the only Non-GMO HTTP library for Python, safe for human consumption.
1.1. install/upgrade
pip install requests
pip install –upgrade requests
2. 快速入门
2.1. make a Request
引用
>>> import requests
请求：
>>> r = requests.get('https://api.github.com/events')
post:
>>> r = requests.post('https://httpbin.org/post', data = {'key':'value'})
the other HTTP request types: PUT, DELETE, HEAD and OPTIONS? These are all just as simple:
>>> r = requests.put('https://httpbin.org/put', data = {'key':'value'})
>>> r = requests.delete('https://httpbin.org/delete')
>>> r = requests.head('https://httpbin.org/get')
>>> r = requests.options('https://httpbin.org/get')
2.2. passing parameters in urls
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get('https://httpbin.org/get', params=payload)
You can see that the URL has been correctly encoded by printing the URL:
>>> print(r.url)
https://httpbin.org/get?key2=value2&key1=value1
2.3. response content
>>> r = requests.get('https://api.github.com/events')
>>> r.text
u'[{"repository":{"open_issues":0,"url":"https://github.com/...
Requests will automatically decode content from the server. Most unicode charsets are seamlessly decoded.
>>> r.encoding
'utf-8'
>>> r.encoding = 'ISO-8859-1'
改编码格式后调用r.text时该格式会应用，但一般没必要改编码格式。
2.4. binary reponse content
>>> r.content
b'[{"repository":{"open_issues":0,"url":"https://github.com/...
The gzip and deflate transfer-encodings are automatically decoded for you.
2.5. raw response content
有时需要获取原始套接字内容：
>>> r = requests.get('https://api.github.com/events', stream=True)
>>> r.raw
<urllib3.response.HTTPResponse object at 0x101194810>
>>> r.raw.read(10)
'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'
注意stream设置。
2.6. custom headers
定制请求头部
>>> url = 'https://api.github.com/some/endpoint'
>>> headers = {'user-agent': 'my-app/0.0.1'}
>>> r = requests.get(url, headers=headers)
2.7. more complicated POST requests
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.post("https://httpbin.org/post", data=payload)
>>> print(r.text)
{
...
"form": {
"key2": "value2",
"key1": "value1"
},
...
}
2.8. response status codes
查看返回状态码
>>> r = requests.get('https://httpbin.org/get')
>>> r.status_code
200
异常信息查看
当返回4xx或5xx状态码时，查看异常信息：
>>> bad_r = requests.get('https://httpbin.org/status/404')
>>> bad_r.status_code
404
>>> bad_r.raise_for_status()
Traceback (most recent call last):
File "requests/models.py", line 832, in raise_for_status
raise http_error
requests.exceptions.HTTPError: 404 Client Error
如果返回状态码是200，该值为空。
>>> r.raise_for_status()
None
2.9. response headers
返回包的头部：
>>> r.headers
{
'content-encoding': 'gzip',
'transfer-encoding': 'chunked',
'connection': 'close',
'server': 'nginx/1.0.4',
'x-runtime': '148ms',
'etag': '"e1ca502697e5c9317743dc078f67693f"',
'content-type': 'application/json'
}
查看指定字段：
>>> r.headers['Content-Type']
'application/json'
>>> r.headers.get('content-type')
'application/json'
2.10. cookies
查看cookies：
>>> url = 'http://example.com/some/cookie/setting/url'
>>> r = requests.get(url)
>>> r.cookies['example_cookie_name']
'example_cookie_value'
在请求中指定cookies:
>>> url = 'https://httpbin.org/cookies'
>>> cookies = dict(cookies_are='working')
>>> r = requests.get(url, cookies=cookies)
>>> r.text
'{"cookies": {"cookies_are": "working"}}'
Cookies are returned in a RequestsCookieJar, which acts like a dict but also offers a more complete interface, suitable for use over multiple domains or paths. Cookie jars can also be passed in to requests:
>>> jar = requests.cookies.RequestsCookieJar()
>>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies')
>>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere')
>>> url = 'https://httpbin.org/cookies'
>>> r = requests.get(url, cookies=jar)
>>> r.text
'{"cookies": {"tasty_cookie": "yum"}}'
2.11. timeouts
指定请求超时。
>>> requests.get('https://github.com/', timeout=0.001)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)
2.12. errors and exceptions
网络报错：In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.
http异常：Response.raise_for_status() will raise an HTTPError if the HTTP request returned an unsuccessful status code.
超时：If a request times out, a Timeout exception is raised.
重定向：If a request exceeds the configured number of maximum redirections, aTooManyRedirects exception is raised.
基类：All exceptions that Requests explicitly raises inherit fromrequests.exceptions.RequestException.
3. 高级用法
这一部分内容主要讲解定制Requests。
3.1. session objects
session对象允许复用参数，典型的是cookies，它使用urllib3的connecton pooling。
如此可以重用tcp底层。
A Session object has all the methods of the main Requests API.
s = requests.Session()
s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get('https://httpbin.org/cookies')
print(r.text)
# '{"cookies": {"sessioncookie": "123456789"}}'
3.2. Request and Reponse object
简单来说，get()做两件事：
- 构造一个Request对象发给server；
- 将返回的数据生成一个Response对象，该对象包含有Request信息。
>>> r = requests.get('https://en.wikipedia.org/wiki/Monty_Python')
获取头部信息：r.headers
获取请求包头部：r.request.headers
3.3. prepared Requests
from requests import Request, Session
s = Session()
req = Request('POST', url, data=data, headers=headers)
prepped = req.prepare()
# do something with prepped.body
prepped.body = 'No, I want exactly this as the body.'
# do something with prepped.headers
del prepped.headers['Content-Type']
resp = s.send(prepped,
stream=stream,
verify=verify,
proxies=proxies,
cert=cert,
timeout=timeout
)
print(resp.status_code)
