python爬虫错误:http.client.HTTPException: got more than 100 headers的解决方法

python爬虫错误:http.client.HTTPException: got more than 100 headers的解决方法

Traceback (most recent call last):
File "C:\Users\Jonariguez\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "C:\Users\Jonariguez\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 387, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "C:\Users\Jonariguez\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 383, in _make_request
httplib_response = conn.getresponse()
File "D:\Software\Python3\lib\http\client.py", line 1331, in getresponse
response.beg

错误
返回的请求头大多，超过了自定义的最大请求头个数

解决方法：
修改最大请求头个数即可。

python3

import http.client

http.client._MAXHEADERS = 1000

 

python2

import httplib

httplib._MAXHEADERS = 1000