实例5：IP地址归属地的自动查询

#ip查询全代码
import requests
import time
url='http://www.ip138.com/ips138.asp?ip=202.204.80.112'
r = requests.get(url)
print(r.status_code)
print(r.request.headers)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='sewer.ip138.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E6B69BD198>: Failed to establish
a new connection: [Errno 11001] getaddrinfo failed',))

尝试了很多方法都不能成功  具体请参考：https://blog.csdn.net/a1007720052/article/details/83383220?utm_medium=distribute.pc_relevant.none-task-blog-baidujs-1

但是都没能解决问题最后

 

kv = {'user-agent':'Mozilla/5.0'}
 r = requests.get(url,headers = kv)

import requests
import time
url="https://m.ip138.com/iplookup.asp?ip="
kv = {'user-agent':'Mozilla/5.0'}
r  = requests.get(url + '202.204.80.112', headers = kv)
print(r.status_code)
r.encoding = r.apparent_encoding
print(r.request.headers)
print(r.text[-2000:-1000])

因此以后每次爬取数据一般都先要

kv = {'user-agent':'Mozilla/5.0'}

200
{'user-agent': 'Mozilla/5.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}

<p class="form-hd">IP地址或者域名：</p>
<div class="form">
<form action="" method="get">
<input type="text" name="ip" size="30" class="form-text" value="202.204.80.112" />
<input type="submit" value="查询" class="form-btn" />
</form>
</div>
<div class="query-hd">ip138.com IP查询(搜索IP地址的地理位置)</div>
<h1 class="query">您查询的IP：202.204.80.112</h1><li><font color="blue">ASN归属地：北京市海淀区 北京理工大学 教育网 </font></li><li>参考数据1：北京北京 北京理工大学 教育网</li><li>参考数据2：北京理工大学 网络
中心</li>

</div>
</div>

<div class="footer">
<span>闽网文[2019]2517-043号</span><br/>
<a href="http://www.beian.miit.gov.cn/" rel="nofollow" target="_blank">闽ICP备15026659号-7</a><br/>
<a href="http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=35020302000956" rel="nofollow" target="_blank">闽公网安备35020302000956号</a>
</div>
</div>

<script type="text/javascript" src="/script/common.js"></script>

<script>
(function(){
var $script = document.createElement('script');

 

 

完整框架

 

 

import requests
url="https://m.ip138.com/iplookup.asp?ip="
kv = {'user-agent':'Mozilla/5.0'}
try:
    r  = requests.get(url + '202.204.80.112', headers = kv)
    r.raise_for_status()
    r.encoding = r.apparent_encoding
    print(r.text[-2000:-1000])   #不要一下全输出
except:
    print('爬取失败')