代理ip的使用

selenium中使用代理ip

# coding=utf8
"""
author:dengjiyun
"""
from selenium import webdriver

options= webdriver.ChromeOptions()

options.add_argument("--proxy-server=http://119.179.139.168:8060")

driver = webdriver.Chrome(chrome_options=options)

# driver.get("http://httpbin.org/ip")

requests 模块中使用代理ip

# coding=utf8
"""
author:dengjiyun
"""

import requests

url='https://www.baidu.com'

proxies={
    'http':'119.179.139.168:8060',

}
res=requests.get(url,proxies=proxies)

print(res.text)

with open('file.html','w',encoding="utf-8") as f:

    f.write(res.text)

3. scrapy中使用代理 ，在中间件中定义一个随机代理类，然后在settings中注册