scrapy.downloadermiddlewares.redirect 302自动重

setting.py加入

COOKIES_ENABLED = False

设置

DEFAULT_REQUEST_HEADERS = {
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',
'Accept-Language': 'zh-CN,zh;q=0.9',
'cookie':'xxxxxxxxx'
}

yield scrapy.Request(url,dont_filter=True)
posted @ 2021-12-12 20:37  myrj  阅读(186)  评论(0)  编辑  收藏  举报