python scrapy 报错 DEBUG: Ignoring response 403

 

DEBUG: Ignoring response <403 http://movie.douban.com/top250>: HTTP status code is not handled or not allowed

怎么回事呢，被屏蔽了，我们来伪装一下，在settings.py里加上USER_AGENT：

 

解决方法：

在setting.py文件中增加USER_AGENT配置：(随便写一条就可以了)

USER_AGENT = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36' 

or

USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.54 Safari/536.5'