scrapy在 cmd创建爬虫在shell中操作

在控制台调试： scrapy shell [url]

scrapy startproject 项目名称      
scrapy genspider 蜘蛛名称 目标网站（example.com）
File->setting->Project Interpretor->add Local
from scrapy cmdline     import execute
os.path.abspath(__path__)
os.path.dirname()
sys.path.append()
execute(["scrapy","crawl","jobbole"])
修改 ROBOTSTXT_OBEY=False
response.xpath()