scrapy 基本命令

创建scrapy项目

$ scrapy startproject project_name

创建爬虫文件

scrapy genspider [-t template] <name> <domain>

$ scrapy genspider -l
Available templates:
  basic
  crawl
  csvfeed
  xmlfeed

爬虫进行抓取

$ scrapy crawl spider_name --logfile spider_name.log

“故障排除,调试”

scrapy shell  [-s "User-Agent"=" "] http://www.baidu.com

response.text

response.body

response.css("  ::text").extract_first()

response.xpath("/text()").extract()[0]