安装好scrapy后,开始创建项目
项目名:zhaopin 爬虫文件名:zhao
1:cmd -- scrapy startproject zhaopin
2:cd zhaopin,进入项目目录
3:scrapy genspider zhao http://sou.zhaopin.com
运行:
1:cmd操作 --- scrapy crawl zhao
如果报错robots.txt 缺失,修改再项目下settings.py 中22行的ROBOTSTXT_OBET = True 改成ROBOTSTXT_OBEY = False
2:pycharm操作 ---
在项目目录下建立main.py
#encoding: utf-8
from scrapy import cmdline cmdline.execute("scrapy crawl zhao".split())