利用scrapy.cmdline里的execute快速调试爬虫

1.一直以来都在pycharm的terminal里输入：

scrapy crawl XX #XX代表spider_name

2.建立main.py来快速调试

import os
import sys

from scrapy.cmdline import execute


def carry_out():
    sys.path.append(os.path.dirname(os.path.abspath(__file__)))
    execute(['scrapy', 'crawl', 'xx'])


if __name__ == '__main__':
    carry_out()

 

注意：main.py 要在和scrapy.cfg建在同一级