摘要: pipeline item 阅读全文
posted @ 2017-05-10 17:29 Erick-LONG 阅读(1726) 评论(0) 推荐(0) 编辑
摘要: rules = [ Rule(SgmlLinkExtractor(allow=('/u012150179/article/details'), restrict_xpaths=('//li[@class="next_article"]')), callback='parse_ite... 阅读全文
posted @ 2017-05-10 16:05 Erick-LONG 阅读(778) 评论(0) 推荐(0) 编辑
摘要: UA池 阅读全文
posted @ 2017-05-10 15:05 Erick-LONG 阅读(517) 评论(0) 推荐(0) 编辑
摘要: 放入项目目录,配置setting.py 阅读全文
posted @ 2017-05-10 14:19 Erick-LONG 阅读(649) 评论(0) 推荐(0) 编辑
摘要: class CsvspiderSpider(CSVFeedSpider): name = 'csvspider' allowed_domains = ['iqianyue.com'] start_urls = ['http://iqianyue.com/feed.csv'] headers = ['id', 'name', 'description', 'imag... 阅读全文
posted @ 2017-05-10 13:51 Erick-LONG 阅读(314) 评论(0) 推荐(0) 编辑
摘要: from scrapy.spiders import XMLFeedSpider from myxml.items import MyxmlItem class XmlspiderSpider(XMLFeedSpider): name = 'xmlspider' allowed_domains = ['sina.com.cn'] start_urls = ['http:... 阅读全文
posted @ 2017-05-10 13:35 Erick-LONG 阅读(209) 评论(0) 推荐(0) 编辑
摘要: import scrapy from Autopjt.items import myItem from scrapy.http import Request class AutospdSpider(scrapy.Spider): name = "fulong_spider" start_urls = 阅读全文
posted @ 2017-05-10 13:15 Erick-LONG 阅读(1678) 评论(0) 推荐(0) 编辑
摘要: pipeline部分 item部分 阅读全文
posted @ 2017-05-10 13:01 Erick-LONG 阅读(545) 评论(0) 推荐(0) 编辑