【python爬虫】scrapy入门2--自定义item

items.py

class LianhezaobaospyderItem(scrapy.Item):
    # define the fields for your item here like:
    # name = scrapy.Field()
    # pass
    body=scrapy.Field()
    link=scrapy.Field()

爬虫.py

from .. import items

def parse_news(self,response):
    item=items.LianhezaobaospyderItem()                
    item['body']=response.xpath("//div[@class='xx']/text()").get()
    item['link']=response.url
    yield item    

item和字典类似,数据量大时,字典可能键值对错误

posted @ 2020-05-07 18:01  HuaBro  阅读(326)  评论(0编辑  收藏  举报