Python爬取王者荣耀信息,存入数据库

爬取王者荣耀英雄信息存入数据库

爬取:requests,lxml
数据库插入数据:pymysql

import requests
from lxml import etree
import pymysql

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36'
}

url = "https://pvp.qq.com/web201605/herolist.shtml"
response = requests.get(url=url, headers=headers)
response.encoding = "gbk"
data = response.text
html = etree.HTML(data)
urls = html.xpath('//ul/li/a/img/@src')
names = html.xpath('//ul/li/a/img/@alt')
for i in range(len(urls)):
    print(names[i],urls[i])
    conn = pymysql.connect(db='数据库名', user='用户', password='密码', host='localhost', port=3306, charset='utf8')
    cur = conn.cursor()
    sql = "INSERT INTO `hero` ( `name`, `img_url`) VALUES (%s,%s);"
    param = (names[i],urls[i])
    recount = cur.execute(sql, param)
    conn.commit()
    cur.close()

数据库内容(先建表)

数据库内容

posted on 2020-05-16 12:39  Huhiseven  阅读(115)  评论(0编辑  收藏  举报

导航