爬虫4:cookie登陆并爬取学习成绩
最近项目太忙了,都没空继续学习爬虫,前几天实验了各种姿势带cookie去登陆,都没有成功,不明觉厉
依然没有放弃,今天用同样的办法登陆以前大学的网站居然成功了,我真是一脸懵逼
又看了一遍以前的学习成绩,真是恨不得给自己两耳屎,青春都让狗吃了
哦,对了,以前还有一段学selenium的学习笔记,空了整理放到博客园里面来
# -*- coding:utf-8 -*- import requests from bs4 import BeautifulSoup head={'cookie':'填入抓包抓来的cookie', 'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0', 'Host':'jxgl.cuit.edu.cn', 'Connection':'keep-alive', 'Referer':'http://210.41.224.117/Login/xLogin/Login.asp', 'Upgrade-Insecure-Requests': '1' } # url = 'http://jxgl.cuit.edu.cn/Jxgl/Xs/MainMenu.asp' # req = requests.get(url, headers=head) detail = requests.get('http://jxgl.cuit.edu.cn/Jxgl//UserPub/GetCjByXh.asp?UTp=Xs',headers=head).text soup = BeautifulSoup(detail, "html.parser") blist = soup.find_all('b') for item in blist: print item.get_text()
我走的很慢,但从不后退