Python编程快速上手-从Web抓取信息

利用webbrowser模块

 　　webbrowser模块的open()函数可以启动一个新浏览器

+ View Code?

1 2 3

>>> import webbrowser >>> webbrowser.open('http://www.baidu.com/') True

 

用requests模块从Web下载文件

用requests.get() 函数下载一个网页

+ View Code?

1 2 3 4 5 6 7 8 9 10 11 12 13

>>> import requests >>> res = requests.get('http://www.gutenberg.org/cache/epub/1112/pg1112.txt') >>> type(res) <class 'requests.models.Response'> >>> len(res.text) 179380 >>> print(res.text[:250]) The Project Gutenberg EBook of Romeo and Juliet, by William Shakespeare     ******************************************************************* THIS EBOOK WAS ONE OF PROJECT GUTENBERG'S EARLY FILES PRODUCED AT A TIME WHEN PROOFING METHODS AND TOO

检查错误

+ View Code?

1 2 3 4 5 6 7 8

>>> res = requests.get('http://inventwithpython.com/page_that_does_not_exist') >>> res.raise_for_status() Traceback (most recent call last):   File "<stdin>", line 1, in <module>   File "C:\Python37\lib\site-packages\requests\models.py", line 960, in raise_for_status     raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://inventwithpython.com/page_that_does_not_exist >>>

　　可以用try 和 except 语句将raise_for_status() 代码行包裹，处理这个错误，不让程序崩溃

+ View Code?

1 2 3 4 5 6

import requests res = requests.get('http://inventwithpython.com/page_that_does_not_exist') try:   res.raise_for_status() except Exception as exc:   print('There was a problem: %s' % (exc))

将下载的文件保存在磁盘

+ View Code?

1 2 3 4 5 6 7 8 9 10 11 12

>>> import requests >>> res = requests.get('http://www.gutenberg.org/cache/epub/1112/pg1112.txt') >>> res.raise_for_status() >>> playFile = open('RomeoAndJuliet.txt', 'wb') >>> for chunk in res.iter_content(1000000): ...   playFile.write(chunk) ... 179382 >>> >>> playFile.close() >>> >>>