BeautifulSoup实例
#coding:utf8
from bs4 import BeautifulSoup
html__doc = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
soup = BeautifulSoup(html__doc, 'html.parser', from_encoding='utf-8')
links = soup.find_all('a')
for link in links:
print (link.name, link['href'], link.get_text())
beautifulsoup的参数:文档字符串、解析器、编码;
创建好soup对象后,获取所有链接(含字符串‘a’);
得到节点的名称、url、文字
运行显示:
(u'a', u'http://example.com/elsie', u'Elsie')
(u'a', u'http://example.com/lacie', u'Lacie')
(u'a', u'http://example.com/tillie', u'Tillie')
print '获取lacie的链接'
link_node = soup.find('a', href='http://example.com/lacie')
print link_node.name, link_node['href'], link_node.get_text()
运行显示:
获取lacie的链接
a http://example.com/lacie Lacie
print '正则匹配'
link_node = soup.find('a', href=re.compile(r"ill"))
print link_node.name, link_node['href'], link_node.get_text()
运行显示:
正则匹配
a http://example.com/tillie Tillie
print '获取p段落文字'
p_node = soup.find('p', class_="title")
print p_node.name, p_node.get_text()
运行显示:
获取p段落文字
p The Dormouse's story