Python爬虫:lxml模块分析并获取网页内容

运用css选择器:

# -*- coding: utf-8 -*-
from lxml import html
page_html = '''
<html><body>
<input id="input_id" value="input value" name="input_a">
</body></html>
'''
page_tree = html.fromstring(page_html.decode('utf-8'))
ele = page_tree.cssselect('#input_id')  # 用css选择器的id选择器获取网页内容
print html.tostring(ele[0]) # <input id="input_id" value="input value" name="input_a">
print ele         # [<InputElement 30133f0 name='input_a' type='text'>]
print ele[0]      # <InputElement 30133f0 name='input_a' type='text'>
print ele[0].get('value')   # input value

获取标签里的内容:

# -*- coding: utf-8 -*-
from lxml import html
page_html = '''
<html><body>
<div class="cl">DIV1</div>
<div class="cl">DIV2</div>
</body></html>
'''
page_tree = html.fromstring(page_html.decode('utf-8'))
ele = page_tree.cssselect('body')[0].findall("div") # findall寻找所有的直接子标签
print ele[0].text_content().strip() # DIV1

若提示如下错误:
from lxml import html
ImportError: DLL load failed: %1 is not a valid Win32 application.
尝试重新安装lxml模块:

python -m pip uninstall lxml
python -m pip install lxml==3.6.0
posted @ 2018-12-28 09:05  xuejianbest  阅读(1301)  评论(0编辑  收藏  举报