数据抓取

1：分析网页

市场信息-最新楼盘

http://www.sjzfgj.gov.cn/plus/scxx_zxlp.php

分住宅和非住宅两个分类

参数：

pageno 页数

type 楼盘分类 1表示住宅，2表示非住宅

住宅uri：

http://www.sjzfgj.gov.cn/plus/scxx_zxlp.php?pageno=1&type=1

非住宅uri：

http://www.sjzfgj.gov.cn/plus/scxx_zxlp.php?pageno=1&type=2

 

数据格式：

<tr align="center">
	<td><a href="/plus/scxx_xminfo.php?id=16440" class="a_lan">东南智汇城（香溪园）</a></td>
	<td>河北中宏置业房地产开发有限公司</td>
	<td>仓丰路161号</td>
	<td>89616020</td>
	</tr>

 

 

 

 

 

有href看出有详情页信息，拼接后uri为：

http://www.sjzfgj.gov.cn/plus/scxx_xmdetail.php?id=16440

点进去后显示信息为：销售许可公示

等等详细信息，查看源代码后发现

 最新楼盘样式为：

<div class="scxx_search">
	<h2 class="new_loupan">最新楼盘</h2>
	<ul class="easyui-tree tree" id="mytree" data-options="
	animate:true,
	url:'cxda_ys_json_menu.php?id=16440'
	">
	</ul>
	</div>

 

 

 

 

 

 

 

项目详细信息样式为：

<div class="scxx_jieguo" id="content" style="height:1200px">
	<h2>东南智汇城（香溪园）</h2>
	<iframe id="main" frameborder="0" width="100%" height="100%" src="scxx_xmdetail.php?id=16440" ></iframe>
	</div>

 

 

 

 

可知数据是通过uri传过来的。

 

可知项目详细信息的uri为：

http://www.sjzfgj.gov.cn/plus/scxx_xmdetail.php?id=16440

数据格式：

<tr>
	<td align="middle" bgcolor="#edf4fa" width="90">
	其他资金
	</td>
	<td bgcolor="#ffffff">
	￥
	9600000000
	</td>
	</tr>

 

 

 

 

 

 

 

最新楼盘拼接uri后为：

http://www.sjzfgj.gov.cn/plus/cxda_ys_json_menu.php?id=16440

网页内容为

[{"id":"2016102108464031325","text":"2016097\u53f7","state":"open","attributes":{"url":"scxx_presale_show.php?projid=16440&id=2016102108464031325"},"children":[{"id":"0200020229002","text":"7\u53f7\u4f4f\u5b85\u697c","state":"open","attributes":{"url":"scxx_floor_show.php?projid=16440&id=0200020229002"},"children":[{"id":"0200020229002","text":"\u5730\u4e0b\u5ba4\u6216\u5546\u94fa","attributes":{"url":"scxx_subroom_show.php?sub=00&id=0200020229002"}},{"id":"0200020229002","text":"01\u5355\u5143","attributes":{"url":"scxx_subroom_show.php?sub=01&id=0200020229002"}},{"id":"0200020229002","text":"02\u5355\u5143","attributes":{"url":"scxx_subroom_show.php?sub=02&id=0200020229002"}}]},{"id":"0200020229003","text":"8\u53f7\u4f4f\u5b85\u697c","state":"open","attributes":{"url":"scxx_floor_show.php?projid=16440&id=0200020229003"},"children":[{"id":"0200020229003","text":"\u5730\u4e0b\u5ba4\u6216\u5546\u94fa","attributes":{"url":"scxx_subroom_show.php?sub=00&id=0200020229003"}},{"id":"0200020229003","text":"01\u5355\u5143","attributes":{"url":"scxx_subroom_show.php?sub=01&id=0200020229003"}},{"id":"0200020229003","text":"02\u5355\u5143","attributes":{"url":"scxx_subroom_show.php?sub=02&id=0200020229003"}}]},{"id":"0200020229009","text":"9\u53f7\u4f4f\u5b85\u697c","state":"open","attributes":{"url":"scxx_floor_show.php?projid=16440&id=0200020229009"},"children":[{"id":"0200020229009","text":"\u5730\u4e0b\u5ba4\u6216\u5546\u94fa","attributes":{"url":"scxx_subroom_show.php?sub=00&id=0200020229009"}},{"id":"0200020229009","text":"01\u5355\u5143","attributes":{"url":"scxx_subroom_show.php?sub=01&id=0200020229009"}},{"id":"0200020229009","text":"02\u5355\u5143","attributes":{"url":"scxx_subroom_show.php?sub=02&id=0200020229009"}}]}]}]

 再次拼接uri后得到

1：http://www.sjzfgj.gov.cn/plus/scxx_presale_show.php?projid=16440&id=2016102108464031325  此为项目的另一个详情页：项目编号，已投资，总投资，开工日期，竣工日期等属性

2：http://www.sjzfgj.gov.cn/plus/scxx_floor_show.php?projid=16440&id=0200020229002 预售申请楼盘信息 如图

3：http://www.sjzfgj.gov.cn/plus/scxx_subroom_show.php?sub=00&id=0200020229002 单元楼信息

查看源代码 格式为：

<td align="center" bgcolor="#FFFFFF">-2</td>
	<td align="center"><img src="/images/Red.gif" alt="状态">
	<a href="scxx_showhouse.php?id=2016102010071780371" class="w">0201</a>
	</td>

 

 

<td align="center" bgcolor="#FFFFFF">-1</td>
	<td align="center"><img src="/images/Green.gif" alt="状态">
	<a href="scxx_showhouse.php?id=2016102010071780329" class="w">0101</a>
	</td>
	<td align="center"><img src="/images/Green.gif" alt="状态">
	<a href="scxx_showhouse.php?id=2016102010071780308" class="w">0102</a>
	</td>

可知图片颜色为状态，Green为可售，Red为不可售

有href 再次拼接uir

得到 http://www.sjzfgj.gov.cn/plus/scxx_showhouse.php?id=2016102010071780329

为户号详情

 

还有多个uri

http://www.sjzfgj.gov.cn/plus/scxx_subroom_show.php?sub=01&id=0200020229003