大众点评反爬虫机制,用户评论自编码替换

本人是研究推荐系统的,最近在看到一篇文章Exploiting Ranking Consistency Principle in Representation Learning for Location Promotion觉得挺有意思的,想深入了解一下,然后想着从大众点评上爬点数据,于是就随手打开大众点评,点开评论,打开源码,研究一下爬虫策略,这才发现,现在大众点评除了在代理IP上下功夫反爬虫外,在源码上也做文章了,具体如下:

网页源码:

<span class="fr-DMrt"></span>店我<span class="fr-Ky0H"></span><span class="fr-QmtS"></span><span class="fr-SHzo"></span><span class="fr-mUfe"></span><span class="fr-mAqU"></span>call,<span class="fr-SHzo"></span><span class="fr-OL5j"></span>前在合肥吃过<span class="fr-QmtS"></span>最好吃<span class="fr-QmtS"></span><span class="fr-DMrt"></span><span class="fr-AaxC"></span>锅店了。<br />首先<span class="fr-Eg8Q"></span><span class="fr-DMrt"></span><span class="fr-QmtS"></span>毛肚<span class="fr-Ky0H"></span><span class="fr-QmtS"></span>超棒,<span class="fr-BK2E"></span><span class="fr-APZx"></span><span class="fr-SHzo"></span>能看<span class="fr-QmtS"></span><span class="fr-gnD7"></span><span class="fr-QmtS"></span>棒,脆脆<span class="fr-QmtS"></span>口感,有些<span class="fr-AaxC"></span>锅店<span class="fr-QmtS"></span>毛肚端上<span class="fr-gnD7"></span><span class="fr-wbqv"></span>显能看出......</p> <a href="javascript:void(0);" class="more fold J-more">展开评论<i class="icon"></i></a> </div> <div class="info J-info-all clearfix Hide"> <p class="desc J-desc">这家店我真<span class="fr-QmtS"></span><span class="fr-SHzo"></span>强烈打call,<span class="fr-SHzo"></span>我目前在合肥吃过<span class="fr-QmtS"></span>最好吃<span class="fr-QmtS"></span><span class="fr-dK0T"></span>家火锅店了。<br />首先他家<span class="fr-QmtS"></span>毛肚真<span class="fr-QmtS"></span>超棒,<span class="fr-BK2E"></span><span class="fr-SHzo"></span>能看<span class="fr-QmtS"></span>出来<span class="fr-QmtS"></span>棒,脆脆<span class="fr-QmtS"></span><span class="fr-2QAf"></span>感,有些火锅店<span class="fr-QmtS"></span>毛肚端上来明显能看出来发白<span class="fr-QmtS"></span>痕迹,明显<span class="fr-SHzo"></span>冻过<span class="fr-QmtS"></span>,而它家<span class="fr-SHzo"></span>冰鲜<span class="fr-ZTIr"></span><span class="fr-QmtS"></span><br />黄喉也<span class="fr-OnYW"></span>好吃,虽然这个<span class="fr-BK2E"></span><span class="fr-QmtS"></span>黄喉合肥有好几家能做到,但<span class="fr-SHzo"></span><span class="fr-wlIq"></span>格没它家便宜<span class="fr-jT0z"></span>!!胜在性<span class="fr-wlIq"></span>比高<span class="fr-jT0z"></span>!!<br /><span class="fr-sMte"></span>还点了<span class="fr-dK0T"></span><span class="fr-xiFS"></span>,哇塞,这个分量你<span class="fr-sMte"></span>可以看图<span class="fr-dK0T"></span>,分量真<span class="fr-QmtS"></span><span class="fr-OnYW"></span><span class="fr-72sB"></span><span class="fr-xiFS"></span>片被片得<span class="fr-OnYW"></span>薄,在火锅里涮几十秒<span class="fr-O0dH"></span>能吃了!<br /><span class="fr-dK0T"></span>直觉得吃火锅<span class="fr-O0dH"></span><span class="fr-SHzo"></span>吃锅底,菜大同小异都差不多,我<span class="fr-sMte"></span><span class="fr-QmtS"></span><span class="fr-SHzo"></span>魔鬼辣,但<span class="fr-SHzo"></span>相信我,<span class="fr-dK0T"></span>点都不辣!!!!!!中间<span class="fr-SHzo"></span>有点药膳味道<span class="fr-QmtS"></span>鸡汤?可能<span class="fr-SHzo"></span>吧,白汤<span class="fr-O0dH"></span><span class="fr-SHzo"></span><span class="fr-OnYW"></span>普通<span class="fr-QmtS"></span>红枣香菇汤。。。涮菜<span class="fr-QmtS"></span>话基本没用白汤。这样感觉有点浪费。强制鸳鸯!!!但辣汤真<span class="fr-QmtS"></span><span class="fr-OnYW"></span>好吃,不咸,不像有<span class="fr-QmtS"></span>店越吃越咸!!!<br />服务<span class="fr-OnYW"></span>好,<span class="fr-vsrl"></span><span class="fr-ALv0"></span>毛巾擦手之<span class="fr-10Lg"></span>!服务<span class="fr-ALv0"></span><span class="fr-2a4U"></span><br /><span class="fr-dK0T"></span>不太好<span class="fr-QmtS"></span><span class="fr-O0dH"></span><span class="fr-SHzo"></span>卫生问题,可能我去<span class="fr-QmtS"></span>时间太晚了,调料区挖葱<span class="fr-QmtS"></span>时候小虫子乱飞,挺影响心<span class="fr-2a4U"></span><span class="fr-QmtS"></span><br />不管怎么说,<span class="fr-SHzo"></span><span class="fr-vsrl"></span>二刷<span class="fr-QmtS"></span>店!</p> <a href="javascript:void(0);" class="more unfold J-more">收起评论

截取其中一句:

<span class="fr-DMrt"></span>店我<span class="fr-Ky0H"></span><span class="fr-QmtS"></span><span class="fr-SHzo"></span><span class="fr-mUfe"></span><span class="fr-mAqU"></span>call,

可以发现其中有些字被网页元素替换了,加载到前端的时候再显示原始评论,可以说很灵性了。

 

posted @ 2018-07-19 20:53  爱吃牛肉面  阅读(5361)  评论(2编辑  收藏  举报