最近要租房子,用Python看一下房源吧..

前言：最近我的朋友想要租房子,为了装个b,决定运用技术去帮助他。

这个网站是什么我也不知道反正是一个房子交易网站 http://www.ljia.net/

设置请求头

headers = {'Accept': '*/*',
               'Accept-Language': 'en-US,en;q=0.8',
               'Cache-Control': 'max-age=0',
               'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36',
               'Connection': 'keep-alive',
               'Referer': 'http://www.baidu.com/'
               }

代码：

import requests
from bs4 import BeautifulSoup
import re;


headers = {'Accept': '*/*',
               'Accept-Language': 'en-US,en;q=0.8',
               'Cache-Control': 'max-age=43200',
               'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36',
               'Connection': 'keep-alive',
               'Referer': 'no-referrer-when-downgrade'
               }

def get_Detail_List(pageindex):
    url = "http://www.ljia.net/zf/p-"+str(pageindex)+'.html'
    response = requests.get(url)
    soup = BeautifulSoup(response.text,'html.parser')
    resultTitle = soup.find_all('h2',{'class','title'})
    for i in resultTitle:
        urler = 'http://www.ljia.net'+i.a['href']
        response = requests.get(urler)
        souper = BeautifulSoup(response.content)
        content = souper.select('div #lpname div[class=floatl]')
        dj = BeautifulSoup(str(content))
        money = souper.find_all('div',{'class','trl-item'})
        moneybs = BeautifulSoup(str(money))
        xinxiList = souper.find_all('div',{'class','tt'})
        xinxi = BeautifulSoup(str(xinxiList))
        consolestr = dj.get_text().replace('[','').replace(']','').strip()+moneybs.get_text().replace('[','').replace(']','').strip()+xinxi.get_text().replace('[','').replace(']','').strip()
        print(consolestr.strip().replace(' ', '').replace('\n', '').replace('\t', '').replace('\r', '').strip())

count = 0
while(count<100):
    get_Detail_List(count)
    count = count+1

哈哈就是这么简单,让我们看下效果图吧！！！

然后,你可以把这些数据放到Mysql,Redis 那我可管不着都可以的了

import pymysql.cursors


# 连接MySQL数据库
connection = pymysql.connect(host='127.0.0.1', port=3306, user='root', password='198876', db='guest', 
                             charset='utf8mb4', cursorclass=pymysql.cursors.DictCursor)

# 通过cursor创建游标
cursor = connection.cursor()

# 创建sql 语句，并执行
sql = "INSERT INTO `users` (`email`, `password`) VALUES ('huzhiheng@itest.info', '123456')"
cursor.execute(sql)

# 提交SQL
connection.commit()