定时爬虫抓当日免费应用:Scrapy + Tkinter + LaunchControl
花了个周末学了下Scrapy,正好一直想买mindnode,于是顺手做了个爬虫,抓取爱范儿每天的限免应用信息。
Thinking
大概思路就是使用LaunchControl每天定时(比如早上9点50,这时正好刚到公司不久)跑一下爬虫脚本,如果找到感兴趣的应用在限免,就使用Tkinter弹出提示。当然,也可以直接用Scrapy做定时任务,以后再说。
Coding
Scrapy + Tkinter
# -*- coding: utf-8 -*-
import scrapy
import Tkinter
from scrapy.shell import inspect_response
import json
# 设置感兴趣的app名称
I_want_apps = set(['mindnode pro', 'u.memory'])
class XianmianSpider(scrapy.Spider):
user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36'
name = "xianmian"
allowed_domains = ["app.so"]
start_urls = (
'http://app.so/api/v1.1/appso/discount/?platform=web&limit=10',
)
def parse(self, response):
jsonresponse = json.loads(response.body_as_unicode())
apps = jsonresponse['objects']
appTitles = {item['display_name'].lower() for item in apps}
self.logger.info('today\' apps are: ' + str(appTitles))
the_apps = appTitles & I_want_apps
if the_apps:
self.showMsg('found the apps: {}'.format(list(the_apps)))
def showMsg(self, msg):
import Tkinter
root = Tkinter.Tk()
root.title('福利到!')
label = Tkinter.Label(root, text=msg)
label.pack()
center_window(root, 300, 240)
root.maxsize(600, 400)
root.minsize(300, 240)
Tkinter.mainloop()
def get_screen_size(window):
return window.winfo_screenwidth(),window.winfo_screenheight()
def get_window_size(window):
return window.winfo_reqwidth(),window.winfo_reqheight()
def center_window(root, width, height):
screenwidth = root.winfo_screenwidth()
screenheight = root.winfo_screenheight()
size = '%dx%d+%d+%d' % (width, height, (screenwidth - width)/2, (screenheight - height)/2)
print(size)
root.geometry(size)
LaunchControl
LaunchControl用起来比较直观。当然,也可以直接用mac自带的launchctl,具体可参考launchctl使用说明