摘要: 融合了scrapy的架构理念和twisted的任务调度思想,目前只是个精简版的,开源地址https://github.com/aware-why/multithreaded_crawler/,已附带demo演示的例子。有兴趣的可以参与进来,有更好想法的人可以直接发起pull request,热烈欢迎大家的贡献。multithreaded_crawlerA condensed crawler framework of “multithreaded model”dependencyAt present, the framework depends on nothing except for mod 阅读全文
posted @ 2014-01-23 17:36 6ruce 阅读(634) 评论(0) 推荐(0) 编辑