浙江省高等学校教师教育理论培训

微信搜索“毛凌志岗前心得”小程序

  博客园  :: 首页  :: 新随笔  :: 联系 :: 订阅 订阅  :: 管理
  12653 随笔 :: 2 文章 :: 617 评论 :: 609万 阅读

Scraping multiple sites in parallel and memory consumption - Google Groups

Ross Chapman
Post reply


More message actions 

 


 

May 4 (2 days ago)

Depending on how thorough you're scraping, you can disable image loading, running javascript, and loading plugins (flash, silverlight, etc...) when constructing casper in the options argument.  This will definitely save memory.
Eg:
var casper = require('casper').create({
    loadImages:  false,
    javascriptEnabled: false,
    loadPlugins: false
});
posted on   lexus  阅读(202)  评论(0编辑  收藏  举报
点击右上角即可分享
微信分享提示