[Node.js] Scraping Dynamic JavaScript Websites with Nightmare

Many websites have more than just simple static content. Dynamic content which is rendered by JavaScript requires browser to be able to scrape data. This video demonstrates how to use Nightmare (which is a wrapper around PhantomJS) to launch a url and scrape dynamic data.

 

import Nightmare from "nightmare";

new Nightmare()
    .goto('http://weather.com')
    .evaluate(function(){
        // access the dom element
        return document.querySelector('.temperature').innerText;
    }, function(temperature){
        // get the return value from the first param
        console.log(temperature);
    })
    .run();