Hadoop Yarn core concepts

   

The fundamental idea of YARN is to split the two major responsibilities of the JobTracker—that is, resource management and job scheduling/monitoring—into separate

daemons: a global ResourceManager and a per-application ApplicationMaster (AM).

The ResourceManager and per-node slave, the NodeManager (NM), form the new,

and generic,operating system for managing applications in a distributed manner.

   

The NodeManager is the per-machine slave, which is responsible for launching the

applications' containers, monitoring their resource usage (CPU, memory, disk, network), and reporting the same to the ResourceManager.

The ApplicationMaster is, in effect, an instanceof a framework-specific libraryand is responsible for negotiating resources from the ResourceManager and working with the NodeManager(s) to

execute and monitor the containers and their resource consumption. It has the responsibility of negotiating for appropriate resource containers from the ResourceManager,

tracking their status, and monitoring progress.