Dream------Hadoop--Hadoop HA QJM （Quorum Journal Manager）

In a typical HA cluster, two separate machines are configured as NameNodes. At any point in time, exactly one of the NameNodes is in an Active state, and the other is in a Standby state. The Active NameNode is responsible for all client operations in the cluster, while the Standby is simply acting as a slave, maintaining enough state to provide a fast failover if necessary.

In order for the Standby node to keep its state synchronized with the Active node, both nodes communicate with a group of separate daemons called "JournalNodes" (JNs). When any namespace modification is performed by the Active node, it durably logs a record of the modification to a majority of these JNs. The Standby node is capable of reading the edits from the JNs, and is constantly watching them for changes to the edit log. As the Standby Node sees the edits, it applies them to its own namespace. In the event of a failover, the Standby will ensure that it has read all of the edits from the JounalNodes before promoting itself to the Active state. This ensures that the namespace state is fully synchronized before a failover occurs.

In order to provide a fast failover, it is also necessary that the Standby node have up-to-date information regarding the location of blocks in the cluster. In order to achieve this, the DataNodes are configured with the location of both NameNodes, and send block location information and heartbeats to both.

It is vital for the correct operation of an HA cluster that only one of the NameNodes be Active at a time. Otherwise, the namespace state would quickly diverge between the two, risking data loss or other incorrect results. In order to ensure this property and prevent the so-called "split-brain scenario," the JournalNodes will only ever allow a single NameNode to be a writer at a time. During a failover, the NameNode which is to become active will simply take over the role of writing to the JournalNodes, which will effectively prevent the other NameNode from continuing in the Active state, allowing the new Active to safely proceed with failover.

 

一个标准的HA集群，要有2个namenode节点，在任何时候，都要保证同一时间只有一个节点是active的，另一个是standby的。active的节点负责

所有客户端的操作。standby仅仅作为一个从节点，时刻准备着进行快速容错。

为了让standby的节点跟active节点的状态保持同步，2个节点通过JN进行通讯。只要在active节点有任何的改变，都要立刻记录到大部分的JN节点中

（注意是大部分而不是全部），standby节点会不停的监听修改日志的的变化，并且有能力读取这些变化。当standby监听到变化的时候，会把这些改变同步到自己的命名空间上（同步源数据）。一旦发生错误，standby节点要确保在变成active之前，要读到JN上所有的变化，这可以保证在容错发生的时候数据是完全同步的。

为了提供一个快速的故障转移，standby必须知道集群中所有块的最新的位置信息，为了能达到这一点，所有的datanode节点必须配置成向2个namenode节点同步块数据，并且也要向2个namenode节点进行心跳检测

 

对一个HA的集群来说，同一时间只有一个namenode节点是active的是至关重要的。否则，这2个namenode的

namespace的状态将会 快速的产生分歧，面临着数据的丢失或者错误的结果。为了确保这个性质（只有一个active的namenode），避免所谓的“精神分裂”，JN在同一时间只允许一个namenode进行写的操作。在故障转移期间，要变成active的那个namenode会接管向JNs写的这个角色，将会强有力的阻止另一个处于active状态的namenode，允许新的active节点安全的进行故障转移。

 

JournalNode 为什么最少是3个

因为在记录修改日志的时候，是往多个JournalNode 节点写的，也就是说只要要有2个节点是正常跑的整个集群才能正常运行，所以如果是3个，

可以容忍1个机器宕机。

Note: There must be at least 3 JournalNode daemons, since edit log modifications must be written to a majority of JNs. This will allow the system to tolerate the failure of a single machine. You may also run more than 3 JournalNodes, but in order to actually increase the number of failures the system can tolerate, you should run an odd number of JNs, (i.e. 3, 5, 7, etc.). Note that when running with N JournalNodes, the system can tolerate at most (N - 1) / 2 failures and continue to function normally.

 

Note that, in an HA cluster, the Standby NameNode also performs checkpoints of the namespace state, and thus it is not necessary to run a Secondary NameNode, CheckpointNode, or BackupNode in an HA cluster. In fact, to do so would be an error. This also allows one who is reconfiguring a non-HA-enabled HDFS cluster to be HA-enabled to reuse the hardware which they had previously dedicated to the Secondary NameNode.

 

Secondary NameNode也没必要有。

 

DT大数据梦工厂，微信公众号是：DT_Spark，每天都会有大数据实战视频发布，请您持续学习。
相关资料：
scala深入浅出实战经典完整视频、PPT、代码下载：
百度云盘：http://pan.baidu.com/s/1c0noOt6
腾讯微云:http://url.cn/TnGbdC
360云盘：http://yunpan.cn/cQ4c2UALDjSKy  访问密码45e2