Hapoop 搭建 （八）YARN HA搭建
结构以及服务器角色
一、yarn-site.xml 文件配置
在centos01的hadoop安装目录的配置文件夹中 配置 yarn-site.xml (路径：/opt/modules/hadoop-2.8.2/etc/hadoop )，完整文件如下：
<configuration>
<!--指定在YARN上面可以运行MapReduce程序 -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<!-- YARN HA 配置-->
<!-- 开启ResourceManager HA高可用 -->
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<!-- 指定RM的cluster id -->
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>yarncluster</value>
</property>
<!-- 集群中ResourceManager的Id列表 -->
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<!-- 分别指定RM的地址 所在节点主机名 -->
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>centos01</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>centos02</value>
</property>
<!-- 分别指定web页面的访问地址 -->
<property>
<name>yarn.resourcemanager.webapp.address.rm1</name>
<value>centos01:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm2</name>
<value>centos02:8088</value>
</property>
<!-- 指定zk集群地址 -->
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>centos01:2181,centos02:2181,centos03:2181</value>
</property>
<!-- 启用自动恢复 -->
<property>
<name>yarn.resourcemanager.recovery.enabled</name>
<value>true</value>
</property>
<!-- 指定resourcemanager的状态信息存储在zookeeper集群上 -->
<property>
<name>yarn.resourcemanager.store.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.log-aggregation.retain-seconds</name>
<value>86400</value>
</property>
</configuration>
二、同步到其它节点
将yarn-site.xml配置文件复制到另2个节点上
scp /opt/modules/hadoop-2.8.2/etc/hadoop/yarn-site.xml hadoop@centos02:/opt/modules/hadoop-2.8.2/etc/hadoop/ scp /opt/modules/hadoop-2.8.2/etc/hadoop/yarn-site.xml hadoop@centos03:/opt/modules/hadoop-2.8.2/etc/hadoop/
三、启动zookeeper集群
停止 sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh stop 启动 sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh start
[hadoop@centos01 ~]$ sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/modules/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: leader [hadoop@centos01 ~]$ sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh stop ZooKeeper JMX enabled by default Using config: /opt/modules/zookeeper-3.4.14/bin/../conf/zoo.cfg Stopping zookeeper ... STOPPED [hadoop@centos01 ~]$ sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/modules/zookeeper-3.4.14/bin/../conf/zoo.cfg Error contacting service. It is probably not running. [hadoop@centos01 ~]$ sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh start ZooKeeper JMX enabled by default Using config: /opt/modules/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [hadoop@centos01 ~]$
四、启动YARN集群
分别进入centos01、centos02的hadoop的安装目录启动ResourceManager
sh /opt/modules/hadoop-2.8.2/sbin/yarn-daemon.sh start resourcemanager
进入3各节点的hadoop安装目录启动NodeManager
sh /opt/modules/hadoop-2.8.2/sbin/yarn-daemon.sh start nodemanager
五、查看节点进程
[hadoop@centos01 sbin]$ jps 96960 ResourceManager 64850 JournalNode 97427 NodeManager 64164 DataNode 34004 QuorumPeerMain 65305 DFSZKFailoverController 98876 Jps 36702 NameNode [hadoop@centos02 ~]$ jps 59364 DataNode 84087 NodeManager 83686 ResourceManager 85798 Jps 61513 DFSZKFailoverController 20457 QuorumPeerMain 59980 JournalNode 82735 NameNode [hadoop@centos03 hadoop]$ jps 62977 NodeManager 49509 JournalNode 65191 Jps 130810 QuorumPeerMain 48845 DataNode
六、测试YARN故障转移
访问centos01活动的ResourceManager
http://192.168.0.171:8088/cluster
访问centos02活动的ResourceManager，此时活动的是centos01所以会自动跳转到centos01上
http://192.168.0.172:8088/cluster
[hadoop@centos01 sbin]$ jps 96960 ResourceManager 64850 JournalNode 97427 NodeManager 64164 DataNode 34004 QuorumPeerMain 65305 DFSZKFailoverController 98876 Jps 36702 NameNode [hadoop@centos01 sbin]$ kill -9 96960 [hadoop@centos01 sbin]$
自动切换到centos02上
