hadoop-0.20.2完全分布式集群

集群规划

准备五台台虚拟机（实验以五台RedHat Enterprise Linux 6.5为例）

防火墙、iptables、和SSH已经在上一篇说过在此就不再赘述，完全分布式相对于伪分布式多了几个注意点，我们按照以下规划步骤来做：

配置hosts文件用于解析IP地址与主机名的映射
建立hadoop运行账号
配置SSH免密码登录
下载解压hadoop的安装包
配置jdk、hadoop环境变量
配置hadoop-env.sh,mapred-site.xml,core-site.xml,hdfs-site.xml,masters,slaves.
将配置好的文件复制到其他节点
格式化namenode并启动hadoop集群
检验集群健康情况

1.配置hosts文件

192.168.186.165 sishen.165
192.168.186.164 sishen.164
192.168.186.163 sishen.163
192.168.186.162 sishen.162
192.168.186.161 sishen.161

[root@sishen ~]# scp /etc/hosts sishen.162:/etc/

[root@sishen ~]# scp /etc/hosts sishen.163:/etc/

[root@sishen ~]# scp /etc/hosts sishen.164:/etc/

[root@sishen ~]# scp /etc/hosts sishen.165:/etc/

使用ping命令检验

[root@sishen ~]# ping -c 2 sishen.162
PING sishen.162 (192.168.186.162) 56(84) bytes of data.
64 bytes from sishen.162 (192.168.186.162): icmp_seq=1 ttl=64 time=29.2 ms
64 bytes from sishen.162 (192.168.186.162): icmp_seq=2 ttl=64 time=0.880 ms

--- sishen.162 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1012ms
rtt min/avg/max/mdev = 0.880/15.040/29.200/14.160 ms

2.建立hadoop运行账户

分别在五台机器上运行以下命令

[root@sishen ~]# useradd hadoop
[root@sishen ~]# echo 123456 | passwd --stdin hadoop

建立hadoop的数据存放目录（注意：分别在五台机器上执行）

[root@sishen ~]# mkdir -p /hadoop-0.20.2/data
[root@sishen ~]# chown -R hadoop:hadoop /hadoop-0.20.2/
[root@sishen ~]# ll -d /hadoop-0.20.2/
drwxr-xr-x 3 hadoop hadoop 4096 Oct 9 14:33 /hadoop-0.20.2/

3.配置SSH免密码登录

分别在五台机器上运行以下命令

[root@sishen ~]# ssh-keygen

……

中间过程直接回车即可

…..

[root@sishen ~]# ssh-copy-id sishen.161

[root@sishen ~]# ssh-copy-id sishen.162

[root@sishen ~]# ssh-copy-id sishen.163

[root@sishen ~]# ssh-copy-id sishen.164

[root@sishen ~]# ssh-copy-id sishen.165

4.解压并配置hadoop软件包