配置分布式集群环境 Distributed Cluster System（多台虚拟机）hadoop 成功

真分布式配置（vmware environment）

 首先安装虚拟机三台

windows下进入

C:\ProgramData\VMware\

 

 

此步骤配置各个虚拟机静态IP

 ref：https://medium.com/shehuawwal/how-to-assign-a-static-ip-address-to-a-vmware-workstation-vm-de7773f9ef19

#node-master
host VMnet8 {
    hardware ethernet 00:0C:29:C8:E2:F8;
    fixed-address 192.168.136.129;
}

#node1
host VMnet8 {
    hardware ethernet 00:0C:29:44:58:5E;
    fixed-address 192.168.136.130;
}

#node2
host VMnet8 {
    hardware ethernet 00:0C:29:2F:6D:BE;
    fixed-address 192.168.136.131;
}

 

 

 

 

 在windows cmd中开启关闭DHCP

 

 

net stop vmnetdhcp
net start vmnetdhcp

启动虚拟机

 在三台主机上安装ssh vim net-tool

sudo apt-get install ssh

sudo apt-get install vim

sudo apt-get update

sudo apt-get install net-tools                     

 三台机器更改/etc/hostname

 node-master

 

node1

node2

更改/etc/hosts

重启

 接下来设置免密钥登录，首先建立文件夹.ssh

 

 把密钥直接复制到authorized_key中

 注意密码配置

$ ssh-keygen -t rsa
NOTE: Leave file name and other things blank.
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys
$ ssh localhost

 

 

 Log into node-master as the hadoop user, download the Hadoop tarball from Hadoop project page

wget http://apache.cs.utah.edu/hadoop/common/current/hadoop-3.1.2.tar.gz
tar -xzf hadoop-3.1.2.tar.gz
mv hadoop-3.1.2 hadoop

Set Environment VariablesPermalink

1. Add Hadoop binaries to your PATH. Edit /home/hadoop/.profile and add the following line:

PATH=/home/hadoop/hadoop/bin:/home/hadoop/hadoop/sbin:$PATH

2.Add Hadoop to your PATH for the shell. Edit vim ~/.bashrc and add the following lines:

export HADOOP_HOME=/home/hadoop/hadoop
export PATH=${PATH}:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin

 安装jdk1.8

sudo apt-get install openjdk-8-jdk

主机按照上文配置

 

 

 

 将这两个文件在虚拟机中配置