Hadoop安装步骤和遇到的坑

• Hadoop

• 安装

安装并配置hadoop-2.6.4(在master中)

(1)将hadoop-2.6.4.tar.gz安装包复制到hadoop文件目录下（与windows环境下类似）。

(2)解压hadoop-2.6.4.tar.gz

cd

cd  hadoop

tar  -zxvf  hadoop-2.6.4.tar.gz

添加环境变量： vim /etc/profile

在最后添加

set hadoop environment

export HADOOP_HOME=/usr/hadoop-2.6.2

export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

生效：source /etc/profile

配置Hadoop (cd /usr/hadoop-2.6.2)

1、vi core-site.xml

hadoop.tmp.dir

/usr/hadoop-2.6.2/tmp

Abase for other temporary directories.

fs.defaultFS

hdfs://master:9000

io.file.buffer.size

4096

2、vi hadoop-env.sh 在开头加上 export JAVA_HOME=/usr/jdk1.7.0_79

3、vi yarn-env.sh 在开头加上 export JAVA_HOME=/usr/jdk1.7.0_79

4、vi hdfs-site.xml

dfs.namenode.name.dir

file:///usr/hadoop-2.6.2/dfs/name

dfs.datanode.data.dir

file:///usr/hadoop-2.6.2/dfs/data

dfs.replication

2

dfs.nameservices

hadoop-cluster1

dfs.namenode.secondary.http-address

master:50090

dfs.webhdfs.enabled

true

5、vi mapred-site.xml

mapreduce.framework.name

yarn

true

mapreduce.jobtracker.http.address

master:50030

mapreduce.jobhistory.address

master:10020

mapreduce.jobhistory.webapp.address

master:19888

mapred.job.tracker

http://master.Hadoop:9001

6、vi yarn-site.xml

yarn.resourcemanager.hostname

master

yarn.nodemanager.aux-services

mapreduce_shuffle

yarn.resourcemanager.address

master:8032

yarn.resourcemanager.scheduler.address

master:8030

yarn.resourcemanager.resource-tracker.address

master:8031

yarn.resourcemanager.admin.address

master:8033

yarn.resourcemanager.webapp.address

master:8088

7、vi slaves

slave1

slave2

8、复制到slave上 scp -r /usr/hadoop-2.6.2/ root@slave1:/usr/

9、配置slave上的Hadoop环境变量

添加环境变量： vim /etc/profile

在最后添加

set hadoop environment

export HADOOP_HOME=/usr/local/hadoop-2.8.4

export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

生效：source /etc/profile

10、启动

先启动hadoop集群start-all.sh