CentOS6.4下伪分布式安装Hadoop-1.2.1

一、配置hadoop

总共需要修改4个文件，"hadoop-env.sh"，"core-site.xml"、"hdfs-site.xml"和"mapred-site.xml"。

1.hadoop-env.sh

export JAVA_HOME=/usr/java/jdk1.7.0_45

2.core-site.xml

<configuration>
	<property>
		<name>hadoop.tmp.dir</name>
		<value>/home/luffy/Development/hadoop-${user.name}</value>
		<description>A base for other temporary directories.</description>
	</property>
	<property> 
		<name>fs.default.name</name> 
		<value>hdfs://localhost:9000</value> 
	</property>
</configuration>

 3.hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

 4.mapred-site.xml

<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>

修改之后，设置环境变量

#set Environment
export JAVA_HOME=/usr/java/jdk1.7.0_45
export HADOOP_INSTALL=/home/luffy/Development/hadoop-1.2.1
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$HADOOP_INSTALL/bin:$PATH

二、配置ssh免密码登陆 

因为CentOS6.4自带安装ssh，所以不需要再安装，但是要配置ssh免密码登陆，需要进行设置

1.ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

2.sudo gedit /etc/selinux/config，并修改"SELINUX=disabled"，然后source /etc/selinux/config

3.chmod 600 ~/.ssh/authorized_keys

chmod 700 ~/.ssh

4.至此，设置已完成，注销再登陆即可。

 三、hadoop测试

1.hadoop namenode -format

2.start-all.sh

3.此时，能访问到http://localhost:50070/和http://localhost:50030/

4.cd /home/luffy/Development/hadoop-1.2.1/

hadoop fs -put conf input

5.hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+'

6.hadoop fs -cat output/*

7.stop-all.sh

以上步骤中，若都没有出现各种错误，则hadoop安装成功。