ubuntu安装hadoop1.x（伪分布式）
参考 kinglau - 二、Ubuntu14.04下安装Hadoop2.4.0 （伪分布模式）
1. 配置core-site.xml
# hadoop1.x.x 的配置文件都在 $HADOOP_HOME/conf/下，hadoop2.x.x 的配置文件都在 $HADOOP_HOME/etc/hadoop/下
# 本文将都已hadoop1.x.x为例，hadoop2.x.x配置对应文件 $ sudo gedit /usr/local/hadoop/conf/core-site.xml # add <configuration> <properties> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </properties>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop/tmp</value>
<description>临时目录,其他临时目录的父目录</description>
<final>true</final>
</property>
</configuration>
2. 配置 hdfs-site.xml
<configuration> <property> <name>dfs.data.dir</name> <value>/usr/local/hadoop/tmp/dfs/data</value> <description>data node的数据目录</description> <final>true</final> </property> <property> <name>dfs.name.dir</name> <value>/usr/local/hadoop/tmp/dfs/name</value> <description>name node的元数据</description> <final>true</final> </property> <property> <name>dfs.replication</name> <value>1</value> <description>文件复制的副本数</description> </property> </configuration>
3. 格式化namenode
$ /usr/local/hadoop/bin/hadoop namenode -format
# when be asked "Re-format filesystem in /usr/local/hadoop/tmp/dfs/namenode ? (Y or N)" input Y
# when you see something like "Storage directory /usr/local/hadoop-1.2.1/tmp/dfs/name has been successfully formatted." ,you succeed.
4. 启动hadoop
$ /usr/local/hadoop/bin/start-dfs.sh # starting namenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-namenode-erbin-ubuntu.out # localhost: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-datanode-erbin-ubuntu.out # localhost: starting secondarynamenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hadoop-secondarynamenode-erbin-ubuntu.out $ jps -m # 14841 DataNode # 14983 SecondaryNameNode # 15073 Jps -m # 14701 NameNode
5. WordCount测试
$ cd /usr/local/hadoop ## 在hdfs创建文件夹 $ bin/hadoop dfs -mkdir /user $ bin/hadoop dfs -ls / #drwxr-xr-x - hadoop supergroup 0 2015-04-24 15:08 /user $ bin/hadoop dfs -mkdir /user/hadoop ## hadoop会默认将/user/hadoop作为home $ bin/hadoop dfs -mkdir input $ bin/hadoop dfs -ls #Found 1 items #drwxr-xr-x - hadoop supergroup 0 2015-04-24 15:12 /user/hadoop/input ## 上传本地文件值hdfs ## hadoop dfs -put [local] [hdfs] $ bin/hadoop dfs -put input input $ bin/hadoop dfs -ls input #Found 7 items #-rw-r--r-- 1 hadoop supergroup 7457 2015-04-24 15:12 /user/hadoop/input/capacity-scheduler.xml #-rw-r--r-- 1 hadoop supergroup 178 2015-04-24 15:12 /user/hadoop/input/core-site.xml #-rw-r--r-- 1 hadoop supergroup 327 2015-04-24 15:12 /user/hadoop/input/fair-scheduler.xml #-rw-r--r-- 1 hadoop supergroup 4644 2015-04-24 15:12 /user/hadoop/input/hadoop-policy.xml #-rw-r--r-- 1 hadoop supergroup 178 2015-04-24 15:12 /user/hadoop/input/hdfs-site.xml #-rw-r--r-- 1 hadoop supergroup 2033 2015-04-24 15:12 /user/hadoop/input/mapred-queue-acls.xml #-rw-r--r-- 1 hadoop supergroup 178 2015-04-24 15:12 /user/hadoop/input/mapred-site.xml $ bin/hadoop jar hadoop-examples-1.x.x.jar wordcount input output $ bin/hadoop dfs -ls output #Found 2 items #-rw-r--r-- 1 hadoop supergroup 0 2015-04-24 15:25 /user/hadoop/output/_SUCCESS #-rw-r--r-- 1 hadoop supergroup 6289 2015-04-24 15:25 /user/hadoop/output/part-r-00000 $ bin/hadoop dfs -cat output/*
6. 关闭hadoop
$ /usr/local/hadoop-1.2.1/bin/stop-dfs.sh
# stopping namenode
# localhost: stopping datanode
# localhost: stopping secondarynamenode
$ jps -m
# 15317 Jps -m
7. 备注
有些文章，在配置core-site.xml,hdfs-site.xml时，里面的文件路径前总是会添加' file:/ '或' file:/// '.这种写法在2.x是可以的，但是在1.x是不可以的 测试过发现不加file前缀的路径写法，在1.x,2.x都是可以通过的。 如果dfs.name.dir和dfs.data.dir文件夹是实现创建的。dfs.data.dir的文件权限可能会报错，如果报错，就按错误提示更改一下文件夹的权限即可。
