Hadoop Windows下安装

1.下载

• winutils：https://github.com/s911415/apache-hadoop-3.1.0-winutils，替换bin文件
• hadoop：3.2.0 tar 解压

2.环境变量

• hadoop version验证
  问题：Error：java_home is incorrectly set
  解决：由于jdk在C盘引起：hadoop-env.cmd 加入 set JAVA_HOME=C:\PROGRA~1\Java\jdk1.8.0_112

3.配置文件

• core-site.xml

<configuration>
	<property>
		<name>fs.defaultFS</name>
		<value>hdfs://localhost:900</value>
	</property>
</configuration>

• hadoop-env.sh

export JAVA_HOME=C:\Program Files\Java\jdk1.8.0_112\bin

• hdfs-site.xml
  在hadoop-3.2.0目录下新键data目录，再新建namenode和datanode目录

<configuration>
	<property>       
	<name>dfs.replication</name>       
	<value>1</value>   
	</property>   
	<property>       
	<name>dfs.namenode.name.dir</name>       
	<value>file:///D:/BigData/Hadoop/hadoop-3.2.0/data/namenode</value>
	</property>   
	<property>       
	<name>dfs.datanode.data.dir</name>     
	<value>file:///D:/BigData/Hadoop/hadoop-3.2.0/data/datanode</value>
	</property>
</configuration>

• mapred-site.xml

<configuration>   
	<property>       
	<name>mapreduce.framework.name</name>       
	<value>yarn</value>   
	</property>
</configuration>

• yarn-site.xml

<configuration>
	<property>
		<name>yarn.nodemanager.aux-services</name>
		<value>mapreduce_shuffle</value>
	</property>
	<property>
		<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
		<value>org.apache.hadoop.mapred.ShuffleHandler</value>
	</property>
</configuration>

4.启动

• bin目录下hdfs namenode –format
  没报错说明配置没问题

• sbin下
  start-dfs.cmd
  访问http://localhost:9870/

start-yarn.cmd
http://localhost:8088/
问题：java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/timelineservice/collector/TimelineColl
解决：将share\hadoop\yarn\timelineservice\hadoop-yarn-server-timelineservice-.jar复制到share\hadoop\yarn\下一份