用MapReduce处理数据
1.打开eclipse,创建一个新的Java Project;选择“Configure Build Path”,选择Library标签,Add External JARs,选择Apache/Hadoop 1.0.4文件夹,选择下列jar文件:(备注:选择的jar视情况而定,参考链接http://www.cnblogs.com/chenyaling/p/5521389.html)
2.创建“com.letsdobigdata”包,在包下创建“MaxTemparatureMapper,MaxTemparatureRedcuer,MaxTemparatureDriver”类;
3.将com.letsdobigdata项目导出为Jar文件,命名为weather.jar;
4.启动Hadoop
复制weather.txt文件到“/home/ chen/data/weather.txt”
5.访问localhost:50070,利用文件选择器选择选择weather.txt;
6.运行MapReduce程序,
“bin/hadoop jar /home/chen/Apache/weatherdata.jar com.letsdobigdata.MaxTemperatureDriver /weather.txt /home/chen/workspace/weather2”;
7.打开浏览器查看输出结果;
8.复制结果到output.txt
bin/hadoop dfs -copyToLocal hdfs:/home/chen/workspace/weather2/part-r-00000 /home/chen/output2.txt
9.查看output.txt。