WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Exception in thread "main" java.io.IOException: No FileSystem for sc F
1、执行脚本程序报如下所示的错误:
1 [hadoop@slaver1 script_hadoop]$ hadoop jar web_click_mr_hive.jar com.bie.hive.mr.ClickStreamThree /home/hadoop/data_hadoop/weblog/preprocess/output/2018-05-19 /home/hadoop/data_hadoop/weblog/preprocess/click_pv_out/2018-05-19 2 18/05/20 17:09:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 3 Exception in thread "main" java.io.IOException: No FileSystem for scheme: C 4 at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584) 5 at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591) 6 at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) 7 at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) 8 at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) 9 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) 10 at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) 11 at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:498) 12 at com.bie.hive.mr.ClickStreamThree.main(ClickStreamThree.java:207) 13 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 14 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 15 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 16 at java.lang.reflect.Method.invoke(Method.java:606) 17 at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 18 [hadoop@slaver1 script_hadoop]$ hadoop jar web_click_mr_hive.jar com.bie.hive.mr.ClickStreamThree hdfs://slaver1:9000/home/hadoop/data_hadoop/weblog/preprocess/output/2018-05-19 hdfs://slaver1:9000/home/hadoop/data_hadoop/weblog/preprocess/click_pv_out/2018-05-19 19 18/05/20 17:10:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20 Exception in thread "main" java.io.IOException: No FileSystem for scheme: C 21 at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584) 22 at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591) 23 at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) 24 at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) 25 at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) 26 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) 27 at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) 28 at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:498) 29 at com.bie.hive.mr.ClickStreamThree.main(ClickStreamThree.java:207) 30 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 31 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 32 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 33 at java.lang.reflect.Method.invoke(Method.java:606) 34 at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 35 [hadoop@slaver1 script_hadoop]$
2、猛一看,WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable,百度一搜这个错误,啥编译32/64位的问题,也有说改配置文件的问题,也许有的可能是这样改就解决问题了,而我想说,出现问题,看看报错原因,也许就可以很快的解决问题了。
然后呢,看下代码,是如下所示导致的错误,将下面的注释了,代码换成上面的就可以使用脚本程序执行代码了:
FileInputFormat.setInputPaths(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); //FileInputFormat.setInputPaths(job, new Path("C:/weblog/output")); //FileOutputFormat.setOutputPath(job, new Path("c:/weblog/pageviews"));
« 上一篇: Caused by: java.net.ConnectException: Connection refused/Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
» 下一篇: WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused
» 下一篇: WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 如何编写易于单元测试的代码
· 10年+ .NET Coder 心语,封装的思维:从隐藏、稳定开始理解其本质意义
· .NET Core 中如何实现缓存的预热?
· 从 HTTP 原因短语缺失研究 HTTP/2 和 HTTP/3 的设计差异
· AI与.NET技术实操系列:向量存储与相似性搜索在 .NET 中的实现
· 周边上新:园子的第一款马克杯温暖上架
· Open-Sora 2.0 重磅开源!
· 分享 3 个 .NET 开源的文件压缩处理库,助力快速实现文件压缩解压功能!
· Ollama——大语言模型本地部署的极速利器
· DeepSeek如何颠覆传统软件测试?测试工程师会被淘汰吗?
2017-05-20 作为程序猿/媛的你在5.20这一天都干了什么???