Hadoop java.lang.ClassNotFoundException: org.apache.commons.lang3.StringUtils

commons-lang3-3.3.2.jar 学习好友推荐案例的时候，提交运行时报错找不到StringUtils

java.lang.ClassNotFoundException: org.apache.commons.lang3.StringUtils
复制代码
[root@node01 ~]# hadoop jar MyFOF.jar com.sxt.hadoop.mr.fof.MyFOF
19/03/30 00:25:37 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
19/03/30 00:25:37 INFO input.FileInputFormat: Total input paths to process : 1
19/03/30 00:25:38 INFO mapreduce.JobSubmitter: number of splits:1
19/03/30 00:25:38 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1553914110426_0006
19/03/30 00:25:38 INFO impl.YarnClientImpl: Submitted application application_1553914110426_0006
19/03/30 00:25:38 INFO mapreduce.Job: The url to track the job: http://node03:8088/proxy/application_1553914110426_0006/
19/03/30 00:25:38 INFO mapreduce.Job: Running job: job_1553914110426_0006
19/03/30 00:25:51 INFO mapreduce.Job: Job job_1553914110426_0006 running in uber mode : false
19/03/30 00:25:51 INFO mapreduce.Job:  map 0% reduce 0%
19/03/30 00:26:01 INFO mapreduce.Job: Task Id : attempt_1553914110426_0006_m_000000_0, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.lang3.StringUtils
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.sxt.hadoop.mr.fof.FMapper.map(FMapper.java:20)
    at com.sxt.hadoop.mr.fof.FMapper.map(FMapper.java:1)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

19/03/30 00:26:14 INFO mapreduce.Job: Task Id : attempt_1553914110426_0006_m_000000_1, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.lang3.StringUtils
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.sxt.hadoop.mr.fof.FMapper.map(FMapper.java:20)
    at com.sxt.hadoop.mr.fof.FMapper.map(FMapper.java:1)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

19/03/30 00:26:24 INFO mapreduce.Job: Task Id : attempt_1553914110426_0006_m_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.lang3.StringUtils
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.sxt.hadoop.mr.fof.FMapper.map(FMapper.java:20)
    at com.sxt.hadoop.mr.fof.FMapper.map(FMapper.java:1)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

19/03/30 00:26:35 INFO mapreduce.Job:  map 100% reduce 100%
19/03/30 00:26:35 INFO mapreduce.Job: Job job_1553914110426_0006 failed with state FAILED due to: Task failed task_1553914110426_0006_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

19/03/30 00:26:35 INFO mapreduce.Job: Counters: 16
    Job Counters 
        Failed map tasks=4
        Killed reduce tasks=1
        Launched map tasks=4
        Other local map tasks=3
        Data-local map tasks=1
        Total time spent by all maps in occupied slots (ms)=36188
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=36188
        Total time spent by all reduce tasks (ms)=0
        Total vcore-milliseconds taken by all map tasks=36188
        Total vcore-milliseconds taken by all reduce tasks=0
        Total megabyte-milliseconds taken by all map tasks=37056512
        Total megabyte-milliseconds taken by all reduce tasks=0
    Map-Reduce Framework
        CPU time spent (ms)=0
        Physical memory (bytes) snapshot=0
        Virtual memory (bytes) snapshot=0
复制代码

 

编译时没问题，提交运行错误，说明在运行环境中找不到StringUtils。

解决办法：

从/opt/sxt/hadoop-2.7.7/share/hadoop/tools/lib目录下拷贝commons-lang3-3.3.2.jar到opt/sxt/hadoop-2.7.7/share/hadoop/common/lib目录下，然后将commons-lang3-3.3.2.jar分别拷贝到各DataNode[root@node01 lib]# scp commons-lang3-3.3.2.jar node02:`pwd`

commons-lang3-3.3.2.jar                                                                    100%  403KB   8.1MB/s   00:00
[root@node01 lib]# scp commons-lang3-3.3.2.jar node03:`pwd`
commons-lang3-3.3.2.jar    　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　100%  403KB   7.6MB/s   00:00
                                                   
[root@node01 lib]# scp commons-lang3-3.3.2.jar node04:`pwd`
commons-lang3-3.3.2.jar                               　　　　　　　　　　　　　　　　　　　　　　100%  403KB   7.0MB/s   00:00 
posted @   梦醒江南·Infinite  阅读(2598)  评论(0)  编辑  收藏  举报
编辑推荐：
· 10年+ .NET Coder 心语，封装的思维：从隐藏、稳定开始理解其本质意义
· .NET Core 中如何实现缓存的预热？
· 从 HTTP 原因短语缺失研究 HTTP/2 和 HTTP/3 的设计差异
· AI与.NET技术实操系列：向量存储与相似性搜索在 .NET 中的实现
· 基于Microsoft.Extensions.AI核心库实现RAG应用
阅读排行：
· 阿里巴巴 QwQ-32B真的超越了 DeepSeek R-1吗？
· 10年+ .NET Coder 心语 ── 封装的思维：从隐藏、稳定开始理解其本质意义
· 【译】Visual Studio 中新的强大生产力特性
· 【设计模式】告别冗长if-else语句：使用策略模式优化代码结构
· 字符编码：从基础到乱码解决
历史上的今天：
2018-03-30 WPF应用程序内存泄漏的一些原因
点击右上角即可分享
微信分享提示
AI IDE Trae