用新版本的job打成jar包在终端运行mapreduce程序遇到的问题 -找不到Map class？

hadoop@ubuntu:~/hadoop-0.20.2/bin$ ./hadoop jar ~/finger.jar Finger kaoqin output

出现的错误：
11/10/14 13:52:07 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
11/10/14 13:52:07 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
11/10/14 13:52:07 INFO input.FileInputFormat: Total input paths to process : 5
11/10/14 13:52:07 INFO mapred.JobClient: Running job: job_201110141041_0014
11/10/14 13:52:08 INFO mapred.JobClient:  map 0% reduce 0%
11/10/14 13:52:19 INFO mapred.JobClient: Task Id : attempt_201110141041_0014_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
    at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:157)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
    ... 4 more

11/10/14 13:52:19 INFO mapred.JobClient: Task Id : attempt_201110141041_0014_m_000001_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
    at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:157)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
    ... 4 more

11/10/14 13:52:25 INFO mapred.JobClient: Task Id : attempt_201110141041_0014_m_000001_1, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
    at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:157)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
    ... 4 more

11/10/14 13:52:25 INFO mapred.JobClient: Task Id : attempt_201110141041_0014_m_000000_1, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
    at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:157)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
    ... 4 more

11/10/14 13:52:31 INFO mapred.JobClient: Task Id : attempt_201110141041_0014_m_000000_2, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
    at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:157)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
    ... 4 more

11/10/14 13:52:31 INFO mapred.JobClient: Task Id : attempt_201110141041_0014_m_000001_2, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
    at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:157)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException: Finger$FingerMapper1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
    ... 4 more

11/10/14 13:52:40 INFO mapred.JobClient: Job complete: job_201110141041_0014
11/10/14 13:52:40 INFO mapred.JobClient: Counters: 3
11/10/14 13:52:40 INFO mapred.JobClient:   Job Counters
11/10/14 13:52:40 INFO mapred.JobClient:     Launched map tasks=8
11/10/14 13:52:40 INFO mapred.JobClient:     Data-local map tasks=8
11/10/14 13:52:40 INFO mapred.JobClient:     Failed map tasks=1

　　从出错的信息看，一开始我以为是要把map和reduce类一起放在主类当中，然后在定制job的时候再设置用到的map和reduce类，后来尝试了一下，重新打包运行发现还是不可以。

后来才发现，No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).与hadoop版本有关，新版本中job好像用不起来。

     解决办法：用 job.setJarByClass(Finger.class);因为 包含map 的  jar 文件是要分发到datanode上面的， 我一开始 也是遇到这个问题， 后来就想通了。测试也好用了～～

      

posted @ 2011-10-13 14:16  glose  阅读(2825)  评论(0)  编辑  收藏  举报