hadoop-执行mapreduce时主机名非法的处理
hadoop搭建正常,在执行mr时,提示一个datanode节点主机名非法:
[tianyc@TeletekHbase bin]$ ~/hadoop/bin/hadoop jar ~/hadoop/contrib/streaming/hadoop-streaming-1.0.4.jar -mapper ~/study/mapred/python/map.py -reducer ~tudy/mapred/python/red.py -input test.dat -output test-output
packageJobJar: [/tmp/hadoop-tianyc/hadoop-unjar7725183963809127230/] [] /tmp/streamjob4354715876493197255.jar tmpDir=null
13/02/19 23:12:56 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/02/19 23:12:56 WARN snappy.LoadSnappy: Snappy native library not loaded
13/02/19 23:12:56 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/19 23:12:56 INFO streaming.StreamJob: getLocalDirs(): [/tmp/hadoop-tianyc/mapred/local]
13/02/19 23:12:56 INFO streaming.StreamJob: Running job: job_201302192306_0001
13/02/19 23:12:56 INFO streaming.StreamJob: To kill this job, run:
13/02/19 23:12:56 INFO streaming.StreamJob: /home/tianyc/hadoop-1.0.4/libexec/../bin/hadoop job -Dmapred.job.tracker=http://TeletekHbase:9001 -kill job_201302192306_0001
13/02/19 23:12:56 INFO streaming.StreamJob: Tracking URL: http://TeletekHbase:50030/jobdetails.jsp?jobid=job_201302192306_0001
13/02/19 23:12:57 INFO streaming.StreamJob: map 100% reduce 100%
13/02/19 23:12:57 INFO streaming.StreamJob: To kill this job, run:
13/02/19 23:12:57 INFO streaming.StreamJob: /home/tianyc/hadoop-1.0.4/libexec/../bin/hadoop job -Dmapred.job.tracker=http://TeletekHbase:9001 -kill job_201302192306_0001
13/02/19 23:12:57 INFO streaming.StreamJob: Tracking URL: http://TeletekHbase:50030/jobdetails.jsp?jobid=job_201302192306_0001
13/02/19 23:12:57 ERROR streaming.StreamJob: Job not successful. Error: Job initialization failed:
java.net.UnknownHostException: TkHbase_2 is not a valid Inet address
at org.apache.hadoop.net.NetUtils.verifyHostnames(NetUtils.java:569)
at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:711)
at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:4207)
at org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
13/02/19 23:12:57 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
这是因为hadoop集群中的主机名不能带有下划线。
将该节点的hostname改为TkHbase2后,重启主机和hadoop,再次执行mr作业,即正常。
参考:
http://www.rqna.net/qna/vrxiv-error-running-mapreduce-jobs-not-a-valid-inet-address.html
http://social.msdn.microsoft.com/Forums/nl-BE/hdinsight/thread/3c2dc886-74a5-4062-a208-a807b99380fa