Hama笔记：错误修复记录

hama上的pagerank，模仿示例代码写的，做了一些简化。然后报错：

task_logs：

13/03/26 14:32:01 INFO sync.ZKSyncClient: Initializing ZK Sync Client
13/03/26 14:32:01 INFO sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At lvxuedong-R458/1.2.3.4:61001
13/03/26 14:32:01 ERROR sync.ZooKeeperSyncClientImpl: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /bsp/job_201303261331_0009/peers
13/03/26 14:32:01 INFO ipc.Server: Starting SocketReader
13/03/26 14:32:02 INFO ipc.Server: IPC Server Responder: starting
13/03/26 14:32:02 INFO ipc.Server: IPC Server listener on 61001: starting
13/03/26 14:32:02 INFO message.HadoopMessageManagerImpl:  BSPPeer address:lvxuedong-R458 port:61001
13/03/26 14:32:02 INFO ipc.Server: IPC Server handler 0 on 61001: starting
13/03/26 14:32:03 INFO ipc.Server: IPC Server listener on 61001: readAndProcess threw exception java.io.EOFException. Count of bytes read: 0
java.io.EOFException
    at java.io.DataInputStream.readFully(DataInputStream.java:180)
    at org.apache.hadoop.io.Text.readFields(Text.java:265)
    at org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:169)
    at org.apache.hama.bsp.BSPMessageBundle.readFields(BSPMessageBundle.java:158)
    at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:237)
    at org.apache.hadoop.ipc.RPC$Invocation.readFields(RPC.java:102)
    at org.apache.hadoop.ipc.Server$Connection.processData(Server.java:1303)
    at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1282)
    at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
    at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
    at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
13/03/26 14:32:03 ERROR bsp.BSPPeerImpl: Error while sending messages
java.io.IOException: Call to lvxuedong-R458/1.2.3.4:61001 failed on local exception: java.io.EOFException
    at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
    at org.apache.hadoop.ipc.Client.call(Client.java:1071)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
    at $Proxy3.put(Unknown Source)
    at org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:111)
    at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
    at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
    at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
    at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
    at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
Caused by: java.io.EOFException
    at java.io.DataInputStream.readInt(DataInputStream.java:375)
    at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
    at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
13/03/26 14:32:03 INFO ipc.Server: IPC Server listener on 61001: readAndProcess threw exception java.lang.IndexOutOfBoundsException. Count of bytes read: 0
java.lang.IndexOutOfBoundsException
    at java.io.DataInputStream.readFully(DataInputStream.java:175)
    at org.apache.hadoop.io.Text.readFields(Text.java:265)
    at org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:169)
    at org.apache.hama.bsp.BSPMessageBundle.readFields(BSPMessageBundle.java:158)
    at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:237)
    at org.apache.hadoop.ipc.RPC$Invocation.readFields(RPC.java:102)
    at org.apache.hadoop.ipc.Server$Connection.processData(Server.java:1303)
    at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1282)
    at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1182)
    at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
    at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
13/03/26 14:32:03 ERROR bsp.BSPPeerImpl: Error while sending messages
java.io.IOException: Call to lvxuedong-R458/1.2.3.4:61001 failed on local exception: java.io.EOFException
    at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
    at org.apache.hadoop.ipc.Client.call(Client.java:1071)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
    at $Proxy3.put(Unknown Source)
    at org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:111)
    at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
    at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
    at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
    at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
    at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
Caused by: java.io.EOFException
    at java.io.DataInputStream.readInt(DataInputStream.java:375)
    at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
    at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
13/03/26 14:32:04 INFO ipc.Server: Stopping server on 61001
13/03/26 14:32:04 INFO ipc.Server: IPC Server handler 0 on 61001: exiting
13/03/26 14:32:04 INFO ipc.Server: Stopping IPC Server listener on 61001
13/03/26 14:32:04 INFO ipc.Server: Stopping IPC Server Responder
13/03/26 14:32:04 INFO metrics.RpcInstrumentation: shut down
13/03/26 14:32:04 ERROR bsp.BSPTask: Shutting down ping service.

提示nonode错误。修改了好长时间都没找到，集群应该没什么问题，用example中的pi例子测试，发现两个datanode都没问题。

肯定是自己的代码写的有问题了....

错误原因：

public static GraphJob createJob(String[] args, HamaConfiguration conf) throws Exception{
     //HamaConfiguration conf = new HamaConfiguration();
     GraphJob myjob = new GraphJob(conf,myhamapr.class);
     myjob.setJobName("mypagerank-hama");
     myjob.setMaxIteration(30);
     myjob.setVertexClass(myPageRankVertex.class);
     myjob.setVertexInputReaderClass(myVertexReader.class);
     myjob.setInputPath(new Path(args[0]));
     myjob.setOutputPath(new Path(args[1]));
     myjob.setAggregatorClass(AverageAggregator.class);
     //
     myjob.setInputKeyClass(LongWritable.class);
     myjob.setInputValueClass(Text.class);
     myjob.setInputFormat(TextInputFormat.class);
     myjob.setPartitioner(HashPartitioner.class);
     myjob.setOutputFormat(TextOutputFormat.class);
     myjob.setOutputKeyClass(Text.class);
     myjob.setOutputValueClass(DoubleWritable.class);
     //
     myjob.setVertexIDClass(Text.class);
     //myjob.setVertexValueClass(DoubleWritable.class);
     myjob.setEdgeValueClass(NullWritable.class);
    return myjob;
     
 }

myjob.setVertexValueClass()是required，不是optional。