Caused by: java.io.IOException: Cannot obtain block length for LocatedBlock

集群入库hive报错：

Caused by: java.io.IOException: Cannot obtain block length for LocatedBlock{BP-433041383-192.168.10.249-1494331993586:blk_1116615973_42940620; getBlockSize()=372; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[192.168.10.234:50010,DS-b8b0240e-837c-42be-9aac-1dd88854370c,DISK], DatanodeInfoWithStorage[192.168.10.239:50010,DS-501a0556-5593-4091-a352-0a2ff434056e,DISK], DatanodeInfoWithStorage[192.168.10.232:50010,DS-f525bc61-ca1e-4d74-a657-adc621136d2f,DISK]]}
    at org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:390)
    at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:333)
    at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:269)
    at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:261)
    at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1540)
    at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:303)
    at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:299)
    at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
    at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:299)
    at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
    at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:108)
    at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
    at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:67)

 

在hadoop测试集群运行job的过程中发现部分运行失败，有Cannot obtain block length for LocatedBlock，使用hdfs dfs -cat ${文件}的时候也报这个错，看过代码后发现是文件未被正常关闭(flume传到hdfs上的，可能由于Namenode长时间gc或者切换或者重启会留下很多.tmp结尾的文件)，为此，使用hadoop命令显示处于打开状态的文件，然后删除这些文件：

hadoop fsck / -openforwrite | egrep -v '^\.+$' | egrep "MISSING|OPENFORWRITE" | grep -o "/[^ ]*" | sed -e "s/:$//" | xargs -i hadoop fs -rmr {};

然后重传这些文件到hdfs。

原文链接：https://blog.csdn.net/oDaiLiDong/article/details/51420701

 参考：flume到底会丢数据吗？