异常-Phoenix HBASE Last region should end with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole

1 详细异常信息

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

RROR: There is a hole in the region chain between \x03\x00\x00\x00\x00\x00\x00\x00\x00 and \x05\x00\x00\x00\x00\x00\x00\x00\x00.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x08\x00\x00\x00\x00\x00\x00\x00\x00 and \x09\x00\x00\x00\x00\x00\x00\x00\x00.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: Last region should end with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. ERROR: Found inconsistency in table WMBIGDATA:T_CHARGING_ORDER ERROR: (region WMBIGDATA:ODS_PLUS_ES_CHARGING_STATION,\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1570851021283.cd3a2e47eae11e7516f40c96ef18f9e0.) First region should start with an empty key.  You need to  create a new region and regioninfo in HDFS to plug the hole. ERROR: There is a hole in the region chain between \x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 and \x06\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x07\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 and \x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: Found inconsistency in table WMBIGDATA:ODS_PLUS_ES_CHARGING_STATION ERROR: (region WMBIGDATA:ODS_ES_CHARGING_EQUIPMENT_EFFECT_INDEX,\x01\x00\x00\x00\x00\x00\x00,1570863380329.262f7bf1c70f3fe0caa2df5f04184d1b.) First region should start with an empty key.  You need to  create a new region and regioninfo in HDFS to plug the hole. ERROR: There is a hole in the region chain between \x02\x00\x00\x00\x00\x00\x00 and \x03\x00\x00\x00\x00\x00\x00.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x04\x00\x00\x00\x00\x00\x00 and \x07\x00\x00\x00\x00\x00\x00.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: Found inconsistency in table WMBIGDATA:ODS_ES_CHARGING_EQUIPMENT_EFFECT_INDEX ERROR: There is a hole in the region chain between \x01 and \x04.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x07 and \x09.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x0A and \x0B.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: Found inconsistency in table WMBIGDATA:WM_USER_ACCOUNT_DEPOSIT_LOG ERROR: There is a hole in the region chain between \x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 and \x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 and \x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: Found inconsistency in table WMBIGDATA:ODS_PLUS_ES_CHARGING_CONNECTOR ERROR: There is a hole in the region chain between \x02 and \x03.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x04 and \x05.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x08 and \x09.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: Last region should end with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. ERROR: Found inconsistency in table WMBIGDATA:WM_TRAVEL_ORDER ERROR: There is a hole in the region chain between \x01 and \x02.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x09 and \x0A.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: Found inconsistency in table WMBIGDATA:WM_ORDER_ER_ORDER ERROR: There is a hole in the region chain between \x01 and \x03.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: There is a hole in the region chain between \x07 and \x08.  You need to create a new .regioninfo and region dir in hdfs to plug the hole. ERROR: Last region should end with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. ERROR: Found inconsistency in table WMBIGDATA:WM_COUPON ERROR: There is a hole in the region chain between \x02 and \x03.  You need to create a new .regioninfo and region dir in hdfs to plug the hole.

2 解决心路历程

　　A尝试命令修复，不起作用

   sudo -u hbase   hbase hbck repair

　　B 尝试重新分配，继续修复，不起作用

   sudo -u hbase hbase hbck -fixAssignments

　　C 删除zookeeper的节点/hbase目录，照样不起作用

　　D 暴力删除 hdfs dfs -rmr /hbase/WALs/ 下面的节点信息，重启集群，也不起作用

3 最终解决

　　上面也是hbase RIT问题能搜索到的很多信息

参考博客: https://cloud.tencent.com/developer/article/1359221

　　然后增加配置hbase-site.xml

　　

<property>

<name>hbase.regionserver.executor.openregion.threads</name>

<value>100</value>

</property>

 

重启集群，服务正常。