HDFS配置HA后,Hive无法读取数据

问题描述:

当cdh集群配置高可用后,hdfs 的 active 节点转 standby 后,hive数据无法读取

FAILED: SemanticException Unable to determine if hdfs://hadoop-prod-namenode:8020/user/hive/warehouse/ods.db/ods_sap_hrp9010 is encrypted: org.apache.hadoop.hive.ql.metadata.HiveException: 
java.net.ConnectException: Call From hadoop-prod-datanode1/10.196.17.27 to hadoop-prod-namenode:8020 failed on connection exception: java.net.ConnectException:
拒绝连接; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused

  

原因解析:(集群命名空间为 nameservice1)

当集群没有配置HA高可用的时候,hive元数据存储格式为

hdfs://hadoop-test-namenode:8020/user/hive/warehouse/ods.db/ods_sap_hrp9001

当集群配置HA高可用的时候,hive元数据存储格式为

hdfs://nameservice1/user/hive/warehouse/ods.db/ods_sap_hrp9001

不难看出hive元数据并没有更改,当 active 节点变更时,hive路径便查询不了了

 

解决方案:

更改hive元数据的俩张表的数据路径即可

update SDS set LOCATION=REPLACE(LOCATION,'hadoop-prod-namenode:8020','nameservice1')
update DBS set DB_LOCATION_URI=REPLACE (DB_LOCATION_URI,'hadoop-prod-namenode:8020','nameservice1');

  

 

posted @ 2021-04-27 14:58  zhengsongsong  阅读(982)  评论(0编辑  收藏  举报