Loading

Kylin 3.1.2 cube构建过程报错 org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

今天在使用kylin构建cube的时候报如下错误:

2022-10-09 16:45:00,932 ERROR [pool-12-thread-9] threadpool.DefaultScheduler:121 : ExecuteException job:f7b41a90-4c10-994d-e9b7-5c559e3d4a32
org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException: java.lang.RuntimeException: java.io.IOException: org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:201)
	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:119)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kylin.job.exception.ExecuteException: java.lang.RuntimeException: java.io.IOException: org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:201)
	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:72)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:180)
	... 4 more
Caused by: java.lang.RuntimeException: java.io.IOException: org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'
	at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:83)
	at org.apache.kylin.engine.mr.steps.CuboidJob.configureMapperInputFormat(CuboidJob.java:177)
	at org.apache.kylin.engine.mr.steps.CuboidJob.run(CuboidJob.java:151)
	at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:156)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:180)
	... 6 more
Caused by: java.io.IOException: org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'
	at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97)
	at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
	at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:80)
	... 10 more
Caused by: org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'
	at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:2079)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:2066)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1578)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1570)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:208)
	at com.sun.proxy.$Proxy73.getTable(Unknown Source)
	at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:191)
	at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105)
	at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88)
	at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
	... 12 more

我在网上查找了相关的错误,发现找到的都是跟spark有关的,但是我的kylin没有使用spark进行构建。后来参考了网上的一些文章,能成功构建啦。
目前服务器的环境是:

hadoop 3.0.0-cdh6.3.2
hive 3.1.2
kylin 3.1.2

错误原因:应该是因为服务器的cdh最近升级到6.3.2,原先的hadoop2和hive2都升级为hadoop3和hive3,导致和kylin有点不兼容。

解决方法:

# 进入kylin的lib目录
cd $KYLIN_HOME/lib

# 将这两个jar包下载到该lib目录下
wget https://repo1.maven.org/maven2/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar
wget https://repo1.maven.org/maven2/commons-configuration/commons-configuration/1.3/commons-configuration-1.3.jar

# 创建目录并进入
mkdir -p $KYLIN_HOME/bin/hadoop3_jars/cdh6
cd $KYLIN_HOME/bin/hadoop3_jars/cdh6

# 将这两个jar包下载到该目录下
wget https://repo1.maven.org/maven2/org/codehaus/woodstox/stax2-api/3.1.4/stax2-api-3.1.4.jar
wget https://cwiki.apache.org/confluence/download/attachments/173081375/hive-exec-1.21.2.3.1.0.0-78.jar?api=v2

# 重启kylin
$KYLIN_HOME/bin/kylin.sh restart

以上步骤完成后,重新到web页面构建cube,发现能成功构建了。

作者:zoufh

出处:https://www.cnblogs.com/zoufh/p/17111325.html

版权:本作品采用「署名-非商业性使用-相同方式共享 4.0 国际」许可协议进行许可。

posted @   爱睡懒觉的我  阅读(349)  评论(0编辑  收藏  举报
相关博文:
阅读排行:
· 25岁的心里话
· 闲置电脑爆改个人服务器(超详细) #公网映射 #Vmware虚拟网络编辑器
· 基于 Docker 搭建 FRP 内网穿透开源项目(很简单哒)
· 零经验选手,Compose 一天开发一款小游戏!
· 通过 API 将Deepseek响应流式内容输出到前端
点击右上角即可分享
微信分享提示
more_horiz
keyboard_arrow_up dark_mode palette
选择主题