Hive hiveserver2 监听端口10000异常 启动失败
先启动metastore
1 | nohup hive --service metastore & |
再启动 hiveserver2
1 | nohup hive --service hiveserver2 & |
用jps和ps -ef查看进程都启动着,但发现10000端口没有被监听
看不出来问题,打开hive日志
1 2 3 4 5 6 | cd $HIVE_HOME/conf cp hive-log4j2.properties.template hive-log4j2.properties vim hive-log4j2.properties property.hive.log.dir = /root/app/hive/log |
查看日志
1 2 3 | org.apache.thrift.transport.TTransportException: Cannot write to null outputStream java.lang.RuntimeException: Error initializing notification event poll Caused by : java.io.IOException: org.apache.thrift.TApplicationException: Internal error processing get_current_notificationEventId |
应该是notification.api.auth的验证问题
修复:
1 2 3 4 5 | 添加hive-site.xml配置如下 <property> <name>hive.metastore. event .db.notification.api.auth</name> <value> false </value> </property> |
再启动hiveserver2
报错:
1 | WARN [main] server.HiveServer2: Error starting HiveServer2 on attempt 1, will retry in 60000ms<br>java.lang.NoClassDefFoundError: org/apache/tez/dag/api/TezConfiguration<br> at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession$AbstractTriggerValidator.startTriggerValidator(TezSessionPoolSession.java:74)<br> at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.initTriggers(TezSessionPoolManager.java:207)<br> at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.startPool(TezSessionPoolManager.java:114)<br> at org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:839)<br> at org.apache.hive.service.server.HiveServer2.startOrReconnectTezSessions(HiveServer2.java:822)<br> at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:745)<br> at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1037)<br> at org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140)<br> at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1305)<br> at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1149)<br> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)<br> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<br> at java.lang.reflect.Method.invoke(Method.java:498)<br> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)<br> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)<br>Caused by : java.lang.ClassNotFoundException: org.apache.tez.dag.api.TezConfiguration<br> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)<br> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)<br> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)<br> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)<br> ... 16 more |
我没有安装tez, 为什么会启动呢?注意如下关联信息:
1 2 | 2020-11-17 00:06:29,576 INFO [main] server.HiveServer2: HS2 interactive HA not enabled. Starting tez sessions.. 2020-11-17 00:06:29,576 INFO [main] server.HiveServer2: Starting/Reconnecting tez sessions.. |
显示HS2 interactive HA没有开启,就导致了开启tez session。那么我们进行如下配置开启HS2 interactive HA:
修复:
1 2 3 4 5 | <property> <name>hive.server2.active.passive.ha.enable</name> <value> true </value> <description>Whether HiveServer2 Active/Passive High Availability be enabled when Hive Interactive sessions are enabled.This will also require hive.server2.support.dynamic.service.discovery to be enabled.</description> </property> |
再次尝试,启动成功
用beeline测试
1 | ./beeline -u jdbc:hive2: //bigdata:10000 |
报错:
1 | Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous (state=08S01,code=0)<em id= "__mceDel" >Beeline version 3.0.0 by Apache Hive</em> |
是root用户验证的问题
修复;
core-stie.xml增加如下配置,重启
1 2 3 4 5 6 7 8 | <property> <name>hadoop.proxyuser.root.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.root.groups</name> <value>*</value> </property> |
再次尝试,beeline正常读取了,但日志仍有Error信息
报错:
1 2 3 4 5 6 7 8 9 10 11 | [HiveServer2-Handler-Pool: Thread-53]: Error occurred during processing of message. java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by : org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:328) at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) |
分析:
1.这是一个可以忽略的异常,不会影响HiveServer2服务的正常使用。
2.该异常会在Thrift0.9.2及更低版本的HiveServer2服务上出现。
3.由HiveServer2服务上的负载均衡器进行Tcp检查引起的,主要用评估服务的健康状况。(详见THRIFT-2268)
https://issues.apache.org/jira/browse/THRIFT-2268?attachmentSortBy=dateTime
参考文章:
https://issues.apache.org/jira/browse/HIVE-19740
https://cloud.tencent.com/developer/article/1176657
https://www.codenong.com/jsa0ad3e1513ef/
https://blog.csdn.net/qq_16633405/article/details/82190440
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 25岁的心里话
· 闲置电脑爆改个人服务器(超详细) #公网映射 #Vmware虚拟网络编辑器
· 基于 Docker 搭建 FRP 内网穿透开源项目(很简单哒)
· 零经验选手,Compose 一天开发一款小游戏!
· 一起来玩mcp_server_sqlite,让AI帮你做增删改查!!