hbase中的Hmaster启动成功后几秒又挂了
hadoop、zookeeper集群启动以及kafka启动成功后,启动Hmaster,jps显示有进程,但是没几秒就挂了,然后排查日志文件logs/目录下的 hbase-hadoop-master-hadoop102.log,如下:
2021-05-18 11:15:30,094 ERROR [master/hadoop102:16000:becomeActiveMaster] master.HMaster: ***** ABORTING master hadoop102,16000,1621307728334: Unhandled exception. Starting shutdown. *****
java.net.ConnectException: Call From hadoop102/192.168.209.102 to hadoop102:9000 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see: [http://wiki.apache.org/hadoop/ConnectionRefused](http://wiki.apache.org/hadoop/ConnectionRefused)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:801)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1493)
at org.apache.hadoop.ipc.Client.call(Client.java:1435)
at org.apache.hadoop.ipc.Client.call(Client.java:1345)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy18.setSafeMode(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.setSafeMode(ClientNamenodeProtocolTranslatorPB.java:691)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
at com.sun.proxy.$Proxy19.setSafeMode(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
at com.sun.proxy.$Proxy20.setSafeMode(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.setSafeMode(DFSClient.java:2143)
at org.apache.hadoop.hdfs.DistributedFileSystem.setSafeMode(DistributedFileSystem.java:1359)
at org.apache.hadoop.hdfs.DistributedFileSystem.setSafeMode(DistributedFileSystem.java:1343)
at org.apache.hadoop.hbase.util.FSUtils.isInSafeMode(FSUtils.java:293)
at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:705)
at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:252)
at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:153)
at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:124)
at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:904)
at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: 拒绝连接
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:685)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:788)
at org.apache.hadoop.ipc.Client$Connection.access$3500(Client.java:410)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1550)
at org.apache.hadoop.ipc.Client.call(Client.java:1381)
... 33 more
2021-05-18 11:15:30,095 INFO [master/hadoop102:16000:becomeActiveMaster] regionserver.HRegionServer: ***** STOPPING region server 'hadoop102,16000,1621307728334' *****
2021-05-18 11:15:30,095 INFO [master/hadoop102:16000:becomeActiveMaster] regionserver.HRegionServer: STOPPED: Stopped by master/hadoop102:16000:becomeActiveMaster
2021-05-18 11:15:33,046 INFO [master/hadoop102:16000] ipc.NettyRpcServer: Stopping server on /192.168.209.102:16000
2021-05-18 11:15:33,054 INFO [master/hadoop102:16000] regionserver.HRegionServer: Stopping infoServer
2021-05-18 11:15:33,059 INFO [master/hadoop102:16000] handler.ContextHandler: Stopped o.e.j.w.WebAppContext@1a6cf771{/,null,UNAVAILABLE}{file:/opt/module/hbase/hbase-webapps/master}
2021-05-18 11:15:33,064 INFO [master/hadoop102:16000] server.AbstractConnector: Stopped ServerConnector@481e91b6{HTTP/1.1,[http/1.1]}{0.0.0.0:16010}
2021-05-18 11:15:33,064 INFO [master/hadoop102:16000] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@8a98f38{/static,file:///opt/module/hbase/hbase-webapps/static/,UNAVAILABLE}
2021-05-18 11:15:33,064 INFO [master/hadoop102:16000] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@68577ba8{/logs,file:///opt/module/hbase/logs/,UNAVAILABLE}
2021-05-18 11:15:33,065 INFO [master/hadoop102:16000] regionserver.HRegionServer: aborting server hadoop102,16000,1621307728334
2021-05-18 11:15:33,071 INFO [master/hadoop102:16000] regionserver.HRegionServer: stopping server hadoop102,16000,1621307728334; all regions closed.
2021-05-18 11:15:33,071 INFO [master/hadoop102:16000] hbase.ChoreService: Chore service for: master/hadoop102:16000 had [] on shutdown
2021-05-18 11:15:33,072 WARN [master/hadoop102:16000] master.ActiveMasterManager: Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null
2021-05-18 11:15:33,076 INFO [master/hadoop102:16000] zookeeper.ZooKeeper: Session: 0x3797d5220330004 closed
2021-05-18 11:15:33,076 INFO [master/hadoop102:16000] regionserver.HRegionServer: Exiting; stopping=hadoop102,16000,1621307728334; zookeeper connection closed.
2021-05-18 11:15:33,076 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x3797d5220330004
2021-05-18 11:15:33,076 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: HMaster Aborted
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:244)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2942)
于是网上查找答案:
1.第一种:在conf/目录下的配置文件 hbase-site.xml 中添加(失败)[1]:
<property>
<name>hbase.unsafe.stream.capability.enforce</name>
<value>false</value>
</property>
然后重启,发现没有用
2.第二种:在conf/目录下的配置文件(成功)[2]:
将hbase-site.xml中的配置项中的端口号8020改为hdfs在运行的端口号8020:
修改前的hbase-site.xml(端口为9000):
<property>
<name>hbase.rootdir</name>
<value>hdfs://hadoop102:9000/hbase</value>
</property>
在core-site.xml中查到hdfs在运行的端口号为8020:
<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop102:8020</value>
</property>
改为与core-site.xml中查到hdfs在运行的端口号为8020一致(保持两者一致):
修改后的hbase-site.xml(端口为8020):
<property>
<name>hbase.rootdir</name>
<value>hdfs://hadoop102:8020/hbase</value>
</property>
最后错误解决,浏览器访问hadoop102:16010成功:
小结一下:
浏览器无法访问服务对应的端口时,有可能是服务未启动(端口不一致),所以一定要先排查日志文件。
参考资料:
[1]https://blog.csdn.net/qq_41497111/article/details/103529971
[2]https://blog.csdn.net/weixin_45627031/article/details/104642502?utm_medium=distribute.pc_relevant.none-task-blog-baidujs_title-0&spm=1001.2101.3001.4242
脚踏实地,仰望星空!