报错背景:
CDH断电后重启失败,解决了种种错误之后,重启成功,但是重启之后的服务器没有任何进程,
查看/opt/cm-5.15.1/log/cloudera-scm-server/cloudera-scm-server.log日志,我来到了这个错误面前。
报错现象:
查看报错日志:
tail -n 200 /opt/cm-5.15.1/log/cloudera-scm-server/cloudera-scm-server.log
2019-04-29 14:44:31,620 INFO WebServerImpl:com.cloudera.server.web.kaiser.components.HealthAdviceManager: Registering 2 CSD bas 2019-04-29 14:44:34,496 INFO WebServerImpl:org.springframework.web.servlet.handler.SimpleUrlHandlerMapping: Root mapping to hanParameterizableViewController] 2019-04-29 14:44:34,569 INFO WebServerImpl:org.springframework.web.servlet.DispatcherServlet: FrameworkServlet 'Spring MVC Disp 2019-04-29 14:44:34,597 INFO WebServerImpl:com.cloudera.server.web.cmon.JobDetailGatekeeper: ActivityMonitor configured to allo 2019-04-29 14:44:36,045 INFO WebServerImpl:com.cloudera.server.web.cmf.AggregatorController: AggregateSummaryScheduler started. 2019-04-29 14:44:36,756 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: I6Z 2019-04-29 14:44:36,870 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: G 2019-04-29 14:44:38,001 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: N 2019-04-29 14:44:38,001 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: G 2019-04-29 14:44:38,023 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: N 2019-04-29 14:44:38,024 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: C 2019-04-29 14:44:38,043 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4 2019-04-29 14:44:38,044 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180 2019-04-29 14:44:38,045 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server. 2019-04-29 14:44:38,238 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: F 2019-04-29 14:44:41,805 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive completed successfully. 2019-04-29 14:44:47,224 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Unable to retrieve remot java.util.concurrent.ExecutionException: java.net.ConnectException: https://archive.cloudera.com/cdh5/parcels/5.15/manifest.jso at com.ning.http.client.providers.netty.NettyResponseFuture.abort(NettyResponseFuture.java:297) at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:104) at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:399) at org.jboss.netty.channel.DefaultChannelFuture.addListener(DefaultChannelFuture.java:145) at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.doConnect(NettyAsyncHttpProvider.java:1041) at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.execute(NettyAsyncHttpProvider.java:858) at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:512) at com.ning.http.client.AsyncHttpClient$BoundRequestBuilder.execute(AsyncHttpClient.java:234) at com.cloudera.parcel.components.ParcelDownloaderImpl.getRepositoryInfoFuture(ParcelDownloaderImpl.java:590) at com.cloudera.parcel.components.ParcelDownloaderImpl.getRepositoryInfo(ParcelDownloaderImpl.java:543) at com.cloudera.parcel.components.ParcelDownloaderImpl.syncRemoteRepos(ParcelDownloaderImpl.java:357) at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:464) at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:459) at com.cloudera.cmf.persist.ReadWriteDatabaseTaskCallable.call(ReadWriteDatabaseTaskCallable.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.ConnectException: https://archive.cloudera.com/cdh5/parcels/5.15/manifest.json at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:100) ... 16 more Caused by: java.nio.channels.UnresolvedAddressException at sun.nio.ch.Net.checkAddress(Net.java:101) at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:139) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:102) at org.jboss.netty.handler.ssl.SslHandler.handleDownstream(SslHandler.java:467) at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:55) at org.jboss.netty.handler.codec.http.HttpClientCodec.handleDownstream(HttpClientCodec.java:97) at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:108) at org.jboss.netty.channel.Channels.connect(Channels.java:642) at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:204) at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:230) at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:183) at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.doConnect(NettyAsyncHttpProvider.java:999) ... 13 more
报错原因:
根据日志里面的信息,我们可以看出,这个问题肯定和manifest.json这个文件有关系。
这时,我想起来这个文件是CDH一开始安装的时候用到的三个文件之一,后来安装kafka的时候,这个文件被替换了。
CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel
CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1
manifest.json
报错解决:
我找到这个目录:/opt/cloudera/parcel-repo/
把以前备份的manifest.json文件放进上面的目录
重启cloudera-scm-server,报错解决。