注册到K8S上的Nacos报错com.alibaba.nacos.api.exception.NacosException: Client not connected, current status:STARTING
近期升级SpringCloudAlibaba版本，从2021.0.1.0升级到2021.0.5
问题现象
注册到K8S平台上2.1.2版本的Nacos报错，错误信息如下：
2023-09-13 09:29:53.712 INFO 19988 — [ main] c.a.n.p.a.s.c.ClientAuthPluginManager : ClientAuthPluginManager Load ClientAuthService com.alibaba.nacos.client.auth.impl.NacosClientAuthServiceImpl success.
2023-09-13 09:29:53.712 INFO 19988 — [ main] c.a.n.p.a.s.c.ClientAuthPluginManager : ClientAuthPluginManager Load ClientAuthService com.alibaba.nacos.client.auth.ram.RamClientAuthServiceImpl success.
2023-09-13 09:30:02.660 ERROR 19988 — [ main] c.a.c.n.registry.NacosServiceRegistry : nacos registry, oneboot register failed...NacosRegistration{nacosDiscoveryProperties=NacosDiscoveryProperties{serverAddr='10.xxx:xxx23', username='nacos', password='***', endpoint='', namespace='', watchDelay=30000, logName='', service='oneboot', weight=1.0, clusterName='DEFAULT', group='DEFAULT_GROUP', namingLoadCacheAtStart='false', metadata=
Unknown macro: {IPv6=null, preserved.register.source=SPRING_CLOUD}
, registerEnabled=true, ip='10.xxxx', networkInterface='', port=8081, secure=false, accessKey='', secretKey='', heartBeatInterval=null, heartBeatTimeout=null, ipDeleteTimeout=null, instanceEnabled=true, ephemeral=true, failureToleranceEnabled=false}, ipDeleteTimeout=null, failFast=true}},
com.alibaba.nacos.api.exception.NacosException: Client not connected, current status:STARTING
at com.alibaba.nacos.common.remote.client.RpcClient.request(RpcClient.java:639) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.common.remote.client.RpcClient.request(RpcClient.java:619) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.remote.gprc.NamingGrpcClientProxy.requestToServer(NamingGrpcClientProxy.java:356) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.remote.gprc.NamingGrpcClientProxy.doRegisterService(NamingGrpcClientProxy.java:209) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.remote.gprc.NamingGrpcClientProxy.registerService(NamingGrpcClientProxy.java:123) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.remote.NamingClientProxyDelegate.registerService(NamingClientProxyDelegate.java:98) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.NacosNamingService.registerInstance(NacosNamingService.java:152) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.cloud.nacos.registry.NacosServiceRegistry.register(NacosServiceRegistry.java:75) ~[spring\-cloud\-starter\-alibaba\-nacos\-discovery\-2021.0.5.0.jar:2021.0.5.0]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.register(AbstractAutoServiceRegistration.java:232) [spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at com.alibaba.cloud.nacos.registry.NacosAutoServiceRegistration.register(NacosAutoServiceRegistration.java:78) [spring\-cloud\-starter\-alibaba\-nacos\-discovery\-2021.0.5.0.jar:2021.0.5.0]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.start(AbstractAutoServiceRegistration.java:133) [spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.bind(AbstractAutoServiceRegistration.java:98) [spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.onApplicationEvent(AbstractAutoServiceRegistration.java:86) [spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.onApplicationEvent(AbstractAutoServiceRegistration.java:47) [spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:421) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:46) [spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) [spring\-context\-5.3.23.jar:5.3.23]
at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0\_144]
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:935) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586) [spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145) ~[spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:745) ~[spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:420) ~[spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) ~[spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1317) ~[spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) ~[spring\-boot\-2.6.13.jar:2.6.13]
at com.yundingtech.Main.main(Main.java:14) ~[classes/:na]
2023-09-13 09:30:02.661 WARN 19988 — [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'; nested exception is java.lang.reflect.UndeclaredThrowableException
2023-09-13 09:30:03.497 INFO 19988 — [ main] o.apache.catalina.core.StandardService : Stopping service Tomcat
2023-09-13 09:30:03.519 INFO 19988 — [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2023-09-13 09:30:03.534 ERROR 19988 — [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'; nested exception is java.lang.reflect.UndeclaredThrowableException
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:181) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) ~[spring\-context\-5.3.23.jar:5.3.23]
at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0\_144]
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:935) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145) ~[spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:745) [spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:420) [spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) [spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1317) [spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) [spring\-boot\-2.6.13.jar:2.6.13]
at com.yundingtech.Main.main(Main.java:14) [classes/:na]
Caused by: java.lang.reflect.UndeclaredThrowableException: null
at org.springframework.util.ReflectionUtils.rethrowRuntimeException(ReflectionUtils.java:147) ~[spring\-core\-5.3.23.jar:5.3.23]
at com.alibaba.cloud.nacos.registry.NacosServiceRegistry.register(NacosServiceRegistry.java:83) ~[spring\-cloud\-starter\-alibaba\-nacos\-discovery\-2021.0.5.0.jar:2021.0.5.0]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.register(AbstractAutoServiceRegistration.java:232) ~[spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at com.alibaba.cloud.nacos.registry.NacosAutoServiceRegistration.register(NacosAutoServiceRegistration.java:78) ~[spring\-cloud\-starter\-alibaba\-nacos\-discovery\-2021.0.5.0.jar:2021.0.5.0]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.start(AbstractAutoServiceRegistration.java:133) ~[spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.bind(AbstractAutoServiceRegistration.java:98) ~[spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.onApplicationEvent(AbstractAutoServiceRegistration.java:86) ~[spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.onApplicationEvent(AbstractAutoServiceRegistration.java:47) ~[spring\-cloud\-commons\-3.1.5.jar:3.1.5]
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:421) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) ~[spring\-context\-5.3.23.jar:5.3.23]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:46) ~[spring\-boot\-2.6.13.jar:2.6.13]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring\-context\-5.3.23.jar:5.3.23]
... 14 common frames omitted
Caused by: com.alibaba.nacos.api.exception.NacosException: Client not connected, current status:STARTING
at com.alibaba.nacos.common.remote.client.RpcClient.request(RpcClient.java:639) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.common.remote.client.RpcClient.request(RpcClient.java:619) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.remote.gprc.NamingGrpcClientProxy.requestToServer(NamingGrpcClientProxy.java:356) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.remote.gprc.NamingGrpcClientProxy.doRegisterService(NamingGrpcClientProxy.java:209) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.remote.gprc.NamingGrpcClientProxy.registerService(NamingGrpcClientProxy.java:123) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.remote.NamingClientProxyDelegate.registerService(NamingClientProxyDelegate.java:98) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.nacos.client.naming.NacosNamingService.registerInstance(NacosNamingService.java:152) ~[nacos\-client\-2.2.0.jar:na]
at com.alibaba.cloud.nacos.registry.NacosServiceRegistry.register(NacosServiceRegistry.java:75) ~[spring\-cloud\-starter\-alibaba\-nacos\-discovery\-2021.0.5.0.jar:2021.0.5.0]
... 27 common frames omitted
2023-09-13 09:30:03.536 WARN 19988 — [ Thread-6] c.a.nacos.common.notify.NotifyCenter : NotifyCenter Start destroying Publisher
2023-09-13 09:30:03.536 WARN 19988 — [ Thread-6] c.a.nacos.common.notify.NotifyCenter : NotifyCenter Destruction of the end
2023-09-13 09:30:03.536 WARN 19988 — [ Thread-9] c.a.n.common.http.HttpClientBeanHolder : HttpClientBeanHolder Start destroying common HttpClient
Process finished with exit code 1
排查过程
1.本地部署2.1.2，注册到本地，成功。
2.降级SpringCloudAlibaba版本，到原来版本，注册到K8S，成功。
3.阅读报错信息，其大意为：建立rpc连接失败。
4.百度搜索相关报错信息关键字，结果均为配置错误导致，再次检查本地配置，无异常。
5.联系K8S同事分析Nacos服务端配置，无异常，重启Nacos后重试，问题仍未解决。
6.检查两版Nacos Client版本区别，发现客户端分别为1.X和2.X，调查两个版本的Nacos区别，发现2.X服务端比1.X多出3个端口，并且客户端对服务端的HTTP心跳改为RPC长连接。
得出结论，8848端口不提供RPC，通过别的端口提供RPC，该端口默认为9848，原因定位到，排查问题结束。
问题解决
1.尝试修改服务端配置，调整RPC端口配置，调查结果：无相关配置。
2.尝试修改客户端配置，调整其连接的RPC端口配置，调查结果：无相关配置。
3.思考其它方案若干
4.最终方案，通过其默认端口偏移量规律（+1000），调整云管映射端口。
规律：
|
端口
|
与主端口的偏移量
|
描述
|
|---|---|---|
| 8848 | 0 | 主端口，客户端、控制台及OpenAPI所使用的HTTP端口 |
| 9848 | 1000 | 客户端gRPC请求服务端端口，用于客户端向服务端发起连接和请求 |
| 9849 | 1001 | 服务端gRPC请求服务端端口，用于服务间同步等 |
| 7848 | -1000 | Jraft请求服务端端口，用于处理服务端间的Raft相关请求 |
K8S Service映射调整结果：
（问题解决，使用新版Nacos客户端可注册K8S上的Nacos成功。）
