RocketMQ之MQClientInstance启动过程
RocketMQ之MQClientInstance启动过程
MQClientInstance#start
public void start() throws MQClientException {
synchronized (this) {
switch (this.serviceState) {
case CREATE_JUST:
this.serviceState = ServiceState.START_FAILED;
// If not specified,looking address from name server
if (null == this.clientConfig.getNamesrvAddr()) {
this.mQClientAPIImpl.fetchNameServerAddr();
}
// Start request-response channel ①启动远程通信服务
this.mQClientAPIImpl.start();
// Start various schedule tasks ②开启定时任务
this.startScheduledTask();
// Start pull service ③启动拉取消息服务
this.pullMessageService.start();
// Start rebalance service ④启动消费者负载均衡服务
this.rebalanceService.start();
// Start push service ⑤启动内部的defaultMQProducer
this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
log.info("the client factory [{}] start OK", this.clientId);
this.serviceState = ServiceState.RUNNING;
break;
case RUNNING:
break;
case SHUTDOWN_ALREADY:
break;
case START_FAILED:
throw new MQClientException("The Factory object[" + this.getClientId() + "] has been created before, and failed.", null);
default:
break;
}
}
}
①启动远程通信服务
②开启定时任务
③启动拉取消息服务
④启动消费者负载均衡服务
⑤启动内部的defaultMQProducer
①启动远程通信服务
mQClientAPIImpl#start
先看MQClientAPIImpl的构造方法
public MQClientAPIImpl(final NettyClientConfig nettyClientConfig,
final ClientRemotingProcessor clientRemotingProcessor,
RPCHook rpcHook, final ClientConfig clientConfig) {
this.clientConfig = clientConfig;
//namesrv地址信息获取地址
topAddressing = new TopAddressing(MixAll.getWSAddr(), clientConfig.getUnitName());
//远程通信客户端
this.remotingClient = new NettyRemotingClient(nettyClientConfig, null);
this.clientRemotingProcessor = clientRemotingProcessor;
this.remotingClient.registerRPCHook(rpcHook);
//远程通信客户端注册处理器
//CHECK_TRANSACTION_STATE 事务回查
this.remotingClient.registerProcessor(RequestCode.CHECK_TRANSACTION_STATE, this.clientRemotingProcessor, null);
//NOTIFY_CONSUMER_IDS_CHANGED 通知消费者变化
this.remotingClient.registerProcessor(RequestCode.NOTIFY_CONSUMER_IDS_CHANGED, this.clientRemotingProcessor, null);
//重置消费端offset
this.remotingClient.registerProcessor(RequestCode.RESET_CONSUMER_CLIENT_OFFSET, this.clientRemotingProcessor, null);
//获取消费端状态
this.remotingClient.registerProcessor(RequestCode.GET_CONSUMER_STATUS_FROM_CLIENT, this.clientRemotingProcessor, null);
//获取消费端运行信息
this.remotingClient.registerProcessor(RequestCode.GET_CONSUMER_RUNNING_INFO, this.clientRemotingProcessor, null);
//消费消息
this.remotingClient.registerProcessor(RequestCode.CONSUME_MESSAGE_DIRECTLY, this.clientRemotingProcessor, null);
}
主要是创建了remotingClient，并注册了几个请求对应的处理器。
接下来MQClientAPIImpl#start方法
public void start() {
this.remotingClient.start();
}
内部调用了remotingClient#start，实现类是NettyRemotingClient
NettyRemotingClient#start
public void start() {
this.defaultEventExecutorGroup = new DefaultEventExecutorGroup(
nettyClientConfig.getClientWorkerThreads(),
new ThreadFactory() {
private AtomicInteger threadIndex = new AtomicInteger(0);
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "NettyClientWorkerThread_" + this.threadIndex.incrementAndGet());
}
});
//设置bootstrap
Bootstrap handler = this.bootstrap.group(this.eventLoopGroupWorker).channel(NioSocketChannel.class)
//关闭Nagle算法 避免产生时延
.option(ChannelOption.TCP_NODELAY, true)
//不启用tcp的心跳机制
.option(ChannelOption.SO_KEEPALIVE, false)
//3秒连接超时
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, nettyClientConfig.getConnectTimeoutMillis())
//发送缓冲区64k
.option(ChannelOption.SO_SNDBUF, nettyClientConfig.getClientSocketSndBufSize())
//接收缓冲区64k
.option(ChannelOption.SO_RCVBUF, nettyClientConfig.getClientSocketRcvBufSize())
.handler(new ChannelInitializer<SocketChannel>() {
@Override
public void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline pipeline = ch.pipeline();
if (nettyClientConfig.isUseTLS()) {
if (null != sslContext) {
pipeline.addFirst(defaultEventExecutorGroup, "sslHandler", sslContext.newHandler(ch.alloc()));
log.info("Prepend SSL handler");
} else {
log.warn("Connections are insecure as SSLContext is null!");
}
}
pipeline.addLast(
defaultEventExecutorGroup,
//消息编码器
new NettyEncoder(),
//消息解码器
new NettyDecoder(),
//心跳检测
new IdleStateHandler(0, 0, nettyClientConfig.getClientChannelMaxIdleTimeSeconds()),
new NettyConnectManageHandler(),
new NettyClientHandler());
}
});
//每秒处理丢弃超时的请求
this.timer.scheduleAtFixedRate(new TimerTask() {
@Override
public void run() {
try {
NettyRemotingClient.this.scanResponseTable();
} catch (Throwable e) {
log.error("scanResponseTable exception", e);
}
}
}, 1000 * 3, 1000);
if (this.channelEventListener != null) {
this.nettyEventExecutor.start();
}
②开启定时任务
MQClientInstance#startScheduledTask
private void startScheduledTask() {
//如果namesrv没指定，开启动态搜索namesrv的定时任务
if (null == this.clientConfig.getNamesrvAddr()) {
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
MQClientInstance.this.mQClientAPIImpl.fetchNameServerAddr();
} catch (Exception e) {
log.error("ScheduledTask fetchNameServerAddr exception", e);
}
}
}, 1000 * 10, 1000 * 60 * 2, TimeUnit.MILLISECONDS);
}
//周期性从namesrv拉取主题信息 30秒拉一次
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
MQClientInstance.this.updateTopicRouteInfoFromNameServer();
} catch (Exception e) {
log.error("ScheduledTask updateTopicRouteInfoFromNameServer exception", e);
}
}
}, 10, this.clientConfig.getPollNameServerInterval(), TimeUnit.MILLISECONDS);
//1.清理下线的broker 2.发送消费者和生产者的心跳信息到broker
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
MQClientInstance.this.cleanOfflineBroker();
MQClientInstance.this.sendHeartbeatToAllBrokerWithLock();
} catch (Exception e) {
log.error("ScheduledTask sendHeartbeatToAllBroker exception", e);
}
}
}, 1000, this.clientConfig.getHeartbeatBrokerInterval(), TimeUnit.MILLISECONDS);
//持久化消费者offset
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
MQClientInstance.this.persistAllConsumerOffset();
} catch (Exception e) {
log.error("ScheduledTask persistAllConsumerOffset exception", e);
}
}
}, 1000 * 10, this.clientConfig.getPersistConsumerOffsetInterval(), TimeUnit.MILLISECONDS);
//动态调整消费者的线程池大小 但是从底层代码来看并没有实现
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
MQClientInstance.this.adjustThreadPool();
} catch (Exception e) {
log.error("ScheduledTask adjustThreadPool exception", e);
}
}
}, 1, 1, TimeUnit.MINUTES);
}
-
如果namesrv没指定，开启动态搜索namesrv的定时任务
-
周期性从namesrv拉取主题路由信息 30秒拉一次
-
清理下线的broker ，发送消费者和生产者的心跳信息到broker
-
持久化消费者offset
-
动态调整消费者的线程池大小 但是从底层代码来看并没有实现
③启动拉取消息服务
pullMessageService#start
public void run() {
log.info(this.getServiceName() + " service started");
while (!this.isStopped()) {
try {
//从请求队列里取出一个请求
PullRequest pullRequest = this.pullRequestQueue.take();
if (pullRequest != null) {
//开始拉取消息
this.pullMessage(pullRequest);
}
} catch (InterruptedException e) {
} catch (Exception e) {
log.error("Pull Message Service Run Method exception", e);
}
}
log.info(this.getServiceName() + " service end");
}
pullMessage拉取消息的逻辑有点多，单独开一个篇章解读
④启动消费者负载均衡服务
rebalanceService#start
public void run() {
log.info(this.getServiceName() + " service started");
while (!this.isStopped()) {
this.waitForRunning(waitInterval);
this.mqClientFactory.doRebalance();
}
log.info(this.getServiceName() + " service end");
}
mqClientFactory#doRebalance进行消息队列负载均衡消费消息，逻辑有点多，单独开一个篇章解读
