RocketMQ存储篇三:追加消息
第一篇介绍了存储的概览,定位到消息追加消息处细看一下
获取mappedFile ——> 追加消息 ——> 处理结果
@DefaultMappedFile#appendMsgInner()
public AppendMessageResult appendMessagesInner(final MessageExt messageExt, final AppendMessageCallback cb, PutMessageContext putMessageContext) { assert messageExt != null; assert cb != null; int currentPos = WROTE_POSITION_UPDATER.get(this); if (currentPos < this.fileSize) { ByteBuffer byteBuffer = appendMessageBuffer().slice(); byteBuffer.position(currentPos); AppendMessageResult result; if (messageExt instanceof MessageExtBatch && !((MessageExtBatch) messageExt).isInnerBatch()) { // traditional batch message result = cb.doAppend(this.getFileFromOffset(), byteBuffer, this.fileSize - currentPos, (MessageExtBatch) messageExt, putMessageContext); } else if (messageExt instanceof MessageExtBrokerInner) { // traditional single message or newly introduced inner-batch message // 使用回调函数来处理append消息的过程 result = cb.doAppend(this.getFileFromOffset(), byteBuffer, this.fileSize - currentPos, (MessageExtBrokerInner) messageExt, putMessageContext); } else { return new AppendMessageResult(AppendMessageStatus.UNKNOWN_ERROR); } WROTE_POSITION_UPDATER.addAndGet(this, result.getWroteBytes()); this.storeTimestamp = result.getStoreTimestamp(); return result; } log.error("MappedFile.appendMessage return null, wrotePosition: {} fileSize: {}", currentPos, this.fileSize); return new AppendMessageResult(AppendMessageStatus.UNKNOWN_ERROR); }
@DefaultAppendMessageCallback#doAppend()
处理逻辑
- 根据bytebuffer中的wroteposition以及mappedfile开头获取物理offset
long wroteOffset = fileFromOffset + byteBuffer.position(); - 构造msgId
- 判断mappedfile是否有足够的空间写msg, 如果没有,就返回一个mappedFile用完的状态,重新分配mappedfile并写入,如果空间足够,就修改encode阶段消息中的一些假值并返回PUT_OK状态
if ((msgLen + END_FILE_MIN_BLANK_LENGTH) > maxBlank) - 根据put的状态来做处理(刷盘以及ha)
那么,消息到底追加到了哪里呢?
如果开启了trasientPoolEnable,就是writeBuffer,否则是mappedByteBuffer
writeBuffer是从trasientPool中borrow的,应用了池化的思想,在刷盘时有作用,后面叙述
mappedByteBuffer就是commitlog文件的映射
protected ByteBuffer appendMessageBuffer() { this.mappedByteBufferAccessCountSinceLastSwap++; return writeBuffer != null ? writeBuffer : this.mappedByteBuffer; }
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 被坑几百块钱后,我竟然真的恢复了删除的微信聊天记录!
· 没有Manus邀请码?试试免邀请码的MGX或者开源的OpenManus吧
· 【自荐】一款简洁、开源的在线白板工具 Drawnix
· 园子的第一款AI主题卫衣上架——"HELLO! HOW CAN I ASSIST YOU TODAY
· Docker 太简单,K8s 太复杂?w7panel 让容器管理更轻松!