graylog RawMessage&RawMessageEvent&MessageEvent&Message 说明
RawMessage 是从graylog input 组件获取到的还没解析处理的消息,是有SimpleChannelInboundHandler 处理的原始消息
RawMessageEvent 是Disruptor 处理的事件包装
MessageEvent 是经过input 之后到ProcessBuffer 中的消息事件(Disruptor 包装的)
Message 是经过input 以及ProcessBufferProcessor 之后生成的消息
参考图
MessageEvent 是包含了RawMessage 以及message经过Disruptor 的事件
RawMessage参考代码
@Override
protected void channelRead0(ChannelHandlerContext ctx, ByteBuf msg) throws Exception {
final byte[] bytes = new byte[msg.readableBytes()];
msg.readBytes(bytes);
final RawMessage raw = new RawMessage(bytes, (InetSocketAddress) ctx.channel().remoteAddress());
input.processRawMessage(raw); // 会通过MessageInput 进行处理,会放到inputbuffer 中
}
RawMessageEvent 参考代码
@Inject
public InputBufferImpl(MetricRegistry metricRegistry,
BaseConfiguration configuration,
Provider<DirectMessageHandler> directMessageHandlerProvider,
Provider<RawMessageEncoderHandler> rawMessageEncoderHandlerProvider,
Provider<JournallingMessageHandler> spoolingMessageHandlerProvider) {
// 基于Disruptor的事件包装,是由 RawMessage 产生的,具体可以通过
final Disruptor<RawMessageEvent> disruptor = new Disruptor<>(
RawMessageEvent.FACTORY,
configuration.getInputBufferRingSize(),
threadFactory(metricRegistry),
ProducerType.MULTI,
configuration.getInputBufferWaitStrategy());
disruptor.setDefaultExceptionHandler(new LoggingExceptionHandler(LOG));
final int numberOfHandlers = configuration.getInputbufferProcessors();
if (configuration.isMessageJournalEnabled()) {
LOG.info("Message journal is enabled.");
final RawMessageEncoderHandler[] handlers = new RawMessageEncoderHandler[numberOfHandlers];
for (int i = 0; i < numberOfHandlers; i++) {
handlers[i] = rawMessageEncoderHandlerProvider.get();
}
disruptor.handleEventsWithWorkerPool(handlers).then(spoolingMessageHandlerProvider.get());
} else {
LOG.info("Message journal is disabled.");
final DirectMessageHandler[] handlers = new DirectMessageHandler[numberOfHandlers];
for (int i = 0; i < numberOfHandlers; i++) {
handlers[i] = directMessageHandlerProvider.get();
}
disruptor.handleEventsWithWorkerPool(handlers);
}
ringBuffer = disruptor.start();
incomingMessages = metricRegistry.meter(name(InputBufferImpl.class, "incomingMessages"));
safelyRegister(metricRegistry, GlobalMetricNames.INPUT_BUFFER_USAGE, new Gauge<Long>() {
@Override
public Long getValue() {
return InputBufferImpl.this.getUsage();
}
});
safelyRegister(metricRegistry, GlobalMetricNames.INPUT_BUFFER_SIZE, constantGauge(ringBuffer.getBufferSize()));
LOG.info("Initialized {} with ring size <{}> and wait strategy <{}>, running {} parallel message handlers.",
this.getClass().getSimpleName(),
configuration.getInputBufferRingSize(),
configuration.getInputBufferWaitStrategy().getClass().getSimpleName(),
numberOfHandlers);
}
MessageEvent 参考处理
ProcessBuffer.java 通过inputbuffer 放到 ProcessBuffer 中
public void insertBlocking(@Nonnull RawMessage rawMessage) {
final long sequence = ringBuffer.next();
final MessageEvent event = ringBuffer.get(sequence);
event.setRaw(rawMessage);
ringBuffer.publish(sequence);
afterInsert(1);
}
Message 提供的能力
后续MessageProcessor 处理的就是此类,同时后续的output 也是使用的此消息进行存储处理
参考资料
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/plugin/journal/RawMessage.java
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/plugin/Message.java
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/shared/buffers/RawMessageEvent.java
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/plugin/inputs/MessageInput.java