graylog RawMessage&RawMessageEvent&MessageEvent&Message 说明

RawMessage 是从graylog input 组件获取到的还没解析处理的消息,是有SimpleChannelInboundHandler 处理的原始消息
RawMessageEvent 是Disruptor 处理的事件包装
MessageEvent 是经过input 之后到ProcessBuffer 中的消息事件(Disruptor 包装的)
Message 是经过input 以及ProcessBufferProcessor 之后生成的消息

参考图

MessageEvent 是包含了RawMessage 以及message经过Disruptor 的事件

 

 

RawMessage参考代码

 
@Override
protected void channelRead0(ChannelHandlerContext ctx, ByteBuf msg) throws Exception {
    final byte[] bytes = new byte[msg.readableBytes()];
    msg.readBytes(bytes);
    final RawMessage raw = new RawMessage(bytes, (InetSocketAddress) ctx.channel().remoteAddress());
    input.processRawMessage(raw); // 会通过MessageInput 进行处理,会放到inputbuffer 中
}

RawMessageEvent 参考代码

@Inject
    public InputBufferImpl(MetricRegistry metricRegistry,
                           BaseConfiguration configuration,
                           Provider<DirectMessageHandler> directMessageHandlerProvider,
                           Provider<RawMessageEncoderHandler> rawMessageEncoderHandlerProvider,
                           Provider<JournallingMessageHandler> spoolingMessageHandlerProvider) {
     // 基于Disruptor的事件包装,是由 RawMessage 产生的,具体可以通过
        final Disruptor<RawMessageEvent> disruptor = new Disruptor<>(
                RawMessageEvent.FACTORY,
                configuration.getInputBufferRingSize(),
                threadFactory(metricRegistry),
                ProducerType.MULTI,
                configuration.getInputBufferWaitStrategy());
        disruptor.setDefaultExceptionHandler(new LoggingExceptionHandler(LOG));
 
        final int numberOfHandlers = configuration.getInputbufferProcessors();
        if (configuration.isMessageJournalEnabled()) {
            LOG.info("Message journal is enabled.");
 
            final RawMessageEncoderHandler[] handlers = new RawMessageEncoderHandler[numberOfHandlers];
            for (int i = 0; i < numberOfHandlers; i++) {
                handlers[i] = rawMessageEncoderHandlerProvider.get();
            }
            disruptor.handleEventsWithWorkerPool(handlers).then(spoolingMessageHandlerProvider.get());
        } else {
            LOG.info("Message journal is disabled.");
            final DirectMessageHandler[] handlers = new DirectMessageHandler[numberOfHandlers];
            for (int i = 0; i < numberOfHandlers; i++) {
                handlers[i] = directMessageHandlerProvider.get();
            }
            disruptor.handleEventsWithWorkerPool(handlers);
        }
 
        ringBuffer = disruptor.start();
 
        incomingMessages = metricRegistry.meter(name(InputBufferImpl.class, "incomingMessages"));
        safelyRegister(metricRegistry, GlobalMetricNames.INPUT_BUFFER_USAGE, new Gauge<Long>() {
            @Override
            public Long getValue() {
                return InputBufferImpl.this.getUsage();
            }
        });
        safelyRegister(metricRegistry, GlobalMetricNames.INPUT_BUFFER_SIZE, constantGauge(ringBuffer.getBufferSize()));
 
        LOG.info("Initialized {} with ring size <{}> and wait strategy <{}>, running {} parallel message handlers.",
                this.getClass().getSimpleName(),
                configuration.getInputBufferRingSize(),
                configuration.getInputBufferWaitStrategy().getClass().getSimpleName(),
                numberOfHandlers);
    }

MessageEvent 参考处理

ProcessBuffer.java 通过inputbuffer 放到 ProcessBuffer 
public void insertBlocking(@Nonnull RawMessage rawMessage) {
    final long sequence = ringBuffer.next();
    final MessageEvent event = ringBuffer.get(sequence);
    event.setRaw(rawMessage);
    ringBuffer.publish(sequence);
    afterInsert(1);
}

Message 提供的能力

后续MessageProcessor 处理的就是此类,同时后续的output 也是使用的此消息进行存储处理

 

 

参考资料

https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/plugin/journal/RawMessage.java
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/plugin/Message.java
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/shared/buffers/RawMessageEvent.java
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/plugin/inputs/MessageInput.java

posted on 2022-10-05 09:46  荣锋亮  阅读(112)  评论(0编辑  收藏  举报

导航