graylog 的InputBuffer 简单说明

InputBuffer 是graylog 实际处理日志的部分，内部处理基于了Disruptor，同时还依赖了Journalling 能力
内部实际处理是依赖LocalKafka的（而且是推荐的）

InputBuffer 处理的几个模式

目前主要包含了3大类 DirectMessageHandler,RawMessageEncoderHandler,JournallingMessageHandler
构造(InputBufferImpl 构造函数)

 

 final Disruptor<RawMessageEvent> disruptor = new Disruptor<>(

                RawMessageEvent.FACTORY,

                configuration.getInputBufferRingSize(),

                threadFactory(metricRegistry),

                ProducerType.MULTI,

                configuration.getInputBufferWaitStrategy());

        disruptor.setDefaultExceptionHandler(new LoggingExceptionHandler(LOG));

        // 处理器核心是disruptor 的workpool 实现负载均衡能力

        final int numberOfHandlers = configuration.getInputbufferProcessors();

        if (configuration.isMessageJournalEnabled()) {

            LOG.info("Message journal is enabled.");

 

            final RawMessageEncoderHandler[] handlers = new RawMessageEncoderHandler[numberOfHandlers];

            for (int i = 0; i < numberOfHandlers; i++) {

                handlers[i] = rawMessageEncoderHandlerProvider.get();

            }

            disruptor.handleEventsWithWorkerPool(handlers).then(spoolingMessageHandlerProvider.get());

        } else {

            LOG.info("Message journal is disabled.");

            final DirectMessageHandler[] handlers = new DirectMessageHandler[numberOfHandlers];

            for (int i = 0; i < numberOfHandlers; i++) {

                handlers[i] = directMessageHandlerProvider.get();

            }

            disruptor.handleEventsWithWorkerPool(handlers);

        }

 

        ringBuffer = disruptor.start();

JournallingMessageHandler 依赖MessageQueueWriter 能力，核心实现是onEvent 方法，调用MessageQueueWriter 的write
同时也会依赖Filter ，进行过滤处理
DirectMessageHandler 依赖ProcessBuffer ，具体处理后边会介绍 ProcessBuffer 也依赖Disruptor 进行处理

insert 处理

通过MessageInput 我们可以知道MessageInput核心是依赖的是InputBuffer 的insert 方法，此处依赖了Disruptor
RingBuffer 的publishEvent 方法，具体消息处理就需要依赖上边的MessageHandler 了

说明

graylog 的InputBuffer 比较依赖Disruptor 进行数据处理，毕竟Disruptor 性能是很不错的，好多高性能框架都依赖这个
进行消息处理

参考资料

https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/shared/buffers/JournallingMessageHandler.java
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/shared/buffers/InputBufferImpl.java
https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/shared/buffers/ProcessBuffer.java