graylog MessageProcessor 简单说明

MessageProcessor 是在ProcessBufferProcessor 使用的，主要进行消息的加工，比如过滤，提取，扩展

ProcessBufferProcessor参考处理

private void handleMessage(@Nonnull Message msg) {

    msg.addStream(defaultStreamProvider.get());

    Messages messages = msg;

    // 消息处理的时候使用MessageProcessor 进行消息的进一步加工

    for (MessageProcessor messageProcessor : orderedMessageProcessors) {

        messages = messageProcessor.process(messages);

    }

 

    for (Message message : messages) {

        message.ensureValidTimestamp();

 

        if (!message.hasField(Message.FIELD_GL2_MESSAGE_ID) || isNullOrEmpty(message.getFieldAs(String.class, Message.FIELD_GL2_MESSAGE_ID))) {

            // Set the message ID once all message processors have finished

            // See documentation of Message.FIELD_GL2_MESSAGE_ID for details

            message.addField(Message.FIELD_GL2_MESSAGE_ID, ulid.nextULID());

        }

 

        // The processing time should only be set once all message processors have finished

        message.setProcessingTime(Tools.nowUTC());

        processingStatusRecorder.updatePostProcessingReceiveTime(message.getReceiveTime());

 

        if(failureSubmissionService.submitProcessingErrors(message)) {

            outputBuffer.insertBlocking(message);

        }

    }

}

MessageProcessor 的实现

目前主要包含以下几类，pipeline，filterchain（组合filter ），以及geoip （特殊处理了）
参考图

 

 

参考资料

https://github.com/Graylog2/graylog2-server/blob/626be1f0d80506705b5ba41fbea33c2ec0164bc0/graylog2-server/src/main/java/org/graylog2/shared/buffers/processors/ProcessBufferProcessor.java