使用python logging 模块将日志打印到Logstash

目的：将脚本运行时的日志打印到ES，通过kibana展示，特别是后台运行时，可知道脚本运行状态。

0.安装logstash并配置

input {
    tcp {
        port => 5044
        codec => "json"
    }
}

output {
  elasticsearch { 
       hosts => ["10.39.40.2:9200","10.39.40.5:9200","10.39.40.6:9200"]
       manage_template => false
       index => "djangologs_%{+YYYY.MM.dd}"
    }
  stdout { codec => rubydebug }
}

1.安装python-logstash模块

pip install python-logstash

2.示例代码

import logging
import logstash

host = '10.39.46.88'

test_logger = logging.getLogger('python-logstash-logger')
test_logger.setLevel(logging.DEBUG)
test_logger.addHandler(logstash.TCPLogstashHandler(host, 5044, version=1))

test_logger.error('python-logstash: test logstash error message.')
test_logger.info('python-logstash: test logstash info message.')
test_logger.warning('python-logstash: test logstash warning message.')

3.通过kibana查看

 4.python-logstash 集成到django 中

LOGGING = {
  ...
  'handlers': {
      'logstash': {
          'level': 'DEBUG',
          'class': 'logstash.LogstashHandler',
          'host': 'localhost',
          'port': 5959, # Default value: 5959
          'version': 1, # Version of logstash event schema. Default value: 0 (for backward compatibility of the library)
          'message_type': 'logstash',  # 'type' field in logstash message. Default value: 'logstash'.
          'fqdn': False, # Fully qualified domain name. Default value: false.
          'tags': ['tag1', 'tag2'], # list of tags. Default: None.
      },
  },
  'loggers': {
      'django.request': {
          'handlers': ['logstash'],
          'level': 'DEBUG',
          'propagate': True,
      },
  },
  ...
}

参考:

https://pypi.org/project/python-logstash/

https://pypi.org/project/python-logstash-async/1.0.0/