zabbix5.0 使用elasticsearch7.6按日期索引存储历史数据

个人学习笔记，谢绝转载！！！

原文：https://www.cnblogs.com/wshenjin/p/15023628.html

---

zabbix5.0和elasticsearch7.6的安装忽略

创建ES mapping

Elasticsearch 支持 Zabbix 的监控项类型：uint，dbl，str，log，text，对应如下：

Zabbix 监控项数据类型	对应 Zabbix 表	对应 Elasticsearch 类型
Numeric(unsigned)(无符号整型)	history_uint	uint
Numeric(float)（浮点型）	history	dbl
Character（字符）	history_str	str
Log（日志）	history_log	log
Text	history_text	text

创建ES中的索引模板：

curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_template/uint_template'  -d '    
{
   "index_patterns": [
      "uint*"
   ],
   "settings": {
      "index": {
         "number_of_replicas": 1,
         "number_of_shards": 5
      }
   },
   "mappings": {
      "properties": {
         "itemid": {
            "type": "long"
         },
         "clock": {
            "format": "epoch_second",
            "type": "date"
         },
         "value": {
            "type": "long"
         }
      }
   }
}'


curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_template/dbl_template'  -d '    
{
   "index_patterns": [
      "dbl*"
   ],
   "settings": {
      "index": {
         "number_of_replicas": 1,
         "number_of_shards": 5
      }
   },
   "mappings": {
      "properties": {
         "itemid": {
            "type": "long"
         },
         "clock": {
            "format": "epoch_second",
            "type": "date"
         },
         "value": {
            "type": "double"
         }
      }
   }
}'

curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_template/log_template'  -d '    
{
   "index_patterns": [
      "log*"
   ],
   "settings": {
      "index": {
         "number_of_replicas": 1,
         "number_of_shards": 5
      }
   },
   "mappings": {
      "properties": {
         "itemid": {
            "type": "long"
         },
         "clock": {
            "format": "epoch_second",
            "type": "date"
         },
         "value": {
            "type": "text"
         }
      }
   }
}'

curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_template/text_template'  -d '    
{
   "index_patterns": [
      "text*"
   ],
   "settings": {
      "index": {
         "number_of_replicas": 1,
         "number_of_shards": 5
      }
   },
   "mappings": {
      "properties": {
         "itemid": {
            "type": "long"
         },
         "clock": {
            "format": "epoch_second",
            "type": "date"
         },
         "value": {
            "type": "text"
         }
      }
   }
}'

curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_template/str_template'  -d '    
{
   "index_patterns": [
      "str*"
   ],
   "settings": {
      "index": {
         "number_of_replicas": 1,
         "number_of_shards": 5
      }
   },
   "mappings": {
      "properties": {
         "itemid": {
            "type": "long"
         },
         "clock": {
            "format": "epoch_second",
            "type": "date"
         },
         "value": {
            "type": "text"
         }
      }
   }
}'

Pipeline是在将数据放入索引之前，对数据的某种预处理。可以使用以下命令，为索引创建pipeline:

curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_ingest/pipeline/uint-pipeline'  -d '
{
   "description": "daily uint index naming",
   "processors": [
      {
         "date_index_name": {
            "field": "clock",
            "date_formats": [
               "UNIX"
            ],
            "index_name_prefix": "uint-",
            "date_rounding": "d"
         }
      }
   ]
}'

curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_ingest/pipeline/dbl-pipeline'  -d '
{
   "description": "daily dbl index naming",
   "processors": [
      {
         "date_index_name": {
            "field": "clock",
            "date_formats": [
               "UNIX"
            ],
            "index_name_prefix": "dbl-",
            "date_rounding": "d"
         }
      }
   ]
}'


curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_ingest/pipeline/log-pipeline' -d '
{
   "description": "daily log index naming",
   "processors": [
      {
         "date_index_name": {
            "field": "clock",
            "date_formats": [
               "UNIX"
            ],
            "index_name_prefix": "log-",
            "date_rounding": "d"
         }
      }
   ]
}'

curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_ingest/pipeline/text-pipeline' -d '
{
   "description": "daily text index naming",
   "processors": [
      {
         "date_index_name": {
            "field": "clock",
            "date_formats": [
               "UNIX"
            ],
            "index_name_prefix": "text-",
            "date_rounding": "d"
         }
      }
   ]
}'


curl -H "Content-Type: application/json" -XPUT 'http://127.0.0.1:9200/_ingest/pipeline/str-pipeline'  -d '{
   "description": "daily str index naming",
   "processors": [
      {
         "date_index_name": {
            "field": "clock",
            "date_formats": [
               "UNIX"
            ],
            "index_name_prefix": "str-",
            "date_rounding": "d"
         }
      }
   ]
}'

配置zabbix

zabbix server, /etc/zabbix/zabbix_server.conf:

HistoryStorageURL=http://127.0.0.1:9200
HistoryStorageTypes=uint,dbl,str,log,text
HistoryStorageDateIndex=1

zabbix web, conf/zabbix.conf.php:

// Zabbix GUI configuration file.
//需要将conf/zabbix.conf.php文件的中$HISTORY配置为全局参数，以确保一切正常:
global $DB, $HISTORY;
.....

// Elasticsearch url (can be string if same url is used for all types).
$HISTORY['url'] = 'http://192.168.3.108:9200';

// Value types stored in Elasticsearch.
$HISTORY['types'] = ['uint','dbl','str','log','text'];

重启zabbix server即可。

参考

https://www.zabbix.com/documentation/5.0/zh/manual/appendix/install/elastic_search_setup
https://blog.51cto.com/u_13520772/2329274