Prometheus-Relabel
- Relabel用来重写target的标签
- 每个Target可以配置多个Relabel动作,按照配置文件顺序应用
- Target包含一些内置的标签(以'__'开头),都可以用于relabel,在relabel时未保留,内置标签将被删除
relabel流程
Target([source_label,…]) -> relabel -> Target ([target_label,…])
Relabel的配置
[ source_labels: '[' <labelname> [, ...] ']' ]
[ separator: <string> | default = ; ]
[ target_label: <labelname> ]
[ regex: <regex> | default = (.*) ]
[ modulus: <uint64> ]
[ replacement: <string> | default = $1 ]
[ action: <relabel_action> | default = replace ]
Relabel的action
ACTION | Regex匹配 | 操作对象 | 重要参数 | 描述 |
---|---|---|---|---|
keep | 标签值 | Target | 源标签、regex | 丢弃指定源标签的标签值没有匹配到regex的target |
Drop | 标签值 | Target | 源标签、regex | 丢弃指定源标签的标签值匹配到regex的target |
labeldrop | 标签名 | Label | Regex | 丢弃匹配到regex 的标签 |
labelkeep | 标签名 | Label | Regex | 丢弃没有匹配到regex 的标签 |
Replace | 标签值 | Label名+值 | 源标签、目标标签、替换(值)、regex(值) | 更改标签名、更改标签值、合并标签 |
hashmod | 无 | 标签名+值 | 源标签、hash长度、target标签 | 将多个源标签的值进行hash,作为target标签的值 |
labelmap | 标签名 | 标签名 | regex、replacement | Regex匹配名->replacement用原标签名的部分来替换名 |
replace是缺省action,可以不配置action
使用labeldrop 和labelkeep Relabel后需要注意保证metrics+labels唯一
Replacement会用到了正则捕获组,需要自行补充相关知识
如何查看源标签
从prometheus-》status-》service Discovery
![](https://upload-images.jianshu.io/upload_images/15672389-4ad5ec371593445e.png?imageMogr2/auto-orient/strip|imageView2/2/w/1132)
image.png
过滤target
- 使用keep,保留标签值匹配regex的targets
scrape_configs:
- …
- job_name: "cephs"
relabel_configs:
- action: keep
source_labels:
- __address__
regex: ceph01.*
relabel结果可以在Prometheus网页的status/ Service Discovery中查看
![](https://upload-images.jianshu.io/upload_images/15672389-7d8daa648008a690.png?imageMogr2/auto-orient/strip|imageView2/2/w/480)
image.png
- 使用drop,丢弃匹配regex的targets
scrape_configs:
- …
- job_name: "cephs"
relabel_configs:
- action: drop
source_labels:
- __address__
regex: ceph01.*
![](https://upload-images.jianshu.io/upload_images/15672389-5f4ccc95bd4e1369.png?imageMogr2/auto-orient/strip|imageView2/2/w/496)
image.png
删除标签
将标签名为job的标签删除
scrape_configs:
- …
- job_name: "cephs"
relabel_configs:
- regex: job
action: labeldrop
![](https://upload-images.jianshu.io/upload_images/15672389-9832bc7dcddd5ec5.png?imageMogr2/auto-orient/strip|imageView2/2/w/554)
image.png
labelKeep和labeldrop不操作’__’开头的标签,要操作需要先改名
修改label名
使用replace将scheme标签改名为protocol
scrape_configs:
- …
- job_name: "cephs"
relabel_configs:
- source_labels:
- __scheme__
target_label: procotol
![](https://upload-images.jianshu.io/upload_images/15672389-2711a4d9dad68298.png?imageMogr2/auto-orient/strip|imageView2/2/w/494)
image.png
这里可以是多个source_labels,只有值匹配到regex,才会进行替换
- 使用labelmap,将原始标签的一部分转换为target标签,这一功能replace无法实现
scrape_configs:
- …
- job_name: "sd_file_mysql"
file_sd_configs:
- files:
- mysql.yml
refresh_interval: 1m
relabel_configs:
- action: labelmap
regex: (.*)(address)(.*)
replacement: ${2}
![](https://upload-images.jianshu.io/upload_images/15672389-0373b5e5f25999f8.png?imageMogr2/auto-orient/strip|imageView2/2/w/443)
image.png
修改label值
配置k8s服务发现
scrape_configs:
- …
- job_name: "sd_k8s_nodes"
kubernetes_sd_configs:
- role: node
bearer_token_file: bearer_token
tls_config:
ca_file: ca.crt
namespaces:
names:
- default
api_server: https://master01:6443
服务发现完成后,默认node的port是10250,会无法取得数据,同通过relabel修改标签.
relabel_configs:
- source_labels:
- __address__
regex: (.*)\:10250
replacement: "${1}:10255"
target_label: __address__
![](https://upload-images.jianshu.io/upload_images/15672389-160ad97314b85669.png?imageMogr2/auto-orient/strip|imageView2/2/w/554)
image.png
多标签合并
标签合并,可以将多个源标签合并为一个目标标签,可以取源标签的值,也可以进行hash,用户target分组
- 在文件服务发现中,将标签filename="mysql.yml" 和sd_type="file"合并为sd=”file;mysql.yml”,标签值使用分号连接
scrape_configs:
- …
- job_name: "sd_file_mysql"
file_sd_configs:
- files:
- mysql.yml
refresh_interval: 1m
relabel_configs:
- source_labels:
- sd_type
- filename
separator: ;
target_label: sd
![](https://upload-images.jianshu.io/upload_images/15672389-45ccb58eaad00df6.png?imageMogr2/auto-orient/strip|imageView2/2/w/554)
image.png
- 将多个标签的值进行hash,形成一个target标签,只要target标签一致,则表示源标签一致,可以用来实现prometheus的负载均衡
scrape_configs:
- …
- job_name: "sd_file_mysql"
file_sd_configs:
- files:
- mysql.yml
refresh_interval: 1m
relabel_configs:
- action: hashmod
source_labels:
- __scheme__
- __metrics_path__
modulus: 64
target_label: hash_id
![](https://upload-images.jianshu.io/upload_images/15672389-c9dc47a82ee4cd3f.png?imageMogr2/auto-orient/strip|imageView2/2/w/554)
image.png
完整案例
以下是一个完整的relabel案例,这个案例包括
- 根据标签值过滤target
- 合并标签值,并进行正则匹配
- 修改标签名
- 直接添加标签名
这个案例说明源标签是可以重复使用的
链接:https://www.jianshu.com/p/c21d399c140a