Prometheus is a system to collect and process metrics, not an event logging system

FAQ | Prometheus https://prometheus.io/docs/introduction/faq/#how-to-feed-logs-into-prometheus

How to feed logs into Prometheus?

Short answer: Don't! Use something like the ELK stack instead.

Longer answer: Prometheus is a system to collect and process metrics, not an event logging system. The Raintank blog post Logs and Metrics and Graphs, Oh My! provides more details about the differences between logs and metrics.

If you want to extract Prometheus metrics from application logs, Google's mtail might be helpful.

 

Logs and Metrics and Graphs, Oh My! | Grafana Labs https://grafana.com/blog/2016/01/05/logs-and-metrics-and-graphs-oh-my/

Comprehensive Toolbox

Logs and metrics are complementary. If you’ve taken the Inclusive Monitoring approach, you will have added instrumentation to thousands of places throughout your codebase. Metrics give you an aggregated view over this instrumentation. Logs give you a view of a smaller number of metrics, but give you information about every single request or event.

Metrics are good as a first port of call when dealing with a problem. Combined with well-designed dashboards they allows you to narrow down to which subsystem of which application is behaving oddly. From there you can bring in profiling tools, data mine your logs and cross-check against the source code itself as you deep dive.

Don’t limit yourself by using only one approach to collecting data for monitoring, incorporate both logs and metrics and get the best of both worlds.