Summary
There were some issues in the past with the Prometheus metrics Forest produces. Apparently, there is no validation of whether they are correct or not. The crate we're using for Prometheus metrics has an open issue for doing some of them prometheus/client_rust#52.
For example:
Error scraping target: invalid metric type "receipt>_6_size_bytes gauge"
resolved via #6065
or #4594
the Info metric is not compliant with the default Prometheus/Grafana stack and so we should not use it (unless proven it works fine).
Those broke existing monitoring setups in the past. Unfortunately, even a small issue in the Prometheus report results in a complete halt to metrics ingestion. Given this, we should have integration tests that assert the metrics are readable.
Completion Criteria
Additional Links & Resources
Summary
There were some issues in the past with the Prometheus metrics Forest produces. Apparently, there is no validation of whether they are correct or not. The crate we're using for Prometheus metrics has an open issue for doing some of them prometheus/client_rust#52.
For example:
resolved via #6065
or #4594
Those broke existing monitoring setups in the past. Unfortunately, even a small issue in the Prometheus report results in a complete halt to metrics ingestion. Given this, we should have integration tests that assert the metrics are readable.
Completion Criteria
Additional Links & Resources