mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-06 08:09:24 +00:00
c9205b0276
This implements a background job for consuming metrics generated by openobsere directly in itself. This is controlled by these env vars - - `ZO_SELF_METRIC_CONSUMPTION_ENABLED` - enable/disable self metric consumption, false by default - `ZO_SELF_METRIC_CONSUMPTION_INTERVAL` - interval to run the job in seconds, 60 seconds by default - `ZO_SELF_METRIC_CONSUMPTION_WHITELIST` - comma separated list of metric names to ingest, empty by default. The job does not run if this is empty. On ingesters, this directly calls the jons metric ingestion function to ingest the metrics from the node. Other nodes do an grpc request to one of the ingesters and send the metrics to it. All metrics are ingested under the `_meta` org, as that seemed sensible to have the metric of openobserve in itself. The Json encode code was taken from https://crates.io/crates/prometheus-json-encoder , which unfortunately does not have any original repo anymore. It had to be changed in some places to fit openobserve's needs, and it is originally licensed under Apache 2.0 . I have kept the original notice header in the file as-is. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced self-consumption of metrics, allowing users to configure which metrics to self-consume and at what intervals. - Added a JSON encoder for Prometheus metrics to facilitate integration with other systems. - Enhanced metrics ingestion capabilities to support JSON data. - New configuration options for self-consumption of metrics, including enabling/disabling and specifying a whitelist. - **Bug Fixes** - Improved error handling for metrics ingestion requests. - **Documentation** - Updated public interfaces to reflect new modules and functions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>