### **User description** fix #7996 sync to main ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Make periodic jobs react to config changes - Introduce reusable `spawn_pausable_job!` macro - Fix env var name `ZO_SEARCH_JOB_RUN_TIMEOUT` - Replace fixed intervals with dynamic getters ___ ### Diagram Walkthrough ```mermaid flowchart LR CFG["get_config() (dynamic)"] MACRO["spawn_pausable_job! macro"] JOBS["Background jobs (GC, compactor, alerts, stats, etc.)"] ENV["Env vars (e.g., ZO_SEARCH_JOB_RUN_TIMEOUT)"] ENV -- "drives" --> CFG CFG -- "intervals/pause_if" --> MACRO MACRO -- "spawns/controls" --> JOBS ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Heartbeat interval now read each loop</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+2/-1</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Fix search job timeout env var name</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>16 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Export pausable_job utilities module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f5df40ca59915d9d58ad04ea2440e6ebf29388afaeb8cb0d6c63d2a979df0c17">+1/-0</a> </td> </tr> <tr> <td><strong>pausable_job.rs</strong><dd><code>Add pausable job spawning macro and helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-7e2dce7a3942789804bae659939c214c4eeb68d89315e23a08139214c339ea24">+143/-0</a> </td> </tr> <tr> <td><strong>disk.rs</strong><dd><code>Use pausable job for disk cache GC</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-af63f2c5d85313779df0204c9689e50a6c3916e3aadc24e54e23ec640a584fcb">+4/-10</a> </td> </tr> <tr> <td><strong>memory.rs</strong><dd><code>Use pausable job for memory cache GC</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-ff00d697a9b223ef2858c5e41111a8918e7cc347dc22891c02bda6982682eee1">+4/-16</a> </td> </tr> <tr> <td><strong>lib.rs</strong><dd><code>Mem persist uses dynamic sleep interval</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-603267e2f712eeb4bc46fdff981f1fad1f3a9d3b989a7db4b3c0984fc5b1ed43">+4/-5</a> </td> </tr> <tr> <td><strong>alert_manager.rs</strong><dd><code>Make alert/search jobs pausable and reactive</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-7e30858abad652a860186eb047347305488f52356112b1495dd53ce740316d7c">+61/-104</a></td> </tr> <tr> <td><strong>compactor.rs</strong><dd><code>Refactor compactor jobs to pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-550629b7ebf6460b36fdd46cdfea466c3d3f1e30023c71ee54b87e7998efc500">+134/-188</a></td> </tr> <tr> <td><strong>parquet.rs</strong><dd><code>Use dynamic interval for file push loop</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-27f710981cfe1b938aeeb53abaed05961596c32e2edf3b8a2a59d3e72b2f542b">+1/-1</a> </td> </tr> <tr> <td><strong>flatten_compactor.rs</strong><dd><code>Flatten generator via pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-dd30b917380ba55bd17aadeaee6aa2c87a67e861257a6a14ba2113a9b3015030">+15/-20</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire jobs with reactive spawning; telemetry</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+21/-13</a> </td> </tr> <tr> <td><strong>promql.rs</strong><dd><code>Make metrics leader push pausable/optional</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-15414a41d64eb87bc5c0cd00a07213b3f99c31b346e8e2c36178f7276a1f058c">+26/-31</a> </td> </tr> <tr> <td><strong>stats.rs</strong><dd><code>Stats jobs via pausable macro; conditional start</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f3ca67d9d2c68beeafd6e5de947dbc9a648ac2d2699c5dc87458c369887d09b9">+51/-42</a> </td> </tr> <tr> <td><strong>telemetry.rs</strong><dd><code>Remove legacy telemetry loop (replaced)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-e82cf6d4c890bfaa5e711293b599d7f7a742bb96180639815ede33958b1bc1d9">+0/-36</a> </td> </tr> <tr> <td><strong>storage.rs</strong><dd><code>Make enrichment merge job pausable</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-335bb1eea9193dc398da6838c6a6a11dd6f09e984a1ed4a2e611618a427754f0">+42/-35</a> </td> </tr> <tr> <td><strong>distinct_values.rs</strong><dd><code>Periodic flush via pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-3fb2320197c6e8e900658f5a24158e922610eadcbf081ab0df230f770ef27e07">+11/-18</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Audit publish via pausable macro, optional run</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-c6255010f7a87d6acdb3ad04954fc9f2c27350ae319c53a1857b120aa06408a3">+17/-15</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___
🚀 10x easier, 🚀 140x lower storage cost, 🚀 high performance, 🚀 petabyte scale - Elasticsearch/Splunk/Datadog alternative for 🚀 (logs, metrics, traces).
OpenObserve (O2 for short) is a cloud-native observability platform built specifically for logs, metrics, traces, analytics, RUM (Real User Monitoring - Performance, Errors, Session Replay) designed to work at petabyte scale.
It is straightforward and easy to operate, in contrast to Elasticsearch, which requires understanding and tuning numerous settings. Get OpenObserve up and running in under 2 minutes.
OpenObserve serves as a seamless replacement for Elasticsearch for users who ingest data using APIs and perform searches. OpenObserve comes with its own user interface, eliminating the need for separate installation.
You can reduce your log storage costs by ~140x compared to Elasticsearch by using OpenObserve. Below, we present the results from pushing logs from our production Kubernetes cluster to both Elasticsearch and OpenObserve using Fluent Bit.
🎥 Introduction Video
🌟 Features:
- Logs, Metrics, Traces: Comprehensive support for various data types.
- OpenTelemetry Support: Full compatibility with OTLP for logs, metrics, and traces.
- Real User Monitoring (RUM): Includes performance tracking, error logging, and session replay.
- Dashboards, Reports, Alerts: Features over 18 different chart types for comprehensive data visualization for on-the-fly analysis and reporting along with alerting.
- Pipelines: Enrich, redact, reduce, normalize data on the fly. Stream processing for logs to metrics and more.
- Advanced Embedded GUI: Intuitive and user-friendly interface.
- SQL and PromQL Support: Query logs and traces with SQL, and metrics with SQL and PromQL.
- Single Binary or HA Installation: Install using a single binary for small deployments or in HA mode for large deployments.
- Versatile Storage Options: Supports local disk, S3, MinIO, GCS, Azure Blob Storage.
- High Availability and Clustering: Ensures reliable and scalable performance.
- Dynamic Schema: Adapts to your data structure seamlessly.
- Built-in Authentication: Secure and ready to use.
- Ease of Operation: Designed for simplicity and efficiency.
- Seamless Upgrades: Hassle-free updates.
- Multilingual UI: Supports 11 languages, including English, Spanish, German, French, Chinese, and more.
For a full list of features, check the documentation.
⚡️ Quick start
🐳 Docker:
docker run -d \
--name openobserve \
-v $PWD/data:/data \
-p 5080:5080 \
-e ZO_ROOT_USER_EMAIL="root@example.com" \
-e ZO_ROOT_USER_PASSWORD="Complexpass#123" \
public.ecr.aws/zinclabs/openobserve:latest
🐙 Docker Compose:
services:
openobserve:
image: public.ecr.aws/zinclabs/openobserve:latest
restart: unless-stopped
environment:
ZO_ROOT_USER_EMAIL: "root@example.com"
ZO_ROOT_USER_PASSWORD: "Complexpass#123"
ports:
- "5080:5080"
volumes:
- data:/data
volumes:
data:
For other ways to quickly install OpenObserve or use OpenObserve cloud, check quickstart documentation.
For installing OpenObserve in HA mode, check HA deployment documentation.
📷 Screenshots
Home
Logs
Traces (OpenTelemetry)
Golden metrics based on traces

Visualizations and Dashboards
Front end monitoring
Alerts
Streams
Ingestion
Pipeline
IAM
RBAC (Role Based Access Control)

SBOM
Software Bill of Materials for OpenObserve
Rust
SBOM can be found here. You can analyze it using dependency track.
In order to generate the SBOM, you can use the following commands:
Install cargo-cyclonedx:
cargo install cargo-cyclonedx
Generate the SBOM:
cargo-cyclonedx cyclonedx
JavaScript
SBOM can be found here. You can analyze it using dependency track.
In order to generate the SBOM, you can use the following commands:
Install cyclonedx-npm:
npm install --global @cyclonedx/cyclonedx-npm
Generate the SBOM:
cd web
cyclonedx-npm > sbom.json
⚖️ License
OpenObserve is licensed under the AGPL-3.0 license. For more details, see the LICENSE.
🌍 Community
🔗 Join OpenObserve community on Slack
Easiest way to get support is to join the Slack channel.
📱 Join OpenObserve community on WeChat

















