Huaijin
17ef03b8d6
Two changes that cut build time, both measured on the same machine (M2
Max,
12 cores) with `cargo build --features mimalloc --profile release`.
## 1. The `web` crate no longer depends on `config`
`src/web` is a leaf crate so the RustEmbed expansion over `web/dist`
only
re-runs when the assets change — that expansion is 149.8s, of which
144.1s is
macro expansion (the crate already sets `#[br = false]` / `#[gzip =
false]`, so
this is not compression).
It also depended on `config`, for exactly one value:
`cfg.common.base_uri`. That
put a 50MB embed downstream of a 724-crate subgraph, so **every `config`
edit
paid the 148.5s embed again** — which is precisely what the crate's own
doc
comment says it exists to avoid.
`ui_routes()` now takes the `<base href>` value from the caller and
`base_href_middleware` carries it as middleware state.
`create_app_router()`
already computed that exact string (`"{base_uri}/web/"`) for its own
redirect,
so it is now resolved once and passed in rather than recomputed per
request
from a second `get_config()` call. `base_uri` is env-derived, normalised
once at
config init and never touched by `refresh_config`, so hoisting it out of
the
request path cannot drift.
The web crate's dependency tree goes **724 → 92 crates**, and `cargo
tree -i
config` no longer lists it.
## 2. Logging and tracing setup moved out of `main.rs`
`setup_logs`, `enable_tracing` and their span-processor / exporter types
compose
very large generic trees. Everything `main.rs` monomorphizes is emitted
into the
binary's own codegen units — the last unit on the critical path,
overlapping
with nothing.
`-Ztime-passes` on the binary unit (91.1s total):
```
LLVM_thinlto 39.8s
LLVM_passes 37.5s
codegen_to_LLVM_IR 5.3s
link_binary 5.3s (run_linker 3.9s)
type_check_crate 0.4s
```
The `openobserve` library does not depend on `openobserve-api-http`, so
its unit
compiles in parallel with the API crates — in the cold build it finished
23.9s
before the binary needed it. Moving the setup into `src/telemetry.rs`
(563
lines) puts that work in the slack instead of at the tail.
Two tests moved with the code and were dropped rather than relocated:
`test_setup_logs` and `test_enable_tracing_error_handling` both install
a global
tracing subscriber, and `setup_logs()` calls `.init()` unguarded, so
they can
only ever pass depending on ordering. They used to race just with each
other
inside the bin test binary; keeping them would have moved that race into
the lib
test binary next to the `cli` and `migration` tests. Neither asserted
anything —
the second wraps the call in `catch_unwind` and discards the result.
`main.rs` 1304 → 712 lines. **Binary unit 88.8s → 80.7s**; the library
unit grows
30.9s → 62.3s and is fully absorbed by the existing slack.
## 3. Unrelated: a dead-code warning in `openobserve-core`
Not a build-time change, just a warning fix noticed while checking the
above.
`COMPARE_ROLLUP_LOOKBACK_MICROS` was gated on `any(feature =
"enterprise",
test)`, but its only two readers (`fetch_arm_aggregate`,
`fetch_arm_failures`)
are `enterprise`-only and the unit tests below it exercise only the pure
aggregation helpers. So in an OSS build `cargo check -p openobserve-core
--all-targets` warned `constant COMPARE_ROLLUP_LOOKBACK_MICROS is never
used`.
Gate narrowed to `enterprise`, and the doc comment — which claimed the
opposite
— corrected. Happy to split this out if you'd rather it not ride along.
## 4. Review follow-ups
- `enable_tracing` became a public library API in §2 without a doc
comment,
unlike `setup_logs` beside it. Documented: which span processors attach
under
which env vars, why the returned provider must be held rather than
dropped,
and that it and `setup_logs` are mutually exclusive since both call
`.init()`.
- Added the `<base href>` test described under Verification below.
Not taken: the suggestion to re-add `test_setup_logs` /
`test_enable_tracing_error_handling` to `src/telemetry.rs`. Both install
a
global subscriber against `.init()`, so they cannot be made
order-independent,
and neither asserted anything — see §2.
`test_resource_key_value_creation` was
deleted for the same reason; it asserted that
`opentelemetry::KeyValue::new`
stores the key it was given.
## Measured and rejected
Recording these so nobody repeats them.
**Moving route registration into the crates that own the handlers.** 317
of the
371 routes were registered in one 622-line function in
`openobserve-api-http`.
Splitting them out to api-management (234), api-search (46),
api-pipelines (23)
and api-ingest (9) worked as designed — api-http went 145.0s → 63.0s —
but the
cold build only improved 654.4s → 637.7s. Splitting across a crate
boundary
**copies** work rather than relocating it: without LTO each crate
instantiates
the axum/tower machinery for itself, so the four receiving crates grew
222.1s
combined and total CPU rose 190.8s. It also just relocated the
bottleneck —
api-management became the new tail at 135.1s, with api-search finishing
only
6.9s behind it, so the tail is a plateau rather than a peak. Not carried
in this
branch.
| Tried | Result |
|---|---|
| sccache | 97.4% hit rate, **0** wall-clock gain; first fill +54% |
| Remove aws-lc-sys from the critical path | **0** — its `dur=72.0s` in
`--timings` is contention, it needs 15.6s alone |
| `codegen-units = 16` on the glue crates | **31.4s slower**, CPU
+297.8s — `lto` defaults to `false`, so thin-local LTO is on and its
cost scales with CGU count |
| Trim DataFusion features | `datafusion-core` depends on
`datafusion-functions` without `default-features = false`, so the
expression features cannot be turned off from the facade at all; what
remains is worth 13.8s of CPU (0.25%) |
A caveat on reading `cargo --timings`: a unit's `duration` is **not**
its
intrinsic cost — it is inflated by CPU contention when the machine is
saturated.
The aws-lc entry above is the clearest example.
## Verification
At this revision:
- `cargo check --all-targets` clean on OSS — 0 errors, 0 warnings — and
each
commit checks clean standalone, so `git bisect` works
- `cargo test -p web` passes
Carried over from the revision these changes were first written against
(the
edits since are doc comments, the `ui_routes` parameter rename, two
deleted
tests, a `Cargo.lock` revert, and the `cfg` narrowing in §3 — which is a
no-op
when `enterprise` is on):
- `cargo check` clean on the enterprise feature combination
- Cold release build succeeds
- Smoke-tested against a running binary: `/healthz` 200, UI and API
routes
resolve, unknown paths 404
The `<base href>` rewrite — the one runtime behaviour §1 touches — is
now
pinned by `base_href_is_rewritten_to_the_mount_point`, and checked end
to end
against the real embedded `index.html`: `ui_routes("/abc/web/")` serves
`<base href="/abc/web/" />`, matching what the old per-request
`format!("{base_uri}/web/")` produced for `ZO_BASE_URI=/abc`. The
committed test
drives a synthetic HTML response instead, so it still asserts something
in
builds where `web/dist` is absent. Verified by mutation: breaking the
rewrite
fails it.
Open source Datadog alternative for logs, metrics, traces, and frontend monitoring. Modern observability platform: 10x easier, 140x lower storage cost, high performance, petabyte scale.
Cloud · Documentation · Slack · Quickstart
OpenObserve (O2) is a cloud-native observability platform for logs, metrics, traces, analytics, Real User Monitoring (RUM), and AI/LLM observability. It's a cost-effective alternative to Datadog, Splunk, and Elasticsearch for teams that need full observability without the complexity or cost — with Parquet columnar storage and an S3-native design that cuts storage costs by up to 140x.
Table of Contents
- Why OpenObserve?
- Quick Start
- Product Tour
- Architecture
- Comparisons
- Production Ready
- Security & Compliance
- Enterprise Edition
- Community & Support
- Contributing
- FAQ
- License
- SBOM
Why OpenObserve?
A single platform for all of your observability signals. Here's why teams choose OpenObserve:
| Benefit | Description |
|---|---|
| 140x lower storage cost | Parquet columnar storage + S3-native architecture dramatically reduce costs vs Elasticsearch |
| Single binary deployment | Up and running in under 2 minutes — no complex cluster setup required |
| OpenTelemetry native | Built on the OpenTelemetry standard — no vendor lock-in |
| Unified platform | Logs, metrics, traces, RUM, dashboards, alerts, and incidents in one tool |
| High performance | Better query performance than Elasticsearch on a quarter of the hardware |
| SQL + PromQL | Query logs and traces with SQL, metrics with SQL or PromQL — no proprietary query language |
| Built in Rust | Memory-safe, high-performance, single binary |
Cost comparison: OpenObserve vs Elasticsearch
Quick Start
OpenObserve Cloud (fastest way)
Get started in minutes without managing infrastructure. The free tier includes up to 50 GB/day of ingestion.
🐳 Docker
docker run -d \
--name openobserve \
-v $PWD/data:/data \
-p 5080:5080 \
-e ZO_ROOT_USER_EMAIL="root@example.com" \
-e ZO_ROOT_USER_PASSWORD="Complexpass#123" \
public.ecr.aws/zinclabs/openobserve:latest
Then open http://localhost:5080 and log in with the credentials above.
For other installation methods, see the quickstart documentation. For clustered deployments, see the High Availability deployment guide.
Product Tour
OpenObserve ships with a powerful, unified web UI for every signal — logs, traces, metrics, dashboards, RUM, alerts, incidents, pipelines, and AI observability.
🏠 Unified Overview
A single home for your workspace — active incidents, service health (error rate, latency, requests), anomalies, and recent events at a glance.
📊 Logs
Centralized log management with full-text search, SQL queries, quick filters, and a visual query builder. Instantly search across all your logs, build dashboards from log data, and set up alerts — all on Parquet columnar storage for 140x lower storage cost than Elasticsearch. Read more →
🔍 Distributed Tracing
Powered by OpenTelemetry, tracing helps you follow requests across services and pinpoint performance bottlenecks. Explore the full request flow with waterfalls, flame graphs, and Gantt charts; click any span to drill into the trace. Read more →
🕸️ Service Graph
Visualize service-to-service dependencies and request flow across your system, with per-edge request counts and health-based coloring (healthy, degraded, warning, critical) to spot problem hotspots at a glance.
📈 Metrics
Explore metrics from your infrastructure and applications, then query them with SQL or PromQL. Browse thousands of metrics with faceted filters, preview them inline, combine multiple queries with formulae, and visualize the results with 19+ chart types. Read more →
📉 Dashboards
Build custom dashboards from any signal with 19+ built-in chart types and 200+ visualization variations, a drag-and-drop panel builder, template variables, and geo maps. Read more →
👀 Frontend Monitoring (RUM)
Real User Monitoring with Core Web Vitals, error tracking, performance analytics, and full session replay — so you can see exactly what your users experience. Read more →
🔔 Alerts
Get notified when something unusual happens on any signal — logs, metrics, or traces. Define thresholds, scheduled or real-time alerts, and notification channels, with alert history and anomaly detection to catch issues early. Read more →
🚨 Incidents
Correlate related alerts into incidents and track them through their lifecycle — open, acknowledged, and resolved — with severity and dimension context for faster response.
🔀 Pipelines
Enrich, redact, reduce, or normalize data at ingest time with a visual editor. Build stream-processing flows — including logs-to-metrics conversion — from source, transform (VRL functions and conditions), and destination nodes. No external tools required. Read more →
🤖 AI Observability
Monitor your GenAI and LLM applications: track cost, tokens, latency percentiles, and error rates across models, with agent graphs, session traces, and evaluation/quality scoring.
✨ O2 AI Assistant
An in-product assistant that writes your SQL, VRL, and PromQL and walks you through logs, traces, metrics, and incidents — turning natural-language questions into queries, dashboards, and alerts.
For the full feature list, see the documentation.
Architecture
OpenObserve achieves 140x lower storage costs and high performance through a modern, cloud-native architecture:
- Parquet columnar storage — efficient compression and fast analytical queries
- S3-native design — inexpensive object storage with intelligent caching
- Built in Rust — memory-safe, high-performance, single binary
- Partitioning, indexing, and smart caching — reduces search space by up to 99% for most queries
- Native multi-tenancy — organizations and streams as first-class concepts with complete data isolation
- Stateless architecture — rapid scaling and low RPO/RTO for disaster recovery
Scale & Deployment
- Thousands of concurrent users can query a single cluster simultaneously
- Single binary scales to terabytes — unique in the observability space
- High Availability mode scales to petabytes for the most demanding workloads
- Multi-region deployments with cluster federation via Super Cluster architecture (Enterprise)
- Federated search across regions and clusters (Enterprise)
High Availability & Disaster Recovery
Deploy in High Availability mode with clustering for mission-critical workloads requiring maximum uptime. OpenObserve's stateless architecture with S3-backed storage enables very low Recovery Point Objective (RPO) and Recovery Time Objective (RTO): stateless nodes restart rapidly, and durability is guaranteed by S3's 99.999999999% (11 nines).
Read the architecture documentation → · Read the enterprise deployment guide →
Comparisons
OpenObserve vs Datadog
| Aspect | OpenObserve | Datadog |
|---|---|---|
| Deployment | Self-hosted or Cloud | SaaS only |
| Pricing model | Per-GB (free up to 200 GB/day) | Per-host + per-GB |
| Open source | Yes (AGPL-3.0) | No |
| OpenTelemetry | Native OTLP | Supported |
| Query language | SQL + PromQL | Proprietary |
| Vendor lock-in | None | High |
OpenObserve vs Elasticsearch
| Aspect | OpenObserve | Elasticsearch |
|---|---|---|
| Storage cost | 140x lower | High (hot/warm/cold tiers) |
| Setup complexity | Single binary | Complex cluster management |
| Query language | SQL | Lucene/KQL |
| Hardware requirements | ~1/4 the resources | High memory/CPU |
OpenObserve vs Splunk
| Aspect | OpenObserve | Splunk |
|---|---|---|
| Licensing | Open source | Expensive enterprise licensing |
| Deployment | Single binary or HA cluster | Complex |
| Query language | SQL + PromQL | SPL (proprietary) |
| Cost | Predictable, low | Unpredictable, high |
OpenObserve vs Grafana/Loki/Prometheus Stack
| Aspect | OpenObserve | Grafana Stack |
|---|---|---|
| Components | Single platform | Multiple tools (Grafana + Loki + Prometheus + Tempo) |
| Management | One binary | Multiple deployments |
| High cardinality | Full support | Loki struggles with high cardinality |
| Query performance | Fast on large volumes | Loki slow on large data |
Production Ready
OpenObserve is battle-tested in production environments worldwide:
- Thousands of active deployments across diverse industries
- Largest deployment: 2+ PB/day ingestion
- Single binary scales to terabytes — unique in the observability space
Security & Compliance
Security Features
- Secure by design with hardened container images
- Data encryption at rest and in transit
- Sensitive Data Redaction (SDR) — automatically redact sensitive data at ingestion and query time (Enterprise)
- Single Sign-On (SSO) — OIDC, OAuth, SAML, LDAP/AD integration (Enterprise)
- Role-Based Access Control (RBAC) — granular permissions (Enterprise) — Learn more →
Compliance Certifications
- ✅ SOC 2 Type II certified
- ✅ ISO 27001 certified
- ✅ GDPR compliant
- ✅ HIPAA ready (BAA available with Enterprise contracts)
OpenObserve meets the stringent security and compliance requirements of regulated industries including finance, healthcare, and government.
Enterprise Edition
OpenObserve is a true open source project. The open source edition is feature-complete and production-ready — logs, metrics, traces, dashboards, alerts, pipelines, and everything you need to run observability at scale. It will always remain actively maintained and free to use without restrictions.
For organizations that need enterprise-grade features and support, an Enterprise edition adds:
Enterprise features
- Single Sign-On (SSO) — OIDC, OAuth, SAML 2.0, LDAP/AD, and major identity providers (Okta, Azure Entra, Google, GitHub, GitLab, Keycloak)
- Advanced RBAC — granular role-based access control with custom roles — Learn more →
- Audit trails — comprehensive immutable audit logs with configurable retention
- Federated search — query across multiple clusters and regions with Super Cluster
- Sensitive Data Redaction (SDR) — automatically redact PII at ingestion and query time
- Advanced encryption — AES-256 SIV cipher keys with Google Tink KeySet and Akeyless integration
- Query & workload management (QoS) — control query resource usage and priorities in multi-tenant environments
Support & SLAs
- Dedicated support with contractual SLA guarantees and priority response times
- Technical account management, architecture review, and deployment assistance
- Migration support from existing tools, plus training and onboarding
Pricing
- Free tier: up to 50 GB/day of ingestion (~1.5 TB/month), including full commercial use (registration required at 50 GB/day)
- Volume discounts and multi-year contracts available
- View the complete feature comparison →
For enterprise inquiries and custom deployments, contact our sales team.
Community & Support
The best way to get help, share ideas, and connect with other OpenObserve users is through our community channels.
🔗 Join us on Slack
Our Slack community is the most active place for installation and configuration help, sharing best practices, discussing the roadmap, and connecting with the core team.
📱 Join the OpenObserve community on WeChat
Other ways to connect
- 💬 GitHub Discussions — longer-form discussions and Q&A
- 🐛 GitHub Issues — report bugs or request features
- 📖 Documentation — guides, tutorials, and API references
Contributing
We welcome contributions from the community! Whether you're fixing bugs, adding features, improving documentation, or sharing feedback, your help makes OpenObserve better for everyone.
To get started, read our Contributing Guide, which covers setting up your development environment, code standards, submitting pull requests, and reporting issues.
FAQ
How does OpenObserve achieve 140x lower storage costs?
Through a combination of Parquet columnar storage (efficient compression) and an S3-native architecture (inexpensive object storage). See the cost comparison chart in the Why OpenObserve? section.
What are the limitations?
All data in OpenObserve is immutable — once ingested, it cannot be modified or deleted (only entire retention periods can be dropped). This is by design and is a feature for logs and compliance use cases, ensuring data integrity and audit trails.
Is this production-ready?
Yes. OpenObserve runs in production across thousands of deployments worldwide, including environments processing in excess of 2 PB/day. See our customer stories for real-world examples.
How does query performance compare to Elasticsearch?
OpenObserve delivers better performance than Elasticsearch for most workloads, with faster search and significantly faster analytics — while using about a quarter of the hardware. The columnar Parquet format is particularly effective for complex aggregations and analytics.
Is there a steep learning curve?
No. OpenObserve is designed to be intuitive from day one:
- Familiar query languages — SQL for logs and traces, PromQL for metrics; no proprietary query language to learn
- Easy-to-use GUI — an intuitive interface with a drag-and-drop dashboard builder
- No complex tuning — unlike Elasticsearch, there are no shards, replicas, or heap sizes to manage. Just install and go.
Most users are productive within hours, not weeks.
License
Open Source Edition — licensed under AGPL-3.0. We chose AGPL to ensure that improvements to OpenObserve remain open source and benefit the entire community, while still allowing free commercial use. Why AGPL, and why it's good for the community →
Enterprise Edition — licensed under a commercial Enterprise License Agreement (not AGPL), which provides additional flexibility for enterprise deployments.
SBOM
Software Bill of Materials for OpenObserve. You can analyze either SBOM with Dependency-Track.
Rust
The SBOM is available here. To regenerate it:
cargo install cargo-cyclonedx
cargo-cyclonedx cyclonedx
JavaScript
The SBOM is available here. To regenerate it:
npm install --global @cyclonedx/cyclonedx-npm
cd web
cyclonedx-npm > sbom.json