mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 08:09:27 +00:00
Huaijin
17ef03b8d6
Two changes that cut build time, both measured on the same machine (M2
Max,
12 cores) with `cargo build --features mimalloc --profile release`.
## 1. The `web` crate no longer depends on `config`
`src/web` is a leaf crate so the RustEmbed expansion over `web/dist`
only
re-runs when the assets change — that expansion is 149.8s, of which
144.1s is
macro expansion (the crate already sets `#[br = false]` / `#[gzip =
false]`, so
this is not compression).
It also depended on `config`, for exactly one value:
`cfg.common.base_uri`. That
put a 50MB embed downstream of a 724-crate subgraph, so **every `config`
edit
paid the 148.5s embed again** — which is precisely what the crate's own
doc
comment says it exists to avoid.
`ui_routes()` now takes the `<base href>` value from the caller and
`base_href_middleware` carries it as middleware state.
`create_app_router()`
already computed that exact string (`"{base_uri}/web/"`) for its own
redirect,
so it is now resolved once and passed in rather than recomputed per
request
from a second `get_config()` call. `base_uri` is env-derived, normalised
once at
config init and never touched by `refresh_config`, so hoisting it out of
the
request path cannot drift.
The web crate's dependency tree goes **724 → 92 crates**, and `cargo
tree -i
config` no longer lists it.
## 2. Logging and tracing setup moved out of `main.rs`
`setup_logs`, `enable_tracing` and their span-processor / exporter types
compose
very large generic trees. Everything `main.rs` monomorphizes is emitted
into the
binary's own codegen units — the last unit on the critical path,
overlapping
with nothing.
`-Ztime-passes` on the binary unit (91.1s total):
```
LLVM_thinlto 39.8s
LLVM_passes 37.5s
codegen_to_LLVM_IR 5.3s
link_binary 5.3s (run_linker 3.9s)
type_check_crate 0.4s
```
The `openobserve` library does not depend on `openobserve-api-http`, so
its unit
compiles in parallel with the API crates — in the cold build it finished
23.9s
before the binary needed it. Moving the setup into `src/telemetry.rs`
(563
lines) puts that work in the slack instead of at the tail.
Two tests moved with the code and were dropped rather than relocated:
`test_setup_logs` and `test_enable_tracing_error_handling` both install
a global
tracing subscriber, and `setup_logs()` calls `.init()` unguarded, so
they can
only ever pass depending on ordering. They used to race just with each
other
inside the bin test binary; keeping them would have moved that race into
the lib
test binary next to the `cli` and `migration` tests. Neither asserted
anything —
the second wraps the call in `catch_unwind` and discards the result.
`main.rs` 1304 → 712 lines. **Binary unit 88.8s → 80.7s**; the library
unit grows
30.9s → 62.3s and is fully absorbed by the existing slack.
## 3. Unrelated: a dead-code warning in `openobserve-core`
Not a build-time change, just a warning fix noticed while checking the
above.
`COMPARE_ROLLUP_LOOKBACK_MICROS` was gated on `any(feature =
"enterprise",
test)`, but its only two readers (`fetch_arm_aggregate`,
`fetch_arm_failures`)
are `enterprise`-only and the unit tests below it exercise only the pure
aggregation helpers. So in an OSS build `cargo check -p openobserve-core
--all-targets` warned `constant COMPARE_ROLLUP_LOOKBACK_MICROS is never
used`.
Gate narrowed to `enterprise`, and the doc comment — which claimed the
opposite
— corrected. Happy to split this out if you'd rather it not ride along.
## 4. Review follow-ups
- `enable_tracing` became a public library API in §2 without a doc
comment,
unlike `setup_logs` beside it. Documented: which span processors attach
under
which env vars, why the returned provider must be held rather than
dropped,
and that it and `setup_logs` are mutually exclusive since both call
`.init()`.
- Added the `<base href>` test described under Verification below.
Not taken: the suggestion to re-add `test_setup_logs` /
`test_enable_tracing_error_handling` to `src/telemetry.rs`. Both install
a
global subscriber against `.init()`, so they cannot be made
order-independent,
and neither asserted anything — see §2.
`test_resource_key_value_creation` was
deleted for the same reason; it asserted that
`opentelemetry::KeyValue::new`
stores the key it was given.
## Measured and rejected
Recording these so nobody repeats them.
**Moving route registration into the crates that own the handlers.** 317
of the
371 routes were registered in one 622-line function in
`openobserve-api-http`.
Splitting them out to api-management (234), api-search (46),
api-pipelines (23)
and api-ingest (9) worked as designed — api-http went 145.0s → 63.0s —
but the
cold build only improved 654.4s → 637.7s. Splitting across a crate
boundary
**copies** work rather than relocating it: without LTO each crate
instantiates
the axum/tower machinery for itself, so the four receiving crates grew
222.1s
combined and total CPU rose 190.8s. It also just relocated the
bottleneck —
api-management became the new tail at 135.1s, with api-search finishing
only
6.9s behind it, so the tail is a plateau rather than a peak. Not carried
in this
branch.
| Tried | Result |
|---|---|
| sccache | 97.4% hit rate, **0** wall-clock gain; first fill +54% |
| Remove aws-lc-sys from the critical path | **0** — its `dur=72.0s` in
`--timings` is contention, it needs 15.6s alone |
| `codegen-units = 16` on the glue crates | **31.4s slower**, CPU
+297.8s — `lto` defaults to `false`, so thin-local LTO is on and its
cost scales with CGU count |
| Trim DataFusion features | `datafusion-core` depends on
`datafusion-functions` without `default-features = false`, so the
expression features cannot be turned off from the facade at all; what
remains is worth 13.8s of CPU (0.25%) |
A caveat on reading `cargo --timings`: a unit's `duration` is **not**
its
intrinsic cost — it is inflated by CPU contention when the machine is
saturated.
The aws-lc entry above is the clearest example.
## Verification
At this revision:
- `cargo check --all-targets` clean on OSS — 0 errors, 0 warnings — and
each
commit checks clean standalone, so `git bisect` works
- `cargo test -p web` passes
Carried over from the revision these changes were first written against
(the
edits since are doc comments, the `ui_routes` parameter rename, two
deleted
tests, a `Cargo.lock` revert, and the `cfg` narrowing in §3 — which is a
no-op
when `enterprise` is on):
- `cargo check` clean on the enterprise feature combination
- Cold release build succeeds
- Smoke-tested against a running binary: `/healthz` 200, UI and API
routes
resolve, unknown paths 404
The `<base href>` rewrite — the one runtime behaviour §1 touches — is
now
pinned by `base_href_is_rewritten_to_the_mount_point`, and checked end
to end
against the real embedded `index.html`: `ui_routes("/abc/web/")` serves
`<base href="/abc/web/" />`, matching what the old per-request
`format!("{base_uri}/web/")` produced for `ZO_BASE_URI=/abc`. The
committed test
drives a synthetic HTML response instead, so it still asserts something
in
builds where `web/dist` is absent. Verified by mutation: breaking the
rewrite
fails it.
21 lines
785 B
Rust
21 lines
785 B
Rust
// Copyright 2026 OpenObserve Inc.
|
|
//
|
|
// This program is free software: you can redistribute it and/or modify
|
|
// it under the terms of the GNU Affero General Public License as published by
|
|
// the Free Software Foundation, either version 3 of the License, or
|
|
// (at your option) any later version.
|
|
//
|
|
// This program is distributed in the hope that it will be useful
|
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
// GNU Affero General Public License for more details.
|
|
//
|
|
// You should have received a copy of the GNU Affero General Public License
|
|
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
#![recursion_limit = "256"]
|
|
|
|
pub mod cli;
|
|
pub mod migration;
|
|
pub mod telemetry;
|