mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 16:09:05 +00:00
046b6b6a3d132667ea851b19a18350ec32c5bf79
126
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Hengfei Yang
|
87d30b71bf | chore(cli): support -s stream key for reset -c stream-stats (#14332) | ||
|
Hengfei Yang
|
ccd254ad00 |
refactor(infra): split ORM read/write clients, retire the sqlite write lock (#13927)
Paired with o2-enterprise `refactor/orm-rw-ro-client` (same branch
name).
## Problem
Every ORM caller went through a single client backed by the sqlite
**write** pool, so writes were kept apart only by `table::get_lock()` —
a convention 365 call sites had to remember. Forgetting it surfaced as
`(code: 517) database is locked`: a deferred transaction that reads and
then upgrades to a write finds the snapshot moved under it, and sqlite
reports that **without ever consulting `busy_timeout`**, which is why
the failures were immediate rather than retried.
## Approach
Serialization now comes from the pool, not from a lock. The sqlite write
pool holds **one connection** — all sqlite allows anyway — so writers
queue on acquire and there is nothing to forget.
That is only affordable because reads move to the read-only pool.
Measured, same load, fresh data dir each run:
| | reads on the write connection | reads on the RO pool |
|---|---|---|
| 200 concurrent reads | p50 **135 ms** | p50 **1 ms** |
| writes during mixed load | p50 **189 ms** | p50 **14 ms** |
Both were 100% success with zero lock errors, so this is purely
throughput — but without the split, reads queue behind the single writer
and drag writes down with them.
## API
`infra::db` now exposes three functions and nothing else:
```rust
pub async fn get_orm_client_ro() -> &'static DatabaseConnection
pub async fn get_orm_client_rw() -> &'static DatabaseConnection
pub async fn get_orm_client_ddl() -> &'static DatabaseConnection
```
The statics behind them, every `connect_to_orm_*`, and the raw
sqlite/postgres pools are now private or `pub(crate)`. `ORM_CLIENT`,
`connect_to_orm` and `table::get_lock()` are gone, as is the raw
`CLIENT_RW` mutex (56 lock acquisitions, plus the 52 guard derefs that
existed only to reach through it).
Final split: **RO 247 / RW 360 / DDL 9**; `get_lock` 0, `ORM_CLIENT` 0,
`CLIENT_RW.lock()` 0.
## How read/write was decided
Per call site, then verified by resolving each call through **its own
file's imports** (bare-name matching is useless here — `get_by_id`,
`list` and `update` collide across modules) and propagating write-ness
along the edges that hand a connection to a callee.
A wrong write client only costs throughput; a wrong read client fails in
production. So the audit is zero-miss in the read direction, and a site
is only downgraded when *every* connection-passing callee resolves.
## Behaviour changes on sqlite
All three make sqlite match what postgres already does today — postgres
has never had this lock:
- check-then-write pairs like `folders::put` are no longer atomic; they
rely on their unique index (`folders_org_folder_id_idx`)
- `db::put` runs two sequential transactions, so concurrent puts to one
key are last-write-wins
- `db::delete` runs several statements outside a transaction
`write_pool_is_single_connection_on_sqlite` is now the **only** guard
against reintroducing 517 — please don't drop it.
## Review guide
The mutex removal is separable if you want to look at it on its own:
`src/infra/src/{db,file_list,scheduler,pipeline,schema/history}/sqlite.rs`.
Everything else is the client split.
## Verification
- `cargo check --workspace --all-targets` and the CI clippy command:
clean
- Tests match a HEAD baseline built with the same `.env`, exactly: infra
1410/2, core 1629/1, db 431/0, api-management 179/0. The three failures
are pre-existing and environmental (two need a postgres that isn't
running locally)
- Static scans, all zero: queries on the pool inside an open
transaction; transactions spanning a second connection acquisition;
ro/rw misclassification
- Runtime smoke on a fresh data dir: 200 concurrent reads, then 100
reads + 100 writes simultaneously — 400/400 HTTP 200, rows persisted,
**0** lock errors, 0 ERROR/WARN lines
- Enterprise verified via the `Cargo.toml.openobserve` swap (`cargo
check --all-targets`): clean
Also fixes `users::tests::set_up`, which truncated shared tables but
released its lock before the test body ran, letting a concurrent test
clear rows mid-test.
https://claude.ai/code/session_016ouf1RxiMpiRw5W7GvCUDh
|
||
 Huaijin
|
37e5f55684 |
feat(metrics): .midx series index for TSID-major files and PromQL pruning through it (#13899)
The query half of the TSID-major layout: #13882 wrote `tsid-major-v3-*` files ordered by `(__hash__, _timestamp)`; this PR adds the `.midx` series index next to them and makes PromQL prune through it. Supersedes the remaining scope of #13839. ## Write side - Every `tsid-major-v3-*` file written by the hour-end merge gets a `.midx` series index (Arrow IPC + zstd), stored like the Tantivy index under a derived root distinct from the data tree: `files/{org}/metrics/{stream}/{date}/{hour}/tsid-major-v3-{id}.parquet` → `files/{org}/midx/{stream}/{date}/{hour}/tsid-major-v3-{id}.midx` (Tantivy: `files/{org}/index/{stream}_{type}/…/{id}.ttv`): **one row per contiguous TSID run** — its `row_count: u32` plus the label columns of the run's first row (source types preserved). Runs tile the data file, so each run's starting row is the prefix sum of the preceding counts and is not stored; `__hash__` is excluded (matchers never reference it). The whole index is one ZSTD-compressed IPC batch. Runs crossing an input RecordBatch boundary appear as two adjacent entries; query-time range coalescing makes that representation equivalent without buffering samples. File and row-group boundaries stay independent of TSID boundaries. - `TsidSeriesIndexWriter` lives inside `search/datafusion/merge/tsid_major.rs` and collects run entries alongside the Parquet writer (one buffered row per run), encoding and compressing once at finish. `MergedFile` carries `series_index: Option<Vec<u8>>`; the compactor uploads it to the `midx/` tree (compliance mode included). - `.midx` is intentionally **not** tracked in `file_list`; its path is derived from the data file's key (`MetricsFileLayout::series_index_path`) and it is deleted together with the parent in the delayed-deletion pass (`deleted.rs`); like the `index/` tree, the `midx/` day directories are covered by retention's local-disk deletion and the gc tool. ## Query side (PromQL) - `promql_service::…::tsid_series_index::search` runs where Tantivy used to, before the metrics table is registered: for each TSID-major file it loads the `.midx` (concurrent, `2×target_partitions ≤ 64`), projects the row-range columns plus the matched labels **by name against each series index's own schema** (label sets / column orders differ between files), builds the matcher filter **per series index** with `physical_expr::create_physical_expr` — the exact same predicates as `apply_matchers` (`matcher_predicates`) — evaluates it over the run rows, coalesces adjacent/overlapping ranges and attaches them as `FileSelection::RowRanges` (converted into a Parquet access plan by the generic helper from #13861, no per-row bitmap). - **Applied per file**: files of any other layout in the same query (still-open hours, legacy data, late arrivals) are left in place and scanned in full. A matched label missing from a series index over-selects that file; a query whose matchers cannot be evaluated at all selects the whole file; any series index failure logs and falls back to a full scan. The final PromQL matchers are always applied, so results are exact in every case. - Evaluated selections are cached (LRU, 256 MiB) keyed by account + series index path + the full matcher set. - The Tantivy path in the PromQL storage scan is removed (`convert_matchers_to_index_condition`, index-field plumbing, `is_add_filter_back` bookkeeping). SQL search is untouched. ## Benchmark (from the #13839 POC, same layout + series index) 478,144,896 rows, 7 CPUs, one-hour PromQL range, step 15s, result cache off, release + mimalloc: | Layout | Filter steady median | No-filter steady median | Data files | Total storage | |---|---:|---:|---:|---:| | main, 2 GiB | 695.709 ms | 4.638 s | 88 | 4,925.81 MiB | | TSID-major + `.midx` | 269.496 ms | 2.793 s | 80 | 1,294.61 MiB | ## Validation - `cargo fmt --check`; `cargo check` / `clippy` on config, search, compaction, promql-service, jobs, search_service (`--tests`) clean - `config tsid_layout` 3 passed (incl. `series_index_path`); `search tsid_major mode:: merge::` 10 passed (rotation test now also asserts the series index, exact-run and coalescing series index tests); `compaction merge deleted` 33 passed; `promql-service` 20 passed + 1 pre-existing wall-clock flake (`cache::test_promql_cache_set_and_get`, fails intermittently on main too) - End-to-end on the #13839 branch (same code paths): hour-end merge writes N `tsid-major-v3-*` + `.midx`; PromQL takes the series index path (`selected 39 ranges across 4/4 files`), counts/sums match the synthetic data, cold-cache identical after restart; a mixed hour (3 major + 1 late `tsid-sorted`) reports `1 files without a series index left to tantivy/full scan` with unchanged results; delayed deletion removes `.parquet` **and** `.midx` |
||
|
Huaijin
|
0641e1e486 |
refactor(merge): decide the merge kind once (MergeMode) and share the DataFusion pipeline (#13866)
## Why The kind of merge — plain time-ordered file, trace time-index aggregation, file-list ordering, enterprise metrics downsampling — was re-derived at every layer, each from stream type / stream name / config / batch time range: - compactor scheduler (`merge_by_stream`): `skip_group_files` (downsampling rule lookup #1) → whole-hour batch vs size groups - `merge_files`: `is_match_downsampling_rule` (lookup #2) → early returns, size-limit bypass - `merge_parquet_files`: lookup #3 + SQL selection by stream type, with `is_ingester: bool` standing in for the writer's role `merge_parquet_files` and `merge_parquet_files_with_downsampling` also carried two copies of the DataFusion context / union table / plan / execute-stream / channel / read-task plumbing. Reading the code meant reconciling three derivations of the same fact to know what a batch would produce. ## What - **`search::datafusion::merge::mode::MergeMode { Classic, TraceTimeIndex, FileList, Downsampling(rule) }`** — decided once: `MergeMode::for_compactor(stream_type, stream_name, max_ts)` by the scheduler (carried in `MergeBatch.mode`), `MergeMode::for_ingester(stream_type, stream_name)` by the movers. Every layer only asks it questions: `merges_whole_batch()`, `input_sort_order()`, `sql(schema)`. - **`MergeOutput { file_format, parquet_compression }`** (`for_ingester` / `for_compactor`) replaces the `is_ingester` bool. - **`merge_parquet_files(schema, tables, bloom, meta, &mode, output)`**: one `run_merge_query()` pipeline for every mode, then the mode's writer. Downsampling is a `MergeMode` arm calling `downsampling::write_files` instead of an early-return second implementation (`merge_parquet_files_with_downsampling` removed, −81 lines). - Compactor: `skip_group_files` / `is_match_downsampling_rule` and their `cfg(enterprise)` twin declarations are gone; `compaction` no longer imports `get_largest_downsampling_rule`. Commit 1 (`672ab315a1`) is pure refactor, no behavior change. ## Behavior change (commit 2, `8030eefd04`): whole-hour merges only for a closed hour, over every file of the hour Metrics downsampling used to merge the whole batch whenever the rule matched the batch's newest sample — including incremental rounds on the still-open hour — and it only ever saw the files below the 95%-of-`ZO_COMPACT_MAX_FILE_SIZE` bound of `query_for_merge`: files already above the bound were never downsampled, and small outputs were re-aggregated round after round. - `MergeMode::for_compactor(stream_type, stream_name, range_end_ts, finalize)` is decided **once per hour, before the file_list query**, from the end of the job's date range; a whole-hour mode (downsampling) is only chosen when `finalize` (the hour is closed). Incremental merges of a matching hour use the plain per-stream mode and leave the aggregation to the hour-end pass. - `query_for_merge` takes the `original_size` bound from the caller (`infra::file_list::merge_max_original_size()` = the classic 95% rule); a whole-hour merge asks for **every** file of the hour (`i64::MAX`), the same input range the TSID-major hour-end merge in #13839 uses. - Rule matching uses the hour's end instead of the batch's max `max_ts` (≤ 1h difference; the hour is decided as a unit). ## Guards (commit 3, cherry-picked from #13839) - compactor: fail the merge instead of deleting the sources when `merge_parquet_files` produced no files - search storage `file_list::get_scan_selection` strips the multi-account `{account}/::/` prefix so selections registered by `set` are found again (+ round-trip test) - `deleted.rs`: `delete_from_storage` takes the file slice and a key closure (`Cow`) instead of a pre-collected `Vec` ## Not in this PR The write stage still dispatches to four writers (`write_parquet`, `write_vortex`, `write_downsampled_parquet`, `write_downsampled_vortex`) that share the open / write / rotate / close skeleton and differ only in the rotation policy and how `FileMeta` is derived. Unifying them into one rotating writer is a follow-up (needs a decision on deriving `min_ts/max_ts` from the output batches for the classic path too). ## Validation - `cargo fmt --check`; `cargo clippy -p search -p compaction -p openobserve-jobs --tests` clean - `cargo test -p search --lib -- merge::` 6 passed (new: `mode_by_stream`, `sql_by_mode`, `output_file_format_uses_parquet_for_ingester_metrics`; trace-time-index merge test now goes through `MergeMode::for_ingester` and asserts `TraceTimeIndex`) - `cargo test -p compaction --lib -- merge` 30 passed; `cargo test -p infra --lib -- file_list` 49 passed - `downsampling.rs` is enterprise-only and not compilable locally; `write_files` / `pub(super) generate_downsampling_sql` were edited by hand and need the enterprise CI. |
||
|
Hengfei Yang
|
d702518446 |
perf(schema): enumerate streams with one grouped cache scan per job cycle (#13536)
Design at: #13465 PR 3 of the plan in #13465 (finding 2: full schema-cache rescans per (org, stream type) pair). ## Background `list_streams_from_cache(org, type)` scans every key in `STREAM_SCHEMAS_LATEST` and allocates a `Vec` per key to parse it. The periodic jobs called it inside `for org { for stream_type { ... } }` loops, so one cycle cost O(orgs × types × streams) key visits. With 100k+ streams, hundreds of orgs and 8 stream types, a single compactor generate cycle performs billions of allocating key parses — and several of these jobs run every cycle (`run_generate_job`, retention, stats) or every 60s (metadata metrics). ## Changes **New API:** `list_all_streams_grouped() → HashMap<org, HashMap<StreamType, Vec<name>>>` — one pass over the cache, allocation-free key parsing (iterator-based split instead of `split().collect::<Vec<_>>()`), same name-truncation and dedup semantics as the per-pair function. Covered by a unit test that also asserts it matches `list_streams_from_cache` output. **Callers rewritten to one scan per cycle:** - compaction `run_generate_job` and (enterprise) `run_generate_downsampling_job` - compaction retention `generate_jobs` - compaction stats `update_stats_from_file_list` - flatten compactor `run_generate` - metadata metrics job (also derives the org list and org-count metric from the same grouped result, dropping the separate `list_organizations_from_cache` full scan) - trace service-graph stream discovery (orgs still come from the org table; the per-org cache scan is gone) - CLI `reset_index_updated_at` and `gc-file-list` Per-type skip conditions (EnrichmentTables/Filelist in retention and gc, Index/Filelist in stats, Logs-only in flatten, Logs/Metrics/Traces in metrics) and the org blocklist checks are unchanged. Iterating the grouped result is equivalent to iterating `ALL_STREAM_TYPES` since that constant covers all `StreamType` variants and absent groups were empty scans before. `list_streams_from_cache` stays for point callers (per-stream lookups, `get_stream_count_cached`'s TTL cache is untouched). ## Testing - new unit test `test_list_all_streams_grouped` (grouping, per-type contents, equivalence with the per-pair enumeration) - test suites pass for db, compaction, jobs, core; the only failures are the two pre-existing sqlite-path retention tests that also fail on main - clippy: no findings in touched files Note: overlaps #13531 (metrics.rs/stats.rs hunks) — whichever merges second rebases trivially. Independent of #13535. |
||
|
Huaijin
|
15275ae7cf |
refactor: split core foundation services (#13405)
## Summary This PR begins decomposing `openobserve-core` into focused service crates while preserving the existing application behavior and dependency direction. It extracts six independently owned areas: - `ingestion-common`: shared ingestion contracts and response types - `schema`: schema inference, validation, evolution, and persistence helpers - `compaction`: merge, retention, and compaction services - `enrichment-data`: enrichment-table lifecycle and runtime support - `promql-service`: PromQL service orchestration - `stream`: stream metadata, settings, retention, and lifecycle operations ## What changed - registered the extracted crates in the workspace and propagated enterprise/cloud/vectorscan/vortex features - moved implementations out of Core and updated API, Jobs, query, and queue consumers to use the new crates - removed Core facade aliases for extracted crates and converted remaining callers to direct crate paths - retained the small cross-service stream-delete coordinator in Core for now because it still coordinates Pipeline and Alert cleanup - restored required root/Core dependencies after validating the full enterprise build - added the missing `anyhow` dependency for enterprise-only `search_service::search_jobs` - updated the lockfile for the new crate dependency graph ## Dependency direction The extracted crates do not depend on `openobserve-core`. Core remains a temporary upper-level consumer while the remaining domain services are split incrementally. ```text ingestion-common -> common/config schema -> ingestion-common/db/infra stream -> schema/enrichment-data/db/infra compaction -> schema/search/search-service/db/infra openobserve-core -> ingestion-common/schema/stream/compaction/... api/jobs -> openobserve-core + extracted services ``` ## Enterprise compatibility The private OpenObserve manifest was updated in openobserve/o2-enterprise#2257 to register the extracted workspace members and dependencies. Related Enterprise PR: https://github.com/openobserve/o2-enterprise/pull/2257 ## Validation OSS manifest: - `cargo fmt --all -- --check` - `cargo clippy -- -D warnings` Using `o2-enterprise/Cargo.toml.openobserve` as the OpenObserve root manifest: - `cargo check --no-default-features --features enterprise` - `cargo check --no-default-features --features cloud` - `cargo clippy --workspace --no-default-features --features enterprise -- -D warnings` - `cargo clippy --workspace --no-default-features --features cloud -- -D warnings` Enterprise repository: - `cargo fmt --all -- --check` - `cargo clippy -- -A clippy::question-mark -D warnings` All checks above pass in the development profile. |
||
|
Huaijin
|
6adcae3a1f |
refactor: remove crate compatibility re-exports (#13381)
## Summary
- remove compatibility re-exports introduced during the recent crate
refactors
- migrate callers to the crates that own each API
- delete five pure facade modules and add the required direct crate
dependencies
- keep OSS and enterprise manifests aligned
## Validation
- cargo fmt --all -- --check
- cargo clippy --workspace -- -D warnings
- cargo test --locked --workspace --no-run
- enterprise overlay: cargo test --offline --workspace
--no-default-features --features enterprise --no-run
- enterprise overlay: cargo clippy --offline --workspace
--no-default-features --features enterprise -- -D warnings
- git diff --check
Companion enterprise PR uses the same branch name:
codex/remove-refactor-reexports.
## TODO / Follow-ups
- [ ] **Re-establish the pipeline write-guardrail lost with
`db::pipeline`.** That module was not a pure facade: it re-exported only
the six read-only accessors from `infra::pipeline` and its doc comment
recorded the invariant that *mutations must go through
`service::pipeline::store`* (which keeps the `ExecutablePipeline` caches
and the coordinator/super-cluster watch events in sync). Deleting it
points callers straight at `infra::pipeline`, which also exposes
`put`/`delete`/`drop_table` with no guardrail. Decide the layering and
enforce it:
- **Reads** may hit `infra` directly (raw persistence, no side effects).
- **Writes** must go through the wrapper that persists *and* emits the
coordinator/super-cluster event + refreshes the in-memory cache. That
wrapper is `db::<entity>::{set,delete}` for most entities (e.g.
`db::alerts::alert::set` →
`infra::coordinator::alerts::emit_put_event`), but for pipeline it is
`core::pipeline::store` (not `db`), because the
`STREAM_EXECUTABLE_PIPELINES` cache it maintains is a core-level
construct the `db` crate cannot build. The super-cluster receiver
(`super_cluster_queue::pipelines`) is the one sanctioned caller of
`infra::pipeline::{put,delete}` directly, and it manually pairs each
with `emit_*_event` to avoid a re-broadcast loop.
- Net invariant to protect: never call `infra::pipeline::{put,delete}`
without emitting the coordinator event — i.e. nowhere outside
`core::pipeline::store` and the super-cluster handler. A bare `infra`
write leaves in-memory caches stale until restart (see the warning
comment still live at `src/core/src/org_cleanup.rs:484`).
- [ ] Consider whether the pipeline write-wrapper living in
`core::pipeline::store` while every other entity's lives in
`db::<entity>` is worth unifying, or at least documenting as an
intentional exception.
|
||
|
Huaijin
|
f6560d7d02 |
refactor: decouple search from core (#13338)
## Summary - extract search orchestration from core into the `search_service` crate - keep the existing low-level `search` crate and directory names unchanged - extract usage reporting and audit services into dedicated crates - replace global async function pointers with narrow publisher traits - keep compatibility re-exports while updating workspace dependencies and feature propagation Companion enterprise PR: https://github.com/openobserve/o2-enterprise/pull/2229 ## Validation - `cargo check --workspace` - `cargo check -p search --tests` - `cargo test -p search_service --lib` (184 passed) - enterprise non-release build with the paired `o2-enterprise` manifest - `cargo fmt --all -- --check` - `git diff --check` |
||
|
Hengfei Yang
|
39b2b3ca77 |
feat: add agent_options.output_format (csv/md_table) to _search response (#13343)
Design at: #13343 ## What `_search` gains an optional `agent_options.output_format` parameter (`json` | `csv` | `md_table`, default `json`). When set to `csv`/`md_table`, the response's `hits` array is rendered as a single compact string block: ```json { "query": { "sql": "...", "start_time": ..., "end_time": ... }, "agent_options": { "output_format": "csv" } } ``` Response then carries three new optional fields (never serialized otherwise): `format` (the format actually used), `data` (the formatted block), `advisory` (note about server-side formatting decisions), with `columns` reporting the column order and `hits` emptied. ## Why Agent/MCP consumers pay per token, and a JSON hits array repeats every key on every row — the repeated keys, not whitespace, are the dominant cost (minify only gets to ~78%). csv removes them: ~40% fewer tokens than minified JSON on typical log results, letting the same context budget hold ~1.7x more rows. `md_table` (~8% more than csv) aligns better for small result sets. This is the first piece of the Agent query-optimization (AX) track; `agent_options` is the extension point for the follow-ups (token-budget pagination, query budget/continuation). ## Design - **Serialization edge only**: formatting runs in the `_search` handler after `SearchService::cache::search` returns. Engine, partition scheduling and result cache always see original JSON hits; cached entries never contain formatted strings, and the cache key is unchanged, so a UI (json) query and an agent (csv) query share the same cache entry. - **Model-friendly escaping**: newlines inside cells become a literal `\n` so no record ever spans physical lines (multi-line RFC 4180 quoted cells are a known LLM parsing hazard); commas/quotes still get standard csv quoting so rows stay machine-parsable. `md_table` escapes `|`. Nested objects/arrays are minified JSON in the cell. - **Column order**: server-provided `columns` when present, otherwise first-seen key order with `_timestamp` promoted to the front. - **Sparse fallback**: results with >= 20 columns and > 50% empty cells (typical `select *` on a wide stream) fall back to ndjson — per-row keys are cheaper and less error-prone there than a mostly-empty grid. The fallback is announced via `advisory`, never silent. Non-object rows fall back the same way. - **Zero impact when unset**: requests without `agent_options` produce byte-for-byte identical responses. The remaining 9 files changed are mechanical `agent_options: None` additions to internal `Request` struct literals. - The `_search` `x-o2-mcp` tool description now mentions the option, and `SearchAgentOptions` flows into the generated OpenAPI/MCP tool schema automatically. ## Testing - 9 unit tests in `agent_format.rs`: json no-op, csv basics + `_timestamp` promotion, newline/comma/quote/nested escaping, missing/null cells, server column order, md_table pipe escaping, sparse fallback, non-object fallback, empty hits. - Verified against a running local instance: default path unchanged (no new keys), csv/md_table output with all escaping cases, sparse stream (32 cols, 90% empty) falls back to ndjson with advisory, invalid `output_format` returns 422 listing valid variants, `SearchAgentOptions` present in `/api-doc/openapi.json`. - `cargo check --workspace`, `cargo clippy -p openobserve-api-query`, `cargo fmt` clean. |
||
|
Huaijin
|
3b44da3eee |
refactor: extract shared common crate (#13252)
## Summary - introduce a `common` workspace crate for shared metadata, infrastructure helpers, and utilities - keep service-backed code in `openobserve-core` while preserving the existing `common` compatibility namespace - flatten `src/core/src/service/` into the standard `src/core/src/` module layout without using `#[path]` - retain `service::*` as compatibility re-exports from `service.rs` - move cipher request/response types from core into the search crate while preserving `openobserve_core::cipher` - remove unused Cargo dependencies from workspace crates Enterprise companion: https://github.com/openobserve/o2-enterprise/pull/2202 ## Validation - `cargo check --workspace --all-targets` - `cargo clippy -p common -p openobserve-core -p search --lib -- -D warnings` - common crate unit tests: 271 passed, 4 ignored - enterprise overlay: `cargo check -p openobserve-core --features cloud` - migrated cipher tests: 8 passed - `cargo deny check licenses` - `cargo fmt --all -- --check` - `git diff --check` |
||
 HuaijinandClaude Fable 5
|
81cc0f2255 |
refactor: extract application core crate (#13168)
## Summary Extract the application core (`src/common`, `src/service`, `src/job`, `src/cipher`) out of the root binary crate into a new workspace crate `src/core` (package `openobserve-core`). The root crate keeps the binary entrypoints and edges: `cli`, `handler`, `router`, `migration`, and `super_cluster_queue`. This builds on #13162 (remove service → handler dependencies), which made the extraction possible. > Naming note: the directory is `src/core` but the package stays `openobserve-core` — a Cargo dependency literally named `core` shadows Rust's built-in `core` crate and breaks the `::core::` paths emitted by derive macros (thiserror, serde, ...). ## What moved - `src/common`, `src/service`, `src/job`, `src/cipher` → `src/core/src/...` (467 files moved; 385 of them byte-identical, the rest only needed `pub(crate)` → `pub` widening and clippy fixes required by the new crate boundary) - `ua_regex/regexes.yaml` → `src/core/ua_regex/` (same content, consumer moved with it) - `src/handler/grpc/request/search/mod.rs` → `src/core/src/service/search/grpc_server.rs` (byte-identical; old path kept via `pub use ... as search`) - `impl From<MetricsQueryRequest> for cluster_rpc::MetricsQueryRequest` moved from `handler/grpc/mod.rs` into `service/promql` (orphan rule; field-for-field identical) - The `impl From<XError> for Response` blocks scattered across HTTP handler files (alerts, dashboards, reports, folders, pipeline, eval_jobs, providers, ratelimit) are consolidated into `src/core/src/service/http.rs` with unchanged status-code mappings ## Compatibility `src/lib.rs` re-exports the moved modules (`pub use openobserve_core::{common, job, service}`, and `cipher` under the `enterprise` feature), so all existing `crate::service::...` / `openobserve::common::...` paths keep working — handlers, tests, and downstream code need no changes. ## Enterprise dependency resolution `openobserve-core` optionally depends on `o2_enterprise` / `o2_dex` / `o2_openfga` / `vortex` behind features. So that the OSS repo resolves on its own, workspace deps point at empty stub packages under `src/enterprise-stubs/` (never compiled into default builds). Enterprise builds replace the workspace manifest with `Cargo.toml.openobserve`, which points these at the real crates. ⚠️ Must be merged together with openobserve/o2-enterprise#2162 (same branch name) — merging either PR alone breaks the enterprise build. ## Feature forwarding Root `enterprise` / `cloud` / `vectorscan` / `vortex` features forward to the corresponding `openobserve-core` features; the core crate forwards to `config` / `infra` / `o2_enterprise` exactly as the root crate did before. ## Unit-test fix The `service::sourcemaps` tests relied on the workspace root being the test cwd (shared zip fixture via a relative path, sqlite db dir/tables created by `init-db` from `coverage.sh`). Unit tests of a sub-crate run with the crate dir as cwd, so they now read the fixture via `CARGO_MANIFEST_DIR` and create the db dir + the tables they touch themselves; `src/core/data` is cleaned between the postgres and sqlite CI passes and gitignored, mirroring `src/infra/data`. ## Cleanup The extracted crate's manifest is pruned to what core actually uses (45 dependency entries copied from the root crate plus a dead `profiling` feature removed), and the stale `roaring` dependency (unused since the tantivy result cache rewrite in #7890) is dropped from the root manifest. All new files carry the standard AGPL header. ## Testing - `cargo check --workspace --all-targets`, `cargo clippy -D warnings`, and `cargo fmt --check` pass - `service::sourcemaps` unit tests pass under `--test-threads=1` (the CI setting; they share one db group and are not parallel-safe, same as before) - No logic changes anywhere: the diff is moves + visibility widening + clippy appeasement; error-to-HTTP-response mappings verified variant-by-variant - e2e label kept so Playwright runs --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Hengfei Yang
|
ad743e9017 |
fix: add ttv inspect (#12743)
```
./target/debug/openobserve ttv-inspect -f data/openobserve/stream/files/default/index/stream_0_logs/2026/06/16/01/747246720234867916913ad.ttv
file : data/openobserve/stream/files/default/index/stream_0_logs/2026/06/16/01/747246720234867916913ad.ttv
total_bytes : 21163306
blob_count : 6
── file properties ──────────────────────────
row_group_size : 131072
── blobs ────────────────────────────────────
# name type offset length comp
0 meta.json o2-ttv-v1 4 1650 none
1 c057f4e04d9840849c7cb2311344be18.term o2-ttv-v1 1654 7466195 none
2 c057f4e04d9840849c7cb2311344be18.fast o2-ttv-v1 7467849 9752452 none
3 c057f4e04d9840849c7cb2311344be18.idx o2-ttv-v1 17220301 3898370 none
4 c057f4e04d9840849c7cb2311344be18.pos o2-ttv-v1 21118671 123 none
5 footer_cache o2-ttv-footer-v1 21118794 43493 none
── tantivy index (meta.json) ────────────────
segments : 1
total_docs : 480000 (deleted: 0)
opstamp : 0
- c057f4e0-4d98-4084-9c7c-b2311344be18: max_doc=480000 deleted=0
schema fields : 5
ip text [indexed,fast, tokenizer=raw]
request_id text [indexed,fast, tokenizer=raw]
trace_id text [indexed,fast, tokenizer=raw]
server_name text [indexed,fast, tokenizer=raw]
_timestamp i64 [fast]
```
|
||
|
Huaijin
|
f548a18de4 | refactor: access plan struct improve (#12646) | ||
|
Hengfei Yang
|
cf8d122394 |
feat: track index_updated_at per index field (#12602)
Design at: #12596 closes #12596 ## Problem `index_updated_at` is a single stream-level timestamp, bumped whenever any field is added to `full_text_search_keys` or `index_fields`. The tantivy optimize path (`SimpleCount` / `SimpleHistogram` / `SimpleMultiHistogram` / `SimpleTopN` / `SimpleTopNMulti` / `SimpleDistinct`) excludes every file with `min_ts < index_updated_at`, because optimize results are used directly as final answers with no DataFusion re-check. Consequence: adding one index field to a long-retention stream knocks every histogram/count/top-N over historical data off the fast path until the cutoff ages out of retention — even when the queried fields have been indexed since day one. ## Design ### Storage `StreamSettings` gains `index_fields_updated_at: HashMap<String, i64>`, recording when each field was added to `full_text_search_keys` or `index_fields`: - A field add stamps only genuinely-new fields. A no-op re-add of an existing field no longer bumps anything (previously it pointlessly bumped the stream-level value and disabled the index for all history). - The stream-level `index_updated_at` is still bumped on every real add, so older queriers in a mixed-version cluster keep the old conservative behavior during rolling upgrades. - A field removal prunes its entry (in `normalize_stream_settings`), so a later re-add stamps a fresh time — files written in the gap lack that field's index, and a single timestamp can't represent a gap. - Bloom-only fields auto-folded into `index_fields` by normalization are stamped like a normal add. - The `reset-index-updated-at` CLI clears the map, since per-field entries override the scalar. - The empty map is skipped during serialization; legacy settings without the key deserialize to an empty map. Fully backward/forward compatible, no migration. A field present in both `full_text_search_keys` and `index_fields` shares one entry (last add wins). That is over-conservative for at most one of the two uses and only in the rare case where the same field is added to both lists at different times. ### Query The cutoff in `flight.rs` is now computed from the fields the query actually reads from the index — the optimize-rule target fields (`SimpleTopN` / `SimpleTopNMulti` / `SimpleMultiHistogram` / `SimpleDistinct`): ``` cutoff = max over referenced fields of: BASE_TIME if the field is a default-enabled index field else the per-field entry, else stream-level index_updated_at, else stream created_at ``` - Default-enabled index fields (`SQL_SECONDARY_INDEX_SEARCH_FIELDS`: `trace_id`, `service_name`, `operation_name`, plus `feature_secondary_index_extra_fields`) are indexed since the stream's first file, so they always resolve to `BASE_TIME` and never constrain the cutoff — even if they also carry a per-field entry. - Index-condition fields (equality and FTS/`match_all`) are deliberately **not** part of the cutoff: each file's tantivy schema is the per-file source of truth there, so the timestamps add nothing. - A query that references no per-stream fields — e.g. a count or histogram without filters — gets cutoff `0`: its tantivy result only depends on data every `.ttv` file has. The existing global `ttv_timestamp_updated_at` / `ttv_secondary_index_updated_at` checks are unchanged and still apply on top. This also covers the comment on #12596 that pure histograms don't need the `index_updated_at` check — it falls out of the rule instead of being special-cased. The regular filter-pushdown path is untouched: it never used `index_updated_at` (it checks the actual per-file tantivy schema and adds the filter back to DataFusion when a field is missing). ### Example Stream created 2024-01 (so `trace_id` etc. indexed since then), `user_id` added to `index_fields` on 2026-06-01: | Query | Before | After | |---|---|---| | top-N on `service_name` (default field) | index only for files after 2026-06-01 | all indexed files (`BASE_TIME`) | | top-N / distinct on `user_id` | files after 2026-06-01 | files after 2026-06-01 (correct: older files lack the `user_id` index) | | `histogram(_timestamp)` / `count(*)` | files after 2026-06-01 | all indexed files | ## Intentionally out of scope The cutoff still conflates data time with index build time: a file the compactor merges *after* a settings change carries a fresh, complete index but keeps its old `min_ts`, so it stays off the optimize path. Fixing that requires fetching `created_at` per file from file_list — one more column over millions of rows on every query — to rescue roughly one compaction cycle of data around the settings change, which self-heals within the next hour. Not worth it. |
||
|
Hengfei YangandClaude Opus 4.8
|
9ccf16f192 |
perf: prune partitions in file_list query_by_ids via time range filter (#12459)
## Problem `query_by_ids` runs `SELECT ... WHERE id IN (...)` with **no partition key filter**. On the partitioned `file_list` table this means: - **planning** must open and lock **every** partition (~101ms on 384 partitions) - **execution** probes **every** partition's id index (~10ms) Measured on production-scale data (2.7B rows / 1.2TB / 384 daily partitions, PostgreSQL 17): **~110ms per call**, vs ~1.5ms on a non-partitioned table. `query_by_ids` is on the search hot path (id → FileKey reverse lookup), so every search pays this cost. ## Fix The ids passed to `query_by_ids` are produced by `query_ids` **using a known time range** — the search path already holds it (`service::file_list::query_by_ids` already takes `time_range` but did not pass it down). This PR passes the range down to the infra layer and derives a `date` filter via the existing `derive_date_range`, enabling plan-time partition pruning: ```sql SELECT ... FROM file_list WHERE id IN (...) AND date >= '2026/05/06/00' AND date < '2026/05/14/00' -- new ``` ## Benchmark From the file_list partitioning performance test (2.7B rows, 384 daily partitions, c6i.8xlarge, PG 17.10, jit=off/work_mem=128MB/rpc=1.1/ANALYZEd). Same in-window batch of 1000 ids, median of 3 runs, total = execution + planning: | Query window | Before (total) | After (total) | Improvement | Partitions scanned | |---|---:|---:|---|---| | 1h | 112.7ms (11.7 + 101.0) | **1.18ms** (0.82 + 0.36) | **95x** | 384 → 1 | | 1d | 110.3ms (9.4 + 100.9) | **1.60ms** (0.97 + 0.62) | **69x** | 384 → 2 | | 7d | 109.4ms (8.4 + 100.9) | **2.93ms** (0.79 + 2.14) | **37x** | 384 → 8 | With the filter, the partitioned table matches or **beats** the non-partitioned baseline (execution 0.8~1.0ms vs 1.2~1.3ms — per-day partition id indexes are shallower than one monolithic 2.7B-row index). ## Safety - **Semantically safe**: the ids were selected by `query_ids` with exactly the same derived date range, so the added filter cannot drop rows. - **No side effect on non-partitioned tables**: measured 1.5ms → 1.7ms (within noise) on the single-table baseline. - **Guarded**: the filter is only applied when `time_range` is `Some((start, end))` with `start > 0 && end >= start`; all other callers pass `None` and keep the previous behavior. - SQLite backend ignores the parameter (not partitioned; already a single index probe). ## Changes - `infra::file_list::FileList::query_by_ids` trait: add `time_range: Option<(i64, i64)>` - postgres impl: derive and append the date filter when a valid range is given - sqlite impl: accept and ignore the parameter - `service::file_list::query_by_ids`: pass its existing `time_range` down (db + local cache) - `cli::basic::bloom`: pass `None` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 Subhra264
|
9a1ed606aa |
feat: org level ingestion token (#11691)
Design at: observability/org-level-ingestion-token |
||
 yujinqiu
|
833b2bdc8e | feat(security): enforce strong password policy (#12327) | ||
|
1afabd35ce |
feat: bloom filter pruning layer above tantivy (#12121)
## Summary Design at: https://github.com/openobserve/designs/blob/main/infrastructure/bloomfilter/DESIGN.md A bloom-filter pruning layer that sits **between `file_list` and tantivy** in the search path. It drops candidate files before any `.ttv` is opened, for **positive equality / IN** lookups on high-cardinality fields (`trace_id`, `request_id`, …). It is built **on top of** tantivy: the `.bf` is constructed by iterating the tantivy term dictionary, so a field must be a tantivy secondary index (`index_fields`) **and** opted into `bloom_filter_fields`. Only the intersection is built — a bloom-only field produces nothing. To make that intersection non-empty by construction, enabling a field for bloom now folds it into `index_fields` automatically (see Config). Values are not inspected; the operator chooses which fields are worth it (see guidance below). Design notes live in the module-level docs under `src/infra/src/bloom/` (`mod.rs`, `writer.rs`, `reader.rs`, `sbbf.rs`). ## Design - **Transposed (block-major) `.bf` layout.** All files in a `.bf` share `num_blocks = B`, so `block_index = fastmap(hash(value), B)` is identical across files and the blocks a query needs form **one contiguous row**. The search side reads **one `M × 32`-byte row per `.bf`** to answer membership for all `M` files — prune cost is `O(groups)`, not `O(files)`. (`infra::bloom`, own ~120-LOC SBBF over gxhash64.) - **Sub-grouping.** A hour's new files are sorted by record count and split into chunks of ≤ `ZO_BLOOM_FILTER_MAX_FILES_PER_BF` (default 256), one `.bf` each. Bounds compactor build memory (`M × B × 32`) and `.bf` object size; query reads become `ceil(files / 256)` rows. - **Footer-only reader + batched fetch.** Footer (≈ `M × 12` B) comes from `BLOOM_FOOTER_CACHE` or one 16 KB suffix GET (with a precise re-read when a large footer overflows the probe); rows are pulled with one `cache_storage::get_ranges` call. `cache_storage` was reworked to do real batched cache reads (memory: 1 lookup + N slices; disk: 1 `File::open` + N `pread`). - **B sizing.** `num_blocks_for(max_records_in_chunk, ZO_BLOOM_FILTER_FPP)` (default fpp `0.01`) — record count is a safe NDV upper bound, so the SBBF never saturates for high-cardinality fields. - **Versioned, with orphan cleanup.** A file's `bloom_ver` is write-once — a row is never re-stamped, so dumped rows keep pointing at the `.bf` they were built into. But `bloom_ver` is **not** append-only at the bucket level: a merge produces new output files with a fresh `bloom_ver` and deletes the small inputs, so the old inputs' `.bf` can become orphaned. After each merge round the compactor retires any `.bf` whose `bloom_ver` is no longer referenced by a live `file_list` row in the bucket (enqueued through the existing `file_list_deleted` path). `bloom_ver` is preserved through every file_list INSERT/SELECT path (incl. the SQLite read-cache) and the dump parquet. - **Observability.** Prometheus metrics for prune (`bloom_prune_keep_ratio`, `bloom_prune_duration_seconds`); `bloom-inspect` CLI dumps a `.bf` footer (fields, B, row bytes, file names); footer-cache hit/miss via the shared `BytesCache` tagged metrics; `Local::get_ranges` now reports `STORAGE_READ_*` (previously bypassed). ## When it helps (measured, S3, no disk cache) Same stream `benchtest`, 170 files / 14.64 GB index, single matching row. The only variable is whether the field's values are time-correlated: | Field | Type | tantivy only | bloom + tantivy | Outcome | |---|---|---|---|---| | `trace_id` (16-byte random) | random | **2584 ms** (opens all 170 `.ttv`) | 24 ms prune + 65 ms = **89 ms** | **~29× faster** | | `request_id` (`…-7e74-…`, UUIDv7) | time-ordered | **154 ms** (sparse index range-prunes) | 42 ms + 154 ms = **196 ms** | ~1.3× **slower** | The whole 170-file group was answered by **one row read of 5440 bytes** (`170 × 32`) + a cached footer. **Guidance — enable only when all three hold (advisory; not enforced):** 1. **High cardinality** field. 2. **Random values** (not time-ordered) — so tantivy can't range-prune and must scan every file. Time-ordered ids (UUIDv7, snowflake) are already pruned by tantivy's sparse index in memory → bloom is pure overhead there. 3. **Many files per hour** — hourly volume ≳ 10 × `ZO_COMPACT_MAX_FILE_SIZE` (≳10 `.ttv` files), so there's something to prune. Bloom turns N file opens into one row read; at 1–2 files/hour it's net overhead. The 29× result above had 170 files/hour all answered by one `.bf`. ❌ Low-cardinality / non-indexed / range / `LIKE` / regex / `!=` / OR: no effect. ## Config - `ZO_BLOOM_FILTER_ENABLED` (default true) — global gate. - Per stream: a field must be in **both** `index_fields` and `bloom_filter_fields`. Adding a field to `bloom_filter_fields` auto-adds it to `index_fields` (bloom is built on the secondary index), and the default index fields include the default bloom fields. - `ZO_BLOOM_FILTER_FPP` (default 0.01) — target false-positive probability; sizes the SBBF block count (smaller fpp = fewer false survivors, larger `.bf`). Out-of-range falls back to 0.01. - `ZO_BLOOM_FILTER_MAX_FILES_PER_BF` (default 256) — files per `.bf`; trades query reads vs build memory. - `ZO_BLOOM_FILTER_DEFAULT_FIELDS` (default empty) — fields bloomed for every stream. - `ZO_BLOOM_FOOTER_CACHE_MAX_SIZE` (default 1% mem, clamped 32–256 MB). Removed in this PR: `ZO_BLOOM_FILTER_DISABLED_ON_SEARCH`, `ZO_BLOOM_FILTER_NDV_RATIO` (B is now sized from record count + fpp, not an ndv ratio). ## Test plan - [x] `cargo build`, `cargo clippy -p infra -p openobserve --lib -- -D warnings`, `cargo fmt --check` clean - [x] Unit tests: `cargo test -p infra --lib bloom`, `cargo test -p openobserve --lib bloom` - [x] E2E on S3 (results above): random field prunes 170→1 and is ~29× faster; transposed read = 1 request / 5440 B per group - [x] Reviewer: confirm gating expectations (operator opts fields in; no auto value detection) - [x] Test: orphan bloom: 2 * 256MB -> merge to 512MB -> merge to 1024MB, the first .bf should be deleted - [x] Test: orphan bloom: 6 * 256MB -> merge to 1GB + 512MB -> 512 MB merge to 1024MB, the first .bf include 2 files(1GB + 512MB), after merge again, the first still have 2 files but one of them already deleted. we should keep the first .bf, because still have files in use. ## Notes - `.bf` format is `VERSION 0x01` (transposed). Not compatible with any pre-merge experimental `.bf`; rebuild by wiping the `bloom/` dir and resetting `bloom_ver=0`. - Hash is gxhash64 (project-wide `config::utils::hash`), not the Parquet spec's xxhash64 — `.bf` is internal, never read by external tools. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Hengfei YangandClaude Opus 4.7
|
31ddecd23b |
feat: add gc-file-list cli to clean stale stream files from storage (#12199)
## What Design at: #12199 Relate to: https://github.com/openobserve/openobserve/issues/12105 https://github.com/openobserve/openobserve/issues/12202 Adds a new CLI command `gc-file-list` that deletes orphaned stream files left on remote object storage (s3/gcs/azure). Sometimes the compactor retention job marks files as deleted in the `file_list` table but fails to remove the physical objects from storage, leaving garbage behind. This command cleans up that leftover data. ## How it works For every stream (or a single one via `-s org/stream_type/stream_name`): - Resolves the effective data retention: `stream_settings.data_retention` if set, otherwise `ZO_COMPACT_DATA_RETENTION_DAYS` (mirrors the compactor `retention` logic). - Deletes date directories whose data is **entirely** past the retention cutoff (`day_end <= now - retention`), so the boundary day is always preserved. - Preserves any day overlapping an `extended_retention_days` ("red day") range. - Cleans both the data dir `files/{org}/{type}/{stream}/` and the derived index dir `files/{org}/index/{stream}_{type}/`. ## Performance To avoid loading every object of a TB-scale stream into memory, it walks the `year/month/day` directory tree using delimiter listings (which only return directory names) and only does a full object listing for the day directories it actually deletes. Supporting changes in `infra::storage`: - New `storage::list_dirs(account, prefix)` returning immediate child directories (common prefixes). - Implemented `list_with_delimiter` for the remote store (previously `NotImplemented`). ## Usage ``` openobserve gc-file-list # all streams openobserve gc-file-list -s default/logs/mystream # one stream openobserve gc-file-list -s default/logs/mystream -d # dry-run preview openobserve gc-file-list -a account2 # override storage account (file_hash setups) ``` ## Safety notes - Only works against remote object storage (errors out on local disk). - Streams with effective retention `<= 1` day are skipped as a safety guard. - Recommend running with `-d` (dry-run) first to review what would be deleted. ## Tests Added unit tests for the date/segment parsing and retention decision helpers (`last_segment`, `day_bounds`, `month_start`, and the deletes-old-day / keeps-boundary-day / keeps-extended-retention-day cases). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Hengfei Yang
|
87e7441580 |
perf: add get_ranges for local cache (#12194)
impl #12195 - [x] perf: add get_ranges for local cache - [x] perf: add compact fast - [x] feat: add CLI reset index-updated-at - [x] ci: bump tantivy version to 2.6 |
||
|
Hengfei YangandClaude Opus 4.7
|
57ee4c72f9 |
perf: store row_group_size in tantivy puffin footer and shrink to 128k (#11965)
## Summary - Drop `PARQUET_MAX_ROW_GROUP_SIZE` from 1M to 128k. The constant was previously load-bearing for row_id → row_group mapping, which is why the old comment said it couldn't change. - Write the row-group-size value in effect at index-build time into each `.ttv` puffin file's footer (`row_group_size` property) so readers can recover it later. - Generate parquet access plans from tantivy hits using the **per-file** value (carried through `TantivyResult::RowIdsBitVec` → `CacheEntry` → `FileKey.row_group_size` → `SegmentInfo` in the SEGMENTS cache). - Falls back to `1024 * 1024` for legacy `.ttv` files written before the property existed, so old data keeps working unchanged. ## Why this matters With the constant in the path, the assumed row-group size at read time had to match the one at write time. Now each tantivy index is self-describing, so the writer's row-group-size limit can be tuned freely going forward without breaking row_id lookups against historical files. ## Notes for reviewers - No DB schema change. No parquet footer reads at query time. No tantivy/puffin binary format break — the property is additive via `PuffinMeta.properties` (already `#[serde(default)]`). - `FileKey.row_group_size` is set locally on the querier after tantivy search; it's not serialized over gRPC (parallels `segment_ids`). - `helpers.rs::generate_parquet_access_plan` now uses the per-file size; legacy `.ttv` falls back to `LEGACY_ROW_GROUP_SIZE = 1024 * 1024`. ## Test plan - [x] `cargo check --all-targets --workspace` clean - [x] `cargo test --package tantivy_utils` — all 153 tests pass - [x] `cargo test --package config --lib test_file_key` — pass - [ ] Ingest with the new 128k size, then search — confirm row_ids resolve correctly - [ ] Search against a pre-existing `.ttv` (no `row_group_size` property) — confirm fallback path still finds rows 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Hengfei Yang
|
17a4b4dc23 |
fix: reset file-list-jobs with stream (#11854)
Usage: ``` openobserve reset -c file-list-jobs -t 1700000000000000 openobserve reset -c file-list-jobs -s default/logs/app openobserve reset -c file-list-jobs -t 1700000000000000 -s default/logs/app ``` |
||
|
Hengfei YangandClaude Opus 4.7
|
057348b308 |
feat: add reset file-list-jobs subcommand (#11535)
## Summary Design at: #11532 Closes [#11532](https://github.com/openobserve/openobserve/issues/11532). Adds a new reset component so operators can re-run compaction over a stream's historical file_list: ``` ./openobserve reset -c file-list-jobs [-t <microseconds>] ``` The command performs three things: 1. **Safety check** — calls `infra::cluster::list_nodes()` and aborts with an error if any dedicated compactor node is online (the operator must stop them first). If a `single_node` (Role::All) node is online, prints a warning that it must be restarted after the reset so its in-memory compact state is cleared. 2. **Reset compact offsets** — `db::compact::files::reset_offset(time)` lists every `/compact/files/` key in meta and rewrites its value to `time` (defaulting to `0`), then clears the in-process offset cache. 3. **Re-queue jobs** — calls `infra_file_list::set_job_pending(&[])`, which now treats an empty ids slice as "reset all rows", flipping every `file_list_jobs` row back to `Pending` and clearing `node`, `started_at`, `updated_at`. `set_job_pending` previously returned `Result<()>`; it now returns `Result<u64>` so callers (and the new CLI output) can report rows affected. The existing call site in `service/compact/mod.rs` is unaffected because it ignores the Ok value. ## Test plan - [x] `./openobserve reset -c file-list-jobs` against a stack with no compactor running — verify offsets are 0 and all jobs are Pending - [x] `./openobserve reset -c file-list-jobs -t 1700000000000000` — verify offsets are written as the supplied microsecond timestamp - [x] Run while a compactor is online — verify the command errors with the offending node name - [x] Run on a single_node deployment — verify the warning is shown and operation still completes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
 ByteBakerandHengfei Yang
|
b948361769 |
chore: update copyright year to 2026 (#11166)
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
Hengfei Yang
|
f83f84d841 |
feat: remove mysql support (#10522)
impl #8276, Design at: #8276 |
||
|
Hengfei Yang
|
33eecaefe3 | refactor: migrate actix-web to axum (#9912) | ||
|
Hengfei Yang
|
511f2389c6 | feat: new db migration (#9774) | ||
|
Hengfei Yang
|
e59474aa1b | perf: reduce nats events (#9712) | ||
 Yashodhan Joshi
|
330a179d1c | refactor: file list dump and stream stats (#9236) | ||
|
Hengfei Yang
|
24b89ebf50 | fix: enhance stream stats handling and reset functionality (#9595) | ||
|
Hengfei Yang
|
3435b80d6d | feat: Add cache reload API and CLI command (#9497) | ||
|
ByteBaker
|
2c75a4fea9 |
fix: broken tests from #9009
https://github.com/openobserve/openobserve/pull/9009 broke a few tests by introducing a new param in a function. Fixed those as well as a few clippy issues. |
||
 Loakesh Indiran
|
6b76fbf785 | feat: clear cache main (#8919) | ||
 udev
|
c09f2822e7 |
feat: custom config file path (#8464)
### **PR Type** Enhancement This feature adds a new cli argument to openobserve binary, i.e. -c which allows user to pass in path to a config file, adding flexibility to load env variables from different files. Earlier versions strictly looked for .env file in the current directory. If this flag is not specified when running o2, the default file path (.env) from current directory will be used to load the config files. This feature also adds a job to watch the config file. By default this job will run every 30 seconds, and can be configured via ZO_CONFIG_WATCHER_INTERVAL env variable. The watcher will watch the config file and reload the config if any changes are detected. By default the watcher watches .env file if no other file is configured using -c flag. ___ ### **Description** - Add CLI config file path support - Introduce config file path manager - Implement auto config reload watcher - Expose watcher interval configuration ___ ### Diagram Walkthrough ```mermaid flowchart LR CLI["CLI adds --config option"] -- "set path + reload" --> Manager["ConfigPathManager stores path/hash"] Manager -- "path available" --> Watcher["ConfigWatcher job (interval)"] Watcher -- "hash change" --> Reload["Reload dotenv + refresh configs"] Reload -- "update hash" --> Manager ConfigInit["Config init"] -- "load from path or .env discovery" --> Manager ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>7 files</summary><table> <tr> <td><strong>cli.rs</strong><dd><code>Add --config option and initial reload handling</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+16/-2</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Config loading via path, hashing, and watcher interval</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+44/-3</a> </td> </tr> <tr> <td><strong>config_path_manager.rs</strong><dd><code>New config path and hash manager</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-20874a97fbe99ee6fd692896482ad236916c5f813c3857f824fe0ff6e8afd72a">+64/-0</a> </td> </tr> <tr> <td><strong>lib.rs</strong><dd><code>Export config path manager module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-c1f72c148113548ea854a05d09e3576632fcd7367279563bf3f42f0aef5454e2">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Refactor enterprise config reload helper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+8/-3</a> </td> </tr> <tr> <td><strong>config_watcher.rs</strong><dd><code>Add periodic config file change watcher and reloader</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-598ec79e0686bf314514227cf1f0b1dc560770b0e056761d6a214289d2eeb7f2">+94/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire up config watcher on job init</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>pausable_job.rs</strong><dd><code>Improve macro documentation punctuation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-7e2dce7a3942789804bae659939c214c4eeb68d89315e23a08139214c339ea24">+3/-3</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Hengfei Yang
|
215cf796cd | feat: remove etcd (#8466) | ||
|
Hengfei Yang
|
53d23c4e1f | fix: improve stream stats reset (#8426) | ||
 Ashish Kolhe
|
34a7341e87 |
chore: node list & user session refresh apis (#8198) (#8200)
### **PR Type** Enhancement ___ ### **Description** - Add CLI commands to refresh caches - Expose HTTP endpoints for refresh actions - Cache node list building in cluster module - Replace retention flag with mode enum ___ ### Diagram Walkthrough ```mermaid flowchart LR CLI["CLI subcommands"] -- triggers --> HTTP["HTTP endpoints"] HTTP -- calls --> Cluster["cluster::cache_node_list"] HTTP -- calls --> Sessions["db::session::cache()"] Config["Config: file_list_deleted_mode"] -- controls --> Retention["Compactor retention logic"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8 files</summary><table> <tr> <td><strong>cli.rs</strong><dd><code>New CLI subcommands for cache refresh</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+6/-0</a> </td> </tr> <tr> <td><strong>http.rs</strong><dd><code>HTTP client helpers for refresh endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-790b7780a51e8eecd4754b623af8d15ad87baaaf29455ba646e2e410f83f5498">+22/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Cache node list and update consistent hash</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+33/-0</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Replace retention flag with mode string config</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+3/-2</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Add FileListBookKeepMode enum and conversions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+29/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add refresh nodes and user sessions endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+20/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire refresh endpoints into basic routes</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+3/-1</a> </td> </tr> <tr> <td><strong>retention.rs</strong><dd><code>Use retention mode for history/deleted behavior</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-6abd048f5ef4e7163cffbb51cb4bee10c07fb88b871469d5b3724f7cd04743dc">+16/-6</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Subhra264
|
6f40d2ffc0 | feat: use nats queue for cluster coordination (#8413) | ||
|
Hengfei Yang
|
55a0c62d3f | fix: root user password reset (#8348) | ||
|
sktrpathi
|
f8cc585408 |
feat: store ratelimit info in vendor extension (#8259)
### **PR Type** Enhancement, Documentation ___ ### **Description** - Replace ratelimit comments with OpenAPI extensions - Add `x-o2-ratelimit` metadata to endpoints - Import `serde_json::json` where needed - Clean up obsolete inline ratelimit markers |
||
 Hengfei YangandAshish Kolhe
|
42e965899b |
feat: support use cookie to do cli command (#8189)
### **User description**
We can use cookie to make requests for CLI command for the case enabled
SSO can't use username & password.
```
ZO_CLI_USER_COOKIE=""
```
For now only used for CLI command.
___
### **PR Type**
Enhancement
___
### **Description**
- Add cookie-based CLI authentication
- Fallback to basic auth when no cookie
- Introduce config `ZO_ROOT_USER_COOKIE`
___
### Diagram Walkthrough
```mermaid
flowchart LR
CFG["Config: add root_user_cookie"] -- "read env ZO_ROOT_USER_COOKIE" --> CLI["CLI HTTP request builder"]
CLI -- "cookie present" --> HDR["Set Cookie header"]
CLI -- "cookie empty" --> BASIC["Use basic_auth(email,password)"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>http.rs</strong><dd><code>Add Cookie header auth with
fallback</code>
</dd></summary>
<hr>
src/cli/basic/http.rs
<ul><li>Read <code>root_user_cookie</code> from config.<br> <li> Use
Cookie header when provided.<br> <li> Fallback to basic auth if cookie
empty.<br> <li> Preserve JSON body handling.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8189/files#diff-790b7780a51e8eecd4754b623af8d15ad87baaaf29455ba646e2e410f83f5498">+7/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Support cookie env in auth
config</code>
</dd></summary>
<hr>
src/config/src/config.rs
<ul><li>Add <code>root_user_cookie</code> to <code>Auth</code>
struct.<br> <li> Map to env var <code>ZO_ROOT_USER_COOKIE</code>.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8189/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+2/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com>
|
||
|
Hengfei Yang
|
48115e1ab9 |
fix: enhance machine id check (#8160)
### **User description** - Add duplicated machine id check - Add 4 random chars for generate parquet filename - Use Atomic replace unsafe code ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Centralize node ID generation logic - Replace unsafe globals with atomics - Detect and resolve duplicate node IDs - Truncate long Snowflake IDs when parsing ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Cluster register (etcd/nats)"] -- "collect node ids" --> B["generate_node_id()"] B -- "new id" --> C["LOCAL_NODE_ID (AtomicI32)"] C -- "reload if changed" --> D["Snowflake ID generator"] E["set_status()"] -- "detect duplicate id" --> B F["CLI parse-id"] -- "truncate to 19 digits" --> G["parse i64"] H["Parquet filename"] -- "append 4-hex random" --> I["unique file path"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>cli.rs</strong><dd><code>Truncate IDs longer than 19 digits before parse</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+8/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>10 files</summary><table> <tr> <td><strong>etcd.rs</strong><dd><code>Use centralized node-id generation and atomics</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-37c2c2803d1c12ebb4ef9299bc79c9e6740fc5f19f7f4d84e7cfbb610b9561a9">+3/-17</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add reusable generate_node_id utility</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+17/-0</a> </td> </tr> <tr> <td><strong>nats.rs</strong><dd><code>Centralize id generation; detect duplicates; reload</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-901773980aa255942e7865b977fe63811f31511ac40bc512d6bbb14dbb54bece">+20/-17</a> </td> </tr> <tr> <td><strong>cluster.rs</strong><dd><code>Replace unsafe globals with AtomicI32</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-68d7195ab7cccf22579c4944a37c8a2f592392c30371ce907fadf62fca6f97dd">+3/-3</a> </td> </tr> <tr> <td><strong>ider.rs</strong><dd><code>Read machine id atomically; support reload</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-6ea42b8eab8ba0708f93f9a8b44056d9a9e6ac42517ffba12f81420d6b0bd2b0">+10/-2</a> </td> </tr> <tr> <td><strong>cluster.rs</strong><dd><code>Node equality includes id; NodeStatus i32</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-a2667c1be522ce095796a7066e02631a7e01acc85a46573e1e3a157d2aefb7c3">+4/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add 4-hex random suffix to filenames</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-30f42c3685037c219f0033617f032b23a7f74624ba4ae6c4e3687af401304591">+2/-1</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Initialize generators using atomic node id</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-4696a44abde2df4ac48fed716f930bfdcc01759bdfcfd8d802821c6dd857bff9">+7/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Generate record ids using atomic node id</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+2/-2</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Use atomic node id for stream generators</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-d9596182a0c7a46a6a7ff5cee822554cdae05ceec312f86c93b538db9f73bbfd">+7/-2</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
sktrpathi
|
9a11a680ae |
chore: add test cases (#8095)
### **User description** - also clear over 400 clippy issues ___ ### **PR Type** Tests, Enhancement ___ ### **Description** Add extensive unit tests for enrichment storage Expose helpers for testability in storage module Modernize tests: clippy-friendly string formatting Normalize constants in time-based tests |
||
|
Ashish Kolhe
|
3458678f3a |
feat: Query Optimiser cli (#7745) (#7748)
### **PR Type**
Enhancement
___
### **Description**
- Add `query-optimiser` subcommand with required flags
- Dispatch to `query_optimiser` handler in CLI
- Introduce `query_optimiser` module with enterprise logic
- Fallback stub when enterprise feature disabled
___
### Diagram Walkthrough
```mermaid
flowchart LR
CLIApp["CLI Application"] --> Dispatch["Subcommand Dispatch"]
Dispatch -- "query-optimiser" --> QueryOpt["query_optimiser Module"]
QueryOpt -- "feature enterprise" --> GetRec["get_recommendations API"]
QueryOpt -- "no enterprise" --> Fallback["Unsupported Handler"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>cli.rs</strong><dd><code>Add query-optimiser command
handling</code>
</dd></summary>
<hr>
src/cli/basic/cli.rs
<ul><li>Added <code>query-optimiser</code> command definition<br> <li>
Defined arguments: url, token, duration, etc.<br> <li> Implemented match
branch for dispatch</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7748/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+70/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>query_optimiser.rs</strong><dd><code>Implement
query_optimiser functionality</code>
</dd></summary>
<hr>
src/cli/basic/query_optimiser.rs
<ul><li>Created <code>query_optimiser</code> module file<br> <li>
Enterprise feature calls <code>get_recommendations</code><br> <li>
Fallback prints unsupported message</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7748/files#diff-69a90f896bb4c373e3f710b55453404c9cfd1d4b17a8d57a835e2b07de9a85fe">+64/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Register query_optimiser
module</code>
</dd></summary>
<hr>
src/cli/basic/mod.rs
- Registered new `query_optimiser` module
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7748/files#diff-8820944bd26c070b4d7d80c21d699937e32f8f8d0f3dad496edf580bd83aa321">+1/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Hengfei Yang
|
3b9fe6f4a7 | feat: add parquet cache size in status (#7929) | ||
|
Huaijin
|
682033437e | chore: refactor sql.rs (#7846) | ||
|
sktrpathi
|
10a2849c32 |
chore: add unit tests (#7837)
### **User description** - also fix FGA tests (lack of import) for `enterprise` feature - consolidate parquet readers for file and bytes into generic |
||
|
udev
|
a5af8cea73 |
ci: update cargo deny (#7791)
### **User description**
add libbz2 to exceptions
dependency from datafusion
___
### **PR Type**
Enhancement
___
### **Description**
- Add libbz2-rs-sys license exception
- Update cargo-deny CI configuration
___
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>deny.toml</strong><dd><code>Add libbz2-rs-sys
BSD-2-Clause exception</code>
</dd></summary>
<hr>
deny.toml
<ul><li>Added <code>libbz2-rs-sys</code> exception entry<br> <li>
Allowed <code>BSD-2-Clause</code> license</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7791/files#diff-1040309c64844eb1b6b63d8fd67938adbf9461f1b3c61f12cf738f064a02d3de">+1/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
sktrpathi
|
f0851c3c7e | chore: add UTs, run fmt and clipyy (#7691) | ||
|
ByteBaker
|
5dc27d173f | chore: add UTs, run fmt and clippy |