mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 00:09:35 +00:00
5fdff20e5c4a6fb679ad5983738eff1533e52b6e
119
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Amey PawarandHengfei Yang
|
f1af2562bd |
docs(alerts): stop promising the create body's folder_id places the alert (#14121)
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
 udevandHengfei Yang
|
e06fe8e686 |
fix: register experiment row sort OpenAPI schema (#14128)
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
 
|
76bdf957ce |
fix: ai obs issues (#14114)
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
|
Hengfei Yang
|
8ed8ec0509 | fix: reject client-forged session trust markers in auth (#14105) | ||
|
Hengfei Yang
|
2feb1f7f1d | fix: stop two handler panics the API fuzzer reaches through query params (#14089) | ||
 Prabhat Sharma
|
478824a41e | feat(alerts): add curated alerts from the library in bulk (#14054) | ||
  
|
1329d6a81a |
feat: pending period for alerts (#14009)
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: ktx-akshay <akshay.patil@kiara.tech> |
||
|
Hengfei Yang
|
9a89aee4b2 | fix: keep search trace collection independent of log level and caller propagation (#14074) | ||
 Vaibhav Mittal
|
4ff12d1194 |
fix: publish nodes only after gRPC listener binds (#14031)
Signed-off-by: Vaibhav mittal <vaibhavmittal929@gmail.com> |
||
 Ashish Kolhe
|
043366ba6d | fix(status-pages): pair the brand logo with the page name and link the OpenObserve mark (#14047) | ||
 Ashish Kolheandgithub-actions[bot]
|
07807995e4 |
feat: public status pages built on synthetics checks (#13974)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
 0xfandomandHengfei Yang
|
906f3c46f4 |
fix: reject empty org_id in POST /api/{org_id}/users (#13806)
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
Prabhat Sharma
|
9a77c6b878 | feat: Slack destination setup via OAuth and app manifest (#14020) | ||
|
9430c5f061 |
feat(synthetics): meter step-based billing usage, shipped dark (#13951)
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Hengfei Yang
|
eedc773cd7 | chore: remove action scripts / action server (#14000) | ||
|
udevandsai nikhil kethe
|
9e5488ed2b |
feat(llm-obs): add Playground and remote task workflows (#13991)
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Hengfei Yang
|
78c18ad300 | feat: org-scoped search inspector with dedicated permission (#14005) | ||
|
Hengfei Yangandgithub-actions[bot]
|
f0df035b49 |
feat(alerts): add is_null/is_not_null/is_empty/is_not_empty condition operators for alerts, pipelines and evaluator filters (#13978)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Hengfei Yang
|
adf55fdba4 | perf(traces): batched per-stream scan for multi-id time range lookup (#14002) | ||
|
|
3e33b0e262 |
fix: workflow UX improvements (#13929)
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Hengfei Yang
|
bbf3754d3b | feat(traces): multi-id and org-wide trace time range lookup (#13992) | ||
|
Hengfei Yang
|
ebdc2f0a4e |
fix(search): release the admission slot and follower task status on cancellation (#13954)
Paired with o2-enterprise `hf/issue-2491-admission-slot-leak` (same branch name). **Stacked on #13953** — base is that branch, so the diff here is just this change. Retargets to `main` automatically once #13953 merges. ## Problem #13953 fixed the wal file locks. `do_get` touches two more trace-id keyed resources, and both had the identical defect: release depended on control flow reaching a call, inside the window where gRPC cancellation drops the future. **The node's admission slot.** Released from three hand-written sites — the two error arms in `do_get` and `FlightEncoderStream::drop`. All three are skipped when the request is cancelled during plan generation, because the encoder stream does not exist yet. **The follower task status.** In the super-cluster path the entry was inserted before plan generation and removed after it: ```rust if is_super_cluster && !SEARCH_SERVER.contain_key(&trace_id).await { SEARCH_SERVER.insert(trace_id.clone(), TaskStatus::new_follower(vec![], false)).await; } let result = get_ctx_and_physical_plan(&trace_id, &req).await; // <-- cancellable if is_super_cluster && !SEARCH_SERVER.is_leader(&trace_id).await { SEARCH_SERVER.remove(&trace_id, false).await; } ``` Cancel in the middle and the entry stays in the map for the life of the process. Nothing else removes it: an explicit `cancel_query` would, but a timeout or a client disconnect would not. ## Approach Both move onto the same ownership chain #13953 established. `SessionGuard` — already armed on the first line of `do_get` and already carried into `FlightEncoderStream` — now releases the admission slot too, so the three call sites collapse into its drop. The follower task status is registered through an `AsyncDefer` that lives in the same scope as plan generation, so it is removed on every exit including a cancelled one. It is now removed only when this node registered it, rather than whenever the entry is not a leader's: a request can no longer clear an entry it did not create. In practice these coincide, since the composite `{trace_id}-{job_id}` is unique per request. ### Drive-by The `physical plan generate error` and `physical plan execution error` logs sat *inside* `if get_o2_config().work_group.max_nodes_per_query > 0`, so a plan failure was silent unless work groups happened to be configured — and silent in every OSS build. They are unconditional now. ## Enterprise side The paired PR gives `Running` reservations a TTL backstop. They had no expiry edge at all, so a slot whose release was lost — this cancellation path, or a leader that dies before sending `Release` — was held until the process restarted. This PR closes the local path; the backstop covers the remote one, which no local guard can. ## Verification - `cargo fmt --all` and the CI clippy command: clean - Enterprise verified via the `Cargo.toml.openobserve` swap against the paired branch: `cargo check --all-targets` clean, and `cargo clippy --no-deps -p openobserve-api-grpc --all-targets` clean. The workspace-wide enterprise clippy still reports the two pre-existing `cognitive_complexity` errors in `core/src/alerts/scheduler/handlers.rs` and `core/src/ofga/mod.rs`, untouched by this change - `api-grpc` flight tests 21/21 ## Also here Three zh-CN strings whose English is *events* said 活动 (activity). Unrelated to the rest of the PR, carried along at review request. | key | en-US | was | now | |---|---|---|---| | `home.docsCountLbl` | Events | 活动 | 事件 | | `rum.searchEvents` | Search Events | 搜索活动 | 搜索事件 | | `logs.noEvents.title` | No events found | 未找到任何活动 | 未找到任何事件 | zh-TW already had 事件 for all three, and every other locale translates them as events. `logs.noEvents` also contradicted itself — `descNoFilters` directly below the title already said 未找到任何事件. ## Not in this PR The `QueryManager` map has a `created_at` on every entry but no reaper. With this change its entries are removed on cancellation, so it is no longer known to grow — but it has no backstop the way the ledger now does. |
||
|
|
b09b56e25a |
feat(llm): add Phase 3.0 experiment workflows (#13778)
## Summary - allow dataset rows without an expected output across management APIs and UI flows - add immutable experiment preview, creation, listing, persistence, RBAC, and super-cluster synchronization - add Experiment as an evaluation target with slot hydration, scoring, and result indexing - harden idempotent creation, scorer-version pinning, score-index setup, and upgrade migrations ## Tickets - Related to [openobserve/o2-enterprise#2398](https://github.com/openobserve/o2-enterprise/issues/2398) - Related to [openobserve/o2-enterprise#2399](https://github.com/openobserve/o2-enterprise/issues/2399) - Related to [openobserve/o2-enterprise#2400](https://github.com/openobserve/o2-enterprise/issues/2400) ## Paired Enterprise PR - [openobserve/o2-enterprise#2413](https://github.com/openobserve/o2-enterprise/pull/2413) ## Validation - focused Rust and frontend tests for the affected dataset and experiment seams - experiment migration test passes - upgraded local environment applies schema version 68 and returns 200 from GET /api/team3/experiments --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Hengfei Yang
|
429e5148fe |
fix(search): tie wal search file locks to guards so cancelled queries release them (#13953)
Fixes o2-enterprise#2491. OSS-only — see "Enterprise side" below.
## Problem
Local wal parquet files that had already been merged and uploaded to
object
storage stayed on disk forever, because their `SEARCHING_FILES` refcount
never
returned to zero. The post-merge cleanup in
`jobs/src/job/files/parquet.rs`
checks `wal::lock_files_exists(file_key)`, defers the deletion when a
search
holds the file, and re-checks every `ZO_FILE_PUSH_INTERVAL` — forever,
if the
lock is never released. Queries stayed correct (`pending_delete::filter`
excludes those files from the search set); the symptom was disk.
The proximate cause is a missing decrement. The root cause is that
**release
depended on control flow reaching a line, while the only thing that can
cancel
the request guarantees no further line runs**.
### The unowned window
`SEARCHING_FILES` and `SEARCHING_REQUESTS` are global maps. Nothing in
the type
system points at an entry, so nothing releases one automatically:
```
lock_files() search_service/src/grpc/wal.rs, in get_file_list
| match_source, parquet footer reads, calculate_files_size, table build
lock_request() search_service/src/grpc/wal.rs
| pack segments, union table, plan rewrite, execute_stream_partitioned
FlightEncoderStreamBuilder::build() api/grpc .../flight/mod.rs <-- first owner with a Drop
```
`lock_request()` is not a handoff to an owner — `SEARCHING_REQUESTS` is
removed
from only by `clear_session_data`, itself reached by control flow. So
the whole
of plan generation was unprotected, not just the gap between the two
calls.
### Why that window is exactly the window cancellation lives in
Both gRPC cancellation paths can only fire *before* `do_get` returns a
response:
- `hyper-1.9.0/src/proto/h2/server.rs` polls `poll_reset` only on the
`Poll::Pending` branch — "check if the client has sent a RST_STREAM
frame
which would cancel the current request" — and drops the service future.
- `tonic-0.14.6/src/transport/service/grpc_timeout.rs` reads the
client's
`grpc-timeout` header and returns `Poll::Ready(Err(TimeoutExpired))`,
dropping
the inner future. Every flight request sets that header
(`make_flight_client` → `req.set_timeout(grpc_timeout)`).
Once the response stream exists, `FlightEncoderStream::drop` handles
teardown
and neither path applies. So the set of cancellable moments and the set
of
unowned moments coincided.
Routine triggers, all of which drop in-flight `do_get` calls to
ingesters:
`query_task.abort()` on leader timeout or user cancel
(`search_service/src/cluster/flight.rs`), `SearchStreamGuard::drop`
cancelling
on client disconnect, and the ingester `grpc-timeout` itself. None of
them are
visible: `remote_scan_exec.rs` treats `Cancelled` / `DeadlineExceeded`
from a
node as an empty partial result, so the query still succeeds.
## Approach
Give the locks an owner for every instant they exist.
- **`wal::SearchingFiles`** owns the raw locks from acquisition until
`into_request` hands them to `SEARCHING_REQUESTS`; `Drop` releases
whatever is
left. `get_file_list` returns it alongside the file list, so the type
forces
every early return to account for it.
- **`flight::SessionGuard`** is armed on the first line of `do_get` —
before
anything can be locked — owns `clear_session_data` for that trace id,
and
moves into `FlightEncoderStream`, whose `Drop` already performed it. It
is a
required argument of `FlightEncoderStreamBuilder::new`, not an optional
setter, so a future call site cannot forget it.
`lock_files`, `release_files` and `lock_request` are module-private now:
the
guard is the only way to take a wal lock. The eight hand-written
`release_files` call sites are gone, and `create_tables_from_files`
loses the
`on_error: F` generic that existed solely to run one of them —
`storage.rs` was
already passing `|| {}`.
Cancellation at any `.await` in the chain now releases:
| cancelled during | released by |
|---|---|
| `get_file_list`, metadata reads, table build | `SearchingFiles::drop`
|
| after `into_request`, before the stream exists | `SessionGuard::drop`
|
| streaming | `FlightEncoderStream::drop` → `SessionGuard::drop` |
### Others on the same chain
- `release_request` no longer early-returns on non-ingesters — the last
step of
an ownership chain should not be conditional — and logs only a release
that
actually happened, so `lock_request` / `release_request` now pair
one-to-one
per trace id and can be reconciled from logs.
- `lock_request` merges instead of overwriting; a second handoff for one
trace
id used to silently drop the first file list.
- `FlightEncoderStream::drop` releases session data unconditionally.
Letting
"am I a super-cluster follower leader?" decide whether wal locks are
released
was the same bug in miniature; the deferred lock only covers the
work-group
slot.
### Observability
Two gauges, both O(1) reads, published by the existing 60s ingester
collector
`update_parquet_metrics` (no stream label):
- `ingest_wal_searching_files` — size of the lock table
- `ingest_wal_pending_delete_files` — uploaded files still awaiting
deletion
A pending-delete backlog that rises and never drains is the leak,
plotted
directly. Previously the only signal was an *absent* `debug!` line.
## Enterprise side
**No paired PR — the fix is entirely in this repo.** The issue suspected
the
super-cluster follower-leader path, but `super_cluster/follower.rs` only
builds
a `RemoteScanExec` and never touches wal locks; `wal::lock_files` has
exactly
one call site, in `grpc/wal.rs`; and o2-enterprise contains no reference
to
`lock_files`, `release_files`, `lock_request` or `release_request`.
Verified by
a full enterprise build after making those three private.
## Verification
- `cargo fmt --all`, and the CI clippy command
(`--workspace --all-targets -W too_many_lines -W cognitive_complexity
-W excessive_nesting -D warnings`): clean
- Enterprise verified via the `Cargo.toml.openobserve` swap,
`cargo check --all-targets`: clean. Clippy on that build reports two
pre-existing `cognitive_complexity` errors in
`core/src/alerts/scheduler/handlers.rs`
and `core/src/ofga/mod.rs`, untouched by this change
- Unit tests: `common` wal 10/10 (4 new, covering drop-release,
duplicate
collapse, `release_one`, and the `into_request` handoff), `api-grpc`
flight
21/21, `search` file_list 4/4
## Not in this PR
- **A timestamped reaper for stale locks.** Redundant once the chain is
unbroken, but it is the lowest-risk first step when backporting to
`branch-v0.40.0` and `branch-v0.80.0`, which carry the same defect.
- **`admission::ledger` and the `SEARCH_SERVER` task status.** Both are
trace-id keyed and both release only by reaching a call, so a
cancellation in
the same window leaks a work-group slot the same way. Same bug class,
different resource, enterprise-gated — worth its own issue.
- **`clean_lock_files()`** (`/node/flush`, `/node/enable`) wipes the
whole
refcount table, including locks held by running queries. It was
survivable
while locks leaked; now it turns a leak into deleting a file a query is
reading. Retiring it is a product call.
|
||
  
|
91e524ce2d |
feat(dashboards): interactive table cells with in-place Logs Explorer drawer (#13675)
Design At: designs/dashboards/interactive-table-cell-drilldown
What changed
New component: DashboardLogDrawer.vue
Clicking a cell in a dashboard table panel now opens a side drawer
scoped to that field/value filter. The drawer shows filtered log events
with a SQL editor (Ctrl+Enter to run, Reset to restore default),
server-side pagination with configurable page size (50/100/200/500), and
an event detail sub-drawer on row click.
Event detail has three tabs:
Insights — Field Anomaly Profile (rarity scoring vs current page), Event
Timeline (SVG histogram of event frequency across the selected time
range), Log Patterns (grouped by structure via /patterns/extract, each
with "Open in Logs" that prefills the log explorer), and Surrounding
Events (events from the same stream in a configurable ±1–10 min window,
paginated 20/page with per-row expand)
Details — key/value field list with copy buttons
JSON — raw event JSON
Modified: PanelSchemaRenderer.vue
Fixed type mismatch: route.query.cell_panel returns string; panel schema
IDs may be number — comparison now uses String() on both sides
Added cell_event_ts to the URL param cleanup list so it's cleared when
the outer log drawer closes
Modified: en-US.json — added complete panel.logExplorer key tree
including the new insights subsection
Modified: design-debt-baseline.json — updated baseline after audit (new
file's rawVarInComponent uses are all in sanctioned color-mix()
positions)
URL persistence
All drawer state is encoded in URL params (cell_panel, cell_field,
cell_value, cell_stream, cell_stype, cell_t0, cell_t1, cell_event_ts).
Sharing the URL reopens both drawers and restores the exact event — on
mount it does a WHERE _timestamp = {ts} targeted fetch (size 1) rather
than searching through paginated results.
Zero backend changes
Uses only existing endpoints: search (SQL), search (histogram via
histogram() function), and patterns/extract.
---------
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
||
|
Hengfei Yang
|
ccd254ad00 |
refactor(infra): split ORM read/write clients, retire the sqlite write lock (#13927)
Paired with o2-enterprise `refactor/orm-rw-ro-client` (same branch
name).
## Problem
Every ORM caller went through a single client backed by the sqlite
**write** pool, so writes were kept apart only by `table::get_lock()` —
a convention 365 call sites had to remember. Forgetting it surfaced as
`(code: 517) database is locked`: a deferred transaction that reads and
then upgrades to a write finds the snapshot moved under it, and sqlite
reports that **without ever consulting `busy_timeout`**, which is why
the failures were immediate rather than retried.
## Approach
Serialization now comes from the pool, not from a lock. The sqlite write
pool holds **one connection** — all sqlite allows anyway — so writers
queue on acquire and there is nothing to forget.
That is only affordable because reads move to the read-only pool.
Measured, same load, fresh data dir each run:
| | reads on the write connection | reads on the RO pool |
|---|---|---|
| 200 concurrent reads | p50 **135 ms** | p50 **1 ms** |
| writes during mixed load | p50 **189 ms** | p50 **14 ms** |
Both were 100% success with zero lock errors, so this is purely
throughput — but without the split, reads queue behind the single writer
and drag writes down with them.
## API
`infra::db` now exposes three functions and nothing else:
```rust
pub async fn get_orm_client_ro() -> &'static DatabaseConnection
pub async fn get_orm_client_rw() -> &'static DatabaseConnection
pub async fn get_orm_client_ddl() -> &'static DatabaseConnection
```
The statics behind them, every `connect_to_orm_*`, and the raw
sqlite/postgres pools are now private or `pub(crate)`. `ORM_CLIENT`,
`connect_to_orm` and `table::get_lock()` are gone, as is the raw
`CLIENT_RW` mutex (56 lock acquisitions, plus the 52 guard derefs that
existed only to reach through it).
Final split: **RO 247 / RW 360 / DDL 9**; `get_lock` 0, `ORM_CLIENT` 0,
`CLIENT_RW.lock()` 0.
## How read/write was decided
Per call site, then verified by resolving each call through **its own
file's imports** (bare-name matching is useless here — `get_by_id`,
`list` and `update` collide across modules) and propagating write-ness
along the edges that hand a connection to a callee.
A wrong write client only costs throughput; a wrong read client fails in
production. So the audit is zero-miss in the read direction, and a site
is only downgraded when *every* connection-passing callee resolves.
## Behaviour changes on sqlite
All three make sqlite match what postgres already does today — postgres
has never had this lock:
- check-then-write pairs like `folders::put` are no longer atomic; they
rely on their unique index (`folders_org_folder_id_idx`)
- `db::put` runs two sequential transactions, so concurrent puts to one
key are last-write-wins
- `db::delete` runs several statements outside a transaction
`write_pool_is_single_connection_on_sqlite` is now the **only** guard
against reintroducing 517 — please don't drop it.
## Review guide
The mutex removal is separable if you want to look at it on its own:
`src/infra/src/{db,file_list,scheduler,pipeline,schema/history}/sqlite.rs`.
Everything else is the client split.
## Verification
- `cargo check --workspace --all-targets` and the CI clippy command:
clean
- Tests match a HEAD baseline built with the same `.env`, exactly: infra
1410/2, core 1629/1, db 431/0, api-management 179/0. The three failures
are pre-existing and environmental (two need a postgres that isn't
running locally)
- Static scans, all zero: queries on the pool inside an open
transaction; transactions spanning a second connection acquisition;
ro/rw misclassification
- Runtime smoke on a fresh data dir: 200 concurrent reads, then 100
reads + 100 writes simultaneously — 400/400 HTTP 200, rows persisted,
**0** lock errors, 0 ERROR/WARN lines
- Enterprise verified via the `Cargo.toml.openobserve` swap (`cargo
check --all-targets`): clean
Also fixes `users::tests::set_up`, which truncated shared tables but
released its lock before the test body ran, letting a concurrent test
clear rows mid-test.
https://claude.ai/code/session_016ouf1RxiMpiRw5W7GvCUDh
|
||
 
|
abc0fdc42d |
feat: model pricing time-based conditonal pricing (#13903)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Hengfei Yang
|
022492fbde |
fix: file list timestamp maybe zero (#13936)
improved on #13925 - [x] Improve db check for timestamp zero - [x] Improve ingestion for timestamp zero |
||
|
Ashish Kolhe
|
b2639c4f9f |
feat(dbm): Database Monitoring (#13918)
## What this is Database Monitoring: an aggregation and presentation layer over telemetry OpenObserve already ingests. It reads three vantages — client spans, server-vantage logs, and collector metrics — and turns them into per-statement, per-instance and per-database views. No new storage engine and no new ingest path. Statements are normalized and rolled up into a stats stream, and the read API queries that stream through the existing search path. ## Shape of the change | | | |---|---| | Read endpoints | 17 | | UI pages | 8 (plus a shell) | | Commits | 141 | **Backend** — `src/core/src/db_monitoring/` holds the non-HTTP half (normalization, rollup, server-vantage parsing). The read API lives in `src/api/management/src/request/db_monitoring/` as a handler / models / service trio; the service layer is split one file per feature. **Frontend** — `web/src/views/DatabaseMonitoring/`, with shared logic in `web/src/composables/dbm/`. ## Feature availability Databases, queries, query detail and history, endpoints, samples, activity, insights, plans, server metrics, server queries, server samples, instances, instance metrics, badges — is available in OSS. ## Reviewing this It is a large diff, and most of it is additive: 327 files, and the two largest non-test files are the shared service module and the deadlocks service. Suggested order: 1. `src/api/management/src/request/db_monitoring/handler.rs` — the 17 entry points and their gating 2. `src/api/management/src/request/db_monitoring/service/common.rs` — auth, SQL building, the shared read path 3. `src/core/src/db_monitoring/rollup.rs` — how the stats stream is produced 4. `web/src/views/DatabaseMonitoring/` — the pages Time semantics are the subtlest part. Rollup rows are stamped at the window **end**, so reads are exclusive at both edges; `stats_read_range` carries the reasoning and a test pins it. ## Known follow-ups - **The server-vantage stream named as `_o2_dbm_server`.** ## Testing - Rust: unit tests across the DBM core and read service, including SQL-building, auth-gating and normalizer cases - Web: unit tests across the DBM pages and composables - Playwright: DBM suite covering the pages and cross-page navigation Verified before opening: OSS and enterprise builds both compile warning-free, `cargo fmt` clean, `vue-tsc` clean, and the full web unit suite passes (44,704 tests). |
||
 
|
5f034c6666 |
feat: folder icons (#13895)
Design at: https://github.com/openobserve/designs/blob/main/ux/folder-icons/design.md --------- Co-authored-by: Kirtan Thakkar <kirtan@kiara.tech> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com> |
||
|
Ashish Kolhe
|
029fbbd3d5 |
fix: stop serving deployment internals on the unauthenticated /config (#13921)
## Problem
`GET /config` is unauthenticated by design
## Fix
Split the endpoint:
- **`GET /config` (unauthenticated)** now returns only the login-page
bootstrap: `sso_enabled`, `native_login_enabled`, the four custom-logo
branding fields, the `rum` init block, `telemetry_enabled`,
`build_type`, and `commit_hash` (kept so the UI's stale-build detection
keeps working; version, license, billing orgs, storage region, etc. no
longer leak). An **exact-key-set test** pins this surface so a new field
cannot land on the unauthenticated endpoint unreviewed.
- **`GET /api/{org_id}/config` (authenticated)** serves the full
configuration, minus `instance`, which is dropped entirely. It goes
through `auth_middleware` like every service route and is classified as
a querier route, matching where `/config` was served before.
- The web app fetches the bootstrap at boot and the full config from
MainLayout once an org is known (with a watcher for fresh sessions), and
the logo-settings / Usage-tab refresh paths use the authenticated
endpoint.
## Paired PR
`o2-enterprise` branch `fix/config-api-exposure` adds the
`ROUTE_PERMISSIONS` entry (`bypass: true`, like the marketplace routes)
so any authenticated org member can fetch `GET /{org}/config` past
OpenFGA.
## Verification
- Router tests: bootstrap exact-key-set, `/config/reload` still 401,
`/{org_id}/config` dispatch + no-`instance` (3/3)
- `cargo test` for config / api-http / api-management crates: green
- Targeted web unit tests (config service, buildVersionChecker, General
settings): green
- Enterprise build verified via the `Cargo.toml.openobserve` swap:
`cargo check --all-targets` clean
- `o2_openfga` route-resolution tests for the new entry: green
- `cargo fmt --check` clean; CI clippy (`--workspace --all-targets -- -D
warnings`) clean
|
||
 Huaijin
|
37e5f55684 |
feat(metrics): .midx series index for TSID-major files and PromQL pruning through it (#13899)
The query half of the TSID-major layout: #13882 wrote `tsid-major-v3-*` files ordered by `(__hash__, _timestamp)`; this PR adds the `.midx` series index next to them and makes PromQL prune through it. Supersedes the remaining scope of #13839. ## Write side - Every `tsid-major-v3-*` file written by the hour-end merge gets a `.midx` series index (Arrow IPC + zstd), stored like the Tantivy index under a derived root distinct from the data tree: `files/{org}/metrics/{stream}/{date}/{hour}/tsid-major-v3-{id}.parquet` → `files/{org}/midx/{stream}/{date}/{hour}/tsid-major-v3-{id}.midx` (Tantivy: `files/{org}/index/{stream}_{type}/…/{id}.ttv`): **one row per contiguous TSID run** — its `row_count: u32` plus the label columns of the run's first row (source types preserved). Runs tile the data file, so each run's starting row is the prefix sum of the preceding counts and is not stored; `__hash__` is excluded (matchers never reference it). The whole index is one ZSTD-compressed IPC batch. Runs crossing an input RecordBatch boundary appear as two adjacent entries; query-time range coalescing makes that representation equivalent without buffering samples. File and row-group boundaries stay independent of TSID boundaries. - `TsidSeriesIndexWriter` lives inside `search/datafusion/merge/tsid_major.rs` and collects run entries alongside the Parquet writer (one buffered row per run), encoding and compressing once at finish. `MergedFile` carries `series_index: Option<Vec<u8>>`; the compactor uploads it to the `midx/` tree (compliance mode included). - `.midx` is intentionally **not** tracked in `file_list`; its path is derived from the data file's key (`MetricsFileLayout::series_index_path`) and it is deleted together with the parent in the delayed-deletion pass (`deleted.rs`); like the `index/` tree, the `midx/` day directories are covered by retention's local-disk deletion and the gc tool. ## Query side (PromQL) - `promql_service::…::tsid_series_index::search` runs where Tantivy used to, before the metrics table is registered: for each TSID-major file it loads the `.midx` (concurrent, `2×target_partitions ≤ 64`), projects the row-range columns plus the matched labels **by name against each series index's own schema** (label sets / column orders differ between files), builds the matcher filter **per series index** with `physical_expr::create_physical_expr` — the exact same predicates as `apply_matchers` (`matcher_predicates`) — evaluates it over the run rows, coalesces adjacent/overlapping ranges and attaches them as `FileSelection::RowRanges` (converted into a Parquet access plan by the generic helper from #13861, no per-row bitmap). - **Applied per file**: files of any other layout in the same query (still-open hours, legacy data, late arrivals) are left in place and scanned in full. A matched label missing from a series index over-selects that file; a query whose matchers cannot be evaluated at all selects the whole file; any series index failure logs and falls back to a full scan. The final PromQL matchers are always applied, so results are exact in every case. - Evaluated selections are cached (LRU, 256 MiB) keyed by account + series index path + the full matcher set. - The Tantivy path in the PromQL storage scan is removed (`convert_matchers_to_index_condition`, index-field plumbing, `is_add_filter_back` bookkeeping). SQL search is untouched. ## Benchmark (from the #13839 POC, same layout + series index) 478,144,896 rows, 7 CPUs, one-hour PromQL range, step 15s, result cache off, release + mimalloc: | Layout | Filter steady median | No-filter steady median | Data files | Total storage | |---|---:|---:|---:|---:| | main, 2 GiB | 695.709 ms | 4.638 s | 88 | 4,925.81 MiB | | TSID-major + `.midx` | 269.496 ms | 2.793 s | 80 | 1,294.61 MiB | ## Validation - `cargo fmt --check`; `cargo check` / `clippy` on config, search, compaction, promql-service, jobs, search_service (`--tests`) clean - `config tsid_layout` 3 passed (incl. `series_index_path`); `search tsid_major mode:: merge::` 10 passed (rotation test now also asserts the series index, exact-run and coalescing series index tests); `compaction merge deleted` 33 passed; `promql-service` 20 passed + 1 pre-existing wall-clock flake (`cache::test_promql_cache_set_and_get`, fails intermittently on main too) - End-to-end on the #13839 branch (same code paths): hour-end merge writes N `tsid-major-v3-*` + `.midx`; PromQL takes the series index path (`selected 39 ranges across 4/4 files`), counts/sums match the synthetic data, cold-cache identical after restart; a mixed hour (3 major + 1 late `tsid-sorted`) reports `1 files without a series index left to tantivy/full scan` with unchanged results; delayed deletion removes `.parquet` **and** `.midx` |
||
|
|
0f4b606e8e |
feat(synthetics): super-cluster support, then move synthetics to OSS (#13795)
> ### ⚠️ This PR carries four changes — review alongside openobserve/o2-enterprise#2420 > > 1. **Super-cluster support (consumer half)** — `8ceb041c43 … 5885c57b3a`. > 2. **Synthetics OSS migration** — `3a380087a3 … 2596300dbf`, nine commits — one per build-order step, plus a ninth for fixes found by running the binary. Moves synthetics out of `o2_enterprise` into `src/synthetics` (`openobserve-synthetics`), leaving only the private-VPC-agent path enterprise: `job_api::lease`, `agent/*`, private locations and the agent-setup install flow. Paired with openobserve/o2-enterprise#2420. > > 3. **Last-check columns read from the results stream** — `6af411e808`, `e7a3f2c759`. Outside the region whose scheduler runs a check, the list showed a stale status and a blank Last check / response time: `last_check_status` replicates only on change, `last_triggered_at` is a runtime column that does not replicate, and `last_response_ms` was never populated anywhere. The `synthetics_results` stream already carries all three and is federated, so the list API reads it instead of replicating the columns — no queue traffic, no schema change. Best-effort: any miss keeps the stored DB values, so search being unavailable cannot take the list down. OSS-only; no enterprise counterpart. > > 4. **Check secrets are readable in every region** — `d47a6b2345` (+ o2-enterprise `da71d7ec`). The per-org DEK is minted by each region for itself, so a check created outside the scheduler region replicated fine and then silently never ran — `resolve` could not decrypt its credentials (o2-enterprise#2451). Secrets now cross the queue **in clear** and the applier re-encrypts them under its own DEK on arrival, fixing the class at the replication boundary with no key to distribute or keep in sync. > > ⚠️ **This puts customer secrets in plaintext on the inter-region bus** — a deliberate, recorded trade-off (`o2oi_` ingest tokens and probe tokens already travel it in clear). Securing the bus itself (TLS on the gateway links, short retention on secret-carrying subjects) is the real fix and is **not** in this PR. > > ⚠️ **Deploy consumers before producers.** The payload's bytes did not change, only the meaning of the secret strings, so an un-upgraded consumer handed the new payload would write customer passwords to disk in clear. The plaintext-carrying messages therefore use new variant names (old ones retired as `CreateEncrypted`/`UpdateEncrypted`), so an old region fails serde and redelivers until upgraded instead of silently persisting plaintext. > > An OSS build can now create, schedule, dispatch to Lambda, take `resolve`/`ack` and report a check. 7 393 lines moved into the new crate; 679 lines stay enterprise. > > **🛑 Deployment prerequisite (2 of 2).** `ZO_SYNTHETICS_ENCRYPTION_KEY` must be set to the **same** base64 64-byte value in every region of a super cluster. Without it synthetics falls back to the per-org DEK and checks carrying secrets silently fail to run outside the scheduler region. It is read once at startup and pinned — changing it later requires a restart, and changing it at all orphans every secret already encrypted under the old value. > > **🛑 Deployment prerequisite (1 of 2).** 14 `O2_SYNTHETICS_*` env vars are hard-renamed to `ZO_SYNTHETICS_*` with **no fallback read**, and `ZO_SYNTHETICS_ENABLED` defaults to `false`. A deployment still setting the old names does not lose tuning — **synthetics stops running entirely, and the first symptom is checks silently not firing.** Update env vars *before* rolling the binary. > > #### Things a reviewer should look at first > > - **`super_cluster_queue` does not depend on `openobserve-synthetics`** (`cargo tree` → 0), which is what keeps the applier's loop-prevention guarantee a property of the crate graph rather than a convention. `a7d5fdd969` adds the assertion that fires if that ever changes. > - **`job_upload` was skipping authorization in OSS builds** (`4c3a0b8f0d`). Unreachable before, because no synthetics route was registered in OSS; registering it would have made it an unauthenticated artifact upload. `authorize_probe` is ungated and the bypass is gone. > - **The dependency edge had to reverse in a single commit** (`3247d4093f`). Cargo rejects an `o2_enterprise` ↔ OSS package cycle at the manifest level — `optional` and feature gating do not help — so the two edges could never coexist, and everything needed for the reversal lands together. This is why the spec's steps 4 and 5 are one commit and the config moved earlier than planned. > > #### Accepted OSS limitations, all documented in the commits > > - No notifications: `notify_check_result` ends in `alerts::alert::dispatch_notification`, which is enterprise. An OSS build runs the check and records the run; it has nowhere to send an alert, exactly as for alerts themselves. > - No per-resource RBAC on synthetics, and no OpenFGA ownership tuples — both are enterprise capabilities across the platform. > - Private locations are refused with a message rather than created. > - Agent-served public `net-*` pools are unavailable, so an OSS user needs a Lambda for every public pool (spec §1). --- Consumer half of synthetics super-cluster support. **Depends on openobserve/o2-enterprise#2420** (producer, message types, queue struct) — and **this side must deploy first**: `MessageType` bytes 83–85 hit a panicking `unreachable!()` in `From<u8>` on a consumer that does not know them, which takes down the subscriber task rather than failing one message. Based on `fix/lease-claimed-by-2270` (#13785), not `main`, so the diff is only this work. ## What this fixes Synthetics writes are not querier routes, so a create lands on ingesters in whichever region received the request and commits to that region's Postgres. There is no synthetics topic on the super-cluster queue, so a check created in a region that runs no scheduler is never seen by the region that does: the UI shows it enabled, the scheduler has never heard of it, and nothing reports the gap. ## Commits | commit | what | | --- | --- | | `8ceb041c43` | orphan-detection query + metrics | | `1c3e10da48` | apply replicated check config | | `79e6abc052` | apply replicated locations and probe tokens | | `16d4fa0fde` | render unknown live status instead of a false pending | | `5885c57b3a` | share the job-cluster predicate | ## Design notes worth reviewing **Config columns only, and the list is not restated.** `synthetics_checks::update` rebuilds its `ActiveModel` from the row already in this region, and `update_mutable_fields` writes only user-editable columns — so `next_run_at` and the alerting counters keep local values for free. Hand-listing them in the consumer would duplicate a list that already exists and let the two drift. A test asserts every runtime column comes back `Unchanged`. **`put()` is deliberately unused.** Its insert branch builds from `build_active_model`, which ends in `..Default::default()` and never assigns `owner`, `tz_offset` or `next_run_at` — a replicated create through it would drop the owner and fire a scheduled-later check immediately. **`create()` gains `use_given_id`**, matching `table::alerts::create`. It generated its own uuid, which would have made every region invent a different primary key for the same check. **Consumers apply through `infra::table` only.** The publish lives one layer up in the enterprise service, and `infra` cannot reach the enterprise crate — that layering is what stops a region re-broadcasting what it just applied. `source_cluster` does not cover this: it only stops a region re-consuming its *own* message. Each consumer greps its own source to keep this true. **Runtime tables are never replicated** — `synthetics_jobs`, `synthetics_runs`, `synthetics_agents`, and the locations `down_notified_at` claim. Asserted at both producer and consumer. ## Known gaps, named not hidden - If two regions mint a token (or location) with the same name for one org before either message crosses, the UNIQUE constraint rejects the arriving row. Local row wins, logged loudly. Erroring would redeliver to dead-letter, which is worse. - Pre-existing and untouched: `write_triggers_stream`/`write_results_stream` in the enterprise reaper post to `_meta` with the *org's* ingest token, which 401s, and neither checks response status. Orphan detection fixes this in its own path only. - Pre-existing: `create` never sets `tz_offset`, so a new check gets DB default 0 until its first edit. Replicates identically everywhere, so it causes no divergence. ## Spec corrections The spec (`docs/synthetics-lcl/2026-08-11-p1-3-super-cluster-spec.md`, local notes) is wrong in five places, all found by reading the tree: §1's invariant evidence (see the enterprise PR), §10's never-run guard, §4's column list (and it lists six runtime columns — there are seven, `alerting` is missing), §3's "private rows read-only when remote", and §7's claim that non-scheduler regions render OFFLINE (they render `pending`). ## Verification infra 42, super_cluster_queue 32, jobs 21, web 111/111 vitest + type-check + design lint, `cargo fmt --check` and clippy clean, full `--features enterprise` build. Pre-existing failures on this branch, unrelated and untouched: `super_cluster_queue::schemas::tests::test_parse_key_{valid,metrics}`, `infra::table::ratelimit::tests::test_add_batch_missing_rule_id`, and 8 web vitest failures in `buildPayload.spec.ts`/`payloadContract.spec.ts`. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
udev
|
84b666ce0b |
fix: annotation_queues item hydration intermittent 500s (#13881)
## Summary
RCA (2026-08-19, 08:20-08:22 UTC): `GET
/api/{org}/annotation_queues/{queue_id}/items/{item_id}` (and its
`/reviews` sibling) intermittently returned 500. Originally attributed
to search work-group admission capacity exhaustion; tracing the actual
code found a different root cause.
**Real mechanism**: `eval_jobs::search::execute()`'s registered
executors (in `src/jobs/src/job/mod.rs`) called
`search_service::search("", ...)` with a hardcoded empty `trace_id`.
With no propagated OTel span in that detached task,
`search_service::search()` falls back to OpenTelemetry's *invalid* trace
ID for every such call - the same `000...000` seen in every failing log
line. Every concurrent internal search therefore shared one
`QueryManager` map key: one query's cleanup (`SEARCH_SERVER.remove`)
deleted another's still in-flight registration, so the next
`insert_sender` failed with "trace_id not found" -> `SearchCancelQuery`
-> 500. `work_group_checking` never reached the real concurrency-wait
branch in this failure mode - "work-group capacity" was never the actual
mechanism.
## Changes (3 commits)
1. **`fix(annotation-queues): merge item detail and reviews into one
call`** - the Queue Workbench always fetches item detail + reviews
together (confirmed: no caller ever needs one without the other,
including after review submission); fold them into one backend call,
cutting the per-item load from 2 HTTP round trips to 1. See companion PR
openobserve/o2-enterprise#2473 for the response-shape change this
depends on.
2. **`fix(annotation-queues): map search admission cancellation to
429`** - `annotation_queue_error_response` unconditionally 500'd on
`AnnotationQueueError::Search`/`Hydration`, even when the underlying
cause already has a defined 429 `http_status()` (`SearchCancelQuery` /
`RatelimitExceeded`). Downcast and map correctly instead.
3. **`fix(search): stop colliding on trace_id in internal eval
searches`** - the actual root-cause fix: generate a real per-call
`trace_id` via `config::ider::generate_trace_id()` at all three internal
`search_service::search("", ...)` call sites instead of passing `""`.
## Dependency
This PR's enterprise build depends on openobserve/o2-enterprise#2473
(same branch name `ud/fix-annotation-queue-hydration-500s`) via the path
dependency in `Cargo.toml.openobserve` - that PR adds the `reviews`
field on `AnnotationQueueItemDetail` this PR's merge (commit 1)
consumes.
## Test plan
- [x] `cargo build --features enterprise`
- [x] `cargo test -p openobserve-api-management --lib --features
enterprise annotation_queue` - 11 passed (incl. 2 new tests for the 429
mapping), no regressions
- [x] `cargo test -p o2_enterprise --lib annotation_queues` (companion
repo) - 16 passed, no regressions
- [x] `cargo clippy --features enterprise -p openobserve-jobs -p
search_service -p openobserve-api-management --no-deps` - clean (no
warnings in touched files)
- [x] `cargo fmt --check` - clean
- [x] Frontend: `npx vitest run` on `llm-queues.service.spec.ts` +
`QueueWorkbenchPage.backTarget.spec.ts` - 9 passed
- [x] `vue-tsc --noEmit` (`type-check:app`) - clean
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
|
71ac1071be |
fix: draft workflows (#13666)
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yashodhan@openobserve.ai> |
||
|
|
69bd367d9b |
feat: notification dependency view — where-used for templates, destinations & alerts (#13820)
## What Adds a **"View dependencies"** affordance to the **Templates**, **Notification Destinations**, and **Alerts** lists. A small `graph-1` icon on each row opens a compact popover that shows everything that entity connects to along the **Template → Destination → Alert** chain, with search, pagination, and per-row **Open** / **Delete**. ## Why The linkage between alert templates, destinations, and alerts is invisible today. There is **no reverse-lookup API** — the only way to learn "which alerts use this destination" is to try deleting it and hit the backend's `409 "Destination is currently used by alert: X"`. That's a dead end, not a map. This surfaces the dependency proactively so you can see the blast-radius *before* you change or delete something, and act on it in place (open or delete a related entity, with the same `409` guard still protecting in-use items). ## Design The three entities are linked **by name** (there are no ID foreign keys): - `Alert.destinations: string[]` → `Destination.name` - `Destination.template` → `Template.name` (a destination's default) - `Alert.template` → `Template.name` (an alert-level override) ```mermaid flowchart LR T["Template"] -->|"Destination.template"| D["Destination"] D -->|"Alert.destinations[]"| A["Alert"] T -. "Alert.template (override)" .-> A ``` Because the alerts **list** DTO previously omitted `destinations`, the whole graph would have needed N per-alert detail fetches. Instead a small **opt-in additive field** on the list item makes the entire graph **three list calls**, cross-referenced client-side and cached per-org: ```mermaid flowchart TD Row["List row · graph-1 icon"] --> Pop["DependencyChainPopover<br/>(OPopover + ConfirmDialog)"] Pop --> Panel["DependencyUsagePanel<br/>search · collapsible sections · pager"] Panel --> Rows["DependencyUsageRow ×N<br/>Open · Delete"] Panel --> Load["useDependencyGraph.loadGraph()<br/>per-org cache"] Load -->|"alerts?include_dependencies=true<br/>+ destinations + templates (3 list calls)"| Build["buildGraph() → nodes + edges"] Panel --> Focus["buildFocusChain(graph, focus)<br/>directional: a shared template never<br/>drags in sibling destinations"] Rows -. "requestDelete(node)" .-> Pop Rows -. "open(node) — same-kind, in-place" .-> Pop Pop -. "ConfirmDialog → service.delete()<br/>(409 'used by' → toast)" .-> Done["emit deleted → list refreshes"] ``` Why the ConfirmDialog lives in the popover **wrapper**, not the panel: a modal can't render inside the popover without dismissing it, so the row/panel only *request* a delete and the wrapper owns the real confirmation. **Open** works the same way for the in-place editors: templates/destinations edit on their own list route (which only opens from the page's mount-time handler), so a same-page open is bubbled to the host page to invoke its native editor rather than a no-op `router.push` to the same route. ## UX / behaviour - **Where-used, grouped:** collapsible **Templates · Destinations · Alerts** sections, each a clickable row (kind-coloured icon, name, and for destinations an "N alerts" **total-usage** badge). - **Search-first:** one search box filters all groups — the key affordance when a destination feeds thousands of alerts. - **Paged:** the alerts group renders **50 at a time** with a pager pinned beside its (sticky) header, so 4,000-alert destinations stay responsive. - **Act in place:** every row has **Open** (templates/destinations → their editor; alerts → the alert's detail page) and **Delete** (the same `ConfirmDialog` the list pages use; the backend `409` still blocks in-use items). - Consistent trigger on all three lists; **purely additive** — no existing button, icon, or column was moved or replaced. ## Backend change `ListAlertsResponseBodyItem` (`src/api/management/src/models/alerts/responses.rs`) gains two additive fields — the linkage the dependency view cross-references: - `destinations: Vec<String>` - `template: Option<String>` (the alert-level override) They serialize **only when the caller passes `?include_dependencies=true`** (which just the dependency popover does); the default paginated alerts-list path is unchanged, so ordinary consumers pay no extra bytes and receive no module-scoped destination/template names. Both are also `skip_serializing_if`-empty. Backward-compatible, OSS, no new endpoint. ## Testing - **Unit (vitest) — 17 tests:** - `useDependencyGraph.spec.ts` — `buildGraph` (linkage, unused/orphan, dangling refs, override edges, shared destinations) and directional `buildFocusChain` per focus kind (incl. the alert-focus total-usage count). - `DependencyUsagePanel.spec.ts` — sections render every kind, search + 50/page paging, collapse, `requestDelete`, the same-kind `open` bubble, and the `include_dependencies` opt-in. - `DependencyChainPopover.spec.ts` — trigger + lazy panel mount, `requestDelete` → ConfirmDialog → API + `deleted`, and the `open` bubble + close. - **Backend:** serialization test asserts `destinations`/`template` are omitted when empty. - `type-check`, `lint:design:strict`, and `prettier` all clean. - A popover e2e (Playwright) is a follow-up in a separate PR — the earlier spec targeted an interim dedicated page that this PR does not ship, so it was removed. ## Notes - Net change is popover-only: an earlier iteration added a dedicated Reliability page + route + rail entry; those were dropped in favour of the on-the-fly popover, so `router.ts` / `navGroups.ts` are untouched vs `main`. - Icon `graph-1` was added to the icon registry (additive) for the popover trigger. - **Deferred (see design doc Non-goals):** a server-side where-used / `?destinations=` endpoint so the popover fetches only the neighbourhood instead of the full alert list; and enterprise RBAC name-filtering on the gated fields. Design at: https://github.com/openobserve/designs/blob/main/alerts/notification-dependencies/design.md (designs#8: `alerts/notification-dependencies/design.md`) --------- Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> |
||
|
Hengfei Yang
|
9d76505882 |
feat(traces): resolve session time ranges through the trace time index (#13837)
Design at: [https://github.com/openobserve/designs/blob/main/infrastructure/file-list/session-time-index.md](https://github.com/openobserve/designs/blob/main/infrastructure/file-list/session-time-index.md) ## Summary - carry an optional `session_id` attribute on trace time index records, resolved from span attributes by the existing `SESSION_ID_COLUMNS` precedence; the first non-empty value in the request wins for the whole trace, stamped onto all of its hour-bucket records - keep the aggregation key `(hour, trace_id)` unchanged; hourly compaction picks the attribute up with `MAX(session_id)`, which ignores NULLs - reuse the locate/expand timer for sessions with a 7-day expand window (traces keep 24h); a UUID v7 session id doubles as a probe anchor and tightens the scan's left bound to the session's lifetime neighborhood - aggregate session lookups per trace, so a lookup resolves the session's range plus every member trace's real range - reshape the time-range endpoint to query parameters so one endpoint serves both lookup kinds - make the session-details caller range optional, union it with the indexed range (never narrowed), take trace ids from the index, and keep the legacy source-stream scan for pre-coverage data - add `session_id` to the default bloom filter fields (gated by the existing `ZO_FEATURE_DEFAULT_INDEX_FIELDS_ENABLED`, folds into the secondary-index field set like `trace_id`); streams without the column build nothing ## Why Session detail lookup depends on a caller-provided time range: the sessions list passes its own picker window into the details request, so an LLM session that started before the window silently loses its earlier turns, and a deep link without a range is a hard 400. The trace time index already turns a `trace_id` into a time range at bounded cost; a session is the same problem one level up — `session_id` → range plus member traces. ## Environment variables No new variables. The existing `ZO_TRACE_TIME_INDEX_ENABLED` (default `true`) gates the feature, session column included. The existing coverage marker is reused; there is no historical backfill. This change must ship in the same release as #13827: reusing the coverage marker is valid only while no release separates the two. ## API changes All timestamp parameters and returned ranges use microseconds. ### Time-range endpoint reshaped ```http GET /api/{org_id}/{stream_name}/traces/time_range?trace_id=xx&hint_ts=... GET /api/{org_id}/{stream_name}/traces/time_range?session_id=xx&hint_ts=... ``` Exactly one of `trace_id` / `session_id` is required; zero or both is `400`. This replaces `GET /{stream_name}/traces/{trace_id}/time_range`; the old path form is removed without a compatibility alias since #13827 has not shipped in a release. ### Session details: range becomes optional `GET /api/{org_id}/{stream_name}/traces/session/details` now accepts `start_time` / `end_time` as an optional pair plus an optional `hint_ts` (defaulting to the caller range's midpoint). The effective range is the union of the caller range and the indexed session range, so it never narrows what the caller asked for; `400` only when neither side yields a range. When the caller range lies entirely inside index coverage the source-stream trace-id scan is skipped; otherwise it still runs over the caller range and its results are merged, so behavior is never worse than before. No frontend changes are required — the frontend keeps sending its list window and the union completes the session. ## Validation - `cargo test -p openobserve-core traces::time_index --lib` — 13 passed - `cargo test -p search datafusion::merge --lib` — 5 passed, including NULL-mixed session aggregation and pre-column file compatibility - `cargo test -p openobserve-api-search --lib` — 149 passed - `cargo test -p openobserve-api-http --lib` — 53 passed, including OpenAPI/MCP initialization - `cargo test -p config router:: --lib` — 8 passed - `cargo test -p infra db::trace_time_index --lib` — passed - `cargo clippy` on all touched crates (`--all-targets`) and `cargo check --workspace` — clean - the enterprise repository has no references to any changed API and no enterprise-gated code was modified |
||
|
udev
|
35860aff9d |
fix: route annotation queue score lookups to queriers (#13801)
## Summary - route Discovery and annotation queue endpoints to queriers so score lookups, item hydration, and review reads do not fall through to ingesters - make Discovery source_stream required at the API response boundary - cover Discovery, queue item detail, and review routing with regression tests The QueueItem remains reference-only. Workbench and enqueue validation derive source_stream from timestamp-bounded _llm_scores queries in the enterprise implementation. Depends on https://github.com/openobserve/o2-enterprise/pull/2423 ## Testing - 5 querier routing tests - Discovery response serialization test with the enterprise feature - 16 enterprise annotation queue tests - 14 enterprise Discovery tests |
||
|
Hengfei Yang
|
41ff3cd9b6 |
feat: replace distinct_values_* streams with tantivy TopK for the _values API (#13834)
Design at: #13822 ## Summary `distinct_values_*` metadata streams existed to serve one reader: the `_values` API. They are a redundant — and lossier — materialization of what the tantivy secondary index already holds, and the index has the collectors needed to replace them outright (`SimpleTopN` / `SimpleDistinct`). This PR removes the distinct streams from the write and read paths and lets the `_values` API be served by the tantivy index optimizer. ### 1. Treat distinct value fields as secondary index fields (read-time union) - `get_stream_setting_index_fields()` — the single accessor used by both the index build side (ingester parquet/pack jobs, compactor merge) and the query side (leader/follower index optimizers) — now unions `distinct_value_fields` into the effective index field set, gated on `enable_distinct_fields`. Fields that cannot be secondary-indexed (full text search keys, partition keys, reserved columns) are skipped; the values API falls back to a normal scan for those. - The distinct list is deliberately **not merged into `settings.index_fields`**: the two stay independent, so removing a distinct value field (or disabling `enable_distinct_fields`) cleanly stops its indexing with no residue, and the user-managed `index_fields` list is never mutated behind their back. - Correctness gate: a distinct-only field is considered indexed since its `added_ts` (`get_stream_setting_index_updated_at_for_fields`), so parquet files older than that are served by DataFusion scan instead of the index (`split_file_list_by_time_range`). `added_ts` is set on add and reset when `enable_distinct_fields` is re-enabled (existing behavior). For streams configured before this release the fields only start being indexed at upgrade time, so `added_ts` is additionally clamped by a release-wide cutoff `/tantivy/distinct_fields/updated_at` — a lazy marker initialized to the first access after upgrade, following the existing `ttv_secondary_index_updated_at` pattern. No data migration and no `DB_SCHEMA_VERSION` bump needed. ### 2. `_values` API always queries the original stream - Removed `can_use_distinct_stream()` and the `distinct_values_*` / `SUM(count)` / `StreamType::Metadata` rewrite from `values_inner` (GET `/_values`, formerly `values_v1`) and `build_search_request_per_field` (POST `/_values_stream`). - The generated SQL shapes are exactly what the index optimizer detects: - `SELECT f AS zo_sql_key, COUNT(*) AS zo_sql_num FROM s GROUP BY zo_sql_key ORDER BY zo_sql_num DESC, zo_sql_key ASC` + limit → `SimpleTopN` - `no_count`: `SELECT f AS zo_sql_key FROM s GROUP BY zo_sql_key ORDER BY zo_sql_key ASC` + limit → `SimpleDistinct` - The `keyword` filter (`str_match_ignore_case`) is a valid index condition, so keyword-filtered values queries stay on the index path too. ### 3. Remove distinct values generation - Dropped the per-record distinct value collection in logs and traces ingestion and the whole `core::metadata` module (channel, mem-table, flush job, shutdown hook). - Removed the now-unused `DISTINCT_FIELDS` static and the `ZO_FEATURE_DISTINCT_EXTRA_FIELDS` / `ZO_DISTINCT_VALUES_INTERVAL` / `ZO_DISTINCT_VALUES_HOURLY` config knobs, including the `SUM(count)` merge rollup the last one gated — legacy `distinct_values_*` streams now compact as ordinary metadata streams. The default fields `DISTINCT_FIELDS` carried (`service_name`, `operation_name`) are already default secondary index fields via `SQL_SECONDARY_INDEX_SEARCH_FIELDS`, so traces `_values` queries on them remain index-served. - Renamed `values_v1` to `values_inner`: the `v2` variant was removed long ago (#5087) and the distinct stream switch inside `v1` is gone now too, so the version suffix was pure history. ### 4. Default fields after this change - `service_name`, `operation_name`: default secondary index fields (unchanged, via `SQL_SECONDARY_INDEX_SEARCH_FIELDS`). - `trace_id`: now also a built-in default **bloom filter** field (`BLOOM_FILTER_DEFAULT_FIELDS`, gated on `ZO_FEATURE_DEFAULT_INDEX_FIELDS_ENABLED`), in addition to being a default index field — bloom sits on top of the secondary index and prunes files for high-cardinality equality lookups. - There is no "default distinct fields" concept anymore. ### 5. Fix surfaced by e2e: SimpleDistinct returned null values as `""` The tantivy index encodes null/missing field values as empty strings (`index_builder` adds `""` for null docs), so the `SimpleDistinct` term-dictionary walk returned `""` as a distinct value — sorted first in ascending order, it surfaced as a blank first option in dashboard variable dropdowns, which the DataFusion `GROUP BY` path never returns (its null group sorts last). This was exposed once distinct fields became index-served; `handle_simple_distinct` now skips empty terms (with a unit test). ### 6. Drive-by: fix the failing `api_fuzz_tests` CI job `CreateAlertRequestSchema` was an untagged documentation union with two identical variants, producing `oneOf: [CreateAlertRequestBody, CreateAlertRequestBody]` in the OpenAPI spec. Under `oneOf` semantics (exactly one branch must match) no instance can ever validate, so schemathesis filtered out every generated example and `api_fuzz_tests` failed on `POST /api/v2/{org_id}/alerts` on every CI run since that schema landed. The handler deserializes `CreateAlertRequestBody` directly, so the annotation now documents exactly that type; runtime behavior is unchanged. ## What stays (intentionally) - `distinct_value_fields` / `enable_distinct_fields` in stream settings: still the user-facing way to declare values-API fields; the list now drives the read-time index-field union described above. - The `distinct_value_fields` DB table that tracks dashboard/report usage of these fields (still gates removal in the settings API). - `DISTINCT_STREAM_PREFIX` helpers, retention sync and stream-deletion cleanup for existing `distinct_values_*` streams — legacy stream data keeps compacting (as ordinary metadata streams) and ages out via retention; no new records are written to them. ## Testing - `cargo check --workspace --all-targets` and `cargo clippy --workspace --all-targets` clean. - Enterprise build verified with `Cargo.toml.openobserve` (`cargo check --all-targets`). - New unit tests for the index-field union (skip rules for FTS/partition/reserved fields, `enable_distinct_fields` gating) and the `added_ts` / release-cutoff indexed-since resolution; schema, infra, config and api-search unit tests pass. A few pre-existing failures on a clean tree (config `ssrf_guard`, one flaky sqlite-backed destinations test, one `.env`-dependent index-fields test) are local-environment issues and unrelated. Closes #13822 |
||
|
Harsh Mahajan
|
7f74925605 | feat: banner as notification (#13667) | ||
|
Prabhat Sharmaandgithub-actions[bot]
|
0dba67d638 |
feat: composite alerts (#13797)
Design at : alerts/composite_alerts/composite_alerts.md ## Summary - add composite alert graph persistence, evaluation, scheduling, lifecycle APIs, and deletion safeguards - add composite alert UI forms, preview, references, list/detail integration, and localization - add Rust, API, Vue, and Playwright contracts ## Verification - cargo fmt --all -- --check - cargo check -p openobserve-core - cargo check -p openobserve-api-management - npm run type-check:composite-alerts - npm run type-check:app - npm run lint:ci -- --quiet - design/token/style checks - targeted Rust/Vitest suites - browser QA against local alerts UI ## Follow-ups (intentionally deferred, documented in the spec) - `RunOutcome::Error`/`NotifyFailed` outcome records on composite delivery failure (§11.5) - Incident detail live-definition resolver for composites + "definition unavailable" retention (§9.6) - `alert_composite_*` observability metrics (§16) - SLO/stream delete-conflict 409 bodies with readable-vs-hidden parent split (§10.3) - Super-cluster startup preflight (§18) <img width="1650" height="979" alt="image" src="https://github.com/user-attachments/assets/3def5246-7bf7-48fe-9a42-c0ac08271ea9" /> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Hengfei Yang
|
3fda083f51 |
feat(traces): add trace time index (#13827)
Design at: [https://github.com/openobserve/designs/blob/main/infrastructure/file-list/trace-time-index.md](https://github.com/openobserve/designs/blob/main/infrastructure/file-list/trace-time-index.md) ## Summary - add an hourly Metadata time index per trace stream to resolve a trace ID to its full time range - add external time-range and trace-details APIs, including bounded locate/expand behavior and standard search responses - integrate index generation into trace ingestion and preserve query visibility through the synchronous Metadata write path - synchronize retention and deletion for associated Metadata streams - reuse the shared trace stream permission check across trace APIs ## Why Trace detail lookup currently depends on a caller-provided time range, which can omit spans outside that window. The time index provides a bounded, best-effort range lookup for `stream + trace_id` without scanning the full trace stream. Closes #13823 ## Environment variables | Variable | Default | Effect | | --- | --- | --- | | `ZO_TRACE_TIME_INDEX_ENABLED` | `true` | Enables per-trace-stream Metadata indexes, index writes during trace ingestion, and indexed time-range lookup. Only traces ingested after the feature is enabled are indexed; there is no historical backfill. | | `ZO_QUERY_TIMEOUT` | 600 seconds | Existing setting reused as the cumulative Trace Timer locate/expand deadline and as the default timeout for the trace-details search. | When enabled, each traces stream can create a related Metadata stream named `trace_time_index_{stream}`. Its retention settings follow the source stream, and it is removed with the source stream. ## API changes All timestamp parameters and returned time ranges use microseconds. ### Get a trace time range ```http GET /api/{org_id}/{stream_name}/traces/{trace_id}/time_range ``` Query parameters: - `hint_ts` (optional): timestamp hint used to start the index lookup Returns the source stream, trace ID, and the resolved `start_time` / `end_time`. The `range` field is `null` when the index cannot resolve the trace. ### Get trace details ```http GET /api/{org_id}/{stream_name}/traces/{trace_id}/details ``` Query parameters: - `start_time` and `end_time` (optional, but must be supplied together): caller-provided range - `hint_ts` (optional): timestamp hint for the time-index lookup - `timeout` (optional): trace-details search timeout in seconds The endpoint unions the caller range with the indexed range, so it never narrows a caller-provided range. If neither the caller range nor the index can provide a range, it returns `400`. The response uses the standard search response and includes the effective range in `new_start_time` and `new_end_time`. ## Validation - `cargo test -p openobserve-api-search --lib` — 145 passed - `cargo test -p openobserve-core traces::time_index::tests --lib` — 7 passed - `cargo test -p infra db::trace_time_index::tests::test_coverage_start_cache -- --exact --nocapture` — passed - `cargo check -p openobserve-api-search` - `cd web && npm run type-check:app` - end-to-end ingestion and query validation with the data generator ## Validation notes - targeted ESLint could not start because the local web dependencies are missing `@eslint/css` - enterprise-only compilation could not be completed from this OSS checkout because the local enterprise overlay is unavailable |
||
|
|
dbee572722 |
fix: slo bugs (#13784)
closes https://github.com/openobserve/o2-enterprise/issues/2397 --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Harsh Mahajan
|
0ed5336bf1 | chore: adding HA mode for o2-ai agent (#13525) | ||
|
Huaijin
|
3901c548a2 |
perf(promql): use Utf8View for metric labels (#13796)
## Summary
- read PromQL metric label columns as Arrow `Utf8View` when the existing
`ZO_UTF8_VIEW_ENABLED` setting is enabled
- preserve `__hash__` and exemplars in their original representation
while retaining schema metadata
- make PromQL equality, negative, and regex matchers type-aware for
`Utf8View`; cast regex operands to `Utf8` to avoid the DataFusion 54
mixed-type rewrite issue
- support `StringViewArray` in series-label loading and keep
compatibility with the process-wide series-label cache
No PromQL-specific environment variable is introduced, and metrics
continue to use the existing global `ZO_BATCH_SIZE`.
## Performance
For the filtered histogram query below, a controlled local A/B using the
same release binary and identical settings except Utf8View measured P50
latency improving from 0.964s to 0.677s, about 29.8%:
```promql
histogram_quantile(0.9, sum by(le, path) (rate(codelab_api_request_duration_seconds_bucket{path="/api/service-1"}[5m])))
```
Test range was one hour with `step=15` and `use_cache=false`.
## Testing
- `cargo fmt --all -- --check`
- `cargo test -p promql --lib utils::tests` (10 passed)
- `cargo test -p promql --lib series_labels::tests` (11 passed)
- `cargo test -p search --lib
datafusion::exec::tests::test_metrics_query_schema` (2 passed)
- `cargo check -p promql-service -p promql`
|
||
|
Ashish Kolhe
|
de0d37e23d |
fix(alerts): stop losing Slack alerts on undeliverable link URLs (#13742) (#13753)
Fixes #13742 Fixes openobserve/o2-enterprise#2394 ## The bug A content template link with a non-http scheme (`javascript:alert(1)`) made Slack reject the **entire** notification with `400 invalid_attachments`. The alert was lost outright and `triggers` recorded `notify_failed`. ## Root cause `safe_url` neutralises a hostile scheme by substituting `BLOCKED_URL` (`"#"`). That is **safe** but not **deliverable** — Slack, Teams, Discord and PagerDuty validate action URLs server-side and reject the whole payload when one is not an absolute `http(s)` URL. A security substitution was turning into total message loss. The two properties are different, and links need both. So the fix splits by medium: | renderer | link handling | |---|---| | Slack buttons, Teams actions, Discord `embed.url`, PagerDuty `href` | `dispatchable_url` → **drop** | | HTML email, email plaintext, SNS plaintext, Discord description | `safe_url` → substitute `#` | | Webhook envelope | unfiltered — versioned machine contract, documented | Dropping beats rewriting: the alert body is worth more than the button, and a button pointing somewhere misleading is worse than no button. `dispatchable_url` requires a **non-empty authority**, not just an allowlisted scheme — `http:javascript:alert(1)` and `https://` are valid *opaque* RFC 3986 http(s) URLs that Slack still rejects. It also rejects raw spaces and control characters, which variable substitution can introduce. ## Related issues fixed in the same area - **Empty `ZO_WEB_URL`** (not in the report): `alert_url` becomes a root-relative `/web/logs?…`, which `safe_url` passes through by design, and it lands in the always-appended "View in OpenObserve" button. Same `invalid_attachments`, on deployments that never authored a hostile template. - **PagerDuty** validates `links[].href` server-side too — same lost-alert class, converted. - **Save-time rejection** (`ContentSpec::validate`): a clear 400 naming the offending link, instead of a template whose link silently never works. Also wired into **preview** and **test-send**, which previously accepted a draft that couldn't be saved and echoed the raw hostile URL back to the UI. - **Slack image-fallback misdiagnosis**: removing the image proves the image was at fault, but only an image pointing at *our* `web_url` justifies tripping the process-wide chart suppression (1h, across every alert) and telling the operator to fix `ZO_WEB_URL`. A custom template's third-party image no longer does either. - **a11y**: `OInput` links its error message to the control via `aria-describedby`; the textarea branch also gained `aria-invalid`. ## o2-enterprise#2394 (fixed here, not in the enterprise repo) Clearing the Monaco body and saving toasted "Please fill required fields" with no indication of *where*. The schema already produced an error on the `body` path — a bare Monaco bridged in via `setFieldValue` just had no `OFormInput` error slot to render it. Now a red border + message driven by the same field state `OFormInput` reads. The aria wiring goes on the **shell**, not on `<QueryEditor>`: `CodeQueryEditor` sets `inheritAttrs: false` and binds `$attrs` to a non-focusable wrapper, so attributes passed to the component would be inert — false assurance for a screen-reader user. > The issue names the title field too. It's an `OFormInput` and already highlights correctly; I added a regression test pinning that rather than changing it. This fix is entirely in OSS (`web/`) since the template form isn't enterprise-gated, so there's no companion enterprise PR. ## Layering note Save-time validation is a **UX gate, not the security boundary**. Link URLs are templates — a scheme that only materialises after `{variable}` substitution can't be judged at save time. The render layer is what actually stops delivery, and it also covers every template saved before this validation existed. Both layers are needed; neither is sufficient. ## Testing - Every fix written test-first (verified RED) and **mutation-tested** — each new test fails when its fix is reverted in substance. - Golden files unchanged: legitimate URLs render byte-identically, so no over-blocking. - Rust: `alerts::notifications` 159 · `config` 3130 · `api-management` 424 — all passing. `cargo clippy --all-targets` and `cargo fmt --check` clean. - Frontend: full suite **41,597 passing** (run because `OInput` is shared library code). `vue-tsc`, `eslint`, and `npm run format:check` clean. - Two independent cold-start review passes; findings from both are folded in, including a P0 where the first version of `dispatchable_url` still let the original bug through. Pre-existing and unrelated: `alerts::destinations::tests::test_alert_destination_template_is_optional` fails locally on a DB connection reset — verified failing on unmodified `main` too. |
||
|
Hengfei Yang
|
238b0d5262 |
fix: support zstd-compressed HTTP requests (#13728)
## Summary - enable tower-http zstd request decompression - add a router middleware regression test using a real zstd-compressed JSON payload - update decompression documentation to list zstd support ## Root cause The Axum router uses `RequestDecompressionLayer`, but the workspace did not enable tower-http's `decompression-zstd` feature. Requests from clients such as Vector configured with `compression: "zstd"` were therefore rejected with `415 Unsupported Media Type` before reaching the ingestion handler. ## User impact Vector HTTP sinks can continue using zstd compression when sending batches to OpenObserve ingestion endpoints. ## Validation - `cargo fmt --all -- --check` - `cargo test -p openobserve-api-http handler::http::router::decompression::tests` - `git diff --check` Fixes #13726 |
||
|
udevandsai nikhil kethe
|
bbd8601c64 |
feat(llm): add annotation queues and datasets (#13719)
## Summary - add annotation queue and dataset storage, management APIs, and review workflows - add discovery, dataset, queue, and review user interfaces - add RBAC synchronization, super-cluster support, tags, version history, and review progress ## Validation - CI should pair this branch with the identically named Enterprise branch for cumulative compilation testing ## Migration note This is an additional cumulative PR created from the committed ud/ebb3 tip. The existing SPR pull requests and branches are intentionally left unchanged. --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Ashish Kolhe
|
b3e6a77270 |
fix(correlation): service-streams P2 behavioral fixes + discovered services flat table (#13710)
Correlation P2 round: behavioral/security fixes across the service-streams backend and correlation FE, plus the Discovered Services UX rework and the correlation e2e suites. **Companion enterprise PR (must merge together):** openobserve/o2-enterprise#2381 — the enterprise side calls the new 3-arg `put(org, record, max_streams_per_type)` introduced here. ## Commit 1 — behavioral fixes (F2–F38) **Backend** - F25/F26: RBAC on `list_services` (`service_streams` GET — the one previously-unenforced endpoint in the module) + validate `distinguish_by` ids on identity-config save - F2: validate trace filter param before WHERE-clause interpolation - F6: `_reset` invalidates caches on all nodes via reload event; org deletion clears service-streams caches - F8: shared coverage-based identity-set ranking for ingest and read - F10: purge rows of removed/changed identity sets on config save - F19: `put` returns deleted orphan keys for cache eviction - F28: discriminate no-match from failure; stop aliasing 404 to no-match - F32: cap DB-side stream union at `max_streams_per_type` - `all_dimensions` unions on put so late-tracked dimensions surface - F14: `StreamInfo.dropped_dimensions` contract field; allow empty `tracked_alias_ids` when identity sets exist **Frontend** - F35/F36/F31: per-stream filter-edit resolution via semantic groups; dimension edits in both key spaces; subject overrides never guess fields - F27/F30: echo `source_stream`/`source_type`; stop guessing another stream's filters; chip dedup - F2/F38: SQL field/value escaping via shared `quoteSqlLiteral` helper ## Commit 2 — Discovered Services UX + e2e - Flat table (one row per instance), same-name services merge their name cell (OTable `pivotRowColumns`); optional per-service collapse with aggregated summary row; search/filters override collapse - Fixes an OTable pivot-merge bug (value-keyed merge map blanked single-column runs) + regression test - Semantic Field Groups: category tabs with counts + cross-category search with match highlighting - Correlation Phase-1 API + Phase-2 UI Playwright suites ## ⚠️ Release note (enterprise + RBAC deployments) Viewing Discovered Services now requires the **Service Streams** permission (previously unenforced). Root/admin users are unaffected; custom roles may need a one-time grant via the role editor (resource already exists in the OFGA model). ## Checks - `cargo fmt --all --check` ✅ · `cargo clippy --all-targets` ✅ enterprise + OSS feature sets (0 warnings) - prettier ✅ · eslint 0 errors · 207 unit tests on touched specs ✅ - Compiles with paired enterprise branch ✅ |
||
|
Prabhat Sharma
|
087e1498ab |
feat(slos): add SLO burn-rate alerts (#13704)
design at /slo ## What Adds **SLO burn-rate alerts** — an alert family that fires off an SLO's error budget rather than a raw query — and finishes the SLO surface around it: navigation entry, detail-page fixes, and the flag removal now that SLOs have shipped. ## Changes **SLO burn-rate alerts (backend)** - New burn-rate evaluation (`config/src/meta/slo/burn.rs`, `condition.rs`) with multi-window burn conditions, and `core/src/slo/evaluate.rs` extended to evaluate them on the SLO job. - Alerts carry an `slo_id`; `infra/src/table/alerts` gains the column, filtering, and query support so an SLO's alerts can be listed and deleted with it. - Alert levels (`meta/alerts/level.rs`) and tags extended for the SLO family; scheduler handlers route SLO alerts through the same notification path. **SLO alert UI** - `SloAlertForm` + `AlertDetailSlo`: create, edit and view burn-rate alerts from the SLO detail page and the alert list. - Payload/routing helpers (`utils/alerts/sloAlertPayload.ts`, `sloAlertRouting.ts`, `alertPayload.ts`) with unit tests. - `AlertDestinationsField` extracted as the complete Destination form field — side label, tooltip, combined destinations/workflows picker, inline error, refresh, and create-in-new-tab. It replaces two verbatim copies in the generic alert form and is reused by the SLO form and synthetics `CheckAlerts` (which sets `supportsWorkflows: false`, since a check has no workflow routing). Fixes the SLO form's previously unwired refresh and Add Destination buttons. **SLO detail page fixes** - Burndown and burn-rate charts drew as bare axes when coverage was sparse: `showSymbol: false` means an isolated measured bucket between gaps has no segment and paints nothing. Symbols are now sized per point, and a sparse-coverage note states how many buckets actually recorded events. - Nothing scrolled below ~700px of viewport height — `OPageLayout`'s fixed flex body plus `OTabPanels` `scroll="none"` clipped the panels. Panels now grow and scroll; the stat strip and tab bar stay pinned. Verified at 1024x500, 1280x620, 1512x945. **Other** - `ZO_SLO_ENABLED` removed — SLOs have shipped, so the flag and its status/route gating are gone. - SLOs surfaced in the main nav. - Dashboard chart panel header gets its own tint via new `--color-panel-bar-bg` / `--color-panel-bar-border` tokens, and the bar is extracted into `PanelBar.vue` (it existed as five hand-copied class strings that had already drifted). - `tsconfig.vitest.tsbuildinfo` untracked; third-party vendor name removed from comments and test names. ## Testing - New Rust tests across `meta/slo/burn.rs`, `condition.rs`, `core/src/alerts/alert.rs` and `infra/src/table/alerts`. - New frontend suites: `sloAlertPayload.spec.ts`, `sloAlertRouting.spec.ts`, `alertPayload.spec.ts`, `AlertDetailSlo.spec.ts`, `SloDetailAlerts.spec.ts`, `SloListDelete.spec.ts`, `SloTimeSlicePreview.spec.ts`. - `cargo clippy --all-targets` and `web` `lint:ci` clean. |