mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 08:09:27 +00:00
e2b35f43f5b48bfaaee033d92bbdd0458aca22ac
6940
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Shrinath Rao
|
e2b35f43f5 |
ci: Auto Merge Queuer — re-add PRs dequeued by failed checks (up to 2x) (#13891)
## What
Adds `.github/workflows/merge-queue-auto-requeue.yml` — **Auto Merge
Queuer**.
When a PR is kicked **out of the merge queue because its required checks
failed** (`reason == CI_FAILURE`) or **timed out** (`reason ==
CI_TIMEOUT`), the workflow adds it straight back to the queue — **up to
twice**.
## Behavior
- **Trigger:** `pull_request: [dequeued]`, gated at the job `if` to
`CI_FAILURE` / `CI_TIMEOUT` only.
- **Manual removals** (`reason == MANUAL`) and every other reason
(`MERGE_CONFLICT`, `QUEUE_CLEARED`, `BRANCH_PROTECTIONS`, …) are
**ignored** — the job never runs for them.
- **Limit:** attempt count tracked via labels `auto-requeue-1` /
`auto-requeue-2` (max 2 re-adds).
- **Re-queue:** GraphQL `enqueuePullRequest` mutation, authenticated
with `secrets.ORG_ADMIN_TOKEN` (a `GITHUB_TOKEN` enqueue would not
re-trigger the `merge_group` check workflows).
- **One sticky comment, edited in place** (found by a hidden marker):
- 1st kick → "added back (attempt 1 of 2)"
- 2nd kick → same comment edited → "re-added a second time (2 of 2) —
last automatic retry"
- 3rd kick → same comment edited → "🛑 no-go for auto re-add" and it does
**not** re-queue.
All `gh` calls run under the PAT, so the comment, labels and enqueue are
authored by the token owner (not `github-actions[bot]`).
## Notes for reviewers
- Requires repo secret `ORG_ADMIN_TOKEN` with merge-queue write (already
used by 26 workflows in this repo).
- `github.event.reason` on the `dequeued` payload drives the gate; worth
confirming on the first real CI-caused dequeue. Safe failure mode if
absent: the job no-ops.
|
||
|
Shrinath Rao
|
503a7054fb |
feat(ci): rebrand automated PR review as "OpenObserve Code Reviewer" (#13871)
## What & why Repackages our existing automated PR reviewer under a branded identity — **OpenObserve Code Reviewer** — and gives its comment a colorful, attention-grabbing format. This is **purely aesthetic + repackaging**. > [!IMPORTANT] > The review **engine is frozen**. The 5 specialist agents, coordinator judgment/rubric, risk tiering, XML finding contract, re-review PATCH-in-place logic, total-outage guard, DeepSeek/opencode wiring, read-only security model, and OTel tracing are all **unchanged**. Only *who signs the comment* and *how it looks* change. ## The two things that move **1. Identity (fail-safe).** The workflow mints a GitHub App installation token via `actions/create-github-app-token`, **gated on the `OO_REVIEWER_APP_ID` variable**. If the app isn't configured, it skips and falls back to `github.token` (posts as `github-actions[bot]`, exactly as today). So this PR is **safe to merge before the app exists** and auto-activates once the secrets are provisioned. **2. Presentation.** The coordinator now emits: - Decision as a native GitHub **callout box** — 🟢 `[!TIP]` approved · 🔵 `[!NOTE]` approved-with-comments · 🟡 `[!WARNING]` minor · 🔴 `[!CAUTION]` significant. - Rebranded heading `## 🔎 OpenObserve Code Review`. - Per-category glyphs 🔒 ⚡ 🧩 📝 📦 on each finding line. `sanitizeReviewBody` normalizes the heading (tolerates a missing emoji / legacy heading, idempotent on re-review) and the deterministic fallback renderer gets the same cosmetic treatment. ## To activate the brand (one-time, org admin) Register a GitHub App "OpenObserve Code Reviewer", install on this repo, then set: - Variable `OO_REVIEWER_APP_ID` (numeric App ID — this is the gate) - Secret `OO_REVIEWER_APP_PRIVATE_KEY` (the app's `.pem`) Remove the variable to instantly roll back to `github-actions[bot]`. ## Verification - JS parses; both workflows valid YAML. - Engine files byte-identical with the o2-enterprise counterpart PR. - 6/6 sanitizer heading cases pass (fresh, no-emoji, legacy, preamble, idempotent, trailing-words). - This PR's own review comment will preview the new format. Paired with the identical o2-enterprise PR (same branch name). |
||
 udev
|
84b666ce0b |
fix: annotation_queues item hydration intermittent 500s (#13881)
## Summary
RCA (2026-08-19, 08:20-08:22 UTC): `GET
/api/{org}/annotation_queues/{queue_id}/items/{item_id}` (and its
`/reviews` sibling) intermittently returned 500. Originally attributed
to search work-group admission capacity exhaustion; tracing the actual
code found a different root cause.
**Real mechanism**: `eval_jobs::search::execute()`'s registered
executors (in `src/jobs/src/job/mod.rs`) called
`search_service::search("", ...)` with a hardcoded empty `trace_id`.
With no propagated OTel span in that detached task,
`search_service::search()` falls back to OpenTelemetry's *invalid* trace
ID for every such call - the same `000...000` seen in every failing log
line. Every concurrent internal search therefore shared one
`QueryManager` map key: one query's cleanup (`SEARCH_SERVER.remove`)
deleted another's still in-flight registration, so the next
`insert_sender` failed with "trace_id not found" -> `SearchCancelQuery`
-> 500. `work_group_checking` never reached the real concurrency-wait
branch in this failure mode - "work-group capacity" was never the actual
mechanism.
## Changes (3 commits)
1. **`fix(annotation-queues): merge item detail and reviews into one
call`** - the Queue Workbench always fetches item detail + reviews
together (confirmed: no caller ever needs one without the other,
including after review submission); fold them into one backend call,
cutting the per-item load from 2 HTTP round trips to 1. See companion PR
openobserve/o2-enterprise#2473 for the response-shape change this
depends on.
2. **`fix(annotation-queues): map search admission cancellation to
429`** - `annotation_queue_error_response` unconditionally 500'd on
`AnnotationQueueError::Search`/`Hydration`, even when the underlying
cause already has a defined 429 `http_status()` (`SearchCancelQuery` /
`RatelimitExceeded`). Downcast and map correctly instead.
3. **`fix(search): stop colliding on trace_id in internal eval
searches`** - the actual root-cause fix: generate a real per-call
`trace_id` via `config::ider::generate_trace_id()` at all three internal
`search_service::search("", ...)` call sites instead of passing `""`.
## Dependency
This PR's enterprise build depends on openobserve/o2-enterprise#2473
(same branch name `ud/fix-annotation-queue-hydration-500s`) via the path
dependency in `Cargo.toml.openobserve` - that PR adds the `reviews`
field on `AnnotationQueueItemDetail` this PR's merge (commit 1)
consumes.
## Test plan
- [x] `cargo build --features enterprise`
- [x] `cargo test -p openobserve-api-management --lib --features
enterprise annotation_queue` - 11 passed (incl. 2 new tests for the 429
mapping), no regressions
- [x] `cargo test -p o2_enterprise --lib annotation_queues` (companion
repo) - 16 passed, no regressions
- [x] `cargo clippy --features enterprise -p openobserve-jobs -p
search_service -p openobserve-api-management --no-deps` - clean (no
warnings in touched files)
- [x] `cargo fmt --check` - clean
- [x] Frontend: `npx vitest run` on `llm-queues.service.spec.ts` +
`QueueWorkbenchPage.backTarget.spec.ts` - 9 passed
- [x] `vue-tsc --noEmit` (`type-check:app`) - clean
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Shrinath Rao
|
6eedf5dd16 |
chore(ci): remove redundant Qodo PR-Agent workflow (#13887)
## What Removes the **Qodo PR-Agent** workflow (`qodo-ai/pr-agent@main`). ## Why - It auto-runs its code-suggestions tool on every PR via OpenAI; that call now fails, so it posts **"Failed to generate code suggestions for PR"** as noise on **every** PR (e.g. o2-enterprise#2475). - It's **superseded by the OpenObserve Code Reviewer** app, which now handles automated PR review. - Keeping it means a failing, redundant OpenAI dependency pinned to a **mutable `@main`** tag. Added Jun 2025 (OSS #7041 / ENT #577); effectively unused since the branded reviewer shipped. ## Scope Deletes only the PR-Agent workflow file. No other CI touched. Paired OSS/ENT change. |
||
|
Shrinath Rao
|
30ef3ec242 |
test(traces): heal service-graph/catalog rail-flyout nav for #13852 ?tab= reroute (#13875)
## What Heals the two Traces-Regression tests that broke after #13852 (\`fix(ui): clean up SLO detail and traces navigation\`). Failing scheduled runs: - ENT: https://github.com/openobserve/o2-enterprise/actions/runs/32213206401/job/95953026991 — \`traces-ent.spec.js\` Service Graph test - OSS: https://github.com/openobserve/openobserve/actions/runs/32213278081 — \`traces-regression.spec.js\` Service Catalog test Both failed all 4 retries with \`locator.click: Timeout 45000ms exceeded\` on the left-rail flyout item. ## Root cause #13852 rerouted the Traces rail-flyout children through the Traces page's in-page \`?tab=\` views. That changed **both**: | | before | after | |---|---|---| | flyout \`data-test\` | \`nav-group-item-serviceGraph\` | \`nav-group-item-traces-service-graph\` | | | \`nav-group-item-servicesCatalog\` | \`nav-group-item-traces-services-catalog\` | | URL | \`/traces/service-graph\` | \`/traces?tab=service-graph\` | | | \`/traces/services\` | \`/traces?tab=services-catalog\` | (\`childDataTest = nav-group-item-\${child.name}-\${child.tab}\`; legacy paths now only exist as redirects.) The stale \`data-test\` made \`.click()\` wait the full 45s action timeout. ## Fix \`serviceGraphPage.navigateToServiceGraph()\` and \`servicesCatalogPage.clickServiceCatalogTab()\`: - point at the new flyout \`data-test\` ids - \`waitFor\` the teleported flyout item to be **visible** before clicking — the flyout opens after ONavGroup's 120ms \`OPEN_DELAY\`, so the item isn't in the DOM the instant \`hover()\` resolves - match the new \`/traces?tab=\` URL Engine/other nav helpers untouched. ## Verification Run locally against a #13852 frontend (localhost:5080, enterprise build): - ✅ Live flyout click → \`/web/traces?org_identifier=default&tab=service-graph\` and \`...&tab=services-catalog\` - ✅ **OSS** \`traces-regression.spec.js\` "Clearing search filter should not blank out Service Catalog table" — **passes** - ✅ **ENT** \`traces-ent.spec.js\` Service Graph nav now works — test reaches the chart assertion (line 54); the only residual local failure is absent service-graph topology (0 nodes locally; CI seeds it via ingestion + the daemon) ## Follow-up (not in this PR / not in any CI matrix) \`Traces/traces-toolbar-tabs.spec.js\` + \`tracesPage.switchToServiceMaps()\` still assert the pre-#13852 "flyout → standalone route" dual-semantics that #13852 removed. Not run by any CI matrix, so left for a separate change aligned with the new single-\`?tab=\` semantics. |
||
 Hengfei Yang
|
c572727863 |
fix: wait for WAL settle window before merging the previous hour (#13880)
## What The compactor's hour-merge gate (`is_past_hour`) now measures the 3 × `ZO_MAX_FILE_RETENTION_TIME` settle window from the **end** of the offset hour instead of its start. ## Why The original design delayed merging the previous hour until the current hour's top plus 3 × `ZO_MAX_FILE_RETENTION_TIME` (default 30 min), giving data still buffered in ingester WAL when the hour closed time to upload. Since the multi-worker compactor refactor (#3526), the gate compared `now - offset` (offset = start of the hour to merge) against the window — a condition that is always already satisfied the moment the hour passes (1 h > 30 min), so the check was dead code and the merge fired right at the top of the hour. Files uploaded afterwards for that hour arrived as small files that the normal path never re-merged; the old-data fallback only picks an hour up once it has ≥ 10 small files, so a handful of stragglers stayed unmerged forever. ## How - `is_past_hour` rounds `offset` up to its hour's end and requires `now - hour_end > 3 * max_file_retention_time`. One fix covers all three call sites: scheduled job generation, the incremental-mode decision in `merge_by_stream` (jobs inside the settle window now correctly run incrementally: seal full-size groups only, carry the remainder), and the file_list dump gate. - Catch-up of older hours is unaffected — their hour end is long past the window, so they pass immediately (the behavior #3526 wanted to preserve). ## Reviewer notes The behavior change operators will see: the previous hour's final merge now starts ~30 minutes (3 × retention) after the hour closes instead of immediately, matching the pre-#3526 design. Streams with the ingester incremental trigger enabled keep getting intra-hour merges during that window. |
||
|
Shrinath Rao
|
be67ed442a |
fix(deps): add missing @noble/hashes@2.3.0 to web lockfile (npm 10 npm ci) (#13872)
## Problem
Every frontend build is red on `npm ci` in `web/`:
```
npm error `npm ci` can only install packages when your package.json and
package-lock.json ... are in sync.
npm error Missing: @noble/hashes@2.3.0 from lock file
```
## Root cause (not a feature change)
- `web/package-lock.json` was last regenerated with **npm 11**, whose
resolver **omits** the explicit `@noble/hashes@2.3.0` entry that
`jsdom`'s transitive `@exodus/bytes@1.15.1` **peer-requires** (`^1.8.0
|| ^2.0.0`).
- The **Playwright build pins `node-version: 22` → npm 10**
(`.github/workflows/playwright.yml`). npm 10's `npm ci` recomputes the
ideal tree, needs that entry, finds it missing, and hard-fails.
- Reproduced locally: `npm@10 ci` **fails**, `npm@11 ci` **passes**, on
the exact committed lock. So it's a **toolchain mismatch**, independent
of any PR — it breaks `main` and every open PR building the frontend.
## Fix
Regenerated the lock with `npm@10 install --package-lock-only` → adds
the nested `node_modules/jsdom/node_modules/@noble/hashes@2.3.0` entry.
**+15 lines, lockfile only**, `lockfileVersion` unchanged (3).
Verified `npm ci` now passes under **both npm 10 and npm 11**.
## Note
The durable fix is aligning the Playwright build to **Node 24 / npm 11**
(matching the AI-review workflow and the Node 20/22 runner deprecation).
This PR is the minimal unblock; the Node bump can follow separately.
|
||
  Yashodhan JoshiandYashodhan Joshi
|
3f40d13f21 |
fix: always create file_list_dump_stats table (#13873)
Currently observed on v0.92.x , when file list dump is not enabled we do not create the dump stats table, but for some reason compactor tries to use the table and fails. Apart from this, even in the case when someone has not initially setup the dump enabled, and later enables it, the table will be missing as migration will not be run if db schema version matches. We should always create all the table schemas, as plain schema and indices will not take much db resources, but would be problematic to later debug and fix if missing. Thus this pr always created the file_list_dump_stats table irrespective of the env value. Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> |
||
 Vamsi Karthik R
|
7b79fdcdbe |
ci(playwright): stop SLO-Measurement backfill from serializing 21 daily chunks (#13870)
## What this fixes The `SLO-Measurement` shard (`slo-timeslice.spec.js` + `slo-alerts.spec.js`) has been taking **~26 minutes** ## Root cause Both specs seed **8 days of history** and create SLOs with a **7-day rolling window**, then block in `beforeAll` until the SLO is actually measured (`waitForSloMeasured`). The server backfills history in `ZO_SLO_BACKFILL_CHUNK_SECS` chunks — default **86400s (1 day)** — and only **one chunk at a time** (`ZO_SCHEDULER_SLO_BACKFILL_CONCURRENCY=1`, by design, so bulk backfill never starves real-time alerts). The shard backfills **3 SLOs** across the two specs, each needing 7 daily chunks to cover its 7-day window: - `slo-timeslice.spec.js` `beforeAll`: 2 SLOs (`sloLt`, `sloGt`) × 7 chunks = 14 - `slo-alerts.spec.js` `beforeAll`: 1 SLO × 7 chunks = 7 **21 chunks total, strictly serial**, at roughly 40s/chunk ≈ **~14 minutes of pure backfill wait**, before any UI assertion runs. This is the server behaving correctly and exactly as designed for production — the test is just paying for it the slow way. ## Fix Neither spec asserts on per-chunk timing or partial coverage, only on the final measured state, so raising the chunk size to cover the whole window in one pass is free wall-clock, not a behavior change. - Threaded `slo_backfill_chunk_secs` through `build-ci-matrix.js`'s existing per-shard passthrough, same pattern already used for `ingest_allowed_upto` / `quick_mode_enabled`. - Set it to `604800` (7 days) for the `SLO-Measurement` shard only — the server-wide default (`86400`) is untouched everywhere else. - Documented the new knob in `tests/ui-testing/ci-matrix/README.md`. ## Validation **Synthetic (API-only), same seed shape as the real specs** (8 days, 1-min interval, deterministic 3h fault window), against a local build: | | Default (86400s chunk) | Fixed (604800s chunk) | |---|---|---| | Time to measured | 246.9s (~4.1 min) for **one** SLO, stepping 1/7 → 2/7 → ... every ~40s | **3.0s** | | Final SLI | 98.21428571428571% | 98.21428571428571% (**identical**) | Same math, same result — only the number of scheduler round-trips changes. **Real shard, end-to-end**, run exactly as CI invokes it (`npx playwright test slo-timeslice.spec.js slo-alerts.spec.js --workers=1`) against a server mirroring this shard's full env including the fix: - **25/25 tests passed**, 0 regressions - Total wall-clock (server start → global teardown): **4m Avg** Local timing won't map 1:1 onto the GitHub Actions runner (different disk/scheduler characteristics — even the "before" baseline was faster locally than the ~26min/~5min-per-spec seen in CI), so the authoritative number is whatever this PR's own CI run reports. But the structural win — 1 chunk instead of 7 per SLO — is verified correct and regression-free independent of machine. ## Companion PR The enterprise repo's `playwright.yml` independently consumes per-shard matrix fields (it explicitly mirrors OSS's `ingest_allowed_upto` handling for the same reason), so it needs the equivalent env passthrough or it'll keep paying the old cost even after this merges. Companion PR: openobserve/o2-enterprise#2460 |
||
|
Shrinath Rao
|
87f7960d48 |
test(traces): accept redirected ?tab=service-graph URL in Service Graph smoke (#13868)
## What The `@enterprise` Service Graph smoke test `P0: Navigate to service graph and verify chart renders` (`tests/ui-testing/playwright-tests/Traces/service-graph.spec.js:128`) hard-asserts: ```js await expect(page).toHaveURL(/\/traces\/service-graph/); ``` ## Why [#13852](https://github.com/openobserve/openobserve/pull/13852) converts the standalone `/traces/service-graph` route into a **redirect** to the canonical `/traces?tab=service-graph` tab. The chart still renders — every other Service Graph spec passes — but this literal URL assertion no longer matches, so the **Playwright E2E Crosscheck (Enterprise gate)** fails on #13852 on every attempt. ## Fix Accept both the legacy route and the `?tab=service-graph` form: ```js await expect(page).toHaveURL(/\/traces(\/service-graph|\?.*\btab=service-graph\b)/); ``` Backward-compatible and test-only: - Against **main** (standalone route) → `…/traces/service-graph?…` → matches ✅ - Against **#13852** (redirect) → `…/traces?…&tab=service-graph` → matches ✅ - Rejects plain `/traces` and `tab=services-catalog` ✅ (verified) ## Ordering note This assertion lives in the OSS repo (`@enterprise`-tagged, runs only in the ENT gate build). Once this merges to `main`, **#13852 must sync/rebase on main** for its crosscheck to pick up the tolerant assertion. |
||
  Chaitanya Sistlaandgithub-actions[bot]
|
351b356620 |
feat(alerts,slos): export as Terraform and OpenTofu (#13849)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
 Prabhat Sharmaandgithub-actions[bot]
|
ecc1d5dbff |
fix(ui): clean up SLO detail and traces navigation (#13852)
## Summary This PR combines the SLO detail cleanup and the Traces navigation fixes into one change set. ### SLO cleanup - remove the internal definition generation value from the SLO detail title - remove the unused translation key and update generated locale files - normalize SLO health icon identifiers - avoid rewriting unchanged locale files during translation updates ### Traces navigation - add Spans as the first item in the Traces flyout - route Spans, Traces, Service Graph, and Service Catalog through /traces?tab=... - redirect legacy Service Graph and Service Catalog paths while preserving query state - support same-route menu switching and retain stream, time-range, and query context - remove the deprecated search_mode parameter from generated and legacy URLs - keep Traces as the default for tabless URLs ## Validation - focused Traces Vitest suites: 436 passed, 2 skipped - app TypeScript check - full frontend ESLint check - Prettier checks - live browser verification of Traces and Spans menu switching --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
 Huaijin
|
0641e1e486 |
refactor(merge): decide the merge kind once (MergeMode) and share the DataFusion pipeline (#13866)
## Why The kind of merge — plain time-ordered file, trace time-index aggregation, file-list ordering, enterprise metrics downsampling — was re-derived at every layer, each from stream type / stream name / config / batch time range: - compactor scheduler (`merge_by_stream`): `skip_group_files` (downsampling rule lookup #1) → whole-hour batch vs size groups - `merge_files`: `is_match_downsampling_rule` (lookup #2) → early returns, size-limit bypass - `merge_parquet_files`: lookup #3 + SQL selection by stream type, with `is_ingester: bool` standing in for the writer's role `merge_parquet_files` and `merge_parquet_files_with_downsampling` also carried two copies of the DataFusion context / union table / plan / execute-stream / channel / read-task plumbing. Reading the code meant reconciling three derivations of the same fact to know what a batch would produce. ## What - **`search::datafusion::merge::mode::MergeMode { Classic, TraceTimeIndex, FileList, Downsampling(rule) }`** — decided once: `MergeMode::for_compactor(stream_type, stream_name, max_ts)` by the scheduler (carried in `MergeBatch.mode`), `MergeMode::for_ingester(stream_type, stream_name)` by the movers. Every layer only asks it questions: `merges_whole_batch()`, `input_sort_order()`, `sql(schema)`. - **`MergeOutput { file_format, parquet_compression }`** (`for_ingester` / `for_compactor`) replaces the `is_ingester` bool. - **`merge_parquet_files(schema, tables, bloom, meta, &mode, output)`**: one `run_merge_query()` pipeline for every mode, then the mode's writer. Downsampling is a `MergeMode` arm calling `downsampling::write_files` instead of an early-return second implementation (`merge_parquet_files_with_downsampling` removed, −81 lines). - Compactor: `skip_group_files` / `is_match_downsampling_rule` and their `cfg(enterprise)` twin declarations are gone; `compaction` no longer imports `get_largest_downsampling_rule`. Commit 1 (`672ab315a1`) is pure refactor, no behavior change. ## Behavior change (commit 2, `8030eefd04`): whole-hour merges only for a closed hour, over every file of the hour Metrics downsampling used to merge the whole batch whenever the rule matched the batch's newest sample — including incremental rounds on the still-open hour — and it only ever saw the files below the 95%-of-`ZO_COMPACT_MAX_FILE_SIZE` bound of `query_for_merge`: files already above the bound were never downsampled, and small outputs were re-aggregated round after round. - `MergeMode::for_compactor(stream_type, stream_name, range_end_ts, finalize)` is decided **once per hour, before the file_list query**, from the end of the job's date range; a whole-hour mode (downsampling) is only chosen when `finalize` (the hour is closed). Incremental merges of a matching hour use the plain per-stream mode and leave the aggregation to the hour-end pass. - `query_for_merge` takes the `original_size` bound from the caller (`infra::file_list::merge_max_original_size()` = the classic 95% rule); a whole-hour merge asks for **every** file of the hour (`i64::MAX`), the same input range the TSID-major hour-end merge in #13839 uses. - Rule matching uses the hour's end instead of the batch's max `max_ts` (≤ 1h difference; the hour is decided as a unit). ## Guards (commit 3, cherry-picked from #13839) - compactor: fail the merge instead of deleting the sources when `merge_parquet_files` produced no files - search storage `file_list::get_scan_selection` strips the multi-account `{account}/::/` prefix so selections registered by `set` are found again (+ round-trip test) - `deleted.rs`: `delete_from_storage` takes the file slice and a key closure (`Cow`) instead of a pre-collected `Vec` ## Not in this PR The write stage still dispatches to four writers (`write_parquet`, `write_vortex`, `write_downsampled_parquet`, `write_downsampled_vortex`) that share the open / write / rotate / close skeleton and differ only in the rotation policy and how `FileMeta` is derived. Unifying them into one rotating writer is a follow-up (needs a decision on deriving `min_ts/max_ts` from the output batches for the classic path too). ## Validation - `cargo fmt --check`; `cargo clippy -p search -p compaction -p openobserve-jobs --tests` clean - `cargo test -p search --lib -- merge::` 6 passed (new: `mode_by_stream`, `sql_by_mode`, `output_file_format_uses_parquet_for_ingester_metrics`; trace-time-index merge test now goes through `MergeMode::for_ingester` and asserts `TraceTimeIndex`) - `cargo test -p compaction --lib -- merge` 30 passed; `cargo test -p infra --lib -- file_list` 49 passed - `downsampling.rs` is enterprise-only and not compilable locally; `write_files` / `pub(super) generate_downsampling_sql` were edited by hand and need the enterprise CI. |
||
|
Hengfei Yang
|
7dd19d78ad |
fix: reduce eval match preview window to one hour (#13859)
## Summary - reduce the new evaluation job matched-target preview from a rolling 24-hour window to one hour - update span, trace, and session preview labels to show `last 1h` - add deterministic coverage for the one-hour query range ## Why Selecting a stream immediately runs a matched-target count query. The hard-coded 24-hour window can scan a large amount of trace data on high-volume streams even though this number is only a creation-time preview. ## Impact This only changes the matched-target preview query and its label. It does not change the evaluation job schedule or execution range. ## Root cause The preview composable and its labels were hard-coded to 24 hours. ## Validation - `npm run test:unit -- --run enterprise/components/onlineEvals/composables/useJobMatchedTargets.spec.ts enterprise/components/onlineEvals/forms/job/JobPreviewPanel.spec.ts` - `npm run type-check:app` - focused ESLint and Prettier checks - `npm run lint:design:strict` ## Companion PR - openobserve/o2-enterprise#2452 fixes the OpenAI token-limit parameter and raises the LLM judge defaults. |
||
|
Hengfei Yang
|
4be23092a7 |
fix: skip session_id in trace time index merge when files lack the column (#13863)
## Problem Ingester merge jobs fail repeatedly for trace time index streams: ``` merge_parquet_files error for stream: default/metadata/trace_time_index__evaluator, ... err: Schema error: No field named session_id. Valid fields are tbl._timestamp, tbl.max_ts, tbl.min_ts, tbl.trace_id. ``` The failing file can never merge, so the job retries it forever and the file stays stuck in the WAL, never uploaded to object storage. ## Root cause - The trace time index write path only inserts a `session_id` value when the trace actually carries one; the field is nullable in the schema. - When the ingester persists WAL files, `merge_record_batches` prunes columns that are entirely null — so a file whose records all lack a session is written **without the `session_id` column at all**. Since most traces have no session id, this is the common case, not an edge case. - The ingester merge job builds the DataFusion table schema from the files' own fields (intentionally, to avoid the wide latest schema), and `merge_parquet_files` hardcoded `MAX(session_id) AS session_id` in the merge SQL for trace time index streams — planning fails when the column is absent. ## Fix Only include `MAX(session_id) AS session_id` in the merge SQL when the resolved schema actually contains the field. Files without the column merge with the original 4-column shape (matching what search already handles via schema adaptation); when any file in the batch has the column, the union schema null-fills the rest and the full query works. This also covers files written before `session_id` was added to the index schema. ## Testing Existing `test_trace_time_index_merge_aggregates_by_trace_id` covers the with-session path through the conditional SQL; verified the module's tests pass. |
||
|
Huaijin
|
4e805038fe |
refactor: merge result / access-plan / deletion helpers groundwork for TSID-major metrics (#13861)
Behavior-neutral refactors split out of #13839 so that PR only carries the TSID-major feature. Nothing here changes what is written, read or deleted; every commit is independently reviewable. ## Commits - `cba75d244b` **promql**: `apply_matchers` split into `matcher_predicates(schema, matchers) -> Vec<Expr>` + `apply_matchers` (unchanged behavior; lets other inputs evaluate the same matcher semantics). - `b5ff6fed21` **parquet**: `new_parquet_writer` / `append_metadata` generic over `W: AsyncFileWriter` instead of `&mut Vec<u8>`; existing callers unchanged. - `b07d84216a` **merge**: `MergeParquetResult::{Single, Multiple}` → `MergeParquetResult { files: Vec<MergedFile { buf, meta }>, file_format }` + `into_single()`. The compactor's two copies of the upload / index / push sequence become one loop (`compressed_size` is set from the buffer for every file, as the `Multiple` arm already did); downsampling maps its `(bufs, metas)` into it; ingester movers use `into_single()` instead of panicking/erroring on `Multiple`. - `014b6c5721` **compactor**: `deleted.rs` `derived_files()` / `delete_from_storage()` replace three copies of collect / `storage::del` / ignore-not-found (data, `.ttv`, flattened). - `7163786362` **search**: `generate_parquet_access_plan_from_ranges` extracted from the bitmap path (`set_slices` → ranges); an out-of-range, empty or unsorted range is rejected with `assert!` (`56de386522`; a producer that violates the sorted/non-overlapping contract is a bug, not something to degrade around) instead of the previous `unreachable!()` on the bitmap path. Adds `FileSelection::RowRanges(Arc<Vec<Range<usize>>>)` as the matching selection kind (no producer yet; Parquet only, Vortex full scan like `RowGroups`). - `754014ccf3` **test**: `ListingTableAdapter` plan-shape test for `FileSortOrder::HashTimestampAsc` (from #13853): three overlapping hash-sorted files → `SortPreservingMergeExec` over three single-file groups, no `SortExec`, globally ordered rows. `tempfile` as `search` dev-dependency. ## Validation - `cargo fmt --check` - `cargo clippy -p config -p promql -p search -p compaction -p openobserve-jobs -p search_service -p promql-service --tests` clean - `cargo test -p promql --lib -- utils` 10, `-p search --lib -- merge:: helpers listing_adapter` 14, `-p compaction --lib -- merge deleted` 31, `-p config --lib -- test_file_key` 4 passed - `merge/downsampling.rs` is enterprise-only and was updated by hand; needs the enterprise CI compile. |
||
 ktx-vaidehi
|
a6c910f015 |
fix(web): logs scroll jump, dashboard tab-rename width, and banner usage (#13858)
Three unrelated UI fixes. ### `fix: logs page wrap scroll` The Logs view delegates its scroll element to a container that also holds the histogram. TanStack Virtual compensates `scrollTop` whenever a measured row differs from its estimate — correct for a self-contained list, wrong for a shared scroller. With wrap on, rows were remeasured on every query refresh and the deltas pushed the whole page downward. Adds `preserveScrollOffsetOnRowResize` to `useTableVirtualization`, which pins `shouldAdjustScrollPositionOnItemSizeChange` to `() => false`. `OTable` opts in only when it has both a delegated `scrollEl` and dynamic row height, so self-contained tables keep the existing behaviour. ### `fix: dashboard tab rename size issue` The rename input's auto-sizer grid track was `minmax(0, max-content)`. Once the tab strip overflowed, the `0` minimum collapsed the track to `0px` and the input vanished. Now a bare `max-content`. ### `fix: ingestion and webinar obanner usage` Replaces two hand-rolled banner `div`s (`Ingestion.vue`, `WebinarBanner.vue`) with `OBanner`. To make that possible without appearance overrides, `OBanner` gains: - a `promo` variant (the existing marketing gold, so the standalone webinar bar still matches); - `center`, which groups message and actions in the middle of a bar instead of stranding the action at the far edge; - a `role` override, for a bar that is page furniture rather than a notification; - `dense` now actually applies in `bar` mode (`py-1` vs `py-2`). ### Tests `TabList.spec.ts` and `OTable.wrapMeasure.spec.ts` updated alongside. |
||
|
ktx-vaidehiandgithub-actions[bot]
|
c53a16b262 |
fix: script i18n (#13840)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
71ac1071be |
fix: draft workflows (#13666)
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yashodhan@openobserve.ai> |
||
|
Huaijin
|
617303a307 |
refactor(search): drop FileSortOrder from the distributed plan wire format (#13856)
Follow-up to #13853. The `(__hash__, _timestamp)` order is only relevant to the compactor merge; distributed query plans never carry it, so the wire-level `FileSortOrder` is unnecessary. ## Changes - `plan.proto` / generated: remove `enum FileSortOrder` and `NewEmptyExecNode.sort_order = 8`; `NewEmptyExecNode` keeps only the legacy `bool sorted_by_time = 6`. - `codec/empty_exec.rs`: encode/decode with the bool only (`FileSortOrder::from_sorted_by_time`); drop the enum-first/legacy-fallback logic and its test. - `sort_order.rs`: drop `to_proto` / `from_proto` and the round-trip test; `HashTimestampAsc` is documented as compactor-merge only. - `empty_exec.rs` tests: cover `None` / `TimestampDesc` output ordering only. Rolling upgrade: #13853 nodes still write and read `sorted_by_time`, nodes with this change only use `sorted_by_time` — no incompatibility. ## Validation - `cargo check -p search -p search_service --tests`, `cargo fmt --check` - `cargo test -p search --lib -- sort_order empty_exec` 26 passed |
||
 ktx-akshay
|
411be8968a |
test: fix flaky report folder move test (#13847)
## Problem `reportFolders.spec.js` failed intermittently in CI. `P2: Move button disabled when source equals destination` failed on all 3 retries with a freshly-created report each time — the signature of poisoned shared state, not timing flake. ## Root causes Four independent test-side defects, all rooted in specs sharing one org, one user, and one `default` folder under `fullyParallel: true, workers: 5`: 1. **Reports bound to a stranger's dashboard.** `createReportViaApi` ignored the dashboard the test had just created and attached to `dashboards[0]` — whichever dashboard sorted first alphabetically in the shared folder. Deleting a dashboard removes its reports, so a neighbouring spec's cleanup made the report vanish mid-test. 2. **Blanket cleanups.** `cleanupDashboards()` deletes *every* dashboard owned by the automation user — every other spec's included (16 in one observed run). `reportFolders` also deleted every report matching `test_report_`, which swept up `scheduledReportsDrawer`'s `test_report_sched_*`. 3. **Bulk select-all across the shared folder.** `reports-bulk-operations` did "select all in `default` → bulk move/delete". Under parallel load that captured other specs' reports. Confirmed on a live server: `reportFolders`' report was found sitting in `bulk_test_folder_*`, created 78ms apart. 4. **A retry that could never succeed.** `expectReportVisibleInTable` retried by re-typing the same value into the search box. The Vue model never changed, and the in-folder search is a client-side filter over already-fetched rows — it cannot pull in a report the page hasn't loaded. The poll burned its full 45s doing nothing. Separately, `clickMove()` swallowed its dialog-close wait, so a move that never landed passed silently and failed a *later* test instead. ## Changes |
||
|
Huaijin
|
516e225bde |
refactor(search): replace sorted_by_time bool with FileSortOrder enum (#13853)
## Summary Every table provider, `NewEmptyExec`, the memtable `SortExec`, the merge `ORDER BY` and the `ListingTable` `file_sort_order` shared a single `sorted_by_time: bool` that implicitly meant `_timestamp DESC`. TSID-major metrics files (see #13839) are sorted by `(__hash__ ASC, _timestamp ASC)` instead, so a bool can no longer describe the physical layout of the files behind a table. This PR introduces `search::datafusion::sort_order::FileSortOrder { None, TimestampDesc, HashTimestampAsc }` as the single place that knows how each order maps to DataFusion logical / physical sort expressions, and threads it through the whole pipeline: - `DataFusionContextBuilder::sort_order` → enables `split_file_groups_by_statistics` for any sorted order - `TableBuilder::sort_order` → `ListingOptions::with_file_sort_order(order.logical_sort_exprs())` - `ListingTableAdapter` → regroups files by statistics using the declared order (`order.physical_ordering(schema)`) instead of a hard-coded `_timestamp DESC`; the "reverse file groups" fallback is kept only for `TimestampDesc` - `NewMemTable` → `SortExec` on the declared order (no-op if a sort column is missing) - `NewEmptyTable` / `NewEmptyExec` → equivalence properties from the declared order; display shows `sort_order=timestamp_desc` - `search_service` (`flight` / `storage` / `wal` / `create_tables_from_files`) pass the enum through - merge `ORDER BY` string derived from `FileSortOrder::TimestampDesc.order_by_clause()` ### Wire format `NewEmptyExecNode` gains `FileSortOrder sort_order = 8`. The legacy `bool sorted_by_time = 6` is still **written** (`true` iff `TimestampDesc`) and **read** as a fallback when `sort_order` is `NONE`, so mixed-version clusters keep working during rolling upgrades. ### No behavior change Every previous `sorted_by_time(true)` maps to `FileSortOrder::TimestampDesc`; `Sql.sorted_by_time` stays a bool (it describes the query, not the files) and is converted at the boundary. `HashTimestampAsc` is not yet used by any caller — that lands with the TSID-major work. ## Validation - `cargo fmt --all -- --check` - `cargo check --workspace --tests` - `cargo clippy -p search -p search_service -p compaction -p openobserve-jobs --tests` clean - `cargo test -p search --lib`: all pass except 12 pre-existing `sql::histogram` failures caused by a local `.env` override (`ZO_HISTOGRAM_BREAKDOWN_FIELDS=""`), unrelated - `cargo test -p search_service --lib -- grpc::` 16 passed - New tests: `sort_order` (logical/physical exprs, ORDER BY clause, proto roundtrip, missing column), `NewEmptyExec` output ordering per variant, codec roundtrip for all variants + legacy-flag fallback decoding |
||
|
Vamsi Karthik R
|
99b1a23773 |
test(slo): E2E + API suite for SLOs (#13836)
Adds automated test coverage for **SLOs** (#13547, #13577, #13579, #13704, #13761, #13784), which shipped with none. **149 tests** — 77 Playwright + 72 pytest — all three SLI types, both query languages, happy / negative / edge paths. Clean branch off `main`. **UI (Playwright · `tests/ui-testing`)** - `pages/sloPages/` — 4 page objects (verified `data-test` selectors, hydration + Monaco-sync waits, OSelect commit-retry, virtualized-list filtering). - `playwright-tests/SLO/slo-crud.spec.js` (24) — create count/time-slice/grouped through the form, detail, edit, SLI-type change, tags, pause, delete, filters; negative: dismissed delete, empty search, cancel, unmeasured SLI is absent not zero. - `playwright-tests/SLO/slo-form-validation.spec.js` (20) — server errors surfacing in the form, conditional fields per SLI type, count + time-slice previews, **PromQL count and time-slice created end to end**; negative: no stream, invalid aggregate, over-long name, target boundaries. - `playwright-tests/SLO/slo-alert-sli.spec.js` (8) — **the `alert` SLI type**: source picker, create, config carries only the alert id, detail names the source; negative: ineligible source offered-but-disabled, grouped alert SLI, slice finer than source cadence. - `playwright-tests/SLO/slo-timeslice.spec.js` (10) — **#13761 P0 regression** (measured SLI, not the preview), PromQL, previews, gaps, language-flip clearing. - `playwright-tests/SLO/slo-alerts.spec.js` (15) — burn-rate + error-budget kinds, presets, edit, freeze door, burndown; negative: no destination, cancel, blank/invalid name. - `playwright-tests/utils/slo-seed.js` — backdated seeding, metrics seeding, eligible-source-alert fixture, ground-truth query, fail-fast env guard, transient-store retry, full fixture teardown (alerts → streams → destinations → templates). **API (pytest · `tests/api-testing`)** - `tests/slos/test_slo_validation.py` (72) — full REST surface (create/get/update/enable/delete/groups/slo-eligible), validation on **both create and update** (name, target, window, slice, config shapes, comparators, float-threshold round-trip), boundaries (target 0.001/99.999, name 256 vs 257, threshold 0/negative/1e12/NaN, unicode names), alert-source eligibility (silence-gated, too-infrequent, cron), PromQL config shapes, auth 401/404. **CI** - Two shards: `SLO` (`workers: 3`) and `SLO-Measurement` (`workers: 1`). Split because the backfill job runs with `ZO_SCHEDULER_SLO_BACKFILL_CONCURRENCY=1`, so measurement specs sharing an instance starve each other's waits — `fullyParallel` races spec *files*, which `mode:'serial'` cannot prevent. The 3-worker pin is SQLite: at 5, concurrent SLO/alert/destination writes trip `database is locked` on the single-node meta store. - New optional per-shard `ingest_allowed_upto` (`240`) alongside the existing `quick_mode_enabled`: the workflow default of 5 **hours** silently drops the 8 days of history an SLO window needs, while still answering `200`. Other shards keep the default. Same precedent as `api-testing.yml`'s `48`. **Feature code**: 9 additive `data-test`/`dataTest` attributes in `SloDetail.vue`, `SloList.vue`, `SloAlertCondition.vue` — no logic changes, `vue-tsc` clean. The stat tiles, tabs and burn-window inputs had no stable selectors; `OStatStrip`/`OTab` already forward the attribute. **Behaviour**: self-cleaning (`e2e_slo_*`, worker-scoped since `afterAll` runs per worker). **Verified in CI** on this branch: `e2e / SLO` **52 passed** (1.8m, workers pinned to 3), `e2e / SLO-Measurement` **25 passed** (11.3m, workers pinned to 1) — the measurement shard passing confirms the per-shard `ingest_allowed_upto` took effect, since those tests are impossible under the 5-hour default. pytest **72 passed** locally; in CI's 599-test run one setup delete hit a transient 500 (see findings) and now retries. **Findings for the feature owner** (reported, not fixed here): - A blank "good when" shows `Request failed with status code 422` — `pruned()` drops the empty key, so the API's real 400 message never reaches `save()`. - A rejected alert name shows no banner: `SloAlertForm.submit()` returns on `nameError` before setting `saveError`. - `avg(latency_ms); DROP TABLE users--` is accepted at create time (parsed only at measurement). - The name-length message says "less than 256 characters" but 256 is accepted — 257 is the first rejection. - Duplicate alert names on one SLO are accepted (duplicate SLO names are a 409), and the panel lists them by name, so they are indistinguishable. - SLO names accept characters that break the list's own `slos-slolist-edit-${name}` selectors; alert names correctly reject them. - All chart/preview root `data-test` ids are unreachable — a parent's fallthrough attribute overrides the child root's. - Both edit forms hydrate asynchronously, so a blank form can flash before the stored values appear. - `DELETE /slos/{id}` on a freshly-created SLO returned **500** once under CI load (not reproducible locally in 12 rapid iterations). Contention is arguably a retryable 503, and the body carried no actionable message. - **A scheduled alert whose source stream is deleted is retried forever** (`Stream "…" not found in schema`), and the SLO backfill lane shares the scheduler's worker budget — so accumulated dead alerts *starve SLO measurement*. Observed: 65 of them held coverage at 1 of 7 chunks indefinitely; deleting them restored 0.143 → 1.0 in 240s. Harmless in CI (fresh instance per shard) but it degrades any long-lived environment. The suite now cleans up after itself; the retry-forever behaviour itself may deserve a look. **Paired ENT PR: openobserve/o2-enterprise#2434.** ENT's matrix is this repo's manifest plus an overlay, merged by this repo's `build-ci-matrix.js`, so both SLO shards reach ENT automatically — verified: 31 merged shards, both SLO entries carrying `ingest_allowed_upto=240` and their worker pins, no other shard affected. What ENT was missing is *consuming* those two fields, which #2434 adds. Merge the two together: this one alone leaves ENT booting the SLO shards on the 5-hour ingest window, and #2434 alone is inert. *(An earlier review reported ENT as using an inline matrix with no SLO shard — that turned out not to be the case.)* |
||
 
|
69bd367d9b |
feat: notification dependency view — where-used for templates, destinations & alerts (#13820)
## What Adds a **"View dependencies"** affordance to the **Templates**, **Notification Destinations**, and **Alerts** lists. A small `graph-1` icon on each row opens a compact popover that shows everything that entity connects to along the **Template → Destination → Alert** chain, with search, pagination, and per-row **Open** / **Delete**. ## Why The linkage between alert templates, destinations, and alerts is invisible today. There is **no reverse-lookup API** — the only way to learn "which alerts use this destination" is to try deleting it and hit the backend's `409 "Destination is currently used by alert: X"`. That's a dead end, not a map. This surfaces the dependency proactively so you can see the blast-radius *before* you change or delete something, and act on it in place (open or delete a related entity, with the same `409` guard still protecting in-use items). ## Design The three entities are linked **by name** (there are no ID foreign keys): - `Alert.destinations: string[]` → `Destination.name` - `Destination.template` → `Template.name` (a destination's default) - `Alert.template` → `Template.name` (an alert-level override) ```mermaid flowchart LR T["Template"] -->|"Destination.template"| D["Destination"] D -->|"Alert.destinations[]"| A["Alert"] T -. "Alert.template (override)" .-> A ``` Because the alerts **list** DTO previously omitted `destinations`, the whole graph would have needed N per-alert detail fetches. Instead a small **opt-in additive field** on the list item makes the entire graph **three list calls**, cross-referenced client-side and cached per-org: ```mermaid flowchart TD Row["List row · graph-1 icon"] --> Pop["DependencyChainPopover<br/>(OPopover + ConfirmDialog)"] Pop --> Panel["DependencyUsagePanel<br/>search · collapsible sections · pager"] Panel --> Rows["DependencyUsageRow ×N<br/>Open · Delete"] Panel --> Load["useDependencyGraph.loadGraph()<br/>per-org cache"] Load -->|"alerts?include_dependencies=true<br/>+ destinations + templates (3 list calls)"| Build["buildGraph() → nodes + edges"] Panel --> Focus["buildFocusChain(graph, focus)<br/>directional: a shared template never<br/>drags in sibling destinations"] Rows -. "requestDelete(node)" .-> Pop Rows -. "open(node) — same-kind, in-place" .-> Pop Pop -. "ConfirmDialog → service.delete()<br/>(409 'used by' → toast)" .-> Done["emit deleted → list refreshes"] ``` Why the ConfirmDialog lives in the popover **wrapper**, not the panel: a modal can't render inside the popover without dismissing it, so the row/panel only *request* a delete and the wrapper owns the real confirmation. **Open** works the same way for the in-place editors: templates/destinations edit on their own list route (which only opens from the page's mount-time handler), so a same-page open is bubbled to the host page to invoke its native editor rather than a no-op `router.push` to the same route. ## UX / behaviour - **Where-used, grouped:** collapsible **Templates · Destinations · Alerts** sections, each a clickable row (kind-coloured icon, name, and for destinations an "N alerts" **total-usage** badge). - **Search-first:** one search box filters all groups — the key affordance when a destination feeds thousands of alerts. - **Paged:** the alerts group renders **50 at a time** with a pager pinned beside its (sticky) header, so 4,000-alert destinations stay responsive. - **Act in place:** every row has **Open** (templates/destinations → their editor; alerts → the alert's detail page) and **Delete** (the same `ConfirmDialog` the list pages use; the backend `409` still blocks in-use items). - Consistent trigger on all three lists; **purely additive** — no existing button, icon, or column was moved or replaced. ## Backend change `ListAlertsResponseBodyItem` (`src/api/management/src/models/alerts/responses.rs`) gains two additive fields — the linkage the dependency view cross-references: - `destinations: Vec<String>` - `template: Option<String>` (the alert-level override) They serialize **only when the caller passes `?include_dependencies=true`** (which just the dependency popover does); the default paginated alerts-list path is unchanged, so ordinary consumers pay no extra bytes and receive no module-scoped destination/template names. Both are also `skip_serializing_if`-empty. Backward-compatible, OSS, no new endpoint. ## Testing - **Unit (vitest) — 17 tests:** - `useDependencyGraph.spec.ts` — `buildGraph` (linkage, unused/orphan, dangling refs, override edges, shared destinations) and directional `buildFocusChain` per focus kind (incl. the alert-focus total-usage count). - `DependencyUsagePanel.spec.ts` — sections render every kind, search + 50/page paging, collapse, `requestDelete`, the same-kind `open` bubble, and the `include_dependencies` opt-in. - `DependencyChainPopover.spec.ts` — trigger + lazy panel mount, `requestDelete` → ConfirmDialog → API + `deleted`, and the `open` bubble + close. - **Backend:** serialization test asserts `destinations`/`template` are omitted when empty. - `type-check`, `lint:design:strict`, and `prettier` all clean. - A popover e2e (Playwright) is a follow-up in a separate PR — the earlier spec targeted an interim dedicated page that this PR does not ship, so it was removed. ## Notes - Net change is popover-only: an earlier iteration added a dedicated Reliability page + route + rail entry; those were dropped in favour of the on-the-fly popover, so `router.ts` / `navGroups.ts` are untouched vs `main`. - Icon `graph-1` was added to the icon registry (additive) for the popover trigger. - **Deferred (see design doc Non-goals):** a server-side where-used / `?destinations=` endpoint so the popover fetches only the neighbourhood instead of the full alert list; and enterprise RBAC name-filtering on the gated fields. Design at: https://github.com/openobserve/designs/blob/main/alerts/notification-dependencies/design.md (designs#8: `alerts/notification-dependencies/design.md`) --------- Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> |
||
|
Hengfei Yang
|
42d4e2a203 |
fix: return empty when usage is not there instead of report error (#13842)
- [x] Should start check job after node start, otherwise it will report gRPC can't connect. - [x] Should skip usage check when there is no usage stream, otherwise it will always report error. |
||
  
|
1c849a9c71 |
refactor(web): migrate templates + shadows to tokens, delete the debt baseline (#13769)
## What
Deletes `web/scripts/design-debt-baseline.json` — **not by relaxing the
rules, by fixing the debt**. Every design-token category is now
zero-tolerance, SCSS is gone from the app, and 144 hand-written shadow
values collapse into one scale.
## The guard was under-reporting (three separate holes)
1. **Comment contamination.** `check-design-consistency.mjs` fed comment
text into its exemption checks, so a `keep()` comment that merely
*named* `:deep(` / `::after` / `@keyframes` silenced the rule after it —
and a `;` or `{` inside a comment corrupted the offsets, so stripping
comments afterwards couldn't help. Comments are now blanked
(offset-preserving) before the scan. **Uncovered 9 occurrences in 6
files.** Real debt was 110, not 101.
2. **`arbShadow` only scanned the template.** `box-shadow:` in a
`<style>` block and `boxShadow:` in a JS/`:style` object were invisible
— 40 literals sat there while the guard reported zero. It now scans all
three syntaxes across `.vue` and `.ts`.
3. **`rawVarInTemplate` missed the fallback form.**
`border-(--color-x,rgba(…))` and `bg-[var(--color-x,#fff)]` slipped
through. The token is always defined, so the fallback can never fire —
it only hid the read.
## stylelint had never run on 14 files
`postcss-scss` isn't installed, so `postcss-html` **silently skipped**
every `<style lang="scss">` block. The CI step named *"Lint styles
(stylelint — bans legacy --o2-* tokens)"* was a no-op for those files —
including `color-no-hex` and the `.body--dark` selector ban. All 14 are
plain CSS now and `vue/block-lang` errors on `scss`/`sass`/`less`, so it
cannot silently recur.
## What moved
| | before | after |
|---|---|---|
| `rawVarInComponent` (in `<style>`) | 110 | **0** |
| template `x-[var(--color-*)]`, utility exists | 225 | **0** |
| template reads, no utility | 27 | **0** |
| `style` / `:style` colour literals | 29 | **0** |
| `color-mix(token N%, transparent)` | 160 | **0** |
| hand-written shadows | 233 (144 distinct) | **0** |
| SFCs using SCSS | 14 | **0** |
Guard categories: 22 → 24. `rawVarInTemplate` and `arbShadow` are new;
`arbPx` was deleted because main's `local/no-hardcoded-px` ESLint rule
(#13541) now owns px across every file type.
## Shadows are a system now
Stripped of colour, the 144 values were the same handful of geometries
rewritten in px vs rem, with and without an explicit `0` spread.
```
--shadow-xs / sm / md / lg elevation
--shadow-sticky-left / -right directional (sticky columns)
--shadow-rail inset status rail
--shadow-glow-sm / -md / -lg coloured, 3 intensities
--shadow-scroll-top / -bottom / -both scroll affordance
--shadow-ring-hairline 1px ring where a utility can't reach
--shadow-ai-focus AI composer focus glow (flips in dark)
```
Colour is a **separate axis**. Tailwind rewrites a `--shadow-*` token as
`<geometry> var(--tw-shadow-color, <default>)` — verified by compiling a
probe, and it holds for multi-layer tokens — so a template writes
`shadow-md shadow-ai-accent/35`. A plain stylesheet can't add a utility,
so the role and glow tokens read `--glow-color`; that keeps the pipeline
canvas colour-coding nodes by type from CSS and lets the status rail be
painted per row from JS.
Also reclassified: `0 0 0 2px <colour>` is a **ring**, not a shadow, and
`0 1px 0 0 <colour>` is a **border**.
## color-mix, analysed by intent
All 331 occurrences were classified rather than treated as one problem:
- **160** mixed a token with `transparent` — only an alpha change, so
it's Tailwind's `/N` modifier written the long way (verified: `/N`
compiles to `color-mix(in oklab, <token> N%, transparent)`, and mixing
with `transparent` is premultiplied, so the interpolation space doesn't
matter)
- **81** blended two colours — a derived colour, so it's a token now
(~50 minted)
- **75** were shadow/ring geometry, handled by the scale
- **8 gradients + 1 `calc()`-driven percentage** genuinely have no
utility form and remain; the guard exempts gradient/mask/filter
contexts, matching what the style-block counter already exempts
## Merge with main
Main was running the same migration in parallel (**#13541 "px to rem"**
removed all SCSS, added `no-hardcoded-px`, and built its own shadow
scale), so the merge collided in 68 files / 155 hunks. Resolved by
taking main's source files and re-applying this branch on top, with the
guards driving convergence (429 → 0) rather than hand-resolving hunks.
On shadows, **main's structure wins** — geometry in `semantic.css`
`@theme`, colours as `--shadow-*-c*` sub-tokens, wrapped in its px
disable; its `--shadow-sm/md/lg`, `--shadow-scroll-*` and
`--shadow-glow` keep their definitions. This branch adds what main
didn't have: `xs`, the directional/inset roles, and the
elevation-matched coloured glows.
## New tokens
No new *colours* — each names a value a component already hardcoded,
because a registered palette utility (`text-orange-500`) is banned by
`rawPalette`, so "just register the ramp" was never available.
`--color-splitter-hover`, `--color-status-flaky` (+`-text`),
`--color-chat-bubble-user-border`, `--color-thread-avatar-bg`,
`--color-thread-accent-strong`, `--color-menu-badge-ring`,
`--color-progress-track-fill`, `--color-ingest-accent-*` (5), ~50
derived blend tokens, plus `--color-border-strong` /
`--color-border-subtle` / `--color-warning-300` registered for utility
use.
## Visual deltas to review
Everything else is value-preserving. These are not:
- **4 hover accents** move from a fixed `primary-600` to `accent` — 600
→ 400 **in dark only** (2 chevrons, an empty-state icon fill, one link)
- **sticky table header** in `TraceDetailsSidebar` was a fixed light
grey that never flipped; now `border-default`, so it themes
- **shadow canonicalization** snaps near-identical geometries together
and moves ~40 hardcoded `rgba(0,0,0,x)` onto the scale's token colour
Highest-surface single file: `AlertSettingsHelpDrawer.vue` — a whole
style block deleted with ~44 class names moved into template utilities.
## Verification
`lint:design` · `lint:tokens` · `lint:token-purity` · `lint:styles` ·
`format:check` · `lint:ci` · `vue-tsc` — all clean.
**94 spec files covering every file this branch touches: 4395 passing, 0
failing.**
Not run: the full 1236-file unit suite, Playwright/E2E, and any browser
visual check.
---------
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com>
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
|
||
|
Hengfei Yang
|
6ca0eec107 |
fix: MCP tool calls return 404 when ZO_BASE_URI is set (#13844)
Fixes #13845 ## What changed The MCP shared HTTP client (used to execute tool calls against the local REST API) now includes `ZO_BASE_URI` in its base URL: `http://localhost:{port}{base_uri}` instead of `http://localhost:{port}`. ## Why When `ZO_BASE_URI` is set (e.g. `/abc`), all HTTP routes are mounted under that prefix. The MCP endpoint itself was reachable (`POST /abc/api/default/mcp` → 200), but every tool execution failed: the internal client sent requests to the unprefixed path and got a 404. ``` INFO rmcp_openapi::http_client: Final URL: http://localhost:5080/api/default/streams INFO config::axum::middlewares::access_log: 127.0.0.1 "GET /api/default/streams HTTP/1.1" 404 ... ``` ## Notes for reviewers - `cfg.common.base_uri` is already normalized at config load to have no trailing slash (empty when unset), so the empty-prefix behavior is unchanged. - The join semantics are safe for path-prefixed base URLs: rmcp-openapi's `with_base_url` appends a trailing slash and `build_url` strips the tool path's leading slash before `Url::join`, covered by the crate's `test_build_url_with_base_url_containing_path`. So `/abc` + `/api/{org}/streams` → `/abc/api/{org}/streams`. - The other `localhost:5080` occurrence in `tools.rs` is a test-only mock client; the OAuth protected-resource metadata handler already includes `base_uri`. - `cargo check -p openobserve-mcp` and `cargo test -p openobserve-mcp` pass. |
||
|
Hengfei Yang
|
9d76505882 |
feat(traces): resolve session time ranges through the trace time index (#13837)
Design at: [https://github.com/openobserve/designs/blob/main/infrastructure/file-list/session-time-index.md](https://github.com/openobserve/designs/blob/main/infrastructure/file-list/session-time-index.md) ## Summary - carry an optional `session_id` attribute on trace time index records, resolved from span attributes by the existing `SESSION_ID_COLUMNS` precedence; the first non-empty value in the request wins for the whole trace, stamped onto all of its hour-bucket records - keep the aggregation key `(hour, trace_id)` unchanged; hourly compaction picks the attribute up with `MAX(session_id)`, which ignores NULLs - reuse the locate/expand timer for sessions with a 7-day expand window (traces keep 24h); a UUID v7 session id doubles as a probe anchor and tightens the scan's left bound to the session's lifetime neighborhood - aggregate session lookups per trace, so a lookup resolves the session's range plus every member trace's real range - reshape the time-range endpoint to query parameters so one endpoint serves both lookup kinds - make the session-details caller range optional, union it with the indexed range (never narrowed), take trace ids from the index, and keep the legacy source-stream scan for pre-coverage data - add `session_id` to the default bloom filter fields (gated by the existing `ZO_FEATURE_DEFAULT_INDEX_FIELDS_ENABLED`, folds into the secondary-index field set like `trace_id`); streams without the column build nothing ## Why Session detail lookup depends on a caller-provided time range: the sessions list passes its own picker window into the details request, so an LLM session that started before the window silently loses its earlier turns, and a deep link without a range is a hard 400. The trace time index already turns a `trace_id` into a time range at bounded cost; a session is the same problem one level up — `session_id` → range plus member traces. ## Environment variables No new variables. The existing `ZO_TRACE_TIME_INDEX_ENABLED` (default `true`) gates the feature, session column included. The existing coverage marker is reused; there is no historical backfill. This change must ship in the same release as #13827: reusing the coverage marker is valid only while no release separates the two. ## API changes All timestamp parameters and returned ranges use microseconds. ### Time-range endpoint reshaped ```http GET /api/{org_id}/{stream_name}/traces/time_range?trace_id=xx&hint_ts=... GET /api/{org_id}/{stream_name}/traces/time_range?session_id=xx&hint_ts=... ``` Exactly one of `trace_id` / `session_id` is required; zero or both is `400`. This replaces `GET /{stream_name}/traces/{trace_id}/time_range`; the old path form is removed without a compatibility alias since #13827 has not shipped in a release. ### Session details: range becomes optional `GET /api/{org_id}/{stream_name}/traces/session/details` now accepts `start_time` / `end_time` as an optional pair plus an optional `hint_ts` (defaulting to the caller range's midpoint). The effective range is the union of the caller range and the indexed session range, so it never narrows what the caller asked for; `400` only when neither side yields a range. When the caller range lies entirely inside index coverage the source-stream trace-id scan is skipped; otherwise it still runs over the caller range and its results are merged, so behavior is never worse than before. No frontend changes are required — the frontend keeps sending its list window and the union completes the session. ## Validation - `cargo test -p openobserve-core traces::time_index --lib` — 13 passed - `cargo test -p search datafusion::merge --lib` — 5 passed, including NULL-mixed session aggregation and pre-column file compatibility - `cargo test -p openobserve-api-search --lib` — 149 passed - `cargo test -p openobserve-api-http --lib` — 53 passed, including OpenAPI/MCP initialization - `cargo test -p config router:: --lib` — 8 passed - `cargo test -p infra db::trace_time_index --lib` — passed - `cargo clippy` on all touched crates (`--all-targets`) and `cargo check --workspace` — clean - the enterprise repository has no references to any changed API and no enterprise-gated code was modified |
||
|
Abhay Padamaniandgithub-actions[bot]
|
f112ac01a5 |
fix: dashboard layout improvements (#13815)
<h2 style="unicode-bidi: plaintext; color: rgb(204, 204, 204);
font-family: -apple-system, "system-ui", "Segoe UI",
Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; letter-spacing: normal; orphans: 2;
text-align: start; text-indent: 0px; text-transform: none; widows: 2;
word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal;
background-color: rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color:
initial;">Dashboard layout & UX improvements</h2><p
style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em;
unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family:
-apple-system, "system-ui", "Segoe UI", Roboto,
sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures:
normal; font-variant-caps: normal; font-weight: 400; letter-spacing:
normal; orphans: 2; text-align: start; text-indent: 0px; text-transform:
none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;">Polish
pass on the dashboard experience: settings layout, drag/resize grid
feedback, empty states, and per-tab panel counts.</p><h3
style="unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family:
-apple-system, "system-ui", "Segoe UI", Roboto,
sans-serif; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; letter-spacing: normal; orphans: 2;
text-align: start; text-indent: 0px; text-transform: none; widows: 2;
word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal;
background-color: rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;">✨
Highlights</h3><p style="white-space: pre-wrap; margin-top: 0.1em;
margin-bottom: 0.2em; unicode-bidi: plaintext; color: rgb(204, 204,
204); font-family: -apple-system, "system-ui", "Segoe
UI", Roboto, sans-serif; font-size: 13px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal; font-weight:
400; letter-spacing: normal; orphans: 2; text-align: start; text-indent:
0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(24, 24, 24);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;"><strong>Grid feedback while
dragging/resizing panels</strong> (<a
href="vscode-webview://0vkpvjgb486eanj04qjfqcnqmnatevjacbbbqdp3ecrs4efo5krm/web/src/views/Dashboards/RenderDashboardCharts.vue"
target="_blank" rel="noopener noreferrer" style="color: rgb(77, 170,
252); text-decoration: rgb(77, 170,
252);">RenderDashboardCharts.vue</a>)</p><ul
style="padding-inline-start: 2em; color: rgb(204, 204, 204);
font-family: -apple-system, "system-ui", "Segoe UI",
Roboto, sans-serif; font-size: 13px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal; font-weight:
400; letter-spacing: normal; orphans: 2; text-align: start; text-indent:
0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; white-space: normal; background-color:
rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;"><li
style="unicode-bidi: plaintext;">Added a live alignment grid that
appears only during a drag or resize, drawn from GridStack's real cell
metrics (192 columns × row height).</li><li style="unicode-bidi:
plaintext;">The grid is<span> </span><strong>operation-aware</strong>:
lines snap to the top/left edges
while<span> </span><em>moving</em><span> </span>and the bottom/right
edges while<span> </span><em>resizing</em>, so the edge you're dragging
always lands on a line and every line is reachable (no skipped
positions).</li><li style="unicode-bidi: plaintext;">Panels are made
opaque during interaction (<code style="font-family: monospace; color:
rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px;
border-radius: 3px; word-break: break-word; font-size:
0.9em;">--color-surface-base</code>) so the grid only shows in the
gutters/empty cells, not through chart or table panels.</li></ul><p
style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em;
unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family:
-apple-system, "system-ui", "Segoe UI", Roboto,
sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures:
normal; font-variant-caps: normal; font-weight: 400; letter-spacing:
normal; orphans: 2; text-align: start; text-indent: 0px; text-transform:
none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color:
initial;"><strong>Empty states</strong> (<a
href="vscode-webview://0vkpvjgb486eanj04qjfqcnqmnatevjacbbbqdp3ecrs4efo5krm/web/src/lib/core/EmptyState/presets.ts"
target="_blank" rel="noopener noreferrer" style="color: rgb(77, 170,
252); text-decoration: rgb(77, 170, 252);">presets.ts</a>, <a
href="vscode-webview://0vkpvjgb486eanj04qjfqcnqmnatevjacbbbqdp3ecrs4efo5krm/web/src/components/dashboards/settings/VariableSettings.vue"
target="_blank" rel="noopener noreferrer" style="color: rgb(77, 170,
252); text-decoration: rgb(77, 170, 252);">VariableSettings.vue</a>, <a
href="vscode-webview://0vkpvjgb486eanj04qjfqcnqmnatevjacbbbqdp3ecrs4efo5krm/web/src/views/Dashboards/ScheduledDashboards.vue"
target="_blank" rel="noopener noreferrer" style="color: rgb(77, 170,
252); text-decoration: rgb(77, 170,
252);">ScheduledDashboards.vue</a>)</p><ul style="padding-inline-start:
2em; color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif;
font-size: 13px; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; font-weight: 400; letter-spacing: normal;
orphans: 2; text-align: start; text-indent: 0px; text-transform: none;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
white-space: normal; background-color: rgb(24, 24, 24);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;"><li style="unicode-bidi:
plaintext;">Replaced the generic<span> </span><code style="font-family:
monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60);
padding: 2px 4px; border-radius: 3px; word-break: break-word; font-size:
0.9em;">NoData</code><span> </span>placeholder with<span> </span><code
style="font-family: monospace; color: rgb(208, 208, 208);
background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px;
word-break: break-word; font-size:
0.9em;">OEmptyState</code><span> </span>for the Variables settings table
and the Scheduled Dashboards list.</li><li style="unicode-bidi:
plaintext;">Added a new<span> </span><code style="font-family:
monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60);
padding: 2px 4px; border-radius: 3px; word-break: break-word; font-size:
0.9em;">no-variables</code><span> </span>preset (illustration + copy +
"Add variable" action); reused the existing<span> </span><code
style="font-family: monospace; color: rgb(208, 208, 208);
background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px;
word-break: break-word; font-size:
0.9em;">no-reports</code><span> </span>preset for scheduled
dashboards.</li><li style="unicode-bidi: plaintext;">Fixed the Variables
table so a long
list<span> </span><strong>scrolls</strong><span> </span>instead of
clipping, and made the Scheduled Dashboards table fill the drawer height
so the empty state centers.</li></ul><p style="white-space: pre-wrap;
margin-top: 0.1em; margin-bottom: 0.2em; unicode-bidi: plaintext; color:
rgb(204, 204, 204); font-family: -apple-system, "system-ui",
"Segoe UI", Roboto, sans-serif; font-size: 13px; font-style:
normal; font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start;
text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(24, 24, 24);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;"><strong>Dashboard settings
layout</strong> (<a
href="vscode-webview://0vkpvjgb486eanj04qjfqcnqmnatevjacbbbqdp3ecrs4efo5krm/web/src/components/dashboards/settings/GeneralSettings.vue"
target="_blank" rel="noopener noreferrer" style="color: rgb(77, 170,
252); text-decoration: rgb(77, 170,
252);">GeneralSettings.vue</a>)</p><ul style="padding-inline-start: 2em;
color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif;
font-size: 13px; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; font-weight: 400; letter-spacing: normal;
orphans: 2; text-align: start; text-indent: 0px; text-transform: none;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
white-space: normal; background-color: rgb(24, 24, 24);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;"><li style="unicode-bidi:
plaintext;">Reworked the General settings layout.</li></ul><p
style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em;
unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family:
-apple-system, "system-ui", "Segoe UI", Roboto,
sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures:
normal; font-variant-caps: normal; font-weight: 400; letter-spacing:
normal; orphans: 2; text-align: start; text-indent: 0px; text-transform:
none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color:
initial;"><strong>Tab panel-count badge</strong> (<a
href="vscode-webview://0vkpvjgb486eanj04qjfqcnqmnatevjacbbbqdp3ecrs4efo5krm/web/src/components/dashboards/tabs/TabList.vue"
target="_blank" rel="noopener noreferrer" style="color: rgb(77, 170,
252); text-decoration: rgb(77, 170, 252);">TabList.vue</a>)</p><ul
style="padding-inline-start: 2em; color: rgb(204, 204, 204);
font-family: -apple-system, "system-ui", "Segoe UI",
Roboto, sans-serif; font-size: 13px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal; font-weight:
400; letter-spacing: normal; orphans: 2; text-align: start; text-indent:
0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; white-space: normal; background-color:
rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;"><li
style="unicode-bidi: plaintext;">Each dashboard tab now shows a badge
with its panel count (section headers excluded), so tab density is
visible without opening each tab.</li></ul><p style="white-space:
pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; unicode-bidi:
plaintext; color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif;
font-size: 13px; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; font-weight: 400; letter-spacing: normal;
orphans: 2; text-align: start; text-indent: 0px; text-transform: none;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color:
initial;"><strong>Variable dependency indicator</strong> (<a
href="vscode-webview://0vkpvjgb486eanj04qjfqcnqmnatevjacbbbqdp3ecrs4efo5krm/web/src/components/dashboards/settings/VariableSettings.vue"
target="_blank" rel="noopener noreferrer" style="color: rgb(77, 170,
252); text-decoration: rgb(77, 170,
252);">VariableSettings.vue</a>)</p><ul style="padding-inline-start:
2em; color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif;
font-size: 13px; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; font-weight: 400; letter-spacing: normal;
orphans: 2; text-align: start; text-indent: 0px; text-transform: none;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
white-space: normal; background-color: rgb(24, 24, 24);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;"><li style="unicode-bidi:
plaintext;">Variables that depend on others show a "Depends on: …" chip
(tree icon + count) in the settings table.</li></ul><p
style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em;
unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family:
-apple-system, "system-ui", "Segoe UI", Roboto,
sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures:
normal; font-variant-caps: normal; font-weight: 400; letter-spacing:
normal; orphans: 2; text-align: start; text-indent: 0px; text-transform:
none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color:
initial;"><strong>Panel header token</strong> (<a
href="vscode-webview://0vkpvjgb486eanj04qjfqcnqmnatevjacbbbqdp3ecrs4efo5krm/web/src/lib/styles/tokens/component.css"
target="_blank" rel="noopener noreferrer" style="color: rgb(77, 170,
252); text-decoration: rgb(77, 170, 252);">component.css</a>)</p><ul
style="padding-inline-start: 2em; color: rgb(204, 204, 204);
font-family: -apple-system, "system-ui", "Segoe UI",
Roboto, sans-serif; font-size: 13px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal; font-weight:
400; letter-spacing: normal; orphans: 2; text-align: start; text-indent:
0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; white-space: normal; background-color:
rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;"><li
style="unicode-bidi: plaintext;">Lightened<span> </span><code
style="font-family: monospace; color: rgb(208, 208, 208);
background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px;
word-break: break-word; font-size:
0.9em;">--color-panel-bar-bg</code><span> </span>(grey-200 → grey-100)
for a softer chart-header band; the grey-300 divider still carries the
header/body edge.</li></ul><h3 style="unicode-bidi: plaintext; color:
rgb(204, 204, 204); font-family: -apple-system, "system-ui",
"Segoe UI", Roboto, sans-serif; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px;
text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; white-space: normal; background-color:
rgb(24, 24, 24); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;">
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
||
|
ktx-akshay
|
33c6e9e0e1 |
test(dashboards): cover CTE/subquery match_all queries through the panel search path (#13838)
## What First test coverage for the dashboard **panel** endpoints and the **dashboard search path**, running CTE/subquery `match_all` queries as real dashboard panels. Nothing in the repo previously touched either: `grep -rn "/panels\|search_type=dashboards" tests/api-testing/` returned zero hits. ## Why Panels query `_search?type=logs&search_type=dashboards&dashboard_id=…` — a different route into the engine than the Logs page (`search_type=ui`). The `match_all` + FST projection rewrite fixed in #13808 runs inside it, so a query that passes via Logs can still regress on a dashboard. The rewrite has to preserve the projection of the block the predicate sits in. When that block is the whole statement, the projection is obvious. When it's a CTE body feeding a JOIN, a window, or a four-branch UNION ALL, it has to preserve the *CTE's* output schema while the outer query reads different columns from it — which is where it went wrong. ## Coverage **71 distinct queries across 29 combination families**, all with `match_all` inside a `WITH …` body or a derived table: | Axis | Variants | |---|---| | Nesting depth | 1, 2, 3, and 5-level CTE chains | | Nesting kind | CTEs and derived tables (and one query using both) | | UNION ALL | 2, 3, and 4 separate `match_all` branches | | JOIN | CTE⋈CTE, derived⋈derived, cross-stream | | Window | `ROW_NUMBER`, `RANK`, `DENSE_RANK`, partitioned aggregates | | Negation | `NOT match_all`, and mixed with positive `match_all` | | Combined with | `COUNT(DISTINCT)`, `CASE`, `HAVING`, `histogram()` | Each case creates a real panel, runs it via the dashboard search path, then asserts row count against the DuckDB oracle and that no expected column was dropped from the projection. A further 30 cases run the same SQL through **both** Logs and dashboard and diff the results — that's what catches a rewrite firing on only one path. Queries are reused from the existing query-agent corpus rather than duplicated, so the two suites can't drift: query-agent verifies the values via the Logs path, this verifies the same SQL survives the dashboard path. Cases are **discovered by classifying the SQL**, not from a hand-written ID list, so new combinations added to the corpus are picked up automatically. Test ids name the shape (`Q1022-cte1_union_cdist_4ma`), so a failure identifies the combination without opening the corpus. Plus 14 tests for the panel endpoints themselves — AddPanel / UpdatePanel / DeletePanel, contract + CRUD lifecycle + 7 error paths derived from `DashboardError` in `src/core/src/http.rs`. ## Changes | File | | |---|---| | `tests/dashboards/test_dashboard_cte_panels.py` | new — 103 tests | | `tests/dashboards/test_dashboard_panels.py` | new — 14 tests | | `tests/dashboards/conftest.py` | new — shared fixtures | | `support/panel_queries.py` | new — discovery + panel helpers | | `support/endpoints/dashboards.py` | +78, additive | | `support/endpoints/search.py` | +36, additive | | `support/factories.py` | +73, additive | The three shared-file edits are **187 added lines, 0 modified** — nothing existing can break. ## Results 168 tests in tests/api-testing/tests/dashboards/ (117 new + 51 pre-existing) 166 passed, 1 skipped, 1 xfailed # full nested-query corpus present 74 passed, 2 skipped, 1 xfailed # current corpus on main Repeated 2× and 4× respectively, no flakes. Discovery scales with the corpus: on main it finds 8 nested `match_all` cases, and 71 once the larger sentinel set is present. Cases that aren't in the corpus skip with an explanatory message instead of failing, so this is safe to merge in either order. |
||
 Julián ValdésandHengfei Yang
|
06c6b515fb |
fix(stats): only read file_list_dump_stats when dumping is enabled (#13830)
Fixes #13829. ## Problem `file_list_dump_stats` is created **conditionally** — `handle_partitioned_tables` only adds it when `ZO_COMPACT_FILE_LIST_DUMP_ENABLED` is set, and that flag defaults to `false`. But `update_stats_from_file_list_for_stream` reads it **unconditionally**. On a PostgreSQL metastore where the table was never created: ``` [STATS] run update stream stats from file list error: SqlxError# error returned from database: relation "file_list_dump_stats" does not exist ``` **The failure is wider than the dump.** The `?` propagates before `set_stream_stats`, so the entire stats update is aborted — every stream reports `doc_num: 0` while ingestion, search and compaction keep working. It reads as a UI problem rather than a failing job, which is what makes it expensive to track down. SQLite creates the table unconditionally, so this only surfaces on deployments with an external PostgreSQL metastore. ## Change Gate the read on the same flag that gates the creation. One call site — `query_dump_stats_by_date_range` has no other caller. When dumping is off there is nothing to merge, so the stats are unchanged: `merge` over an empty result was already a no-op. The difference is that the surrounding update now completes instead of aborting. ## Alternative considered Always creating the table would also fix it, at the cost of an empty partitioned table on every deployment. Gating the read seemed closer to the existing intent — the table is deliberately optional — but I am happy to flip it if you prefer the schema to be unconditional. ## Testing Reproduced on v0.92.1 with `ZO_META_STORE=postgres` and the flag at its default: stats fail every cycle and all counters sit at zero. Creating the table by hand with the DDL from `create_partitioned_dump_stats_table` makes the error and the zeros go away, which confirms the missing table is the whole cause. I have not been able to build the project locally to run the test suite — no Rust toolchain on the machine I reproduced this on. The change is a single conditional and the import follows the one already used in `src/compaction/src/merge.rs` (`use config::{get_config, ...}`), but it deserves a CI run before merging. Signed-off-by: Julián Valdés <julianvaldes24@gmail.com> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
  Neha PandClaude Opus 4.7
|
8a4f7ea23b |
test(query-agent): add 70 match_all sentinel queries for FST rewrite path coverage (#13824)
## Summary Adds Q1003–Q1072 (70 queries) to `tests/test-data/query-agent/queries/full_text_search.json`. All exercise the `match_all` + FST projection rewrite path fixed in #13808, under diverse plan shapes. Structural patterns mirror real production customer queries (fan-out UNION ALL, multi-CTE, CSE-triggering scalar wrappers on the FTS field) but use **only test-fixture fields and values** — no customer identifiers. ## Shape coverage | Family | Range | Count | What it exercises | |---|---|---|---| | A. Subquery + `match_all` + GROUP BY | Q1003–Q1012 | 10 | Simple inline-view aggregations | | B. CTE with fan-out UNION ALL, `match_all` per branch | Q1013–Q1027 | 15 | Dominant customer pattern (multi-branch UNION + per-branch FTS + LIKE/regex + outer aggregate) | | C. CTE + scalar wrapper on `log` (trim/upper/split_part/regexp_replace/substring/concat/length) + `match_all` | Q1028–Q1037 | 10 | **CSE-trigger sentinels** — the wrappers cause DataFusion CSE to insert the intermediate `ProjectionExec` that #13808 fixes | | D. Deep multi-CTE + `match_all` + regex/LIKE/CASE combos | Q1038–Q1047 | 10 | Deep plan trees, many projection candidates | | E. Edge cases (DISTINCT, HAVING, CASE, IN-list) | Q1048–Q1052 | 5 | Corner combinations | | F. `match_all` inside JOINs (self-join, LEFT JOIN, cross-stream) | Q1053–Q1058 | 6 | JOIN builders inject their own projections | | G. `match_all` + window functions (ROW_NUMBER, RANK, DENSE_RANK, SUM/COUNT OVER) | Q1059–Q1063 | 5 | Window planning inserts projections above filter | | H. Negated `match_all` (NOT, AND NOT, UNION of pos/neg) | Q1064–Q1066 | 3 | Rewriter's negation code path | | I. Multiple `match_all` in same query (AND, OR, nested) | Q1067–Q1069 | 3 | Two FST fields injected simultaneously | | J. `histogram` + `match_all` time-series | Q1070–Q1072 | 3 | Common customer pattern; histogram optimizer + FST rewriter together | ## Verification - All 70 queries have DuckDB oracle results auto-populated via `compute_counts.py`. - Two initial cross-stream JOIN queries (Q1055, Q1058) were rewritten to disambiguate the FTS field between streams — now they scope `match_all` inside a per-stream CTE before joining. - Audit for customer-token leakage (pharma, tomcat, o2_test, usgm, storenumber, tagname, cash, credit_card, etc.) returns **zero hits** across Q1003–Q1072. ## Implementation notes - Added generator script [tests/test-data/query-agent/gen_q1003_q1052.py](tests/test-data/query-agent/gen_q1003_q1052.py) alongside existing `gen_q201_q400.py` / `gen_q511_q556.py` convention. Deterministic regeneration. - Wide JSON churn across the other 12 category files is `compute_counts.py` re-baselining expected results against a fresh `BASE_TS` (`base_ts_override.json` is untracked); tuple-order shuffles are harmless under `results_mode: rowsort`. - Coverage is most meaningful under the `remove_filter_with_index_false` CI leg (added in #13810), where `match_all` actually reaches `RewriteMatchPhysical`. Under the default flag (`true`), the index optimizer removes `match_all` before the rewriter runs, masking the bug path. ## Test plan - [ ] `api_query_agent / single_node_opt_true` — green - [ ] `api_query_agent / single_node_opt_false` — green - [ ] `api_query_agent / remove_filter_with_index_false` — green (this is the leg that actually validates the rewriter under all 70 shapes) - [ ] Q1003–Q1072 all pass on both memtable and parquet phases 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
udev
|
35860aff9d |
fix: route annotation queue score lookups to queriers (#13801)
## Summary - route Discovery and annotation queue endpoints to queriers so score lookups, item hydration, and review reads do not fall through to ingesters - make Discovery source_stream required at the API response boundary - cover Discovery, queue item detail, and review routing with regression tests The QueueItem remains reference-only. Workbench and enqueue validation derive source_stream from timestamp-bounded _llm_scores queries in the enterprise implementation. Depends on https://github.com/openobserve/o2-enterprise/pull/2423 ## Testing - 5 querier routing tests - Discovery response serialization test with the enterprise feature - 16 enterprise annotation queue tests - 14 enterprise Discovery tests |
||
|
udev
|
d770d0245a |
fix: preserve online evaluation provider API keys (#13739)
## Summary - preserve the stored provider API key when an edit submits a blank value - replace the stored key when a nonblank value is supplied - show the create-time required marker only for providers that require authentication - retain keyless creation for self-hosted providers ## Testing - ProviderFormPage Vitest suite passed: 8 tests - Targeted ESLint passed - cargo fmt check passed - git diff check passed - Focused Rust test compilation is blocked by unrelated existing alert-state model errors Related to #13655. Companion enterprise PR: https://github.com/openobserve/o2-enterprise/pull/2390 |
||
|
Bhargav
|
b9e11bb885 |
fix(web): classify unquoted LIKE patterns wherever the parser stops (#13818)
`x LIKE E00%` parses as the identifier `E00` followed by the modulo operator, so the PEG parser fails on a later token and the existing `found === "%"` check never fires — the error fell through to the allow-list suppression and the editor showed no diagnostic at all. Only patterns whose first character terminates the identifier (`LIKE BIN-%`) were caught. Add hasUnquotedLikePattern(), which inspects the LIKE/ILIKE/RLIKE operand nearest the error position and reports badLikePattern when it is a bare wildcard token. It runs immediately before the suppression return, so it can only reclassify errors that were previously dropped; quoted patterns, parenthesised expressions and function calls never match. Detection on the unquote_like_pattern mutation set goes 6/13 -> 13/13, so raise that threshold from 0.5 to 0.9. The old 0.5 sat exactly on the sample boundary, which is why adding one query to the shared query-agent corpus in #13808 turned the suite red. Also stop the per-mutation tests from regenerating the whole mutation pool (~1,000 queries x 16 mutations, each re-parsed) on every case: memoize generateBrokenQueries() and give the heavy tests an explicit 30s timeout. Spec runtime drops from ~112s to ~48s and the 5s-default timeouts go away. |
||
 Omkar Kesarkhaneandgithub-actions[bot]
|
ddb67f3efb |
fix(synthetics): record from a specific step, and stop silently downgrading recorded actions (#13799)
Supersedes #13777, which carried the record-from-step commits alone. Closing that in favour of this, because the two changes land in the same three files (`types/synthetics.ts`, `useSyntheticsRecorder.ts`, `JourneySteps.vue`) and reviewing them apart means resolving the same conflict twice. ## Problem **A recorded hover was stored as a click.** Playwright 1.56 added `hover` to the recorder's action model, reachable from its "Choose action" picker. The extension's action mapper seeds every step with `action: 'click'` as a placeholder for its switch to overwrite, and returned that placeholder from `default:` — so any action name the switch did not know became a click the author never performed. On the usual hover target — a nav trigger, a dropdown opener — clicking it can navigate or submit. The run still reports green. There are three separate guards meant to stop exactly this, and none of them fired, because the relabel happens upstream of all of them: | Guard | Why it missed | |---|---| | `UNSUPPORTED_REPLAY_ACTIONS` (extension) | step already said `click` | | `RETIRED_ACTIONS` → `isStorableAction` | step already said `click` | | `V2_STEP_ACTIONS` (Rust validation) | `click` is valid | The same picker offers **Right click** and **Double click** as explicit choices. Both were flattened too: `click_count` was dropped by the mapper and never existed downstream, and the probe's executor called `locator.click({ timeout })` while reading neither `button` nor `click_count`. A deliberately recorded right click replayed as a left click; a double click as a single one. **Recording could only start from scratch.** Fixing step 12 of a 15-step journey meant re-recording all 15. ## Change **Record from a specific step.** Each step gets a record action that replays the prefix before it, then flips the recorder into insert mode at that point. A restore that cannot reach the requested step reports where it stopped (`prefixFailed`) rather than silently recording from the wrong place. **Every action Playwright can record is now stored and replayed as itself.** - `hover` is a first-class v2 action — mapped, stored, validated, and executed by `Locator.hover` in the probe and `Frame.hover` in the extension player. - `button` and `click_count` travel to storage, so right click and double click replay as recorded. - An action name the mapper does not know now **throws** instead of returning the `click` placeholder. This is the part that is not hover-specific: upstream added one action in nine minors and will add more, and the next one must be a deliberate decision rather than a silent downgrade. **The extension handshake reports capabilities, not just presence.** `getStatus` now returns `extVersion` and a `capabilities` array; O2 gates affordances on `hasCapability()` with a defined pre-handshake fallback, and warns when the installed build is older than the minimum. Previously the protocol carried no version at all — Chrome updates the extension independently of when O2 deploys, so the two could disagree about the wire with no way to say so. **Two dead message declarations, opposite problems.** `setSources` no longer exists in playwright-core (sources arrive alongside the actions), so its declared variant could never fire — while its `generatedCode` field is exactly what an eject-to-code feature would reach for. Deleted. `elementPicked` is the reverse: it fires on every "Pick locator" and every click in inspecting mode, and was being discarded, which made inspect mode a dead end. Now kept. ### Compatibility `button` and `click_count` are optional with defined absent-behaviour — absent means a single left click. Every journey stored today stays valid, means the same thing, and serialises byte-identically. **No schema version bump and no migration.** A test pins that rather than leaving it as an intention. `hover` is a new vocabulary entry with no default, so the probe must be able to execute one before O2 is allowed to store one — the probe change ships first. ## Implementation | File | | |---|---| | `src/config/src/meta/synthetics.rs` | `hover` joins `V2_STEP_ACTIONS`/`V2_ELEMENT_ACTIONS`; `button` and `click_count` added to `BrowserStepV2` with validation (`left\|middle\|right`, `1..=3`) | | `web/src/types/synthetics.ts` | `button`/`clickCount` on both step types; `extVersion` + `capabilities` on `RecorderStatus`; `setSources` variant deleted and `elementPicked` typed; `prefixFailed` and insert-mode fields on `recordingStarted` | | `web/src/utils/synthetics/mapRecordedStep.ts` | promotes `button`/`clickCount` from the wire, like every other stored field | | `web/src/utils/synthetics/buildV2Steps.ts` | `hover` in `ACTION_TO_V2`; crosses camelCase → snake_case for the two click fields | | `web/src/constants/synthetics.ts` | `hover` out of `RETIRED_ACTIONS`; `MIN_EXTENSION_VERSION` | | `web/src/composables/useSyntheticsRecorder.ts` | `startRecordingFrom`, `hasCapability`, `pickedSelector`, `extensionOutdated` | | `web/src/components/synthetics/journey/{BrowserJourney,JourneySteps}.vue` | per-step record action and its wiring | `click_count` is capped at 3: the recorder only ever emits 1 or 2, and an unbounded count is a way to hold a page's event loop open. The two click fields cross from camelCase to snake_case in `buildV2Step` and nowhere else — reading `step.wire` directly inside it would have worked and invented a second path into storage. ### Depends on a recorder extension build `hover`, the click fields and the capability handshake all need the matching extension build. O2 degrades honestly without it: `hasCapability()` falls back to the pre-handshake set, and an older build raises the outdated-extension warning instead of failing on a message shape it does not recognise. ## Tests - **Rust** — hover accepted as an element action; `button`/`click_count` validated at both bounds; a step carrying neither still validates (the compatibility guarantee). Two existing tests encoded "hover is rejected" and now say the opposite. 101 pass. - **Probe** — hover hovers rather than clicks; a right click sends the recorded button; a double click sends two clicks; a journey stored before these fields replays unchanged. The action union is asserted against the Rust vocabulary, so the next divergence fails a test instead of reaching an executor with no case for it. Also pinned: a selector carrying a frame hop resolves into the iframe, which had no coverage anywhere. - **Web** — 1213 across 60 files. Nine existing tests encoded "hover is retired" across constants, `buildV2Steps`, `buildPayload` and the save-gate schema; each now states the opposite deliberately. - **Extension** — the unmapped-action throw, hover round-tripping both directions, and a double click surviving the round trip as two clicks. `vue-tsc`, `cargo clippy -D warnings`, and `prettier --check` are clean. The last commit is formatting only: the three record-from-step files arrived failing `format:check`, which is a gate of its own and would not have been caught by lint. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Hengfei Yang
|
192f11f7ea |
fix(i18n): correct mistranslations in zh-CN locale (#13835)
## Summary
Fixes machine-translation errors in the Simplified Chinese locale
(`web/src/locales/languages/zh-CN.json`), found by sweeping the file
against the English source. Three batches of the same class of error:
### Nonsense translations (wrong sense of the English word)
| Before | After | English source |
|---|---|---|
| 痕迹 / 踪迹 (physical traces) | 追踪 | Traces (nav, tabs, buttons — 8 keys)
|
| 朗姆酒 (the liquor) | RUM | RUM |
| 漂亮 / 崩溃了 (beautiful / crashed) | 格式化 / 折叠 | Pretty / Collapsed view
toggle |
| 汽车 (a car) | 自动 | Auto (layout mode) |
| 转 / 转弯 (road turns) | 回合 | Turn / Turns (conversation) |
| {n} 个电话 (phone calls) | {n} 次调用 | {n} calls (tool calls) |
| oTel 收藏家/代理人 (art collector) | OTel Collector / 代理 | OTel Collector /
Agent |
| 空缺评测任务 (job vacancy) | 打开评估任务 | Open Eval Jobs |
| 简历 (a CV) | 恢复 | Resume (pause/resume) |
| 出口 (trade export, ×3) | 导出 | Export |
| 逃脱了 (ran away) | 已转义 | Escaped (string escaping) |
| 测试赛 (sports match) | 测试匹配 | Test Match |
| 比分 (sports score) | 分数 | scores (eval scores) |
| 好吧 ("well, fine") | 正常 | Ok (status) |
| 领域 (domain/field) | 区域 | Region (bucket region) |
| 全球 (worldwide, ×4) | 全局 | Global (scope) |
| 商店 (a shop) | 存储 | Store (key store) |
| 尺寸 (physical size, ×5) | 维度 | Dimensions (metrics) |
| 用法 / 使用密钥 (how-to / secret key) | 用量 / 用量键 | Usage / Usage key |
| 滤镜 (photo filters) | 筛选条件 | Filters |
| 网络缓存影响 (network) | 缓存净影响 | Net Cache Impact |
| 申请并搜索 (apply for) | 应用并搜索 | Apply & Search |
| 压缩后的尺寸 | 压缩后大小 | Compressed Size |
| 等得紧，我们正在搜寻你的踪迹。 | 正在加载追踪... | Loading Traces... |
### Terminology unification
- **GenAI agents → 智能体** (agent graph, agent behavior, agent mapping,
sessions — ~40 strings). Infrastructure agents (host collectors,
synthetics runners) keep the conventional 代理.
- **Eval jobs → 评估任务** (was a mix of 评估作业 / 评测任务 / 作业). Batch jobs
elsewhere (backfill, search jobs) keep 作业.
- **Enrichment table → 富化表** (was a mix of 充实表 / 丰富表 / 扩充表).
- **Cipher Key → 加密密钥** (was mixed with 密码密钥); **Actions → 操作** (was 行动
in 4 places); **Running Queries → 运行中的查询**; synthetics **journey → 流程**
(was 旅程); assistant 助理 → 助手.
- `shortcuts.actions.tracesSearch` resynced with current English source
("Run query" → 运行查询).
### Cosmetic
- Shortcuts cheatsheet: corrupted ⌘ symbol (rendered as ▼) restored;
ungrammatical strings fixed (按？任何地方 → 在任意位置按 ?, 焦点搜索栏 → 聚焦搜索栏, 导入控制面板 →
导入仪表板); untranslated example values restored to `input`/`output`;
half-translated strings completed (Spans 表格视图 → 跨度表格视图, Stacktrace 已复制 →
堆栈跟踪已复制).
Notes:
- zh-TW.json was checked for the same error patterns and is clean.
- 跟踪 vs 追踪 for "trace" is still mixed in a few longer sentences; left
as-is since 跟踪 is not incorrect, only inconsistent.
## Test plan
- JSON validated with `python3 -c 'json.load(...)'` after each batch;
keys unchanged, only values edited (~200 lines).
- Grep confirms none of the wrong terms above remain in zh-CN.json.
|
||
|
Hengfei Yang
|
41ff3cd9b6 |
feat: replace distinct_values_* streams with tantivy TopK for the _values API (#13834)
Design at: #13822 ## Summary `distinct_values_*` metadata streams existed to serve one reader: the `_values` API. They are a redundant — and lossier — materialization of what the tantivy secondary index already holds, and the index has the collectors needed to replace them outright (`SimpleTopN` / `SimpleDistinct`). This PR removes the distinct streams from the write and read paths and lets the `_values` API be served by the tantivy index optimizer. ### 1. Treat distinct value fields as secondary index fields (read-time union) - `get_stream_setting_index_fields()` — the single accessor used by both the index build side (ingester parquet/pack jobs, compactor merge) and the query side (leader/follower index optimizers) — now unions `distinct_value_fields` into the effective index field set, gated on `enable_distinct_fields`. Fields that cannot be secondary-indexed (full text search keys, partition keys, reserved columns) are skipped; the values API falls back to a normal scan for those. - The distinct list is deliberately **not merged into `settings.index_fields`**: the two stay independent, so removing a distinct value field (or disabling `enable_distinct_fields`) cleanly stops its indexing with no residue, and the user-managed `index_fields` list is never mutated behind their back. - Correctness gate: a distinct-only field is considered indexed since its `added_ts` (`get_stream_setting_index_updated_at_for_fields`), so parquet files older than that are served by DataFusion scan instead of the index (`split_file_list_by_time_range`). `added_ts` is set on add and reset when `enable_distinct_fields` is re-enabled (existing behavior). For streams configured before this release the fields only start being indexed at upgrade time, so `added_ts` is additionally clamped by a release-wide cutoff `/tantivy/distinct_fields/updated_at` — a lazy marker initialized to the first access after upgrade, following the existing `ttv_secondary_index_updated_at` pattern. No data migration and no `DB_SCHEMA_VERSION` bump needed. ### 2. `_values` API always queries the original stream - Removed `can_use_distinct_stream()` and the `distinct_values_*` / `SUM(count)` / `StreamType::Metadata` rewrite from `values_inner` (GET `/_values`, formerly `values_v1`) and `build_search_request_per_field` (POST `/_values_stream`). - The generated SQL shapes are exactly what the index optimizer detects: - `SELECT f AS zo_sql_key, COUNT(*) AS zo_sql_num FROM s GROUP BY zo_sql_key ORDER BY zo_sql_num DESC, zo_sql_key ASC` + limit → `SimpleTopN` - `no_count`: `SELECT f AS zo_sql_key FROM s GROUP BY zo_sql_key ORDER BY zo_sql_key ASC` + limit → `SimpleDistinct` - The `keyword` filter (`str_match_ignore_case`) is a valid index condition, so keyword-filtered values queries stay on the index path too. ### 3. Remove distinct values generation - Dropped the per-record distinct value collection in logs and traces ingestion and the whole `core::metadata` module (channel, mem-table, flush job, shutdown hook). - Removed the now-unused `DISTINCT_FIELDS` static and the `ZO_FEATURE_DISTINCT_EXTRA_FIELDS` / `ZO_DISTINCT_VALUES_INTERVAL` / `ZO_DISTINCT_VALUES_HOURLY` config knobs, including the `SUM(count)` merge rollup the last one gated — legacy `distinct_values_*` streams now compact as ordinary metadata streams. The default fields `DISTINCT_FIELDS` carried (`service_name`, `operation_name`) are already default secondary index fields via `SQL_SECONDARY_INDEX_SEARCH_FIELDS`, so traces `_values` queries on them remain index-served. - Renamed `values_v1` to `values_inner`: the `v2` variant was removed long ago (#5087) and the distinct stream switch inside `v1` is gone now too, so the version suffix was pure history. ### 4. Default fields after this change - `service_name`, `operation_name`: default secondary index fields (unchanged, via `SQL_SECONDARY_INDEX_SEARCH_FIELDS`). - `trace_id`: now also a built-in default **bloom filter** field (`BLOOM_FILTER_DEFAULT_FIELDS`, gated on `ZO_FEATURE_DEFAULT_INDEX_FIELDS_ENABLED`), in addition to being a default index field — bloom sits on top of the secondary index and prunes files for high-cardinality equality lookups. - There is no "default distinct fields" concept anymore. ### 5. Fix surfaced by e2e: SimpleDistinct returned null values as `""` The tantivy index encodes null/missing field values as empty strings (`index_builder` adds `""` for null docs), so the `SimpleDistinct` term-dictionary walk returned `""` as a distinct value — sorted first in ascending order, it surfaced as a blank first option in dashboard variable dropdowns, which the DataFusion `GROUP BY` path never returns (its null group sorts last). This was exposed once distinct fields became index-served; `handle_simple_distinct` now skips empty terms (with a unit test). ### 6. Drive-by: fix the failing `api_fuzz_tests` CI job `CreateAlertRequestSchema` was an untagged documentation union with two identical variants, producing `oneOf: [CreateAlertRequestBody, CreateAlertRequestBody]` in the OpenAPI spec. Under `oneOf` semantics (exactly one branch must match) no instance can ever validate, so schemathesis filtered out every generated example and `api_fuzz_tests` failed on `POST /api/v2/{org_id}/alerts` on every CI run since that schema landed. The handler deserializes `CreateAlertRequestBody` directly, so the annotation now documents exactly that type; runtime behavior is unchanged. ## What stays (intentionally) - `distinct_value_fields` / `enable_distinct_fields` in stream settings: still the user-facing way to declare values-API fields; the list now drives the read-time index-field union described above. - The `distinct_value_fields` DB table that tracks dashboard/report usage of these fields (still gates removal in the settings API). - `DISTINCT_STREAM_PREFIX` helpers, retention sync and stream-deletion cleanup for existing `distinct_values_*` streams — legacy stream data keeps compacting (as ordinary metadata streams) and ages out via retention; no new records are written to them. ## Testing - `cargo check --workspace --all-targets` and `cargo clippy --workspace --all-targets` clean. - Enterprise build verified with `Cargo.toml.openobserve` (`cargo check --all-targets`). - New unit tests for the index-field union (skip rules for FTS/partition/reserved fields, `enable_distinct_fields` gating) and the `added_ts` / release-cutoff indexed-since resolution; schema, infra, config and api-search unit tests pass. A few pre-existing failures on a clean tree (config `ssrf_guard`, one flaky sqlite-backed destinations test, one `.env`-dependent index-fields test) are local-environment issues and unrelated. Closes #13822 |
||
 Harsh Mahajan
|
7f74925605 | feat: banner as notification (#13667) | ||
|
Hengfei Yang
|
c246119809 |
fix(i18n): stop translated placeholder names crashing localized pages (#13833)
Fixes #13751 ## The crash vue-i18n compiles messages just-in-time (`__INTLIFY_JIT_COMPILATION__`), and a compile error is **thrown**, not warned — it tears down whatever component asked for the message. The translation pipeline had been localizing placeholder *names*, which is exactly that kind of error: ``` en-US "Are you sure you want to delete {identifier}?" zh-CN "您确定要删除 {标识符} 吗？" -> SyntaxError: 2 (INVALID_TOKEN_IN_PLACEHOLDER) ``` IAM → Service Accounts rendered blank in Chinese for this reason. The two zh-CN strings named in the issue had already been patched by hand in #13707, but the *class* was never swept. ## What was actually broken Compiling every message in every locale file with vue-i18n's own compiler found three groups: | | count | symptom | |---|---|---| | Does not compile | 132 | page blanks with `SyntaxError` — 8 locales: es, fr, it, ja, ko, nl, pt, tr | | Compiles, unknown name | 147 | `{Anzahl}` for `{count}` → value renders as an empty string | | Stale text, dropped placeholder | 5 | `{org}`, `{payer}`, `{endpoint}`, `{service}` — English source no longer has them | Worst of the first group are mangled vue-i18n literal escapes, e.g. `{'{'}team{'}'}` coming back as `{'{'} '}` — an unbalanced brace. All 279 are corrected here. The existing translated prose is kept verbatim; only the tokens are restored. The 5 stale ones are retranslated from the current English text, since restoring a token that no longer exists is not an option. ## Why it can't come back **`web/src/locales/enUsMessages.spec.ts` already did this — for en-US only.** That left the machine-translated locales, the actual source of the bug, uncovered. Renamed to `localeMessages.spec.ts` and extended to every locale, plus a second assertion that a translation only interpolates names en-US defines. That second check is a **subset** rule, not equality: dropping a placeholder the target language does not need stays legal (an English plural suffix like `{plural}` has no Japanese equivalent). Introducing a name en-US never passes a value for is always a bug. Placeholder names are collected by *running* the compiled message against a recording context rather than by pattern-matching the source, because only the compiler knows which braces are placeholders and which are literal escapes — `{'}'}` is one token whose body is the character a `{...}` regex stops at. All plural branches are evaluated. **`scripts/translations/translator.py` — `_validate` had the same regex blind spot** (`{[^{}]*}`) and never checked that the output compiles at all, which is why these landed. It now tokenizes the way vue-i18n does and rejects anything vue-i18n would refuse, so the pipeline stops emitting them rather than CI merely catching them afterwards. Escape drift (`{'{'} rows {'} rows {'}`) is caught too, because literal escapes are now part of the compared token multiset. ## Verification - The Python scanner was cross-checked against vue-i18n's compiler over **all 170,055 messages** in the 15 locale files — identical compile verdicts and identical token sets, 0 divergences. - Injecting the original `{标识符}` string back into zh-CN fails the new spec; injecting a compiling-but-wrong `{bilv}` fails the second assertion. - `npx vitest run --environment jsdom --root src/ locales/` → 45 passed. `prettier --check` and `tsc` clean on the spec. - Only `en-US.json` is imported anywhere in `web/src`, so no existing test depends on the changed locale data. ## Not in scope A handful of translations are awkward but compile and bind correctly (e.g. ko-KR renders "ingestion limit" with the eating sense of the word). Left alone to keep this diff auditable as a mechanical token restoration. |
||
|
Prabhat Sharmaandgithub-actions[bot]
|
0dba67d638 |
feat: composite alerts (#13797)
Design at : alerts/composite_alerts/composite_alerts.md ## Summary - add composite alert graph persistence, evaluation, scheduling, lifecycle APIs, and deletion safeguards - add composite alert UI forms, preview, references, list/detail integration, and localization - add Rust, API, Vue, and Playwright contracts ## Verification - cargo fmt --all -- --check - cargo check -p openobserve-core - cargo check -p openobserve-api-management - npm run type-check:composite-alerts - npm run type-check:app - npm run lint:ci -- --quiet - design/token/style checks - targeted Rust/Vitest suites - browser QA against local alerts UI ## Follow-ups (intentionally deferred, documented in the spec) - `RunOutcome::Error`/`NotifyFailed` outcome records on composite delivery failure (§11.5) - Incident detail live-definition resolver for composites + "definition unavailable" retention (§9.6) - `alert_composite_*` observability metrics (§16) - SLO/stream delete-conflict 409 bodies with readable-vs-hidden parent split (§10.3) - Super-cluster startup preflight (§18) <img width="1650" height="979" alt="image" src="https://github.com/user-attachments/assets/3def5246-7bf7-48fe-9a42-c0ac08271ea9" /> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
 TessaIO
|
30b938a1ed |
feat: introduce compactor ZO_COMPACT_DELETE_FILES_DELAY_MINUTES (#13828)
Resolves #13819 Signed-off-by: TessaIO <ahmedgrati1999@gmail.com> |
||
|
Hengfei Yang
|
3fda083f51 |
feat(traces): add trace time index (#13827)
Design at: [https://github.com/openobserve/designs/blob/main/infrastructure/file-list/trace-time-index.md](https://github.com/openobserve/designs/blob/main/infrastructure/file-list/trace-time-index.md) ## Summary - add an hourly Metadata time index per trace stream to resolve a trace ID to its full time range - add external time-range and trace-details APIs, including bounded locate/expand behavior and standard search responses - integrate index generation into trace ingestion and preserve query visibility through the synchronous Metadata write path - synchronize retention and deletion for associated Metadata streams - reuse the shared trace stream permission check across trace APIs ## Why Trace detail lookup currently depends on a caller-provided time range, which can omit spans outside that window. The time index provides a bounded, best-effort range lookup for `stream + trace_id` without scanning the full trace stream. Closes #13823 ## Environment variables | Variable | Default | Effect | | --- | --- | --- | | `ZO_TRACE_TIME_INDEX_ENABLED` | `true` | Enables per-trace-stream Metadata indexes, index writes during trace ingestion, and indexed time-range lookup. Only traces ingested after the feature is enabled are indexed; there is no historical backfill. | | `ZO_QUERY_TIMEOUT` | 600 seconds | Existing setting reused as the cumulative Trace Timer locate/expand deadline and as the default timeout for the trace-details search. | When enabled, each traces stream can create a related Metadata stream named `trace_time_index_{stream}`. Its retention settings follow the source stream, and it is removed with the source stream. ## API changes All timestamp parameters and returned time ranges use microseconds. ### Get a trace time range ```http GET /api/{org_id}/{stream_name}/traces/{trace_id}/time_range ``` Query parameters: - `hint_ts` (optional): timestamp hint used to start the index lookup Returns the source stream, trace ID, and the resolved `start_time` / `end_time`. The `range` field is `null` when the index cannot resolve the trace. ### Get trace details ```http GET /api/{org_id}/{stream_name}/traces/{trace_id}/details ``` Query parameters: - `start_time` and `end_time` (optional, but must be supplied together): caller-provided range - `hint_ts` (optional): timestamp hint for the time-index lookup - `timeout` (optional): trace-details search timeout in seconds The endpoint unions the caller range with the indexed range, so it never narrows a caller-provided range. If neither the caller range nor the index can provide a range, it returns `400`. The response uses the standard search response and includes the effective range in `new_start_time` and `new_end_time`. ## Validation - `cargo test -p openobserve-api-search --lib` — 145 passed - `cargo test -p openobserve-core traces::time_index::tests --lib` — 7 passed - `cargo test -p infra db::trace_time_index::tests::test_coverage_start_cache -- --exact --nocapture` — passed - `cargo check -p openobserve-api-search` - `cd web && npm run type-check:app` - end-to-end ingestion and query validation with the data generator ## Validation notes - targeted ESLint could not start because the local web dependencies are missing `@eslint/css` - enterprise-only compilation could not be completed from this OSS checkout because the local enterprise overlay is unavailable |
||
|
ktx-akshay
|
bd28872ddc | test: improve stability of Alpha environment tests (#13817) | ||
|
|
dbee572722 |
fix: slo bugs (#13784)
closes https://github.com/openobserve/o2-enterprise/issues/2397 --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Omkar Kesarkhane
|
6927c2f335 |
fix(synthetics): collapse the variables panel by default (#13825)
## What The variables panel on the browser-test **Journey** step now starts collapsed. ## Why The journey is what that page is for, and the panel was taking a share of the width before the author had asked for it. The toolbar toggle sits right next to the journey and is labelled, so bringing the panel in is one click — and the splitter position is still shared with Configure, so nothing about the panel's own behaviour changes. ## Scope One line plus a comment. `variablesPanelOpen` is journey-only by design (the existing comment above it explains why it is not shared with Configure), so this cannot affect the Configure step. ```diff -const variablesPanelOpen = ref(true); +const variablesPanelOpen = ref(false); ``` ## Testing `npx vitest run src/views/synthetics/CreateBrowserTest.spec.ts` — 44 passing. Split out of a larger synthetics branch so it can land on its own. |
||
|
Harsh Mahajan
|
0ed5336bf1 | chore: adding HA mode for o2-ai agent (#13525) | ||
|
Huaijin
|
3901c548a2 |
perf(promql): use Utf8View for metric labels (#13796)
## Summary
- read PromQL metric label columns as Arrow `Utf8View` when the existing
`ZO_UTF8_VIEW_ENABLED` setting is enabled
- preserve `__hash__` and exemplars in their original representation
while retaining schema metadata
- make PromQL equality, negative, and regex matchers type-aware for
`Utf8View`; cast regex operands to `Utf8` to avoid the DataFusion 54
mixed-type rewrite issue
- support `StringViewArray` in series-label loading and keep
compatibility with the process-wide series-label cache
No PromQL-specific environment variable is introduced, and metrics
continue to use the existing global `ZO_BATCH_SIZE`.
## Performance
For the filtered histogram query below, a controlled local A/B using the
same release binary and identical settings except Utf8View measured P50
latency improving from 0.964s to 0.677s, about 29.8%:
```promql
histogram_quantile(0.9, sum by(le, path) (rate(codelab_api_request_duration_seconds_bucket{path="/api/service-1"}[5m])))
```
Test range was one hour with `step=15` and `use_cache=false`.
## Testing
- `cargo fmt --all -- --check`
- `cargo test -p promql --lib utils::tests` (10 passed)
- `cargo test -p promql --lib series_labels::tests` (11 passed)
- `cargo test -p search --lib
datafusion::exec::tests::test_metrics_query_schema` (2 passed)
- `cargo check -p promql-service -p promql`
|
||
|
ktx-vaidehi
|
e490a5c347 |
fix(logs): load the stream list on Visualize when no stream is selected (#13812)
Entering Visualize from the logs page with no stream selected left the
Stream dropdown on "No options found" — enabled, openable, and empty, so
there was no way to pick one.
PanelFieldList defers its initial fetch in edit mode until a stream is
set, exempting only "metrics". The Visualize tab ("build") needs the
same exemption: its select is the user's own, and the list is keyed on
stream_type alone, so waiting for a stream buys nothing.
Also arms the stream_type watcher at mount, so changing Stream Type with
a blank stream now reloads the list instead of doing nothing.
- https://github.com/openobserve/o2-enterprise/issues/2165
|
||
|
Hengfei Yang
|
6a2dfbb2b4 |
fix(compact): stop retention from deleting one extra day (#13804)
Fixes #13800 ## Problem With a 7 day retention the stream only kept ~6 days of data. From the issue: retention is 7 days, but the stream data range starts at `2026-08-07T00:00 UTC` while now is `2026-08-13T04:52 UTC` — the whole `2026-08-06` partition is gone. Retention deletes data at day granularity (`delete_by_date` wipes a full `[day, day+1)` partition), but the job generator stepped through the days with: ```rust while start < time_range.end { ... } // time_range.end == lifecycle_end ``` `lifecycle_end = now - retention_days` is a point in the middle of a day, so the day it falls into also matched the condition and got a delete job. Deleting that partition also removes everything between `lifecycle_end` and the end of that day, which is still inside the retention period. Effective retention became `(N-1, N]` days instead of `[N, N+1)`. This is a regression from #8058, which switched retention from one job per range to one job per day. Before that change the range end was formatted as `%Y-%m-%d`, so the boundary day was never included. The day-aligned fast path in `generate_jobs()` already assumes the boundary day is retained — it skips the stream when `doc_time_min >= retention_day_start` — so the two paths disagreed today. ## Fix Only generate delete jobs for days that lie fully before the retention boundary. Extracted the day splitting into `generate_deletion_dates()` so it can be unit tested without a database. Retained window goes back to `[N, N+1)` days: with a 7 day retention the stream now holds 7–8 days of data depending on the time of day, which is the expected behaviour for day-granular deletion. Day-aligned range ends (the ranges produced by extended retention splitting) are unaffected — `start + 1day <= end` and `start < end` agree there. ## Tests Three new unit tests for `generate_deletion_dates`: a mid-day boundary (the regression case), a day-aligned end, and ranges shorter than a day. ``` test retention::tests::test_generate_deletion_dates_keeps_the_day_of_the_retention_boundary ... ok test retention::tests::test_generate_deletion_dates_with_day_aligned_end ... ok test retention::tests::test_generate_deletion_dates_less_than_a_day ... ok ``` `cargo clippy -p compaction --tests` and `cargo fmt --check` are clean. `test_generate_retention_job` / `test_delete_all` fail locally on `main` as well (sqlite data dir missing in my checkout), unrelated to this change. ## Note on the `time_range_start >= time_range_end` guard The removed loop had a `continue` when the two formatted dates compared equal, but it could never fire: both dates come from the same `start` variable, formatted before and after `start += day_micros(1)`, and in UTC adding 24h always advances the calendar date. Simulating the old loop with the numbers from the issue (`lifecycle_end = 2026-08-06T04:52 UTC`) produces jobs for `08-04`, `08-05` **and `08-06`** — the guard never triggers. It was meaningful before #8058, where the two dates came from the two ends of the range (`format(range.start)` / `format(range.end)`) and could land on the same day — and where `format(lifecycle_end)` as the exclusive end is exactly what used to keep the boundary day alive. Second commit removes it as dead code. |