## What
Adds backfill support for pipelines whose only destination is a **remote
destination**.
## Why
Backfilling a remote-only pipeline failed at runtime:
`get_destination_stream_from_pipeline` only recognized local `Stream`
nodes, so a pipeline with only a `RemoteStream` destination returned "No
destination streams found". That error path **deleted the scheduler
trigger** while keeping the `backfill_jobs` config row, which then
surfaced as:
- `GET .../backfill/{job_id}` returning **404** (`DbError# key
<org>/backfill/<job_id> does not exist`)
- the backfill row **disappearing from the UI** on the next refresh
Remote destinations were never actually unsupported by execution —
`ExecutablePipeline::process_batch` (already called by the backfill
handler) routes remote records to the remote WAL. The pipeline was just
being rejected by an over-strict pre-execution gate before it could run.
## Changes
- `get_destination_stream_from_pipeline` now returns an empty list (not
an error) for a remote-only pipeline, and only errors when there is no
destination leaf of either kind. Remote records continue to be handled
inside `process_batch`.
- Added a defensive guard so `delete_before_backfill` is skipped (with a
clear log) when a pipeline has no local destination streams — the
edit-after-create case. Creation already rejects
`delete_before_backfill` for remote pipelines with a user-facing
message.
- Added unit tests for remote-only (→ `Ok(empty)`) and mixed
local+remote (→ locals only) pipelines.
## Testing
- `cargo test -p openobserve get_destination_stream` → 8 passed.
- ⚠️ Enterprise build not compiled locally (private `o2_enterprise`
crate unavailable). The remote-WAL write path should be smoke-tested on
an enterprise build with a real remote-only pipeline backfill before
merge.
## Out of scope
The pre-existing trigger-orphaning inconsistency (get 404s while list
silently skips when a trigger is missing) is not addressed here.
OSelect's trigger lacked role="combobox", so a focused select wasn't
recognized as an input by the shortcut guard — pressing a single-letter
key (e.g. "s" on logs) fired the page-level shortcut instead of typing
into the select. ODialog's auto-focus also skipped selects entirely and
never re-focused after a body swap (e.g. Create/Update toggle), leaving
focus on a button where keystrokes leaked past the modal.
- OSelect: mark the trigger role="combobox" and add type-to-search so a
focused, closed trigger opens the dropdown and seeds the filter.
- ODialog: auto-focus falls back to a combobox trigger when there's no
text field, re-focuses the first field after body content swaps, and now
swallows any bare printable keydown that doesn't land in an input-like
element so page shortcuts can never fire while the dialog is open,
regardless of what has focus.
design at: cloud/org%20deletion/org-deletion-design.md
## Summary
Self-service cloud org deletion with a **recoverable grace period**,
followed by complete, retryable, auditable hard cleanup.
**Lifecycle:** `active → pending_deletion → deleting → gone`
- **Delete** soft-deletes into `pending_deletion` — data intact, org
hidden from the switcher + all API access blocked, `deleted_at`
recorded. With `grace=0` it deletes immediately (legacy path).
- A **promotion scheduler** flips expired pending orgs to `deleting`
after the grace window; the background **cleanup worker** then tears
down all data across object store, file_list, DB, OpenFGA, and billing —
per-step tracked and retried (`MAX_ATTEMPTS`).
- **Resurrect** (`_meta`/root only) flips `pending_deletion` back to
`active` — no data was ever touched.
## Key changes
- New `org_cleanup_tasks` ledger table +
`organizations.status`/`deleted_at` columns (2 migrations);
`DB_SCHEMA_VERSION` 47→48.
- `OrgStatus::PendingDeletion` cache + coordinator events + watch;
`is_blocked` middleware hides & blocks pending/deleting orgs.
- Grace window from enterprise `O2_ORG_DELETION_GRACE_PERIOD_DAYS`
(default 30), read from OSS behind `#[cfg]`.
- Cleanup worker + promotion scheduler run on **compactor** nodes;
dist-lock + atomic CAS make execution exactly-once cluster-wide. Task
rows purged immediately on hard-delete completion and on resurrect (no
retention sweep).
- Audit-trail entry to `_meta` on every status transition
(fire-and-forget).
- Resurrect endpoint (dual-impl, cloud-gated) + list surfaces
`deleted_at`/grace days.
- UI: pending-deletion badge + countdown + resurrect action,
deletion-progress dialog, Danger Zone restyle with type-to-confirm.
## RBAC
In-handler role checks (no OFGA middleware): **delete** = root OR org
Admin; **resurrect** = root only via `_meta`.
## Companion PR
Enterprise side: `openobserve/o2-enterprise` #2128 (config + ofga +
cloud billing).
## Testing
Unit suites pass; full flow live-verified end-to-end (soft-delete →
resurrect with data intact, grace=0 immediate delete, audit entries,
RBAC 403s).
> Requires ≥1 compactor node running for deletions to progress (always
true for single-binary / `ZO_NODE_ROLE=all`).
## Summary
Follow-up to #13109. Two aggregation-side bottlenecks showed up at high
cardinality (3.17M series), both single-threaded loops that scale with
series count:
1. **Parallel group signature computation** — `eval_aggregate` computed
each series' grouping signature (clone projected labels + hash)
sequentially: 1.17s of a ~3.8s `sum by(le, path)` query. Now fanned out
with rayon (~95ms); `group_series_by_labels` (used by `topk`/`bottomk`)
gets the same treatment.
2. **Parallel chunked accumulation for huge groups** — an aggregation
without a modifier (`sum(rate(m[5m]))`) puts every series into a single
group, so the "parallel per-group" aggregation ran on one thread: ~250ms
for 66M samples at a 15-min window, and 2-3s at a 1-hour window. Large
groups (≥64k series) are now split into 32k-series chunks aggregated in
parallel and merged via a new `Accumulate::merge` (implemented by all 9
accumulators; equivalence covered by a unit test).
Plus a refactor commit: `eval_aggregate` had reimplemented
`group_series_by_labels` inline just to keep per-series projected labels
it only needed once per group — both now share `projected_labels()`, and
a 3.17M-entry `(hash, Labels)` side vector (and its drop) disappears.
## Benchmarks
Same setup as #13109: single node (M-series Mac, 32GB), 1.34B rows /
3.17M series histogram bucket stream, result cache disabled, warm = mean
of runs 2-4.
**15-minute window:**
| Query | main (#13109) | this PR | speedup |
|---|---|---|---|
| `histogram_quantile(0.9, sum by(le,path)(rate(m[5m])))` | 3.81s |
3.02s | **1.26x** |
| `count by(path) (rate(m[5m]))` | 3.54s | 2.86s | **1.24x** |
| `sum by(region) (rate(m[5m]))` | 3.40s | 2.83s | **1.20x** |
| `max(rate(m[5m]))` | 2.60s | 2.34s | 1.11x |
| `sum(rate(m[5m]))` | 2.69s | 2.46s | 1.09x |
| `topk(5, rate(m[5m]))` | 10.37s | 10.39s | parity |
| selective filter (132k series) | 0.46s | 0.45s | parity |
**1-hour window (1.2B samples):** `sum(rate(m[5m]))` 12.1s → 8.6s
(**1.4x** — the single-threaded accumulation scales with samples, so the
fix matters more at longer ranges), `histogram_quantile` 13.0s → 11.8s,
`sum by(region)` 14.0s → 13.3s. Peak RSS unchanged.
Output verified against main: `max` bitwise identical; `sum`/`avg`
differ by ≤1.7e-11 relative (float summation order over 3.17M values;
run-to-run noise on main alone is ~1e-14).
## Tried and rejected (kept out of this PR)
Each was implemented, benchmarked A/B on the same dataset, and reverted
— recorded here so they aren't retried blindly:
- **Pre-reserving the partition-map merge**: zero effect — the merge is
hidden behind the slowest scan partition.
- **O(1) rate windows via a counter-reset prefix**: only -2~4%
end-to-end; the per-window reset rescan is cheap branch-predictable
compares, and the per-series prefix allocation ate the savings.
- **DataFusion `array_agg` for the sample load**: +17-55% slower — DF's
vectorized grouping is fast, but converting its Arrow list output into
engine structures is a second full materialization.
- **Sharded concurrent series map instead of `RepartitionExec`**:
+10-14% slower — the repartition copy is vectorized and buys each
consumer a purely local, cache-friendly map build.
Conclusion from the last two: the sample load (~60% of a 1-hour query)
is at a local optimum for the current row-at-a-time `Vec<Sample>` model;
further gains there need a columnar (SoA) sample layout, which is out of
scope for this PR.
## Notes
- Query-side only; no storage or config changes, no new env vars.
- `Accumulate` gains `merge`/`into_any`; all merges are
order-insensitive combines (add / min / max / union / concat).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Enrichment table queries (top k values and search) should ignore max
query range because enrichment tables, even stored as a stream in
backend, are not exactly logically a stream. It contains the data used
for enrichment like a csv file.
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
Bumps [ws](https://github.com/websockets/ws) from 7.5.10 to 7.5.11.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/websockets/ws/releases">ws's
releases</a>.</em></p>
<blockquote>
<h2>7.5.11</h2>
<h1>Bug fixes</h1>
<ul>
<li>Backported 2b2abd45 to the 7.x release line (e14c4586).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/websockets/ws/commit/fd36cd864fcdf62a08273a99e19a7d975401fee8"><code>fd36cd8</code></a>
[dist] 7.5.11</li>
<li><a
href="https://github.com/websockets/ws/commit/e14c45861deca0cef60dec0f9109b694abebdf52"><code>e14c458</code></a>
[security] Limit retained message parts</li>
<li>See full diff in <a
href="https://github.com/websockets/ws/compare/7.5.10...7.5.11">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/openobserve/openobserve/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Summary
This PR improves the Tantivy `SimpleSelect` path for queries using
`ORDER BY _timestamp ASC|DESC LIMIT n`.
Previously, each index file could return up to `limit` rows, causing
extra Parquet reads. The new path collects `(_timestamp, doc_id)`
candidates, safely stops when remaining file groups cannot produce
better results, and merges the global Top-N before building row
selections. On the fully indexed path, only the global `LIMIT`
candidates reach Parquet.
## Performance
https://github.com/openobserve/o2-ck-benchmark use this repo to generate
100M recores.
```sql
SELECT
*
FROM
k8s_logs
WHERE
kubernetes_pod_name = 'api-gateway-9a2c1e6f4-8923f'
ORDER BY _timestamp DESC
LIMIT 100
```
**main branch: 550ms
this branch: 250ms**
## Changes
- Add `SimpleSelectPruner` for safe early stopping, global Top-N
merging, and file/row pruning.
- Add timestamp-aware candidate results and cache entries.
- Fall back to full row-ID collection and DataFusion filtering when
Tantivy skips any condition.
- Conservatively retain unindexed, overlapping, or timestamp-tied files
unless they are provably dominated.
- Avoid caching results for files only partially covered by the query
time range.
- Separate SimpleSelect, SimpleCount, and generic row-count aggregation,
with minor query and scan-repartition cleanup.
## Correctness
- Supports both ascending and descending timestamp order.
- Uses strict timestamp bounds so ties are not pruned incorrectly.
- Keeps uncertain files when fewer than `limit` exact candidates are
available.
- No public API, configuration, or wire-format changes.
## Testing
- `cargo test --lib service::search::tantivy::` — 84 passed.
🤖 Auto-generated E2E tests for #11813 — **healed to passing** against a
live OpenObserve before this PR was opened.
- Branch `autoe2e/pr-11813` (stacked on `main`), so the tests run
against the feature code and pass before it merges.
- Spec:
`tests/ui-testing/playwright-tests/Logs/logsVisualizePersistence.spec.js`
The repo Playwright CI re-validates here. Review is for **quality +
coverage** — the tests already pass.
_Generated by the E2E Council pipeline._
**Test cases added / changed:**
| Test case | Type | What it verifies |
|---|---|---|
| `should persist live mode / auto-run toggle across navigation` |
extend | live-mode toggle state-guard added before navigation to catch
silent no-op when auto_query_enabled is false |
---------
Co-authored-by: Shrinath-O2 <214651540+Shrinath-O2@users.noreply.github.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
Auto-generated translation updates from the English source file
(`web/src/locales/languages/en.json`).
🤖 Generated with automated translation workflow
Co-authored-by: oasisk <35160958+oasisk@users.noreply.github.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
## What
Adds an **"Enable usage reporting" call-to-action** to the Cloud Billing
→ Usage tab for organizations that have self-usage reporting turned off,
a **"waiting for usage data" illustrated state** for orgs that have it
on but haven't reported any usage yet, and flips the per-org default for
the setting to **OFF** so the CTA is the entry point.
Previously, when `usage_stream_enabled` was off the Usage tab showed
only the billing-cycle cards and nothing where the daily chart would be.
And when it was on but the org's `usage` stream didn't exist yet, the
panel showed a raw `Search stream not found: usage` error. Now:
- **Reporting OFF** → an illustrated empty state (`OEmptyState`,
`wave-bars`) invites the user to enable reporting. Clicking **Enable**
opens a `ConfirmDialog` explaining the per-org, reversible effect before
the merge-and-POST enable flow runs.
- **Reporting ON, no data yet** → the raw "stream not found" error is
suppressed (on this panel only) and replaced with an illustrated
**"Waiting for usage data"** state. When usage lands, the search
succeeds, the error clears, and the daily chart shows through
automatically.
- **Reporting ON, has data** → unchanged: the daily line chart
(Ingestion + Search + Pipelines) renders.
## Behavior change (please review)
`default_usage_stream_enabled()` now returns **`false` in every build**
(previously cloud builds defaulted to ON when
`ZO_USAGE_REPORT_TO_OWN_ORG` was set). `ZO_USAGE_REPORT_TO_OWN_ORG`
remains the deploy-wide master switch but no longer flips the per-org
default. This is the intentional pairing with the CTA — an "enable"
prompt only makes sense when reporting is off by default. Explicit
stored values (opt-in or opt-out) still win in every build.
## How the waiting state is scoped
The "stream not found" suppression is set via `error_config` **only on
the usage panel's schema** (`usageDailyPanelSchema.ts`) —
`custom_error_handeling: true` with no `custom_error_message` silences
PanelSchemaRenderer's own error blocks. `usage.vue` catches the panel's
`@error`, detects the stream-not-found case specifically (unrelated
query errors still surface), and overlays the graphic. No other chart is
affected.
## Testing
- `web`: 417/417 billing specs pass, incl. new coverage for the enable
CTA, merged-settings POST, the schema silencing the renderer, and the
waiting overlay showing only for stream-not-found (not other errors).
- `cargo check -p openobserve` (non-release): clean.
## Note
These commits were cherry-picked onto a fresh branch from `main` — the
original branch (`chore/usage-ui-cloud-changes`) had already merged its
base work via #13112 and carried divergent, pre-squash history that
would have produced a conflicting diff.
## Summary
Fixes#13062.
On deployments served over plain HTTP (`isSecureContext = false`) the
Clipboard API is disabled, so `writeClipboard` falls back to
`document.execCommand("copy")`. The hidden textarea was appended to
`document.body` — outside reka-ui's `FocusScope` subtree — so the
drawer's focus trap immediately reclaimed focus, cleared the textarea
selection, and `execCommand` ran on an empty selection. Chrome returns
`true` for an empty-selection copy, so no error was thrown and the
"Content Copied Successfully!" toast appeared even though nothing was
copied.
## Changes (`web/src/utils/clipboard.ts`)
**1. Append inside the dialog, not `document.body`**
```ts
const activeEl = document.activeElement as HTMLElement | null;
const container: HTMLElement =
activeEl?.closest<HTMLElement>('[role="dialog"]') ?? document.body;
container.appendChild(textarea);
```
The temporary textarea is now inserted inside the nearest
`[role="dialog"]` ancestor (the drawer's `DialogContent`). The
`FocusScope` trap does not fire for focus changes that stay within its
own subtree, so the selection survives until `execCommand` runs.
**2. Gate the copy on focus still being held**
```ts
const focused = document.activeElement === textarea;
const copied = focused && document.execCommand("copy");
if (!copied) throw new Error("execCommand copy failed");
```
If focus was reclaimed (edge case) the copy is a known no-op; we throw
so `copyToClipboard` shows an error toast instead of a false-success.
**3. Fall through from `navigator.clipboard` on permission error**
The `navigator.clipboard.writeText()` call is now wrapped in its own
`try/catch` so a `NotAllowedError` (permission denied) falls through to
the `execCommand` path rather than propagating as an unhandled
rejection.
## Test plan
- [ ] HTTP deployment (non-secure context): open Source Details drawer →
click "Copy to clipboard" → paste confirms content is copied; no
false-success toast on failure
- [ ] HTTPS / localhost: `navigator.clipboard` path still used;
behaviour unchanged
- [ ] Deny clipboard permission in browser settings on HTTPS: falls
through to `execCommand` path gracefully
Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
## Summary
Fixes the "pin dashboard to Home" feature going stale when the pinned
dashboard is **moved** or **deleted**.
- **o2-enterprise#2139** (P1) — pin a dashboard, delete it, click the
pin → **404**.
- **o2-enterprise#2140** (P2) — pin a dashboard in Folder B, move it to
Folder A, click the pin → navigates to the **stale folder**.
### Root cause
The pin is stored as an org setting `home_dashboard` holding a static
snapshot `{ dashboardId, folderId, label }`, written once at pin-time
and never reconciled with the dashboard's live state. Both issues are
instances of the same class: a stored reference that drifts out of sync
on move/delete/rename.
## Approach — defense in depth
**L1 — authoritative server-side reconciliation (root fix).**
`move_dashboard` / `delete_dashboard` (OSS
`src/service/dashboards/mod.rs`) now keep the `home_dashboard` org
setting in sync at the mutation source:
- **move** → update the stored `folderId` to the destination folder.
- **delete** → clear the setting.
This is authoritative across every client and API caller (incl. **bulk
delete**, which loops `delete_dashboard`). Reconciliation is
**best-effort** — it runs *after* the primary op succeeds and
logs-and-swallows any error, so it can never fail the move/delete. No
`folder_id` is added to the GET response (no folder-membership info
leak).
**L2 — read-side re-sync (cross-system convergence).** The FE re-reads
the now-authoritative setting when the pinned tab is opened and when the
Dashboards page mounts, so a snapshot that went stale on **another
system** self-corrects before navigation.
**L3 — graceful degradation.** A deleted pin clears + toasts instead of
routing to a 404 (both the Home pinned tab and the Dashboards shortcut
button). Deliberate unpin (× button) stays silent.
## Deferred (follow-ups, out of scope)
- **Super-cluster cross-region reconciliation** — the replication queue
consumer (`src/super_cluster_queue/dashboards.rs`) applies moves/deletes
via table ops directly (bypassing the service layer where L1 lives), and
`system_settings` is not super-cluster-replicated. In a multi-region
deployment the non-originating region's pin isn't reconciled on **move**
(delete still converges via L3). Follow-up: add the same reconcile calls
to the queue consumer.
- **403-on-move (RBAC)** — moving a pin into a folder the current user
can't access. L3 handles 404 only for now; whether to clear the org-wide
setting vs. hide per-user is a separate design.
## Testing
- **Backend:** 6 new unit tests for the reconcile decision logic
(match/no-match, folder-changed, malformed value). `cargo test -p
openobserve home_pin` ✅
- **Frontend:** extended `HomeView.spec.ts` — re-sync on tab activation,
error toast on unavailable, silent deliberate unpin. `HomeView` +
`PinnedDashboardTab` specs: **40 passing** ✅
- **rcheck:** `cargo fmt --all --check` + `cargo clippy -- -D warnings`
clean ✅
- The `openHomeDashboard` shortcut guard is a small guard mirroring the
already-tested `PinnedDashboardTab` unavailable path; it's covered by
manual repro rather than a brittle full-mount of the 1400-line
`Dashboards.vue`.
### Manual reproduction
- **#2140:** pin in B → move to A → pin opens in **A** (correct
URL/breadcrumb); setting shows `folderId: "A"`.
- **#2139:** pin → delete → click pin/shortcut → no 404, toast,
pin/tab/shortcut gone; setting removed.
- **Cross-system:** move on client B while client A sits on Home/AI tab
→ A opens the pinned tab → resolves to the new folder.
## Layer compliance
All changes are OSS and unconditional (no enterprise gating, no
migrations, no new HTTP handlers) — `home_dashboard` is a generic system
setting readable by both builds. Reconciliation lives in the service
layer and reuses `src/service/db/system_settings.rs`.
## What
Adds a **daily usage view** to Cloud Billing → Usage for orgs with usage
reporting enabled, layered on top of the existing billing-cycle cards.
Cloud-only.
### Frontend (cloud billing components)
- **Merged into the existing Usage tab** (no separate tab): the billable
usage cards are always shown; when self-usage reporting is on, a
dashboards **date-range picker** (in the toolbar next to the GB/MB
toggle) drives **both** the cards and a daily chart.
- **Daily line chart** — a single query via `PanelSchemaRenderer` over
the org's own `usage` stream, broken down by event into **Ingestion /
Search / Pipeline / Remote Pipeline** lines (connect-nulls on).
- **Six billable usage tiles** on one row with **automatic unit
scaling** (GB→TB→PB / MB→GB→…) so large numbers stay readable; AI
Credits shown as a count.
- Cards re-fetch over the calendar range (derived `<N>days`, hardened
against bad values).
### Backend (all cloud-gated)
- `usage_stream_enabled` **defaults to `ZO_USAGE_REPORT_TO_OWN_ORG`** in
cloud builds, so orgs report to their own `usage` stream by default; a
per-org opt-out persists an explicit `false`. OSS / self-hosted default
stays `false`.
- **Reserved self-reporting streams** (`usage`, `stats`, `triggers`,
`errors`, `data_retention_usage`) are blocked from user **create /
ingest / delete** — cloud-only; the internal self-reporting writer
(`IngestionRequest::Usage`) is exempt.
- `get_org_usage`: a **missing `usage` / `data_retention_usage` stream
is tolerated** (treated as zero) instead of failing the whole request
with a 400.
## Notes
- No enterprise-repo or DB-migration changes; backend logic stays in OSS
behind `#[cfg(feature = "cloud")]`, per the OSS/Enterprise layer map.
- `Cargo.toml` / `Cargo.lock` are intentionally **not** included (local
`downloadO2.sh` dev-setup).
## Testing
- `cargo check` (OSS) and `cargo check --features cloud` — pass; `cargo
fmt` clean.
- Frontend: **411 billing unit tests** pass; `vue-tsc` typecheck clean;
production build succeeds.
1. Variable dropdown closes automatically after selecting "Select All"
in multi-select mode.
2. not able to unselect all.
3. for multi select dropdown, requires to click 2 times to close
dropdown list.
- #13102
## Logging: dev filtering + production strip
### Setup
1. Add to `web/.env` (git-ignored):
```ini
VITE_DEBUG_GROUPS=dashboards,auth:* # comma-separated, * wildcard
supported
```
2. Run `npm run dev` (editing `.env` auto-restarts Vite — just reload
the page).
3. Log with the feature group as the first argument:
```ts
console.debug('dashboards', 'panel refreshed', panelId);
```
4. In DevTools, enable the **Verbose** level to see `console.debug`.
### Default behavior (`VITE_DEBUG_GROUPS` unset)
- **Dev:** all `console.log/debug/info` are hidden — logging is opt-in
per feature group.
- **Production build:** `console.log/debug/info` are always stripped
from the bundle, regardless of env.
- `console.error` / `console.warn` are never filtered or stripped.
Design At: designs/ui/feature-group-console-logging.md
## What changed
- **Removed** the native `scrollIntoView()` call from `SpanBlock.vue`'s
`onMounted` hook, which was fighting the user's manual scroll in a
virtualized list.
- **Added** `cancelScroll()` to `TraceTree.vue` — clears pending
timeouts and aborts the TanStack virtualizer retry loop by resetting
`currentScrollToIndex`.
- **Wired cancellations** in `TraceDetails.vue` — calls
`traceTreeRef.value?.cancelScroll?.()` when the sidebar closes, trace
details reset, or a new trace is loaded.
- **Added `onBeforeUnmount`** cleanup in `TraceTree.vue` so a queued
scroll can't fire against a torn-down virtualizer.
- **Added regression tests** in `SpanBlock.spec.ts`,
`TraceTree.spec.ts`, and `TraceDetails.spec.ts` covering all
scroll-cancellation paths.
## Why
When `TraceDetails.vue` is opened with a `span_id` query parameter,
`SpanBlock.vue` called `element.scrollIntoView()` on mount. Because rows
are virtualized via `@tanstack/vue-virtual`, a `SpanBlock` remounts
every time it scrolls into the viewport — so each remount snapped the
viewport back to the pre-selected span, fighting the user's manual
scroll. The issue only manifested when a `span_id` was pre-selected at
page load.
## What changed
- **`web/src/plugins/traces/SearchBar.vue`**: Fixed the Run Query button
by changing `variant="ghost"` to `variant="primary"`, restoring its
primary action styling (matches Logs SearchBar behavior).
- **`web/src/components/TenstackTable.vue`**: Fixed skeleton loader
background color to match the logs version. Added
`bg-(--o2-log-table-row-bg)` and `border-(--o2-log-table-row-border)`
tokens to skeleton rows. Replaced inline shimmer gradients with a
reusable `.o2-skel-pill` CSS class that uses OTable skeleton token
variables (`--color-skeleton-base`, `--color-skeleton-highlight`).
## Why
The Traces Run Query button had lost its primary styling and appeared
visually broken — it was rendering as a ghost button instead of the
prominent primary action button. The generic `TenstackTable.vue`
skeleton loaders had a different background color than the logs-specific
`TenstackTable.vue`, causing visual inconsistency.
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
## Reduce `Logs-Builder-Basic` e2e shard runtime — ~10 min → ~5 min
### 🎯 Result (measured on CI)
| Metric | Before | After | Saved |
|--------|--------|-------|-------|
| **`Logs-Builder-Basic` shard** | **10m 05s** | **5m 12s** | **~4m 53s
(≈48% faster)** |
The shard runtime is cut roughly in half. Most of the old time was fixed
sleeps, waits that always ran their full timeout, and duplicated
per-test setup — not actual test logic. **No test cases, steps, or
assertions were removed.**
### What we changed
| # | Area | Before | After | Saving |
|---|------|--------|-------|--------|
| 1 | `initQueryBuilderTest` setup | Logs page loaded twice (helper
`goto` + `selectStream` navigates again) | Navigate once | 1 page load ×
~41 tests |
| 2 | `waitForBuildTabLoaded` | Always waited up to 10s for editor to
fill, even when it's meant to be empty | Skip when SQL mode is off |
~10s dead wait on many tests |
| 3 | `enableSqlModeIfNeeded` | 2s watcher wait + 500ms sleep, even
where the watcher never fires | Gated to Build tab; waits on real signal
| ~2.5s × ~35 tests |
| 4 | `disableSqlModeIfNeeded` | Fixed 500–600ms sleeps | Deterministic
wait on editor update | Blind sleeps removed |
| 5 | `applyQueryButton` | Clicked Run blindly | Waits for button ready
first | Fewer early-click retries |
| 6 | `pagination.spec` | Fresh stream ingest + index-wait per test (9×)
| One ingest per worker, reused | 8 redundant ingests + waits |
| 7 | `pagination.spec` | Dead `waitForTimeout(2000)` per streaming test
| Removed | ~2s × 5 tests |
| 8 | `unflattened.spec` | Search run twice with 3s + 1s sleeps | Single
run | ~5s × ~8 calls |
| 9 | `unflattened` row scan | Fixed 3s sleep per row | Wait for content
to render | 15-row scan: ~97s → seconds |
| 10 | `unflattened.spec` | Blind sleep after ingest | Poll until stream
queryable | No over-waiting |
### Bonus
Items 2–5 live in the shared `logsPage` page object, so
`Logs-Builder-Advanced`, `Logs-Core`, and `Logs-Features` shards get
faster for free too.
### Notes
| Item | Detail |
|------|--------|
| `playwright.yml` | Unchanged — shard layout stays as-is |
| Assertions | Preserved everywhere (e.g. pagination still checks the
same record counts) |
| Validation | All 52 tests still discovered by `playwright --list`;
shard confirmed green on CI at 5m 12s |
## Summary
Fast fields are used by search optimizations such as timestamp
filtering, histograms, multi-histograms, and top-N queries. Previously,
`build_footer_cache` only warmed indexed field metadata, so fast field
access could still trigger additional reads later. This change primes
fast field handles during footer cache construction so those reads can
be served from the cached directory.
## Changes
- Warm fast field handles while building the puffin footer cache.
- Refactor `warm_up_terms` to operate on a `SegmentReader`.
- Update the Tantivy search warm-up call site for the new
`warm_up_terms` signature.
- Add a unit test to verify footer cache generation includes fast field
cache entries.
## Testing
- Added `test_build_footer_cache_with_fast_field`.
- Ran formatting and clippy fixes.
## Summary
- Skip OTLP histogram `_sum`, `_min`, and `_max` records when the
corresponding optional stat is absent.
- Preserve existing `_count` and bucket records.
- Add coverage for histograms whose optional stats are missing.
## Root Cause
OTLP `HistogramDataPoint` defines `sum`, `min`, and `max` as optional
values. The previous ingestion path converted missing optional stats
into JSON `null` and still emitted derived metric records such as
`<metric>_min` and `<metric>_max`. The flattening layer drops `null`
fields, so those derived streams could be created without a `value`
field.
Fixes#13019.
## Validation
- `cargo test -p openobserve test_process_hist_data_point --quiet`
- `cargo test -p openobserve
otlp::tests::metric_name_processing_tests::test_histogram_metric_name_suffixes
--quiet`
- `git diff --check`
## What
Lets an org pin a single dashboard as a tab on the **Home** page for the
whole organization. The chosen dashboard is stored server-side (per org)
so it persists across devices/browsers, and every member of the org sees
the same home tab.
## How
- **New `useHomeDashboard` composable** — single-value, server-backed
store using the existing `settings/v2` KV API (`setting_key:
"home_dashboard"`, org scope). Optimistic writes with revert-on-failure
(403 → permission toast), a falsy-org guard, and label auto-sync that
persists.
- **Loads in the existing `getOrganizationSettings()` hook** (boot + org
switch), so it is naturally **org-scoped** — switching orgs shows that
org's home dashboard, with no cross-org leak.
- **HomeView** renders the single home tab (read-only
`PinnedDashboardTab` embedding `RenderDashboardCharts` with its own time
picker); the tab × removes it.
- **Set / remove** from the dashboard list ⋮ overflow menu and the
ViewDashboard header button, with an at-a-glance indicator on the home
dashboard.
## Scope
- **Frontend only — no backend changes.** Reuses the existing
`system_settings` table + `settings/v2` handler + its RBAC gate.
- Single value (max 1); pinning a new dashboard replaces the previous
one.
- Permissions are enforced by the settings API (a 403 reverts the
optimistic change + toasts) — no client-side RBAC logic.
## Testing
- **74 unit tests**: composable (optimistic/revert/403/falsy-org
guard/label persist), HomeView (single tab render, ×
pointerdown-activation regression), PinnedDashboardTab (time/panel
contracts, µs time shape, tab-id provide).
- **Verified live against a local server**: set → server persists →
**fresh browser context with no localStorage** renders the tab → replace
swaps the single value → remove clears it. **Org isolation confirmed**
(`default` vs `demo` resolve independent values).
- 0 type errors in touched files.
Single squashed commit; 10 files, +1154/-11.
---------
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Auto-generated translation updates from the English source file
(`web/src/locales/languages/en.json`).
🤖 Generated with automated translation workflow
Co-authored-by: github-merge-queue[bot] <118344674+github-merge-queue[bot]@users.noreply.github.com>
The two metrics were using an empty string as the label name even though
no label names were registered in the metrics. Hence, whenever this
metric is emitted, there is a runtime panic. This pr fixes that.
Auto-generated translation updates from the English source file
(`web/src/locales/languages/en.json`).
🤖 Generated with automated translation workflow
Co-authored-by: github-merge-queue[bot] <118344674+github-merge-queue[bot]@users.noreply.github.com>
## Scheduler: per-module job pullers with dedicated worker pools (A3 +
A4)
### Why
Scheduled jobs are dispatched late at scale. A single shared puller uses
one FIFO, one cadence, and one worker pool across every module, so a
backlog or slow handler in one module (e.g. an alert burst or a bulk
backfill) starves all others. This is Part A of the scheduler
scalability design.
### What
Give **each `TriggerModule` its own pull loop, cadence, LIMIT budget,
channel, and worker pool**, behind a flag. Off by default — zero
behavior change until explicitly enabled.
- **`ZO_SCHEDULER_PER_MODULE_PULLERS`** (default `false`) — master
switch.
- **off** → single shared puller (legacy, byte-identical SQL + same
global advisory lock).
- **on** → one lane per module (Alert, Report, DerivedStream, Backfill,
AnomalyDetection, QueryRecommendations).
- **Module-filtered pull** — `pull(...)` gains `module:
Option<TriggerModule>`; when `Some(m)` it filters `WHERE module = m` and
takes a **per-module advisory lock** (`scheduler_pull_lock:{module}`,
strategy C3) so modules no longer serialize against each other. `None`
keeps the exact legacy query + global lock.
- **Per-module budgets/cadence** — `ZO_SCHEDULER_<MODULE>_CONCURRENCY` /
`_INTERVAL` env vars, `0` = inherit the global alert default. Backfill
defaults to the smallest budget (1) so bulk jobs can't crowd out
latency-sensitive modules.
### Changes
| Area | Files |
|---|---|
| Flag + per-module config | `src/config/src/config.rs` |
| `module` param on `pull` (trait + wrappers) |
`src/infra/src/scheduler/{mod,postgres,sqlite}.rs`,
`src/service/db/scheduler.rs` |
| Per-module lanes (puller + pool + channel) |
`src/service/alerts/scheduler/worker.rs` |
### Notes
- No new SQL statement variants beyond two (module-filtered / legacy);
bind ordinals preserved, `is_realtime/is_silenced` exclusion unchanged.
- The keep-alive task and channel-full back-pressure retry are preserved
verbatim.
- Deferred: per-module Prometheus metrics (A6) and `FOR UPDATE SKIP
LOCKED` benchmark (A2/C2).
### Rollout
Flip the flag **cluster-wide**. During a mixed on/off window the
per-module and global locks don't mutually exclude, but the atomic
`UPDATE ... WHERE status=Waiting ... RETURNING` still claims each row
once (no double-dispatch) — only transient loss of cross-node FIFO. Size
the Postgres/sqlx pool for the extra pullers before enabling at scale.
### Testing
- 11 unit tests in `worker.rs`: legacy single-lane path,
one-lane-per-module, concurrency floor, and full
`resolve_module_configs` coverage (inherit/override/backfill floor).
- `cargo fmt --all -- --check` clean · `cargo clippy -- -D warnings`
clean.
Subhra264
ktx-mihirandAbhay Padamani
ktx-vaidehi
Ashish Kolhe
Bhargav
Hengfei YangandClaude Fable 5
Huaijin
dependabot[bot]
Simran KumariandHengfei Yang
Omkar Kesarkhane
udev
Alexander peczon
ktx-akshay
