mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-06 00:09:22 +00:00
ee71df7ce73cec00a9dba13dbe4b2e910e16ebfc
1064
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
18a53716f6 |
test(traces): heal service-graph Metrics-tab correlation detection (#13674)
## What Fixes the one remaining alpha1 Traces failure after the service-graph route deploy: **`service-graph.spec.js` → P1: Show telemetry correlation from node panel via Metrics tab**. Test-only, no product change. ## Root cause The test asserted on two hooks that never reach the DOM: - `service-graph-side-panel-metrics` sits on an `OTabPanel` whose `data-test` is **swallowed by `<Transition>`**. - `service-graph-side-panel-metrics-dashboard` is passed to `<TelemetryCorrelationDashboard>`, which has **multiple top-level roots** (dialog `ODrawer` / embedded `div` / `ODialog`) → it renders a **Vue fragment**, and Vue 3 **drops fallthrough attrs on fragment roots**, so the `data-test` is never applied. On alpha (which seeds only traces, no metrics), the correlation returns a **zero-stream dashboard object**: the dashboard branch renders (so the parent empty-state is suppressed), but there are no stream rows and the dashboard hook doesn't exist → the test's `else` branch `expect(hasError || hasEmpty)` fails. The test only ever passed before when correlation happened to return *empty*; it broke as soon as real correlation data appeared. Confirmed live on the deployed build: after clicking Metrics, the panel wrapper `data-test` is absent, while the correlation view's real content (`correlation-event-header-*`, metric selector) renders fine. ## Fix Wait on the correlation view's **real rendered content** — the correlation event header, or a terminal empty/error state — instead of the swallowed/dropped wrappers, and assert only that the Metrics tab **resolves** (the helper throws if nothing renders, catching a genuinely broken tab). Metric-stream rows are asserted only on the happy path when data is present. Deploy-independent: every element it waits on ships on the current build (verified), so it won't reintroduce the main-vs-deployed skew. ## Validation `--workers=1 --retries=0` against alpha on the previously-failing test: - 1 run → passed - `--repeat-each=2` → 2/2 passed `Metrics tab resolved: streamsPresent=false → resolved with no correlated streams (acceptable) → Test PASSED` ## Context The other 9 service-graph failures were the missing `/traces/service-graph` route (#13659) and cleared on deploying latest `main`. Sibling fixes from the same alpha1 triage: openobserve#13672 (alerts move flake), o2-enterprise#2370 (per-org cleanup). |
||
|
|
e1a265d7f0 |
test(alerts): heal Move-across-folders selection-wipe flake (#13672)
## What
Heals the recurring flake in `alerts-e2e-flow.spec.js:160` — **E2E:
Alert Folder Management - Move, Search, Verify** — which fails with
*"Move drawer failed to open after retries"* (the
`alert-list-move-across-folders-btn` never becomes visible). Also
hardens the sibling caller in `alerts-ui-operations.spec.js`.
**Test-only change. No `web/` product code is touched.**
## Root cause
`AlertList.vue`'s `getAlertsFn()` runs `selectedAlerts.value = []` at
the **start of every folder fetch**. The move / export / pause buttons
are `v-if="selectedAlerts.length > 0"`.
- Switching to a freshly-created (**uncached**) folder kicks off an
async `listByFolderId` fetch.
- `navigateToFolder` returns as soon as the folder tab is active and a
table is on screen — so `select-all` can fire **before** that fetch
resolves.
- When it resolves it **wipes the selection** → the move button
disappears → the drawer never opens.
- **Cached** folders serve synchronously with no wipe.
This precisely explains the observed signature:
- **Always failed attempt 1** (folder uncached), **passed attempt 2**
(retry warmed the cache).
- **Failed all 3 attempts under parallel load** — the slower API
guarantees the wipe lands after `select-all` every time.
`networkidle` doesn't catch it because streaming/websocket
(`ZO_STREAMING_ENABLED=true`) keeps connections open.
## The fix (`pages/alertsPages/alertBulkOperations.js`)
1. Before selecting, wait for the source folder's **own** row to render
(`expectAlertName`) — this proves the fetch resolved (wipe already
happened) and that we're on the right folder, not a stale table. Falls
back to "any row" when no name is passed (backward compatible).
2. Retry **select-all → click Move → assert drawer open** as **one
atomic `expect(...).toPass()`**. A fixed stability window isn't enough
(a second fetch can resolve later and wipe). The drawer is driven by its
own `showMoveAlertDialog` ref and captures the selection **at click
time**, so once open a later wipe cannot close it — the click just needs
to land while the selection is valid, which it always eventually does
once the folder is cached.
Both spec callers now pass `{ expectAlertName: alertName }`.
## Validation
Ran against the alpha cloud env on the exact failing scenario
(freshly-created uncached source folder, `--workers=1 --retries=0`):
- 1 initial run → **passed**
- `--repeat-each=3` → **3/3 passed**
`Move drawer opened with a valid selection` → `Moved all alerts to
target folder` on every run.
## Note
Not related to the concurrent Traces service-graph failures on alpha —
those are a separate main-vs-RC route skew (#13659 not in the deployed
RC build) and resolve on deploying latest `main`.
|
||
|
|
32d02d6781 |
chore(traces): make Service Graph and Services their own routes (#13659)
## What Service Graph and Services Catalog were tabs on the Traces page, switched via a `?tab=` query param on a single keep-alive route. They are now real routes, reached from a hover flyout on the Traces rail tile: | Route | Notes | |---|---| | `/traces` | Traces search — keeps the **Spans / Traces** toggle | | `/traces/service-graph` | enterprise-gated; redirects to Traces on OSS | | `/traces/services` | | **Spans stays an in-page toggle** on purpose. It's a view *granularity* that shares the query, stream and time range — you switch to it mid-investigation and expect context to carry over. Datadog's Trace Explorer does the same. Service Graph and Services are destinations; Datadog, New Relic and Grafana all treat them that way. ## Why this wasn't just cosmetic As tabs, all three views shared one `meta.title`, one history entry, and the traces page's `searchObj`. That last part is why Service Graph and Services Catalog inherited the traces page's selected stream — the coupling this PR removes structurally. ## UI Both pages now use the Agent Graph header pattern (`OPageLayout` + title/subtitle/icon, date picker and last-run refresh in `#actions`), with a subnav row carrying stream selector + search on the left and view-type + layout on the right. ## Handoff `view-traces` no longer mutates the shared store from a sibling tab. Both views build the same filter string — extracted verbatim into `viewTracesHandoff.ts` — and navigate to Traces with it as query params, so the URL survives a reload. (Traces then canonicalises `filter=` to its own base64 `query=` via existing code in `useTraces.ts:290`.) ## Two bugs this surfaced Both were things the tab views inherited for free and now have to set themselves — and both would have hit real users deep-linking these pages: - **`organizationIdentifier`** was only assigned inside the Traces page's `loadPageData()`. Without it the catalog issued requests to `/api//_search_stream` (empty org segment) and rendered nothing. - **`?period=` / `?from=`+`?to=`** were only applied by the Traces page's `restoreUrlQueryParams()`. Without it a shared link carrying a time range silently opened on the 15-minute default. ## Tests | Suite | Result | |---|---| | `Traces/service-graph.spec.js` | 14 / 14 | | `Traces/service-catalog.spec.js` | 10 / 10 | | `RegressionSet/Traces/traces-ent.spec.js` | 1 / 1 | | Unit (`Navbar` + `plugins/traces`) | 2514 pass | Also clean: `type-check`, `eslint` (0 errors), `lint:design:strict`. Test-side changes: navigation POMs go through the rail flyout instead of the removed toggles; the graph POM pins an explicit window, since on its own route the graph no longer inherits the traces page's range. The shared auth fixture now verifies against `navbar-main-nav` — it keyed off `menu-link-/-item`, which the current rail no longer renders, so suites failed setup even after a successful login. Filter-building coverage moved with the code into `viewTracesHandoff.spec.ts` (21 tests) rather than being deleted alongside the component tests that asserted the old in-page handlers. ## Known / follow-up - `RegressionSet/Traces/traces-regression.spec.js` has 2 failures (trace column sorting, pagination). **Pre-existing** — they fail identically against production on `main`, and the Service Catalog test in that file passes. - Both views still default their stream picker to the traces page's selected stream. Defaulting Service Graph and Services to **all streams** is a deliberate follow-up: on our own instance the `default` stream shows 22 topology nodes vs 71 across all streams, and services like `o2-ai` span 4 streams. Graph is a one-line change (the API already supports it); the catalog needs a backend decision, so it's out of scope here. |
||
|
|
fbe558b463 |
test(autoe2e): generated E2E tests for #13652 (stream-schema-timezone) (#13657)
🤖 Auto-generated E2E tests for #13652 — **healed to passing** against a live OpenObserve before this PR was opened. - Branch `autoe2e/pr-13652` (stacked on `main`), so the tests run against the feature code and pass before it merges. - Spec: `tests/ui-testing/playwright-tests/Logs/streamSchemaTimezone.spec.js` The repo Playwright CI re-validates here. Review is for **quality + coverage** — the tests already pass. _Generated by the E2E Council pipeline._ --------- Co-authored-by: Shrinath-O2 <214651540+Shrinath-O2@users.noreply.github.com> |
||
|
|
4d91f91a41 |
test(logs): fix alpha1 unflattened drawer-close detach flake (#13656)
## Problem On the **alpha1 (cloud)** e2e run, the **Logs-Core** shard failed and did not recover across all 3 retries. The single hard failure was: `Logs/unflattened.spec.js:220 › Unflattened testcases › stream to toggle store original data toggle and display o2 id` ### Root cause After switching to the **unflattened** tab, the test closed the log-detail drawer with a bare `closeDialog.waitFor()` + `closeDialog.click()`. Switching that tab re-renders the drawer, so the close-button node **detaches mid-click** — Playwright logs `element was detached from the DOM, retrying` and the click hangs for the full 45s, failing every retry identically. It's a deterministic race, which is why retries never helped. The secondary retry-failure signature (JSON detail panel lagging behind tab activation) is the same drawer instability, wider on the slower shared alpha env. ## Fix - Close the drawer via the file's existing resilient `closeLogDetailDrawerIfOpen()` helper (Escape fallback + wait-for-hidden) in **both** the failing test and its identical quick-mode sibling (`unflattened.spec.js` lines ~283 and ~426). - Harden `openJsonDetailTab()` (`pages/logsPages/unflattened.js`) to re-assert the JSON tab and re-wait once when the panel content lags, then fall back to the original throwing wait so a genuine non-render still fails loudly. Test-only change; no product code touched. ## Verification Seeded the **Run E2E Tests on Alpha1 (Cloud)** workflow against this branch (`o2_opensource_repo=test/alpha1-unflatten-drawer-close-flake`, enterprise on `main`): [run 31067277452](https://github.com/openobserve/o2-enterprise/actions/runs/31067277452) — Logs-Core and the overall run passed green. |
||
|
|
58c7dafa55 |
fix: notification templates v2 — rich content templates + alert chart images (#13640)
## Summary
Two coupled alerting features:
**1. Notification Templates v2 ("content" templates)**
- New `kind: content` template type: a structured spec (title, body
markdown, fields, links, rows, per-channel title overrides,
severity-filtered visibility) rendered natively per channel — existing
`custom` (raw body) templates are untouched and byte-identical on the
send path.
- Per-channel renderers: Slack Block Kit, Teams (Adaptive Card +
MessageCard), Discord embeds, PagerDuty Events v2, Opsgenie, ServiceNow,
SNS, HTML/text email, and a stable machine `webhook` envelope.
- Escaping is renderer-owned (mrkdwn entities, HTML encoding, markdown
stripping) with a shared URL-scheme allowlist; golden-fixture tests per
channel.
- Template editor UI: guided content form (fields/links row tables,
compact rows config, channel title overrides), live preview panel with
per-channel preview + payload tab, test-send (rate-limited,
`[TEST]`-marked).
- Migration `m20260802_000001_add_template_kind`; prebuilt default
template + org backfill.
**2. Alert chart images (stateless, storage-free)**
- Alerts can attach a metric-history chart PNG rendered from the
triggers self-reporting stream (`actual_value` per evaluation), with
warn/crit threshold lines and the firing point appended.
- Zero storage: the chart URL *is* the data — deflate+base64url payload
signed with truncated HMAC-SHA256 (org-bound, expiry embedded), rendered
on fetch by the HTTP-terminating node (`GET
/api/v2/{org}/alerts/charts/render`). Signature verified before inflate
(decompression-bomb guard), uniform 404s, per-node render semaphore +
per-minute query budget + small LRU.
- Renderer: plotters + embedded DejaVu Sans (Bitstream Vera license
included), gap-broken line segments so sparse evaluation history doesn't
draw false plateaus.
- Channel delivery: Slack image block, email inline CID (no remote
fetch), Discord multipart attachment, Teams/PagerDuty image URLs;
per-destination opt-in via template chart flag.
- Slack layout live-A/B'd on a real channel: content + buttons/chart
split across two same-colored attachments so the severity stripe spans
everything while staying under Slack's "Show more" fold; clickable link
title; "View in OpenObserve" keeps a reserved primary-button slot.
- Slack image-delivery fallback: Slack validates image URLs server-side
and rejects the ENTIRE message with `400 invalid_attachments` when its
proxy can't fetch the chart (VPN-only/private `ZO_WEB_URL`) —
live-verified. On that rejection the send strips the image blocks and
resends once (alert always delivers), sets a process-wide flag so later
sends pre-strip (reachability is deployment-wide, not per-destination),
and re-probes hourly. Zero config; observable via `slack_image_rejected`
/ `slack_image_pre_stripped` metric events.
## Testing
- 147 notification unit tests (resolve/render/preview/test-send incl.
per-channel goldens and hostile-input/injection coverage), 8 chart
payload tests, renderer PNG tests.
- Live-verified end to end: Slack (Slackbot chart fetches ~100ms,
cache-immutable semantics), SES email inline image, multi-severity
firing scenario (warning→critical) with correct threshold lines.
- FE: AddTemplate/ContentTemplateForm/preview specs; UX audit pass with
fixes.
- `cargo clippy -D warnings` clean in both OSS and enterprise
arrangements.
## Enterprise pairing
Must merge together with openobserve/o2-enterprise#2353 (workspace
twin-manifest mirror for the new OSS deps).
|
||
|
|
13acdd61da |
fix(web): hide LLM Model Pricing settings on OSS builds (#13651)
## What Hide **LLM Model Pricing** on OSS builds and make it an enterprise/cloud-only feature, end to end (UI + e2e coverage). Paired with o2-enterprise PR on branch `fix/hide-llm-model-pricing-oss`. ## Why Model Pricing was surfaced on OSS (settings rail + navbar flyout), gated only on the `model_pricing_enabled` backend flag with **no** enterprise/cloud check — unlike its sibling AI feature **LLM Providers** (`(isEnterprise || isCloud) && online_evals_enabled`). Product decision: it should not appear on OSS. This brings it in line with that sibling. ## Changes **Web (hide on OSS)** - `web/src/components/settings/index.vue` — settings rail item now `visible: (isEnt || isCloud) && !!z.model_pricing_enabled`. - `web/src/lib/core/Navbar/navGroups.ts` — `modelPricing` gate predicate now requires enterprise/cloud (mirrors `llmProviders`), so the collapsed-navbar flyout matches the rail. - `web/src/composables/shared/useManagementRoutes.ts` — `model_pricing` + `model_pricing/edit` routes registered only inside the `isEnterprise || isCloud` block (defense-in-depth; matches `llm_providers`), so the route isn't reachable by direct URL on OSS. **E2E (move coverage to ENT)** - `tests/ui-testing/ci-matrix/ci_matrix.json` — drop `model-pricing.spec.js` from the OSS `GeneralTests` shard. It's added back on ENT via the enterprise overlay (`ci_matrix.ent.json`, in the paired ENT PR), so the merged ENT matrix runs it exactly once (verified with `build-ci-matrix.js`). - `.github/workflows/playwright.yml` — remove `ZO_MODEL_PRICING_ENABLED: true` (OSS no longer exercises the feature; ENT `playwright.yml` keeps it). - `tests/ui-testing/playwright-tests/GeneralTests/settings-form-validation.spec.js` — gate the *ModelPricingEditor* describe block to a clean SKIP on OSS, mirroring the existing Domain Management / Org Storage / Correlation blocks. - `tests/ui-testing/pages/generalPages/settingsFormValidationPage.js` — `navigateToModelPricing()` now returns a boolean so the suite can detect the OSS build and skip (mirrors `navigateToDomainManagement()`). ## Not changed (deliberately) The backend `ZO_MODEL_PRICING_ENABLED` default (`true`, [config.rs](src/config/src/config.rs)) is **left as-is**. It lives in the shared config crate used by both editions, so flipping it to `false` would also regress real ENT/Cloud deployments (feature off by default). The UI gate already ensures the page never shows on OSS regardless of the flag. Making the backend itself enterprise-only would be a separate, backend-owned change. ## Behavior | Build | Model Pricing UI | e2e | |-------|------------------|-----| | OSS | Hidden | `model-pricing.spec.js` not run; settings-form-validation block SKIPPED | | Enterprise / Cloud | Shown (if flag set) — unchanged | `model-pricing.spec.js` runs (via ENT overlay) | |
||
|
|
98e03bca15 |
test(e2e/alpha1): stabilize the alpha1 cloud suite — auth self-heal, infra, and per-test heals (#13626)
## Stabilize the Alpha1 (cloud) e2e suite + regression carryover Root-caused and fixed the failures surfacing on the Alpha1 cloud runs — investigated live via Playwright MCP against alpha and, where possible, a local build. Grouped by theme. Every fix is non-masking (still fails if the underlying is genuinely broken). Pairs with ENT #2351 (`max-parallel` + Logs-SelectStar `quick_mode`). ### Runtime / infra resilience (whole-suite) - **Auth self-heal on 401 / session expiry** — the shared cloud session + per-org passcode are minted once and reused all run; on load/contention they expire mid-run (`User not authenticated`, `HTTP 401 Unauthorized`). `cloud-auth.authedRequest` refreshes the passcode and retries on 401 (escalating to a full Dex re-login if the session is dead); `navigateToBase` re-authenticates and resumes instead of throwing. Also routed the alert **template-creation** API through it (401 there cascaded into "Failed to navigate to templates"). Gated on `isCloudEnvironment()` — no self-hosted change. - **`--disable-dev-shm-usage`** on the alpha1 Chromium launch — the EKS pods used the default 64MB `/dev/shm`; under 5 workers on long/heavy shards Chromium exhausted it and the pod OOM-killed → "runner lost communication" (frozen step + 404 logs). Route shared memory to /tmp; eliminates the infra deaths. - **Service graph queries a wide window** (`period=6h`) — the tab defaulted to Past 15 Minutes and rendered "No service graph data" before the daemon-processed topology fell inside the window. ### Pipelines - `queryStream(stream, expectedMinCount)` **polls until the record count lands** (pipeline output arrives async) — was single-shot ("Expected 5, got 3"). - `openPipelineForEdit` **searches + polls-with-reload** for the edit button (list not refreshed after save → 45s click timeout). - `exploreStreamAndNavigateToPipeline` **reloads the streams list** while polling for Explore, and **falls back to a direct logs URL** when a freshly-ingested source stream never surfaces in the list (stream-stats lag). ### Alerts - Alert-destination URL → **public `example.com` webhook** (self-host URL hit the cloud **SSRF guard** → form never closed). - **Force-click the virtualized folder item** (#9484) — the row re-rendered mid-click → 45s "not stable". - **Export button readiness** — it's rendered only on selection; wait for visible+enabled (re-selecting if needed) before clicking. - **`deleteAlertByRow` cleanup readiness** — the just-created alert's more-options kebab needs longer than 5s under load. ### RegressionSet - **Use the shard's org** (`getOrgIdentifier()`), not hardcoded `org_identifier=default`, in `logs-bugs` — on cloud the data lives in `auto_org_N`, so the logs page never rendered. - **Unique stream names** for `logs-9754` / `logs-bugs:1419` — fixed names collided with concurrent shared-org runs ("stream is being deleted"). ### Traces / correlation - **Migrated `logs-bugs:995` (bug-11469)** to the redesigned correlation UI — the old `log-correlation-btn` / `correlation-dashboard-close` are gone; repointed to the inline `correlated-traces-tab`, assertion (`expectNoContextMenuVisible`) unchanged and **verified live on alpha** (no fake-pass). ### Workflows (ENT-only) + misc - **Healed the workflows destination flow** and forced-type Connection-step assertion; alert/report list-readiness hardening. - **Enrichment tables list** readiness (30s + re-nav retry) — "slow under load". ### Notes - Validated live via MCP (alpha) and a local build where the feature exists. - Alpha1 config already runs `retries: 2`; the remaining first-attempt flakes are shared-env load, addressed on the ENT side via `max-parallel`. |
||
|
|
dafc1ce7f7 | feat(ui): dashboard query editor redesign (#13637) | ||
|
|
3285a31e6f |
fix: i18n lint enforcement (#13439)
Co-authored-by: Kirtan Thakkar <kirtan@kiara.tech> |
||
|
|
98777be8be | fix: update enterprise feature (#13639) | ||
|
|
a384dd5ddd |
test(e2e/alpha1): robustness — auth self-heal + service-graph wide window (#13622)
## Alpha1 e2e robustness — self-healing auth + service-graph window Builds on **#13621** (re-auth self-heal). Both branches share the slug `test/alpha1-e2e-robustness` is stacked on `test/alpha1-reauth-on-401`; review/merge #13621 first (or merge this, which contains it). Root-caused against alpha1 runs 30892800970 / 30900015839 / 30902942856 (each investigated live via Playwright MCP). ### 1. Auth self-heal (from #13621, included here) Shared cloud session + per-org passcode are minted once and reused all run; on long runs / org-contention they expire mid-run → `User not authenticated` (UI) and `HTTP 401 Unauthorized` (API). Now: `authedRequest` refreshes the passcode and retries on 401; `navigateToBase` re-authenticates and resumes instead of throwing. **Verified working** in run 30902942856 — the auth storm cleared (Metrics/SDR/Streams recovered, zero `User not authenticated`), and `[auth] Refreshed cloud passcode after 401` fired and recovered mid-run. ### 2. Service graph queries a wide window All 9 `service-graph.spec.js` chart tests failed with `service-graph-chart` "element(s) not found" **despite** `beforeAll` confirming topology via API (25 nodes). **Reproduced live**: the service-graph tab defaults to **Past 15 Minutes** and shows *"No service graph data / 0 entities"* (no chart element) when the daemon-processed topology sits outside that window. The tab **honours the `period` URL param** (verified: label → "Past 6 Hours"). → `navigateToServiceGraphUrl` now navigates with `period=6h`; added `navigateToServiceGraphAndWaitForChart()` (poll + refresh, tolerates daemon lag). Non-masking. ### Honest scope - **Not included (can't validate locally / not test-code):** - **Pipelines ×5** (`Explore`/`update-pipeline` 45s timeout) — persistent across *every* run; pipeline/stream data unavailable. Env/data, needs CI-side repro. - **OTLP metrics ingestion 401** (a PromQL-prereq `beforeAll`) — a node-`fetch` ingest path with no page context to re-fetch the passcode; narrow, needs a small refactor to self-heal. - Singles: `logoManagement` (@enterprise `_meta`), `shareLink` (known race), `alerts-ui-operations:131` (#9484) — pre-existing flaky/env. - Service-graph is **enterprise/feature-gated** (403 in my org), so its fix is evidence-reasoned + validated at the URL-param level, but final proof is the seeded CI run. |
||
|
|
601ef75e2e | fix: IAM page UI changes (#13618) | ||
|
|
7999fd33f3 |
test: enhance logs visualization error handling and persistence tests (#13591)
## Summary Adds **10 E2E tests** covering the two defects fixed in #13244, plus the `SELECT *` behaviour change that shipped alongside them. Also introduces per-shard backend configuration so the `SELECT *` tests can actually run in CI. ## Test coverage ### `logsVisualizePersistence.spec.js` — 7 tests (quick mode **off**) | # | Test | Defect | Fix code exercised | |---|---|---|---| | 1 | Renders without the false required-fields error | A | `convertPanelData.ts` `!query?.customQuery` | | 2 | Restores chart type, query and chart after URL refresh (h-bar) | B | `setupLogsTab` re-run + `isInitialVisualizeRestore` | | 3 | Re-renders after reload with the default chart type | B | permanent No-Data half | | 4 | Restores a non-default chart type (area) after refresh | B | `restoreVisualizationFromUrlOnLoad()` | | 5 | Chart still renders after reload with a VRL function applied | B | `prepareVisualizeMode()` VRL copy | | 6 | Never shows required-fields error toggling Visualize → Logs → Visualize | A | manual-toggle rebuild path | | 7 | No `SELECT *` toast opening Visualize with no stream in the URL | B | page-load ordering guard | **Defect A** — false "Please select required fields to render the chart" on the custom-query Timechart panel. **Defect B** — URL refresh produced a `SELECT *` toast plus permanent No Data. ### `logsVisualizeSelectStar.spec.js` — 3 tests (quick mode **on**, dedicated shard) | # | Test | Behaviour verified | Call site | |---|---|---|---| | 8 | Table chart on `SELECT *` → toast shown **and** chart type reverts | Table is blocked | `Index.vue:2265` (watcher) | | 9 | Histogram chart on `SELECT *` → no toast, renders | Line/bar/area/scatter allowed | entry path | | 10 | `SELECT *` line chart survives browser refresh → no toast, state restored | Restore path doesn't re-emit | `Index.vue:2465` (restore) | ## Infrastructure changes | File | Change | Why | |---|---|---| | `ci_matrix.json` | New `Logs-SelectStar` shard with `quick_mode_enabled: "true"` | `isSelectStarForTable()` only exists when the server runs with quick mode on | | `playwright.yml` | Per-shard `ZO_QUICK_MODE_ENABLED` env on `Start OpenObserve` | The flag is read once at server start — a shard is the smallest unit that can vary it | | `build-ci-matrix.js` | Emit `quick_mode_enabled` (defaults `"false"`) | The field was being silently stripped by the emit whitelist | | `logsPage.js` | 13 assertion helpers + fixed chart-type detection | Detection read `data-selected` off the wrong element and matched retired CSS classes, so it always returned `null` | All 24 existing shards are unaffected — they omit the field and default to `false`. ## Notes on test quality - Every negative assertion ("no toast", "no error") is sequenced after a deterministic positive end-state, and gated on a precondition so it cannot pass vacuously. - Toast absence is proved with a MutationObserver recorder installed before the navigation under test, since toasts auto-dismiss. - The quick-mode precondition is a hard assertion, not `test.skip` — a misconfigured shard fails loudly instead of silently reporting "skipped". |
||
|
|
cc759bb6e0 |
test(e2e): fix Alpha1 shard failures (Alerts SSRF, sanity time-range, lazy Monaco, traces settle) (#13614)
## Alpha1 (Cloud) E2E — shard-by-shard failure fixes Fixes real, root-caused failures from Alpha1 run [30876689608](https://github.com/openobserve/o2-enterprise/actions/runs/30876689608) (5 red shards, 16 hard failures). Each root cause was reproduced live against alpha via Playwright MCP. **8 of 16 hard failures are fixed here in code**; the remaining 8 are environment/data-timing driven (backend instability during the run's 04:15–04:18 ingest window) and need a re-run, not a code change. ### Fixes | Shard | Fix | Root cause | |---|---|---| | **Alerts** (4) | `alerts-advanced.spec.js` destination URL → `https://example.com/webhook` | The self-host URL `${ZO_BASE_URL}/…/alerts_notify_sink/_json` resolves to a **private IP** on cloud, so the backend **SSRF guard rejects the save** → the New Destination form never closes → `createDestination()` times out, failing all 4 Alerts-Advanced tests. Verified live: private-IP URL is blocked, `example.com` (public, IANA-reserved, never delivered to) saves and closes the form. | | **GeneralTests** (2 + 2 flaky) | sanity `beforeEach` → `setDateTimeToPast1Hour()` (+ new `logsPage` helper) | The result-summary/pagination tests query the default **Past 15 Minutes**. `e2e_automate` ingestion is bursty on shared/cloud envs — **0 rows in 15m vs 7,850 in 1h** (confirmed via API) — so the result table never renders. This is exactly the failed-vs-flaky split seen on the run. | | **Logs-Features** (1) | `monaco-query-prefill.spec.js` → poll `getQueryEditorTextWhenReady()` | Lazy Monaco prefills its model asynchronously after a saved view is applied on a fresh page load. A one-shot read after a fixed 2s caught an empty editor (`Received: ""`). Now polls until the query lands. Non-masking: if it never prefills, the poll still times out and the test fails. | | **Traces** (1 of 2) | `tracesSearch.spec.js` → poll for a settled results/no-results state | The "no stream selected state" test read `hasTraceResults()`/`isNoResultsVisible()` once each; the async search settle left both momentarily false. Now polls until one terminal state renders. | ### Not changed (deliberately) - **Traces `traceQueryEditor.spec.js:151`** — the detail-tree check already waits 10s; its failure means the error-trace detail didn't open, which needs error-trace data in-window to validate. Weakening the assertion would mask real behavior. - **GeneralTests `enrichment.spec.js:101`, `logoManagement.spec.js:18`** — ran during the backend-instability window and can't be reproduced live. `[name='home']` is **not** stale (present live) and `_meta` access is fine for the test user. Env/timing. - **Pipelines (all 5)** — UI-click timeouts (`…-update-pipeline` / `Explore` button not found) because pipeline creation / stream data was unavailable while the backend returned *"No online node" / "Proxy request" / "Expected N records, got 0"*. Same selectors 42 passing tests in the shard use. Env-driven → re-run to confirm. ### Validation - All fixes reproduced/verified live on alpha via Playwright MCP (`e2etest1@o2-qa.com`, `auto_org_1`). - All 4 edited specs parse and list (`playwright test --list`). - Full local runs are blocked by an unrelated issue: the Dex login UI changed (new "Continue with Email" flow) so `global-setup.js` can't authenticate locally; CI uses the shared-auth artifact. (Worth a separate fix.) |
||
|
|
624425a08a |
feat(editor): rebuild query autocomplete, and fix the value path behind it (#13586)
Design at autocomplete/autocomplete.md
Rewrites the query editor's autocomplete across every surface that has
one, and
follows the field-value path far enough to fix the bugs it was hiding.
53
commits, in six phases; each phase was written test-first, reviewed
adversarially, and checked in a browser before the next one started.
## What was wrong
The completion providers were wired inconsistently. Six surfaces bound a
resolver that was never defined, so their value completion could not
resolve at
all — SLO, dashboards and anomaly detection among them. Traces never
bound
`:suggestions`. Alerts never called `updateStreamKeywords`. PromQL ran a
second
editor runtime purely for its grammar, and asked
`/prometheus/api/v1/series`
— the heaviest endpoint available — for data the stream schema already
has.
## Phases
**1 — icons, staleness, quoting.** Correct `CompletionItemKind` glyphs,
stream
keywords refreshed when the stream changes, `spath` quoting.
**2 — the function catalog.** Custom functions get one tab stop per
argument,
org VRL functions stop appearing twice, column types read from
`dataType` as
well as `type`.
**3 — signature help, hover, async completion.** Providers return real
`CompletionList`s with `incomplete: true`, so a list that arrives late
is
re-queried rather than pinned.
**4 — numeric ranking.** Numeric columns sort first inside a numeric
aggregate.
**5 — values for streams nobody has searched.** The value cache only
ever held
what a search had already returned, so a stream you had not queried
offered
nothing. Values are now fetched on demand, bounded (10s timeout, 60s
cooldown,
one in-flight request per key), and shared between the editor and the
builder.
PromQL joins the same path: labels from the stream schema, values from
the
shared cache. `/series` now has zero production callers.
**6 — one way to fetch a value.** Item 22 only, in this PR.
## The dashboard filter 400
Clicking the ✕ beside "Select Field" in a filter's dropdown sets the
column to
`{}`, and the watcher on `condition.column` then asks for that column's
values.
`loadFilterItem` built `fields: [row.field]` out of `undefined`,
JSON.stringify
turned that into `null`, and the server answered
```
400 {"code":400,"message":"SerdeJsonError# invalid type: null, expected a string"}
```
`_values_stream` answers 400, not an empty result, for each way a piece
of the
payload can go missing — and every one is reachable: an unresolved
stream alias
(`.find(…)?.stream`), a stream not yet chosen (`""`), a range not yet
set. The
range fails differently: `meta.dateTime` starts as `{start_time: ""}`
and
`""?.toISOString()` does not short-circuit, so it throws and both
callers
surface "Something went wrong!" for a lookup the user never asked to
fail.
Guarded at the two callers, where the payload is built and where the
throw
happens. Not at `fetchFieldValues` — a guard there is too late to stop
the
throw and would duplicate validation the callers need anyway. The
trigger is
left alone: re-picking a field must still load its values.
## Notable removals
CodeMirror is gone — it was still bundled for the PromQL grammar alone.
Bundle 26.3 MB → 25.43 MB; the PromQL chunks 442.7 KB → 20.6 KB. The
PromQL
term catalog went from 7 entries to 113, generated from Prometheus's own
tables
and vendored with its MIT notice.
## Verification
- 76 files, +10.5k/−1.4k, ~9,100 tests passing across the touched areas.
- `vue-tsc` clean, `npm run build` exit 0.
- Every phase checked in a running browser against a live instance,
including
the 400 above: the ✕ now produces no request and no toast, adding a
filter and
re-picking a field each still fetch once, and the value list still
populates.
## Known, deliberately out of scope
- A filter still fetches twice (once on add, once on open) — that is
item 23,
the cache read, not this PR.
- `useMetricsExplorer.loadFilterItem` has the same unguarded
`toISOString()`
pattern on a different surface.
- `services/metrics.ts` and `services/search.ts` still carry dead
`get_promql_series` wrappers now that nothing calls `/series`.
|
||
|
|
d76c0a6bbb |
ci(playwright): single-source ALL shard matrices via ci-matrix/ manifests (#13606)
## Why
The OSS and ENT Playwright shard matrices (`run_files` per `testfolder`)
were hand-maintained inline in each `playwright.yml` and had to be
synced on every spec add — error-prone and drift-prone (ENT was even
missing `dashboard-favorites`). This makes the shard list a **single
source of truth** for **every** Playwright workflow.
## What (OSS side)
All matrices are generated at run time from JSON manifests in
`tests/ui-testing/ci-matrix/`:
- **`ci-matrix/ci_matrix.json`** — PR-gate base (24 shards). The only
place shared shards live.
- **`ci-matrix/ci_matrix_regression.json`** — regression base (10
shards); `RegressionSet/<Feature>` mapping now carried in
`actual_folder`.
- **`.github/scripts/build-ci-matrix.js`** — merges base (+ optional ENT
overlay) → GitHub matrix; validates uniqueness / no dup specs / no spec
both active+disabled. Reused by ENT from its OSS checkout.
- **`ci-matrix/README.md`** — manifest index + how to add/move/disable a
spec.
- Workflows `playwright.yml` + `playwright_regression.yml`: new
`generate_matrix` job → `strategy.matrix: fromJSON(...)`; `case` blocks
→ `${{ matrix.actual_folder }}`.
Disabling a spec (JSON has no `//`): move it into that shard's
`disabled: [{file, reason}]` array — never emitted (won't run), record
survives. Every shard ships a `"disabled": []` placeholder.
## Safety (verified)
- **OSS `playwright.yml` and `playwright_regression.yml` matrices are
byte-identical to before** — zero behavior change.
- Companion ENT PR (same branch slug
`test/playwright-matrix-single-source`) derives its matrices from these
manifests + overlays.
## Validate
Add the `e2e` label; confirm `generate_matrix` prints the shard count
and the `e2e /` jobs fan out.
|
||
|
|
49448334ed |
test(alpha1): split shard logins across multiple Dex users (#13602)
## What Lets alpha1 shards log in as **different Dex users** instead of all hammering the cloud env as one user. Paired with **openobserve/o2-enterprise#2333** (the workflow wiring). ## Why - **Load distribution** — spreads API/session load across users if the env rate-limits per user. - **Blast radius** — one user's account/session issue stops ~1/3 of shards, not all. - Test **data isolation is unchanged** (still per-org); this is purely about the login user. ## Changes (OSS) - `global-setup-alpha1.js` — new `ALPHA1_USER_INDEX` (1|2|3) selects the Dex user: email resolves from `ALPHA1_USER_EMAIL_<N>`, **falling back to `ALPHA1_USER_EMAIL`** when the indexed secret isn't set (safe incremental rollout). Password shared (`ALPHA1_USER_PASSWORD`). - Auth **filenames stay canonical** (`user.json` / `cloud-config.json`) — ~20 specs + shared utils read those exact paths. The per-user split happens at the CI layer (each shard downloads only its user's artifact into the canonical path), so no spec changes needed. - New `auth-warm.spec.js` — a login-only spec the barrier uses to mint auth artifacts for users 2..N **without** running the heavy org-wide cleanup. ## Default behaviour With no index set, everything resolves to user 1 — identical to today. ## Enterprise PR openobserve/o2-enterprise#2333 |
||
|
|
503eca6264 |
fix(e2e): deflake logs autocomplete value-suggestion capture (#13593)
## Problem In [o2-enterprise run 30791574230](https://github.com/openobserve/o2-enterprise/actions/runs/30791574230) the **Logs-Features** shard logged **2 failed, 6 flaky, 3 skipped, 63 passed** with **52 retries**. Seven of the eight troubled tests are in `logs-autocomplete-suggestions.spec.js` (the eighth `code`-field test shares the cause), and every one dies on the identical assertion: ``` Error: Expected to find a field with captured values expect(received).not.toBeNull() Received: null ``` — i.e. the shared `findFieldWithStringValues()` helper returned `null`. ## Root cause A **click-time race**, not missing data (the `e2e_automate` stream holds 3848 rows ingested at shard start). The helper clicked a field's sidebar expand button and then polled IndexedDB for 10s — but the 10s window started at **click time** and raced the `/_values_stream` fetch. In the failing run the runner was slow (a perf assertion showed queries ~2.6× over baseline: 1055ms vs <403ms). The values response landed **after** the IndexedDB poll had already given up, so the record read as `null` and the assertion failed hard. Because `findFieldWithStringValues()` is shared across ~8 tests, one slow window knocked out the whole cluster, and each burned up to 4 attempts → 52 retries. An intended fix (`clickFieldExpandAndWaitValues`, a deterministic `/_values_stream` response wait) already existed in the file but was **never wired in**. ## Fix - **`expandFieldAndCaptureValues()`** — anchor the IndexedDB confirmation **after** the actual `/_values_stream` response instead of racing it from click time; re-click to re-expand if a prior collapse meant no fetch fired. Replaces the dead `clickFieldExpandAndWaitValues` helper and the bare click+poll in the `code`-field test. - **`findFieldWithStringValues()`** — scan via the anchored capture, with a bounded refresh-and-retry round so a transient empty `/_values` response self-heals rather than failing the test. - **Readiness gate** — add `waitForStreamData()` to every `beforeEach` so freshly-ingested data is searchable before the UI drives value capture (WAL → index lag). Best-effort so an already-warm shared stream never blocks. ## Testing - `node --check` passes; no linter is configured for the suite. - The slow-runner condition can't be reproduced deterministically locally; the change is a test-harness reliability fix (deterministic response-anchoring + readiness gate) that removes the timing dependency that caused the flake. Will confirm green via the ENT playwright regression this PR dispatches. |
||
|
|
de95f7f2d8 |
fix: restore Table default log-detail view (#13368) + heal alert-name inline-edit read (#13585)
## What & why Two **Playwright Regression** nightly failures, reproduced identically on both OSS ([run 30785096054](https://github.com/openobserve/openobserve/actions/runs/30785096054)) and ENT ([run 30785071382](https://github.com/openobserve/o2-enterprise/actions/runs/30785071382)) — same two shared-`tests/` specs, all 3 retries failed each. Both are OSS-owned specs (ENT overlays only its own `tests/*` and builds OSS `web/`), so this single OSS PR fixes both nightlies. ### 1. Logs — real product regression (`logs-regression.spec.js:1158`) `Log detail sidebar opens with Table tab by default (#13368)` → `Error: Table tab should be selected by default`. **Root cause:** [#13368](https://github.com/openobserve/openobserve/pull/13368) (Jul 23) deliberately made the log-detail sidebar open on the **Table** tab. [#13451](https://github.com/openobserve/openobserve/pull/13451) (Jul 30, *"table migration to same components"*) silently flipped `detailTableInitialTab` back to `"json"` in `SearchResult.vue` (both the `ref` init and the per-open reset). The unchanged test correctly caught the reverted behavior. **Fix:** restore `"table"` as the default log-detail tab. Product fix — no test change. ### 2. Alerts — stale test vs redesigned component (`alerts-stream-switching.spec.js:443`) `Bug #11577: Alert preview chart y-axis displays numeric values` → `TimeoutError: locator.inputValue: Timeout 45000ms exceeded`. **Root cause:** the alert name became an **`OInlineEdit`** title (`OFormInlineEdit`, from the [#13547](https://github.com/openobserve/openobserve/pull/13547) Alerts revamp). In display mode the value lives in the `…-value` span; the `…-input` only exists **while editing**. The test read `.inputValue()` on the non-existent input → 45s timeout. **Fix:** read the committed display value via a new `getAlertName()` page-object helper (with an edit-mode `inputValue()` fallback). ## Verification - **Alerts #11577** — ✅ passed locally against a local O2 build (`1 passed`); `getAlertName()` reads the name and the full save/verify/delete flow works. - **Logs #13368** — restores the exact `detailTableInitialTab = "table"` value #13368 set and the unchanged test already validated as green. Confirmed by this run's Logs-Regression job + the dispatched ENT e2e gate. ## Scope - `web/src/plugins/logs/SearchResult.vue` — 2 lines (product) - `tests/ui-testing/playwright-tests/RegressionSet/Alerts/alerts-stream-switching.spec.js` — read display value - `tests/ui-testing/pages/alertsPages/alertsPage.js` — `alertNameValue` locator + `getAlertName()` helper |
||
|
|
d99ecff3e8 |
feat(nav): add Reliability menu, restore deferred SLO routes (#13577)
Groups the alerting surface under one left-rail tile, and un-defers the SLO backend that shipped disabled. > **Lands with** openobserve/o2-enterprise#2321, which adds `ROUTE_PERMISSIONS` for the restored SLO endpoints — the route-coverage test pairs the two, so this one alone fails it. ## Reliability menu New `reliability` nav group absorbing **Alerts, SLOs, Incidents**, plus **Notification Destinations** and **Templates**, which move out of Settings — they are alerting configuration, not deployment configuration. Settings keeps Pipeline Destinations (that one really is pipeline config); its group is renamed "Destinations". Their routes are **top-level and flat** (`/alert-destinations`, `/alert-templates`) rather than nested under `/alerts`. They are siblings of Alerts, not sub-pages of it — nesting them made the URL claim otherwise and the rail believed it, highlighting Alerts alongside them. - Route **names** are unchanged, so the ~15 call sites that navigate by name are untouched. - Old `/settings/*` paths redirect, with the query preserved — `?action=import` deep links keep working. ## SLO routes `/api/{org}/slos` answered **404**. The routes were deliberately unregistered before the last release (`6977b4020d`, `ac78a622e1`) while the rest of the feature — `core/src/slo/*`, the infra tables, both migrations, the `slo_maintenance` job, all seven handlers — stayed in the tree. This restores the five registrations and the OpenAPI entries, recovered from the pre-strip commit rather than rewritten. `ZO_SLO_ENABLED` still **defaults to false**. It is now published as `slo_enabled` on `/config`, so the menu entry follows the flag instead of offering a page the API answers with 501. ## Nav active-state Active state was decided per child, so any section whose path prefixed the current route lit up alongside it. It is now resolved once per flyout: exact route-name match wins, else the **deepest** path prefix — so a genuine drill-down like `/alerts/detail/:id` is still attributed to Alerts. `placeAfter` also accepts a group key, letting Data anchor on the Reliability tile it follows rather than on an item that tile absorbs. ## Tests `tests/ui-testing` referenced `[data-test="alert-destinations-tab"]` / `alert-templates-tab` in six files — those data-tests were on the Settings rail items removed here, so the locators would have matched nothing and hung until timeout. Those paths now go through the existing `openNavFlyoutChild` helper, extended with the Reliability group. Two URL regexes that would have silently mismatched on the hyphen are fixed (one ended in `.catch(() => {})`, so it would have degraded to a silent 15s stall). ## Verification - 3597 unit tests pass; `vue-tsc` and eslint clean; enterprise build compiles. - `playwright test --list`: all 141 tests across 23 files load, no module errors. - Against a live server: `GET /slos` → 200, `POST /slos/move` → 422 (so the literal wins over the `{slo_id}` catch-all), `GET /slos/x` → handler-level `{"code":404,"message":"SLO not found"}`. - In the running app: `/alert-destinations`, `/alert-templates` and `/alerts` each highlight exactly one row; `/settings/alert_destinations?...&action=import` redirects with the query intact; rail order unchanged. |
||
|
|
83d12fff04 |
feat(web): inline-rename page titles with auto-generated default names (#13520)
## What changed
Naming an alert, panel, workflow, pipeline or function meant inventing a
name up front in a form field before anything else could be configured.
Two pieces replace that.
**Inline rename** — the page title *is* the input. `OInlineEdit` /
`OFormInlineEdit` (`web/src/lib/forms/InlineEdit`): click to rename,
Enter or blur commits, Escape restores the pre-edit value. A `#trail`
slot carries the "Auto" badge in display mode only.
**Auto-generated names** — `utils/autoName.ts` builds the name,
`composables/useAutoName.ts` decides when it's allowed to win. A name is
AUTO until the user types, then it's theirs for good; clearing the field
and blurring re-arms auto. The generators describe only what is actually
configured, so a half-built panel yields a shorter name rather than a
speculative one, and an empty string when there is nothing to say.
- Panel names read as prose — `Avg of duration by service`, `Record
count`, `k8s_logs overview`. Both live panel-field shapes are handled
(builder `functionName` + `args`, and legacy flat `column` +
`aggregationFunction`); the time column is read from
`zoConfig.timestamp_column` rather than hardcoded. Hand-written queries
fall back to the stream, since naming a panel after `x_axis_1` is worse
than saying nothing.
- Alert names are identifiers — `anomaly_k8s_logs`,
`k8s_logs_status_gte_500` — built to survive the backend's
`RE_OFGA_UNSUPPORTED_NAME` rule, with operators spelled out and a
`slugify` backstop. A leading underscore is preserved so internal
streams (`_rumdata`) keep their names.
### Supporting library changes
| Component | Change |
| --- | --- |
| `OPageHeader` / `OPageLayout` | New `titleOverflow="visible"` — the
`<h1>`'s `truncate` clips an interactive title's focus ring and the
error message it floats below itself |
| `OSelect` | New `appearance="inline"` — borderless, no fixed height,
sized to content, for a value that reads as a word inside running text
("Add Alert **in KTX**") |
All new strings go through i18n (`common.inlineEdit.*`,
`dashboard.autoName.*`, `alerts.autoName.*`, per-module `renameHint`).
Interpolated identifiers — column names, SQL functions, stream names —
stay verbatim.
## Wired into
AddPanel, AddAlert (+ `useAlertForm`), WorkflowEditor, PipelineEditor,
Functions / FunctionsToolbar, IncidentDetailDrawer,
InlineSelectFolderDropdown.
## Tested
244 unit tests across the 5 affected suites, all passing — including new
specs for `OInlineEdit`, `useAutoName`, and `autoName`.
One pre-existing expectation in `autoName.spec.ts` was updated: it
asserted `Avg of duration, max of duration +1 more`, capitalising only
the first word of a multi-measure name. `measureLabel` capitalises per
measure by design, so the expectation now reads `Avg of duration, Max of
duration +1 more`.
---------
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
|
||
|
|
4772df1723 |
feat: per-group and per-series alerting, and SLO measurement (#13547)
Design at: alerts_2.md Per-group and per-series alerting, SLO measurement, and the UI for both. ## Alerts **Multi-alerts (Feature 3).** A grouped alert can opt in to evaluating and paging **per group** rather than collapsing to one verdict. Opt-in only (`multi_alert`), so no existing alert changes behaviour: the flag cannot be present in JSON written before it existed. Per-group state, transitions, silence fingerprints, disappearance/reaping and a cardinality cap, with the count-gate rules that make "most severe group" and the legacy collapsed level provably the same verdict. **Per-series alerting for PromQL.** The same feature for the metrics family, where a group is one returned **series** keyed by its full label set — that is a series' identity in Prometheus, and the expression's own `by (…)` clause is already where the label set is chosen, so a second picker could only disagree with it. One additive nullable column; NULL means off, so stored alerts are untouched. **Alert detail page**, replacing the row-click side panel: stat strip, evaluation chart with threshold marklines, per-group table and history. Simple alerts get a real history too — evaluations (one record per run) by default, level changes alongside. ## SLOs Count / time-slice / alert SLIs, error-budget and burn-rate arithmetic, rolling windows, per-group slices with an exact rollup, coverage gating, backfill, and SLO-based alerts. Two capabilities the model could not express, both added tests-first: - **Metrics-native counting** (`CountSource::PromQl`) — counter metrics have no rows for a predicate to classify, so "good" exists only as arithmetic between series, and correct counter arithmetic is `increase()`. - **Freshness semantics** (`absent_is_bad`) — for a pipeline SLO, silence IS the failure, which the default gap rule can never report. A *failed* query still freezes rather than pages; only a query that PROVED emptiness counts as bad. **SLO form**: live good/bad preview so a wrong predicate is caught before saving rather than days later, real query editors with field autocomplete, folders shared with alerts, and a move endpoint. ## Notes for review - The SLO and composite features are **deferred (TODO)**: `ZO_SLO_ENABLED` stays default **false** and the SLOs menu entry is hidden until they ship; the SLO backend and UI remain in the tree behind the flag, and composites stay pure logic + tests. - One migration (schema 61): a nullable column for the PromQL opt-in. No backfill. - Two fixes outside the feature that it surfaced: query-editor completions were duplicated per mounted editor (Monaco registers providers per *language*), and the alert detail chart had no PromQL branch, so it charted row counts against the series-count gate. - Lands together with the matching o2-enterprise PR — `QueryType::Slo` is new here and the enterprise dedup matches are exhaustive over it. - `ratelimit::test_add_batch_missing_rule_id` fails intermittently on `main` as well; unrelated to this branch. --------- Co-authored-by: Shrinath Rao <shnath@openobserve.ai> Co-authored-by: hengfeiyang <hengfei.yang@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
|
015a20a184 |
refactor: remove HTTP API compatibility re-exports (#13560)
## Summary - remove the HTTP request/model compatibility re-exports introduced by the domain crate split - migrate router, OpenAPI, enterprise action-server, and integration-test callers to the crates that own each API - remove the ingest Content-Type re-export and declare the direct root/test dependencies - delete the pure `handler::http::models` facade module ## Validation - `cargo fmt --all -- --check` - `cargo clippy -p openobserve-api-ingest -p openobserve-api-http -- -D warnings` - `cargo test --test integration_test --no-run` - `cargo check -p openobserve --no-default-features` - enterprise overlay: `cargo test --offline --workspace --no-default-features --features enterprise --no-run` - cloud overlay: `cargo test --offline --workspace --no-default-features --features cloud --no-run` - `git diff --check` |
||
|
|
6fb5eecdc5 |
fix: table migration to same components (#13451)
## What changed Every table in the app now renders through the shared `OTable` (`web/src/lib/core/Table`). Both legacy implementations are deleted — `components/TenstackTable.vue` and `plugins/logs/TenstackTable.vue` — for ~7,000 lines removed and one table behaviour contract instead of three. To get there, `OTable` gained: per-cell hover actions, per-column value filter, opt-in column close "x", sticky pivot total row + right-pinned total columns, pivot row-field cell merge, caller-owned pagination bar, delegated scroll (shared scroller with the logs/traces histogram), and variable-height virtual rows. ## Tables migrated | Area | Tables | | --- | --- | | Logs | Search results grid, correlated logs | | Traces | Search results grid, trace details sidebar (attributes / events / links), service graph node panel | | Dashboards | Table panel + pivot (incl. PromQL), panel pagination controls, import error panel | | RUM | Session player traces tab | | Pipelines | Scheduled pipeline SQL preview | ## Tested 775 unit tests across the 12 affected suites, all passing. Manually verified per table: - **Columns** — resize, reorder, close "x", persistence across reload; horizontal scroll on wide grids; wrap toggle with virtual scroll; client + 3-state server sort - **Pagination** — custom records-per-page, enabling pagination after mount, scroll-to-top on page change, no duplicate pager - **Logs** — FTS highlighting, row expand → JSON preview and its actions, hover actions + AI button, cell font / row density / status spine, histogram pinned while scrolling sideways - **Dashboards** — pivot multi-level headers, cell merge, sticky totals staying aligned under horizontal scroll, cell colouring (auto palette / value mapping / conditional rules / overrides), CSV + JSON export - **Traces, RUM, Pipelines** — events table expansion, error-row border, row click → trace navigation, SQL preview copy + send-to-AI --------- Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com> Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
|
|
6d4eb704ed |
fix: allow testing workflow without saving (#13537)
This PR adds a fix so that workflows can be tested without having to save. --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
|
c49d156e7e |
test(alpha1): multi-org sharding for the alpha1 e2e run (#13530)
Companion OSS change to enterprise 2278 (alpha1 workflow). Same branch slug on both repos. ## What Supports running the Alpha1 (cloud) e2e matrix with its shards **divided across multiple orgs** for data isolation, on top of the shared-auth login (one Dex login, reused by all shards). Both changes are in \`global-setup-alpha1.js\`. ## Changes 1. **Ingest into the shard's own org (\`ORGNAME\`)** instead of the org baked into \`cloud-config.json\`. The shared-auth barrier logs into one org; without this, every shard ingested into that one org even when \`ORGNAME\` differed. 2. **Re-fetch the shard's own org passcode in the shared-auth path.** Each org has its own \`o2oi_\` default ingestion token (see \`core/organization.rs\` \`get_passcode\`). The barrier captured only its org's token, so shards on other orgs got \`401 Unauthorized Access\` on ingest. \`verifySharedAuth\` now calls \`fetchCloudConfig(page)\` to overwrite \`cloud-config.json\` with the current shard's passcode (fetched via the session, which is a member of every org). Keeps the single shared login — no login storm — while giving each shard the correct per-org credential. ## Context Part of the alpha1 parallelization effort: - Shared-auth artifact removes the 13-way concurrent Dex login storm (enterprise 2278). - Shards divided across 6 orgs (enterprise 2278 matrix) for data isolation. - These two OSS fixes make per-org ingestion actually work. ## Testing Validated by dispatching the enterprise \`playwright_alpha1.yml\` with \`o2_opensource_repo=test/alpha1-parallel-shards\`. Verifying each shard now ingests \`e2e_automate\` into its own org with \`status:200\` (previously \`401\` for non-barrier orgs). |
||
|
|
4231ca5833 |
perf(schema): share stream settings via Arc and centralize cache mutation (#13535)
Design at: #13465 PR 2 of the plan in #13465 (finding 1: the settings cache O(S) deep clone per mutation). ## Background The stream settings cache is a locked map (`STREAM_SETTINGS`) plus an ArcSwap read snapshot (`STREAM_SETTINGS_ATOMIC`) introduced in `405794db08` for lock-free ingestion reads. That architecture is kept — the read/write ratio justifies it. The problem was the constant factor: - every mutation republished the snapshot by **deep-cloning every cached `StreamSettings`** (vectors, UDS field lists, maps). The snapshot is republished on every schema change event cluster-wide (any new stream or field, on every node) plus every cache-miss backfill — at 100k+ streams this is a full deep copy of the map at minute-level frequency; - every point read (`get_settings`, called on each ingestion request) deep-cloned the returned `StreamSettings`, including UDS lists that can hold hundreds of strings; - the two maps each held a complete deep copy of all settings. ## Changes **Arc-ified values.** Both maps now store `Arc<StreamSettings>`. Snapshot republish is a shallow clone (String keys + refcount bumps), point reads return an `Arc` clone, and the maps share one set of values, roughly halving the cache's memory. **Centralized mutation.** The map statics are private; all writes go through `put_stream_settings` / `put_stream_settings_batch` / `remove_stream_settings`, which republish the snapshot under the write lock so snapshots cannot be published out of order. Startup load collects settings and publishes once via the batch API. Callers can no longer accidentally rebuild the snapshot — the pattern that caused the original quadratic startup. Lock-free reads stay available via `get_stream_settings_atomic`; the status API uses a new `get_stream_settings_stats()` accessor. **API ripple, kept minimal.** `get_settings()` now returns `Option<Arc<StreamSettings>>`. Field access and `unwrap_or_default()` call sites compile unchanged; the few paths that edit a local copy before persisting (`update_stream_settings`, LLM/gen-ai settings, dashboard distinct fields) do an explicit `(*s).clone()`. The `get_stream_setting_*` helpers are generic over `Borrow<StreamSettings>`, so both `&Option<StreamSettings>` and `&Option<Arc<StreamSettings>>` callers work — including the downstream enterprise call sites, which were checked and need no changes. **`SchemaCache::fields_map` Arc-ified** — cloning a cached schema entry no longer deep-copies the field-name index, which holds thousands of entries for wide streams. ## Measurements (criterion bench used during development, not kept in-tree) | operation | result | |---|---| | bulk load 10k / 50k / 100k streams | 0.80ms / 4.4ms / 8.7ms — linear, ~87ns per stream | | point write against 100k-entry cache | 3.5ms (shallow snapshot republish) | | point read against 100k-entry cache | 7.9ns (lock-free) | For comparison, the field report in #13465 measured ~27ms per stream insert at 58k cached streams under the old per-mutation deep clone; reads previously deep-cloned the settings value on every call. ## Testing - new unit test for the mutation API (put/batch/remove, snapshot sharing via `Arc::ptr_eq`, republish keeps other keys) - full test suites pass for all touched crates (infra, db, schema, stream, core, api-search, api-management, search_service, compaction, enrichment-data); the only failures are pre-existing local-environment ones also failing on main (sqlite-path retention tests, one env-dependent index test, one enterprise doctest) - `cargo clippy --workspace --all-targets`: no findings in touched files Finding 2 (grouped stream enumeration) landed in #13536; this branch has been rebased onto main (the #13531 overlap is resolved). |
||
|
|
184268ff1b |
test(ci): loud stdout banner when Playwright retries a test (#13529)
## What Adds a small **log-only** Playwright reporter that prints a LOUD, unmissable banner to stdout **the moment any test starts a retry attempt**, plus an end-of-shard summary listing every test that was retried. ## Why In CI the reporter is `blob`, which writes nothing to stdout. So when a failed test is retried (`retries: process.env.CI ? 3 : 0` in `playwright.config.js`) there is **no signal in the GitHub Actions logs that a retry is even happening** — a test silently re-runs up to 3 more times and the log just shows the next `Test Started` line. This makes flaky-vs-real failures hard to spot when scanning CI output. The retry is internal to `npx playwright test`, so the workflow YAML can't see it — it has to be emitted from the Playwright reporter layer. This is that layer. ## What it looks like ``` 🔁🔁🔁══════════════════════════════════════════════════════════🔁🔁🔁 🔁 R E T R Y # 1 · attempt 2 of 4 (test previously failed — re-running) 🔁 ▶ chromium › logsqueries.cte.spec.js › should query with simple TotalCols CTE 🔁 ▶ Logs/logsqueries.cte.spec.js 🔁🔁🔁══════════════════════════════════════════════════════════🔁🔁🔁 ``` …and once per shard at the end: ``` 🔁 RETRY SUMMARY — N test(s) were retried in this shard: 🔁 • [2 retries] chromium › … › should query … (Logs/logsqueries.cte.spec.js) ``` ## Scope / safety - Added to the **CI** reporter list only, alongside `blob` (multiple reporters are fine). It only `console.log`s — never sends anything, never fails the run. - **ENT gets this for free**: the ENT Playwright workflow checks out and runs the OSS `tests/ui-testing` tree, so this one OSS change makes retries loud in both OSS and ENT CI. There is no ENT test tree to change. - Verified locally with a forced-failure probe (`CI=true`, `retries: 2`): the per-retry banner fires on attempt 2 and 3, and the summary lists the retried test. ## Files - `tests/ui-testing/playwright-tests/utils/retry-banner-reporter.js` — new reporter - `tests/ui-testing/playwright.config.js` — wire it into the CI reporter array |
||
|
|
c9fc8d3b55 | fix: dashboard builder redesign (#13419) | ||
|
|
7d212f4621 |
feat: workflows incident connection (#13472)
This connects incidents to workflows, so any lifecycle event of incident will trigger workflow. design at: https://github.com/openobserve/designs/blob/main/infrastructure/workflows/design.md --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
|
|
e9596a10a3 |
feat: Mobile rum sdk v2 (#13503)
Design at: design/mobile-rum-sdk-evaluation/design-docs Backend changes exist to make the OpenObserve RUM intake accept the mobile SDK wire format alongside the existing browser format. src/api/src/handler/http/request/rum/ingest.rs 200 → 202 Accepted on /rum, /logs, /replay | ---> OpenObserve-family SDKs (which the mobile forks derive from) treat any non-202 as an unexpected error and drop the batch. #[serde(default)] on Event ---> Mobile replay omits creation_reason / index_in_view that the browser format sends. Multi-segment replay handling ---> Browser sends 1 segment + 1 event object; mobile sends N segment parts + one event JSON array. Handler now collects all segments, parses event as array-or-object, pairs them, emits NDJSON → one _sessionreplay row per segment. MAX_SEGMENTS=200, 16 MB decompressed cap via .take() ---> Decompression-bomb guard, newly needed now that N segments are accepted. |
||
|
|
6e0a7374e5 |
test: add DashboardFavorites page object and associated tests for fav… (#13362)
## Summary Adds a Playwright E2E test suite covering the dashboard Favorites feature — favoriting/unfavoriting a dashboard, the `Favorites` pseudo-folder view, and its interaction with single and bulk delete flows. ## What's covered New `dashboard-favorites.spec.js` (6 tests) + `DashboardFavorites` page object: - Add a dashboard to favorites and verify it appears in the Favorites view - Remove a dashboard from favorites — confirms it drops from the Favorites view while the dashboard itself remains untouched - Favorite state persists across a page reload - Bulk-delete a favorited dashboard from the Favorites view completes successfully with no error - Deleting a dashboard directly also removes it from Favorites automatically - Bulk-deleting a dashboard from the Favorites view doesn't leave a stale copy behind in its original folder ## CI `dashboard-favorites.spec.js` runs in its own dedicated `Dashboards-Isolated` shard with a fresh backend instance, keeping it independent from the rest of the dashboard test suite. |
||
|
|
ef3617f157 |
refactor: split HTTP APIs by domain (#13479)
## Summary
- organize API crates under
`src/api/{common,grpc,http,ingest,management,pipelines,search}`
- keep ingest, management, pipelines, and search independent from one
another; compose them only in the HTTP frontend
- move pattern APIs into search and simplify `main.rs` wiring
- update workspace manifests, lockfile, integration tests, and
cargo-deny entries
- keep `web/` unchanged
## Validation
- `cargo fmt --all -- --check`
- `cargo clippy --all-targets -- -D warnings`
- `cargo deny check licenses`
- enterprise overlay: `cargo build --no-default-features --features
enterprise`
- enterprise overlay: `cargo build --no-default-features --features
cloud`
- enterprise overlay: `cargo clippy --all-targets -- -D warnings`
The matching `o2-enterprise` overlay PR will be linked after creation.
|
||
|
|
442ca09688 |
test(functions): un-skip JS-in-all-orgs tests + add per-event runtime-error coverage (#13341)
Replaces the `skip` band-aids that PR #13156 ("workflows v1") left on the **Functions** tests with real, running coverage of the current behaviour — on the correct edition — plus new tests for behaviour #13156 introduced but never covered. Test files only, no feature-code churn. Clean branch off `main`. ## Background — what #13156 actually changed #13156 bundled a Functions backend change unrelated to Workflows, and the old tests still asserted the *pre*-#13156 rules, so they were silenced with `pytest.skip(...)` / `test.skip(...)`. Two rules changed: 1. **JS is no longer `_meta`-only at the backend.** `save_function` / `test_run_function` have **no org gate on any edition** — JS (`transType:1`) is accepted in every org, including `default`, on **both OSS and Enterprise**. The `_meta`-only rule now lives **only in the frontend** (`AddFunction.vue` gates the JS radio on `isEnterprise || isCloud || org === "_meta"`, baked at build time by `VITE_OPENOBSERVE_ENTERPRISE`). 2. **The compile-check reports only *syntax* errors, not *runtime* errors.** A syntax error still returns **400**; code that compiles but throws at runtime now returns **200** with the error surfaced **per-event** in `results[].message` — for **both VRL and JS**. Because the API tests hit the backend directly, there is **no edition difference to assert for them** — the earlier "OSS asserts 400 / ENT asserts 200" framing was incorrect. Only the **e2e** (frontend JS-radio) is genuinely edition-dependent. ## API tests (pytest · `tests/api-testing`) **`test_functions.py`** - Keep the compile-error → 400 test (`test_test_endpoint_with_invalid_vrl_returns_400`) — it was never actually failing; `test`/`=====` are genuine VRL *compile* errors (`undefined variable` / syntax) and still 400. - Add `test_test_endpoint_with_js_runtime_error_returns_200` and `test_test_endpoint_with_vrl_runtime_error_returns_200` — runtime errors now return 200 with a per-event `results[].message`. **`test_functions_meta_org_restriction.py`** - Flip the 3 previously-skipped "blocked in default" tests → assert **200** (JS allowed in every org): `..._blocked` → `..._allowed`. - Repurpose `TestErrorMessages` (its old restriction-message assertions no longer exist) into per-event runtime-error coverage: JS `ReferenceError` content, VRL + JS surfacing consistency, partial-failure (only the failing event carries a message), and a syntax-error → 400 regression guard. - Update the class/module docstrings to the real contract. **`test_js_enrichment.py`** - Flip `test_js_function_in_default_org_returns_400` → `test_js_function_in_default_org_allowed` (200, transform runs). ## e2e (Playwright · `tests/ui-testing/Functions/js-transform-type.spec.js`) The JS-radio rule *is* edition-dependent. Instead of a skip (or an env-var hack), the two tests now detect edition at runtime via `EditionFeaturesPage.detectEdition()` — which reads the header edition button, driven by the **same** build flag that gates the JS radio, so it's the exact source of truth and needs **no CI change**: - **OSS build** → assert JS radio **hidden** in a non-meta org. - **Enterprise / cloud build** → assert JS radio **visible** in every org. The edition-agnostic steps that were trapped inside the skipped tests now **always run**: VRL-function create in a non-meta org (a P0 smoke path), and JS-visible in `_meta`. Added `P1: VRL runtime error surfaces in error-details` (mirrors the existing JS `ReferenceError` case; covers the VRL half of the per-event-error change). ## Skips removed (before → after) | Old (skipped) | New (runs & passes) | |---|---| | `test_create_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_test_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_update_vrl_to_js_in_default_org_blocked` | `..._default_org_allowed` | | `test_error_message_content` / `_consistency` | per-event runtime-error tests | | `test_js_function_in_default_org_returns_400` (enrichment) | `..._default_org_allowed` | | e2e `P0: Non-meta org - JS hidden, VRL works` | `P0: Non-meta org - JS radio edition-gated, VRL works` | | e2e `P1: Radio visibility across orgs` (skipped) | same name, edition-aware, no `test.skip` | `grep -rn "JS fns are now allowed" tests/` and `grep -rn "JS is allowed in all orgs on enterprise" tests/` → **both empty**. (The 7 remaining pytest skips are the unrelated `JS_HELPER_UNAVAILABLE` enrichment-helper cases — a separate pre-existing limitation, not part of this change.) ## Verification (local, against current `main` source) Built the current-`main` OSS binary and ran everything live: - **API:** `32 passed`, `0` target skips (7 unrelated `JS_HELPER_UNAVAILABLE` skips remain). - **e2e OSS build:** `7/7` — `detectEdition=opensource`, JS hidden in non-meta. - **e2e Enterprise-frontend build:** `3/3` edition tests — `detectEdition=enterprise`, JS visible everywhere, VRL runtime error renders in error-details. All backend behaviour was confirmed by probing the live `/functions` + `/functions/test` endpoints, not assumed. ## What CI proves, and where - **This PR (openobserve):** `api-testing.yml` runs the API tests; `playwright.yml` (`Functions` shard) runs the e2e on the **OSS** edition (JS-hidden branch). - **Enterprise edition:** dispatch `o2-enterprise` → `playwright.yml` (workflow_dispatch) with `o2_opensource_repo = test/functions-skipped-tests-13318` to run the same specs on the **enterprise** build (JS-visible branch). Pairs with the ENT Functions work in #13318 / o2-enterprise#2217. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
|
7ade62610a |
test: heal logs scroll-retention selector after histogram-pinning change (#13460)
## Problem The scheduled Playwright Regression runs failed on `Logs-Regression` in **both repos** on the same 3 tests in `logs-9044-7354.spec.js` (issue 9044 scroll-retention): - OSS run [30150748275](https://github.com/openobserve/openobserve/actions/runs/30150748275) - ENT run [30150717648](https://github.com/openobserve/o2-enterprise/actions/runs/30150717648) All three failed identically — `expect(scroll).toBeGreaterThan(10)` received `0`, i.e. scrolling the results container no longer moved it. ## Root cause `web/src/plugins/logs/SearchResult.vue` now **pins the histogram** and hands the scrollbar to the results table (`TenstackTable` → `[data-test="logs-search-result-logs-table"]`). Its scroll parent is the `ref="parentRef"` div (`.o2-scroll-container`, `overflow: auto`). The outer `scrollContainerRef` pane is now `overflow: hidden` in the logs view. The page-object helpers `getScrollContainerPosition()` / `scrollToResultsBottom()` walked *down* from the pagination header looking for a scrollable sibling, which latched onto a non-overflowing element — so `scrollTop` stayed `0`. (This is the same area healed once before in #13316; the histogram-pinning change moved the scroller again.) ## Fix Discover the scroll container from the results table itself: walk **up** from `[data-test="logs-search-result-logs-table"]` to the nearest ancestor with a scrollable computed `overflow-y` (the `.o2-scroll-container`). Matched by computed style so class renames don't break it. The old pagination sibling-walk is kept as a fallback for the patterns view (which still scrolls as one pane). Only the shared OSS page object `logsPage.js` changes, so this heals both OSS and ENT (ENT merges the OSS test folder). ## Verification Ran against a fresh local build (localhost:5080): ``` 3 passed (46.8s) Scrolled to: 358 → Scroll correctly reset to top on page change ``` Confirmed via a DOM probe that `.o2-scroll-container` is the real scroller (scrollHeight 1112 > clientHeight 754; scrollTop → 358 after scrolling to bottom). |
||
|
|
1d99426df9 | fix: browser sdk package version upgraded to 0.4.1 (#13455) | ||
|
|
3f9ce27542 |
refactor: split service dependencies out of core (#13446)
## Summary - decouple stream cleanup from enrichment-data and remove the transitive compaction/DataFusion dependency from stream - extract sourcemaps and organization storage services from core into dedicated crates - extract GitHub built-in regex access and model pricing into openobserve-builtins - rename pipeline and organization-storage persistence modules from store to db - remove now-unused direct dependencies from core - synchronize the enterprise workspace overlay in openobserve/o2-enterprise#2266 ## Validation OSS: - cargo check -p openobserve-builtins -p openobserve-core -p openobserve-jobs -p openobserve-api - cargo test -p openobserve-builtins --lib - cargo test -p openobserve-builtins --doc - cargo clippy -p openobserve-builtins -- -D warnings - cargo check -p openobserve-org-storage --features enterprise Enterprise overlay: - copied ../o2-enterprise/Cargo.toml.openobserve over Cargo.toml - cargo fmt --all -- --check - cargo check --all-targets --no-default-features --features enterprise - cargo clippy --all-targets --no-default-features --features enterprise -- -D warnings The default enterprise + vectorscan build was attempted and reached vectorscan-rs-sys, but the validation host does not have cmake installed. The enterprise-only build and clippy run completed successfully. |
||
|
|
c6b2829a5e |
test(autoe2e): generated E2E tests for #13025 (logs-stream-quick-pick) (#13030)
🤖 Auto-generated E2E tests for #13025 — **healed to passing** against a live OpenObserve before this PR was opened. - Branch `autoe2e/pr-13025` (stacked on `issue-ent-2038`), so the tests run against the feature code and pass before it merges. - Spec: `tests/ui-testing/playwright-tests/Logs/logsQuickPick.spec.js` The repo Playwright CI re-validates here. Review is for **quality + coverage** — the tests already pass. _Generated by the E2E Council pipeline._ **Test cases added / changed:** | Test case | Type | What it verifies | |---|---|---| | `should quick pick a stream from the sidebar` | new | clicking a sidebar quick pick button selects the stream and loads the field list | | `should show hero state on fresh arrival with no stream selected` | new | the main content area shows both hero action cards when no stream is selected | | `should open stream dropdown when clicking select stream card` | new | clicking the "Select a stream" hero card opens the stream dropdown popover | | `should reappear quick pick after deselecting and re-select works` | new | after deselecting a stream the quick pick reappears and re-selection works correctly | | `should show more footer when org has more than 8 streams` | new | the quick pick more footer is visible with the correct remaining count when org has > 8 streams (skips otherwise) | | `should select stream via hero recent chip` | new | clicking a recent-stream chip in the hero state selects the stream and triggers field list loading | | `should open query guide documentation in new tab` | new | clicking the "Read the query guide" card opens the OpenObserve docs in a new browser tab | --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
15275ae7cf |
refactor: split core foundation services (#13405)
## Summary This PR begins decomposing `openobserve-core` into focused service crates while preserving the existing application behavior and dependency direction. It extracts six independently owned areas: - `ingestion-common`: shared ingestion contracts and response types - `schema`: schema inference, validation, evolution, and persistence helpers - `compaction`: merge, retention, and compaction services - `enrichment-data`: enrichment-table lifecycle and runtime support - `promql-service`: PromQL service orchestration - `stream`: stream metadata, settings, retention, and lifecycle operations ## What changed - registered the extracted crates in the workspace and propagated enterprise/cloud/vectorscan/vortex features - moved implementations out of Core and updated API, Jobs, query, and queue consumers to use the new crates - removed Core facade aliases for extracted crates and converted remaining callers to direct crate paths - retained the small cross-service stream-delete coordinator in Core for now because it still coordinates Pipeline and Alert cleanup - restored required root/Core dependencies after validating the full enterprise build - added the missing `anyhow` dependency for enterprise-only `search_service::search_jobs` - updated the lockfile for the new crate dependency graph ## Dependency direction The extracted crates do not depend on `openobserve-core`. Core remains a temporary upper-level consumer while the remaining domain services are split incrementally. ```text ingestion-common -> common/config schema -> ingestion-common/db/infra stream -> schema/enrichment-data/db/infra compaction -> schema/search/search-service/db/infra openobserve-core -> ingestion-common/schema/stream/compaction/... api/jobs -> openobserve-core + extracted services ``` ## Enterprise compatibility The private OpenObserve manifest was updated in openobserve/o2-enterprise#2257 to register the extracted workspace members and dependencies. Related Enterprise PR: https://github.com/openobserve/o2-enterprise/pull/2257 ## Validation OSS manifest: - `cargo fmt --all -- --check` - `cargo clippy -- -D warnings` Using `o2-enterprise/Cargo.toml.openobserve` as the OpenObserve root manifest: - `cargo check --no-default-features --features enterprise` - `cargo check --no-default-features --features cloud` - `cargo clippy --workspace --no-default-features --features enterprise -- -D warnings` - `cargo clippy --workspace --no-default-features --features cloud -- -D warnings` Enterprise repository: - `cargo fmt --all -- --check` - `cargo clippy -- -A clippy::question-mark -D warnings` All checks above pass in the development profile. |
||
|
|
9842922521 |
test(pipelines): fix addPipeline mount-wait flake (isVisible timeout is a no-op) (#13420)
## What flaked `Pipelines-Regression` → *"should validate scheduled pipeline SQL query with metrics stream type"* (`pipeline-regression.spec.js:73`) timed out in the ENT gate waiting for the editor mount signal `[data-test="pipeline-node-sidebar-collapse-btn"]` (15s, all retries). It's a **load-timing flake, not a real break**: OSS CI ([run 30005672782](https://github.com/openobserve/openobserve/actions/runs/30005672782)) and the ENT `main` baseline ([run 30005554375](https://github.com/openobserve/o2-enterprise/actions/runs/30005554375)) both pass this same test. ## Root cause `addPipeline()` meant to poll for the mount signal across 3 attempts of 15s (re-clicking the add button non-destructively if the click was dropped before navigation): ```js for (let attempt = 1; attempt <= 3; attempt++) { if (await railToggle.isVisible({ timeout: 15000 }).catch(() => false)) break; ... } await railToggle.waitFor({ state: 'visible', timeout: 15000 }); ``` But **`locator.isVisible({ timeout })` does not wait** — it returns the current state immediately and ignores `timeout` (Playwright 1.50). So the loop spun through instantly and the editor effectively got only the **final 15s** `waitFor` to mount. Under concurrent cloud load VueFlow's mount lags past 15s → timeout. ## Fix (page-object only) Make each poll attempt actually block using `waitFor({ state: 'visible' })` inside the loop (try/catch, re-click on the list page between attempts), restoring the intended **~45s** of resilience before the final assert. ## Verification Local verification is currently blocked by a stale local build (the selector was added in #13397 and isn't in the local `web/dist`), so this is gated on a fresh CI regression run against the branch. The change only lengthens the effective mount wait; it can't regress the happy path (which already passes on fresh builds). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
0e477f9918 |
test(workflows): de-flake K9 canary — log Save-intercept, don't hard-assert it (#13418)
## How this was found The **Playwright E2E Crosscheck** bot (the ENT gate that runs enterprise-only specs against OSS PRs) flagged a failing spec on an unrelated logs PR ([#13414 comment](https://github.com/openobserve/openobserve/pull/13414#issuecomment-5058151842)): > `Workflows/workflows-negative.spec.js` — *"enterprise-only e2e spec failed against this PR … wouldn't show up in OSS CI."* Digging showed it wasn't caused by that PR — the same test was **also failing on #13413 and #13315**. It's a pre-existing **flake** in the ENT gate, so this PR fixes it at the source. ## Evidence it's a flake, not a real break The **same branch** produced both outcomes: | ENT crosscheck run | PR | K9 test | |---|---|---| | 30005431754 | #13414 | ✅ passed (~50s, slow path) | | 30008606974 | #13414 | ❌ failed | | 30006996158 | #13315 | ❌ failed | | 30006419247 | #13413 | ❌ failed | ## Root cause `CT-19 K9` hard-asserted that a normal `.click()` on Save is intercepted by a tooltip overlay: ```js expect(await pm.workflowsPage.normalSaveClickIsIntercepted(4000)).toBe(true); ``` But the Save button has **no tooltip of its own** (`WorkflowEditor.vue`) — the overlay that intercepts it is a **transient/adjacent tooltip** that often never appears on a programmatic hover in headless CI. The probe therefore conflates *"blocked by overlay"* with *"click landed"*, and the assertion flips run to run. (A first attempt — waiting for the overlay before probing — didn't help: the overlay simply isn't reliably summoned.) ## Fix (test-only, no product/Vue changes) - Treat interception as an **informational canary** — `testLogger.info(...)` instead of a hard assert. - Keep the **stable** assertion: the JS-click Save workaround still reaches the app (`msg.length > 0`) — the adjacent trigger-only-save test already exercises that same path and passes consistently. - If K9 is ever genuinely fixed, the canary log shows `intercepted:false` consistently — the signal to drop the JS-click workaround. This removes the flaky gate without losing meaningful coverage. Changes are limited to the spec and its page object. ## Verification `Workflows/*` are enterprise-only (skipped on OSS CI), so verified via the **ENT gate against this OSS branch** (`e2e` label). The removed assertion was the only flaky part; the retained assertion matches an already-stable adjacent test. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
1daa36b0f8 |
fix(logs): preserve restored SQL query on shared-link open + align log-detail default-tab test (13368) (#13414)
> **This PR contains two independent logs changes:**
> **(A)** a product fix for the shared-link SQL-query restore race, and
> **(B)** the log-detail default-tab test alignment (original scope of
this PR).
---
# (A) fix(logs): preserve restored SQL query on shared-link open
(fullSQLMode watcher race)
## The bug
Opening a **shared link** to an SQL search **sometimes lands on an empty
query editor** ("SQL query is missing or invalid") instead of the shared
search. Intermittent, worse under load — it's the recurring
Logs-Features alpha flake (`monaco-query-prefill` P0 "query editor never
contained SELECT").
**Not the short-URL mechanism** — the short URL correctly carries
`sql_mode=true` + the base64 query. The loss happens in the browser
during restore.
## Root cause — a watcher race
`restoreUrlQueryParams()` raises `shouldIgnoreWatcher`, flips
`meta.sqlMode = true`, then writes the decoded query into the editor.
Flipping `sqlMode` fires the **`fullSQLMode`** watcher in `Index.vue`,
whose two branches were **asymmetric**:
- **switch OFF** branch → already checks `shouldIgnoreWatcher` ✅
- **switch ON** branch → **did not** ❌ — it called `setQuery()`,
rebuilding a *default* `SELECT … FROM stream` over the value the restore
was mid-writing.
Under load the watcher wins the race; the editor briefly empties, SQL
mode auto-detects back **off**, and the editor is cleared entirely.
## The fix
Make the "switch ON" branch honour `shouldIgnoreWatcher` too — symmetric
with "switch OFF". During any programmatic restore the watcher **stands
down** and the restored query is the last word. (`shouldIgnoreWatcher`
is raised only by programmatic-state paths whose purpose is to suppress
watcher side-effects, so honouring it here is consistent.)
## Verification (A)
- ✅ **New unit test**: with `shouldIgnoreWatcher` set, flipping
`sqlMode` ON does **not** call `setQuery`/`updateUrlQueryParams`, and
the restored `SELECT` query is preserved.
- ✅ Existing `fullSQLMode` watcher test still passes; full
`Index.spec.ts` green (**69 passed**).
- ✅ Reproduced the failure on **alpha under CI-load** beforehand
(share-link P0 flaky, "query editor never contained SELECT").
- ⚠️ The race is only observable against a **deployed** frontend (the
alpha e2e suite runs against alpha's frontend, not a local build), so
the final "flake gone on alpha under load" check requires this deployed
to alpha, then re-running the Logs-Features share-link tests.
Files: `web/src/plugins/logs/Index.vue`,
`web/src/plugins/logs/Index.spec.ts`.
---
# (B) test(logs): align log-detail default-tab test with Table default
(13368)
## What
Updates the log-detail sidebar default-tab regression test (originally
Bug #9724) to match the **intentionally shipped** behavior from [PR
#13368](https://github.com/openobserve/openobserve/pull/13368) — *"logs
sidebar table will be default view and draggable tabs"*.
## Why
#13368 changed the log-detail sidebar default from the **JSON** tab to
the **Table** tab (`SearchResult.vue` — `detailTableInitialTab.value =
"table"`). The Bug #9724 test still asserted JSON-as-default, so it
fails on `main`: the JSON tab's `data-state` stays `inactive` for the
full timeout across all retries, on both OSS and ENT Logs-Regression.
Not a flake and not a product bug — the test expectation was simply out
of date after a deliberate default-view change.
## Changes
- Rename test → **"Log detail sidebar opens with Table tab by default
(#13368)"**.
- Add `verifyTableTabSelectedByDefault()` page-object helper
(auto-retrying `toHaveAttribute('data-state','active')`); remove the
now-invalid `verifyJsonTabSelectedByDefault()`.
- Reorder switch steps: assert Table default (+ wrap toggle, which
renders only on the Table tab), switch to JSON and back to Table, reopen
and confirm Table default persists.
## Note on #13407
Supersedes the earlier heal
[#13407](https://github.com/openobserve/openobserve/pull/13407), which
misdiagnosed the failure as a `data-state` timing flake. A
locally-hosted stale pre-#13368 build had masked the change.
Files: `tests/ui-testing/pages/logsPages/logsPage.js`,
`tests/ui-testing/playwright-tests/RegressionSet/Logs/logs-regression.spec.js`.
```
npx playwright test playwright-tests/RegressionSet/Logs/logs-regression.spec.js \
-g "Table tab by default" --project=chromium
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
2cec0a0c94 | fix: pipeline and workflows UI change (#13397) | ||
|
|
8886a03b99 |
feat(dashboards): redesign panel config sections (#13371)
- Collapse all config sections by default (was expanded) - Add open-state accent to OCollapsible sidebar variant: left accent bar, tinted header, and accent-colored icon/chevron when expanded - Move sidebar chevron to the right of the label (rotates on open) - Give every config section a leading icon via SECTION_ICONS - Add theme-aware collapsible open-state tokens (light + dark) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
8da11deec7 |
test(logs): heal Bug 9724 JSON-tab-default flake with retrying data-state assertion (#13407)
## What Heals the flaky **Logs-Regression** spec `Log detail sidebar opens with JSON tab by default (Bug #9724)`, which failed the scheduled main regression run [29990712453](https://github.com/openobserve/openobserve/actions/runs/29990712453) (and its re-run) on all retries. ## Root cause `logsPage.verifyJsonTabSelectedByDefault()` read the Reka `OTab` (`TabsTrigger`) `data-state` attribute with a **one-shot** `.evaluate(el => el.getAttribute('data-state') === 'active')`. `DetailTable.vue` is loaded via `defineAsyncComponent`, and its tabs moved to Reka UI in the design-token migration (13173). On drawer open, the trigger can render a tick before Reka's reactive `data-state` settles to `active`. A raw `getAttribute` read does **not** auto-retry, so under CI load it captured the pre-settle value and failed. JSON **is** the genuine product default (`DetailTable.vue` `tab = ref(initialTab || "json")`, `SearchResult.vue` `detailTableInitialTab = "json"`) — this is a load flake, not a product regression. ## Fix Replace the one-shot reads with the auto-retrying `expect(tab).toHaveAttribute('data-state', 'active', { timeout: 10000 })` pattern already used elsewhere in the same page object (regex-pattern / builder-type toggles), across all three tab-verify helpers: - `verifyJsonTabSelectedByDefault()` - `verifyTableTabSelected()` - `verifyJsonTabSelected()` Test-only change; no product code touched. ## Verification Local run against `localhost:5080`: ``` npx playwright test playwright-tests/RegressionSet/Logs/logs-regression.spec.js \ -g "Bug #9724" --project=chromium --repeat-each=3 # 3 passed (16.9s) ``` The single spec exercises all three healed helpers (default-JSON at steps 3 + reopen, Table tab at step 6, JSON switch-back at step 7). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
ef2efdf5fc |
fix: logs sidebar table will be default view and draggable tabs (#13368)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
4f458a0fab |
test(sdr): add traces SDR e2e suite (redact/drop/hash × ingestion/query) (#13390)
## What Adds an end-to-end **traces SDR (Sensitive Data Redaction)** test suite, extending SDR coverage from logs to traces. This validates the fix in this PR branch (`feat/fix-traces-sdr-bug`) — SDR regex patterns were previously applied to logs only; these tests assert the same policies now apply to **trace span attributes** at both ingestion and query time. Targets `feat/fix-traces-sdr-bug` so the tests land together with the fix they cover. ## Tests added **6 new spec files → 12 tests** (each spec = 1 `setup: cleanup patterns` + 1 combined functional test), covering **24 field-level policy assertions** (4 sensitive fields × 6 specs). All tagged `@enterprise @sdr @traces`. | # | Spec | Policy | Timing | What it asserts | |---|------|--------|--------|-----------------| | 1 | `tracesIngestionTimeRedaction.spec.js` | Redact | Ingestion | Span-attribute values matching the pattern are stored as `[REDACTED]` | | 2 | `tracesQueryTimeRedaction.spec.js` | Redact | Query | Stored values are untouched; the search response returns `[REDACTED]` | | 3 | `tracesIngestionTimeDrop.spec.js` | Drop | Ingestion | Matching fields are removed from the record at write time | | 4 | `tracesQueryTimeDrop.spec.js` | Drop | Query | Fields present in storage are dropped from the search response | | 5 | `tracesIngestionTimeHash.spec.js` | Hash | Ingestion | Matching values are stored as `[REDACTED:<hash>]` | | 6 | `tracesQueryTimeHash.spec.js` | Hash | Query | Search response returns `[REDACTED:<hash>]`; storage untouched | Each combined test drives the full flow for **4 fields / patterns**: - **email** (`user_email` = `john.doe@example.com`) — `^[\w\.-]+@[\w\.-]+\.\w{2,}$` - **US phone** (`phone` = `5551234567`) — `^\d{10}$` - **credit card** (`cc_number` = `1234 5678 9012 3456`) — `^\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}$` - **SSN** (`ssn` = `123-45-6789`) — `^\d{3}-\d{2}-\d{4}$` (drop/hash specs use `website_url`, `user_password`, `server_ip`, `theme_color` with URL / password / IPv4 / hex-color patterns.) Each functional test follows: ingest → assert field visible (no pattern) → create 4 patterns → associate to the traces stream's fields → ingest/re-query → assert the field is redacted / dropped / hashed. ## How it works Traces reuse the **shared** pattern create/associate/verify flow with logs — only the ingest and read seams differ: - **`sdrTracesIngestionPage.js`** (new): ingests via OTLP `/api/{org}/v1/traces` (`stream-name` header). Each test field is a span attribute that flattens to an identical top-level field; every span also carries an `sdr_test_id` marker for batch isolation. - **`sdrVerificationPage` / `streamAssociationPage`**: parameterized by `streamType` — `?type=traces` search, Traces stream-type tab selection, and schema-field filtering so a field row in the (virtualized, ~20-field) traces schema is reliably found. - **`apiCleanup`**: `fetchStreams` / `deleteStream` / `cleanupStreams` / `isStreamStillDeleting` / `waitForStreamDeletion` take an optional `streamType` (default `'logs'` — no change for existing callers). ## Artifact cleanup - `cleanup.spec.js` sweeps `sdr_traces_*` streams via `streamType: 'traces'` (the logs sweep is `type=logs`-only and never listed them). - `cleanupRegexPatterns` adds the `traces_` prefix. - **Verified live**: the sweep deleted 8 leftover traces streams + 32 traces patterns; 0 remained. ## Verification - All **6 specs / 12 tests pass** against a local enterprise build carrying the fix. - The existing **logs** SDR suite still passes unchanged (no regression from the shared page-object changes). ## Companion CI wiring is a paired o2-enterprise PR (branch `test/traces-sdr-e2e`) that splits the SDR matrix into `SDR-Logs` and `SDR-Traces` shards so the new specs run as their own shard. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Chaitanya Sistla <sistlachaitanya2@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
68946ed063 |
test: harden org-selector switch flows against virtualized dropdown (changeOrg + logoManagement) (#13391)
## What
Heals the two `GeneralTests` specs that flake on the redesigned
**`OrganizationSelector.vue`** (reka-ui `ODropdown` +
`@tanstack/vue-virtual` virtualized list, from the design-token
migration): `changeOrg.spec.js` and `logoManagement.spec.js`. Test-only
— no product/frontend change.
## Why it flakes (CI parallel load only — passes single-worker locally)
- **changeOrg** — virtual rows carry a `transform: translateY(...)` that
keeps shifting while the popper animates in and the list settles, so
Playwright's actionability *stability* check burns the click budget and
times out. A force-click on a still-moving row can also land on a
**neighbouring row** (often `default`) — the menu still closes, so the
wrong org is selected and the swallowed `waitForURL(...).catch(()=>{})`
hid it until a later assertion failed.
- **logoManagement** — `managementOrg` had no "menu opened" gate; a
blind `waitForTimeout` raced ahead of the popper open, so
`organization-search-input-field` never rendered and `fill()` timed out
(45s).
This is **not a regression from any single PR** — it reproduces on every
branch's ENT `GeneralTests` run (the org-selector redesign was never
hardened for the virtualized dropdown).
## Fix
- `homePage.js` `clickOrgRow`: `scrollIntoViewIfNeeded` + `force` click
+ gate on the menu **hiding**, so a missed click retries instead of
leaving an un-selected menu.
- `homePage.js` `selectOrgByIdentifier` (new): retry open → search →
click until the **URL** carries the target identifier — the only
authoritative signal that the right row was selected. `homePageOrg`'s
identifier branch uses it.
- `logoManagementPage.js` `managementOrg`: retry the open until the menu
list is visible, fill the `-field` input with `force`, hardened row
click with the same menu-hidden verification. Added
`organizationMenuList` locator.
## Verification (local enterprise build v0.92.0-rc1)
| Test set | Workers | Result |
|----------|---------|--------|
| changeOrg "change organisation validation" (14) ×3 | 4 | 14/14 pass
each (was 1-failed pre-fix) |
| changeOrg full spec | 4 | 28 pass, 0 fail (no regressions) |
| logoManagement ×2 | 1 | pass |
## Enterprise coverage
Carries the `e2e` label so the **ENT e2e gate** dispatches the
enterprise suite against this branch (built against ENT main) and
mirrors the result here — validating the `@enterprise` logoManagement
spec and ENT GeneralTests without a separate ENT PR.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
871b9b69b7 |
test(traces): heal bug #11531 spec for flame-graph-default trace detail (#13392)
## What Heals the **Traces-Regression** failure on `main` (scheduled run [29935986660](https://github.com/openobserve/openobserve/actions/runs/29935986660/job/88982390601)). The Bug #11531 test in `traces-bugs.spec.js` (`traces spans tab should not reference non-existent method field`) clicks a trace result and immediately asserts the span tree via `expectTraceDetailsVisible()`, which waits for `[data-test="trace-details-tree"]`. ## Why it started failing PR 13347 (`fix: traces page default flame graph and drag-drop option added`) changed the trace detail view to **open on the flame graph by default** (`DEFAULT_TRACE_TAB = "flame-graph"`) instead of the waterfall. The span tree lives inside `v-if="activeTab === 'waterfall'"`, so it no longer renders on open — the assertion timed out (15s) and the job exited 1. This is **not a product bug** — it's an intentional product change. 13347 added a `openTraceDetailsTab()` helper and updated the other traces specs, but this Bug #11531 test was missed. ## Fix Select the waterfall tab explicitly before reading the tree, matching the pattern 13347 already applied elsewhere: ```js // Since 13347 the trace detail opens on the flame graph by default. await pm.tracesPage.openTraceDetailsTab('waterfall'); await pm.tracesPage.expectTraceDetailsVisible(); ``` Test-only change; no product code touched. ## Verification (local, hosted O2 @ localhost:5080, `default` traces stream w/ data) | State | Result | |-------|--------| | Before fix (unpatched) | ❌ `expectTraceDetailsVisible` times out at `tracesPage.js:310` | | After fix (patched) | ✅ `1 passed (22.3s)` — "Trace details visible" logged | 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |