Reduces three independent e2e flakes. All test-infra only; no product
code.
## 1. Pipelines — `ingestData` "Premature close" (the original failure)
`LogsPage.ingestData` failed all records with `Failed to ingest N out of
N`. Server returned HTTP 200 and ingested every record, but node-fetch
2.7.0 reused a keep-alive socket the server closed during the 500ms
inter-record gap → "Premature close" reading the body (0/384 succeeded).
Fix: disable keep-alive (fresh connection per request, per-protocol
agent) + per-record retry. Verified: Pipelines shard green.
## 2. Dashboards-Core — `dashboard-chartJson.spec.js` popup flake
Custom-query tests (1 failed + 3 flaky) failed at the JSON/non-timestamp
checkbox with "popper intercepts pointer events" / detached timeouts.
Root cause (reproduced 100% locally): the field-property ODropdown
auto-closes within ~300ms when opened right after the panel's first
render — a late field-list re-render dismisses it. `toBeVisible()`
passes at t=0 but the checkbox is gone by click time. Fix:
`openFieldPropertyPopup` now opens, confirms the popup STAYS open, and
re-opens if it auto-closed (throws if it never stays open — no masking).
Verified locally across repeated runs.
## 3. Logs-Builder — `unflattened.spec.js` `_o2_id` readiness
Failed with `Failed to find _o2_id field after 3 attempts`; CI log shows
the UI detail was empty (`fields: []`) — re-ingested data wasn't
search-visible yet on the contended runner. The readiness gate existed
but its result was ignored, so the test fell through to a doomed UI scan
and burned all 4 retries (~26 min). Fix: respect the gate's result (on
timeout, wait once more and log), and widen the UI row scan 10→15. The
final assertion is unchanged — a genuinely absent `_o2_id` still throws.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
Applies the same node-fetch v2 fix from PR #12799 (pipelines) to all
remaining CI jobs that show \"Premature close\" / ECONNRESET failures:
- **apiUtils.js** (`sendRequest`): adds `compress:false` +
`noKeepAliveAgent` — fixes **Streams** job
(`multiselect-stream.spec.js`, `streamname.spec.js`)
- **apiCleanup.js** (`_fetch`): same fix on the node-fetch fallback path
— fixes **Reports** job (dashboard/report API calls)
- **model-pricing.spec.js**: wraps `node-fetch` with a local safe-fetch
shim — fixes **GeneralTests** job (`/llm/models` premature close)
- **streamsPage.js**: replaces four dynamic `node-fetch` imports with a
shared `_nodeFetchSafe` helper
- **ingestionPage.js**: adds `fetchWithRetry` around native fetch to
handle transient `ECONNRESET`/premature close in `beforeEach` ingestion
**SDR fix:** `regexPatternManagement.spec.js` — `AddRegexPattern.vue`
migrated to `OForm` with submit-time Zod validation
(`AddRegexPattern.schema.ts`). The `ODrawer` primary button is no longer
disabled for empty fields; validation triggers on click and keeps the
drawer open on error. Replaces stale `expect(isDisabled).toBeTruthy()`
checks with click-then-expect-drawer-still-visible assertions.
## Root cause (same as #12799)
`node-fetch` v2 defaults to `keepAlive: true` (connection pooling) and
`compress: true` (Gunzip stream). When the server closes a pooled
connection early, the Gunzip stream throws \"Premature close\". Because
the dead socket stays in the pool, retries hit the same socket.
`keepAlive: false` + `compress: false` eliminates both failure modes.
## Test plan
- [ ] GeneralTests / `model-pricing.spec.js` — no more premature close
on `/llm/models`
- [ ] Streams / `multiselect-stream.spec.js` + `streamname.spec.js` — no
more premature close on ingestion
- [ ] Reports / `reportsScheduleNow.spec.js` — no more premature close
during beforeEach ingestion
- [ ] SDR / `regexPatternManagement.spec.js` — validation test passes
with updated assertion
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Problem
CI job **Pipelines-Regression** consistently fails on:
> `[chromium] › pipeline-regression.spec.js:73 › should validate
scheduled pipeline SQL query with metrics stream type`
with `FetchError: Invalid response body … Premature close` on every
attempt.
PR #12793 added `fetchWithRetry` to handle transient network errors, but
the underlying cause — **node-fetch v2 keep-alive pooling + gzip
decompression** — can survive retries when every connection attempt
picks up the same dead socket from the pool.
## Root cause
`node-fetch` v2 defaults:
- `compress: true` → sends `Accept-Encoding: gzip` and pipes response
through `Gunzip`
- Implicit HTTP agent with `keepAlive: true` → reuses connections across
requests
When the server closes a pooled connection before the response body is
fully read, `Gunzip` throws "Premature close." Because the bad
connection stays in the pool, retries can hit the same socket and fail
identically.
## Fix
- Use `keepAlive: false` agent → every fetch gets a fresh connection
- Set `compress: false` → bypass the `Gunzip` stream where the error
originates
- Keep the retry wrapper as a safety net for any remaining transient
faults
## Test plan
- [ ] CI: `Pipelines-Regression` job passes (especially the metrics
stream type test)
- [ ] Full `pipeline-regression.spec.js` suite passes locally
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## Problem
The **Pipelines-Regression** CI job was failing on the P0 test:
> `[chromium] › RegressionSet/Pipelines/pipeline-regression.spec.js:73 ›
should validate scheduled pipeline SQL query with metrics stream type`
Failure (all 4 attempts):
```
FetchError: Invalid response body while trying to fetch
http://localhost:5080/api/default/ingest/metrics/_json: Premature close
```
The error originates in `beforeEach`, inside
`pipelinesPage.ingestMetricsData()`. In the same setup, the global log
ingestion, trace ingestion, and RUM ingestion all succeeded — only the
metrics ingestion `fetch` threw.
## Root cause
`Premature close` / `ECONNRESET` / `socket hang up` are **transient**
`node-fetch` errors that happen when the server closes a keep-alive
socket before the response body is fully read. They are not real
ingestion failures. Because the throw happens in the `beforeEach` of a
**serial** describe block, a single flaky socket error fails the whole
suite.
There was no retry on any of the ingestion `fetch` calls in
`pipelinesPage.js`.
## Fix
Add a small `fetchWithRetry()` helper and use it for the three ingestion
calls (metrics, traces, bulk). It retries (with linear backoff) **only**
on transient network errors; real HTTP failures (non-2xx) are untouched
and still throw immediately.
## Verification
Ran locally against `localhost:5080`:
- Failing P0 test in isolation — **passed** (36.9s)
- Full `pipeline-regression.spec.js` suite — **7 passed** (3.4m)
## Notes
- Only `tests/ui-testing/pages/pipelinesPages/pipelinesPage.js` is
changed.
- The transient socket flake can't be reproduced against a healthy local
server; the local runs confirm the retry wrapper is transparent on
success (no regression).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
- Adds a matrix strategy to the query agent SQL regression suite jobs in
both `api-testing.yml` and `query-agent.yml`
- Each job now runs with `ZO_FEATURE_SINGLE_NODE_OPTIMIZE_ENABLED=false`
and `ZO_FEATURE_SINGLE_NODE_OPTIMIZE_ENABLED=true`
- Verifies both DataFusion physical optimizer paths produce identical
query results
- Matrix legs run in parallel with `fail-fast: false`
- All artifact/log names include the matrix label to avoid collisions
## Test plan
- [ ] PR CI shows two `api_query_agent` matrix legs
(single_node_opt_false and single_node_opt_true), both pass
- [ ] `/query-agent` PR comment triggers both modes and posts per-mode
results comments
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## PR #12726 — Change Summary
### Problem
Test `should render table with histogram(_timestamp) as X and count
aggregation as Y` was flaky with:
Error: locator.scrollIntoViewIfNeeded: Element is not attached to the
DOM
at dashboard-chart.js:163
Root cause: After searchInput.fill(fieldName), Vue reactively re-renders
the field list
multiple times (debounce / virtual scroll). waitFor({ state: "visible"
}) passed during
an intermediate render, then Vue triggered another re-render, detaching
the element before
scrollIntoViewIfNeeded() could act on it.
---
### Changes Made
1. tests/ui-testing/pages/dashboardPages/dashboard-chart.js — Flakiness
Fix
- Removed redundant scrollIntoViewIfNeeded() — hover() already
auto-scrolls
- Wrapped hover() in a 3-attempt retry loop that re-waits for the
element after
each DOM detachment
- Raised initial waitFor timeout from 5000ms → 10000ms for heavier field
lists
2.
tests/ui-testing/playwright-tests/Dashboards/dashboard-table-chart.spec.js
— Two changes
a) Replaced waitForStreamComplete with page.waitForResponse + explicit
status assertion
- waitForStreamComplete silently timed out on non-200 responses — no
assertion
- page.waitForResponse resolves on any matching URL response, then
expect(apiResponse.status()).toBe(200) fails clearly on 401/500
b) Fixed vacuous assertion (Test Inspector finding)
- Original predicate had && r.status() === 200 — waitForResponse would
only resolve
on 200, making expect(apiResponse.status()).toBe(200) always true
(no-op)
- Removed status condition from predicate so the expect() is now a real
assertion
---
### Test Inspector Findings Addressed
| Finding | Status |
|----------------------------------------------------------------------|---------------------|
| Vacuous assertion — expect(apiResponse.status()).toBe(200) was no-op |
Fixed |
| Selector cross-validation — o-field-list-row-* matches
OFieldList.vue:67 | Verified |
## Summary
- The AI integrations regression test (`datasources-regression.spec.js`)
was using `.first()` to select the first integration in the sidebar, but
manifest-backed rich-content cards placed at the top of each category
render `AIRichSetupCard` or `AIIntegrationCard` — neither contains the
`[data-test="rum-content-text"]` element that the test expects.
- Fix: navigate directly to the "agno" child route and click the
agno-specific tab (`[data-test="ai-integrations-item-agno"]`). Agno is a
known stub integration without rich content that always uses the
`CopyContent` fallback.
## Test plan
- [ ] CI Playwright run: `Data Sources Regression Bug Fixes` suite
passes on chromium
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## Summary
Fixes a test gap where `match_all` Tantivy queries could return wrong
results after a DataFusion upgrade without being caught by existing
assertions.
### Problem
After a DataFusion upgrade merge, the DataFusion access plan was no
longer used for `match_all` queries, causing Tantivy to return wrong
results. The existing tests only compared row counts against a DuckDB
oracle that emulates `match_all` with `LIKE '%term%'` — so wrong rows
that still matched the LIKE pattern passed undetected.
Additionally, `node/flush` does not generate Tantivy indexes
immediately. The default `ZO_MAX_FILE_RETENTION_TIME=600s` means
WAL→parquet→Tantivy conversion can lag, so FTS queries may fall back to
a non-Tantivy path during tests.
### Changes
1. **Content verification**
([conftest.py](tests/api-testing/tests/query_agent/conftest.py)): Every
hit from a `match_all` query that returns the `log` field is now checked
— each search term word must be present in `log`. A hard assertion fails
if Tantivy returns rows that don't contain the search terms.
2. **Tantivy readiness gate**
([test_2_parquet.py](tests/api-testing/tests/query_agent/test_2_parquet.py)):
After `node/flush`, the test now polls with a `match_all('warehouse')
LIMIT 1` query (up to 120s timeout) to confirm Tantivy indexes are built
before running FTS assertions. Previously only COUNT(*) was checked.
3. **Multi-word match_all emulation**
([compute_counts.py](tests/test-data/query-agent/compute_counts.py)):
`match_all('ACK batch')` now translates to `(log LIKE '%ACK%' AND log
LIKE '%batch%')` — per-token AND conditions matching Tantivy's
token-level search semantics, instead of the old phrase match `log LIKE
'%ACK batch%'`.
4. **Regenerated expected results** across all query JSON files.
## Test plan
- [ ] Run query agent test suite against an environment with Tantivy
enabled
- [ ] Verify `match_all` content assertions catch intentionally
corrupted results
- [ ] Confirm post-flush Tantivy readiness poll works within the timeout
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Huaijin <haohuaijin@gmail.com>
## Summary
- Add E2E test coverage for SQL queries containing `--` inline comments
in the Logs search page
- Fix verified: queries like `-- comment\nSELECT _timestamp FROM stream`
no longer produce `SQL ParserError("Expected: end of statement, found:
_timestamp")`
- Add two reusable page object methods to `logsPage.js` for asserting
SQL error state
## What Was Broken
SQL queries with `--` style comments followed by a `SELECT` statement
were failing with:
> `SQL error: ParserError("Expected: end of statement, found: _timestamp
at Line: 1, Column: 45")`
The parser was not stripping comment lines before executing the query,
causing it to misread the `SELECT` column list as a second statement.
## Test Scenarios Covered
| # | Scenario | Tag |
|---|----------|-----|
| 1 | Single-line comment before SELECT | `@P0 @smoke` |
| 2 | Multiple comment lines before SELECT | `@P1` |
| 3 | Inline comment at end of a line (`SELECT ... -- comment`) | `@P1`
|
| 4 | Comment in the middle of a multi-line SQL query | `@P1` |
| 5 | Comment + WHERE clause using `str_match_ignore_case` | `@P1` |
| 6 | Comment-only query — must NOT produce `_timestamp` parser error
(edge case) | `@P2` |
## Files Changed
-
`tests/ui-testing/playwright-tests/Logs/logsqueries.sql-comments.spec.js`
← new spec file
- `tests/ui-testing/pages/logsPages/logsPage.js` ← two new assertion
methods added
- `.github/workflows/playwright.yml` ← spec registered in CI pipeline
---------
Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com>
🤖 Auto-generated E2E tests for #12607 — **healed to passing** against a
live OpenObserve before this PR was opened.
- Branch `autoe2e/pr-12607` (stacked on `main`), so the tests run
against the feature code and pass before it merges.
- Spec:
`tests/ui-testing/playwright-tests/Metrics/metrics-share-deep-link.spec.js`
The repo Playwright CI re-validates here. Review is for **quality +
coverage** — the tests already pass.
_Generated by the E2E Council pipeline._
**Test cases added / changed:**
| Test case | Type | What it verifies |
|---|---|---|
| `Default state — page loads with defaults, no auto-run` | new |
metrics page loads with default config, no metrics_data blob in URL, and
core UI elements visible |
| `Deep-link with bad metrics_data blob falls back to defaults` | new |
corrupt base64 blob does not crash the page — falls back silently to
default panel state |
| `Deep-link with unknown version blob falls back to defaults` | new |
blob with version 9999 is ignored and page shows default panel
configuration |
| `Deep-link with override params only builds panel from scratch` | new
| chart_type and query override params configure the panel without a
metrics_data blob |
| `URL syncs metrics_data blob after manual query and Apply` | new |
running a query adds a valid metrics_data blob to the URL with correct
version |
| `metrics_data blob excludes volatile data (id, title, description)` |
new | decoded blob does not contain id, title, or description keys |
| `Deep-link restores absolute from/to time range` | new | deep-link
with explicit from/to unix timestamps restores the time range correctly
|
| `Deep-link restores relative time period (period param wins)` | new |
period param takes precedence over from/to and restores relative time
range |
| `Share button disabled state — detects when web_url is not configured`
| new | share button is visible but disabled when web_url is not set |
| `Share button copies shortened URL to clipboard` | new | clicking
share generates a short URL containing /short/ and copies it to
clipboard with success toast |
| `Share button shows loading state during API call` | new | share
button enters loading spinner state during API call and clears after
completion |
| `Deep-link blob restore auto-runs query on mount` | new | valid
metrics_data blob restores panel config and auto-executes query without
manual Apply |
| `Deep-link restores auto-refresh interval` | new | refresh=30s param
sets the auto-refresh interval on the page |
| `Deep-link without stream — query populated, no stream selected` | new
| a deep-link with query but no stream_name does not crash the page |
| `Share URL contains absolute from/to timestamps, not period` | new |
shared URL freezes time to absolute from/to seconds, not relative period
|
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
- [x] ~add sort when to many groups, we can do this, but for now do not
do this~
- [x] remove the unsafe for memory pool
- [x] add approx_distinct improve, already fork, will add in next pr
- [x] fork vortex to add cast fix, and export
`.exclude_schemes([IntDictScheme.id()])`
- [x] ~enable_physical_uncorrelated_scalar_subquery test~
## Summary
- Fixes 2 hard-failing regression tests (traces autocomplete + pipeline
destination save)
- Improves 6 flaky pipeline timeout tests
## Root causes
### Pipeline destination save test (line 513)
`destinationListAddBtn`, `destinationTypeCard`, `toastError`, and
`toastSuccess` were referenced in the test but never defined in
`PipelinesPage`. Added the four missing locators with their verified
`data-test` attributes.
### Traces autocomplete test (line 139)
The test clicked `.view-lines` (a display-only div) and used
`page.keyboard.type()`, but Monaco only routes keyboard input through
its hidden `.inputarea` textarea. Switched to Monaco's
`editor.trigger('keyboard', 'type', {text})` API which properly updates
the editor model and fires the autocomplete provider.
### Pipeline timeout tests (lines 73, 181, 289, 368, 451, 478)
Reduced `addPipeline` timeout from 30s to 15s so tests fail faster
instead of hanging when the pipeline page doesn't load. Combined with
`slowMo: 500`, the old 30s timeout amplified cascading delays in serial
test mode.
## Verification
Both hard-failure tests pass against the pentest environment:
- `traces-regression-bugs.spec.js:139` — PASSED (editor content:
"select", suggestions: true)
- `pipeline-regression.spec.js:513` — PASSED (no duplicate
notifications)
## Test plan
- [x] Traces autocomplete test passes
- [x] Pipeline destination save test passes
- [ ] Pipeline timeout tests run within CI timeout (improved, not fully
resolved)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## Summary
- Adds `alerts-metrics-notification.spec.js` — a 3-test UI suite
verifying the full metrics alert chain
- Test 1 (P1): PromQL alert triggers via scheduler, verified in history
UI — self-contained using `ensureMetricsIngested()`
- Test 2 (P2): Template body contains required alert variables
(`{alert_url}`, `{alert_name}`, `{stream_name}`)
- Test 3 (P1): Manual trigger from alert list more-options menu sends
notification
- Includes `alert-metrics-notification-findings.md` documenting manual
testing results (API schema, PromQL vs SQL, trigger indicators)
## Test plan
- [ ] Test 1: Alert triggers on freshly ingested `cpu_usage` metrics and
history rows appear in UI
- [ ] Test 2: Template editor (Monaco) shows expected variables
- [ ] Test 3: Manual trigger menu option calls PATCH /trigger and
succeeds
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## Problem
The **Logs-Builder-Basic** e2e shard is frequently flaky — it times out
on `unflattened.spec.js` and, because it's a big shard, a single timeout
re-runs the whole thing. Example: [run
27521591530](https://github.com/openobserve/openobserve/actions/runs/27521591530/job/81341218030).
In that run:
- `unflattened.spec.js:205` — hard failure, **timed out (300s) on all 3
retries**
- `unflattened.spec.js:100` and `logsQueryBuilder-chart.spec.js:148` —
flaky (passed on retry)
## Root cause
The failures are **timeouts, not assertion bugs**. The unflattened
tests:
1. Slept **~76s of fixed `waitForTimeout`** across the file (pure dead
time).
2. Ran a retry loop that **re-ingested 3,848 records up to 5×/test**
while waiting for the `_o2_id` field to index after enabling *Store
Original Data*.
Each test + setup is ~80s on a fast local machine; CI runners are ~3-4×
slower **and** run all 4 specs in the shard in parallel against one
backend. That pushes individual tests right onto the 300s/test ceiling.
The re-ingestion balloon also hammered the shared backend, slowing the
other specs (`logsQueryBuilder`, `pagination`) — which is why they
flaked as collateral.
## Fix
- Add `waitForO2IdQueryable()` to the unflattened page object: **polls
the search API until `_o2_id` is actually queryable** instead of
sleeping a fixed interval and hoping indexing caught up.
- Gate the UI scan on that readiness signal, removing the 15s/5s blind
schema-propagation sleeps.
- Turn the expensive **re-ingest** retry loop into a cheap **re-query**
loop (attempts 5 → 3) — the gate guarantees the data is already in the
backend, so a UI miss is only render lag.
No change to what the tests actually assert.
## Verification (local, against self-hosted OpenObserve)
| Check | Result |
|---|---|
| Unflattened solo ×2 (cleanup between) | 2 passed each, `_o2_id` on
**row 0 / attempt 1** every time, **2.3m** (was 2.8m) |
| `logsQueryBuilder-chart:148` isolated ×5 | **5/5 passed** → not
independently flaky (collateral) |
| **Full 4-spec shard, 4 workers (CI scenario)** | **52 passed, 0
failed, 0 flaky** |
The retry loop now never fires — the readiness gate makes the first UI
scan succeed every run, killing both the timeout and the cross-spec
contention.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## What
Removes `tests/ui-testing/playwright-tests/API/` and its sole file,
`dashboards-multi-window-vrl-api.spec.js` (708 lines).
## Why it is safe to remove
**1. It is never run.**
- Absent from the Playwright CI matrix in
[.github/workflows/playwright.yml](https://github.com/openobserve/openobserve/blob/main/.github/workflows/playwright.yml)
— CI runs only the per-folder `run_files` lists, and there is no `API`
`testfolder`.
- Not referenced by any config, script, `package.json`, or workflow in
the repo (only self-references inside the file).
- It would execute solely via a bare local `npx playwright test`, never
in CI.
**2. It is not actually a UI test.**
It drives no browser UI — it uses `page.request.*` to call REST
endpoints directly (`/api/{org}/dashboards`, `/_search_multi_stream`)
and parses SSE. It lived under `ui-testing/` only to borrow the
Playwright auth/fixtures.
**3. Coverage is fully duplicated by the pytest API suite, which does
run in CI.**
The pytest twin was added in the same PR and runs via
[.github/workflows/api-testing.yml](https://github.com/openobserve/openobserve/blob/main/.github/workflows/api-testing.yml)
(`rye run pytest` over `tests/`; `dashboards/` is not ignored). The 10
JS tests map 1:1 to
`tests/api-testing/tests/dashboards/test_multi_window_vrl.py`:
| JS spec test | pytest equivalent |
|---|---|
| multi-window no VRL | `test_create_dashboard_multi_window_no_vrl` |
| multi-window + non-ResultArray VRL |
`test_create_dashboard_multi_window_non_result_array_vrl` |
| multi-window + ResultArray VRL |
`test_create_dashboard_multi_window_result_array_vrl` |
| all 3 panel types | `test_create_dashboard_all_panel_types_combined` |
| `_search_multi_stream` no VRL | `test_search_multi_stream_no_vrl` |
| `_search` baseline VRL | `test_search_single_stream_vrl_baseline` |
| non-ResultArray VRL per hit |
`test_search_multi_stream_non_result_array_vrl` |
| ResultArray VRL diff | `test_search_multi_stream_result_array_vrl` |
| ResultArray VRL SSE structure |
`test_search_multi_stream_result_array_vrl_response_structure` |
| per_query_response=false |
`test_search_multi_stream_without_per_query_response` |
`test_multi_window_vrl_v2.py` adds 5 further cases. Both the JS and
pytest files originated together in PR #10860 (multi-window dashboard
VRL fix); this removes the unused JS duplicate and keeps the maintained
pytest version.
## Risk
None to test coverage — the underlying behavior remains covered (and
exceeded) by the CI-executed pytest suite.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## What
Fixes the failing **Logs-Regression** test `should show correct error
message identifying the problematic field @errorMessage @P0 @regression`
([logs-regression-bugs.spec.js:330](tests/ui-testing/playwright-tests/RegressionSet/logs-regression-bugs.spec.js#L330)).
Failed identically in the latest scheduled runs:
- OSS:
https://github.com/openobserve/openobserve/actions/runs/27405296335
- ENT:
https://github.com/openobserve/o2-enterprise/actions/runs/27405221785
(The spec is shared from OSS and merged into the enterprise test tree,
so this single OSS fix covers both.)
## Root cause
Not a product bug — the test wasn't updated after the query-error UI
redesign (`QueryErrorState.vue` + `ErrorDetailPanel.vue` /
`useQueryError`). The backend message `No field named
nonexistent_field_xyz_12345. Valid fields are e2e_automate._timestamp.`
is now split at the first `". "`:
- **summary line** (`data-test="error-detail-summary"`, always visible)
→ `No field named nonexistent_field_xyz_12345.` — carries the field name
- **detail body** (`data-test="error-detail-body"`, behind the expand
toggle) → `Valid fields are e2e_automate._timestamp.`
The page-object helper `getDetailedErrorDialogText()` read **only** the
expanded detail body, so it never captured the field name the assertion
checks for (`Received string: "valid fields are
e2e_automate._timestamp."`).
## Fix
Rewrote `getDetailedErrorDialogText()` in
[logsPage.js](tests/ui-testing/pages/logsPages/logsPage.js) to expand
the detail panel and return the **whole** `logs-search-error-state`
container text (summary line + detail body), falling back to the detail
body if the container is unavailable. Test-only change — no assertion or
product code touched; the assertion was always correct.
## Verification
Run locally against a hosted instance (`http://localhost:5080`):
- `should show correct error message identifying the problematic field`
→ **PASS** (21.7s)
- Stability `--repeat-each=2` → **PASS x2**, no flakiness
Only one caller of `getDetailedErrorDialogText()` exists (this test), so
the change is fully contained.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Review feedback addressed
- **Robust fallback:** when the full error container isn't visible (e.g.
a rendering race), the helper now stitches the summary line
(`error-detail-summary`) + detail body (`error-detail-body`) together
explicitly, instead of falling back to the detail body alone — so the
field name can never be dropped.
- **JSDoc:** `@returns` now documents that the value is the full error
text (summary line + detail body).
No change to the test flow or assertion — only the page-object helper
that reads the error text was touched.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Close#12345
## Problem
OTLP histogram metrics sent over HTTP/JSON (e.g. otel-collector with
`encoding: json`) created streams but stored 0 events, with no
`_count`/`_sum`/`_bucket` streams. Int-valued counters/gauges were
silently recorded as 0. The protobuf path was unaffected, which is why
this only surfaced for collectors explicitly configured with JSON
encoding.
## Root cause
`otlp_json` deserializes the body directly into opentelemetry-proto's
serde-derived types, which only partially implement the proto3 JSON
mapping (64-bit integers encoded as strings, enums by name, zero/empty
fields omitted). `Metric.data` is a flattened `Option` oneof, and serde
converts any inner deserialization error of a flattened `Option` into
`None` instead of failing — so the whole metric silently became `data:
None`, while the stream itself had already been created by the schema
setup. Same mechanism on `NumberDataPoint.value` turned string `asInt`
values into 0.
## Changes
- Upgrade opentelemetry stack 0.31 → 0.32, which fixes string-encoded
u64 handling for plain histograms and summaries upstream.
- Add `otlp_json_compat` normalizer for the gaps still present in 0.32
(each verified empirically, with unit tests):
- string `asInt` values on data points and exemplars
- `aggregationTemporality` as enum name
- `Exemplar`'s value oneof is not flattened in 0.32
- missing `serde(default)` on
`ExponentialHistogramDataPoint`/`Buckets`/`ValueAtQuantile`/`Exemplar`,
which rejected omitted zero/empty fields
- Count and log metrics whose data oneof failed to decode
(`partial_success.rejected_data_points`) instead of silently skipping
them, and stop overwriting specific partial-success error messages with
the generic retention message.
- Adapt to OTLP 0.32 API additions (`KeyValue.key_strindex`,
`AnyValue::StringValueStrindex`).
- Upgrade workspace reqwest 0.12 → 0.13, required because
opentelemetry-http 0.32 only implements `HttpClient` for reqwest 0.13
(`rustls` feature replaces `rustls-tls-native-roots` via the platform
verifier; `json`/`query`/`form` are now explicit features). The
`segment` crate is pinned to reqwest 0.12, so `TELEMETRY_CLIENT` moved
into `telemetry.rs` as a plain `reqwest::Client` that posts the Track
message itself, replicating `segment::HttpClient::send`.
- Remove the unused pyroscope profiling integration (optional feature,
dependencies, agent setup, `ZO_PROF_PYROSCOPE_*` config).
## Verification
- 10 unit tests in `otlp_json_compat` covering the full payload matrix
(collector-style string integers, enum names, omitted zero fields,
exemplars with and without trace context, already-normalized payloads,
malformed input).
- End-to-end: posted the issue's collector-style JSON payload to
`/api/default/v1/metrics` — `_bucket` (3 events, correct cumulative
counts with exemplars attached), `_count`, `_sum` populated for
histogram and exponential histogram, summary quantiles stored, `asInt`
counter stored as 42 instead of 0.
- `cargo clippy -- -D warnings` clean; lib test failures are
pre-existing local-environment issues that reproduce identically on
main.
## Summary
- Adds a single GeneralTests spec (`edition-features.spec.js`) that
validates the header edition button, the `EnterpriseUpgradeDialog`,
every enterprise feature card, and the `o2.ws/*` documentation links
each card opens.
- Two tests live in the spec; exactly one runs per environment. The
active edition is detected from the rendered header button label (most
direct signal of what the user sees, tolerates frontend/backend
build-flag mismatches like an OSS frontend served by an ENT backend).
- Pairs with the same-named branch on `openobserve/o2-enterprise` so ENT
CI tree-merges this spec in and runs the ENT variant.
## What the spec validates
- Header button label per edition (`Get OpenObserve Enterprise (Free)`
vs `Edition: Enterprise`)
- Hero panel (title, badge, primary button, features title, Learn More
button)
- 20 enterprise feature cards (presence + external-link icon + BETA/HA
badges)
- Cloud-only `Bring Your Own Bucket` card stays hidden outside Cloud
- Every feature card click opens a popup that lands on the expected
`openobserve.ai` doc page
- Hero external link buttons (`Download Now`/`Learn More` on OSS, `Learn
More` on ENT) — `Get Free License` intentionally not clicked because it
switches the active org to the meta org
## Files
-
`tests/ui-testing/playwright-tests/GeneralTests/edition-features.spec.js`
(new)
- `tests/ui-testing/pages/generalPages/editionFeaturesPage.js` (new —
page object + truth table of slugs / expected doc paths)
- `tests/ui-testing/pages/page-manager.js` (register
`pm.editionFeaturesPage`)
## Test plan
- [x] OSS local run: 1 passed, 1 skipped (~2.0m)
- [x] ENT local run: 1 passed, 1 skipped (~2.0m)
- [x] Sentinel audit passed (POM compliance, no raw selectors in spec,
no hardcoded secrets, no console.log)
- [ ] OSS CI green on this PR
- [ ] ENT CI green on companion PR
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
## What changed
- Extracted parser-limitation guards into `isParserLimitation()` and
refactored `useSearchQuery.ts` to **not block the search** when the PEG
parser flags a false-positive error — the squiggle still shows but the
query proceeds to the backend
- Added a 4th parser-limitation guard for `found="("` inside window
function clauses (covers `SUM(COUNT(*)) OVER (...)`, `NTILE(N) OVER
(PARTITION BY COALESCE(...)...)`)
- Unified all four `fetchRumEventsForTrace` queries to a consistent
±60,000,000 µs buffer around the trace window
- Added `allowedTracingUrls` config + `propagatorTypes` to the RUM SDK
config snippet in `FrontendRumConfig.vue`
- Added an "Explain" query dropdown menu item in the logs SearchBar
- Fixed various traces/RUM UI padding and border issues
(PlayerEventsSidebar, TraceDetails, TracesMetricsDashboard,
SessionViewer)
- Updated i18n keys across all 12 language files
## Why
Two valid SQL queries produced false-positive validation errors
(`SUM(COUNT(*)) OVER (...)`, `NTILE(...) OVER (PARTITION BY
COALESCE(...)...)`), and even after suppressing those, the query was
still **blocked from executing** by `useSearchQuery.ts`. The PEG parser
has known gaps that the DataFusion backend handles correctly — there was
no reason to block valid queries. The RUM event queries also had
inconsistent time buffers across different calls (some used ±10s, others
±60s), causing missed RUM events in trace correlation.
## Approach
- Extracted the 4 parser-limitation guards from `validateSql()` into a
pure sync `isParserLimitation()` function exported from
`sqlDiagnostics.ts`, then reused it in `useSearchQuery.ts` to let query
execution continue when only parser limitations are detected
- Replaced inline constants with a shared `RUM_TIME_BUFFER_US =
60_000_000` in `useRumSpanBuilder.ts` so all four
`fetchRumEventsForTrace` queries use the same ±60s window
- Added `{ timeout: 30000 }` to the aggregate mutation test that was
timing out on ~5,500 broken queries
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
## Summary
- **Root cause**: `alertManagement.js` used
`alert-list-search-across-folders-toggle-btn` but `AlertList.vue`
renders a plain `<button>` with
`data-test="alert-list-search-across-folders-toggle"` — no `-btn`
suffix.
- Fixed the same stale `-btn` selector in `dashboardPage.js` (dashboard
toggle is also a plain `<button>`).
## Failing tests fixed
- `alerts-regression.spec.js:128` — Bug #9967: PromQL alert creation
- `alerts-stream-switching-regression.spec.js:338` — Bug #11577: Alert
preview chart y-axis
Both were timing out at 45s trying to click the nonexistent
`*-toggle-btn` selector.
## Files changed
- `tests/ui-testing/pages/alertsPages/alertManagement.js` — 2 selector
fixes (lines 214, 481)
- `tests/ui-testing/pages/dashboardPages/dashboardPage.js` — 1 selector
fix + removed stale OSwitch comment
## Test plan
- [ ] Run `npx playwright test
playwright-tests/RegressionSet/alerts-regression.spec.js` locally
- [ ] Run `npx playwright test
playwright-tests/RegressionSet/alerts-stream-switching-regression.spec.js`
locally
- [ ] Verify CI Alerts-Regression job passes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- TestDino quota (~75k/month) was being consumed at ~4x rate because
each failing test uploads one result entry per retry attempt (original +
3 retries = 4 entries per test)
- This PR adds deduplication: if all retry results share the same
`status` and error first-line, only the final attempt is uploaded
- Retries with differing status or error message are preserved in full —
they carry real diagnostic signal (e.g. a test that times out then fails
with a different error)
## How it works
```
getErrorFingerprint(result) → first line of error.message (stable across retries)
allIdentical = every retry has same status AND same error fingerprint
→ true: upload only results[last] (strips ~3 entries per test)
→ false: upload all results as before (keeps diagnostic data)
```
## Expected quota impact
Based on last month's data: ~65k attempts → ~16k (estimated ~75%
reduction for OSS Portal alone)
## Test plan
- [ ] Trigger a CI run that has failures with retries — confirm the
workflow log shows `stripped N identical retry results`
- [ ] Trigger a run with a genuinely flaky/environment-variable failure
— confirm all retries are preserved in TestDino
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- **13 Playwright E2E journey tests** for the Model Pricing feature
(\`/settings/model_pricing\`), consolidating all P0–P2 scenarios into
journey-based flows
- New page object \`ModelPricingPage\` under \`pages/generalPages/\`
- API cleanup integrated into the centralized \`APICleanup\` class and
called from \`cleanup.spec.js\`
- CI matrix updated: \`ZO_MODEL_PRICING_ENABLED=true\` env var added;
\`model-pricing.spec.js\` added to the \`GeneralTests\` job
- **Bug fix**: \`waitForBuildTabLoaded\` now polls the Monaco editor for
non-empty content (Phase 3), fixing a timing race where Build tab SQL
tests read an empty query string on slower CI machines
## Test coverage
| Journey | Scenarios covered |
|---------|------------------|
| List, tabs & search | Controls, tab switching, search, built-in tab |
| Create & validation | All form validation cases + successful save |
| Edit, cancel & delete | Full lifecycle |
| Duplicate & clone | Org model duplicate + built-in clone |
| Bulk actions | Select, export JSON, bulk delete |
| Toggle | Disable/re-enable with correct visual state |
| Test match dialog | Empty state, no-result, clear, match result |
| Import | Valid JSON, correction flow, cancel |
| Editor features | Multi-tier, pricing template, pattern examples |
| Pricing drawer | Overflow link → drawer with all price keys |
| Empty state | CTA when org has no custom models |
| Cost applied | LLM span matched → cost stored correctly |
| No matching model | Unknown model name → zero cost stored |
## Bug fix: Build tab query editor timing
\`waitForBuildTabLoaded\`'s Phase 2 (chart-type-selected signal) fires
as soon as \`initializeFromQuery\` sets the chart type — before
\`makeAutoSQLQuery\` has run. On slower CI machines this left
\`queries[0].query\` empty when tests read the editor, causing the "SQL
ON + Auto mode" and "Auto → Custom" tests to receive \`""\` instead of
generated SQL.
Phase 3 polls the Monaco editor
(\`[data-test="logs-search-bar-query-editor"]\`) for non-empty content,
confirming \`onBuildQueryGenerated\` has been called and the query is
ready.
## Test plan
- [x] All 13 model pricing tests pass locally
- [x] \`logsQueryBuilder-editor.spec.js\` (23 tests) — all pass with
Phase 3 fix applied
- [x] Sentinel code audit passed (no blocking issues)
- [x] API cleanup added to \`cleanup.spec.js\` via
\`apiCleanup.cleanupModelPricingModels()\` — 404-safe for instances
where the feature is disabled
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
## Summary
- Adds **239 new tests** covering the vortex columnar file format across
78 scenarios from the test plan
- **`tests/api-testing/tests/vortex/`** — 9 test modules (83 tests):
format conversion, schema evolution, data integrity, FTS, downsampling,
API format awareness, server config, and performance/stress
- **`tests/api-testing/tests/query_agent/test_3_vortex.py`** — 156
tests: the full 155-query SQL correctness suite + scenario 39
(self-join) run post-flush against vortex files
- **`api-testing.yml`** — adds `--ignore=tests/vortex` to the OSS
default-set run; vortex tests require the enterprise binary with
`ZO_FILE_FORMAT=vortex` and are gated to enterprise CI only
## Test plan coverage
| Module | Scenarios |
|--------|-----------|
| `test_format_conversion.py` | 1–13, 7, 8, 9, 10 |
| `test_schema_evolution.py` | 14–25 |
| `test_data_integrity.py` | 55–62 |
| `test_fts.py` | 51–54 |
| `test_downsampling.py` | 47–50 |
| `test_api_format.py` | 71–73 |
| `test_config.py` | 66–67 |
| `test_performance.py` | 74–78 |
| `test_3_vortex.py` | 26–46 (155 queries + JOIN) |
## Enterprise CI
Two dedicated jobs added in the companion `openobserve/o2-enterprise` PR
on the same branch:
- **`api_query_agent_vortex`** — runs `test_3_vortex.py` with
`ZO_FILE_FORMAT=vortex`
- **`api_vortex_format_tests`** — runs the full `tests/vortex/` suite
with 4-phase server lifecycle (parquet→vortex→parquet→vortex+compaction)
OSS CI is unchanged except for the ignore rule — no vortex tests run
against the OSS binary.
## Test checklist
- [ ] `tests/vortex/` collects cleanly: `83 tests collected`
- [ ] `tests/query_agent/test_3_vortex.py` collects cleanly: `156 tests
collected`
- [ ] Enterprise CI `api_query_agent_vortex` job passes
- [ ] Enterprise CI `api_vortex_format_tests` job passes
🤖 Generated with [Claude Code](https://claude.ai/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- **Streams-Regression** — `isQueryEditorExpanded()` used
`container.locator('.editor-fullscreen')` which searches *descendants*,
but Vue adds `editor-fullscreen` directly to the container element
itself. Fixed by using combined CSS selector
`.query-editor-container.editor-fullscreen`.
- **Logs-Regression** — `enableHistogram()` checked canvas visibility to
detect histogram state. Canvas is absent before the first query runs, so
it always assumed histogram was OFF and toggled it OFF when it was
already ON. This removed `[data-test="logs-search-result-bar-chart"]`
from the DOM (it is `v-if`-gated), breaking
`expectBarChartHasContent()`. Fixed by using
`logsQueryPage.isHistogramOn()` (reads button `data-state` attribute),
consistent with how `ensureHistogramState()` already works.
## Failing tests fixed
- `should allow query expand icon to work after navigating from stream
explorer (#9337)` — Streams-Regression
- `should not show blank histogram after cancelling query @bug-4315` —
Logs-Regression
- `collapsing/expanding index list should redraw histogram chart` —
Logs-Regression
## Test plan
- [ ] Ran `streams-regression.spec.js --grep "9337"` locally — passes
- [ ] Ran `logs-regression-bugs.spec.js --grep "blank histogram after
cancelling"` locally — passes
- [ ] Ran `logs-regression-bugs-2.spec.js --grep "collapsing/expanding
index list"` locally — passes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- Adds `|| echo '{"jobs":[]}'` fallback on the `curl` call so a network
error or GitHub API rate-limit response never produces an empty/invalid
variable
- Adds `2>/dev/null || echo "0"` on both `jq` calls so a malformed API
response (non-JSON error body) falls back to `"0"` instead of
propagating a non-zero exit under `-eo pipefail`
- Changes the arithmetic comparison to `[ "${CURRENT_ATTEMPT_JOBS:-0}"
-eq "${TOTAL_MATRIX_JOBS:-0}" ]` with `2>/dev/null` so a `null` jq
output can't trigger "integer expression expected" and kill the step
**Failure mode fixed:** On re-runs, multiple matrix jobs query the same
GitHub API endpoint concurrently with the shared `GITHUB_TOKEN`. A
rate-limit `403` or transient network hiccup returns non-JSON, `jq`
exits non-zero, and with `set -eo pipefail` the whole "Check if this is
a rerun" step fails — blocking the job before any tests run.
**Degraded behaviour:** When the API is unavailable, both counts fall to
`0`, `0 -eq 0` → `rerun_type=all` → step succeeds → all tests run (same
as a normal rerun-all). The TestDino optimisation is skipped but nothing
is blocked.
Companion PR: openobserve/o2-enterprise#<ENT_PR>
## Test plan
- [ ] Trigger a re-run of a workflow run — "Check if this is a rerun"
step should pass even if GitHub API is slow/rate-limited
- [ ] Verify `rerun_type` is still correctly set to `failed` when the
API responds normally
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
Two high-value improvements to the query agent regression test data:
1. **Data variety** — Per-field rotation offsets (`qi*13 + i*7`) replace
the old uniform `idx % len(pool)` approach. Previously all fields picked
from the same pool position, creating only 5 fixed row shapes per query.
Now field-value combinations vary independently per field.
2. **NULL stress** — 8 fields (`bot_flag`, `info_tag`,
`upstream_error_code`, `page_url`, `endpoint_path`, `resource_path`,
`variant_tag`, `threat_flag`) get `None` for 40% of records per query.
This exercises `IS NULL` / `IS NOT NULL` / `COALESCE` paths that were
previously untested.
## Changes
- `data_gen.py`: Rewrote `make_record()` with per-field rotation and
NULL injection
- 24 queries: Adjusted WHERE/HAVING filters to match new data
distribution
- All 12 query JSONs: Regenerated expected results via
`compute_counts.py`
## Impact
- 400 queries all have results (18 pre-existing legacy/skip queries
unchanged)
- No change to record count (still 5/query, 2000 total) — test time
unaffected (~7 min)
## Test plan
- [x] `compute_counts.py` regenerates all expected results
- [ ] Run `pytest tests/query_agent/ -v --tb=short` against dev
environment
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Shrinath RaoandClaude Opus 4.8
Neha PandClaude Sonnet 4.6
ktx-mihirandgithub-actions[bot]
ktx-kirtanandktx-vaidehi
Sai Nikhil
ktx-akshay
Hengfei Yang
ktx-vaidehi
Bhargavandcopilot-swe-agent[bot]
udev
Omkar Kesarkhaneandgithub-actions[bot]
