mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 00:09:35 +00:00
Vamsi Karthik RandClaude Opus 4.8
899bb681a8
## What
Consolidated stabilization of the **Alerts**, **SDR** and **Pipelines**
shards of the
alpha1 cloud E2E suite (`playwright_alpha1.yml`) to green — the shards
still failing
after the earlier per-module stabilization PRs merged. Rebased on
`main`.
Across these shards the original run had: **Alerts** 1 hard fail,
**SDR** 1 hard fail +
1 flake, **Pipelines** 1 hard fail + several flakes. Root causes were an
ambiguous
operator-select selector plus cloud data/UI-readiness races (editor &
node-sidebar mount,
hover-flyout nav, freshly-ingested destination streams lagging their
list/table) and
transient backend 5xx on ingestion.
**No `waitForTimeout` added as a fix, no `force:true`, no
`.first()`/`.nth()` band-aids,
no retry-count bumps to hide failures, no weakened assertions. No
worker-count,
parallelism, playwright-config or CI/yml changes** — test code only.
## Root cause & fixes
**Alerts**
1. **Operator select — strict-mode violation** (`alerts-e2e-flow`, state
management).
The operator control is an OSelect that renders an aria-hidden native
`<select><option>=</option>` alongside the visible popover item, so the
unscoped
`getByText('=', {exact})` matched **two** nodes. Scope the option click
to
`[data-test="alert-conditions-operator-select-popover"]` +
`getByRole('option')`
(excludes the aria-hidden native select, mirrors the working
`alertCreationWizard.js`
pattern), and gate on the popover open/close instead of the trailing
`waitForTimeout`s.
(`alertManagement.js`)
**SDR**
2. **Stream Detail button — 45s timeout under load**
(`multipleSDRPatterns` @slow,
`ingestionTimeHash`). `openStreamDetail` resolved the button by the
icon's title text,
which the OTable can render after the name cell, and the retry only
re-clicked a
possibly-detached row without re-fetching the list. Resolve by the
stable
`data-test="log-stream-schema-btn"` and reload + re-search the streams
list between
attempts, keyed to the real "detail sidebar open" signal
(`schema-update-settings-button`). (`streamAssociationPage.js`)
**Pipelines**
3. **Editor / node-sidebar mount race** (`selectStream`, all pipeline
flows). A bare click
on the stream-input button dead-waited the 45s action timeout when the
VueFlow editor +
NodeSidebar missed the first render under concurrent load. Moved a
**non-destructive**
mount guard into `addPipeline`: after clicking Add, if the button hasn't
rendered and
we're still on the list (add button visible), re-click to re-trigger
navigation — no
`page.reload()` (a reload of the editor route re-runs the auth callback
on cloud and
stalls the shard). (`pipelinesPage.js`)
4. **`openPipelineMenu` — hover-flyout nav timeout.** Routed through the
left-nav hover
flyout, which self-closes on settle and, under load, burned ~30s
retrying to open before
its goto fallback fired — tipping long condition tests over the 180s
test timeout.
Promoted the already-proven `goto` to the primary path and skipped the
flyout (it adds
no coverage here). (`pipelinesPage.js`)
5. **Destination-stream data-flow / render latency**
(`pipeline-dynamic`, query-source
flow). Extended `openTimestampMenu`'s re-search loop (destination stream
indexing can
lag past the old budget) and `waitForQueryNodeVisible` 30s→45s
(query-node canvas render
lags under full-shard load) — both within the CI test timeout, both exit
early on the
real signal. (`pipelinesPage.js`)
**Ingestion resilience (Pipelines + shared helper)**
6. **Transient backend 5xx aborted ingestion.** `fetchWithRetry` only
retried thrown
network errors; a `503`/`502` is a "successful" fetch, so ingestion
threw on the first
one when the env returned transient 5xx during load spikes. Retry 5xx
with backoff in
both ingestion helpers — a persistent 5xx still surfaces after
`maxRetries`, so genuine
outages aren't masked. (`pipelinesPage.js`, `ingestionPage.js`)
## Validation
- **CI — Alerts / SDR / Pipelines green (alpha cloud):**
[Pipeline](https://github.com/openobserve/o2-enterprise/actions/runs/29809833252).
[SDR,
Alerts](https://github.com/openobserve/o2-enterprise/actions/runs/29771293789)
- **Original failing run:**
https://github.com/openobserve/o2-enterprise/actions/runs/29720987587
- **Local (real alpha cloud, `playwright-alpha1.config.js`):** each
previously-failing test
re-run to green — `pipeline-conditions.spec.js` @ `--workers=3` 6/6,
`pipeline-dynamic`
72 & 97, `pipelines:180` (query source) solo; Alerts operator flow and
SDR
`openStreamDetail` verified against alpha. 5xx-retry control flow
verified (2xx/4xx
unchanged, transient 503 recovers, persistent 503 still surfaces).
## Notes
- 4 files change (3 page objects + the shared ingestion helper). No
specs, shared
cleanup/setup, config, or workflow files touched.
- Any residual Pipelines tail is cloud latency / env backend health
(e.g. runs where the
env returns sustained 503s) — not test code; the durable lever there is
cross-shard
concurrency / env capacity, out of scope for this PR.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>