mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 16:09:05 +00:00
Shrinath Rao
98e03bca15
## Stabilize the Alpha1 (cloud) e2e suite + regression carryover Root-caused and fixed the failures surfacing on the Alpha1 cloud runs — investigated live via Playwright MCP against alpha and, where possible, a local build. Grouped by theme. Every fix is non-masking (still fails if the underlying is genuinely broken). Pairs with ENT #2351 (`max-parallel` + Logs-SelectStar `quick_mode`). ### Runtime / infra resilience (whole-suite) - **Auth self-heal on 401 / session expiry** — the shared cloud session + per-org passcode are minted once and reused all run; on load/contention they expire mid-run (`User not authenticated`, `HTTP 401 Unauthorized`). `cloud-auth.authedRequest` refreshes the passcode and retries on 401 (escalating to a full Dex re-login if the session is dead); `navigateToBase` re-authenticates and resumes instead of throwing. Also routed the alert **template-creation** API through it (401 there cascaded into "Failed to navigate to templates"). Gated on `isCloudEnvironment()` — no self-hosted change. - **`--disable-dev-shm-usage`** on the alpha1 Chromium launch — the EKS pods used the default 64MB `/dev/shm`; under 5 workers on long/heavy shards Chromium exhausted it and the pod OOM-killed → "runner lost communication" (frozen step + 404 logs). Route shared memory to /tmp; eliminates the infra deaths. - **Service graph queries a wide window** (`period=6h`) — the tab defaulted to Past 15 Minutes and rendered "No service graph data" before the daemon-processed topology fell inside the window. ### Pipelines - `queryStream(stream, expectedMinCount)` **polls until the record count lands** (pipeline output arrives async) — was single-shot ("Expected 5, got 3"). - `openPipelineForEdit` **searches + polls-with-reload** for the edit button (list not refreshed after save → 45s click timeout). - `exploreStreamAndNavigateToPipeline` **reloads the streams list** while polling for Explore, and **falls back to a direct logs URL** when a freshly-ingested source stream never surfaces in the list (stream-stats lag). ### Alerts - Alert-destination URL → **public `example.com` webhook** (self-host URL hit the cloud **SSRF guard** → form never closed). - **Force-click the virtualized folder item** (#9484) — the row re-rendered mid-click → 45s "not stable". - **Export button readiness** — it's rendered only on selection; wait for visible+enabled (re-selecting if needed) before clicking. - **`deleteAlertByRow` cleanup readiness** — the just-created alert's more-options kebab needs longer than 5s under load. ### RegressionSet - **Use the shard's org** (`getOrgIdentifier()`), not hardcoded `org_identifier=default`, in `logs-bugs` — on cloud the data lives in `auto_org_N`, so the logs page never rendered. - **Unique stream names** for `logs-9754` / `logs-bugs:1419` — fixed names collided with concurrent shared-org runs ("stream is being deleted"). ### Traces / correlation - **Migrated `logs-bugs:995` (bug-11469)** to the redesigned correlation UI — the old `log-correlation-btn` / `correlation-dashboard-close` are gone; repointed to the inline `correlated-traces-tab`, assertion (`expectNoContextMenuVisible`) unchanged and **verified live on alpha** (no fake-pass). ### Workflows (ENT-only) + misc - **Healed the workflows destination flow** and forced-type Connection-step assertion; alert/report list-readiness hardening. - **Enrichment tables list** readiness (30s + re-nav retry) — "slow under load". ### Notes - Validated live via MCP (alpha) and a local build where the feature exists. - Alpha1 config already runs `retries: 2`; the remaining first-attempt flakes are shared-env load, addressed on the ENT side via `max-parallel`.