mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 00:09:35 +00:00
c49d156e7ea77e2bf64e2d62ba173b4b4479c2d4
1038
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Shrinath Rao
|
c49d156e7e |
test(alpha1): multi-org sharding for the alpha1 e2e run (#13530)
Companion OSS change to enterprise 2278 (alpha1 workflow). Same branch slug on both repos. ## What Supports running the Alpha1 (cloud) e2e matrix with its shards **divided across multiple orgs** for data isolation, on top of the shared-auth login (one Dex login, reused by all shards). Both changes are in \`global-setup-alpha1.js\`. ## Changes 1. **Ingest into the shard's own org (\`ORGNAME\`)** instead of the org baked into \`cloud-config.json\`. The shared-auth barrier logs into one org; without this, every shard ingested into that one org even when \`ORGNAME\` differed. 2. **Re-fetch the shard's own org passcode in the shared-auth path.** Each org has its own \`o2oi_\` default ingestion token (see \`core/organization.rs\` \`get_passcode\`). The barrier captured only its org's token, so shards on other orgs got \`401 Unauthorized Access\` on ingest. \`verifySharedAuth\` now calls \`fetchCloudConfig(page)\` to overwrite \`cloud-config.json\` with the current shard's passcode (fetched via the session, which is a member of every org). Keeps the single shared login — no login storm — while giving each shard the correct per-org credential. ## Context Part of the alpha1 parallelization effort: - Shared-auth artifact removes the 13-way concurrent Dex login storm (enterprise 2278). - Shards divided across 6 orgs (enterprise 2278 matrix) for data isolation. - These two OSS fixes make per-org ingestion actually work. ## Testing Validated by dispatching the enterprise \`playwright_alpha1.yml\` with \`o2_opensource_repo=test/alpha1-parallel-shards\`. Verifying each shard now ingests \`e2e_automate\` into its own org with \`status:200\` (previously \`401\` for non-barrier orgs). |
||
 Hengfei Yang
|
4231ca5833 |
perf(schema): share stream settings via Arc and centralize cache mutation (#13535)
Design at: #13465 PR 2 of the plan in #13465 (finding 1: the settings cache O(S) deep clone per mutation). ## Background The stream settings cache is a locked map (`STREAM_SETTINGS`) plus an ArcSwap read snapshot (`STREAM_SETTINGS_ATOMIC`) introduced in `405794db08` for lock-free ingestion reads. That architecture is kept — the read/write ratio justifies it. The problem was the constant factor: - every mutation republished the snapshot by **deep-cloning every cached `StreamSettings`** (vectors, UDS field lists, maps). The snapshot is republished on every schema change event cluster-wide (any new stream or field, on every node) plus every cache-miss backfill — at 100k+ streams this is a full deep copy of the map at minute-level frequency; - every point read (`get_settings`, called on each ingestion request) deep-cloned the returned `StreamSettings`, including UDS lists that can hold hundreds of strings; - the two maps each held a complete deep copy of all settings. ## Changes **Arc-ified values.** Both maps now store `Arc<StreamSettings>`. Snapshot republish is a shallow clone (String keys + refcount bumps), point reads return an `Arc` clone, and the maps share one set of values, roughly halving the cache's memory. **Centralized mutation.** The map statics are private; all writes go through `put_stream_settings` / `put_stream_settings_batch` / `remove_stream_settings`, which republish the snapshot under the write lock so snapshots cannot be published out of order. Startup load collects settings and publishes once via the batch API. Callers can no longer accidentally rebuild the snapshot — the pattern that caused the original quadratic startup. Lock-free reads stay available via `get_stream_settings_atomic`; the status API uses a new `get_stream_settings_stats()` accessor. **API ripple, kept minimal.** `get_settings()` now returns `Option<Arc<StreamSettings>>`. Field access and `unwrap_or_default()` call sites compile unchanged; the few paths that edit a local copy before persisting (`update_stream_settings`, LLM/gen-ai settings, dashboard distinct fields) do an explicit `(*s).clone()`. The `get_stream_setting_*` helpers are generic over `Borrow<StreamSettings>`, so both `&Option<StreamSettings>` and `&Option<Arc<StreamSettings>>` callers work — including the downstream enterprise call sites, which were checked and need no changes. **`SchemaCache::fields_map` Arc-ified** — cloning a cached schema entry no longer deep-copies the field-name index, which holds thousands of entries for wide streams. ## Measurements (criterion bench used during development, not kept in-tree) | operation | result | |---|---| | bulk load 10k / 50k / 100k streams | 0.80ms / 4.4ms / 8.7ms — linear, ~87ns per stream | | point write against 100k-entry cache | 3.5ms (shallow snapshot republish) | | point read against 100k-entry cache | 7.9ns (lock-free) | For comparison, the field report in #13465 measured ~27ms per stream insert at 58k cached streams under the old per-mutation deep clone; reads previously deep-cloned the settings value on every call. ## Testing - new unit test for the mutation API (put/batch/remove, snapshot sharing via `Arc::ptr_eq`, republish keeps other keys) - full test suites pass for all touched crates (infra, db, schema, stream, core, api-search, api-management, search_service, compaction, enrichment-data); the only failures are pre-existing local-environment ones also failing on main (sqlite-path retention tests, one env-dependent index test, one enterprise doctest) - `cargo clippy --workspace --all-targets`: no findings in touched files Finding 2 (grouped stream enumeration) landed in #13536; this branch has been rebased onto main (the #13531 overlap is resolved). |
||
|
Shrinath Rao
|
184268ff1b |
test(ci): loud stdout banner when Playwright retries a test (#13529)
## What Adds a small **log-only** Playwright reporter that prints a LOUD, unmissable banner to stdout **the moment any test starts a retry attempt**, plus an end-of-shard summary listing every test that was retried. ## Why In CI the reporter is `blob`, which writes nothing to stdout. So when a failed test is retried (`retries: process.env.CI ? 3 : 0` in `playwright.config.js`) there is **no signal in the GitHub Actions logs that a retry is even happening** — a test silently re-runs up to 3 more times and the log just shows the next `Test Started` line. This makes flaky-vs-real failures hard to spot when scanning CI output. The retry is internal to `npx playwright test`, so the workflow YAML can't see it — it has to be emitted from the Playwright reporter layer. This is that layer. ## What it looks like ``` 🔁🔁🔁══════════════════════════════════════════════════════════🔁🔁🔁 🔁 R E T R Y # 1 · attempt 2 of 4 (test previously failed — re-running) 🔁 ▶ chromium › logsqueries.cte.spec.js › should query with simple TotalCols CTE 🔁 ▶ Logs/logsqueries.cte.spec.js 🔁🔁🔁══════════════════════════════════════════════════════════🔁🔁🔁 ``` …and once per shard at the end: ``` 🔁 RETRY SUMMARY — N test(s) were retried in this shard: 🔁 • [2 retries] chromium › … › should query … (Logs/logsqueries.cte.spec.js) ``` ## Scope / safety - Added to the **CI** reporter list only, alongside `blob` (multiple reporters are fine). It only `console.log`s — never sends anything, never fails the run. - **ENT gets this for free**: the ENT Playwright workflow checks out and runs the OSS `tests/ui-testing` tree, so this one OSS change makes retries loud in both OSS and ENT CI. There is no ENT test tree to change. - Verified locally with a forced-failure probe (`CI=true`, `retries: 2`): the per-retry banner fires on attempt 2 and 3, and the summary lists the retried test. ## Files - `tests/ui-testing/playwright-tests/utils/retry-banner-reporter.js` — new reporter - `tests/ui-testing/playwright.config.js` — wire it into the CI reporter array |
||
 ktx-vaidehi
|
c9fc8d3b55 | fix: dashboard builder redesign (#13419) | ||
  
|
7d212f4621 |
feat: workflows incident connection (#13472)
This connects incidents to workflows, so any lifecycle event of incident will trigger workflow. design at: https://github.com/openobserve/designs/blob/main/infrastructure/workflows/design.md --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
 Bhargav
|
e9596a10a3 |
feat: Mobile rum sdk v2 (#13503)
Design at: design/mobile-rum-sdk-evaluation/design-docs Backend changes exist to make the OpenObserve RUM intake accept the mobile SDK wire format alongside the existing browser format. src/api/src/handler/http/request/rum/ingest.rs 200 → 202 Accepted on /rum, /logs, /replay | ---> OpenObserve-family SDKs (which the mobile forks derive from) treat any non-202 as an unexpected error and drop the batch. #[serde(default)] on Event ---> Mobile replay omits creation_reason / index_in_view that the browser format sends. Multi-segment replay handling ---> Browser sends 1 segment + 1 event object; mobile sends N segment parts + one event JSON array. Handler now collects all segments, parses event as array-or-object, pairs them, emits NDJSON → one _sessionreplay row per segment. MAX_SEGMENTS=200, 16 MB decompressed cap via .take() ---> Decompression-bomb guard, newly needed now that N segments are accepted. |
||
 ktx-akshay
|
6e0a7374e5 |
test: add DashboardFavorites page object and associated tests for fav… (#13362)
## Summary Adds a Playwright E2E test suite covering the dashboard Favorites feature — favoriting/unfavoriting a dashboard, the `Favorites` pseudo-folder view, and its interaction with single and bulk delete flows. ## What's covered New `dashboard-favorites.spec.js` (6 tests) + `DashboardFavorites` page object: - Add a dashboard to favorites and verify it appears in the Favorites view - Remove a dashboard from favorites — confirms it drops from the Favorites view while the dashboard itself remains untouched - Favorite state persists across a page reload - Bulk-delete a favorited dashboard from the Favorites view completes successfully with no error - Deleting a dashboard directly also removes it from Favorites automatically - Bulk-deleting a dashboard from the Favorites view doesn't leave a stale copy behind in its original folder ## CI `dashboard-favorites.spec.js` runs in its own dedicated `Dashboards-Isolated` shard with a fresh backend instance, keeping it independent from the rest of the dashboard test suite. |
||
 Huaijin
|
ef3617f157 |
refactor: split HTTP APIs by domain (#13479)
## Summary
- organize API crates under
`src/api/{common,grpc,http,ingest,management,pipelines,search}`
- keep ingest, management, pipelines, and search independent from one
another; compose them only in the HTTP frontend
- move pattern APIs into search and simplify `main.rs` wiring
- update workspace manifests, lockfile, integration tests, and
cargo-deny entries
- keep `web/` unchanged
## Validation
- `cargo fmt --all -- --check`
- `cargo clippy --all-targets -- -D warnings`
- `cargo deny check licenses`
- enterprise overlay: `cargo build --no-default-features --features
enterprise`
- enterprise overlay: `cargo build --no-default-features --features
cloud`
- enterprise overlay: `cargo clippy --all-targets -- -D warnings`
The matching `o2-enterprise` overlay PR will be linked after creation.
|
||
 
|
442ca09688 |
test(functions): un-skip JS-in-all-orgs tests + add per-event runtime-error coverage (#13341)
Replaces the `skip` band-aids that PR #13156 ("workflows v1") left on the **Functions** tests with real, running coverage of the current behaviour — on the correct edition — plus new tests for behaviour #13156 introduced but never covered. Test files only, no feature-code churn. Clean branch off `main`. ## Background — what #13156 actually changed #13156 bundled a Functions backend change unrelated to Workflows, and the old tests still asserted the *pre*-#13156 rules, so they were silenced with `pytest.skip(...)` / `test.skip(...)`. Two rules changed: 1. **JS is no longer `_meta`-only at the backend.** `save_function` / `test_run_function` have **no org gate on any edition** — JS (`transType:1`) is accepted in every org, including `default`, on **both OSS and Enterprise**. The `_meta`-only rule now lives **only in the frontend** (`AddFunction.vue` gates the JS radio on `isEnterprise || isCloud || org === "_meta"`, baked at build time by `VITE_OPENOBSERVE_ENTERPRISE`). 2. **The compile-check reports only *syntax* errors, not *runtime* errors.** A syntax error still returns **400**; code that compiles but throws at runtime now returns **200** with the error surfaced **per-event** in `results[].message` — for **both VRL and JS**. Because the API tests hit the backend directly, there is **no edition difference to assert for them** — the earlier "OSS asserts 400 / ENT asserts 200" framing was incorrect. Only the **e2e** (frontend JS-radio) is genuinely edition-dependent. ## API tests (pytest · `tests/api-testing`) **`test_functions.py`** - Keep the compile-error → 400 test (`test_test_endpoint_with_invalid_vrl_returns_400`) — it was never actually failing; `test`/`=====` are genuine VRL *compile* errors (`undefined variable` / syntax) and still 400. - Add `test_test_endpoint_with_js_runtime_error_returns_200` and `test_test_endpoint_with_vrl_runtime_error_returns_200` — runtime errors now return 200 with a per-event `results[].message`. **`test_functions_meta_org_restriction.py`** - Flip the 3 previously-skipped "blocked in default" tests → assert **200** (JS allowed in every org): `..._blocked` → `..._allowed`. - Repurpose `TestErrorMessages` (its old restriction-message assertions no longer exist) into per-event runtime-error coverage: JS `ReferenceError` content, VRL + JS surfacing consistency, partial-failure (only the failing event carries a message), and a syntax-error → 400 regression guard. - Update the class/module docstrings to the real contract. **`test_js_enrichment.py`** - Flip `test_js_function_in_default_org_returns_400` → `test_js_function_in_default_org_allowed` (200, transform runs). ## e2e (Playwright · `tests/ui-testing/Functions/js-transform-type.spec.js`) The JS-radio rule *is* edition-dependent. Instead of a skip (or an env-var hack), the two tests now detect edition at runtime via `EditionFeaturesPage.detectEdition()` — which reads the header edition button, driven by the **same** build flag that gates the JS radio, so it's the exact source of truth and needs **no CI change**: - **OSS build** → assert JS radio **hidden** in a non-meta org. - **Enterprise / cloud build** → assert JS radio **visible** in every org. The edition-agnostic steps that were trapped inside the skipped tests now **always run**: VRL-function create in a non-meta org (a P0 smoke path), and JS-visible in `_meta`. Added `P1: VRL runtime error surfaces in error-details` (mirrors the existing JS `ReferenceError` case; covers the VRL half of the per-event-error change). ## Skips removed (before → after) | Old (skipped) | New (runs & passes) | |---|---| | `test_create_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_test_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_update_vrl_to_js_in_default_org_blocked` | `..._default_org_allowed` | | `test_error_message_content` / `_consistency` | per-event runtime-error tests | | `test_js_function_in_default_org_returns_400` (enrichment) | `..._default_org_allowed` | | e2e `P0: Non-meta org - JS hidden, VRL works` | `P0: Non-meta org - JS radio edition-gated, VRL works` | | e2e `P1: Radio visibility across orgs` (skipped) | same name, edition-aware, no `test.skip` | `grep -rn "JS fns are now allowed" tests/` and `grep -rn "JS is allowed in all orgs on enterprise" tests/` → **both empty**. (The 7 remaining pytest skips are the unrelated `JS_HELPER_UNAVAILABLE` enrichment-helper cases — a separate pre-existing limitation, not part of this change.) ## Verification (local, against current `main` source) Built the current-`main` OSS binary and ran everything live: - **API:** `32 passed`, `0` target skips (7 unrelated `JS_HELPER_UNAVAILABLE` skips remain). - **e2e OSS build:** `7/7` — `detectEdition=opensource`, JS hidden in non-meta. - **e2e Enterprise-frontend build:** `3/3` edition tests — `detectEdition=enterprise`, JS visible everywhere, VRL runtime error renders in error-details. All backend behaviour was confirmed by probing the live `/functions` + `/functions/test` endpoints, not assumed. ## What CI proves, and where - **This PR (openobserve):** `api-testing.yml` runs the API tests; `playwright.yml` (`Functions` shard) runs the e2e on the **OSS** edition (JS-hidden branch). - **Enterprise edition:** dispatch `o2-enterprise` → `playwright.yml` (workflow_dispatch) with `o2_opensource_repo = test/functions-skipped-tests-13318` to run the same specs on the **enterprise** build (JS-visible branch). Pairs with the ENT Functions work in #13318 / o2-enterprise#2217. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
Shrinath Rao
|
7ade62610a |
test: heal logs scroll-retention selector after histogram-pinning change (#13460)
## Problem The scheduled Playwright Regression runs failed on `Logs-Regression` in **both repos** on the same 3 tests in `logs-9044-7354.spec.js` (issue 9044 scroll-retention): - OSS run [30150748275](https://github.com/openobserve/openobserve/actions/runs/30150748275) - ENT run [30150717648](https://github.com/openobserve/o2-enterprise/actions/runs/30150717648) All three failed identically — `expect(scroll).toBeGreaterThan(10)` received `0`, i.e. scrolling the results container no longer moved it. ## Root cause `web/src/plugins/logs/SearchResult.vue` now **pins the histogram** and hands the scrollbar to the results table (`TenstackTable` → `[data-test="logs-search-result-logs-table"]`). Its scroll parent is the `ref="parentRef"` div (`.o2-scroll-container`, `overflow: auto`). The outer `scrollContainerRef` pane is now `overflow: hidden` in the logs view. The page-object helpers `getScrollContainerPosition()` / `scrollToResultsBottom()` walked *down* from the pagination header looking for a scrollable sibling, which latched onto a non-overflowing element — so `scrollTop` stayed `0`. (This is the same area healed once before in #13316; the histogram-pinning change moved the scroller again.) ## Fix Discover the scroll container from the results table itself: walk **up** from `[data-test="logs-search-result-logs-table"]` to the nearest ancestor with a scrollable computed `overflow-y` (the `.o2-scroll-container`). Matched by computed style so class renames don't break it. The old pagination sibling-walk is kept as a fallback for the patterns view (which still scrolls as one pane). Only the shared OSS page object `logsPage.js` changes, so this heals both OSS and ENT (ENT merges the OSS test folder). ## Verification Ran against a fresh local build (localhost:5080): ``` 3 passed (46.8s) Scrolled to: 358 → Scroll correctly reset to top on page change ``` Confirmed via a DOM probe that `.o2-scroll-container` is the real scroller (scrollHeight 1112 > clientHeight 754; scrollTop → 358 after scrolling to bottom). |
||
|
Bhargav
|
1d99426df9 | fix: browser sdk package version upgraded to 0.4.1 (#13455) | ||
|
Huaijin
|
3f9ce27542 |
refactor: split service dependencies out of core (#13446)
## Summary - decouple stream cleanup from enrichment-data and remove the transitive compaction/DataFusion dependency from stream - extract sourcemaps and organization storage services from core into dedicated crates - extract GitHub built-in regex access and model pricing into openobserve-builtins - rename pipeline and organization-storage persistence modules from store to db - remove now-unused direct dependencies from core - synchronize the enterprise workspace overlay in openobserve/o2-enterprise#2266 ## Validation OSS: - cargo check -p openobserve-builtins -p openobserve-core -p openobserve-jobs -p openobserve-api - cargo test -p openobserve-builtins --lib - cargo test -p openobserve-builtins --doc - cargo clippy -p openobserve-builtins -- -D warnings - cargo check -p openobserve-org-storage --features enterprise Enterprise overlay: - copied ../o2-enterprise/Cargo.toml.openobserve over Cargo.toml - cargo fmt --all -- --check - cargo check --all-targets --no-default-features --features enterprise - cargo clippy --all-targets --no-default-features --features enterprise -- -D warnings The default enterprise + vectorscan build was attempted and reached vectorscan-rs-sys, but the validation host does not have cmake installed. The enterprise-only build and clippy run completed successfully. |
||
|
Shrinath RaoandClaude Opus 4.8
|
c6b2829a5e |
test(autoe2e): generated E2E tests for #13025 (logs-stream-quick-pick) (#13030)
🤖 Auto-generated E2E tests for #13025 — **healed to passing** against a live OpenObserve before this PR was opened. - Branch `autoe2e/pr-13025` (stacked on `issue-ent-2038`), so the tests run against the feature code and pass before it merges. - Spec: `tests/ui-testing/playwright-tests/Logs/logsQuickPick.spec.js` The repo Playwright CI re-validates here. Review is for **quality + coverage** — the tests already pass. _Generated by the E2E Council pipeline._ **Test cases added / changed:** | Test case | Type | What it verifies | |---|---|---| | `should quick pick a stream from the sidebar` | new | clicking a sidebar quick pick button selects the stream and loads the field list | | `should show hero state on fresh arrival with no stream selected` | new | the main content area shows both hero action cards when no stream is selected | | `should open stream dropdown when clicking select stream card` | new | clicking the "Select a stream" hero card opens the stream dropdown popover | | `should reappear quick pick after deselecting and re-select works` | new | after deselecting a stream the quick pick reappears and re-selection works correctly | | `should show more footer when org has more than 8 streams` | new | the quick pick more footer is visible with the correct remaining count when org has > 8 streams (skips otherwise) | | `should select stream via hero recent chip` | new | clicking a recent-stream chip in the hero state selects the stream and triggers field list loading | | `should open query guide documentation in new tab` | new | clicking the "Read the query guide" card opens the OpenObserve docs in a new browser tab | --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Huaijin
|
15275ae7cf |
refactor: split core foundation services (#13405)
## Summary This PR begins decomposing `openobserve-core` into focused service crates while preserving the existing application behavior and dependency direction. It extracts six independently owned areas: - `ingestion-common`: shared ingestion contracts and response types - `schema`: schema inference, validation, evolution, and persistence helpers - `compaction`: merge, retention, and compaction services - `enrichment-data`: enrichment-table lifecycle and runtime support - `promql-service`: PromQL service orchestration - `stream`: stream metadata, settings, retention, and lifecycle operations ## What changed - registered the extracted crates in the workspace and propagated enterprise/cloud/vectorscan/vortex features - moved implementations out of Core and updated API, Jobs, query, and queue consumers to use the new crates - removed Core facade aliases for extracted crates and converted remaining callers to direct crate paths - retained the small cross-service stream-delete coordinator in Core for now because it still coordinates Pipeline and Alert cleanup - restored required root/Core dependencies after validating the full enterprise build - added the missing `anyhow` dependency for enterprise-only `search_service::search_jobs` - updated the lockfile for the new crate dependency graph ## Dependency direction The extracted crates do not depend on `openobserve-core`. Core remains a temporary upper-level consumer while the remaining domain services are split incrementally. ```text ingestion-common -> common/config schema -> ingestion-common/db/infra stream -> schema/enrichment-data/db/infra compaction -> schema/search/search-service/db/infra openobserve-core -> ingestion-common/schema/stream/compaction/... api/jobs -> openobserve-core + extracted services ``` ## Enterprise compatibility The private OpenObserve manifest was updated in openobserve/o2-enterprise#2257 to register the extracted workspace members and dependencies. Related Enterprise PR: https://github.com/openobserve/o2-enterprise/pull/2257 ## Validation OSS manifest: - `cargo fmt --all -- --check` - `cargo clippy -- -D warnings` Using `o2-enterprise/Cargo.toml.openobserve` as the OpenObserve root manifest: - `cargo check --no-default-features --features enterprise` - `cargo check --no-default-features --features cloud` - `cargo clippy --workspace --no-default-features --features enterprise -- -D warnings` - `cargo clippy --workspace --no-default-features --features cloud -- -D warnings` Enterprise repository: - `cargo fmt --all -- --check` - `cargo clippy -- -A clippy::question-mark -D warnings` All checks above pass in the development profile. |
||
|
Shrinath RaoandClaude Opus 4.8
|
9842922521 |
test(pipelines): fix addPipeline mount-wait flake (isVisible timeout is a no-op) (#13420)
## What flaked `Pipelines-Regression` → *"should validate scheduled pipeline SQL query with metrics stream type"* (`pipeline-regression.spec.js:73`) timed out in the ENT gate waiting for the editor mount signal `[data-test="pipeline-node-sidebar-collapse-btn"]` (15s, all retries). It's a **load-timing flake, not a real break**: OSS CI ([run 30005672782](https://github.com/openobserve/openobserve/actions/runs/30005672782)) and the ENT `main` baseline ([run 30005554375](https://github.com/openobserve/o2-enterprise/actions/runs/30005554375)) both pass this same test. ## Root cause `addPipeline()` meant to poll for the mount signal across 3 attempts of 15s (re-clicking the add button non-destructively if the click was dropped before navigation): ```js for (let attempt = 1; attempt <= 3; attempt++) { if (await railToggle.isVisible({ timeout: 15000 }).catch(() => false)) break; ... } await railToggle.waitFor({ state: 'visible', timeout: 15000 }); ``` But **`locator.isVisible({ timeout })` does not wait** — it returns the current state immediately and ignores `timeout` (Playwright 1.50). So the loop spun through instantly and the editor effectively got only the **final 15s** `waitFor` to mount. Under concurrent cloud load VueFlow's mount lags past 15s → timeout. ## Fix (page-object only) Make each poll attempt actually block using `waitFor({ state: 'visible' })` inside the loop (try/catch, re-click on the list page between attempts), restoring the intended **~45s** of resilience before the final assert. ## Verification Local verification is currently blocked by a stale local build (the selector was added in #13397 and isn't in the local `web/dist`), so this is gated on a fresh CI regression run against the branch. The change only lengthens the effective mount wait; it can't regress the happy path (which already passes on fresh builds). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Shrinath RaoandClaude Opus 4.8
|
0e477f9918 |
test(workflows): de-flake K9 canary — log Save-intercept, don't hard-assert it (#13418)
## How this was found The **Playwright E2E Crosscheck** bot (the ENT gate that runs enterprise-only specs against OSS PRs) flagged a failing spec on an unrelated logs PR ([#13414 comment](https://github.com/openobserve/openobserve/pull/13414#issuecomment-5058151842)): > `Workflows/workflows-negative.spec.js` — *"enterprise-only e2e spec failed against this PR … wouldn't show up in OSS CI."* Digging showed it wasn't caused by that PR — the same test was **also failing on #13413 and #13315**. It's a pre-existing **flake** in the ENT gate, so this PR fixes it at the source. ## Evidence it's a flake, not a real break The **same branch** produced both outcomes: | ENT crosscheck run | PR | K9 test | |---|---|---| | 30005431754 | #13414 | ✅ passed (~50s, slow path) | | 30008606974 | #13414 | ❌ failed | | 30006996158 | #13315 | ❌ failed | | 30006419247 | #13413 | ❌ failed | ## Root cause `CT-19 K9` hard-asserted that a normal `.click()` on Save is intercepted by a tooltip overlay: ```js expect(await pm.workflowsPage.normalSaveClickIsIntercepted(4000)).toBe(true); ``` But the Save button has **no tooltip of its own** (`WorkflowEditor.vue`) — the overlay that intercepts it is a **transient/adjacent tooltip** that often never appears on a programmatic hover in headless CI. The probe therefore conflates *"blocked by overlay"* with *"click landed"*, and the assertion flips run to run. (A first attempt — waiting for the overlay before probing — didn't help: the overlay simply isn't reliably summoned.) ## Fix (test-only, no product/Vue changes) - Treat interception as an **informational canary** — `testLogger.info(...)` instead of a hard assert. - Keep the **stable** assertion: the JS-click Save workaround still reaches the app (`msg.length > 0`) — the adjacent trigger-only-save test already exercises that same path and passes consistently. - If K9 is ever genuinely fixed, the canary log shows `intercepted:false` consistently — the signal to drop the JS-click workaround. This removes the flaky gate without losing meaningful coverage. Changes are limited to the spec and its page object. ## Verification `Workflows/*` are enterprise-only (skipped on OSS CI), so verified via the **ENT gate against this OSS branch** (`e2e` label). The removed assertion was the only flaky part; the retained assertion matches an already-stable adjacent test. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Shrinath RaoandClaude Opus 4.8
|
1daa36b0f8 |
fix(logs): preserve restored SQL query on shared-link open + align log-detail default-tab test (13368) (#13414)
> **This PR contains two independent logs changes:**
> **(A)** a product fix for the shared-link SQL-query restore race, and
> **(B)** the log-detail default-tab test alignment (original scope of
this PR).
---
# (A) fix(logs): preserve restored SQL query on shared-link open
(fullSQLMode watcher race)
## The bug
Opening a **shared link** to an SQL search **sometimes lands on an empty
query editor** ("SQL query is missing or invalid") instead of the shared
search. Intermittent, worse under load — it's the recurring
Logs-Features alpha flake (`monaco-query-prefill` P0 "query editor never
contained SELECT").
**Not the short-URL mechanism** — the short URL correctly carries
`sql_mode=true` + the base64 query. The loss happens in the browser
during restore.
## Root cause — a watcher race
`restoreUrlQueryParams()` raises `shouldIgnoreWatcher`, flips
`meta.sqlMode = true`, then writes the decoded query into the editor.
Flipping `sqlMode` fires the **`fullSQLMode`** watcher in `Index.vue`,
whose two branches were **asymmetric**:
- **switch OFF** branch → already checks `shouldIgnoreWatcher` ✅
- **switch ON** branch → **did not** ❌ — it called `setQuery()`,
rebuilding a *default* `SELECT … FROM stream` over the value the restore
was mid-writing.
Under load the watcher wins the race; the editor briefly empties, SQL
mode auto-detects back **off**, and the editor is cleared entirely.
## The fix
Make the "switch ON" branch honour `shouldIgnoreWatcher` too — symmetric
with "switch OFF". During any programmatic restore the watcher **stands
down** and the restored query is the last word. (`shouldIgnoreWatcher`
is raised only by programmatic-state paths whose purpose is to suppress
watcher side-effects, so honouring it here is consistent.)
## Verification (A)
- ✅ **New unit test**: with `shouldIgnoreWatcher` set, flipping
`sqlMode` ON does **not** call `setQuery`/`updateUrlQueryParams`, and
the restored `SELECT` query is preserved.
- ✅ Existing `fullSQLMode` watcher test still passes; full
`Index.spec.ts` green (**69 passed**).
- ✅ Reproduced the failure on **alpha under CI-load** beforehand
(share-link P0 flaky, "query editor never contained SELECT").
- ⚠️ The race is only observable against a **deployed** frontend (the
alpha e2e suite runs against alpha's frontend, not a local build), so
the final "flake gone on alpha under load" check requires this deployed
to alpha, then re-running the Logs-Features share-link tests.
Files: `web/src/plugins/logs/Index.vue`,
`web/src/plugins/logs/Index.spec.ts`.
---
# (B) test(logs): align log-detail default-tab test with Table default
(13368)
## What
Updates the log-detail sidebar default-tab regression test (originally
Bug #9724) to match the **intentionally shipped** behavior from [PR
#13368](https://github.com/openobserve/openobserve/pull/13368) — *"logs
sidebar table will be default view and draggable tabs"*.
## Why
#13368 changed the log-detail sidebar default from the **JSON** tab to
the **Table** tab (`SearchResult.vue` — `detailTableInitialTab.value =
"table"`). The Bug #9724 test still asserted JSON-as-default, so it
fails on `main`: the JSON tab's `data-state` stays `inactive` for the
full timeout across all retries, on both OSS and ENT Logs-Regression.
Not a flake and not a product bug — the test expectation was simply out
of date after a deliberate default-view change.
## Changes
- Rename test → **"Log detail sidebar opens with Table tab by default
(#13368)"**.
- Add `verifyTableTabSelectedByDefault()` page-object helper
(auto-retrying `toHaveAttribute('data-state','active')`); remove the
now-invalid `verifyJsonTabSelectedByDefault()`.
- Reorder switch steps: assert Table default (+ wrap toggle, which
renders only on the Table tab), switch to JSON and back to Table, reopen
and confirm Table default persists.
## Note on #13407
Supersedes the earlier heal
[#13407](https://github.com/openobserve/openobserve/pull/13407), which
misdiagnosed the failure as a `data-state` timing flake. A
locally-hosted stale pre-#13368 build had masked the change.
Files: `tests/ui-testing/pages/logsPages/logsPage.js`,
`tests/ui-testing/playwright-tests/RegressionSet/Logs/logs-regression.spec.js`.
```
npx playwright test playwright-tests/RegressionSet/Logs/logs-regression.spec.js \
-g "Table tab by default" --project=chromium
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Sai Nikhil
|
2cec0a0c94 | fix: pipeline and workflows UI change (#13397) | ||
|
ktx-vaidehiandClaude Opus 4.8
|
8886a03b99 |
feat(dashboards): redesign panel config sections (#13371)
- Collapse all config sections by default (was expanded) - Add open-state accent to OCollapsible sidebar variant: left accent bar, tinted header, and accent-colored icon/chevron when expanded - Move sidebar chevron to the right of the label (rotates on open) - Give every config section a leading icon via SECTION_ICONS - Add theme-aware collapsible open-state tokens (light + dark) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Shrinath RaoandClaude Opus 4.8
|
8da11deec7 |
test(logs): heal Bug 9724 JSON-tab-default flake with retrying data-state assertion (#13407)
## What Heals the flaky **Logs-Regression** spec `Log detail sidebar opens with JSON tab by default (Bug #9724)`, which failed the scheduled main regression run [29990712453](https://github.com/openobserve/openobserve/actions/runs/29990712453) (and its re-run) on all retries. ## Root cause `logsPage.verifyJsonTabSelectedByDefault()` read the Reka `OTab` (`TabsTrigger`) `data-state` attribute with a **one-shot** `.evaluate(el => el.getAttribute('data-state') === 'active')`. `DetailTable.vue` is loaded via `defineAsyncComponent`, and its tabs moved to Reka UI in the design-token migration (13173). On drawer open, the trigger can render a tick before Reka's reactive `data-state` settles to `active`. A raw `getAttribute` read does **not** auto-retry, so under CI load it captured the pre-settle value and failed. JSON **is** the genuine product default (`DetailTable.vue` `tab = ref(initialTab || "json")`, `SearchResult.vue` `detailTableInitialTab = "json"`) — this is a load flake, not a product regression. ## Fix Replace the one-shot reads with the auto-retrying `expect(tab).toHaveAttribute('data-state', 'active', { timeout: 10000 })` pattern already used elsewhere in the same page object (regex-pattern / builder-type toggles), across all three tab-verify helpers: - `verifyJsonTabSelectedByDefault()` - `verifyTableTabSelected()` - `verifyJsonTabSelected()` Test-only change; no product code touched. ## Verification Local run against `localhost:5080`: ``` npx playwright test playwright-tests/RegressionSet/Logs/logs-regression.spec.js \ -g "Bug #9724" --project=chromium --repeat-each=3 # 3 passed (16.9s) ``` The single spec exercises all three healed helpers (default-JSON at steps 3 + reopen, Table tab at step 6, JSON switch-back at step 7). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
BhargavandClaude Opus 4.8
|
ef2efdf5fc |
fix: logs sidebar table will be default view and draggable tabs (#13368)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
4f458a0fab |
test(sdr): add traces SDR e2e suite (redact/drop/hash × ingestion/query) (#13390)
## What Adds an end-to-end **traces SDR (Sensitive Data Redaction)** test suite, extending SDR coverage from logs to traces. This validates the fix in this PR branch (`feat/fix-traces-sdr-bug`) — SDR regex patterns were previously applied to logs only; these tests assert the same policies now apply to **trace span attributes** at both ingestion and query time. Targets `feat/fix-traces-sdr-bug` so the tests land together with the fix they cover. ## Tests added **6 new spec files → 12 tests** (each spec = 1 `setup: cleanup patterns` + 1 combined functional test), covering **24 field-level policy assertions** (4 sensitive fields × 6 specs). All tagged `@enterprise @sdr @traces`. | # | Spec | Policy | Timing | What it asserts | |---|------|--------|--------|-----------------| | 1 | `tracesIngestionTimeRedaction.spec.js` | Redact | Ingestion | Span-attribute values matching the pattern are stored as `[REDACTED]` | | 2 | `tracesQueryTimeRedaction.spec.js` | Redact | Query | Stored values are untouched; the search response returns `[REDACTED]` | | 3 | `tracesIngestionTimeDrop.spec.js` | Drop | Ingestion | Matching fields are removed from the record at write time | | 4 | `tracesQueryTimeDrop.spec.js` | Drop | Query | Fields present in storage are dropped from the search response | | 5 | `tracesIngestionTimeHash.spec.js` | Hash | Ingestion | Matching values are stored as `[REDACTED:<hash>]` | | 6 | `tracesQueryTimeHash.spec.js` | Hash | Query | Search response returns `[REDACTED:<hash>]`; storage untouched | Each combined test drives the full flow for **4 fields / patterns**: - **email** (`user_email` = `john.doe@example.com`) — `^[\w\.-]+@[\w\.-]+\.\w{2,}$` - **US phone** (`phone` = `5551234567`) — `^\d{10}$` - **credit card** (`cc_number` = `1234 5678 9012 3456`) — `^\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}$` - **SSN** (`ssn` = `123-45-6789`) — `^\d{3}-\d{2}-\d{4}$` (drop/hash specs use `website_url`, `user_password`, `server_ip`, `theme_color` with URL / password / IPv4 / hex-color patterns.) Each functional test follows: ingest → assert field visible (no pattern) → create 4 patterns → associate to the traces stream's fields → ingest/re-query → assert the field is redacted / dropped / hashed. ## How it works Traces reuse the **shared** pattern create/associate/verify flow with logs — only the ingest and read seams differ: - **`sdrTracesIngestionPage.js`** (new): ingests via OTLP `/api/{org}/v1/traces` (`stream-name` header). Each test field is a span attribute that flattens to an identical top-level field; every span also carries an `sdr_test_id` marker for batch isolation. - **`sdrVerificationPage` / `streamAssociationPage`**: parameterized by `streamType` — `?type=traces` search, Traces stream-type tab selection, and schema-field filtering so a field row in the (virtualized, ~20-field) traces schema is reliably found. - **`apiCleanup`**: `fetchStreams` / `deleteStream` / `cleanupStreams` / `isStreamStillDeleting` / `waitForStreamDeletion` take an optional `streamType` (default `'logs'` — no change for existing callers). ## Artifact cleanup - `cleanup.spec.js` sweeps `sdr_traces_*` streams via `streamType: 'traces'` (the logs sweep is `type=logs`-only and never listed them). - `cleanupRegexPatterns` adds the `traces_` prefix. - **Verified live**: the sweep deleted 8 leftover traces streams + 32 traces patterns; 0 remained. ## Verification - All **6 specs / 12 tests pass** against a local enterprise build carrying the fix. - The existing **logs** SDR suite still passes unchanged (no regression from the shared page-object changes). ## Companion CI wiring is a paired o2-enterprise PR (branch `test/traces-sdr-e2e`) that splits the SDR matrix into `SDR-Logs` and `SDR-Traces` shards so the new specs run as their own shard. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Chaitanya Sistla <sistlachaitanya2@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Shrinath RaoandClaude Opus 4.8
|
68946ed063 |
test: harden org-selector switch flows against virtualized dropdown (changeOrg + logoManagement) (#13391)
## What
Heals the two `GeneralTests` specs that flake on the redesigned
**`OrganizationSelector.vue`** (reka-ui `ODropdown` +
`@tanstack/vue-virtual` virtualized list, from the design-token
migration): `changeOrg.spec.js` and `logoManagement.spec.js`. Test-only
— no product/frontend change.
## Why it flakes (CI parallel load only — passes single-worker locally)
- **changeOrg** — virtual rows carry a `transform: translateY(...)` that
keeps shifting while the popper animates in and the list settles, so
Playwright's actionability *stability* check burns the click budget and
times out. A force-click on a still-moving row can also land on a
**neighbouring row** (often `default`) — the menu still closes, so the
wrong org is selected and the swallowed `waitForURL(...).catch(()=>{})`
hid it until a later assertion failed.
- **logoManagement** — `managementOrg` had no "menu opened" gate; a
blind `waitForTimeout` raced ahead of the popper open, so
`organization-search-input-field` never rendered and `fill()` timed out
(45s).
This is **not a regression from any single PR** — it reproduces on every
branch's ENT `GeneralTests` run (the org-selector redesign was never
hardened for the virtualized dropdown).
## Fix
- `homePage.js` `clickOrgRow`: `scrollIntoViewIfNeeded` + `force` click
+ gate on the menu **hiding**, so a missed click retries instead of
leaving an un-selected menu.
- `homePage.js` `selectOrgByIdentifier` (new): retry open → search →
click until the **URL** carries the target identifier — the only
authoritative signal that the right row was selected. `homePageOrg`'s
identifier branch uses it.
- `logoManagementPage.js` `managementOrg`: retry the open until the menu
list is visible, fill the `-field` input with `force`, hardened row
click with the same menu-hidden verification. Added
`organizationMenuList` locator.
## Verification (local enterprise build v0.92.0-rc1)
| Test set | Workers | Result |
|----------|---------|--------|
| changeOrg "change organisation validation" (14) ×3 | 4 | 14/14 pass
each (was 1-failed pre-fix) |
| changeOrg full spec | 4 | 28 pass, 0 fail (no regressions) |
| logoManagement ×2 | 1 | pass |
## Enterprise coverage
Carries the `e2e` label so the **ENT e2e gate** dispatches the
enterprise suite against this branch (built against ENT main) and
mirrors the result here — validating the `@enterprise` logoManagement
spec and ENT GeneralTests without a separate ENT PR.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Shrinath RaoandClaude Opus 4.8
|
871b9b69b7 |
test(traces): heal bug #11531 spec for flame-graph-default trace detail (#13392)
## What Heals the **Traces-Regression** failure on `main` (scheduled run [29935986660](https://github.com/openobserve/openobserve/actions/runs/29935986660/job/88982390601)). The Bug #11531 test in `traces-bugs.spec.js` (`traces spans tab should not reference non-existent method field`) clicks a trace result and immediately asserts the span tree via `expectTraceDetailsVisible()`, which waits for `[data-test="trace-details-tree"]`. ## Why it started failing PR 13347 (`fix: traces page default flame graph and drag-drop option added`) changed the trace detail view to **open on the flame graph by default** (`DEFAULT_TRACE_TAB = "flame-graph"`) instead of the waterfall. The span tree lives inside `v-if="activeTab === 'waterfall'"`, so it no longer renders on open — the assertion timed out (15s) and the job exited 1. This is **not a product bug** — it's an intentional product change. 13347 added a `openTraceDetailsTab()` helper and updated the other traces specs, but this Bug #11531 test was missed. ## Fix Select the waterfall tab explicitly before reading the tree, matching the pattern 13347 already applied elsewhere: ```js // Since 13347 the trace detail opens on the flame graph by default. await pm.tracesPage.openTraceDetailsTab('waterfall'); await pm.tracesPage.expectTraceDetailsVisible(); ``` Test-only change; no product code touched. ## Verification (local, hosted O2 @ localhost:5080, `default` traces stream w/ data) | State | Result | |-------|--------| | Before fix (unpatched) | ❌ `expectTraceDetailsVisible` times out at `tracesPage.js:310` | | After fix (patched) | ✅ `1 passed (22.3s)` — "Trace details visible" logged | 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Huaijin
|
6adcae3a1f |
refactor: remove crate compatibility re-exports (#13381)
## Summary
- remove compatibility re-exports introduced during the recent crate
refactors
- migrate callers to the crates that own each API
- delete five pure facade modules and add the required direct crate
dependencies
- keep OSS and enterprise manifests aligned
## Validation
- cargo fmt --all -- --check
- cargo clippy --workspace -- -D warnings
- cargo test --locked --workspace --no-run
- enterprise overlay: cargo test --offline --workspace
--no-default-features --features enterprise --no-run
- enterprise overlay: cargo clippy --offline --workspace
--no-default-features --features enterprise -- -D warnings
- git diff --check
Companion enterprise PR uses the same branch name:
codex/remove-refactor-reexports.
## TODO / Follow-ups
- [ ] **Re-establish the pipeline write-guardrail lost with
`db::pipeline`.** That module was not a pure facade: it re-exported only
the six read-only accessors from `infra::pipeline` and its doc comment
recorded the invariant that *mutations must go through
`service::pipeline::store`* (which keeps the `ExecutablePipeline` caches
and the coordinator/super-cluster watch events in sync). Deleting it
points callers straight at `infra::pipeline`, which also exposes
`put`/`delete`/`drop_table` with no guardrail. Decide the layering and
enforce it:
- **Reads** may hit `infra` directly (raw persistence, no side effects).
- **Writes** must go through the wrapper that persists *and* emits the
coordinator/super-cluster event + refreshes the in-memory cache. That
wrapper is `db::<entity>::{set,delete}` for most entities (e.g.
`db::alerts::alert::set` →
`infra::coordinator::alerts::emit_put_event`), but for pipeline it is
`core::pipeline::store` (not `db`), because the
`STREAM_EXECUTABLE_PIPELINES` cache it maintains is a core-level
construct the `db` crate cannot build. The super-cluster receiver
(`super_cluster_queue::pipelines`) is the one sanctioned caller of
`infra::pipeline::{put,delete}` directly, and it manually pairs each
with `emit_*_event` to avoid a re-broadcast loop.
- Net invariant to protect: never call `infra::pipeline::{put,delete}`
without emitting the coordinator event — i.e. nowhere outside
`core::pipeline::store` and the super-cluster handler. A bare `infra`
write leaves in-memory caches stale until restart (see the warning
comment still live at `src/core/src/org_cleanup.rs:484`).
- [ ] Consider whether the pipeline write-wrapper living in
`core::pipeline::store` while every other entity's lives in
`db::<entity>` is worth unifying, or at least documenting as an
intentional exception.
|
||
|
Shrinath RaoandClaude Opus 4.8
|
415d781941 |
test: remove redundant networkidle waits in Logs-Features specs (alpha stability + speed) (#13383)
## Why
The **Logs-Features** alpha1 shard runs 18–23 min and intermittently
trips *"self-hosted runner lost communication"* — a **duration-driven
infra flake**, not a test failure (the tests were passing right up to
the runner death). A long job keeps the self-hosted runner under
sustained load, raising the odds it drops off the network. Shrink the
runtime and the flake probability drops with it.
A large part of that runtime is `waitForLoadState('networkidle')`.
`networkidle` means "wait for 500ms of zero network activity" — but the
cloud SPA is constantly chattering (telemetry / RUM / streaming search),
so that quiet moment basically never comes. Each call therefore runs to
its **full 30s ceiling**, then `.catch(() => {})` throws the result
away. Locally there's no background traffic, so the same line returns in
<1s — which is exactly why these specs fly locally but crawl on alpha.
## What
In `monaco-query-prefill.spec.js`, all **10** `networkidle` calls sit
right after a `page.goto()` (which already awaits the `load` event)
**and** right before a deterministic gate:
`selectStream`, `waitForQueryEditorContent`,
`waitForQueryEditorVisible`, or `waitForRedirectedQueryEditorContent`.
They add nothing to correctness — the following targeted wait is the
real readiness signal — so they're removed. No new timers introduced;
the existing targeted waits do the gating.
## Verified on alpha (live env, full spec)
| | Runtime | Result |
|---|---|---|
| **Before** (original) | 5.1m | 1 flaky failure (P2), last test skipped
(serial mode) |
| **After** (this change) | 4.3m | **5/5 passed** |
The after-run ran **more** tests (all 5) in **less** time, and was fully
green. A second confirmation run is in progress and will be posted
below.
## Scope
Pilot for the same cleanup. The heavy file is `searchPatterns.spec.js`
(**46** of these waits); this validates the pattern on a small,
well-understood spec first before rolling out.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
 Harsh Mahajan
|
9c66eda99f |
fix: frontend fix. found by playwright tests (#13389)
https://github.com/openobserve/o2-enterprise/actions/runs/29932597095/job/89019125595?pr=2227 |
||
|
Bhargav
|
e0fe62a9a0 | fix: traces page default flame graph and drag-drop option added (#13347) | ||
|
Huaijin
|
e236c7e653 |
refactor: extract database layer into db crate (#13340)
## Summary - extract persistence, cache, watcher data access, and DB service modules from openobserve-core into a standalone workspace db crate - keep application orchestration that depends on search, organization provisioning, and other Core services behind the existing Core db facade - preserve existing call paths through re-exports while making the dependency direction openobserve-core -> db - move HTTP error mapping back to the Core/API boundary so db has no direct axum dependency - avoid provider/provisioner callback traits and function-pointer hooks ## Validation - cargo fmt --all - cargo check --all-targets - cargo check -p db --all-targets - Enterprise workspace: cargo check --all-targets --offline - Enterprise workspace: cargo check -p db --all-targets --features enterprise --offline Companion Enterprise manifest PR: openobserve/o2-enterprise#2230. |
||
 Prabhat Sharma
|
21d0907342 |
fix(logs): redesign log patterns UI (#13337)
## What Redesign of the log patterns UI. - **`PatternCard`** reworked: template tokens, a badges row (trend / rarity / anomaly), and inline frequency + percentage replace the old fixed count and action columns. - **`PatternVolumeCell`** (new) + **`usePatternVolume`** (new) render per-pattern volume sparklines. - **`PatternDetailsDialog`** now owns the include / exclude / create-alert actions that previously lived on each card. - **`patternUtils`** / **`usePatternActions`** extended to back the above. ## Test plan - `web` unit tests for the touched areas: 356 passing, 0 failing (`plugins/logs/patterns`, `plugins/logs/data-table`, `SearchResult`, `FieldValuesPanel`). - New spec coverage for `PatternVolumeCell`, `usePatternVolume`, and `patternUtils`. - `npm run type-check` clean. - `lint:design`, `lint:tokens`, `lint:token-purity` all clean. ## Notes Rebased onto latest `main`. Five conflicts came from main's repo-wide type-check pass (#13174), which made `OIcon`'s `name` prop required. Main's fix (`name=""` + slot) was kept in `CodeQueryEditor.vue`, `FieldValuesPanel.vue`, and `CellActions.vue`; in `PatternCard.vue` the surrounding block was removed by this redesign, so the deletion stands. |
||
|
Vamsi Karthik RandClaude Opus 4.8
|
899bb681a8 |
test(e2e): optimize/fix alpha1 Alerts, SDR & Pipelines shards (operator select scoping + editor/nav/data-flow races + ingestion 5xx resilience) (#13336)
## What
Consolidated stabilization of the **Alerts**, **SDR** and **Pipelines**
shards of the
alpha1 cloud E2E suite (`playwright_alpha1.yml`) to green — the shards
still failing
after the earlier per-module stabilization PRs merged. Rebased on
`main`.
Across these shards the original run had: **Alerts** 1 hard fail,
**SDR** 1 hard fail +
1 flake, **Pipelines** 1 hard fail + several flakes. Root causes were an
ambiguous
operator-select selector plus cloud data/UI-readiness races (editor &
node-sidebar mount,
hover-flyout nav, freshly-ingested destination streams lagging their
list/table) and
transient backend 5xx on ingestion.
**No `waitForTimeout` added as a fix, no `force:true`, no
`.first()`/`.nth()` band-aids,
no retry-count bumps to hide failures, no weakened assertions. No
worker-count,
parallelism, playwright-config or CI/yml changes** — test code only.
## Root cause & fixes
**Alerts**
1. **Operator select — strict-mode violation** (`alerts-e2e-flow`, state
management).
The operator control is an OSelect that renders an aria-hidden native
`<select><option>=</option>` alongside the visible popover item, so the
unscoped
`getByText('=', {exact})` matched **two** nodes. Scope the option click
to
`[data-test="alert-conditions-operator-select-popover"]` +
`getByRole('option')`
(excludes the aria-hidden native select, mirrors the working
`alertCreationWizard.js`
pattern), and gate on the popover open/close instead of the trailing
`waitForTimeout`s.
(`alertManagement.js`)
**SDR**
2. **Stream Detail button — 45s timeout under load**
(`multipleSDRPatterns` @slow,
`ingestionTimeHash`). `openStreamDetail` resolved the button by the
icon's title text,
which the OTable can render after the name cell, and the retry only
re-clicked a
possibly-detached row without re-fetching the list. Resolve by the
stable
`data-test="log-stream-schema-btn"` and reload + re-search the streams
list between
attempts, keyed to the real "detail sidebar open" signal
(`schema-update-settings-button`). (`streamAssociationPage.js`)
**Pipelines**
3. **Editor / node-sidebar mount race** (`selectStream`, all pipeline
flows). A bare click
on the stream-input button dead-waited the 45s action timeout when the
VueFlow editor +
NodeSidebar missed the first render under concurrent load. Moved a
**non-destructive**
mount guard into `addPipeline`: after clicking Add, if the button hasn't
rendered and
we're still on the list (add button visible), re-click to re-trigger
navigation — no
`page.reload()` (a reload of the editor route re-runs the auth callback
on cloud and
stalls the shard). (`pipelinesPage.js`)
4. **`openPipelineMenu` — hover-flyout nav timeout.** Routed through the
left-nav hover
flyout, which self-closes on settle and, under load, burned ~30s
retrying to open before
its goto fallback fired — tipping long condition tests over the 180s
test timeout.
Promoted the already-proven `goto` to the primary path and skipped the
flyout (it adds
no coverage here). (`pipelinesPage.js`)
5. **Destination-stream data-flow / render latency**
(`pipeline-dynamic`, query-source
flow). Extended `openTimestampMenu`'s re-search loop (destination stream
indexing can
lag past the old budget) and `waitForQueryNodeVisible` 30s→45s
(query-node canvas render
lags under full-shard load) — both within the CI test timeout, both exit
early on the
real signal. (`pipelinesPage.js`)
**Ingestion resilience (Pipelines + shared helper)**
6. **Transient backend 5xx aborted ingestion.** `fetchWithRetry` only
retried thrown
network errors; a `503`/`502` is a "successful" fetch, so ingestion
threw on the first
one when the env returned transient 5xx during load spikes. Retry 5xx
with backoff in
both ingestion helpers — a persistent 5xx still surfaces after
`maxRetries`, so genuine
outages aren't masked. (`pipelinesPage.js`, `ingestionPage.js`)
## Validation
- **CI — Alerts / SDR / Pipelines green (alpha cloud):**
[Pipeline](https://github.com/openobserve/o2-enterprise/actions/runs/29809833252).
[SDR,
Alerts](https://github.com/openobserve/o2-enterprise/actions/runs/29771293789)
- **Original failing run:**
https://github.com/openobserve/o2-enterprise/actions/runs/29720987587
- **Local (real alpha cloud, `playwright-alpha1.config.js`):** each
previously-failing test
re-run to green — `pipeline-conditions.spec.js` @ `--workers=3` 6/6,
`pipeline-dynamic`
72 & 97, `pipelines:180` (query source) solo; Alerts operator flow and
SDR
`openStreamDetail` verified against alpha. 5xx-retry control flow
verified (2xx/4xx
unchanged, transient 503 recovers, persistent 503 still surfaces).
## Notes
- 4 files change (3 page objects + the shared ingestion helper). No
specs, shared
cleanup/setup, config, or workflow files touched.
- Any residual Pipelines tail is cloud latency / env backend health
(e.g. runs where the
env returns sustained 503s) — not test code; the durable lever there is
cross-shard
concurrency / env capacity, out of scope for this PR.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Shrinath RaoandClaude Opus 4.8
|
b1604cc7b6 |
test(workflows): e2e + API tests for Workflows (#13318)
Adds automated test coverage for the Enterprise-only **Workflows** feature (now merged to main). Clean branch off `main` — test files only, no feature-code churn. **UI (Playwright · `tests/ui-testing`)** - `pages/workflowsPages/workflowsPage.js` — page object (verified `data-test` selectors, Save-tooltip JS-click workaround, slow-list waits, enterprise-availability probe). - `playwright-tests/Workflows/workflows-crud.spec.js` — build/name validation, list + search. - `playwright-tests/Workflows/workflows-negative.spec.js` — negative + NFR sweep (long/unicode/emoji names, list latency, console/network hygiene). - `pages/page-manager.js`, `pages/apiCleanup.js` (`cleanupWorkflows()` + cascade), `playwright-tests/cleanup.spec.js` (`wf_auto_*` prefixes). - `.github/workflows/playwright.yml` — `Workflows` testfolder in the matrix. **API (pytest · `tests/api-testing`)** - `tests/workflows/test_workflows.py` — full REST surface (create/list/update/enable/test/history/errors/retry/delete) + validation, auth (401), cross-org. **Behaviour**: Enterprise-gated (skips on OSS where the feature is off); self-cleaning (`wf_auto_*`). Verified live: **11 Playwright + 13 pytest passing**. Paired with the ENT matrix PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Shrinath RaoandClaude Opus 4.8
|
6dfcb8a44a |
ci: remove TestDino from e2e workflows (#13326)
## What Removes **all TestDino integration** from the Playwright e2e CI workflows. ### Removed - **Reporting/upload to TestDino** — the `upload_testdino` job, the `UPLOAD_TO_TESTDINO` env/var, and all `TESTDINO_API_TOKEN` usages. - **Rerun optimization** — the `check_rerun` step and the `tdpw last-failed` `RERUN_MODE` dispatch. ### Files - `.github/workflows/playwright.yml` (−444) - `.github/workflows/playwright_regression.yml` (−350) - `.github/workflows/playwright-auto-rerun.yml` (stale notice line) - `tests/ui-testing/playwright.config.js` (comment reword only) ### Behavior change "Re-run failed jobs" no longer targets only the previously-failed tests — a re-run now runs the **full matrix job**. Native GitHub re-run-failed-*jobs* still works. ### Kept `report.json` / merge-reports generation is untouched — still consumed by the Kinora and OpenObserve reporters. ### Follow-up (after merge) Revoke `TESTDINO_API_TOKEN` (repo secret) + delete the `UPLOAD_TO_TESTDINO` repo variable, and revoke the token in the TestDino dashboard. Paired ENT PR uses the same branch `test/remove-testdino`. ### Validation `actionlint` clean (no new findings) and zero TestDino references remain. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
ktx-vaidehi
|
784459fcf4 | fix: login form migration (#13334) | ||
|
Shrinath RaoandClaude Fable 5
|
79c524c7f9 |
test: heal dark-mode detection and logs scroll-container selectors after design-token migration (#13316)
## Why The 2026-07-21 scheduled Playwright Regression runs failed on **both** OSS ([run 29801668855](https://github.com/openobserve/openobserve/actions/runs/29801668855)) and ENT ([run 29801639539](https://github.com/openobserve/o2-enterprise/actions/runs/29801639539)) — same 4 tests, same shards (Traces-Regression, Logs-Regression). Root cause: the design-token migration (#13173, `f15e27e5fd`, merged 2026-07-20 evening — the only code change between the last green scheduled run and the failing one) removed Quasar entirely, breaking two page-object seams: | Failing test | Broken seam | |---|---| | `traces-bugs.spec.js` › trace values should be visible on hover in dark mode (@bug-4151) | `homePage.isDarkMode()` read the retired Quasar `body--dark` class on `<body>`. Dark mode is now the `.dark` class on `<html>` (set by `utils/theme.ts`). `themePage.js` was already migrated inside #13173 — `homePage.js` was missed. | | `logs-9044-7354.spec.js` › all 3 scroll-retention tests (#9044) | `getScrollContainerPosition()` / `scrollToResultsBottom()` navigated `pagination → .search-list → [class*="overflow-y-auto"]` — both hooks are gone. The results scroller is now the combined scroll area (`ref="scrollContainerRef"` in `logs/SearchResult.vue`). Helpers returned the `-1` "not found" sentinel, so `expect(scroll).toBeGreaterThan(10)` failed. | ## What Test-side only (no `web/` changes): - `homePage.isDarkMode()` → checks `document.documentElement.classList.contains('dark')`. - Logs scroll helpers → walk up from the `logs-search-result-pagination` anchor and pick the sibling child whose **computed** `overflow-y` is `auto`/`scroll`, so future utility-class renames don't break it again. Pagination helpers (`-page-N`, `-next`, `aria-current="page"`) already matched the new OPagination markup and needed no change. ## ENT ENT overrides none of these files — its regression workflow checks out OSS tests (`cp -r o2-enterprise/tests/* openobserve/tests/` overlays only ENT-owned files). The ENT scheduled run heals automatically once this merges; verified pre-merge by dispatching ENT `playwright_regression.yml` with `oss_branch: test/heal-dark-mode-scroll-selectors`. ## Verification All 4 previously-failing tests pass locally against a post-migration build (`main` @ `c523d0034c`, enterprise features): - `logs-9044-7354.spec.js` — 3 passed (49s) - `traces-bugs.spec.js -g "trace values should be visible on hover in dark mode"` — 1 passed (18s) OSS + ENT regression runs seeded against this branch (links in comments). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
fd85566742 |
feat: workflows version 1 (#13156)
This PR introduces workflows design at: https://github.com/openobserve/designs/blob/main/infrastructure/workflows/design.md --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
 
|
f15e27e5fd |
refactor: design token migration + CI guards (#13173)
# Design System Unification — one visual language, enforced end-to-end
Establishes **one source of truth for every visual decision**, wires the
whole app to it, and **locks it in with CI** so it can't drift back.
### Foundations
- **Design-token system ("one knob per decision").** Every colour, text
size, spacing, corner radius, and shadow is a named token defined in
exactly one place — change the token, every screen updates. Collapsed
hundreds of near-duplicate definitions and removed "fake" duplicates
that pretended to be different but weren't.
- **Shared component library (O2).** Buttons, tables, dialogs, drawers,
form fields, badges, etc. already know the correct colours, spacing, and
dark-mode behaviour; new screens compose them instead of hand-styling.
Added new primitives on this branch (`OContent`, `KpiCard`) and
standardized existing ones.
- **One page structure everywhere** (`PageLayout` / `AppPageHeader` /
`AppPage` / `OContent`) — titles, breadcrumbs, spacing, and content
edges line up identically across every screen.
- **One table implementation** — all tabular data uses the same table
(sorting, density, sticky headers, empty states).
- **Styling consolidated onto Tailwind; old systems removed** —
hand-written CSS driven to the practical minimum, the legacy **Quasar**
framework fully removed, surviving stylesheets required to justify
themselves.
- **Two-tier corner radius** — a tight **4px** for controls and a soft
**12px** for surfaces (dialogs, cards, the app shell), plus a
full-circle option — replacing five radius names that all secretly meant
the same thing.
### Detailed improvements across screens
**Page structure & alignment** — unified page shell rolled across ~40+
screens (IAM, Settings, Alerts, Dashboards, Functions, Pipelines, RUM,
Metrics, Synthetics, Billing, editors, lists); one header height; one
page-edge grid driving headers, tables, rails, and tabs; unified
side-rail widths; inset rail selection/hover pills; every tab strip on
the grid line; tightened title/subtitle spacing.
**Tables (cross-screen)** — action columns consistently centered (incl.
Organizations); unified row-number column with no "#" truncation;
non-standard tables restyled to match; loading skeletons made
geometrically identical to the loaded table; symmetric edge inset; IAM
first-column grid alignment fixed.
**Traces** — flame graph + span-detail correlation panel fixes;
correlated logs/metrics placeholder fit; sticky headers; removed a
double scrollbar + a 1px sliver; aligned table fonts/badges; polished
the service detail drawer; stream-selector + correlation alignment;
aligned loading spinners; floating overlay arrows for the
disabled-services strip.
**Logs** — field-list controls aligned (focus ring no longer clipped);
detail-drawer spacing + metrics-tab overflow fixed; Run Query toolbar
tidied; listing sizing + stray border fixed.
**Metrics** — search added to the "Type" facet rail and aligned with the
other rails; proper empty states for the explorer and facet rail.
**Dashboards** — removed inconsistent card shadows; tightened
borders/resizers/field-list spacing; aligned View Dashboard chrome to
the grid; chart colours via the token helper.
**Home** — redesigned the Usage tab; added reusable KPI stat cards so
summary strips stop drifting.
**Dark mode** — new theme-switch animation; fixed a 1px content shift on
toggle and a white flash on load; fixed AI-chat text and assorted
light/dark mismatches; completed the token-driven dark migration.
**Empty states, controls & small polish** — restored 3 broken
empty-state animations; adopted the standard empty state; added the
sliding indicator to the toggle control; fixed the Quality-page inset +
drawer badge; refreshed Search History / Schedulers; fixed the
alert-history drawer scrollbar + timeline hover; removed a stray
menu-bar border.
### Dead code & cruft removal
- **Deleted the entire legacy `utilities.css`** global stylesheet (well
over a thousand lines) — evacuated in waves, roughly half of it dead
once traced.
- **Removed Quasar entirely**, including dead `.q-*` styles and ~25
phantom `--q-*` variables that were silently breaking bits of the UI.
- Stripped hundreds of **dead CSS classes and tokens that compiled to
nothing** — e.g. 46 uses of a colour class with no backing token, a
no-op shadow class on 51 elements, dead layout/utility classes, a
leftover `card-container` block, a dead table-actions token, and a
mis-registered gradient that rendered nothing.
- Cut **unused keyframe animations (19 → 3)** and removed **unused
components** + a stray build artifact.
- Net effect: a much smaller, leaner styling surface — less to maintain,
fewer dead ends to copy.
## Safety & verification
Large but low-risk: this is overwhelmingly a **styling and structure**
change, not product behaviour or data. Guarded by the
design-token/consistency/type checks above plus the existing test suite,
and reviewed screen-by-screen.
---------
Co-authored-by: Kirtan Thakkar <kirtan@kiara.tech>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: ktx-mihir <mihir.patel@kiara.tech>
|
||
|
Vamsi Karthik RandClaude Opus 4.8
|
ba57f76e9d |
test(e2e): stabilize alpha1 Traces & Pipelines shards (error-toggle data seeding + flyout-nav / loading-button / indexing races) (#13246)
## What Stabilizes the **Traces** and **Pipelines** shards of the alpha1 cloud E2E suite (`playwright_alpha1.yml`) to green. Rebased on `main` — builds on #13191's protocol-aware ingestion agent (`selectAgent`), so no duplicate transport fix here. The failing run had 1 hard failure + 1 flake in Traces and 7 failures + 1 flake in Pipelines. Root causes: a data dependency (error traces not guaranteed), hover-flyout navigation force-clicks that miss, a refresh button disabled while loading, and logs indexing latency. All fixed with deterministic data/waits — no `waitForTimeout` as a fix, no `force:true`, no retry-count bumps, no weakened assertions. ## Root cause & fixes **Traces** 1. **`tracesAnalyzeDimensions` "P0: Error-only toggle filters trace results"** (hard fail, both attempts). The error-only toggle is the error-count badge in `SearchResult.vue`, rendered `v-if="errorCount != null && errorCount > 0"` — it exists only when the search returns an error span. The shared `default` trace stream has no guaranteed error traces (global-setup ingests only logs), so the badge was present or absent by luck of the window (hence the intermittent CI greens). Fix: `beforeAll` seeds guaranteed error traces via a new `ingestTraces(page, n, { forceScenario: 'error' })` option in `trace-ingestion.js`, and `tracesPage.waitForErrorBadgeAfterSearch()` re-runs the search until the seeded errors are indexed — a bounded wait for backend processing, not a sleep. 2. **`traceQueryEditor` "P2: View error spans and their details"** (flaky). `clickFirstErrorTrace()` used a blind `waitForTimeout(2000)` that raced the trace-details render before the test asserted on it. Now waits for `trace-details-tree` to be visible. **Pipelines** (`pipelinesPage.js`) 3. **`pipeline-section-tab-streamPipelines` 45s timeout** (5 tests: `pipeline-core:56/159/238`, `pipelines:479`, `pipeline-conditions:297`). `openPipelineMenu()` / `navigateToStreams()` navigate via `openNavFlyoutChild`, which hovers a nav group tile and force-clicks the teleported flyout child. The flyout self-closes on settle, so the click can miss and the page never navigates — the follow-up tab/refresh click then dead-waits the 45s action timeout. Both now verify the target page rendered (`pipeline-list-page` / `log-stream-table`) and fall back to a direct `page.goto()` of the route otherwise. (A failure screenshot showed exactly this: the Data flyout open over a blank page.) 4. **`log-stream-refresh-stats-btn` 45s timeout** (3 tests: `pipeline-dynamic:72/97/113`). The refresh button is an `OButton` with `:loading` bound to the stream-stats fetch, and `OButton` renders `:disabled="disabled || loading"`. Clicking during the (slow, on loaded alpha) stats load waited on actionability until timeout. `refreshStreamStats()` now waits `toBeEnabled()` before clicking; `navigateToStreams()` waits for the table instead of a blind 1s. 5. **`openTimestampMenu` empty-table dead-wait.** After exploring the destination stream, a bare `.click()` on the first logs row timed out 45s when the freshly-ingested rows weren't indexed yet (high indexing latency on loaded envs) — a latent flake exposed once fix 3 let the test reach this step. Now re-runs the logs query until a row appears, then clicks. ## Validation - **CI — both shards green (workers=3, alpha cloud):** https://github.com/openobserve/o2-enterprise/actions/runs/29563931877 (`alpha1-e2e / Traces` ✅ · `alpha1-e2e / Pipelines` ✅) - **Original failing run:** https://github.com/openobserve/o2-enterprise/actions/runs/29481437880 - **Local (alpha, CI conditions):** each previously-failing test re-run to green — Traces error-toggle & error-spans; `pipeline-core:56/159/238`; `pipeline-dynamic:72`. ## Notes - Only 4 files change (`pipelinesPage.js`, `tracesPage.js`, `tracesAnalyzeDimensions.spec.js`, `trace-ingestion.js`). The https-ingestion agent fix these depend on already landed on `main` in #13191 — this branch relies on it rather than re-adding it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vamsi Karthik RandClaude Opus 4.8
|
fddf2cf264 |
test(e2e): stabilize alpha1 SDR, GeneralTests & Metrics shards (stream/pattern-readiness + langTranslation OOM + Reka-table & editor races) (#13259)
## What
Stabilizes the **SDR**, **GeneralTests** and **Metrics** shards of the
alpha1 cloud
E2E suite (`playwright_alpha1.yml`) to green — the last three shards
remaining after
the rest of the suite was greened. Rebased on `main`.
The original failing run had, across these shards: **SDR** 1 hard fail +
1 flake,
**GeneralTests** 21 fails + 2 flakes, **Metrics** a load-sensitive
query-persistence
flake. Root causes were cloud data/UI-readiness races (freshly-created
streams &
regex patterns lagging their list/table; browser OOM under parallelism;
revamped Reka
UI re-rendering mid-interaction), all fixed with deterministic waits on
real signals.
**No `waitForTimeout` as a fix, no `force:true`, no `.first()`/`.nth()`
band-aids, no
retry-count bumps to hide failures, no weakened assertions. No
worker-count,
parallelism, playwright-config or CI/yml changes** — test code only.
## Root cause & fixes
**SDR**
1. **`scrollIntoViewIfNeeded` 45s timeout** (`multipleSDRPatterns`,
`queryTimeDrop`).
The revamped Reka streams/schema table re-renders rows mid-animation, so
the
scroll's stability polling spun to the 45s action timeout even though
the element
was visible. Guard with `expect(el).toBeVisible()` + an instant native
`scrollIntoView`; removed the trailing sleeps.
(`streamAssociationPage.js`)
2. **Stream Detail `click()` 45s timeout.** One-shot JS-then-Playwright
click raced the
re-rendering row. Replaced with a `toPass` retry that re-resolves the
button fresh,
fires a native DOM click, and verifies the detail sidebar opened.
3. **Regex-pattern create→list eventual consistency** (6 combined tests:
`ingestionTime*`,
`queryTime*`). Create is accepted (`verifyPatternCreatedSuccess` passes)
before it
propagates to the list read-replica, so the UI text-count check threw
"not found in list". Gate on the backend source of truth:
`waitForPatternCreated`
polls `GET /re_patterns` until present then confirms the UI rendered;
`waitForPatternDeleted` fixes the symmetric "Unable to delete … Reason:
unknown"
delete race. (`sdrPatternsPage.js`)
4. **Freshly-ingested stream lags the `/streams` table**
(`multipleSDRPatterns` @slow).
`openStreamDetail` reloads + re-searches until the stream's row appears
(bounded),
instead of a one-shot 5s wait. (Local-caught gotcha:
`isVisible({timeout})` does
**not** auto-wait — switched to `waitFor({state:'visible'})`.)
5. Transient cloud-gateway **connection timeouts** on `page.request`
pattern calls wrapped
in a connection-error-only retry.
**GeneralTests**
6. **cipherKeys ×6 — tab never visible.** Cipher Keys shares the exact
`visible: isEnt`
gate as Regex Patterns (`settings/index.vue`), and SDR's regex
management passes on
this same cloud env — so the tab **does** render; the test was clicking
the Settings
menu, which lands on the SectionHub where per-tab data-tests aren't
rendered. Fixed
by a **direct `goto('/web/settings/cipher_keys')`**, mirroring the
working regex page.
(Not skipped — that would have masked a real, fixable nav bug.)
7. **enrichment ×7 — `pipeline-section-tab-enrichmentTables` 45s
force-click.** The tab
only mounts after the pipelines page header renders; the
flyout→pipelines→section-tab
nav is fragile under load. Replaced with direct SPA nav to the
enrichment-tables route
+ `goto` fallback + a `toPass` settle-retry Add-button click.
(`pipelinesPage.js`)
8. **languageTranslation ×8 + landingPage ×1 — browser OOM cascade**
("Target page/context/
browser has been closed"). 10 exhaustive ~27-page full-app crawls ran
`mode:parallel`
under `--workers=5`; a worker's browser was OOM-killed and every test
after it
cascaded. **Kept parallel** (serializing overran the shard timeout) —
the crawl now
navigates `waitUntil:'domcontentloaded'` + a bounded settle instead of
waiting up to
10–30s for `networkidle` on chart/websocket-heavy pages, cutting each
heavy page's
peak-memory dwell ~10s→~3s (so 5 concurrent crawls no longer spike
together) and making
each crawl ~3–4× faster. Added fail-fast on a closed page/context so any
residual crash
surfaces at its true origin instead of being swallowed/retried.
9. **schema ×2 — `selectStream` can't find just-created
`test1`/`test3`.** Local reproduction
proved the root cause: the **fixed** stream names hit `400 "stream is
being deleted"`
(a prior run's cleanup still finishing) → the stream got no data → never
listed. Fixed
with **unique per-invocation stream names**; also hardened the ingestion
helper
(status-check + transient retry) and added a streams-API readiness gate
before
`selectStream`. (`schema.spec.js`)
10. **enrichment `@upload` ×2 — `_search` `waitForResponse` timeout.**
The 30s search wait
was armed **before** the up-to-120s `selectStream`, so it expired
mid-selection. Re-armed
it (200-gated) immediately before `applyQuery`, plus a streams-API
readiness gate for
`e2e_automate`. `schemaload` gated on the `streams?type=logs` response
before reload.
**Metrics**
11. Replaced `waitForTimeout`s in `metricsPage.js` with real signals
(`data-selected="true"`,
`data-state="on"`) and fixed a reka `OToggleGroupItem` state-detection
bug (active state
is `data-state="on"`, not a `selected` class).
12. **`metrics-promql-query-persistence` load-sensitive flake.**
`selectFirstStream` returned
before the async auto-populate finished, and persistence tests read the
query editor
before Monaco re-rendered after tab switches. Added editor-readiness
gates on real
signals (auto-populated query present; Monaco mounted + laid out) in
`metricsQueryEditorPage.js`; assertions unchanged.
## Validation
- **CI — SDR / GeneralTests / Metrics green (alpha cloud):**
https://github.com/openobserve/o2-enterprise/actions/runs/29633444186
- **Original failing run:**
https://github.com/openobserve/o2-enterprise/actions/runs/29481437880
- **Local (alpha cloud, CI conditions, single-worker):** each
previously-failing test re-run
to green — `schema.spec.js` 3/3, `multipleSDRPatterns.spec.js` 2/2.
## Notes
- 21 test files change (8 page objects + 13 specs). No shared
cleanup/setup, config, or
workflow files touched.
- RegressionSet was unaffected by this branch — a transient hang in the
workflow's
"pre-test cleanup" step (before any test runs), unrelated to these
changes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Vamsi Karthik RandClaude Opus 4.8
|
96bbed8a6e |
test(alerts): stabilize alpha1 Alerts shard on cloud — robust empty-name validation + hardened self-seeded stream registration (#13270)
## What Follow-up to #13209. In the full-suite alpha1 CI (runs 29576418415 → 29655287388) the **Alerts** shard was failing on two independent issues. This PR makes both deterministic so the shard goes green. Test-infra / page-object only — **no product code**. ✅ Green: alpha1-e2e / Alerts — 20 passed, 4 flaky (recovered on retry), 3 skipped, 0 failures https://github.com/openobserve/o2-enterprise/actions/runs/29655287388/job/88116296475 ## Root cause & fixes ### 1) `alerts-ui-operations:191` — empty-name validation assertion **Root cause:** the check asserted an inline field-error element (`add-alert-name-input-error`) that isn't reliably rendered for the topbar name input, so the assertion timed out even though the empty-name Save is correctly blocked. Verified against alpha with a live session: on empty-name Save the app surfaces the block as a **toast — "Alert name is required."** **Fix:** assert the toast (the app's actual, visible signal), keeping the inline field error as an `.or()` fallback so the check stays robust regardless of which surface renders. Validated locally against alpha (PASS) and green in CI. ### 2) `alerts-e2e-flow:106/160` + `alerts-import:52/165` — self-seeded stream registration **Root cause:** these specs seed a unique per-test stream (`alert_e2e_*` / `alert_import_*`) with custom columns, then wait for it to register in `GET /api/{org}/streams?type=logs` before opening the wizard. On the shared cloud alpha org under full multi-shard concurrency, that registration **intermittently takes minutes, not ~1s**, to reflect in the list — reproduced with both Playwright `page.request` and a raw HTTP client, so it is **not** a test-auth issue. The wizard's stream dropdown reads the same list, so the stream is absent there too. The old poll waited only 15/60s and **threw**, hard-failing in `beforeEach`. **Fix:** widen the registration poll to **90s** (matches global-setup's indexing budget) and make it **non-fatal** — warn and proceed instead of throwing. The stream is already ingested, and the point-of-use dropdown selection already retries (in-place ×4 + wizard re-entry ×2); with `retries: 2` those slow-to-register streams now recover as *flaky* rather than hard-fail. In the green run, 6 streams hit the 90s warning and all recovered. ## Known follow-up (infra, not test-fixable here) The deeper cause of the stream flakes is a **server-side streams-list propagation lag with session-sticky routing** on the shared cloud alpha org — a freshly-ingested stream can take minutes to appear in `/streams` for a given session (some sessions land on a stale node). No test-side wait fully resolves it; the retries absorb it. Flagged for infra investigation. ## Scope / safety - 2 files: `pages/commonActions.js`, `pages/alertsPages/alertsPage.js`. No product code. - No tests added/deleted/skipped, no `.only`, no assertions weakened (191 asserts the real user-facing signal; the `.or()` keeps inline-error coverage). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Prabhat Sharma
|
eaba77fa34 |
fix(i18n): DeepSeek translation pipeline, BCP-47/en-US locales, +ru/pl/vi, full backfill (#13263)
## Summary
Replaces the AWS Translate i18n pipeline with a DeepSeek (LLM) backend,
standardizes locale files to BCP-47 naming with an `en-US` base, adds
new languages, and backfills **every locale to 100%**.
## What changed
### Translation engine (DeepSeek)
- Swap AWS Translate → DeepSeek via its OpenAI-compatible API
(`scripts/translations/`). Auth via the `DEEPSEEK_API_KEY` secret;
per-request 60s timeout.
- **Batched** translation (50/call) with **per-string validation**:
interpolation placeholders (`{count}`, `%s`, `@:key`), vue-i18n
**pluralization pipes** (`{n} Day \| {n} Days`), and literal escapes
(`{'@'}`, `{'{'}`, `{'\|'}`) are preserved or the string is rejected and
retried.
- Robust recovery: index-aligned batches, split-on-mismatch to isolate a
single bad string, temperature-perturbed retries, and rejection of
empty/whitespace output (never blanks a label). Removed the old
`TRANSLATION_MAX_KEYS` safety cap.
### Locale naming + base language
- All 15 locale files renamed to **BCP-47 hyphen** form (`en-US.json`,
`zh-CN.json`, `zh-TW.json`, `fr-FR.json`, …) — matching
`navigator.language` / `Intl` / HTML `lang`.
- **English base switched `en-gb` → `en-us`** (app code,
`fallbackLocale`, BCP-47 map, selector, defaults). British-Pound
currency and date-formatting `en-GB` usages left untouched. Existing
`en-gb` cookies fall back cleanly to English — no migration needed.
### New languages
- **Russian (`ru`), Polish (`pl`), Vietnamese (`vi`)** — fully generated
and wired into `localeFileMap`, `APP_LOCALE_TO_BCP47`, and the language
selector.
- **Traditional Chinese (`zh-TW`)** now maintained by the translator
alongside Simplified (`zh-CN`).
- RTL languages (`ar`, `fa`) intentionally **deferred** until the app
has `dir="rtl"` support.
### Full backfill (one-time)
- Every pre-existing locale filled from ~5.9k up to the full **8,269
keys** (~26.9k strings). `.translation_state.json` rebuilt complete, so
future runs are incremental.
### Nav i18n bug fix
- The **Dashboards** and **Data** nav group labels were hardcoded
English (bypassing i18n) and stayed English in every locale. Now
resolved via `t()` (`menu.dashboard`, new `menu.data`), localized and
re-resolving on language change.
## Verification
- Full web unit suite: **39,035 passed / 414 skipped / 0 failed** (1,167
files).
- `type-check` clean (0 TS errors).
- Locale registry sync spec green; all 15 files = 8,269 leaves.
- Translation engine verified live against `deepseek-v4-flash`.
## Notes
- Since all locales are now 100%, the post-merge `update-translations`
workflow will find nothing pending (no follow-up translation PR).
- `ar`/`fa` to be added with RTL support in a follow-up.
|
||
|
Prabhat Sharma
|
21cd5ca977 |
fix: standardize service account credential terminology in web UI (#13255)
## What Cleans up the wording and i18n coverage around **service accounts** in the web UI so the credential is consistently called a **token**, and fixes a mislabeled toast. ### Changes - **Standardized credential wording** — `en.json` service-account labels drifted between "Service Token", "Service Account Token", and "Token". Unified on **"Token"** (kept "Service Account Token" only as the standalone reveal-dialog title). - **Localized hardcoded strings** — moved all `ServiceAccountsList.vue` toasts (create / update / delete / rotate / loading / copy / bulk-delete) into a new `serviceAccounts.toast.*` namespace; localized the tab labels in `EditRole.vue` / `EditGroup.vue` and the search placeholder in `GroupServiceAccounts.vue` (added `iam.permissions`). - **Fixed a mislabeled toast** — the org-passcode flow in `Ingestion.vue` showed `"API Key not found."`; it's the org **passcode** (sibling button is "Update Passcode"), so it now reads `"Passcode not found."` (spec updated to match). ### Notes - `en.json` is updated as the source of truth; the other locale files sync via the normal translation flow. - Dead `apikeyLabel` / `copyapikey` / `deleteAPIKeyMessage` keys (no component references) were left untouched — out of scope for this change. ## Testing - `vitest` unit suite for the touched files passes (134/134, incl. updated `Ingestion.spec.ts` assertions). ## Related - Docs counterpart (new Service Accounts page + credential comparison): openobserve/openobserve-docs PR. |
||
|
Prabhat Sharma
|
d9d128d61c |
fix: Serviceaccount1 (#13257)
Improve UI for creating service accounts. Improve UI for accessing MCP server. |
||
|
Huaijin
|
3b44da3eee |
refactor: extract shared common crate (#13252)
## Summary - introduce a `common` workspace crate for shared metadata, infrastructure helpers, and utilities - keep service-backed code in `openobserve-core` while preserving the existing `common` compatibility namespace - flatten `src/core/src/service/` into the standard `src/core/src/` module layout without using `#[path]` - retain `service::*` as compatibility re-exports from `service.rs` - move cipher request/response types from core into the search crate while preserving `openobserve_core::cipher` - remove unused Cargo dependencies from workspace crates Enterprise companion: https://github.com/openobserve/o2-enterprise/pull/2202 ## Validation - `cargo check --workspace --all-targets` - `cargo clippy -p common -p openobserve-core -p search --lib -- -D warnings` - common crate unit tests: 271 passed, 4 ignored - enterprise overlay: `cargo check -p openobserve-core --features cloud` - migrated cipher tests: 8 passed - `cargo deny check licenses` - `cargo fmt --all -- --check` - `git diff --check` |
||
|
ktx-mihirandktx-vaidehi
|
eaebff9e57 |
fix: alerts module migration with zod (#13227)
- alert module migration with zod --------- Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com> |
||
|
Vamsi Karthik RandClaude Opus 4.8
|
30b736e6e4 |
test: stabilize alpha1 Logs-Queries & Logs-Features shards on cloud (monolithic download-test split + share-link redirect hydration + SQL-commit reliability) (#13241)
## What Stabilizes the **Logs-Queries** and **Logs-Features** shards of the alpha1 cloud E2E suite to green. Test-infra + page-object changes only; **no product code** (5 files under `tests/ui-testing/`). > **Stacked on #13240** (Logs-Core). Base is `fix/logs-core-alpha1-e2e` — please merge > #13240 first; GitHub will then retarget this PR to `main`. The diff here is only the > Logs-Queries/Features work (5 files). ## Root causes & fixes ### Logs-Queries **`logsDownloads.spec.js` — monolithic test exceeded the test timeout (hard failure).** All 14 download scenarios ran sequentially in **one** `test()` (each a full `page.reload` + re-setup), needing ~7–8 min against a 360s budget. It hit 360s mid-run, Playwright tore the page down, and the remaining scenarios failed with **"Target page/context closed"** (a cascade, not a download bug). A `try/catch` wrapper also swallowed per-scenario errors and rethrew an aggregate, hiding the real stack. → **Split into 14 isolated per-scenario tests.** The `beforeEach` already yields a fresh, query-loaded page, so no per-scenario reload is needed; each test is short, isolated (one failing download can't close the page for the others), and parallelizes. Removed the error-swallowing wrapper; `@downloadAll` tag preserved. **`searchpartition.spec.js` — Reka OSwitch state asserted within a hardcoded 5s (flaky).** `verifyHistogramState` waits for the histogram OSwitch `data-state="unchecked"` element, but the Reka switch animates the checked→unchecked transition and the settled state can appear >5s after the toggle+execute under CI load. → Raised the poll ceiling to 15s (still asserts the real end-state, no sleep). ### Logs-Features All the share-link failures were **flaky, not product bugs** — the redirect fully restores every param; the tests raced the SPA under CI load. **Reading UI/URL state before the short-URL redirect finished hydrating.** `getQueryEditorText()` returned `""` (lazy Monaco visible but not yet pre-filled) and `captureCurrentState()` read `show_histogram=null` (the SPA rewrites URL params in stages). → New page-object helpers `waitForQueryEditorContent`, `waitForUrlParam`, and URL-stabilization inside `waitForRedirectComplete` (`_waitForUrlStable`). No fixed `waitForTimeout`. **Bare `clickRefresh()` cancelled the in-flight SQL-mode auto-search** ("Running query cancelled" toast), so `sql_mode`/query never committed to the short URL → empty editor after redirect. → Commit via `runQueryAndWaitForResults()` (waits for the Cancel state to clear first), then confirm `sql_mode` is in the URL before sharing. **SQL setup itself was unreliable.** `enableSqlModeIfNeeded` (Vue `meta.sqlMode` flag) didn't always stick, and `clearAndFillQueryEditor`'s focus-sensitive `Ctrl+A` select-all silently appended in headless CI (**doubled query**). → New `setupSqlQueryForShare(query)`: sets the Vue flag + writes the exact query via Monaco `executeEdits` (verified) + `runQueryAndWaitForResults`, then confirms `sql_mode` committed to the URL — **retries once, then strict-asserts** (writing SELECT text alone does **not** flip `meta.sqlMode`; the flag is what the app serializes to `sql_mode`). `waitForUrlParam` gained a `required` mode that throws instead of silently letting a bad state through to the share. **Residual Monaco lazy-mount race after the short-URL hop** — the editor occasionally mounted without applying the query from the (resolved) URL. → `waitForRedirectedQueryEditorContent` self-heals with **one** reload of the resolved URL (the reliable URL-param prefill path). **Share success toast strict-mode violation** — the "Link Copied Successfully!" toast coexisted with the cancelled-query info toast, so the broad selector matched 2 elements. → Match the share-result toast by message with `.first()` in the share helpers. **`searchPatterns.spec.js` — async pattern extraction still empty/timeout after 60s** under alpha indexing load. → `setupPatternsView` re-runs the query and waits once more before giving up (resilience for the data dependency; callers still assert the returned state). ## Scope / safety - **5 files:** `pages/logsPages/logsPage.js` + specs `logsDownloads`, `monaco-query-prefill`, `shareLink`, `searchPatterns`. **No product code.** - **No tests deleted, no `.only`, no unconditional skips, no assertions removed.** - Every fix replaces a fixed sleep / wrong assumption with a deterministic wait on the real condition — no `waitForTimeout`, no `force`, no retry-based masking. ## Validation - Local on cloud alpha (`retries:0`, stricter than CI's `retries:1`): - `logsDownloads` **14/14** (`--workers=3`) — no timeouts, no "Target page closed". - Previously-flaky share-link / SQL / searchpartition tests **12/12** under `--repeat-each=3` (`sql_mode` committed on the first attempt every time; retry never needed). - Full `shareLink` + `monaco-query-prefill` files **15/15** — no regressions. - alpha1 Logs-Queries + Logs-Features CI: green — https://github.com/openobserve/o2-enterprise/actions/runs/29539009573 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
9c32b03b34 |
test: stabilize alpha1 Logs-Core shard on cloud (quick-mode detection + search-toolbar sync + scheduler-list hidden-button recovery) (#13240)
## What
Stabilizes the **Logs-Core** shard of the alpha1 cloud E2E suite to
green, running
**fully parallel** (`--workers=5`). Test-infra + page-object changes
only; **no product
code** (7 files under `tests/ui-testing/`).
## Root causes & fixes
### Quick mode / interesting fields (7 tests)
- **`enableQuickModeIfDisabled` treated the wrong element as the
quick-mode signal.** It
returned early whenever the field-list **user-defined-schema toggle
group** was visible —
but `FieldListPagination` renders that group whenever a stream exposes a
UDS (always true
on cloud), independent of `searchObj.meta.quickMode`. The per-field
interesting (star)
buttons are gated purely on `quickMode`, so they never rendered and
every
`ensureFieldIsInteresting` call timed out. Now verifies the **actual**
Quick Mode switch
state via the utilities menu and confirms by waiting for a per-field
interesting button.
- **`sqlModeOrderBy` hung on `waitForLoadState('networkidle')`** —
streaming keeps the
network busy so it never settles; switched to `domcontentloaded` (the
editor assertion is
the real readiness gate).
### Search-toolbar synchronization (logspage / histogram flakes)
- **`clickRefreshButton` / `clickRefresh` clicked the refresh button
mid-state-swap.** The
Run-query button swaps to a "Cancel query" variant (and briefly
detaches/disables) while a
prior/auto search is in flight; clicking then cancelled that search or
hit a
not-visible/detached node (`element is not visible`), and a subsequent
`waitForResponse('/_search')` never fired (60s timeout). Both now wait
for the run-mode
button to be **visible AND idle** (not Cancel, not disabled/`aria-busy`)
before clicking —
no `force`.
- **`clickLiveMode5Sec` + `clickRelative6WeeksButton` popovers shifted
mid-click.** The
refresh-interval dropdown and the date picker are popovers anchored to
the toolbar; while a
search is in flight the toolbar reflows and the popover repositions, so
the target never
reaches Playwright's "stable" state (~45s click timeout) or the click
misses (date range
stayed "Past 15 Minutes"). Both now wait for the toolbar to settle
(`_waitForQueryButtonIdle`) first; the date preset additionally verifies
its trigger label
and retries once.
### Histogram error-handling
- **Query-propagation race** — `clearAndFillQueryEditor` fills Monaco
but the value reaches
the store on a debounce, so `clickRefresh()` could run the **previous**
query and the
no-events/error state never appeared. Now waits for the Monaco model to
reflect the new
query (`waitForQueryEditorValue`) before running.
### Logs-table field management
- **Fixed `waitForTimeout(2000)` in `beforeEach` →
`waitForFieldListReady()`** — the
expandable field rows only render once the stream's fields have indexed
**values**, and
cloud indexing after ingestion lags under parallel load, so the sidebar
was empty
(`fieldCount === 0`, "field expand button not found"). Gates on the
data-backed field list
(60s, matching the global-setup indexing budget).
- **Severity-colors** read status bars after a fixed 3s → poll until
they render.
- **Case-insensitivity** counted matching fields after a fixed 500ms →
`expect.poll`.
- **Persist-after-refresh (app timing)** — the added column survives
reload only via the
`logFilterField` localStorage entry, written **after** the column
renders; reloading before
that commit lost it (now waits for the localStorage write). The restore
is then
intermittently slow/dropped on the first reload under cloud load, so it
reloads up to 2×
and polls for the restored column.
### jobSearch — scheduler-list hidden button (app defect, worked around)
- **App defect:** the scheduler list renders under
`v-show="showSearchScheduler"`, and the
logs page **resets that flag whenever the URL query loses the
`action=search_scheduler`
param** (`updateUrlQueryParams` strips it after a search). The "Get
Jobs" button is then in
the DOM but `display:none` and never recovers — the whole jobSearch CRUD
suite timed out.
`_getJobRowIndex` now **re-navigates to the scheduler-list URL**
(restores the param →
re-shows the list) and retries; validated firing and recovering live on
alpha. Written up
for a frontend fix (`fix/search-scheduler-job-issue`).
- **`validateAddJob` "Job added successfully" toast made best-effort** —
it's transient and
`searchSchedulerSubmit` already awaited the POST (returned the jobId),
so the toast can
auto-dismiss before the assertion runs; creation is already confirmed by
the jobId + the
scheduler-list lookup. Same re-open recovery applied to its Get-Jobs
click.
## Scope / safety
- **7 files:** `Logs` page-objects (`logsPage`, `logsQueryPage`,
`logsPageEP`,
`jobScheduler`) + Logs specs (`logshistogram`, `logsquickmode`,
`logstable`). **No product
code.**
- **No tests deleted, no `.only`, no unconditional skips, no assertions
removed.**
- Every fix replaces a fixed sleep / wrong assumption with a
deterministic wait on the real
condition — no `waitForTimeout`, no `force` in the touched paths.
## Validation
- Local on cloud alpha: `logsquickmode` 8/8, `logshistogram` 5/5,
`logspage` 23/23,
`logstable` field-management + persist (`--repeat-each` clean),
jobSearch scheduler-list
recovery observed firing and recovering live.
- alpha1 Logs-Core CI: green —
https://github.com/openobserve/o2-enterprise/actions/runs/29471282407
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
|
||
|
|
3906bdc296 |
test: stabilize alpha1 Alerts shard on cloud (fix auto_playwright_stream being-deleted race + concurrency hardening) (#13209)
## What
Stabilizes the **Alerts** shard of the alpha1 cloud E2E suite to green,
running **fully
parallel** (`workers:3`, `fullyParallel:true`) — no serialization.
Test-infra +
page-object changes only; **no product code**. The one product-side
defect surfaced (the
prebuilt destination form unmounts its fields on template auto-load) is
worked around in
the test layer and written up separately for a frontend fix.
## Approach change since the first revision
The first version serialized the contended tests (`@serialAlpha1` + a
dedicated
`chromium-serial` project). Per review feedback ("solve it in
parallel"), that was
**removed** — the shard now runs **fully parallel**, and the flakiness
is fixed at the
source (fixture / registration / render races) instead of by
serializing.
## Root causes & fixes
### Streams / fixtures
- **`auto_playwright_stream` "being deleted" race** — the 3
`alerts-ui-operations` tests
now use the shared, always-protected **`e2e_automate`** stream
(pre-ingested by
global-setup, never deleted by cleanup), eliminating the shared
fixed-name delete race
entirely. `cleanup.spec.js` no longer deletes `auto_playwright_stream`;
`commonActions.ingestTestData` checks `response.ok()`, retries the "is
being deleted"
race, and polls the streams API until the stream is registered before
the wizard opens.
### Alert wizard
- **Dedup alert blocked at save** —
`createScheduledAlertWithDeduplication` typed its SQL
into the Monaco editor without clearing the default `SELECT * FROM
"<stream>"`, producing
malformed SQL → "Please fix the SQL error before saving." Now clears the
editor first.
- **Stream-select ordering + registration poll** — drive the stream
dropdown only after its
list fetch resolves; poll registration; page-reload recovery when the
cached list is stale.
### Prebuilt destinations (the deep one)
- **App defect:** on a *reused* form (2nd+ destination), selecting a
prebuilt type
intermittently **unmounts the credential/name fields ~0.5 s later**
(template auto-load
re-render) and they stay gone. Confirmed via DOM timeline (`count:0` for
25 s+). Since a
wait can't recover an unmounted element, `selectDestinationType` clicks
the card once,
waits for the template network to settle, confirms the fields are
**stable** (not just
present), and — if they unmount — **remounts the form** (cancel+reopen →
full page reload)
and re-selects. Written up separately for a frontend fix.
- **List verifications raced the OTable loading skeleton** —
`expectDestinationInList` / `verifyDestinationExists` /
`verifyAlertCreated` now wait for
`o2-table-skeleton-body` to clear and poll the **list** endpoint the UI
actually reads
(not just the single-GET) before asserting the row.
### Form-validation specs
- `alerts-form-validation` + `anomaly-detection-form-validation` flipped
serial→parallel and
made self-sufficient: `beforeAll` seeds a worker-scoped
template+destination via API (the
"Add Alert" button is hard-disabled with zero destinations), `afterAll`
removes them,
cleanup patterns registered.
### Misc
- Cloud-correct (https) ingestion agent; alert search-input +
`updateAlert` edit-form wait
hardening for shared-org load.
## Cloud environment gates (legitimate — not coverage drops)
`test.skip(isCloudEnvironment())` (dedup validation), `if
(!isCloudEnvironment())` (import
trigger round-trip), and external destination URLs — all because cloud's
**SSRF guard**
blocks self-referencing destinations by design. All still run fully on
OSS/self-hosted.
## Scope / safety
- 15 files: Alerts page-objects, alpha1 config + global-setup, Alerts
specs, cleanup. **No product code.**
- **No tests deleted, no `.only`, no unconditional skips, no assertions
removed.**
- The temporary Alerts-only playwright matrix (added for faster
iteration) has been
**reverted** — the full OSS e2e matrix is restored.
## Validation
- Prebuilt P1 validated locally on cloud alpha (repeated clean passes;
every field-unmount
recovered by the remount).
- alpha1 Alerts CI:
https://github.com/openobserve/o2-enterprise/actions/runs/29391293698
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
|
||
|
|
3c8f106425 |
test(e2e): stabilize alpha1 cloud E2E — shared https-ingestion fix + Streams revamp/cloud-race/CI-flake fixes (#13191)
## What Stabilizes the **Streams** shard of the alpha1 cloud E2E suite (`playwright_alpha1.yml`) to green after the UX revamp — plus a shared node-fetch protocol fix that unblocks ingestion for several non-Streams shards too. Rebased on `main` (no longer stacked on #13155). Un-skips the multistream **interesting-fields** test, and rewrites the **extended-retention** flow for the revamped UI (page-object retained, but the test itself is currently re-skipped — see fix 8). ## Root cause & fixes 1. **https ingestion rejected in shared fetch helpers.** `apiUtils.js`, `apiCleanup.js`, `pipelinesPage.js`, `streamsPage.js`, `model-pricing.spec.js` still forced an `http.Agent` on node-fetch → `Protocol "https:" not supported` against alpha's https ingestion URL, killing every multiselect / streamname / streaming ingestion. Replaced the single `http.Agent` with a protocol-selecting agent (an `http.Agent` for `http://` and an `https.Agent` for `https://`), passed as a function so node-fetch picks per-URL. Same fix as #13155's `ingestionPage.js`. 2. **Add Stream dialog selectors** (`streamsPage.js`). The revamped form is an `ODialog`: the panel is `add-stream-dialog`; save/cancel/close are the ODialog-owned `o-dialog-primary/secondary/close-btn` ids (scoped under the panel). The old `add-stream-title` / `save-stream-btn` / `add-stream-cancel/close-btn` no longer exist on the dialog. 3. **Streams-page navigation** (`streamsFormValidationPage.js`). The `/streams` left-nav tile is now a hover-flyout group — navigate by URL (`/web/streams?org_identifier=…`) and wait for the Add Stream button (rendered only after zoConfig loads, `v-if="isSchemaUDSEnabled"`) before clicking. 4. **Casing-preservation test skipped on cloud** (`streamname.spec.js`). Alpha runs the server default `ZO_FORMAT_STREAM_NAME_TO_LOWERCASE=true` (local CI sets it `false`), so stream names are always lowercased there — the feature under test is off. `test.skip(isCloudEnvironment(), …)`. 5. **TC002 cloud async-delete race** (`streamsPage.js`). A just-deleted stream lingers "being deleted" server-side after it leaves the streams list, so the recreate POST 400s; the create eventually goes through, so a follow-up "already exists" is treated as created. Also re-clicks Save when no POST fires (submit occasionally doesn't reach the form under parallel load), asserts name/retention values stick before saving, and adds `_waitForStreamGoneCloud` so `deleteStreamViaAPI` polls the streams list until the stream is actually gone. Success is now signalled by the dialog closing (the toast auto-dismisses), and both create and retention saves capture the API response so a server-side failure shows in the test log. 6. **streaming field-list waits** (`streamsPage.js`). `kubernetesContainerName` filters the field list and waits for the expand toggle before clicking — the cloud field list populates slowly. 7. **`managementPage.goToManagement` retry.** Under parallel-worker load the first settings-nav click can land before the nav is interactive; retry (`expect().toPass`) until the settings page renders. 8. **Un-skipped / rewritten for the revamped UI:** - **interesting-fields** (`multiselect-stream.spec.js`): **un-skipped.** Enable quick mode and hover the field row so the (ⓘ) toggle renders. Locators live in the page object (`logsPage.hoverAndClickInterestingFieldLast()` + reused `clickLogTableColumnSource()`), not in the spec. - **extended-retention** (`stream-settings.spec.js`): the page-object flow was rewritten for the new date-range calendar (`daterangecalendar-cell-*`), OTable row checkbox, and shared `ConfirmDialog` — **but the test is currently `test.skip`ed.** Clicking the Extended Retention tab collapses the entire Stream Detail drawer on alpha's deployed build (DOM elements drop 274 → 2), so the picker never renders. Verified by live DOM diff; sibling Settings-tab tests still pass, so it's a deployed-frontend regression, not test flakiness. Skipped with a documented reason; re-enable once the tab renders its panel again. 9. **CI-flake stabilization pass** (`logsPage.js`, `streamsPage.js`) — from alpha1 Streams CI run 29504364410: - **`clickInterestingFields`** (`logsPage.js`): an instant `isVisible()` check raced the field list re-rendering after the query completed, then toggled quick mode (already on) which churned the utilities menu. Now asserts quick mode on once (idempotent) and waits deterministically (20s + fallback) for the interesting buttons to render. - **`navigateToExtendedRetention`** (`streamsPage.js`): replaced fixed `waitForUI(2000)/(1000)` sleeps with real waits — schema dialog tab bar visible, then the date-picker button visible — so "date-time-btn not visible" can't recur. - **`navigateToStreamExplorer` / `searchStream` / `clickAddStreamButton`** (`streamsPage.js`): wait for the streams search input / add-stream button to actually render before interacting instead of a fixed 1s wait, removing the 45s click timeouts on slow CI. ## Validation - **Local (alpha, CI conditions — 3 workers):** Streams shard green (30 passed / 2 skipped / 0 failed; TC002 stable 3/3) prior to the extended-retention re-skip. With extended-retention now skipped (fix 8), that test moves to the skipped column. - **CI:** https://github.com/openobserve/o2-enterprise/actions/runs/29328193145 · flake triage from run 29504364410. ## Notes - `pipelinesPage.js` / `model-pricing.spec.js` receive the same shared protocol fix as collateral, so a couple of non-Streams shards benefit too. - The extended-retention page-object rewrite (calendar / OTable / ConfirmDialog) is kept and correct; only the test invocation is skipped pending the alpha drawer fix. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
|
Huaijin
|
66ca3d2439 |
refactor: extract PromQL evaluator into dedicated crate (#13239)
## Summary - add a dedicated promql workspace crate for parsing and evaluation - keep cluster routing, cache, WAL, file-list, and request orchestration in openobserve-core - preserve existing service::promql call sites through a compatibility facade; the facade re-exports an explicit item list (not a glob), and the promql search orchestration in core depends on the promql crate directly - inject enterprise query cancellation through TableProvider instead of coupling the evaluator to core search state - replace the core regex-UDF dependency with DataFusion regexp_like expressions ## Companion PR - openobserve/o2-enterprise#2198 updates the enterprise workspace override for the new promql crate ## Validation - cargo fmt --all -- --check - cargo check --workspace - cargo clippy -p promql --all-targets -- -D warnings - cargo clippy -p openobserve-core - cargo test -p promql: 363 passed - verified the promql dependency tree has no openobserve-core or search dependency - cargo check -p o2_enterprise --lib in the paired private workspace ## Notes The evaluator moves atomically in this PR. The existing service::promql::search subtree remains in core because it owns OpenObserve-specific orchestration rather than PromQL evaluation. The two web/ file changes only update source-path references in comments that pointed at the moved Rust files (src/service/promql/... -> src/promql/src/...). |