mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 08:09:27 +00:00
edcb3c835e6ef41e4a42853a42d2617c85170ff6
201
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Shrinath RaoandShrinath-O2
|
edcb3c835e |
test(autoe2e): generated E2E tests for #14613 (splunk-hec-ingestion-window) (#14641)
Co-authored-by: Shrinath-O2 <214651540+Shrinath-O2@users.noreply.github.com> |
||
 Neha P
|
80d9d6ff91 | test: regression coverage for 4 closed Needs-Automation bugs, and repair the alert wizard's stream select (#14689) | ||
|
Shrinath Rao
|
b2e058a202 | test: regression coverage for 4 Needs-Automation issues (1 API + 3 Logs UI regression) (#14538) | ||
|
Neha P
|
8dcaf5e8e8 | test: cover report folders (#11219) and the incident status filter (#12611) (#14646) | ||
|
Neha P
|
b607d0a623 | fix(alerts): repair three clone-dialog bugs, and cover the clone surface (#14621) | ||
|
Neha P
|
2c341ca4ae | test(api): cover 8 closed Needs-Automation issues (#14598) | ||
|
Shrinath Rao
|
b85030e7d9 | test(promql): guard left-open range-selector boundary (#14546) (#14584) | ||
 
|
3be8821d93 |
fix: correctly handle silence for pending period (main) (#14564)
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
|
Neha P
|
f3db1f5477 | test(workflows): cover folders and folder-scoped RBAC [merge after o2-enterprise#2653] (#14577) | ||
|
Neha P
|
5cecdc0f7f | fix(alerts): send a name-resolved expression summary on composite list rows (#14458) | ||
 Hengfei Yang
|
1263f05595 | fix(ci): honor fuzz health-check settings under the CI profile (#14527) | ||
|
Neha P
|
94a82a2ef8 | test(alerts): real E2E coverage for composite alerts (#14456) | ||
  Ashish Kolheandgithub-actions[bot]
|
2a19a61a31 |
fix(dbm): Metrics tab — database load, themed catalog, health ratios, and the o2_dbm_kind panel-failure fix (#14401)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
  udevandsai nikhil kethe
|
f27440841b |
fix(ai): improve provider testing, playground feedback, and experiment results (#14174)
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Neha P
|
0fb66d4f26 | test: fail-loud when WORKFLOWS_REQUIRED=1 instead of silent skip (#14040) | ||
|
Neha P
|
e74d5ea210 | test(workflows): parametrized condition-eval matrix (17 cases) (#13999) | ||
 Prabhat Sharma
|
fea71e02d4 |
feat(alerts): alert library — browse, preview and install curated alerts (#13922)
## Alert Library A curated catalog at `/alert-library`: browse packs of ready-made alerts, see which ones can actually run against the streams this org already receives, preview a verdict against real data, and install one or many. The library is a **peer page** of Alerts / Notification Destinations / Templates — flat routing, presented as tabs via a shared `AlertSectionTabs` in `OPageLayout #header-tabs` (the pattern `PipelineSectionTabs` already ships). ### What landed, by phase | Phase | What | | --- | --- | | Data layer | S3 manifest fetch, cache, readiness resolution (`useAlertLibrary`, `constants/alertLibrary`, `types/alertLibrary`) | | Gallery | Rail (packs / categories, multi-select, segmented switch), stat-strip availability facets, cards | | Drawer | Detail view with query preview and customizable tunables | | Install | Bulk-install wizard — destination selection, per-alert tunables, stale-session hardening | | Prefill | Carry a library alert (query, trigger, threshold, severity) into the alert editor | ### Design decisions worth knowing - **Rail navigates, strip filters.** Availability is deliberately not in the rail — carrying both overflowed it and pushed a facet group below the fold. - **Colour marks the exception only.** No "Ready" pill (it would describe ~75 of 86 cards and signal nothing); only unusable cards carry a label. - **A failed `/streams` call leaves readiness unknown**, rather than claiming "none of your data matches" — that claim would be about the request, not the org, and it is the most alarming thing this page can say. Counts read `—` until there is an answer, and a failed *refresh* keeps the verdict it already had. - **No "Installed" tile** — installed detection doesn't exist yet, and a tile reading a permanent zero is dishonest UI. - **`severity: warning` renders as `medium`** — the badge group's scale is critical/high/medium/low/info, and medium is the amber it also gives P3, which is exactly the priority a warning alert installs as. - Pack and category names come from the manifest, so they render as `raw()` data — a pack added upstream needs no frontend release. ### Design-system changes (used by the library, applied app-wide) - **`warning-quiet` badge tone** — one variant whose values differ by theme: the soft amber wash in light, a neutral chip in dark. Neither the filled amber chip nor the neutral one was right in both, because the two ramps aren't symmetric. - **Border tokens now meet WCAG 2.1 SC 1.4.11.** Every border token measured under 3:1 (`border-strong` was 2.52:1). `border-default` firms to grey-300 and controls get a token that clears 3:1. - Nav: Reliability's flyout groups its four alerting entries under one **Alerts** header, declared in the same order as the tab strip those pages carry. Flyout levels are sized by depth. - `sensors-off` icon added to the `OIcon` registry. ### Fixes that rode along - **metrics**: a shared URL whose `metrics_data` blob omits panel query fields no longer blanks the whole metrics page — blob queries are filled out to the panel schema before being laid over. - **dashboards**: a PromQL series with no legend template was named `JSON.stringify(metric)`, so fifty series read identically for the width of the chip. Series are now named by the labels that actually differ, preferring the ones that say *which workload* it is (namespace / service / pod / host). Correlation's semantic groups refine that when loaded, but nothing is fetched to render a chart — that endpoint is enterprise-only and a chart must render the same on OSS. - **alerts**: preview evaluates against the alert's real trigger; deleting a row no longer reloads the list behind it; peer tabs no longer shift when switching alerting pages. - **ui**: badge-only select options no longer render bold. ### Tests - `test(search)`: pins the streaming-aggregate **wire contract** at the HTTP boundary. Under streaming aggregates each partition carries the cumulative merged state (last frame is the answer) while ordinary partitions are disjoint and must be summed. No pytest had ever received `streaming_aggs: true` — the emitter could have flipped to per-partition deltas with the whole suite green while dashboards, Logs and the alert preview all silently over-counted. - `test(e2e)`: four helpers that passed only while an org stayed small — an assertion reading past a 20-row page, an unfiltered `OSelect` that virtualises past 50 entries, a tab click landing on the rename pencil, and two builder tests asserting on whichever metric sorted first. - New unit specs across the library components, composable, facets, tunables and install path. ### Notes - **i18n**: en-US only; CI generates the other 14 locales. - **No backend changes** — this branch is entirely `web/` plus test helpers. |
||
 Vamsi Karthik R
|
99b1a23773 |
test(slo): E2E + API suite for SLOs (#13836)
Adds automated test coverage for **SLOs** (#13547, #13577, #13579, #13704, #13761, #13784), which shipped with none. **149 tests** — 77 Playwright + 72 pytest — all three SLI types, both query languages, happy / negative / edge paths. Clean branch off `main`. **UI (Playwright · `tests/ui-testing`)** - `pages/sloPages/` — 4 page objects (verified `data-test` selectors, hydration + Monaco-sync waits, OSelect commit-retry, virtualized-list filtering). - `playwright-tests/SLO/slo-crud.spec.js` (24) — create count/time-slice/grouped through the form, detail, edit, SLI-type change, tags, pause, delete, filters; negative: dismissed delete, empty search, cancel, unmeasured SLI is absent not zero. - `playwright-tests/SLO/slo-form-validation.spec.js` (20) — server errors surfacing in the form, conditional fields per SLI type, count + time-slice previews, **PromQL count and time-slice created end to end**; negative: no stream, invalid aggregate, over-long name, target boundaries. - `playwright-tests/SLO/slo-alert-sli.spec.js` (8) — **the `alert` SLI type**: source picker, create, config carries only the alert id, detail names the source; negative: ineligible source offered-but-disabled, grouped alert SLI, slice finer than source cadence. - `playwright-tests/SLO/slo-timeslice.spec.js` (10) — **#13761 P0 regression** (measured SLI, not the preview), PromQL, previews, gaps, language-flip clearing. - `playwright-tests/SLO/slo-alerts.spec.js` (15) — burn-rate + error-budget kinds, presets, edit, freeze door, burndown; negative: no destination, cancel, blank/invalid name. - `playwright-tests/utils/slo-seed.js` — backdated seeding, metrics seeding, eligible-source-alert fixture, ground-truth query, fail-fast env guard, transient-store retry, full fixture teardown (alerts → streams → destinations → templates). **API (pytest · `tests/api-testing`)** - `tests/slos/test_slo_validation.py` (72) — full REST surface (create/get/update/enable/delete/groups/slo-eligible), validation on **both create and update** (name, target, window, slice, config shapes, comparators, float-threshold round-trip), boundaries (target 0.001/99.999, name 256 vs 257, threshold 0/negative/1e12/NaN, unicode names), alert-source eligibility (silence-gated, too-infrequent, cron), PromQL config shapes, auth 401/404. **CI** - Two shards: `SLO` (`workers: 3`) and `SLO-Measurement` (`workers: 1`). Split because the backfill job runs with `ZO_SCHEDULER_SLO_BACKFILL_CONCURRENCY=1`, so measurement specs sharing an instance starve each other's waits — `fullyParallel` races spec *files*, which `mode:'serial'` cannot prevent. The 3-worker pin is SQLite: at 5, concurrent SLO/alert/destination writes trip `database is locked` on the single-node meta store. - New optional per-shard `ingest_allowed_upto` (`240`) alongside the existing `quick_mode_enabled`: the workflow default of 5 **hours** silently drops the 8 days of history an SLO window needs, while still answering `200`. Other shards keep the default. Same precedent as `api-testing.yml`'s `48`. **Feature code**: 9 additive `data-test`/`dataTest` attributes in `SloDetail.vue`, `SloList.vue`, `SloAlertCondition.vue` — no logic changes, `vue-tsc` clean. The stat tiles, tabs and burn-window inputs had no stable selectors; `OStatStrip`/`OTab` already forward the attribute. **Behaviour**: self-cleaning (`e2e_slo_*`, worker-scoped since `afterAll` runs per worker). **Verified in CI** on this branch: `e2e / SLO` **52 passed** (1.8m, workers pinned to 3), `e2e / SLO-Measurement` **25 passed** (11.3m, workers pinned to 1) — the measurement shard passing confirms the per-shard `ingest_allowed_upto` took effect, since those tests are impossible under the 5-hour default. pytest **72 passed** locally; in CI's 599-test run one setup delete hit a transient 500 (see findings) and now retries. **Findings for the feature owner** (reported, not fixed here): - A blank "good when" shows `Request failed with status code 422` — `pruned()` drops the empty key, so the API's real 400 message never reaches `save()`. - A rejected alert name shows no banner: `SloAlertForm.submit()` returns on `nameError` before setting `saveError`. - `avg(latency_ms); DROP TABLE users--` is accepted at create time (parsed only at measurement). - The name-length message says "less than 256 characters" but 256 is accepted — 257 is the first rejection. - Duplicate alert names on one SLO are accepted (duplicate SLO names are a 409), and the panel lists them by name, so they are indistinguishable. - SLO names accept characters that break the list's own `slos-slolist-edit-${name}` selectors; alert names correctly reject them. - All chart/preview root `data-test` ids are unreachable — a parent's fallthrough attribute overrides the child root's. - Both edit forms hydrate asynchronously, so a blank form can flash before the stored values appear. - `DELETE /slos/{id}` on a freshly-created SLO returned **500** once under CI load (not reproducible locally in 12 rapid iterations). Contention is arguably a retryable 503, and the body carried no actionable message. - **A scheduled alert whose source stream is deleted is retried forever** (`Stream "…" not found in schema`), and the SLO backfill lane shares the scheduler's worker budget — so accumulated dead alerts *starve SLO measurement*. Observed: 65 of them held coverage at 1 of 7 chunks indefinitely; deleting them restored 0.143 → 1.0 in 240s. Harmless in CI (fresh instance per shard) but it degrades any long-lived environment. The suite now cleans up after itself; the retry-forever behaviour itself may deserve a look. **Paired ENT PR: openobserve/o2-enterprise#2434.** ENT's matrix is this repo's manifest plus an overlay, merged by this repo's `build-ci-matrix.js`, so both SLO shards reach ENT automatically — verified: 31 merged shards, both SLO entries carrying `ingest_allowed_upto=240` and their worker pins, no other shard affected. What ENT was missing is *consuming* those two fields, which #2434 adds. Merge the two together: this one alone leaves ENT booting the SLO shards on the 5-hour ingest window, and #2434 alone is inert. *(An earlier review reported ENT as using an inline matrix with no SLO shard — that turned out not to be the case.)* |
||
 ktx-akshay
|
33c6e9e0e1 |
test(dashboards): cover CTE/subquery match_all queries through the panel search path (#13838)
## What First test coverage for the dashboard **panel** endpoints and the **dashboard search path**, running CTE/subquery `match_all` queries as real dashboard panels. Nothing in the repo previously touched either: `grep -rn "/panels\|search_type=dashboards" tests/api-testing/` returned zero hits. ## Why Panels query `_search?type=logs&search_type=dashboards&dashboard_id=…` — a different route into the engine than the Logs page (`search_type=ui`). The `match_all` + FST projection rewrite fixed in #13808 runs inside it, so a query that passes via Logs can still regress on a dashboard. The rewrite has to preserve the projection of the block the predicate sits in. When that block is the whole statement, the projection is obvious. When it's a CTE body feeding a JOIN, a window, or a four-branch UNION ALL, it has to preserve the *CTE's* output schema while the outer query reads different columns from it — which is where it went wrong. ## Coverage **71 distinct queries across 29 combination families**, all with `match_all` inside a `WITH …` body or a derived table: | Axis | Variants | |---|---| | Nesting depth | 1, 2, 3, and 5-level CTE chains | | Nesting kind | CTEs and derived tables (and one query using both) | | UNION ALL | 2, 3, and 4 separate `match_all` branches | | JOIN | CTE⋈CTE, derived⋈derived, cross-stream | | Window | `ROW_NUMBER`, `RANK`, `DENSE_RANK`, partitioned aggregates | | Negation | `NOT match_all`, and mixed with positive `match_all` | | Combined with | `COUNT(DISTINCT)`, `CASE`, `HAVING`, `histogram()` | Each case creates a real panel, runs it via the dashboard search path, then asserts row count against the DuckDB oracle and that no expected column was dropped from the projection. A further 30 cases run the same SQL through **both** Logs and dashboard and diff the results — that's what catches a rewrite firing on only one path. Queries are reused from the existing query-agent corpus rather than duplicated, so the two suites can't drift: query-agent verifies the values via the Logs path, this verifies the same SQL survives the dashboard path. Cases are **discovered by classifying the SQL**, not from a hand-written ID list, so new combinations added to the corpus are picked up automatically. Test ids name the shape (`Q1022-cte1_union_cdist_4ma`), so a failure identifies the combination without opening the corpus. Plus 14 tests for the panel endpoints themselves — AddPanel / UpdatePanel / DeletePanel, contract + CRUD lifecycle + 7 error paths derived from `DashboardError` in `src/core/src/http.rs`. ## Changes | File | | |---|---| | `tests/dashboards/test_dashboard_cte_panels.py` | new — 103 tests | | `tests/dashboards/test_dashboard_panels.py` | new — 14 tests | | `tests/dashboards/conftest.py` | new — shared fixtures | | `support/panel_queries.py` | new — discovery + panel helpers | | `support/endpoints/dashboards.py` | +78, additive | | `support/endpoints/search.py` | +36, additive | | `support/factories.py` | +73, additive | The three shared-file edits are **187 added lines, 0 modified** — nothing existing can break. ## Results 168 tests in tests/api-testing/tests/dashboards/ (117 new + 51 pre-existing) 166 passed, 1 skipped, 1 xfailed # full nested-query corpus present 74 passed, 2 skipped, 1 xfailed # current corpus on main Repeated 2× and 4× respectively, no flakes. Discovery scales with the corpus: on main it finds 8 nested `match_all` cases, and 71 once the larger sentinel set is present. Cases that aren't in the corpus skip with an explanatory message instead of failing, so this is safe to merge in either order. |
||
|
Hengfei Yang
|
41ff3cd9b6 |
feat: replace distinct_values_* streams with tantivy TopK for the _values API (#13834)
Design at: #13822 ## Summary `distinct_values_*` metadata streams existed to serve one reader: the `_values` API. They are a redundant — and lossier — materialization of what the tantivy secondary index already holds, and the index has the collectors needed to replace them outright (`SimpleTopN` / `SimpleDistinct`). This PR removes the distinct streams from the write and read paths and lets the `_values` API be served by the tantivy index optimizer. ### 1. Treat distinct value fields as secondary index fields (read-time union) - `get_stream_setting_index_fields()` — the single accessor used by both the index build side (ingester parquet/pack jobs, compactor merge) and the query side (leader/follower index optimizers) — now unions `distinct_value_fields` into the effective index field set, gated on `enable_distinct_fields`. Fields that cannot be secondary-indexed (full text search keys, partition keys, reserved columns) are skipped; the values API falls back to a normal scan for those. - The distinct list is deliberately **not merged into `settings.index_fields`**: the two stay independent, so removing a distinct value field (or disabling `enable_distinct_fields`) cleanly stops its indexing with no residue, and the user-managed `index_fields` list is never mutated behind their back. - Correctness gate: a distinct-only field is considered indexed since its `added_ts` (`get_stream_setting_index_updated_at_for_fields`), so parquet files older than that are served by DataFusion scan instead of the index (`split_file_list_by_time_range`). `added_ts` is set on add and reset when `enable_distinct_fields` is re-enabled (existing behavior). For streams configured before this release the fields only start being indexed at upgrade time, so `added_ts` is additionally clamped by a release-wide cutoff `/tantivy/distinct_fields/updated_at` — a lazy marker initialized to the first access after upgrade, following the existing `ttv_secondary_index_updated_at` pattern. No data migration and no `DB_SCHEMA_VERSION` bump needed. ### 2. `_values` API always queries the original stream - Removed `can_use_distinct_stream()` and the `distinct_values_*` / `SUM(count)` / `StreamType::Metadata` rewrite from `values_inner` (GET `/_values`, formerly `values_v1`) and `build_search_request_per_field` (POST `/_values_stream`). - The generated SQL shapes are exactly what the index optimizer detects: - `SELECT f AS zo_sql_key, COUNT(*) AS zo_sql_num FROM s GROUP BY zo_sql_key ORDER BY zo_sql_num DESC, zo_sql_key ASC` + limit → `SimpleTopN` - `no_count`: `SELECT f AS zo_sql_key FROM s GROUP BY zo_sql_key ORDER BY zo_sql_key ASC` + limit → `SimpleDistinct` - The `keyword` filter (`str_match_ignore_case`) is a valid index condition, so keyword-filtered values queries stay on the index path too. ### 3. Remove distinct values generation - Dropped the per-record distinct value collection in logs and traces ingestion and the whole `core::metadata` module (channel, mem-table, flush job, shutdown hook). - Removed the now-unused `DISTINCT_FIELDS` static and the `ZO_FEATURE_DISTINCT_EXTRA_FIELDS` / `ZO_DISTINCT_VALUES_INTERVAL` / `ZO_DISTINCT_VALUES_HOURLY` config knobs, including the `SUM(count)` merge rollup the last one gated — legacy `distinct_values_*` streams now compact as ordinary metadata streams. The default fields `DISTINCT_FIELDS` carried (`service_name`, `operation_name`) are already default secondary index fields via `SQL_SECONDARY_INDEX_SEARCH_FIELDS`, so traces `_values` queries on them remain index-served. - Renamed `values_v1` to `values_inner`: the `v2` variant was removed long ago (#5087) and the distinct stream switch inside `v1` is gone now too, so the version suffix was pure history. ### 4. Default fields after this change - `service_name`, `operation_name`: default secondary index fields (unchanged, via `SQL_SECONDARY_INDEX_SEARCH_FIELDS`). - `trace_id`: now also a built-in default **bloom filter** field (`BLOOM_FILTER_DEFAULT_FIELDS`, gated on `ZO_FEATURE_DEFAULT_INDEX_FIELDS_ENABLED`), in addition to being a default index field — bloom sits on top of the secondary index and prunes files for high-cardinality equality lookups. - There is no "default distinct fields" concept anymore. ### 5. Fix surfaced by e2e: SimpleDistinct returned null values as `""` The tantivy index encodes null/missing field values as empty strings (`index_builder` adds `""` for null docs), so the `SimpleDistinct` term-dictionary walk returned `""` as a distinct value — sorted first in ascending order, it surfaced as a blank first option in dashboard variable dropdowns, which the DataFusion `GROUP BY` path never returns (its null group sorts last). This was exposed once distinct fields became index-served; `handle_simple_distinct` now skips empty terms (with a unit test). ### 6. Drive-by: fix the failing `api_fuzz_tests` CI job `CreateAlertRequestSchema` was an untagged documentation union with two identical variants, producing `oneOf: [CreateAlertRequestBody, CreateAlertRequestBody]` in the OpenAPI spec. Under `oneOf` semantics (exactly one branch must match) no instance can ever validate, so schemathesis filtered out every generated example and `api_fuzz_tests` failed on `POST /api/v2/{org_id}/alerts` on every CI run since that schema landed. The handler deserializes `CreateAlertRequestBody` directly, so the annotation now documents exactly that type; runtime behavior is unchanged. ## What stays (intentionally) - `distinct_value_fields` / `enable_distinct_fields` in stream settings: still the user-facing way to declare values-API fields; the list now drives the read-time index-field union described above. - The `distinct_value_fields` DB table that tracks dashboard/report usage of these fields (still gates removal in the settings API). - `DISTINCT_STREAM_PREFIX` helpers, retention sync and stream-deletion cleanup for existing `distinct_values_*` streams — legacy stream data keeps compacting (as ordinary metadata streams) and ages out via retention; no new records are written to them. ## Testing - `cargo check --workspace --all-targets` and `cargo clippy --workspace --all-targets` clean. - Enterprise build verified with `Cargo.toml.openobserve` (`cargo check --all-targets`). - New unit tests for the index-field union (skip rules for FTS/partition/reserved fields, `enable_distinct_fields` gating) and the `added_ts` / release-cutoff indexed-since resolution; schema, infra, config and api-search unit tests pass. A few pre-existing failures on a clean tree (config `ssrf_guard`, one flaky sqlite-backed destinations test, one `.env`-dependent index-fields test) are local-environment issues and unrelated. Closes #13822 |
||
|
Prabhat Sharmaandgithub-actions[bot]
|
0dba67d638 |
feat: composite alerts (#13797)
Design at : alerts/composite_alerts/composite_alerts.md ## Summary - add composite alert graph persistence, evaluation, scheduling, lifecycle APIs, and deletion safeguards - add composite alert UI forms, preview, references, list/detail integration, and localization - add Rust, API, Vue, and Playwright contracts ## Verification - cargo fmt --all -- --check - cargo check -p openobserve-core - cargo check -p openobserve-api-management - npm run type-check:composite-alerts - npm run type-check:app - npm run lint:ci -- --quiet - design/token/style checks - targeted Rust/Vitest suites - browser QA against local alerts UI ## Follow-ups (intentionally deferred, documented in the spec) - `RunOutcome::Error`/`NotifyFailed` outcome records on composite delivery failure (§11.5) - Incident detail live-definition resolver for composites + "definition unavailable" retention (§9.6) - `alert_composite_*` observability metrics (§16) - SLO/stream delete-conflict 409 bodies with readable-vs-hidden parent split (§10.3) - Super-cluster startup preflight (§18) <img width="1650" height="979" alt="image" src="https://github.com/user-attachments/assets/3def5246-7bf7-48fe-9a42-c0ac08271ea9" /> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Shrinath Rao
|
673bedb468 |
test(alerts): migrate multi-alert API specs from Playwright to pytest (Phase 2) (#13791)
## What **Phase 2** of the Playwright→pytest API-test migration (Phase 1 = Correlation, #13788 / o2-enterprise#2419). Ports the 4 **pure-API** Alerts 4.0 (multi-alert) specs into pytest and deletes the Playwright versions — they launched a browser only to assert the raw API. ## Changes — `tests/api-testing/tests/alerts/` | File | Ported from | |---|---| | `multialert_helpers.py` | `utils/alerts-api-helpers.js` (v1/v2 endpoints, canonical payloads, seed fixtures, ingest, scheduler-outcome polling) | | `conftest.py` | `alerts` fixture — seeds template + destination + stream, auto-cleans created alerts | | `test_multialert_api.py` | `alerts-multialert-api.spec.js` — 8 tests (save-time guardrails, round-trip type contract, B1 group-endpoint RBAC guard, tag/priority facets) | | `test_multialert_regression.py` | `alerts-multialert-regression.spec.js` — 4 tests (zero-behaviour-change guards) | | `test_multialert_firing.py` | `alerts-multialert-firing.spec.js` — 4 tests (fires at critical/warning, per-group fan-out, simple rollup) | | `test_multialert_transitions.py` | `alerts-multialert-transitions.spec.js` — 1 test (Ok→Warning→Critical history) | **17 tests, 1:1 port.** Deletes the 4 API specs and trims them from `ci_matrix.json`'s Alerts `run_files`. Keeps `alerts-multialert-ui.spec.js` (UI) and `alerts-api-helpers.js` (still used by the UI + priority-tags specs). ## No workflow change needed Unlike the Correlation pilot (which needed a dedicated server env), the main `api_integration_tests` job **already** runs `tests/alerts/`, and its server env **already** sets `ZO_ALERT_SCHEDULE_INTERVAL: 3` — the exact scheduler cadence the firing/transitions tests need. So these tests are picked up automatically with no CI edits. ## Verified locally Full suite against a fresh single-node enterprise build: **17/17 passed** (~52s), including the timing-dependent firing (critical/warning/per-group/rollup) and level-transition history tests. |
||
|
|
6d4eb704ed |
fix: allow testing workflow without saving (#13537)
This PR adds a fix so that workflows can be tested without having to save. --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
|
7d212f4621 |
feat: workflows incident connection (#13472)
This connects incidents to workflows, so any lifecycle event of incident will trigger workflow. design at: https://github.com/openobserve/designs/blob/main/infrastructure/workflows/design.md --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
 Bhargav
|
e9596a10a3 |
feat: Mobile rum sdk v2 (#13503)
Design at: design/mobile-rum-sdk-evaluation/design-docs Backend changes exist to make the OpenObserve RUM intake accept the mobile SDK wire format alongside the existing browser format. src/api/src/handler/http/request/rum/ingest.rs 200 → 202 Accepted on /rum, /logs, /replay | ---> OpenObserve-family SDKs (which the mobile forks derive from) treat any non-202 as an unexpected error and drop the batch. #[serde(default)] on Event ---> Mobile replay omits creation_reason / index_in_view that the browser format sends. Multi-segment replay handling ---> Browser sends 1 segment + 1 event object; mobile sends N segment parts + one event JSON array. Handler now collects all segments, parses event as array-or-object, pairs them, emits NDJSON → one _sessionreplay row per segment. MAX_SEGMENTS=200, 16 MB decompressed cap via .take() ---> Decompression-bomb guard, newly needed now that N segments are accepted. |
||
|
442ca09688 |
test(functions): un-skip JS-in-all-orgs tests + add per-event runtime-error coverage (#13341)
Replaces the `skip` band-aids that PR #13156 ("workflows v1") left on the **Functions** tests with real, running coverage of the current behaviour — on the correct edition — plus new tests for behaviour #13156 introduced but never covered. Test files only, no feature-code churn. Clean branch off `main`. ## Background — what #13156 actually changed #13156 bundled a Functions backend change unrelated to Workflows, and the old tests still asserted the *pre*-#13156 rules, so they were silenced with `pytest.skip(...)` / `test.skip(...)`. Two rules changed: 1. **JS is no longer `_meta`-only at the backend.** `save_function` / `test_run_function` have **no org gate on any edition** — JS (`transType:1`) is accepted in every org, including `default`, on **both OSS and Enterprise**. The `_meta`-only rule now lives **only in the frontend** (`AddFunction.vue` gates the JS radio on `isEnterprise || isCloud || org === "_meta"`, baked at build time by `VITE_OPENOBSERVE_ENTERPRISE`). 2. **The compile-check reports only *syntax* errors, not *runtime* errors.** A syntax error still returns **400**; code that compiles but throws at runtime now returns **200** with the error surfaced **per-event** in `results[].message` — for **both VRL and JS**. Because the API tests hit the backend directly, there is **no edition difference to assert for them** — the earlier "OSS asserts 400 / ENT asserts 200" framing was incorrect. Only the **e2e** (frontend JS-radio) is genuinely edition-dependent. ## API tests (pytest · `tests/api-testing`) **`test_functions.py`** - Keep the compile-error → 400 test (`test_test_endpoint_with_invalid_vrl_returns_400`) — it was never actually failing; `test`/`=====` are genuine VRL *compile* errors (`undefined variable` / syntax) and still 400. - Add `test_test_endpoint_with_js_runtime_error_returns_200` and `test_test_endpoint_with_vrl_runtime_error_returns_200` — runtime errors now return 200 with a per-event `results[].message`. **`test_functions_meta_org_restriction.py`** - Flip the 3 previously-skipped "blocked in default" tests → assert **200** (JS allowed in every org): `..._blocked` → `..._allowed`. - Repurpose `TestErrorMessages` (its old restriction-message assertions no longer exist) into per-event runtime-error coverage: JS `ReferenceError` content, VRL + JS surfacing consistency, partial-failure (only the failing event carries a message), and a syntax-error → 400 regression guard. - Update the class/module docstrings to the real contract. **`test_js_enrichment.py`** - Flip `test_js_function_in_default_org_returns_400` → `test_js_function_in_default_org_allowed` (200, transform runs). ## e2e (Playwright · `tests/ui-testing/Functions/js-transform-type.spec.js`) The JS-radio rule *is* edition-dependent. Instead of a skip (or an env-var hack), the two tests now detect edition at runtime via `EditionFeaturesPage.detectEdition()` — which reads the header edition button, driven by the **same** build flag that gates the JS radio, so it's the exact source of truth and needs **no CI change**: - **OSS build** → assert JS radio **hidden** in a non-meta org. - **Enterprise / cloud build** → assert JS radio **visible** in every org. The edition-agnostic steps that were trapped inside the skipped tests now **always run**: VRL-function create in a non-meta org (a P0 smoke path), and JS-visible in `_meta`. Added `P1: VRL runtime error surfaces in error-details` (mirrors the existing JS `ReferenceError` case; covers the VRL half of the per-event-error change). ## Skips removed (before → after) | Old (skipped) | New (runs & passes) | |---|---| | `test_create_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_test_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_update_vrl_to_js_in_default_org_blocked` | `..._default_org_allowed` | | `test_error_message_content` / `_consistency` | per-event runtime-error tests | | `test_js_function_in_default_org_returns_400` (enrichment) | `..._default_org_allowed` | | e2e `P0: Non-meta org - JS hidden, VRL works` | `P0: Non-meta org - JS radio edition-gated, VRL works` | | e2e `P1: Radio visibility across orgs` (skipped) | same name, edition-aware, no `test.skip` | `grep -rn "JS fns are now allowed" tests/` and `grep -rn "JS is allowed in all orgs on enterprise" tests/` → **both empty**. (The 7 remaining pytest skips are the unrelated `JS_HELPER_UNAVAILABLE` enrichment-helper cases — a separate pre-existing limitation, not part of this change.) ## Verification (local, against current `main` source) Built the current-`main` OSS binary and ran everything live: - **API:** `32 passed`, `0` target skips (7 unrelated `JS_HELPER_UNAVAILABLE` skips remain). - **e2e OSS build:** `7/7` — `detectEdition=opensource`, JS hidden in non-meta. - **e2e Enterprise-frontend build:** `3/3` edition tests — `detectEdition=enterprise`, JS visible everywhere, VRL runtime error renders in error-details. All backend behaviour was confirmed by probing the live `/functions` + `/functions/test` endpoints, not assumed. ## What CI proves, and where - **This PR (openobserve):** `api-testing.yml` runs the API tests; `playwright.yml` (`Functions` shard) runs the e2e on the **OSS** edition (JS-hidden branch). - **Enterprise edition:** dispatch `o2-enterprise` → `playwright.yml` (workflow_dispatch) with `o2_opensource_repo = test/functions-skipped-tests-13318` to run the same specs on the **enterprise** build (JS-visible branch). Pairs with the ENT Functions work in #13318 / o2-enterprise#2217. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
Shrinath RaoandClaude Opus 4.8
|
b1604cc7b6 |
test(workflows): e2e + API tests for Workflows (#13318)
Adds automated test coverage for the Enterprise-only **Workflows** feature (now merged to main). Clean branch off `main` — test files only, no feature-code churn. **UI (Playwright · `tests/ui-testing`)** - `pages/workflowsPages/workflowsPage.js` — page object (verified `data-test` selectors, Save-tooltip JS-click workaround, slow-list waits, enterprise-availability probe). - `playwright-tests/Workflows/workflows-crud.spec.js` — build/name validation, list + search. - `playwright-tests/Workflows/workflows-negative.spec.js` — negative + NFR sweep (long/unicode/emoji names, list latency, console/network hygiene). - `pages/page-manager.js`, `pages/apiCleanup.js` (`cleanupWorkflows()` + cascade), `playwright-tests/cleanup.spec.js` (`wf_auto_*` prefixes). - `.github/workflows/playwright.yml` — `Workflows` testfolder in the matrix. **API (pytest · `tests/api-testing`)** - `tests/workflows/test_workflows.py` — full REST surface (create/list/update/enable/test/history/errors/retry/delete) + validation, auth (401), cross-org. **Behaviour**: Enterprise-gated (skips on OSS where the feature is off); self-cleaning (`wf_auto_*`). Verified live: **11 Playwright + 13 pytest passing**. Paired with the ENT matrix PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
fd85566742 |
feat: workflows version 1 (#13156)
This PR introduces workflows design at: https://github.com/openobserve/designs/blob/main/infrastructure/workflows/design.md --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Neha PandClaude Opus 4.7
|
f202a0c94f |
test: repro vortex utf8→i64 cast pushdown bug (o2-enterprise#2179) (#13210)
## Summary - Adds `TestVortexUtf8Int64CastPushdown` in [tests/api-testing/tests/query_agent/test_3_vortex.py](tests/api-testing/tests/query_agent/test_3_vortex.py) — regression coverage for the bug fixed by [o2-enterprise#2179](https://github.com/openobserve/o2-enterprise/pull/2179). - The fixture ingests a batch as **i64** and then a batch as **utf8** on the same field into a fresh stream, flushes to force vortex file generation, then runs `SELECT * FROM {stream} WHERE value > 0`. - The bug: DataFusion coerces the predicate to `CAST(value AS Int64) > 0`; vortex used to accept the cast for pushdown but had no `utf8→i64` cast kernel, so execution failed with `No CastReduce to cast constant array from utf8? to i64?` / `No CastKernel …`. - Assertion: status 200 + empty `partial_error` — the query must not 5xx or partially fail. ## Where this runs - Runs in the enterprise `api_query_agent_vortex` job (server booted with `ZO_FILE_FORMAT=vortex`). - Not enabled in OSS `api-testing.yml` (test file is enterprise-only per the file header). ## Test plan - [ ] `api_query_agent_vortex` on the paired ent PR passes - [ ] Ent PR (same branch name) needed to fire ent's `api-testing.yml` — the OSS `ent-build-status-check` only runs `cargo check/test`, not the Python vortex suite 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Neha PandClaude Opus 4.7
|
9af9a47fea |
test: add error path, explain analyze, and concurrency stress tests for query agent (#12809)
## Summary - **Phase 4 — Error Paths** (`test_4_error_paths.py`): 28 cases across 8 categories that send intentionally invalid SQL and assert 4xx not 5xx. Self-contained, runs in ~24s without data dependency. - **Phase 5 — Concurrency Stress** (`test_5_concurrency.py`): 26 queries (2 per category) run simultaneously at 5/10/15 worker levels via ThreadPoolExecutor. Each thread gets its own client. Catches deadlocks and thread-safety bugs. - **EXPLAIN ANALYZE** (`test_explain_analyze.py`): Verifies EXPLAIN/EXPLAIN ANALYZE return 200 with plan output and no `partial_error`. Also validates EXPLAIN ANALYZE on cross-schema dot-notation queries returns 4xx (not 500 crash). - **Fix**: `load_all_queries()` now skips JSON files without a `"queries"` key. ## Test plan - [x] Error path tests pass on pentest (28/28, all return 400) - [ ] CI run: `rye run pytest tests/query_agent/test_4_error_paths.py -v` - [ ] CI run: `rye run pytest tests/query_agent/test_5_concurrency.py -v` - [ ] CI run: `rye run pytest tests/query_agent/test_explain_analyze.py -v` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
BhargavandClaude Opus 4.8
|
2a3416ee35 |
fix: Rum automation (#13064)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Neha PandClaude Sonnet 4.6
|
afd3758972 |
test: add schema evolution stress tests — scenarios 26–59 (diverge-then-converge + 9 scenario groups) (#12888)
## What Extends `tests/api-testing/tests/vortex/test_schema_evolution.py` with **34 new tests** across 10 new classes covering all realistic field drift scenarios for petabyte-scale schema evolution. Grows the schema evolution suite from 13 scenarios (14–25) to **46 scenarios (14–59)**. ## The scenario (shared across all groups) Think of a post office logbook. In January every letter has *From, To, Weight*. In February the form changes — Weight is dropped, Stamp Color is added. In March they go back plus add more fields. ``` Batch 1 [fa, fb, fc] — initial deployment Batch 2 [fa, fb, fd] — update: fc dropped, fd introduced Batch 3 [fa, fb, fc, fd, fe] — redeployment: fc back, fd kept, fe added ``` Each batch is flushed to a separate on-disk file before the next is ingested, forcing the query planner to reconcile three distinct schemas — the exact code path touched by `perf: Batch schema evolution checks` (v0.80.0). ## Test classes and scenarios added | Class | Scenarios | What it checks | |---|---|---| | `TestSchemaDivergenceConvergence` | 26–28 | Nullability per batch, cross-batch aggregation, field coexistence isolation | | `TestNullHandling` | 29–31 | IS NULL / IS NOT NULL exhaustive for absent fields | | `TestAggregationsOnPartialFields` | 32–36 | COUNT, SUM, MIN/MAX, COUNT DISTINCT on partial fields | | `TestGroupByOrderByEvolved` | 37–40 | GROUP BY / ORDER BY when grouped field is absent in some batches | | `TestTypeChangeOnReappearance` | 41–44 | Field returns as different type; OO type-preservation behavior documented | | `TestMultipleDivergenceCycles` | 45–46 | Field disappears and reappears multiple times | | `TestComplexQueriesEvolved` | 47–51 | SELECT *, CASE WHEN, subquery, HAVING on evolved schemas | | `TestTimeRangeEvolved` | 52–55 | Time-range filters correctly isolate individual batch windows | | `TestFTSEvolved` | 56–58 | Full-text search across batches with absent fields | | `TestHighFieldCountDivergence` | 59 | Wide schema: 20-field batch diverging to 10-field batch | ## Key findings documented in tests - **OO string-null behavior**: absent string fields are stored as `''` not SQL NULL — correct check is `IS NOT NULL AND != ''`; numeric fields use `IS NOT NULL` only - **OO type preservation**: when a field returns as a different numeric type, OO preserves the original type on disk; test_42 documents this behavior explicitly ## Code review fixes included - Added missing `fb` assertion in test_26 - Removed incorrect `!= ''` check on numeric `latency_ms` field (tests 27, 32–35) - Added comment in test_28 explaining the OO string-null quirk - Updated module docstring to list all 9 scenario groups - Added docstrings to all `_ingest` / `_ingest_three_batches` helper methods ## Why The v0.80.0 commit changed how OpenObserve batches schema reconciliation across multiple on-disk files. A bug in that path can: - Silently return fewer records than expected (data loss with no error) - Return wrong nullability for fields that reappear after being absent - Mis-count aggregations over fields that span only some of the files None of the 13 existing tests covered the diverge-then-converge pattern. This PR closes that gap across all query types. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Neha PandClaude Opus 4.7
|
7d3e67903e |
ci: add single_node_opt matrix mode to query agent tests (#12756)
## Summary - Adds a matrix strategy to the query agent SQL regression suite jobs in both `api-testing.yml` and `query-agent.yml` - Each job now runs with `ZO_FEATURE_SINGLE_NODE_OPTIMIZE_ENABLED=false` and `ZO_FEATURE_SINGLE_NODE_OPTIMIZE_ENABLED=true` - Verifies both DataFusion physical optimizer paths produce identical query results - Matrix legs run in parallel with `fail-fast: false` - All artifact/log names include the matrix label to avoid collisions ## Test plan - [ ] PR CI shows two `api_query_agent` matrix legs (single_node_opt_false and single_node_opt_true), both pass - [ ] `/query-agent` PR comment triggers both modes and posts per-mode results comments 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Hengfei Yang
|
ad743e9017 |
fix: add ttv inspect (#12743)
```
./target/debug/openobserve ttv-inspect -f data/openobserve/stream/files/default/index/stream_0_logs/2026/06/16/01/747246720234867916913ad.ttv
file : data/openobserve/stream/files/default/index/stream_0_logs/2026/06/16/01/747246720234867916913ad.ttv
total_bytes : 21163306
blob_count : 6
── file properties ──────────────────────────
row_group_size : 131072
── blobs ────────────────────────────────────
# name type offset length comp
0 meta.json o2-ttv-v1 4 1650 none
1 c057f4e04d9840849c7cb2311344be18.term o2-ttv-v1 1654 7466195 none
2 c057f4e04d9840849c7cb2311344be18.fast o2-ttv-v1 7467849 9752452 none
3 c057f4e04d9840849c7cb2311344be18.idx o2-ttv-v1 17220301 3898370 none
4 c057f4e04d9840849c7cb2311344be18.pos o2-ttv-v1 21118671 123 none
5 footer_cache o2-ttv-footer-v1 21118794 43493 none
── tantivy index (meta.json) ────────────────
segments : 1
total_docs : 480000 (deleted: 0)
opstamp : 0
- c057f4e0-4d98-4084-9c7c-b2311344be18: max_doc=480000 deleted=0
schema fields : 5
ip text [indexed,fast, tokenizer=raw]
request_id text [indexed,fast, tokenizer=raw]
trace_id text [indexed,fast, tokenizer=raw]
server_name text [indexed,fast, tokenizer=raw]
_timestamp i64 [fast]
```
|
||
|
4f3c0fcba3 |
fix(tests): verify match_all FTS result correctness and enforce Tantivy readiness (#12717)
## Summary
Fixes a test gap where `match_all` Tantivy queries could return wrong
results after a DataFusion upgrade without being caught by existing
assertions.
### Problem
After a DataFusion upgrade merge, the DataFusion access plan was no
longer used for `match_all` queries, causing Tantivy to return wrong
results. The existing tests only compared row counts against a DuckDB
oracle that emulates `match_all` with `LIKE '%term%'` — so wrong rows
that still matched the LIKE pattern passed undetected.
Additionally, `node/flush` does not generate Tantivy indexes
immediately. The default `ZO_MAX_FILE_RETENTION_TIME=600s` means
WAL→parquet→Tantivy conversion can lag, so FTS queries may fall back to
a non-Tantivy path during tests.
### Changes
1. **Content verification**
([conftest.py](tests/api-testing/tests/query_agent/conftest.py)): Every
hit from a `match_all` query that returns the `log` field is now checked
— each search term word must be present in `log`. A hard assertion fails
if Tantivy returns rows that don't contain the search terms.
2. **Tantivy readiness gate**
([test_2_parquet.py](tests/api-testing/tests/query_agent/test_2_parquet.py)):
After `node/flush`, the test now polls with a `match_all('warehouse')
LIMIT 1` query (up to 120s timeout) to confirm Tantivy indexes are built
before running FTS assertions. Previously only COUNT(*) was checked.
3. **Multi-word match_all emulation**
([compute_counts.py](tests/test-data/query-agent/compute_counts.py)):
`match_all('ACK batch')` now translates to `(log LIKE '%ACK%' AND log
LIKE '%batch%')` — per-token AND conditions matching Tantivy's
token-level search semantics, instead of the old phrase match `log LIKE
'%ACK batch%'`.
4. **Regenerated expected results** across all query JSON files.
## Test plan
- [ ] Run query agent test suite against an environment with Tantivy
enabled
- [ ] Verify `match_all` content assertions catch intentionally
corrupted results
- [ ] Confirm post-flush Tantivy readiness poll works within the timeout
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Huaijin <haohuaijin@gmail.com>
|
||
|
Huaijin
|
0831badd02 |
perf: upgrade datafusion to 54 (#12665)
- [x] ~add sort when to many groups, we can do this, but for now do not do this~ - [x] remove the unsafe for memory pool - [x] add approx_distinct improve, already fork, will add in next pr - [x] fork vortex to add cast fix, and export `.exclude_schemes([IntDictScheme.id()])` - [x] ~enable_physical_uncorrelated_scalar_subquery test~ |
||
|
udev
|
6761ce2dbf |
fix: refactor online evals (#12654)
move online evals to enterprise repo |
||
|
Hengfei Yang
|
9c82beeeed |
fix: OTLP/JSON metrics silently dropping histograms and int values (#12615)
Close #12345 ## Problem OTLP histogram metrics sent over HTTP/JSON (e.g. otel-collector with `encoding: json`) created streams but stored 0 events, with no `_count`/`_sum`/`_bucket` streams. Int-valued counters/gauges were silently recorded as 0. The protobuf path was unaffected, which is why this only surfaced for collectors explicitly configured with JSON encoding. ## Root cause `otlp_json` deserializes the body directly into opentelemetry-proto's serde-derived types, which only partially implement the proto3 JSON mapping (64-bit integers encoded as strings, enums by name, zero/empty fields omitted). `Metric.data` is a flattened `Option` oneof, and serde converts any inner deserialization error of a flattened `Option` into `None` instead of failing — so the whole metric silently became `data: None`, while the stream itself had already been created by the schema setup. Same mechanism on `NumberDataPoint.value` turned string `asInt` values into 0. ## Changes - Upgrade opentelemetry stack 0.31 → 0.32, which fixes string-encoded u64 handling for plain histograms and summaries upstream. - Add `otlp_json_compat` normalizer for the gaps still present in 0.32 (each verified empirically, with unit tests): - string `asInt` values on data points and exemplars - `aggregationTemporality` as enum name - `Exemplar`'s value oneof is not flattened in 0.32 - missing `serde(default)` on `ExponentialHistogramDataPoint`/`Buckets`/`ValueAtQuantile`/`Exemplar`, which rejected omitted zero/empty fields - Count and log metrics whose data oneof failed to decode (`partial_success.rejected_data_points`) instead of silently skipping them, and stop overwriting specific partial-success error messages with the generic retention message. - Adapt to OTLP 0.32 API additions (`KeyValue.key_strindex`, `AnyValue::StringValueStrindex`). - Upgrade workspace reqwest 0.12 → 0.13, required because opentelemetry-http 0.32 only implements `HttpClient` for reqwest 0.13 (`rustls` feature replaces `rustls-tls-native-roots` via the platform verifier; `json`/`query`/`form` are now explicit features). The `segment` crate is pinned to reqwest 0.12, so `TELEMETRY_CLIENT` moved into `telemetry.rs` as a plain `reqwest::Client` that posts the Track message itself, replicating `segment::HttpClient::send`. - Remove the unused pyroscope profiling integration (optional feature, dependencies, agent setup, `ZO_PROF_PYROSCOPE_*` config). ## Verification - 10 unit tests in `otlp_json_compat` covering the full payload matrix (collector-style string integers, enum names, omitted zero fields, exemplars with and without trace context, already-normalized payloads, malformed input). - End-to-end: posted the issue's collector-style JSON payload to `/api/default/v1/metrics` — `_bucket` (3 events, correct cumulative counts with exemplars attached), `_count`, `_sum` populated for histogram and exponential histogram, summary quantiles stored, `asInt` counter stored as 42 instead of 0. - `cargo clippy -- -D warnings` clean; lib test failures are pre-existing local-environment issues that reproduce identically on main. |
||
|
Sai Nikhilandudev
|
57e57a56c9 |
fix: import and export functionality for score and score configs in llm evals (#12517)
Co-authored-by: udev <uddhav@openobserve.ai> |
||
|
Neha PandClaude Sonnet 4.6
|
146d5e7612 |
feat(tests): vortex file format test suite (scenarios 1–83) (#12472)
## Summary - Adds **239 new tests** covering the vortex columnar file format across 78 scenarios from the test plan - **`tests/api-testing/tests/vortex/`** — 9 test modules (83 tests): format conversion, schema evolution, data integrity, FTS, downsampling, API format awareness, server config, and performance/stress - **`tests/api-testing/tests/query_agent/test_3_vortex.py`** — 156 tests: the full 155-query SQL correctness suite + scenario 39 (self-join) run post-flush against vortex files - **`api-testing.yml`** — adds `--ignore=tests/vortex` to the OSS default-set run; vortex tests require the enterprise binary with `ZO_FILE_FORMAT=vortex` and are gated to enterprise CI only ## Test plan coverage | Module | Scenarios | |--------|-----------| | `test_format_conversion.py` | 1–13, 7, 8, 9, 10 | | `test_schema_evolution.py` | 14–25 | | `test_data_integrity.py` | 55–62 | | `test_fts.py` | 51–54 | | `test_downsampling.py` | 47–50 | | `test_api_format.py` | 71–73 | | `test_config.py` | 66–67 | | `test_performance.py` | 74–78 | | `test_3_vortex.py` | 26–46 (155 queries + JOIN) | ## Enterprise CI Two dedicated jobs added in the companion `openobserve/o2-enterprise` PR on the same branch: - **`api_query_agent_vortex`** — runs `test_3_vortex.py` with `ZO_FILE_FORMAT=vortex` - **`api_vortex_format_tests`** — runs the full `tests/vortex/` suite with 4-phase server lifecycle (parquet→vortex→parquet→vortex+compaction) OSS CI is unchanged except for the ignore rule — no vortex tests run against the OSS binary. ## Test checklist - [ ] `tests/vortex/` collects cleanly: `83 tests collected` - [ ] `tests/query_agent/test_3_vortex.py` collects cleanly: `156 tests collected` - [ ] Enterprise CI `api_query_agent_vortex` job passes - [ ] Enterprise CI `api_vortex_format_tests` job passes 🤖 Generated with [Claude Code](https://claude.ai/claude-code) --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Neha PandClaude Opus 4.7
|
008398163b |
test: add more queries to query regression (#12487)
## Summary - Adds **100 new structural-pattern queries** (Q411-Q510) across all 13 categories, bringing the suite from 425 to **525 queries** - Adds **cross-stream JOIN/UNION support**: `stream_offset` in data_gen.py, `STREAM2` in conftest, dual-table DuckDB oracle - 15 cross-stream queries (Q401-Q415) covering INNER/LEFT/RIGHT JOIN, EXISTS/NOT EXISTS, UNION ALL across streams - Bumps data_gen.py from 510 to 520 query capacity ## Changes - **Infrastructure**: [conftest.py](tests/api-testing/tests/query_agent/conftest.py), [compute_counts.py](tests/test-data/query-agent/compute_counts.py), [data_gen.py](tests/test-data/query-agent/data_gen.py) - **New file**: [cross_stream.json](tests/test-data/query-agent/queries/cross_stream.json) — 15 queries - **12 categories**: 85 queries across aggregation, basic_select, combined, cte_subquery, date_time, full_text_search, histogram, math_functions, pagination, string_functions, union, window ## Test plan - [x] `compute_counts.py` generates expected results for all 525 queries - [ ] Run `pytest tests/query_agent/ -v` against target environment (565 tests expected) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Neha PandClaude Opus 4.7
|
4d48bbb21b |
test: expand query agent regression suite (#12479)
## Summary - Expands the SQL regression catalog from **155 to 200 queries** across all 12 categories - Adds **7 queries** extracted from abc.sql (Q156-Q162) — synthesized from real-world OpenObserve SQL patterns with all data/fields/streams generated from scratch - Adds **230 auto-generated queries** (Q163-Q392) from templates to cover query format diversity - Both test phases pass: **200 memtable + 200 parquet = 400 total** ## Key changes - **data_gen.py**: 43 new fields (record_type, site_name, latency_ms, payload_size, alert_tags, phase_data, db_info, etc.), generates 3920 deterministic records - **compute_counts.py**: new translations for regexp_like, str_match_ignore_case, approx_distinct, json_get_str/json_get_json chaining; empty-aggregate result detection (OO returns 0 rows vs DuckDB's 1 NULL row) - **conftest.py**: PID-based stream name suffix for cross-invocation isolation (prevents data accumulation) ## Test plan - [x] test_1_memtable.py — 392 passed - [x] test_2_parquet.py — 392 passed - [x] All 7 abc.sql queries pass (Q156-Q162) - [x] No real stream names, field names, or data copied from external sources 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
0aab1ae261 |
feat: online evaluations (#12352)
- **feat(online-eval): add PipelineKind::Evaluation and dual realtime pipeline support** - **feat(online-eval): add provider, score_config, scorer, and online_eval_jobs DB entities** - **feat(online-eval): add config CRUD services and API handlers** - **feat(online-eval): reconcile jobs into hidden evaluation pipelines** - **feat(online-eval): add scorer preparation and execution backends** - **feat(online-eval): write scores and evaluator traces to system streams** - **refactor(online-eval): remove legacy eval templates backend** - **test(online-eval): add API integration coverage for eval resources** --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Neha PandClaude Sonnet 4.6
|
73ce75c883 |
fix(tests): bump query_agent ingest timeout 120s → 300s (#12444)
## Summary - ENT CI runners are slower than local/OSS runners — the `ingest_query_agent_data` fixture was hitting the 120s timeout before 1550 records became searchable, causing all 310 query_agent tests to fail at setup - Bumped `wait_until` timeout to 300s ## Test plan - [ ] OSS `api_query_agent_tests` passes as before - [ ] ENT `api_query_agent_tests` no longer hits WaitTimeout on ingest 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Neha PandClaude Opus 4.7
|
1196045636 |
test: add org-level ingestion token API and Playwright tests (#12439)
PR #11691 introduces org-level ingestion tokens with o2oi_ prefix (37 chars). This commit adds comprehensive automated test coverage: ## API Tests (35 tests in test_ingestion_tokens.py) ### Token CRUD (9 tests) - Create token returns o2oi_ prefix and 37-char length - Create token persists name, created_by, enabled=true, is_default=false - Create token with name only (no description) — description is empty - Create token with hyphens and underscores allowed - Duplicate name returns 400 with "already exists" message - Empty name returns 400 with "cannot be empty" message - Whitespace-only name returns 400 with "cannot be empty" message - Name too long (>256 chars) returns 400 with non-empty error message - Parametrized: special chars (spaces, @, ., /) rejected with "alphanumeric" message ### List Tokens (3 tests) - GET returns data array - Created token appears in list - Token values are unmasked in list response ### Enable/Disable (5 tests) - Disable token via PATCH {enabled: false} - Enable disabled token via PATCH {enabled: true} - Double disable is a no-op (idempotent) - Toggle nonexistent token returns 404 - DELETE returns 405 (not supported) ### Ingestion Auth — Org Token (6 tests) - Ingest with org token (org_id:o2oi_token) succeeds - Ingest with disabled org token fails (401) - Ingest with nonexistent o2oi_ token fails (401) - Ingest to wrong org with valid token fails (401) - Org token blocked on search endpoint (401) - Org token blocked on dashboards endpoint (401) ### Ingestion Auth — Backward Compat (6 tests) - User password ingestion still works (pre-PR behavior) - Non-o2oi_ service account token ingestion works (falls through to user lookup) - User password on search succeeds (200) — mirrors org token blocking tests - User password on dashboards succeeds (200) — mirrors org token blocking tests - Non-o2oi_ SA token NOT blocked by o2oi_ prefix check on search (≠401) - Non-o2oi_ SA token NOT blocked by o2oi_ prefix check on dashboards (≠401) ### Multi-token and Security (6 tests) - Create 5 tokens, all visible in list - Disable one token, others remain enabled - SQL injection in token name rejected (special chars → 400) - Empty name, whitespace name, too-long name (with message assertions) - Special chars parametrized: spaces, @, ., / ## Playwright UI Tests (7 tests in ingestionTokens.spec.js) - Ingestion Tokens tab renders with table and columns (P0) - Create token via UI dialog with name and description (P0) - Create token with empty name — button disabled, enabled on fill, disabled on clear - Revealed token dialog — copy token with clipboard verification - Toggle token enable/disable from table actions (P0) - Manage Tokens button on ingestion page navigates to IAM - Ingestion page token selector dropdown shows available tokens ## Page Object (ingestionTokensPage.js) - data-test selectors for IAM nav, token list, create dialog, toggle, toast, revealed dialog - Registered in centralized PageManager --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Shrinath Rao
|
a647e6aee9 |
test: add retry_on_db_lock helper to support/wait.py (#12434)
## Summary
- Adds `retry_on_db_lock()` to `tests/api-testing/support/wait.py`
- Companion to the enterprise-side fix (openobserve/o2-enterprise#TBD)
that applies the helper across all RBAC entity base tests
## Root cause
The `ent_rbac` CI shard runs with `-n 2` (two pytest-xdist workers).
SQLite's single-writer model means parallel workers racing on write
operations produce transient `500 {"message": "database is locked"}`
responses (SQLite error codes 5 / 517). These surface as flaky assertion
failures across many PRs.
## Fix
`retry_on_db_lock(fn)` wraps an HTTP call and retries up to 3 times with
exponential backoff (0.5 → 1 → 2 s) only when the response is `500` with
`"database is locked"` in the body. Parallelism is preserved; the brief
retry window absorbs the time the other writer needs to commit and
release the lock.
## Test plan
- [ ] OSS CI passes with this branch
- [ ] Enterprise PR with the same branch name picks up the new utility
and applies it in all RBAC entity base tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Neha PandClaude Opus 4.7
|
eb7cce9b96 |
test: query agent SQL regression suite — 155 queries, sqllogictest-style, two-phase (#12384)
## Summary
155-query SQL regression test suite covering 12 categories. Adding a new
query requires zero test code changes — drop a JSON entry into the right
category file and run `compute_counts.py`.
**Sqllogictest-style comparison**: DuckDB pre-computes full expected
result sets (not just row counts). The test runner extracts actual
values from OO hits, sorts both sides, and compares cell-by-cell with 5%
relative float tolerance. This catches value-level regressions that
row-count-only checks miss.
Two-phase execution: **memtable** (pre-flush) then **parquet**
(post-flush + Tantivy). **310/310 pass**.
## Architecture
```
tests/
├── api-testing/tests/query_agent/
│ ├── conftest.py # Shared fixtures, run_query(), tolerance comparison
│ ├── test_1_memtable.py # Phase 1: all queries pre-flush (FTS skipped)
│ └── test_2_parquet.py # Phase 2: post-flush (module-scoped fixture triggers flush)
└── test-data/query-agent/
├── data_gen.py # Deterministic data generator (single source of truth)
├── compute_counts.py # DuckDB oracle: computes expected result sets
├── base_ts_override.json # Locks BASE_TS so compute and tests share same timestamps
└── queries/ # 12 JSON files, 155 queries with expected results
```
### How it works
1. **Deterministic data** — `data_gen.py` generates 10 records per query
(1550 total) with a warehouse/supply-chain IoT schema. Each query Q{N}
gets isolated timestamps. `BASE_TS` is locked via
`base_ts_override.json` so compute and tests share identical timestamps
(prevents histogram bucket drift on minute boundaries).
2. **Unique stream per session** — `STREAM =
f"query_agent_test_{int(time.time())}"` prevents accumulated-data bleed
from prior runs.
3. **DuckDB oracle** — `compute_counts.py` loads the same dataset into
DuckDB, creates a pre-filtered VIEW per query (so time filters work for
CTEs, subqueries, UNIONs — no fragile SQL injection), translates
OO-specific functions, and writes `expected.results` blocks into the
query JSONs.
4. **Histogram translation** — `histogram(_timestamp, 'N unit')` is
translated to integer arithmetic matching OO's `date_bin` with origin at
2001-01-01T00:00:00 UTC. Verified against DataFusion CLI — 100/100
records match across all interval types (15s, 30s, 1m, 2m, 3m, 4m, 5m).
ISO timestamp post-processing converts epoch-µs bucket values to
`YYYY-MM-DDTHH:MM:SS` strings matching OO's serialization format.
5. **Sqllogictest comparison** — `run_query()` checks for
`expected.results`. When present, it extracts column values from hits as
strings, sorts both actual and expected (`results_mode: "rowsort"`), and
compares cell-by-cell with 5% relative float tolerance (handles
cross-engine floating-point drift in AVG/STDDEV/etc.).
6. **Two-phase testing** — `test_1_memtable.py` runs all queries
pre-flush (FTS sqllogictest skipped — Tantivy indexes don't exist yet).
`test_2_parquet.py` has a module-scoped `post_flush` fixture that
triggers `/node/flush`, re-polls until data is searchable, then runs all
queries with full assertions.
### Comparison modes
| Mode | Count | Mechanism |
|------|-------|-----------|
| **Sqllogictest** | 140 | Cell-by-cell comparison with 5% float
tolerance |
| **Skip sqllogictest** | 9 | DuckDB results for documentation, fall
back to legacy assertions |
| **Legacy** | 6 | 5 auto-interval histograms + Q122 (STRING_AGG
DISTINCT ORDER BY) |
**Skip sqllogictest queries** — known OO-DuckDB divergence:
| Query | Reason |
|-------|--------|
| Q037 | ROW_NUMBER tie-breaking differs between engines |
| Q058 | Self-join pairings non-deterministic across engines |
| Q072 | FULL OUTER JOIN histogram — NULL handling differs |
| Q080 | FTS `match_all` + histogram — Tantivy vs SQL LIKE |
| Q111 | ROW_NUMBER ties in pagination |
| Q117 | STRING_AGG ordering differs between engines |
| Q121 | STRING_AGG concatenation order |
| Q127 | UNION across 5 overlapping groups — boundary records compete
for LIMIT spots |
| Q152 | PERCENT_RANK tie-breaking differs between engines |
### Key design decisions
| Decision | Why |
|----------|-----|
| Full result-set comparison | Catches value-level regressions (e.g.,
ROUND producing wrong decimal) |
| `rowsort` mode | DuckDB and OO order ties differently — sorting both
sides eliminates spurious failures |
| 5% float tolerance | Cross-engine AVG/STDDEV/PERCENTILE differ at
least-significant digits |
| `''` = `'None'` normalization | OO serializes NULL as empty string;
DuckDB uses `"None"` |
| DuckDB VIEW for time filtering | WHERE injection into CTEs/subqueries
is fragile — VIEW filters at scan level |
| Dynamic columns excluded | `_timestamp` and `_time` differ between
compute time and test time |
| OO-dropped LAG columns excluded | `prev_tput`, `hours_2_back`,
`delta_from_prev`, `temp_diff` |
| Histogram → DuckDB integer math | DuckDB lacks `date_bin`; integer
formula verified via DataFusion CLI |
| BASE_TS override file | Prevents histogram timestamp drift when
compute and test run at different minutes |
| 1s time window margins | Minimizes cross-query data bleed while
capturing all 10 records |
| `pytest.fail()` on fixture failure | Stops all tests instead of
running 310 against an empty stream |
### Categories covered
| Category | Queries | Description |
|---|---|---|
| aggregation | Q001-Q020 | GROUP BY, COUNT, AVG, SUM, MIN, MAX, FILTER,
HAVING |
| basic_select | Q021-Q035 | WHERE, IN, BETWEEN, LIKE, ORDER BY,
DISTINCT, CASE |
| combined | Q036-Q045 | JOINs, multi-condition filters, combined
aggregations |
| cte_subquery | Q046-Q062 | CTEs, subqueries, derived tables, EXISTS |
| date_time | Q063-Q073 | Date parsing, formatting, extraction, interval
arithmetic, histogram |
| full_text_search | Q074-Q082 | match_all, match against fields |
| histogram | Q083-Q093 | histogram() bucketing, aggregation over
buckets |
| math_functions | Q094-Q102 | ROUND, SQRT, ABS, POWER, arithmetic
expressions |
| pagination | Q103-Q112 | LIMIT, OFFSET, combination with filters and
sorts |
| string_functions | Q113-Q122 | SUBSTR, CONCAT, UPPER, LOWER, TRIM,
LIKE patterns |
| union | Q123-Q134 | UNION ALL, UNION (dedup), CTE + UNION combinations
|
| window | Q135-Q155 | ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD, SUM/AVG
OVER |
## Test plan
- [x] All 155 queries pass in both phases (310/310)
- [x] 140 queries with sqllogictest cell-by-cell comparison
- [x] 17 histogram queries converted from legacy to sqllogictest
- [x] 9 queries with skip_sqllogictest for known OO-DuckDB divergence
- [x] 6 queries in legacy mode (auto-interval histograms + Q122)
- [x] BASE_TS locked via override file — no timestamp drift
- [x] Dedicated CI job (`api_query_agent_tests`) runs in parallel
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Hengfei YangandClaude Opus 4.8
|
ad9fef2242 |
fix: fuzzy tests (#12405)
- [x] Bulk ingestion panic - [x] Create function panic - [x] Dashboards/Alerts/Incidents panic when list data - [x] Match http response error code - [x] Skip some endpoint that fuzz can't generate data --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Shrinath RaoandClaude Opus 4.7
|
b0118090e1 |
test: api-testing follow-ups — pytest-xdist dep + standardize test user display name (#12393)
## Summary
Companion PR to o2-enterprise PR 1841 (api-testing follow-ups). All
ENT-side test/CI work lands there; this OSS PR carries the two changes
that must live on the OSS side because they touch shared code.
Both PRs live on the same-named branch (`test/api-test-followups`) so
ENT CI's cp-merge picks up this branch and sees both changes together.
## What's in this PR
### 1 — Add `pytest-xdist` dependency
Adds `pytest-xdist>=3.0` to `tests/api-testing/pyproject.toml`. Needed
by the ENT api-testing CI's `api_integration / ent_rbac` shard to run
RBAC tests in parallel (`-n 4`).
CI-verified speedup: **~70 s → 21.7 s (~69% reduction)** on the ENT RBAC
shard's 280+ tests, with zero failures.
Foundation only — no test code in this repo uses xdist yet. Available
for any future shard that wants it.
### 2 — Standardize automated-test user display name to "Automation
TestUser"
Two helper files in `tests/api-testing/helpers/workflow/pages/` had a
personal name (`"Shyam P"` / `"Shyam Panjiyar"`) hardcoded in their user
/ service-account create payloads. Those helpers are consumed by the
workflow chain (`helpers/workflow/create_objects.py` +
`edit_objects.py`), which doesn't reliably clean up its created users —
so the names persisted in the IAM users page and looked confusingly like
real teammates' accounts.
Files changed:
- `tests/api-testing/helpers/workflow/pages/user_page.py`
- `tests/api-testing/helpers/workflow/pages/serviceaccount_page.py`
Both now use:
```
first_name: "Automation"
last_name: "TestUser"
```
Convention going forward: any account named `Automation TestUser` in the
IAM users page is automatically identifiable as test-created and safe to
clean up. Email addresses still use random suffixes
(`d4m21_{random_int}`) so uniqueness isn't affected — only the display
name standardizes.
## Test plan
- [ ] CI green on this branch
- [ ] Companion ENT PR 1841 (which depends on the `pytest-xdist` dep
added here) stays green
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Shrinath RaoandClaude Opus 4.7
|
61aa772d57 |
test: revamp api-testing suite (framework + 16 file rewrites + CI matrix) (#12372)
## Summary
Comprehensive rework of the `tests/api-testing/` suite. 30+ commits
implementing all 6 phases of the [revamp
plan](../../tests/ui-testing/MD_Files/api%20tests%20revamp/API_TESTS_REVAMP.md).
**Before → After:**
| | Baseline | Final | Δ |
|---|---|---|---|
| passed | 421 | **443** | **+22 ✓** |
| failed | 3 | **2** | -1 (only same pre-existing) |
| skipped | 53 | 39 | -14 (unskipped dead tests now run) |
| xfailed | 1 | 2 | +1 (new visible backend-bug signal) |
| runtime | 588s | **387s** | **-201s (34% faster)** |
**Zero tests that passed in baseline are now failing.** The 2 remaining
failures are the exact pre-existing ones
(`test_enrichment_table_join.py` backend panic + `test_workflow.py` SMTP
env-miss).
## What's in the diff
- **New `support/` framework** — `OpenObserveClient`, endpoint wrappers
(`alerts`, `dashboards`, `streams`, `search`, etc.), `wait_until()`,
payload factories, pytest fixtures, SSE parser. 14 framework selfcheck
tests.
- **16 of 16 OLD-tier test files rewritten** — using the new framework,
unique-suffix names, body validation, `wait_until` instead of
`time.sleep`, parametrize where appropriate.
- **29 files moved into 12 feature folders** mirroring the UI suite
layout
(`tests/{alerts,dashboards,ingest,orgs,pipelines,regression,rum,search,streams,users,workflow}/`).
- **−6,500 LOC net** (copy-paste + dead code removal; new framework adds
back ~1,500).
- **Ruff config** in pyproject.toml: T201 ban on `print()`, PT pytest
rules, B bugbear, UP pyupgrade.
## CI workflow changes
- New job `api_regression_tests`: `strategy.matrix` over the 4 env-flag
combinations (UTF8_VIEW × JOIN_MATCH_ONE) — now runs in parallel instead
of as 4 sequential server reboots in the main job.
- New job `api_fuzz_tests`: schemathesis OpenAPI-driven fuzz against
`/api-doc/openapi.json` (214 endpoints tested). Informational
(`continue-on-error: true`). Each run surfaces **7–10 distinct 5xx
panics** (e.g., empty `sql` body returning 500 instead of 400, traces
endpoints crashing on unescaped path params that get interpolated into
SQL). Filed to backend team for separate triage.
- Server `o2_*.log` files uploaded as artifacts on failure (was:
silently lost).
- `api_tests_summary` now requires BOTH integration + regression to
pass.
### CI infrastructure fixes
A short hardening pass landed after the initial CI run surfaced several
issues:
- **pytest-sugar count inflation** — dropped `--force-sugar` from both
pytest invocations. The flag double-registered sugar on top of the
auto-loaded entry point, inflating the CI test counter ~3x (display
showed 1364 when collection was 486). Counter is now honest.
- **schemathesis 4.x flag migration** — `--base-url` → `-u`,
`--hypothesis-max-examples` → `-n`, `--hypothesis-deadline` →
`--request-timeout`, `--data-generation-method` → `-m`, dropped
`--auth-type basic`. The fuzz job was failing immediately on flag-parse
before this.
- **Regression matrix binary path** — used `${{ github.workspace
}}/target/debug/openobserve` so the absolute path resolves correctly
under `working-directory: tests/api-testing/` (was: every matrix job
failed with "No such file or directory").
- **`permissions: contents: read`** added to all 4 workflow jobs —
closes CodeQL alert #369.
- **Removed redundant server boot** in `api_integration_tests` (leftover
from main's structure after the regression matrix split; was silently
failing to bind to :5080 and getting masked by the first server).
## Backend bugs surfaced (for backend team to track)
The rework uncovered and visibly flagged these as `xfail`/documented
test comments:
1. `DELETE /api/organizations/{id}` returns 404 unconditionally even for
just-created orgs — endpoint appears unimplemented.
2. `test_dashboard_missing_required_fields` triggers backend panic
(returns 500 instead of 400/422).
3. JS `getEnrichmentTableData()` restricted to `_meta` org AND not
exposed in the `/functions/test` endpoint runtime — `ReferenceError`
even with explicit `enrichment_tables: [...]`.
4. Bulk API silently sanitizes invalid `_index` values (`****` → `_`)
without flagging — user data lands in unexpected index.
5. `AVG` of a single value drifts by ~1e-14 (likely float reduction
order; documented tolerance in `test_advanced_sql.py`).
6. Org name validation rejects hyphens (only
`A-Z/a-z/0-9/spaces/underscores`) — not in spec.
Plus 7–10 additional 5xx panics surfaced per fuzz run (input validation
gaps — separate triage list in fuzz job logs).
## Test bugs uncovered + fixed
- `test_pipeline.py`: had two function definitions named
`test_pipeline_creation_and_action_with_schedule` — Python's last-wins
meant the first test's body never executed for years.
- `test_alertsV2.py`: `assert alert_disabled == "False"` was indented
inside an `else:` branch that never executed because the response is
always a dict; AND `enabled` is a bool, not the string `"False"`. Two
layers of broken.
- `test_bulk.py`: variable typo (`valid_bulk_payload` reused on the POST
call) made the invalid-index test silently duplicate the valid case.
- `test_organisations.py`: `test_create_organization` used pre-pytest-4
`yield`-style fixtures — silently ignored by pytest 8.
- `test_js_result_array.py`: 8 tests skipped with `"Temporarily disabled
- failing test"` — the actual cause was that JS functions are restricted
to `_meta` org. One-line fix unskipped all 8.
- `test_fuzzy.py`: 4 tests `@pytest.mark.skip`-ed because they used a
1-minute time window against older static test data → 0 hits → flake →
skip. Wider window unskips them.
- 547 LOC of zombie commented-out test blocks deleted (15 commented `#
def test_` definitions, including one orphan case where the `def` was
commented but the indented body wasn't).
## Folder layout
```
tests/api-testing/
├── support/ # NEW framework
│ ├── client.py # OpenObserveClient
│ ├── endpoints/ # 7 per-resource wrappers
│ ├── factories.py # unique_name, time_window, *_payload
│ ├── wait.py # wait_until + WaitTimeout
│ ├── sse.py # read_sse_response
│ └── fixtures.py # client, temp_user_email, temp_stream_name, ...
├── helpers/ # Non-test shared code (was scattered in tests/)
│ ├── sourcemap_helpers.py
│ └── workflow/ # workflow chain + pages/ live here now
├── fixtures/
│ └── rum/ # Standalone RUM ingest scripts (was in tests/)
└── tests/
├── alerts/ # 3 files
├── dashboards/ # 4 files
├── ingest/ # 4 files
├── orgs/ # 2 files
├── pipelines/ # 10 files
├── regression/ # 2 files (env-flag matrix targets)
├── rum/ # 3 files
├── search/ # 8 files
├── streams/ # 1 file
├── users/ # 1 file
├── workflow/ # 3 files
├── conftest.py
└── test_framework_selfcheck.py
```
## Test plan
- [x] `rye run pytest --collect-only` — 486 tests discovered
- [x] Full default-set run (`--ignore=tests/regression`) — 443 pass / 2
fail / 39 skip / 2 xfail in 387s
- [x] `rye run pytest -m framework` — 14/14 selfcheck tests pass
- [x] `rye run ruff check support/ tests/test_framework_selfcheck.py` —
clean
- [x] CI run on this PR — `api_integration_tests` + all 4
`api_regression_tests` matrix jobs green
- [x] CI run — `api_fuzz_tests` runs cleanly (informational; surfaces
7–10 unique 5xx panics per run)
## Note on test count
The `test_streaming.py` rewrite folded 7 "tests" (`test_ingest_data`,
`test_ingest_join`, `test_disable_streaming`,
`test_update_max_query_range`, `test_enable_streaming`,
`test_delete_stream`, `test_delete_stream_join`) into pytest fixtures.
Each only checked `status_code == 200` of a setup/teardown operation —
they were imperative steps masquerading as tests. The fixtures still
perform the same operations and assert their success; they're just no
longer counted as discrete pytest items. **Functional coverage is
identical.**
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|