mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 16:09:05 +00:00
d16d693ba99bc20621a58b948ecbcad7ec7fa882
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vamsi Karthik R
|
1b50f7ca0a | test: regression coverage for retested bug fixes (#14376, #14317, #14648, #11283, #14636, #14640, #14917) (#14927) | ||
|
Vamsi Karthik R
|
33c0239410 | test: cover the bug fixes confirmed by retest, and drop the #13243 workarounds (#14842) | ||
 Prabhat Sharma
|
fea71e02d4 |
feat(alerts): alert library — browse, preview and install curated alerts (#13922)
## Alert Library A curated catalog at `/alert-library`: browse packs of ready-made alerts, see which ones can actually run against the streams this org already receives, preview a verdict against real data, and install one or many. The library is a **peer page** of Alerts / Notification Destinations / Templates — flat routing, presented as tabs via a shared `AlertSectionTabs` in `OPageLayout #header-tabs` (the pattern `PipelineSectionTabs` already ships). ### What landed, by phase | Phase | What | | --- | --- | | Data layer | S3 manifest fetch, cache, readiness resolution (`useAlertLibrary`, `constants/alertLibrary`, `types/alertLibrary`) | | Gallery | Rail (packs / categories, multi-select, segmented switch), stat-strip availability facets, cards | | Drawer | Detail view with query preview and customizable tunables | | Install | Bulk-install wizard — destination selection, per-alert tunables, stale-session hardening | | Prefill | Carry a library alert (query, trigger, threshold, severity) into the alert editor | ### Design decisions worth knowing - **Rail navigates, strip filters.** Availability is deliberately not in the rail — carrying both overflowed it and pushed a facet group below the fold. - **Colour marks the exception only.** No "Ready" pill (it would describe ~75 of 86 cards and signal nothing); only unusable cards carry a label. - **A failed `/streams` call leaves readiness unknown**, rather than claiming "none of your data matches" — that claim would be about the request, not the org, and it is the most alarming thing this page can say. Counts read `—` until there is an answer, and a failed *refresh* keeps the verdict it already had. - **No "Installed" tile** — installed detection doesn't exist yet, and a tile reading a permanent zero is dishonest UI. - **`severity: warning` renders as `medium`** — the badge group's scale is critical/high/medium/low/info, and medium is the amber it also gives P3, which is exactly the priority a warning alert installs as. - Pack and category names come from the manifest, so they render as `raw()` data — a pack added upstream needs no frontend release. ### Design-system changes (used by the library, applied app-wide) - **`warning-quiet` badge tone** — one variant whose values differ by theme: the soft amber wash in light, a neutral chip in dark. Neither the filled amber chip nor the neutral one was right in both, because the two ramps aren't symmetric. - **Border tokens now meet WCAG 2.1 SC 1.4.11.** Every border token measured under 3:1 (`border-strong` was 2.52:1). `border-default` firms to grey-300 and controls get a token that clears 3:1. - Nav: Reliability's flyout groups its four alerting entries under one **Alerts** header, declared in the same order as the tab strip those pages carry. Flyout levels are sized by depth. - `sensors-off` icon added to the `OIcon` registry. ### Fixes that rode along - **metrics**: a shared URL whose `metrics_data` blob omits panel query fields no longer blanks the whole metrics page — blob queries are filled out to the panel schema before being laid over. - **dashboards**: a PromQL series with no legend template was named `JSON.stringify(metric)`, so fifty series read identically for the width of the chip. Series are now named by the labels that actually differ, preferring the ones that say *which workload* it is (namespace / service / pod / host). Correlation's semantic groups refine that when loaded, but nothing is fetched to render a chart — that endpoint is enterprise-only and a chart must render the same on OSS. - **alerts**: preview evaluates against the alert's real trigger; deleting a row no longer reloads the list behind it; peer tabs no longer shift when switching alerting pages. - **ui**: badge-only select options no longer render bold. ### Tests - `test(search)`: pins the streaming-aggregate **wire contract** at the HTTP boundary. Under streaming aggregates each partition carries the cumulative merged state (last frame is the answer) while ordinary partitions are disjoint and must be summed. No pytest had ever received `streaming_aggs: true` — the emitter could have flipped to per-partition deltas with the whole suite green while dashboards, Logs and the alert preview all silently over-counted. - `test(e2e)`: four helpers that passed only while an org stayed small — an assertion reading past a 20-row page, an unfiltered `OSelect` that virtualises past 50 entries, a tab click landing on the rename pencil, and two builder tests asserting on whichever metric sorted first. - New unit specs across the library components, composable, facets, tunables and install path. ### Notes - **i18n**: en-US only; CI generates the other 14 locales. - **No backend changes** — this branch is entirely `web/` plus test helpers. |
||
 ktx-akshay
|
33c6e9e0e1 |
test(dashboards): cover CTE/subquery match_all queries through the panel search path (#13838)
## What First test coverage for the dashboard **panel** endpoints and the **dashboard search path**, running CTE/subquery `match_all` queries as real dashboard panels. Nothing in the repo previously touched either: `grep -rn "/panels\|search_type=dashboards" tests/api-testing/` returned zero hits. ## Why Panels query `_search?type=logs&search_type=dashboards&dashboard_id=…` — a different route into the engine than the Logs page (`search_type=ui`). The `match_all` + FST projection rewrite fixed in #13808 runs inside it, so a query that passes via Logs can still regress on a dashboard. The rewrite has to preserve the projection of the block the predicate sits in. When that block is the whole statement, the projection is obvious. When it's a CTE body feeding a JOIN, a window, or a four-branch UNION ALL, it has to preserve the *CTE's* output schema while the outer query reads different columns from it — which is where it went wrong. ## Coverage **71 distinct queries across 29 combination families**, all with `match_all` inside a `WITH …` body or a derived table: | Axis | Variants | |---|---| | Nesting depth | 1, 2, 3, and 5-level CTE chains | | Nesting kind | CTEs and derived tables (and one query using both) | | UNION ALL | 2, 3, and 4 separate `match_all` branches | | JOIN | CTE⋈CTE, derived⋈derived, cross-stream | | Window | `ROW_NUMBER`, `RANK`, `DENSE_RANK`, partitioned aggregates | | Negation | `NOT match_all`, and mixed with positive `match_all` | | Combined with | `COUNT(DISTINCT)`, `CASE`, `HAVING`, `histogram()` | Each case creates a real panel, runs it via the dashboard search path, then asserts row count against the DuckDB oracle and that no expected column was dropped from the projection. A further 30 cases run the same SQL through **both** Logs and dashboard and diff the results — that's what catches a rewrite firing on only one path. Queries are reused from the existing query-agent corpus rather than duplicated, so the two suites can't drift: query-agent verifies the values via the Logs path, this verifies the same SQL survives the dashboard path. Cases are **discovered by classifying the SQL**, not from a hand-written ID list, so new combinations added to the corpus are picked up automatically. Test ids name the shape (`Q1022-cte1_union_cdist_4ma`), so a failure identifies the combination without opening the corpus. Plus 14 tests for the panel endpoints themselves — AddPanel / UpdatePanel / DeletePanel, contract + CRUD lifecycle + 7 error paths derived from `DashboardError` in `src/core/src/http.rs`. ## Changes | File | | |---|---| | `tests/dashboards/test_dashboard_cte_panels.py` | new — 103 tests | | `tests/dashboards/test_dashboard_panels.py` | new — 14 tests | | `tests/dashboards/conftest.py` | new — shared fixtures | | `support/panel_queries.py` | new — discovery + panel helpers | | `support/endpoints/dashboards.py` | +78, additive | | `support/endpoints/search.py` | +36, additive | | `support/factories.py` | +73, additive | The three shared-file edits are **187 added lines, 0 modified** — nothing existing can break. ## Results 168 tests in tests/api-testing/tests/dashboards/ (117 new + 51 pre-existing) 166 passed, 1 skipped, 1 xfailed # full nested-query corpus present 74 passed, 2 skipped, 1 xfailed # current corpus on main Repeated 2× and 4× respectively, no flakes. Discovery scales with the corpus: on main it finds 8 nested `match_all` cases, and 71 once the larger sentinel set is present. Cases that aren't in the corpus skip with an explanatory message instead of failing, so this is safe to merge in either order. |
||
 Shrinath Rao
|
a647e6aee9 |
test: add retry_on_db_lock helper to support/wait.py (#12434)
## Summary
- Adds `retry_on_db_lock()` to `tests/api-testing/support/wait.py`
- Companion to the enterprise-side fix (openobserve/o2-enterprise#TBD)
that applies the helper across all RBAC entity base tests
## Root cause
The `ent_rbac` CI shard runs with `-n 2` (two pytest-xdist workers).
SQLite's single-writer model means parallel workers racing on write
operations produce transient `500 {"message": "database is locked"}`
responses (SQLite error codes 5 / 517). These surface as flaky assertion
failures across many PRs.
## Fix
`retry_on_db_lock(fn)` wraps an HTTP call and retries up to 3 times with
exponential backoff (0.5 → 1 → 2 s) only when the response is `500` with
`"database is locked"` in the body. Parallelism is preserved; the brief
retry window absorbs the time the other writer needs to commit and
release the lock.
## Test plan
- [ ] OSS CI passes with this branch
- [ ] Enterprise PR with the same branch name picks up the new utility
and applies it in all RBAC entity base tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
 Shrinath RaoandClaude Opus 4.7
|
61aa772d57 |
test: revamp api-testing suite (framework + 16 file rewrites + CI matrix) (#12372)
## Summary
Comprehensive rework of the `tests/api-testing/` suite. 30+ commits
implementing all 6 phases of the [revamp
plan](../../tests/ui-testing/MD_Files/api%20tests%20revamp/API_TESTS_REVAMP.md).
**Before → After:**
| | Baseline | Final | Δ |
|---|---|---|---|
| passed | 421 | **443** | **+22 ✓** |
| failed | 3 | **2** | -1 (only same pre-existing) |
| skipped | 53 | 39 | -14 (unskipped dead tests now run) |
| xfailed | 1 | 2 | +1 (new visible backend-bug signal) |
| runtime | 588s | **387s** | **-201s (34% faster)** |
**Zero tests that passed in baseline are now failing.** The 2 remaining
failures are the exact pre-existing ones
(`test_enrichment_table_join.py` backend panic + `test_workflow.py` SMTP
env-miss).
## What's in the diff
- **New `support/` framework** — `OpenObserveClient`, endpoint wrappers
(`alerts`, `dashboards`, `streams`, `search`, etc.), `wait_until()`,
payload factories, pytest fixtures, SSE parser. 14 framework selfcheck
tests.
- **16 of 16 OLD-tier test files rewritten** — using the new framework,
unique-suffix names, body validation, `wait_until` instead of
`time.sleep`, parametrize where appropriate.
- **29 files moved into 12 feature folders** mirroring the UI suite
layout
(`tests/{alerts,dashboards,ingest,orgs,pipelines,regression,rum,search,streams,users,workflow}/`).
- **−6,500 LOC net** (copy-paste + dead code removal; new framework adds
back ~1,500).
- **Ruff config** in pyproject.toml: T201 ban on `print()`, PT pytest
rules, B bugbear, UP pyupgrade.
## CI workflow changes
- New job `api_regression_tests`: `strategy.matrix` over the 4 env-flag
combinations (UTF8_VIEW × JOIN_MATCH_ONE) — now runs in parallel instead
of as 4 sequential server reboots in the main job.
- New job `api_fuzz_tests`: schemathesis OpenAPI-driven fuzz against
`/api-doc/openapi.json` (214 endpoints tested). Informational
(`continue-on-error: true`). Each run surfaces **7–10 distinct 5xx
panics** (e.g., empty `sql` body returning 500 instead of 400, traces
endpoints crashing on unescaped path params that get interpolated into
SQL). Filed to backend team for separate triage.
- Server `o2_*.log` files uploaded as artifacts on failure (was:
silently lost).
- `api_tests_summary` now requires BOTH integration + regression to
pass.
### CI infrastructure fixes
A short hardening pass landed after the initial CI run surfaced several
issues:
- **pytest-sugar count inflation** — dropped `--force-sugar` from both
pytest invocations. The flag double-registered sugar on top of the
auto-loaded entry point, inflating the CI test counter ~3x (display
showed 1364 when collection was 486). Counter is now honest.
- **schemathesis 4.x flag migration** — `--base-url` → `-u`,
`--hypothesis-max-examples` → `-n`, `--hypothesis-deadline` →
`--request-timeout`, `--data-generation-method` → `-m`, dropped
`--auth-type basic`. The fuzz job was failing immediately on flag-parse
before this.
- **Regression matrix binary path** — used `${{ github.workspace
}}/target/debug/openobserve` so the absolute path resolves correctly
under `working-directory: tests/api-testing/` (was: every matrix job
failed with "No such file or directory").
- **`permissions: contents: read`** added to all 4 workflow jobs —
closes CodeQL alert #369.
- **Removed redundant server boot** in `api_integration_tests` (leftover
from main's structure after the regression matrix split; was silently
failing to bind to :5080 and getting masked by the first server).
## Backend bugs surfaced (for backend team to track)
The rework uncovered and visibly flagged these as `xfail`/documented
test comments:
1. `DELETE /api/organizations/{id}` returns 404 unconditionally even for
just-created orgs — endpoint appears unimplemented.
2. `test_dashboard_missing_required_fields` triggers backend panic
(returns 500 instead of 400/422).
3. JS `getEnrichmentTableData()` restricted to `_meta` org AND not
exposed in the `/functions/test` endpoint runtime — `ReferenceError`
even with explicit `enrichment_tables: [...]`.
4. Bulk API silently sanitizes invalid `_index` values (`****` → `_`)
without flagging — user data lands in unexpected index.
5. `AVG` of a single value drifts by ~1e-14 (likely float reduction
order; documented tolerance in `test_advanced_sql.py`).
6. Org name validation rejects hyphens (only
`A-Z/a-z/0-9/spaces/underscores`) — not in spec.
Plus 7–10 additional 5xx panics surfaced per fuzz run (input validation
gaps — separate triage list in fuzz job logs).
## Test bugs uncovered + fixed
- `test_pipeline.py`: had two function definitions named
`test_pipeline_creation_and_action_with_schedule` — Python's last-wins
meant the first test's body never executed for years.
- `test_alertsV2.py`: `assert alert_disabled == "False"` was indented
inside an `else:` branch that never executed because the response is
always a dict; AND `enabled` is a bool, not the string `"False"`. Two
layers of broken.
- `test_bulk.py`: variable typo (`valid_bulk_payload` reused on the POST
call) made the invalid-index test silently duplicate the valid case.
- `test_organisations.py`: `test_create_organization` used pre-pytest-4
`yield`-style fixtures — silently ignored by pytest 8.
- `test_js_result_array.py`: 8 tests skipped with `"Temporarily disabled
- failing test"` — the actual cause was that JS functions are restricted
to `_meta` org. One-line fix unskipped all 8.
- `test_fuzzy.py`: 4 tests `@pytest.mark.skip`-ed because they used a
1-minute time window against older static test data → 0 hits → flake →
skip. Wider window unskips them.
- 547 LOC of zombie commented-out test blocks deleted (15 commented `#
def test_` definitions, including one orphan case where the `def` was
commented but the indented body wasn't).
## Folder layout
```
tests/api-testing/
├── support/ # NEW framework
│ ├── client.py # OpenObserveClient
│ ├── endpoints/ # 7 per-resource wrappers
│ ├── factories.py # unique_name, time_window, *_payload
│ ├── wait.py # wait_until + WaitTimeout
│ ├── sse.py # read_sse_response
│ └── fixtures.py # client, temp_user_email, temp_stream_name, ...
├── helpers/ # Non-test shared code (was scattered in tests/)
│ ├── sourcemap_helpers.py
│ └── workflow/ # workflow chain + pages/ live here now
├── fixtures/
│ └── rum/ # Standalone RUM ingest scripts (was in tests/)
└── tests/
├── alerts/ # 3 files
├── dashboards/ # 4 files
├── ingest/ # 4 files
├── orgs/ # 2 files
├── pipelines/ # 10 files
├── regression/ # 2 files (env-flag matrix targets)
├── rum/ # 3 files
├── search/ # 8 files
├── streams/ # 1 file
├── users/ # 1 file
├── workflow/ # 3 files
├── conftest.py
└── test_framework_selfcheck.py
```
## Test plan
- [x] `rye run pytest --collect-only` — 486 tests discovered
- [x] Full default-set run (`--ignore=tests/regression`) — 443 pass / 2
fail / 39 skip / 2 xfail in 387s
- [x] `rye run pytest -m framework` — 14/14 selfcheck tests pass
- [x] `rye run ruff check support/ tests/test_framework_selfcheck.py` —
clean
- [x] CI run on this PR — `api_integration_tests` + all 4
`api_regression_tests` matrix jobs green
- [x] CI run — `api_fuzz_tests` runs cleanly (informational; surfaces
7–10 unique 5xx panics per run)
## Note on test count
The `test_streaming.py` rewrite folded 7 "tests" (`test_ingest_data`,
`test_ingest_join`, `test_disable_streaming`,
`test_update_max_query_range`, `test_enable_streaming`,
`test_delete_stream`, `test_delete_stream_join`) into pytest fixtures.
Each only checked `status_code == 200` of a setup/teardown operation —
they were imperative steps masquerading as tests. The fixtures still
perform the same operations and assert their success; they're just no
longer counted as discrete pytest items. **Functional coverage is
identical.**
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|