mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 16:09:05 +00:00
main
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
fea71e02d4 |
feat(alerts): alert library — browse, preview and install curated alerts (#13922)
## Alert Library A curated catalog at `/alert-library`: browse packs of ready-made alerts, see which ones can actually run against the streams this org already receives, preview a verdict against real data, and install one or many. The library is a **peer page** of Alerts / Notification Destinations / Templates — flat routing, presented as tabs via a shared `AlertSectionTabs` in `OPageLayout #header-tabs` (the pattern `PipelineSectionTabs` already ships). ### What landed, by phase | Phase | What | | --- | --- | | Data layer | S3 manifest fetch, cache, readiness resolution (`useAlertLibrary`, `constants/alertLibrary`, `types/alertLibrary`) | | Gallery | Rail (packs / categories, multi-select, segmented switch), stat-strip availability facets, cards | | Drawer | Detail view with query preview and customizable tunables | | Install | Bulk-install wizard — destination selection, per-alert tunables, stale-session hardening | | Prefill | Carry a library alert (query, trigger, threshold, severity) into the alert editor | ### Design decisions worth knowing - **Rail navigates, strip filters.** Availability is deliberately not in the rail — carrying both overflowed it and pushed a facet group below the fold. - **Colour marks the exception only.** No "Ready" pill (it would describe ~75 of 86 cards and signal nothing); only unusable cards carry a label. - **A failed `/streams` call leaves readiness unknown**, rather than claiming "none of your data matches" — that claim would be about the request, not the org, and it is the most alarming thing this page can say. Counts read `—` until there is an answer, and a failed *refresh* keeps the verdict it already had. - **No "Installed" tile** — installed detection doesn't exist yet, and a tile reading a permanent zero is dishonest UI. - **`severity: warning` renders as `medium`** — the badge group's scale is critical/high/medium/low/info, and medium is the amber it also gives P3, which is exactly the priority a warning alert installs as. - Pack and category names come from the manifest, so they render as `raw()` data — a pack added upstream needs no frontend release. ### Design-system changes (used by the library, applied app-wide) - **`warning-quiet` badge tone** — one variant whose values differ by theme: the soft amber wash in light, a neutral chip in dark. Neither the filled amber chip nor the neutral one was right in both, because the two ramps aren't symmetric. - **Border tokens now meet WCAG 2.1 SC 1.4.11.** Every border token measured under 3:1 (`border-strong` was 2.52:1). `border-default` firms to grey-300 and controls get a token that clears 3:1. - Nav: Reliability's flyout groups its four alerting entries under one **Alerts** header, declared in the same order as the tab strip those pages carry. Flyout levels are sized by depth. - `sensors-off` icon added to the `OIcon` registry. ### Fixes that rode along - **metrics**: a shared URL whose `metrics_data` blob omits panel query fields no longer blanks the whole metrics page — blob queries are filled out to the panel schema before being laid over. - **dashboards**: a PromQL series with no legend template was named `JSON.stringify(metric)`, so fifty series read identically for the width of the chip. Series are now named by the labels that actually differ, preferring the ones that say *which workload* it is (namespace / service / pod / host). Correlation's semantic groups refine that when loaded, but nothing is fetched to render a chart — that endpoint is enterprise-only and a chart must render the same on OSS. - **alerts**: preview evaluates against the alert's real trigger; deleting a row no longer reloads the list behind it; peer tabs no longer shift when switching alerting pages. - **ui**: badge-only select options no longer render bold. ### Tests - `test(search)`: pins the streaming-aggregate **wire contract** at the HTTP boundary. Under streaming aggregates each partition carries the cumulative merged state (last frame is the answer) while ordinary partitions are disjoint and must be summed. No pytest had ever received `streaming_aggs: true` — the emitter could have flipped to per-partition deltas with the whole suite green while dashboards, Logs and the alert preview all silently over-counted. - `test(e2e)`: four helpers that passed only while an org stayed small — an assertion reading past a 20-row page, an unfiltered `OSelect` that virtualises past 50 entries, a tab click landing on the rename pencil, and two builder tests asserting on whichever metric sorted first. - New unit specs across the library components, composable, facets, tunables and install path. ### Notes - **i18n**: en-US only; CI generates the other 14 locales. - **No backend changes** — this branch is entirely `web/` plus test helpers. |
||
|
|
61aa772d57 |
test: revamp api-testing suite (framework + 16 file rewrites + CI matrix) (#12372)
## Summary
Comprehensive rework of the `tests/api-testing/` suite. 30+ commits
implementing all 6 phases of the [revamp
plan](../../tests/ui-testing/MD_Files/api%20tests%20revamp/API_TESTS_REVAMP.md).
**Before → After:**
| | Baseline | Final | Δ |
|---|---|---|---|
| passed | 421 | **443** | **+22 ✓** |
| failed | 3 | **2** | -1 (only same pre-existing) |
| skipped | 53 | 39 | -14 (unskipped dead tests now run) |
| xfailed | 1 | 2 | +1 (new visible backend-bug signal) |
| runtime | 588s | **387s** | **-201s (34% faster)** |
**Zero tests that passed in baseline are now failing.** The 2 remaining
failures are the exact pre-existing ones
(`test_enrichment_table_join.py` backend panic + `test_workflow.py` SMTP
env-miss).
## What's in the diff
- **New `support/` framework** — `OpenObserveClient`, endpoint wrappers
(`alerts`, `dashboards`, `streams`, `search`, etc.), `wait_until()`,
payload factories, pytest fixtures, SSE parser. 14 framework selfcheck
tests.
- **16 of 16 OLD-tier test files rewritten** — using the new framework,
unique-suffix names, body validation, `wait_until` instead of
`time.sleep`, parametrize where appropriate.
- **29 files moved into 12 feature folders** mirroring the UI suite
layout
(`tests/{alerts,dashboards,ingest,orgs,pipelines,regression,rum,search,streams,users,workflow}/`).
- **−6,500 LOC net** (copy-paste + dead code removal; new framework adds
back ~1,500).
- **Ruff config** in pyproject.toml: T201 ban on `print()`, PT pytest
rules, B bugbear, UP pyupgrade.
## CI workflow changes
- New job `api_regression_tests`: `strategy.matrix` over the 4 env-flag
combinations (UTF8_VIEW × JOIN_MATCH_ONE) — now runs in parallel instead
of as 4 sequential server reboots in the main job.
- New job `api_fuzz_tests`: schemathesis OpenAPI-driven fuzz against
`/api-doc/openapi.json` (214 endpoints tested). Informational
(`continue-on-error: true`). Each run surfaces **7–10 distinct 5xx
panics** (e.g., empty `sql` body returning 500 instead of 400, traces
endpoints crashing on unescaped path params that get interpolated into
SQL). Filed to backend team for separate triage.
- Server `o2_*.log` files uploaded as artifacts on failure (was:
silently lost).
- `api_tests_summary` now requires BOTH integration + regression to
pass.
### CI infrastructure fixes
A short hardening pass landed after the initial CI run surfaced several
issues:
- **pytest-sugar count inflation** — dropped `--force-sugar` from both
pytest invocations. The flag double-registered sugar on top of the
auto-loaded entry point, inflating the CI test counter ~3x (display
showed 1364 when collection was 486). Counter is now honest.
- **schemathesis 4.x flag migration** — `--base-url` → `-u`,
`--hypothesis-max-examples` → `-n`, `--hypothesis-deadline` →
`--request-timeout`, `--data-generation-method` → `-m`, dropped
`--auth-type basic`. The fuzz job was failing immediately on flag-parse
before this.
- **Regression matrix binary path** — used `${{ github.workspace
}}/target/debug/openobserve` so the absolute path resolves correctly
under `working-directory: tests/api-testing/` (was: every matrix job
failed with "No such file or directory").
- **`permissions: contents: read`** added to all 4 workflow jobs —
closes CodeQL alert #369.
- **Removed redundant server boot** in `api_integration_tests` (leftover
from main's structure after the regression matrix split; was silently
failing to bind to :5080 and getting masked by the first server).
## Backend bugs surfaced (for backend team to track)
The rework uncovered and visibly flagged these as `xfail`/documented
test comments:
1. `DELETE /api/organizations/{id}` returns 404 unconditionally even for
just-created orgs — endpoint appears unimplemented.
2. `test_dashboard_missing_required_fields` triggers backend panic
(returns 500 instead of 400/422).
3. JS `getEnrichmentTableData()` restricted to `_meta` org AND not
exposed in the `/functions/test` endpoint runtime — `ReferenceError`
even with explicit `enrichment_tables: [...]`.
4. Bulk API silently sanitizes invalid `_index` values (`****` → `_`)
without flagging — user data lands in unexpected index.
5. `AVG` of a single value drifts by ~1e-14 (likely float reduction
order; documented tolerance in `test_advanced_sql.py`).
6. Org name validation rejects hyphens (only
`A-Z/a-z/0-9/spaces/underscores`) — not in spec.
Plus 7–10 additional 5xx panics surfaced per fuzz run (input validation
gaps — separate triage list in fuzz job logs).
## Test bugs uncovered + fixed
- `test_pipeline.py`: had two function definitions named
`test_pipeline_creation_and_action_with_schedule` — Python's last-wins
meant the first test's body never executed for years.
- `test_alertsV2.py`: `assert alert_disabled == "False"` was indented
inside an `else:` branch that never executed because the response is
always a dict; AND `enabled` is a bool, not the string `"False"`. Two
layers of broken.
- `test_bulk.py`: variable typo (`valid_bulk_payload` reused on the POST
call) made the invalid-index test silently duplicate the valid case.
- `test_organisations.py`: `test_create_organization` used pre-pytest-4
`yield`-style fixtures — silently ignored by pytest 8.
- `test_js_result_array.py`: 8 tests skipped with `"Temporarily disabled
- failing test"` — the actual cause was that JS functions are restricted
to `_meta` org. One-line fix unskipped all 8.
- `test_fuzzy.py`: 4 tests `@pytest.mark.skip`-ed because they used a
1-minute time window against older static test data → 0 hits → flake →
skip. Wider window unskips them.
- 547 LOC of zombie commented-out test blocks deleted (15 commented `#
def test_` definitions, including one orphan case where the `def` was
commented but the indented body wasn't).
## Folder layout
```
tests/api-testing/
├── support/ # NEW framework
│ ├── client.py # OpenObserveClient
│ ├── endpoints/ # 7 per-resource wrappers
│ ├── factories.py # unique_name, time_window, *_payload
│ ├── wait.py # wait_until + WaitTimeout
│ ├── sse.py # read_sse_response
│ └── fixtures.py # client, temp_user_email, temp_stream_name, ...
├── helpers/ # Non-test shared code (was scattered in tests/)
│ ├── sourcemap_helpers.py
│ └── workflow/ # workflow chain + pages/ live here now
├── fixtures/
│ └── rum/ # Standalone RUM ingest scripts (was in tests/)
└── tests/
├── alerts/ # 3 files
├── dashboards/ # 4 files
├── ingest/ # 4 files
├── orgs/ # 2 files
├── pipelines/ # 10 files
├── regression/ # 2 files (env-flag matrix targets)
├── rum/ # 3 files
├── search/ # 8 files
├── streams/ # 1 file
├── users/ # 1 file
├── workflow/ # 3 files
├── conftest.py
└── test_framework_selfcheck.py
```
## Test plan
- [x] `rye run pytest --collect-only` — 486 tests discovered
- [x] Full default-set run (`--ignore=tests/regression`) — 443 pass / 2
fail / 39 skip / 2 xfail in 387s
- [x] `rye run pytest -m framework` — 14/14 selfcheck tests pass
- [x] `rye run ruff check support/ tests/test_framework_selfcheck.py` —
clean
- [x] CI run on this PR — `api_integration_tests` + all 4
`api_regression_tests` matrix jobs green
- [x] CI run — `api_fuzz_tests` runs cleanly (informational; surfaces
7–10 unique 5xx panics per run)
## Note on test count
The `test_streaming.py` rewrite folded 7 "tests" (`test_ingest_data`,
`test_ingest_join`, `test_disable_streaming`,
`test_update_max_query_range`, `test_enable_streaming`,
`test_delete_stream`, `test_delete_stream_join`) into pytest fixtures.
Each only checked `status_code == 200` of a setup/teardown operation —
they were imperative steps masquerading as tests. The fixtures still
perform the same operations and assert their success; they're just no
longer counted as discrete pytest items. **Functional coverage is
identical.**
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|