Two regression sentinels for the FST projection rewrite path fixed in
openobserve/openobserve#13808 (currently in the merge queue on
branch-v0.40.0 and awaiting the main-branch counterpart):
Q1001 — CTE with trim(log) alias, match_all in WHERE, column-pruned
outer SELECT. Mirrors the shape of the PR's own unit test
(test_cse_projection_schema_propagation) — CSE inserts a
ProjectionExec between filter and datasource; a regression in the
rewriter's f_down handling would surface here.
Q1002 — five-CTE pipeline (filtered → in_shop → scoped → normalized
→ cleaned) mixing match_all with LIKE, split_part, regexp_replace ID
normalization, and GROUP BY aggregation. Higher structural pressure:
more intermediate ProjectionExec nodes and more chances for a stale
column index to leak through.
Uses only test-fixture fields (log, resource_path, page_url,
http_method); no customer identifiers.
Also adds a third leg to the query-agent CI matrix that runs with
ZO_FEATURE_QUERY_REMOVE_FILTER_WITH_INDEX=false — exercises the
non-index-filter path that the match_all projection rewriter also
touches.
The wide JSON churn across the other query files is compute_counts.py
re-baselining expected results against a fresh BASE_TS
(base_ts_override.json is untracked); tuple-order shuffles are harmless
under results_mode: rowsort.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Problem
The API/DB integration test jobs intermittently fail — often, and
retries don't reliably help. CI reports the failure on the **`Pin
cpython`** step:
```
rye: command not found
##[error]Process completed with exit code 127
```
But that step is a **victim, not the cause**. The real failure is the
preceding **`Setup rye`** step (`eifinger/setup-rye@v4`). It downloads
and extracts the `rye` binary fine, then runs `rye self install`, which
downloads a **python-build-standalone toolchain (3.11.7)** from GitHub
to bootstrap Rye's own venv:
```
Downloading Rye from ".../rye/releases/download/0.27.0/rye-x86_64-linux.gz" ... ✓
Extracting downloaded archive... ✓
Determined RYE_TOOLCHAIN_VERSION: 3.11.7
Installing Rye into /opt/hostedtoolcache/setup-rye-.../0.27.0/x86_64
##[error]The process '.../_temp/...' failed with exit code 1 ← real failure
```
That download flakes and exits 1, so the `rye` binary is never placed on
`PATH`, and the next step (`rye pin cpython@3.11.6`) dies with `command
not found` (exit 127). Retries only pass when the download happens to
succeed.
Example failing run:
https://github.com/openobserve/openobserve/actions/runs/31679986019/job/94383108990
## Fix
Every one of these jobs already provisions CPython 3.11.6 via
`actions/setup-python`. Point **`RYE_TOOLCHAIN`** at that interpreter so
`rye self install` reuses it instead of downloading a second toolchain —
removing the flaky network fetch entirely:
```yaml
- name: Setup Python
uses: actions/setup-python@v7
id: setup-python
with:
python-version: "3.11.6"
- name: Setup rye
uses: eifinger/setup-rye@v4
env:
RYE_TOOLCHAIN: ${{ steps.setup-python.outputs.python-path }}
with:
...
```
The `setup-python` step's official `python-path` output is the absolute
path to the interpreter it provisioned — used directly (no hardcoded
`/bin/python`), and present in every affected job (verified 1:1).
## Scope
- `.github/workflows/api-testing.yml` — 4 `Setup rye` blocks
- `.github/workflows/db-testing.yml` — 1 `Setup rye` block
Additive, low-risk: only redirects Rye's internal bootstrap; `rye pin` /
`rye sync` are untouched.
## Companion PR
Enterprise: openobserve/o2-enterprise#2424 — same fix on
`api-testing.yml` (8 blocks) + `api-testing_env.yml` (2 blocks).
Two refinements to the `e2e` label gate (Playwright + API, OSS + ENT).
## 1. Adding the `e2e` label always runs the suite
Previously the auto-bypass could override the label — add `e2e` to an
otherwise-skippable PR and nothing ran, which was confusing. Now the
label is an explicit **"run it"**: it always runs the full suite.
Without the label, the skip logic still decides.
| Event | `e2e` label | Change | Result |
|---|---|---|---|
| PR | **yes** | anything (even docs) | **run full suite** → pass/fail
on results |
| PR | no | relevant (code) | **block** (fail, "add label") |
| PR | no | skip category (docs / unrelated workflow) | **pass, no run**
|
| push / merge queue / dispatch | n/a | relevant | run |
| push / merge queue / dispatch | n/a | nothing relevant | pass, no run
|
Gate flips from `has_changes && (non-PR || label)` to `(PR && label) ||
(non-PR && has_changes)`; the summary is reworked to match.
## 2. Unrelated workflow changes no longer require e2e
`check_changes` treated **any** `.github/workflows/**` change as
relevant, so a Dependabot bump to an unrelated workflow
(openobserve#13060, `update-translations.yml`) demanded the label. Now
workflow files are bypassed by default; only a change to the suite's
**own** workflow (`playwright.yml` / `api-testing.yml`) triggers it.
Implemented with captured non-ignored lines instead of `grep -qv`
(portable across grep builds).
## Note
This PR changes `playwright.yml` + `api-testing.yml`, so it correctly
**needs the `e2e` label** to run. Sibling PR on the other repo uses the
same branch `ci/e2e-skip-unrelated-workflows`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
## Why
Our OSS Playwright + API CI compute (ubicloud build + eks matrix) is a
major cost driver. Two sources of avoidable spend:
1. **Every merged PR ran the full heavy suite up to 3×** — on the PR
(label-gated), in the merge queue (`merge_group`), and *again* on `push:
main`. The `push: main` run is a duplicate of the `merge_group` run
(both build/test the exact commit landing on main) **and** it bypasses
the `e2e` label gate, clawing back the gate's savings.
2. **Regression runs every 4h regardless of whether `main` moved** —
rebuilding and re-running the full matrix on identical code
overnight/weekends.
## Changes
**`playwright.yml` + `api-testing.yml`**
- Remove the `push: main` trigger. `merge_group` remains the trusted
main-ward validation.
- **Cache fix (important):** the Rust cache was only *saved* on `push:
main` runs (`save-if: github.ref == 'refs/heads/main'`). Deleting `push`
alone would stop the cache ever refreshing → cold builds → *more*
compute. So `save-if` now also fires on the `merge_group` run, which
builds the exact soon-to-be-main commit.
**`playwright_regression.yml`**
- New `check_main_changed` gate: compares `main`'s HEAD to the last
run's `head_sha` and skips build+matrix when nothing merged since.
Manual `workflow_dispatch` always runs. The skip cascades cleanly
(`playwright_summary` already treats skipped build/ui as success → stays
green).
## Impact
- ~1 full build + 146-shard matrix eliminated per merged PR (the
`push:main` duplicate).
- Idle-period regression runs (no merges) skipped instead of running the
full matrix.
- No coverage loss: merge queue still gates every merge; cache stays
warm.
## Not in this PR (deferred per discussion)
- `db-testing.yml` / `unit-tests.yml` `push:main` removal (cheaper,
later).
- ENT merge-queue adoption.
- OSS→ENT dispatch (semgrep / endpoint-auth-probe) gating.
Paired ENT change (regression gate): openobserve/o2-enterprise
`ci/trim-redundant-runs`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## What
Gate the full **Playwright (UI)** and **API** test suites on PRs behind
a single **`e2e` label**, and **auto-bypass** PRs that only touch files
which can't affect those suites. Covers `playwright.yml` and
`api-testing.yml`.
## Behaviour (both suites)
| Situation | build/test jobs | summary check (`playwright_summary` /
`api_tests_summary`) | Mergeable? |
|---|---|---|---|
| PR with `e2e` label | full suite runs | green if tests pass | ✅ if
green |
| PR, no label, relevant code changed | skipped | **fails** ("add `e2e`
label") | ❌ blocked |
| PR, no label, docs/meta only | skipped | **passes** (auto-bypass) | ✅
|
| Remove the `e2e` label | skipped (fresh run) | **fails** again | ❌
until re-added |
| push to `main` / merge queue / dispatch | full suite runs | green if
tests pass | n/a |
One `e2e` label runs **both** UI + API. `push`, `merge_group`, and
`workflow_dispatch` are never label-gated.
## Why the gate lives in the summary job
A job that `if:`-skips reports `skipped`, which branch protection treats
as **pass**. So a no-label PR can't be blocked by merely skipping jobs —
the summary job **actively fails** when the change is relevant and the
`e2e` label is absent.
## Auto-bypass (`check_changes`)
No label needed for: `*.md/.mdx/.markdown/.rst/.adoc`, `docs/`,
images/media, `LICENSE/NOTICE/AUTHORS/CONTRIBUTORS/CHANGELOG/.mailmap`,
`CODEOWNERS`, GitHub issue/PR/funding templates,
`dependabot`/`renovate`,
`.gitignore/.gitattributes/.editorconfig/.gitkeep/.husky/.vscode/.idea`.
The **API** workflow additionally bypasses `web/` + UI-test dirs
(frontend changes don't affect API tests).
**Not** bypassed (still trigger): `.github/workflows/**` (incl. these
workflow files), source, `package*.json`/lockfiles, configs, `*.txt`.
## Rollout prerequisites
- Create the **`e2e`** label in this repo.
- Required status checks = **`playwright_summary`** and
**`api_tests_summary`** — the individual shard jobs must NOT be required
(they legitimately skip and would auto-pass).
Sibling PR on the other repo uses the same branch `ci/e2e-label-gate`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Why
Two CI-observability gaps, both about **per-test/per-shard counts** that
the run-level metrics couldn't show:
1. **Flaky vs failed was conflated.** "Failed Shards by Module" counted
shard-*job* failures — but a shard that **failed then passed on rerun is
flaky, not failed**. A bare count told you nothing and misclassified
recovered flakes as failures.
2. **API had no test counts at all.** API `ci_test_runs` rows carried
only run health (conclusion/duration) — `tests_total` was NULL — so the
Test Counts dashboard had nothing for API.
## What
### Per-test flaky **and** failed → `ci_test_results`
- The **merge job** parses the merged `report.json` for tests with
`status="flaky"` (failed→passed on retry) or `status="unexpected"`
(failed after retries) via the new commented
**`.github/scripts/extract-flaky.js`**, emitting **title + module +
status**.
- **report_to_openobserve** posts **one row per such test** to
**`ci_test_results`**.
- Row schema: `{run_id, repo, suite, workflow, ingest_source, status
("flaky"|"failed"), module, test_title, test_file}` — powers accurate
per-module **flaky** *and* **failed** dashboards, no misclassification.
### Per-shard API test counts → `ci_test_counts` (+ run-doc total)
- Each API pytest job emits `--junitxml` and uploads an
**`api-junit__<category>__<leg>`** artifact (category = the test shard:
integration / query_agent / regression / … ENT also cli / vortex_* /
oss_in_ent / ent_top / ent_rbac).
- **report_to_openobserve** runs
**`.github/scripts/parse-api-junit.py`**, which reads the category per
artifact and **dedups config-leg reruns by test identity** (query_agent
opt true|false, regression flag combos) so the same suite isn't counted
twice.
- The deduped **total** folds into the run doc (`ci_test_runs`,
suite=api) so API shows in the same KPI/pie panels as UI; **per-category
rows** go to the new **`ci_test_counts`** stream for the per-shard
breakdown (the API analog of UI's `ci_test_modules`).
## Notes
- **Forward-only** — historical runs never stored junit / per-test names
(only counts), so no backfill; both streams accumulate from merge.
- All **non-fatal**: any parse/ingest error only warns, never fails the
workflow. `_timestamp` unset → ingest time.
- Pairs with the same-named branch on the other repo (OSS ⇄ ENT).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## What
Makes the per-run test dashboards **live**. Adds non-fatal
`report_to_openobserve` jobs that emit **one retry-aware run-summary doc
per run** to OpenObserve, following the metrics work merged earlier
(regression/docgen/council). Companion PR in the other repo mirrors
this.
**Scope (this PR):**
1. **`playwright.yml` + `api-testing.yml`** → new report jobs →
`ci_test_runs` (`suite=ui` / `suite=api`).
2. **`playwright_regression.yml`** → retry-aware reporting →
`ci_regression` (OSS: augments the existing job; ENT: adds the job — was
uninstrumented).
## How it works
- **Shared `.github/scripts/o2-run-report.sh`** (used by playwright/api)
derives the run's `conclusion` from job results, wall-clock from the
jobs API, and **sums cost across all attempts**. Regression jobs do the
same inline (they also emit shard/build/test fields).
- Emits: `run_id`, `run_attempt`, `retries`, `was_retried`,
`conclusion`, `suite`, `repo`, `branch`, `author`, `commit_sha`,
`pr_number`, `trigger`, `final_duration_sec`, `total_duration_sec`,
`retry_wasted_sec`.
- **Dedup model:** re-runs append one doc per attempt; dashboards keep
the **latest attempt per `run_id`** via `row_number() OVER (PARTITION BY
run_id ORDER BY run_attempt DESC)`. Retries become an *attribute*, never
duplicate rows; `total_duration_sec` on the kept row carries cumulative
compute.
- **Safe by construction:** `continue-on-error` + the never-fail poster
— can't fail/slow a test run; skips when no tests ran; **no-ops until
`O2_REPORTING_*` secrets are set**.
## Review
AI review addressed (mktemp payload, `GH_TOKEN` guard, non-negative
duration clamp). YAML + jq unit-tested + `actionlint` clean on the added
jobs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
- Adds a matrix strategy to the query agent SQL regression suite jobs in
both `api-testing.yml` and `query-agent.yml`
- Each job now runs with `ZO_FEATURE_SINGLE_NODE_OPTIMIZE_ENABLED=false`
and `ZO_FEATURE_SINGLE_NODE_OPTIMIZE_ENABLED=true`
- Verifies both DataFusion physical optimizer paths produce identical
query results
- Matrix legs run in parallel with `fail-fast: false`
- All artifact/log names include the matrix label to avoid collisions
## Test plan
- [ ] PR CI shows two `api_query_agent` matrix legs
(single_node_opt_false and single_node_opt_true), both pass
- [ ] `/query-agent` PR comment triggers both modes and posts per-mode
results comments
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## Summary
Fixes a test gap where `match_all` Tantivy queries could return wrong
results after a DataFusion upgrade without being caught by existing
assertions.
### Problem
After a DataFusion upgrade merge, the DataFusion access plan was no
longer used for `match_all` queries, causing Tantivy to return wrong
results. The existing tests only compared row counts against a DuckDB
oracle that emulates `match_all` with `LIKE '%term%'` — so wrong rows
that still matched the LIKE pattern passed undetected.
Additionally, `node/flush` does not generate Tantivy indexes
immediately. The default `ZO_MAX_FILE_RETENTION_TIME=600s` means
WAL→parquet→Tantivy conversion can lag, so FTS queries may fall back to
a non-Tantivy path during tests.
### Changes
1. **Content verification**
([conftest.py](tests/api-testing/tests/query_agent/conftest.py)): Every
hit from a `match_all` query that returns the `log` field is now checked
— each search term word must be present in `log`. A hard assertion fails
if Tantivy returns rows that don't contain the search terms.
2. **Tantivy readiness gate**
([test_2_parquet.py](tests/api-testing/tests/query_agent/test_2_parquet.py)):
After `node/flush`, the test now polls with a `match_all('warehouse')
LIMIT 1` query (up to 120s timeout) to confirm Tantivy indexes are built
before running FTS assertions. Previously only COUNT(*) was checked.
3. **Multi-word match_all emulation**
([compute_counts.py](tests/test-data/query-agent/compute_counts.py)):
`match_all('ACK batch')` now translates to `(log LIKE '%ACK%' AND log
LIKE '%batch%')` — per-token AND conditions matching Tantivy's
token-level search semantics, instead of the old phrase match `log LIKE
'%ACK batch%'`.
4. **Regenerated expected results** across all query JSON files.
## Test plan
- [ ] Run query agent test suite against an environment with Tantivy
enabled
- [ ] Verify `match_all` content assertions catch intentionally
corrupted results
- [ ] Confirm post-flush Tantivy readiness poll works within the timeout
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Huaijin <haohuaijin@gmail.com>
## Summary
- Adds **239 new tests** covering the vortex columnar file format across
78 scenarios from the test plan
- **`tests/api-testing/tests/vortex/`** — 9 test modules (83 tests):
format conversion, schema evolution, data integrity, FTS, downsampling,
API format awareness, server config, and performance/stress
- **`tests/api-testing/tests/query_agent/test_3_vortex.py`** — 156
tests: the full 155-query SQL correctness suite + scenario 39
(self-join) run post-flush against vortex files
- **`api-testing.yml`** — adds `--ignore=tests/vortex` to the OSS
default-set run; vortex tests require the enterprise binary with
`ZO_FILE_FORMAT=vortex` and are gated to enterprise CI only
## Test plan coverage
| Module | Scenarios |
|--------|-----------|
| `test_format_conversion.py` | 1–13, 7, 8, 9, 10 |
| `test_schema_evolution.py` | 14–25 |
| `test_data_integrity.py` | 55–62 |
| `test_fts.py` | 51–54 |
| `test_downsampling.py` | 47–50 |
| `test_api_format.py` | 71–73 |
| `test_config.py` | 66–67 |
| `test_performance.py` | 74–78 |
| `test_3_vortex.py` | 26–46 (155 queries + JOIN) |
## Enterprise CI
Two dedicated jobs added in the companion `openobserve/o2-enterprise` PR
on the same branch:
- **`api_query_agent_vortex`** — runs `test_3_vortex.py` with
`ZO_FILE_FORMAT=vortex`
- **`api_vortex_format_tests`** — runs the full `tests/vortex/` suite
with 4-phase server lifecycle (parquet→vortex→parquet→vortex+compaction)
OSS CI is unchanged except for the ignore rule — no vortex tests run
against the OSS binary.
## Test checklist
- [ ] `tests/vortex/` collects cleanly: `83 tests collected`
- [ ] `tests/query_agent/test_3_vortex.py` collects cleanly: `156 tests
collected`
- [ ] Enterprise CI `api_query_agent_vortex` job passes
- [ ] Enterprise CI `api_vortex_format_tests` job passes
🤖 Generated with [Claude Code](https://claude.ai/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- ENT CI runners are slower than local/OSS runners — the
`ingest_query_agent_data` fixture was hitting the 120s timeout before
1550 records became searchable, causing all 310 query_agent tests to
fail at setup
- Bumped `wait_until` timeout to 300s
## Test plan
- [ ] OSS `api_query_agent_tests` passes as before
- [ ] ENT `api_query_agent_tests` no longer hits WaitTimeout on ingest
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- [x] Bulk ingestion panic
- [x] Create function panic
- [x] Dashboards/Alerts/Incidents panic when list data
- [x] Match http response error code
- [x] Skip some endpoint that fuzz can't generate data
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Summary
Comprehensive rework of the `tests/api-testing/` suite. 30+ commits
implementing all 6 phases of the [revamp
plan](../../tests/ui-testing/MD_Files/api%20tests%20revamp/API_TESTS_REVAMP.md).
**Before → After:**
| | Baseline | Final | Δ |
|---|---|---|---|
| passed | 421 | **443** | **+22 ✓** |
| failed | 3 | **2** | -1 (only same pre-existing) |
| skipped | 53 | 39 | -14 (unskipped dead tests now run) |
| xfailed | 1 | 2 | +1 (new visible backend-bug signal) |
| runtime | 588s | **387s** | **-201s (34% faster)** |
**Zero tests that passed in baseline are now failing.** The 2 remaining
failures are the exact pre-existing ones
(`test_enrichment_table_join.py` backend panic + `test_workflow.py` SMTP
env-miss).
## What's in the diff
- **New `support/` framework** — `OpenObserveClient`, endpoint wrappers
(`alerts`, `dashboards`, `streams`, `search`, etc.), `wait_until()`,
payload factories, pytest fixtures, SSE parser. 14 framework selfcheck
tests.
- **16 of 16 OLD-tier test files rewritten** — using the new framework,
unique-suffix names, body validation, `wait_until` instead of
`time.sleep`, parametrize where appropriate.
- **29 files moved into 12 feature folders** mirroring the UI suite
layout
(`tests/{alerts,dashboards,ingest,orgs,pipelines,regression,rum,search,streams,users,workflow}/`).
- **−6,500 LOC net** (copy-paste + dead code removal; new framework adds
back ~1,500).
- **Ruff config** in pyproject.toml: T201 ban on `print()`, PT pytest
rules, B bugbear, UP pyupgrade.
## CI workflow changes
- New job `api_regression_tests`: `strategy.matrix` over the 4 env-flag
combinations (UTF8_VIEW × JOIN_MATCH_ONE) — now runs in parallel instead
of as 4 sequential server reboots in the main job.
- New job `api_fuzz_tests`: schemathesis OpenAPI-driven fuzz against
`/api-doc/openapi.json` (214 endpoints tested). Informational
(`continue-on-error: true`). Each run surfaces **7–10 distinct 5xx
panics** (e.g., empty `sql` body returning 500 instead of 400, traces
endpoints crashing on unescaped path params that get interpolated into
SQL). Filed to backend team for separate triage.
- Server `o2_*.log` files uploaded as artifacts on failure (was:
silently lost).
- `api_tests_summary` now requires BOTH integration + regression to
pass.
### CI infrastructure fixes
A short hardening pass landed after the initial CI run surfaced several
issues:
- **pytest-sugar count inflation** — dropped `--force-sugar` from both
pytest invocations. The flag double-registered sugar on top of the
auto-loaded entry point, inflating the CI test counter ~3x (display
showed 1364 when collection was 486). Counter is now honest.
- **schemathesis 4.x flag migration** — `--base-url` → `-u`,
`--hypothesis-max-examples` → `-n`, `--hypothesis-deadline` →
`--request-timeout`, `--data-generation-method` → `-m`, dropped
`--auth-type basic`. The fuzz job was failing immediately on flag-parse
before this.
- **Regression matrix binary path** — used `${{ github.workspace
}}/target/debug/openobserve` so the absolute path resolves correctly
under `working-directory: tests/api-testing/` (was: every matrix job
failed with "No such file or directory").
- **`permissions: contents: read`** added to all 4 workflow jobs —
closes CodeQL alert #369.
- **Removed redundant server boot** in `api_integration_tests` (leftover
from main's structure after the regression matrix split; was silently
failing to bind to :5080 and getting masked by the first server).
## Backend bugs surfaced (for backend team to track)
The rework uncovered and visibly flagged these as `xfail`/documented
test comments:
1. `DELETE /api/organizations/{id}` returns 404 unconditionally even for
just-created orgs — endpoint appears unimplemented.
2. `test_dashboard_missing_required_fields` triggers backend panic
(returns 500 instead of 400/422).
3. JS `getEnrichmentTableData()` restricted to `_meta` org AND not
exposed in the `/functions/test` endpoint runtime — `ReferenceError`
even with explicit `enrichment_tables: [...]`.
4. Bulk API silently sanitizes invalid `_index` values (`****` → `_`)
without flagging — user data lands in unexpected index.
5. `AVG` of a single value drifts by ~1e-14 (likely float reduction
order; documented tolerance in `test_advanced_sql.py`).
6. Org name validation rejects hyphens (only
`A-Z/a-z/0-9/spaces/underscores`) — not in spec.
Plus 7–10 additional 5xx panics surfaced per fuzz run (input validation
gaps — separate triage list in fuzz job logs).
## Test bugs uncovered + fixed
- `test_pipeline.py`: had two function definitions named
`test_pipeline_creation_and_action_with_schedule` — Python's last-wins
meant the first test's body never executed for years.
- `test_alertsV2.py`: `assert alert_disabled == "False"` was indented
inside an `else:` branch that never executed because the response is
always a dict; AND `enabled` is a bool, not the string `"False"`. Two
layers of broken.
- `test_bulk.py`: variable typo (`valid_bulk_payload` reused on the POST
call) made the invalid-index test silently duplicate the valid case.
- `test_organisations.py`: `test_create_organization` used pre-pytest-4
`yield`-style fixtures — silently ignored by pytest 8.
- `test_js_result_array.py`: 8 tests skipped with `"Temporarily disabled
- failing test"` — the actual cause was that JS functions are restricted
to `_meta` org. One-line fix unskipped all 8.
- `test_fuzzy.py`: 4 tests `@pytest.mark.skip`-ed because they used a
1-minute time window against older static test data → 0 hits → flake →
skip. Wider window unskips them.
- 547 LOC of zombie commented-out test blocks deleted (15 commented `#
def test_` definitions, including one orphan case where the `def` was
commented but the indented body wasn't).
## Folder layout
```
tests/api-testing/
├── support/ # NEW framework
│ ├── client.py # OpenObserveClient
│ ├── endpoints/ # 7 per-resource wrappers
│ ├── factories.py # unique_name, time_window, *_payload
│ ├── wait.py # wait_until + WaitTimeout
│ ├── sse.py # read_sse_response
│ └── fixtures.py # client, temp_user_email, temp_stream_name, ...
├── helpers/ # Non-test shared code (was scattered in tests/)
│ ├── sourcemap_helpers.py
│ └── workflow/ # workflow chain + pages/ live here now
├── fixtures/
│ └── rum/ # Standalone RUM ingest scripts (was in tests/)
└── tests/
├── alerts/ # 3 files
├── dashboards/ # 4 files
├── ingest/ # 4 files
├── orgs/ # 2 files
├── pipelines/ # 10 files
├── regression/ # 2 files (env-flag matrix targets)
├── rum/ # 3 files
├── search/ # 8 files
├── streams/ # 1 file
├── users/ # 1 file
├── workflow/ # 3 files
├── conftest.py
└── test_framework_selfcheck.py
```
## Test plan
- [x] `rye run pytest --collect-only` — 486 tests discovered
- [x] Full default-set run (`--ignore=tests/regression`) — 443 pass / 2
fail / 39 skip / 2 xfail in 387s
- [x] `rye run pytest -m framework` — 14/14 selfcheck tests pass
- [x] `rye run ruff check support/ tests/test_framework_selfcheck.py` —
clean
- [x] CI run on this PR — `api_integration_tests` + all 4
`api_regression_tests` matrix jobs green
- [x] CI run — `api_fuzz_tests` runs cleanly (informational; surfaces
7–10 unique 5xx panics per run)
## Note on test count
The `test_streaming.py` rewrite folded 7 "tests" (`test_ingest_data`,
`test_ingest_join`, `test_disable_streaming`,
`test_update_max_query_range`, `test_enable_streaming`,
`test_delete_stream`, `test_delete_stream_join`) into pytest fixtures.
Each only checked `status_code == 200` of a setup/teardown operation —
they were imperative steps masquerading as tests. The fixtures still
perform the same operations and assert their success; they're just no
longer counted as discrete pytest items. **Functional coverage is
identical.**
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add merge_group event trigger to all required CI workflows so they run
when PRs enter the merge queue. Fix github.event.pull_request references
in dispatch-event-enterprise.yml to handle merge_group context.
Add comprehensive API tests for logs-to-logs joins with asymmetric data
sizes:
- 29 test cases covering INNER/LEFT/RIGHT/FULL joins, aggregations,
filters
- Subquery tests with IN/NOT IN, substr, length, CONCAT functions
- Tests run with both ZO_FEATURE_JOIN_MATCH_ONE_ENABLED=true and false
- Test data: logs_small.json (10 records) and logs_large.json (50
records)
---------
Co-authored-by: Claude <noreply@anthropic.com>
Shrinath Rao
dependabot[bot]
Hengfei Yang
Neha P
Ashish Kolheandgithub-actions[bot]
Vamsi Karthik R
Loakesh Indiran
Thibaud-Vdb
Prabhat Sharma
udev
Yashodhan JoshiandYashodhan Joshi
Subhra264
Simha G
