Files
ktx-akshay 33c6e9e0e1 test(dashboards): cover CTE/subquery match_all queries through the panel search path (#13838)
## What

First test coverage for the dashboard **panel** endpoints and the
**dashboard
search path**, running CTE/subquery `match_all` queries as real
dashboard panels.

Nothing in the repo previously touched either:
`grep -rn "/panels\|search_type=dashboards" tests/api-testing/` returned
zero hits.

## Why

Panels query `_search?type=logs&search_type=dashboards&dashboard_id=…` —
a
different route into the engine than the Logs page (`search_type=ui`).
The
`match_all` + FST projection rewrite fixed in #13808 runs inside it, so
a query
that passes via Logs can still regress on a dashboard.

The rewrite has to preserve the projection of the block the predicate
sits in.
When that block is the whole statement, the projection is obvious. When
it's a
CTE body feeding a JOIN, a window, or a four-branch UNION ALL, it has to
preserve
the *CTE's* output schema while the outer query reads different columns
from it —
which is where it went wrong.

## Coverage

**71 distinct queries across 29 combination families**, all with
`match_all`
inside a `WITH …` body or a derived table:

| Axis | Variants |
|---|---|
| Nesting depth | 1, 2, 3, and 5-level CTE chains |
| Nesting kind | CTEs and derived tables (and one query using both) |
| UNION ALL | 2, 3, and 4 separate `match_all` branches |
| JOIN | CTE⋈CTE, derived⋈derived, cross-stream |
| Window | `ROW_NUMBER`, `RANK`, `DENSE_RANK`, partitioned aggregates |
| Negation | `NOT match_all`, and mixed with positive `match_all` |
| Combined with | `COUNT(DISTINCT)`, `CASE`, `HAVING`, `histogram()` |

Each case creates a real panel, runs it via the dashboard search path,
then
asserts row count against the DuckDB oracle and that no expected column
was
dropped from the projection. A further 30 cases run the same SQL through
**both**
Logs and dashboard and diff the results — that's what catches a rewrite
firing on
only one path.

Queries are reused from the existing query-agent corpus rather than
duplicated,
so the two suites can't drift: query-agent verifies the values via the
Logs path,
this verifies the same SQL survives the dashboard path.

Cases are **discovered by classifying the SQL**, not from a hand-written
ID list,
so new combinations added to the corpus are picked up automatically.
Test ids
name the shape (`Q1022-cte1_union_cdist_4ma`), so a failure identifies
the
combination without opening the corpus.

Plus 14 tests for the panel endpoints themselves — AddPanel /
UpdatePanel /
DeletePanel, contract + CRUD lifecycle + 7 error paths derived from
`DashboardError` in `src/core/src/http.rs`.

## Changes

| File | |
|---|---|
| `tests/dashboards/test_dashboard_cte_panels.py` | new — 103 tests |
| `tests/dashboards/test_dashboard_panels.py` | new — 14 tests |
| `tests/dashboards/conftest.py` | new — shared fixtures |
| `support/panel_queries.py` | new — discovery + panel helpers |
| `support/endpoints/dashboards.py` | +78, additive |
| `support/endpoints/search.py` | +36, additive |
| `support/factories.py` | +73, additive |

The three shared-file edits are **187 added lines, 0 modified** —
nothing
existing can break.

## Results

168 tests in tests/api-testing/tests/dashboards/ (117 new + 51
pre-existing)
166 passed, 1 skipped, 1 xfailed     # full nested-query corpus present
74 passed, 2 skipped, 1 xfailed     # current corpus on main


Repeated 2× and 4× respectively, no flakes.

Discovery scales with the corpus: on main it finds 8 nested `match_all`
cases,
and 71 once the larger sentinel set is present. Cases that aren't in the
corpus
skip with an explanatory message instead of failing, so this is safe to
merge in
either order.
2026-08-17 09:47:17 +00:00

234 lines
7.0 KiB
Python

"""Payload factory functions for API requests.
Each factory returns a dict that can be passed to `client.<resource>.create(...)`.
Sensible defaults are merged with caller-supplied overrides via `**kwargs`.
Pattern: small functions, named per the API resource. No classes — these are
just typed dict builders.
"""
from __future__ import annotations
import uuid
from datetime import datetime, timedelta, UTC
from typing import Any
def _ms_epoch(dt: datetime) -> int:
"""OpenObserve time fields are microseconds-since-epoch as int."""
return int(dt.timestamp() * 1_000_000)
def time_window(*, minutes: int = 15) -> tuple[int, int]:
"""Return (start, end) microsecond-epoch ints for the last `minutes` minutes."""
now = datetime.now(UTC)
end = _ms_epoch(now)
start = _ms_epoch(now - timedelta(minutes=minutes))
return start, end
def unique_name(prefix: str = "pytest") -> str:
"""Generate a unique, lowercase, alphanumeric resource name.
Used to prevent cross-test resource collisions. The original suite had
fixed names like `pytests@gmail.com` that caused cascading failures and
blocked parallel runs.
"""
return f"{prefix}_{uuid.uuid4().hex[:8]}"
def unique_email(prefix: str = "pytest") -> str:
"""Generate a unique test email (using .invalid TLD per RFC 2606)."""
return f"{prefix}_{uuid.uuid4().hex[:8]}@test.invalid"
# ----- payload factories -----
def search_payload(
sql: str,
*,
start_time: int | None = None,
end_time: int | None = None,
size: int = 100,
from_: int = 0,
quick_mode: bool = False,
track_total_hits: bool = False,
) -> dict[str, Any]:
"""SQL search request body for POST /api/{org}/_search.
Note: `track_total_hits` defaults to **False**. When True, OO switches
to histogram-aggregate mode and returns `{"zo_sql_num": <count>}` instead
of raw hits, which surprises callers expecting individual records.
Callers that genuinely want the aggregate count can pass True explicitly.
"""
if start_time is None or end_time is None:
s, e = time_window(minutes=15)
start_time = start_time if start_time is not None else s
end_time = end_time if end_time is not None else e
return {
"query": {
"sql": sql,
"start_time": start_time,
"end_time": end_time,
"from": from_,
"size": size,
"quick_mode": quick_mode,
"track_total_hits": track_total_hits,
}
}
def user_payload(
*,
email: str | None = None,
password: str = "Complexpass#123",
first_name: str = "Pytest",
last_name: str = "User",
role: str = "admin",
organization: str = "default",
) -> dict[str, Any]:
"""User create/update body."""
return {
"email": email or unique_email(),
"password": password,
"first_name": first_name,
"last_name": last_name,
"role": role,
"organization": organization,
}
def dashboard_payload(
*,
title: str | None = None,
description: str = "",
folder_id: str = "default",
version: int = 8,
tabs: list | None = None,
) -> dict[str, Any]:
"""Dashboard create body."""
return {
"version": version,
"title": title or unique_name("dashboard"),
"description": description,
"folder_id": folder_id,
"tabs": tabs if tabs is not None else [],
}
def axis_item(alias: str, *, label: str | None = None, column: str | None = None) -> dict[str, Any]:
"""One entry for a panel's `x` / `y` / `breakdown` array.
`label` is a required String in the v8 schema (not optional) — omitting it
is a 422. For `customQuery` panels the `alias` must match the SELECT alias
verbatim, or the axis renders blank.
"""
return {
"label": label if label is not None else alias,
"alias": alias,
"column": column if column is not None else alias,
"color": None,
}
def panel_payload(
*,
stream: str,
sql: str,
x_alias: str,
y_alias: str,
panel_id: str | None = None,
title: str | None = None,
description: str = "",
panel_type: str = "bar",
stream_type: str = "logs",
layout: dict[str, int] | None = None,
) -> dict[str, Any]:
"""A v8 `Panel` carrying hand-written SQL, for the AddPanel/UpdatePanel APIs.
`customQuery` is hard-coded True: with False, OO discards `sql` and rebuilds
the query from the field config, which silently defeats any test asserting on
the SQL it supplied.
Note `config` (PanelConfig) is snake_case while `fields`/`queries` are
camelCase — the v8 schema genuinely mixes both.
"""
return {
"id": panel_id or unique_name("panel"),
"type": panel_type,
"title": title or unique_name("Panel"),
"description": description,
"config": {
"show_legends": True,
"legends_position": None,
},
"queryType": "sql",
"queries": [
{
"query": sql,
"vrlFunctionQuery": None,
"customQuery": True,
"fields": {
"stream": stream,
"stream_type": stream_type,
"x": [axis_item(x_alias)],
"y": [axis_item(y_alias)],
"z": [],
# A bare [] is rejected: PanelFilter is an untagged enum and
# the empty list matches neither variant. It must be a group.
"filter": {
"filterType": "group",
"logicalOperator": "AND",
"conditions": [],
},
},
"config": {"promql_legend": ""},
}
],
"layout": layout if layout is not None else {"x": 0, "y": 0, "w": 24, "h": 9, "i": 1},
}
def alert_template_payload(
*,
name: str | None = None,
body: str | None = None,
type_: str = "http",
title: str = "",
) -> dict[str, Any]:
"""Alert template create body."""
return {
"name": name or unique_name("template"),
"body": body or '{"text": "alert {alert_name} on {stream_name} active"}',
"type": type_,
"title": title,
}
def webhook_destination_payload(
*,
name: str | None = None,
url: str = "http://localhost:0/sink", # test default — override per test
method: str = "post",
template: str,
headers: dict | None = None,
skip_tls_verify: bool = False,
) -> dict[str, Any]:
"""Webhook destination create body."""
return {
"name": name or unique_name("dest"),
"url": url,
"method": method,
"template": template,
"headers": headers or {},
"skip_tls_verify": skip_tls_verify,
}
def folder_payload(*, name: str | None = None, description: str = "") -> dict[str, Any]:
"""Folder create body (alerts or dashboards)."""
return {
"name": name or unique_name("folder"),
"description": description,
}