starred/openobserve
1
0
Fork 0
mirror of https://github.com/openobserve/openobserve.git synced 2026-11-04 16:09:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
8,007 Commits 635 Branches 244 Tags
ee690179ef211fcde83575a2aaec9be77cddfe20
Commit Graph
13 Commits
Author SHA1 Message Date
Neha PandClaude Opus 4.7 8a4f7ea23b test(query-agent): add 70 match_all sentinel queries for FST rewrite path coverage (#13824) 
## Summary

Adds Q1003–Q1072 (70 queries) to
`tests/test-data/query-agent/queries/full_text_search.json`. All
exercise the `match_all` + FST projection rewrite path fixed in #13808,
under diverse plan shapes.

Structural patterns mirror real production customer queries (fan-out
UNION ALL, multi-CTE, CSE-triggering scalar wrappers on the FTS field)
but use **only test-fixture fields and values** — no customer
identifiers.

## Shape coverage

| Family | Range | Count | What it exercises |
|---|---|---|---|
| A. Subquery + `match_all` + GROUP BY | Q1003–Q1012 | 10 | Simple
inline-view aggregations |
| B. CTE with fan-out UNION ALL, `match_all` per branch | Q1013–Q1027 |
15 | Dominant customer pattern (multi-branch UNION + per-branch FTS +
LIKE/regex + outer aggregate) |
| C. CTE + scalar wrapper on `log`
(trim/upper/split_part/regexp_replace/substring/concat/length) +
`match_all` | Q1028–Q1037 | 10 | **CSE-trigger sentinels** — the
wrappers cause DataFusion CSE to insert the intermediate
`ProjectionExec` that #13808 fixes |
| D. Deep multi-CTE + `match_all` + regex/LIKE/CASE combos | Q1038–Q1047
| 10 | Deep plan trees, many projection candidates |
| E. Edge cases (DISTINCT, HAVING, CASE, IN-list) | Q1048–Q1052 | 5 |
Corner combinations |
| F. `match_all` inside JOINs (self-join, LEFT JOIN, cross-stream) |
Q1053–Q1058 | 6 | JOIN builders inject their own projections |
| G. `match_all` + window functions (ROW_NUMBER, RANK, DENSE_RANK,
SUM/COUNT OVER) | Q1059–Q1063 | 5 | Window planning inserts projections
above filter |
| H. Negated `match_all` (NOT, AND NOT, UNION of pos/neg) | Q1064–Q1066
| 3 | Rewriter's negation code path |
| I. Multiple `match_all` in same query (AND, OR, nested) | Q1067–Q1069
| 3 | Two FST fields injected simultaneously |
| J. `histogram` + `match_all` time-series | Q1070–Q1072 | 3 | Common
customer pattern; histogram optimizer + FST rewriter together |

## Verification

- All 70 queries have DuckDB oracle results auto-populated via
`compute_counts.py`.
- Two initial cross-stream JOIN queries (Q1055, Q1058) were rewritten to
disambiguate the FTS field between streams — now they scope `match_all`
inside a per-stream CTE before joining.
- Audit for customer-token leakage (pharma, tomcat, o2_test, usgm,
storenumber, tagname, cash, credit_card, etc.) returns **zero hits**
across Q1003–Q1072.

## Implementation notes

- Added generator script
[tests/test-data/query-agent/gen_q1003_q1052.py](tests/test-data/query-agent/gen_q1003_q1052.py)
alongside existing `gen_q201_q400.py` / `gen_q511_q556.py` convention.
Deterministic regeneration.
- Wide JSON churn across the other 12 category files is
`compute_counts.py` re-baselining expected results against a fresh
`BASE_TS` (`base_ts_override.json` is untracked); tuple-order shuffles
are harmless under `results_mode: rowsort`.
- Coverage is most meaningful under the `remove_filter_with_index_false`
CI leg (added in #13810), where `match_all` actually reaches
`RewriteMatchPhysical`. Under the default flag (`true`), the index
optimizer removes `match_all` before the rewriter runs, masking the bug
path.

## Test plan

- [ ] `api_query_agent / single_node_opt_true` — green
- [ ] `api_query_agent / single_node_opt_false` — green
- [ ] `api_query_agent / remove_filter_with_index_false` — green (this
is the leg that actually validates the rewriter under all 70 shapes)
- [ ] Q1003–Q1072 all pass on both memtable and parquet phases

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-08-17 06:42:06 +00:00
Neha PandClaude Opus 4.7 7fbce61fd1 test: add Q1001 + Q1002 sentinels for match_all/CSE + widen CI matrix (#13810) 
Two regression sentinels for the FST projection rewrite path fixed in
openobserve/openobserve#13808 (currently in the merge queue on
branch-v0.40.0 and awaiting the main-branch counterpart):

  Q1001 — CTE with trim(log) alias, match_all in WHERE, column-pruned
  outer SELECT. Mirrors the shape of the PR's own unit test
  (test_cse_projection_schema_propagation) — CSE inserts a
  ProjectionExec between filter and datasource; a regression in the
  rewriter's f_down handling would surface here.

  Q1002 — five-CTE pipeline (filtered → in_shop → scoped → normalized
  → cleaned) mixing match_all with LIKE, split_part, regexp_replace ID
  normalization, and GROUP BY aggregation. Higher structural pressure:
  more intermediate ProjectionExec nodes and more chances for a stale
  column index to leak through.

Uses only test-fixture fields (log, resource_path, page_url,
http_method); no customer identifiers.

Also adds a third leg to the query-agent CI matrix that runs with
ZO_FEATURE_QUERY_REMOVE_FILTER_WITH_INDEX=false — exercises the
non-index-filter path that the match_all projection rewriter also
touches.

The wide JSON churn across the other query files is compute_counts.py
re-baselining expected results against a fresh BASE_TS
(base_ts_override.json is untracked); tuple-order shuffles are harmless
under results_mode: rowsort.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-08-13 09:46:15 +00:00
HuaijinNeha PClaude Opus 4.7
 		7f647109fc fix: preserve projection schema when rewriting match_all (#13808) 
## Summary

Forward-port of #13798 to `main`.

- rebuild `ProjectionExec` expressions against the rewritten child
schema before DataFusion validates the projection
- carry full-text search fields through intermediate projections and
preserve the original `FilterExec` output schema
- reject a full-text search field that a projection already exposes as
an alias for a different expression, instead of silently searching the
wrong data
- add a regression for a projection sitting between the filter and the
datasource

## Root cause

`RewriteMatchPhysical` expanded the `NewEmptyExec` projection with
full-text search fields. When an optimizer rule (CSE, for example) had
inserted an intermediate `ProjectionExec`, adding the FST field changed
the child column positions while the projection still referenced its
columns by their old indices, so DataFusion rejected the plan.

The old `filter_projection` was also computed as positions inside
`parquet_projection`, i.e. in the datasource output space. That is only
valid when the filter reads the datasource directly. It is now mapped by
name against the actual filter input schema, and the field order of
`filter.schema()` is preserved so the operators above the filter keep
reading the same columns.

Two latent issues are fixed along the way:

- a scan with `projection == None` (all columns) was narrowed down to
only the FST fields
- `schema.index_of(...).unwrap()` in the datasource rewrite panicked
instead of returning a plan error

## Differences from #13798

`main` is on DataFusion 54 and builds the filter through
`FilterExecBuilder`, so step 5 uses `apply_projection` and keeps
`with_fetch`.

The regression test also differs. The CTE query used in #13798 no longer
produces an intermediate projection on DataFusion 54 — CSE does not
extract `btrim(log)` there — so the test would be vacuous.
`test_projection_between_filter_and_scan` builds the offending plan
shape directly instead, which does not depend on optimizer internals.

## Verification

- `cargo test -p search --no-default-features --lib
datafusion::optimizer::physical_optimizer::rewrite_match` — 7 passed, 0
failed
- `test_projection_between_filter_and_scan` fails on `main` without the
fix (`SchemaError: Unable to get field named "__common_expr_1"`)

---------

Co-authored-by: Neha P <neha@openobserve.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-08-13 07:53:44 +00:00
Neha PandClaude Opus 4.7 b0f4b0e46f test: expand query-agent suite from 672 to 1,000 queries (#12856) 
## Summary

Expands the query-agent regression test suite from **672 → 1,000
queries** by adding 328 complex queries (Q672–Q999) across all 13
categories.

## What changed

- **`data_gen.py`**: `NUM_QUERIES` bumped from 675 → 1000 so
`build_dataset()` generates records for all query IDs
- **13 query JSON files**: 328 new queries appended, with
DuckDB-computed expected results via `compute_counts.py`
- **`gen_q672_q999.py`**: new generation script (idempotent — re-running
skips all existing queries)

## New query patterns added

Targets DataFusion-upgrade-fragile areas identified from past breakage:

| Pattern | Examples |
|---|---|
| 3-stream JOINs + correlated EXISTS | Q672–Q677 |
| FULL OUTER JOIN between streams | Q693–Q696 |
| Window + DISTINCT (FIRST_VALUE, LAST_VALUE, NTH_VALUE) | Q717–Q730 |
| Named WINDOW clauses | Q731–Q736 |
| LAG/LEAD with CASE trend detection | Q742–Q751 |
| match_all + GROUP BY + HAVING | Q792–Q796 |
| INTERSECT/EXCEPT with CTEs | Q822–Q849 |
| Deep CTE chains (3+ CTEs, self-joins) | Q757–Q791 |
| Correlated scalar subqueries in SELECT | Q769–Q776 |
| Histogram with JOIN between streams | Q901–Q905 |

## Issues fixed

- **4 duplicate IDs** (Q545–Q548) existed in both `cte_subquery.json`
and `full_text_search.json` — renumbered FTS copies to available gaps
(Q025, Q611, Q631, Q1000)
- **27 queries** using OO-specific functions (match_all, spath,
regexp_like, window-inside-aggregate) annotated with `skip_sqllogictest:
true`

## Final breakdown

| Category | Before | After |
|---|---|---|
| cte_subquery | 89 | 124 |
| aggregation | 71 | 99 |
| string_functions | 70 | 92 |
| combined | 73 | 88 |
| full_text_search | 58 | 88 |
| window | 48 | 88 |
| basic_select | 71 | 86 |
| histogram | 57 | 79 |
| union | 34 | 62 |
| cross_stream | 16 | 61 |
| date_time | 37 | 60 |
| pagination | 22 | 42 |
| math_functions | 26 | 31 |
| **Total** | **672** | **1,000** |

- **956** queries use sqllogictest mode (DuckDB-verified results)
- **44** use legacy assertion mode (OO-specific functions, histograms)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-24 14:24:11 +00:00
Neha PandClaude Opus 4.7 494025f83b test: add ~100 query-agent queries covering re_match, production patterns, deep nesting, edge cases (#12735) 
## Summary
- Expands the query agent test suite from **574 → 675 queries** across
13 category files
- **20 re_match queries** — testing `\d`, `[abc]`, `^anchor`, `a|b`,
coercion, boolean mode, COALESCE+CASE patterns found 224x in production
- **19 production-inspired queries** —
histogram+re_match+CASE+percentage, COALESCE chains,
re_match+re_not_match combined
- **19 deep-nesting queries** — INTERSECT, EXCEPT, subquery-in-HAVING,
double-NOT-EXISTS, 2-level correlated subqueries
- **15 edge-case queries** — empty results, division-by-zero via NULLIF,
STDDEV of single row, NULL grouping, BETWEEN edge
- **10 window-depth queries** — multiple window functions, WINDOW clause
reuse, NTH_VALUE, NTILE, CUME_DIST
- **5 histogram+re_match queries** — covering production pattern of
histogram + filtered metric extraction
- **5 FTS combo queries** — match_all +
histogram/re_match/window/aggregation
- **5 aggregation edge queries** — multiple FILTER clauses, STRING_AGG
with ORDER BY, HAVING complexity
- Expected values regenerated by DuckDB oracle

## Test plan
- [ ] CI passes (memtable + parquet phases)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-17 12:51:13 +00:00
Neha PClaude Opus 4.7Huaijin
 		4f3c0fcba3 fix(tests): verify match_all FTS result correctness and enforce Tantivy readiness (#12717) 
## Summary

Fixes a test gap where `match_all` Tantivy queries could return wrong
results after a DataFusion upgrade without being caught by existing
assertions.

### Problem

After a DataFusion upgrade merge, the DataFusion access plan was no
longer used for `match_all` queries, causing Tantivy to return wrong
results. The existing tests only compared row counts against a DuckDB
oracle that emulates `match_all` with `LIKE '%term%'` — so wrong rows
that still matched the LIKE pattern passed undetected.

Additionally, `node/flush` does not generate Tantivy indexes
immediately. The default `ZO_MAX_FILE_RETENTION_TIME=600s` means
WAL→parquet→Tantivy conversion can lag, so FTS queries may fall back to
a non-Tantivy path during tests.

### Changes

1. **Content verification**
([conftest.py](tests/api-testing/tests/query_agent/conftest.py)): Every
hit from a `match_all` query that returns the `log` field is now checked
— each search term word must be present in `log`. A hard assertion fails
if Tantivy returns rows that don't contain the search terms.

2. **Tantivy readiness gate**
([test_2_parquet.py](tests/api-testing/tests/query_agent/test_2_parquet.py)):
After `node/flush`, the test now polls with a `match_all('warehouse')
LIMIT 1` query (up to 120s timeout) to confirm Tantivy indexes are built
before running FTS assertions. Previously only COUNT(*) was checked.

3. **Multi-word match_all emulation**
([compute_counts.py](tests/test-data/query-agent/compute_counts.py)):
`match_all('ACK batch')` now translates to `(log LIKE '%ACK%' AND log
LIKE '%batch%')` — per-token AND conditions matching Tantivy's
token-level search semantics, instead of the old phrase match `log LIKE
'%ACK batch%'`.

4. **Regenerated expected results** across all query JSON files.

## Test plan

- [ ] Run query agent test suite against an environment with Tantivy
enabled
- [ ] Verify `match_all` content assertions catch intentionally
corrupted results
- [ ] Confirm post-flush Tantivy readiness poll works within the timeout

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Huaijin <haohuaijin@gmail.com>
 2026-06-17 08:30:33 +00:00
Neha PandClaude Opus 4.7 1484f920a9 test: expand query agent suite to 570 queries with production SQL patterns (#12534) 
## Summary

- Added **46 new queries** (Q511-Q556) to the query agent regression
suite, bringing the total from 524 to **570 unique queries**
- Mined from real production SQL patterns (june7.sql) and adapted to use
existing test data fields
- Fixed **15 pre-existing duplicate IDs** (Q226-Q240 existed in both
aggregation.json and basic_select.json — renumbered to Q557-Q571)

## New query distribution

| Category | New | Description |
|---|---|---|
| string_functions | +10 | spath() + str_match_ignore_case, split_part()
+ chr(), multi-COALESCE chains, CAST AS VARCHAR |
| histogram | +7 | 30-second/min intervals, COUNT FILTER, SUM(CASE
WHEN), NULL breakdowns |
| date_time | +6 | arrow_cast(), date_trunc(), to_timestamp_micros(),
timestamp arithmetic, LAG for gaps |
| combined | +6 | count(distinct) in CASE WHEN, re_match + re_not_match
combo, FILTER in aggregation, HAVING percentages |
| full_text_search | +5 | re_match + re_not_match, regexp_like,
match_all(), str_match_ignore_case with COALESCE |
| cte_subquery | +4 | CTE with spath, CTE with COALESCE fallbacks,
correlated subquery |
| aggregation | +2 | IS NOT NULL exploration patterns, multi-column
GROUP BY |
| window | +2 | NTILE with COALESCE PARTITION, LAG with delta
calculation |
| basic_select | +2 | Complex COALESCE + CAST chains, NULLS LAST
ordering |
| cross_stream | +1 | LEFT JOIN on dual keys across streams |
| union | +1 | UNION ALL with different metric types |

## New OO-specific functions covered

Four functions previously had **zero test coverage**:
- `chr()` — 1 query (string splitting by character code)
- `arrow_cast()` — 1 query (DataFusion timestamp arithmetic)
- `date_trunc()` — 1 query (minute-level timestamp bucketing)
- `to_timestamp_micros()` — 1 query (microsecond-to-timestamp
conversion)

Plus expanded coverage of rare functions: `spath()` (+3), `split_part()`
(+3).

## Test modes

- **38/46** new queries have full sqllogictest comparison (DuckDB
computed expected results)
- **8/46** use legacy assertions for OO-specific functions that DuckDB
cannot replicate (spath, arrow_cast, date_trunc, to_timestamp_micros)
- Overall suite: **533** sqllogictest, **37** legacy

## Changes

| File | Change |
|---|---|
| `data_gen.py` | `NUM_QUERIES` 520 → 575 |
| `gen_q511_q556.py` | New: query generation script |
| 13 category JSON files | Updated with new queries + dedup fix |

## Verification

- `compute_counts.py` ran successfully on all 570 queries
- 0 duplicate IDs across all 13 category files
- Only gap: Q025 (pre-existing)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-10 06:24:40 +00:00
Neha PandClaude Sonnet 4.6 146d5e7612 feat(tests): vortex file format test suite (scenarios 1–83) (#12472) 
## Summary

- Adds **239 new tests** covering the vortex columnar file format across
78 scenarios from the test plan
- **`tests/api-testing/tests/vortex/`** — 9 test modules (83 tests):
format conversion, schema evolution, data integrity, FTS, downsampling,
API format awareness, server config, and performance/stress
- **`tests/api-testing/tests/query_agent/test_3_vortex.py`** — 156
tests: the full 155-query SQL correctness suite + scenario 39
(self-join) run post-flush against vortex files
- **`api-testing.yml`** — adds `--ignore=tests/vortex` to the OSS
default-set run; vortex tests require the enterprise binary with
`ZO_FILE_FORMAT=vortex` and are gated to enterprise CI only

## Test plan coverage

| Module | Scenarios |
|--------|-----------|
| `test_format_conversion.py` | 1–13, 7, 8, 9, 10 |
| `test_schema_evolution.py` | 14–25 |
| `test_data_integrity.py` | 55–62 |
| `test_fts.py` | 51–54 |
| `test_downsampling.py` | 47–50 |
| `test_api_format.py` | 71–73 |
| `test_config.py` | 66–67 |
| `test_performance.py` | 74–78 |
| `test_3_vortex.py` | 26–46 (155 queries + JOIN) |

## Enterprise CI

Two dedicated jobs added in the companion `openobserve/o2-enterprise` PR
on the same branch:
- **`api_query_agent_vortex`** — runs `test_3_vortex.py` with
`ZO_FILE_FORMAT=vortex`
- **`api_vortex_format_tests`** — runs the full `tests/vortex/` suite
with 4-phase server lifecycle (parquet→vortex→parquet→vortex+compaction)

OSS CI is unchanged except for the ignore rule — no vortex tests run
against the OSS binary.

## Test checklist

- [ ] `tests/vortex/` collects cleanly: `83 tests collected`
- [ ] `tests/query_agent/test_3_vortex.py` collects cleanly: `156 tests
collected`
- [ ] Enterprise CI `api_query_agent_vortex` job passes
- [ ] Enterprise CI `api_vortex_format_tests` job passes

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-08 13:57:17 +00:00
Neha PandClaude Opus 4.7 008398163b test: add more queries to query regression (#12487) 
## Summary
- Adds **100 new structural-pattern queries** (Q411-Q510) across all 13
categories, bringing the suite from 425 to **525 queries**
- Adds **cross-stream JOIN/UNION support**: `stream_offset` in
data_gen.py, `STREAM2` in conftest, dual-table DuckDB oracle
- 15 cross-stream queries (Q401-Q415) covering INNER/LEFT/RIGHT JOIN,
EXISTS/NOT EXISTS, UNION ALL across streams
- Bumps data_gen.py from 510 to 520 query capacity

## Changes
- **Infrastructure**:
[conftest.py](tests/api-testing/tests/query_agent/conftest.py),
[compute_counts.py](tests/test-data/query-agent/compute_counts.py),
[data_gen.py](tests/test-data/query-agent/data_gen.py)
- **New file**:
[cross_stream.json](tests/test-data/query-agent/queries/cross_stream.json)
— 15 queries
- **12 categories**: 85 queries across aggregation, basic_select,
combined, cte_subquery, date_time, full_text_search, histogram,
math_functions, pagination, string_functions, union, window

## Test plan
- [x] `compute_counts.py` generates expected results for all 525 queries
- [ ] Run `pytest tests/query_agent/ -v` against target environment (565
tests expected)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-08 04:38:40 +00:00
Neha PandClaude Opus 4.7 0334dda9a3 test: add data variety and NULL stress to query agent data generation (#12485) 
## Summary

Two high-value improvements to the query agent regression test data:

1. **Data variety** — Per-field rotation offsets (`qi*13 + i*7`) replace
the old uniform `idx % len(pool)` approach. Previously all fields picked
from the same pool position, creating only 5 fixed row shapes per query.
Now field-value combinations vary independently per field.

2. **NULL stress** — 8 fields (`bot_flag`, `info_tag`,
`upstream_error_code`, `page_url`, `endpoint_path`, `resource_path`,
`variant_tag`, `threat_flag`) get `None` for 40% of records per query.
This exercises `IS NULL` / `IS NOT NULL` / `COALESCE` paths that were
previously untested.

## Changes
- `data_gen.py`: Rewrote `make_record()` with per-field rotation and
NULL injection
- 24 queries: Adjusted WHERE/HAVING filters to match new data
distribution
- All 12 query JSONs: Regenerated expected results via
`compute_counts.py`

## Impact
- 400 queries all have results (18 pre-existing legacy/skip queries
unchanged)
- No change to record count (still 5/query, 2000 total) — test time
unaffected (~7 min)

## Test plan
- [x] `compute_counts.py` regenerates all expected results
- [ ] Run `pytest tests/query_agent/ -v --tb=short` against dev
environment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-07 08:11:47 +00:00
Neha PandClaude Opus 4.7 5eb63395f1 test: broaden coverage with structural query templates (#12482) 
## Summary
- Adds 200 new SQL queries (Q201-Q400) across all 12 categories,
extracted as structural templates from production `june6.sql` query
patterns
- All queries use the existing synthetic `FIELD_POOL` schema — zero real
stream/field names
- Bumps `data_gen.py` from 200 → 400 queries (4000 records total, 10 per
query)
- Introduces new structural patterns previously missing from the suite

## New patterns
- **CONCAT + COALESCE chains**: Multi-field string construction for
display messages
- **CASE WHEN inside histogram COUNT**: Platform mapping + conditional
aggregation
- **Multi-field LIKE / NOT LIKE OR chains**: Text search across 4-8
fields
- **NOT IN + IS NOT NULL**: Exclusion filters with null safety
- **Long IN lists**: 5+ values in WHERE clauses
- **Percentage calculations**: `100.0 * COUNT(CASE WHEN ...) / COUNT(*)`
patterns
- **approx_percentile_cont in subqueries**: P95/P99 in nested SELECTs
- **Histogram with breakdown**: `GROUP BY _time, breakdown` pattern
- **char_length CASE fallbacks**: `CASE WHEN char_length(f) > 0 THEN f
ELSE fallback END`
- **CTE with multiple subquery references**: Self-join and multi-CTE
patterns

## Distribution
| Category | New | Range |
|----------|-----|-------|
| aggregation | 25 | Q201-Q225 |
| basic_select | 25 | Q226-Q250 |
| combined | 20 | Q251-Q270 |
| cte_subquery | 25 | Q271-Q295 |
| date_time | 10 | Q296-Q305 |
| full_text_search | 20 | Q306-Q325 |
| histogram | 25 | Q326-Q350 |
| math_functions | 8 | Q351-Q358 |
| pagination | 7 | Q359-Q365 |
| string_functions | 15 | Q366-Q380 |
| union | 12 | Q381-Q392 |
| window | 8 | Q393-Q400 |

## Test plan
- [x] `compute_counts.py` generates DuckDB expected results for all 400
queries
- [ ] CI: 800 tests (400 × 2 phases) pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-07 02:31:06 +00:00
Neha PandClaude Opus 4.7 4d48bbb21b test: expand query agent regression suite (#12479) 
## Summary
- Expands the SQL regression catalog from **155 to 200 queries** across
all 12 categories
- Adds **7 queries** extracted from abc.sql (Q156-Q162) — synthesized
from real-world OpenObserve SQL patterns with all data/fields/streams
generated from scratch
- Adds **230 auto-generated queries** (Q163-Q392) from templates to
cover query format diversity
- Both test phases pass: **200 memtable + 200 parquet = 400 total**

## Key changes
- **data_gen.py**: 43 new fields (record_type, site_name, latency_ms,
payload_size, alert_tags, phase_data, db_info, etc.), generates 3920
deterministic records
- **compute_counts.py**: new translations for regexp_like,
str_match_ignore_case, approx_distinct, json_get_str/json_get_json
chaining; empty-aggregate result detection (OO returns 0 rows vs
DuckDB's 1 NULL row)
- **conftest.py**: PID-based stream name suffix for cross-invocation
isolation (prevents data accumulation)

## Test plan
- [x] test_1_memtable.py — 392 passed
- [x] test_2_parquet.py — 392 passed
- [x] All 7 abc.sql queries pass (Q156-Q162)
- [x] No real stream names, field names, or data copied from external
sources

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-06 04:59:20 +00:00
Neha PandClaude Opus 4.7 eb7cce9b96 test: query agent SQL regression suite — 155 queries, sqllogictest-style, two-phase (#12384) 
## Summary

155-query SQL regression test suite covering 12 categories. Adding a new
query requires zero test code changes — drop a JSON entry into the right
category file and run `compute_counts.py`.

**Sqllogictest-style comparison**: DuckDB pre-computes full expected
result sets (not just row counts). The test runner extracts actual
values from OO hits, sorts both sides, and compares cell-by-cell with 5%
relative float tolerance. This catches value-level regressions that
row-count-only checks miss.

Two-phase execution: **memtable** (pre-flush) then **parquet**
(post-flush + Tantivy). **310/310 pass**.

## Architecture

```
tests/
├── api-testing/tests/query_agent/
│   ├── conftest.py            # Shared fixtures, run_query(), tolerance comparison
│   ├── test_1_memtable.py     # Phase 1: all queries pre-flush (FTS skipped)
│   └── test_2_parquet.py      # Phase 2: post-flush (module-scoped fixture triggers flush)
└── test-data/query-agent/
    ├── data_gen.py             # Deterministic data generator (single source of truth)
    ├── compute_counts.py       # DuckDB oracle: computes expected result sets
    ├── base_ts_override.json   # Locks BASE_TS so compute and tests share same timestamps
    └── queries/                # 12 JSON files, 155 queries with expected results
```

### How it works

1. **Deterministic data** — `data_gen.py` generates 10 records per query
(1550 total) with a warehouse/supply-chain IoT schema. Each query Q{N}
gets isolated timestamps. `BASE_TS` is locked via
`base_ts_override.json` so compute and tests share identical timestamps
(prevents histogram bucket drift on minute boundaries).

2. **Unique stream per session** — `STREAM =
f"query_agent_test_{int(time.time())}"` prevents accumulated-data bleed
from prior runs.

3. **DuckDB oracle** — `compute_counts.py` loads the same dataset into
DuckDB, creates a pre-filtered VIEW per query (so time filters work for
CTEs, subqueries, UNIONs — no fragile SQL injection), translates
OO-specific functions, and writes `expected.results` blocks into the
query JSONs.

4. **Histogram translation** — `histogram(_timestamp, 'N unit')` is
translated to integer arithmetic matching OO's `date_bin` with origin at
2001-01-01T00:00:00 UTC. Verified against DataFusion CLI — 100/100
records match across all interval types (15s, 30s, 1m, 2m, 3m, 4m, 5m).
ISO timestamp post-processing converts epoch-µs bucket values to
`YYYY-MM-DDTHH:MM:SS` strings matching OO's serialization format.

5. **Sqllogictest comparison** — `run_query()` checks for
`expected.results`. When present, it extracts column values from hits as
strings, sorts both actual and expected (`results_mode: "rowsort"`), and
compares cell-by-cell with 5% relative float tolerance (handles
cross-engine floating-point drift in AVG/STDDEV/etc.).

6. **Two-phase testing** — `test_1_memtable.py` runs all queries
pre-flush (FTS sqllogictest skipped — Tantivy indexes don't exist yet).
`test_2_parquet.py` has a module-scoped `post_flush` fixture that
triggers `/node/flush`, re-polls until data is searchable, then runs all
queries with full assertions.

### Comparison modes

| Mode | Count | Mechanism |
|------|-------|-----------|
| **Sqllogictest** | 140 | Cell-by-cell comparison with 5% float
tolerance |
| **Skip sqllogictest** | 9 | DuckDB results for documentation, fall
back to legacy assertions |
| **Legacy** | 6 | 5 auto-interval histograms + Q122 (STRING_AGG
DISTINCT ORDER BY) |

**Skip sqllogictest queries** — known OO-DuckDB divergence:

| Query | Reason |
|-------|--------|
| Q037 | ROW_NUMBER tie-breaking differs between engines |
| Q058 | Self-join pairings non-deterministic across engines |
| Q072 | FULL OUTER JOIN histogram — NULL handling differs |
| Q080 | FTS `match_all` + histogram — Tantivy vs SQL LIKE |
| Q111 | ROW_NUMBER ties in pagination |
| Q117 | STRING_AGG ordering differs between engines |
| Q121 | STRING_AGG concatenation order |
| Q127 | UNION across 5 overlapping groups — boundary records compete
for LIMIT spots |
| Q152 | PERCENT_RANK tie-breaking differs between engines |

### Key design decisions

| Decision | Why |
|----------|-----|
| Full result-set comparison | Catches value-level regressions (e.g.,
ROUND producing wrong decimal) |
| `rowsort` mode | DuckDB and OO order ties differently — sorting both
sides eliminates spurious failures |
| 5% float tolerance | Cross-engine AVG/STDDEV/PERCENTILE differ at
least-significant digits |
| `''` = `'None'` normalization | OO serializes NULL as empty string;
DuckDB uses `"None"` |
| DuckDB VIEW for time filtering | WHERE injection into CTEs/subqueries
is fragile — VIEW filters at scan level |
| Dynamic columns excluded | `_timestamp` and `_time` differ between
compute time and test time |
| OO-dropped LAG columns excluded | `prev_tput`, `hours_2_back`,
`delta_from_prev`, `temp_diff` |
| Histogram → DuckDB integer math | DuckDB lacks `date_bin`; integer
formula verified via DataFusion CLI |
| BASE_TS override file | Prevents histogram timestamp drift when
compute and test run at different minutes |
| 1s time window margins | Minimizes cross-query data bleed while
capturing all 10 records |
| `pytest.fail()` on fixture failure | Stops all tests instead of
running 310 against an empty stream |

### Categories covered

| Category | Queries | Description |
|---|---|---|
| aggregation | Q001-Q020 | GROUP BY, COUNT, AVG, SUM, MIN, MAX, FILTER,
HAVING |
| basic_select | Q021-Q035 | WHERE, IN, BETWEEN, LIKE, ORDER BY,
DISTINCT, CASE |
| combined | Q036-Q045 | JOINs, multi-condition filters, combined
aggregations |
| cte_subquery | Q046-Q062 | CTEs, subqueries, derived tables, EXISTS |
| date_time | Q063-Q073 | Date parsing, formatting, extraction, interval
arithmetic, histogram |
| full_text_search | Q074-Q082 | match_all, match against fields |
| histogram | Q083-Q093 | histogram() bucketing, aggregation over
buckets |
| math_functions | Q094-Q102 | ROUND, SQRT, ABS, POWER, arithmetic
expressions |
| pagination | Q103-Q112 | LIMIT, OFFSET, combination with filters and
sorts |
| string_functions | Q113-Q122 | SUBSTR, CONCAT, UPPER, LOWER, TRIM,
LIKE patterns |
| union | Q123-Q134 | UNION ALL, UNION (dedup), CTE + UNION combinations
|
| window | Q135-Q155 | ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD, SUM/AVG
OVER |

## Test plan

- [x] All 155 queries pass in both phases (310/310)
- [x] 140 queries with sqllogictest cell-by-cell comparison
- [x] 17 histogram queries converted from legacy to sqllogictest
- [x] 9 queries with skip_sqllogictest for known OO-DuckDB divergence
- [x] 6 queries in legacy mode (auto-interval histograms + Q122)
- [x] BASE_TS locked via override file — no timestamp drift
- [x] Dedicated CI job (`api_query_agent_tests`) runs in parallel

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-03 09:30:02 +00:00