greptimedb/tests/perf/fixture-format.md

# Direct-SST fixture format

The phase-1 generator should be a reusable fixture generator, not a collection
of issue-specific data loaders.

## Inputs

Each case describes:

```toml
[case]
name = "example_metric"
description = "example direct-readable-SST regression"

[scenario]
kind = "direct_readable_sst"
seed = 12345

[[scenario.tables]]
database = "public"
name = "example_metric"
engine = "mito"
append_mode = true
sst_format = "flat"
primary_key = ["host", "instance"]
time_index = "ts"

[[scenario.tables.columns]]
name = "host"
type = "STRING"
semantic = "tag"
distribution = { kind = "cardinality", values = 100, prefix = "host" }

[[scenario.tables.columns]]
name = "value"
type = "DOUBLE"
semantic = "field"
distribution = { kind = "deterministic_wave", min = 0.0, max = 100.0 }

[[scenario.tables.columns]]
name = "ts"
type = "TIMESTAMP(9)"
semantic = "timestamp"

[scenario.layout]
regions = 1
sst_count = 1024
rows_per_sst = 4096
row_group_size = 512
time_range_layout = "non_overlapping_per_sst"
series_layout = "round_robin"

[[scenario.queries]]
name = "count_all"
kind = "sql"
query = "SELECT count(*) FROM example_metric"
warmup = 0
iterations = 1
```

### Value distributions

Tag columns use `cardinality` and produce one label per series. Double field
columns accept either a NULL-free wave:

```toml
distribution = { kind = "deterministic_wave", min = 0.0, max = 100.0 }
```

or a wave with periodic NULLs:

```toml
distribution = { kind = "nullable_wave", min = 0.0, max = 100.0, null_every = 4 }
```

`nullable_wave` also accepts `null_every = 0` to disable NULLs. It writes NULL on
every `null_every`-th sample of each series and wave values everywhere else, so a
NULL is one missing sample of one series rather than a missing row. The cadence
is counted in samples of the series, not in generated rows, so every series
carries the same null ratio under any `series_layout`; when one SST holds a
multiple of `null_every` samples per series (`rows_per_sst / series_count`), the
pattern also repeats with the same period in every SST. Counting the cadence in
generated rows instead would make it constant per series whenever `series_count`
is a multiple of `null_every` (128 and 4, for example), leaving series either
entirely NULL or entirely NULL-free. Counting per series keeps NULL and non-NULL
samples mixed inside range-function windows, which is what the nullable branches
of the PromQL range functions observe.

### Query kinds

`kind = "prom_http"` runs a Prometheus range query by POSTing form fields to
`/v1/prometheus/api/v1/query_range`. `query`, `start`, `end`, and `step` are
sent as form fields; `start` and `end` define the range, and `step` defines its
evaluation interval. `start`, `end`, and `step` are optional in the case model
and default to empty form values when omitted. The table database is sent as the
`db` URL query parameter, rather than as a form field.

The measurement starts before the HTTP request is sent and ends after the full
response body is read and parsed as JSON. It therefore includes request send,
server-side Prometheus JSON response building, and response-body read and JSON
parsing; it is not server-side-only latency.

`series_layout = "round_robin"` advances the timestamp once per generated row and
cycles series labels across rows. `series_layout = "timestamp_major"` writes all
series for one timestamp before advancing to the next timestamp; use it for
Prometheus-like high-cardinality scrape fixtures where short query windows should
still contain many raw samples. `timestamp_major` requires `rows_per_sst` to be
divisible by `series_count`. `per_sst` uses one series selected from the SST
index. These describe logical generation order. Before either flat or primary-key
SST format is written, each completed generated batch is physically sorted by
encoded primary key ascending, timestamp ascending, and sequence descending, as
required by the Mito Parquet writer; sorting preserves every generated column and
row, including deterministic-wave values.

`[scenario]` is required. Other scenario variants are intentionally unsupported
for now, but `scenario.kind` leaves room for future `write_then_query` and
`cache_warm_query` configuration.

The generator should use these declarations to produce:

- object-store SST files written through the real Mito SST writer
- manifest checkpoint and `_last_checkpoint`
- fixture summary with file IDs, row counts, time ranges, and generated schema

## Why this is generic

The same fixture format should support different query-regression families:

- PromQL/TQL time-index pushdown
- SQL predicate pruning
- projection and row-group pruning
- series scan behavior
- joins or aggregation over controlled layouts

Issue-specific behavior belongs in case configs and thresholds, not in the
generator implementation.

## Direct generation vs realism

Direct SST generation is phase-1 because it is fast and reproducible:

- SST count, file time ranges, row groups, and label distributions are fixed by
  the case config.
- The same fixture can be used for base and candidate builds.
- Large pruning-sensitive datasets can be created without spending CI time on
  ingestion, memtable flush, or compaction.

It is less realistic than ingestion-path data because it bypasses writes,
memtables, flush scheduling, and compaction. That tradeoff is intentional for
PR-level query regression. A later nightly/release suite can add ingestion-based
cases for end-to-end realism.

Multi-table cases are supported by generating one fixture directory per table.
Each table is still limited to one region, and the runner passes `--table`, the
discovered `--region-id`, and the discovered `--table-dir` for each table before
materializing all generated region subtrees into the same datanode data home.
This supports JOIN regression cases without changing existing single-table case
files.

Multi-table cases must use unique table names and unique `(database, name)`
pairs. The runner derives each fixture subdirectory from table index, database,
and table name, sanitizing path-unsafe characters to avoid collisions and unsafe
paths.

The preferred compatibility path is:

1. create an empty table with the target build to seed catalog/table metadata;
2. stop the process;
3. generate readable SSTs and replacement manifest checkpoints offline using the
   seeded region metadata;
4. restart and query the fixture.

Fully synthetic metadata is useful for generator smoke tests, but seeded metadata
is safer for end-to-end query performance cases.