A spawned partition queues up to two chunks and its task holds a third
while the queue is full. None of them was charged to a reservation, and
since they carry materialized labels their size is unbounded even though
their count is not.
Each chunk is now charged to the query's memory pool when the task reads
it, and the charge travels with the chunk: it is released when the chunk
is handed to the gather, which charges what it holds back itself, or
when the chunk is dropped in the queue or in an aborted task. A pool
that cannot hold the next chunk ends the partition in an error.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
A one-to-one comparison between two selectors, such as `a > on(host) b`
or `a == bool b`, kept the existing plan: the pair admission accepted
only arithmetic operators. The kernel already pairs comparisons by the
same matching rules and only differs in the value a pair keeps (the
left sample when the comparison holds, or 1/0 with `bool`), so the
operator check is dropped. This covers one table and two, and the
point-wise wrappers around a pair, such as `(a > b) * 100`.
Adds an end-to-end case over two tables comparing explicit rows, the
legacy answer and the native placement for a filter and a `bool`
comparison.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Reuse pack_evaluated_series for fused Frame output, preserving the single native driver and PlainRows/partial timing. Reject mismatched logical frame schemas and use the reset scan plan in integration tests. Add evaluated-packer lifecycle and invalid-sequence regressions, and address the review follow-ups.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## What's changed and what's your intention?
Enable vmbench-lite 05/06/08 cross-metric arithmetic using two
independently planned native operands and the existing DataFusion binary
join, PromQL matching and arithmetic projection.
Physical shape: `SeriesScan A(N) -> Fused A(N)` and `SeriesScan B(M) ->
Fused B(M)` execute region-side; separate MergeScan branches feed the
existing frontend Join. No native Join, second matching kernel or
Exchange is added; same-table fused admission is preserved.
A bounded analyzer pass locates each direct Fused(TableScan) through
Alias/Projection/Filter/Window wrappers, rewrites only the fused
subtree, and leaves all wrappers frontend-side. This preserves global
uniqueness checks. Subquery bodies disable native admission and restore
the prior flag before returning either success or error, avoiding use of
outer evaluation bounds.
### Verification
- `make fmt-check`
- `cargo check --locked -p query -p tests-integration --all-targets`
- Focused query admission/analyzer tests: 2 passed
- `cargo nextest run --locked -p tests-integration --lib -E
'test(native_hybrid_binary_05_06_08)'`: passed, independently rerun on
final source
- `cargo clippy --locked -p query --all-targets --all-features -- -D
warnings`
- Independent source review approved.
Standalone integration asserts two independent region-local
Fused->SeriesScan branches, four nonempty partitions per input, frontend
join placement, native/legacy complete-row parity, unmatched-series
removal (05), outer scalar projection (06), and independent rate inputs
(08). Additional cases cover subquery legacy fallback, valid
reduced-label `on(job)` matching, frontend uniqueness Window plus
`prom_assert_unique_match_group` Filter above MergeScan, and meaningful
duplicate-match execution errors under both routes.
### Scope and limitations
Standalone single-region per table, multiple scan partitions. Arithmetic
and admitted one-to-one matching only. No distributed Join/Exchange,
cross-region routing, group_left/right, set operators, native subqueries
or multi-field expansion. No full workspace suite or performance
benchmark is claimed. Based directly on narthex, independent of the
global-TopK PR.
## PR Checklist
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the matching labels.
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## What's changed and what's your intention?
Enable vmbench-lite 09, `topk(10, sum by(hostname)(rate(...[5m])))`,
through the existing native aggregate-finalize path. Split TopK/BottomK
into the global suffix above fully merged aggregate values. Other vector
operations retain their existing admission behavior. No new physical
operator or persisted/wire format is introduced.
Physical shape: `SeriesScan(N) -> FusedSeries(N, partial sum) ->
merge/coalesce -> AggregateFinalize(global TopK) -> rows`. TopK is never
partition-local.
The standalone regression uses one region, four nonempty scan
partitions, 28 series and two evaluation timestamps with different
winning hosts. It compares complete native/legacy results and
independently expected values, checks producer/finalizer plans, and
records original partial-state streams to prove the same hostname
contributes from distinct partitions at both timestamps. Partition
output counts were [14,14,14,14].
### Verification
- `make fmt-check`
- `cargo check --locked -p promql -p tests-integration --all-targets`
- `cargo nextest run --locked -p promql -E 'test(aggregate_finalize)'`:
28 passed
- `cargo nextest run --locked -p tests-integration --lib -E
'test(native_aggregate_test)'`: 2 passed
- `cargo clippy --locked -p promql --all-targets --all-features -- -D
warnings`
- Independent source review approved; main-agent targeted execution
checks passed.
### Scope and limitations
Standalone single-region, multiple scan partitions only. No distributed
Exchange, cross-region merge/routing, or new matching/subquery support.
No full workspace suite or performance benchmark is claimed. This PR is
independent of the hybrid-binary PR and is based directly on narthex.
## PR Checklist
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the matching labels.
A multi-input FusedSeriesExec drained every partition of its first input
before it touched the second, and RawSeriesSourceExec drained its scan
partitions one after another. The series distributor feeding those
partitions then waited on the one partition being read, the two inputs'
scans never overlapped, and all decoding ran on the one consuming task.
Read every partition through SeriesGather instead. It polls all of them
and hands a chunk on unless another partition of the same input has a
series open; such a chunk is held until that series closes, so each
input still sees contiguous series. A partition holding a closed series
is not polled again until it is handed over, which bounds what is held
to one series per partition, and that is charged to a memory
reservation. Inputs do not constrain each other.
When there is more than one partition, each is driven on its own query
task with a two-chunk channel, so the scan conversion, frame codec and
label decode of different partitions run in parallel. Dropping the
stream aborts those tasks, and a task that panics ends its partition in
an error rather than a silent end.
On the o11ybench vmlite/vmlong data (standalone, native ON, median of 10
alternating runs; results byte-equal to the sequential reader):
sequential* this change
vmbench-lite-05 4.4ms 3.0ms
vmbench-lite-06 7.3ms 4.2ms
vmbench-lite-08 5.0ms 3.3ms
vmbench-long-range-03 54.4ms 39.6ms
cpu same-table pair 27.6ms 15.3ms
long fs same-table pair 104.2ms 83.6ms
* with the series distributor fix; without it the sequential reader
stalled for up to 12.8s on these queries, and this change does not
need that fix.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Under `or` the right table's tags join the left side's columns, and one of
them can share a name with the left side's field or time index; building
the output schema then failed instead of leaving the query to the existing
plan. Such a union is now declined. The output-label walk also declines an
aggregate or whole-vector node instead of passing its input's labels
through, which would describe the wrong schema if one were ever reached.
Adds an end-to-end case over two tables whose tags, field and time index
are all named differently, partly flushed and partly in the memtable,
under both series scan modes.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
A binary or set operation between selectors of two tables, such as
`rate(a[5m]) / rate(b[5m])`, fell back to the existing plan: the native
route required every input to share one metric, time index and tag list,
and a fused node is placed inside the one region it reads.
Two tables are two regions, so such a pair now runs on the frontend. Each
input becomes a `RawSeriesSource` that rides in its own merge scan: the
region binds its scan to that table's metric, tags and fields, decodes the
labels and answers one explicit-label raw frame stream, reading its scan
partitions one after another so each series stays contiguous. The fused
node consumes those frames directly. Pairs on one table keep running in
the region; partitioned tables keep the existing plan.
A point-wise wrapper around an admitted pair, such as
`100 * (1 - a / b)`, is admitted too, on one table and across two. The
output labels are now derived from the whole tree with each input's own
tags, and the physical node takes them from the output schema instead of
narrowing by the root alone.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## Refer to a related PR or issue link (optional)
Native tree operator wiring (#25): set operators. The kernel came in
#9679; the two-input route comes from #9870 / #9871 / #9896.
## What's changed and what's your intention?
`and`, `or` and `unless`, with `on(...)` / `ignoring(...)`, now compile
into the native tree and run natively. Before, the compiler refused them
and the query fell back to the legacy plan.
- **Plan.** New `NativeExpr::Set { left, right, op: BoundSetOp { op,
labels } }`. There is no cardinality, `bool` or fill field, because
Prometheus rejects those modifiers on set operators. Validation requires
two evaluated vectors on the same grid; the output takes the left side's
fields.
- **Runtime.** `BinaryRuntime` runs either kernel (`PairKernel::{Binary,
Set}`), so a set node gets the same operand retention, assembly,
reservation, queueing and contiguity checks as vector–vector binary.
Errors name the set node.
- **Kernel.** `set_vectors` charged no memory.
`set_vectors_with_reservation` now charges it the way the binary kernel
does: output schema, filter-side group table, kept rows, the sort phase,
error text and the merge. Membership is decided from one presence bitmap
per filter match group (one bit per grid point), so each kept sample is
an O(1) lookup. Before, a coarse `on()` scanned every filter candidate
at every kept point, O(K·P·F). Results are unchanged; the Prometheus
golden set cases still pass.
- **Compiler.** `set_operation` binds the operator and the matching
labels, and nests on either side and above. It refuses a fill value:
promql-parser accepts one, but Prometheus rejects "filling in missing
series not allowed for set operators". It also refuses any side that is
not a single field. Legacy refuses multi-field set operations (both
sides for `or`, the left side for `and` / `unless`), and a per-field
presence rule is not defined by Prometheus, so those queries keep the
legacy plan.
- **Planner.** Same-table set operations take the existing two-scan
route, where `FusedSeriesExec` reads every partition of both scans into
one execution, so a set operation sees every series of both sides.
- **Partitioned tables.** A two-selector native plan on a table with
partition columns failed in `DistPlannerAnalyzer` ("only accept plan
with at most one child") instead of falling back. This already happened
for arithmetic pairs such as `x{kind="a"} - on(host) x{kind="b"}`, and
set operators would have hit it too. The planner now declines
two-selector plans on such tables, so both keep the legacy plan.
Not covered yet: set operations across two tables (`rate(x[5m]) and
on(job) up`), and those with aggregate operands (`sum by (job)(a) or sum
by (job)(b)`). Both still use the legacy plan. The presence bitmaps take
`groups × grid points / 8` bytes. That can exceed the filter samples
only when groups are very sparse (under about 1/64 occupancy); such a
query is refused by the memory pool rather than running over budget.
Allocating each bitmap over its group's position span would tighten
this, and is left for later.
## PR Checklist
Please convert it to a draft if some of the following conditions are not
met.
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the `backport-<target>` labels (e.g. `backport-v1.3` targets
`release/v1.3`).
A range query `a or on(host) b` where one host's left series covers only
the middle steps: native must keep the left sample where it exists and
take the right sample with its own labels at the other steps, matching
the explicit rows and the legacy plan. The set test's row reader moves
to module level so both cases share it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
A node that reads its whole input (binary, set, vector) queues the
series its kernel produced and settles its reservation on the queue.
Each queued result was held for `evaluated_bytes`, which covers the
chunk and its fields but not the label set the chunk owns, so once the
operands were released the reservation dropped below what the queue
actually held. A set over many long-labelled one-point series made the
gap large.
Hold each result that does not share the input for `series_bytes`, the
complete owned estimate including labels. The figure stored with the
entry is still the one released when it is handed on.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
The set kernel checked every filter series of a kept series' match group
at every kept point, so a coarse `on()` / `on(...)` cost
O(kept series x points x group size). Prometheus decides a step from
the set of signatures present on the filter side.
Build one bitmap per filter match group, one bit per grid point, from
the filter side's rows, and look each kept sample up in O(1). Each
bitmap is charged to the walk phase when its group is first met and is
released with the group table before the sort phase. Results are
unchanged.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
The legacy planner refuses `and`, `or` and `unless` over several fields
(MultiFieldsNotSupported), and Prometheus has one value per sample, so
per-field presence has no reference semantics. The compiler now declines
a set operation unless each side has exactly one field, and the query
keeps the legacy plan and its error.
Errors raised by the shared two-operand runtime now name the set node
instead of calling every such node "the binary node".
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
A two-scan FusedSeries over a table split into several regions has no
placement: DistPlannerAnalyzer rejects it with "only accept plan with at
most one child", so the query failed instead of falling back. This hit
the existing arithmetic pair and, since set operators now compile, set
operators too. Decline the native route for two selectors when the table
has partition columns.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Bind `and`, `or` and `unless` (with `on`/`ignoring`) to a new
`NativeExpr::Set` node instead of refusing them. The node runs the
existing MATCH-SET kernel through the binary runtime, which retains both
operands until the input ends, so a set operation reads every series of
both sides in one execution.
The set kernel now reports its workspace like the binary kernel does:
kept rows, the filter side's match groups, the sort and the merge are
charged to the node's reservation.
The compiler refuses a fill value on a set operator, which Prometheus
rejects and the parser accepts, and sides with different field counts.
Same-table set operations over two scans take the existing two-scan
native route.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## Refer to a related PR or issue link (optional)
- **Stacked on #9942** (PR A, the unified SeriesFrame codec).
- GitHub cannot target a fork branch, so this PR's base reads `narthex`.
**Review #9942 first.**
- The commits unique to this PR are `b32573f9cbf`, `16b3e428951`,
`d5547c55bdf` and
`f2041d8d9de` (test-only lint cleanup). The chain is
`narthex (771347df5d) → #9942 (8bc53c4871) → this PR (CLEANUP_SHA)`.
- Merge order remains **#9942 → #9943**. Once #9942 lands, rebase this
PR onto the resulting
`narthex`; do not merge the codec changes twice.
- Spec: Narthex Unified SeriesFrame refactor, PR B of two.
## What's changed and what's your intention?
The fused native node could only answer one way: a row table with the
grid timestamps, one column
per series field and one per declared tag. That is what a query returns,
but it is not what a
consumer of the native path needs when it has to tell series and
positions apart — the labels of a
series, and which grid positions it actually holds, are both flattened
away by the row shape.
This adds a second output format, **fixed when the plan is built**
rather than chosen at run time:
```rust
pub enum FusedOutputFormat { PlainRows, SeriesFrame }
```
| | `PlainRows` (default) | `SeriesFrame` |
| --- | --- | --- |
| output schema | the existing row table (time / fields / tags) | the
codec's declared evaluated frame |
| encoding | `series_to_record_batch` | straight from `EvaluatedChunk`
via `SeriesFrameCursor` |
| reachable from a query | yes | no |
| `EXPLAIN` | byte-identical | appends ` (output: series_frame)` |
The format is part of the plan because it decides the node's output
schema, so it is part of the
node's identity (`PartialEq`/`Hash`/`PartialOrd`), its `EXPLAIN` text,
and its rebuild over a new
child. A format error is refused while the plan is built, not when the
first poll happens.
The `SeriesFrame` schema is derived from the
**`NativeTree.root().output` evaluated `SeriesSchema`**,
never from the original scan schema or the input tags: after a label
rewrite or an aggregate, the
result's labels and fields are what count.
### Independent chains remain independent
`is_last` closes the current chain, not the entire label set. Separate
chains may therefore carry
identical complete labels and each close independently. This PR adds no
plan-owned global label
identity management; combining contributions across partitions remains
the consumer's responsibility.
Tests check exact positions, payload bits and separate finality for both
consecutive same-label
chains in one encoder and chains in separate partitions.
### Parallelism is unchanged
For the supported single-source Frame path, output partition count
equals the input scan's
partition count. No `CoalescePartitionsExec`, `RepartitionExec` or
round-robin is introduced in
either format.
`series_frame_preserves_four_input_partitions` checks the in-memory
boundary. The standalone
integration test additionally captures a real single-region
`SeriesScanExec` at `parallelism=4`
and executes every partition in each output format.
### Implementation notes worth knowing
1. **Stream and batch schemas agree.** The codec-derived output schema
includes the explicit
`contiguous` ordering declaration. Published batches are checked against
that declared schema.
2. **The end of the execution must flush.** The frame builder buffers up
to a batch across chunks,
so missing the final flush loses the tail rows.
3. **`Poll::Pending` also flushes.** Otherwise a closing row stays in
the builder behind an input
that may never become ready — the same convention PR A's `PackStream`
already follows.
### Key code locations
- `src/promql/src/extension_plan.rs`: exports `FusedOutputFormat`.
- `src/promql/src/extension_plan/fused_series.rs`: default/explicit
format constructors, root-output
schema derivation, plan identity/rebuild/EXPLAIN, direct chunk encoding
and stream flushing.
- `src/promql/src/extension_plan/fused_series/tests.rs` and
`tests/frame_output.rs`: internal
logical/physical execution and lifecycle/semantic regressions.
- `tests-integration/tests/http.rs`: real standalone scan replay in both
output formats.
### Review fixes and acceptance
- **Batch-level backpressure:** the stream retains the existing
`SeriesFrameCursor` and advances
only until one batch is ready, then returns control to the consumer. It
no longer encodes an
entire evaluated chunk into a `ready` queue before returning the first
batch. Pending native
output is drained before more upstream input; a pre-EOF stall does not
close the executor.
PlainRows and partial-state conversion keep their original timing.
- **Real scan acceptance:** the standalone File integration fixture has
one physical region and
`parallelism=4`. The test captures its actual `SeriesScanExec`, replays
the internal fused plan
in both formats, executes all four nonempty partitions, and checks the
exact 24 expected rows,
eight Frame chain closings and per-partition native output metrics.
Frame is not exposed as a
user query's final result to perform this test.
- Replaced the vacuous sparse fixture with exact nonempty positions
`0,2,5`, two fields with
partial NULLs and preserved ordinary-NaN bits, and a PlainRows
comparison. Added fixed-value
`abs(rate(...))` and label-rewrite comparisons, including a new label
outside the input tag
projection and two independent same-label chains in one encoder.
Cross-partition chains also
check exact positions and value bits.
- Added retained-cursor/one-batch checks, buffered upstream-error
termination, and early-drop
ownership checks. Reintroducing eager encoding compiles but makes the
backpressure regression
fail at its queue assertion; restoring the reviewed source makes it
pass.
- Removed the redundant physical scalar/Frame guard; the shared
root-output validator remains.
- Independent correctness and anti-over-design review: **APPROVE** for
`8bc53c48714..d5547c55bdf`. Review approval is separate from new-head
CI.
### Limitations
- `SeriesFrame` is **not reachable from a user query** yet (production
defaults to `PlainRows`).
Per spec §8.7 the acceptance executes the internal physical plan and
decodes its output, instead
of exposing the frame as a query's final result format.
- No sqlness result changes: the production planner still constructs
`PlainRows`. The existing
standalone integration case is extended to execute and decode the
internal Frame plan.
- Rebased onto `narthex` at `771347df5d9`, including #9936, #9882 and
#9886. The native zero-source
scalar, `timestamp()`/instant-`@`, multi-input and partial-state paths
are preserved. A Frame over
a scalar root is rejected because this format requires evaluated
samples.
- The multi-source case is refused for frames rather than implemented:
this node still answers only
partition 0 when it has more than one raw source, so a frame over
several sources would have to
change that first.
- Consumer contract: a sparse chunk decodes into **several** `Series`
(one frame row per grid run),
so a consumer must concatenate by position and cannot assume "one chunk
→ one decoded result".
Nothing consumes frames in this PR, but Complete Aggregate's SeriesFrame
input must respect it.
- Not verified: Windows; the results below are Linux.
### Does this break API or data compatibility?
`PlainRows` remains the default and the only format a user query
reaches. This increment leaves
its conversion and plan display paths in place. The added tests compare
exact Frame semantics
without changing the user-facing result format. No persisted format is
touched.
### Tests
Coverage includes codec-derived schema, logical rebuild and EXPLAIN,
dense and genuinely sparse
output, multi-field NULL/NaN values, composed math and label rewrites,
four in-memory and four real
scan partitions, independent same-label chains, partial-state schema,
constructor refusals,
zero-sample closing, Pending/EOF flush, buffered error, cancellation and
incremental encoding.
<!-- CURRENT_VERIFICATION_START -->
Main verification on `d5547c55bdf` (Linux; default features):
- `make fmt-check`: passed.
- `cargo nextest run -p datatypes -p common-query -p table -p promql -p
query --retries 0`:
**2698 passed, 2 skipped**.
- `cargo nextest run -p tests-integration --test main -E
'test(test_native_promql_route_plans_the_query) & test(file)' --retries
0 --success-output immediate`:
**1 passed, 771 skipped**. Both formats produced per-partition decoded
row counts
**[15, 3, 3, 3]**; exact 24-row results and eight Frame chain closings
were asserted.
- `cargo check --workspace --all-targets`: passed.
- The final strict Clippy pass found two test-only hygiene issues in the
new tests: an
unnecessary `f64` cast and two `println!` calls that the workspace's
`print_stdout` lint
rejects. The follow-up types the existing float literal explicitly,
removes the cast, and drops
the diagnostics, whose values were already asserted by the surrounding
exact checks. It changes
no production code or assertions.
- Final test-only cleanup `f2041d8d9de` on top of the reviewed
`d5547c55bdf`:
`make fmt-check` passed; `cargo clippy -p promql -p query -p
tests-integration --all-targets -- -D warnings`
passed; the affected
`abs_rate_frame_matches_plain_rows_with_exact_values_and_labels`
(1 passed) and real File scan replay (1 passed, 771 skipped) passed
again. The full five-crate
suite above was run at `d5547c55bdf`; this last commit touches tests
only and was not followed by
another full-suite run.
Other focused evidence: the eager-encoding mutation compiled and then
failed the queue-boundary
assertion; restored production source passed. License/TOML/diff checks
passed. These are local
Linux results, not a claim of Windows or new-head CI success; full
all-feature workspace Clippy
and the full workspace test suite were not run.
<!-- CURRENT_VERIFICATION_END -->
## PR Checklist
Please convert it to a draft if some of the following conditions are not
met.
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the `backport-<target>` labels (e.g. `backport-v1.3` targets
`release/v1.3`).
The workspace denies the default Clippy warnings, and the new tests tripped
three of them without changing what any of them proves:
- an expected value was built through `scale as f64` even though the tuple
already carries an `f64`, so the conversion is spelled in the literal and the
cast is gone;
- the standalone replay printed the executed plan and the per-partition counts
even though the surrounding assertions already require four nonempty
partitions, 24 rows in total and eight closing frames, so the diagnostics
are gone rather than exempted from the workspace lint.
No production code, assertion or fixture changes.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
The frame output was covered for its happy path and for the plan-level decisions, but three things
the spec asks about had no test of their own at this boundary, and one test did not prove what it
claimed.
The pending test is the one that mattered. It awaited the stream, and `StagedStream` wakes itself
right after reporting pending, so the await ran straight through the boundary: every later assertion
held whether or not the buffered rows were handed over at the pending poll. It now polls that single
poll by hand and requires it to answer the rows it already encoded. Deleting the `flush_frames` call
from the pending branch makes it fail, which it did not before.
The three boundaries now have their own tests: a chain that closes with no position survives as one
row where the row table answers nothing at all; a sparse series carries exactly the positions the
row table answers and no filled hole; and two chains that close on one label set are both kept, with
values that differ so a lost chain would show.
Also drops the `as_str` helper, which only asked whether to show a format and then spelled the same
string twice, and corrects two comments: `convert` produces three outputs rather than two, and a
built plan is not a promise that execution cannot fail.
Signed-off-by: discord9 <discord9@users.noreply.github.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
The fused node answers one way: a row table with the grid timestamps, one column per series field
and one per declared tag. That is what a query returns, but a consumer of the native path needs to
tell series and positions apart, and the row shape flattens both away - which series a value belongs
to, and which grid positions that series actually holds.
This adds a second output format, fixed when the plan is built rather than chosen at run time:
PlainRows the existing row table, unchanged and still the only one a query reaches
SeriesFrame the same evaluated chunks handed over as the unified series frame
The format is part of the plan because it decides the node's output schema, so it takes part in the
node's equality, its EXPLAIN text and its rebuild over new children. A row-table plan prints exactly
what it printed before; a frame plan says so on its own line.
A frame is encoded straight from each evaluated chunk, never by flattening it into rows and packing
it again, so a sparse series keeps its runs and a closing chunk with no position is still one row.
The output keeps one partition per input partition: a frame is not a reason to coalesce, and no
exchange is inserted in either format.
The partial-state output that a complete aggregate reads is a third thing again, and the format does
not reach it. When a node exports partition-local state, the format is `PlainRows` by construction,
`convert` hands every chunk to the fold before any format is consulted, and the EXPLAIN line keeps
its `partial state:` suffix. Nothing in the aggregate path is rewritten to make room for frames.
A tree whose root is not evaluated samples is refused while the plan is built, because aggregate
state is a contribution to a later merge and a scalar has no series to name. So is a frame output
over more than one raw source, which this node does not run in parallel yet.
Signed-off-by: discord9 <discord9@users.noreply.github.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## Refer to a related PR or issue link (optional)
- Spec: Narthex Unified SeriesFrame refactor, PR A of two.
- #9914, #9868, #9937, #9896, #9933, #9936, #9882 and #9886 are all
merged, so this is a plain increment on
`narthex` with no stacked dependency.
## What's changed and what's your intention?
SeriesFrame is the internal transport of the Narthex native PromQL path,
and it could only carry
raw samples: an evaluated result had to fall back to a plain row table.
This makes **one
self-describing Arrow format carry both**, so a receiver decodes a frame
from the frame alone,
without the scanner, region, or planning context its producer had.
What is unified is the **physical representation and its codec**, not
the runtime data structures.
`RawSeriesChunk`, `Series` and `EvaluatedChunk` are unchanged; only the
adapter boundaries branch
on the sample shape.
### Base
- base: `narthex` @ `771347df5d9`
- HEAD: `8bc53c48714`
- **no stacked dependency**: everything this PR built on has landed
This is a plain increment on the current `narthex` tip, with focused
follow-up fixes for review
and CI. Nothing from the earlier stack is carried: #9914 is merged, so
no `MetricName` rename appears here. The codec itself, the
adapters, the tests and the two frozen wire fixtures are all that moved,
plus the call sites that
had to follow the renamed API.
### The format (`layout = v2`)
One frame family, three legal combinations. Every other combination is
refused on both the writing
and the reading side:
| content | identity | time | producer |
| --- | --- | --- | --- |
| `raw` | `key` | `timestamps` | a storage raw-series scan |
| `raw` | `explicit` | `timestamps` | a raw input whose labels are
already decoded |
| `evaluated` | `explicit` | `grid_spans` | a compute node's output |
`evaluated` + `key` is refused because a computation may rewrite its
labels, so it cannot claim a
storage key as its identity. `raw` + `grid_spans` and `evaluated` +
`timestamps` are refused because
raw samples have no grid and computed values have no timestamps of their
own.
Column layout (`series_frame_schema` in
`src/datatypes/src/series/frame.rs`):
```text
read_key: Binary (NOT NULL for key mode; NULLABLE for explicit mode)
is_last: Boolean NOT NULL
time: Struct<timestamps: List<Int64>> // raw
Struct<spans: List<Struct<start: u64, len: u64>>> // evaluated
fields: Struct<<field_name>: SamplePayload, ...>
name: Utf8 NULLABLE // explicit only, __name__
labels: List<Struct<name, value>> // explicit only, no __name__, no duplicates
```
`read_key` must be populated for every raw row, including raw/explicit,
and must be NULL for
evaluated rows. `SamplePayload` (float / native histogram / mixed) keeps
its existing representation
and is not redesigned.
Protocol metadata lives under `greptime.series.*`.
`SeriesFrameDeclaration::parse` reads the
layout, identity, time and ordering declaration. Schemas explicitly
declare `contiguous` by default;
raw streams may declare `read_key_then_time`, while evaluated streams
accept only `contiguous`.
Missing ordering, unknown values and unknown reserved keys are refused
by declaration parsing,
direct decoding (including empty batches), and stream admission.
| key | value |
| --- | --- |
| `layout` | `v2` |
| `content` | `raw` / `evaluated` |
| `labels` | `key` / `explicit` |
| `time` | `timestamps` / `grid_spans` |
| `grid` | required for evaluated, forbidden for raw; canonical
`GridSpec` JSON |
| `label_decode` | required for key, forbidden for explicit |
| `ordering` | `contiguous` / `read_key_then_time` |
### Time axis
- raw: every row carries its own strictly increasing `i64` millisecond
timestamps.
- evaluated: the `GridSpec` is declared once in schema metadata
(`{start_ms, step_ms, len}`), and a row carries only the **absolute grid
positions** as spans.
A sparse grid is never expanded and its holes are never filled; `start +
len` is checked against
the grid with checked arithmetic.
- A zero-sample closing chunk is legal for evaluated and **is kept** as
one frame row with zero
positions and `is_last = true`, because dropping it would drop the
finality it exists to carry.
A zero-sample chunk that closes nothing is refused, and a raw frame
still refuses an empty chunk.
### Series chains and `is_last`
`is_last` closes **one chunk chain**, not the end of a PromQL label set.
After a native label
rewrite, two different input series can produce **the same complete
labels**, and each closes
itself. The writer allows that and the reader does not treat "same
labels" as a re-opened series.
A chain that changes identity before closing, and EOF with an open
chain, are both errors.
### Key code locations
- `src/datatypes/src/series/frame.rs`: shared schema, declaration
parser, bounded builder,
cursor and raw/evaluated decoder.
- `src/common/query/src/series_frame.rs`: typed pack/unpack adapters and
chain validators.
- `src/datatypes/src/series/time.rs`: canonical `GridSpec`
serialization.
### Review and CI fixes
- Repaired the two table PK-binding fixtures with real `host` and `rack`
Tag columns and matching
primary-key metadata. Production `LabelDecodeSpec::validate()` stays
strict. The mismatch test
retains all six cases, including changed tag order, and checks rejection
before polling.
- Removed the missing-ordering fallback and added one private ordering
validator shared by the
declaration parser and direct decoder. Invalid evaluated ordering cannot
bypass stream checks.
- Compatibility metadata is tested exactly as written, without merging
canonical metadata into
malformed inputs. The direct-decoder matrix covers raw/evaluated,
empty/nonempty, missing,
unknown and interleaved ordering, and valid raw versus invalid evaluated
`read_key_then_time`.
- Independent source and anti-over-design review: **APPROVE** for
`771347df5d9..8bc53c48714`. Review approval is separate from the CI
status of the new head.
### Limitations
- Not reachable from a query yet: `FusedSeriesExec` still answers
`PlainRows` by default. The
second output format is PR B.
- Not verified: Windows; the results below are Linux.
- Deliberately **not** done, per the spec: evaluated **input** wiring in
`FusedSeriesExec`,
Complete Aggregate's SeriesFrame input, Partial/Final physical plans,
cross-region exchange,
remote NativeTree serialization, NodePath rework, #9892
storage-interface changes, #9844 remote
read wiring.
- The unreleased v1 frame format is **not** read. In-tree producers,
consumers, tests and the two
frozen wire fixtures all move to v2 together.
### Does this break API or data compatibility?
No published format changes. SST, WAL, manifest and the generic
Flight/Protobuf formats are
untouched. The only versioned thing here is the unreleased internal
SeriesFrame layout, which goes
from `v1` to `v2` in one step and is not read in both directions.
### Tests
The existing tests include real pack → batch stream → unpack round
trips, not only schema
snapshots:
- raw/key with its decoded PK, raw/explicit with complete labels, both
across batches and through
Arrow IPC
- evaluated dense, sparse (multiple spans, holes not filled),
multi-field (a position can hold a
value in one field and nothing in another; NULL ≠ NaN), native histogram
and mixed payloads
(payload kind and values survive, no silent float-ification)
- evaluated empty closing chunk kept as one row; multi-batch streams
keeping order and finality;
two consecutive chains with equal labels each closing independently; EOF
with an open chain
- malformed frames refused: non-increasing timestamps,
overlapping/out-of-order/out-of-bounds/zero
length spans, payload length mismatch, span-count overflow, missing
grid, malformed `GridSpec`,
missing label decode, contradictory metadata, wrong layout version, an
evaluated row carrying a
storage key, an empty row that closes nothing
<!-- CURRENT_VERIFICATION_START -->
Verified on Linux at `8bc53c48714`; every command below exited 0:
```text
make fmt-check
-> clean (rustfmt and repository source checks)
cargo nextest run -p datatypes -p common-query -p table --retries 0
-> 690 passed, 0 skipped; includes both reported CI failures
cargo nextest run -p mito-codec -p promql -p query --retries 0
-> 2055 passed, 2 skipped
cargo nextest run -p mito2 -E 'test(engine::scan_test::test_mito_handle_query_raw_series) | test(engine::scan_test::test_two_phase_native_raw_stream) | test(engine::scan_test::test_two_phase_raw_) | test(engine::scan_test::test_mito_raw_series_)' --retries 0
-> 8 passed, 1733 skipped (focused raw-scan selection)
cargo check --workspace --all-targets
-> passed
cargo clippy -p datatypes -p common-query -p table -p mito-codec -p promql -p query --all-targets -- -D warnings
-> passed (scoped crates, default features)
```
License headers, TOML formatting and `git diff --check` also pass. These
are local Linux
results, not a claim that the new-head GitHub checks are green. Full
`make test`, workspace
all-features Clippy and Windows execution were not run locally in this
round.
<!-- CURRENT_VERIFICATION_END -->
## PR Checklist
Please convert it to a draft if some of the following conditions are not
met.
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the `backport-<target>` labels (e.g. `backport-v1.3` targets
`release/v1.3`).
SeriesFrame is the internal transport of the Narthex native PromQL path, and
it could only carry raw samples: an evaluated result had to fall back to a
plain row table. This makes one self-describing Arrow format carry both, so a
receiver decodes a frame from the frame alone, without the scanner, region, or
planning context its producer had.
What is unified is the physical representation and its codec, not the runtime
data structures. RawSeriesChunk, Series, and EvaluatedChunk stay as they are;
only the adapter boundaries branch on the sample shape.
Three combinations are legal and every other one is refused in both
directions:
raw + key + timestamps a storage raw-series scan
raw + explicit + timestamps a raw input whose labels are decoded
evaluated + explicit + grid_spans a compute node's output
An evaluated frame cannot claim a storage key, because the computation may
rewrite its labels; raw samples have no grid to be addressed on. The schema
builder and the declaration parser both enforce that, so a frame cannot be
written or read in a shape no producer could mean.
An evaluated frame declares its GridSpec once in schema metadata and carries
absolute grid positions per row as spans, so a sparse grid costs its runs and
not its holes: nothing is expanded, and no position a chunk does not hold is
filled in. A chunk with no position is legal exactly when it closes its chain,
and it is kept as one frame row, because dropping it would drop the finality
it exists to carry.
Layout is v2 and the writer and reader of the reserved greptime.series.*
metadata live in one place each, so a producer and a consumer cannot spell one
frame two ways. The unreleased v1 format is not read: the in-tree producers,
consumers, tests, and the two frozen wire fixtures all move to v2 together.
Signed-off-by: discord9 <discord9@users.noreply.github.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 176f6f6479)
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## Refer to a related PR or issue link (optional)
Test-only follow-up to #9882. Merge #9882 into `narthex` first, then
merge this PR. The PR base remains `feat/narthex-instant-at` until its
parent lands.
## What's changed and what's your intention?
Add regression coverage for the existing native multi-field
`timestamp()` refusal. The net change against the refreshed #9882 is
exactly **one file, `src/query/src/promql/native_compile/tests.rs`, with
43 added lines and no production changes**.
- `timestamp(x)` over two fields reaches the raw-selector guard;
`timestamp(abs(x))` reaches the evaluated-vector guard. Both must return
`NotImplemented` naming `timestamp() over a multi-value input` as the
reason.
- The same two expressions over one field must compile. These positive
controls establish that the refusal is about field count rather than
blanket rejection of timestamp expressions.
- A small local multi-field schema fixture leaves the existing
single-field fixture unchanged.
The old branch history was not mechanically replayed: only this final
test delta is retained atop the new parent. Production refusal, public
single-field native timestamp admission and the paired public-planner
multi-field fallback test belong to #9882. This PR does not implement
multi-field timestamp results or add fallback machinery.
### Verification
Verified clean final HEAD
**`9a702209747646a68a67fc09a3d44a3c30cfc5cf`**, directly atop #9882
**`24e214b7d4b3d277ce04a7e40d3aaf3d7673c70a`** (rebased onto
`narthex=aa30968d43b3`, including #9936).
- `make fmt-check` and the exact-parent `git diff --check`: passed.
- `cargo check --locked -p query -p promql --all-targets`: passed.
- `cargo nextest run --locked -p query -p promql --retries 0
--no-fail-fast`: **1990 passed, 2 skipped**.
- `cargo clippy --locked -p query -p promql --all-targets --all-features
-- -D warnings`: passed.
Old `06695dcf4a5` CI results are not attributed to this rewritten HEAD.
The refreshed Checks, Rust CI and Integration CI results for this exact
immutable HEAD are required before merge; this PR still follows #9882.
The 43 added test lines and their context were compared with the
original `3e80f822595..93ad42b76d5` test-only delta. They are identical;
only Git blob IDs and hunk line numbers move because #9936 changed the
surrounding baseline. Parent standalone Native/Complete Aggregate
verification is reported in #9882 rather than attributed to this
test-only change.
## PR Checklist
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the `backport-<target>` labels (e.g. `backport-v1.3` targets
`release/v1.3`).
This test-only change alters no API, schema, stored data or wire format,
and requires no release backport.
Refreshed CI runs for this published immutable HEAD are pending:
[Checks](https://github.com/GreptimeTeam/greptimedb/actions/runs/40266409294),
[Rust
CI](https://github.com/GreptimeTeam/greptimedb/actions/runs/40266409205),
[Integration
CI](https://github.com/GreptimeTeam/greptimedb/actions/runs/40266409248).
No merge is performed until these new-HEAD checks pass.
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## Refer to a related PR or issue link (optional)
Follows the merged multi-input wiring in #9896, Complete Aggregate in
#9933, and zero-input Scalar/literal broadcasts in #9936. Merge this PR
before its test-only follow-up #9886.
## What's changed and what's your intention?
Connect instant-selector `@` and `timestamp()` to the standalone native
PromQL planner and recursive executor on current `narthex`.
- Supports fixed `@ <timestamp>`, `@ start()`, `@ end()` and
positive/negative offsets. The selector reads at `anchor - offset`, with
the existing lookback and staleness rules, and replays its answer on the
statement's evaluation grid. Its scan bounds follow the anchor even when
the selected data predates the statement.
- `timestamp(raw_selector)` reports the selected source sample's
original timestamp in seconds. `timestamp(abs(raw_selector))` and other
supported evaluated-vector arguments report evaluation-grid seconds only
where the argument has a value. These remain distinct
compilation/runtime paths; an offset changes which source sample is
chosen, not the timestamp projected from that sample.
- Adds `timestamp` to public native planner admission for supported
single-field expressions. This is no longer only a compiler/executor API
capability.
- Retains recursive RuntimeNode ownership, InputId routing, per-node
metrics, SeriesLabels and the current frame decoder. The evaluated
Timestamp node reuses existing NodeState accounting and lifecycle, with
projection workspace reserved before allocation.
### Integration with Complete Aggregate
The migration combines Timestamp recursion with the latest scalar-aware
`reads_across_series` handling. AggregateFinalize receives only the
necessary Timestamp traversal arms and test-fixture updates. Existing
`partial_fold`, Complete Aggregate admission, scalar-binary
prefixes/suffixes, `FusedSeries.children: Vec<LogicalPlan>` and
`BoundInput.source_index` are preserved; no old planner/compiler files
or flat NodePath runtime are restored.
This remains a single-region standalone experimental Native PoC.
Multi-field `timestamp()` still declines native compilation and keeps
the existing planner. Timestamp/aggregate shapes outside the current
split grammar likewise retain the existing plan. This PR does not add
general fallback machinery, cross-table routing, Join, Exchange or
distributed execution. `timestamp(x @ t offset d)` preserves
GreptimeDB's existing selection semantics rather than introducing
Prometheus' special-case offset behavior.
### Synchronization with merged Scalar roots
Rebased the reviewed B-only change onto `narthex` at
`aa30968d43b3bf9b5c335c6ac3a9f38be58d653b` (#9936 merged). This is a
single B implementation commit directly on that baseline, followed
separately by #9886's test-only commit. The only textual conflict was
adjacent added planner tests; both timestamp tests and the
constant-root/broadcast test remain intact. No production merge fix or
fallback expansion was needed. The zero-child Scalar route, literal
broadcasting, compiler/admission guards, recursive runtime accounting
and Complete Aggregate remain present.
### Verification
Verified clean final HEAD
**`24e214b7d4b3d277ce04a7e40d3aaf3d7673c70a`**, rebased directly onto
`narthex` **`aa30968d43b3bf9b5c335c6ac3a9f38be58d653b`**.
- `make fmt-check` and `git diff aa30968d43b3..HEAD --check`: passed.
- `cargo check --locked -p query -p promql --all-targets` and the
separate `-p tests-integration --all-targets` check: passed.
- `cargo nextest run --locked -p query -p promql --retries 0
--no-fail-fast`: **1989 passed, 2 skipped**.
- `cargo nextest run --locked -p tests-integration --lib --retries 0
--no-fail-fast -E 'test(native_promql_) |
test(native_aggregate_complete_over_four_nonempty_mito_partitions)'`:
**17 passed**. This includes the instant/timestamp storage query,
retained native routes and Complete Aggregate over four nonempty Mito
partitions, including #9936's scalar broadcasts.
- `cargo nextest run --locked -p tests-integration --test main --retries
0 --no-fail-fast -E
'test(integration_http_file_test::test_prometheus_native_promql_route) |
test(integration_http_file_test::test_native_promql_route_plans_the_query)'`:
**2 passed**, preserving #9936's File-backed HTTP native and planning
routes.
- `cargo clippy --locked -p query -p promql --all-targets --all-features
-- -D warnings` and the separate `-p tests-integration` invocation:
passed.
- The added `native_instant_at.rs` was copied byte-for-byte for focused
`hawkeye format` followed by `hawkeye check`, using repository inception
year 2023; both passed and the copy remained identical.
Built fresh `greptime` and `sqlness-runner` binaries from this exact
HEAD, then ran isolated standalone SQLness compatibility cases
`at_modifier`, `fused_multi_standalone` and `scalar_entry`: **all 3
passed**. Expected `.result` files remained byte-identical; the runner
stopped the temporary instance and removed its state. These are
compatibility checks, not substitutes for the Native execution tests
above. GitHub CI results for the old `d09e4fdc057` HEAD are not reused:
refreshed Checks, Rust CI and Integration CI must complete successfully
on the new immutable HEAD before merge.
The new actual standalone test checks independent expected timestamps,
values and host labels, then compares with the legacy route. Its cases
cover fixed/start/end anchors, pre-statement anchors, signed offsets,
the exclusive lookback boundary and exact zero-lookback selection,
missing values, exact-bit stale markers, and raw-sample versus
evaluation-grid timestamps. For a bare selector whose newest source
value is NULL, the existing Arrow representations differ: Native emits
no row, whereas legacy keeps a selected NULL-valued row; the test
asserts both representations explicitly, including the legacy
timestamp/label, without filtering results or changing production
semantics. It collects the output, observes `FusedSeriesExec` and checks
native output metrics, rather than treating a logical plan or old CI run
as execution evidence.
The existing ordinary single-input/two-independent-scan tests and
four-nonempty-partition Complete Aggregate test are retained as separate
regressions. The two scans are still the #9896 same-table capability;
this PR does not claim true cross-table dual input.
## PR Checklist
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [ ] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the `backport-<target>` labels (e.g. `backport-v1.3` targets
`release/v1.3`).
The native tree additions change an internal experimental Rust
interface; no persisted metadata or wire format is changed. No release
backport is requested.
Refreshed CI runs for this published immutable HEAD are pending:
[Checks](https://github.com/GreptimeTeam/greptimedb/actions/runs/40266408649),
[Rust
CI](https://github.com/GreptimeTeam/greptimedb/actions/runs/40266408660),
[Integration
CI](https://github.com/GreptimeTeam/greptimedb/actions/runs/40266408658).
No merge is performed until these new-HEAD checks pass.
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## Refer to a related PR or issue link (optional)
Stacked on #9868 (`feat/narthex-fused-wiring`, currently at
`990d932198ab0c5e41b724a54d990e5a8ffc8acb`). This change needs that PR's
unmerged `FusedSeries`/`SeriesScan`/in-process native route, so the base
is its branch; GitHub will retarget to `narthex` once #9868 merges.
## What's changed and what's your intention?
Enable the existing opt-in native PromQL route for literal vector/scalar
arithmetic such as `x * 100`, unary negation such as `-x`, and constant
scalar statements such as `1 + 2`. These public entries return ordinary
nonempty query RecordBatches with the same values and output schema as
the existing planner.
- Traverse unary and binary expression operands when locating the one
selector and its range requirements. Constant-only statements reuse the
existing constant evaluator and statement grid; unsupported shapes still
use the existing planner.
- Execute constant Scalar roots as one unlabelled evaluated chunk on the
full statement grid, with existing memory admission, output metrics and
error/cleanup paths. A no-input tree has genuinely zero logical/physical
children rather than a discarded placeholder scan.
- Keep each raw input partition attached to its own `FusedSeriesExec`
evaluator. This change adds neither an input-side Coalesce nor a
SeriesFrame exchange. Ordinary output collection continues to use
DataFusion.
- Cover scalar NaN/negative-zero/EOF/memory boundaries, child arity and
output schemas, independent lazy input progress, and the actual public
HTTP/query-handler route. The standalone single-region fixture persists
four disjoint input batches: for both broadcast and unary queries, all
four input/native partitions perform work and the complete result is
exactly 24 labelled timestamp/value rows. A four-stream barrier test
collected by DataFusion verifies concurrent progress; this is not a
CPU-overlap or speedup benchmark.
- Add a generated SQLness compatibility case for scalar arithmetic,
broadcast and unary results, rerun successfully without result changes.
Scope: constant scalar roots and literal broadcasts only.
`scalar(vector)` adaptation, dynamic per-grid scalar
arithmetic/broadcast, global Vector/Select and Complete Aggregate wiring
are separate work and are not enabled here. The experimental
configuration/request gates and current unsupported-shape fallback
remain in place. No persisted or remote wire format changes are
introduced; internal Fused constructor callers are migrated together.
Verification on this exact HEAD
(`d9eb9778f113dd9bd6125305a9ea447b4aa6ab1d`, restacked onto the current
dependency
`990d932198ab0c5e41b724a54d990e5a8ffc8acb`), exit 0:
```text
cargo fmt --all -- --check
git diff --check
cargo check --locked -p promql -p query --tests
cargo check --locked -p tests-integration --lib --test main
cargo nextest run --locked -p promql -p query --retries 0
1870 passed, 2 skipped
cargo nextest run --locked -p tests-integration --test main --retries 0 \
-E 'test(=integration_http_file_test::test_native_promql_route_plans_the_query) | test(=integration_http_file_test::test_prometheus_native_promql_route)'
2 passed, 770 filtered out (four real input partitions, exact 24 values)
cargo nextest run --locked -p tests-integration --lib --retries 0 -E 'test(native_promql_)'
4 passed, 251 filtered out (upstream VIEW/metric/ANALYZE compatibility)
cargo clippy --locked -p promql -p query --all-targets -- -D warnings
cargo clippy --locked -p tests-integration --lib --test main -- -D warnings
```
SQLness (generated, then rerun with an unchanged result):
```text
/mnt/nvme_rust/rust-targets-2/d-scalar-entry/debug/sqlness-runner bare \
--bins-dir .../debug --case-dir <isolated root> --test-filter '^standalone:scalar_entry$' --jobs 1
```
This SQLness case uses the default public compatibility route, not the
native request flag; it does not replace the native execution assertions
above. Full workspace `make test`/`make clippy`/udeps and external CI
have not been run for this change.
## PR Checklist
Please convert it to a draft if some of the following conditions are not
met.
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the `backport-<target>` labels (e.g. `backport-v1.3` targets
`release/v1.3`).
Dennis Zhuang
discord9
discord9
