A one-to-one comparison between two selectors, such as `a > on(host) b`
or `a == bool b`, kept the existing plan: the pair admission accepted
only arithmetic operators. The kernel already pairs comparisons by the
same matching rules and only differs in the value a pair keeps (the
left sample when the comparison holds, or 1/0 with `bool`), so the
operator check is dropped. This covers one table and two, and the
point-wise wrappers around a pair, such as `(a > b) * 100`.
Adds an end-to-end case over two tables comparing explicit rows, the
legacy answer and the native placement for a filter and a `bool`
comparison.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Reuse pack_evaluated_series for fused Frame output, preserving the single native driver and PlainRows/partial timing. Reject mismatched logical frame schemas and use the reset scan plan in integration tests. Add evaluated-packer lifecycle and invalid-sequence regressions, and address the review follow-ups.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## What's changed and what's your intention?
Enable vmbench-lite 05/06/08 cross-metric arithmetic using two
independently planned native operands and the existing DataFusion binary
join, PromQL matching and arithmetic projection.
Physical shape: `SeriesScan A(N) -> Fused A(N)` and `SeriesScan B(M) ->
Fused B(M)` execute region-side; separate MergeScan branches feed the
existing frontend Join. No native Join, second matching kernel or
Exchange is added; same-table fused admission is preserved.
A bounded analyzer pass locates each direct Fused(TableScan) through
Alias/Projection/Filter/Window wrappers, rewrites only the fused
subtree, and leaves all wrappers frontend-side. This preserves global
uniqueness checks. Subquery bodies disable native admission and restore
the prior flag before returning either success or error, avoiding use of
outer evaluation bounds.
### Verification
- `make fmt-check`
- `cargo check --locked -p query -p tests-integration --all-targets`
- Focused query admission/analyzer tests: 2 passed
- `cargo nextest run --locked -p tests-integration --lib -E
'test(native_hybrid_binary_05_06_08)'`: passed, independently rerun on
final source
- `cargo clippy --locked -p query --all-targets --all-features -- -D
warnings`
- Independent source review approved.
Standalone integration asserts two independent region-local
Fused->SeriesScan branches, four nonempty partitions per input, frontend
join placement, native/legacy complete-row parity, unmatched-series
removal (05), outer scalar projection (06), and independent rate inputs
(08). Additional cases cover subquery legacy fallback, valid
reduced-label `on(job)` matching, frontend uniqueness Window plus
`prom_assert_unique_match_group` Filter above MergeScan, and meaningful
duplicate-match execution errors under both routes.
### Scope and limitations
Standalone single-region per table, multiple scan partitions. Arithmetic
and admitted one-to-one matching only. No distributed Join/Exchange,
cross-region routing, group_left/right, set operators, native subqueries
or multi-field expansion. No full workspace suite or performance
benchmark is claimed. Based directly on narthex, independent of the
global-TopK PR.
## PR Checklist
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the matching labels.
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## What's changed and what's your intention?
Enable vmbench-lite 09, `topk(10, sum by(hostname)(rate(...[5m])))`,
through the existing native aggregate-finalize path. Split TopK/BottomK
into the global suffix above fully merged aggregate values. Other vector
operations retain their existing admission behavior. No new physical
operator or persisted/wire format is introduced.
Physical shape: `SeriesScan(N) -> FusedSeries(N, partial sum) ->
merge/coalesce -> AggregateFinalize(global TopK) -> rows`. TopK is never
partition-local.
The standalone regression uses one region, four nonempty scan
partitions, 28 series and two evaluation timestamps with different
winning hosts. It compares complete native/legacy results and
independently expected values, checks producer/finalizer plans, and
records original partial-state streams to prove the same hostname
contributes from distinct partitions at both timestamps. Partition
output counts were [14,14,14,14].
### Verification
- `make fmt-check`
- `cargo check --locked -p promql -p tests-integration --all-targets`
- `cargo nextest run --locked -p promql -E 'test(aggregate_finalize)'`:
28 passed
- `cargo nextest run --locked -p tests-integration --lib -E
'test(native_aggregate_test)'`: 2 passed
- `cargo clippy --locked -p promql --all-targets --all-features -- -D
warnings`
- Independent source review approved; main-agent targeted execution
checks passed.
### Scope and limitations
Standalone single-region, multiple scan partitions only. No distributed
Exchange, cross-region merge/routing, or new matching/subquery support.
No full workspace suite or performance benchmark is claimed. This PR is
independent of the hybrid-binary PR and is based directly on narthex.
## PR Checklist
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the matching labels.
Under `or` the right table's tags join the left side's columns, and one of
them can share a name with the left side's field or time index; building
the output schema then failed instead of leaving the query to the existing
plan. Such a union is now declined. The output-label walk also declines an
aggregate or whole-vector node instead of passing its input's labels
through, which would describe the wrong schema if one were ever reached.
Adds an end-to-end case over two tables whose tags, field and time index
are all named differently, partly flushed and partly in the memtable,
under both series scan modes.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
A binary or set operation between selectors of two tables, such as
`rate(a[5m]) / rate(b[5m])`, fell back to the existing plan: the native
route required every input to share one metric, time index and tag list,
and a fused node is placed inside the one region it reads.
Two tables are two regions, so such a pair now runs on the frontend. Each
input becomes a `RawSeriesSource` that rides in its own merge scan: the
region binds its scan to that table's metric, tags and fields, decodes the
labels and answers one explicit-label raw frame stream, reading its scan
partitions one after another so each series stays contiguous. The fused
node consumes those frames directly. Pairs on one table keep running in
the region; partitioned tables keep the existing plan.
A point-wise wrapper around an admitted pair, such as
`100 * (1 - a / b)`, is admitted too, on one table and across two. The
output labels are now derived from the whole tree with each input's own
tags, and the physical node takes them from the output schema instead of
narrowing by the root alone.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
A range query `a or on(host) b` where one host's left series covers only
the middle steps: native must keep the left sample where it exists and
take the right sample with its own labels at the other steps, matching
the explicit rows and the legacy plan. The set test's row reader moves
to module level so both cases share it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
The legacy planner refuses `and`, `or` and `unless` over several fields
(MultiFieldsNotSupported), and Prometheus has one value per sample, so
per-field presence has no reference semantics. The compiler now declines
a set operation unless each side has exactly one field, and the query
keeps the legacy plan and its error.
Errors raised by the shared two-operand runtime now name the set node
instead of calling every such node "the binary node".
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
A two-scan FusedSeries over a table split into several regions has no
placement: DistPlannerAnalyzer rejects it with "only accept plan with at
most one child", so the query failed instead of falling back. This hit
the existing arithmetic pair and, since set operators now compile, set
operators too. Decline the native route for two selectors when the table
has partition columns.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Bind `and`, `or` and `unless` (with `on`/`ignoring`) to a new
`NativeExpr::Set` node instead of refusing them. The node runs the
existing MATCH-SET kernel through the binary runtime, which retains both
operands until the input ends, so a set operation reads every series of
both sides in one execution.
The set kernel now reports its workspace like the binary kernel does:
kept rows, the filter side's match groups, the sort and the merge are
charged to the node's reservation.
The compiler refuses a fill value on a set operator, which Prometheus
rejects and the parser accepts, and sides with different field counts.
Same-table set operations over two scans take the existing two-scan
native route.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
The workspace denies the default Clippy warnings, and the new tests tripped
three of them without changing what any of them proves:
- an expected value was built through `scale as f64` even though the tuple
already carries an `f64`, so the conversion is spelled in the literal and the
cast is gone;
- the standalone replay printed the executed plan and the per-partition counts
even though the surrounding assertions already require four nonempty
partitions, 24 rows in total and eight closing frames, so the diagnostics
are gone rather than exempted from the workspace lint.
No production code, assertion or fixture changes.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Remove plan-global label-set occupancy and the speculative split-series tree guard. Keep the native evaluator, finality, memory budgets, errors and ordinary RecordBatch output local to each scan partition.
Narrow native admission to selectors and the small unary POC subset. Document that cross-partition final-label conflicts and semantic merging are outside this POC, and cover real rate/abs(rate) execution across dense and sparse storage spanning SST and memtable data.
Signed-off-by: discord9 <discord9@163.com>
Keep native queries available for unflushed memtables and SSTs with incomplete columnar series or samples coverage by selecting the ordinary raw-series reader. Preserve structural validation and propagate artifact read errors instead of retrying them through legacy execution.
Cover all four incomplete-readiness states in public PromQL queries, both storage scan modes and ordering, complete prepared-source execution, and artifact error propagation.
Signed-off-by: discord9 <discord9@163.com>