Reuse pack_evaluated_series for fused Frame output, preserving the single native driver and PlainRows/partial timing. Reject mismatched logical frame schemas and use the reset scan plan in integration tests. Add evaluated-packer lifecycle and invalid-sequence regressions, and address the review follow-ups.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## What's changed and what's your intention?
Enable vmbench-lite 05/06/08 cross-metric arithmetic using two
independently planned native operands and the existing DataFusion binary
join, PromQL matching and arithmetic projection.
Physical shape: `SeriesScan A(N) -> Fused A(N)` and `SeriesScan B(M) ->
Fused B(M)` execute region-side; separate MergeScan branches feed the
existing frontend Join. No native Join, second matching kernel or
Exchange is added; same-table fused admission is preserved.
A bounded analyzer pass locates each direct Fused(TableScan) through
Alias/Projection/Filter/Window wrappers, rewrites only the fused
subtree, and leaves all wrappers frontend-side. This preserves global
uniqueness checks. Subquery bodies disable native admission and restore
the prior flag before returning either success or error, avoiding use of
outer evaluation bounds.
### Verification
- `make fmt-check`
- `cargo check --locked -p query -p tests-integration --all-targets`
- Focused query admission/analyzer tests: 2 passed
- `cargo nextest run --locked -p tests-integration --lib -E
'test(native_hybrid_binary_05_06_08)'`: passed, independently rerun on
final source
- `cargo clippy --locked -p query --all-targets --all-features -- -D
warnings`
- Independent source review approved.
Standalone integration asserts two independent region-local
Fused->SeriesScan branches, four nonempty partitions per input, frontend
join placement, native/legacy complete-row parity, unmatched-series
removal (05), outer scalar projection (06), and independent rate inputs
(08). Additional cases cover subquery legacy fallback, valid
reduced-label `on(job)` matching, frontend uniqueness Window plus
`prom_assert_unique_match_group` Filter above MergeScan, and meaningful
duplicate-match execution errors under both routes.
### Scope and limitations
Standalone single-region per table, multiple scan partitions. Arithmetic
and admitted one-to-one matching only. No distributed Join/Exchange,
cross-region routing, group_left/right, set operators, native subqueries
or multi-field expansion. No full workspace suite or performance
benchmark is claimed. Based directly on narthex, independent of the
global-TopK PR.
## PR Checklist
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the matching labels.
I hereby agree to the terms of the [GreptimeDB
CLA](https://github.com/GreptimeTeam/.github/blob/main/CLA.md).
## What's changed and what's your intention?
Enable vmbench-lite 09, `topk(10, sum by(hostname)(rate(...[5m])))`,
through the existing native aggregate-finalize path. Split TopK/BottomK
into the global suffix above fully merged aggregate values. Other vector
operations retain their existing admission behavior. No new physical
operator or persisted/wire format is introduced.
Physical shape: `SeriesScan(N) -> FusedSeries(N, partial sum) ->
merge/coalesce -> AggregateFinalize(global TopK) -> rows`. TopK is never
partition-local.
The standalone regression uses one region, four nonempty scan
partitions, 28 series and two evaluation timestamps with different
winning hosts. It compares complete native/legacy results and
independently expected values, checks producer/finalizer plans, and
records original partial-state streams to prove the same hostname
contributes from distinct partitions at both timestamps. Partition
output counts were [14,14,14,14].
### Verification
- `make fmt-check`
- `cargo check --locked -p promql -p tests-integration --all-targets`
- `cargo nextest run --locked -p promql -E 'test(aggregate_finalize)'`:
28 passed
- `cargo nextest run --locked -p tests-integration --lib -E
'test(native_aggregate_test)'`: 2 passed
- `cargo clippy --locked -p promql --all-targets --all-features -- -D
warnings`
- Independent source review approved; main-agent targeted execution
checks passed.
### Scope and limitations
Standalone single-region, multiple scan partitions only. No distributed
Exchange, cross-region merge/routing, or new matching/subquery support.
No full workspace suite or performance benchmark is claimed. This PR is
independent of the hybrid-binary PR and is based directly on narthex.
## PR Checklist
- [x] I have written the necessary rustdoc comments.
- [x] I have added the necessary unit tests and integration tests.
- [ ] This PR requires documentation updates.
- [x] API changes are backward compatible.
- [x] Schema or data changes are backward compatible.
- [ ] This PR needs to be backported to release branches, and I have
added the matching labels.
A range query `a or on(host) b` where one host's left series covers only
the middle steps: native must keep the left sample where it exists and
take the right sample with its own labels at the other steps, matching
the explicit rows and the legacy plan. The set test's row reader moves
to module level so both cases share it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
The legacy planner refuses `and`, `or` and `unless` over several fields
(MultiFieldsNotSupported), and Prometheus has one value per sample, so
per-field presence has no reference semantics. The compiler now declines
a set operation unless each side has exactly one field, and the query
keeps the legacy plan and its error.
Errors raised by the shared two-operand runtime now name the set node
instead of calling every such node "the binary node".
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
A two-scan FusedSeries over a table split into several regions has no
placement: DistPlannerAnalyzer rejects it with "only accept plan with at
most one child", so the query failed instead of falling back. This hit
the existing arithmetic pair and, since set operators now compile, set
operators too. Decline the native route for two selectors when the table
has partition columns.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Bind `and`, `or` and `unless` (with `on`/`ignoring`) to a new
`NativeExpr::Set` node instead of refusing them. The node runs the
existing MATCH-SET kernel through the binary runtime, which retains both
operands until the input ends, so a set operation reads every series of
both sides in one execution.
The set kernel now reports its workspace like the binary kernel does:
kept rows, the filter side's match groups, the sort and the merge are
charged to the node's reservation.
The compiler refuses a fill value on a set operator, which Prometheus
rejects and the parser accepts, and sides with different field counts.
Same-table set operations over two scans take the existing two-scan
native route.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
The workspace denies the default Clippy warnings, and the new tests tripped
three of them without changing what any of them proves:
- an expected value was built through `scale as f64` even though the tuple
already carries an `f64`, so the conversion is spelled in the literal and the
cast is gone;
- the standalone replay printed the executed plan and the per-partition counts
even though the surrounding assertions already require four nonempty
partitions, 24 rows in total and eight closing frames, so the diagnostics
are gone rather than exempted from the workspace lint.
No production code, assertion or fixture changes.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Remove plan-global label-set occupancy and the speculative split-series tree guard. Keep the native evaluator, finality, memory budgets, errors and ordinary RecordBatch output local to each scan partition.
Narrow native admission to selectors and the small unary POC subset. Document that cross-partition final-label conflicts and semantic merging are outside this POC, and cover real rate/abs(rate) execution across dense and sparse storage spanning SST and memtable data.
Signed-off-by: discord9 <discord9@163.com>
Keep native queries available for unflushed memtables and SSTs with incomplete columnar series or samples coverage by selecting the ordinary raw-series reader. Preserve structural validation and propagate artifact read errors instead of retrying them through legacy execution.
Cover all four incomplete-readiness states in public PromQL queries, both storage scan modes and ordering, complete prepared-source execution, and artifact error propagation.
Signed-off-by: discord9 <discord9@163.com>
The flow and nested-MergeScan ingress tests hardcoded "follower" among
the read-preference values that must be rejected, coupling them to the
current leader-only ReadPreference enum. Assert only genuinely invalid
values there, matching the wire-values test which already limits itself
to ["", "unknown"]. This keeps the ingress tests valid for builds that
extend ReadPreference with more variants.
Signed-off-by: Ning Sun <sunning@greptime.com>
* refactor(operator): route shared requests through NodeManager
Remove the concrete NodeClients requester API and use the existing node manager for owned and shared requests. Preserve the original write hint while the legacy error adapter remains in place.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(meta): preserve retry policy through error hints
Propagate client errors through External and let DDL read retry hints while preserving cleanup and terminal-child policies. Region transport failures retain the legacy Retryable classification, which now also participates in bounded write retries.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(operator): restore sparse partition tags when rebuilding writes
Decode retained primary keys when refreshed partition expressions need different tags. Keep the partitioner and multi-region sparse write regression tests with the fix.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(operator): stop detached write retries when callers cancel
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat(frontend): configure retries across distributed write paths
Wire the shared dispatcher through frontend and logical batching, preserving partition columns and request headers. Keep standalone retries disabled and include configuration, batching, and storage regression coverage.
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
discord9
discord9
Dennis Zhuang
Yingwen
Ning Sun
Weny Xu
jeremyhi