feat(query): place a native label-set merge above the regions (#10191)

* feat(query): place the canonical merge above the regions

A canonical producer merges the pieces of one label set that its own input
partitions hold, and the canonical router FusedSeries::to_physical_plan inserts
above it collects them from exactly those partitions. In process the producer
reads the region's own scans, so the region holds every piece its router has to
see and that placement is the only one it has.

Over the wire the producer reads RawSeriesSource children instead - the wrapper
that reconciles fragments several regions hold. Classifying that shape
commutative shipped producer -> router as the region sub-plan, one producer and
one router per region, each collecting only its own region's pieces, so a label
set a time partition spread over several regions was answered from one region's
share. Classify a canonical producer by its own contract only: commutative when
every child is a bare scan, non-commutative otherwise, and never fall through to
the per-series rules, which describe what a node computes and say nothing about
where the pieces of one label set are. The analyzer then cuts at the raw source,
so the frontend holds

  consumer -> SeriesRepartitionExec(CanonicalLabels) -> producer
           -> [raw router] -> MergeScan(RawSeriesSource(TableScan))

and the canonical router collects each label set from the producers of every
region, while the fragments of each series are still reconciled below the
producer by the raw-primary-key router the interleaved raw input asks for.

Admission follows the placement: the in-process route still declines a canonical
merge over a partitioned table, because a region-local producer is the only
placement it has, and the remote route now admits it. Acceptance is a real
two-datanode test over a three-partition metric table: the rows equal an
unpartitioned control's, the producer and its canonical router are frontend
nodes above the merge scans, and a collision two regions hold reports the
Prometheus duplicate-labelset error that no single region can see.

Signed-off-by: discord9 <discord9@163.com>

* test(promql): partition the canonical acceptance table on a tag

GreptimeDB has no implicit time partitioning: a table has more than one
region only from a declared partition, and metric tables are normally
partitioned on a tag. The acceptance fixture partitioned on the time index
instead, which made the test read as if time partitioning were the everyday
case. Partition on `host` in three ranges - `early`/`first`/`late`/`second`
still land in different regions, so the merge pair and the collision pair keep
their cross-region geometry - and rename the test accordingly.

A partition on a primary-key column that excludes the time index also proves
the regions hold disjoint series, so the two assertions that described
fragmented series state what this fixture actually proves instead: the merge
scan publishes whole series (`contiguous`), and no raw-key repartition sits
below the frontend producer because only the label set crosses regions.

Signed-off-by: discord9 <discord9@163.com>

* refactor(query): ask the transport where a canonical merge runs

`canonical_boundary_region_node` inferred "this execution reads the whole
table" from the node's children: a canonical producer whose children were all
bare scans was admitted inside a region sub-plan. Over the wire every selector
input is a `RawSeriesSource`, so the inference happened to agree with the rule
that matters, but a wire plan that ever held a bare scan under a producer would
have been admitted silently and answered a label set from one region's share.

The analyzer already holds the fact. `in_process_region_transport` is what it
was built with, and it is exactly the condition under which a region read is
the whole table. Pass it down to `Categorizer::check_plan` and
`check_extension_plan` and require it instead of inspecting child shapes.

Guard the hole the inference hid: a canonical producer over a bare table scan
classifies `Commutative` in process and `NonCommutative` on the wire.

Signed-off-by: discord9 <discord9@163.com>

* test(promql): cover a fragmented canonical merge too

A partition on a primary-key column proves the regions hold disjoint series,
so the tag-partitioned fixture could not exercise the other geometry the
frontend merge has to read: fragments. In a metric table a tag is always a
primary-key column and a field must be Float64, so the only declared partition
that proves nothing about where a series lives is the time index.

Run the same two fixture rows over both geometries in one loop. The tag case
still publishes whole series and has no router below the producer; the time
case declares interleaved and asserts exactly one RawPrimaryKey reconciling
router between the merge scans and the frontend producer. Both keep the
unpartitioned control's rows, the frontend placement of the producer and its
canonical router, and the cross-region duplicate-labelset error.

Signed-off-by: discord9 <discord9@163.com>

* Revert "test(promql): cover a fragmented canonical merge too"

This reverts commit 8ff9f43031.

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
This commit is contained in:
discord9
2026-11-06 14:53:33 +08:00
committed by GitHub
parent 595abae174
commit 23afa9697e
5 changed files with 455 additions and 44 deletions
+19 -2
View File
@@ -446,6 +446,14 @@ enum RewriterStatus {
#[derive(Debug, Default)]
struct PlanRewriter {
/// Whether every region this pass reads is reached in process.
///
/// [`Categorizer`] needs it to place canonical producers: their router collects the pieces of
/// one label set from the partitions the producer runs on, so the producer belongs inside a
/// region only when region reads are served in process. It is a fact about the transport, not
/// about the plan, and every check this pass makes is a placement decision for the same
/// transport.
in_process_region_transport: bool,
/// Whether the whole plan encodes, allowing per-node checks to be skipped.
whole_plan_encodable: bool,
/// Whether an in-process root fused plan was recognized for placement-gate evaluation.
@@ -638,6 +646,7 @@ impl PlanRewriter {
.encode(plan, DefaultSerializer)
.is_ok();
Self {
in_process_region_transport,
in_process_fused_root,
whole_plan_encodable,
..Default::default()
@@ -688,7 +697,11 @@ impl PlanRewriter {
}
if self.expand_on_next_part_cond_trans_commutative {
let comm = Categorizer::check_plan(plan, self.partition_cols.clone())?;
let comm = Categorizer::check_plan(
plan,
self.partition_cols.clone(),
self.in_process_region_transport,
)?;
match comm {
Commutativity::PartialCommutative => {
// a small difference is that for partial commutative, we still need to
@@ -713,7 +726,11 @@ impl PlanRewriter {
}
}
match Categorizer::check_plan(plan, self.partition_cols.clone())? {
match Categorizer::check_plan(
plan,
self.partition_cols.clone(),
self.in_process_region_transport,
)? {
Commutativity::Commutative => {
// PATCH: we should reconsider SORT's commutativity instead of doing this trick.
// explain: for a fully commutative SeriesDivide, its child Sort plan only serves it. I.e., that
+129 -32
View File
@@ -32,28 +32,29 @@ use crate::dist_plan::analyzer::AliasMapping;
use crate::dist_plan::merge_sort::{MergeSortLogicalPlan, merge_sort_transformer};
use crate::promql::planner::native::{independent_scan_pair, pointwise_operand, same_table_scans};
/// Whether `fused` is a canonical producer whose prefix and sources can run in one region.
/// Its physical canonical router assembles the complete label sets; ordinary consumers above it
/// do not carry the producer marker and retain their normal placement requirements.
fn canonical_boundary_region_node(fused: &FusedSeries) -> bool {
/// Whether `fused` is a canonical producer that may run inside one region.
///
/// A canonical producer merges the pieces of one label set that its own input partitions hold, and
/// its physical canonical router collects those pieces from exactly those partitions. It may only
/// run where those partitions cover the whole table. In process that is the region's own read: the
/// region holds every piece its router has to see and the node belongs there. Ordinary consumers
/// above it do not carry the producer marker and retain their normal placement requirements.
///
/// Over the wire every selector input is a [`RawSeriesSource`] answered per region, so no region
/// holds every piece of a label set. A producer placed below the raw sources would run once per
/// region and its router would collect the pieces of the region it runs in and no others, which
/// answers a label set from one region's share whenever the pieces are spread. Declining this
/// classification is what keeps that from happening: the analyzer expands each raw source into a
/// merge scan and the producer, with the router it inserts above itself, stays on the frontend over
/// every region.
fn canonical_boundary_region_node(fused: &FusedSeries, in_process_region_transport: bool) -> bool {
if !fused.is_canonical_producer() {
return false;
}
fused.format() == FusedOutputFormat::SeriesFrame
in_process_region_transport
&& fused.format() == FusedOutputFormat::SeriesFrame
&& fused.canonical_producer_prefix_is_local()
&& !fused.children().is_empty()
&& fused.children().iter().all(|child| match child {
LogicalPlan::TableScan(_) => true,
LogicalPlan::Extension(extension)
if extension.node.as_any().is::<RawSeriesSource>() =>
{
matches!(
extension.node.inputs().as_slice(),
[LogicalPlan::TableScan(_)]
)
}
_ => false,
})
}
pub struct StepTransformAction {
@@ -121,6 +122,7 @@ impl Categorizer {
pub fn check_plan(
plan: &LogicalPlan,
partition_cols: Option<AliasMapping>,
in_process_region_transport: bool,
) -> DfResult<Commutativity> {
// Subquery is treated separately in `inspect_plan_with_subquery`. To avoid rewrite the
// "maybe rewritten" plan, stop the check here.
@@ -214,9 +216,11 @@ impl Categorizer {
Commutativity::Unimplemented
}
}
LogicalPlan::Extension(extension) => {
Self::check_extension_plan(extension.node.as_ref() as _, &partition_cols)
}
LogicalPlan::Extension(extension) => Self::check_extension_plan(
extension.node.as_ref() as _,
&partition_cols,
in_process_region_transport,
),
LogicalPlan::Distinct(_) => {
if partition_cols.is_empty() {
Commutativity::Commutative
@@ -242,6 +246,7 @@ impl Categorizer {
pub fn check_extension_plan(
plan: &dyn UserDefinedLogicalNode,
partition_cols: &AliasMapping,
in_process_region_transport: bool,
) -> Commutativity {
match plan.name() {
name if name == SeriesDivide::name() => {
@@ -269,8 +274,17 @@ impl Categorizer {
}
name if name == FusedSeries::name() => {
let fused = plan.as_any().downcast_ref::<FusedSeries>().unwrap();
if canonical_boundary_region_node(fused) {
return Commutativity::Commutative;
if fused.is_canonical_producer() {
// A canonical producer is placed by its own contract and not by the shape below
// it: either the region holds every piece its router collects, or the node stays
// above the raw sources with its router. It must not fall through to the
// per-series rules below, which describe what a node computes and say nothing
// about where the pieces of one label set are.
return if canonical_boundary_region_node(fused, in_process_region_transport) {
Commutativity::Commutative
} else {
Commutativity::NonCommutative
};
}
// The raw-series reader supports one scan, or the narrow same-table multi-scan
// one-to-one case. This classification is for region-local execution only.
@@ -450,7 +464,7 @@ mod tests {
};
use promql::native::plan::{
AggregateKind, AggregateStage, AggregateStateSchema, BoundAggregate, BoundInput, Grouping,
NativeExpr, NativeNode, NativeType,
NativeExpr, NativeNode, NativeTree, NativeType,
};
use promql_parser::parser;
use store_api::storage::RegionId;
@@ -476,7 +490,8 @@ mod tests {
BTreeSet::from([Column::from_name("some_partition_col")]),
)]);
let commutativity = Categorizer::check_extension_plan(&series_divide, &partition_cols);
let commutativity =
Categorizer::check_extension_plan(&series_divide, &partition_cols, false);
assert!(matches!(commutativity, Commutativity::Commutative));
}
@@ -617,6 +632,52 @@ mod tests {
})
}
/// A canonical producer of `some_metric` over `input`.
///
/// The tree says what the compiler says at a canonical boundary - one evaluated vector merged by
/// canonical label set - and the `Merge` root is what marks the node as the producer of that
/// collection. The children are exactly what the caller supplies, so a test can hand the node
/// the region's own scan and ask where such a node may run.
fn canonical_producer_plan(input: LogicalPlan) -> LogicalPlan {
let stmt = native_statement();
let grid = statement_grid(&stmt).unwrap();
let evaluated = NativeType::Samples(Arc::new(SeriesSchema {
fields: Arc::from([FieldSpec { name: "v0".into() }]),
eval_grid: Some(Arc::clone(&grid)),
}));
let merge = NativeNode {
expr: NativeExpr::Merge {
input: Box::new(NativeNode {
expr: NativeExpr::Input(BoundInput {
output: evaluated.clone(),
source_index: 0,
ordering: SeriesOrdering::KeyThenTime,
}),
output: evaluated.clone(),
}),
},
output: evaluated,
};
let binding = FusedSeriesBinding {
grid,
time_index: "ts".to_string(),
tags: Vec::new(),
metric_name: Some(Arc::from("some_metric")),
};
let tree = NativeTree::try_new(merge).unwrap();
LogicalPlan::Extension(Extension {
node: Arc::new(
FusedSeries::try_new_canonical_producer(
Arc::from("some_metric"),
tree,
binding,
vec![input],
)
.unwrap(),
),
})
}
/// The region's own view of the metric's table: a scan whose provider is the region provider.
fn table_scan_plan() -> LogicalPlan {
let provider = mock_table_provider(RegionId::new(1, 1));
@@ -786,7 +847,7 @@ mod tests {
let plan = fused_series_plan(table_scan_plan());
assert!(matches!(
Categorizer::check_plan(&plan, None).unwrap(),
Categorizer::check_plan(&plan, None, true).unwrap(),
Commutativity::Commutative
));
}
@@ -804,7 +865,7 @@ mod tests {
let plan = fused_series_plan(input);
assert!(matches!(
Categorizer::check_plan(&plan, None).unwrap(),
Categorizer::check_plan(&plan, None, true).unwrap(),
Commutativity::NonCommutative
));
}
@@ -816,7 +877,35 @@ mod tests {
let plan = fused_series_plan_named(table_scan_plan(), None);
assert!(matches!(
Categorizer::check_plan(&plan, None).unwrap(),
Categorizer::check_plan(&plan, None, true).unwrap(),
Commutativity::NonCommutative
));
}
/// The transport decides where a canonical producer runs, not the shape of its children: its
/// router collects the pieces of one label set from exactly the partitions the producer runs
/// on, so those partitions have to be the region's whole read - which is only true in process.
/// A bare scan underneath must not on its own admit the producer to a wire plan.
#[test]
fn a_canonical_producer_is_not_commutative_on_the_wire_even_over_a_bare_scan() {
let plan = canonical_producer_plan(table_scan_plan());
let LogicalPlan::Extension(extension) = &plan else {
unreachable!();
};
assert!(matches!(
Categorizer::check_extension_plan(
extension.node.as_ref(),
&AliasMapping::default(),
true
),
Commutativity::Commutative
));
assert!(matches!(
Categorizer::check_extension_plan(
extension.node.as_ref(),
&AliasMapping::default(),
false
),
Commutativity::NonCommutative
));
}
@@ -878,7 +967,11 @@ mod tests {
unreachable!();
};
assert!(matches!(
Categorizer::check_extension_plan(extension.node.as_ref(), &AliasMapping::default()),
Categorizer::check_extension_plan(
extension.node.as_ref(),
&AliasMapping::default(),
true
),
Commutativity::Commutative
));
@@ -897,7 +990,11 @@ mod tests {
unreachable!();
};
assert!(matches!(
Categorizer::check_extension_plan(extension.node.as_ref(), &AliasMapping::default()),
Categorizer::check_extension_plan(
extension.node.as_ref(),
&AliasMapping::default(),
true
),
Commutativity::NonCommutative
));
}
@@ -1167,7 +1264,7 @@ mod tests {
fn a_split_aggregate_is_commutative_only_below_the_cut() {
let plan = split_aggregate_plan(table_scan_plan());
assert!(matches!(
Categorizer::check_plan(&plan, None).unwrap(),
Categorizer::check_plan(&plan, None, true).unwrap(),
Commutativity::NonCommutative
));
@@ -1176,7 +1273,7 @@ mod tests {
panic!("expected the final node to have one partial child: {plan}");
};
assert!(matches!(
Categorizer::check_plan(partial, None).unwrap(),
Categorizer::check_plan(partial, None, true).unwrap(),
Commutativity::Commutative
));
}
@@ -1363,7 +1460,7 @@ mod test {
fetch: None,
});
assert!(matches!(
Categorizer::check_plan(&plan, Some(Default::default())).unwrap(),
Categorizer::check_plan(&plan, Some(Default::default()), false).unwrap(),
Commutativity::Commutative
));
}
+22 -10
View File
@@ -180,8 +180,8 @@ type DiscoveredSelectorMetadata = (Arc<SeriesSchema>, String, Vec<String>, Arc<s
///
/// `partitioned` says whether the table behind that scan is split into regions along a partition
/// column. Nothing in the scan itself carries this: a partition is a placement fact about the
/// table, and the canonical route has to know it before it can claim a label-set answer covers the
/// whole query rather than one region's share of it.
/// table, and the in-process canonical route has to know it before it can claim a label-set answer
/// covers the whole query rather than one region's share of it.
type PreparedInput = (
LogicalPlan,
Arc<SeriesSchema>,
@@ -368,9 +368,10 @@ impl PromPlanner {
};
let table = self.table_from_source(&scan.source)?;
let table_meta = &table.table_info().meta;
// A table split along a partition column lives in more than one region. The canonical
// boundary's router only collects the pieces its own region's producer holds, so the caller
// needs this fact to tell a whole-query label-set answer from one region's share of it.
// A table split along a partition column lives in more than one region. In process the
// canonical boundary's router only collects the pieces its own region's producer holds, so
// the caller needs this fact to tell a whole-query label-set answer from one region's share
// of it.
let partitioned = table_meta.partition_column_names().next().is_some();
// An engine without a raw-series scanner has no frame to give this node, and by the
// time the physical planner asks, the logical plan is already this node: the refusal
@@ -1124,11 +1125,22 @@ impl PromPlanner {
// region-local producer only sees the partitions of the region it runs in. A table split
// along a partition column is planned region by region, so each region would answer a
// canonical label set from the piece it holds while the consumer above has no collection of
// the other regions' pieces. Without a proof that this query reads one region, decline the
// route instead of claiming a global canonical answer; the existing planner reads every
// region and reports what Prometheus reports. A tree with no canonical merge claims no such
// answer, so it is untouched: a per-series query on a partitioned table keeps fusing.
if any_partitioned && contains_canonical_merge(root) {
// the other regions' pieces.
//
// In process the producer reads the region's own scans, so that placement is the only one it
// has: without a proof that this query reads one region, decline the route instead of
// claiming a global canonical answer, and let the existing planner - which reads every
// region and reports what Prometheus reports - answer. A tree with no canonical merge claims
// no such answer, so it is untouched: a per-series query on a partitioned table keeps fusing.
//
// Over the wire the producer reads `RawSeriesSource` children, and
// `Categorizer::check_extension_plan` keeps such a producer out of a region
// (`canonical_boundary_region_node` admits only a region's own scans). The analyzer expands
// the raw sources into merge scans and the producer runs on the frontend above every region,
// with its canonical router above all of them, so each label set is collected from the
// producers of every region. That placement is what the remote route claims, and the shape it
// builds is the one the acceptance test asserts: no canonical producer below a merge scan.
if !self_contained && any_partitioned && contains_canonical_merge(root) {
return Ok(None);
}
// The kind the tree's root answers with, which every output column follows. It is not the
+2
View File
@@ -3989,6 +3989,8 @@ async fn native_fused_info_over_its_own_table_reads_two_scans_in_the_region() {
crate::dist_plan::Categorizer::check_extension_plan(
fused as &dyn datafusion::logical_expr::UserDefinedLogicalNode,
&Default::default(),
// The node reads two scans of the region's own table, which is the in-process read.
true,
),
crate::dist_plan::Commutativity::Commutative
));