mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-11-17 16:24:07 +00:00
feat(promql): run subqueries through the native prefix (#10085)
* feat(promql): run subqueries through the native prefix A subquery evaluates its inner expression on its own grid, so the native route now plans it instead of sending every statement containing one to the existing planner. - `collect_selectors` recurses into a subquery's expression, so its selectors are prepared like any other and each gets its scan bounds from `raw_scan_bounds`. - `calls_rate_or_increase` recurses too, so a delta-temporality table still keeps a rate or an increase inside a subquery on the existing planner. - A window whose input is already evaluated - a subquery - is admitted above the split aggregate when its declared grid equals its direct input's grid, and the suffix is rebuilt with it. A raw-sample window still cannot sit above the cut. - A `FusedSeries` whose root is such a window publishes on that window's output grid, which is not necessarily the statement's; the binding follows it at every construction site, and the split's partial state keeps the cut aggregate's own grid. An evaluated-input window does not need an owner router for its sources, and a segment whose root is one is cut on its own grid. Verified with `-p promql`, `-p query`, `tests-integration --lib` (native_promql) and clippy. Signed-off-by: discord9 <discord9@163.com> * fix(promql): keep a subquery's aggregate operand and gate an evaluated cut Two shapes reached the native route and failed while planning instead of keeping the plan they had. A hybrid-binary operand folds a group one subquery down (`avg_over_time((sum by (job, host) (pm))[5m:1m]) + on(job, host) pn`): `single_selector_operand` counts the subquery's selector, so the operand looked admissible, while `contains_aggregate_expression` stopped at `PromExpr::Subquery` and never saw the aggregate. The admission then built a `FusedSeries` the physical planner could not convert (`No installed planner was able to convert the custom node to an execution plan ...`). The walk descends into a subquery again, and both statements keep the existing plan with the same rows. A window over an evaluated input was cut as a segment regardless of locality. For any such window whose input is not a bare `Input` - a raw window, a `label_replace` - `reads_across_series` is true and `pointwise_operand` false, so the `FusedSeries` arm of the dist analyzer answers `NonCommutative`: the segment stays above the `MergeScan` and nothing converts it (`sum by (job) (max_over_time(rate(pm[2m])[5m:1m]))` and two neighbours failed that way at instant and range evaluation). `region_local_segment` mirrors that arm's decision from the tree, and `cut_as_segment` declines an evaluated-input window it refuses, cutting the per-series prefix inside the window instead; where there is none, the statement keeps the plan it had. The analyzer is not extended: every range function but `last_over_time`/`first_over_time` drops the metric name, so such a window is genuinely cross-series and a region-local segment would publish two series under one label set. `build_evaluated_segment` stays, and so does its caller. The canonical pass inserts a merge under a label rewrite, and that boundary is only ever built from the frames the loop has published, so the cut needs the fallback; deleting it took every rewritten fold off the route, which is what `sum(label_replace(...))` and the histogram-over-rewrite boundary caught. The frames-bound segment itself is not the defect - the gate is: the evaluated-input window is declined before any cut is taken, so the fallback only sees cuts the analyzer places. Tests that asserted the older rejection are decided on evidence: `sum_over_time(rate(m[5m])[1h:])` is admitted now and its rows match the existing plan, so the plan-level test asserts the route and the integration case proves the rows; the aggregate-subquery case in `native_aggregate_complete_over_four_nonempty_mito_partitions` is answered by the split (one `Partial` producer, one frontend merge) and returns the same 15.0 on both routes, so it moved to the split cases rather than being dropped. Two fold-over-subquery shapes in `native_promql_subqueries_match_legacy` (`sum by (job) (max_over_time(rate(...)[5m:1m]))` and the fold above it) are answered natively for the same reason - the fallback builds their cut - and their rows equal the existing plan's, so they assert the route instead of the fallback; the rewrite shape still declines, because its window reads across series and there is no per-series prefix inside it to take instead. Signed-off-by: discord9 <discord9@163.com> * test(promql): assert anchored subqueries do not re-fold across steps Signed-off-by: discord9 <discord9@163.com> * test(promql): drop the unused fallback-case helper Signed-off-by: discord9 <discord9@163.com> * test(promql): update subquery expectations for the native route Signed-off-by: discord9 <discord9@163.com> * fix(promql): publish subquery segments on their declared grid Signed-off-by: discord9 <discord9@163.com> * refactor(promql): drop unreachable segment arms and pin subquery routes Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>
This commit is contained in:
@@ -1779,7 +1779,13 @@ fn tree_inputs(tree: &NativeTree) -> Vec<&crate::native::plan::BoundInput> {
|
||||
fn partitioned_window_sources(node: &NativeNode) -> DataFusionResult<Vec<usize>> {
|
||||
fn visit(node: &NativeNode, sources: &mut Vec<usize>) -> DataFusionResult<()> {
|
||||
match &node.expr {
|
||||
NativeExpr::Window { input, .. } => {
|
||||
NativeExpr::Window { input, op } => {
|
||||
// A subquery window reads an evaluated input, whose fragments belong to one series
|
||||
// wherever they were produced, so it needs no owner of its own. A raw window
|
||||
// anywhere below it still does, so the walk continues into the input.
|
||||
if op.input_grid.is_some() {
|
||||
return visit(input, sources);
|
||||
}
|
||||
let NativeExpr::Input(bound) = &input.expr else {
|
||||
return Err(DataFusionError::Plan(
|
||||
"partitioned FusedSeries windows require a direct Input under Window"
|
||||
|
||||
@@ -70,8 +70,10 @@ pub(crate) fn histogram_group_is_single(group: &Grouping) -> bool {
|
||||
/// the merged values. `topk`, a second aggregate and vector-vector matching still need a whole-vector
|
||||
/// boundary.
|
||||
///
|
||||
/// An instant selection or window above the cut cannot be applied to the merged values under the
|
||||
/// tree's current binding: the cut output is evaluated, whereas these nodes consume raw samples.
|
||||
/// An instant selection or a window over raw samples above the cut cannot be applied to the merged
|
||||
/// values under the tree's current binding: the cut output is evaluated, whereas these nodes consume
|
||||
/// raw samples. A subquery window instead reads an evaluated input, so it is admitted when its
|
||||
/// declared grid equals its direct input's grid.
|
||||
pub(crate) fn split(
|
||||
node: &NativeNode,
|
||||
fields: &[FieldSpec],
|
||||
@@ -228,6 +230,34 @@ pub(crate) fn split(
|
||||
..
|
||||
} => Err(declined("a whole-vector operation above a split aggregate")),
|
||||
NativeExpr::Merge { .. } => Err(declined("an aggregate split across a canonical merge")),
|
||||
// A window whose input is already evaluated - a subquery - applies to the merged values the
|
||||
// suffix reads, provided it evaluates on that input's own grid. The declared grid must equal
|
||||
// its direct input's grid; the cut's grid need not be that grid, because a window between
|
||||
// them may already have re-gridded the values. A raw-sample window cannot sit above the cut.
|
||||
NativeExpr::Window { input, op } => {
|
||||
let Some(mut inner) = split(input, fields, time_index)? else {
|
||||
return Ok(None);
|
||||
};
|
||||
let input_grid = input
|
||||
.output
|
||||
.samples()
|
||||
.and_then(|schema| schema.eval_grid.as_ref());
|
||||
if op.input_grid.as_ref().is_none_or(|declared| {
|
||||
input_grid.is_none_or(|input| declared.spec() != input.spec())
|
||||
}) {
|
||||
return Err(declined(
|
||||
"a window above an aggregate with a different evaluation grid",
|
||||
));
|
||||
}
|
||||
inner.suffix = NativeNode {
|
||||
expr: NativeExpr::Window {
|
||||
input: Box::new(inner.suffix),
|
||||
op: op.clone(),
|
||||
},
|
||||
output: node.output.clone(),
|
||||
};
|
||||
Ok(Some(inner))
|
||||
}
|
||||
// These read values the merged state does not answer: an instant selection or window reads
|
||||
// raw samples, `scalar`/`vector` answers one value or one series instead of the vector, and a
|
||||
// parameterized map reads the whole vector at a point. `ParameterizedTopK` declines here
|
||||
@@ -235,7 +265,6 @@ pub(crate) fn split(
|
||||
// a bound one does, so it needs the same whole-vector boundary. Admitting the one and
|
||||
// refusing the other would be an inconsistency rather than a capability.
|
||||
NativeExpr::Instant { input, .. }
|
||||
| NativeExpr::Window { input, .. }
|
||||
| NativeExpr::Timestamp { input }
|
||||
| NativeExpr::Vector { input, .. }
|
||||
| NativeExpr::VectorOf { input }
|
||||
@@ -541,9 +570,9 @@ impl Split {
|
||||
|
||||
/// Puts `merged` at the cut and returns the suffix above it.
|
||||
///
|
||||
/// A suffix is the spine of point-wise maps, constant-sided binaries and at most one histogram
|
||||
/// operation between the cut and the root, with scalar constants and the evaluation time preserved.
|
||||
/// Other whole-vector operations are refused by `split`.
|
||||
/// A suffix is the spine of point-wise maps, constant-sided binaries, evaluated-input windows and at
|
||||
/// most one histogram operation between the cut and the root, with scalar constants and the
|
||||
/// evaluation time preserved. Other whole-vector operations are refused by `split`.
|
||||
fn enter_suffix(node: NativeNode, merged: &NativeNode) -> DfResult<NativeNode> {
|
||||
let NativeNode { expr, output } = node;
|
||||
let rejected = || {
|
||||
@@ -557,6 +586,11 @@ fn enter_suffix(node: NativeNode, merged: &NativeNode) -> DfResult<NativeNode> {
|
||||
NativeExpr::Scalar(_) | NativeExpr::EvalTime { .. } => {
|
||||
return Ok(NativeNode { expr, output });
|
||||
}
|
||||
// A subquery window above the cut is rebuilt over the merged values, on its own grid.
|
||||
NativeExpr::Window { input, op } => NativeExpr::Window {
|
||||
input: Box::new(enter_suffix(*input, merged)?),
|
||||
op,
|
||||
},
|
||||
NativeExpr::Map { input, op } => NativeExpr::Map {
|
||||
input: Box::new(enter_suffix(*input, merged)?),
|
||||
op,
|
||||
|
||||
@@ -1279,12 +1279,13 @@ impl PromPlanner {
|
||||
let Some(rawinput) = children.into_iter().next() else {
|
||||
return Ok(None);
|
||||
};
|
||||
let publish_grid = fused_publish_grid(final_tree.root(), &grid);
|
||||
let node = plan_split_aggregate(
|
||||
Arc::from(prom_expr.to_string()),
|
||||
partial,
|
||||
final_tree,
|
||||
FusedSeriesBinding {
|
||||
grid,
|
||||
grid: publish_grid,
|
||||
time_index: time.clone(),
|
||||
tags,
|
||||
metric_name: metric,
|
||||
@@ -1374,11 +1375,12 @@ impl PromPlanner {
|
||||
} else {
|
||||
(tags, metric)
|
||||
};
|
||||
let publish_grid = fused_publish_grid(tree.root(), &grid);
|
||||
let node = FusedSeries::try_new(
|
||||
Arc::from(prom_expr.to_string()),
|
||||
tree,
|
||||
FusedSeriesBinding {
|
||||
grid,
|
||||
grid: publish_grid,
|
||||
time_index: time,
|
||||
tags,
|
||||
metric_name: metric,
|
||||
@@ -1631,15 +1633,49 @@ fn exportable(node: &NativeNode) -> bool {
|
||||
.is_some_and(|schema| schema.eval_grid.is_some())
|
||||
}
|
||||
|
||||
/// Whether `node` is a window over an evaluated input - the shape a subquery over anything but a
|
||||
/// bare selection compiles to.
|
||||
fn evaluated_input_window(node: &NativeNode) -> bool {
|
||||
matches!(&node.expr, NativeExpr::Window { op, .. } if op.input_grid.is_some())
|
||||
}
|
||||
|
||||
/// Whether the dist analyzer places a segment over `node` region-locally.
|
||||
///
|
||||
/// A segment is a `FusedSeries` node the analyzer has to move below the `MergeScan` for the physical
|
||||
/// planner to read the scans it names. The `FusedSeries` arm of `Categorizer::check_extension_plan`
|
||||
/// (`src/query/src/dist_plan/commutativity.rs`) is what decides that, from the bound tree alone: a
|
||||
/// root that [`reads_across_series`] is `NonCommutative` unless it is a per-series evaluation or the
|
||||
/// `Partial` half of a split fold. A segment it refuses stays above the merge, where the physical
|
||||
/// planner finds neither a region scan nor an explicit-label frame below it, so the statement fails
|
||||
/// while being planned instead of keeping the plan it had. Segmentation therefore asks that question
|
||||
/// before it cuts, and [`cut_as_segment`] declines the nodes this refuses.
|
||||
fn region_local_segment(node: &NativeNode) -> bool {
|
||||
!reads_across_series(node) || pointwise_operand(node) || independent_scan_pair(node)
|
||||
}
|
||||
|
||||
/// Whether `node` becomes a segment of its own under a parent that must gather.
|
||||
///
|
||||
/// Cut local prefixes, global consumers and canonical merges separately so the prefix keeps its
|
||||
/// scan width, and every global consumer sees the canonical boundary below it. A pointwise wrapper
|
||||
/// of a global consumer is cut as well, so it cannot hide that consumer from recursive lowering.
|
||||
fn cut_as_segment(node: &NativeNode, canonical_segmentation: bool) -> bool {
|
||||
// A window over an evaluated input answers per series on its own grid, so its own shape says
|
||||
// nothing about where a segment over it can go: it drops the metric name, so two of the series
|
||||
// it answers can end under one label set, and the dist analyzer reads it as cross-series
|
||||
// ([`region_local_segment`]). Cutting one where the analyzer will not place it leaves a
|
||||
// `FusedSeries` above the `MergeScan` that the physical planner cannot build, so the segment is
|
||||
// only taken where the analyzer places it region-locally; otherwise the per-series prefix
|
||||
// inside the window is cut instead, and where there is none the statement keeps the plan it had.
|
||||
if evaluated_input_window(node) && !region_local_segment(node) {
|
||||
return false;
|
||||
}
|
||||
// A window over an evaluated input is answered per series on its own grid, so it is not the
|
||||
// global gather a raw window's locality reports.
|
||||
let evaluated_window = evaluated_input_window(node);
|
||||
if !exportable(node)
|
||||
|| matches!(node.expr, NativeExpr::Input(_))
|
||||
|| (!canonical_segmentation
|
||||
&& !evaluated_window
|
||||
&& matches!(input_locality(node), InputLocality::SinglePartition))
|
||||
{
|
||||
return false;
|
||||
@@ -1873,6 +1909,21 @@ fn cut_segments(node: &NativeNode, canonical_segmentation: bool) -> Segment {
|
||||
}
|
||||
}
|
||||
|
||||
/// The grid one `FusedSeries` publishes on: the grid its root declares it produces, or the
|
||||
/// statement's grid when the root declares none.
|
||||
///
|
||||
/// A root that produces one value per grid point says which grid that is, and that declared grid is
|
||||
/// the one its consumer reads it on: a window over an evaluated input - a subquery - produces on the
|
||||
/// window's own output grid, and a non-window root inside such a window, like the `Map` an
|
||||
/// expression of a subquery compiles to, produces on the subquery's grid rather than on the
|
||||
/// statement's. A statement root declares the statement's grid, so the two agree there. A root with
|
||||
/// no declared grid - raw samples - is read where it was scanned and keeps the caller's grid.
|
||||
fn fused_publish_grid(root: &NativeNode, statement_grid: &Arc<EvalGrid>) -> Arc<EvalGrid> {
|
||||
root.output
|
||||
.grid()
|
||||
.map_or_else(|| Arc::clone(statement_grid), Arc::clone)
|
||||
}
|
||||
|
||||
/// Lowers a tree whose root is a canonical merge into a producer and a canonical router.
|
||||
///
|
||||
/// The tree says the statement answers one series per canonical label set; what it cannot say is
|
||||
@@ -2120,11 +2171,12 @@ fn build_evaluated_segment(
|
||||
}
|
||||
let frame = FusedSeries::series_frame_schema(&tree).ok()?;
|
||||
let frame = DFSchema::try_from(frame.as_ref().clone()).ok()?;
|
||||
let publish_grid = fused_publish_grid(tree.root(), grid);
|
||||
let plan = FusedSeries::try_new_with_format(
|
||||
Arc::from("segment"),
|
||||
tree,
|
||||
FusedSeriesBinding {
|
||||
grid: Arc::clone(grid),
|
||||
grid: publish_grid,
|
||||
time_index: time_index.to_string(),
|
||||
tags: output_tags.clone(),
|
||||
metric_name: None,
|
||||
@@ -2168,6 +2220,11 @@ fn build_inner_segment(
|
||||
.collect::<Option<Vec<_>>>()?;
|
||||
let tree = NativeTree::try_new(rewrite_inner_segment(node, &sources)).ok()?;
|
||||
let output_tags = native_output_tags(tree.root(), &source_tags)?;
|
||||
// A segment publishes on the grid its own root declares, which is the grid its parent reads it
|
||||
// on: the segments of a subquery's inner expression are evaluated on the subquery's grid, not on
|
||||
// the statement's, whatever their root computes. The statement's grid is only the fallback for a
|
||||
// root that declares none.
|
||||
let segment_grid = fused_publish_grid(tree.root(), grid);
|
||||
let (inputs, metric, time_index, tags) = if let [index] = sources.as_slice() {
|
||||
let (metric, time_index, tags) = per_source.get(*index)?;
|
||||
(scans, metric.clone(), time_index.clone(), tags.clone())
|
||||
@@ -2209,7 +2266,7 @@ fn build_inner_segment(
|
||||
Arc::from("segment"),
|
||||
tree,
|
||||
FusedSeriesBinding {
|
||||
grid: Arc::clone(grid),
|
||||
grid: segment_grid,
|
||||
time_index: time_index.clone(),
|
||||
tags: tags.clone(),
|
||||
metric_name: metric.clone(),
|
||||
@@ -2251,7 +2308,15 @@ fn segment_tree(
|
||||
root_kind: FieldKind,
|
||||
remote_raw_boundary: bool,
|
||||
) -> Result<Option<LogicalPlan>> {
|
||||
if !matches!(input_locality(tree.root()), InputLocality::SinglePartition) {
|
||||
// Only a tree the analyzer gathers is worth splitting: its per-series prefixes then run where
|
||||
// their input is read. A window over an evaluated input can need that split even where
|
||||
// [`input_locality`] reports a local tree - it drops the metric name, so [`reads_across_series`]
|
||||
// reads it as cross-series - and the per-series prefix inside it is what keeps the partitions it
|
||||
// scanned. Proceeding only admits the attempt: every cut is admitted by [`cut_as_segment`]
|
||||
// individually, which is where the analyzer's placement is asked.
|
||||
if !matches!(input_locality(tree.root()), InputLocality::SinglePartition)
|
||||
&& !evaluated_input_window(tree.root())
|
||||
{
|
||||
return Ok(None);
|
||||
}
|
||||
// A segment boundary has to carry something, and the only thing it can carry is a per-series
|
||||
@@ -2259,6 +2324,10 @@ fn segment_tree(
|
||||
// such subtree inside it has nothing to cut, and cutting anything else - an aggregate's own
|
||||
// state, a constant - is either impossible or a plain recomputation of the same work in one
|
||||
// more node. Cutting on locality alone would accept one here.
|
||||
//
|
||||
// The window this tree may be rooted at is not one of those consumers, but it is no boundary
|
||||
// either: `cut_as_segment` never cuts one, and what keeps its input's partitions is the
|
||||
// per-series prefix below it, which the cuts below are admitted for.
|
||||
// A grouping by `__name__` keeps that label while the fused node's declared columns cannot
|
||||
// name it. With multiple sources, groups differing only by metric name would therefore be
|
||||
// reported under one label set. A single-source tree cannot merge distinct metric-name groups
|
||||
@@ -2273,10 +2342,13 @@ fn segment_tree(
|
||||
let mut first_iteration = true;
|
||||
let mut changed = false;
|
||||
loop {
|
||||
// A tree the analyzer keeps partition-local needs no segment; a window over an evaluated
|
||||
// input is still worth attempting one for, so its per-series prefix keeps its partitions.
|
||||
if !matches!(
|
||||
input_locality(current_tree.root()),
|
||||
InputLocality::SinglePartition
|
||||
) {
|
||||
) && !evaluated_input_window(current_tree.root())
|
||||
{
|
||||
break;
|
||||
}
|
||||
let segment = cut_segments(current_tree.root(), canonical_segmentation);
|
||||
@@ -2339,6 +2411,12 @@ fn segment_tree(
|
||||
if built.is_none() && contains_global_consumer(cut) {
|
||||
return Ok(None);
|
||||
}
|
||||
// A cut the first iteration cannot build from the scans is built from the frames this
|
||||
// loop has already published: the canonical pass inserts a merge under a label rewrite,
|
||||
// and that boundary - rebuilt here as a producer, a frame and a router - is what answers
|
||||
// one series per label set across partitions. Which cuts are taken is
|
||||
// [`cut_as_segment`]'s decision, and it declines the evaluated-input window the analyzer
|
||||
// will not place region-locally, so this fallback only ever sees the cuts it admitted.
|
||||
let built = built.or_else(|| {
|
||||
build_evaluated_segment(
|
||||
prom_expr,
|
||||
@@ -2472,11 +2550,12 @@ fn segment_tree(
|
||||
)
|
||||
};
|
||||
let (segment_metric, segment_time_index) = segment_binding(¤t_source_ids);
|
||||
let publish_grid = fused_publish_grid(parent.root(), grid);
|
||||
let Ok(plan) = FusedSeries::try_new_with_format(
|
||||
Arc::from(prom_expr.to_string()),
|
||||
parent,
|
||||
FusedSeriesBinding {
|
||||
grid: Arc::clone(grid),
|
||||
grid: publish_grid,
|
||||
time_index: segment_time_index,
|
||||
tags: tags.to_vec(),
|
||||
metric_name: segment_metric,
|
||||
@@ -2604,10 +2683,20 @@ fn plan_split_aggregate(
|
||||
)))
|
||||
.context(DataFusionPlanningSnafu);
|
||||
}
|
||||
// The partial tree publishes aggregate state on the cut aggregate's grid, which is not
|
||||
// necessarily the statement's when the suffix re-grids the values - a subquery above the cut
|
||||
// produces on its own grid. The final tree's binding is the one the caller derived for it.
|
||||
let partial_binding = FusedSeriesBinding {
|
||||
grid: match &partial.root().output {
|
||||
NativeType::AggregateState(state) => Arc::clone(&state.grid),
|
||||
_ => Arc::clone(&binding.grid),
|
||||
},
|
||||
..binding.clone()
|
||||
};
|
||||
let partial = FusedSeries::try_new_state_export(
|
||||
Arc::clone(&promql),
|
||||
partial,
|
||||
binding.clone(),
|
||||
partial_binding,
|
||||
vec![rawinput],
|
||||
)
|
||||
.context(DataFusionPlanningSnafu)?;
|
||||
@@ -3236,6 +3325,12 @@ fn contains_aggregate_expression(expr: &PromExpr) -> bool {
|
||||
.args
|
||||
.iter()
|
||||
.any(|arg| contains_aggregate_expression(arg)),
|
||||
// A subquery evaluates its expression on its own grid, but a fold inside it still folds a
|
||||
// group no single region owns, so an operand holding one needs the split route just as a
|
||||
// direct aggregate does. [`single_selector_operand`] counts a subquery's selector as the
|
||||
// operand's one selector, so this is what keeps a subquery around an aggregate out of the
|
||||
// join.
|
||||
PromExpr::Subquery(subquery) => contains_aggregate_expression(&subquery.expr),
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
@@ -3326,7 +3421,11 @@ fn collect_selectors<'a>(
|
||||
None => true,
|
||||
};
|
||||
}
|
||||
PromExpr::Subquery(_) => return false,
|
||||
// A subquery evaluates its expression on its own grid; its selectors are prepared like any
|
||||
// other, each with the scan bounds `raw_scan_bounds` derives from its own range and step.
|
||||
PromExpr::Subquery(subquery) => {
|
||||
return collect_selectors(stmt, &subquery.expr, out);
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
true
|
||||
@@ -4139,6 +4238,12 @@ mod tests {
|
||||
// the shapes that produced a `tag_0` planning error before the aggregate gate existed.
|
||||
"sum(A) / B",
|
||||
"A / sum(B)",
|
||||
// The same fold one subquery down. The subquery's selector is the operand's one
|
||||
// selector, so only the aggregate gate keeps this operand out of the join - without it
|
||||
// the admitted operand fails at execution, where no planner can build the fused node
|
||||
// over the gathered subquery.
|
||||
"avg_over_time((sum by (job, host) (A))[5m:1m]) + on(job, host) B",
|
||||
"A + on(job, host) avg_over_time((sum by (job, host) (B))[5m:1m])",
|
||||
] {
|
||||
assert!(!hybrid_admission(query), "{query}");
|
||||
}
|
||||
@@ -4154,6 +4259,30 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// An aggregate inside a subquery is still an aggregate, so a hybrid operand that holds one is
|
||||
/// not the per-series evaluation the join needs.
|
||||
///
|
||||
/// This is the admission decision behind a statement that used to fail while executing: both
|
||||
/// shapes below were admitted, and the operand's subquery then had to be gathered for a join
|
||||
/// that no installed planner can build. Refusing them keeps the plan the existing route builds.
|
||||
#[test]
|
||||
fn subquery_aggregate_operand_is_refused() {
|
||||
for query in [
|
||||
"avg_over_time((sum by (job, host) (A))[5m:1m]) + on(job, host) B",
|
||||
"A + on(job, host) avg_over_time((sum by (job, host) (B))[5m:1m])",
|
||||
] {
|
||||
assert!(!hybrid_admission(query), "{query}");
|
||||
}
|
||||
// The subquery on its own is admitted: what the gate refuses is the fold inside it, not the
|
||||
// subquery that carries it.
|
||||
for query in [
|
||||
"avg_over_time((A)[5m:1m]) + on(job, host) B",
|
||||
"A + on(job, host) avg_over_time((B)[5m:1m])",
|
||||
] {
|
||||
assert!(hybrid_admission(query), "{query}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn millisecond_scan_bounds_preserve_open_closed_and_maximum_edges() {
|
||||
let inclusive = scan_time_filter(0, 10, true, "ts").to_string();
|
||||
|
||||
@@ -3488,25 +3488,22 @@ async fn native_fused_depth_limit_falls_back_to_legacy_plan() {
|
||||
);
|
||||
}
|
||||
|
||||
/// What the compiler has no node for still keeps the existing plan.
|
||||
/// A window over an evaluated input enters the native route, and only when it is asked for.
|
||||
///
|
||||
/// This test used to assert the opposite: a subquery evaluates its own body on a grid of its own,
|
||||
/// which the route did not offer, so the statement kept the existing plan. The route compiles a
|
||||
/// subquery now, and the window's per-series prefix - the `rate` over the raw scan - is the segment
|
||||
/// it is cut as, so the statement is planned natively. What the answer is worth is checked row for
|
||||
/// row against the existing plan in `native_promql_subqueries_match_legacy` (tests-integration),
|
||||
/// which runs both routes over the same fixture.
|
||||
#[tokio::test]
|
||||
async fn native_fused_unsupported_shape_falls_back() {
|
||||
for query in [
|
||||
// A subquery evaluates its own body on a grid of its own, which this route does not offer.
|
||||
"sum_over_time(rate(some_metric[5m])[1h:])",
|
||||
] {
|
||||
async fn native_fused_subquery_window_enters_the_native_route() {
|
||||
for query in ["sum_over_time(rate(some_metric[5m])[1h:])"] {
|
||||
let plan = build_native_fused_plan(query, true).await;
|
||||
let display = plan.display_indent().to_string();
|
||||
assert!(
|
||||
!display.contains("FusedSeries"),
|
||||
"`{query}` must not enter the native route: {display}"
|
||||
);
|
||||
// Falling back is the existing plan, not an approximation of it.
|
||||
let without_flag = build_native_fused_plan(query, false).await;
|
||||
assert_eq!(
|
||||
display,
|
||||
without_flag.display_indent().to_string(),
|
||||
"`{query}` must keep the existing plan"
|
||||
display.contains("FusedSeries"),
|
||||
"`{query}` must enter the native route: {display}"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -5501,17 +5498,17 @@ async fn native_fused_preparation_decline_preserves_legacy_plan() {
|
||||
);
|
||||
}
|
||||
|
||||
/// A subquery is planned natively: it is a window over an evaluated input, which the split and
|
||||
/// the segment machinery both understand, so it no longer keeps the existing plan.
|
||||
#[tokio::test]
|
||||
async fn native_fused_subquery_falls_back_to_legacy_plan() {
|
||||
async fn native_fused_subquery_plans_natively() {
|
||||
for query in [
|
||||
"rate(some_metric[5m:1m])",
|
||||
"rate(rate(some_metric[5m:1m])[10m:1m])",
|
||||
] {
|
||||
let plan = build_native_fused_plan(query, true).await;
|
||||
let display = plan.display_indent().to_string();
|
||||
assert!(!display.contains("FusedSeries"), "{query}: {display}");
|
||||
let legacy = build_native_fused_plan(query, false).await;
|
||||
assert_eq!(display, legacy.display_indent().to_string(), "{query}");
|
||||
assert!(display.contains("FusedSeries"), "{query}: {display}");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user