feat(promql): run subqueries through the native prefix (#10085)

* feat(promql): run subqueries through the native prefix

A subquery evaluates its inner expression on its own grid, so the native route now plans it
instead of sending every statement containing one to the existing planner.

- `collect_selectors` recurses into a subquery's expression, so its selectors are prepared
  like any other and each gets its scan bounds from `raw_scan_bounds`.
- `calls_rate_or_increase` recurses too, so a delta-temporality table still keeps a rate or
  an increase inside a subquery on the existing planner.
- A window whose input is already evaluated - a subquery - is admitted above the split
  aggregate when its declared grid equals its direct input's grid, and the suffix is rebuilt
  with it. A raw-sample window still cannot sit above the cut.
- A `FusedSeries` whose root is such a window publishes on that window's output grid, which
  is not necessarily the statement's; the binding follows it at every construction site, and
  the split's partial state keeps the cut aggregate's own grid. An evaluated-input window
  does not need an owner router for its sources, and a segment whose root is one is cut on
  its own grid.

Verified with `-p promql`, `-p query`, `tests-integration --lib` (native_promql) and clippy.

Signed-off-by: discord9 <discord9@163.com>

* fix(promql): keep a subquery's aggregate operand and gate an evaluated cut

Two shapes reached the native route and failed while planning instead of keeping the plan they had.

A hybrid-binary operand folds a group one subquery down
(`avg_over_time((sum by (job, host) (pm))[5m:1m]) + on(job, host) pn`): `single_selector_operand`
counts the subquery's selector, so the operand looked admissible, while
`contains_aggregate_expression` stopped at `PromExpr::Subquery` and never saw the aggregate. The
admission then built a `FusedSeries` the physical planner could not convert
(`No installed planner was able to convert the custom node to an execution plan ...`). The walk
descends into a subquery again, and both statements keep the existing plan with the same rows.

A window over an evaluated input was cut as a segment regardless of locality. For any such window
whose input is not a bare `Input` - a raw window, a `label_replace` - `reads_across_series` is true
and `pointwise_operand` false, so the `FusedSeries` arm of the dist analyzer answers
`NonCommutative`: the segment stays above the `MergeScan` and nothing converts it
(`sum by (job) (max_over_time(rate(pm[2m])[5m:1m]))` and two neighbours failed that way at instant
and range evaluation). `region_local_segment` mirrors that arm's decision from the tree, and
`cut_as_segment` declines an evaluated-input window it refuses, cutting the per-series prefix inside
the window instead; where there is none, the statement keeps the plan it had. The analyzer is not
extended: every range function but `last_over_time`/`first_over_time` drops the metric name, so such
a window is genuinely cross-series and a region-local segment would publish two series under one
label set.

`build_evaluated_segment` stays, and so does its caller. The canonical pass inserts a merge under a
label rewrite, and that boundary is only ever built from the frames the loop has published, so the
cut needs the fallback; deleting it took every rewritten fold off the route, which is what
`sum(label_replace(...))` and the histogram-over-rewrite boundary caught. The frames-bound segment
itself is not the defect - the gate is: the evaluated-input window is declined before any cut is
taken, so the fallback only sees cuts the analyzer places.

Tests that asserted the older rejection are decided on evidence: `sum_over_time(rate(m[5m])[1h:])`
is admitted now and its rows match the existing plan, so the plan-level test asserts the route and the
integration case proves the rows; the aggregate-subquery case in
`native_aggregate_complete_over_four_nonempty_mito_partitions` is answered by the split (one `Partial`
producer, one frontend merge) and returns the same 15.0 on both routes, so it moved to the split cases
rather than being dropped. Two fold-over-subquery shapes in `native_promql_subqueries_match_legacy`
(`sum by (job) (max_over_time(rate(...)[5m:1m]))` and the fold above it) are answered natively for the
same reason - the fallback builds their cut - and their rows equal the existing plan's, so they assert
the route instead of the fallback; the rewrite shape still declines, because its window reads across
series and there is no per-series prefix inside it to take instead.

Signed-off-by: discord9 <discord9@163.com>

* test(promql): assert anchored subqueries do not re-fold across steps

Signed-off-by: discord9 <discord9@163.com>

* test(promql): drop the unused fallback-case helper

Signed-off-by: discord9 <discord9@163.com>

* test(promql): update subquery expectations for the native route

Signed-off-by: discord9 <discord9@163.com>

* fix(promql): publish subquery segments on their declared grid

Signed-off-by: discord9 <discord9@163.com>

* refactor(promql): drop unreachable segment arms and pin subquery routes

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
This commit is contained in:
discord9
2026-11-09 21:46:03 +08:00
committed by GitHub
parent ae9fb131e2
commit 63dfaf8915
6 changed files with 613 additions and 115 deletions
@@ -1779,7 +1779,13 @@ fn tree_inputs(tree: &NativeTree) -> Vec<&crate::native::plan::BoundInput> {
fn partitioned_window_sources(node: &NativeNode) -> DataFusionResult<Vec<usize>> {
fn visit(node: &NativeNode, sources: &mut Vec<usize>) -> DataFusionResult<()> {
match &node.expr {
NativeExpr::Window { input, .. } => {
NativeExpr::Window { input, op } => {
// A subquery window reads an evaluated input, whose fragments belong to one series
// wherever they were produced, so it needs no owner of its own. A raw window
// anywhere below it still does, so the walk continues into the input.
if op.input_grid.is_some() {
return visit(input, sources);
}
let NativeExpr::Input(bound) = &input.expr else {
return Err(DataFusionError::Plan(
"partitioned FusedSeries windows require a direct Input under Window"
+40 -6
View File
@@ -70,8 +70,10 @@ pub(crate) fn histogram_group_is_single(group: &Grouping) -> bool {
/// the merged values. `topk`, a second aggregate and vector-vector matching still need a whole-vector
/// boundary.
///
/// An instant selection or window above the cut cannot be applied to the merged values under the
/// tree's current binding: the cut output is evaluated, whereas these nodes consume raw samples.
/// An instant selection or a window over raw samples above the cut cannot be applied to the merged
/// values under the tree's current binding: the cut output is evaluated, whereas these nodes consume
/// raw samples. A subquery window instead reads an evaluated input, so it is admitted when its
/// declared grid equals its direct input's grid.
pub(crate) fn split(
node: &NativeNode,
fields: &[FieldSpec],
@@ -228,6 +230,34 @@ pub(crate) fn split(
..
} => Err(declined("a whole-vector operation above a split aggregate")),
NativeExpr::Merge { .. } => Err(declined("an aggregate split across a canonical merge")),
// A window whose input is already evaluated - a subquery - applies to the merged values the
// suffix reads, provided it evaluates on that input's own grid. The declared grid must equal
// its direct input's grid; the cut's grid need not be that grid, because a window between
// them may already have re-gridded the values. A raw-sample window cannot sit above the cut.
NativeExpr::Window { input, op } => {
let Some(mut inner) = split(input, fields, time_index)? else {
return Ok(None);
};
let input_grid = input
.output
.samples()
.and_then(|schema| schema.eval_grid.as_ref());
if op.input_grid.as_ref().is_none_or(|declared| {
input_grid.is_none_or(|input| declared.spec() != input.spec())
}) {
return Err(declined(
"a window above an aggregate with a different evaluation grid",
));
}
inner.suffix = NativeNode {
expr: NativeExpr::Window {
input: Box::new(inner.suffix),
op: op.clone(),
},
output: node.output.clone(),
};
Ok(Some(inner))
}
// These read values the merged state does not answer: an instant selection or window reads
// raw samples, `scalar`/`vector` answers one value or one series instead of the vector, and a
// parameterized map reads the whole vector at a point. `ParameterizedTopK` declines here
@@ -235,7 +265,6 @@ pub(crate) fn split(
// a bound one does, so it needs the same whole-vector boundary. Admitting the one and
// refusing the other would be an inconsistency rather than a capability.
NativeExpr::Instant { input, .. }
| NativeExpr::Window { input, .. }
| NativeExpr::Timestamp { input }
| NativeExpr::Vector { input, .. }
| NativeExpr::VectorOf { input }
@@ -541,9 +570,9 @@ impl Split {
/// Puts `merged` at the cut and returns the suffix above it.
///
/// A suffix is the spine of point-wise maps, constant-sided binaries and at most one histogram
/// operation between the cut and the root, with scalar constants and the evaluation time preserved.
/// Other whole-vector operations are refused by `split`.
/// A suffix is the spine of point-wise maps, constant-sided binaries, evaluated-input windows and at
/// most one histogram operation between the cut and the root, with scalar constants and the
/// evaluation time preserved. Other whole-vector operations are refused by `split`.
fn enter_suffix(node: NativeNode, merged: &NativeNode) -> DfResult<NativeNode> {
let NativeNode { expr, output } = node;
let rejected = || {
@@ -557,6 +586,11 @@ fn enter_suffix(node: NativeNode, merged: &NativeNode) -> DfResult<NativeNode> {
NativeExpr::Scalar(_) | NativeExpr::EvalTime { .. } => {
return Ok(NativeNode { expr, output });
}
// A subquery window above the cut is rebuilt over the merged values, on its own grid.
NativeExpr::Window { input, op } => NativeExpr::Window {
input: Box::new(enter_suffix(*input, merged)?),
op,
},
NativeExpr::Map { input, op } => NativeExpr::Map {
input: Box::new(enter_suffix(*input, merged)?),
op,
+138 -9
View File
@@ -1279,12 +1279,13 @@ impl PromPlanner {
let Some(rawinput) = children.into_iter().next() else {
return Ok(None);
};
let publish_grid = fused_publish_grid(final_tree.root(), &grid);
let node = plan_split_aggregate(
Arc::from(prom_expr.to_string()),
partial,
final_tree,
FusedSeriesBinding {
grid,
grid: publish_grid,
time_index: time.clone(),
tags,
metric_name: metric,
@@ -1374,11 +1375,12 @@ impl PromPlanner {
} else {
(tags, metric)
};
let publish_grid = fused_publish_grid(tree.root(), &grid);
let node = FusedSeries::try_new(
Arc::from(prom_expr.to_string()),
tree,
FusedSeriesBinding {
grid,
grid: publish_grid,
time_index: time,
tags,
metric_name: metric,
@@ -1631,15 +1633,49 @@ fn exportable(node: &NativeNode) -> bool {
.is_some_and(|schema| schema.eval_grid.is_some())
}
/// Whether `node` is a window over an evaluated input - the shape a subquery over anything but a
/// bare selection compiles to.
fn evaluated_input_window(node: &NativeNode) -> bool {
matches!(&node.expr, NativeExpr::Window { op, .. } if op.input_grid.is_some())
}
/// Whether the dist analyzer places a segment over `node` region-locally.
///
/// A segment is a `FusedSeries` node the analyzer has to move below the `MergeScan` for the physical
/// planner to read the scans it names. The `FusedSeries` arm of `Categorizer::check_extension_plan`
/// (`src/query/src/dist_plan/commutativity.rs`) is what decides that, from the bound tree alone: a
/// root that [`reads_across_series`] is `NonCommutative` unless it is a per-series evaluation or the
/// `Partial` half of a split fold. A segment it refuses stays above the merge, where the physical
/// planner finds neither a region scan nor an explicit-label frame below it, so the statement fails
/// while being planned instead of keeping the plan it had. Segmentation therefore asks that question
/// before it cuts, and [`cut_as_segment`] declines the nodes this refuses.
fn region_local_segment(node: &NativeNode) -> bool {
!reads_across_series(node) || pointwise_operand(node) || independent_scan_pair(node)
}
/// Whether `node` becomes a segment of its own under a parent that must gather.
///
/// Cut local prefixes, global consumers and canonical merges separately so the prefix keeps its
/// scan width, and every global consumer sees the canonical boundary below it. A pointwise wrapper
/// of a global consumer is cut as well, so it cannot hide that consumer from recursive lowering.
fn cut_as_segment(node: &NativeNode, canonical_segmentation: bool) -> bool {
// A window over an evaluated input answers per series on its own grid, so its own shape says
// nothing about where a segment over it can go: it drops the metric name, so two of the series
// it answers can end under one label set, and the dist analyzer reads it as cross-series
// ([`region_local_segment`]). Cutting one where the analyzer will not place it leaves a
// `FusedSeries` above the `MergeScan` that the physical planner cannot build, so the segment is
// only taken where the analyzer places it region-locally; otherwise the per-series prefix
// inside the window is cut instead, and where there is none the statement keeps the plan it had.
if evaluated_input_window(node) && !region_local_segment(node) {
return false;
}
// A window over an evaluated input is answered per series on its own grid, so it is not the
// global gather a raw window's locality reports.
let evaluated_window = evaluated_input_window(node);
if !exportable(node)
|| matches!(node.expr, NativeExpr::Input(_))
|| (!canonical_segmentation
&& !evaluated_window
&& matches!(input_locality(node), InputLocality::SinglePartition))
{
return false;
@@ -1873,6 +1909,21 @@ fn cut_segments(node: &NativeNode, canonical_segmentation: bool) -> Segment {
}
}
/// The grid one `FusedSeries` publishes on: the grid its root declares it produces, or the
/// statement's grid when the root declares none.
///
/// A root that produces one value per grid point says which grid that is, and that declared grid is
/// the one its consumer reads it on: a window over an evaluated input - a subquery - produces on the
/// window's own output grid, and a non-window root inside such a window, like the `Map` an
/// expression of a subquery compiles to, produces on the subquery's grid rather than on the
/// statement's. A statement root declares the statement's grid, so the two agree there. A root with
/// no declared grid - raw samples - is read where it was scanned and keeps the caller's grid.
fn fused_publish_grid(root: &NativeNode, statement_grid: &Arc<EvalGrid>) -> Arc<EvalGrid> {
root.output
.grid()
.map_or_else(|| Arc::clone(statement_grid), Arc::clone)
}
/// Lowers a tree whose root is a canonical merge into a producer and a canonical router.
///
/// The tree says the statement answers one series per canonical label set; what it cannot say is
@@ -2120,11 +2171,12 @@ fn build_evaluated_segment(
}
let frame = FusedSeries::series_frame_schema(&tree).ok()?;
let frame = DFSchema::try_from(frame.as_ref().clone()).ok()?;
let publish_grid = fused_publish_grid(tree.root(), grid);
let plan = FusedSeries::try_new_with_format(
Arc::from("segment"),
tree,
FusedSeriesBinding {
grid: Arc::clone(grid),
grid: publish_grid,
time_index: time_index.to_string(),
tags: output_tags.clone(),
metric_name: None,
@@ -2168,6 +2220,11 @@ fn build_inner_segment(
.collect::<Option<Vec<_>>>()?;
let tree = NativeTree::try_new(rewrite_inner_segment(node, &sources)).ok()?;
let output_tags = native_output_tags(tree.root(), &source_tags)?;
// A segment publishes on the grid its own root declares, which is the grid its parent reads it
// on: the segments of a subquery's inner expression are evaluated on the subquery's grid, not on
// the statement's, whatever their root computes. The statement's grid is only the fallback for a
// root that declares none.
let segment_grid = fused_publish_grid(tree.root(), grid);
let (inputs, metric, time_index, tags) = if let [index] = sources.as_slice() {
let (metric, time_index, tags) = per_source.get(*index)?;
(scans, metric.clone(), time_index.clone(), tags.clone())
@@ -2209,7 +2266,7 @@ fn build_inner_segment(
Arc::from("segment"),
tree,
FusedSeriesBinding {
grid: Arc::clone(grid),
grid: segment_grid,
time_index: time_index.clone(),
tags: tags.clone(),
metric_name: metric.clone(),
@@ -2251,7 +2308,15 @@ fn segment_tree(
root_kind: FieldKind,
remote_raw_boundary: bool,
) -> Result<Option<LogicalPlan>> {
if !matches!(input_locality(tree.root()), InputLocality::SinglePartition) {
// Only a tree the analyzer gathers is worth splitting: its per-series prefixes then run where
// their input is read. A window over an evaluated input can need that split even where
// [`input_locality`] reports a local tree - it drops the metric name, so [`reads_across_series`]
// reads it as cross-series - and the per-series prefix inside it is what keeps the partitions it
// scanned. Proceeding only admits the attempt: every cut is admitted by [`cut_as_segment`]
// individually, which is where the analyzer's placement is asked.
if !matches!(input_locality(tree.root()), InputLocality::SinglePartition)
&& !evaluated_input_window(tree.root())
{
return Ok(None);
}
// A segment boundary has to carry something, and the only thing it can carry is a per-series
@@ -2259,6 +2324,10 @@ fn segment_tree(
// such subtree inside it has nothing to cut, and cutting anything else - an aggregate's own
// state, a constant - is either impossible or a plain recomputation of the same work in one
// more node. Cutting on locality alone would accept one here.
//
// The window this tree may be rooted at is not one of those consumers, but it is no boundary
// either: `cut_as_segment` never cuts one, and what keeps its input's partitions is the
// per-series prefix below it, which the cuts below are admitted for.
// A grouping by `__name__` keeps that label while the fused node's declared columns cannot
// name it. With multiple sources, groups differing only by metric name would therefore be
// reported under one label set. A single-source tree cannot merge distinct metric-name groups
@@ -2273,10 +2342,13 @@ fn segment_tree(
let mut first_iteration = true;
let mut changed = false;
loop {
// A tree the analyzer keeps partition-local needs no segment; a window over an evaluated
// input is still worth attempting one for, so its per-series prefix keeps its partitions.
if !matches!(
input_locality(current_tree.root()),
InputLocality::SinglePartition
) {
) && !evaluated_input_window(current_tree.root())
{
break;
}
let segment = cut_segments(current_tree.root(), canonical_segmentation);
@@ -2339,6 +2411,12 @@ fn segment_tree(
if built.is_none() && contains_global_consumer(cut) {
return Ok(None);
}
// A cut the first iteration cannot build from the scans is built from the frames this
// loop has already published: the canonical pass inserts a merge under a label rewrite,
// and that boundary - rebuilt here as a producer, a frame and a router - is what answers
// one series per label set across partitions. Which cuts are taken is
// [`cut_as_segment`]'s decision, and it declines the evaluated-input window the analyzer
// will not place region-locally, so this fallback only ever sees the cuts it admitted.
let built = built.or_else(|| {
build_evaluated_segment(
prom_expr,
@@ -2472,11 +2550,12 @@ fn segment_tree(
)
};
let (segment_metric, segment_time_index) = segment_binding(&current_source_ids);
let publish_grid = fused_publish_grid(parent.root(), grid);
let Ok(plan) = FusedSeries::try_new_with_format(
Arc::from(prom_expr.to_string()),
parent,
FusedSeriesBinding {
grid: Arc::clone(grid),
grid: publish_grid,
time_index: segment_time_index,
tags: tags.to_vec(),
metric_name: segment_metric,
@@ -2604,10 +2683,20 @@ fn plan_split_aggregate(
)))
.context(DataFusionPlanningSnafu);
}
// The partial tree publishes aggregate state on the cut aggregate's grid, which is not
// necessarily the statement's when the suffix re-grids the values - a subquery above the cut
// produces on its own grid. The final tree's binding is the one the caller derived for it.
let partial_binding = FusedSeriesBinding {
grid: match &partial.root().output {
NativeType::AggregateState(state) => Arc::clone(&state.grid),
_ => Arc::clone(&binding.grid),
},
..binding.clone()
};
let partial = FusedSeries::try_new_state_export(
Arc::clone(&promql),
partial,
binding.clone(),
partial_binding,
vec![rawinput],
)
.context(DataFusionPlanningSnafu)?;
@@ -3236,6 +3325,12 @@ fn contains_aggregate_expression(expr: &PromExpr) -> bool {
.args
.iter()
.any(|arg| contains_aggregate_expression(arg)),
// A subquery evaluates its expression on its own grid, but a fold inside it still folds a
// group no single region owns, so an operand holding one needs the split route just as a
// direct aggregate does. [`single_selector_operand`] counts a subquery's selector as the
// operand's one selector, so this is what keeps a subquery around an aggregate out of the
// join.
PromExpr::Subquery(subquery) => contains_aggregate_expression(&subquery.expr),
_ => false,
}
}
@@ -3326,7 +3421,11 @@ fn collect_selectors<'a>(
None => true,
};
}
PromExpr::Subquery(_) => return false,
// A subquery evaluates its expression on its own grid; its selectors are prepared like any
// other, each with the scan bounds `raw_scan_bounds` derives from its own range and step.
PromExpr::Subquery(subquery) => {
return collect_selectors(stmt, &subquery.expr, out);
}
_ => {}
}
true
@@ -4139,6 +4238,12 @@ mod tests {
// the shapes that produced a `tag_0` planning error before the aggregate gate existed.
"sum(A) / B",
"A / sum(B)",
// The same fold one subquery down. The subquery's selector is the operand's one
// selector, so only the aggregate gate keeps this operand out of the join - without it
// the admitted operand fails at execution, where no planner can build the fused node
// over the gathered subquery.
"avg_over_time((sum by (job, host) (A))[5m:1m]) + on(job, host) B",
"A + on(job, host) avg_over_time((sum by (job, host) (B))[5m:1m])",
] {
assert!(!hybrid_admission(query), "{query}");
}
@@ -4154,6 +4259,30 @@ mod tests {
}
}
/// An aggregate inside a subquery is still an aggregate, so a hybrid operand that holds one is
/// not the per-series evaluation the join needs.
///
/// This is the admission decision behind a statement that used to fail while executing: both
/// shapes below were admitted, and the operand's subquery then had to be gathered for a join
/// that no installed planner can build. Refusing them keeps the plan the existing route builds.
#[test]
fn subquery_aggregate_operand_is_refused() {
for query in [
"avg_over_time((sum by (job, host) (A))[5m:1m]) + on(job, host) B",
"A + on(job, host) avg_over_time((sum by (job, host) (B))[5m:1m])",
] {
assert!(!hybrid_admission(query), "{query}");
}
// The subquery on its own is admitted: what the gate refuses is the fold inside it, not the
// subquery that carries it.
for query in [
"avg_over_time((A)[5m:1m]) + on(job, host) B",
"A + on(job, host) avg_over_time((B)[5m:1m])",
] {
assert!(hybrid_admission(query), "{query}");
}
}
#[test]
fn millisecond_scan_bounds_preserve_open_closed_and_maximum_edges() {
let inclusive = scan_time_filter(0, 10, true, "ts").to_string();
+16 -19
View File
@@ -3488,25 +3488,22 @@ async fn native_fused_depth_limit_falls_back_to_legacy_plan() {
);
}
/// What the compiler has no node for still keeps the existing plan.
/// A window over an evaluated input enters the native route, and only when it is asked for.
///
/// This test used to assert the opposite: a subquery evaluates its own body on a grid of its own,
/// which the route did not offer, so the statement kept the existing plan. The route compiles a
/// subquery now, and the window's per-series prefix - the `rate` over the raw scan - is the segment
/// it is cut as, so the statement is planned natively. What the answer is worth is checked row for
/// row against the existing plan in `native_promql_subqueries_match_legacy` (tests-integration),
/// which runs both routes over the same fixture.
#[tokio::test]
async fn native_fused_unsupported_shape_falls_back() {
for query in [
// A subquery evaluates its own body on a grid of its own, which this route does not offer.
"sum_over_time(rate(some_metric[5m])[1h:])",
] {
async fn native_fused_subquery_window_enters_the_native_route() {
for query in ["sum_over_time(rate(some_metric[5m])[1h:])"] {
let plan = build_native_fused_plan(query, true).await;
let display = plan.display_indent().to_string();
assert!(
!display.contains("FusedSeries"),
"`{query}` must not enter the native route: {display}"
);
// Falling back is the existing plan, not an approximation of it.
let without_flag = build_native_fused_plan(query, false).await;
assert_eq!(
display,
without_flag.display_indent().to_string(),
"`{query}` must keep the existing plan"
display.contains("FusedSeries"),
"`{query}` must enter the native route: {display}"
);
}
}
@@ -5501,17 +5498,17 @@ async fn native_fused_preparation_decline_preserves_legacy_plan() {
);
}
/// A subquery is planned natively: it is a window over an evaluated input, which the split and
/// the segment machinery both understand, so it no longer keeps the existing plan.
#[tokio::test]
async fn native_fused_subquery_falls_back_to_legacy_plan() {
async fn native_fused_subquery_plans_natively() {
for query in [
"rate(some_metric[5m:1m])",
"rate(rate(some_metric[5m:1m])[10m:1m])",
] {
let plan = build_native_fused_plan(query, true).await;
let display = plan.display_indent().to_string();
assert!(!display.contains("FusedSeries"), "{query}: {display}");
let legacy = build_native_fused_plan(query, false).await;
assert_eq!(display, legacy.display_indent().to_string(), "{query}");
assert!(display.contains("FusedSeries"), "{query}: {display}");
}
}
@@ -47,12 +47,10 @@
//! satisfy. No other counter query reads that tag: an aggregate drops the labels it does not group
//! by, so their answers are the ones the untagged fixture gives.
//!
//! A query the native route declines is checked the other way round. [`check_fallback_case`] runs
//! shapes the compiler has no node for - `scalar(v)` and a subquery - with the native request on and
//! with it off, and requires that no native node ran in either plan and that both runs report rows
//! derived from the fixture by hand. A hint that quietly
//! took the route for one of them would show a fused producer or a frontend merge in the plans that
//! executed, which is what is observed.
//! Two shapes that used to be declined are checked on the plans that really ran. `scalar(v)` reads
//! the whole vector in one fused node and needs no merge; a subquery over a fold splits into one
//! producer and one frontend merge. A hint that quietly kept the existing route would show no native
//! node, which is what the plan observer checks, and both routes must answer the hand-derived rows.
//!
//! A binary operation whose sibling is a literal scalar is carried by the split on either side of
//! the fold: below the aggregate it is computed in the per-series prefix (`sum(8 * rate(...))`,
@@ -739,9 +737,16 @@ async fn native_hybrid_binary_05_06_08() {
);
let (legacy, legacy_capture) = run_query(&instance, &subquery, false, &observer).await;
let (native, native_capture) = run_query(&instance, &subquery, true, &observer).await;
// A subquery now runs on the native route, so the existing plan must hold no native node while
// the native run must exercise one. The rows below are what says the route answers what the
// existing plan does.
assert!(
legacy_capture.fused.is_empty() && native_capture.fused.is_empty(),
"hybrid must be disabled inside subquery: {subquery}"
legacy_capture.fused.is_empty(),
"the existing route must not plan native nodes: {subquery}"
);
assert!(
!native_capture.fused.is_empty(),
"the subquery must run on the native route: {subquery}"
);
assert_eq!(
legacy.rows.len(),
@@ -1606,15 +1611,14 @@ fn assert_finalize_reads_region_state(
}
}
/// Checks one query the native route declines: it must stay declined on both routes, and both must
/// answer what the fixture derives by hand.
/// Checks one query the native route answers by splitting the aggregate below a subquery.
///
/// The native request is on for one run and off for the other, so the hint being asked for and the
/// hint being absent are both exercised: neither plan may hold a fused producer or a frontend merge,
/// and the rows - labels, evaluation time, values and their multiplicity - are the same hand-derived
/// ones for both. The observer is asked to have seen a plan at all, because a run that handed nothing
/// over would otherwise satisfy "no native node" without observing anything.
async fn check_fallback_case(
/// The per-series prefix runs in a `Partial` producer on every partition and the window above it
/// finishes the answer in the `Final` half over the merged values, so the native run must show
/// exactly one producer and one frontend merge - the opposite of a fallback case - while the existing
/// plan shows neither. Both routes must answer what the fixture derives by hand, which is what says
/// the split's answer is the statement's answer and not an approximation of it.
async fn check_split_case(
instance: &Arc<Instance>,
observer: &NativePlanObserver,
case: &AggregateCase,
@@ -1630,13 +1634,32 @@ async fn check_fallback_case(
"`{query}`: the {route} run handed no plan to the observer, so nothing about the plan \
that ran was observed"
);
assert!(
capture.fused.is_empty() && capture.finalize.is_empty(),
"`{query}`: the {route} run must keep the existing plan, but the plans that ran held {} \
fused producers and {} frontend merges",
capture.fused.len(),
capture.finalize.len()
);
if native {
assert_eq!(
1,
capture.fused.len(),
"`{query}`: the {route} run must split the aggregate into one producer, but the \
plans that ran held {} producers and {} frontend merges",
capture.fused.len(),
capture.finalize.len()
);
assert_eq!(
1,
capture.finalize.len(),
"`{query}`: the {route} run must finish the fold in one frontend merge, but the \
plans that ran held {} producers and {} frontend merges",
capture.fused.len(),
capture.finalize.len()
);
} else {
assert!(
capture.fused.is_empty() && capture.finalize.is_empty(),
"`{query}`: the {route} run must keep the existing plan, but the plans that ran \
held {} fused producers and {} frontend merges",
capture.fused.len(),
capture.finalize.len()
);
}
assert_rows(&result.rows, &case.expected, &format!("{route} `{query}`"));
}
}
@@ -2481,20 +2504,6 @@ async fn native_aggregate_complete_over_four_nonempty_mito_partitions() {
)
.await;
// A shape the native compiler declines, checked on the plans that really ran: it may hold
// neither a fused producer nor a frontend merge, and both routes must answer what the fixture
// derives by hand. A subquery evaluates its own body on a grid of its own: it reads across
// series and cannot be answered by a producer per partition, so it keeps the existing plan. The
// subquery's window spans the two inner points `EVAL_SECS - 1` and `EVAL_SECS`, at each of
// which the same extrapolated rate answers 15 - both samples stay further from the window's
// ends than the 66s threshold - so the maximum is 15 as well.
let fallback_case = case(
format!("max_over_time((sum(rate({COUNTER_TABLE}[5m])))[2s:1s])"),
&[(&[], 15.0)],
false,
);
check_fallback_case(&instance, &observer, &fallback_case).await;
// `scalar` reads the whole vector and the native route answers it, so it is no longer one of
// the declined shapes. `scalar` reports the one value present at a point only when exactly one
// series is present at it, so the five counter series make it NaN; a route that folded one
@@ -2540,6 +2549,20 @@ async fn native_aggregate_complete_over_four_nonempty_mito_partitions() {
&legacy.rows,
"native `scalar` against the existing plan",
);
// A subquery over a fold is not declined: the compiler binds a window over an evaluated input to
// that input's own grid, so the split carries it - the per-series prefix runs in a `Partial`
// producer on every partition and the window above it finishes the answer in the `Final` half
// over the merged values. The subquery's window spans the two inner points `EVAL_SECS - 1` and
// `EVAL_SECS`, at each of which the same extrapolated rate answers 15 - both samples stay further
// from the window's ends than the 66s threshold - so the maximum is 15 as well.
let split_cases = [case(
format!("max_over_time((sum(rate({COUNTER_TABLE}[5m])))[2s:1s])"),
&[(&[], 15.0)],
false,
)];
for case in &split_cases {
check_split_case(&instance, &observer, case).await;
}
}
/// Histogram functions above the cut merge Partial state before evaluating the complete buckets.
+352 -43
View File
@@ -2028,7 +2028,11 @@ async fn native_site_rows(output: Output) -> Vec<(i64, String, String, f64)> {
rows
}
async fn native_result_rows(output: Output) -> Vec<(i64, String, f64)> {
/// The rows of a TQL answer, labelled by `label`.
///
/// `None` labels each row with every label column it carries, by name, so an answer whose label set
/// is not the `host` one can be compared with the existing route's the same way.
async fn native_rows(output: Output, label: Option<&str>) -> Vec<(i64, String, f64)> {
let batches = match output.data {
OutputData::Stream(stream) => collect_batches(stream).await.unwrap(),
OutputData::RecordBatches(batches) => batches,
@@ -2043,30 +2047,60 @@ async fn native_result_rows(output: Output) -> Vec<(i64, String, f64)> {
.as_any()
.downcast_ref::<TimestampMillisecondArray>()
.unwrap();
let hosts = cast(
batch.column_by_name("host").unwrap().as_ref(),
&DataType::Utf8,
)
.unwrap();
let hosts = hosts.as_any().downcast_ref::<StringArray>().unwrap();
let value_columns = batch
.columns()
.iter()
.filter(|column| column.data_type() == &DataType::Float64)
.enumerate()
.filter(|(_, column)| column.data_type() == &DataType::Float64)
.collect::<Vec<_>>();
assert_eq!(value_columns.len(), 1, "expected one Float64 value column");
let values = value_columns[0]
let (value_index, value_column) = value_columns[0];
let labels = match label {
Some(label) => {
let labels = cast(
batch.column_by_name(label).unwrap().as_ref(),
&DataType::Utf8,
)
.unwrap();
let labels = labels.as_any().downcast_ref::<StringArray>().unwrap();
assert_eq!(labels.null_count(), 0);
(0..batch.num_rows())
.map(|row| labels.value(row).to_string())
.collect::<Vec<_>>()
}
// Every label column, by name: the value column and the time index are the two that
// are not one, and a route names its value differently from the other.
None => (0..batch.num_rows())
.map(|row| {
batch
.schema
.column_schemas()
.iter()
.enumerate()
.filter(|(index, column)| *index != value_index && column.name != "ts")
.filter_map(|(_, column)| {
let values = batch.column_by_name(&column.name)?.as_ref();
let values = cast(values, &DataType::Utf8).ok()?;
let values = values.as_any().downcast_ref::<StringArray>()?;
(!values.is_null(row))
.then(|| format!("{}={}", column.name, values.value(row)))
})
.collect::<Vec<_>>()
.join(",")
})
.collect::<Vec<_>>(),
};
let values = value_column
.as_any()
.downcast_ref::<Float64Array>()
.unwrap();
assert_eq!(timestamps.null_count(), 0);
assert_eq!(hosts.null_count(), 0);
assert_eq!(values.null_count(), 0);
(0..batch.num_rows())
.map(|row| {
(
timestamps.value(row),
hosts.value(row).to_string(),
labels[row].clone(),
values.value(row),
)
})
@@ -2077,6 +2111,10 @@ async fn native_result_rows(output: Output) -> Vec<(i64, String, f64)> {
rows
}
async fn native_result_rows(output: Output) -> Vec<(i64, String, f64)> {
native_rows(output, Some("host")).await
}
#[tokio::test(flavor = "multi_thread")]
async fn native_promql_clamp_aggregates_execute_on_native_segments() {
common_telemetry::init_default_ut_logging();
@@ -7248,38 +7286,41 @@ async fn native_promql_repeated_selector_binary_uses_independent_scans() {
native_result_rows(native_tql_output(&instance, &different_matchers, false).await).await;
assert_eq!(different_matcher_rows, different_matcher_legacy);
// Group matching executes natively; the subquery case remains on the ordinary path, unchanged.
let grouped_expected = expected[..HOSTS].to_vec();
for (index, (sql, expected_rows)) in [
(
format!(
r#"TQL EVAL (0, 0, '1s') {table}{{host!=""}} - on(host) group_left() ({table}{{host!=""}} * 2)"#
),
grouped_expected,
),
(
format!(r#"TQL EVAL (1, 1, '1s') rate({table}{{host="h000"}}[2s:1s])"#),
vec![(1_000, "h000".to_string(), 0.5)],
),
]
.into_iter()
.enumerate()
{
observer.clear();
let rows = native_result_rows(native_tql_output(&instance, &sql, true).await).await;
if index == 0 {
let plan = observer.two_input_fused();
assert_two_source_native_plan(&observer, &plan, false);
} else {
assert!(
observer.fused().is_none(),
"a fallback shape must not plan a fused node: {sql}"
);
}
assert_eq!(rows, expected_rows, "{sql}");
let legacy_rows = native_result_rows(native_tql_output(&instance, &sql, false).await).await;
assert_eq!(rows, legacy_rows, "{sql}");
}
// Group matching executes natively: it keeps the many side's series, so the route answers it
// with one node over two same-table scans.
let grouped_sql = format!(
r#"TQL EVAL (0, 0, '1s') {table}{{host!=""}} - on(host) group_left() ({table}{{host!=""}} * 2)"#
);
observer.clear();
let grouped_rows =
native_result_rows(native_tql_output(&instance, &grouped_sql, true).await).await;
let grouped_plan = observer.two_input_fused();
assert_two_source_native_plan(&observer, &grouped_plan, false);
assert_eq!(grouped_rows, expected[..HOSTS].to_vec(), "{grouped_sql}");
let grouped_legacy =
native_result_rows(native_tql_output(&instance, &grouped_sql, false).await).await;
assert_eq!(grouped_rows, grouped_legacy, "{grouped_sql}");
// A subquery is planned natively: `rate` over an evaluated input reads the grid the subquery
// produces, and the route compiles that window instead of sending the statement to the ordinary
// planner. The rows are what says the plan it runs answers the statement it was given, so the
// answer is pinned here and compared with the existing plan's on the same fixture.
let subquery_sql = format!(r#"TQL EVAL (1, 1, '1s') rate({table}{{host="h000"}}[2s:1s])"#);
observer.clear();
let subquery_rows =
native_result_rows(native_tql_output(&instance, &subquery_sql, true).await).await;
assert!(
observer.fused().is_some(),
"a subquery is not a fallback shape: {subquery_sql}"
);
assert_eq!(
subquery_rows,
vec![(1_000, "h000".to_string(), 0.5)],
"{subquery_sql}"
);
let subquery_legacy =
native_result_rows(native_tql_output(&instance, &subquery_sql, false).await).await;
assert_eq!(subquery_rows, subquery_legacy, "{subquery_sql}");
}
/// Empty-key grouped matches retain the many side's labels and report kernel cardinality errors.
@@ -8540,6 +8581,30 @@ async fn native_promql_delta_temporality_rate_and_increase_match_legacy() {
);
}
// A subquery recurses through the expression tree for the same delta-temporality guard.
// Compare against the existing route and assert that native execution declines both functions.
for query in [
format!("max_over_time(rate({TABLE}[2m])[2m:1m])"),
format!("max_over_time(increase({TABLE}[2m])[2m:1m])"),
] {
let sql = format!("TQL EVAL (120, 120, '1m') {query}");
let legacy_rows = native_marker_rows(native_tql_output(&instance, &sql, false).await).await;
assert!(
!legacy_rows.is_empty(),
"legacy subquery `{query}` must return rows"
);
observer.clear();
let native_rows = native_marker_rows(native_tql_output(&instance, &sql, true).await).await;
assert_eq!(
native_rows, legacy_rows,
"`{query}`: same rows on both routes"
);
assert!(
observer.fused().is_none(),
"`{query}`: delta temporality inside a subquery must decline native planning"
);
}
// The control: a marker-free table (samples 10 and 30 here, unlike the table above) still fuses
// and agrees with the existing plan, so the decline is about the marker, not the table shape.
let control_sql = format!("TQL EVAL (120, 120, '1m') sum_over_time({CONTROL}[2m])");
@@ -12250,3 +12315,247 @@ async fn native_promql_root_canonical_merge_keeps_the_scan_width() {
);
}
}
/// A subquery evaluates its inner expression on its own grid, on the native route.
///
/// A window over an aggregate is the split's one admitted evaluated-input shape: the subquery's
/// declared input grid equals the suffix's, so the split carries it rather than declining. Each
/// answer is compared with the existing planner's, and the route is asserted through the observer.
///
/// A window over an evaluated input is not a per-series prefix, though: it drops the metric name, so
/// the dist analyzer reads it as cross-series and will not place a segment over one. The shapes below
/// it - a raw window's `rate`, a `label_replace` map - are cut instead, and where nothing below the
/// window can carry the tree the statement keeps the plan the existing route builds. Either way it
/// answers the rows Prometheus does, and never the "no installed planner" failure a segment the
/// analyzer refuses produces.
#[tokio::test(flavor = "multi_thread")]
async fn native_promql_subqueries_match_legacy() {
common_telemetry::init_default_ut_logging();
let config = mito2::config::MitoConfig {
experimental_series_scan_v2: true,
..Default::default()
};
let (standalone, observer) = native_test_instance(
GreptimeDbStandaloneBuilder::new("native_promql_subqueries").with_mito_config(config),
)
.await;
let instance = standalone.fe_instance().clone();
execute_all(
&instance,
r#"CREATE TABLE native_subq (host STRING, job STRING, ts TIMESTAMP(3) TIME INDEX, val DOUBLE, PRIMARY KEY (host, job));
INSERT INTO native_subq (host, job, ts, val) VALUES
('h0', 'a', 60000, 0), ('h0', 'a', 120000, 10), ('h0', 'a', 180000, 20),
('h1', 'a', 60000, 0), ('h1', 'a', 120000, 30), ('h1', 'a', 180000, 60),
('h0', 'b', 60000, 0), ('h0', 'b', 120000, 40), ('h0', 'b', 180000, 80),
('h1', 'b', 60000, 0), ('h1', 'b', 120000, 50), ('h1', 'b', 180000, 100)"#,
QueryContext::arc(),
)
.await;
for sql in [
"TQL EVAL (180, 180, '1m') avg_over_time(native_subq[2m:1m])",
"TQL EVAL (180, 180, '1m') max_over_time(rate(native_subq[2m])[2m:1m])",
"TQL EVAL (180, 180, '1m') avg_over_time((sum by (job, host) (rate(native_subq[2m])))[2m:1m])",
"TQL EVAL (180, 180, '1m') avg_over_time((sum by (job, host) (rate(native_subq[2m])))[2m:1m] @ 180)",
"TQL EVAL (180, 180, '1m') max_over_time(max_over_time(native_subq[2m:1m])[2m:1m])",
// A subquery body that is a mapped selection rather than a fold: the segment under the
// window is the `Map` that clamps, and it has to publish on the subquery's own grid - the
// two points at 120000 and 180000 at one-minute steps - rather than on the statement's,
// which holds the single point at 180000. The two grids therefore differ in both their
// start and their length, not just in identity.
"TQL EVAL (180, 180, '1m') avg_over_time(clamp(native_subq, 0, 1)[2m:1m])",
] {
observer.clear();
let mut native = native_result_rows(native_tql_output(&instance, sql, true).await).await;
assert!(
observer.fused().is_some(),
"{sql} did not execute a FusedSeriesExec: {:#?}",
observer.root_plan()
);
observer.clear();
let mut legacy = native_result_rows(native_tql_output(&instance, sql, false).await).await;
// The two routes emit the same rows but not in the same order: the helper sorts within a
// batch, and a series' label is one column of that batch's own order.
assert!(
observer.fused().is_none(),
"{sql} must keep the existing plan when the native route is off"
);
native.sort_by(|a, b| (a.0, &a.1).cmp(&(b.0, &b.1)).then(a.2.total_cmp(&b.2)));
legacy.sort_by(|a, b| (a.0, &a.1).cmp(&(b.0, &b.1)).then(a.2.total_cmp(&b.2)));
assert_eq!(native, legacy, "{sql}");
}
// The anchored shape is compared with the existing route only at its anchor. Past the anchor the
// existing route re-folds the subquery's window on every outer step, and the split must not copy
// that bug, so the later steps are held to the invariant on the native answer alone: an anchored
// subquery's range vector is fixed at 180, so each outer step has to publish 180's value for a
// series rather than fold a fresh one.
let anchored = "TQL EVAL (180, 360, '1m') \
avg_over_time((sum by (job, host) (rate(native_subq[2m])))[2m:1m] @ 180)";
observer.clear();
let anchored_rows = native_rows(native_tql_output(&instance, anchored, true).await, None).await;
assert!(
observer.fused().is_some(),
"{anchored} did not execute a FusedSeriesExec: {:#?}",
observer.root_plan()
);
// Every label column names the series, so `job` is part of the key as well as `host`.
let mut series =
std::collections::BTreeMap::<&str, std::collections::BTreeMap<i64, f64>>::new();
for (ts, label, value) in &anchored_rows {
assert!(
series
.entry(label.as_str())
.or_default()
.insert(*ts, *value)
.is_none(),
"{anchored} answered {label} twice at {ts}"
);
}
assert!(
!series.is_empty(),
"{anchored} answered no series: {:#?}",
observer.root_plan()
);
for step in [180000, 240000, 300000] {
assert!(
series.values().all(|steps| steps.contains_key(&step)),
"{anchored} did not answer {step} for every series: {anchored_rows:?}"
);
}
// An anchored subquery must not re-fold when the outer step advances.
for (label, steps) in &series {
let anchor = steps[&180000];
for (step, value) in steps {
assert_eq!(
anchor, *value,
"{anchored}: {label} holds the anchored value at 180000 but folded {value} at {step}"
);
}
}
// A fold over a subquery whose input is not a bare selection. The window is not itself cut as a
// segment - it drops the metric name, so the analyzer reads it as cross-series - and what keeps
// the scan width is the per-series prefix below it or the canonical boundary a rewrite needs.
// `fused` says which plan answers, and every case is compared with the existing route row by
// row. The one shape that still keeps the existing plan is the rewrite: its window's input is
// cross-series, so `region_local_segment` refuses the cut and there is no per-series prefix
// inside the window to take instead.
for (sql, label, fused) in [
(
"TQL EVAL (180, 180, '1m') sum by (job) (max_over_time(rate(native_subq[2m])[5m:1m]))",
Some("job"),
true,
),
// The rewrite writes a new label, so its map is per-series: the subquery's inner map is
// the segment, and it publishes on the subquery's grid rather than the statement's. It
// runs natively now; the rows below are what say it answers what the existing plan does.
(
"TQL EVAL (180, 180, '1m') sum by (x) \
(avg_over_time(label_replace(native_subq, \"x\", \"$1\", \"job\", \"(.*)\")[5m:1m]))",
Some("x"),
true,
),
(
"TQL EVAL (180, 180, '1m') max_over_time(sum by (job) \
(max_over_time(rate(native_subq[2m])[3m:1m]))[5m:1m])",
Some("job"),
true,
),
// This shape used to keep the existing plan, on the grounds that a subquery evaluates its
// own body on a grid of its own. It runs natively now: the `rate` over the raw scan is the
// per-series segment and the window above it publishes, so the fused node below is the plan
// that answers. The rows are what says that plan answers what the existing one does.
(
"TQL EVAL (180, 180, '1m') sum_over_time(rate(native_subq[5m])[1h:])",
None,
true,
),
] {
observer.clear();
let mut native = native_rows(native_tql_output(&instance, sql, true).await, label).await;
assert_eq!(
fused,
observer.fused().is_some(),
"{sql}: the native request must {} a FusedSeriesExec: {:#?}",
if fused {
"run"
} else {
"keep the existing plan for"
},
observer.root_plan()
);
assert!(
!native.is_empty(),
"{sql} answered no rows: {:#?}",
observer.root_plan()
);
let mut legacy = native_rows(native_tql_output(&instance, sql, false).await, label).await;
native.sort_by(|a, b| (a.0, &a.1).cmp(&(b.0, &b.1)).then(a.2.total_cmp(&b.2)));
legacy.sort_by(|a, b| (a.0, &a.1).cmp(&(b.0, &b.1)).then(a.2.total_cmp(&b.2)));
assert_eq!(native, legacy, "{sql}");
}
}
/// An aggregate inside a subquery is still an aggregate, so it keeps that operand out of the
/// hybrid-binary join.
///
/// Both statements below are two single-selector vector operands on the same matching labels, which
/// is what the join admits - but one operand folds a group one subquery down. With the fold counted
/// as a fold, both run on the native route through `build_evaluated_segment` and answer the same
/// rows the existing route does.
#[tokio::test(flavor = "multi_thread")]
async fn native_promql_aggregate_subquery_operands_run_natively() {
common_telemetry::init_default_ut_logging();
let config = mito2::config::MitoConfig {
experimental_series_scan_v2: true,
..Default::default()
};
let (standalone, observer) = native_test_instance(
GreptimeDbStandaloneBuilder::new("native_promql_aggregate_subquery")
.with_mito_config(config),
)
.await;
let instance = standalone.fe_instance().clone();
execute_all(
&instance,
r#"CREATE TABLE native_pm (host STRING, job STRING, ts TIMESTAMP(3) TIME INDEX, val DOUBLE, PRIMARY KEY (host, job));
INSERT INTO native_pm (host, job, ts, val) VALUES
('h0', 'a', 60000, 0), ('h0', 'a', 120000, 10), ('h0', 'a', 180000, 20),
('h1', 'a', 60000, 0), ('h1', 'a', 120000, 30), ('h1', 'a', 180000, 60),
('h0', 'b', 60000, 0), ('h0', 'b', 120000, 40), ('h0', 'b', 180000, 80),
('h1', 'b', 60000, 0), ('h1', 'b', 120000, 50), ('h1', 'b', 180000, 100);
CREATE TABLE native_pn (host STRING, job STRING, ts TIMESTAMP(3) TIME INDEX, val DOUBLE, PRIMARY KEY (host, job));
INSERT INTO native_pn (host, job, ts, val) VALUES
('h0', 'a', 60000, 1), ('h0', 'a', 120000, 2), ('h0', 'a', 180000, 3),
('h1', 'a', 60000, 4), ('h1', 'a', 120000, 5), ('h1', 'a', 180000, 6),
('h0', 'b', 60000, 7), ('h0', 'b', 120000, 8), ('h0', 'b', 180000, 9),
('h1', 'b', 60000, 10), ('h1', 'b', 120000, 11), ('h1', 'b', 180000, 12)"#,
QueryContext::arc(),
)
.await;
for sql in [
"TQL EVAL (180, 180, '1m') avg_over_time((sum by (job, host) (native_pm))[5m:1m]) + on(job, host) native_pn",
"TQL EVAL (180, 180, '1m') native_pm + on(job, host) avg_over_time((sum by (job, host) (native_pn))[5m:1m])",
] {
observer.clear();
let (_, mut native) =
native_labelled_rows(native_tql_output(&instance, sql, true).await).await;
assert!(
!native.is_empty(),
"{sql} answered no rows: {:#?}",
observer.root_plan()
);
assert!(
observer.fused().is_some(),
"{sql} must run on the native route"
);
observer.clear();
let (_, mut legacy) =
native_labelled_rows(native_tql_output(&instance, sql, false).await).await;
native.sort_by(|a, b| (a.0, &a.1).cmp(&(b.0, &b.1)).then(a.2.total_cmp(&b.2)));
legacy.sort_by(|a, b| (a.0, &a.1).cmp(&(b.0, &b.1)).then(a.2.total_cmp(&b.2)));
assert_eq!(native, legacy, "{sql}");
}
}