From ee54ded60fce10cb9295383735aedb63a72a1cfc Mon Sep 17 00:00:00 2001 From: discord9 Date: Wed, 21 Oct 2026 17:24:07 +0800 Subject: [PATCH] docs(promql): state the real reason an instant selection needs no ordering check (#9785) * docs(promql): state the real reason an instant selection needs no ordering check `select_instants` reads one series' complete axis, so the executor must collect a series to its last chunk before evaluating it: a single buffer closed by `is_last` and keyed by nothing else. That needs each series' chunks contiguous, which `Contiguous` and `KeyThenTime` both guarantee, and `Interleaved` never reaches it because `unpack_raw_series` rejects that ordering at the raw-frame boundary. The comment said the opposite: that interleaved input was enough because `select_instants` holds no cross-series state. The conclusion stood, but the stated reason named a property the executor does not have. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * docs(promql): stop deriving the instant arm's ordering check from another path The comment justified skipping an ordering check here by pointing at `unpack_raw_series`, which rejects `Interleaved` at the raw-frame boundary. This validator never runs through that call, so the rejection was never a premise it could use, and the conclusion did not follow. The requirement it was reaching for does exist, one layer down: `NativeInstantManipulateExec` collects a series to its `is_last` chunk into one buffer keyed by nothing else, so it needs each series' chunks contiguous. This validator does not check that. `Instant` has no implementation in the native tree yet and `try_to_native_execution_plan` has no callers, so the requirement is unenforced rather than met in advance; it is settled when `Instant` lands. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * feat(promql): refuse an interleaved scan at the compiler entry, and in the validator The `Instant` validation arm replaced an ordering check with a comment saying no check was needed. The argument was that `unpack_raw_series` rejects `Interleaved`, which holds only for the legacy `InstantManipulate` path and says nothing about this tree. Meanwhile the refusal in `compile` sat after the bare-selector early return, so `compile(eval_stmt("x"), schema, Interleaved)` returned `Ok` against `compile`'s own rustdoc, and the same hole was open to every nested shape. The requirement is real for every shape: a native evaluation reads one series to its last chunk before the next. The refusal now runs before `compile` dispatches on the shape, so the bare selector, the range function and its parameterised form all inherit it, and a shape added later inherits it too. `NativeNode::validate` keeps a check of its own, for a tree built some other way: `Instant` and `Window` now share `ensure_not_interleaved` instead of one branch having the check and the other a comment about it. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * docs(promql): link the validator by the node that has it `NativeTree` has no `validate`; the method lives on `NativeNode`. Both doc comments linked the tree, so rustdoc reports an unresolved link and the published API docs lose the reference. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test(promql): assert the control records the scan's ordering The contiguous control in the interleaved-refusal test only checked that the bare selector compiled, while its comment said the ordering is recorded. Assert it on the tree's input instead. Drop the plain range case from the loop, which the ordering table already compiles under `Interleaved`. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --- src/promql/src/native/plan.rs | 38 ++++++++++------ src/promql/src/native/plan/tests.rs | 48 ++++++++++++++++---- src/query/src/promql/native_compile.rs | 39 +++++++++------- src/query/src/promql/native_compile/tests.rs | 48 ++++++++++++++++++-- 4 files changed, 129 insertions(+), 44 deletions(-) diff --git a/src/promql/src/native/plan.rs b/src/promql/src/native/plan.rs index 6cca42134f3..f5715b21f72 100644 --- a/src/promql/src/native/plan.rs +++ b/src/promql/src/native/plan.rs @@ -122,8 +122,8 @@ pub struct AggregateStateSchema { /// A scan or exchange boundary bound as a node input. /// /// The binding records what the input was created for: the output a consumer type-checks against, -/// and the ordering it guarantees. A window reads the ordering, rejecting an interleaved input; no -/// other consumer reads it yet. +/// and the ordering it guarantees. An instant selection and a window both read the ordering, +/// rejecting an interleaved input; no other consumer reads it yet. #[derive(Clone, Debug)] pub struct BoundInput { /// What the input produces: raw samples from a scan, or state from an exchange boundary. @@ -329,14 +329,13 @@ impl NativeNode { ))); } } + ensure_not_interleaved(input)?; ensure_samples(&self.output, "an instant selection")?; // A selection is one source row per grid point: `select_instants` picks row // indices and every field is taken at them, so it carries one value per input // field. ensure_same_fields(&input.output, &self.output, "an instant selection")?; ensure_grid(&self.output, &op.output_grid, "an instant selection")?; - // `select_instants` reads one series at a time and holds no cross-series state, so - // an interleaved input is enough for it: no ordering check here. } NativeExpr::Window { input, op } => { input.validate()?; @@ -354,17 +353,7 @@ impl NativeNode { "a window reads an empty range: its range must be positive", )); } - // A range evaluator consumes one series to its last chunk before it starts the - // next, so interleaved chunks of different series cannot be fed to it. Only an - // input node carries an ordering, so only there is there one to check. - if let NativeExpr::Input(binding) = &input.expr - && binding.ordering == SeriesOrdering::Interleaved - { - return Err(invalid( - "a window reads one series to its last chunk before the next, so \ - interleaved chunks cannot be fed to it", - )); - } + ensure_not_interleaved(input)?; // What is checked here and just below is that the declared input grid equals the // input's grid, and that the output grid matches the declared one. The grid is // not checked to cover the window's read range @@ -624,6 +613,25 @@ fn ensure_grid(ty: &NativeType, expected: &Arc, what: &str) -> Result< } } +/// Checks that `input`, when it is a [`NativeExpr::Input`], does not declare interleaved series. +/// +/// A native evaluation reads one series to its last chunk before the next: a range evaluator folds +/// a series completely before it starts another, and an instant selection's consumer collects one +/// series' chunks into one buffer. Neither can be handed chunks of different series that +/// interleave, which is the one rule both consumers state. Only an input node carries an ordering, +/// so only there is there one to check. +fn ensure_not_interleaved(input: &NativeNode) -> Result<()> { + if let NativeExpr::Input(binding) = &input.expr + && binding.ordering == SeriesOrdering::Interleaved + { + return Err(invalid( + "a native evaluation reads one series to its last chunk before the next, so \ + interleaved chunks cannot be fed to it", + )); + } + Ok(()) +} + fn invalid(reason: &str) -> DataFusionError { // A rejected tree is a compiler defect, not a query error: the compiler only builds a node // after deciding that this combination is supported. `Internal` keeps that meaning on the wire diff --git a/src/promql/src/native/plan/tests.rs b/src/promql/src/native/plan/tests.rs index 1c42c3b32af..ba01eb53cdd 100644 --- a/src/promql/src/native/plan/tests.rs +++ b/src/promql/src/native/plan/tests.rs @@ -96,6 +96,18 @@ fn input_with_fields(count: usize) -> NativeNode { } } +/// A scan of raw samples whose chunks of different series may interleave. +fn interleaved_input() -> NativeNode { + let output = raw(); + NativeNode { + expr: NativeExpr::Input(BoundInput { + output: output.clone(), + ordering: SeriesOrdering::Interleaved, + }), + output, + } +} + /// A constant scalar, which is a scalar type evaluated on the statement's grid. fn scalar_on(value: f64, grid: Arc) -> NativeNode { NativeNode { @@ -457,15 +469,35 @@ fn a_window_checks_its_declared_input_grid() { /// chunks of different series may not interleave. #[test] fn a_window_cannot_read_interleaved_input() { - let output = raw(); - let interleaved = NativeNode { - expr: NativeExpr::Input(BoundInput { - output: output.clone(), - ordering: SeriesOrdering::Interleaved, - }), - output, + let error = window_over(interleaved_input(), None) + .validate() + .unwrap_err(); + assert!( + error + .to_string() + .contains("interleaved chunks cannot be fed to it"), + "{error}" + ); +} + +/// An instant selection's consumer collects one series' chunks into one buffer, so it reads a +/// series to its last chunk before the next: it refuses an interleaved input for the reason a +/// window does. +#[test] +fn an_instant_selection_cannot_read_interleaved_input() { + let out = grid(0, 60_000, 4); + let node = NativeNode { + expr: NativeExpr::Instant { + input: Box::new(interleaved_input()), + op: BoundInstant { + lookback_ms: 300_000, + offset_ms: 0, + output_grid: Arc::clone(&out), + }, + }, + output: evaluated(Arc::new([]), out), }; - let error = window_over(interleaved, None).validate().unwrap_err(); + let error = node.validate().unwrap_err(); assert!( error .to_string() diff --git a/src/query/src/promql/native_compile.rs b/src/query/src/promql/native_compile.rs index 88e5023aa42..1be936d7980 100644 --- a/src/query/src/promql/native_compile.rs +++ b/src/query/src/promql/native_compile.rs @@ -34,7 +34,9 @@ //! The statement's grid, the selector's lookback, window, offset and anchor are the only numbers //! derived here; the legality of the tree is not checked here. [`NativeTree::try_new`] is the //! single authority on that, so a node added later cannot be legal in one place and illegal in -//! another. +//! another. What is refused before a tree exists is the scan's shape: no shape below can consume +//! chunks of different series that interleave, so an `Interleaved` ordering is refused once, at the +//! entry, and [`NativeNode::validate`] refuses the same ordering for a tree built some other way. use std::sync::Arc; use std::time::Duration; @@ -72,12 +74,12 @@ const SUPPORTED: &str = "a bare selector, a range function over a selector — \ /// one. /// /// `ordering` is what the caller's scan guarantees, and the tree records it rather than assuming -/// one. Only `Contiguous` and `KeyThenTime` are accepted: a native range window consumes one series -/// to its last chunk before the next, so chunks of different series may not interleave, and an -/// `Interleaved` declaration is refused here — which is where the refusal belongs, since a tree that -/// declares it under a window cannot be built by this compiler at all. [`NativeTree::validate`]'s -/// window branch checks the same thing, so if both land this refusal comes first and the tree check -/// is a backstop for a tree built some other way. +/// one. Only `Contiguous` and `KeyThenTime` are accepted: every shape this compiler accepts reads +/// one series to its last chunk before the next, so an `Interleaved` declaration is refused at the +/// entry, before any shape is built — the bare selector included, which returns before a window +/// exists — and a shape added later inherits the refusal instead of restating it. +/// [`NativeNode::validate`] refuses the same ordering in its `Instant` and `Window` branches, as a +/// backstop for a tree built some other way. /// /// The tree carries no metric name and no matchers, so `rate(x{job="a"}[5m])` and `rate(y[5m])` /// compile to identical trees, and an `EXPLAIN` built on this display cannot tell them apart. That @@ -88,6 +90,17 @@ pub fn compile( input: Arc, ordering: SeriesOrdering, ) -> Result { + // Every shape this compiler accepts reads one series to its last chunk before the next, so raw + // samples whose chunks of different series interleave cannot be fed to any of them. The refusal + // is here rather than in a shape's branch so that a shape added later inherits it, and it comes + // before the early return below for a bare selector. + if ordering == SeriesOrdering::Interleaved { + return Err(not_implemented( + "raw samples whose series interleave, which this compiler cannot consume: a native \ + evaluation reads one series to its last chunk before the next, so chunks of different \ + series may not interleave", + )); + } // A bare selector is the other shape: an instant selection over the same raw input. if let PromExpr::VectorSelector(selector) = unparen(&stmt.expr) { return instant_selection(stmt, selector, input, ordering); @@ -126,16 +139,8 @@ pub fn compile( func.name, argument ))); }; - // A native range window reads one series to its last chunk before it reads the next, so raw - // samples whose chunks of different series interleave cannot be folded by it. The declaration - // is refused here rather than recorded onto an input the tree is built over and fails on. - if ordering == SeriesOrdering::Interleaved { - return Err(not_implemented( - "raw samples whose series interleave, which this compiler cannot consume: its range \ - window reads one series to its last chunk before the next, so chunks of different \ - series may not interleave", - )); - } + // The ordering was refused at the entry above, so every shape below is built over a scan whose + // series' chunks do not interleave. window_selection(stmt, function, vs, range, input, ordering) } diff --git a/src/query/src/promql/native_compile/tests.rs b/src/query/src/promql/native_compile/tests.rs index 5e375858111..05b4d07ead8 100644 --- a/src/query/src/promql/native_compile/tests.rs +++ b/src/query/src/promql/native_compile/tests.rs @@ -631,11 +631,11 @@ fn the_numeric_edges_of_a_statement_are_bound_as_given() { SeriesOrdering::KeyThenTime, Expected::Rejected(Rejection::Plan), ), - // A range window reads one series to its last chunk before the next, so a scan whose - // chunks of different series interleave is refused instead of recorded onto an input the - // window would fail on. + // Every shape reads one series to its last chunk before the next, so a scan whose chunks + // of different series interleave is refused instead of recorded onto an input that shape + // would fail on. ( - "an interleaved ordering, which a window cannot consume to its last chunk", + "an interleaved ordering, which no shape can consume to its last chunk", eval_stmt("rate(x[5m])"), SeriesOrdering::Interleaved, Expected::Rejected(Rejection::NotImplemented), @@ -707,6 +707,46 @@ fn the_numeric_edges_of_a_statement_are_bound_as_given() { } } +/// An interleaved scan is refused whatever shape the statement has. The refusal sits at the entry +/// of the compiler rather than in one shape's branch, because a bare selector returns before any +/// window exists and would otherwise be accepted with an ordering it cannot be evaluated on. +#[test] +fn an_interleaved_scan_is_refused_whatever_the_shape() { + // The plain range function is the ordering table's case, which compiles `rate(x[5m])` under + // `Interleaved` and expects the same refusal; these two cover the shapes that table cannot, + // because one returns before a window exists and the other takes a parameter. + for query in ["x", "quantile_over_time(0.9, x[5m])"] { + let error = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::Interleaved) + .expect_err(&format!("`{query}` compiled over an interleaved scan")); + assert!( + matches!(error, DataFusionError::NotImplemented(_)), + "`{query}` did not report NotImplemented: {error}" + ); + let message = error.to_string(); + assert!( + message.contains("reads one series to its last chunk before the next"), + "`{query}` did not say why the ordering is refused: {message}" + ); + } + // The same scan compiles a bare selector when its chunks are contiguous, and the ordering it + // was compiled under is recorded on the tree's input rather than dropped: that is what makes + // the refusals above about the ordering rather than about the shape. + let tree = compile(&eval_stmt("x"), raw_schema(), SeriesOrdering::Contiguous) + .expect("a contiguous scan compiles a bare selector"); + let (input, _) = root_instant(&tree); + let NativeExpr::Input(bound) = &input.expr else { + panic!( + "an instant selection's input is not an input: {:?}", + input.expr + ); + }; + assert_eq!( + bound.ordering, + SeriesOrdering::Contiguous, + "the instant selection dropped the scan's ordering" + ); +} + /// A shape outside the subset is refused, and the error says what was found so the query can be /// rewritten rather than guessed at. #[test]