feat(promql): compile a constant, a negation and a scalar-vector operation

Three shapes reach the tree now: a constant binds as a `Scalar` on the statement's
grid, a negation binds as the multiplication by `-1` it is, and a binary operation
between one vector and one constant binds as a `Binary` node whose output is the
vector side's own binding. The arithmetic and comparison operators all map, and
`return_bool` is carried as its own field so `x > 0` (which filters) and
`x > bool 0` (which reports zero or one) are one node with one field changed.

The operands keep the order they were written in, because most operators are not
commutative: `1 / x` is not `x / 1`, and which side the constant is on is the only
thing that tells them apart. A constant is bound by the same `constant_value` fold a
call's parameters already use, so `x * (1 + 2)` carries `3` rather than an addition
the tree would have to know how to run.

Three shapes are refused rather than approximated. An operation between two vectors
needs each `Input` to name the selector it reads, which the one binding does not
carry, so `a + b` and `a + a` would compile to the same tree; a set operator
combines two vectors' series and has no node at all; and a statement whose value is
a scalar has nothing to scan, so a constant is only bound next to a vector that
gives it something to broadcast over. Each refusal names what it found.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit e672a9c526)
This commit is contained in:
discord9
2026-10-21 18:52:24 +08:00
parent b4fcfe5b84
commit 77ea4d1f4e
2 changed files with 720 additions and 24 deletions
+223 -15
View File
@@ -23,15 +23,20 @@
//! <range function>(<selector>[<range>]) e.g. rate(x[5m])
//! <range function>(<constant>, <selector>[<range>]) e.g. quantile_over_time(0.9, x[5m])
//! <math function>(<expression>[, <constant>]*) e.g. abs(rate(x[5m])), clamp_min(x, 0)
//! <expression> <operator> <constant> e.g. x * 100, x > bool 0
//! <constant> <operator> <expression> e.g. 1 / x, 0 < x
//! -<expression> e.g. -x
//! ```
//!
//! which become `Input` (raw samples) → `Instant` (the last sample within the statement's lookback),
//! `Input` (raw samples) → `Window` (the range function over the statement's grid) and
//! `<expression>` → `Map` (a pointwise math function over what its argument evaluates to). A
//! selector may carry `offset` in either selection shape and, in the window shape, an `@` anchor.
//! Everything else is a [`DataFusionError::NotImplemented`] that names what was found and quotes the
//! shapes above, because rejecting a query the compiler does not understand is the only honest
//! alternative to compiling it into something else.
//! `Input` (raw samples) → `Window` (the range function over the statement's grid),
//! `<expression>` → `Map` (a pointwise math function over what its argument evaluates to) and, over
//! any of those, `Binary` (an arithmetic or comparison operation with a constant on one side) and
//! `Scalar` (that constant). A selector may carry
//! `offset` in either selection shape and, in the window shape, an `@` anchor. Everything else is a
//! [`DataFusionError::NotImplemented`] that names what was found and quotes the shapes above,
//! because rejecting a query the compiler does not understand is the only honest alternative to
//! compiling it into something else.
//!
//! Two decisions keep the recursion from growing a branch per query. A constant argument — a
//! quantile, a rounding step, a clamp bound — is folded out of the statement by [`constant_value`],
@@ -42,6 +47,20 @@
//! alone, so the number this compiler binds and the number the kernel computes cannot disagree
//! about `%`, `^` or `atan2`.
//!
//! Two more are about what a node means rather than about which node to build. A binary operation
//! keeps its operands in the order they were written, because most operators are not commutative:
//! `1 / x` is not `x / 1`, and which side the constant is on is the only thing that tells them
//! apart. `return_bool` is carried as its own field, so `x > 0` (which filters) and `x > bool 0`
//! (which reports zero or one) are one node with one field changed. A negation then needs no node of
//! its own: `-x` binds as the multiplication by `-1` that it is, which the multiplication kernel
//! already computes.
//!
//! What a tree cannot yet express is deliberate too. The one input binding carries the ordering it
//! guarantees and what it produces, but not the selector it reads, so a tree with two selectors
//! cannot tell its inputs apart and an operation between two vectors is refused. A statement whose
//! value is a scalar has nothing to read, so it is refused as well: a constant is bound next to a
//! vector that gives it something to broadcast over.
//!
//! The statement's grid, the selector's lookback, window, offset and anchor are the only numbers
//! derived here; the legality of the tree is not checked here. [`NativeTree::try_new`] is the
//! single authority on that, so a node added later cannot be legal in one place and illegal in
@@ -58,14 +77,16 @@ use datatypes::series::samples::SeriesSchema;
use datatypes::series::time::{EvalGrid, GridSpec};
use promql::map::{ArithmeticOp, MathOp};
use promql::native::plan::{
BoundInput, BoundInstant, BoundMap, BoundWindow, NativeExpr, NativeNode, NativeTree, NativeType,
BoundInput, BoundInstant, BoundMap, BoundScalar, BoundWindow, NativeExpr, NativeNode,
NativeTree, NativeType,
};
use promql::range_eval::{RangeFunction, RangeWindow};
use promql::vector_match::{BinaryMatching, BinaryOp, Cardinality, FillValues, MatchingLabels};
use promql_parser::parser::token::TokenType;
use promql_parser::parser::value::ValueType;
use promql_parser::parser::{
AtModifier, Call, EvalStmt, Expr as PromExpr, MatrixSelector, NumberLiteral, Offset,
VectorSelector, token,
AtModifier, BinaryExpr, Call, EvalStmt, Expr as PromExpr, MatrixSelector, NumberLiteral,
Offset, VectorMatchCardinality, VectorMatchFillValues, VectorSelector, token,
};
use crate::promql::label_values::signed_millis_since_epoch;
@@ -73,9 +94,11 @@ use crate::promql::label_values::signed_millis_since_epoch;
/// The shapes this compiler accepts, quoted in every rejection.
const SUPPORTED: &str = "a selector, a range function over a range selector — \
`<range function>(<selector>[<range>])`, with its constants in front of \
the selector as in `quantile_over_time(0.9, x[5m])` — or a math function \
over any of them, as in `abs(rate(x[5m]))`: for example `x`, \
`rate(x[5m])`, `quantile_over_time(0.9, x[5m])` or `abs(rate(x[5m]))`";
the selector as in `quantile_over_time(0.9, x[5m])` — a math function over \
any of them, a binary operation between one of them and a constant, or a \
negation of one of them, as in `x`, `rate(x[5m])`, \
`quantile_over_time(0.9, x[5m])`, `abs(rate(x[5m]))`, `2 / (x + 1)` or \
`-x`";
/// Compiles `stmt` into a tree that reads `input` and evaluates the statement's expression over it.
///
@@ -114,6 +137,18 @@ pub fn compile(
series may not interleave",
));
}
// A tree reads a scan and reports evaluated samples, so an expression whose value is not a
// vector has nothing to compile into: `1 + 2` is a scalar, and the scalar node is bound as a
// constant next to a vector that gives it something to broadcast over rather than on its own.
// The check is on the statement's type rather than on the compiled node, so it does not depend on
// which kinds of expression happen to compile to a scalar today.
if stmt.expr.value_type() != ValueType::Vector {
return Err(not_implemented(&format!(
"the expression `{}`, which is a {} where a tree reports evaluated samples",
stmt.expr,
stmt.expr.value_type()
)));
}
let context = Context {
stmt,
input,
@@ -145,16 +180,190 @@ struct Context<'a> {
/// its argument, so a query nests as deeply as it was written. A kind this compiler has no node for
/// is refused with what was found, which is why the rejection is useful without a coverage list.
fn compile_expr(context: &Context, expr: &PromExpr) -> Result<NativeNode> {
// A constant is a value rather than a computation, so it binds as the number it folds to, by the
// same fold a call's parameters use: `x * (1 + 2)` carries `3` rather than an addition the tree
// would have to know how to run. The fold is checked first so that no shape below has to say
// what it does with a constant operand.
if let Some(value) = constant_value(expr) {
return scalar_node(context, value);
}
match expr {
// Parentheses group, so they change how a statement was typed rather than what it asks for:
// the expression inside them is the one that compiles.
PromExpr::Paren(paren) => compile_expr(context, &paren.expr),
PromExpr::Call(call) => compile_call(context, call),
PromExpr::VectorSelector(selector) => instant_selection(context, selector),
PromExpr::Unary(unary) => negate(context, &unary.expr),
PromExpr::Binary(binary) => binary_operation(context, binary),
found => Err(not_implemented(&format!("the expression `{found}`"))),
}
}
/// Binds `value` as a constant scalar on the statement's grid.
///
/// The grid is the statement's rather than the constant's own: a constant has no timestamps, so the
/// only positions it can be broadcast over are the ones the statement is evaluated at, and
/// [`NativeTree::validate`] compares a binary operation's two grids by spec rather than by
/// allocation.
fn scalar_node(context: &Context, value: f64) -> Result<NativeNode> {
Ok(NativeNode {
expr: NativeExpr::Scalar(BoundScalar { value }),
output: NativeType::Scalar(statement_grid(context.stmt)?),
})
}
/// Compiles a negation as the multiplication it is: `-x` is `-1 * x`.
///
/// The tree has no negate node and needs none: broadcasting a constant over the expression's value
/// is the same arithmetic, and the multiplication kernel is the one that already computes it. A
/// dedicated negate kernel would be an optimization of this node's work rather than a different
/// meaning, so binding the multiplication does not bind anything the query did not ask for.
fn negate(context: &Context, operand: &PromExpr) -> Result<NativeNode> {
// The constant goes on the left, where the minus was: `-x` subtracts nothing, but a reader of
// the tree should see the same order the query was written in.
Ok(broadcast(
scalar_node(context, -1.0)?,
compile_expr(context, operand)?,
BinaryOp::Mul,
false,
))
}
/// Compiles a binary operation whose operands are not both vectors.
///
/// The tree holds one input binding, and that binding carries the ordering it guarantees and what it
/// produces but not the selector it reads, so two vectors cannot be told apart: `a + b` and `a + a`
/// would compile to the same tree. That is why a vector–vector operation is refused until an input
/// names its selector, and why an operation over one vector and one scalar is the shape the tree can
/// bind: a scalar carries no labels, so broadcasting it is unambiguous.
fn binary_operation(context: &Context, binary: &BinaryExpr) -> Result<NativeNode> {
// A set operator combines two vectors' series rather than their values, and the tree has no node
// for it at all. Its name is reported before the operands are looked at, so a set operation
// between two vectors is refused as the set operation it is rather than as a binary operation
// the compiler has no node for.
if binary.op.is_set_operator() {
return Err(not_implemented(&format!(
"the set operator `{}` in `{binary}`, which has no native node: it combines two \
vectors' series, and every binary node here is an arithmetic or comparison operation \
over values",
binary.op
)));
}
let Some(op) = binary_op(&binary.op) else {
return Err(not_implemented(&format!(
"the binary operator `{}` in `{binary}`",
binary.op
)));
};
// Two vectors are refused before the modifier is looked at, so a vector–vector query with
// matching says what is actually missing. The refusal names the missing identity rather than the
// query, because that is what whoever adds it has to add.
if binary.lhs.value_type() == ValueType::Vector && binary.rhs.value_type() == ValueType::Vector
{
return Err(not_implemented(&format!(
"the binary operation `{binary}` between two vectors, which needs each input to name \
the selector it reads: an input binding carries its ordering and what it produces, but \
not which selector it was bound to"
)));
}
// What is left has a constant on at least one side, and a modifier is refused rather than
// dropped: a scalar carries no labels, so there is nothing to match or name, and Prometheus
// rejects these words on a scalar operand for the same reason. A fill value is refused because
// the tree's matching has no field for one, so binding the operation would compute it without
// the fill the query asked for.
if let Some(modifier) = &binary.modifier
&& (modifier.matching.is_some()
|| modifier.card != VectorMatchCardinality::OneToOne
|| modifier.fill_values != VectorMatchFillValues::default())
{
return Err(not_implemented(&format!(
"the vector matching in `{binary}`, which a constant operand has nothing to match: \
`on`, `ignoring`, `group_left` and `group_right` name labels of a vector, and a fill \
value fills in a match group that could miss"
)));
}
// Both operands are compiled before the node is built, so `2 / (x + 1)` nests. The constant side
// is bound by [`compile_expr`]'s fold, which is what makes it a scalar node rather than an
// expression the tree would have to run.
Ok(broadcast(
compile_expr(context, &binary.lhs)?,
compile_expr(context, &binary.rhs)?,
op,
binary.return_bool(),
))
}
/// Wraps two compiled operands in the binary node that applies `op` to them.
///
/// The output is the vector side's own binding, because a scalar carries no fields: a broadcast keeps
/// the vector's fields and the grid it is evaluated on, which is the statement's grid on both sides.
/// Two scalars are possible as a nested operand — a pair that [`constant_value`] refuses to fold, such
/// as a comparison between two constants — and the node reports the left one's grid, which is the
/// statement's too. The operands keep the order they were written in, because a non-commutative
/// operator means what that order says.
fn broadcast(left: NativeNode, right: NativeNode, op: BinaryOp, return_bool: bool) -> NativeNode {
// `compile_expr` binds a constant to a scalar node and a selection or a function to a vector, so
// this is where the side the operation reports is decided.
let output = if matches!(left.output, NativeType::Scalar(_)) {
right.output.clone()
} else {
left.output.clone()
};
NativeNode {
expr: NativeExpr::Binary {
left: Box::new(left),
right: Box::new(right),
matching: scalar_matching(op, return_bool),
},
output,
}
}
/// The matching a vector-and-constant operation records.
///
/// Nothing is matched: a scalar carries no labels, so [`NativeTree::validate`] broadcasts it and
/// reads neither `labels` nor `cardinality`. They carry the default a vector–vector operation of the
/// same operator would have, which is also the only matching PromQL allows here — it rejects
/// `on`, `ignoring`, `group_left` and `group_right` on a scalar–vector operation. `return_bool` is
/// the one modifier that changes what this operation computes: `x > 0` filters the vector and
/// `x > bool 0` reports zero or one for each of its values.
fn scalar_matching(op: BinaryOp, return_bool: bool) -> BinaryMatching {
BinaryMatching {
op,
return_bool,
labels: MatchingLabels::Ignoring(Vec::new()),
cardinality: Cardinality::OneToOne,
// A fill value fills in a match group that could miss, which a broadcast over a constant never
// does: the constant exists at every grid point, so there is no missing side to fill.
fill: FillValues::default(),
}
}
/// The binary operator a PromQL token denotes, or `None` when there is no node for it.
///
/// A comparison is bound like an arithmetic operator, because the node carries `return_bool`
/// separately: `x > 0` and `x > bool 0` are one operator with one field changed. [`arithmetic_op`]
/// is not reused because it decides what a *fold* may do at compile time and a comparison folds
/// nothing, while this map decides what a *node* computes once the query runs.
fn binary_op(op: &TokenType) -> Option<BinaryOp> {
match op.id() {
token::T_ADD => Some(BinaryOp::Add),
token::T_SUB => Some(BinaryOp::Sub),
token::T_MUL => Some(BinaryOp::Mul),
token::T_DIV => Some(BinaryOp::Div),
token::T_MOD => Some(BinaryOp::Mod),
token::T_POW => Some(BinaryOp::Pow),
token::T_ATAN2 => Some(BinaryOp::Atan2),
token::T_EQLC => Some(BinaryOp::Eq),
token::T_NEQ => Some(BinaryOp::Ne),
token::T_GTR => Some(BinaryOp::Gt),
token::T_LSS => Some(BinaryOp::Lt),
token::T_GTE => Some(BinaryOp::Ge),
token::T_LTE => Some(BinaryOp::Le),
_ => None,
}
}
/// Compiles a call: a range function opens a window over the selector it is given, and a math
/// function maps the value of its first argument.
///
@@ -168,9 +377,8 @@ fn compile_call(context: &Context, call: &Call) -> Result<NativeNode> {
return window_call(context, func.name, signature, &args.args);
}
// The name is looked up before the call's arguments are read, so a name this compiler has no
// kernel for is refused as such whichever arguments it was given: `pi()` and `time()` have no
// vector to map, and the rejection names the name rather than describing a vector they never
// wrote.
// kernel for is refused as such whichever argument it was given: `absent(x)` is refused as a
// name with no kernel, not as a selector this compiler could not compile.
let Some((op, vector)) = math_op(func.name, &args.args)? else {
return Err(not_implemented(&format!(
"a call to `{}`, which has no native kernel",
+497 -9
View File
@@ -23,7 +23,10 @@ use promql::native::plan::{
BoundInstant, BoundMap, BoundWindow, NativeExpr, NativeNode, NativeTree, NativeType,
};
use promql::range_eval::RangeFunction;
use promql_parser::parser::{EvalStmt, Expr as PromExpr, NumberLiteral};
use promql::vector_match::{BinaryMatching, BinaryOp};
use promql_parser::parser::{
BinModifier, EvalStmt, Expr as PromExpr, NumberLiteral, VectorMatchFillValues,
};
use super::compile;
@@ -70,6 +73,19 @@ fn with_selector_range(range: Duration) -> EvalStmt {
stmt
}
/// `query`'s statement with its binary operation's modifier edited by `edit`.
///
/// The parser rewrites and rejects some modifiers, so a modifier the compiler has to refuse as
/// written can only reach it on a tree the parser accepted: this is the way in.
fn with_binary_modifier(query: &str, edit: impl FnOnce(&mut BinModifier)) -> EvalStmt {
let mut stmt = eval_stmt(query);
let PromExpr::Binary(binary) = &mut stmt.expr else {
unreachable!("`{query}` parses to a binary operation");
};
edit(binary.modifier.get_or_insert_with(BinModifier::default));
stmt
}
/// `query`'s statement with its call's arguments edited by `edit`.
///
/// The parser checks arity, so a call with the wrong number of arguments cannot be parsed: the only
@@ -140,6 +156,42 @@ fn evaluated(node: &NativeNode) -> &Arc<SeriesSchema> {
schema
}
/// The two operands and the matching of a `Binary` node.
fn binary_parts(node: &NativeNode) -> (&NativeNode, &NativeNode, &BinaryMatching) {
let NativeExpr::Binary {
left,
right,
matching,
} = &node.expr
else {
panic!("the node is not a binary operation: {:?}", node.expr);
};
(left, right, matching)
}
/// The constant a `Scalar` node binds, if it is one.
fn constant(node: &NativeNode) -> Option<f64> {
match &node.expr {
NativeExpr::Scalar(scalar) => Some(scalar.value),
_ => None,
}
}
/// The grid spec a `Scalar` node reports, as `(start_ms, step_ms, len)`.
fn scalar_grid(node: &NativeNode) -> (i64, i64, u64) {
let NativeType::Scalar(grid) = &node.output else {
panic!("the node is not a scalar: {:?}", node.output);
};
let spec = grid.spec();
(spec.start_ms, spec.step_ms, spec.len)
}
/// The grid spec a node's evaluated output carries, as `(start_ms, step_ms, len)`.
fn grid_of(node: &NativeNode) -> (i64, i64, u64) {
let spec = evaluated(node).eval_grid.as_ref().unwrap().spec();
(spec.start_ms, spec.step_ms, spec.len)
}
/// `rate(x[5m])` becomes a window over the raw input, on the statement's grid.
#[test]
fn a_rate_over_a_selector_becomes_a_window_over_raw_input() {
@@ -706,13 +758,14 @@ fn the_numeric_edges_of_a_statement_are_bound_as_given() {
Expected::Ordering(SeriesOrdering::Contiguous),
),
// A shape outside the subset is refused for its shape, whatever its timing: the statement
// range of an unsupported query is not read, so it cannot turn a refusal into a plan error.
// range of an unsupported shape is not read, so it cannot turn a refusal into a plan
// error. A subquery is the reachable case of a shape with no node at all.
(
"an unsupported shape with an inverted range",
EvalStmt {
end: UNIX_EPOCH,
start: UNIX_EPOCH + Duration::from_secs(100),
..eval_stmt("x + 1")
..eval_stmt("x[5m:]")
},
SeriesOrdering::KeyThenTime,
Expected::Rejected(Rejection::NotImplemented),
@@ -810,8 +863,6 @@ fn an_interleaved_scan_is_refused_whatever_the_shape() {
#[test]
fn an_unsupported_shape_is_rejected_and_names_what_it_found() {
let cases = [
("x + 1", "the expression `x + 1`"),
("sum(x)", "the expression `sum(x)`"),
("x[5m:]", "the expression `x[5m:]`"),
// Nested shapes are refused by the argument check, not skipped over. A subquery is the
// reachable case of an argument that is not a range selector: the parser type-checks
@@ -829,10 +880,6 @@ fn an_unsupported_shape_is_rejected_and_names_what_it_found() {
"absent(x)",
"a call to `absent`, which has no native kernel",
),
// The two functions PromQL calls with no arguments at all: neither has a kernel, and the
// rejection names the name rather than describing a vector they were never given.
("pi()", "a call to `pi`, which has no native kernel"),
("time()", "a call to `time`, which has no native kernel"),
];
for (query, named) in cases {
@@ -876,6 +923,45 @@ fn the_display_of_a_compiled_tree_is_the_tree() {
);
}
/// The display of a tree the batch added nodes to, checked as a whole: the shapes are read by
/// someone reading an `EXPLAIN`, so a wrong nesting, operator, constant or grouping has to show up on
/// a line rather than only in an assertion over the tree.
#[test]
fn the_display_of_a_tree_the_batch_added_to_is_the_query() {
let cases: [(&str, &[&str]); 2] = [
(
"-x",
&[
"NativeTree",
" root Binary * ignoring [] one-to-one",
" left Scalar -1",
" right Instant lookback 5000ms offset 0 on grid 0/5000x21",
" input Input raw samples",
],
),
(
"0 < x",
&[
"NativeTree",
" root Binary < ignoring [] one-to-one",
" left Scalar 0",
" right Instant lookback 5000ms offset 0 on grid 0/5000x21",
" input Input raw samples",
],
),
];
for (query, expected) in cases {
let tree = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.unwrap_or_else(|error| panic!("`{query}` was refused: {error}"));
assert_eq!(
tree.to_string(),
expected.join("\n"),
"`{query}` is displayed as a different query"
);
}
}
/// Parentheses group; they do not change what a statement asks for. A compiler that refused them
/// would reject a query on how it was typed rather than on what it means, and a statement written
/// with them is otherwise exactly the shape the compiler accepts.
@@ -1159,3 +1245,405 @@ fn a_math_function_over_a_non_vector_argument_is_refused_and_names_it() {
);
}
}
/// A constant on the right of a vector binds as the number it is, next to the vector the rest of the
/// query compiles to: `x * 100` is `x`'s instant selection multiplied by `100`.
#[test]
fn a_constant_binds_as_a_scalar_next_to_the_vector() {
let input = raw_schema();
let tree = compile(
&eval_stmt("x * 100"),
Arc::clone(&input),
SeriesOrdering::KeyThenTime,
)
.unwrap();
let (left, right, matching) = binary_parts(tree.root());
assert_eq!(matching.op, BinaryOp::Mul);
assert!(
!matching.return_bool,
"`x * 100` is not a comparison and has no bool"
);
let NativeExpr::Instant { .. } = &left.expr else {
panic!(
"the left operand is not the selector's instant: {:?}",
left.expr
);
};
assert_eq!(
constant(right),
Some(100.0),
"the right operand is not the constant the query wrote"
);
assert_eq!(
scalar_grid(right),
(0, 5_000, 21),
"the constant is not on the statement's grid"
);
// A scalar carries no fields, so the operation reports the vector side's layout and grid.
let output = evaluated(tree.root());
assert!(
Arc::ptr_eq(&output.fields, &input.fields),
"the broadcast did not keep the vector's fields"
);
assert_eq!(
grid_of(tree.root()),
(0, 5_000, 21),
"the operation does not land on the statement's grid"
);
}
/// The operands keep the order they were written in, because most operators are not commutative:
/// `1 / x` is not `x / 1`. The constant side is what tells them apart, so a compiler that sorted the
/// operands would compile both queries to the same, wrong tree.
#[test]
fn a_non_commutative_operation_keeps_the_operands_where_they_were_written() {
let cases = [
// (query, operator, the constant, whether the constant is on the left)
("1 / x", BinaryOp::Div, 1.0, true),
("x / 1", BinaryOp::Div, 1.0, false),
("2 - x", BinaryOp::Sub, 2.0, true),
("x - 2", BinaryOp::Sub, 2.0, false),
("x % 2", BinaryOp::Mod, 2.0, false),
("2 ^ x", BinaryOp::Pow, 2.0, true),
];
for (query, op, value, constant_on_the_left) in cases {
let tree = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.unwrap_or_else(|error| panic!("`{query}` was refused: {error}"));
let (left, right, matching) = binary_parts(tree.root());
assert_eq!(matching.op, op, "`{query}` bound the wrong operator");
let (scalar_side, vector_side) = if constant_on_the_left {
(left, right)
} else {
(right, left)
};
assert_eq!(
constant(scalar_side),
Some(value),
"`{query}` put the constant on the wrong side"
);
assert!(
!matches!(vector_side.expr, NativeExpr::Scalar(_)),
"`{query}` compiled the vector operand to a constant: {:?}",
vector_side.expr
);
}
}
/// A binary operation compiles by compiling both operands, so it nests as deeply as the query was
/// written: `2 / (x + 1)` is a division whose right operand is the addition, not a shape the
/// compiler knows by name.
#[test]
fn a_binary_operation_nests_over_the_operands_it_was_written_with() {
let tree = compile(
&eval_stmt("2 / (x + 1)"),
raw_schema(),
SeriesOrdering::KeyThenTime,
)
.unwrap_or_else(|error| panic!("`2 / (x + 1)` was refused: {error}"));
let (left, right, matching) = binary_parts(tree.root());
assert_eq!(matching.op, BinaryOp::Div);
assert_eq!(constant(left), Some(2.0), "the divisor was not bound");
let (inner_left, inner_right, inner_matching) = binary_parts(right);
assert_eq!(inner_matching.op, BinaryOp::Add);
let NativeExpr::Instant { .. } = &inner_left.expr else {
panic!(
"the sum's left operand is not the selector: {:?}",
inner_left.expr
);
};
assert_eq!(
constant(inner_right),
Some(1.0),
"the summand was not bound as a constant"
);
// Every node is on the statement's grid, which is what lets the operation align them.
assert_eq!(grid_of(tree.root()), (0, 5_000, 21));
assert_eq!(grid_of(right), (0, 5_000, 21));
}
/// `return_bool` is a field of the operation rather than a different tree: `x > 0` filters and
/// `x > bool 0` reports zero or one, and the two differ in nothing else. Both directions of a
/// comparison are asserted, because which side the constant is on decides what the filter keeps and
/// a compiler that always put the constant on one side would compile `0 < x` as `x > 0`.
#[test]
fn a_comparison_carries_whether_it_filters_or_reports_bool() {
let cases = [
("x > 0", BinaryOp::Gt, false),
("x > bool 0", BinaryOp::Gt, true),
("0 < x", BinaryOp::Lt, false),
("0 < bool x", BinaryOp::Lt, true),
("x == 0", BinaryOp::Eq, false),
("x != 0", BinaryOp::Ne, false),
("x >= 0", BinaryOp::Ge, false),
("x <= 0", BinaryOp::Le, false),
];
for (query, op, return_bool) in cases {
let tree = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.unwrap_or_else(|error| panic!("`{query}` was refused: {error}"));
let (_, _, matching) = binary_parts(tree.root());
assert_eq!(matching.op, op, "`{query}` bound the wrong operator");
assert_eq!(
matching.return_bool, return_bool,
"`{query}` bound the wrong comparison mode"
);
}
// The two modes are one tree otherwise: the query text is what differs, and a tree that recorded
// the mode somewhere else would fail the assertions above rather than this one.
let filters = compile(
&eval_stmt("x > 0"),
raw_schema(),
SeriesOrdering::KeyThenTime,
)
.unwrap();
let reports = compile(
&eval_stmt("x > bool 0"),
raw_schema(),
SeriesOrdering::KeyThenTime,
)
.unwrap();
assert_eq!(grid_of(filters.root()), grid_of(reports.root()));
}
/// A negation binds as the multiplication it is: `-x` is `-1 * x`. The tree has no negate node, and
/// the multiplication kernel computes exactly the value the query asks for.
#[test]
fn a_negation_binds_as_a_multiplication_by_minus_one() {
for query in ["-x", "-(x)"] {
let tree = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.unwrap_or_else(|error| panic!("`{query}` was refused: {error}"));
let (left, right, matching) = binary_parts(tree.root());
assert_eq!(
matching.op,
BinaryOp::Mul,
"`{query}` is not bound as a multiplication"
);
assert_eq!(
constant(left),
Some(-1.0),
"`{query}` did not bind `-1` on the left"
);
let NativeExpr::Instant { .. } = &right.expr else {
panic!("`{query}`'s operand is not the selector: {:?}", right.expr);
};
}
}
/// The negation applies to whatever its operand compiles to, so a window can be negated: `-rate(x[5m])`
/// is a multiplication over the window rather than a new shape.
#[test]
fn a_negation_applies_to_the_node_its_operand_compiles_to() {
let tree = compile(
&eval_stmt("-rate(x[5m])"),
raw_schema(),
SeriesOrdering::KeyThenTime,
)
.unwrap_or_else(|error| panic!("`-rate(x[5m])` was refused: {error}"));
let (left, right, matching) = binary_parts(tree.root());
assert_eq!(matching.op, BinaryOp::Mul);
assert_eq!(constant(left), Some(-1.0));
let NativeExpr::Window { op, .. } = &right.expr else {
panic!("the negation's operand is not a window: {:?}", right.expr);
};
assert_eq!(op.function, RangeFunction::Rate);
assert_eq!(op.window.range_ms, 300_000);
}
/// A constant written as an expression binds the number it folds to, by the fold a call's parameter
/// already uses. The value is asserted exactly, so a fold that produced a neighbouring number would
/// fail here.
#[test]
fn a_constant_operand_binds_the_value_it_folds_to() {
let cases = [
("x * (1 + 2)", 3.0),
("x + 2 ^ 10", 1024.0),
// `^` binds tighter than `*`, so this is `x * 8` rather than `(x * 2) ^ 3`.
("x * 2 ^ 3", 8.0),
("x * -0.5", -0.5),
("x * (1 - 0.1)", 0.9),
];
for (query, value) in cases {
let tree = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.unwrap_or_else(|error| panic!("`{query}` was refused: {error}"));
let (left, right, _) = binary_parts(tree.root());
let scalar_side = if constant(left).is_some() {
left
} else {
right
};
assert_eq!(
constant(scalar_side),
Some(value),
"`{query}` did not bind the number it folds to"
);
}
}
/// An operation between two vectors is refused because the tree cannot say which selector each
/// `Input` reads: `a + b` and `a + a` would compile to the same tree. The refusal names the missing
/// identity rather than the query it found, so whoever adds it knows what to add.
#[test]
fn an_operation_between_two_vectors_is_refused_for_the_missing_selector_identity() {
for query in [
"a + b",
"a > b",
"rate(a[5m]) * rate(b[5m])",
"abs(a) + abs(b)",
] {
let error = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.expect_err(&format!("`{query}` compiled, and it should not have"));
assert!(
matches!(error, DataFusionError::NotImplemented(_)),
"`{query}` did not report NotImplemented: {error}"
);
let message = error.to_string();
assert!(
message.contains("between two vectors"),
"`{query}` did not say what it found: {message}"
);
assert!(
message.contains("selector"),
"`{query}` did not name what an input is missing: {message}"
);
}
}
/// A set operator has no node at all, so it is refused as the set operator it is rather than as an
/// arithmetic operation the compiler could not bind: `and`, `or` and `unless` combine two vectors'
/// series, which is not what a binary value node does.
#[test]
fn a_set_operator_is_refused_as_a_set_operator() {
for (query, name) in [
("a and b", "and"),
("a or b", "or"),
("a unless b", "unless"),
] {
let error = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.expect_err(&format!("`{query}` compiled, and it should not have"));
assert!(
matches!(error, DataFusionError::NotImplemented(_)),
"`{query}` did not report NotImplemented: {error}"
);
let message = error.to_string();
assert!(
message.contains(&format!("set operator `{name}`")),
"`{query}` did not name the operator it found: {message}"
);
}
}
/// A statement with no vector has nothing to read, so it is refused: a tree reads a scan and reports
/// evaluated samples, and a constant is bound next to a vector rather than on its own. The refusal
/// names the expression and its type, because the type is why it cannot be bound.
#[test]
fn a_statement_with_no_vector_is_refused() {
// `pi()` and `time()` are the two functions PromQL calls with no arguments. Both are scalars, so
// they reach this refusal before any function-name lookup: what a tree cannot do with them is
// report them, not fold them. The rejection still names the expression it found, so a reader
// learns which function it was.
for (query, found) in [
("1 + 2", "which is a scalar"),
("1", "which is a scalar"),
("-3", "which is a scalar"),
("pi()", "which is a scalar"),
("time()", "which is a scalar"),
] {
let error = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.expect_err(&format!("`{query}` compiled, and it should not have"));
assert!(
matches!(error, DataFusionError::NotImplemented(_)),
"`{query}` did not report NotImplemented: {error}"
);
let message = error.to_string();
assert!(
message.contains(&format!("`{query}`")),
"`{query}` did not name the expression it found: {message}"
);
assert!(
message.contains(found),
"`{query}` did not name the type it found: {message}"
);
}
}
/// A vector-matching modifier has nothing to match when one operand is a constant, and the tree's
/// matching has no field for a fill value. Both are refused rather than dropped: dropping one would
/// compile a query into an operation other than the one it wrote. `bool` is not a matching modifier
/// and is bound, which is why it is not in this list.
#[test]
fn a_vector_matching_modifier_over_a_constant_is_refused() {
let cases = [
(eval_stmt("x + ignoring() 1"), "x + ignoring() 1"),
(eval_stmt("x + on() group_left 1"), "x + on() group_left 1"),
(
eval_stmt("x + on() group_right 1"),
"x + on() group_right 1",
),
// A fill value is a matching extension the tree has no field for, so a fill is refused
// rather than bound without it. The parser accepts a fill only between two vectors, so the
// value is set on a tree the parser accepted.
(
with_binary_modifier("x + 1", |modifier| {
modifier.fill_values = VectorMatchFillValues::new(0.0, 0.0);
}),
"x + 1 with a fill of 0 on both sides",
),
];
for (stmt, query) in cases {
let error = compile(&stmt, raw_schema(), SeriesOrdering::KeyThenTime)
.expect_err(&format!("`{query}` compiled, and it should not have"));
assert!(
matches!(error, DataFusionError::NotImplemented(_)),
"`{query}` did not report NotImplemented: {error}"
);
let message = error.to_string();
assert!(
message.contains("vector matching"),
"`{query}` did not say what it found: {message}"
);
assert!(
message.contains("constant operand"),
"`{query}` blamed the wrong operand: {message}"
);
}
}
/// The shared acceptance queries of this batch, compiled whole: each one combines the shapes the
/// batch added, so a compiler that bound any of them wrongly would fail here rather than in a
/// narrower test.
#[test]
fn the_batch_acceptance_queries_compile() {
let cases = [
// Recursion plus a constant on the left.
"2 / (x + 1)",
// Filter against bool comparison, in both directions.
"x > 0",
"0 < x",
"x > bool 0",
// A negation.
"-x",
];
for query in cases {
let tree = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.unwrap_or_else(|error| panic!("`{query}` was refused: {error}"));
// Every tree the compiler accepts reports a grid, whether it ends in a comparison or a
// scalar broadcast.
assert_eq!(
grid_of(tree.root()),
(0, 5_000, 21),
"`{query}` does not land on the statement's grid"
);
}
}