rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-11-14 06:46:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(promql): add evaluated series arithmetic and basic aggregates (#9682)

Browse Source 
* feat(promql): add evaluated series math kernels

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(promql): add evaluated series scalar arithmetic

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(promql): complete evaluated series math functions

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(promql): add series comparisons and scalar results

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(promql): add sparse evaluated series basic aggregates

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(promql): align time-varying scalar maps on evaluated grids

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): drop metric name in evaluated series math like Prometheus

Prometheus' simpleFloatFunc drops __name__ together with __type__ and
__unit__. Check abs, ceil, floor, sqrt and round against fixtures
generated by the Prometheus v3.15.0 engine.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): drop empty labels with the metric name in series math

Prometheus has no empty-valued labels, so a kept empty value would hide
a same-labelset collision from the caller. Round shares its calculation
with the UDF, so check it only against the Prometheus fixtures.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): align evaluated series map kernels with Prometheus

Follow Prometheus v3.15.0 simpleFloatFunc, clamp, VectorscalarBinop and
vectorElemBinop: drop __name__, __type__ and __unit__ for math,
arithmetic and bool comparisons; use ports of Go math (including the
amd64 Log and Exp assembly) for ^, %, atan2 and the transcendental
functions; empty a clamp with max < min; keep -0 in sgn and Prometheus'
operation order in deg and rad. Fields are compared independently and
filtered output is trimmed with GridSeriesBuilder.

Check the kernels against fixtures generated by the Prometheus engine
and Go math results built for amd64.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): align evaluated series aggregation with Prometheus

Follow Prometheus aggregation and generateGroupingLabels: no modifier is
by (), without drops __name__, empty label values are absent, sum and
avg use Neumaier compensation and avg switches to an incremental mean
on overflow. Fields are aggregated independently, so a field without
values at a row is null also for count and group. Output is trimmed
with GridSeriesBuilder.

Check against fixtures generated by the Prometheus engine and drop the
main UDAF oracle and the common-function dependency.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): validate whole segments in series comparisons

map_compare checked sample types one by one, so a histogram or mixed
segment without a present sample passed. Validate segments up front as
the other kernels do, check them before an empty clamp, share the
empty-label rule with aggregation, and pin exact bits in the deg, rad
and incremental avg tests.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* chore: exclude promql golden fixtures from typos

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* docs(promql): point series round at Prometheus funcRound

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): keep Go math ports only where std leaves the tolerance

Series math now follows the relative-1e-10 comparison rule: functions use
Rust's standard library unless Go's amd64 result differs beyond the
tolerance or in overflow and sign. asin, atan, tanh, asinh, acosh and
atanh use std; the Exp assembly port is reduced to its overflow edge,
and sinh/cosh build on it above 21.

Golden and Go math fixtures are compared with almost.Equal. The Go cases
are evaluated through the kernels' own per-sample functions, and each
retained port must have a case where std fails, so a port that is no
longer needed, or a swap back to std, fails a test.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* refactor(promql): validate aggregate input like series maps

aggregate_float repeated map's schema, grid-axis and float-segment
checks in a different order, so one bad series could fail with a
different error category in each. Both now use validate_float_series,
and map_math/map_negate/map_arithmetic run it before mapping instead of
their own axis check.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(promql): pin avg compensation to Prometheus bits

With fixtures compared within the tolerance, dropping or misplacing the
Kahan compensation in avg only moves the last bits. Three Prometheus
cases where it does are compared exactly: compensation on the
incremental mean, the separate division of sum and compensation, and
rescaling the compensation when avg switches to the incremental mean.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(promql): pin the zero sign of clamp and the Neumaier swap

Under the fixture tolerance, +0 and -0 compare equal and no case had an
addend larger than the running sum, so two branches lost their tests:
Go's Max picking +0 over -0, whose sign a later 1/x turns into the sign
of an infinity, and the Neumaier swap, without which
sum(1, 1e100, 1, -1e100) is 1 instead of 2.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* chore: allow the Unicode category Nd in typos

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* perf(promql): keep aggregate state in fixed-size grid blocks

Aggregation kept one B-tree entry and one Vec allocation per populated
evaluation point, plus a tree lookup per sample.

State is now stored in 64-point blocks keyed by absolute grid index /
64, holding one state per field and point. A block is allocated only
once a sample falls in it, so sparse input never expands to the full
grid, and each field's run through a block costs one lookup. Points
without samples are skipped on output, which also keeps the last block
from reaching past the grid.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* perf(promql): drop reserved and empty labels in one rebuild

drop_metric_name removed empty-valued labels one at a time, and each MetricName::remove copies the whole label array, so the cost grew with the square of the number of empty labels. Filter the labels once and rebuild the name with MetricName::try_new.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(promql): pin that aggregation keeps type and unit labels

Prometheus generateGroupingLabels drops only the metric name, unlike math and arithmetic. Nothing failed when grouping dropped __type__ or __unit__.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): return aggregate groups in first-seen order

Groups were collected in a BTreeMap and returned sorted by label. Prometheus' rangeEvalAgg appends groups in the order their first input appears, and a nested aggregation folds them in that order: for sum(sum by (g) (m)) over 1e308, -1e308, 1e308 with g = a, c, b, Prometheus gives 1e308 and label order overflows to +Inf. Look groups up by label key and keep them in a Vec.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(promql): pin which zero min and max keep

Prometheus replaces the min/max accumulator only on a strict comparison, so of +0 and -0 the first is kept. The golden tolerance treats the zeros as equal, and a non-strict comparison passed every test.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
This commit is contained in:
Lei, HUANG
2026-10-19 16:00:14 +08:00
committed by GitHub
parent 50002c6830
commit aa4abe554a
18 changed files with 8942 additions and 377 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/promql/src/aggregate.rs
+308
View File
@@ -0,0 +1,308 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Local float aggregation over evaluated series, following Prometheus' `aggregation`. State
//! is keyed by absolute grid index, never by local row number; sparse state and trimmed output
//! do not expand the full grid.
#[cfg(test)]
mod tests;
use std::collections::{BTreeMap, HashMap};
use std::sync::Arc;
use datafusion::error::{DataFusionError, Result};
use datatypes::series::MetricName;
use datatypes::series::samples::{SampleRef, Series, SeriesSchema};
use datatypes::series::time::TimeAxis;
use crate::grid_series::GridSeriesBuilder;
use crate::map::{drop_empty_labels, validate_float_series};
/// Basic float aggregations.
#[derive(Clone, Copy, Debug)]
pub enum AggregateOp {
Sum,
Count,
Avg,
Min,
Max,
Group,
}
/// Grouping by series labels. No modifier is `By(&[])`, as in Prometheus.
#[derive(Clone, Copy, Debug)]
pub enum Grouping<'a> {
By(&'a [&'a str]),
Without(&'a [&'a str]),
}
/// Aggregates each field at each absolute evaluation instant.
///
/// Each field is aggregated independently; an output row exists where any field has a value,
/// and a field without input values at that row is null. Ordinary NaNs count as values.
///
/// Inputs are folded in slice order. Prometheus folds them in storage postings order, and the
/// compensated `sum`/`avg` depend on that order in the last bits, so callers must keep it.
/// Groups are returned in the order their first input appears, as in Prometheus: a nested
/// aggregation folds them in that order.
///
/// Only float evaluated inputs on the shared schema/grid are admitted. `by` on a value field
/// needs per-point labels and an output schema adapter, and is rejected. There is no fallback.
pub fn aggregate_float(
schema: &Arc<SeriesSchema>,
input: &[Series],
op: AggregateOp,
grouping: Grouping<'_>,
) -> Result<Vec<Series>> {
validate_input(schema, input, grouping)?;
let mut group_index = HashMap::new();
let mut groups = Vec::new();
for series in input {
let metric = grouping.apply(&series.metric)?;
let index = *group_index.entry(label_key(&metric)).or_insert_with(|| {
groups.push(Group {
metric,
blocks: BTreeMap::new(),
});
groups.len() - 1
});
groups[index].accumulate(series, op)?;
}
let mut builder = GridSeriesBuilder::try_new(schema.clone())?;
let mut output = Vec::with_capacity(groups.len());
let mut row = Vec::with_capacity(schema.fields.len());
for group in groups {
for (&block, states) in &group.blocks {
// The last block can extend past the grid; only points with a sample are pushed.
for (offset, point) in states.chunks_exact(schema.fields.len()).enumerate() {
if !point.iter().any(|state| state.seen) {
continue;
}
row.clear();
row.extend(point.iter().map(|state| state.finish(op)));
builder.push(block * BLOCK_POINTS + offset, &row)?;
}
}
output.extend(builder.finish(group.metric)?);
}
Ok(output)
}
/// Schema validation is repeated because Series exposes its fields publicly.
fn validate_input(schema: &SeriesSchema, input: &[Series], grouping: Grouping<'_>) -> Result<()> {
if schema.eval_grid.is_none() {
return Err(DataFusionError::Execution(
"Aggregate requires an evaluated grid".into(),
));
}
grouping.validate(schema)?;
for series in input {
validate_float_series(schema, series)?;
}
Ok(())
}
impl Grouping<'_> {
fn validate(self, schema: &SeriesSchema) -> Result<()> {
if let Self::By(labels) = self
&& schema
.fields
.iter()
.any(|field| labels.contains(&field.name.as_ref()))
{
return Err(DataFusionError::NotImplemented(
"aggregate by value field requires a per-point label/schema adapter".into(),
));
}
Ok(())
}
/// Prometheus `generateGroupingLabels`; empty values are dropped as absent labels.
fn apply(self, input: &MetricName) -> Result<MetricName> {
let mut metric = input.clone();
match self {
Self::By(labels) => metric.keep_only(labels),
Self::Without(labels) => {
metric.remove_all(labels);
metric.remove("__name__");
}
}
drop_empty_labels(&metric)
}
}
fn label_key(metric: &MetricName) -> Vec<(Arc<str>, Arc<str>)> {
let mut labels = Vec::with_capacity(metric.tags().len() + 1);
if let Some(name) = metric.name() {
labels.push((Arc::from("__name__"), Arc::from(name)));
}
labels.extend(
metric
.tags()
.iter()
.map(|label| (label.name.clone(), label.value.clone())),
);
labels
}
/// Grid points per state block. A block is allocated only once a sample falls in it, so sparse
/// input never expands to the full grid and dense input costs one allocation per block.
const BLOCK_POINTS: usize = 64;
struct Group {
metric: MetricName,
/// Keyed by absolute grid index / `BLOCK_POINTS`; each block holds one state per field for
/// each of its points, point by point.
blocks: BTreeMap<usize, Box<[FloatState]>>,
}
impl Group {
fn accumulate(&mut self, series: &Series, op: AggregateOp) -> Result<()> {
let TimeAxis::Grid(range) = &series.time else {
return Err(DataFusionError::Execution(
"Aggregate requires an evaluated grid".into(),
));
};
let width = series.fields.len();
let mut pending = Vec::with_capacity(BLOCK_POINTS);
for (field_index, field) in series.fields.iter().enumerate() {
let mut block = range.start() / BLOCK_POINTS;
for (offset, sample) in field.iter().enumerate() {
let index = range.start() + offset;
if index / BLOCK_POINTS != block {
self.update_block(block, width, field_index, op, &mut pending);
block = index / BLOCK_POINTS;
}
if let SampleRef::Float(sample) = sample {
pending.push((index % BLOCK_POINTS, sample));
}
}
self.update_block(block, width, field_index, op, &mut pending);
}
Ok(())
}
/// Applies one field's samples within a block, so a block costs one lookup per field run.
fn update_block(
&mut self,
block: usize,
width: usize,
field_index: usize,
op: AggregateOp,
pending: &mut Vec<(usize, f64)>,
) {
if pending.is_empty() {
return;
}
let states = self.blocks.entry(block).or_insert_with(|| {
vec![FloatState::default(); BLOCK_POINTS * width].into_boxed_slice()
});
for (offset, sample) in pending.drain(..) {
states[offset * width + field_index].update(op, sample);
}
}
}
/// The float part of Prometheus' `groupedAggregation`.
#[derive(Clone, Copy, Default)]
struct FloatState {
seen: bool,
value: f64,
mean: f64,
compensation: f64,
count: f64,
incremental_mean: bool,
}
impl FloatState {
fn update(&mut self, op: AggregateOp, sample: f64) {
if !self.seen {
*self = Self {
seen: true,
value: if matches!(op, AggregateOp::Group) {
1.0
} else {
sample
},
mean: sample,
count: 1.0,
..Self::default()
};
return;
}
match op {
AggregateOp::Sum => {
(self.value, self.compensation) = kahan_inc(sample, self.value, self.compensation);
}
AggregateOp::Avg => self.update_avg(sample),
AggregateOp::Count => self.count += 1.0,
AggregateOp::Max => {
if self.value < sample || self.value.is_nan() {
self.value = sample;
}
}
AggregateOp::Min => {
if self.value > sample || self.value.is_nan() {
self.value = sample;
}
}
AggregateOp::Group => {}
}
}
/// Sums directly until the running sum would overflow, then switches to an incremental mean.
fn update_avg(&mut self, sample: f64) {
self.count += 1.0;
if !self.incremental_mean {
let (value, compensation) = kahan_inc(sample, self.value, self.compensation);
if !value.is_infinite() {
(self.value, self.compensation) = (value, compensation);
return;
}
self.incremental_mean = true;
self.mean = self.value / (self.count - 1.0);
self.compensation /= self.count - 1.0;
}
let q = (self.count - 1.0) / self.count;
(self.mean, self.compensation) =
kahan_inc(sample / self.count, q * self.mean, q * self.compensation);
}
fn finish(self, op: AggregateOp) -> Option<f64> {
if !self.seen {
return None;
}
Some(match op {
AggregateOp::Sum => self.value + self.compensation,
AggregateOp::Avg if self.incremental_mean => self.mean + self.compensation,
AggregateOp::Avg => self.value / self.count + self.compensation / self.count,
AggregateOp::Count => self.count,
AggregateOp::Min | AggregateOp::Max | AggregateOp::Group => self.value,
})
}
}
/// Prometheus `kahansum.Inc`, with the Neumaier swap.
fn kahan_inc(inc: f64, sum: f64, compensation: f64) -> (f64, f64) {
let t = sum + inc;
let compensation = if t.is_infinite() {
0.0
} else if sum.abs() >= inc.abs() {
compensation + ((sum - t) + inc)
} else {
compensation + ((inc - t) + sum)
};
(t, compensation)
}
src/promql/src/aggregate/prometheus_cases.txt
+1380
View File
File diff suppressed because it is too large Load Diff
src/promql/src/aggregate/tests.rs
+637
View File
@@ -0,0 +1,637 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use datatypes::arrow::array::{Float64Array, UInt32Array};
use datatypes::series::Label;
use datatypes::series::histogram::HistogramBatchBuilder;
use datatypes::series::samples::{FieldSpec, Samples, SeriesFields};
use datatypes::series::time::{EvalGrid, GridRange, GridSpec};
use super::*;
use crate::golden::{assert_result, cases};
const CASES: &str = include_str!("prometheus_cases.txt");
const OPS: [AggregateOp; 6] = [
AggregateOp::Sum,
AggregateOp::Count,
AggregateOp::Avg,
AggregateOp::Min,
AggregateOp::Max,
AggregateOp::Group,
];
fn schema(fields: &[&str]) -> Arc<SeriesSchema> {
Arc::new(SeriesSchema {
fields: fields
.iter()
.map(|name| FieldSpec {
name: (*name).into(),
})
.collect(),
eval_grid: Some(Arc::new(
EvalGrid::try_new(GridSpec::try_new(-1000, 250, 128).unwrap()).unwrap(),
)),
})
}
fn metric(name: &str, labels: &[(&str, &str)]) -> MetricName {
MetricName::try_new(
Some(name.into()),
labels
.iter()
.map(|(name, value)| Label {
name: (*name).into(),
value: (*value).into(),
})
.collect(),
)
.unwrap()
}
/// Each field is split at a different offset into chunks with nonzero Arrow offsets.
fn input(
schema: &SeriesSchema,
metric: MetricName,
start: usize,
fields: &[Vec<Option<f64>>],
) -> Series {
let len = fields[0].len();
let fields = fields
.iter()
.enumerate()
.map(|(index, samples)| {
let mut padded = vec![Some(999.0)];
padded.extend(samples);
padded.push(Some(999.0));
let array = Float64Array::from(padded);
let split = (index + 1).min(len);
Samples::try_from_chunks(vec![
Samples::from_float(array.slice(1, split)),
Samples::from_float(array.slice(1 + split, len - split)),
])
.unwrap()
})
.collect();
Series::try_new(
schema,
metric,
TimeAxis::Grid(GridRange::try_new(schema.eval_grid.clone().unwrap(), start, len).unwrap()),
SeriesFields::try_from_vec(fields).unwrap(),
)
.unwrap()
}
fn values(series: &Series, field: usize) -> (usize, Vec<Option<f64>>) {
let TimeAxis::Grid(range) = &series.time else {
panic!("lost grid")
};
let values = series
.fields
.get(field)
.unwrap()
.iter()
.map(|sample| match sample {
SampleRef::Float(value) => Some(value),
SampleRef::Missing => None,
_ => panic!("unexpected histogram"),
})
.collect();
(range.start(), values)
}
fn grouping<'a>(header: &[String], labels: &'a [&'a str]) -> Grouping<'a> {
match header[2].as_str() {
"none" | "by" => Grouping::By(labels),
"without" => Grouping::Without(labels),
other => panic!("unmapped grouping {other}"),
}
}
#[test]
fn aggregates_match_prometheus() {
let mut matched = BTreeMap::<String, usize>::new();
for case in cases(CASES) {
let h = &case.header;
let op = match h[1].as_str() {
"sum" => AggregateOp::Sum,
"count" => AggregateOp::Count,
"avg" => AggregateOp::Avg,
"min" => AggregateOp::Min,
"max" => AggregateOp::Max,
"group" => AggregateOp::Group,
other => panic!("unmapped aggregate {other}"),
};
let labels = h[3]
.split(',')
.filter(|label| *label != "-")
.collect::<Vec<_>>();
let output = aggregate_float(&case.schema, &case.series, op, grouping(h, &labels)).unwrap();
assert_result(&case, output);
*matched.entry(format!("{} {}", h[1], h[2])).or_default() += 1;
}
for op in ["sum", "count", "avg", "min", "max", "group"] {
for grouping in ["none", "by", "without"] {
let count = matched
.get(&format!("{op} {grouping}"))
.copied()
.unwrap_or_default();
assert!(count >= 8, "{op} {grouping}: {count}");
}
}
}
#[test]
fn compensated_sum_and_overflowing_avg_follow_prometheus() {
let schema = schema(&["value"]);
let series = [1e16, 1.0, -1e16]
.into_iter()
.enumerate()
.map(|(i, value)| {
input(
&schema,
metric("m", &[("i", &i.to_string())]),
3,
&[vec![Some(value)]],
)
})
.collect::<Vec<_>>();
// A plain left fold loses the 1.
let sum = aggregate_float(&schema, &series, AggregateOp::Sum, Grouping::By(&[])).unwrap();
assert_eq!(values(&sum[0], 0), (3, vec![Some(1.0)]));
let avg = aggregate_float(&schema, &series, AggregateOp::Avg, Grouping::By(&[])).unwrap();
assert_eq!(values(&avg[0], 0), (3, vec![Some(1.0 / 3.0)]));
let series = [f64::MAX, f64::MAX, f64::MAX / 2.0]
.into_iter()
.enumerate()
.map(|(i, value)| {
input(
&schema,
metric("m", &[("i", &i.to_string())]),
3,
&[vec![Some(value)]],
)
})
.collect::<Vec<_>>();
// The direct sum overflows, so the mean switches to the incremental form.
let avg = aggregate_float(&schema, &series, AggregateOp::Avg, Grouping::By(&[])).unwrap();
let (_, avg) = values(&avg[0], 0);
// Bits from the same calculation in Go on amd64; a plain mean is +Inf.
assert_eq!(avg, vec![Some(f64::from_bits(0x7feaaaaaaaaaaaa9))]);
let sum = aggregate_float(&schema, &series, AggregateOp::Sum, Grouping::By(&[])).unwrap();
assert_eq!(values(&sum[0], 0), (3, vec![Some(f64::INFINITY)]));
}
/// Kahan compensation is part of Prometheus' algorithm, but its effect on these means stays
/// within the fixture tolerance, so the Prometheus bits (from `aggregate/prometheus_cases.txt`)
/// are compared exactly.
#[test]
fn avg_compensation_matches_prometheus_bits() {
let schema = schema(&["value"]);
let cases: [(&[u64], u64); 3] = [
// Compensation added to the incremental mean.
(
&[
0x7fefffffffffffff,
0x7fe2b55f24962471,
0x7f7b95c7ed3a8f06,
0x7f75d53a379fb517,
0x7f71a21ceafb8ae4,
0xffe1f8b2ce599b68,
],
0x7fc62c8462e82319,
),
// The direct mean divides sum and compensation separately.
(
&[
0x406c76fff31cc498,
0xc300281b07086372,
0xc2b33ace5f269b56,
0xc2807b0afee3570f,
0x40527b1b3a5c44e6,
],
0xc2daea473b33e9f1,
),
// The compensation is rescaled when the mean switches to the incremental form.
(
&[
0xffe985777be7cf98,
0x7fb976769842f3a1,
0xffe6ef0261aba506,
0x7fbc6cc7f64a3057,
],
0xffd4dc0905e0e810,
),
];
for (inputs, expected) in cases {
let series = inputs
.iter()
.enumerate()
.map(|(i, bits)| {
input(
&schema,
metric("a", &[("i", &i.to_string())]),
1,
&[vec![Some(f64::from_bits(*bits))]],
)
})
.collect::<Vec<_>>();
let avg = aggregate_float(&schema, &series, AggregateOp::Avg, Grouping::By(&[])).unwrap();
let (_, avg) = values(&avg[0], 0);
assert_eq!(avg[0].map(f64::to_bits), Some(expected));
}
}
#[test]
fn fields_are_aggregated_independently() {
// A field without input values at a row is null there, even for count and group; the row
// exists because another field has a value. Single-field projections are the oracle.
let multi = schema(&["a", "b"]);
let single = schema(&["value"]);
let samples = [
None,
Some(-0.0),
Some(0.0),
Some(-5.0),
Some(7.5),
Some(f64::NAN),
Some(f64::INFINITY),
Some(f64::NEG_INFINITY),
Some(1e16),
];
let mut seed = 17_u64;
let mut next = || {
seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1);
(seed >> 33) as usize
};
let mut series = Vec::new();
for i in 0..24 {
let (start, len) = (next() % 80, 1 + next() % 20);
let fields = [0, 1].map(|_| (0..len).map(|_| samples[next() % samples.len()]).collect());
let host = ["h1", "h2", ""][i % 3];
series.push(input(
&multi,
metric(["a", "b"][i % 2], &[("host", host), ("i", &i.to_string())]),
start,
&fields,
));
}
let project = |field: usize| -> Vec<Series> {
series
.iter()
.filter_map(|s| {
let (start, values) = values(s, field);
let first = values.iter().position(Option::is_some)?;
let last = values.iter().rposition(Option::is_some)?;
Some(input(
&single,
s.metric.clone(),
start + first,
&[values[first..=last].to_vec()],
))
})
.collect()
};
for grouping in [
Grouping::By(&[]),
Grouping::By(&["host"]),
Grouping::Without(&["i"]),
] {
for op in OPS {
let output = aggregate_float(&multi, &series, op, grouping).unwrap();
for field in [0, 1] {
let expected = aggregate_float(&single, &project(field), op, grouping).unwrap();
let expected = expected
.iter()
.map(|s| (s.metric.clone(), values(s, 0)))
.collect::<Vec<_>>();
for actual in &output {
let (start, actual_values) = values(actual, field);
let present = actual_values
.iter()
.enumerate()
.filter_map(|(i, v)| v.map(|v| (start + i, v.to_bits())))
.collect::<Vec<_>>();
let reference = expected
.iter()
.find(|(metric, _)| metric == &actual.metric)
.map_or_else(Vec::new, |(_, (start, v))| {
v.iter()
.enumerate()
.filter_map(|(i, v)| v.map(|v| (start + i, v.to_bits())))
.collect()
});
assert_eq!(present, reference, "{op:?} {grouping:?} field {field}");
}
assert!(
expected
.iter()
.all(|(m, _)| output.iter().any(|s| &s.metric == m))
);
}
}
}
}
#[test]
fn state_blocks_keep_absolute_indices_across_boundaries_and_gaps() {
// A 512-point grid spans eight state blocks; inputs straddle a block edge, leave whole
// blocks empty and land in a far block, in both fields.
let schema = Arc::new(SeriesSchema {
fields: vec![
FieldSpec { name: "a".into() },
FieldSpec { name: "b".into() },
]
.into(),
eval_grid: Some(Arc::new(
EvalGrid::try_new(GridSpec::try_new(0, 1, 512).unwrap()).unwrap(),
)),
});
let mut gap = vec![None; 200];
gap[0] = Some(1.0);
gap[199] = Some(2.0);
let inputs = [
input(
&schema,
metric("m", &[("i", "0")]),
60,
&[vec![Some(1.0); 8], vec![Some(10.0); 8]],
),
input(
&schema,
metric("m", &[("i", "1")]),
100,
&[gap.clone(), vec![None; 200]],
),
input(
&schema,
metric("m", &[("i", "2")]),
62,
&[vec![Some(3.0); 4], vec![None, None, Some(5.0), None]],
),
input(
&schema,
metric("m", &[("i", "3")]),
509,
&[vec![Some(4.0); 3], vec![Some(6.0); 3]],
),
];
let output = aggregate_float(&schema, &inputs, AggregateOp::Sum, Grouping::By(&[])).unwrap();
assert_eq!(output.len(), 1);
let mut a = vec![None; 452];
let mut b = vec![None; 452];
for i in 60..68 {
a[i - 60] = Some(if (62..66).contains(&i) { 4.0 } else { 1.0 });
b[i - 60] = Some(if i == 64 { 15.0 } else { 10.0 });
}
a[100 - 60] = Some(1.0);
a[299 - 60] = Some(2.0);
for i in 509..512 {
a[i - 60] = Some(4.0);
b[i - 60] = Some(6.0);
}
assert_eq!(values(&output[0], 0), (60, a));
assert_eq!(values(&output[0], 1), (60, b));
}
#[test]
fn grouping_labels_follow_prometheus_and_treat_empty_values_as_absent() {
let schema = schema(&["value"]);
let series = [
input(
&schema,
metric("a", &[("host", ""), ("job", "x")]),
0,
&[vec![Some(1.0)]],
),
input(&schema, metric("b", &[("job", "x")]), 0, &[vec![Some(2.0)]]),
];
let labels = |grouping| {
aggregate_float(&schema, &series, AggregateOp::Sum, grouping)
.unwrap()
.into_iter()
.map(|s| {
let values = values(&s, 0).1;
(s.metric, values)
})
.collect::<Vec<_>>()
};
let job = MetricName::try_new(None, metric("m", &[("job", "x")]).tags().to_vec()).unwrap();
assert_eq!(
labels(Grouping::By(&["host", "job"])),
[(job.clone(), vec![Some(3.0)])]
);
// `without` always drops the name; only `by (__name__)` keeps it.
assert_eq!(
labels(Grouping::Without(&["host"])),
[(job, vec![Some(3.0)])]
);
let by_name = labels(Grouping::By(&["__name__"]));
assert_eq!(by_name.len(), 2);
assert_eq!(
by_name[0].0,
MetricName::try_new(Some("a".into()), vec![]).unwrap()
);
let unnamed = [
input(&schema, metric("", &[]), 0, &[vec![Some(1.0)]]),
input(
&schema,
MetricName::try_new(None, vec![]).unwrap(),
0,
&[vec![Some(2.0)]],
),
];
let by_name = aggregate_float(
&schema,
&unnamed,
AggregateOp::Sum,
Grouping::By(&["__name__"]),
)
.unwrap();
assert_eq!(by_name.len(), 1);
assert_eq!(by_name[0].metric.name(), None);
let none = labels(Grouping::By(&[]));
assert_eq!(
none,
[(MetricName::try_new(None, vec![]).unwrap(), vec![Some(3.0)])]
);
}
#[test]
fn min_and_max_keep_the_first_of_equal_zeros() {
// Prometheus replaces the accumulator only on a strict comparison; the tolerance treats
// the zeros as equal, but their sign shows through a later `1 / x`.
let schema = schema(&["value"]);
for (first, second) in [(0.0, -0.0), (-0.0, 0.0)] {
let series = [("0", first), ("1", second)]
.map(|(i, v)| input(&schema, metric("m", &[("i", i)]), 0, &[vec![Some(v)]]));
for op in [AggregateOp::Min, AggregateOp::Max] {
let output = aggregate_float(&schema, &series, op, Grouping::By(&[])).unwrap();
let value = values(&output[0], 0).1[0].unwrap();
assert_eq!(
value.to_bits(),
f64::to_bits(first),
"{op:?} {first} {second}"
);
}
}
}
#[test]
fn groups_keep_first_seen_order_for_nested_aggregation() {
// `sum(sum by (g) (m))` from the Prometheus engine with series created as g=a, c, b: the
// outer sum folds 1e308, -1e308, 1e308 and gets 1e308; label order would overflow.
let schema = schema(&["value"]);
let series = [("0", "a", 1e308), ("1", "c", -1e308), ("2", "b", 1e308)].map(|(a, g, v)| {
input(
&schema,
metric("m", &[("a", a), ("g", g)]),
0,
&[vec![Some(v)]],
)
});
let inner = aggregate_float(&schema, &series, AggregateOp::Sum, Grouping::By(&["g"])).unwrap();
let order = inner
.iter()
.map(|s| s.metric.get("g").unwrap())
.collect::<Vec<_>>();
assert_eq!(order, ["a", "c", "b"]);
let outer = aggregate_float(&schema, &inner, AggregateOp::Sum, Grouping::By(&[])).unwrap();
assert_eq!(values(&outer[0], 0), (0, vec![Some(1e308)]));
}
#[test]
fn grouping_keeps_type_and_unit_labels() {
// Unlike math and arithmetic, Prometheus `generateGroupingLabels` drops only the name.
let schema = schema(&["value"]);
let series = [input(
&schema,
metric(
"a",
&[("__type__", "gauge"), ("__unit__", "seconds"), ("job", "x")],
),
0,
&[vec![Some(1.0)]],
)];
let labels = |grouping| {
aggregate_float(&schema, &series, AggregateOp::Sum, grouping).unwrap()[0]
.metric
.clone()
};
assert_eq!(
labels(Grouping::Without(&["job"])),
MetricName::try_new(
None,
metric("a", &[("__type__", "gauge"), ("__unit__", "seconds")])
.tags()
.to_vec()
)
.unwrap()
);
assert_eq!(
labels(Grouping::By(&["__unit__"])),
MetricName::try_new(
None,
metric("a", &[("__unit__", "seconds")]).tags().to_vec()
)
.unwrap()
);
}
#[test]
fn empty_inputs_produce_no_groups() {
let schema = schema(&["a", "b"]);
let empty = input(&schema, metric("m", &[]), 127, &[vec![None], vec![None]]);
for op in OPS {
assert!(
aggregate_float(&schema, &[], op, Grouping::By(&[]))
.unwrap()
.is_empty()
);
let output =
aggregate_float(&schema, std::slice::from_ref(&empty), op, Grouping::By(&[])).unwrap();
assert!(output.is_empty());
}
}
#[test]
fn unsupported_shapes_and_mutated_schema_fail_without_partial_output() {
let schema = schema(&["a", "b"]);
let source = input(
&schema,
metric("m", &[]),
11,
&[vec![Some(1.0)], vec![Some(2.0)]],
);
assert!(matches!(
aggregate_float(
&schema,
std::slice::from_ref(&source),
AggregateOp::Sum,
Grouping::By(&["a"])
),
Err(DataFusionError::NotImplemented(_))
));
let mut histogram = source.clone();
let batch = HistogramBatchBuilder::new().finish().unwrap();
histogram.fields = SeriesFields::try_from_vec(vec![
source.fields.get(0).unwrap().clone(),
Samples::try_from_histogram(UInt32Array::from(vec![None]), batch).unwrap(),
])
.unwrap();
for op in OPS {
assert!(matches!(
aggregate_float(
&schema,
&[source.clone(), histogram.clone()],
op,
Grouping::By(&[])
),
Err(DataFusionError::NotImplemented(_))
));
}
let mut invalid = source.clone();
invalid.fields = SeriesFields::one(source.fields.get(0).unwrap().clone());
// Wrong in two ways at once, the series fails as it does in Map.
let mut raw_invalid = invalid.clone();
raw_invalid.time = TimeAxis::try_new(vec![1750].into()).unwrap();
assert_eq!(
aggregate_float(
&schema,
&[raw_invalid.clone()],
AggregateOp::Sum,
Grouping::By(&[])
)
.unwrap_err()
.to_string(),
crate::map::map_math(&schema, &raw_invalid, crate::map::MathOp::Abs)
.unwrap_err()
.to_string()
);
assert!(aggregate_float(&schema, &[invalid], AggregateOp::Sum, Grouping::By(&[])).is_err());
let mut raw = source.clone();
raw.time = TimeAxis::try_new(vec![1750].into()).unwrap();
assert!(aggregate_float(&schema, &[raw], AggregateOp::Sum, Grouping::By(&[])).is_err());
let mut other_grid = schema.as_ref().clone();
other_grid.eval_grid = Some(Arc::new(
EvalGrid::try_new(GridSpec::try_new(0, 250, 128).unwrap()).unwrap(),
));
let other_grid = Arc::new(other_grid);
assert!(aggregate_float(&other_grid, &[source], AggregateOp::Sum, Grouping::By(&[])).is_err());
let mut raw_schema = schema.as_ref().clone();
raw_schema.eval_grid = None;
let raw_schema = Arc::new(raw_schema);
assert!(aggregate_float(&raw_schema, &[], AggregateOp::Sum, Grouping::By(&[])).is_err());
}
src/promql/src/golden.rs
+224
View File
@@ -0,0 +1,224 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Fixtures evaluated by the Prometheus engine; each fixture's header names its generator.
use std::sync::Arc;
use datatypes::arrow::array::Float64Array;
use datatypes::series::samples::{
FieldSpec, SampleRef, Samples, Series, SeriesFields, SeriesSchema,
};
use datatypes::series::time::{EvalGrid, GridRange, GridSpec, TimeAxis};
use datatypes::series::{Label, MetricName};
type Points = Vec<(usize, f64)>;
pub(crate) struct Case {
pub(crate) header: Vec<String>,
pub(crate) expr: String,
pub(crate) schema: Arc<SeriesSchema>,
pub(crate) series: Vec<Series>,
pub(crate) scalar: Option<Series>,
pub(crate) result: Vec<(MetricName, Points)>,
}
/// Parses a fixture of `case`/`expr`/`grid`/`series`/`scalar`/`result`/`end` blocks.
pub(crate) fn cases(text: &str) -> Vec<Case> {
let mut cases = Vec::new();
let mut lines = text.lines().filter(|line| !line.starts_with('#'));
while let Some(line) = lines.next() {
let header = line.strip_prefix("case ").expect("case line");
let expr = lines
.next()
.unwrap()
.strip_prefix("expr ")
.unwrap()
.to_string();
let grid: Vec<i64> = lines.next().unwrap()["grid ".len()..]
.split(' ')
.map(|v| v.parse().unwrap())
.collect();
let schema = Arc::new(SeriesSchema {
fields: vec![FieldSpec {
name: "value".into(),
}]
.into(),
eval_grid: Some(Arc::new(
EvalGrid::try_new(GridSpec::try_new(grid[0], grid[1], grid[2] as u64).unwrap())
.unwrap(),
)),
});
let mut case = Case {
header: header.split(' ').map(String::from).collect(),
expr,
series: Vec::new(),
scalar: None,
result: Vec::new(),
schema,
};
for line in lines.by_ref() {
if line == "end" {
break;
}
let (head, points) = line.split_once(';').unwrap();
let (kind, labels) = head
.trim_end()
.split_once(' ')
.unwrap_or((head.trim_end(), "-"));
let (metric, points) = (parse_metric(labels), parse_points(points));
match kind {
"series" => {
if let Some(series) = grid_series(&case.schema, metric, &points) {
case.series.push(series);
}
}
"scalar" => case.scalar = grid_series(&case.schema, metric, &points),
"result" => case.result.push((metric, points)),
_ => panic!("unknown line {line}"),
}
}
cases.push(case);
}
cases
}
fn parse_metric(labels: &str) -> MetricName {
let mut name = None;
let mut tags = Vec::new();
for label in labels
.split(',')
.filter(|label| *label != "-" && !label.is_empty())
{
let (key, value) = label.split_once('=').unwrap();
if key == "__name__" {
name = Some(Arc::from(value));
} else {
tags.push(Label {
name: key.into(),
value: value.into(),
});
}
}
MetricName::try_new(name, tags).unwrap()
}
fn parse_points(points: &str) -> Points {
points
.split_whitespace()
.map(|point| {
let (index, bits) = point.split_once(':').unwrap();
(
index.parse().unwrap(),
f64::from_bits(u64::from_str_radix(bits, 16).unwrap()),
)
})
.collect()
}
pub(crate) fn hex(bits: &str) -> f64 {
f64::from_bits(u64::from_str_radix(bits, 16).unwrap())
}
/// Builds the trimmed series Eval produces: from the first to the last point, nulls between.
fn grid_series(schema: &SeriesSchema, metric: MetricName, points: &Points) -> Option<Series> {
let (&(start, _), &(last, _)) = points.first().zip(points.last())?;
let mut values = vec![None; last - start + 1];
for &(index, value) in points {
values[index - start] = Some(value);
}
let range = GridRange::try_new(schema.eval_grid.clone().unwrap(), start, values.len()).unwrap();
Some(
Series::try_new(
schema,
metric,
TimeAxis::Grid(range),
SeriesFields::one(Samples::from_float(Float64Array::from(values))),
)
.unwrap(),
)
}
/// Compares labels and the trimmed absolute range exactly, and values with [`close`].
pub(crate) fn assert_result(case: &Case, mut actual: Vec<Series>) {
actual.sort_by_key(|series| label_key(&series.metric));
let mut expected = case.result.iter().collect::<Vec<_>>();
expected.sort_by_key(|(metric, _)| label_key(metric));
let context = || format!("{} ({})", case.expr, case.header.join(" "));
assert_eq!(actual.len(), expected.len(), "{}", context());
for (actual, (metric, points)) in actual.iter().zip(expected) {
assert_eq!(&actual.metric, metric, "{}", context());
let TimeAxis::Grid(range) = &actual.time else {
panic!("{}: lost grid axis", context());
};
let mut present = Vec::new();
for field in actual.fields.iter() {
for (offset, sample) in field.iter().enumerate() {
match sample {
SampleRef::Float(value) => present.push((range.start() + offset, value)),
SampleRef::Missing => {}
_ => panic!("{}: non-float output", context()),
}
}
}
assert_eq!(
(range.start(), range.start() + range.len()),
(points[0].0, points.last().unwrap().0 + 1),
"{}: untrimmed range",
context()
);
assert_eq!(present.len(), points.len(), "{}", context());
for (&(index, actual), &(expected_index, expected)) in present.iter().zip(points) {
assert_eq!(index, expected_index, "{}", context());
assert!(
close(actual, expected),
"{} at {index}: {actual:e} ({:016x}) != {expected:e} ({:016x})",
context(),
actual.to_bits(),
expected.to_bits(),
);
}
}
}
/// Prometheus `almost.Equal` with relative tolerance 1e-10, treating results below 1e-10 as
/// equal and any NaN as equal to NaN, as promql-compat compares engines.
pub(crate) fn close(a: f64, b: f64) -> bool {
const EPSILON: f64 = 1e-10;
if a.is_nan() || b.is_nan() {
return a.is_nan() && b.is_nan();
}
if a == b {
return true;
}
let (diff, sum) = ((a - b).abs(), a.abs() + b.abs());
let equal = if a == 0.0 || b == 0.0 || sum < f64::MIN_POSITIVE {
diff < EPSILON * f64::MIN_POSITIVE
} else {
diff / sum.min(f64::MAX) < EPSILON
};
equal || (a.abs() < EPSILON && b.abs() < EPSILON)
}
fn label_key(metric: &MetricName) -> Vec<(String, String)> {
let mut key = metric
.tags()
.iter()
.map(|label| (label.name.to_string(), label.value.to_string()))
.collect::<Vec<_>>();
if let Some(name) = metric.name() {
key.insert(0, ("__name__".into(), name.into()));
}
key
}
src/promql/src/lib.rs
+3
View File
@@ -12,10 +12,13 @@
// See the License for the specific language governing permissions and
// limitations under the License.
pub mod aggregate;
pub mod error;
pub mod extension_plan;
pub mod functions;
mod go_sort;
#[cfg(test)]
mod golden;
pub mod grid_series;
pub mod histogram_eval;
pub mod instant_eval;
src/promql/src/map.rs
+301 -27
View File
@@ -17,14 +17,25 @@
//! Expression column naming and omission of missing points belong to the planner/output adapter;
//! these kernels retain the shared field schema and represent missing grid positions by validity.
mod broadcast;
mod comparison;
mod go_math;
#[cfg(test)]
mod tests;
pub use broadcast::{ScalarSeriesOperand, map_arithmetic_series, map_compare_series};
pub use comparison::{Comparison, ComparisonOp, map_compare, scalar_arithmetic, scalar_compare};
use std::f64::consts::PI;
use std::sync::Arc;
use datafusion::error::{DataFusionError, Result};
use datatypes::arrow::array::Float64Array;
use datatypes::series::MetricName;
use datatypes::series::samples::{DenseSamplesRef, Samples, Series, SeriesFields, SeriesSchema};
use datatypes::series::time::TimeAxis;
use datatypes::series::samples::{
DenseSamplesRef, SampleRef, Samples, Series, SeriesFields, SeriesSchema,
};
use datatypes::series::time::{GridRange, TimeAxis};
use crate::functions::Round;
@@ -40,7 +51,57 @@ pub enum MathOp {
/// IEEE square root, returning NaN for negative input.
Sqrt,
/// Rounds to a constant multiple; use `1.0` for PromQL's default argument.
Round { nearest: f64 },
Round {
nearest: f64,
},
Exp,
Ln,
Log2,
Log10,
Sin,
Cos,
Tan,
Asin,
Acos,
Atan,
Sinh,
Cosh,
Tanh,
Asinh,
Acosh,
Atanh,
Deg,
Rad,
Sgn,
Clamp {
min: f64,
max: f64,
},
ClampMin {
min: f64,
},
ClampMax {
max: f64,
},
}
/// Float arithmetic between a vector and a scalar, as in Prometheus.
#[derive(Clone, Copy, Debug)]
pub enum ArithmeticOp {
Add,
Subtract,
Multiply,
Divide,
Modulo,
Power,
Atan2,
}
/// The scalar's position in a binary expression; noncommutative operators retain this order.
#[derive(Clone, Copy, Debug)]
pub enum ScalarOperand {
Left(f64),
Right(f64),
}
/// Applies float math to every field of one evaluated series and drops the metric name, like
@@ -56,74 +117,287 @@ pub enum MathOp {
/// admitted to the ordinary path before execution; this function neither falls back nor returns
/// partial results.
///
/// Returns `None` for `clamp` with `max < min`, which yields no series in Prometheus.
///
/// Distinct inputs can share a label set once the name is dropped. Merging them, or failing
/// when both have a value at one step, is left to the caller (Prometheus
/// `mergeSeriesWithSameLabelset`).
pub fn map_math(schema: &SeriesSchema, series: &Series, op: MathOp) -> Result<Series> {
if !matches!(series.time, TimeAxis::Grid(_)) {
return Err(DataFusionError::Execution(
"Map math requires an evaluated grid time axis".into(),
));
pub fn map_math(schema: &SeriesSchema, series: &Series, op: MathOp) -> Result<Option<Series>> {
if op.is_empty_clamp() {
validate_float_series(schema, series)?;
return Ok(None);
}
let mut output = map_float_fields(schema, series, |floats| op.apply(floats))?;
output.metric = drop_metric_name(&output.metric)?;
Ok(Some(output))
}
/// Negates every float field and drops the metric name, like Prometheus' unary minus.
///
/// Shares the evaluated-grid, missingness, unsupported-type and same-labelset contract of
/// [`map_math`].
pub fn map_negate(schema: &SeriesSchema, series: &Series) -> Result<Series> {
let mut output = map_float_fields(schema, series, |floats| floats.unary(|v| -v))?;
output.metric = drop_metric_name(&output.metric)?;
Ok(output)
}
/// Applies arithmetic against a constant scalar to every field and drops the metric name.
///
/// Shares the evaluated-grid, missingness, unsupported-type and same-labelset contract of
/// [`map_math`]. Results follow Prometheus `vectorElemBinop`, including Go's `math.Pow`,
/// `math.Mod` and `math.Atan2`.
pub fn map_arithmetic(
schema: &SeriesSchema,
series: &Series,
op: ArithmeticOp,
scalar: ScalarOperand,
) -> Result<Series> {
let mut output = map_float_fields(schema, series, |floats| op.apply(floats, scalar))?;
output.metric = drop_metric_name(&output.metric)?;
Ok(output)
}
/// Produces an unlabelled scalar over exactly the requested evaluated range.
///
/// Scalar results require a single-field schema. No points outside `range` are allocated.
pub fn constant_series(schema: &SeriesSchema, range: GridRange, scalar: f64) -> Result<Series> {
let fields = SeriesFields::one(Samples::from_float(Float64Array::from(vec![
scalar;
range.len()
])));
Series::try_new(
schema,
MetricName::try_new(None, vec![]).map_err(series_error)?,
TimeAxis::Grid(range),
fields,
)
.map_err(series_error)
}
/// Evaluates `pi()` without a vector input, preserving the requested short grid range.
pub fn map_pi(schema: &SeriesSchema, range: GridRange) -> Result<Series> {
constant_series(schema, range, PI)
}
fn map_float_fields(
schema: &SeriesSchema,
series: &Series,
map: impl Fn(&Float64Array) -> Float64Array,
) -> Result<Series> {
validate_float_series(schema, series)?;
let fields = series
.fields
.iter()
.map(|samples| map_samples(samples, op))
.map(|samples| map_samples(samples, &map))
.collect::<Result<Vec<_>>>()?;
let fields = SeriesFields::try_from_vec(fields).map_err(series_error)?;
let metric = drop_metric_name(&series.metric)?;
Series::try_new(schema, metric, series.time.clone(), fields).map_err(series_error)
Series::try_new(schema, series.metric.clone(), series.time.clone(), fields)
.map_err(series_error)
}
/// Prometheus `DropReserved(schema.IsMetadataLabel)` drops the type and unit labels with the
/// name. Empty values are dropped too: Prometheus has no empty labels, and keeping them would
/// hide a same-labelset collision from the caller.
pub(crate) fn drop_metric_name(metric: &MetricName) -> Result<MetricName> {
// One rebuild: removing labels one by one copies the label array each time.
rebuild_labels(metric, None, |name| {
!matches!(name, "__type__" | "__unit__")
})
}
/// Drops labels with empty values, which Prometheus treats as absent.
pub(crate) fn drop_empty_labels(metric: &MetricName) -> Result<MetricName> {
let name = metric.name().filter(|name| !name.is_empty()).map(Arc::from);
rebuild_labels(metric, name, |_| true)
}
/// One rebuild: removing labels one by one copies the label array each time.
fn rebuild_labels(
metric: &MetricName,
name: Option<Arc<str>>,
keep: impl Fn(&str) -> bool,
) -> Result<MetricName> {
let tags = metric
.tags()
.iter()
.filter(|label| {
!label.value.is_empty() && !matches!(label.name.as_ref(), "__type__" | "__unit__")
})
.filter(|label| !label.value.is_empty() && keep(&label.name))
.cloned()
.collect();
MetricName::try_new(None, tags).map_err(series_error)
MetricName::try_new(name, tags).map_err(series_error)
}
impl MathOp {
fn apply(self, floats: &Float64Array) -> Float64Array {
floats.unary(|sample| self.calculate(sample))
}
/// The function applied to one sample.
pub(crate) fn calculate(self, v: f64) -> f64 {
match self {
Self::Abs => floats.unary(f64::abs),
Self::Ceil => floats.unary(f64::ceil),
Self::Floor => floats.unary(f64::floor),
Self::Sqrt => floats.unary(f64::sqrt),
Self::Round { nearest } => {
floats.unary(|sample| Round::round_to_nearest(sample, nearest))
Self::Exp => go_math::exp(v),
Self::Ln => go_math::log(v),
Self::Log2 => go_math::log2(v),
Self::Log10 => go_math::log10(v),
Self::Sin => go_math::sin(v),
Self::Cos => go_math::cos(v),
Self::Tan => go_math::tan(v),
Self::Asin => v.asin(),
Self::Acos => go_math::acos(v),
Self::Atan => v.atan(),
Self::Sinh => go_math::sinh(v),
Self::Cosh => go_math::cosh(v),
Self::Tanh => v.tanh(),
Self::Asinh => v.asinh(),
Self::Acosh => v.acosh(),
Self::Atanh => v.atanh(),
// Prometheus' operation order, which rounds differently from to_degrees/to_radians.
Self::Deg => v * 180.0 / PI,
Self::Rad => v * PI / 180.0,
// Zeros and NaN are returned unchanged, so -0 keeps its sign.
Self::Sgn => {
if v < 0.0 {
-1.0
} else if v > 0.0 {
1.0
} else {
v
}
}
Self::Clamp { min, max } => go_max(min, go_min(max, v)),
Self::ClampMin { min } => go_max(min, v),
Self::ClampMax { max } => go_min(max, v),
Self::Abs => v.abs(),
Self::Ceil => v.ceil(),
Self::Floor => v.floor(),
Self::Sqrt => v.sqrt(),
Self::Round { nearest } => Round::round_to_nearest(v, nearest),
}
}
/// Prometheus returns an empty vector when `max < min`; NaN bounds do not compare.
fn is_empty_clamp(self) -> bool {
match self {
Self::Clamp { min, max } => max < min,
_ => false,
}
}
}
fn map_samples(samples: &Samples, op: MathOp) -> Result<Samples> {
/// Go `math.Max`: +Inf wins over NaN, and +0 over -0.
fn go_max(x: f64, y: f64) -> f64 {
if x == f64::INFINITY || y == f64::INFINITY {
f64::INFINITY
} else if x.is_nan() || y.is_nan() {
f64::NAN
} else if x == 0.0 && x == y {
if x.is_sign_negative() { y } else { x }
} else if x > y {
x
} else {
y
}
}
/// Go `math.Min`: -Inf wins over NaN, and -0 over +0.
fn go_min(x: f64, y: f64) -> f64 {
if x == f64::NEG_INFINITY || y == f64::NEG_INFINITY {
f64::NEG_INFINITY
} else if x.is_nan() || y.is_nan() {
f64::NAN
} else if x == 0.0 && x == y {
if x.is_sign_negative() { x } else { y }
} else if x < y {
x
} else {
y
}
}
impl ArithmeticOp {
/// Prometheus `vectorElemBinop` for two floats.
pub(crate) fn calculate(self, left: f64, right: f64) -> f64 {
match self {
Self::Add => left + right,
Self::Subtract => left - right,
Self::Multiply => left * right,
Self::Divide => left / right,
Self::Modulo => go_math::fmod(left, right),
Self::Power => go_math::pow(left, right),
Self::Atan2 => go_math::atan2(left, right),
}
}
fn apply(self, floats: &Float64Array, scalar: ScalarOperand) -> Float64Array {
match scalar {
ScalarOperand::Left(scalar) => floats.unary(|sample| self.calculate(scalar, sample)),
ScalarOperand::Right(scalar) => floats.unary(|sample| self.calculate(sample, scalar)),
}
}
}
fn map_samples(samples: &Samples, map: &impl Fn(&Float64Array) -> Float64Array) -> Result<Samples> {
let chunks = samples
.segments()
.iter()
.map(|segment| map_float_segment(segment, op))
.map(|segment| map_float_segment(segment, map))
.collect::<Result<Vec<_>>>()?;
Samples::try_from_chunks(chunks).map_err(series_error)
}
fn map_float_segment(segment: &Samples, op: MathOp) -> Result<Samples> {
fn map_float_segment(
segment: &Samples,
map: &impl Fn(&Float64Array) -> Float64Array,
) -> Result<Samples> {
let Some(DenseSamplesRef::Float(floats)) = segment.dense() else {
return Err(DataFusionError::NotImplemented(
"Map math supports only float samples; histogram/mixed samples are not supported"
"Series operators support only float samples; histogram/mixed samples are not supported"
.into(),
));
};
Ok(Samples::from_float(op.apply(floats)))
Ok(Samples::from_float(map(floats)))
}
fn series_error(error: datatypes::error::Error) -> DataFusionError {
DataFusionError::External(Box::new(error))
}
/// Checks an evaluated float series up front, so no segment of another type yields a result.
pub(crate) fn validate_float_series<'a>(
schema: &SeriesSchema,
series: &'a Series,
) -> Result<&'a GridRange> {
let TimeAxis::Grid(range) = &series.time else {
return Err(DataFusionError::Execution(
"Series operators require an evaluated grid time axis".into(),
));
};
Series::try_new(
schema,
series.metric.clone(),
series.time.clone(),
series.fields.clone(),
)
.map_err(series_error)?;
for field in series.fields.iter() {
if field
.segments()
.iter()
.any(|segment| !matches!(segment.dense(), Some(DenseSamplesRef::Float(_))))
{
return Err(DataFusionError::NotImplemented(
"Series operators support only float samples; histogram/mixed samples are not supported"
.into(),
));
}
}
Ok(range)
}
/// A sample of a series accepted by [`validate_float_series`].
fn float_sample(sample: SampleRef<'_>) -> Option<f64> {
match sample {
SampleRef::Float(sample) => Some(sample),
SampleRef::Missing => None,
_ => unreachable!("validated as float"),
}
}
src/promql/src/map/broadcast.rs
+141
View File
@@ -0,0 +1,141 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::sync::Arc;
use datafusion::error::{DataFusionError, Result};
use datatypes::series::MetricName;
use datatypes::series::samples::{FieldSpec, Samples, Series, SeriesSchema};
use datatypes::series::time::GridRange;
use crate::grid_series::GridSeriesBuilder;
use crate::map::{
ArithmeticOp, Comparison, drop_metric_name, float_sample, series_error, validate_float_series,
};
/// A time-varying, unlabelled single-field scalar, with explicit operand direction.
#[derive(Clone, Copy, Debug)]
pub enum ScalarSeriesOperand<'a> {
Left(&'a Series),
Right(&'a Series),
}
/// Applies arithmetic between vector fields and a time-varying scalar, aligned by absolute grid
/// index, and drops the metric name.
///
/// Prometheus scalars exist at every step; a missing scalar point here yields a missing output.
/// The output is trimmed to its first and last valid rows, and `None` means no valid row. The
/// scalar must be unlabelled and single-field on the same grid; histogram/mixed/raw inputs are
/// rejected even outside the overlap. Two unlabelled single-field inputs also evaluate
/// scalar–scalar operations. Same-labelset merging is left to the caller, as in
/// [`crate::map::map_math`].
pub fn map_arithmetic_series(
schema: &Arc<SeriesSchema>,
vector: &Series,
op: ArithmeticOp,
scalar: ScalarSeriesOperand<'_>,
) -> Result<Option<Series>> {
let metric = drop_metric_name(&vector.metric)?;
map_grid_scalar(schema, vector, scalar, metric, |left, right, _| {
Some(op.calculate(left, right))
})
}
/// Compares against a time-varying scalar. Filter keeps vector values and labels; bool emits
/// 0/1 and drops the metric name.
///
/// Shares [`map_arithmetic_series`]'s alignment and trimming contract. Each field is compared
/// independently; a row is kept where any field passes a filter.
pub fn map_compare_series(
schema: &Arc<SeriesSchema>,
vector: &Series,
comparison: Comparison,
scalar: ScalarSeriesOperand<'_>,
) -> Result<Option<Series>> {
let metric = if matches!(comparison, Comparison::Bool(_)) {
drop_metric_name(&vector.metric)?
} else {
vector.metric.clone()
};
map_grid_scalar(schema, vector, scalar, metric, |left, right, sample| {
comparison.apply(left, right, sample)
})
}
/// Calls `calculate(left, right, vector_sample)` at each index where both operands exist.
fn map_grid_scalar(
schema: &Arc<SeriesSchema>,
vector: &Series,
scalar: ScalarSeriesOperand<'_>,
metric: MetricName,
calculate: impl Fn(f64, f64, f64) -> Option<f64>,
) -> Result<Option<Series>> {
let vector_range = validate_float_series(schema, vector)?;
let (scalar_series, scalar_left) = match scalar {
ScalarSeriesOperand::Left(s) => (s, true),
ScalarSeriesOperand::Right(s) => (s, false),
};
let scalar_range = validate_scalar(schema, scalar_series)?;
let start = vector_range.start().max(scalar_range.start());
let end =
(vector_range.start() + vector_range.len()).min(scalar_range.start() + scalar_range.len());
let mut builder = GridSeriesBuilder::try_new(schema.clone())?;
if start >= end {
return builder.finish(metric);
}
let len = end - start;
let scalar_samples = scalar_series.fields.as_slice()[0]
.slice(start - scalar_range.start(), len)
.map_err(series_error)?;
let fields = vector
.fields
.iter()
.map(|field| field.slice(start - vector_range.start(), len))
.collect::<std::result::Result<Vec<_>, _>>()
.map_err(series_error)?;
let mut fields = fields.iter().map(Samples::iter).collect::<Vec<_>>();
let mut row = Vec::with_capacity(fields.len());
for (offset, scalar) in scalar_samples.iter().enumerate() {
let scalar = float_sample(scalar);
row.clear();
for field in &mut fields {
let sample = float_sample(field.next().expect("sliced to the same length"));
row.push(scalar.zip(sample).and_then(|(scalar, sample)| {
if scalar_left {
calculate(scalar, sample, sample)
} else {
calculate(sample, scalar, sample)
}
}));
}
builder.push(start + offset, &row)?;
}
builder.finish(metric)
}
fn validate_scalar<'a>(schema: &SeriesSchema, scalar: &'a Series) -> Result<&'a GridRange> {
if scalar.metric.name().is_some() || !scalar.metric.tags().is_empty() {
return Err(DataFusionError::Execution(
"grid scalar must be unlabelled".into(),
));
}
let schema = SeriesSchema {
fields: vec![FieldSpec {
name: "scalar".into(),
}]
.into(),
eval_grid: schema.eval_grid.clone(),
};
validate_float_series(&schema, scalar)
}
src/promql/src/map/comparison.rs
+128
View File
@@ -0,0 +1,128 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::sync::Arc;
use datafusion::error::Result;
use datatypes::series::samples::{Series, SeriesSchema};
use datatypes::series::time::GridRange;
use crate::grid_series::GridSeriesBuilder;
use crate::map::{
ArithmeticOp, ScalarOperand, constant_series, drop_metric_name, float_sample, map_arithmetic,
validate_float_series,
};
/// IEEE comparisons: NaN only satisfies `NotEqual`, and signed zeros compare equal.
#[derive(Clone, Copy, Debug)]
pub enum ComparisonOp {
Equal,
NotEqual,
Less,
LessEqual,
Greater,
GreaterEqual,
}
/// Filter keeps the vector value and labels; bool emits 0/1 and drops the metric name.
#[derive(Clone, Copy, Debug)]
pub enum Comparison {
Filter(ComparisonOp),
Bool(ComparisonOp),
}
/// Compares a vector with a constant scalar, like Prometheus `VectorscalarBinop`.
///
/// Each field is compared independently; a row is kept where any field passes a filter. The
/// output is trimmed to its first and last kept rows, and `None` means nothing passed. Raw axes
/// and histogram/mixed inputs share [`crate::map::map_math`]'s rejection contract.
pub fn map_compare(
schema: &Arc<SeriesSchema>,
series: &Series,
comparison: Comparison,
scalar: ScalarOperand,
) -> Result<Option<Series>> {
let range = validate_float_series(schema, series)?;
let mut builder = GridSeriesBuilder::try_new(schema.clone())?;
let mut fields = series
.fields
.iter()
.map(|field| field.iter())
.collect::<Vec<_>>();
let mut row = Vec::with_capacity(fields.len());
for index in range.start()..range.start() + range.len() {
row.clear();
for field in &mut fields {
let sample = float_sample(field.next().expect("validated length"));
row.push(sample.and_then(|sample| {
let (left, right) = match scalar {
ScalarOperand::Left(scalar) => (scalar, sample),
ScalarOperand::Right(scalar) => (sample, scalar),
};
comparison.apply(left, right, sample)
}));
}
builder.push(index, &row)?;
}
let metric = if matches!(comparison, Comparison::Bool(_)) {
drop_metric_name(&series.metric)?
} else {
series.metric.clone()
};
builder.finish(metric)
}
/// Evaluates scalar–scalar arithmetic as an unlabelled, single-field series on `range`.
pub fn scalar_arithmetic(
schema: &SeriesSchema,
range: GridRange,
op: ArithmeticOp,
operands: [f64; 2],
) -> Result<Series> {
let left = constant_series(schema, range, operands[0])?;
map_arithmetic(schema, &left, op, ScalarOperand::Right(operands[1]))
}
/// Evaluates scalar–scalar comparison, which requires bool in PromQL, on `range`.
pub fn scalar_compare(
schema: &SeriesSchema,
range: GridRange,
op: ComparisonOp,
operands: [f64; 2],
) -> Result<Series> {
constant_series(schema, range, f64::from(op.apply(operands[0], operands[1])))
}
impl Comparison {
/// The output for one `left op right` pair, where `vector` is the vector operand's value.
pub(crate) fn apply(self, left: f64, right: f64, vector: f64) -> Option<f64> {
match self {
Self::Filter(op) => op.apply(left, right).then_some(vector),
Self::Bool(op) => Some(f64::from(op.apply(left, right))),
}
}
}
impl ComparisonOp {
fn apply(self, left: f64, right: f64) -> bool {
match self {
Self::Equal => left == right,
Self::NotEqual => left != right,
Self::Less => left < right,
Self::LessEqual => left <= right,
Self::Greater => left > right,
Self::GreaterEqual => left >= right,
}
}
}
src/promql/src/map/go_math.rs
+592
View File
@@ -0,0 +1,592 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! The Go `math` functions whose results on amd64 differ from Rust's standard library beyond
//! the comparison tolerance (relative 1e-10), or in whether they overflow or keep their sign.
//! Everything else uses the standard library. Inputs where the standard library fails, all in
//! `go_math_cases.txt`:
//!
//! - `exp`, `sinh`, `cosh`: +Inf from `exp(709.44)` on, where libm is finite up to 709.78.
//! - `log`, `log10`: `log_amd64.s` does not normalize subnormals (`log(5e-324)` is -709.09,
//! not -744.44).
//! - `log2`: `log(frac) / ln 2 + exp` cancels near 1 (`log2(1.0000000000882365)`).
//! - `acos`: `π/2 - asin(x)` cancels near 1 (`acos(0.999999992587986)`).
//! - `sin`, `cos`, `tan`: `trigReduce` loses precision from 2^29 on (`sin(1.0446683604636763e12)`).
//! - `pow`: repeated squaring accumulates error (`pow(1.0000006850274261, 9.80019037e8)`), and
//! subnormal bases go through `log`.
//! - `atan2`: `atan(y / x)` underflows to +0 and picks the wrong half-turn
//! (`atan2(-1e-300, -1e300)` is +π).
//!
//! `asin` and `atan` are kept only as dependencies of `acos` and `atan2`.
#![allow(clippy::excessive_precision)]
use std::f64::consts::{FRAC_1_SQRT_2, FRAC_PI_2, FRAC_PI_4, LOG2_E, PI};
#[cfg(test)]
mod tests;
const MASK: u64 = 0x7FF;
const SHIFT: u32 = 64 - 11 - 1;
const BIAS: i64 = 1023;
// Go folds constant expressions exactly before rounding, so derived constants are spelled
// as Go's bits where a Rust expression would round differently.
const FOUR_OVER_PI: f64 = f64::from_bits(0x3ff45f306dc9c883);
const THREE_PI_4: f64 = f64::from_bits(0x4002d97c7f3321d2);
const INV_LN10: f64 = f64::from_bits(0x3fdbcb7b1526e50e);
const INV_LN2: f64 = f64::from_bits(0x3ff71547652b82fe);
/// Go `math.Exp` on amd64 overflows once `x * log2(e)` rounds to 1024, from about 709.44
/// (`exp_amd64.s`); below that libm is within the tolerance.
pub(crate) fn exp(x: f64) -> f64 {
if x.is_finite() && (LOG2_E * x).round_ties_even() >= 1024.0 {
f64::INFINITY
} else {
x.exp()
}
}
/// Go `math.Mod`. Both Go's loop and C `fmod` compute the exact remainder with the sign of
/// `x`, and return NaN for a zero divisor, an infinite dividend or NaN input.
pub(crate) fn fmod(x: f64, y: f64) -> f64 {
x % y
}
/// Go `math.Pow`.
pub(crate) fn pow(x: f64, y: f64) -> f64 {
if y == 0.0 || x == 1.0 {
return 1.0;
}
if y == 1.0 {
return x;
}
if x.is_nan() || y.is_nan() {
return f64::NAN;
}
if x == 0.0 {
if y < 0.0 {
return if x.is_sign_negative() && is_odd_int(y) {
f64::NEG_INFINITY
} else {
f64::INFINITY
};
}
return if x.is_sign_negative() && is_odd_int(y) {
x
} else {
0.0
};
}
if y.is_infinite() {
return if x == -1.0 {
1.0
} else if (x.abs() < 1.0) == (y > 0.0) {
0.0
} else {
f64::INFINITY
};
}
if x.is_infinite() {
if x < 0.0 {
return pow(1.0 / x, -y);
}
return if y < 0.0 { 0.0 } else { f64::INFINITY };
}
if y == 0.5 {
return x.sqrt();
}
if y == -0.5 {
return 1.0 / x.sqrt();
}
let (mut yi, mut yf) = (y.abs().trunc(), y.abs().fract());
if yf != 0.0 && x < 0.0 {
return f64::NAN;
}
if yi >= (1u64 << 63) as f64 {
return if x == -1.0 {
1.0
} else if (x.abs() < 1.0) == (y > 0.0) {
0.0
} else {
f64::INFINITY
};
}
let mut a1 = 1.0;
let mut ae: i64 = 0;
if yf != 0.0 {
if yf > 0.5 {
yf -= 1.0;
yi += 1.0;
}
a1 = exp(yf * log(x));
}
let (mut x1, mut xe) = frexp(x);
let mut i = yi as i64;
while i != 0 {
if !(-(1 << 12)..=(1 << 12)).contains(&xe) {
ae += xe;
break;
}
if i & 1 == 1 {
a1 *= x1;
ae += xe;
}
x1 *= x1;
xe <<= 1;
if x1 < 0.5 {
x1 += x1;
xe -= 1;
}
i >>= 1;
}
if y < 0.0 {
a1 = 1.0 / a1;
ae = -ae;
}
ldexp(a1, ae)
}
fn is_odd_int(x: f64) -> bool {
if x.abs() >= (1u64 << 53) as f64 {
return false;
}
x.fract() == 0.0 && (x.trunc() as i64) & 1 == 1
}
fn normalize(x: f64) -> (f64, i64) {
if x.abs() < f64::MIN_POSITIVE {
return (x * (1u64 << 52) as f64, -52);
}
(x, 0)
}
/// Go `math.Frexp`.
fn frexp(f: f64) -> (f64, i64) {
if f == 0.0 || f.is_infinite() || f.is_nan() {
return (f, 0);
}
let (f, mut exp) = normalize(f);
let mut x = f.to_bits();
exp += ((x >> SHIFT) & MASK) as i64 - BIAS + 1;
x &= !(MASK << SHIFT);
x |= ((-1 + BIAS) as u64) << SHIFT;
(f64::from_bits(x), exp)
}
/// Go `math.Ldexp`.
fn ldexp(frac: f64, exp: i64) -> f64 {
if frac == 0.0 || frac.is_infinite() || frac.is_nan() {
return frac;
}
let (frac, e) = normalize(frac);
let mut exp = exp + e;
let mut x = frac.to_bits();
exp += ((x >> SHIFT) & MASK) as i64 - BIAS;
if exp < -1075 {
return 0.0_f64.copysign(frac);
}
if exp > 1023 {
return if frac < 0.0 {
f64::NEG_INFINITY
} else {
f64::INFINITY
};
}
let mut m = 1.0;
if exp < -1022 {
exp += 53;
m = 1.0 / (1u64 << 53) as f64;
}
x &= !(MASK << SHIFT);
x |= ((exp + BIAS) as u64) << SHIFT;
m * f64::from_bits(x)
}
/// Go `math.Atan2`.
pub(crate) fn atan2(y: f64, x: f64) -> f64 {
if y.is_nan() || x.is_nan() {
return f64::NAN;
}
if y == 0.0 {
if x >= 0.0 && !x.is_sign_negative() {
return 0.0_f64.copysign(y);
}
return PI.copysign(y);
}
if x == 0.0 {
return FRAC_PI_2.copysign(y);
}
if x.is_infinite() {
if x > 0.0 {
return if y.is_infinite() {
FRAC_PI_4.copysign(y)
} else {
0.0_f64.copysign(y)
};
}
return if y.is_infinite() {
THREE_PI_4.copysign(y)
} else {
PI.copysign(y)
};
}
if y.is_infinite() {
return FRAC_PI_2.copysign(y);
}
let q = atan(y / x);
if x < 0.0 {
if q <= 0.0 {
return q + PI;
}
return q - PI;
}
q
}
/// Go `math.Atan`.
fn atan(x: f64) -> f64 {
if x == 0.0 {
return x;
}
if x > 0.0 { satan(x) } else { -satan(-x) }
}
fn satan(x: f64) -> f64 {
const MOREBITS: f64 = 6.123233995736765886130e-17;
const TAN3PIO8: f64 = 2.41421356237309504880;
if x <= 0.66 {
return xatan(x);
}
if x > TAN3PIO8 {
return FRAC_PI_2 - xatan(1.0 / x) + MOREBITS;
}
FRAC_PI_4 + xatan((x - 1.0) / (x + 1.0)) + 0.5 * MOREBITS
}
fn xatan(x: f64) -> f64 {
const P0: f64 = -8.750608600031904122785e-01;
const P1: f64 = -1.615753718733365076637e+01;
const P2: f64 = -7.500855792314704667340e+01;
const P3: f64 = -1.228866684490136173410e+02;
const P4: f64 = -6.485021904942025371773e+01;
const Q0: f64 = 2.485846490142306297962e+01;
const Q1: f64 = 1.650270098316988542046e+02;
const Q2: f64 = 4.328810604912902668951e+02;
const Q3: f64 = 4.853903996359136964868e+02;
const Q4: f64 = 1.945506571482613964425e+02;
let mut z = x * x;
z = z * ((((P0 * z + P1) * z + P2) * z + P3) * z + P4)
/ (((((z + Q0) * z + Q1) * z + Q2) * z + Q3) * z + Q4);
x * z + x
}
/// Go `math.Asin`.
fn asin(x: f64) -> f64 {
if x == 0.0 {
return x;
}
let (x, sign) = if x < 0.0 { (-x, true) } else { (x, false) };
if x > 1.0 {
return f64::NAN;
}
let mut temp = (1.0 - x * x).sqrt();
if x > 0.7 {
temp = FRAC_PI_2 - satan(temp / x);
} else {
temp = satan(x / temp);
}
if sign { -temp } else { temp }
}
/// Go `math.Acos`.
pub(crate) fn acos(x: f64) -> f64 {
FRAC_PI_2 - asin(x)
}
const PI4A: f64 = 7.85398125648498535156e-1;
const PI4B: f64 = 3.77489470793079817668e-8;
const PI4C: f64 = 2.69515142907905952645e-15;
const REDUCE_THRESHOLD: f64 = (1u64 << 29) as f64;
const SIN: [f64; 6] = [
1.58962301576546568060e-10,
-2.50507477628578072866e-8,
2.75573136213857245213e-6,
-1.98412698295895385996e-4,
8.33333333332211858878e-3,
-1.66666666666666307295e-1,
];
const COS: [f64; 6] = [
-1.13585365213876817300e-11,
2.08757008419747316778e-9,
-2.75573141792967388112e-7,
2.48015872888517045348e-5,
-1.38888888888730564116e-3,
4.16666666666665929218e-2,
];
/// The octant and the reduced argument of a non-negative finite `x`.
fn reduce(x: f64) -> (u64, f64) {
if x >= REDUCE_THRESHOLD {
return trig_reduce(x);
}
let mut j = (x * FOUR_OVER_PI) as u64;
let mut y = j as f64;
if j & 1 == 1 {
j += 1;
y += 1.0;
}
(j, ((x - y * PI4A) - y * PI4B) - y * PI4C)
}
fn sin_poly(z: f64, zz: f64) -> f64 {
z + z
* zz
* ((((((SIN[0] * zz) + SIN[1]) * zz + SIN[2]) * zz + SIN[3]) * zz + SIN[4]) * zz + SIN[5])
}
fn cos_poly(zz: f64) -> f64 {
1.0 - 0.5 * zz
+ zz * zz
* ((((((COS[0] * zz) + COS[1]) * zz + COS[2]) * zz + COS[3]) * zz + COS[4]) * zz
+ COS[5])
}
/// Go `math.Cos`.
pub(crate) fn cos(x: f64) -> f64 {
if x.is_nan() || x.is_infinite() {
return f64::NAN;
}
let (mut j, z) = reduce(x.abs());
j &= 7;
let mut sign = false;
if j > 3 {
j -= 4;
sign = !sign;
}
if j > 1 {
sign = !sign;
}
let zz = z * z;
let y = if j == 1 || j == 2 {
sin_poly(z, zz)
} else {
cos_poly(zz)
};
if sign { -y } else { y }
}
/// Go `math.Sin`.
pub(crate) fn sin(x: f64) -> f64 {
if x == 0.0 || x.is_nan() {
return x;
}
if x.is_infinite() {
return f64::NAN;
}
let (x, mut sign) = if x < 0.0 { (-x, true) } else { (x, false) };
let (mut j, z) = reduce(x);
j &= 7;
if j > 3 {
sign = !sign;
j -= 4;
}
let zz = z * z;
let y = if j == 1 || j == 2 {
cos_poly(zz)
} else {
sin_poly(z, zz)
};
if sign { -y } else { y }
}
/// Go `math.Tan`.
pub(crate) fn tan(x: f64) -> f64 {
const P: [f64; 3] = [
-1.30936939181383777646e4,
1.15351664838587416140e6,
-1.79565251976484877988e7,
];
const Q: [f64; 5] = [
1.00000000000000000000e0,
1.36812963470692954678e4,
-1.32089234440210967447e6,
2.50083801823357915839e7,
-5.38695755929454629881e7,
];
if x == 0.0 || x.is_nan() {
return x;
}
if x.is_infinite() {
return f64::NAN;
}
let (x, sign) = if x < 0.0 { (-x, true) } else { (x, false) };
let (j, z) = reduce(x);
let zz = z * z;
let mut y = if zz > 1e-14 {
z + z
* (zz * (((P[0] * zz) + P[1]) * zz + P[2])
/ ((((zz + Q[1]) * zz + Q[2]) * zz + Q[3]) * zz + Q[4]))
} else {
z
};
if j & 2 == 2 {
y = -1.0 / y;
}
if sign { -y } else { y }
}
const M_PI4: [u64; 20] = [
0x0000000000000001,
0x45f306dc9c882a53,
0xf84eafa3ea69bb81,
0xb6c52b3278872083,
0xfca2c757bd778ac3,
0x6e48dc74849ba5c0,
0x0c925dd413a32439,
0xfc3bd63962534e7d,
0xd1046bea5d768909,
0xd338e04d68befc82,
0x7323ac7306a673e9,
0x3908bf177bf25076,
0x3ff12fffbc0b301f,
0xde5e2316b414da3e,
0xda6cfd9e4f96136e,
0x9e8c7ecd3cbfd45a,
0xea4f758fd7cbe2f6,
0x7a0e73ef14a525d4,
0xd7f6bf623f1aba10,
0xac06608df8f6d757,
];
/// Go `trigReduce`: Payne–Hanek reduction for `x >= 2^29`.
fn trig_reduce(x: f64) -> (u64, f64) {
if x < FRAC_PI_4 {
return (0, x);
}
let mut ix = x.to_bits();
let exp = ((ix >> SHIFT) & MASK) as i64 - BIAS - SHIFT as i64;
ix &= !(MASK << SHIFT);
ix |= 1 << SHIFT;
let (digit, bitshift) = (((exp + 61) / 64) as usize, ((exp + 61) % 64) as u32);
let join = |hi: u64, lo: u64| shl(hi, bitshift) | shr(lo, 64 - bitshift);
let z0 = join(M_PI4[digit], M_PI4[digit + 1]);
let z1 = join(M_PI4[digit + 1], M_PI4[digit + 2]);
let z2 = join(M_PI4[digit + 2], M_PI4[digit + 3]);
let z2hi = ((z2 as u128 * ix as u128) >> 64) as u64;
let z1prod = z1 as u128 * ix as u128;
let (z1hi, z1lo) = ((z1prod >> 64) as u64, z1prod as u64);
let z0lo = z0.wrapping_mul(ix);
let (lo, carry) = z1lo.overflowing_add(z2hi);
let mut hi = z0lo.wrapping_add(z1hi).wrapping_add(carry as u64);
let mut j = hi >> 61;
hi = (hi << 3) | (lo >> 61);
let lz = hi.leading_zeros();
let e = (BIAS as u64).wrapping_sub(lz as u64 + 1);
hi = shl(hi, lz + 1) | shr(lo, 64u32.wrapping_sub(lz + 1));
hi >>= 64 - SHIFT;
hi |= e << SHIFT;
let mut z = f64::from_bits(hi);
if j & 1 == 1 {
j += 1;
j &= 7;
z -= 1.0;
}
(j, z * FRAC_PI_4)
}
// Go shifts by 64 or more produce 0 instead of panicking.
fn shl(x: u64, n: u32) -> u64 {
x.checked_shl(n).unwrap_or(0)
}
fn shr(x: u64, n: u32) -> u64 {
x.checked_shr(n).unwrap_or(0)
}
/// Go `math.Sinh`, which is `exp(|x|) / 2` above 21 and so overflows with [`exp`].
pub(crate) fn sinh(x: f64) -> f64 {
if x.abs() > 21.0 {
(exp(x.abs()) * 0.5).copysign(x)
} else {
x.sinh()
}
}
/// Go `math.Cosh`, which is `exp(|x|) / 2` above 21 and so overflows with [`exp`].
pub(crate) fn cosh(x: f64) -> f64 {
if x.abs() > 21.0 {
exp(x.abs()) * 0.5
} else {
x.cosh()
}
}
const LN2_HI: f64 = 6.93147180369123816490e-01;
const LN2_LO: f64 = 1.90821492927058770002e-10;
const L1: f64 = 6.666666666666735130e-01;
const L2: f64 = 3.999999999940941908e-01;
const L3: f64 = 2.857142874366239149e-01;
const L4: f64 = 2.222219843214978396e-01;
const L5: f64 = 1.818357216161805012e-01;
const L6: f64 = 1.531383769920937332e-01;
const L7: f64 = 1.479819860511658591e-01;
/// Go `math.Log`, following `log_amd64.s` operation by operation. Its inline frexp does not
/// normalize subnormal input, which changes the result there.
pub(crate) fn log(x: f64) -> f64 {
let bits = x.to_bits();
if bits & !(1 << 63) == 0 {
return f64::NEG_INFINITY;
}
if (bits as i64) < 0 {
return f64::NAN;
}
if bits >= f64::INFINITY.to_bits() {
return x;
}
let mut f1 = f64::from_bits((bits & 0x000FFFFFFFFFFFFF) | 0.5f64.to_bits());
let mut k = (((bits >> 52) & 0x7FF) as i32 - 0x3FE) as f64;
// CMPSD predicate 5 is "not less than": Sqrt2/2 >= f1.
if FRAC_1_SQRT_2 >= f1 {
k -= 1.0;
f1 *= 2.0;
}
let f = f1 - 1.0;
let s = f / (2.0 + f);
let s2 = s * s;
let s4 = s2 * s2;
let t1 = s2 * (((L7 * s4 + L5) * s4 + L3) * s4 + L1);
let t2 = s4 * ((L6 * s4 + L4) * s4 + L2);
let r = t1 + t2;
let hfsq = 0.5 * f * f;
k * LN2_HI - ((hfsq - (s * (hfsq + r) + k * LN2_LO)) - f)
}
/// Go `math.Log2`.
pub(crate) fn log2(x: f64) -> f64 {
// Go returns `exp - 1` for `frac == 0.5`, which is also what the general formula gives.
let (frac, exp) = frexp(x);
log(frac) * INV_LN2 + exp as f64
}
/// Go `math.Log10`.
pub(crate) fn log10(x: f64) -> f64 {
log(x) * INV_LN10
}
src/promql/src/map/go_math/tests.rs
+158
View File
@@ -0,0 +1,158 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::collections::BTreeMap;
use super::*;
use crate::golden::close;
use crate::map::{ArithmeticOp, MathOp};
const CASES: &str = include_str!("../go_math_cases.txt");
fn hex(s: &str) -> f64 {
f64::from_bits(u64::from_str_radix(s, 16).unwrap())
}
/// Each function as the kernels evaluate it.
fn evaluate(name: &str, args: &[f64]) -> f64 {
let op = match (name, args) {
("atan2", [y, x]) => return ArithmeticOp::Atan2.calculate(*y, *x),
("mod", [x, y]) => return ArithmeticOp::Modulo.calculate(*x, *y),
("pow", [x, y]) => return ArithmeticOp::Power.calculate(*x, *y),
("sin", [_]) => MathOp::Sin,
("cos", [_]) => MathOp::Cos,
("tan", [_]) => MathOp::Tan,
("asin", [_]) => MathOp::Asin,
("acos", [_]) => MathOp::Acos,
("atan", [_]) => MathOp::Atan,
("sinh", [_]) => MathOp::Sinh,
("cosh", [_]) => MathOp::Cosh,
("tanh", [_]) => MathOp::Tanh,
("asinh", [_]) => MathOp::Asinh,
("acosh", [_]) => MathOp::Acosh,
("atanh", [_]) => MathOp::Atanh,
("log", [_]) => MathOp::Ln,
("log2", [_]) => MathOp::Log2,
("log10", [_]) => MathOp::Log10,
("exp", [_]) => MathOp::Exp,
_ => panic!("unknown case {name} {args:?}"),
};
op.calculate(args[0])
}
/// The standard library function each port replaces.
fn evaluate_std(name: &str, args: &[f64]) -> Option<f64> {
Some(match (name, args) {
("sin", [x]) => x.sin(),
("cos", [x]) => x.cos(),
("tan", [x]) => x.tan(),
("acos", [x]) => x.acos(),
("sinh", [x]) => x.sinh(),
("cosh", [x]) => x.cosh(),
("log", [x]) => x.ln(),
("log2", [x]) => x.log2(),
("log10", [x]) => x.log10(),
("exp", [x]) => x.exp(),
("atan2", [y, x]) => y.atan2(*x),
("pow", [x, y]) => x.powf(*y),
_ => return None,
})
}
fn cases() -> impl Iterator<Item = (&'static str, &'static str, Vec<f64>, f64)> {
CASES
.lines()
.filter(|line| !line.starts_with('#'))
.map(|line| {
let words = line.split(' ').collect::<Vec<_>>();
let args = words[1..words.len() - 1]
.iter()
.map(|word| hex(word))
.collect();
(line, words[0], args, hex(words[words.len() - 1]))
})
}
#[test]
fn kernels_match_go_math_on_amd64() {
let mut counts = BTreeMap::<&str, usize>::new();
for (line, name, args, expected) in cases() {
let actual = evaluate(name, &args);
assert!(
close(actual, expected),
"{line}: got {:016x} ({actual:e})",
actual.to_bits()
);
*counts.entry(name).or_default() += 1;
}
// Lower bounds keep coverage from silently shrinking when the cases are regenerated.
let minimums = [
("sin", 44),
("cos", 44),
("tan", 44),
("asin", 32),
("acos", 33),
("atan", 36),
("sinh", 33),
("cosh", 31),
("tanh", 31),
("asinh", 37),
("acosh", 41),
("atanh", 34),
("log", 40),
("log2", 33),
("log10", 33),
("exp", 40),
("atan2", 64),
("mod", 34),
("pow", 72),
];
assert_eq!(counts.len(), minimums.len());
for (name, minimum) in minimums {
assert!(
counts[name] >= minimum,
"{name}: only {} cases",
counts[name]
);
}
}
/// Every port needs a case the standard library fails, or it should be the standard library.
#[test]
fn every_port_has_a_case_std_fails() {
let mut failing = BTreeMap::<&str, usize>::new();
for (_, name, args, expected) in cases() {
if let Some(actual) = evaluate_std(name, &args) {
let entry = failing.entry(name).or_default();
if !close(actual, expected) {
*entry += 1;
}
}
}
let unneeded = failing
.iter()
.filter(|(_, count)| **count == 0)
.map(|(name, _)| *name)
.collect::<Vec<_>>();
assert_eq!(failing.len(), 12);
assert!(unneeded.is_empty(), "std passes every case of {unneeded:?}");
}
/// Within the tolerance, but `floor(log10(1e15))` is 14 in Prometheus, so `log10` must not
/// become `f64::log10`, which returns exactly 15.
#[test]
fn log10_keeps_go_rounding_below_powers_of_ten() {
assert_eq!(log10(1e15).floor(), 14.0);
}
src/promql/src/map/go_math_cases.txt
+758
View File
@@ -0,0 +1,758 @@
# Generated by promql-compat cmd/narthex-golden-gomath (go1.26.5, GOARCH=amd64, AVX+FMA CPU, seed 1).
# Each line is `<function> <arg1> [<arg2>] <result>` as hex bit patterns of Go math results.
sin 7ff8000000000001 7ff8000000000001
sin 7ff0000000000000 7ff8000000000001
sin fff0000000000000 7ff8000000000001
sin 0000000000000000 0000000000000000
sin 8000000000000000 8000000000000000
sin 0000000000000001 0000000000000001
sin 8000000000000001 8000000000000001
sin 000fffffffffffff 000fffffffffffff
sin 8010000000000000 8010000000000000
sin 7fefffffffffffff 3f7452fc98b34eb0
sin ffefffffffffffff bf7452fc98b34eb0
sin 3ff0000000000000 3feaed548f090cee
sin bff0000000000000 bfeaed548f090cee
sin 3fe0000000000000 3fdeaee8744b05f0
sin c000000000000000 bfed18f6ead1b445
sin 3fe921fb54442d17 3fe6a09e667f3bcc
sin 3fe921fb54442d18 3fe6a09e667f3bcc
sin 3fe921fb54442d19 3fe6a09e667f3bcd
sin 41bfffffffffffff 3fd4e67c4aa3dd2f
sin 41c0000000000000 3fd4e67c0e2622df
sin 41c0000000000001 3fd4e67b952aadfe
sin 3ff921fb54442d17 3ff0000000000000
sin 3ff921fb54442d18 3ff0000000000000
sin 3ff921fb54442d19 3ff0000000000000
sin 400921fb54442d17 3cc469898cc51700
sin 400921fb54442d18 3ca1a62633145c00
sin 400921fb54442d19 bcb72cece675d200
sin 3e7ad7f29abcaf47 3e7ad7f29abcaf3a
sin 3e7ad7f29abcaf48 3e7ad7f29abcaf3b
sin 3e7ad7f29abcaf49 3e7ad7f29abcaf3c
sin 4202a05f20000000 bfdf334c7896a4e4
sin 4480f0cf064dd592 bfeb453ab76bf398
sin 7e37e43c8800759c bfea2c16b010e386
sin 41132cbd0fba43a8 3dbaa8ecfb278780
sin c30c6bf526340000 bfeb76f88136ceba
sin 3e45798ee2308c3a 3e45798ee2308c3a
sin 426e6762c4e1f5a4 3e238fd3cd5ef108
sin 424e15ae48b2b67c 3fefffffffffffff
sin c27e8f300e08ad06 bfe30678f36bcc55
sin c237a6e57c5284e2 3fe745e704a91c92
sin 41ec904d43d89136 3fc70ae0ffe8a1c6
sin c043c29c909c1016 bfef0053a506e970
sin 406b38c9777f52bf bfeb020e9e147e8b
sin c20ea7242ab02d20 bfdbe0d09ff65d3d
cos 7ff8000000000001 7ff8000000000001
cos 7ff0000000000000 7ff8000000000001
cos fff0000000000000 7ff8000000000001
cos 0000000000000000 3ff0000000000000
cos 8000000000000000 3ff0000000000000
cos 0000000000000001 3ff0000000000000
cos 8000000000000001 3ff0000000000000
cos 000fffffffffffff 3ff0000000000000
cos 8010000000000000 3ff0000000000000
cos 7fefffffffffffff bfefffe62ecfab75
cos ffefffffffffffff bfefffe62ecfab75
cos 3ff0000000000000 3fe14a280fb5068c
cos bff0000000000000 3fe14a280fb5068c
cos 3fe0000000000000 3fec1528065b7d50
cos c000000000000000 bfdaa22657537205
cos 3fe921fb54442d17 3fe6a09e667f3bce
cos 3fe921fb54442d18 3fe6a09e667f3bcd
cos 3fe921fb54442d19 3fe6a09e667f3bcc
cos 41bfffffffffffff bfee3edd238bb17a
cos 41c0000000000000 bfee3edd2dfeef90
cos 41c0000000000001 bfee3edd42e56b61
cos 3ff921fb54442d17 3cb469898cc51700
cos 3ff921fb54442d18 3c91a62633145c00
cos 3ff921fb54442d19 bca72cece675d200
cos 400921fb54442d17 bff0000000000000
cos 400921fb54442d18 bff0000000000000
cos 400921fb54442d19 bff0000000000000
cos 3e7ad7f29abcaf47 3fefffffffffffd3
cos 3e7ad7f29abcaf48 3fefffffffffffd3
cos 3e7ad7f29abcaf49 3fefffffffffffd3
cos 4202a05f20000000 3febf098901c931a
cos 4480f0cf064dd592 3fe0be2cef01c8f3
cos 7e37e43c8800759c bfe2699022adc4c1
cos 41132cbd0fba43a8 3ff0000000000000
cos c30c6bf526340000 bfe06c154609d33e
cos 3e45798ee2308c3a 3ff0000000000000
cos 426e6762c4e1f5a4 bff0000000000000
cos 424e15ae48b2b67c 3e4916ae56ee5415
cos 407738e0f02dd61d 3fe5311f6d02caef
cos c13442eab43a81fc 3feffe4f7851cc5d
cos c0683ca4acf01acb 3fe44c82521caab9
cos bfd792e6ec8216cc 3fedda87725a5070
cos c1e2334fb929b159 bfe59d9dd16fa979
cos bfc0418aabaaf62a 3fefbe0630f2ce5b
tan 7ff8000000000001 7ff8000000000001
tan 7ff0000000000000 7ff8000000000001
tan fff0000000000000 7ff8000000000001
tan 0000000000000000 0000000000000000
tan 8000000000000000 8000000000000000
tan 0000000000000001 0000000000000001
tan 8000000000000001 8000000000000001
tan 000fffffffffffff 000fffffffffffff
tan 8010000000000000 8010000000000000
tan 7fefffffffffffff bf74530cfe72949d
tan ffefffffffffffff 3f74530cfe72949d
tan 3ff0000000000000 3ff8eb245cbee3a5
tan bff0000000000000 bff8eb245cbee3a5
tan 3fe0000000000000 3fe17b4f5bf3474a
tan c000000000000000 40017af62e0950f8
tan 3fe921fb54442d17 3feffffffffffffd
tan 3fe921fb54442d18 3ff0000000000000
tan 3fe921fb54442d19 3ff0000000000000
tan 41bfffffffffffff bfd61cd928a38268
tan 41c0000000000000 bfd61cd8e0ffa035
tan 41c0000000000001 bfd61cd851b7dc63
tan 3ff921fb54442d17 4329153d9443ed0d
tan 3ff921fb54442d18 434d02967c31cdc0
tan 3ff921fb54442d19 c33617a154947677
tan 400921fb54442d17 bcc469898cc51700
tan 400921fb54442d18 bca1a62633145c00
tan 400921fb54442d19 3cb72cece675d200
tan 3e7ad7f29abcaf47 3e7ad7f29abcaf47
tan 3e7ad7f29abcaf48 3e7ad7f29abcaf48
tan 3e7ad7f29abcaf49 3e7ad7f29abcaf62
tan 4202a05f20000000 bfe1de000f443f51
tan 4480f0cf064dd592 bffa0f79c1b6b259
tan 7e37e43c8800759c 3ff6be411f37ac76
tan 41132cbd0fba43a8 3dbaa8ecfb278780
tan c30c6bf526340000 3ffac23600a95be5
tan 3e45798ee2308c3a 3e45798ee2308c3a
tan 426e6762c4e1f5a4 be238fd3cd5ef108
tan 424e15ae48b2b67c 4194685d861d2ae7
tan c063120eaccf52cc 40140947240353ee
tan bffbeb9eea0105a6 4016b9c19d3aa369
tan 3fe4581c6fc6e67e 3fe79d6e14271946
tan c27fcb137176ce42 3feb30b2ef58568e
tan c209027869d95ed3 bff657be9c221051
tan c21f1120be3fe33b 403c87f1f6b27777
asin 7ff8000000000001 7ff8000000000001
asin 7ff0000000000000 7ff8000000000001
asin fff0000000000000 7ff8000000000001
asin 0000000000000000 0000000000000000
asin 8000000000000000 8000000000000000
asin 0000000000000001 0000000000000001
asin 8000000000000001 8000000000000001
asin 000fffffffffffff 000fffffffffffff
asin 8010000000000000 8010000000000000
asin 7fefffffffffffff 7ff8000000000001
asin ffefffffffffffff 7ff8000000000001
asin 3ff0000000000000 3ff921fb54442d18
asin bff0000000000000 bff921fb54442d18
asin 3fe0000000000000 3fe0c152382d7366
asin c000000000000000 7ff8000000000001
asin 3fe6666666666665 3fe8d00e692afd93
asin 3fe6666666666666 3fe8d00e692afd95
asin 3fe6666666666667 3fe8d00e692afd96
asin 3fefffffffffffff 3ff921fb50442d18
asin 3ff0000000000000 3ff921fb54442d18
asin 3ff0000000000001 7ff8000000000001
asin 3fe68f5c28f5c28f 3fe9099c43e0fdac
asin 3fe6a7ef9db22d0e 3fe92c5621d2671e
asin 3e45798ee2308c3a 3e45798ee2308c3b
asin 3fefffffca501acb 3ff920266447a7d1
asin bfd3333333333333 bfd380159e14f6ff
asin 3fc05410006eeb4b 3fc05f7bba74ffc4
asin bf1e539083b9cce4 bf1e539084dc54cf
asin 3e15e73d3f65e1a4 3e15e73d3f65e1a4
asin 3f2ad365bb3e31f4 3f2ad365be628bb7
asin bf22f399bda4ada9 bf22f399bec04a3a
asin bf467e227c75ece0 bf467e229a184d4c
acos 7ff8000000000001 7ff8000000000001
acos 7ff0000000000000 7ff8000000000001
acos fff0000000000000 7ff8000000000001
acos 0000000000000000 3ff921fb54442d18
acos 8000000000000000 3ff921fb54442d18
acos 0000000000000001 3ff921fb54442d18
acos 8000000000000001 3ff921fb54442d18
acos 000fffffffffffff 3ff921fb54442d18
acos 8010000000000000 3ff921fb54442d18
acos 7fefffffffffffff 7ff8000000000001
acos ffefffffffffffff 7ff8000000000001
acos 3ff0000000000000 0000000000000000
acos bff0000000000000 400921fb54442d18
acos 3fe0000000000000 3ff0c152382d7365
acos c000000000000000 7ff8000000000001
acos 3fe6666666666665 3fe973e83f5d5c9d
acos 3fe6666666666666 3fe973e83f5d5c9b
acos 3fe6666666666667 3fe973e83f5d5c9a
acos 3fefffffffffffff 3e50000000000000
acos 3ff0000000000000 0000000000000000
acos 3ff0000000000001 7ff8000000000001
acos bff0000000000001 7ff8000000000001
acos bff0000000000000 400921fb54442d18
acos bfefffffffffffff 400921fb52442d18
acos 3e45798ee2308c3a 3ff921fb5194fb3c
acos bfefffffca501acb 40092110dc45ea74
acos 3feffffffc054cf1 3f1feac532850000
acos be1352c1f4a03468 3ff921fb54917820
acos bfe24e1c05975004 40017030f3a99a5a
acos beb7f8016b436d6a 3ff921fcd3c443cc
acos 3e9dba14cfc3749e 3ff921fadd5bd9d9
acos 3fdba2dd2e98670e 3ff1fd195bafe240
acos 3f71859a7695ff58 3ff91075b64cff73
atan 7ff8000000000001 7ff8000000000001
atan 7ff0000000000000 3ff921fb54442d18
atan fff0000000000000 bff921fb54442d18
atan 0000000000000000 0000000000000000
atan 8000000000000000 8000000000000000
atan 0000000000000001 0000000000000001
atan 8000000000000001 8000000000000001
atan 000fffffffffffff 000fffffffffffff
atan 8010000000000000 8010000000000000
atan 7fefffffffffffff 3ff921fb54442d18
atan ffefffffffffffff bff921fb54442d18
atan 3ff0000000000000 3fe921fb54442d18
atan bff0000000000000 bfe921fb54442d18
atan 3fe0000000000000 3fddac670561bb4f
atan c000000000000000 bff1b6e192ebbe44
atan 3fe51eb851eb851e 3fe2aafdde4d0c9e
atan 3fe51eb851eb851f 3fe2aafdde4d0c9f
atan 3fe51eb851eb8520 3fe2aafdde4d0c9f
atan 4003504f333f9de5 3ff2d97c7f3321d2
atan 4003504f333f9de6 3ff2d97c7f3321d2
atan 4003504f333f9de7 3ff2d97c7f3321d2
atan 400350e560418937 3ff2d9a87a576c56
atan 40035c28f5c28f5c 3ff2dcf3400d72c6
atan 400399999999999a 3ff2eeaf2ac0364c
atan 4008000000000000 3ff3fc176b7a8560
atan 4014000000000000 3ff5f97315254856
atan 401c000000000000 3ff6dcc57bb565fc
atan 01a56e1fc2f8f359 01a56e1fc2f8f359
atan 7e37e43c8800759c 3ff921fb54442d18
atan c008000000000000 bff3fc176b7a8560
atan 3efab63dd043666e 3efab63dd02a953c
atan bf27e538b7b61815 bf27e538b3451767
atan 419e76fa16b54ed4 3ff921fb522a5fd4
atan 3e385cba3985f6c2 3e385cba3985f6c2
atan 3f42d10b9a6e361c 3f42d10b77bb2f5a
atan 3f47ccac013ad9bc 3f47ccabbb04f5ef
sinh 7ff8000000000001 7ff8000000000001
sinh 7ff0000000000000 7ff0000000000000
sinh fff0000000000000 fff0000000000000
sinh 0000000000000000 0000000000000000
sinh 8000000000000000 8000000000000000
sinh 0000000000000001 0000000000000001
sinh 8000000000000001 8000000000000001
sinh 000fffffffffffff 000fffffffffffff
sinh 8010000000000000 8010000000000000
sinh 7fefffffffffffff 7ff0000000000000
sinh ffefffffffffffff fff0000000000000
sinh 3ff0000000000000 3ff2cd9fc44eb982
sinh bff0000000000000 bff2cd9fc44eb982
sinh 3fe0000000000000 3fe0acd00fe63b97
sinh c000000000000000 c00d03cf63b6e1a0
sinh 3fdfffffffffffff 3fe0acd00fe63b96
sinh 3fe0000000000000 3fe0acd00fe63b97
sinh 3fe0000000000001 3fe0acd00fe63b98
sinh 4034ffffffffffff 41c3a6e1fd9eecea
sinh 4035000000000000 41c3a6e1fd9eecfd
sinh 4035000000000001 41c3a6e1fd9eed11
sinh 3fe199999999999a 3fe28037cac6a8a7
sinh 3fe28f5c28f5c28f 3fe39e450b3fd0e8
sinh 4024000000000000 40c5829dced69992
sinh 4086300000000000 7ff0000000000000
sinh 4086380000000000 7ff0000000000000
sinh bfd0000000000000 bfd02accd9d08102
sinh be38cf2f2fb9c8d2 be38cf2f2fb9c8d2
sinh 403fc7091e85dcb4 42bcbe9b6a57ffae
sinh c062c5b669b56823 cd69501b80c21d04
sinh 407e0cb613376c9a 6b38ede7bdce39b7
sinh 3e44aef20e5b0856 3e44aef20e5b0856
sinh 3e5aa84ef8ec5e7f 3e5aa84ef8ec5e7f
cosh 7ff8000000000001 7ff8000000000001
cosh 7ff0000000000000 7ff0000000000000
cosh fff0000000000000 7ff0000000000000
cosh 0000000000000000 3ff0000000000000
cosh 8000000000000000 3ff0000000000000
cosh 0000000000000001 3ff0000000000000
cosh 8000000000000001 3ff0000000000000
cosh 000fffffffffffff 3ff0000000000000
cosh 8010000000000000 3ff0000000000000
cosh 7fefffffffffffff 7ff0000000000000
cosh ffefffffffffffff 7ff0000000000000
cosh 3ff0000000000000 3ff8b07551d9f550
cosh bff0000000000000 3ff8b07551d9f550
cosh 3fe0000000000000 3ff20ac1862ae8d0
cosh c000000000000000 400e18fa0df2d9bc
cosh 4034ffffffffffff 41c3a6e1fd9eecea
cosh 4035000000000000 41c3a6e1fd9eecfd
cosh 4035000000000001 41c3a6e1fd9eed11
cosh 4024000000000000 40c5829dd053712e
cosh 4086340000000000 7ff0000000000000
cosh 4086380000000000 7ff0000000000000
cosh c008000000000000 402422a497d6185e
cosh 3e112e0be826d695 3ff0000000000000
cosh 40862c0000000000 7ff0000000000000
cosh c086300000000000 7ff0000000000000
cosh c06fbeaf943b88c6 56c4e5e38d86bb30
cosh 3e39028660d37252 3ff0000000000000
cosh be1b4a2d189d5f04 3ff0000000000000
cosh 3eb034c893b713de 3ff0000000000835
cosh 3e3cbf84cecc924f 3ff0000000000000
cosh 3e8c6cd42e9d466b 3ff0000000000065
tanh 7ff8000000000001 7ff8000000000001
tanh 7ff0000000000000 3ff0000000000000
tanh fff0000000000000 bff0000000000000
tanh 0000000000000000 0000000000000000
tanh 8000000000000000 8000000000000000
tanh 0000000000000001 0000000000000001
tanh 8000000000000001 8000000000000001
tanh 000fffffffffffff 000fffffffffffff
tanh 8010000000000000 8010000000000000
tanh 7fefffffffffffff 3ff0000000000000
tanh ffefffffffffffff bff0000000000000
tanh 3ff0000000000000 3fe85efab514f394
tanh bff0000000000000 bfe85efab514f394
tanh 3fe0000000000000 3fdd9353d7568af3
tanh c000000000000000 bfeed9505e1bc3d4
tanh 3fe3ffffffffffff 3fe1bf47eabb8f94
tanh 3fe4000000000000 3fe1bf47eabb8f96
tanh 3fe4000000000001 3fe1bf47eabb8f96
tanh 404601e678fc457a 3ff0000000000000
tanh 404601e678fc457b 3ff0000000000000
tanh 404601e678fc457c 3ff0000000000000
tanh 4024000000000000 3feffffffdc96f35
tanh c028000000000000 bfeffffffff59f7c
tanh 3ddb7cdfd9d7bdbb 3ddb7cdfd9d7bdbb
tanh bfd3333333333333 bfd2a4dda7d914fa
tanh 3eb576b6a60a0ab6 3eb576b6a609fdd6
tanh 3eab40d12af7b28e 3eab40d12af7abf7
tanh 3ec92cb567ea6d54 3ec92cb567ea1a3b
tanh 3f67adcf71ad7a97 3f67adcb1f4c1ed8
tanh 3f6b20b4999f0b22 3f6b20ae19fc1780
tanh 3ec0adbe9d1e93c4 3ec0adbe9d1e7b9a
asinh 7ff8000000000001 7ff8000000000001
asinh 7ff0000000000000 7ff0000000000000
asinh fff0000000000000 fff0000000000000
asinh 0000000000000000 0000000000000000
asinh 8000000000000000 8000000000000000
asinh 0000000000000001 0000000000000001
asinh 8000000000000001 8000000000000001
asinh 000fffffffffffff 000fffffffffffff
asinh 8010000000000000 8010000000000000
asinh 7fefffffffffffff 408633ce8fb9f87d
asinh ffefffffffffffff c08633ce8fb9f87d
asinh 3ff0000000000000 3fec34366179d427
asinh bff0000000000000 bfec34366179d427
asinh 3fe0000000000000 3fdecc2caec5160a
asinh c000000000000000 bff719218313d087
asinh 3e2fffffffffffff 3e2fffffffffffff
asinh 3e30000000000000 3e30000000000000
asinh 3e30000000000001 3e30000000000001
asinh 3fffffffffffffff 3ff719218313d087
asinh 4000000000000000 3ff719218313d087
asinh 4000000000000001 3ff719218313d088
asinh 41afffffffffffff 403419ecb712c481
asinh 41b0000000000000 403419ecb712c481
asinh 41b0000000000001 403419ecb712c480
asinh 3e60000000000000 3e5fffffffffffff
asinh 3e601b2b29a4692b 3e601b2b29a4692a
asinh 400199999999999a 3ff8797d488657e0
asinh 40311882ca4b952e 400c433cf6065ed6
asinh 40609d0b2f8e7820 401654cd9cc84077
asinh 40b1862a91f03ce4 4022342c5d5dba6f
asinh 7fe6c8e5ca239029 40863116f85cb427
asinh 40e919aa0bd75796 402714cf4ab33f27
asinh c012528e8487618a c001d0428452da3b
asinh c1a58530db4002e1 c033b45bf13b454f
asinh 404d65dfd7f5ee36 401311b4dd6c7a28
asinh bf803833bd049a44 bf803828a07a5f37
asinh bf52b967f4d411a0 bf52b967b0724c32
acosh 7ff8000000000001 7ff8000000000001
acosh 7ff0000000000000 7ff0000000000000
acosh fff0000000000000 7ff8000000000001
acosh 0000000000000000 7ff8000000000001
acosh 8000000000000000 7ff8000000000001
acosh 0000000000000001 7ff8000000000001
acosh 8000000000000001 7ff8000000000001
acosh 000fffffffffffff 7ff8000000000001
acosh 8010000000000000 7ff8000000000001
acosh 7fefffffffffffff 408633ce8fb9f87d
acosh ffefffffffffffff 7ff8000000000001
acosh 3ff0000000000000 0000000000000000
acosh bff0000000000000 7ff8000000000001
acosh 3fe0000000000000 7ff8000000000001
acosh c000000000000000 7ff8000000000001
acosh 3fefffffffffffff 7ff8000000000001
acosh 3ff0000000000000 0000000000000000
acosh 3ff0000000000001 3e56a09e667f3bcd
acosh 3fffffffffffffff 3ff5124271980434
acosh 4000000000000000 3ff5124271980434
acosh 4000000000000001 3ff5124271980436
acosh 41afffffffffffff 403419ecb712c481
acosh 41b0000000000000 403419ecb712c480
acosh 41b0000000000001 403419ecb712c480
acosh 400199999999999a 3ff6ce81ff1a70f3
acosh 4003333333333333 3ff85a6fe512c39d
acosh 4008000000000000 3ffc34366179d426
acosh 4012000000000000 40017a268632d3b9
acosh 401c000000000000 4005124271980435
acosh 4024000000000000 4007f21ed1ce05d4
acosh 4034000000000000 400d818b3e5d4c5f
acosh 4041800000000000 4010fe400585b88f
acosh 4049000000000000 40126b9783d0a89d
acosh 4054000000000000 40144cf01fbfd077
acosh 7e37e43c8800759c 40859bbfd8b83e43
acosh c005597208859522 7ff8000000000001
acosh c11046c4bef318dd 7ff8000000000001
acosh 4144b5913e9a2daa 402f03b398c93fc0
acosh 40ccf87431ed4e21 40249872c6942263
acosh c0819644082a5278 7ff8000000000001
acosh c0807567fb142384 7ff8000000000001
atanh 7ff8000000000001 7ff8000000000001
atanh 7ff0000000000000 7ff8000000000001
atanh fff0000000000000 7ff8000000000001
atanh 0000000000000000 0000000000000000
atanh 8000000000000000 8000000000000000
atanh 0000000000000001 0000000000000001
atanh 8000000000000001 8000000000000001
atanh 000fffffffffffff 000fffffffffffff
atanh 8010000000000000 8010000000000000
atanh 7fefffffffffffff 7ff8000000000001
atanh ffefffffffffffff 7ff8000000000001
atanh 3ff0000000000000 7ff0000000000000
atanh bff0000000000000 fff0000000000000
atanh 3fe0000000000000 3fe193ea7aad030a
atanh c000000000000000 7ff8000000000001
atanh 3e2fffffffffffff 3e2fffffffffffff
atanh 3e30000000000000 3e30000000000000
atanh 3e30000000000001 3e30000000000001
atanh 3fdfffffffffffff 3fe193ea7aad030a
atanh 3fe0000000000000 3fe193ea7aad030a
atanh 3fe0000000000001 3fe193ea7aad030c
atanh 3fefffffffffffff 4032b708872320e2
atanh 3ff0000000000000 7ff0000000000000
atanh 3ff0000000000001 7ff8000000000001
atanh 3e60000000000000 3e60000000000001
atanh 3e601b2b29a4692b 3e601b2b29a4692d
atanh 3fe199999999999a 3fe3c9c79bc850a0
atanh bfe8000000000000 bfef2272ae325a57
atanh bfae0c1dbe9120c4 bfae14f717dffc09
atanh be1475606e08af5d be1475606e08af5d
atanh 3f83dba51ea57cb3 3f83dbcde84afe09
atanh 3deccd3fdb63b60e 3deccd3fdb63b60e
atanh befb3a1ea9912a98 befb3a1ea9ab727f
atanh bf3bdc5064b62c20 bf3bdc5080dec2eb
log 7ff8000000000001 7ff8000000000001
log 7ff0000000000000 7ff0000000000000
log fff0000000000000 7ff8000000000001
log 0000000000000000 fff0000000000000
log 8000000000000000 fff0000000000000
log 0000000000000001 c08628b76e3a7b61
log 8000000000000001 7ff8000000000001
log 000fffffffffffff c086232bdd7abcd2
log 8010000000000000 7ff8000000000001
log 7fefffffffffffff 40862e42fefa39ef
log ffefffffffffffff 7ff8000000000001
log 3ff0000000000000 0000000000000000
log bff0000000000000 7ff8000000000001
log 3fe0000000000000 bfe62e42fefa39ef
log c000000000000000 7ff8000000000001
log 3fe6a09e667f3bcc bfd62e42fefa39f1
log 3fe6a09e667f3bcd bfd62e42fefa39ee
log 3fe6a09e667f3bce bfd62e42fefa39eb
log 3ff6a09e667f3bcc 3fd62e42fefa39ee
log 3ff6a09e667f3bcd 3fd62e42fefa39f0
log 3ff6a09e667f3bce 3fd62e42fefa39f3
log 3fefffffffffffff bca0000000000000
log 3ff0000000000000 0000000000000000
log 3ff0000000000001 3cafffffffffffff
log 3fdfffffffffffff bfe62e42fefa39f0
log 3fe0000000000000 bfe62e42fefa39ef
log 3fe0000000000001 bfe62e42fefa39ed
log 0166a09e667f3bcd c085abf335603bdb
log 3f46a09e667f3bcd c01d1cb7eea86c0a
log 4086a09e667f3bcd 401a56ef8ec924cc
log 4626a09e667f3bcd 40513df614348309
log 7e66a09e667f3bcd 4085a667a4a07d4c
log 0000000000004000 c08628b76e3a7b41
log 7e37e43c8800759c 4085963447f87fb5
log 616c4cb7d66b2fd2 40773676cc3f5fd8
log 2add5877c2929b68 c06d35ab928b93ee
log 39db456c5d80a243 c050d94aa8801dc5
log 51554045f66e7a04 40681f51cb3bf5f1
log 5eec3feb39f815ac 40757ad2522acaa3
log 595226ac2f095ecc 407198b3cc1833f7
log2 7ff8000000000001 7ff8000000000001
log2 7ff0000000000000 7ff0000000000000
log2 fff0000000000000 7ff8000000000001
log2 0000000000000000 fff0000000000000
log2 8000000000000000 fff0000000000000
log2 0000000000000001 c090c80000000000
log2 8000000000000001 7ff8000000000001
log2 000fffffffffffff c08ff00000000000
log2 8010000000000000 7ff8000000000001
log2 7fefffffffffffff 4090000000000000
log2 ffefffffffffffff 7ff8000000000001
log2 3ff0000000000000 0000000000000000
log2 bff0000000000000 7ff8000000000001
log2 3fe0000000000000 bff0000000000000
log2 c000000000000000 7ff8000000000001
log2 3fe6a09e667f3bcc bfe0000000000001
log2 3fe6a09e667f3bcd bfdffffffffffffe
log2 3fe6a09e667f3bce bfdffffffffffffa
log2 3fefffffffffffff bca71547652b82fe
log2 3ff0000000000000 0000000000000000
log2 3ff0000000000001 3cb8000000000000
log2 401fffffffffffff 4008000000000000
log2 4020000000000000 4008000000000000
log2 4020000000000001 4008000000000001
log2 0000000000004000 c090900000000000
log2 000012688b70e62b c0901730dabca5f6
log2 3ff0000000061046 3de17eeb00000000
log2 d823a26aa6a33ed4 7ff8000000000001
log2 2686f97ffe8dda36 c07967a60470ab68
log2 906658fcefca0e90 7ff8000000000001
log2 ae74b8d6c26d0ef7 7ff8000000000001
log2 73c1cc5f64c78f03 4089e93ab759727d
log2 5a29cb27fd2c2c9c 407a3b05e6032815
log10 7ff8000000000001 7ff8000000000001
log10 7ff0000000000000 7ff0000000000000
log10 fff0000000000000 7ff8000000000001
log10 0000000000000000 fff0000000000000
log10 8000000000000000 fff0000000000000
log10 0000000000000001 c0733f424bcb5221
log10 8000000000000001 7ff8000000000001
log10 000fffffffffffff c0733a7146f72a42
log10 8010000000000000 7ff8000000000001
log10 7fefffffffffffff 40734413509f79ff
log10 ffefffffffffffff 7ff8000000000001
log10 3ff0000000000000 0000000000000000
log10 bff0000000000000 7ff8000000000001
log10 3fe0000000000000 bfd34413509f79ff
log10 c000000000000000 7ff8000000000001
log10 3fe6a09e667f3bcc bfc34413509f7a00
log10 3fe6a09e667f3bcd bfc34413509f79fe
log10 3fe6a09e667f3bce bfc34413509f79fb
log10 3fefffffffffffff bc8bcb7b1526e50e
log10 3ff0000000000000 0000000000000000
log10 3ff0000000000001 3c9bcb7b1526e50d
log10 4023ffffffffffff 3fefffffffffffff
log10 4024000000000000 3ff0000000000000
log10 4024000000000001 3ff0000000000000
log10 408f400000000000 4008000000000000
log10 000012688b70e62b c0733f3a51bf529c
log10 430c6bf526340000 402dffffffffffff
log10 26ebbf2fed09c31e c05e1e54f33fbafb
log10 77dfd697e91f5264 4070d6b6de196d40
log10 f4c1a373dda62828 7ff8000000000001
log10 8c08784870b934af 7ff8000000000001
log10 55bfdd0f35666977 405a43afeaf2cdfb
log10 9140deece79b4d03 7ff8000000000001
exp 7ff8000000000001 7ff8000000000001
exp 7ff0000000000000 7ff0000000000000
exp fff0000000000000 0000000000000000
exp 0000000000000000 3ff0000000000000
exp 8000000000000000 3ff0000000000000
exp 0000000000000001 3ff0000000000000
exp 8000000000000001 3ff0000000000000
exp 000fffffffffffff 3ff0000000000000
exp 8010000000000000 3ff0000000000000
exp 7fefffffffffffff 7ff0000000000000
exp ffefffffffffffff 0000000000000000
exp 3ff0000000000000 4005bf0a8b145769
exp bff0000000000000 3fd78b56362cef38
exp 3fe0000000000000 3ffa61298e1e069c
exp c000000000000000 3fc152aaa3bf81cc
exp 3fdfffffffffffff 3ffa61298e1e069b
exp 3fe0000000000000 3ffa61298e1e069c
exp 3fe0000000000001 3ffa61298e1e069d
exp 40862e42fefa39ee 7ff0000000000000
exp 40862e42fefa39ef 7ff0000000000000
exp 40862e42fefa39f0 7ff0000000000000
exp c0874910d52d3052 0000000000000000
exp c0874910d52d3051 0000000000000001
exp c0874910d52d3050 0000000000000001
exp 40862c0000000000 7ff0000000000000
exp 40862b3333333333 7fe5d303de3ac8fd
exp c086240000000000 000e6cf6d08897ac
exp c08628cccccccccd 0007eabddec4fd38
exp c0874f3333333333 0000000000000000
exp c087540000000000 0000000000000000
exp fe37e43c8800759c 0000000000000000
exp 7e37e43c8800759c 7ff0000000000000
exp 3bc79ca10c924223 3ff0000000000000
exp 4044000000000000 438a220d397972eb
exp c08631b15f366464 00029adbadf5571d
exp c07fe1defdef5055 11f0a488943d9284
exp c0362305cc9e861c 3df0b7d4a11a4ca8
exp 3eb95db124645c62 3ff0000195db2662
exp 3e933b2cb8e67288 3ff000004cecb39d
exp be336c86dd2ef1ec 3feffffffd926f24
atan2 7ff8000000000001 7ff8000000000001 7ff8000000000001
atan2 7ff8000000000001 7ff0000000000000 7ff8000000000001
atan2 7ff8000000000001 fff0000000000000 7ff8000000000001
atan2 7ff8000000000001 0000000000000000 7ff8000000000001
atan2 7ff8000000000001 8000000000000000 7ff8000000000001
atan2 7ff8000000000001 4004000000000000 7ff8000000000001
atan2 7ff8000000000001 c004000000000000 7ff8000000000001
atan2 7ff0000000000000 7ff8000000000001 7ff8000000000001
atan2 7ff0000000000000 7ff0000000000000 3fe921fb54442d18
atan2 7ff0000000000000 fff0000000000000 4002d97c7f3321d2
atan2 7ff0000000000000 0000000000000000 3ff921fb54442d18
atan2 7ff0000000000000 8000000000000000 3ff921fb54442d18
atan2 7ff0000000000000 4004000000000000 3ff921fb54442d18
atan2 7ff0000000000000 c004000000000000 3ff921fb54442d18
atan2 fff0000000000000 7ff8000000000001 7ff8000000000001
atan2 fff0000000000000 7ff0000000000000 bfe921fb54442d18
atan2 fff0000000000000 fff0000000000000 c002d97c7f3321d2
atan2 fff0000000000000 0000000000000000 bff921fb54442d18
atan2 fff0000000000000 8000000000000000 bff921fb54442d18
atan2 fff0000000000000 4004000000000000 bff921fb54442d18
atan2 fff0000000000000 c004000000000000 bff921fb54442d18
atan2 0000000000000000 7ff8000000000001 7ff8000000000001
atan2 0000000000000000 7ff0000000000000 0000000000000000
atan2 0000000000000000 fff0000000000000 400921fb54442d18
atan2 0000000000000000 0000000000000000 0000000000000000
atan2 0000000000000000 8000000000000000 400921fb54442d18
atan2 0000000000000000 4004000000000000 0000000000000000
atan2 0000000000000000 c004000000000000 400921fb54442d18
atan2 8000000000000000 7ff8000000000001 7ff8000000000001
atan2 8000000000000000 7ff0000000000000 8000000000000000
atan2 8000000000000000 fff0000000000000 c00921fb54442d18
atan2 8000000000000000 0000000000000000 8000000000000000
atan2 8000000000000000 8000000000000000 c00921fb54442d18
atan2 8000000000000000 4004000000000000 8000000000000000
atan2 8000000000000000 c004000000000000 c00921fb54442d18
atan2 4004000000000000 7ff8000000000001 7ff8000000000001
atan2 4004000000000000 7ff0000000000000 0000000000000000
atan2 4004000000000000 fff0000000000000 400921fb54442d18
atan2 4004000000000000 0000000000000000 3ff921fb54442d18
atan2 4004000000000000 8000000000000000 3ff921fb54442d18
atan2 4004000000000000 4004000000000000 3fe921fb54442d18
atan2 4004000000000000 c004000000000000 4002d97c7f3321d2
atan2 c004000000000000 7ff8000000000001 7ff8000000000001
atan2 c004000000000000 7ff0000000000000 8000000000000000
atan2 c004000000000000 fff0000000000000 c00921fb54442d18
atan2 c004000000000000 0000000000000000 bff921fb54442d18
atan2 c004000000000000 8000000000000000 bff921fb54442d18
atan2 c004000000000000 4004000000000000 bfe921fb54442d18
atan2 c004000000000000 c004000000000000 c002d97c7f3321d2
atan2 0000000000000001 fe37e43c8800759c 400921fb54442d18
atan2 8000000000000001 fe37e43c8800759c 400921fb54442d18
atan2 81a56e1fc2f8f359 fe37e43c8800759c 400921fb54442d18
atan2 40aea8b61a6fd041 40e6ec94d2138db8 3fb55955a8245661
atan2 bff3d712f2f6139a c0161a29f66d448e c0075ddf868b0c86
atan2 3f565e8204f04675 bf244f648da0b843 3ffaf0df520cfef0
atan2 412d2cd852efff7e c061046d9d136d04 3ff92290a61b3c94
atan2 3f04423fc4e56d1e 401f82ce01c21660 3ed492bd1bbf3aee
atan2 3f7359ecdda1c91a 40b8175190917d74 3ea9b4426204573a
atan2 402f2b22209e6a50 4090c1c0ee80fd48 3f8dc234e90f5df1
atan2 bf63c737c1125a90 3ee5fb3bb3edefdf bff9103315d21b75
atan2 3f322f680d3df6da bf90a31e340fdd0c 4008ff01fa2ade6b
atan2 c08c039326160b54 408bf8cabaecea40 bfe928251c5ce306
atan2 c052eec25e890aac bf1c7d09cc124ab2 bff921fcd579d013
atan2 c0b6711fe66c4f08 c06fe147e68c681f bff9d7b19f172d50
mod 4016000000000000 4000000000000000 3ff8000000000000
mod c016000000000000 4000000000000000 bff8000000000000
mod 4016000000000000 c000000000000000 3ff8000000000000
mod c010000000000000 4000000000000000 8000000000000000
mod 4010000000000000 c000000000000000 0000000000000000
mod 7e37e43c8800759c 4008000000000000 0000000000000000
mod fe37e43c8800759c 01d2c05bca99d4ee 81ce478ecb1573bc
mod 4008000000000000 7ff0000000000000 4008000000000000
mod c008000000000000 fff0000000000000 c008000000000000
mod 7ff0000000000000 4008000000000000 7ff8000000000001
mod 4008000000000000 0000000000000000 7ff8000000000001
mod 8000000000000000 4008000000000000 8000000000000000
mod 3ff0000000000000 0000000000000001 0000000000000000
mod 000080dbd0164b2d 00000000000017b8 00000000000016d5
mod 800080dbd0164b2d 00000000000017b8 80000000000016d5
mod 7fefffffffffffff 0000000000000001 0000000000000000
mod 7ff8000000000001 3ff0000000000000 7ff8000000000001
mod 3ff0000000000000 7ff8000000000001 7ff8000000000001
mod 3fd3333333333333 3fb999999999999a 3fb9999999999998
mod bfd3333333333333 3fb999999999999a bfb9999999999998
mod 3ff0000000000000 3ff0000000000000 0000000000000000
mod 4000000000000000 3ff0000000000000 0000000000000000
mod c37b44dbf4422d92 403e3bd0e406e87a bffb8a85c8074f60
mod 41ee397ebdb50539 3e81b7033b170344 3e7265807c0baff8
mod be6607791d7153b6 3f5ba03dd74ec012 be6607791d7153b6
mod c2fd954a033aa585 bf4e34a2f6d02d69 bf002548e3820e60
mod 3f8f8dba152fdd38 3e719f1abc786ae8 3e682f2fdef88870
mod 4381092ebb744468 bf277561b95af638 3f08653c8af49180
mod bf9c366cd2ba18e0 c02c1c045f3f0318 bf9c366cd2ba18e0
mod c11b86fc57bf2a5e 400d9220769fcb80 bfe08b8ffee99c00
mod c300147779c31dc1 c0714b8a0429051a c06544f2783877d0
mod 438d03db0245846b bfa57a38686161d5 3f84d32a41d32fc8
mod 4147e4ed5ea7f202 bfe4ca79c70b2a37 3fda88f525448f90
mod 3f8ba31159d39518 3fca4dc77621a84c 3f8ba31159d39518
pow 0000000000000000 bff0000000000000 7ff0000000000000
pow 8000000000000000 bff0000000000000 fff0000000000000
pow 8000000000000000 c000000000000000 7ff0000000000000
pow 8000000000000000 4008000000000000 8000000000000000
pow 0000000000000000 4008000000000000 0000000000000000
pow 0000000000000000 bfe0000000000000 7ff0000000000000
pow c000000000000000 4008000000000000 c020000000000000
pow c000000000000000 4010000000000000 4030000000000000
pow c000000000000000 c008000000000000 bfc0000000000000
pow c000000000000000 3fe0000000000000 fff8000000000000
pow c020000000000000 3fd5555555555555 7ff8000000000001
pow 4000000000000000 3fe0000000000000 3ff6a09e667f3bcd
pow 4000000000000000 bfe0000000000000 3fe6a09e667f3bcc
pow 7ff8000000000001 0000000000000000 3ff0000000000000
pow 3ff0000000000000 7ff8000000000001 3ff0000000000000
pow bff0000000000000 7ff0000000000000 3ff0000000000000
pow 3fe0000000000000 7ff0000000000000 0000000000000000
pow 3fe0000000000000 fff0000000000000 7ff0000000000000
pow 4000000000000000 7ff0000000000000 7ff0000000000000
pow 4000000000000000 fff0000000000000 0000000000000000
pow fff0000000000000 4008000000000000 fff0000000000000
pow fff0000000000000 c008000000000000 8000000000000000
pow fff0000000000000 4000000000000000 7ff0000000000000
pow 7ff0000000000000 c000000000000000 0000000000000000
pow 7ff0000000000000 3fe0000000000000 7ff0000000000000
pow 4000000000000000 43e0000000000000 7ff0000000000000
pow 3fe0000000000000 43e0000000000000 0000000000000000
pow bff0000000000000 43f0000000000000 3ff0000000000000
pow 3ff000001ad7f29b 43d0000000000000 7ff0000000000000
pow 4000000000000000 408ff80000000000 7fe0000000000000
pow 4000000000000000 4090000000000000 7ff0000000000000
pow 4000000000000000 c090c80000000000 0000000000000001
pow 4000000000000000 c090cc0000000000 0000000000000000
pow 4024000000000000 4072c00000000000 7e37e43c880075a0
pow 4024000000000000 c072c00000000000 01a56e1fc2f8f355
pow 3ff8000000000000 409f400000000000 7ff0000000000000
pow 3ff8000000000000 409a900000000000 7e15a63b3b6d05e3
pow 3fefffffffffffff 43d0000000000000 11c44104df1f819a
pow 4000000000000000 c090d00000000000 0000000000000000
pow 4000000000000000 c090d40000000000 0000000000000000
pow 298d12ed0af1a281 4008000000000000 0000000000000001
pow 4008000000000000 c085380000000000 0000000000000000
pow 0000000000000001 3fe0000000000000 1e60000000000000
pow 0000000000000001 4000000000000000 0000000000000000
pow 01a56e1fc2f8f359 c000000000000000 7ff0000000000000
pow 4008000000000000 3ff8000000000000 4014c8dc2e423980
pow 4008000000000000 400599999999999a 40336b452019ac38
pow 401c000000000000 bfd0000000000000 3fe3ac58345c9b0b
pow 3ff00000b7e2b9ed 41cd34f3ae800000 7c773d083e152846
pow 0000000000000001 3f50624dd2f1a9fc 3fdf7e6f7d7c3075
pow 403c04a18a010b48 c037000000000000 3905400e74a4a20e
pow 3fa04ae232e371ad 4043800000000000 33d03753047875fe
pow 3fb83e208526ca62 c03a000000000000 45754fd6230748c9
pow 4047aaa650be3c74 c032000000000000 39ac855c70434e2d
pow 3fd5e0db4331d3c5 402fda7806f364a3 3e6433b309d51b79
pow 3f89ef9d13e70104 c043994be1d47ea8 4f60c358bd4d37e8
pow 4073fb9bc9765e94 c024fa1ab0a6ee90 3a7a830aa218d6e7
pow 3f83fec6613ccf28 4041000000000000 31be90af878fa640
pow 4092315575a9db01 c036d4c6fa78572c 3165fa8cd0d939ee
pow 402a88dc3e79cdb8 c01c000000000000 3e4dadd9d0aa2d77
pow 3f9726d47e5ae63c 4041000000000000 34516e2b068a7b59
pow 3fa66220e7eded1c c0333d2e80a9da90 455d5ffcc89c7eba
pow 3fa4807ab9b713f7 4028f11b8e4b02c0 3c513883932d86b9
pow 3f6edf631c58d557 4021f9e6b484472c 3b68c3be1657038e
pow 407faec6330b694a 4042800000000000 54b621e1f06b8d57
pow 405d0bdb64068826 403ef95eeca0f86f 4d367d208ea83211
pow c005182e1a73e74e c026000000000000 bef879d4bbfd7c35
pow c0297004356ebf48 c028000000000000 3d2f69fcf296bf44
pow bff8fe018d9396e0 c022000000000000 bf927fa0e585f2b8
pow bfd8b25190d49e78 3ff0000000000000 bfd8b25190d49e78
pow bff3d1828e7f3a0e 4008000000000000 bffe680d02bc0fd1
pow bfbc53feb70a1f92 4032000000000000 3c5c8a45b3416914
src/promql/src/map/prometheus_cases.txt
+3237 -131
View File
File diff suppressed because it is too large Load Diff
src/promql/src/map/tests.rs
+21 -15
View File
@@ -12,6 +12,12 @@
// See the License for the specific language governing permissions and
// limitations under the License.
mod arithmetic;
mod broadcast;
mod comparison;
mod math_functions;
mod prometheus;
use std::sync::Arc;
use common_query::prometheus::PROMETHEUS_STALE_NAN_BITS;
@@ -31,8 +37,6 @@ use datatypes::series::{Label, MetricName};
use super::*;
use crate::functions::IeeeSqrt;
mod prometheus;
fn schema(names: &[&str]) -> SeriesSchema {
SeriesSchema {
fields: names
@@ -181,7 +185,7 @@ fn math_preserves_missing_and_uses_ieee_float_semantics() {
),
];
for (op, expected) in cases {
let output = map_math(&schema, &input, op).unwrap();
let output = map_math(&schema, &input, op).unwrap().unwrap();
let expected = expected
.into_iter()
.map(Some)
@@ -206,7 +210,7 @@ fn short_multifield_slices_keep_absolute_grid_and_labels() {
.unwrap(),
];
let input = series(&schema, 7, fields);
let output = map_math(&schema, &input, MathOp::Floor).unwrap();
let output = map_math(&schema, &input, MathOp::Floor).unwrap().unwrap();
let TimeAxis::Grid(range) = &output.time else {
panic!("lost grid axis")
};
@@ -262,7 +266,7 @@ fn empty_and_all_missing_ranges_are_not_expanded() {
MathOp::Sqrt,
MathOp::Round { nearest: 0.0 },
] {
let output = map_math(&schema, &input, op).unwrap();
let output = map_math(&schema, &input, op).unwrap().unwrap();
let TimeAxis::Grid(range) = &output.time else {
panic!("lost grid axis")
};
@@ -285,7 +289,7 @@ fn evaluated_missingness_depends_on_validity_not_nan_payloads() {
Some(NullBuffer::from(vec![false, true, true])),
))],
);
let output = map_math(&schema, &input, MathOp::Abs).unwrap();
let output = map_math(&schema, &input, MathOp::Abs).unwrap().unwrap();
assert_floats(
output.fields.get(0).unwrap(),
&[None, Some(ordinary_nan), Some(1.0)],
@@ -299,12 +303,12 @@ fn evaluated_missingness_depends_on_validity_not_nan_payloads() {
// must not reinterpret that computed value as a missing sample or reject the evaluated input.
let negative_nan = f64::from_bits(PROMETHEUS_STALE_NAN_BITS | (1_u64 << 63));
let input = series(&schema, 1, vec![floats(&[Some(negative_nan)])]);
let absolute = map_math(&schema, &input, MathOp::Abs).unwrap();
let absolute = map_math(&schema, &input, MathOp::Abs).unwrap().unwrap();
let SampleRef::Float(computed_nan) = absolute.fields.get(0).unwrap().sample(0) else {
panic!("lost computed NaN")
};
assert_eq!(computed_nan.to_bits(), PROMETHEUS_STALE_NAN_BITS);
let rounded = map_math(&schema, &absolute, MathOp::Ceil).unwrap();
let rounded = map_math(&schema, &absolute, MathOp::Ceil).unwrap().unwrap();
assert_floats(rounded.fields.get(0).unwrap(), &[Some(f64::NAN)]);
}
@@ -347,10 +351,12 @@ fn histogram_and_mixed_fields_are_rejected_without_partial_success() {
for unsupported in [histogram, mixed, null_histogram, chunks] {
let prefix = floats(&vec![Some(-2.0); unsupported.len()]);
let input = series(&schema, 2, vec![prefix, unsupported]);
assert!(matches!(
map_math(&schema, &input, MathOp::Abs),
Err(DataFusionError::NotImplemented(_))
));
for op in [MathOp::Abs, MathOp::Clamp { min: 2.0, max: 1.0 }] {
assert!(matches!(
map_math(&schema, &input, op),
Err(DataFusionError::NotImplemented(_))
));
}
assert_floats(
input.fields.get(0).unwrap(),
&vec![Some(-2.0); input.time.len()],
@@ -402,7 +408,7 @@ fn ordinary_math(op: MathOp, input: Float64Array) -> Float64Array {
MathOp::Ceil => math::ceil(),
MathOp::Floor => math::floor(),
MathOp::Sqrt => Arc::new(IeeeSqrt::scalar_udf()),
MathOp::Round { .. } => unreachable!("round is checked against Prometheus fixtures"),
_ => unreachable!("main agrees with Prometheus only for abs, ceil, floor and sqrt"),
};
let arg_fields = args
.iter()
@@ -471,7 +477,7 @@ fn math_matches_main_udfs_across_float_boundaries() {
// Round shares its calculation with the UDF; the Prometheus fixtures check it instead.
let cases = [MathOp::Abs, MathOp::Ceil, MathOp::Floor, MathOp::Sqrt];
for op in cases {
let output = map_math(&schema, &series, op).unwrap();
let output = map_math(&schema, &series, op).unwrap().unwrap();
let expected = ordinary_math(op, input.clone()).iter().collect::<Vec<_>>();
for field in output.fields.iter() {
assert_floats(field, &expected);
@@ -492,7 +498,7 @@ fn math_drops_reserved_metadata_and_empty_labels_with_the_name() {
for label in ["__type__", "__unit__", "__other__"] {
input.metric.set(label, "x".into()).unwrap();
}
let output = map_math(&schema, &input, MathOp::Abs).unwrap();
let output = map_math(&schema, &input, MathOp::Abs).unwrap().unwrap();
let names = output
.metric
.tags()
src/promql/src/map/tests/arithmetic.rs
+200
View File
@@ -0,0 +1,200 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::*;
const OPERATIONS: [ArithmeticOp; 7] = [
ArithmeticOp::Add,
ArithmeticOp::Subtract,
ArithmeticOp::Multiply,
ArithmeticOp::Divide,
ArithmeticOp::Modulo,
ArithmeticOp::Power,
ArithmeticOp::Atan2,
];
#[test]
fn unary_negation_drops_name_and_keeps_missing_positions_and_payloads() {
let schema = schema(&["value"]);
let stale = f64::from_bits(PROMETHEUS_STALE_NAN_BITS);
let input = series(
&schema,
6,
vec![floats(&[Some(-0.0), Some(2.5), Some(-stale), None])],
);
let mut typed = input.clone();
typed.metric.set("__type__", "gauge".into()).unwrap();
let output = map_negate(&schema, &typed).unwrap();
let field = output.fields.get(0).unwrap();
assert_floats(field, &[Some(0.0), Some(-2.5), Some(stale), None]);
let SampleRef::Float(negated) = field.iter().nth(2).unwrap() else {
panic!("missing negated NaN");
};
assert_eq!(negated.to_bits(), PROMETHEUS_STALE_NAN_BITS);
assert_eq!(output.metric.name(), None);
assert_eq!(output.metric.tags(), &input.metric.tags()[1..]);
assert_eq!(output.time.timestamp(0), Some(4000));
}
#[test]
fn noncommutative_arithmetic_keeps_operand_order_across_multifield_slices() {
let schema = schema(&["first", "second"]);
let first = floats(&[Some(99.0), Some(2.0), None, Some(4.0), Some(99.0)]);
let second = floats(&[Some(99.0), None, Some(8.0), Some(-2.0), Some(99.0)]);
let input = series(
&schema,
9,
vec![
Samples::try_from_chunks(vec![first.slice(1, 1).unwrap(), first.slice(2, 2).unwrap()])
.unwrap(),
Samples::try_from_chunks(vec![
second.slice(1, 2).unwrap(),
second.slice(3, 1).unwrap(),
])
.unwrap(),
],
);
let left = map_arithmetic(
&schema,
&input,
ArithmeticOp::Divide,
ScalarOperand::Left(8.0),
)
.unwrap();
let right = map_arithmetic(
&schema,
&input,
ArithmeticOp::Divide,
ScalarOperand::Right(8.0),
)
.unwrap();
assert_floats(left.fields.get(0).unwrap(), &[Some(4.0), None, Some(2.0)]);
assert_floats(left.fields.get(1).unwrap(), &[None, Some(1.0), Some(-4.0)]);
assert_floats(right.fields.get(0).unwrap(), &[Some(0.25), None, Some(0.5)]);
assert_floats(
right.fields.get(1).unwrap(),
&[None, Some(1.0), Some(-0.25)],
);
for (actual, source) in left.fields.iter().zip(input.fields.iter()) {
let lengths = |samples: &Samples| {
samples
.segments()
.iter()
.map(Samples::len)
.collect::<Vec<_>>()
};
assert_eq!(lengths(actual), lengths(source));
}
assert_eq!(left.time.timestamp(0), Some(5500));
assert_eq!(left.time.timestamp(2), Some(6500));
}
#[test]
fn power_and_modulo_follow_go_special_cases() {
let schema = schema(&["value"]);
let input = series(
&schema,
0,
vec![floats(&[
Some(0.0),
Some(-0.0),
Some(f64::NAN),
Some(-7.0),
None,
])],
);
let field = |op, scalar| {
map_arithmetic(&schema, &input, op, scalar)
.unwrap()
.fields
.get(0)
.unwrap()
.clone()
};
// Main's DataFusion power fails on a zero base with a negative exponent.
assert_floats(
&field(ArithmeticOp::Power, ScalarOperand::Right(-1.0)),
&[
Some(f64::INFINITY),
Some(f64::NEG_INFINITY),
Some(f64::NAN),
Some(-1.0 / 7.0),
None,
],
);
assert_floats(
&field(ArithmeticOp::Power, ScalarOperand::Right(0.0)),
&[Some(1.0), Some(1.0), Some(1.0), Some(1.0), None],
);
assert_floats(
&field(ArithmeticOp::Power, ScalarOperand::Left(1.0)),
&[Some(1.0), Some(1.0), Some(1.0), Some(1.0), None],
);
assert_floats(
&field(ArithmeticOp::Modulo, ScalarOperand::Right(3.0)),
&[Some(0.0), Some(-0.0), Some(f64::NAN), Some(-1.0), None],
);
assert_floats(
&field(ArithmeticOp::Modulo, ScalarOperand::Left(5.0)),
&[
Some(f64::NAN),
Some(f64::NAN),
Some(f64::NAN),
Some(5.0),
None,
],
);
}
#[test]
fn arithmetic_rejects_unevaluated_and_histogram_inputs() {
let schema = schema(&["value"]);
let histogram =
Samples::try_from_histogram(UInt32Array::from(vec![0]), histogram_batch()).unwrap();
let input = series(&schema, 2, vec![histogram]);
assert!(matches!(
map_negate(&schema, &input),
Err(DataFusionError::NotImplemented(_))
));
for op in OPERATIONS {
assert!(matches!(
map_arithmetic(&schema, &input, op, ScalarOperand::Right(2.0)),
Err(DataFusionError::NotImplemented(_))
));
}
let raw_schema = SeriesSchema {
eval_grid: None,
..schema
};
let raw = Series::try_new(
&raw_schema,
input.metric,
TimeAxis::try_new(vec![2000].into()).unwrap(),
SeriesFields::one(floats(&[Some(1.0)])),
)
.unwrap();
assert!(matches!(
map_negate(&raw_schema, &raw),
Err(DataFusionError::Execution(_))
));
assert!(matches!(
map_arithmetic(
&raw_schema,
&raw,
ArithmeticOp::Add,
ScalarOperand::Left(1.0)
),
Err(DataFusionError::Execution(_))
));
}
src/promql/src/map/tests/broadcast.rs
+313
View File
@@ -0,0 +1,313 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::*;
fn scalar(schema: &SeriesSchema, start: usize, samples: Samples) -> Series {
let scalar_schema = SeriesSchema {
fields: vec![FieldSpec {
name: "scalar".into(),
}]
.into(),
eval_grid: schema.eval_grid.clone(),
};
Series::try_new(
&scalar_schema,
MetricName::try_new(None, vec![]).unwrap(),
TimeAxis::Grid(
GridRange::try_new(schema.eval_grid.clone().unwrap(), start, samples.len()).unwrap(),
),
SeriesFields::one(samples),
)
.unwrap()
}
fn grid_range(series: &Series) -> (usize, usize) {
let TimeAxis::Grid(range) = &series.time else {
panic!("lost grid axis")
};
(range.start(), range.len())
}
#[test]
fn time_varying_scalar_aligns_absolute_indices_across_independent_chunks() {
// The vector covers 7..15 and the scalar 5..16, each chunked at different offsets.
let schema = Arc::new(schema(&["a", "b"]));
let first = Float64Array::from(vec![
Some(1.0),
Some(2.0),
None,
Some(4.0),
Some(5.0),
Some(6.0),
Some(7.0),
Some(8.0),
]);
let second = Float64Array::from(vec![
None,
Some(20.0),
Some(30.0),
Some(40.0),
Some(50.0),
Some(60.0),
Some(70.0),
None,
]);
let input = series(
&schema,
7,
vec![
Samples::try_from_chunks(vec![
Samples::from_float(first.slice(0, 2)),
Samples::from_float(first.slice(2, 6)),
])
.unwrap(),
Samples::try_from_chunks(vec![
Samples::from_float(second.slice(0, 5)),
Samples::from_float(second.slice(5, 3)),
])
.unwrap(),
],
);
let scalar_samples = Float64Array::from(vec![
Some(-1.0),
Some(-2.0),
Some(100.0),
Some(200.0),
Some(300.0),
None,
Some(500.0),
Some(600.0),
Some(700.0),
Some(800.0),
Some(-3.0),
]);
let scalar = scalar(
&schema,
5,
Samples::try_from_chunks(vec![
Samples::from_float(scalar_samples.slice(0, 3)),
Samples::from_float(scalar_samples.slice(3, 3)),
Samples::from_float(scalar_samples.slice(6, 5)),
])
.unwrap(),
);
let output = map_arithmetic_series(
&schema,
&input,
ArithmeticOp::Subtract,
ScalarSeriesOperand::Left(&scalar),
)
.unwrap()
.unwrap();
assert_eq!(grid_range(&output), (7, 8));
assert_floats(
output.fields.get(0).unwrap(),
&[
Some(99.0),
Some(198.0),
None,
None,
Some(495.0),
Some(594.0),
Some(693.0),
Some(792.0),
],
);
assert_floats(
output.fields.get(1).unwrap(),
&[
None,
Some(180.0),
Some(270.0),
None,
Some(450.0),
Some(540.0),
Some(630.0),
None,
],
);
assert_eq!(output.metric.name(), None);
assert_eq!(output.metric.tags(), &input.metric.tags()[1..]);
let output = map_arithmetic_series(
&schema,
&input,
ArithmeticOp::Subtract,
ScalarSeriesOperand::Right(&scalar),
)
.unwrap()
.unwrap();
assert_floats(
output.fields.get(0).unwrap(),
&[
Some(-99.0),
Some(-198.0),
None,
None,
Some(-495.0),
Some(-594.0),
Some(-693.0),
Some(-792.0),
],
);
}
#[test]
fn time_varying_comparison_keeps_vector_values_and_trims() {
let schema = Arc::new(schema(&["value"]));
let input = series(
&schema,
6,
vec![floats(&[
Some(-0.0),
Some(2.0),
None,
Some(f64::NAN),
Some(4.0),
Some(1.0),
])],
);
let scalar = scalar(
&schema,
6,
floats(&[
Some(0.0),
None,
Some(5.0),
Some(f64::NAN),
Some(2.0),
Some(9.0),
]),
);
let output = map_compare_series(
&schema,
&input,
Comparison::Filter(ComparisonOp::Less),
ScalarSeriesOperand::Left(&scalar),
)
.unwrap()
.unwrap();
assert_eq!(grid_range(&output), (10, 1));
assert_floats(output.fields.get(0).unwrap(), &[Some(4.0)]);
assert_eq!(output.metric, input.metric);
let output = map_compare_series(
&schema,
&input,
Comparison::Bool(ComparisonOp::Equal),
ScalarSeriesOperand::Right(&scalar),
)
.unwrap()
.unwrap();
assert_eq!(grid_range(&output), (6, 6));
assert_floats(
output.fields.get(0).unwrap(),
&[Some(1.0), None, None, Some(0.0), Some(0.0), Some(0.0)],
);
assert_eq!(output.metric.name(), None);
}
#[test]
fn disjoint_or_missing_overlap_produces_no_series() {
let schema = Arc::new(schema(&["value"]));
let input = series(
&schema,
5,
vec![floats(&[Some(10.0), Some(20.0), Some(30.0)])],
);
for (start, samples, expected) in [
(
3,
vec![Some(1.0), Some(2.0), Some(3.0)],
Some(((5, 1), vec![Some(7.0)])),
),
(
7,
vec![Some(3.0), Some(4.0)],
Some(((7, 1), vec![Some(27.0)])),
),
(
4,
vec![None, Some(1.0), None, None],
Some(((5, 1), vec![Some(9.0)])),
),
(5, vec![None, None, None], None),
(8, vec![Some(1.0)], None),
(2, vec![], None),
(64, vec![], None),
] {
let scalar = scalar(&schema, start, floats(&samples));
let output = map_arithmetic_series(
&schema,
&input,
ArithmeticOp::Subtract,
ScalarSeriesOperand::Right(&scalar),
)
.unwrap();
match (output, expected) {
(None, None) => {}
(Some(output), Some((range, values))) => {
assert_eq!(grid_range(&output), range);
assert_floats(output.fields.get(0).unwrap(), &values);
}
(output, expected) => panic!("{start}: {output:?} != {expected:?}"),
}
}
}
#[test]
fn time_varying_scalar_preflight_rejects_unsupported_data_even_outside_overlap() {
let schema = Arc::new(schema(&["value"]));
let input = series(&schema, 5, vec![floats(&[Some(2.0)])]);
let mut named = scalar(&schema, 5, floats(&[Some(1.0)]));
named.metric.set("host", "a".into()).unwrap();
let mut multi = scalar(&schema, 5, floats(&[Some(1.0)]));
multi.fields =
SeriesFields::try_from_vec(vec![floats(&[Some(1.0)]), floats(&[Some(1.0)])]).unwrap();
let mut other_grid = scalar(&schema, 5, floats(&[Some(1.0)]));
other_grid.time = TimeAxis::Grid(
GridRange::try_new(
Arc::new(EvalGrid::try_new(GridSpec::try_new(0, 500, 64).unwrap()).unwrap()),
5,
1,
)
.unwrap(),
);
let histogram = scalar(
&schema,
20,
Samples::try_from_histogram(UInt32Array::from(vec![0]), histogram_batch()).unwrap(),
);
let mut raw = scalar(&schema, 5, floats(&[Some(1.0)]));
raw.time = TimeAxis::try_new(vec![3500].into()).unwrap();
for scalar in [named, multi, other_grid, histogram, raw] {
assert!(
map_arithmetic_series(
&schema,
&input,
ArithmeticOp::Add,
ScalarSeriesOperand::Right(&scalar)
)
.is_err()
);
assert!(
map_compare_series(
&schema,
&input,
Comparison::Bool(ComparisonOp::Equal),
ScalarSeriesOperand::Right(&scalar)
)
.is_err()
);
}
}
src/promql/src/map/tests/comparison.rs
+176
View File
@@ -0,0 +1,176 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::*;
fn grid_range(series: &Series) -> (usize, usize) {
let TimeAxis::Grid(range) = &series.time else {
panic!("lost grid axis")
};
(range.start(), range.len())
}
#[test]
fn filter_compares_fields_independently_and_trims_the_output() {
let schema = Arc::new(schema(&["a", "b"]));
let input = series(
&schema,
4,
vec![
floats(&[Some(1.0), Some(1.0), Some(5.0), None, Some(7.0), Some(1.0)]),
Samples::try_from_chunks(vec![
floats(&[None, Some(9.0)]),
floats(&[None, Some(0.0), None, Some(f64::NAN)]),
])
.unwrap(),
],
);
let output = map_compare(
&schema,
&input,
Comparison::Filter(ComparisonOp::Greater),
ScalarOperand::Right(4.0),
)
.unwrap()
.unwrap();
assert_eq!(grid_range(&output), (5, 4));
assert_floats(
output.fields.get(0).unwrap(),
&[None, Some(5.0), None, Some(7.0)],
);
assert_floats(
output.fields.get(1).unwrap(),
&[Some(9.0), None, None, None],
);
assert_eq!(output.metric, input.metric);
for scalar in [ScalarOperand::Right(100.0), ScalarOperand::Right(f64::NAN)] {
let filter = Comparison::Filter(ComparisonOp::Greater);
assert!(
map_compare(&schema, &input, filter, scalar)
.unwrap()
.is_none()
);
}
}
#[test]
fn filter_with_left_scalar_keeps_the_vector_value() {
let schema = Arc::new(schema(&["value"]));
let input = series(
&schema,
5,
vec![floats(&[Some(1.0), Some(3.0), None, Some(4.0)])],
);
let output = map_compare(
&schema,
&input,
Comparison::Filter(ComparisonOp::Less),
ScalarOperand::Left(2.0),
)
.unwrap()
.unwrap();
assert_eq!(grid_range(&output), (6, 3));
assert_floats(output.fields.get(0).unwrap(), &[Some(3.0), None, Some(4.0)]);
}
#[test]
fn bool_emits_every_present_field_and_drops_the_name() {
let schema = Arc::new(schema(&["a", "b"]));
let input = series(
&schema,
4,
vec![
floats(&[Some(-0.0), None, Some(2.0)]),
Samples::try_from_chunks(vec![floats(&[Some(0.0)]), floats(&[Some(f64::NAN), None])])
.unwrap(),
],
);
let output = map_compare(
&schema,
&input,
Comparison::Bool(ComparisonOp::Equal),
ScalarOperand::Right(0.0),
)
.unwrap()
.unwrap();
assert_eq!(grid_range(&output), (4, 3));
assert_floats(output.fields.get(0).unwrap(), &[Some(1.0), None, Some(0.0)]);
assert_floats(output.fields.get(1).unwrap(), &[Some(1.0), Some(0.0), None]);
assert_eq!(output.metric.name(), None);
assert_eq!(output.metric.tags(), &input.metric.tags()[1..]);
}
#[test]
fn scalar_operations_produce_unlabelled_short_ranges() {
let schema = schema(&["value"]);
let range = GridRange::try_new(schema.eval_grid.clone().unwrap(), 8, 3).unwrap();
let output =
scalar_arithmetic(&schema, range.clone(), ArithmeticOp::Power, [0.0, -1.0]).unwrap();
assert_floats(output.fields.get(0).unwrap(), &[Some(f64::INFINITY); 3]);
assert!(output.metric.name().is_none() && output.metric.tags().is_empty());
assert!(matches!(&output.time, TimeAxis::Grid(actual) if actual == &range));
let output = scalar_compare(
&schema,
range.clone(),
ComparisonOp::NotEqual,
[f64::NAN, f64::NAN],
)
.unwrap();
assert_floats(output.fields.get(0).unwrap(), &[Some(1.0); 3]);
assert!(output.metric.name().is_none() && output.metric.tags().is_empty());
assert!(matches!(&output.time, TimeAxis::Grid(actual) if actual == &range));
}
#[test]
fn comparisons_reject_raw_axes_and_late_histogram_chunks() {
let schema = Arc::new(schema(&["value"]));
let unsupported = Samples::try_from_chunks(vec![
floats(&[Some(1.0)]),
Samples::try_from_histogram(UInt32Array::from(vec![0]), histogram_batch()).unwrap(),
])
.unwrap();
let input = series(&schema, 3, vec![unsupported]);
// A segment of another type is rejected even when it holds no present sample.
let all_missing = Samples::try_from_chunks(vec![
floats(&[Some(1.0)]),
Samples::try_from_histogram(UInt32Array::from(vec![None]), histogram_batch()).unwrap(),
])
.unwrap();
let all_missing = series(&schema, 3, vec![all_missing]);
let mut raw_schema = schema.as_ref().clone();
raw_schema.eval_grid = None;
let raw_schema = Arc::new(raw_schema);
let raw = Series::try_new(
&raw_schema,
input.metric.clone(),
TimeAxis::try_new(vec![2500, 3000].into()).unwrap(),
SeriesFields::one(floats(&[Some(1.0), None])),
)
.unwrap();
for comparison in [
Comparison::Bool(ComparisonOp::Equal),
Comparison::Filter(ComparisonOp::Equal),
] {
for input in [&input, &all_missing] {
assert!(matches!(
map_compare(&schema, input, comparison, ScalarOperand::Right(1.0)),
Err(DataFusionError::NotImplemented(_))
));
}
assert!(matches!(
map_compare(&raw_schema, &raw, comparison, ScalarOperand::Right(1.0)),
Err(DataFusionError::Execution(_))
));
}
}
src/promql/src/map/tests/math_functions.rs
+171
View File
@@ -0,0 +1,171 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::*;
#[test]
fn clamp_follows_go_min_max_and_empties_reversed_bounds() {
let schema = schema(&["value"]);
let input = series(
&schema,
7,
vec![floats(&[
Some(f64::INFINITY),
Some(f64::NEG_INFINITY),
Some(-0.0),
Some(0.0),
Some(f64::NAN),
None,
])],
);
let output = |op| {
map_math(&schema, &input, op)
.unwrap()
.map(|series| series.fields.get(0).unwrap().clone())
};
// Go's Max lets +Inf win over NaN and Min lets -Inf win; +0/-0 are ordered.
let infinite = output(MathOp::ClampMin { min: f64::NAN }).unwrap();
assert_floats(
&infinite,
&[
Some(f64::INFINITY),
Some(f64::NAN),
Some(f64::NAN),
Some(f64::NAN),
Some(f64::NAN),
None,
],
);
let infinite = output(MathOp::ClampMax { max: f64::NAN }).unwrap();
assert_floats(
&infinite,
&[
Some(f64::NAN),
Some(f64::NEG_INFINITY),
Some(f64::NAN),
Some(f64::NAN),
Some(f64::NAN),
None,
],
);
let zero = output(MathOp::Clamp { min: 0.0, max: 0.0 }).unwrap();
assert_floats(
&zero,
&[
Some(0.0),
Some(0.0),
Some(0.0),
Some(0.0),
Some(f64::NAN),
None,
],
);
let zero = output(MathOp::Clamp {
min: -0.0,
max: -0.0,
})
.unwrap();
assert_floats(
&zero,
&[
Some(-0.0),
Some(-0.0),
Some(-0.0),
Some(-0.0),
Some(f64::NAN),
None,
],
);
// Either zero can win in Go's Max; the sign shows through a later `1 / x`.
let zero = output(MathOp::ClampMin { min: -0.0 }).unwrap();
assert_floats(
&zero,
&[
Some(f64::INFINITY),
Some(-0.0),
Some(-0.0),
Some(0.0),
Some(f64::NAN),
None,
],
);
assert!(output(MathOp::Clamp { min: 2.0, max: 1.0 }).is_none());
assert!(
output(MathOp::Clamp {
min: f64::NAN,
max: 1.0
})
.is_some()
);
}
#[test]
fn sgn_deg_and_rad_keep_prometheus_operation_order() {
let schema = schema(&["value"]);
let input = series(
&schema,
0,
vec![floats(&[Some(-0.0), Some(f64::NAN), Some(11.0), Some(3.0)])],
);
let field = |op| {
map_math(&schema, &input, op)
.unwrap()
.unwrap()
.fields
.get(0)
.unwrap()
.clone()
};
assert_floats(
&field(MathOp::Sgn),
&[Some(-0.0), Some(f64::NAN), Some(1.0), Some(1.0)],
);
// Bits from Go on amd64; 11 and 3 round differently under to_degrees/to_radians.
assert_floats(
&field(MathOp::Deg),
&[
Some(-0.0),
Some(f64::NAN),
Some(f64::from_bits(0x4083b2075224955b)),
Some(f64::from_bits(0x40657c6513cad17a)),
],
);
assert_ne!(11.0 * 180.0 / PI, 11.0_f64.to_degrees());
assert_floats(
&field(MathOp::Rad),
&[
Some(-0.0),
Some(f64::NAN),
Some(f64::from_bits(0x3fc893011f31982e)),
Some(f64::from_bits(0x3faacee9f37bebd5)),
],
);
assert_ne!(3.0 * PI / 180.0, 3.0_f64.to_radians());
}
#[test]
fn pi_produces_an_unlabelled_scalar_only_on_the_requested_range() {
let schema = schema(&["value"]);
for len in [0, 1, 7] {
let range = GridRange::try_new(schema.eval_grid.clone().unwrap(), 13, len).unwrap();
let output = map_pi(&schema, range.clone()).unwrap();
assert!(matches!(&output.time, TimeAxis::Grid(actual) if actual == &range));
assert!(output.metric.name().is_none());
assert!(output.metric.tags().is_empty());
assert_floats(output.fields.get(0).unwrap(), &vec![Some(PI); len]);
}
let invalid = super::schema(&["a", "b"]);
let range = GridRange::try_new(invalid.eval_grid.clone().unwrap(), 0, 1).unwrap();
assert!(map_pi(&invalid, range).is_err());
}
src/promql/src/map/tests/prometheus.rs
+194 -204
View File
@@ -12,223 +12,213 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//! Golden cases evaluated by the Prometheus engine; see the header of `prometheus_cases.txt`.
//! Map kernels against `prometheus_cases.txt`, evaluated by the Prometheus engine.
use std::sync::Arc;
use std::collections::BTreeMap;
use datatypes::arrow::array::Float64Array;
use datatypes::series::samples::{
FieldSpec, SampleRef, Samples, Series, SeriesFields, SeriesSchema,
use datatypes::series::samples::Series;
use datatypes::series::time::GridRange;
use crate::golden::{Case, assert_result, cases, hex};
use crate::map::{
ArithmeticOp, Comparison, ComparisonOp, MathOp, ScalarOperand, ScalarSeriesOperand,
map_arithmetic, map_arithmetic_series, map_compare, map_compare_series, map_math, map_negate,
scalar_arithmetic, scalar_compare,
};
use datatypes::series::time::{EvalGrid, GridRange, GridSpec, TimeAxis};
use datatypes::series::{Label, MetricName};
use crate::map::{MathOp, map_math};
const CASES: &str = include_str!("../prometheus_cases.txt");
type Points = Vec<(usize, f64)>;
pub(super) struct Case {
pub(super) header: Vec<String>,
pub(super) expr: String,
pub(super) schema: SeriesSchema,
pub(super) series: Vec<Series>,
pub(super) scalar: Option<Series>,
pub(super) result: Vec<(MetricName, Points)>,
}
pub(super) fn cases() -> Vec<Case> {
let mut cases = Vec::new();
let mut lines = CASES.lines().filter(|line| !line.starts_with('#'));
while let Some(line) = lines.next() {
let header = line.strip_prefix("case ").expect("case line");
let expr = lines
.next()
.unwrap()
.strip_prefix("expr ")
.unwrap()
.to_string();
let grid: Vec<i64> = lines.next().unwrap()["grid ".len()..]
.split(' ')
.map(|v| v.parse().unwrap())
.collect();
let schema = SeriesSchema {
fields: vec![FieldSpec {
name: "value".into(),
}]
.into(),
eval_grid: Some(Arc::new(
EvalGrid::try_new(GridSpec::try_new(grid[0], grid[1], grid[2] as u64).unwrap())
.unwrap(),
)),
};
let mut case = Case {
header: header.split(' ').map(String::from).collect(),
expr,
series: Vec::new(),
scalar: None,
result: Vec::new(),
schema,
};
for line in lines.by_ref() {
if line == "end" {
break;
}
let (kind, rest) = line.split_once(' ').unwrap();
let (labels, points) = rest.split_once(" ;").unwrap();
let (metric, points) = (parse_metric(labels), parse_points(points));
match kind {
"series" => {
if let Some(series) = grid_series(&case.schema, metric, &points) {
case.series.push(series);
}
}
"scalar" => case.scalar = grid_series(&case.schema, metric, &points),
"result" => case.result.push((metric, points)),
_ => panic!("unknown line {line}"),
}
}
cases.push(case);
}
cases
}
fn parse_metric(labels: &str) -> MetricName {
let mut name = None;
let mut tags = Vec::new();
for label in labels
.split(',')
.filter(|label| *label != "-" && !label.is_empty())
{
let (key, value) = label.split_once('=').unwrap();
if key == "__name__" {
name = Some(Arc::from(value));
} else {
tags.push(Label {
name: key.into(),
value: value.into(),
});
}
}
MetricName::try_new(name, tags).unwrap()
}
fn parse_points(points: &str) -> Points {
points
.split_whitespace()
.map(|point| {
let (index, bits) = point.split_once(':').unwrap();
(
index.parse().unwrap(),
f64::from_bits(u64::from_str_radix(bits, 16).unwrap()),
)
})
.collect()
}
pub(super) fn hex(bits: &str) -> f64 {
f64::from_bits(u64::from_str_radix(bits, 16).unwrap())
}
/// Builds the trimmed series Eval produces: from the first to the last point, nulls between.
fn grid_series(schema: &SeriesSchema, metric: MetricName, points: &Points) -> Option<Series> {
let (&(start, _), &(last, _)) = points.first().zip(points.last())?;
let mut values = vec![None; last - start + 1];
for &(index, value) in points {
values[index - start] = Some(value);
}
let range = GridRange::try_new(schema.eval_grid.clone().unwrap(), start, values.len()).unwrap();
Some(
Series::try_new(
schema,
metric,
TimeAxis::Grid(range),
SeriesFields::one(Samples::from_float(Float64Array::from(values))),
)
.unwrap(),
)
}
/// Compares labels, the trimmed absolute range and value bits; any NaN matches any NaN.
pub(super) fn assert_result(case: &Case, mut actual: Vec<Series>) {
actual.sort_by_key(|series| label_key(&series.metric));
let mut expected = case.result.iter().collect::<Vec<_>>();
expected.sort_by_key(|(metric, _)| label_key(metric));
let context = || format!("{} ({})", case.expr, case.header.join(" "));
assert_eq!(actual.len(), expected.len(), "{}", context());
for (actual, (metric, points)) in actual.iter().zip(expected) {
assert_eq!(&actual.metric, metric, "{}", context());
let TimeAxis::Grid(range) = &actual.time else {
panic!("{}: lost grid axis", context());
};
let mut present = Vec::new();
for field in actual.fields.iter() {
for (offset, sample) in field.iter().enumerate() {
match sample {
SampleRef::Float(value) => present.push((range.start() + offset, value)),
SampleRef::Missing => {}
_ => panic!("{}: non-float output", context()),
}
}
}
assert_eq!(
(range.start(), range.start() + range.len()),
(points[0].0, points.last().unwrap().0 + 1),
"{}: untrimmed range",
context()
);
assert_eq!(present.len(), points.len(), "{}", context());
for (&(index, actual), &(expected_index, expected)) in present.iter().zip(points) {
assert_eq!(index, expected_index, "{}", context());
assert!(
actual.to_bits() == expected.to_bits() || (actual.is_nan() && expected.is_nan()),
"{} at {index}: {actual:e} ({:016x}) != {expected:e} ({:016x})",
context(),
actual.to_bits(),
expected.to_bits(),
);
}
}
}
fn label_key(metric: &MetricName) -> Vec<(String, String)> {
let mut key = metric
.tags()
.iter()
.map(|label| (label.name.to_string(), label.value.to_string()))
.collect::<Vec<_>>();
if let Some(name) = metric.name() {
key.insert(0, ("__name__".into(), name.into()));
}
key
}
fn math_op(header: &[String]) -> Option<MathOp> {
Some(match header[1].as_str() {
fn math_op(header: &[String]) -> MathOp {
let arg = |index: usize| hex(&header[index]);
match header[1].as_str() {
"abs" => MathOp::Abs,
"ceil" => MathOp::Ceil,
"floor" => MathOp::Floor,
"sqrt" => MathOp::Sqrt,
"round" => MathOp::Round {
nearest: hex(&header[2]),
"exp" => MathOp::Exp,
"ln" => MathOp::Ln,
"log2" => MathOp::Log2,
"log10" => MathOp::Log10,
"sin" => MathOp::Sin,
"cos" => MathOp::Cos,
"tan" => MathOp::Tan,
"asin" => MathOp::Asin,
"acos" => MathOp::Acos,
"atan" => MathOp::Atan,
"sinh" => MathOp::Sinh,
"cosh" => MathOp::Cosh,
"tanh" => MathOp::Tanh,
"asinh" => MathOp::Asinh,
"acosh" => MathOp::Acosh,
"atanh" => MathOp::Atanh,
"deg" => MathOp::Deg,
"rad" => MathOp::Rad,
"sgn" => MathOp::Sgn,
"round" => MathOp::Round { nearest: arg(2) },
"clamp" => MathOp::Clamp {
min: arg(2),
max: arg(3),
},
_ => return None,
})
"clamp_min" => MathOp::ClampMin { min: arg(2) },
"clamp_max" => MathOp::ClampMax { max: arg(2) },
other => panic!("unmapped math function {other}"),
}
}
fn arithmetic_op(op: &str) -> ArithmeticOp {
match op {
"+" => ArithmeticOp::Add,
"-" => ArithmeticOp::Subtract,
"*" => ArithmeticOp::Multiply,
"/" => ArithmeticOp::Divide,
"%" => ArithmeticOp::Modulo,
"^" => ArithmeticOp::Power,
"atan2" => ArithmeticOp::Atan2,
other => panic!("unmapped arithmetic {other}"),
}
}
fn comparison(op: &str, mode: &str) -> Comparison {
let op = match op {
"==" => ComparisonOp::Equal,
"!=" => ComparisonOp::NotEqual,
">" => ComparisonOp::Greater,
"<" => ComparisonOp::Less,
">=" => ComparisonOp::GreaterEqual,
"<=" => ComparisonOp::LessEqual,
other => panic!("unmapped comparison {other}"),
};
match mode {
"filter" => Comparison::Filter(op),
"bool" => Comparison::Bool(op),
other => panic!("unmapped comparison mode {other}"),
}
}
fn scalar_operand(side: &str, value: f64) -> ScalarOperand {
match side {
"left" => ScalarOperand::Left(value),
_ => ScalarOperand::Right(value),
}
}
fn evaluate(case: &Case) -> Vec<Series> {
let h = &case.header;
let schema = &case.schema;
let full_grid =
|| GridRange::try_new(schema.eval_grid.clone().unwrap(), 0, grid_len(case)).unwrap();
let each = |f: &dyn Fn(&Series) -> Option<Series>| case.series.iter().filter_map(f).collect();
let scalar_series = || {
let scalar = case.scalar.as_ref().expect("scalar operand");
match h[h.len() - 1].as_str() {
"left" => ScalarSeriesOperand::Left(scalar),
_ => ScalarSeriesOperand::Right(scalar),
}
};
match h[0].as_str() {
"math" => each(&|s| map_math(schema, s, math_op(h)).unwrap()),
"neg" => each(&|s| Some(map_negate(schema, s).unwrap())),
"arith" => {
let operand = scalar_operand(&h[2], hex(&h[3]));
each(&|s| Some(map_arithmetic(schema, s, arithmetic_op(&h[1]), operand).unwrap()))
}
"cmp" => {
let operand = scalar_operand(&h[3], hex(&h[4]));
each(&|s| map_compare(schema, s, comparison(&h[1], &h[2]), operand).unwrap())
}
"arith_series" => each(&|s| {
map_arithmetic_series(schema, s, arithmetic_op(&h[1]), scalar_series()).unwrap()
}),
"cmp_series" => each(&|s| {
map_compare_series(schema, s, comparison(&h[1], &h[2]), scalar_series()).unwrap()
}),
"scalar_arith" => vec![
scalar_arithmetic(
schema,
full_grid(),
arithmetic_op(&h[1]),
[hex(&h[2]), hex(&h[3])],
)
.unwrap(),
],
"scalar_cmp" => vec![
scalar_compare(
schema,
full_grid(),
match comparison(&h[1], "bool") {
Comparison::Bool(op) | Comparison::Filter(op) => op,
},
[hex(&h[2]), hex(&h[3])],
)
.unwrap(),
],
other => panic!("unmapped case kind {other}"),
}
}
fn grid_len(case: &Case) -> usize {
case.schema.eval_grid.as_ref().unwrap().timestamps().len()
}
#[test]
fn math_matches_prometheus() {
let mut matched = 0;
for case in cases().iter().filter(|case| case.header[0] == "math") {
let op = math_op(&case.header).unwrap_or_else(|| panic!("unmapped {}", case.expr));
let output = case
.series
.iter()
.map(|series| map_math(&case.schema, series, op).unwrap())
.collect();
assert_result(case, output);
matched += 1;
fn map_kernels_match_prometheus() {
let mut matched = BTreeMap::<String, usize>::new();
for case in cases(CASES) {
assert_result(&case, evaluate(&case));
*matched.entry(case.header[0].clone()).or_default() += 1;
if case.header[0] == "math" {
*matched
.entry(format!("math {}", case.header[1]))
.or_default() += 1;
}
}
// Floors keep coverage from shrinking silently when the fixture is regenerated.
for (kind, floor) in [
("math", 180),
("neg", 10),
("arith", 70),
("cmp", 48),
("arith_series", 42),
("cmp_series", 36),
("scalar_arith", 42),
("scalar_cmp", 24),
] {
let count = matched.get(kind).copied().unwrap_or_default();
assert!(count >= floor, "{kind}: {count} < {floor}");
}
for function in [
"abs",
"ceil",
"floor",
"sqrt",
"exp",
"ln",
"log2",
"log10",
"sin",
"cos",
"tan",
"asin",
"acos",
"atan",
"sinh",
"cosh",
"tanh",
"asinh",
"acosh",
"atanh",
"deg",
"rad",
"sgn",
"round",
"clamp",
"clamp_min",
"clamp_max",
] {
let count = matched
.get(&format!("math {function}"))
.copied()
.unwrap_or_default();
assert!(count >= 4, "math {function}: {count}");
}
assert!(matched >= 48, "only {matched} math cases");
}