dennis zhuang
7152ca9264
* fix(promql): treat NULL field values as absent samples in range functions Range functions read the value column through `Float64Array::values()`, which returns the raw buffer and ignores the null bitmap. A NULL field value means the series has no sample at that timestamp, so the padding under a null slot (0.0 in practice) was counted as a real sample. `rate`, `increase`, `delta`, `changes`, `resets`, `idelta`, `irate`, `quantile_over_time` and `avg_over_time` now work on samples instead of slots. `stddev_over_time` and `stdvar_over_time` used to panic on a NULL slot, and tokio swallowed the panic so the query returned success with that series missing. Null handling is gated on `null_count() == 0` over the whole backing array, checked once per batch, so tables without NULLs keep the existing code path. `deriv` and `predict_linear` already had this guard through `linear_regression_slices` and are untouched. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix(promql): keep other fields when one has no sample in the window Review follow-up. Two things surfaced once range functions started returning NULL for a window without samples. The filter after a function call required every field column to be non-NULL, so on a multi-field table one field with no samples in a window would drop the other fields' results with it. It now keeps a row when any field has a sample, which is the shape a selector already emits. On a single field column the two predicates are identical. `quantile_over_time` returned NaN rather than NULL for a window without samples, so the row survived that filter. Prometheus returns an empty vector there, so the emptiness check now sits in the range UDF; the shared quantile kernel still yields NaN for an empty slice, matching upstream's `quantile()` helper that `quantile_aggr` depends on. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * docs(promql): correct two comments on the null handling The planner one did not say why `preserve_any_value` is hardcoded at that call site, which is the question a reader arrives with. The `quantile_over_time` one described the empty-window behaviour while sitting on the `has_nulls` line, and that behaviour had moved into `window_quantile`. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix(promql): clamp extrapolation before snapping a counter to zero Prometheus clamps `durationToStart` to half an average interval once the first sample is past the extrapolation threshold, and only then lets the counter zero-snap shorten it further, so the snap can never lengthen the leading extrapolation. Running the snap first let it rescue a duration the clamp should have cut, and `rate` and `increase` over-extrapolated to the left. For samples 1@0s and 2@1s in a 4s window at 1s, upstream gives 0.375 and this returned 0.5. `extrapolation_matches_prometheus_on_seeded_windows` carries a line-by-line port of upstream `extrapolatedRate` as an oracle and diffs it against the UDF over seeded windows, so the order stays pinned. `factor` also picked up upstream's guard against a zero sampled interval, which previously divided by zero. Two sqlness results move, both verified against the upstream algorithm. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(promql): fold range_presence_null into null_samples #9104 landed its own NULL-sample case whose data is the same series this one already used: one host with interior NULLs, one host with nothing but NULLs. Keeping both means two files asserting the same semantics on the same rows. The merged case keeps every query from both, so the presence functions still cover the trailing-NULL window, the count of two, the empty left-open window at t=7, and the all-NULL windows. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Sqlness Test
Sqlness manual
Case file
Sqlness has two types of file:
.sql: test input, SQL only.result: expected test output, SQL and its results
.result is the output (execution result) file. If you see .result files is changed,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.
You only need to write test SQL in .sql file, and run the test.
Case organization
The root dir of input cases is tests/cases. It contains several subdirectories stand for different test
modes. E.g., standalone/ contains all the tests to run under greptimedb standalone start mode.
Under the first level of subdirectory (e.g. the cases/standalone), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.
Kafka WAL
Sqlness supports Kafka WAL. You can either provide a Kafka cluster or let sqlness to start one for you.
To run test with kafka, you need to pass the option -w kafka. If no other options are provided, sqlness will use conf/kafka-cluster.yml to start a Kafka cluster. This requires docker and docker-compose commands in your environment.
Otherwise, you can additionally pass the your existing kafka environment to sqlness with -k option. E.g.:
cargo sqlness bare -w kafka -k localhost:9092
In this case, sqlness will not start its own kafka cluster and the one you provided instead.
Run the test
Unlike other tests, this harness is in a binary target form. You can run it with:
cargo sqlness bare
It automatically finishes the following procedures: compile GreptimeDB, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check if the .result files are changed.
If not, congratulations, the test is passed 🥳!