discord9
01b0076f3e
* fix(promql): collapse duplicate samples at one timestamp before range functions A read path that does not deduplicate, such as an append-mode region, can hand PromQL two rows for one series at one timestamp. Every range function reads its samples from RangeManipulate, so the duplicate entries were treated as independent samples: count_over_time counted both, idelta and irate returned NULL because the last two entries shared a timestamp, and delta/increase/rate deflated because the average sampling interval divides the observed span by sample_count - 1. Prometheus sample identity is one sample per series per timestamp, so its range functions can never see this input. Restore that identity once, at the operator every range function shares, instead of special-casing each function's arithmetic. The collapse compares native ticks, never the millisecond payload, so distinct sub-millisecond samples that share a millisecond stay distinct as calculate_range documents and tests. Batches without duplicate timestamps take a single adjacency scan and are returned untouched; only when duplicates exist is a take built, keeping tag and value columns aligned. Keeping the last row of a group matches MergeMode::LastRow. The normalize test that fed three rows at one timestamp now uses distinct timestamps: it exercises histogram start-timestamp offsetting, and its repeated timestamp was incidental, not asserted behavior. Signed-off-by: discord9 <discord9@163.com> * test(promql): cover duplicate samples end to end and keep the newer value The operator now keeps the first row of a tied timestamp group, which is the row a deduplicating read keeps: storage orders equal primary key and timestamp by sequence descending, and FlatDedupReader takes the first row of each group. The end-to-end case pins that choice by writing 100 and then 101 at the same timestamp: the window reports increase 6, while keeping the stale 100 would report 8. The sqlness case writes duplicate rows into an append_mode region, which is the read path that does not deduplicate, and asserts every range function plus the unaffected ordinary table in both the standalone and distributed environments. Signed-off-by: discord9 <discord9@163.com> * docs(promql): qualify duplicate-timestamp conflict selection Signed-off-by: discord9 <discord9@163.com> * perf(promql): avoid copying tags when collapsing duplicate samples Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>
Sqlness Test
Sqlness manual
Case file
Sqlness has two types of file:
.sql: test input, SQL only.result: expected test output, SQL and its results
.result is the output (execution result) file. If you see .result files is changed,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.
You only need to write test SQL in .sql file, and run the test.
Case organization
The root dir of input cases is tests/cases. It contains several subdirectories stand for different test
modes. E.g., standalone/ contains all the tests to run under greptimedb standalone start mode.
Under the first level of subdirectory (e.g. the cases/standalone), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.
Kafka WAL
Sqlness supports Kafka WAL. You can either provide a Kafka cluster or let sqlness to start one for you.
To run test with kafka, you need to pass the option -w kafka. If no other options are provided, sqlness will use conf/kafka-cluster.yml to start a Kafka cluster. This requires docker and docker-compose commands in your environment.
Otherwise, you can additionally pass the your existing kafka environment to sqlness with -k option. E.g.:
cargo sqlness bare -w kafka -k localhost:9092
In this case, sqlness will not start its own kafka cluster and the one you provided instead.
Run the test
Unlike other tests, this harness is in a binary target form. You can run it with:
cargo sqlness bare
It automatically finishes the following procedures: compile GreptimeDB, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check if the .result files are changed.
If not, congratulations, the test is passed 🥳!