dennis zhuang
8174697c96
* fix(promql): correct counter reset accumulation in rate windows
`prom_rate` and `prom_increase` reused the previous window's counter-reset
correction when the next window slid forward by exactly one sample, adding
the entering reset and subtracting the leaving one. Running a sum through
addition and subtraction does not restore the earlier terms in f64: a large
reset absorbs the smaller ones that must survive it, and an expired infinity
leaves a NaN that no later window can clear. `prom_delta` shares the code but
is not a counter function, so it never took that path.
Index the reset positions of the value array once instead, and reduce each
window over the resets it contains, in sample order. The result is
bit-identical to scanning the window directly, so windows keep the direct
reduction when they request fewer sample pairs than the input has.
Also sweep the query step in the rate benchmarks: the cost of the reset
correction depends on how much the windows overlap, which no existing case
varied.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* test(promql): cover counter reset precision over adjacent rate windows
The unit tests build the range windows directly, so they do not show that a
plain PromQL range query produces the window layout that lost the correction.
This case does: with a query step equal to the sample interval, `increase`
over the second window returns 1.333 before the fix and 2.667 after it.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* perf(promql): advance the counter reset bounds instead of searching
Locating a window's resets with two binary searches costs more than the
reduction it replaces once a series resets often enough for the searches to
get deep: on a 20k-sample counter resetting every 37 samples, stepping the
windows by one sample was 2.7x slower than the previous code, against 1.2x
for a counter that never resets.
Windows normally advance, so walk the bounds forward from the previous
window and only search when they move back. The cost then no longer depends
on the reset density.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* perf(promql): cut the per-window cost of the counter reset index
Two costs the index added showed up on a one-sample query step, where the
removed fast path used to answer each window with two comparisons.
Cache the two reset positions that bound the active slice. A window that only
advanced and reached neither of them covers the same resets as the previous
one, so the common case is four integer comparisons and no lookup at all.
Stop summing the requested sample pairs once they exceed one pass over the
values. The sum only decides which side of that comparison the input falls on,
and a query with a short lookback and a long step settles it after a few
windows instead of after every key.
Together these take the one-sample step from 25-32% slower than the previous
code down to 6-11%, measured as before / after / before to bound drift. No
other step value regresses, and a ten-sample step stays about 88% faster.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* fix(promql): accumulate counter resets into the running result
`prom_rate` and `prom_increase` summed a window's counter-reset corrections
on their own and added that sum to `last - first`. Prometheus folds each
reset into the running result instead, and so did this code before #7880.
The two are not interchangeable in f64: over samples `[1e16, 1, 0, 1]` the
isolated sum rounds `1e16 + 1.0` back to `1e16`, which then cancels against
the first sample and reports no increase at all, where folding the resets in
one at a time keeps the 1.0.
Restore the original order. The reset index accumulates into the result the
same way, so it still matches a direct scan of the window bit for bit, but a
window's contribution can no longer be cached as a standalone value and is
re-added from its own difference each time. The bounds are still cached, so
a window that did not cross a reset skips the lookup, and one that holds no
resets returns without touching the index at all.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* test(promql): note which reset boundaries the stride of one walks
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
---------
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
(cherry picked from commit 0f625a7e92)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Sqlness Test
Sqlness manual
Case file
Sqlness has two types of file:
.sql: test input, SQL only.result: expected test output, SQL and its results
.result is the output (execution result) file. If you see .result files is changed,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.
You only need to write test SQL in .sql file, and run the test.
Case organization
The root dir of input cases is tests/cases. It contains several subdirectories stand for different test
modes. E.g., standalone/ contains all the tests to run under greptimedb standalone start mode.
Under the first level of subdirectory (e.g. the cases/standalone), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.
Kafka WAL
Sqlness supports Kafka WAL. You can either provide a Kafka cluster or let sqlness to start one for you.
To run test with kafka, you need to pass the option -w kafka. If no other options are provided, sqlness will use conf/kafka-cluster.yml to start a Kafka cluster. This requires docker and docker-compose commands in your environment.
Otherwise, you can additionally pass the your existing kafka environment to sqlness with -k option. E.g.:
cargo sqlness bare -w kafka -k localhost:9092
In this case, sqlness will not start its own kafka cluster and the one you provided instead.
Run the test
Unlike other tests, this harness is in a binary target form. You can run it with:
cargo sqlness bare
It automatically finishes the following procedures: compile GreptimeDB, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check if the .result files are changed.
If not, congratulations, the test is passed 🥳!