jeremyhi
77cb39b5d1
* feat(mito2): record pending WAL tails in the region manifest Adds `pending_tails` and `handoff_floors` to the region manifest and its checkpoint, and an optional `pending_tails` / `handoff_floor` pair to `RegionEdit`. An edit without them leaves the records unchanged. A floor is dropped once the flushed entry id reaches it; a tail stays until a commit removes it. Neither is written when empty, so manifests without them encode as before. Opening a region replays its WAL after this node's tails and floor, and WAL releases at open and after a flush stay below this node's tails. Nothing writes these records yet, so every region behaves as before. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * feat(mito2): hold back reads and region changes while WAL tails are pending A region with pending WAL tails is missing data that only those tails hold. Until they are recovered, scans and memtable and SST stats fail with the retryable `RegionPendingTails` error. Truncate, entering staging, a repartition flush, a close-time flush on request, and ALTERs other than adding nullable columns without a default or changing options are rejected before they change any region state. Compaction pauses. A full-listing GC pass skips the region while it has pending tails, or while a newer manifest may exist or the manifest listing fails. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * docs(mito2): name the WAL releases that keep pending tails Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix(mito2): reject no-op ALTERs of restricted kinds while WAL tails are pending Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test(mito2): check that reopening restores the pending-tail read gate Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test(mito2): pin what the pending-tail fixtures and rejections promise Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor(mito2): name the replay start in the open log and share the pending-tail check Signed-off-by: jeremyhi <fengjiachun@gmail.com> * docs(mito2): state the per-node order of pending tails and who keeps later writes out of them Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test(mito2): check that a cancelled compaction starts nothing while WAL tails are pending When a compaction cancelled by task id finishes, the queued manual request goes through handle_pending_compaction_request, whose pending-tails check replies Ok(0). The reply is read with try_recv so that a dispatched request fails the test at once instead of hanging it. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix(mito2): drop the pending compaction task state when WAL tails skip it Skipping a pending manual compaction while WAL tails are pending replied Ok(0) but left its task state in place, so task inspection kept listing it as queued and cancelling it by id still succeeded. Clear the pending task state and publish before replying. Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Sqlness Test
Sqlness manual
Case file
Sqlness has two types of file:
.sql: test input, SQL only.result: expected test output, SQL and its results
.result is the output (execution result) file. If you see .result files is changed,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.
You only need to write test SQL in .sql file, and run the test.
Case organization
The root dir of input cases is tests/cases. It contains several subdirectories stand for different test
modes. E.g., standalone/ contains all the tests to run under greptimedb standalone start mode.
Under the first level of subdirectory (e.g. the cases/standalone), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.
Kafka WAL
Sqlness supports Kafka WAL. You can either provide a Kafka cluster or let sqlness to start one for you.
To run test with kafka, you need to pass the option -w kafka. If no other options are provided, sqlness will use conf/kafka-cluster.yml to start a Kafka cluster. This requires docker and docker-compose commands in your environment.
Otherwise, you can additionally pass the your existing kafka environment to sqlness with -k option. E.g.:
cargo sqlness bare -w kafka -k localhost:9092
In this case, sqlness will not start its own kafka cluster and the one you provided instead.
Run the test
Unlike other tests, this harness is in a binary target form. You can run it with:
cargo sqlness bare
It automatically finishes the following procedures: compile GreptimeDB, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check if the .result files are changed.
If not, congratulations, the test is passed 🥳!