* feat(meta-srv): bind candidate region leases to the migration's registration A region migration now gives its candidate region registration a random non-zero candidate lease id. Leases granted because of that registration carry it, but only while the table route exists, and the upgrade instruction carries the same id. The datanode passes the id of each lease to the engine, and Mito records the lease of the latest heartbeat per region outside the region instance, so it survives a reopen during catchup. The migration also drops the candidate registration before it restores the route on rollback, and before the procedure leaves the migration tracker when it ends without a rollback. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix(mito2): record a lease only while the region stays open Record a granted lease under the same lock that removes it after the region closes, so a lease recorded during a close does not outlive the region. The metric engine records the data region's lease when the metadata region ignores it. Remove the roles-only extraction of operating regions, and cover a present candidate lease id in the upgrade JSON. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * build: point greptime-proto at the candidate lease id change Use the greptime-proto commit that adds GrantedRegion.candidate_lease_id (greptime-proto#357), which also brings in the RawSeriesSource Substrait payload message. Rely on the declaration order of RegionMigrationProcedure to drop the candidate registration before the tracker entry, and document it. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix(mito2): drop lease records when workers stop Remove the lease records of the regions a stopping worker closes. Check under the removal lock deterministically in tests, cover a lease that arrives after a region is removed, assert the candidate lease id on the migration's real registration and its retry, and match ids per region in a batched upgrade. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test(mito2): drop a redundant helper comment Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test(store-api): pin the candidate lease id wire bytes Signed-off-by: jeremyhi <fengjiachun@gmail.com> * docs: document candidate registration and lease record lifecycles Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: keep candidate lease ids through customized lease renewers The heartbeat handler now sets the candidate lease id of each lease a customized renewer returns from the renewed region with the same id. The metric engine records a lease for the data region only when the metadata region ignored it, since the data region records its own leases. The countdown reset command carries a named struct instead of a tuple. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test(metric-engine): cover a lease the data region ignores Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor: tighten lease id lookups and lease test assertions Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor: align lease naming and share the txn-observing kv backend Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor: tidy lease naming, docs and shared test helpers Signed-off-by: jeremyhi <fengjiachun@gmail.com> * docs: drop field comments that restate their names Signed-off-by: jeremyhi <fengjiachun@gmail.com> * docs: drop method comments that restate their names Signed-off-by: jeremyhi <fengjiachun@gmail.com> * docs: say which heartbeat a lease's send time refers to Signed-off-by: jeremyhi <fengjiachun@gmail.com> * docs: drop the reset deadline field comment Signed-off-by: jeremyhi <fengjiachun@gmail.com> * docs: state when an operating region's role and id reach a lease Signed-off-by: jeremyhi <fengjiachun@gmail.com> * chore: pin greptime-proto to the merged candidate lease id change Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: add the candidate lease id argument to the direct demotion test Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor: order new keeper helpers after public items and test equal send times Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Sqlness Test
Sqlness manual
Case file
Sqlness has two types of file:
.sql: test input, SQL only.result: expected test output, SQL and its results
.result is the output (execution result) file. If you see .result files is changed,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.
You only need to write test SQL in .sql file, and run the test.
Case organization
The root dir of input cases is tests/cases. It contains several subdirectories stand for different test
modes. E.g., standalone/ contains all the tests to run under greptimedb standalone start mode.
Under the first level of subdirectory (e.g. the cases/standalone), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.
Kafka WAL
Sqlness supports Kafka WAL. You can either provide a Kafka cluster or let sqlness to start one for you.
To run test with kafka, you need to pass the option -w kafka. If no other options are provided, sqlness will use conf/kafka-cluster.yml to start a Kafka cluster. This requires docker and docker-compose commands in your environment.
Otherwise, you can additionally pass the your existing kafka environment to sqlness with -k option. E.g.:
cargo sqlness bare -w kafka -k localhost:9092
In this case, sqlness will not start its own kafka cluster and the one you provided instead.
Run the test
Unlike other tests, this harness is in a binary target form. You can run it with:
cargo sqlness bare
It automatically finishes the following procedures: compile GreptimeDB, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check if the .result files are changed.
If not, congratulations, the test is passed 🥳!