Lei, HUANG
f5c1020f5f
* feat(mito2): add shared SST storage and reclamation foundations Add shared-level isolation, receipt replay and recovery, migration support, and fail-closed file protection. Keep coordinator submission and verified publication in the next stack layer. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito2): isolate table compaction manifest semantics Keep atomic manifest extension envelopes and lifecycle safety in the common engine. Gate table-specific receipts and remap admission with enterprise, preserving their wire format and rejecting unsupported state before replay or GC. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito2): remove manifest-backed compaction task state Keep file-only edits, shared-reference admission and GC safety. Task retries no longer require persisted receipts or task epochs; reject unpublished prototype task-state records rather than silently dropping them. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): guard staging manifests and reclaim shared range indexes Reject prototype task state when decoding staging blobs, matching checkpoint and delta decoding. Delete the region-scoped range index when a shared SST reference is retired, while leaving the shared object to global GC. Document the level-2 merge-split rationale and bind the delta guard's action tag with a test. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito2): simplify GC deletion and lingering rules Require unknown files in a directory without a manifest to pass the lingering window before deletion, so an unpublished table-compaction output cannot be reclaimed out from under an active attempt. Replace the per-batch deletion loop with a single publication recheck taken before remote deletion, and drop the batching constant, the deferred-deletion accounting, and their test. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): keep regions open after manifest validation rejection Classify InvalidRequest from pre-write applicability checks as definitely not persisted. Cover both shared-removal and unsupported-level rejections without stopping the manifest manager, while keeping ambiguous storage errors conservative. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): reject ambiguous shared SST reclassification Reject edits that both add a shared SST and remove the same file ID before admission and under the manifest lock. Keep ordinary edit semantics and verify add-only L0 reclassification survives reopen without diverging from the live Version. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test(mito2): define truncate semantics for shared SST references Keep explicit truncation applicable to shared references and document the distinction from physical reclamation. Exercise full and time-range truncation with real shared SSTs, an old query, another referencing region, and reopen. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): define shared copy results and reject metadata imports Document copied_file_ids as installed logical references, including no-op retries. Reject shared SSTs in metadata-region copies before copying any objects or changing the target, and cover shared, mixed and repeated copy behavior. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * style(mito2): unify shared SST level references and bounds docs Use module-qualified shared-level constants consistently and document MAX_LEVEL as an exclusive bound for SST version operations. No runtime behavior changes. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito2): move the shared SST level to a max sentinel Use the u8 maximum as the shared classification so future ordinary levels can never collide with it, and map wire levels to fixed storage slots instead of indexing by value. Shared metadata, admission and filters keep their semantics; unallocated levels stay rejected. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): keep dropped-region tombstones while files are deferred Young unknown objects and hook-protected candidates on a dropped region must report needs_retry so the meta tombstone survives until they are actually reclaimed; reporting success would let the procedure drop the tombstone and orphan the retained objects. Cover both deferral paths plus the eventual clear. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): reject adopting an existing L1 SST as shared Adopting a shared reference is L0-only: the manifest would replace an existing L1 entry while the live version keeps both handles, duplicating rows until reopen. Reject the edit at admission before any state change and cover the rejection with a real SST. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Sqlness Test
Sqlness manual
Case file
Sqlness has two types of file:
.sql: test input, SQL only.result: expected test output, SQL and its results
.result is the output (execution result) file. If you see .result files is changed,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.
You only need to write test SQL in .sql file, and run the test.
Case organization
The root dir of input cases is tests/cases. It contains several subdirectories stand for different test
modes. E.g., standalone/ contains all the tests to run under greptimedb standalone start mode.
Under the first level of subdirectory (e.g. the cases/standalone), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.
Kafka WAL
Sqlness supports Kafka WAL. You can either provide a Kafka cluster or let sqlness to start one for you.
To run test with kafka, you need to pass the option -w kafka. If no other options are provided, sqlness will use conf/kafka-cluster.yml to start a Kafka cluster. This requires docker and docker-compose commands in your environment.
Otherwise, you can additionally pass the your existing kafka environment to sqlness with -k option. E.g.:
cargo sqlness bare -w kafka -k localhost:9092
In this case, sqlness will not start its own kafka cluster and the one you provided instead.
Run the test
Unlike other tests, this harness is in a binary target form. You can run it with:
cargo sqlness bare
It automatically finishes the following procedures: compile GreptimeDB, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check if the .result files are changed.
If not, congratulations, the test is passed 🥳!