mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-07-06 05:50:38 +00:00
3758911a2e
* feat: add close-first soft-drop table flow Soft-drop now tombstones table metadata and closes datanode regions instead of issuing physical drop requests, while preserving hard-drop cleanup semantics and blocking conflicting drops of recreated table names. Files: - `src/common/meta/src/ddl.rs` - `src/common/meta/src/ddl/drop_table.rs` - `src/common/meta/src/ddl/drop_table/executor.rs` - `src/common/meta/src/error.rs` - `src/common/meta/src/ddl_manager.rs` - `src/meta-srv/src/metasrv/builder.rs` - `src/cmd/src/standalone.rs` - `src/common/meta/src/test_util.rs` - `src/meta-srv/src/procedure/utils.rs` - `tests-integration/src/standalone.rs` - `src/common/meta/src/ddl/tests/drop_table.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: add undrop and purge table procedures Add soft-drop recovery and cleanup procedures, wire their DDL task handling, and update \`greptime-proto\` so the new tasks can round-trip through protobuf. Files: - \`Cargo.toml\` - \`Cargo.lock\` - \`src/common/meta/src/ddl.rs\` - \`src/common/meta/src/ddl/undrop_table.rs\` - \`src/common/meta/src/ddl/purge_dropped_table.rs\` - \`src/common/meta/src/ddl_manager.rs\` - \`src/common/meta/src/rpc/ddl.rs\` - \`src/common/meta/src/key.rs\` - \`src/common/meta/src/ddl/tests/drop_table.rs\` - \`src/mito2/src/engine/open_test.rs\` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: reopen soft-dropped regions before purge Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: disable soft-drop operations for metric logical tables Prevent soft-dropping, undropping, and purging of metric engine logical tables by explicitly returning unsupported errors. This introduces `is_metric_engine_logical_table` to identify metric logical tables and adds corresponding test cases. Files: - `src/common/meta/src/ddl/drop_table/metadata.rs` - `src/common/meta/src/ddl/purge_dropped_table.rs` - `src/common/meta/src/ddl/tests/drop_table.rs` - `src/common/meta/src/ddl/undrop_table.rs` - `src/common/meta/src/ddl/utils.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: sync failure detectors during soft-drop lifecycle Keep region failure detector state aligned as soft-dropped tables close, reopen, and purge regions so stale detectors do not trigger failover for unavailable or deleted regions. Files: - \`src/common/meta/src/ddl.rs\` - \`src/common/meta/src/ddl/drop_table.rs\` - \`src/common/meta/src/ddl/undrop_table.rs\` - \`src/common/meta/src/ddl/purge_dropped_table.rs\` - \`src/common/meta/src/ddl/tests/drop_table.rs\` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * refactor: simplify soft-drop table tests Consolidate redundant soft-drop lifecycle assertions into existing end-to-end tests and share dropped-table metadata setup to keep the branch coverage focused. Files: - `src/common/meta/src/ddl/tests/drop_table.rs` - `src/common/meta/src/key.rs` - `src/mito2/src/engine/open_test.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: reopen follower regions during undrop Reopen all replicas when restoring dropped physical tables so recovered replicated tables do not leave follower regions closed. Files: - `src/common/meta/src/ddl/undrop_table.rs` - `src/common/meta/src/ddl/tests/drop_table.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: use dropped table ddl expr protos Update greptime-proto and adapt dropped table DDL task conversions to the shared expression wrappers required by the proto API. Files: - `Cargo.toml` - `Cargo.lock` - `src/api/src/helper.rs` - `src/common/meta/src/ddl/drop_table/executor.rs` - `src/common/meta/src/rpc/ddl.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * revert: use inline dropped table task protos Point GreptimeDB at the proto revision that restores direct dropped table task fields and remove wrapper-expression conversion code. Files: - `Cargo.toml` - `Cargo.lock` - `src/api/src/helper.rs` - `src/common/meta/src/rpc/ddl.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: various soft-drop improvements - Bump `greptime-proto` dependency revision (`Cargo.toml`, `Cargo.lock`) - Pass `region_wal_options` directly without pre-serialization in undrop flow (`src/common/meta/src/ddl/undrop_table.rs`, `src/common/meta/src/key.rs`) - Remove unused `RegionNumber` import (`src/common/meta/src/ddl/utils.rs`) - Add `reset_failure_detectors` to test mock (`src/common/meta/src/ddl/tests/drop_table.rs`) - Add JSON roundtrip tests for `UndropTableTask` and `PurgeDroppedTableTask` (`src/common/meta/src/rpc/ddl.rs`) Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: validate table name match in undrop procedure Add a guard in `UndropTableProcedure::on_prepare()` to check that the dropped table name matches the undrop task name, returning `TableNotFound` on mismatch. This prevents undropping a table by a different name when only the table ID is known. - `src/common/meta/src/ddl/undrop_table.rs` — add table-name validation - `src/common/meta/src/ddl/tests/drop_table.rs` — add test for name mismatch Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor: simplify UndropTableTask to use table_id only Remove catalog, schema, and table name fields from `UndropTableTask` since the table name can be derived from the dropped table metadata in the procedure itself. This eliminates redundant fields and the associated name-validation test. Simplify locking in `UndropTableProcedure` to only use `TableLock`. Update `greptime-proto` dependency revision. - `Cargo.toml`, `Cargo.lock` - `src/common/meta/src/rpc/ddl.rs` - `src/common/meta/src/ddl/undrop_table.rs` - `src/common/meta/src/ddl/tests/drop_table.rs` Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: detect table name conflict during tombstone restore in undrop - Added `require_dest_not_exists` parameter to tombstone `move_values` to check destination key existence during restore - Added `TombstoneTargetAlreadyExists` error variant - Map tombstone conflict to `TableAlreadyExists` in undrop procedure - Added test for undrop failing when live name created after prepare Files: - `src/common/meta/src/ddl/tests/drop_table.rs` - `src/common/meta/src/ddl/undrop_table.rs` - `src/common/meta/src/error.rs` - `src/common/meta/src/key/tombstone.rs` Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: make PurgeDroppedTableTask table_id-only Streamline the purge-dropped-table flow by requiring a table_id instead of allowing name-based fallback. - Refactored `PurgeDroppedTableTask` to hold only `table_id` in `src/common/meta/src/rpc/ddl.rs` - Simplified purge procedure in `src/common/meta/src/ddl/purge_dropped_table.rs` - Adapted tests in `src/common/meta/src/ddl/tests/drop_table.rs` - Bumped `greptime-proto` dependency Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(tombstone): chunk values by per-key txn ops instead of fixed divisor Replaced the fixed `max_txn_ops() / 2` chunk size with operation-aware constants (`MOVE_VALUE_TXN_OPS_PER_KEY=4`, `RESTORE_VALUE_TXN_OPS_PER_KEY=6`) to correctly account for per-key transaction operations. Added `TxnOpLimitKvBackend` test helper and two new tests (`test_restore_chunks_by_total_txn_ops_limit`, `test_create_chunks_by_total_txn_ops_limit`) verifying chunking under tight txn op limits. Affected file: - `src/common/meta/src/key/tombstone.rs` — chunk size fix, `TxnOpLimitKvBackend` helper, two new tests Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat(soft-drop): deregister failure detectors and handle replayed open-regions during purge - `src/common/meta/src/ddl/drop_table.rs`: deregister failure detectors before transitioning to DeleteTombstone state - `src/common/meta/src/ddl/undrop_table.rs`: refactor `open_regions` into `open_regions_inner` with an `ignore_region_not_found` flag; expose `open_regions_ignore_region_not_found` for purge replayer - `src/common/meta/src/ddl/purge_dropped_table.rs`: use `open_regions_ignore_region_not_found` in replayed purge procedures - `src/common/meta/src/ddl/tests/drop_table.rs`: add tests for undrop idempotency and purge replay tolerance of dropped regions Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * chore: fix clippy Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(soft-drop): open regions before restoring undrop metadata Restore undropped table metadata only after physical regions have been reopened, keeping the table hidden while regions are still closed. Preserve the live-name conflict check before opening regions and cover the ordering with a regression test. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(tombstone): fail fast on invalid txn op budget Fail before issuing `TombstoneManager` transactions when the configured `max_txn_ops` cannot fit one key. Add coverage for undersized restore budgets in `src/common/meta/src/key/tombstone.rs`. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * chore: bump greptime-proto to main branch commit Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>