* feat: add repartition procedure factory support to DdlManager
- Introduce RepartitionProcedureFactory trait for creating and registering
repartition procedures
- Implement DefaultRepartitionProcedureFactory for metasrv with full support
- Implement StandaloneRepartitionProcedureFactory for standalone (unsupported)
- Add procedure loader registration for RepartitionProcedure and
RepartitionGroupProcedure
- Add helper methods to TableMetadataAllocator for allocator access
- Add error types for repartition procedure operations
- Update DdlManager to accept and use RepartitionProcedureFactoryRef
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat: integrate repartition procedure into DdlManager
- Add submit_repartition_task() to handle repartition from alter table
- Route Repartition operations in submit_alter_table_task() to repartition factory
- Refactor: rename submit_procedure() to execute_procedure_and_wait()
- Make all DDL operations wait for completion by default
- Add submit_procedure() for fire-and-forget submissions
- Add CreateRepartitionProcedure error type
- Add placeholder Repartition handling in grpc-expr (unsupported)
- Update greptime-proto dependency
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat: implement ALTER TABLE REPARTITION procedure submission
Signed-off-by: WenyXu <wenymedia@gmail.com>
* refactor(repartition): handle central region in apply staging manifest
- Introduce ApplyStagingManifestInstructions struct to organize instructions
- Add special handling for central region when applying staging manifests
- Transition state from UpdateMetadata to RepartitionEnd after applying staging manifests
- Remove next_state() method in RepartitionStart and inline state transitions
- Improve logging and expression serialization in DDL statement executor
- Move repartition tests from standalone to distributed test suite
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: apply suggestions from CR
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: update proto
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat: implement deallocate regions for repartition procedure
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat(metric-engine): add force flag to drop physical regions with associated logical regions
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat: update table metadata after deallocating regions
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: apply suggestions
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: update proto
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat: impl vector index building
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: supports flat format
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* ci: add vector_index feature to test
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: apply suggestions
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: apply suggestions from copilot
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: update proto to include native histogram
* ci: add a CI check to ensure whitelisted dependencies are using their main branch
* chore: add changes to Cargo.toml to trigger CI
* chore: update proto
* test: update test to include histogram
* refactor(cli): unify storage configuration for export command
- Utilize ObjectStoreConfig to unify storage configuration for export command
- Support export command for Fs, S3, OSS, GCS and Azblob
- Fix the Display implementation for SecretString always returned the string
"SecretString([REDACTED])" even when the internal secret was empty.
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
* refactor(cli): unify storage configuration for export command
- Change the encapsulation permissions of each configuration
options for every storage backend to public access.
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
* refactor(cli): unify storage configuration for export command
- Update the implementation of ObjectStoreConfig::build_xxx() using macro solutions
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
* refactor(cli): unify storage configuration for export command
- Introduce config validation for each storage type
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
* refactor(cli): unify storage configuration for export command
- Enable trait-based polymorphism for storage type handling
(from inherent impl to trait impl)
- Extract helper functions to reduce code duplication
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
* refactor(cli): unify storage configuration for export command
- Improve SecretString handling and validation
(Distinguishing between "not provided" and "empty string")
- Add validation when using filesystem storage
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
* refactor(cli): unify storage configuration for export command
- Refactor storage field validation with macro
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
* refactor(cli): unify storage configuration for export command
- support GCS Application Default Credentials (like GKE, Cloud Run, or local development with ) in export
(Enabling ADC without validating or to be present)
(Making optional in GCS validation (defaults to https://storage.googleapis.com))
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
* refactor(cli): unify storage configuration for export command
This commit refactors the validation logic for object store configurations in the CLI to leverage clap features and reduce boilerplate.
Key changes:
- Update wrap_with_clap_prefix macro to use clap's requires attribute.
This ensures that storage-specific options (e.g., --s3-bucket) are only accepted when the corresponding backend is enabled (e.g., --s3).
- Simplify FieldValidator trait by removing the is_provided method, as dependency checks are now handled by clap.
- Introduce validate_backend! macro to standardize the validation of required fields for enabled backends.
- Refactor ExportCommand to remove explicit validation calls (validate_s3, etc.) and rely on the validation within backend constructors.
- Add integration tests for ExportCommand to verify build success with S3, OSS, GCS, and Azblob configurations.
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
* refactor(cli): unify storage configuration for export command
- Use macros to simplify storage export implementation
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
* refactor(cli): unify storage configuration for export command
- Rollback StorageExport trait implementation to not using macro for better code clarity and maintainability
- Introduce format_uri helper function to unify URI formatting logic
- Fix OSS URI path bug inherited from legacy code
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
* refactor(cli): unify storage configuration for export command
- Remove unnecessary async_trait
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
---------
Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
* feat: adds the foundational types and SQL parsing support for vector index
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: by suggestions
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: ensure index option values must be greater than zero
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: validate connectivity strictly
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: compile error
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: disable SIMD for ci
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: regression with shortcutted statement on postgres extended query
* chore: typo fix
* feat: also add more type support for parameters
* chore: remove dbg print
* feat/change-tsid-gen:
perf(metric-engine): replace mur3 with fxhash for faster TSID generation
- Switches from mur3::Hasher128 to fxhash::FxHasher for TSID hashing
- Pre-computes label-name hash when no nulls are present, avoiding redundant work
- Adds fast-path for rows without nulls; falls back to slow path otherwise
- Updates Cargo.toml and lockfile to reflect dependency change
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/change-tsid-gen:
fix: only check primary-key labels for null when re-using cached hash
- Rename has_null() → has_null_labels() and restrict the check to the
primary-key columns so that non-label NULLs do not force a full
TSID re-computation.
- Update expected hashes in tests to match the new logic.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/change-tsid-gen:
test: add comprehensive TSID generation tests for label ordering and null handling
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/change-tsid-gen:
bench: add criterion benchmark for TSID generator
- Compare original mur3 vs current fxhash fast/slow paths
- Test 2, 5, 10 label sets plus null-value slow path
- Add mur3 & criterion dev-deps; register bench target
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/change-tsid-gen:
test: stabilize metric-engine tests by fixing non-deterministic row order
- Add ORDER BY to SELECTs in TTL tests to ensure consistent output
- Update expected __tsid values after hash function change
- Swap expected OTLP metric rows to match new ordering
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/change-tsid-gen:
refactor: simplify Default impls and remove redundant code
- Replace manual Default for TsidGenerator with derive
- Remove unnecessary into_iter() call
- Simplify Option::unwrap_or_else to unwrap_or
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat: gc ctx&procedure
Signed-off-by: discord9 <discord9@163.com>
* fix: handle region not found case
Signed-off-by: discord9 <discord9@163.com>
* docs: more explain&todo
Signed-off-by: discord9 <discord9@163.com>
* per review
Signed-off-by: discord9 <discord9@163.com>
* chore: add time for region gc
Signed-off-by: discord9 <discord9@163.com>
* fix: explain why loader for gc region should fail
Signed-off-by: discord9 <discord9@163.com>
---------
Signed-off-by: discord9 <discord9@163.com>
