mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2025-12-22 22:20:02 +00:00
refactor(cli): unify storage configuration for export command (#7280)
* refactor(cli): unify storage configuration for export command - Utilize ObjectStoreConfig to unify storage configuration for export command - Support export command for Fs, S3, OSS, GCS and Azblob - Fix the Display implementation for SecretString always returned the string "SecretString([REDACTED])" even when the internal secret was empty. Signed-off-by: McKnight22 <tao.wang.22@outlook.com> * refactor(cli): unify storage configuration for export command - Change the encapsulation permissions of each configuration options for every storage backend to public access. Signed-off-by: McKnight22 <tao.wang.22@outlook.com> Co-authored-by: WenyXu <wenymedia@gmail.com> * refactor(cli): unify storage configuration for export command - Update the implementation of ObjectStoreConfig::build_xxx() using macro solutions Signed-off-by: McKnight22 <tao.wang.22@outlook.com> Co-authored-by: WenyXu <wenymedia@gmail.com> * refactor(cli): unify storage configuration for export command - Introduce config validation for each storage type Signed-off-by: McKnight22 <tao.wang.22@outlook.com> * refactor(cli): unify storage configuration for export command - Enable trait-based polymorphism for storage type handling (from inherent impl to trait impl) - Extract helper functions to reduce code duplication Signed-off-by: McKnight22 <tao.wang.22@outlook.com> * refactor(cli): unify storage configuration for export command - Improve SecretString handling and validation (Distinguishing between "not provided" and "empty string") - Add validation when using filesystem storage Signed-off-by: McKnight22 <tao.wang.22@outlook.com> * refactor(cli): unify storage configuration for export command - Refactor storage field validation with macro Signed-off-by: McKnight22 <tao.wang.22@outlook.com> * refactor(cli): unify storage configuration for export command - support GCS Application Default Credentials (like GKE, Cloud Run, or local development with ) in export (Enabling ADC without validating or to be present) (Making optional in GCS validation (defaults to https://storage.googleapis.com)) Signed-off-by: McKnight22 <tao.wang.22@outlook.com> * refactor(cli): unify storage configuration for export command This commit refactors the validation logic for object store configurations in the CLI to leverage clap features and reduce boilerplate. Key changes: - Update wrap_with_clap_prefix macro to use clap's requires attribute. This ensures that storage-specific options (e.g., --s3-bucket) are only accepted when the corresponding backend is enabled (e.g., --s3). - Simplify FieldValidator trait by removing the is_provided method, as dependency checks are now handled by clap. - Introduce validate_backend! macro to standardize the validation of required fields for enabled backends. - Refactor ExportCommand to remove explicit validation calls (validate_s3, etc.) and rely on the validation within backend constructors. - Add integration tests for ExportCommand to verify build success with S3, OSS, GCS, and Azblob configurations. Signed-off-by: McKnight22 <tao.wang.22@outlook.com> * refactor(cli): unify storage configuration for export command - Use macros to simplify storage export implementation Signed-off-by: McKnight22 <tao.wang.22@outlook.com> Co-authored-by: WenyXu <wenymedia@gmail.com> * refactor(cli): unify storage configuration for export command - Rollback StorageExport trait implementation to not using macro for better code clarity and maintainability - Introduce format_uri helper function to unify URI formatting logic - Fix OSS URI path bug inherited from legacy code Signed-off-by: McKnight22 <tao.wang.22@outlook.com> Co-authored-by: WenyXu <wenymedia@gmail.com> * refactor(cli): unify storage configuration for export command - Remove unnecessary async_trait Signed-off-by: McKnight22 <tao.wang.22@outlook.com> Co-authored-by: jeremyhi <jiachun_feng@proton.me> --------- Signed-off-by: McKnight22 <tao.wang.22@outlook.com> Co-authored-by: WenyXu <wenymedia@gmail.com> Co-authored-by: jeremyhi <jiachun_feng@proton.me>
This commit is contained in:
1
Cargo.lock
generated
1
Cargo.lock
generated
@@ -1786,6 +1786,7 @@ dependencies = [
|
||||
"common-recordbatch",
|
||||
"common-runtime",
|
||||
"common-telemetry",
|
||||
"common-test-util",
|
||||
"common-time",
|
||||
"common-version",
|
||||
"common-wal",
|
||||
|
||||
Reference in New Issue
Block a user