rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-08-19 12:38:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,834 Commits 164 Branches 267 Tags
aec7ba34c9b5dfc482e98565484fd1583f45151a
Commit Graph

100 Commits

Author SHA1 Message Date
Lei, HUANG 4873fc4f18 feat(meta): add retention GC for soft-dropped tables (#8526) 
* feat(meta): persist soft-drop retention metadata

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): complete soft-drop retention coverage

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): finalize soft-drop metadata after prepare

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(meta): purge expired soft-dropped tables

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): keep soft-drop GC responsive

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): deduplicate soft-drop purge tasks

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): release purge reservations safely

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* refactor(meta): simplify soft-drop GC wiring

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(meta): persist soft-drop retention deadlines

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): retain soft-drop config for recovery

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: complete standalone DDL test context

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): recheck retention before automatic purge

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): keep experimental soft drop disabled

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): rotate soft-drop purge candidates

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): address soft-drop retention review

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): protect retained table tombstones

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): preserve post-cleanup purge state

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): fence purge by drop generation

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
 2026-07-16 08:47:03 +00:00
jeremyhi 619460f742 test(cli): add minio export-import v2 e2e (#8314) 
* test(cli): add minio export-import v2 e2e

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): reuse e2e connection helper in minio test

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): avoid stderr in minio e2e cleanup

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): report minio snapshot cleanup failures

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: run import-v2 minio e2e

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): cleanup minio snapshot on failure

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-24 09:25:10 +00:00
jeremyhi f74dfacfad test(cli): harden incomplete import-v2 snapshot e2e (#8313) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-23 06:55:21 +00:00
jeremyhi 47581acbd3 test(cli): harden import-v2 schema filter e2e (#8312) 
* test(cli): harden import-v2 schema filter e2e

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): assert filtered export includes both schemas

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): assert import-v2 schema filter result

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-22 12:23:28 +00:00
jeremyhi a3e01e0018 test(cli): add import-v2 resume e2e coverage (#8311) 
* test(cli): harden export import v2 e2e roundtrip

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): add import-v2 resume e2e coverage

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): make import-v2 resume replay check non-idempotent

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): share export import e2e setup helpers

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): assert import-v2 resume row identities

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-18 09:02:24 +00:00
jeremyhi 3e36778b0e test(cli): harden export/import-v2 e2e roundtrip (#8310) 
test(cli): harden export import v2 e2e roundtrip

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-18 07:41:54 +00:00
Weny Xu e520ff8300 feat: decouple error retryability from status codes (#8301) 
* feat: add retry hint to common error

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: propagate retry hints in core errors

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: propagate retry hint through RPC metadata

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fallback retry hints to status codes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: add explicit retry hints for retryable errors

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: remove status code retry fallback

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: classify io retry hints

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: preserve retry hints across error wrappers

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: minior

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: preserve datasource retry hints in query

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: preserve metric engine retry hints

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: preserve flow and frontend retry hints

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: share rskafka retry hint mapping

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-06-17 08:27:53 +00:00
jeremyhi fd6c9e710d feat(cli): allow overriding import-v2 state path (#8302) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-16 09:53:19 +00:00
jeremyhi c22f5d741b feat(cli): add import-v2 task parallelism (#8300) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-16 02:55:52 +00:00
jeremyhi 90606af070 feat(cli): add export-v2 progress reporting (#8294) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-15 09:56:31 +00:00
Weny Xu 15597047c9 feat: pass Kafka pruned entry id when creating regions (#8282) 
* refactor: use typed region wal options

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: pass kafka pruned entry id to regions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: remove unused error

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: clarify wal options serialization errors

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: cover legacy region wal options encoding

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: recover legacy create table wal options

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: lock remote wal during table creation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix unit tests-s

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: refresh remote wal prune hints

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-06-15 08:51:50 +00:00
jeremyhi 336e4cdb84 feat(cli): add export-v2 chunk parallelism (#8292) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-15 06:00:53 +00:00
jeremyhi 34c4cae9bb feat(cli): add import-v2 progress UI (#8289) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-12 09:45:45 +00:00
jeremyhi c567a6562a feat(cli): add import-v2 progress mode (#8283) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-12 06:13:27 +00:00
jeremyhi 0a2dc6d3f2 feat(cli): add export import progress abstraction (#8270) 
Signed-off-by: fengjiachun <3860496+fengjiachun@users.noreply.github.com>
Co-authored-by: fengjiachun <3860496+fengjiachun@users.noreply.github.com>
 2026-06-10 11:35:32 +00:00
jeremyhi 6ceae4498f refactor(cli): stream export-v2 verify data listing (#8268) 
Signed-off-by: fengjiachun <fengjiachun@gmail.com>
 2026-06-10 02:59:27 +00:00
jeremyhi aaffe1a048 refactor(cli): split export-v2 verify into plan/scan/reconcile (#8206) 
* refactor(cli): split export-v2 verify into plan/scan/reconcile

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-06-03 03:10:10 +00:00
jeremyhi c84462bdc1 feat(cli): add export-v2 delete command (#8162) 
* feat(cli): add export-v2 celete command

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat(cli): refine delete confirmation flag

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-05-26 04:28:23 +00:00
jeremyhi 72434ee5d7 feat: add export-v2 snapshot verification (#8111) 
* feat: add export-v2 snapshot verification

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: reject full export snapshots without chunks

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: enforce strict export snapshot data mainifest

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-05-19 07:15:51 +00:00
jeremyhi 0518567e5a feat: add export-v2 snapshot listing (#8096) 
* feat: add export-v2 snapshot listing

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: allow export-v2 list at storage roots

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-05-13 03:04:04 +00:00
jeremyhi d709fd29ef feat: import resume part2 (#8070) 
* feat: import resume part2

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: condig docs

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-05-12 02:56:52 +00:00
shuiyisong bdaad9db77 refactor: extract functions for building mysql/pg's kvbackend and electionref (#8076) 
chore: extract functions for building mysql/pg's kvbackend and electionref

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
 2026-05-07 08:48:49 +00:00
jeremyhi 266734d7ef fix: win cli lock (#8049) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-04-29 10:02:06 +00:00
shuiyisong 0effc30778 chore: update the opendal to 0.56 rc2 (#8003) 
* chore: update opendal version

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: update opendal version

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: fix test

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: grpc init

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: dep versions

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: remove aws-lc-rs in reqwest

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: rebase main and fix compile

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: remove unused deps

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* Revert "fix: remove aws-lc-rs in reqwest"

This reverts commit 90bfafca06f877befb36f3e54bb72fcfc4c56778.

* chore: remove aws-lc-sys from blacklist

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: fix sqlness

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: add tls deps

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: idemptent install in rds

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: test

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: use aws-lc-sys as possible

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: lint

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: address comments

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: address CR issue

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: sync opendal compat adapter with upstream

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: address compat clippy warnings

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
 2026-04-26 09:59:48 +00:00
jeremyhi 10f3f59a01 test(cli): harden import state lock test (#8030) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-04-24 06:38:05 +00:00
jeremyhi e316797fff feat(cli): add retry and import state fundations (#8007) 
* feat(cli): add retry and import state fiundations

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by human comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-04-23 04:38:36 +00:00
Weny Xu 39f9c7b4a4 feat(cli): add metadata put key and table commands (#7989) 
* feat: add validated metadata put cli

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add table metadata put commands

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: share metadata table selector

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(cli): read metadata put values from stdin

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-04-23 04:34:05 +00:00
Weny Xu 24e3364142 fix: relax azblob validation requirements (#7970) 2026-04-15 08:08:39 +00:00
jeremyhi e3f7ea8783 feat(cli): implement import-v2 data import pipeline (#7898) 
* feat(cli): implement import-v2 data import pipeline

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: cargo fmt

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by AI comments

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(cli): harden import-v2 snapshot validation

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: excape sql

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(cli): redact escaped secrets in copy sql logs

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cli): tighten export v2 e2e helpers

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(cli): log execution time

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-04-14 03:15:36 +00:00
jeremyhi f0ea87f52f fix: windows ci (#7905) 
* fix: windows ci

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: typo

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: use common create_temp_dir

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-04-03 02:17:42 +00:00
jeremyhi ab10696657 feat: implement export-v2 chunked data export flow (#7841) 
* feat: implement export-v2 chunked data export flow

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by codex comment

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by gemini comment

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: clippy

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: by comment

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: handle empty export ranges consistently

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: validate resume config

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: file-uri paths

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat: check args on schema-only mode

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-03-31 22:43:40 +00:00
Ning Sun e14404c677 chore: update rust toolchain to 2026-03-21 (#7849) 
* chore: update rust toolchain to 2026-03-21

* chore: new format

* fix: lint

* chore: resolve lint issues

* chore: remove as_millis_f64

* chore: deps up
 2026-03-30 12:13:14 +00:00
jeremyhi 805536aed1 fix: windows file path (#7839) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-03-20 08:19:41 +00:00
jeremyhi 16fcbb2729 feat: export import v2 pr1 (#7785) 
* feat: v2 schema handling

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat: impl m1.5 ddl export/import and schema tests

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: git ignore update

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: add license header

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: make fmt-check happy

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: Run imported DDL against the intended schema

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: Canonicalize schema names after case-insensitive check

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: escape sql funcs

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: Fixed by carrying explicit execution_schema in DdlStatement instead of parsing schema from SQL

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: Fixed by encoding schema names as safe path segments in shared DDL path helpers

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor(cli): make export/import v2 schema recovery DDL-only

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: by clippy

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: follow our styling

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(cli): reject remote snapshot URIs with empty root

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(cli): dedupe schema filters after canonicalization

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(cli): schema-scoped detection to cover external tables

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-03-19 21:26:41 +00:00
Weny Xu b17947b5d8 feat: add COPY support for GCS/Azblob with OSS/GCS/Azblob integration coverage (#7743) 
* feat: add gcs and azblob backends to copy datasource

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add copy to/from oss integration coverage

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add copy to/from gcs/azure integration coverage

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: use existing layers

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: hide `credential_path` in cli

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: remove logs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-03-04 04:21:21 +00:00
dependabot[bot] b0fb4abbdf chore(deps): bump tracing-subscriber from 0.3.19 to 0.3.20 (#7720) 
* chore(deps): bump tracing-subscriber from 0.3.19 to 0.3.20

Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.19 to 0.3.20.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.19...tracing-subscriber-0.3.20)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-version: 0.3.20
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: remove ansi term depdenncy

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
 2026-02-25 23:56:55 +00:00
Weny Xu df04267c54 fix(repartition): reject writes on deallocating regions during region merge (#7694) 
* feat(meta): add write route policy to region route with backward compatibility

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): use partition_expr compatibility accessor in repartition matching

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(meta): introduce staging partition rule enum for repartition instructions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(datanode): plumb staging partition rule enum through heartbeat handlers

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(meta): mark pending-deallocate regions as reject-all during merge staging

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(partition): exclude reject-all regions from write partitioning

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(mito): store staging partition rule enum in region state

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(mito): reject writes in staging when partition rule is reject-all

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(meta): send enter staging instruction with reject-all

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(repartition): preserve reject-all on exit, merge enter-staging instructions, and allow staged bulk writes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: refactor to ignore all writes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: rename StagingPartitionRule to StagingPartitionDirective across staging flow

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: clippy

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: nit

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: rename

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-02-25 07:04:38 +00:00
LFC 8c23b29725 refactor: remove the RawTableMeta and RawTableInfo to make codes more concise (#7626) 
* refactor: remove the `RawTableMeta` and `RawTableInfo` to make codes more concise

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2026-02-10 07:10:04 +00:00
LFC fcfab97d8c feat: cli tool to repair partition columns mismatch (#7605) 
* feat: cli tool to repair partition columns mismatch

Signed-off-by: luofucong <luofc@foxmail.com>

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* fix: harden partition column repair

Signed-off-by: MichaelScofield <MichaelScofield@users.noreply.github.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: MichaelScofield <MichaelScofield@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-09 02:58:25 +00:00
fys 8f48d233ef fix: unit tests for export module (#7607) 
fix: unit test about export
 2026-01-23 06:11:33 +00:00
Weny Xu 25687bb282 feat: add ddl timeout/wait options, repartition WITH parsing, meta-client startup refactor (#7589) 
* feat: add ddl request timeouts and unify meta client startup

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: omplement ALTER TABLE repartition DDL options parsing

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add sqlness tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: pass timeout argument to procedure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: refine comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: assert timeout

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-01-20 09:26:53 +00:00
Weny Xu 4e35028e28 test(cli): fix snapshot test paths on Windows (#7574) 
* test(cli): fix snapshot test paths on Windows

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix bugs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: styling

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-01-16 06:54:24 +00:00
Weny Xu ed171d9264 feat(cli): support RaftEngine store backend for metadata snapshot operations (#7467) 
* feat(cli): support RaftEngine store backend for metadata snapshot operations

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: allow empty aksk

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-01-14 02:24:36 +00:00
Weny Xu f3e2d333e4 feat(repartition): implement region allocation for repartition procedure (#7534) 
* refactor: rename WalOptionsAllocator to WalProvider

The name "WalOptionsAllocator" was misleading because:
- For RaftEngine variant, it doesn't actually allocate anything
- The actual allocation logic lives in KafkaTopicPool

"WalProvider" better describes its role as providing WAL options
based on the configured WAL backend (RaftEngine or Kafka).

Changes:
- Rename `WalOptionsAllocator` to `WalProvider`
- Rename `WalOptionsAllocatorRef` to `WalProviderRef`
- Rename `build_wal_options_allocator` to `build_wal_provider`
- Rename module `wal_options_allocator` to `wal_provider`
- Rename error types: `BuildWalOptionsAllocator` -> `BuildWalProvider`,
  `StartWalOptionsAllocator` -> `StartWalProvider`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): extract allocator traits from TableMetadataAllocator

Refactor TableMetadataAllocator to use trait-based dependency injection
for better testability and separation of concerns.

Changes:
- Add `ResourceIdAllocator` trait to abstract ID allocation
- Add `WalOptionsAllocator` trait to abstract WAL options allocation
- Implement traits for `Sequence` and `WalProvider`
- Remove duplicate `allocate_region_wal_options` function
- Rename `table_id_sequence` to `table_id_allocator` for consistency
- Rename `TableIdSequenceHandler` to `TableIdAllocatorHandler`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(meta): add max_region_number tracking to PhysicalTableRouteValue

Add `max_region_number` field to track the highest region number ever
allocated for a table. This value only increases when regions are added
and never decreases when regions are dropped, ensuring unique region
numbers across the table's lifetime.

Changes:
- Add `max_region_number` field to `PhysicalTableRouteValue`
- Implement custom `Deserialize` for backward compatibility
- Update `update_region_routes` to maintain max_region_number
- Calculate max_region_number from region_routes in `new()`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: extract TableRouteAllocator trait from TableMetadataAllocator

- Add TableRouteAllocator trait for abstracting region route allocation
- Implement blanket impl for all PeerAllocator types
- Add PeerAllocator impl for Arc<T> to support trait object delegation
- Update TableMetadataAllocator to use TableRouteAllocatorRef

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: rename TableRouteAllocator to RegionRoutesAllocator

- Rename table_route.rs to region_routes.rs
- Rename TableRouteAllocator trait to RegionRoutesAllocator
- Rename wal_option.rs to wal_options.rs for consistency
- Update TableMetadataAllocator to use new naming

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(meta-srv): implement region allocation for repartition procedure

This commit implements the region allocation phase of the repartition procedure,
which handles allocating new regions when a table needs to be split into more partitions.

Key changes:
- Refactor `RegionRoutesAllocator::allocate` to accept `(region_number, partition_expr)` tuples
  for more flexible region number assignment
- Simplify `AllocationPlanEntry` by removing `regions_to_allocate` and `regions_to_deallocate`
  fields (now derived from source/target counts)
- Add `convert_allocation_plan_to_repartition_plan` function to handle allocation, equal,
  and deallocation cases
- Fix `RepartitionPlanEntry::allocate_regions()` to return target regions (was incorrectly
  returning source regions)
- Implement complete `AllocateRegion` state with:
  - Region route allocation via `RegionRoutesAllocator`
  - WAL options allocation via `WalOptionsAllocator`
  - Operating region registration for concurrency control
  - Region creation on datanodes via `CreateTableExecutor`
  - Table route metadata update
- Add `TableRouteValue::max_region_number()` helper method
- Add comprehensive unit tests for plan conversion and allocation logic

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-01-08 11:03:58 +00:00
Weny Xu ada4666e10 refactor: remove region_numbers from TableMeta and TableInfo (#7519) 
* refactor: remove `region_numbers` from `TableMeta` and `TableInfo`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: create partitions from region route

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix build

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-01-06 13:21:36 +00:00
Weny Xu 9343da7fe8 feat(meta-srv): fallback to non-TLS connection when etcd TLS prefer mode fail (#7507) 
* feat(meta-srv): fallback to non-TLS connection when etcd TLS prefer mode fail

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore(ci): set timeout for deploy cluster

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: simplify etcd TLS prefer mode handling

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-12-31 10:03:34 +00:00
Lanqing Yang 8d9afc83e3 feat: allow auto schema creation for pg (#7459) 
Signed-off-by: lyang24 <lanqingy93@gmail.com>
 2025-12-23 08:55:24 +00:00
Weny Xu 4d66bd96b8 feat: make distributed time constants and client timeouts configurable (#7433) 
Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-12-19 02:23:20 +00:00
shuiyisong c088d361a4 chore: expose disable_ec2_metadata option (#7439) 
chore: add option for disable ec2 metadata

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
 2025-12-18 11:55:08 +00:00
McKnight22 72eede8b38 refactor(cli): unify storage configuration for export command (#7280) 
* refactor(cli): unify storage configuration for export command

- Utilize ObjectStoreConfig to unify storage configuration for export command
- Support export command for Fs, S3, OSS, GCS and Azblob
- Fix the Display implementation for SecretString always returned the string
  "SecretString([REDACTED])" even when the internal secret was empty.

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>

* refactor(cli): unify storage configuration for export command

- Change the encapsulation permissions of each configuration
  options for every storage backend to public access.

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>

* refactor(cli): unify storage configuration for export command

- Update the implementation of ObjectStoreConfig::build_xxx() using macro solutions

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>

* refactor(cli): unify storage configuration for export command

- Introduce config validation for each storage type

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>

* refactor(cli): unify storage configuration for export command

- Enable trait-based polymorphism for storage type handling
  (from inherent impl to trait impl)
- Extract helper functions to reduce code duplication

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>

* refactor(cli): unify storage configuration for export command

- Improve SecretString handling and validation
  (Distinguishing between "not provided" and "empty string")
- Add validation when using filesystem storage

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>

* refactor(cli): unify storage configuration for export command

- Refactor storage field validation with macro

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>

* refactor(cli): unify storage configuration for export command

- support GCS Application Default Credentials (like GKE, Cloud Run, or local development with ) in export
  (Enabling ADC without validating  or  to be present)
  (Making  optional in GCS validation (defaults to https://storage.googleapis.com))

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>

* refactor(cli): unify storage configuration for export command

This commit refactors the validation logic for object store configurations in the CLI to leverage clap features and reduce boilerplate.

Key changes:
- Update wrap_with_clap_prefix macro to use clap's requires attribute.
  This ensures that storage-specific options (e.g., --s3-bucket) are only accepted when the corresponding backend is enabled (e.g., --s3).
- Simplify FieldValidator trait by removing the is_provided method, as dependency checks are now handled by clap.
- Introduce validate_backend! macro to standardize the validation of required fields for enabled backends.
- Refactor ExportCommand to remove explicit validation calls (validate_s3, etc.) and rely on the validation within backend constructors.
- Add integration tests for ExportCommand to verify build success with S3, OSS, GCS, and Azblob configurations.

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>

* refactor(cli): unify storage configuration for export command

- Use macros to simplify storage export implementation

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>

* refactor(cli): unify storage configuration for export command

- Rollback StorageExport trait implementation to not using macro for better code clarity and maintainability
- Introduce format_uri helper function to unify URI formatting logic
- Fix OSS URI path bug inherited from legacy code

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>

* refactor(cli): unify storage configuration for export command

- Remove unnecessary async_trait

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>

---------

Signed-off-by: McKnight22 <tao.wang.22@outlook.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
 2025-12-18 03:16:53 +00:00