rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-16 21:10:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,315 Commits 133 Branches 252 Tags
1.0.1_patch_opendal
Commit Graph

862 Commits

Author SHA1 Message Date
shuiyisong
65f1a41fbb chore: update the opendal to 0.56 rc2 (#8003) 
* chore: update opendal version

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: update opendal version

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: fix test

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: grpc init

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: dep versions

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: remove aws-lc-rs in reqwest

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: rebase main and fix compile

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: remove unused deps

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* Revert "fix: remove aws-lc-rs in reqwest"

This reverts commit 90bfafca06.

* chore: remove aws-lc-sys from blacklist

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: fix sqlness

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: add tls deps

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: idemptent install in rds

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: test

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: use aws-lc-sys as possible

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: lint

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: address comments

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: address CR issue

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: sync opendal compat adapter with upstream

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: address compat clippy warnings

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
 2026-04-27 10:37:29 +08:00
Weny Xu
8d2f92c01a chore: cherry pick fixes and bum version to v1.0.1 (#8024) 
* fix: remap peer addresses during retries (#7933)

* fix: remap peer addresses during retries

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: styling

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: using uint64 datatype for postgres prepared statement parameters (#7942)

* feat: add support for decimal parameter type, remove string replacement fallback

* chore: format

* fix: add support for using unsigned bigint in postgres

* chore: format toml

* refactor: cleanup duplicated code

* fix: rescale decimal

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix current version comparison logic for pre-releases (#7946)

Signed-off-by: liyang <daviderli614@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(index): intersect bitmaps before early exit in predicates applier (#7867)

* fix(index): intersect bitmaps before early exit in predicates applier

The loop skipped intersecting when the next bitmap was empty, which left
the accumulator unchanged instead of zeroing it. Intersect first, then
break when the result is empty.

Signed-off-by: Weixie Cui <cuiweixie@gmail.com>

* per gemini

* style(index): format predicates applier loop

* fix(index): remove unused mut in predicates applier

---------

Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
Co-authored-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: randomize standalone test ports in cli export test (#7955)

fix/flaky-test:
 ### Add Dynamic Port Selection for Standalone Tests

 - **`cli.rs`**: Implemented functions `random_standalone_addrs` and `choose_random_unused_port_offset` to dynamically select unused ports for standalone tests, enhancing test reliability.
 - Updated `test_export_create_table_with_quoted_names` to use dynamically assigned ports for HTTP, RPC, MySQL, and PostgreSQL addresses.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: fix git cliff errors in latest version (#7947)

* chore: fix git cliff errors in latest version

- Fix errors in v2.12.0
- Do not generate logs for beta/rc tags between the compared commits

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: preserve blank line before release date in changelog

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: match term zh (#7952)

* fix: match term zh

Signed-off-by: discord9 <discord9@163.com>

* chore: per gemini

Signed-off-by: discord9 <discord9@163.com>

* chore: revert accident change

Signed-off-by: discord9 <discord9@163.com>

* feat: unicode script han

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: set upload timeout for uploading artifacts to S3 (#7958)

* ci: set upload timeout for uploading artifacts to S3

Signed-off-by: liyang <daviderli614@gmail.com>

* Update upload-artifacts-to-s3.sh

---------

Signed-off-by: liyang <daviderli614@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: cargo check -p common-meta (#7964)

fix: moka feature
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: always skip field pruning when using merge mode (#7957)

* test: add prefilter regressions for last_row null filters

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: skip fields in all merge mode

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: simplify pre-filter skip fields handling

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: update test

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: mysql prepare correctly returns error instead of panic (#7963)

feat: mysql writer support multiple statement execution

Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: relax azblob validation requirements (#7970)

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(mito2): allow CompactionOutput to succeed independently (#7948)

* refactor(mito2): improve compaction error handling and file removal

Refactor compaction task execution to enhance error handling and robustness.
- Implemented parallel execution of compaction tasks with proper error capture and logging for individual task failures.
- Ensured JoinSnafu is no longer directly used in error propagation, instead handling errors within the task processing loop.
- Adjusted file removal logic to correctly include expired SSTs after compaction merges.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* refactor(mito2): extract SstMerger trait for testability in compaction

Extract SstMerger trait and DefaultSstMerger implementation to improve the testability of DefaultCompactor.

The DefaultCompactor is now generic over SstMerger, allowing mock implementations to be injected for unit testing without relying on the full object storage access layer. This refactoring separates the concerns of SST file merging from the overall compaction orchestration logic.

Additionally:
- Updated CompactionScheduler to use DefaultCompactor::default().
- Added unit tests for DefaultCompactor using a MockMerger.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix(compaction): propagate join error during sst flush

Correctly propagates the error when joining SST flush handles during compaction. Previously, the error was logged but not returned, leading to potential silent failures.
Also reorders some imports for consistency.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* perf(compaction): pre-allocate capacity for compacted_inputs

Pre-allocates capacity for the compacted_inputs vector based on the estimated total size of inputs and expired SSTs. This optimization aims to reduce vector reallocations during the compaction process.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* feat/allow-partial-compaction:
 ### Commit Message

 Enhance `DefaultCompactor` and `MockMerger` for Improved Flexibility

 - **`compactor.rs`**:
   - Added `Clone` trait to `DefaultSstMerger` and `MockMerger` to allow cloning.
   - Removed `Arc` wrapping from `DefaultCompactor`'s `merger` field for direct usage.
   - Updated `merge_ssts` method to require `Clone` trait for `SstMerger`.
   - Modified `MockMerger` to use `Arc<Mutex>` for `results` and `call_idx` to ensure thread safety.
   - Adjusted error handling to use `error::InvalidMetaSnafu` directly.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

---------

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: propagate staging leader through lease and heartbeat (#7950)

* feat(mito): expose staging leader role state

* fix(region): clear staging metadata on leader exit

* feat: propagate staging leader role through heartbeat and metasrv

* chore: update comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(region): unify staging exit role transitions

* chore: update proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: cancel local compaction for enter staging (#7885)

* feat(mito2): support cancelling active local compaction

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(mito2): cover compaction cancellation return paths

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: cancel remaining tasks

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: move group rollback ownership to parent repartition (#7967)

* refactor(meta-srv): move group rollback ownership to parent repartition procedure

- Parent procedure now owns partial rollback based on failed/unknown subprocedures
- rollback order: group metadata first, then allocated-region cleanup
- original_target_routes captured during build-plan, persisted in RepartitionPlanEntry
- rollback_group_metadata_routes moved to utils as parent-owned helper
- Group subprocedure no longer supports rollback (rollback_supported = false)
- Removed UpdateMetadata::RollbackStaging from group state machine
- Deleted redundant group rollback tests and helpers

BREAKING CHANGE: group Procedure no longer handles rollback; parent procedure
is responsible for crash recovery and selecting which plans to roll back.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: use PreFilterMode::All if only one source in the partition range (#7973)

* feat: use PrefilterMode::All if only one source in the partition range

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: consider append_mode

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: skip merge if only one source

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix test

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): renew operating region leases from keeper roles (#7971)

* refactor(meta): store operating region roles in memory keeper

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): register operating region roles from region routes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): require explicit operating region roles

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): renew operating region leases from keeper roles

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(common-meta): cover region route role helpers

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(meta): cover operating region role propagation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): prefer metadata roles for region lease renewal

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add an index page (#7975)

* feat: include an index page

* fix: address code review

* fix: let / auth gated

* refactor: rename public-apis to public-api-prefix

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: remove redundant error messages in admin functions (#7953)

Closes #7938

Signed-off-by: yxrxy <yxrxytrigger@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf: better jieba cut (#7984)

* perf: better jieba cut

Signed-off-by: discord9 <discord9@163.com>

* fix: also filter pun mark

Signed-off-by: discord9 <discord9@163.com>

* chore

Signed-off-by: discord9 <discord9@163.com>

* docs: explain why

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: allow ipv4_num_to_string to accept valid integers (#7994)

* fix: allow ipv4_num_to_string to accept valid integers

Signed-off-by: Johannes Sluis <joesluis51@gmail.com>

* test: update sqlness result file

Signed-off-by: Johannes Sluis <joesluis51@gmail.com>

* fix: use coercible integer signature for ipv4_num_to_string

Signed-off-by: Johannes Sluis <joesluis51@gmail.com>

---------

Signed-off-by: Johannes Sluis <joesluis51@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: update manifest state before deleting delta files (#8001)

* fix: update state before deleting deltas

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: update comment

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: update log level

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: upgrade mysql metadata value limit to mediumblob (#7985)

* fix: upgrade mysql metadata values to mediumblob

* fix: fail mysql metadata startup on upgrade check errors

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: zh same underscore behavior (#8002)

* fix: zh same underscore behavior

Signed-off-by: discord9 <discord9@163.com>

* fix: only add token with _ from en analyzer

Signed-off-by: discord9 <discord9@163.com>

* test: neg sqlness case

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: manifest recovery scans after last version if possible (#8009)

* feat: suppport scan with start after

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: add start_after test

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: adjust remove dir warning

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: test list_with_start_after

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: update get_paths call with start_after arg in checkpoint test

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: log scan metrics

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: fix start_after on manifest dir

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add a standalone flag in plugins during startup (#7974)

* chore: add a standalone flag in plugins during startup

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: add derive

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: bump version to v1.0.1

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: liyang <daviderli614@gmail.com>
Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: yxrxy <yxrxytrigger@gmail.com>
Signed-off-by: Johannes Sluis <joesluis51@gmail.com>
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
Co-authored-by: liyang <daviderli614@gmail.com>
Co-authored-by: cui <cuiweixie@gmail.com>
Co-authored-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
Co-authored-by: yxrxy <1532529704@qq.com>
Co-authored-by: Joe Sluis <43276756+JoeS51@users.noreply.github.com>
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>
 2026-04-23 17:37:27 +08:00
Yingwen
f3dbf34c74 chore: bump version to 1.0.0 (#7935) 
* chore: bump version to 1.0.0

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix sqlness test

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix cluster info sqlness

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: reorder regex in cluster_info

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: fix pg catalog

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-04-08 15:15:27 +00:00
discord9
b623cb1aa2 perf: no longer window sort when limit (#7912) 
* perf: no longer window sort when limit

Signed-off-by: discord9 <discord9@163.com>

* test: confusing vector sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: redact sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: redact every thing

Signed-off-by: discord9 <discord9@163.com>

* REDACTED

Signed-off-by: discord9 <discord9@163.com>

* what

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-04-08 02:54:22 +00:00
Yingwen
233e35c0c9 feat!: switch default sst format to flat (#7909) 
* feat: support alter from primary_key to flat

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: alter flat to primary_key

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: change default_experimental_flat_format to true

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: compute channel size from splitted batch size

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: add tests for split and channel size

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: always set sst_format from manifest on region open

sanitize_region_options did not set options.sst_format when the
default (PrimaryKey) matched the manifest value, leaving it as None
after reopen. This caused the alter format change to appear lost.

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix tests

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: show create table after alteration

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor!: rename default_experimental_flat_format to default_flat_format

The flat format is no longer experimental. Remove "experimental" from
the config field name, doc comments, and all references.

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: fix clippy

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-04-03 04:14:02 +00:00
Ning Sun
08ded45c7a feat: add common_version customization (#7869) 
* feat: add product name customization

* chore: update tests
 2026-03-26 10:18:06 +00:00
Ruihang Xia
59dd418629 feat: simplify nested aggr inside count query (#7859) 
* as optimizer rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* dump changes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* perf: tighten count-count optimizer rewrite

* extend inner op set

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify and more coverage

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove prom-non-null

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* preserve value column through pruning

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* more sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* enforce is not null before inner aggr

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* finalize

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-03-26 00:08:38 +00:00
Boudewijn van Groos
ec9d57cecc fix: nested views not working (#7857) 
Signed-off-by: Boudewijn van Groos <boudewijn.vangroos@foundationzero.org>
 2026-03-25 17:58:45 +00:00
discord9
30e895abbe fix: prom cast to f64 (#7840) 
* fix: cast to f64

Signed-off-by: discord9 <discord9@163.com>

* test: div case

Signed-off-by: discord9 <discord9@163.com>

* test: int test

Signed-off-by: discord9 <discord9@163.com>

* chore: sqlness update

Signed-off-by: discord9 <discord9@163.com>

* chore: test

Signed-off-by: discord9 <discord9@163.com>

* chore: update test

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-03-24 06:24:52 +00:00
dennis zhuang
223f6cfdf7 feat: supports sst_format for x-greptime-hints and database options (#7843) 
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-03-24 02:05:16 +00:00
Ning Sun
d14817bfa6 fix: resolve optimization issue for extended query (#7824) 
* fix: resolve optimization issue for extended query

* fix: type cast from subquery

* chore: update error information in sqlness

* chore: switch to released pgwire

* refactor: remove optimize function completely

* chore: add more tests

* test: attempt to fix the fuzz issue

* fix: try to resolve the test issue
 2026-03-20 03:58:39 +00:00
Ning Sun
306e8398cf fix: correct unicode representation for jsonb_to_string (#7810) 
* fix: correct unicode representation for jsonb_to_string

* refactor: correct function name and behavior

* fix: fix json_to_string and provide tests
 2026-03-16 03:01:02 +00:00
LFC
74ff5c37ea refactor: customize standalone instance build (#7807) 
* refactor: customize standalone instance build

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2026-03-13 09:25:21 +00:00
discord9
3beb538aa8 fix: rm useless analyzer (#7797) 
* fix: rm useless analyzer

Signed-off-by: discord9 <discord9@163.com>

* test: rm related test

Signed-off-by: discord9 <discord9@163.com>

* test: flow tql avg

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-03-12 10:53:47 +00:00
discord9
922f9cb3d6 feat: use dyn filter (#7545) 
* parent b2074e3863
author discord9 <discord9@163.com> 1767869295 +0800
committer discord9 <discord9@163.com> 1772529023 +0800

feat: use dyn filter

Signed-off-by: discord9 <discord9@163.com>

not supported

Signed-off-by: discord9 <discord9@163.com>

refactor: use make_mut instead

Signed-off-by: discord9 <discord9@163.com>

refactor: rm need to clone stream ctx

Signed-off-by: discord9 <discord9@163.com>

r

Signed-off-by: discord9 <discord9@163.com>

pcr

Signed-off-by: discord9 <discord9@163.com>

test: wait for datafusion update

Signed-off-by: discord9 <discord9@163.com>

refactor: use arc swap for dyn filters

Signed-off-by: discord9 <discord9@163.com>

* test: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: comment out sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness fix

Signed-off-by: discord9 <discord9@163.com>

* refactor: predicate without option

Signed-off-by: discord9 <discord9@163.com>

* feat: print dyn filters& more tests

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness vector result update

Signed-off-by: discord9 <discord9@163.com>

* chore: log

Signed-off-by: discord9 <discord9@163.com>

* test: properly redact

Signed-off-by: discord9 <discord9@163.com>

* test: better data dist for non empty dyn filter

Signed-off-by: discord9 <discord9@163.com>

* test: properly redacted

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* properly redact

Signed-off-by: discord9 <discord9@163.com>

* docs:  explain why not  do it

Signed-off-by: discord9 <discord9@163.com>

* chore: rename update to add as its more proper

Signed-off-by: discord9 <discord9@163.com>

* chore: rm no need clone

Signed-off-by: discord9 <discord9@163.com>

* docs: per review

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-03-11 03:06:24 +00:00
Ning Sun
9b1288f66a feat: add function rewrite rule for json_get with cast (#7631) 
* feat: initial function rewriter for json_get

* feat: make sure rewrite rule is applied

* feat: keep analyzer's default rules

* feat: implement rewriter for arrow_cast

* test: add unit test for tht rewriter

* chore: format

* refactor: extract some more functions

* Apply suggestion from @waynexia

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2026-03-09 02:39:48 +00:00
discord9
d1151b665b feat: flow tql cte (#7702) 
* feat: flow tql cte

Signed-off-by: discord9 <discord9@163.com>

* fix: creating flow TQL CTE source tables lose cte part in query

Signed-off-by: discord9 <discord9@163.com>

* test: update sqlness result

Signed-off-by: discord9 <discord9@163.com>

* chore

Signed-off-by: discord9 <discord9@163.com>

* fix: properly canonicalize ident

Signed-off-by: discord9 <discord9@163.com>

* feat: even stricter check

Signed-off-by: discord9 <discord9@163.com>

* chore: sqlness update

Signed-off-by: discord9 <discord9@163.com>

* chore: after rebase fix

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-03-06 03:36:42 +00:00
Ruihang Xia
39140058d0 feat: adapt new name of holt winters fn (#7700) 
* feat: adapt new name of holt winters fn

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update parser

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* alias old fn name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-03-04 23:32:37 +00:00
LFC
b2074e3863 chore: upgrade DataFusion family, again (#7578) 
* chore: upgrade DataFusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: switch to released version of datafusion-pg-catalog

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
 2026-03-03 07:36:39 +00:00
Ning Sun
0f6b8ff815 feat: implement postgres copy to stdout (#7709) 
* feat: update pgwire

* feat: add special parser for copy to stdout

* feat: implement copy to stdout

* fix: improve code

* fix: expect optional with

* fix: lint

* feat: correct encoder using and refactor

* chore: fmt

* refactor: update api

* chore: use released dependencies

* fix: update datafusion-pg-catalog to support schema query

* fix: support for double quoted identifier

* feat: update datafusion-postgres to support schema.table

* refactor: use pgsqlparser container

* refactor: remove unquote which is no longer needed

* fix: correctly handle invalid query

* fix: correct handle null in nano timestamp

* test: add a new test for additional close )
 2026-02-26 03:41:33 +00:00
dennis zhuang
6d998c043e feat: implements anomaly_score_iqr, anomaly_score_mad etc. (#7681) 
* feat: implements anomaly_score_iqr, anomaly_score_mad and anomaly_score_zscore functions

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: forgot result file

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* feat: return NaN and zero for scale=0

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: comment

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: signature, cast and degenerate case

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: tweak comment

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: format

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* feat: update min samples threshold

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: missing closing quote

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: mock engine

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: oudated comment and test name

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: todo

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: apply suggestions

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-25 07:01:22 +00:00
Yingwen
42ad842434 feat: support changing table's append_mode to true (#7669) 
* feat: support alter append_mode to true

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: add sqlness test

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: remove comment

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: fix compiler errors

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: clear merge mode in mito when setting append mode

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: sanitize open request and options with both append/merge mode

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: clear merge mode when append mode is true

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-02-25 04:11:23 +00:00
Ning Sun
bcfbd01582 fix: use full DDL of flow in information_schema.flows.flow_definition (#7704) 
* fix: use full DDL of flow in information_schema.flows.flow_definition

* fix: add schema name in sink table
 2026-02-12 00:09:40 +00:00
Ruihang Xia
77013d9085 feat: report flow stats from streaming and batching engines (#7701) 
* fix: report flow stats from streaming and batching engines

* handle restart report handler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename fn name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-02-11 07:38:09 +00:00
Ruihang Xia
db46849f40 feat: track flow source tables for TQL and info schema (#7697) 
* feat: track flow source tables for TQL and info schema

* handle schema matcher

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sqlness tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* cover __name__ case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-02-11 03:19:09 +00:00
fys
1aa80d9363 fix: incorrect-tql-explain result (#7675) 2026-02-11 02:30:15 +00:00
LFC
8c23b29725 refactor: remove the RawTableMeta and RawTableInfo to make codes more concise (#7626) 
* refactor: remove the `RawTableMeta` and `RawTableInfo` to make codes more concise

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2026-02-10 07:10:04 +00:00
Ruihang Xia
4ac73a7e9e feat: adapt prometheus 3.x selector and lookback behavior (#7688) 
* feat: adapt prometheus 3.x selector and lookback behavior

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove commit hash from comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-02-10 04:00:45 +00:00
Ruihang Xia
a10d40e552 feat: adapt prometheus 3.x matrix selector behavior change (#7671) 
feat: adapt prometheus 3.x matrix selector behavior

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-02-06 03:06:33 +00:00
dennis zhuang
8883022742 refactor(vector-index): use protobuf for metadata and align code (#7648) 
* refactor(vector-index): use protobuf for metadata and introduce lifecycle traits

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: minor change

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: by suggestions

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: format

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: style

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: remove usearch from mito2

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: tweak errors

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: update index size in result

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: clippy

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: update proto deps

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-02-05 02:41:48 +00:00
Lin Yihai
64ac9b60e9 feat(copy_to_json): add date_format/timestamp_format/time_format for JSON format copy (#7633) 
* feat(copy_to_json): add `date_format`/`timestamp_format`/`time_format` for  JSON format.

Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>

* Update src/common/datasource/src/file_format/json.rs

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>

* chore: Use predefined constants as the time format.

Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>

---------

Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-04 13:30:41 +00:00
Ruihang Xia
26f7c12ffd fix: drop rhs columns on promql filter join (#7665) 
* fix: drop rhs columns on promql filter join

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* choose instant vector to operate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-02-04 13:12:20 +00:00
discord9
f3882aac88 chore: use forked sqlness (#7664) 
* use forked sqlness

Signed-off-by: discord9 <discord9@163.com>

* merge

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-02-04 11:46:19 +00:00
Ruihang Xia
2dc4c294cf feat: support group by op in promql (#7663) 
* feat: support group by op in promql

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* restrict to single field table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-02-04 09:43:17 +00:00
Ruihang Xia
f9030a84c8 fix: incorrect column statistics of promql plans (#7662) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-02-04 04:58:38 +00:00
dennis zhuang
c08f3a4472 test: adds sqlness test for vector index (#7634) 
* test: adds sqlness test for vector index

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: CI

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: redacted flat map and size

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: simplify the replace rules

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: update comments and tests

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-02-04 03:54:47 +00:00
Weny Xu
969a64d483 fix(copy): treat null inferred columns as Utf8 to preserve later values (#7658) 
Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-02-03 10:20:35 +00:00
Weny Xu
1a594f3d1b chore: bump version rc.1 (#7645) 
* chore: bump version rc.1

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update sqlness result

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-02-02 02:49:31 +00:00
discord9
a6e69bc180 feat: flow last non null (#7646) 
* feat: flow last non null support

Signed-off-by: discord9 <discord9@163.com>

* clippy

Signed-off-by: discord9 <discord9@163.com>

* test: report error when sink is not last non null

Signed-off-by: discord9 <discord9@163.com>

* fix: error if query column matches nothing if partial

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-01-30 12:34:33 +00:00
Ruihang Xia
c83868c4eb feat: partition rule simplifier (#7622) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reuse collider

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify range helpers

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* notes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update unit test resule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-01-27 14:31:20 +00:00
Ruihang Xia
65f46b6a2f feat: ensure datanode plan's dist and ordering for tsid col (#7614) 
* ensure datanode plan's dist and ordering

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add one more comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-01-26 11:36:59 +00:00
Ning Sun
079ee8615f feat: UDF json_get with user specified return type (#7554) 
* feat: add return_field_from_args

* feat: add JsonGetWithType

* port json_get_float and json_get_bool to new implementation, add
json_get with third argument accepting a scalar value for type.

* fix: lint fix

* chore: add sqlness tests

* chore: update tests
 2026-01-26 09:53:45 +00:00
discord9
085e9dfc7b fix: use phy table when need fitler by tsid/table id (#7609) 
* fix: use phy table when need fitler by tsid/table id

Signed-off-by: discord9 <discord9@163.com>

* feat: add subquery alias

Signed-off-by: discord9 <discord9@163.com>

* test: update after alias

Signed-off-by: discord9 <discord9@163.com>

* test: sort&redact

Signed-off-by: discord9 <discord9@163.com>

* test: per review

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2026-01-26 06:11:16 +00:00
Ruihang Xia
2f82e7525f fix: correct histogram result (#7608) 
correct result by removing tsid

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-01-23 07:46:46 +00:00
Ruihang Xia
cc1dbd108b perf: skip unnecessary label columns in promql query (#7602) 
* feat: fully install tsid to promql planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sqlness tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unused field

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-01-21 13:31:23 +00:00
Ruihang Xia
966ade7565 feat: use tsid on promql planner (#7590) 
* expose tsid on logical table's schema and use it on planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* detect table type on planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplification

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* filter out internal columns

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reset tsid flag

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-01-21 06:25:36 +00:00
Weny Xu
25687bb282 feat: add ddl timeout/wait options, repartition WITH parsing, meta-client startup refactor (#7589) 
* feat: add ddl request timeouts and unify meta client startup

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: omplement ALTER TABLE repartition DDL options parsing

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add sqlness tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: pass timeout argument to procedure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: refine comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: assert timeout

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-01-20 09:26:53 +00:00
discord9
aa3daf7053 fix: read filter's column (#7579) 
* fix: read columns need for filter

Signed-off-by: discord9 <discord9@163.com>

* c

Signed-off-by: discord9 <discord9@163.com>

* feat: add support for explicit read columns in projection mappers

Signed-off-by: discord9 <discord9@163.com>

* test: add compatibility tests for projection mappers

Signed-off-by: discord9 <discord9@163.com>

* c

Signed-off-by: discord9 <discord9@163.com>

* fix: rename variable for clarity and improve column ID retrieval logic

Signed-off-by: discord9 <discord9@163.com>

* fix: update scan input construction to include read column IDs

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness for projection filter

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

* chore: more redacting

Signed-off-by: discord9 <discord9@163.com>

* chore: more redact

Signed-off-by: discord9 <discord9@163.com>

* c

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-01-20 08:19:50 +00:00
Weny Xu
2ae20daa62 feat: add sync region instruction for repartition procedure (#7562) 
* feat: add sync region instruction for repartition procedure

This commit introduces a new sync region instruction and integrates it
into the repartition procedure flow, specifically for metric engine tables.

Changes:
- Add SyncRegion instruction type and SyncRegionsReply in instruction.rs
- Implement SyncRegionHandler in datanode to handle sync region requests
- Add SyncRegion state in repartition procedure to sync newly allocated regions
- Integrate sync region step after enter_staging_region for metric engine tables
- Add sync_region flag and allocated_region_ids to PersistentContext
- Make SyncRegionFromRequest serializable for instruction transmission
- Add test utilities and mock support for sync region operations

The sync region step is conditionally executed based on the table engine type,
ensuring that newly allocated regions in metric engine tables are properly
synced from their source regions before proceeding with manifest remapping.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add logs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(repartition): improve staging region handling and support metric engine repartition
- Reorder sync region flow: move SyncRegion from EnterStagingRegion to RepartitionStart to sync before applying staging
- Add ExitStaging metadata update state to properly clear staging leader info after repartition completes
- Update build_template_from_raw_table_info to optionally skip metric engine internal columns when creating region requests
- Fix region state transition: set_dropping now expects specific state (Staging or Writable) for proper validation
- Adjust region drop and copy handlers to handle staging regions correctly
- Add comprehensive test cases for metric engine SPLIT/MERGE partition operations on physical tables with logical tables
- Improve logging for table route updates, region drops, and repartition operations

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: removes code duplication

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: update result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: refine comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add error strategy support for flush region and flush pending deallocate regions

- **Add `ErrorStrategy` enum** in `procedure/utils.rs`:
  - Supports `Ignore` and `Retry` strategies for error handling
  - Refactor `flush_region` to accept `error_strategy` parameter
  - Extract `handle_flush_region_reply` helper function for better code organization

- **Add pending deallocate region support**:
  - Add `pending_deallocate_region_ids` field to `PersistentContext`
  - Implement `flush_pending_deallocate_regions` in `EnterStagingRegion` state
  - Flush pending deallocate regions before entering staging regions to ensure data consistency

- **Update error handling**:
  - `flush_leader_region`: Use `ErrorStrategy::Ignore` to skip unreachable datanodes
  - `sync_region`: Use `ErrorStrategy::Retry` for critical operations
  - `enter_staging_region`: Use `ErrorStrategy::Retry` when flushing pending deallocate regions

This change improves the robustness of the repartition procedure by:
1. Providing flexible error handling strategies for flush operations
2. Ensuring pending deallocate regions are properly flushed before repartitioning
3. Preventing data inconsistency during region migration

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: compile

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-01-15 04:52:57 +00:00
LFC
e64c31e59a chore: upgrade DataFusion family (#7558) 
* chore: upgrade DataFusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* use main proto

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2026-01-14 14:02:31 +00:00