rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-15 20:40:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,315 Commits 132 Branches 252 Tags
1.0.1_patch_opendal
Commit Graph

54 Commits

Author SHA1 Message Date
Weny Xu
8d2f92c01a chore: cherry pick fixes and bum version to v1.0.1 (#8024) 
* fix: remap peer addresses during retries (#7933)

* fix: remap peer addresses during retries

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: styling

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: using uint64 datatype for postgres prepared statement parameters (#7942)

* feat: add support for decimal parameter type, remove string replacement fallback

* chore: format

* fix: add support for using unsigned bigint in postgres

* chore: format toml

* refactor: cleanup duplicated code

* fix: rescale decimal

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix current version comparison logic for pre-releases (#7946)

Signed-off-by: liyang <daviderli614@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(index): intersect bitmaps before early exit in predicates applier (#7867)

* fix(index): intersect bitmaps before early exit in predicates applier

The loop skipped intersecting when the next bitmap was empty, which left
the accumulator unchanged instead of zeroing it. Intersect first, then
break when the result is empty.

Signed-off-by: Weixie Cui <cuiweixie@gmail.com>

* per gemini

* style(index): format predicates applier loop

* fix(index): remove unused mut in predicates applier

---------

Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
Co-authored-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: randomize standalone test ports in cli export test (#7955)

fix/flaky-test:
 ### Add Dynamic Port Selection for Standalone Tests

 - **`cli.rs`**: Implemented functions `random_standalone_addrs` and `choose_random_unused_port_offset` to dynamically select unused ports for standalone tests, enhancing test reliability.
 - Updated `test_export_create_table_with_quoted_names` to use dynamically assigned ports for HTTP, RPC, MySQL, and PostgreSQL addresses.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: fix git cliff errors in latest version (#7947)

* chore: fix git cliff errors in latest version

- Fix errors in v2.12.0
- Do not generate logs for beta/rc tags between the compared commits

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: preserve blank line before release date in changelog

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: match term zh (#7952)

* fix: match term zh

Signed-off-by: discord9 <discord9@163.com>

* chore: per gemini

Signed-off-by: discord9 <discord9@163.com>

* chore: revert accident change

Signed-off-by: discord9 <discord9@163.com>

* feat: unicode script han

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: set upload timeout for uploading artifacts to S3 (#7958)

* ci: set upload timeout for uploading artifacts to S3

Signed-off-by: liyang <daviderli614@gmail.com>

* Update upload-artifacts-to-s3.sh

---------

Signed-off-by: liyang <daviderli614@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: cargo check -p common-meta (#7964)

fix: moka feature
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: always skip field pruning when using merge mode (#7957)

* test: add prefilter regressions for last_row null filters

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: skip fields in all merge mode

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: simplify pre-filter skip fields handling

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: update test

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: mysql prepare correctly returns error instead of panic (#7963)

feat: mysql writer support multiple statement execution

Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: relax azblob validation requirements (#7970)

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(mito2): allow CompactionOutput to succeed independently (#7948)

* refactor(mito2): improve compaction error handling and file removal

Refactor compaction task execution to enhance error handling and robustness.
- Implemented parallel execution of compaction tasks with proper error capture and logging for individual task failures.
- Ensured JoinSnafu is no longer directly used in error propagation, instead handling errors within the task processing loop.
- Adjusted file removal logic to correctly include expired SSTs after compaction merges.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* refactor(mito2): extract SstMerger trait for testability in compaction

Extract SstMerger trait and DefaultSstMerger implementation to improve the testability of DefaultCompactor.

The DefaultCompactor is now generic over SstMerger, allowing mock implementations to be injected for unit testing without relying on the full object storage access layer. This refactoring separates the concerns of SST file merging from the overall compaction orchestration logic.

Additionally:
- Updated CompactionScheduler to use DefaultCompactor::default().
- Added unit tests for DefaultCompactor using a MockMerger.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix(compaction): propagate join error during sst flush

Correctly propagates the error when joining SST flush handles during compaction. Previously, the error was logged but not returned, leading to potential silent failures.
Also reorders some imports for consistency.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* perf(compaction): pre-allocate capacity for compacted_inputs

Pre-allocates capacity for the compacted_inputs vector based on the estimated total size of inputs and expired SSTs. This optimization aims to reduce vector reallocations during the compaction process.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* feat/allow-partial-compaction:
 ### Commit Message

 Enhance `DefaultCompactor` and `MockMerger` for Improved Flexibility

 - **`compactor.rs`**:
   - Added `Clone` trait to `DefaultSstMerger` and `MockMerger` to allow cloning.
   - Removed `Arc` wrapping from `DefaultCompactor`'s `merger` field for direct usage.
   - Updated `merge_ssts` method to require `Clone` trait for `SstMerger`.
   - Modified `MockMerger` to use `Arc<Mutex>` for `results` and `call_idx` to ensure thread safety.
   - Adjusted error handling to use `error::InvalidMetaSnafu` directly.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

---------

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: propagate staging leader through lease and heartbeat (#7950)

* feat(mito): expose staging leader role state

* fix(region): clear staging metadata on leader exit

* feat: propagate staging leader role through heartbeat and metasrv

* chore: update comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(region): unify staging exit role transitions

* chore: update proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: cancel local compaction for enter staging (#7885)

* feat(mito2): support cancelling active local compaction

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(mito2): cover compaction cancellation return paths

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: cancel remaining tasks

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: move group rollback ownership to parent repartition (#7967)

* refactor(meta-srv): move group rollback ownership to parent repartition procedure

- Parent procedure now owns partial rollback based on failed/unknown subprocedures
- rollback order: group metadata first, then allocated-region cleanup
- original_target_routes captured during build-plan, persisted in RepartitionPlanEntry
- rollback_group_metadata_routes moved to utils as parent-owned helper
- Group subprocedure no longer supports rollback (rollback_supported = false)
- Removed UpdateMetadata::RollbackStaging from group state machine
- Deleted redundant group rollback tests and helpers

BREAKING CHANGE: group Procedure no longer handles rollback; parent procedure
is responsible for crash recovery and selecting which plans to roll back.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: use PreFilterMode::All if only one source in the partition range (#7973)

* feat: use PrefilterMode::All if only one source in the partition range

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: consider append_mode

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: skip merge if only one source

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix test

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): renew operating region leases from keeper roles (#7971)

* refactor(meta): store operating region roles in memory keeper

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): register operating region roles from region routes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): require explicit operating region roles

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): renew operating region leases from keeper roles

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(common-meta): cover region route role helpers

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(meta): cover operating region role propagation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): prefer metadata roles for region lease renewal

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add an index page (#7975)

* feat: include an index page

* fix: address code review

* fix: let / auth gated

* refactor: rename public-apis to public-api-prefix

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: remove redundant error messages in admin functions (#7953)

Closes #7938

Signed-off-by: yxrxy <yxrxytrigger@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf: better jieba cut (#7984)

* perf: better jieba cut

Signed-off-by: discord9 <discord9@163.com>

* fix: also filter pun mark

Signed-off-by: discord9 <discord9@163.com>

* chore

Signed-off-by: discord9 <discord9@163.com>

* docs: explain why

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: allow ipv4_num_to_string to accept valid integers (#7994)

* fix: allow ipv4_num_to_string to accept valid integers

Signed-off-by: Johannes Sluis <joesluis51@gmail.com>

* test: update sqlness result file

Signed-off-by: Johannes Sluis <joesluis51@gmail.com>

* fix: use coercible integer signature for ipv4_num_to_string

Signed-off-by: Johannes Sluis <joesluis51@gmail.com>

---------

Signed-off-by: Johannes Sluis <joesluis51@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: update manifest state before deleting delta files (#8001)

* fix: update state before deleting deltas

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: update comment

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: update log level

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: upgrade mysql metadata value limit to mediumblob (#7985)

* fix: upgrade mysql metadata values to mediumblob

* fix: fail mysql metadata startup on upgrade check errors

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: zh same underscore behavior (#8002)

* fix: zh same underscore behavior

Signed-off-by: discord9 <discord9@163.com>

* fix: only add token with _ from en analyzer

Signed-off-by: discord9 <discord9@163.com>

* test: neg sqlness case

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: manifest recovery scans after last version if possible (#8009)

* feat: suppport scan with start after

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: add start_after test

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: adjust remove dir warning

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: test list_with_start_after

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: update get_paths call with start_after arg in checkpoint test

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: log scan metrics

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: fix start_after on manifest dir

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add a standalone flag in plugins during startup (#7974)

* chore: add a standalone flag in plugins during startup

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: add derive

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: bump version to v1.0.1

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: liyang <daviderli614@gmail.com>
Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: yxrxy <yxrxytrigger@gmail.com>
Signed-off-by: Johannes Sluis <joesluis51@gmail.com>
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
Co-authored-by: liyang <daviderli614@gmail.com>
Co-authored-by: cui <cuiweixie@gmail.com>
Co-authored-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
Co-authored-by: yxrxy <1532529704@qq.com>
Co-authored-by: Joe Sluis <43276756+JoeS51@users.noreply.github.com>
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>
 2026-04-23 17:37:27 +08:00
Ruihang Xia
0f2f20d4b7 feat: reduce unit test suite wall time (#7657) 
* feat: reduce wait timt of  from 57s to 0.6s

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* , , ,

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test_query_concurrently

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-02-03 13:32:11 +00:00
Ning Sun
124478f577 feat: use arrow-pg for postgres data encoding (#7591) 
* feat: use arrow-pg for encode_row

* refactor: remove bytea and datetime module

* feat: port more encodings to arrow-pg

* feat: implement intervalstyle

* chore: format

* chore: remove error that is no longer used

* chore: use released arrow-pg

* Apply suggestions from code review

Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>

---------

Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
 2026-01-28 02:34:02 +00:00
LFC
fdab75ce27 feat: simple read write new json type values (#7175) 
feat: basic json read and write

Signed-off-by: luofucong <luofc@foxmail.com>
 2025-11-27 12:40:35 +00:00
LFC
4a7c16586b refactor: remove Vectors from RecordBatch completely (#7184) 
* refactor: remove `Vector`s from `RecordBatch` completely

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2025-11-21 08:53:35 +00:00
LFC
2f637a262e chore: update datafusion to 50 (#7076) 
* chore: update datafusion to 50

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* fix: update datafusion_pg_catalog import

* chore: fix toml format

* chore: fix toml format again

* fix nextest

Signed-off-by: luofucong <luofc@foxmail.com>

* fix sqlness

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: switch datafusion-orc to upstream tag

* fix sqlness

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
 2025-10-23 07:18:36 +00:00
Alan Tang
8073e552df feat: add updated_on to tablemeta with a default of created_on (#7072) 
* feat: add updated_on to tablemeta with a default of created_on

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* feat: support the update_on on alter procedure

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* feat: add updated_on into information_schema.tables

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* fix: make sqlness happy

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* test: add test case for tablemeta update

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* fix: fix failing test for ALTER TABLE

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* feat: use created_on as default for updated_on when missing

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

---------

Signed-off-by: Alan Tang <jmtangcs@gmail.com>
 2025-10-15 11:12:27 +00:00
LFC
8fe17d43d5 chore: update rust to nightly 2025-10-01 (#7069) 
* chore: update rust to nightly 2025-10-01

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: nix update

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
 2025-10-11 07:30:52 +00:00
dennis zhuang
91a727790d feat: supports permission mode for static user provider (#7017) 
* feat: supports permission mode for static user provider

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: style

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: comment

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-09-25 03:45:31 +00:00
Ruihang Xia
c9377e7c5a build: bump rust edition to 2024 (#6920) 
* bump edition

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* gen keyword

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* lifetime and env var

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* one more gen fix

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* lifetime of temporaries in tail expressions

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* format again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clippy nested if

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clippy let and return

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-09-08 02:37:18 +00:00
LFC
f9d2a89a0c chore: update datafusion family (#6675) 
* chore: update datafusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* use official otel-arrow-rust

Signed-off-by: luofucong <luofc@foxmail.com>

* rebase

Signed-off-by: luofucong <luofc@foxmail.com>

* use the official orc-rust

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* remove the empty lines

Signed-off-by: luofucong <luofc@foxmail.com>

* try following PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2025-08-15 12:41:49 +00:00
zyy17
777da35b0d refactor: unify the event recorder (#6689) 
* refactor: unify the event recorder

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add `table_name()` in `Event` trait

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: add `slow_query_options` in `Instance`

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add `EventHandlerOptions` and `options()` in `EventHandler` trait

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: add `aggregate_events_by_type()` and support log mode of slow query

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: polish the code

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* fix: clippy errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: support to set ttl by using extension of query context

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: refine the configs fields

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* fix: sqlness test errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: use `Duration` type instead of `String` for ttl fields

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: use pre-allocation for building RowInsertRequests

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: fix clippy errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: code review

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: fix integration errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: polish code for `group_events_by_type()` and `build_row_inserts_request()`, also add the unit tests

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: refine comments

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-08-12 18:26:12 +00:00
zyy17
3a9f0220b5 fix: unable to record slow query (#6590) 
* refactor: add process manager for prometheus query

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: modify `register_query()` API to accept parsed statement(`catalog::process_manager::QueryStatement`)

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add the slow query timer in the `Tikcet` of ProcessManager

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* test: add integration tests

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add process manager in `do_exec_plan()`

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* tests: add `test_postgres_slow_query` integration test

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: polish the code

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: create a query ticket and slow query timer if the statement is a query in `query_statement()`

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* fix: sqlness errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-08-06 03:35:12 +00:00
dennis zhuang
a4bd11fb9c fix: empty statements hang (#6480) 
* fix: empty statements hang

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* tests: add cases

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2025-07-09 02:13:14 +00:00
dennis zhuang
0d81400bb4 feat: supports select @@session.time_zone (#6212) 2025-06-03 02:32:19 +00:00
dennis zhuang
9afc61f778 feat: supports @@session.time_zone for mysql (#6210) 
* feat: supports @@session.time_zone for mysql

* test: assert timezone
 2025-05-29 05:26:49 +00:00
LFC
4b5ab75312 refactor: remove some async in ServerHandlers (#6057) 
* refactor: remove some async in ServerHandlers

* address PR comments
 2025-05-07 03:57:16 +00:00
jeremyhi
02af9dd21a refactor!: remove datetime type (#5506) 
* feat remove datetime type

* chore: fix unit test

* chore: add column test

* refactor: move create and alter validation to one place

* chore: minor refactor ut

* refactor: rename expr_factory to expr_helper

* chore: remove unnecessary args
 2025-02-13 08:01:16 +00:00
Ning Sun
3133f3fb4e feat: add cursor statements (#5094) 
* feat: add sql parsers for cursor operations

* feat: cursor operator

* feat: implement RecordBatchStreamCursor

* feat: implement cursor storage and execution

* test: add tests

* chore: update docstring

* feat: add a temporary sql rewrite for cast in limit

this issue is described in #5097

* test: add more sql for cursor integration test

* feat: reject non-select query for cursor statement

* refactor: address review issues

* test: add empty result case

* feat: address review comments
 2024-12-06 09:32:22 +00:00
Zhenchi
0aab68c23b feat(vector): add conversion between vector and string (#5029) 
* feat(vector): add conversion between vector and string

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix sqlness

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-11-20 08:42:00 +00:00
Zhenchi
d616bd92ef feat: introduce vector type (#4964) 
* feat: introduce vector type

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: fix prepared stmt

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: add grpc test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: parse vector value

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: column to row

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: sqlness

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: merge issue

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: add check for bytes size

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* Update tests/cases/standalone/common/types/vector/vector.sql

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: update proto

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: simplify cargo

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: address comment

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-11-12 08:28:44 +00:00
dennis zhuang
a34035a1f2 fix: set transaction variables not working in mysql protocol (#4912) 2024-10-30 10:59:13 +00:00
Yohan Wal
8ed5bc5305 refactor: json conversion (#4893) 
* refactor: json type update

* test: update test

* fix: convert when needed

* revert: leave sqlness tests unchanged

* fix: fmt

* refactor: just refactor

* Apply suggestions from code review

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* refactor: parse jsonb first

* test: add bad cases

* Update src/datatypes/src/vectors/binary.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* fix: fmt

* fix: fix clippy/check

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-10-29 15:46:24 +00:00
Ning Sun
0f99218386 feat: list/array/timezone support for postgres output (#4727) 
* feat: list/array support for postgres output

* fix: implement time zone support for postgrsql

* feat: add a geohash function that returns array

* fix: typo

* fix: lint warnings

* test: add sqlness test

* refactor: check resolution range before convert value

* fix: test result for sqlness

* feat: upgrade pgwire apis
 2024-09-22 02:39:38 +00:00
Yohan Wal
04e7dd6fd5 feat: add json data type (#4619) 
* feat: add json type and vector

* fix: allow to create and insert json data

* feat: udf to query json as string

* refactor: remove JsonbValue and JsonVector

* feat: show json value as strings

* chore: make ci happy

* test: adunit test and sqlness test

* refactor: use binary as grpc value of json

* fix: use non-preserve-order jsonb

* test: revert changed test

* refactor: change udf get_by_path to jq

* chore: make ci happy

* fix: distinguish binary and json in proto

* chore: delete udf for future pr

* refactor: remove Value(Json)

* chore: follow review comments

* test: some tests and checks

* test: fix unit tests

* chore: follow review comments

* chore: corresponding changes to proto

* fix: change grpc and pgsql server behavior alongside with sqlness/crud tests

* chore: follow review comments

* feat: udf of conversions between json and strings, used for grpc server

* refactor: rename to_string to json_to_string

* test: add more sqlness test for json

* chore: thanks for review :)

* Apply suggestions from code review

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-09-09 11:41:36 +00:00
Weny Xu
4c93fe6c2d chore: bump rust-postgres to 0.7.11 (#4504) 2024-08-05 04:26:46 +00:00
dennis zhuang
be3ea0fae7 feat: improve datafusion external error and mysql error (#4362) 
* feat: improve datafusion external error and mysql error

* chore: address CR comments and fix tests

---------

Co-authored-by: evenyag <realevenyag@gmail.com>
 2024-07-16 07:01:09 +00:00
Ruihang Xia
185953e586 fix: support unary operator in default value, partition rule and prepare statement (#4301) 
* handle unary operator

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add prepare test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add test and context

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix rebase error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix merge error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-07-09 08:59:06 +00:00
dennis zhuang
d2f6daf7b7 fix: prepare inserting with column defaults not work, #4244 (#4272) 
* fix: prepare inserting with column defaults not work, #4244

* fix: build column_defaults every time when creating adapters

* feat: cache the column_defaults in table

* test: assert ts column

* fix: unit

* chore: style

Co-authored-by: Yingwen <realevenyag@gmail.com>

* fix: typo

* chore: style

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-07-05 06:50:12 +00:00
Yingwen
669a6d84e9 test: gracefully shutdown postgres client in sql tests (#3958) 
* chore: debug log

* test: gracefully shutdown pg client
 2024-05-16 11:50:45 +00:00
LFC
314f2704d4 build(deps): update datafusion to latest and arrow to 51.0 (#3661) 
* chore: update datafusion

* update sqlness case of time.sql

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: adjust range query partition

* fix: hisogram incorrect result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: ignore filter pushdown temporarily

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: update limit sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: histogram with wrong distribution

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: update negative ordinal sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: bump df to cd7a00b

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve conflicts

* ignore test_range_filter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix promql exec panic

* fix "select count(*)" exec error

* re-enable the "test_range_filter" test since the filter push down seems not necessary to be removed

* fix: range query schema error

* update sqlness results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve conflicts

* update datafusion, again

* fix pyo3 compile error, and update some sqlness results

* update decimal sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: promql literal

* fix udaf tests

* fix filter pushdown sqlness tests

* fix?: test_cast

* fix: rspy test fail due to datafusion `sin` signature change

* rebase main to see if there are any failed tests

* debug ci

* debug ci

* debug ci

* enforce input partition

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* debug ci

* fix ci

* fix ci

* debug ci

* debug ci

* debug ci

* fix sqlness

* feat: do not return error while creating a filter

* chore: remove array from error

* chore: replace todo with unimplemented

* Update src/flow/clippy.toml

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WUJingdi <taylor-lagrange@qq.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
 2024-04-18 12:07:18 +00:00
JohnsonLee
d6b2d1dfb8 feat: Support outputting various date styles for postgresql (#3602) 
* test: add integration_test for datetime style

* feat: support various datestyle for postgres

* doc: rewrite the comment about merge_datestyle_value

* test: add more test to illustrate valid datestyle input
 2024-04-01 07:31:36 +00:00
JohnsonLee
83643eb195 feat: Support printing postgresql's bytea data type in its "hex" and "escape" format (#3567) 
* feat: support set variable statement of session

* feat: support printing postgresql's bytea data type in its "hex" and "escape" format in ugly way

* refactor: add 'SessionConfigValue' type and unify the name

* doc: add license header

* refactor: confine coupling with 'sql::ast::Value' in SessionConfigValue

* refactor: move all bytea wrapper into bytea.rs

* fix: remove unused import in context.rs and postgres.rs

* refactor: rename 'set_configuration_parameter' to 'set_session_config'

rename 'set_configuration_parameter' in statement_.rs to 'set_session_config'

* refactor: use mod to organize options via macro

* refactor: re-model the session config value with static type

* test: add integration test

* refactor: move the encode bytea by format type logic into encoder

refactor: use Arc<DashMap> instead of DashMap in QueryContext

refactor: use Arc<DashMap> instead of DashMap in QueryContext

    Avoid expensive clone

refactor: use unreachable!() instead of unimplemented!()

refactor: move the encode bytea by format type logic into encoder

test: add binary format integration test case

* test: add ut for byte related type

* doc: remove TODO of bytea_output

* refactor: simplify the implementation with simple struct instead of complex typing

* fix: typo of 'Available'

* fix compile

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
 2024-03-27 01:54:41 +00:00
x³u³
24886b9530 test: add a parameter type mismatch test case to sql integration test (#3568) 2024-03-22 17:43:20 +00:00
Weny Xu
a309cd018a fix: fix incorrect COM_STMT_PREPARE reply (#3463) 
* fix: fix incorrect `COM_STMT_PREPARE` reply

* chore: use column name instead of index
 2024-03-08 07:31:20 +00:00
WU Jingdi
816d94892c feat: support HTTP&gRPC&pg set timezone (#3125) 
* feat: support HTTP&gRPC&pg set timezone

* chore: fix code advice

* chore: fix code advice
 2024-01-15 06:29:31 +00:00
WU Jingdi
d1ee1ba56a feat: support set timezone in db (#2992) 
* feat: support set timezone in db

* chore: fix  ci

* chore: fix code advice

* fix: rename `time_zone` to `timezone`
 2023-12-27 09:19:39 +00:00
LFC
c7b36779c1 fix: correctly decode mysql timestamp binary value (#2946) 
* fix: correctly decode mysql timestamp binary value

* fix: ut

* fix: resolve PR comments
 2023-12-19 11:05:28 +00:00
Ning Sun
85eebcb16f fix: correct mysql timestamp encoding for binary protocol (#2797) 
* fix: correct mysql timestamp encoding for binary protocol

* test: add tests for mysql timestamp encoding
 2023-11-23 18:54:59 +00:00
Ning Sun
0ad3fb6040 fix: mysql timezone settings (#2534) 
* fix: restore time zone settings for mysql

* test: add integration test for time zone

* test: fix unit test for check
 2023-10-07 10:21:32 +00:00
Ruihang Xia
e352fb4495 fix: check for table scan before expanding (#2491) 
* fix: check for table scan before expanding

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change assert_ok to unwrap

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy warning

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* don't skip dml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* uncomment ignored tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-26 12:12:08 +00:00
Ruihang Xia
34d6288945 feat: bring back sqlness and integration tests (#2448) 
* enable integration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable sqlness region failover

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* enable sqlness in CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort unstable result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* set require_lease_before_startup to true

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: fix inconsistent cache

* replace windows path chars

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* ignore some integration cases in windows

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Revert "ignore some integration cases in windows"

This reverts commit 122478b7c1.

* disable windows for now

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: fix close region bug in RegionHeartbeatResponseHandler

* disable failover tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
 2023-09-20 09:17:30 +00:00
LFC
4b13c88752 fix: resolve more integration tests (#2406) 
* fix: resolve more integration tests

* Update tests-integration/tests/http.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2023-09-15 09:43:16 +00:00
JeremyHi
58d07e0e62 feat: v04 rm unused exprs (#2285) 
* feat: rm compact and flush exprs

* refactor: continue to rm compact and flush
 2023-09-12 07:57:15 -05:00
shuiyisong
7f51141ed0 refactor: auth crate (#2148) 
* chore: move user_info to auth crate

* chore: temp commit before resolving tests compile error

* chore: fix compile issue

* chore: minor fix

* chore: tmp save

* chore: change user_info to trait

* chore: minor change & use auth result user info in pg session setup

* chore: add as_any to user_info

* chore: rename user_info

* chore: remove ice file

* chore: add permission checker

* chore: add grpc permission check

* chore: add session spawn user_info to query_ctx

* chore: minor update

* chore: add permission checker to sql handler & temp save

* chore: add permission checker to prometheus handler

* chore: add permission checker to opentsdb handler

* chore: add permission checker to other handlers

* chore: add test

* chore: add user_info setting on http entrance

* chore: fix toml

* chore: remove box in permission req

* chore: cr issue

* chore: cr issue
 2023-08-14 02:51:26 +00:00
Ning Sun
d18eb18b32 feat: use server inferenced types on statement describe (#2032) 
* feat: use server inferenced types on statement describe

* feat: add support for server inferenced type

* feat: allow parameter type inferencing

* chore: update comments

* fix: lint issue

* style: comfort rustfmt

* Update src/servers/src/postgres/types.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-08-09 02:57:56 +00:00
shuiyisong
5eb2c609a3 fix: auth in grpc (#2056) 
* fix: auth in grpc

* fix: change to return err

* fix: add grpc test

* fix: add http test

* fix: add mysql and pg test
 2023-08-01 15:18:31 +00:00
Ning Sun
f293126315 feat: add logical plan based prepare statement for postgresql (#1813) 
* feat: add logical plan based prepare statement for postgresql

* refactor: correct more types

* Update src/servers/src/postgres/types.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* fix: address review issues

* test: add datetime in integration tests

---------

Co-authored-by: LFC <bayinamine@gmail.com>
 2023-07-11 11:07:18 +08:00
Lei, HUANG
f287d3115b chore: replace result assertions (#1840) 
* s/assert!\((.*)\.is_ok\(\)\);/\1.unwrap\(\);/g

* s/assert!\((.*)\.is_some\(\)\);/\1.unwrap\(\);/g
 2023-06-27 19:14:48 +08:00
LFC
fcff66e039 chore: deny unused results (#1825) 
* chore: deny unused results

* rebase
 2023-06-27 15:33:53 +08:00