Ruihang Xia
8b60c27c2e
feat: enhance windowed-sort optimizer rule ( #4910 )
...
* add RegionScanner::metadata
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* skip PartSort when there is no tag column
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add more sqlness test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* handle desc
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: should keep part sort on DESC
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-10-31 06:15:45 +00:00
Weny Xu
6e776d5f98
feat: support to reject write after flushing ( #4759 )
...
* refactor: use `RegionRoleState` instead of `RegionState`
* feat: introducing `RegionLeaderState::Downgrading`
* refactor: introduce `set_region_role_state_gracefully`
* refactor: use `set_region_role` instead of `set_writable`
* feat: support to reject write after flushing
* fix: fix unit tests
* test: add unit test for `should_reject_write`
* chore: add comments
* chore: refine comments
* fix: fix unit test
* test: enable fuzz tests for Local WAL
* chore: add logs
* chore: rename `RegionStatus` to `RegionState`
* feat: introduce `DowngradingLeader`
* chore: rename
* refactor: refactor `set_role_state` tests
* test: ensure downgrading region will reject write
* chore: enhance logs
* chore: refine name
* chore: refine comment
* test: add tests for `set_role_role_state`
* fix: fix unit tests
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2024-09-30 08:28:51 +00:00
Weny Xu
4045298cb2
feat: add region_statistics table ( #4771 )
...
* refactor: introduce `region_statistic`
* refactor: move DatanodeStat related structs to common_meta
* chore: add comments
* feat: implement `list_region_stats` for `ClusterInfo` trait
* feat: add `region_statistics` table
* feat: add table_id and region_number fields
* chore: rename unused snafu
* chore: udpate sqlness results
* chore: avoid to print source in error msg
* chore: move `procedure_info` under `greptime` catalog
* chore: apply suggestions from CR
* Update src/common/meta/src/datanode.rs
Co-authored-by: jeremyhi <jiachun_feng@proton.me >
---------
Co-authored-by: jeremyhi <jiachun_feng@proton.me >
2024-09-27 09:54:52 +00:00
Lei, HUANG
216bce6973
perf: count(*) for append-only tables ( #4545 )
...
* feat: support fast count(*) for append-only tables
* fix: total_rows stats in time series memtable
* fix: sqlness result changes for SinglePartitionScanner -> StreamScanAdapter
* fix: some cr comments
2024-08-13 09:27:50 +00:00
Weny Xu
cb4cffe636
chore: bump opendal version to 0.48 ( #4499 )
2024-08-04 00:46:04 +00:00
Ruihang Xia
fce65c97e3
feat: make RegionScanner aware of PartitionRange ( #4170 )
...
* define PartitionRange
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add optimizer rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* implement interfaces
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* impl aggr stream
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add fallback method
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add document and rename struct
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add more comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-06-21 09:54:22 +00:00
LFC
cc2f7efb98
chore: bump datafusion version to fix last_value regression ( #4169 )
...
* chore: bump datafusion version to fix `last_value` regression
* fix: resolve PR comments
* fix ci
2024-06-19 07:47:17 +00:00
Ruihang Xia
45fee948e9
fix: display error in correct format ( #4082 )
...
* fix: display error in correct format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add address to RegionServer error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-05-31 09:25:14 +00:00
Weny Xu
eab309ff7e
fix: avoid acquiring lock during reading stats ( #4070 )
...
* fix: avoid acquiring lock during reading stats
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2024-05-30 07:08:04 +00:00
tison
f9db5ff0d6
build(deps): upgrade opendal to 0.46 ( #4037 )
...
* build(deps): upgrade opendal to 0.46
Signed-off-by: tison <wander4096@gmail.com >
* migrate writes
Signed-off-by: tison <wander4096@gmail.com >
* migrate reads
Signed-off-by: tison <wander4096@gmail.com >
* fixup object safety
Signed-off-by: tison <wander4096@gmail.com >
* fixup names
Signed-off-by: tison <wander4096@gmail.com >
* fixup compilation
Signed-off-by: tison <wander4096@gmail.com >
* fixup compilation
Signed-off-by: tison <wander4096@gmail.com >
* a few Buffer to Vec
Signed-off-by: tison <wander4096@gmail.com >
* Make greptime buildable with opendal 0.46 (#5 )
Signed-off-by: Xuanwo <github@xuanwo.io >
* fixup toml check
Signed-off-by: tison <wander4096@gmail.com >
* test_orc_opener
Signed-off-by: tison <wander4096@gmail.com >
* Fix lru cache (#6 )
Signed-off-by: Xuanwo <github@xuanwo.io >
* clippy
Signed-off-by: tison <wander4096@gmail.com >
* improve comments
Signed-off-by: tison <wander4096@gmail.com >
* address comments
Signed-off-by: tison <wander4096@gmail.com >
* reduce buf copy
Signed-off-by: tison <wander4096@gmail.com >
* upgrade to reqwest 0.12
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
Signed-off-by: Xuanwo <github@xuanwo.io >
Co-authored-by: Xuanwo <github@xuanwo.io >
2024-05-27 09:12:23 +00:00
maco
40c585890a
refactor: replace Expr with datafusion::Expr ( #3995 )
...
* refactor: replace Expr with datafusion::Expr
* fix: fmt-toml
* fix: cr comment
2024-05-21 06:40:29 +00:00
Yingwen
179c8c716c
feat: Adds RegionScanner trait ( #3948 )
...
* feat: define region scanner
* feat: single partition scanner
* feat: use single partition scanner
* feat: implement ExecutionPlan wip
* feat: mito engine returns single partition scanner
* feat: implement DisplayAs for region server
* feat: dummy table provider use handle_partitioned_query()
* test: update sqlness test
* feat: table provider use ReadFromRegion
* refactor: remove StreamScanAdapter
* chore: update lock
* style: fix clippy
* refactor: remove handle_query from the RegionEngine trait
* chore: address CR comments
* refactor: rename methods
* refactor: rename ReadFromRegion to RegionScanExec
2024-05-20 11:52:00 +00:00
Yingwen
d0820bb26d
refactor: Remove PhysicalPlan trait and use ExecutionPlan directly ( #3894 )
...
* refactor: remove PhysicalPlan
* refactor: remove physical_plan mod
* refactor: import
* fix merge error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-05-11 07:38:03 +00:00
Ruihang Xia
115c74791d
build(deps): bump snafu to 0.8 ( #3911 )
...
* change Cargo.toml
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* global replace
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* handle alias in script engine
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-05-10 13:36:25 +00:00
LFC
314f2704d4
build(deps): update datafusion to latest and arrow to 51.0 ( #3661 )
...
* chore: update datafusion
* update sqlness case of time.sql
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: adjust range query partition
* fix: hisogram incorrect result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: ignore filter pushdown temporarily
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: update limit sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: histogram with wrong distribution
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: update negative ordinal sqlness case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: bump df to cd7a00b
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve conflicts
* ignore test_range_filter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix promql exec panic
* fix "select count(*)" exec error
* re-enable the "test_range_filter" test since the filter push down seems not necessary to be removed
* fix: range query schema error
* update sqlness results
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve conflicts
* update datafusion, again
* fix pyo3 compile error, and update some sqlness results
* update decimal sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: promql literal
* fix udaf tests
* fix filter pushdown sqlness tests
* fix?: test_cast
* fix: rspy test fail due to datafusion `sin` signature change
* rebase main to see if there are any failed tests
* debug ci
* debug ci
* debug ci
* enforce input partition
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* debug ci
* fix ci
* fix ci
* debug ci
* debug ci
* debug ci
* fix sqlness
* feat: do not return error while creating a filter
* chore: remove array from error
* chore: replace todo with unimplemented
* Update src/flow/clippy.toml
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: WUJingdi <taylor-lagrange@qq.com >
Co-authored-by: discord9 <discord9@163.com >
Co-authored-by: evenyag <realevenyag@gmail.com >
Co-authored-by: tison <wander4096@gmail.com >
2024-04-18 12:07:18 +00:00
tison
50ae4dc174
refactor: merge RegionHandleResult into RegionHandleResponse ( #3721 )
...
* refactor: merge RegionHandleResult into RegionHandleResponse
Signed-off-by: tison <wander4096@gmail.com >
* RegionResponse to api::region
Signed-off-by: tison <wander4096@gmail.com >
* order
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-04-17 10:03:20 +00:00
Ruihang Xia
2b2fd80bf4
feat: return new added columns in region server's extension response ( #3533 )
...
* feat: adapt the new proto response
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update interfaces
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* write columns to extension
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use physical column's schema
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* sort logical columns by name
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* format code
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* return physical table's column
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/common/meta/src/datanode_manager.rs
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* implement sort column logic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* proxy create table procedure to create logical table
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add unit test for sort_columns
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-03-23 09:31:16 +00:00
shuiyisong
aa953dcc34
fix: impl RecordBatchStream method explicitly ( #3482 )
...
fix: impl RecordBatchStream method explicitly
2024-03-11 09:07:10 +00:00
SteveLauC
e9a2b0a9ee
chore: use workspace-wide lints ( #3352 )
...
* chore: use workspace-wide lints
* respond to review
2024-02-22 01:01:10 +00:00
LFC
ec43b9183d
feat: table route for metric engine ( #3053 )
...
* feat: table route for metric engine
* feat: register logical regions
* fix: open logical region (#96 )
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-04 06:30:17 +00:00
niebayes
7d509e97f6
chore: move some commonly referenced crates to workspace Cargo.toml ( #2981 )
...
fix: resolve conflicts
2023-12-22 09:13:18 +00:00
Weny Xu
97c3755ab6
feat(mito): add skip_wal_replay option to OpenRegionRequest ( #2955 )
...
* feat(mito): add skip_replay_wal option to OpenRegionRequest
* test: add tests for skip replay wal
* chore: rename `skip_replay_wal` to `skip_wal_replay`
2023-12-20 03:31:48 +00:00
Ruihang Xia
6b8dbcfb54
chore: update toolchain to 20231219 ( #2932 )
...
* update toolchain file, remove unused feature gates
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update action file
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update to 12-19
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-19 07:24:08 +00:00
tison
f74715ce52
refactor: RegionEngine::handle_request always returns affected rows ( #2874 )
...
* refactor: RegionEngine::handle_request -> handle_execution
Signed-off-by: tison <wander4096@gmail.com >
* propagate refactor
Signed-off-by: tison <wander4096@gmail.com >
* revert spell change
Signed-off-by: tison <wander4096@gmail.com >
* propagate refactor
Signed-off-by: tison <wander4096@gmail.com >
* cargo clippy
Signed-off-by: tison <wander4096@gmail.com >
* propagate refactor
Signed-off-by: tison <wander4096@gmail.com >
* cargo fmt
Signed-off-by: tison <wander4096@gmail.com >
* more name clarification
Signed-off-by: tison <wander4096@gmail.com >
* revert rename
Signed-off-by: tison <wander4096@gmail.com >
* wrap affected rows into RegionResponse
Signed-off-by: tison <wander4096@gmail.com >
* flatten return AffectedRows
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2023-12-06 13:27:19 +00:00
Weny Xu
9e58bba363
feat: add set_readonly_gracefully for engine ( #2787 )
...
* feat: add set_readonly_gracefully for engine
* chore: apply suggestions from CR
* chore: rename to set_readonly_test
* chore: apply suggestions from CR
2023-11-24 10:59:05 +00:00
Yingwen
142035340d
fix: compiler errors under non-default features ( #2761 )
...
* feat: bump prost and fix pprof feature compiler errors
* feat: fix compiler errors on tokio-console
* chore: fix compiler errors
* ci: add all features check to ci
2023-11-16 19:43:44 +00:00
Ruihang Xia
93f21b188d
chore: unify workspace dependencies and remove some unused ( #2713 )
...
* log-store
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* base64
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* udeps
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* snafu
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* snafu style
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change the workspace deps' style
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-11-09 07:34:22 +00:00
Weny Xu
54ed7529ca
feat: introduce the region role ( #2640 )
...
* feat: introduce region role
* chore: apply suggestions from CR
2023-10-31 06:59:22 +00:00
Wei
36c0742c45
feat: region disk usage statistic ( #2665 )
...
* feat: region usage
* chore: code comment
* chore: unit test
* feat: region usage for engine
* chore: rename to region usage
* feat: add to heartbeat
* chore: u64 to i64 convert
* chore: cr comment.
Co-authored-by: Yingwen <realevenyag@gmail.com >
* Update src/file-engine/src/engine.rs
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2023-10-31 03:57:37 +00:00
Zhenchi
201acd152d
fix: missing file engine with default options ( #2519 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2023-09-28 10:25:12 +00:00
Zhenchi
1f1d72bdb8
feat: defensively specify limit parameter for file stream ( #2517 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2023-09-28 06:14:27 +00:00
Ruihang Xia
515ce825bd
feat: stack trace style debug print for error ( #2489 )
...
* impl macro stack_trace_debug
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* manually mark external error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use debug print
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplify the error and warn macro
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix ut
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add docs
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* replace snafu backtrace with location
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-09-26 11:23:21 +00:00
Yingwen
c9f8b9c7c3
feat: update proto and remove create_if_not_exists ( #2467 )
2023-09-22 03:24:49 +00:00
shuiyisong
20f4f7971a
refactor: remove source and location in snafu display ( #2428 )
...
* refactor: remove source pt 1
* refactor: remove source pt 2
* refactor: remove source pt 3
* refactor: remove location pt 1
* refactor: remove location pt 2
* chore: remove rustc files
* chore: fix error case
* chore: fix test
* chore: fix test
* chore: fix cr issue
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com >
---------
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com >
2023-09-21 02:55:24 +00:00
Zhenchi
3a39215f11
feat: migrate file engine from table to reigon ( #2365 )
...
* feat: migrate file engine from table to reigon
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* Update src/file-engine/src/engine.rs
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* feat: specify ts index for file engine
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: handle time index for external table
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: some integration testsg
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: add file schema and table schema compatibility
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: compatible file schema to region schema
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: add error msg
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: simplify close
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: implement set_writable
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: tests-integration compilation
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local >
2023-09-18 08:02:43 +00:00
