Yingwen
2fcb95f50a
fix!: fix regression caused by unbalanced partitions and splitting ranges ( #5090 )
...
* feat: assign partition ranges by rows
* feat: balance partition rows
* feat: get uppoer bound for part nums
* feat: only split in non-compaction seq scan
* fix: parallel scan on multiple sources
* fix: can split check
* feat: scanner prepare by request
* feat: remove scan_parallelism
* docs: upate docs
* chore: update comment
* style: fix clippy
* feat: skip merge and dedup if there is only one source
* chore: Revert "feat: skip merge and dedup if there is only one source"
Since memtable won't do dedup jobs
This reverts commit 2fc7a54b11
2024-12-09 12:50:57 +00:00
Ruihang Xia
be72d3bedb
feat: simple limit impl in PartSort ( #4922 )
...
* feat: simple limit impl in PartSort
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: update time_index method to return a non-optional String
Co-authored-by: Yingwen <realevenyag@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use builtin limit
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add more info to analyze display
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-11-01 09:25:03 +00:00
Ruihang Xia
8b60c27c2e
feat: enhance windowed-sort optimizer rule ( #4910 )
...
* add RegionScanner::metadata
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* skip PartSort when there is no tag column
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add more sqlness test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* handle desc
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: should keep part sort on DESC
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-10-31 06:15:45 +00:00
Ruihang Xia
03f2fa219d
feat: optimizer rule for windowed sort ( #4874 )
...
* basic impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* implement physical rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: install windowed sort physical rule and optimize partition ranges
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add logs and sqlness test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: introduce PartSortExec for partitioned sorting
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tune exec nodes' properties and metrics
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clean up
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* debug: add more info on very wrong
* debug: also print overlap ranges
* feat: add check when emit PartSort Stream
* dbg: info on overlap working range
* feat: check batch range is inside part range
* set distinguish partition range param
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: more logs
* update sqlness
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tune optimizer
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clean up
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix lints
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix windowed sort rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: early terminate sort stream
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: remove min/max check
* chore: remove unused windowed_sort module, uuid feature and refactor region_scanner to synchronous
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: print more fuzz log
* chore: more log
* fix: part sort should skip empty part
* chore: remove insert logs
* tests: empty PartitionRange
* refactor: testcase
* docs: update comment&tests: all empty
* ci: enlarge etcd cpu limit
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: discord9 <discord9@163.com >
Co-authored-by: evenyag <realevenyag@gmail.com >
2024-10-29 07:46:05 +00:00
discord9
a8ed3db0aa
feat: Merge sort Logical plan ( #4768 )
...
* feat(WIP): MergeSort
* wip
* feat: MergeSort LogicalPlan
* update sqlness result
* Apply suggestions from code review
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
* refactor: per review advice
* refactor: more per review
* chore: per review
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
2024-10-09 09:37:27 +00:00
Ruihang Xia
2feddca1cb
feat: include order by to commutativity rule set ( #4753 )
...
* feat: include order by to commutativity rule set
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tune sqlness replace interceptor
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-23 08:35:06 +00:00
Ruihang Xia
8f0959fa9f
fix: fix incorrect result of topk with cte ( #4523 )
...
* fix: fix incorrect result of topk with cte
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clean up cargo toml
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-08-07 09:13:38 +00:00
Ruihang Xia
a7aa556763
feat: output multiple partition in MergeScanExec ( #4227 )
...
* feat: output multiple partition in MergeScanExec
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix range manipulate
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-06-28 13:45:22 +00:00
LFC
314f2704d4
build(deps): update datafusion to latest and arrow to 51.0 ( #3661 )
...
* chore: update datafusion
* update sqlness case of time.sql
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: adjust range query partition
* fix: hisogram incorrect result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: ignore filter pushdown temporarily
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: update limit sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: histogram with wrong distribution
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: update negative ordinal sqlness case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: bump df to cd7a00b
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve conflicts
* ignore test_range_filter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix promql exec panic
* fix "select count(*)" exec error
* re-enable the "test_range_filter" test since the filter push down seems not necessary to be removed
* fix: range query schema error
* update sqlness results
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve conflicts
* update datafusion, again
* fix pyo3 compile error, and update some sqlness results
* update decimal sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: promql literal
* fix udaf tests
* fix filter pushdown sqlness tests
* fix?: test_cast
* fix: rspy test fail due to datafusion `sin` signature change
* rebase main to see if there are any failed tests
* debug ci
* debug ci
* debug ci
* enforce input partition
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* debug ci
* fix ci
* fix ci
* debug ci
* debug ci
* debug ci
* fix sqlness
* feat: do not return error while creating a filter
* chore: remove array from error
* chore: replace todo with unimplemented
* Update src/flow/clippy.toml
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: WUJingdi <taylor-lagrange@qq.com >
Co-authored-by: discord9 <discord9@163.com >
Co-authored-by: evenyag <realevenyag@gmail.com >
Co-authored-by: tison <wander4096@gmail.com >
2024-04-18 12:07:18 +00:00
Ruihang Xia
92a9802343
feat: canonicalize all unquoted identifier to lowercase ( #2828 )
...
* feat: canonicalize all unquoted identifier to lowercase
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add more tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* test altering table
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* primary key declare
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix primary key declare
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* partition by and time index
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove redundent call to canonicalize
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-11-29 06:40:10 +00:00
Ruihang Xia
56fc77e573
fix: add missing error display message ( #2791 )
...
* fix: add missing error display message
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-11-23 02:59:49 +00:00
Wei
ae27fbc7f2
feat: impl Vector, DataType traits for decimal128 ( #2727 )
...
* feat: decimal128 vector
* feat: Decimal128Type in datatypes
* refactor: cast function
* feat: decimal128 type in ConcreteDataType
* chore: decimal
* chore: sql_data_type_to_concrete_data_type
* chore: cr comment
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
* chore: fix finish()
* chore: add debug_assert to new decimal128
---------
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-11-16 06:44:26 +00:00
Ruihang Xia
5d8b0e8154
chore: bump dependencies ( #2659 )
...
* fix all hard error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix nextest
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* trivial changes
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix order by
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix sql keyword and data type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix range exec's input partitioning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix cover input type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix explain analyze
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update distributed mode sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix lints
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update locks
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* downgrade dlv-list
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-10-26 12:24:14 +00:00
Ning Sun
0fbde48655
feat: hide internal error and unknown error message from end user ( #2544 )
...
* feat: use fixed error message for unknown error
* feat: return fixed message for internal error as well
* chore: include status code in error message
* test: update tests for asserts of error message
* feat: change status code of some datafusion error
* fix: make CollectRecordbatch an query error
* test: update sqlness results
2023-10-23 03:07:35 +00:00
JeremyHi
52ac093110
fix: drop table 0 rows affected ( #2515 )
2023-09-28 06:21:18 +00:00
Ruihang Xia
34d6288945
feat: bring back sqlness and integration tests ( #2448 )
...
* enable integration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* disable sqlness region failover
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* enable sqlness in CI
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* sort unstable result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* set require_lease_before_startup to true
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: fix inconsistent cache
* replace windows path chars
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore some integration cases in windows
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Revert "ignore some integration cases in windows"
This reverts commit 122478b7c1waynestxia@gmail.com >
* fix: fix close region bug in RegionHeartbeatResponseHandler
* disable failover tests
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: WenyXu <wenymedia@gmail.com >
2023-09-20 09:17:30 +00:00
dennis zhuang
db89235474
feat: only allow timestamp type as time index ( #2281 )
...
* feat: only allow timestamp data type as time index
* test: update sqltest cases, todo: need some fixes
* fix: sqlness tests
* fix: forgot adding back cte test
* chore: style
2023-09-12 07:57:15 -05:00
dennis zhuang
272f649b22
fix: some TODO in sqlness cases and refactor meta-client error ( #2207 )
...
* fix: some TODO in sqlness cases and refactor meta-client error
* fix: delete tests/cases/standalone/alter/drop_col_not_null_next.output
2023-08-18 10:09:11 +00:00
Ruihang Xia
bee8323bae
chore: bump sqlness to 0.5.0 ( #1877 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-07-04 19:49:12 +08:00
Ruihang Xia
2615718999
feat: merge scan for distributed execution ( #1660 )
...
* generate exec plan
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* move DatanodeClients to client crate
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* wip MergeScanExec::to_stream
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix default catalog
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix expand order of new stage
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* move sqlness cases contains plan out of common dir
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refactor information schema to allow duplicated scan call
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: ignore two cases due to substrait
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* reorganise sqlness common cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typos
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* redact round robin partition number
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: Yingwen <realevenyag@gmail.com >
* skip tranforming projection
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* revert common/order
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/query/src/dist_plan/merge_scan.rs
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result again
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore region failover IT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result again and again
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* unignore some tests about projection
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* enable failover tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2023-06-02 06:42:54 +00:00
LFC
ff2784da0f
test: add SELECT ... LIMIT ... test cases for distributed mode ( #1419 )
2023-04-18 23:05:43 +08:00
Ruihang Xia
a6e41cdd7b
chore: bump arrow, parquet, datafusion and tonic ( #1386 )
...
* bump arrow, parquet, datafusion, tonic and greptime-proto
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add analyzer and fix test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-04-15 00:03:15 +08:00
Ruihang Xia
b5e5f8e555
chore(deps): bump arrow and parquet to 36.0.0, and datafusion to the latest ( #1282 )
...
* chore: update arrow, parquet to 36.0 and datafusion
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update deps
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: LFC <bayinamine@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: LFC <bayinamine@gmail.com >
2023-03-30 16:24:10 +08:00
LFC
11d45e2918
refactor: upgrade DataFusion, Arrow and Sqlparser ( #1074 )
...
* refactor: upgrade DataFusion, Arrow and Sqlparser
* fix: resolve PR comments
2023-02-27 22:20:08 +08:00
LFC
9ad6c45913
test: Sqlness tests for distribute mode ( #979 )
...
* test: Sqlness tests for distribute mode
* ci
* fix: resolve PR comments
* fix: resolve PR comments
2023-02-14 10:24:09 +08:00
