Ruihang Xia
93f21b188d
chore: unify workspace dependencies and remove some unused ( #2713 )
...
* log-store
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* base64
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* udeps
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* snafu
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* snafu style
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change the workspace deps' style
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-11-09 07:34:22 +00:00
Ruihang Xia
5d8b0e8154
chore: bump dependencies ( #2659 )
...
* fix all hard error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix nextest
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* trivial changes
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix order by
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix sql keyword and data type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix range exec's input partitioning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix cover input type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix explain analyze
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update distributed mode sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix lints
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update locks
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* downgrade dlv-list
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-10-26 12:24:14 +00:00
Ruihang Xia
515ce825bd
feat: stack trace style debug print for error ( #2489 )
...
* impl macro stack_trace_debug
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* manually mark external error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use debug print
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplify the error and warn macro
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix ut
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add docs
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* replace snafu backtrace with location
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-09-26 11:23:21 +00:00
shuiyisong
20f4f7971a
refactor: remove source and location in snafu display ( #2428 )
...
* refactor: remove source pt 1
* refactor: remove source pt 2
* refactor: remove source pt 3
* refactor: remove location pt 1
* refactor: remove location pt 2
* chore: remove rustc files
* chore: fix error case
* chore: fix test
* chore: fix test
* chore: fix cr issue
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com >
---------
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com >
2023-09-21 02:55:24 +00:00
Ning Sun
c39de9072f
refactor: use workspace dependencies for internal modules ( #2119 )
...
* refactor: use workspace dependencies for internal modules
* fix: resolve issue with mock module in datanode
* refactor: update test modules
2023-08-08 11:02:34 +00:00
LFC
46fa3eb629
chore: upgrade rust toolchain to latest nightly ( #2049 )
...
* chore: upgrade rust toolchain to latest nightly
* rebase develop
* update rust toolchain in ci
2023-08-08 07:17:51 +00:00
shuiyisong
7efff2d704
fix: introduce taplo.toml and sort Cargo.toml ( #2096 )
...
* fix: add taplo.toml
* fix: introduce taplo.toml & sort cargo.toml
* chore: remove option in ci too
2023-08-04 06:44:45 +00:00
Ruihang Xia
44f3ed2f74
chore(deps): bump datafusion to the latest commit ( #1967 )
...
* bump deps
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile except pyo3 backend
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix promql-parser metric name matcher
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix pyarrow convert
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix pyo3 compiling
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove deadcode
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update stream adapter display format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix physical optimizer rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-08-01 02:10:49 +00:00
Ruihang Xia
5bd80a74ab
feat: prepare for implementing considering partition key in the distributed planner ( #2000 )
...
* basic impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix frontend logic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add sqlness test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* check substrait compatibility before pushdown
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* going to revert some rules
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix test and clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove println
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-07-31 12:36:23 +00:00
Ben Baoyi
9ff7670adf
refactor:remove common_error::prelude ( #1946 )
...
* feat:Remove common_error::prelude
* fix merge error
* cr comment
* fix error
2023-07-13 10:36:36 +08:00
Ruihang Xia
9153191819
fix: resolve catalog and schema in dist planner ( #1891 )
...
* try resolve catalog and schema
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* upload sqlness case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix information schema case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix unnamed table name
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-07-06 08:08:44 +00:00
Ruihang Xia
8c78368374
refactor: replace #[snafu(backtrace)] with Location ( #1753 )
...
* remove snafu backtrace
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clean up
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-06-12 11:55:33 +08:00
Ruihang Xia
2615718999
feat: merge scan for distributed execution ( #1660 )
...
* generate exec plan
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* move DatanodeClients to client crate
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* wip MergeScanExec::to_stream
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix default catalog
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix expand order of new stage
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* move sqlness cases contains plan out of common dir
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refactor information schema to allow duplicated scan call
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: ignore two cases due to substrait
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* reorganise sqlness common cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typos
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* redact round robin partition number
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: Yingwen <realevenyag@gmail.com >
* skip tranforming projection
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* revert common/order
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/query/src/dist_plan/merge_scan.rs
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result again
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore region failover IT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result again and again
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* unignore some tests about projection
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* enable failover tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2023-06-02 06:42:54 +00:00
Ruihang Xia
ac3666b841
chore(deps): bump arrow/parquet to 40.0, datafuson to the latest HEAD ( #1677 )
...
* fix compile error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove deprecated substrait
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update deps
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* downgrade opendal to 0.33.1
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change finish's impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test results
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore failing cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-05-31 18:55:02 +08:00
Lei, HUANG
8e7ec4626b
refactor: remove useless error ( #1624 )
...
* refactor: remove useless
* fix: remove useless error variant
2023-05-22 22:55:27 +08:00
Lei, HUANG
fb9978e95d
refactor: catalog ( #1454 )
...
* wip
* add schema_async
* remove CatalogList
* remove catalog provider and schema provider
* fix
* fix: rename table
* fix: sqlness
* fix: ignore tonic error metadata
* fix: table engine name
* feat: rename catalog_async to catalog
* respect engine name in table regional value when deregistering tables
* fix: CR
2023-04-26 08:36:40 +00:00
fys
2287db7ff7
fix: execute sql query in another catalog ( #1457 )
2023-04-25 10:30:35 +08:00
Ruihang Xia
dc85a4b5bb
feat: migrate substrait to datafusion official implementation ( #1238 )
...
* some test cases will fail
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* revert version changes
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update substrait-proto version
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update df again
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clean up
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/common/substrait/Cargo.toml
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* revert COPY FROM / COPY TO sqlness to standalone only
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2023-04-17 15:54:35 +08:00
Ruihang Xia
a6e41cdd7b
chore: bump arrow, parquet, datafusion and tonic ( #1386 )
...
* bump arrow, parquet, datafusion, tonic and greptime-proto
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add analyzer and fix test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-04-15 00:03:15 +08:00
LFC
a3e47955b8
feat: information schema ( #1327 )
...
* feat: basic information schema
* show information schema only for current catalog
* fix: fragile tests
2023-04-07 16:50:14 +08:00
Ruihang Xia
da66138e80
refactor(error): remove backtrace, and introduce call-site location for debugging ( #1329 )
...
* wip: global replace
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unneeded tests of errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix ErrorExt trait implementator
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix pyo3 tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-04-06 04:06:00 +00:00
Lei, HUANG
d10de46e03
feat: support timestamp precision on creating table ( #1332 )
...
* feat: support timestamp precision on creating table
* fix sqlness
* fix: substrait representation of different timestamp precision
2023-04-06 11:18:20 +08:00
Weny Xu
451f9d2d4e
feat: support multi table engines ( #1277 )
...
* feat: support multi table engines
* refactor: adapt SqlHandler to support multiple table engines
* refactor: refactor TableEngineManager
* chore: apply review suggestions
* chore: apply review suggestions
* chore: apply review suggestions
* chore: snafu context styling
2023-04-03 14:49:12 +00:00
Ruihang Xia
b5e5f8e555
chore(deps): bump arrow and parquet to 36.0.0, and datafusion to the latest ( #1282 )
...
* chore: update arrow, parquet to 36.0 and datafusion
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update deps
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: LFC <bayinamine@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: LFC <bayinamine@gmail.com >
2023-03-30 16:24:10 +08:00
LFC
11d45e2918
refactor: upgrade DataFusion, Arrow and Sqlparser ( #1074 )
...
* refactor: upgrade DataFusion, Arrow and Sqlparser
* fix: resolve PR comments
2023-02-27 22:20:08 +08:00
Ning Sun
4b58a8a18d
feat: update substrait and prost version ( #1080 )
2023-02-27 15:18:12 +08:00
Ruihang Xia
4cc3ac37d5
feat: add DictionaryVector DataType ( #1061 )
...
* fix stddev and stdvar. try build range function expr
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: add dictionary data type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* preserve timestamp column in range manipulator
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* plan range functions
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-02-23 20:31:07 +08:00
Yingwen
ddbc97befb
refactor: changes CreateTableRequest::schema to RawSchema ( #1018 )
...
* refactor: changes CreateTableRequest::schema to RawSchema
* refactor(grpc-expr): create_table_schema returns RawSchema
2023-02-16 16:04:17 +08:00
Ruihang Xia
36a13dafb7
build(deps): bump tokio to 1.24.2 ( #900 )
...
deps: bump tokio to 1.24.2
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-01-29 11:13:37 +08:00
LFC
d1730a9577
refactor: simplify how Frontend instance handles other protocols ( #831 )
...
* refactor: make influxdb, opentsdb and prometheus read/write goes through GRPC interface, to unify and simplify the Frontend instance either in standalone or distributed mode
2023-01-06 12:19:38 +08:00
Lei, HUANG
8ffc078f88
fix: license header ( #815 )
2023-01-03 15:09:49 +08:00
Ruihang Xia
26a3e93ca7
chore: util workspace deps in more places ( #792 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2022-12-27 16:26:59 +08:00
LFC
dc52a51576
chore: upgrade to Arrow 29.0 and use workspace package and dependencies ( #782 )
...
* chore: upgrade to Arrow 29.0 and use workspace package and dependencies
* fix: resolve PR comments
Co-authored-by: luofucong <luofucong@greptime.com >
2022-12-23 14:28:37 +08:00
Yingwen
b6d29afcd1
ci: Use lld for coverage ( #778 )
...
* ci: Use lld for coverage
* style: Fix clippy
2022-12-22 16:10:37 +08:00
LFC
ea9af42091
chore: upgrade Rust to nightly 2022-12-20 ( #772 )
...
* chore: upgrade Rust to nightly 2022-12-20
* chore: upgrade Rust to nightly 2022-12-20
Co-authored-by: luofucong <luofucong@greptime.com >
2022-12-21 19:32:30 +08:00
LFC
77182f5024
chore: upgrade Arrow to version 28, and DataFusion to 15 ( #771 )
...
Co-authored-by: luofucong <luofucong@greptime.com >
2022-12-21 17:02:11 +08:00
Lei, HUANG
0653301754
feat: replace arrow2 with official implementation 🎉 ( #753 )
...
* chore: kick off. change datafusion/arrow/parquet to target version
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: replace one last datafusion dep
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: arrow_array switch to arrow
* chore: update dep of binary vector
* chore: fix wrong merge commit
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: Switch to datatypes2
* feat: Make recordbatch compile
* chore: sort Cargo.toml
* feat: Fix common::recordbatch compiler errors
* feat: Fix recordbatch test compiling issue
* fix: api crate (#708 )
* fix: rename ConcreteDataType::timestamp_millis_type to ConcreteDataType::timestamp_millisecond_type. fix other warnings regarding timestamp
* fix: revert changes in datatypes2
* fix: helper
* chore: delete datatypes based on arrow2
* feat: Fix some compiler errors in common::query (#710 )
* feat: Fix some compiler errors in common::query
* feat: test_collect use vectors api
* fix: common-query subcrate (#712 )
* fix: record batch adapter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix error enum
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: Fix common::query compiler errors (#713 )
* feat: Move conversion to ScalarValue to value.rs
* fix: Fix common::query compiler errors
This commit also make InnerError pub(crate)
* feat: Implements diff accumulator using WrapperType (#715 )
* feat: Remove usage of opaque error from common::recordbatch
* feat: Remove opaque error from common::query
* feat: Fix diff compiler errors
Now common_function just use common_query's Error and Result. Adds
a LargestType associated type to LogicalPrimitiveType to get the largest
type a logical primitive type can cast to.
* feat: Remove LargestType from NativeType trait
* chore: Update comments
* feat: Restrict Scalar::RefType of WrapperType to itself
Add trait bound `for<'a> Scalar<RefType<'a> = Self>` to WrapperType
* chore: Address CR comments
* chore: Format codes
* fix: fix compile error for mean/polyval/pow/interp ops
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Revert "fix: fix compile error for mean/polyval/pow/interp ops"
This reverts commit fb0b4eb826#716 )
* fix: Fix compiler errors in argmax/rate/median/norm_cdf
* chore: Address CR comments
* fix: fix compile error for mean/polyval/pow/interp ops (#717 )
* fix: fix compile error for mean/polyval/pow/interp ops
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplify type bounds
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: fix argmin/percentile/clip/interp/scipy_stats_norm_pdf errors (#718 )
fix: fix argmin/percentile/clip/interp/scipy_stats_norm_pdf compiler errors
* fix: fix other compile error in common-function (#719 )
* further fixing
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix all compile errors in common function
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: Fix tests and clippy for common-function subcrate (#726 )
* further fixing
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix all compile errors in common function
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* revert test changes
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: row group pruning (#725 )
* fix: row group pruning
* chore: use macro to simplify stats implemetation
* fxi: CR comments
* fix: row group metadata length mismatch
* fix: simplify code
* fix: Fix common::grpc compiler errors (#722 )
* fix: Fix common::grpc compiler errors
This commit refactors RecordBatch and holds vectors in the RecordBatch
struct, so we don't need to cast the array to vector when doing
serialization or iterating the batch.
Now we use the vector API instead of the arrow API in grpc crate.
* chore: Address CR comments
* fix common record batch
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: Fix compile error in server subcrate (#727 )
* fix: Fix compile error in server subcrate
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused type alias
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* explicitly panic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/storage/src/sst/parquet.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix: Fix common grpc expr (#730 )
* fix compile errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* rename fn names
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix styles
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix wranings in common-time
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: pre-cast to avoid tremendous match arms (#734 )
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: upgrade storage crate to arrow and parquet offcial impl (#738 )
* fix: compile erros
* fix: parquet reader and writer
* fix: parquet reader and writer
* fix: WriteBatch IPC encode/decode
* fix: clippy errors in storage subcrate
* chore: remove suspicious unwrap
* fix: some cr comments
* fix: CR comments
* fix: CR comments
* fix: Fix compiler errors in catalog and mito crates (#742 )
* fix: Fix compiler errors in mito
* fix: Fix compiler errors in catalog crate
* style: Fix clippy
* chore: Fix use
* Merge pull request #745
* fix nyc-taxi and util
* Merge branch 'replace-arrow2' into fix-others
* fix substrait
* fix warnings and error in test
* fix: Fix imports in optimizer.rs
* fix: errors in optimzer
* fix: remove unwrap
* fix: Fix compiler errors in query crate (#746 )
* fix: Fix compiler errors in state.rs
* fix: fix compiler errors in state
* feat: upgrade sqlparser to 0.26
* fix: fix datafusion engine compiler errors
* fix: Fix some tests in query crate
* fix: Fix all warnings in tests
* feat: Remove `Type` from timestamp's type name
* fix: fix query tests
Now datafusion already supports median, so this commit also remove the
median function
* style: Fix clippy
* feat: Remove RecordBatch::pretty_print
* chore: Address CR comments
* Update src/query/src/query_engine/state.rs
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* fix: frontend compile errors (#747 )
fix: fix compile errors in frontend
* fix: Fix compiler errors in script crate (#749 )
* fix: Fix compiler errors in state.rs
* fix: fix compiler errors in state
* feat: upgrade sqlparser to 0.26
* fix: fix datafusion engine compiler errors
* fix: Fix some tests in query crate
* fix: Fix all warnings in tests
* feat: Remove `Type` from timestamp's type name
* fix: fix query tests
Now datafusion already supports median, so this commit also remove the
median function
* style: Fix clippy
* feat: Remove RecordBatch::pretty_print
* chore: Address CR comments
* feat: Add column_by_name to RecordBatch
* feat: modify select_from_rb
* feat: Fix some compiler errors in vector.rs
* feat: Fix more compiler errors in vector.rs
* fix: fix table.rs
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: Fix compiler errors in coprocessor
* fix: Fix some compiler errors
* fix: Fix compiler errors in script
* chore: Remove unused imports and format code
* test: disable interval tests
* test: Fix test_compile_execute test
* style: Fix clippy
* feat: Support interval
* feat: Add RecordBatch::columns and fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* fix: Fix All The Tests! (#752 )
* fix: Fix several tests compile errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: some compile errors in tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: compile errors in frontend tests
* fix: compile errors in frontend tests
* test: Fix tests in api and common-query
* test: Fix test in sql crate
* fix: resolve substrait error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: add more test
* test: Fix tests in servers
* fix instance_test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* test: Fix tests in tests-integration
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Lei, HUANG <mrsatangel@gmail.com >
Co-authored-by: evenyag <realevenyag@gmail.com >
* fix: clippy errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: evenyag <realevenyag@gmail.com >
2022-12-15 18:49:12 +08:00
LFC
8959dbcef8
feat: Substrait logical plan ( #704 )
...
* feat: use Substrait logical plan to query data from Datanode in Frontend in distributed mode
* fix: resolve PR comments
* fix: resolve PR comments
* fix: resolve PR comments
Co-authored-by: luofucong <luofucong@greptime.com >
2022-12-06 19:21:57 +08:00
Ruihang Xia
d82a3a7d58
feat: implement most of scalar function and selection conversion in substrait ( #678 )
...
* impl to_df_scalar_function
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* part of scalar functions
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* conjunction over filters
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change the ser/de target to substrait::Plan
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* basic test coverage
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typos and license header
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* logs unsupported extension
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/common/substrait/src/df_expr.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
* address review comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* replace context with with_context
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2022-12-02 14:46:05 +08:00
Dongxu Wang
b6fa316c65
chore: correct typos ( #589 ) ( #592 )
2022-11-21 14:07:45 +08:00
Ruihang Xia
7ba512980a
chore: add APACHE-2.0 license header ( #518 )
...
* feat: add license checker workflow
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix existing header
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* specify license for internal sub-crate
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix rustfmt
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2022-11-15 18:05:46 +08:00
Ruihang Xia
1565c8d236
chore: specify import style in rustfmt ( #460 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2022-11-15 15:58:54 +08:00
Lei, Huang
2137587091
feat: datanode heartbeat ( #377 )
...
* chore: refactor dir for local catalog manager
* refactor: CatalogProvider returns Result
* refactor: SchemaProvider returns Result
* feat: add kv operations to remote catalog
* chore: refactor some code
* feat: impl catalog initialization
* feat: add register table and register system table function
* refactor: add table_info method for Table trait
* chore: add some tests
* chore: add register schema test
* chore: fix build issue after rebase onto develop
* refactor: mock to separate file
* build: failed to compile
* fix: use a container struct to bridge KvBackend and Accessor trait
* feat: upgrade opendal to 0.17
* test: add more tests
* chore: add catalog name and schema name to table info
* chore: add catalog name and schema name to table info
* chore: rebase onto develop
* refactor: common-catalog crate
* chore: refactor dir for local catalog manager
* refactor: CatalogProvider returns Result
* refactor: SchemaProvider returns Result
* feat: add kv operations to remote catalog
* chore: refactor some code
* feat: impl catalog initialization
* feat: add register table and register system table function
* refactor: add table_info method for Table trait
* chore: add some tests
* chore: add register schema test
* chore: fix build issue after rebase onto develop
* refactor: mock to separate file
* build: failed to compile
* fix: use a container struct to bridge KvBackend and Accessor trait
* feat: upgrade opendal to 0.17
* test: add more tests
* chore: add catalog name and schema name to table info
* chore: add catalog name and schema name to table info
* chore: rebase onto develop
* refactor: common-catalog crate
* refactor: remove remote catalog related files
* fix: compilation
* feat: add table version to TableKey
* feat: add node id to TableValue
* fix: some CR comments
* chore: change async fn create_expr_to_request to sync
* fix: add backtrace to errors
* fix: code style
* refactor: merge refactor/catalog-crate
* feat: table key with version
* feat: impl KvBackend for MetaClient
* fix: integrate metaclient
* fix: catalog use local table info as baseline
* fix: sync metsrv
* fix: wip
* fix: update remote catalog on register and deregister
* refactor: CatalogProvider
* refactor: CatalogManager
* fix: catalog key filtering
* fix: pass some test
* refactor: catalog iterating
* fix: CatalogManager::table also requires both catalog_name and schema_name
* chore: merge develop
* chore: merge catalog crate
* fix: adapt to recent meta-client api change
* feat: databode lease
* feat: remote catalog (#356 )
* chore: refactor dir for local catalog manager
* refactor: CatalogProvider returns Result
* refactor: SchemaProvider returns Result
* feat: add kv operations to remote catalog
* chore: refactor some code
* feat: impl catalog initialization
* feat: add register table and register system table function
* refactor: add table_info method for Table trait
* chore: add some tests
* chore: add register schema test
* chore: fix build issue after rebase onto develop
* refactor: mock to separate file
* build: failed to compile
* fix: use a container struct to bridge KvBackend and Accessor trait
* feat: upgrade opendal to 0.17
* test: add more tests
* chore: add catalog name and schema name to table info
* chore: add catalog name and schema name to table info
* chore: rebase onto develop
* refactor: common-catalog crate
* chore: refactor dir for local catalog manager
* refactor: CatalogProvider returns Result
* refactor: SchemaProvider returns Result
* feat: add kv operations to remote catalog
* chore: refactor some code
* feat: impl catalog initialization
* feat: add register table and register system table function
* refactor: add table_info method for Table trait
* chore: add some tests
* chore: add register schema test
* chore: fix build issue after rebase onto develop
* refactor: mock to separate file
* build: failed to compile
* fix: use a container struct to bridge KvBackend and Accessor trait
* feat: upgrade opendal to 0.17
* test: add more tests
* chore: add catalog name and schema name to table info
* chore: add catalog name and schema name to table info
* chore: rebase onto develop
* refactor: common-catalog crate
* refactor: remove remote catalog related files
* fix: compilation
* feat: add table version to TableKey
* feat: add node id to TableValue
* fix: some CR comments
* chore: change async fn create_expr_to_request to sync
* fix: add backtrace to errors
* fix: code style
* refactor: merge refactor/catalog-crate
* feat: table key with version
* feat: impl KvBackend for MetaClient
* fix: integrate metaclient
* fix: catalog use local table info as baseline
* fix: sync metsrv
* fix: wip
* fix: update remote catalog on register and deregister
* refactor: CatalogProvider
* refactor: CatalogManager
* fix: catalog key filtering
* fix: pass some test
* refactor: catalog iterating
* fix: CatalogManager::table also requires both catalog_name and schema_name
* chore: merge develop
* chore: merge catalog crate
* fix: adapt to recent meta-client api change
* feat: datanode heartbeat (#355 )
* feat: add heartbeat task to instance
* feat: add node_id datanode opts
* fix: use real node id in heartbeat and meta client
* feat: distribute table in frontend
* test: distribute read demo
* test: distribute read demo
* test: distribute read demo
* add write spliter
* fix: node id changed to u64
* feat: datanode uses remote catalog implementation
* dist insert integrate table
* feat: specify region ids on creating table (#359 )
* fix: compiling issues
* feat: datanode lease (#354 )
* Some glue code about dist_insert
* fix: correctly wrap string value with quotes
* feat: create route
* feat: frontend catalog (#362 )
* feat: integrate catalog to frontend
* feat: preserve partition rule on create
* fix: print tables on start
* chore: log in create route
* test: distribute read demo
* feat: support metasrv addr command line options
* feat: optimize DataNodeInstance creation (#368 )
* chore: remove unnecessary changes
* chore: revert changes to src/api
* chore: revert changes to src/datanode/src/server.rs
* chore: remove opendal backend
* chore: optimize imports
* chore: revert changes to instance and region ids
* refactor: MetaKvBackend range
* fix: remove some wrap
* refactor: initiation of catalog
* feat: add region id to create table request and add heartbeat task to datanode instance
* fix: fix auto reconnect for heartbeat task
* chore: change TableValue::region_numbers to vec<u32>.
* fix: some tests
* fix: avoid concurrently start Heartbeat task by compare_exchange
* fix: some cr comments
* fix: fix unit tests
Co-authored-by: jiachun <jiachun_fjc@163.com >
Co-authored-by: luofucong <luofucong@greptime.com >
Co-authored-by: fys <1113014250@qq.com >
Co-authored-by: Jiachun Feng <jiachun_feng@proton.me >
2022-11-07 17:10:43 +08:00
Ruihang Xia
e2c28fe374
feat: support data type and schema conversion ( #351 )
...
* feat: type and schema transformer
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* test schema codec
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* support projection and schema
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy warning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* project schema
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typos
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/common/substrait/src/df_logical.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
* more document about type variations
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2022-10-31 15:16:13 +08:00
Lei, Huang
932b30d299
refactor: catalog crate ( #331 )
...
* chore: refactor dir for local catalog manager
* refactor: CatalogProvider returns Result
* refactor: SchemaProvider returns Result
* feat: add kv operations to remote catalog
* chore: refactor some code
* feat: impl catalog initialization
* feat: add register table and register system table function
* refactor: add table_info method for Table trait
* chore: add some tests
* chore: add register schema test
* chore: fix build issue after rebase onto develop
* refactor: mock to separate file
* build: failed to compile
* fix: use a container struct to bridge KvBackend and Accessor trait
* feat: upgrade opendal to 0.17
* test: add more tests
* chore: add catalog name and schema name to table info
* chore: add catalog name and schema name to table info
* chore: rebase onto develop
* refactor: common-catalog crate
* refactor: remove remote catalog related files
* fix: compilation
* feat: add table version to TableKey
* feat: add node id to TableValue
* fix: some CR comments
* chore: change async fn create_expr_to_request to sync
* fix: add backtrace to errors
* fix: code style
* fix: CatalogManager::table also requires both catalog_name and schema_name
* chore: merge develop
2022-10-26 10:50:39 +08:00
Ruihang Xia
7fe39e9187
feat: support quering with logical plan in gRPC layer ( #344 )
...
* impl logical exec & example
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* test on upper api
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add todo to prost dep
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* sign the TODO
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2022-10-25 16:05:53 +08:00
Ruihang Xia
8ab43b65ea
feat: serialize/deserialize logical and execution plan via substrait ( #317 )
...
* fix: change Utf8Array indice type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refactor: remove unused sub-crate
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: impl for both Logical and Execution plan
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refactor: move test-util subcrate into table
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* test: table scan logical plan round trip
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* drop support of physical plan
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* rename trait fns to encode/decode
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* address review comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2022-10-24 15:29:33 +08:00
