Ruihang Xia
548e1988ab
refactor: remove unused symbols ( #5193 )
...
chore: remove unused symbols
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-18 11:24:43 +00:00
shuiyisong
9d7fea902e
chore: remove unused dep ( #5163 )
...
* chore: remove unused dep
* chore: remove more unused dep
2024-12-16 06:17:27 +00:00
discord9
d931389a4c
fix(flow): minor fix about count(*)&sink keyword ( #5061 )
...
* fix: SiNk
* fix: sink&count(*)
* tests: SiNk
* refactor: per review
2024-11-29 03:06:27 +00:00
discord9
4d8fe29ea8
feat: CREATE OR REPLACE FLOW ( #5001 )
...
* feat: Replace flow
* refactor: better show create flow&tests: better check
* tests: sqlness result update
* tests: unit test for update
* refactor: cmp with raw bytes
* refactor: rename
* refactor: per review
2024-11-19 08:44:57 +00:00
discord9
a70b4d7eba
chore: update greptime-proto to e1070a ( #4992 )
...
* chore: update protot to e1070a
* fix: update proto usage
2024-11-14 11:59:00 +00:00
discord9
35898f0b2e
test: more sqlness tests for flow ( #4988 )
...
* tests: more flow testcase
* tests(WIP): more tests
* tests: more flow tests
* test: wired regex for sqlness
* refactor: put blog&example to two files
2024-11-14 07:40:14 +00:00
LFC
2e9737c01d
refactor: pass LogicalPlan to promql execution interceptor ( #4937 )
2024-11-05 08:49:05 +00:00
dennis zhuang
7a4276c24a
fix: typo ( #4931 )
...
fix/database-base-ttl:
Fix typos in comments and function names across multiple modules
- Correct spelling of 'parallelism' in region_server, engine, and scan_region modules
- Amend typo in TODO comment from 'persisent' to 'persistent' in server module
- Update incorrect test query from 'versiona' to 'version' in federated module tests
Co-authored-by: Lei, HUANG <mrsatangel@gmail.com >
2024-11-04 01:56:11 +00:00
LFC
fd8eba36a8
refactor: make use of the "pre_execute" in sql execution interceptor ( #4875 )
...
* feat: dynamic definition of plugin options
* rebase
* revert
* fix ci
2024-10-30 09:16:46 +00:00
Lei, HUANG
e328c7067c
chore: udapte Rust toolchain to 2024-10-19 ( #4857 )
...
* update rust toolchain
* change toolchain to 2024-10-17
* fix: clippy
* fix: ut
* bump shadow-rs
* fix: use nightly-2024-10-19
* fix: clippy
* chore/udapte-toolchain-2024-10-17: Update DEV_BUILDER_IMAGE_TAG to 2024-10-19-a5c00e85-20241024184445 in Makefile
2024-10-25 00:23:32 +00:00
discord9
ff38abde2e
chore: better column schema check for flow ( #4855 )
...
* chore: better column schema check for flow
* chore: better msg
* tests: clean up after tests
* chore: better msg
* chore: per review
* tests: sqlness
2024-10-24 09:43:32 +00:00
Yingwen
2f2b4b306c
feat!: implement interval type by multiple structs ( #4772 )
...
* define structs and methods
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: re-implement interval types in time crate
* feat: use new
* feat: interval value
* feat: query crate interval
* feat: pg and mysql interval
* chore: remove unused imports
* chore: remove commented codes
* feat: make flow compile but may not work
* feat: flow datetime
* test: fix some tests
* test: fix some flow tests(WIP)
* chore: some fix test&docs
* fix: change interval order
* chore: remove unused codes
* chore: fix cilppy
* chore: now signature change
* chore: remove todo
* feat: update error message
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: discord9 <discord9@163.com >
2024-10-14 03:09:03 +00:00
zyy17
e39a9e6feb
feat: add StatementStatistics for slow query logging implementation ( #4719 )
...
* feat: log slow query
* feat: log slow query for sql
* refactor: add slow query logging options
* ci: fix errors
* feat: add StatementStatistics
* chore: revert modification of servers crate
* docs: update config docs
* fix: clippy errors
2024-09-30 03:26:50 +00:00
Ruihang Xia
f5cf25b0db
refactor: remove DfPlan wrapper ( #4733 )
...
* refactor: remove DfPlan wrapper
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clean up
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix test assertion
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-19 12:29:33 +00:00
discord9
a3d567f0c9
perf(flow): use batch mode for flow ( #4599 )
...
* generic bundle trait
* feat: impl get/let
* fix: drop batch
* test: tumble batch
* feat: use batch eval flow
* fix: div use arrow::div not mul
* perf: not append batch
* perf: use bool mask for reduce
* perf: tiny opt
* perf: refactor slow path
* feat: opt if then
* fix: WIP
* perf: if then
* chore: use trace instead
* fix: reduce missing non-first batch
* perf: flow if then using interleave
* docs: add TODO
* perf: remove unnecessary eq
* chore: remove unused import
* fix: run_available no longer loop forever
* feat: blocking on high input buf
* chore: increase threhold
* chore: after rebase
* chore: per review
* chore: per review
* fix: allow empty values in reduce&test
* tests: more flow doc example tests
* chore: per review
* chore: per review
2024-09-11 03:31:52 +00:00
Ruihang Xia
93f202694c
refactor: remove unused error variants ( #4666 )
...
* add python script
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix all negative cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* setup CI
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add license header
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-03 13:19:38 +00:00
discord9
8c8499ce53
perf(flow): Map&Reduce Operator use batch to reduce alloc ( #4567 )
...
* feat: partial impl mfp
* feat: eval batch inner
* chore: fmt
* feat: mfp eval_batch
* WIP
* feat: Collection generic over row&Batch
* feat: render source batch
* chore: chore
* feat: render mfp batch
* feat: render reduce batch(WIP)
* feat(WIP): render reduce
* feat: reduce batch
* feat: render sink batch
* feat: render constant batch
* chore: error handling& mfp batch test
* test: mfp batch
* chore: rm import
* test: render reduce batch
* chore: add TODO
* chore: per bot review
* refactor: per review
* chore: cmt
* chore: rename
* docs: update no panic
2024-08-29 07:28:13 +00:00
discord9
9f9c1dab60
feat(flow): use DataFusion's optimizer ( #4489 )
...
* feat: use datafusion optimization
refactor: mv `sql_to_flow_plan` elsewhere
feat(WIP): use df optimization
WIP analyzer rule
feat(WIP): avg expander
fix: transform avg expander
fix: avg expand
feat: names from substrait
fix: avg rewrite
test: update `test_avg`&`test_avg_group_by`
test: fix `test_sum`
test: fix some tests
chore: remove unused flow plan transform
feat: tumble expander
test: update tests
* chore: clippy
* fix: tumble lose `group expr`
* test: sqlness test update
* test: rm unused cast
* test: simplify sqlness
* refactor: per review
* chore: after rebase
* fix: remove a outdated test
* test: add comment
* fix: report error when not literal
* chore: update sqlness test after rebase
* refactor: per review
2024-08-29 02:52:00 +00:00
Weny Xu
25cd61b310
chore: upgrade toolchain to nightly-2024-08-07 ( #4549 )
...
* chore: upgrade toolchain to `nightly-2024-08-07`
* chore(ci): upgrade toolchain
* fix: fix unit test
2024-08-22 11:02:18 +00:00
discord9
2c3fccb516
feat(flow): add eval_batch for ScalarExpr ( #4551 )
...
* refactor: better perf flow
* feat(WIP): batching proc
* feat: UnaryFunc::eval_batch untested
* feat: BinaryFunc::eval_batch untested
* feat: VariadicFunc::eval_batch un tested
* feat: literal eval_batch
* refactor: move DfScalarFunc to separate file
* chore: remove unused imports
* feat: eval_batch df func&ifthen
* chore: remove unused file
* refactor: use Batch type
* chore: remove unused
* chore: remove a done TODO
* refactor: per review
* chore: import
* refactor: eval_batch if then
* chore: typo
2024-08-14 11:29:30 +00:00
discord9
cbb06cd0c6
feat(flow): add some metrics ( #4539 )
...
* feat: add some metrics
* fix: tmp rate limiter
* feat: add task count metrics
* refactor: use bounded channel anyway
* refactor: better metrics
2024-08-14 03:23:49 +00:00
discord9
e6cc4df8c8
feat: flow recreate on reboot ( #4509 )
...
* feat: flow reboot clean
* refactor: per review
* refactor: per review
* test: sqlness flow reboot
2024-08-05 13:57:48 +00:00
Ruihang Xia
7daf24c47f
feat: remove dedicated runtime for grpc, mysql and pg protocols ( #4436 )
...
* feat: remove dedicated runtime for grpc, mysql and pg protocols
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove other runtimes
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* spawn compact task into compact_runtime
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refine naming
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/servers/tests/mysql/mysql_server_test.rs
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* turnoff fuzz test matrix fail fast option
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: update rt config for ci tests
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-07-30 06:17:58 +00:00
discord9
021ec7b6ac
feat(flow): flush_flow function ( #4416 )
...
* refactor: df err variant
* WIP
* chore: update proto version
* chore: revert mistaken rust-toolchain
* feat(WIP): added FlowService to QueryEngine
* refactor: move flow service to operator
* refactor: flush use flow name not id
* refactor: use full path in macro
* feat: flush flow
* feat: impl flush flow
* chore: remove unused
* chore: meaninful response
* chore: remove unused
* chore: clippy
* fix: flush_flow with proper blocking
* test: sqlness tests added back for flow
* test: better predicate for flush_flow
* refactor: rwlock
* fix: flush lock
* fix: flush lock write then drop
* test: add a new flow sqlness test
* fix: sqlness testcase
* chore: style
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-07-26 23:04:13 +00:00
discord9
9fa9156bde
feat: FLOWS table in information_schema&SHOW FLOWS ( #4386 )
...
* feat(WIP): flow info table
refactor: better err handling&log
feat: add flow metadata to info schema provider
feat(WIP): info_schema.flows
feat: info_schema.flows table
* fix: err after rebase
* fix: wrong comparsion op
* feat: SHOW FLOWS&tests
* refactor: per review
* chore: unused
* refactor: json error
* chore: per review
* test: sqlness
* chore: rm inline error
* refactor: per review
2024-07-19 09:29:36 +00:00
discord9
2b912d93fb
feat: flow perf&fix df func call ( #4347 )
...
* feat: flow perf&fix df func call
feat: make source sender `send` non-blocking
feat: better control of flow worker freq
feat: support transform interval
fix: const folding df func args&tests
tests: update cast const fold
chore: adjust flow work's freq
refactor: batch split
feat: adaptive run freq flow worker&check for errors
chore: better debug log
* refactor: per review
* chore: per zc's review
* chore: per bot review
* chore: remove some `TODO` completed
* docs: add comments for a test
2024-07-15 09:20:04 +00:00
discord9
1ddf19d886
feat: flownode use Inserter to write to database ( #4323 )
...
* feat: use `Inserter` as Frontend
* fix: enable procedure in flownode
* docs: remove `frontend_addr` opts
* chore: rm fe addr in test runner
* refactor: int test also use inserter invoker
* feat: flow shutdown&refactor: remove `Frontendinvoker`
* refactor: rename `RemoteFrontendInvoker` to `FrontendInvoker`
* refactor: per review
* refactor: remove a layer of box
* fix: standalone use `node_manager`
* fix: remove a `Arc` cycle
2024-07-09 10:44:22 +00:00
Zhenchi
00308218b3
feat(fulltext_index): allow enable full-text index in SQL and gRPC way ( #4310 )
...
* feat(fulltext_index): allow enable full-text index in SQL and gRPC way
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: typo
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: test_fulltext_intm_path
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: explicitly build column options
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* test: fix error msg
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-07-08 20:18:48 +00:00
discord9
09fff24ac4
feat: make flow distributed work&tests ( #4256 )
...
feat: flownode frontend client&test
feat: Frontend Client
feat: set frontend invoker for flownode
feat: set frontend invoker for flownode
chore: test script
WIP: test flow distributed
feat: hard coded demo
docs: flownode example toml
feat: add flownode support in runner
docs: comments for node
chore: after rebase
docs: add a todo
tests: move flow tests to common
fix: flownode sqlness dist test
chore: per review
docs: make
fix: make doc
2024-07-05 14:46:44 +00:00
discord9
c21e969329
fix: call df_func with literal ( #4265 )
...
* fix: call df_func with literal
* chore: rm dbg log forget to remove
2024-07-05 06:21:22 +00:00
Jeremyhi
ddf2e6a3c0
feat: provide a simple way to create metaclient ( #4257 )
...
* feat: provide a simple way to create metaclient
* chore: minor refactor using metaclient
* chore: minor refactor using metaclient
2024-07-03 08:11:55 +00:00
discord9
f035a7c79c
feat: flow cli for distributed ( #4226 )
...
* feat(WIP): add FlownodeInstance for flow cli
* feat(WIP): cli
* feat: add merge opts func
* refactor: move server&error to src dir
* feat: flownode cli build
* feat: add `flownode` subcmd to cli
* refactor: per review
* refactor!: BREAKING remove alias `metasrv-addr`
* chore: after rebase
* feat: cache invalide flownode cache
* chore: small refactor per review
* chore: fix a typo
* feat!: revert breaking change
* chore: per review
* refactor: not accept `metasrv-addr` only for flownode
2024-07-01 09:56:15 +00:00
Jeremyhi
214fd38f69
feat: add build info for flow heartbeat task ( #4228 )
...
* chore: refactor load region stats
* feat: add build info for flow heartbeat
2024-07-01 03:19:25 +00:00
discord9
b6585e3581
refactor(flow): make from_substrait_* async& worker handle refactor ( #4210 )
...
* refactor: use oneshot to receive result
* refactor: make from_substrait_* async
* refacrot: remove serde for plan&expr
2024-06-27 17:17:46 +00:00
dennis zhuang
a779cb36ec
fix: wrong frontend registration address ( #4199 )
...
* fix: frontend registration address is wrong, #4186
* fix: license header
* chore: adds hostname to frontend grpc
* fix: forgot run make config-docs
* chore: warn when using bind_addr
* fix: flow node heartbeat carrying address
2024-06-26 06:13:07 +00:00
discord9
a44fe627ce
feat: heartbeat task&peer lookup in proc ( #4179 )
...
* feat: herat beat task
* feat: use real flow peer allocator when building
* feat: add peer look up in ddl context
* fix: drop flow test
* refactor: per review(WIP)
* refactor: not check if is alive
* refactor: per review
* refactor: remove useless `reset`
* refactor: per bot advices
* refactor: alive peer
* chore: bot review
2024-06-24 15:06:33 +00:00
discord9
517917453d
fix(flow): fix call df func bug&sqlness test ( #4165 )
...
* tests: flow sqlness tests
* tests: WIP df func test
* fix: use schema before expand for transform expr
* tests: some basic flow tests
* tests: unit test
* chore: dep use rev not patch
* fix: wired sqlness error?
* refactor: per review
* fix: temp sqlness bug
* fix: use fixed sqlness
* fix: impl drop as async shutdown
* refactor: per bot's review
* tests: drop worker handler both sync/async
* docs: add rationale for test
* refactor: per review
* chore: fmt
2024-06-24 07:52:45 +00:00
LFC
cc2f7efb98
chore: bump datafusion version to fix last_value regression ( #4169 )
...
* chore: bump datafusion version to fix `last_value` regression
* fix: resolve PR comments
* fix ci
2024-06-19 07:47:17 +00:00
discord9
cd9705ccd7
feat(flow): support datafusion scalar function ( #4142 )
...
* chore: call df function types
* feat: RelationDesc to DfSchema
* refactor: use RelationDesc instead of Type
* chore: WIP get to phy expr
* feat: custom deserialize
* chore: fmt
* refactor: renaming to DfScalarFunction
* feat: eval df func(untested)
* fix: had to spawn a thread for calling async
* chore: per review advices
* tests: test df scalar function
2024-06-18 12:34:38 +00:00
discord9
1b00526de5
fix(flow): infer table schema correctly ( #4113 )
...
* refactor: make individual col name optional
* chore: rename TypedPlan's `typ` to `schema`
* feat: add optional col name to typed plan
* feat: pass col name all along
* feat: correct infer output table schema
* chore: unused import
* fix: error when key is not projected
* refactor: per review
* chore: fmt
2024-06-11 08:57:47 +00:00
discord9
0a07130931
fix(flow): mfp operator missing rows ( #4084 )
...
* fix: mfp missing rows if run twice in same tick
* tests: run mfp for multiple times
* refactor: make mfp less hacky
* feat: make channel larger
* chore: typos
2024-06-04 09:07:13 +00:00
shuiyisong
d1838fb28d
refactor: move define_into_tonic_status to common-error ( #4095 )
...
* chore: finish cherry-pick
* chore: remove unused code
2024-06-04 03:29:15 +00:00
Ruihang Xia
45fee948e9
fix: display error in correct format ( #4082 )
...
* fix: display error in correct format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add address to RegionServer error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-05-31 09:25:14 +00:00
discord9
ea49f8a5c4
feat(flow): make write path faster with shared lock ( #4073 )
...
* feat(WIP): make write faster
* feat: read lock on fast path
* chore: per review
2024-05-31 06:50:22 +00:00
dennis zhuang
24612f62dd
feat: querying from view works ( #3952 )
...
* feat: querying from view works
* feat: use MemoryCatalogProviderList instead of DummyCatalogList
* refactor: revert src/query/src/dummy_catalog.rs
* chore: clean code
* fix: make clippy happy
* fix: toml format
* fix: sqlness
* fix: forgot files
* fix: make sqlness happy
* test: table source, serializer and decoder
* fix: fail to decode plan because of invalid table names
* test: adds more sqlness test for view
* chore: remove unused errors
* fix: comments
* fix: typo
* fix: invalidate view info cache after creating view successfully
* chore: apply suggestion
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* chore: apply suggestion
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* fix: compile error after rebeasing
* chore: style
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* fix: don't export table_name in common-meta
* chore: change ViewInfo::new signature
* docs: leave a TODO for optimize param
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-05-30 21:45:56 +00:00
discord9
7de336f087
fix(flow): correctness bugs ( #4018 )
...
* fix: optional args of tumble
* fix(WIP): choose
* feat: rename default ts to GREPTIME_TIMESTAMP
* fix: default timestamp name
* fix: reorder write requests
* fix: expire state
* fix: test of tumble
* fix: send buf clear
* fix: ts<start time correct window
* fix: window_start when ts<start_time
* mend
* fix: range begin>range end
* refactor: per reviews
* feat!: ts placeholder rename to __ts_placeholder
* refactor: better condition
* tests(WIP): func sig choose
* tests(WIP): tumble func
* feat: make run_available optional blocking
* tests: tumble transform
* chore: clippy
* fix?: lagged missing data
* fix: flow source break on empty chnl
2024-05-30 03:49:11 +00:00
Ruihang Xia
1de17aec74
feat: change EXPIRE WHEN to EXPIRE AFTER ( #4002 )
...
* feat: change EXPIRE WHEN to EXPIRE AFTER
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change remaining
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* rename create_if_not_exist to create_if_not_exists
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* parse interval expr
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update comment
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: Jeremyhi <jiachun_feng@proton.me >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Jeremyhi <jiachun_feng@proton.me >
2024-05-27 04:05:55 +00:00
discord9
533ada70ca
chore: remove a dbg! forget to remove ( #3990 )
...
* chore: remove a dbg! forget to remove
* remove other dbg! and add lint
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix pyo3 feature
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-05-20 08:34:47 +00:00
discord9
3477fde0e5
feat(flow): tumble window func ( #3968 )
...
* feat(WIP): tumble window rewrite parser
* tests: tumble func
* feat: add `update_at` column for all flow output
* chore: cleanup per review
* fix: update_at not as time index
* fix: demo tumble
* fix: tests&tumble signature&accept both ts&datetime
* refactor: update_at now ts millis type
* chore: per review advices
2024-05-17 12:10:28 +00:00
discord9
a45017ad71
feat(flow): expire arrange according to time_index type ( #3956 )
...
* feat: render_reduce's arrangement expire after time passed
* feat: set expire when create flow
2024-05-16 11:41:03 +00:00
