discord9
7caa88abc7
refactor: flow replace check&better error msg ( #5277 )
...
* chore: better error msg
* chore eof newline
* refactor: move replace check to flow worker
* chore: add ctx to insert flow failure
* chore: Update src/flow/src/adapter/flownode_impl.rs
* test: add order by for deterministic
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2025-01-04 02:12:27 +08:00
discord9
5e3c5945c4
fix: flow handle reordered inserts ( #5275 )
...
* fix: reorder correct schema
* tests: reorder insert handled correctly
* chore: rm unused
* refactor: per review
* chore: more comment
* chore: per review
2025-01-04 02:12:27 +08:00
discord9
cf605ecccc
fix(flow): flow's table schema cache ( #5251 )
...
* fix: flow schema cache
* refactor: location for `to_meta_err`
* chore: endfile emptyline
* chore: review(partially)
* chore: per review
* refactor: per review
* refactor: per review
2025-01-04 02:12:27 +08:00
discord9
1f5c2b32e5
fix: flow compare null values ( #5234 )
...
* fix: flow compare null values
* fix: fix again ck ty before cmp
* chore: rm comment
* fix: handle null
* chore: typo
* docs: update comment
* refactor: per review
* tests: more sqlness
* tests: sqlness not show create table
2025-01-04 02:12:27 +08:00
discord9
3ff5754b5a
feat(flow): check sink table mismatch on flow creation ( #5112 )
...
* tests: more mismatch errors
* feat: check sink table schema if exists&prompt nice err msg
* chore: rm unused variant
* chore: fmt
* chore: cargo clippy
* feat: check schema on create
* feat: better err msg when mismatch
* tests: fix a schema mismatch
* todo: create sink table
* feat: create sink table
* fix: find time index
* tests: auto created sink table
* fix: remove empty keys
* refactor: per review
* chore: fmt
* test: sqlness
* chore: after rebase
2025-01-04 02:12:27 +08:00
discord9
9e7121c1bb
fix(flow): batch builder with type ( #5195 )
...
* fix: typed builder
* chore: clippy
* chore: rename
* fix: unit tests
* refactor: per review
2024-12-20 14:12:19 +08:00
discord9
f5e743379f
feat: show flow's mem usage in INFORMATION_SCHEMA.FLOWS ( #4890 )
...
* feat: add flow mem size to sys table
* chore: rm dup def
* chore: remove unused variant
* chore: minor refactor
* refactor: per review
2024-12-20 14:12:19 +08:00
discord9
7aa8c28fe4
test: flow rebuild ( #5162 )
...
* tests: rebuild flow
* tests: more rebuild
* tests: restart
* chore: drop clean
2024-12-20 14:12:19 +08:00
discord9
8b944268da
feat: ttl=0/instant/forever/humantime&ttl refactor ( #5089 )
...
* feat: ttl zero filter
* refactor: use TimeToLive enum
* fix: unit test
* tests: sqlness
* refactor: Option<TTL> None means UNSET
* tests: sqlness
* fix: 10000 years --> forever
* chore: minor refactor from reviews
* chore: rename back TimeToLive
* refactor: split imme request from normal requests
* fix: use correct lifetime
* refactor: rename immediate to instant
* tests: flow sink table default ttl
* refactor: per review
* tests: sqlness
* fix: ttl alter to instant
* tests: sqlness
* refactor: per review
* chore: per review
* feat: add db ttl type&forbid instant for db
* tests: more unit test
2024-12-06 09:20:42 +00:00
discord9
d931389a4c
fix(flow): minor fix about count(*)&sink keyword ( #5061 )
...
* fix: SiNk
* fix: sink&count(*)
* tests: SiNk
* refactor: per review
2024-11-29 03:06:27 +00:00
discord9
4d8fe29ea8
feat: CREATE OR REPLACE FLOW ( #5001 )
...
* feat: Replace flow
* refactor: better show create flow&tests: better check
* tests: sqlness result update
* tests: unit test for update
* refactor: cmp with raw bytes
* refactor: rename
* refactor: per review
2024-11-19 08:44:57 +00:00
discord9
35898f0b2e
test: more sqlness tests for flow ( #4988 )
...
* tests: more flow testcase
* tests(WIP): more tests
* tests: more flow tests
* test: wired regex for sqlness
* refactor: put blog&example to two files
2024-11-14 07:40:14 +00:00
discord9
ff38abde2e
chore: better column schema check for flow ( #4855 )
...
* chore: better column schema check for flow
* chore: better msg
* tests: clean up after tests
* chore: better msg
* chore: per review
* tests: sqlness
2024-10-24 09:43:32 +00:00
discord9
a3d567f0c9
perf(flow): use batch mode for flow ( #4599 )
...
* generic bundle trait
* feat: impl get/let
* fix: drop batch
* test: tumble batch
* feat: use batch eval flow
* fix: div use arrow::div not mul
* perf: not append batch
* perf: use bool mask for reduce
* perf: tiny opt
* perf: refactor slow path
* feat: opt if then
* fix: WIP
* perf: if then
* chore: use trace instead
* fix: reduce missing non-first batch
* perf: flow if then using interleave
* docs: add TODO
* perf: remove unnecessary eq
* chore: remove unused import
* fix: run_available no longer loop forever
* feat: blocking on high input buf
* chore: increase threhold
* chore: after rebase
* chore: per review
* chore: per review
* fix: allow empty values in reduce&test
* tests: more flow doc example tests
* chore: per review
* chore: per review
2024-09-11 03:31:52 +00:00
discord9
9f9c1dab60
feat(flow): use DataFusion's optimizer ( #4489 )
...
* feat: use datafusion optimization
refactor: mv `sql_to_flow_plan` elsewhere
feat(WIP): use df optimization
WIP analyzer rule
feat(WIP): avg expander
fix: transform avg expander
fix: avg expand
feat: names from substrait
fix: avg rewrite
test: update `test_avg`&`test_avg_group_by`
test: fix `test_sum`
test: fix some tests
chore: remove unused flow plan transform
feat: tumble expander
test: update tests
* chore: clippy
* fix: tumble lose `group expr`
* test: sqlness test update
* test: rm unused cast
* test: simplify sqlness
* refactor: per review
* chore: after rebase
* fix: remove a outdated test
* test: add comment
* fix: report error when not literal
* chore: update sqlness test after rebase
* refactor: per review
2024-08-29 02:52:00 +00:00
dennis zhuang
63f2463273
feat!: impl admin command ( #4600 )
...
* feat: impl admin statement parser
* feat: introduce AsyncFunction and implements it for admin functions
* feat: execute admin functions
* fix: license header
* fix: panic in test
* chore: fixed by code review
2024-08-26 07:53:40 +00:00
discord9
e6cc4df8c8
feat: flow recreate on reboot ( #4509 )
...
* feat: flow reboot clean
* refactor: per review
* refactor: per review
* test: sqlness flow reboot
2024-08-05 13:57:48 +00:00
discord9
021ec7b6ac
feat(flow): flush_flow function ( #4416 )
...
* refactor: df err variant
* WIP
* chore: update proto version
* chore: revert mistaken rust-toolchain
* feat(WIP): added FlowService to QueryEngine
* refactor: move flow service to operator
* refactor: flush use flow name not id
* refactor: use full path in macro
* feat: flush flow
* feat: impl flush flow
* chore: remove unused
* chore: meaninful response
* chore: remove unused
* chore: clippy
* fix: flush_flow with proper blocking
* test: sqlness tests added back for flow
* test: better predicate for flush_flow
* refactor: rwlock
* fix: flush lock
* fix: flush lock write then drop
* test: add a new flow sqlness test
* fix: sqlness testcase
* chore: style
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-07-26 23:04:13 +00:00
discord9
9fa9156bde
feat: FLOWS table in information_schema&SHOW FLOWS ( #4386 )
...
* feat(WIP): flow info table
refactor: better err handling&log
feat: add flow metadata to info schema provider
feat(WIP): info_schema.flows
feat: info_schema.flows table
* fix: err after rebase
* fix: wrong comparsion op
* feat: SHOW FLOWS&tests
* refactor: per review
* chore: unused
* refactor: json error
* chore: per review
* test: sqlness
* chore: rm inline error
* refactor: per review
2024-07-19 09:29:36 +00:00
discord9
c595a56ac8
test(flow): ignore flow tests for now ( #4377 )
...
* tests: ignore flow tests for now
* chore: typo
* test: remove file altogether
* fix: last_value sqlness test regex
* fix: last_value part REDACTED
2024-07-16 10:07:21 +00:00
discord9
b2c5f8eefa
test: more sleep when flow insert makes it serial ( #4373 )
...
* tests: more sleep
* fix: wait more time
* fix: update result file
2024-07-16 03:36:31 +00:00
discord9
2b912d93fb
feat: flow perf&fix df func call ( #4347 )
...
* feat: flow perf&fix df func call
feat: make source sender `send` non-blocking
feat: better control of flow worker freq
feat: support transform interval
fix: const folding df func args&tests
tests: update cast const fold
chore: adjust flow work's freq
refactor: batch split
feat: adaptive run freq flow worker&check for errors
chore: better debug log
* refactor: per review
* chore: per zc's review
* chore: per bot review
* chore: remove some `TODO` completed
* docs: add comments for a test
2024-07-15 09:20:04 +00:00
discord9
09fff24ac4
feat: make flow distributed work&tests ( #4256 )
...
feat: flownode frontend client&test
feat: Frontend Client
feat: set frontend invoker for flownode
feat: set frontend invoker for flownode
chore: test script
WIP: test flow distributed
feat: hard coded demo
docs: flownode example toml
feat: add flownode support in runner
docs: comments for node
chore: after rebase
docs: add a todo
tests: move flow tests to common
fix: flownode sqlness dist test
chore: per review
docs: make
fix: make doc
2024-07-05 14:46:44 +00:00
