* fix(query): validate DistAnalyzeExec child count (#8510)
* fix(query): validate DistAnalyzeExec child count
Signed-off-by: discord9 <discord9@163.com>
* test(query): avoid implicit clone lint
Signed-off-by: discord9 <discord9@163.com>
---------
Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 623145e635)
* fix: avoid panic when negating MIN-valued literals (#8484)
Value::try_negative and the temporal negative() helpers negated with raw
unary minus, which panics (debug) or wraps (release) on MIN values such as
-i64::MIN. try_negative already returns None for the unsigned arms; make
the signed and temporal arms honor that contract via checked_neg /
checked_negative so a MIN literal produces a clean error instead.
Signed-off-by: raphaelroshan <raphaelroshan@gmail.com>
(cherry picked from commit 1a476f76e2)
* fix: preserve nulls in timestamp arrays (#8508)
Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit c7bcddacc5)
* fix: timestamp display precision should respect column schema (#8227) (#8238)
* fix: timestamp display precision should respect column schema (#8227)
Previously, the MySQL writer converted timestamps to NaiveDateTime via
to_chrono_datetime_with_timezone(), then passed the NaiveDateTime to
write_col(). NaiveDateTime::Display uses a fixed 6-digit fractional-
second format, causing:
- TIMESTAMP(3) to show '.195000' instead of '.195'
- TIMESTAMP(9) to show '.195123' instead of '.195123456'
Fix: use Timestamp::to_timezone_aware_string() directly, which formats
with chrono's '%.f' specifier — it strips trailing zeros and preserves
full nanosecond fidelity based on the actual stored unit.
Also add comprehensive unit tests in mysql_writer_test.rs covering:
- All concrete data type → MySQL column type mappings
- UNSIGNED_FLAG propagation for unsigned integer types
- Timestamp precision for all four units (0/3/6/9 decimal places)
- Edge cases: zero subseconds, trailing-zero stripping, Unix epoch,
negative timestamps, and timezone offset shifts
- Column-def ordering, empty schema, and decimal variants
Signed-off-by: Divyansh <anshmcs@gmail.com>
* perf: reuse format buffer for timestamp serialization to avoid per-row heap allocation
Signed-off-by: Divyansh <anshmcs@gmail.com>
* refactor: rename test variable and update visibility of create_mysql_column function
Signed-off-by: Divyansh <anshmcs@gmail.com>
* test: fix timestamp test constants and add sqlness case for display precision
- Correct the base epoch constant: 1_748_836_200 is 2025-06-02, not
2026-06-02; use 1_780_372_200 to match the documented instant.
- Fix subsecond expectations: chrono's %.f renders fractional digits in
groups of 3 (.100 / .010), it does not strip to .1 / .01.
- Add sqlness case timestamp_precision_display reproducing issue #8227
over the MySQL protocol (TIMESTAMP(0/3/6/9) rendering).
- cargo fmt reflow of create_mysql_column signature.
Signed-off-by: Divyansh <anshmcs@gmail.com>
* fix: support binary protocol for timestamp column
Signed-off-by: Divyansh <anshmcs@gmail.com>
* test: update sqlness results for new timestamp display precision
Signed-off-by: Divyansh <anshmcs@gmail.com>
---------
Signed-off-by: Divyansh <anshmcs@gmail.com>
(cherry picked from commit 1acdec9334)
* fix(mito): chunk manifest object writes (#8567)
Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit d1e61a4ad2)
* fix: display codes in metasrv client errors (#8558)
* fix: display codes in metasrv client errors
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: normalize meta client errors in sqlness
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit b1580c98f6)
* fix(mito): notify bulk writes on WAL error (#8563)
* fix(mito): notify bulk writes on WAL error
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito): move write ctx tests after items
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs: mention PR template in agent guide
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
(cherry picked from commit 25909bd1aa)
* fix: ignore dropping marker during GC (#8588)
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
(cherry picked from commit 036db3d1ec)
* fix(query): preserve bare plan names in analyze json (#8519)
Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 4a0dd8f9bc)
* fix(flow): lower routine batching messages to debug (#8592)
fix(flow): reduce expected batching log noise
Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 9f5a11e82a)
* fix(flow): downgrade disabled incremental checkpoint log (#8572)
Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 4c5be47400)
* fix: demote expected remote dynamic filter misses (#8574)
Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 45fc51ebd2)
* fix: prevent credential leaks in sanitize_connection_string (#8539)
Parse PostgreSQL DSNs (URL or libpq keyword) with tokio_postgres::Config —
the backend's own parser — and log its Debug, which redacts the password.
This matches the backend grammar exactly (multi-host URIs, backslash
escapes, any Unicode whitespace, percent-encoded query keys, and '&'/';'/
'://' inside values) rather than approximating it by hand. Other URLs are
redacted via the url crate; a best-effort keyword fallback covers inputs
neither parser accepts.
Signed-off-by: raphaelroshan <raphaelroshan@gmail.com>
(cherry picked from commit e19df8960e)
* fix(mito2): remove stale compaction status when next compaction is throttled (#8618)
When a compaction finishes with no pending request/DDL, on_compaction_finished
leaves the region's CompactionStatus in the map with no active compaction. If
the worker then skips schedule_next_compaction because min_compaction_interval
has not passed, the stale status is never cleaned up: all subsequent compaction
requests are swallowed by the "status exists" branch (regular waiters hang,
manual StrictWindow requests pend forever) and the region never compacts again
until restart.
Add CompactionScheduler::remove_inactive_status() and call it from
handle_compaction_finished when the interval gate skips scheduling the next
compaction, restoring the invariant that a status in the map always has an
active compaction.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
(cherry picked from commit d1ac9e2da7)
* fix(meta): configure gRPC message limits (#8616)
* fix(meta): configure gRPC message limits
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(meta): preserve follower range limits
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 82de463940)
* fix(flow): convert streaming expiration to milliseconds (#8481)
* fix(flow): convert streaming expiration to milliseconds
Flow EXPIRE AFTER values are stored in seconds, while streaming flow timestamps and durations use milliseconds. Passing the value through unchanged expires state and limits refill scans 1000 times too early.
Convert the value at streaming create and refill boundaries, reject overflow, and document the CreateFlowArgs unit.
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
* fix(flow): reject negative streaming expiration
Reject negative EXPIRE AFTER values at the streaming adapter boundary so refill ranges cannot move into the future. Keep zero valid and retain checked seconds-to-milliseconds conversion.
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
---------
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
* fix: qualify scalar-subquery tables in persisted views (#8581)
fix(query): traverse scalar subqueries in views
Signed-off-by: discord9 <discord9@163.com>
* chore: bump version
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: raphaelroshan <raphaelroshan@gmail.com>
Signed-off-by: Divyansh <anshmcs@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: raphaelroshan <49832307+raphaelroshan@users.noreply.github.com>
Co-authored-by: Div <anshmcs@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: QuakeWang <45645138+QuakeWang@users.noreply.github.com>
* feat: support alter from primary_key to flat
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: alter flat to primary_key
Signed-off-by: evenyag <realevenyag@gmail.com>
* feat: change default_experimental_flat_format to true
Signed-off-by: evenyag <realevenyag@gmail.com>
* feat: compute channel size from splitted batch size
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: add tests for split and channel size
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: always set sst_format from manifest on region open
sanitize_region_options did not set options.sst_format when the
default (PrimaryKey) matched the manifest value, leaving it as None
after reopen. This caused the alter format change to appear lost.
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: fix tests
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: show create table after alteration
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor!: rename default_experimental_flat_format to default_flat_format
The flat format is no longer experimental. Remove "experimental" from
the config field name, doc comments, and all references.
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: fix clippy
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor: add datafusion-postgres dependency
* refactor: move and include pg_catalog udfs
* chore: update upstream
* feat: register table function pg_get_keywords
* feat: bridge CatalogInfo for our CatalogManager
Signed-off-by: Ning Sun <sunning@greptime.com>
* feat: convert pg_catalog table to our system table
* feat: bridge system catalog with datafusion-postgres
Signed-off-by: Ning Sun <sunning@greptime.com>
* feat: add more udfs
* feat: add compatibility rewriter to postgres handler
* fix: various fix
* fmt: fix
* fix: use functions from pg_catalog library
* fmt
* fix: sqlness runner
Signed-off-by: Ning Sun <sunning@greptime.com>
* test: adopt arrow 56.0 to 56.1 memory size change
* fix: add additional udfs
* chore: format
* refactor: return None when creating system table failed
Signed-off-by: Ning Sun <sunning@greptime.com>
* chore: provide safety comments about expect usage
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
* feat: use flow batching engine
broken: try using logical plan
fix: use dummy catalog for logical plan
fix: insert plan exec&sqlness grpc addr
feat: use frontend instance in flownode in standalone
feat: flow type in metasrv&fix: flush flow out of sync& column name alias
tests: sqlness update
tests: sqlness flow rebuild udpate
chore: per review
refactor: keep chnl mgr
refactor: use catalog mgr for get table
tests: use valid sql
fix: add more check
refactor: put flow type determine to frontend
* chore: update proto
* chore: update proto to main branch
* fix: add locks for create/drop flow&docs: update docs
* feat: flush_flow flush all ranges now
* test: add align time window test
* docs: explain `nodeid` use in check task
* refactor: AddAutoColumnRewriter check for Projection
* refactor: per review
* fix: query without time window also clean dirty time window
* chore: better logging
* chore: add comments per review
* refactor: per review
* chore: per review
* chore: per review rename args
* refactor: per review partially
* chore: update docs
* chore: use better error variant
* chore: better error variant
* refactor: rename FlowWorkerManager to FlowStreamingEngine
* rename again
* refactor: per review
* chore: rebase after #5963 merged
* refactor: rename all flow_worker_manager occurs
* docs: rm resolved TODO
* feat: add mysql election
* feat: add mysql election
* chore: fix deps
* chore: fix deps
* fix: duplicate container
* fix: duplicate setup for sqlness
* fix: call once
* fix: do not use NOWAIT for mysql 5.7
* chore: apply comments
* fix: no parallel sqlness for mysql
* chore: comments and minor revert
* chore: apply comments
* chore: apply comments
* chore: add to table name
* ci: use 2 metasrv to detect election bugs
* refactor: better election logic
* chore: apply comments
* chore: apply comments
* feat: version check before startup
* feat(flow): (Part 1) refill utils
* chore: after rebase fix
* chore: more rebase
* rm refill.rs to reduce pr size
* chore: simpler args
* refactor: per review
* docs: more explain for instant requests
* refactor: per review
* feat: simple version switch
* chore: remove debug print
* chore: add common folder
* tests: add drop table
* feat: pull versioned binary
* chore: don't use native-tls
* chore: rm outdated docs
* chore: new line
* fix: save old bin dir
* fix: switch version restart all node
* feat: use etcd
* fix: wait for election
* fix: normal sqlness
* refactor: hashmap for bin dir
* test: past 3 major version compat crate table
* refactor: allow using without setup etcd
* feat: use `Inserter` as Frontend
* fix: enable procedure in flownode
* docs: remove `frontend_addr` opts
* chore: rm fe addr in test runner
* refactor: int test also use inserter invoker
* feat: flow shutdown&refactor: remove `Frontendinvoker`
* refactor: rename `RemoteFrontendInvoker` to `FrontendInvoker`
* refactor: per review
* refactor: remove a layer of box
* fix: standalone use `node_manager`
* fix: remove a `Arc` cycle
feat: flownode frontend client&test
feat: Frontend Client
feat: set frontend invoker for flownode
feat: set frontend invoker for flownode
chore: test script
WIP: test flow distributed
feat: hard coded demo
docs: flownode example toml
feat: add flownode support in runner
docs: comments for node
chore: after rebase
docs: add a todo
tests: move flow tests to common
fix: flownode sqlness dist test
chore: per review
docs: make
fix: make doc
* refactor: bring metrics to http output
* chore: remove unwrap
* chore: make walk plan accumulate
* chore: change field name and comment
* chore: add metrics to http resp header
* chore: move PrometheusJsonResponse to a separate file and impl IntoResponse
* chore: put metrics in prometheus resp header too
* feat: let TypeConversionRule aware query context timezone setting
* chore: don't optimize explain command
* feat: parse string into timestamp with timezone
* fix: compile error
* chore: check the scalar value type in predicate
* chore: remove mut for engine context
* chore: return none if the scalar value is utf8 in time range predicate
* fix: some fixme
* feat: let Date and DateTime parsing from string value be aware of timezone
* chore: tweak
* test: add datetime from_str test with timezone
* feat: construct function context from query context
* test: add timezone test for to_unixtime and date_format function
* fix: typo
* chore: apply suggestion
* test: adds string with timezone
* chore: apply CR suggestion
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
* chore: apply suggestion
---------
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Weny Xu
Ruihang Xia
QuakeWang
Yingwen
LFC
dennis zhuang
yihong
fys
Ning Sun
Cason Kervis
liyang
discord9
Yohan Wal
jeremyhi
JohnsonLee
Lei, HUANG
Lanqing Yang
tison
dimbtp
shuiyisong