Commit Graph
150 Commits
Author SHA1 Message Date
+1 c60311973d fix: backport v1.2.0 correctness and liveness fixes (#9023)
* fix(mito2): fence checkpoints during region transitions (#8847)

* fix: fence checkpoints during region transitions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(datanode): fix transient downgrade setup

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(mito2): fix checkpoint lifecycle test setup

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(mito2): cover cancelled downgrade waiter retry

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): fence direct follower transitions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: trim checkpoint transition coverage

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(mito2): clarify checkpoint task lifecycle

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit a502dfdefd)

* fix(mito2): split SSTs at primary key series boundaries (#8888)

* fix(mito2): split SSTs at series boundaries

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(mito2): cover SST splitting without primary key

Also document the sortedness precondition and the series boundary
split semantics on write_all_flat/write_all_flat_as_primary_key and
the new split helpers.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(mito2): avoid per-row Vec allocation for empty primary key

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* Update src/mito2/src/sst/parquet/writer.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
(cherry picked from commit 4ac3423261)

* fix(meta): allow manual migration from offline datanodes (#8934)

* fix(meta): allow migration from offline datanodes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: fix offline migration event actor

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: read migration routes from metadata

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 8a473c5bf0)

* fix(flight): bound DoGet response wait (#8943)

* fix(flight): defer datanode query initialization

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(client): retain Flight stream peer context

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(client): improve Flight stream diagnostics

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 28398138ec)

* fix(query): keep INSERT timestamp conversion out of the source query (#8911)

* fix(query): keep INSERT timestamp conversion out of the source query

Interpreting an INSERT's string timestamps used to work by pushing the
conversion down into the source query, which changed what that query
means. Two consequences:

- Pushing through a UNION's DISTINCT moved the dedup key from the raw
  strings to parsed instants, so rows spelling the same instant
  differently collapsed into one. On an append-only table that is a
  silently dropped row.
- A UNION branch that needed no conversion (a NULL, or an explicit cast)
  made the whole column give up, leaving sibling branches on UTC while
  the rest of the row used the session timezone.

Convert at the assignment instead, by routing its cast through a
timezone-carrying timestamp type and back. Arrow applies the timezone
when a cast target carries one, and stripping it afterwards preserves
the value. The source query is no longer touched, so both cases go away
and the tree-walking rewrite (roughly 160 lines) is deleted.

The rewrite reads source types, so it now runs TypeCoercion first: a
UNION still carries its loose per-branch schema before coercion, and
retargeting a cast whose input later becomes a timestamp would shift the
value rather than reinterpret it.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(query): address review on INSERT assignment rewrite

- Clone the input `Arc` instead of the whole subtree, and only rebuild it
  when a `Values` row actually changes.
- Defer cloning the cast source until the literal-folding path has been
  ruled out.
- Move the UTC check onto `Timezone::is_utc`, replacing a bare string
  compare.
- Cover a prepared `INSERT ... VALUES (?)`: an untyped placeholder types
  as `Null`, so the assignment cast is left for parameter substitution.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 1851f6bf4d)

* test: renew etcd TLS certificates (#8956)

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 144f83528d)

* fix: update tokio-postgres and correct explain/fetch cursor output schema (#8955)

* chore(deps): update tokio-postgres

* fix: describing fetch cursor and analyze

(cherry picked from commit b31f05eb59)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(flow): avoid insert select HTTP/2 stalls

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit d4a3d88cd7)

* fix(query): share record batch forwarding for DML

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit c34bec197c)

* fix(mysql): interpret prepared statement datetime params in session timezone (#8923)

* fix(mysql): interpret prepared statement datetime params in session timezone

Binary DATETIME parameters of server-side prepared statements were
converted as if UTC, ignoring the session timezone set via SET time_zone.
Convert them with the session timezone and add an integration test
covering prepared inserts and predicates under Asia/Shanghai.

Signed-off-by: wy471x <wy471x@gmail.com>

* refactor: share naive datetime timezone policy via common-time

Address review feedback on the prepared-statement timezone fix:

- Expose Timestamp::from_naive_datetime in common-time so the DST policy
  (gap -> error, ambiguous -> earlier instant) lives in one place, shared
  by the text protocol (Timestamp::from_str) and the MySQL binary protocol.
- Route the MySQL prepared-statement datetime conversion through it.
- Match the target type before converting datetime params so
  PreparedStmtTypeMismatch fails fast without wasted conversion.
- Use the short Timezone import form for consistency with the rest of servers.

Signed-off-by: wy471x <wy471x@gmail.com>

---------

Signed-off-by: wy471x <wy471x@gmail.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
(cherry picked from commit aaa843104b)

* fix(mito2): use target sequence for foreign SSTs (#8946)

* fix(mito2): use target sequence for foreign SSTs

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* chore(mito2): address foreign SST review feedback

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit bd7d2c1dfa)

* fix: postgres describe for more statements (#8974)

* fix: postgres describe for more statements

Signed-off-by: Ning Sun <sunning@greptime.com>

* fix: cover more show statements

Signed-off-by: Ning Sun <sunning@greptime.com>

* fix: address review comments

- add missing `clippy::too_many_arguments` allow on
  `query_from_information_schema_dataframe` (CI clippy failure)
- take `&ShowKind` in the information-schema dataframe helper so `kind`
  is no longer cloned at every call site; only the WHERE arm (which needs
  an owned expression for `sql_to_expr`) clones internally
- document why re-applying TQL explain formats never overwrites an
  existing value (per-query context state)

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: trim comments to essentials

Signed-off-by: Ning Sun <sunning@greptime.com>

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
(cherry picked from commit d32cd77505)

* fix(promql): resolve derived labels in aggregation arithmetic (#8994)

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
(cherry picked from commit c4dafb5826)

* fix(deps): narrow postgres lockfile update

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(deps): normalize remaining x509 lock reference

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(client): isolate query and control transports

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs(frontend): clarify adaptive window purpose

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(grpc): satisfy clippy in transport isolation coverage

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(pipeline): coalesce concurrent pipeline cache misses (#9022)

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* Revert "test(grpc): satisfy clippy in transport isolation coverage"

This reverts commit 888bcedc96.

(cherry picked from commit e0216b4074485bcffaae53348a17091532cb3600)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* Revert "docs(frontend): clarify adaptive window purpose"

This reverts commit 84cf69d099.

(cherry picked from commit da30e605ad7ebfb2a3fa86ea44b67405e5afde98)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* Revert "fix(client): isolate query and control transports"

This reverts commit a1d2dbb050.

(cherry picked from commit 03f5c31f544d3d3852beddfa87cff34d04a646ae)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor(json2): add bounded auto-expansion to the JSON2 vector builder (#8909)

* refactor(json2): add bounded auto-expansion to the JSON2 vector builder

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
(cherry picked from commit 76f08d2b3f)
(cherry picked from commit 640fa63bd8ee098fac8e8d62d6fb7ca5af4ddcd0)

* refactor(json2): optimize JSON2 building without auto-expanded paths (#8928)

* refactor(json2): optimize JSON2 building without auto-expanded paths

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* avoid panicking memtable write

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
(cherry picked from commit 7fd0a7bb98)
(cherry picked from commit 63c68ee60101e2a09e14420051f8f889b41a2a3b)

* refactor(json2): support querying v2 storage layout (#8940)

* feat(json2): support querying v2 storage layout

- route missing JSON2 paths to the v2 remainder field
- reconstruct complete values from explicit fields and remainder data
- preserve root JSON2 columns across projections and filters
- support nested JSON values in json_get string results
- add and reorganize JSON2 sqlness coverage

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
(cherry picked from commit 932f87f7a8)
(cherry picked from commit 5af1394a1f319bb99cafa9474b8a5a075be1e805)

* feat(pipeline): support table-aware JSON2 transforms (#8964)

* feat(pipeline): support table-aware JSON2 transforms

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* feat(pipeline): support JSON2 type hints in transforms

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix(pipeline): default failed JSON2 transforms to null

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* refactor(json2): distinguish invalid settings from layout errors

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
(cherry picked from commit fb86f6573e)
(cherry picked from commit 18a21a9ce017b6887c857a3f3bbb9a83aa7f968b)

* refactor: json2 v2 storage layout (#8979)

* refactor: json2 v2 storage layout

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* rethinking when "needs_remainder"

Signed-off-by: luofucong <luofc@foxmail.com>

* restore "ReadColumns"

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
(cherry picked from commit 529f046110)
(cherry picked from commit 7628d795197c786872730c3a5ef36c5c662d4322)

* feat(json2): support JSON2 paths in SQL functions (#9007)

feat(query): support JSON2 paths in SQL functions

Update the DataFusion fork to expose scalar function planning hooks.

Infer JSON2 path output types from scalar, aggregate, and window function signatures, while preserving the default Utf8View behavior for functions that accept arbitrary inputs.

Add unit and sqlness coverage for type conflicts, mixed typed and untyped JSON paths, filters, aggregates, and window functions.

Signed-off-by: luofucong <luofc@foxmail.com>
(cherry picked from commit 15317a131b)
(cherry picked from commit a945cbc7a4398e6bee74d4d2c67528772faf398d)

* feat(json2): support empty and null JSON2 value (#9010)

* feat(json2): support empty and null JSON2 value

Signed-off-by: fys <fengys1996@gmail.com>

* test(json2): cover explicit NULL and omitted-column inserts

Signed-off-by: fys <fengys1996@gmail.com>

* fix: cargo fmt

Signed-off-by: fys <fengys1996@gmail.com>

* fix: infer empty JSON object as object type

Signed-off-by: fys <fengys1996@gmail.com>

---------

Signed-off-by: fys <fengys1996@gmail.com>
(cherry picked from commit 05c65f54a8)
(cherry picked from commit 05c27452ea5e958fab85e0ba4b82f5df205a5d08)

* feat(json2): support list indexing for JSON2 columns (#9013)

feat(query): support list indexing for JSON2 columns

Extend JSON2 paths through DataFusion field-access planning, including nested list indexes and object fields following an index.

Preserve Variant reads for bracket JSONPath expressions and normalize dot accesses after subscripts to work around the current DataFusion planner limitation.

Add unit and sqlness coverage for nested indexes, type conflicts, missing paths, flushes, and compacted SSTs.

Signed-off-by: luofucong <luofc@foxmail.com>
(cherry picked from commit d62a5a990a)
(cherry picked from commit c11d8647feb30ecdd5ba1bc8ed4df78999801672)

* refactor(json2): concretize JSON2 schemas at merge scan boundaries (#9016)

* refactor(json2): concretize JSON2 schemas at merge scan boundaries

Infer concrete JSON2 output types from remote plans and expose them on MergeScanLogicalPlan before physical planning. Recompute affected local schemas and remove the JSON2-specific rewrite from MergeScanExec.

Add SQLness coverage for whole JSON2 columns in windows and joins.

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
(cherry picked from commit 84bd993131)
(cherry picked from commit 51c98a7e31a929fc33ff5133714bb47108f7539b)

* fix(json2): keep empty structs in remainder (#9027)

Signed-off-by: luofucong <luofc@foxmail.com>
(cherry picked from commit d67d3501a9)
(cherry picked from commit b3f2c1cc5bf0cfddb7c0670ac8f3522eb67c35a0)

* fix(mito2): remove stale scan test imports

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit d63bbe6aef298f5b8e960bb02eed08743a6684ee)

* test(query): expect bounded merge scan partitions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 7bcef46b1c010d2e73ec49be7581c216e6b5b1b6)

* style(mito2): remove trailing whitespace

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(sqlness): refresh bounded partition plans

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(sqlness): normalize CPU-dependent partition plans

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: wy471x <wy471x@gmail.com>
Signed-off-by: Ning Sun <sunning@greptime.com>
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: fys <fengys1996@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: Lei, HUANG <ratuthomm@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
Co-authored-by: wy471x <wy471x@gmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
Co-authored-by: shuiyisong <xixing.sys@gmail.com>
Co-authored-by: luofucong <luofc@foxmail.com>
Co-authored-by: fys <fengys1996@gmail.com>
2026-09-07 19:29:54 +08:00
jeremyhi 448f973593 fix: sandbox SQL local filesystem access (#8708)
* fix: sandbox SQL local filesystem access

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: address local file sandbox review findings

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: support Windows local copy paths

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: improve sandbox path errors

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: simplify local path error context

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* perf: stream secure filesystem listings

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* style: derive local file access default

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: improve local file access errors

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: address local file access review findings

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test: simplify local file access coverage

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: harden sandboxed local file backends

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: reject directory copy targets before creation

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: avoid implicit string clone in file table listing

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-07-31 13:23:15 +00:00
discord9 5ed7128f06 perf(query): prune RangeSelect input projections (#8570)
* perf(query): prune RangeSelect input projections

Signed-off-by: discord9 <discord9@163.com>

* test(query): stabilize RangeSelect projection snapshot

Signed-off-by: discord9 <discord9@163.com>

* test(query): update RangeSelect nested plan snapshot

Signed-off-by: discord9 <discord9@163.com>

* fix(query): handle nested RangeSelect aliases

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-07-30 07:09:49 +00:00
discord9 9776d88dc8 fix: preserve dictionary regex filter semantics (#8688)
* fix: preserve dictionary regex filter semantics

Signed-off-by: discord9 <discord9@163.com>

* test(query): update dictionary regex plan results

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-07-30 06:20:57 +00:00
fys 3e67c67607 refactor: simplify scan projection to root column indices (#8629)
* refactor: simplify scan projection to root column indices

Signed-off-by: fys <fengys1996@gmail.com>

* fix: sqlness test

Signed-off-by: fys <fengys1996@gmail.com>

* chore: code style adjust

Signed-off-by: fys <fengys1996@gmail.com>

* fix: cr

Signed-off-by: fys <fengys1996@gmail.com>

* add comment and test

Signed-off-by: fys <fengys1996@gmail.com>

* minor change

Signed-off-by: fys <fengys1996@gmail.com>

* fix: unit test

Signed-off-by: fys <fengys1996@gmail.com>

---------

Signed-off-by: fys <fengys1996@gmail.com>
2026-07-27 12:04:19 +00:00
Ruihang Xia 0479c05092 perf: preserve dictionary-encoded query labels (#8541)
* perf: preserve dictionary-encoded query labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(client): skip dictionary Flight batches

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(servers): decode dictionary labels in HTTP output

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test(mito2): support dictionary tags in series scans

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(servers): preserve dictionary child nulls in SQL

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(promql): compare dictionary tags by logical nulls

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): keep dictionary tags within query paths

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor(query): scope PK dictionary encoding to reads

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): preserve pushdown for dictionary labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): handle dictionary labels in query operators

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): complete label type matching

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): align dictionary query schemas

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): decode incompatible OR labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): preserve dictionary partition pruning

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: handle dictionary query edge cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: import dictionary downcast macro

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-07-17 13:57:10 +00:00
discord9 b30765e5e9 fix: preserve distributed topk merge ordering (#8432)
* fix: preserve distributed topk for scalar latest

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve scalar latest ordering across merge scan

Signed-off-by: discord9 <discord9@163.com>

* fix: carry merge scan ordering from planner rewrite

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan partition ordering gate

Signed-off-by: discord9 <discord9@163.com>

* fix: require adjacent merge sort for ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: update ordering sqlness plans

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve distributed topk merge ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover latest per series queries

Signed-off-by: discord9 <discord9@163.com>

* test: update distributed merge sort sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering partition gate

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan over-partition ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover distributed latest with low parallelism

Signed-off-by: discord9 <discord9@163.com>

* fix: keep distributed merge sort opaque

Signed-off-by: discord9 <discord9@163.com>

* test: assert distributed scalar latest merge sort

Signed-off-by: discord9 <discord9@163.com>

* test: clarify merge scan ordering helper

Signed-off-by: discord9 <discord9@163.com>

* fix: complete merge sort exec delegation

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort limit pushdown

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer opacity

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve merge sort child topk

Signed-off-by: discord9 <discord9@163.com>

* test: update order by topk plan

Signed-off-by: discord9 <discord9@163.com>

* fix: recognize merge sort global fetch

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-07-14 12:53:07 +00:00
Weny Xu 7039e54835 fix: require metasrv GC for repartition (#8497)
* fix: require GC for repartition

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: update sqlness result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: enable GC for repartition integration tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-07-14 12:04:03 +00:00
jeremyhi 03b304522f refactor: model distributed inspect fan-in as exec (#8447)
* refactor: model distributed inspect fan-in as exec

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: harden distributed inspect exec fan-in

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: show distributed inspect scan in plans

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: stabilize distributed inspect sqlness

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: apply sqlness replacement to inspect explain

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: redact inspect repartition plan

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-07-10 08:53:23 +00:00
discord9 ff2fa71d50 fix: repartition subset partition key joins (#8460)
Signed-off-by: discord9 <discord9@163.com>
2026-07-10 06:10:28 +00:00
jeremyhi 5bcc2e9d6b fix: global limit for distributed inspect streams (#8412)
* fix: global limit for distributed inspect streams

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: honor tighter filter fetch under global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: restore distribution after global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: preserve inherited distribution for global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: propagate inherited hash partitioning

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-07-06 07:02:54 +00:00
discord9 4f5dccf6bb fix(query): push down PromQL cast filters (#8407)
* fix(query): push down PromQL cast filters

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover cast preimage pushdown

Signed-off-by: discord9 <discord9@163.com>

* test(query): expand cast preimage coverage

Signed-off-by: discord9 <discord9@163.com>

* test(query): compare cast normalization rules

Signed-off-by: discord9 <discord9@163.com>

* test(query): update cast pushdown sqlness plans

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-07-03 09:10:00 +00:00
discord9 d3cc1b1888 feat(flow): stabilize eval interval scheduling (#8360)
* feat(flow): stabilize eval interval scheduling

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): satisfy eval schedule clippy

Signed-off-by: discord9 <discord9@163.com>

* test(flow): trim eval schedule coverage

Signed-off-by: discord9 <discord9@163.com>

* test(flow): cover stable eval scheduling

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): reserve scheduled runtime hint

Signed-off-by: discord9 <discord9@163.com>

* test(flow): trim sqlness result eof

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): harden eval schedule edges

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): address scheduled flow review

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): clean scheduled config handling

Signed-off-by: discord9 <discord9@163.com>

* test(flow): add eval interval compat case

Signed-off-by: discord9 <discord9@163.com>

* test(flow): cover show create flow in compat

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): drop scheduled time from flow context

Signed-off-by: discord9 <discord9@163.com>

* test(flow): assert scheduled now binding

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-06-29 09:55:27 +00:00
discord9 9cf071808a fix(query): run optimizer rules before MergeScan (#8339)
* fix(query): push down join filters before MergeScan

Signed-off-by: discord9 <discord9@163.com>

* fix(query): run optimizer before MergeScan pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): narrow pre-MergeScan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): refine pre-MergeScan optimizer prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): satisfy predicate extractor clippy

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover pre-MergeScan optimizer edges

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover set comparison prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): guard remote scan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve subquery planning errors

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve usable scan predicates

Signed-off-by: discord9 <discord9@163.com>

* fix(query): simplify scan predicate extraction

Signed-off-by: discord9 <discord9@163.com>

* fix(query): keep scan filter extraction scoped

Signed-off-by: discord9 <discord9@163.com>

* docs(query): explain pre-MergeScan optimizer

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-06-23 12:15:40 +00:00
Ruihang Xia 428cd8137d feat: identify noneffective binary modifiers (#8230)
* perf(promql): use tsid for full-label modifier joins

* fix(promql): collect narrow range joins by tsid

* verify on for both sides

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* check if tsid exists

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* correct behavior

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-06-05 08:44:50 +00:00
discord9 9fafd879ed fix: window sort off by one precision TimeRange&better alias track (#8019)
* fix: window sort track alias&off by one precision TimeRange

Signed-off-by: discord9 <discord9@163.com>

* chore: more test

Signed-off-by: discord9 <discord9@163.com>

* refactor: clear helper

Signed-off-by: discord9 <discord9@163.com>

* dedup a bit

Signed-off-by: discord9 <discord9@163.com>

* feat: even more guard

Signed-off-by: discord9 <discord9@163.com>

* fix: case insensitive

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-04-24 13:56:51 +00:00
QuakeWanganddennis zhuang e8a86a4273 feat: expose flownode addrs in information_schema flows (#7992)
* feat: expose flownode addrs in information_schema flows

Signed-off-by: QuakeWang <1677980708@qq.com>

* fix: always include flownode addrs

Signed-off-by: QuakeWang <1677980708@qq.com>

* test: split flownode_addrs sqlness cases by mode

Signed-off-by: QuakeWang <1677980708@qq.com>

* test: fix flow_tql sqlness snapshots

Signed-off-by: QuakeWang <1677980708@qq.com>

* test: fix sqlness information_schema snapshot

Signed-off-by: QuakeWang <1677980708@qq.com>

---------

Signed-off-by: QuakeWang <1677980708@qq.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
2026-04-22 07:06:44 +00:00
Yingwen f3dbf34c74 chore: bump version to 1.0.0 (#7935)
* chore: bump version to 1.0.0

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix sqlness test

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix cluster info sqlness

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: reorder regex in cluster_info

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: fix pg catalog

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2026-04-08 15:15:27 +00:00
discord9 b623cb1aa2 perf: no longer window sort when limit (#7912)
* perf: no longer window sort when limit

Signed-off-by: discord9 <discord9@163.com>

* test: confusing vector sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: redact sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: redact every thing

Signed-off-by: discord9 <discord9@163.com>

* REDACTED

Signed-off-by: discord9 <discord9@163.com>

* what

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-04-08 02:54:22 +00:00
discord9 30e895abbe fix: prom cast to f64 (#7840)
* fix: cast to f64

Signed-off-by: discord9 <discord9@163.com>

* test: div case

Signed-off-by: discord9 <discord9@163.com>

* test: int test

Signed-off-by: discord9 <discord9@163.com>

* chore: sqlness update

Signed-off-by: discord9 <discord9@163.com>

* chore: test

Signed-off-by: discord9 <discord9@163.com>

* chore: update test

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-03-24 06:24:52 +00:00
discord9 d1151b665b feat: flow tql cte (#7702)
* feat: flow tql cte

Signed-off-by: discord9 <discord9@163.com>

* fix: creating flow TQL CTE source tables lose cte part in query

Signed-off-by: discord9 <discord9@163.com>

* test: update sqlness result

Signed-off-by: discord9 <discord9@163.com>

* chore

Signed-off-by: discord9 <discord9@163.com>

* fix: properly canonicalize ident

Signed-off-by: discord9 <discord9@163.com>

* feat: even stricter check

Signed-off-by: discord9 <discord9@163.com>

* chore: sqlness update

Signed-off-by: discord9 <discord9@163.com>

* chore: after rebase fix

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-03-06 03:36:42 +00:00
b2074e3863 chore: upgrade DataFusion family, again (#7578)
* chore: upgrade DataFusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: switch to released version of datafusion-pg-catalog

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
2026-03-03 07:36:39 +00:00
Ruihang Xia db46849f40 feat: track flow source tables for TQL and info schema (#7697)
* feat: track flow source tables for TQL and info schema

* handle schema matcher

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sqlness tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* cover __name__ case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-02-11 03:19:09 +00:00
fys 1aa80d9363 fix: incorrect-tql-explain result (#7675) 2026-02-11 02:30:15 +00:00
Ruihang Xia 4ac73a7e9e feat: adapt prometheus 3.x selector and lookback behavior (#7688)
* feat: adapt prometheus 3.x selector and lookback behavior

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove commit hash from comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-02-10 04:00:45 +00:00
Ruihang Xia c83868c4eb feat: partition rule simplifier (#7622)
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reuse collider

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify range helpers

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* notes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update unit test resule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-01-27 14:31:20 +00:00
discord9andRuihang Xia 085e9dfc7b fix: use phy table when need fitler by tsid/table id (#7609)
* fix: use phy table when need fitler by tsid/table id

Signed-off-by: discord9 <discord9@163.com>

* feat: add subquery alias

Signed-off-by: discord9 <discord9@163.com>

* test: update after alias

Signed-off-by: discord9 <discord9@163.com>

* test: sort&redact

Signed-off-by: discord9 <discord9@163.com>

* test: per review

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2026-01-26 06:11:16 +00:00
Weny Xu 25687bb282 feat: add ddl timeout/wait options, repartition WITH parsing, meta-client startup refactor (#7589)
* feat: add ddl request timeouts and unify meta client startup

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: omplement ALTER TABLE repartition DDL options parsing

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add sqlness tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: pass timeout argument to procedure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: refine comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: assert timeout

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-01-20 09:26:53 +00:00
Weny Xu 2ae20daa62 feat: add sync region instruction for repartition procedure (#7562)
* feat: add sync region instruction for repartition procedure

This commit introduces a new sync region instruction and integrates it
into the repartition procedure flow, specifically for metric engine tables.

Changes:
- Add SyncRegion instruction type and SyncRegionsReply in instruction.rs
- Implement SyncRegionHandler in datanode to handle sync region requests
- Add SyncRegion state in repartition procedure to sync newly allocated regions
- Integrate sync region step after enter_staging_region for metric engine tables
- Add sync_region flag and allocated_region_ids to PersistentContext
- Make SyncRegionFromRequest serializable for instruction transmission
- Add test utilities and mock support for sync region operations

The sync region step is conditionally executed based on the table engine type,
ensuring that newly allocated regions in metric engine tables are properly
synced from their source regions before proceeding with manifest remapping.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add logs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(repartition): improve staging region handling and support metric engine repartition
- Reorder sync region flow: move SyncRegion from EnterStagingRegion to RepartitionStart to sync before applying staging
- Add ExitStaging metadata update state to properly clear staging leader info after repartition completes
- Update build_template_from_raw_table_info to optionally skip metric engine internal columns when creating region requests
- Fix region state transition: set_dropping now expects specific state (Staging or Writable) for proper validation
- Adjust region drop and copy handlers to handle staging regions correctly
- Add comprehensive test cases for metric engine SPLIT/MERGE partition operations on physical tables with logical tables
- Improve logging for table route updates, region drops, and repartition operations

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: removes code duplication

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: update result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: refine comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add error strategy support for flush region and flush pending deallocate regions

- **Add `ErrorStrategy` enum** in `procedure/utils.rs`:
  - Supports `Ignore` and `Retry` strategies for error handling
  - Refactor `flush_region` to accept `error_strategy` parameter
  - Extract `handle_flush_region_reply` helper function for better code organization

- **Add pending deallocate region support**:
  - Add `pending_deallocate_region_ids` field to `PersistentContext`
  - Implement `flush_pending_deallocate_regions` in `EnterStagingRegion` state
  - Flush pending deallocate regions before entering staging regions to ensure data consistency

- **Update error handling**:
  - `flush_leader_region`: Use `ErrorStrategy::Ignore` to skip unreachable datanodes
  - `sync_region`: Use `ErrorStrategy::Retry` for critical operations
  - `enter_staging_region`: Use `ErrorStrategy::Retry` when flushing pending deallocate regions

This change improves the robustness of the repartition procedure by:
1. Providing flexible error handling strategies for flush operations
2. Ensuring pending deallocate regions are properly flushed before repartitioning
3. Preventing data inconsistency during region migration

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: compile

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-01-15 04:52:57 +00:00
LFC e64c31e59a chore: upgrade DataFusion family (#7558)
* chore: upgrade DataFusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* use main proto

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
2026-01-14 14:02:31 +00:00
Ruihang Xia 170f94fc08 feat: enable pruning for manipulate plans (#7565)
* feat: enable pruning for manipulate plans

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* apply to other plans and add sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix scalar manipulate and histogram fold for missing some columns

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* don't drop every columns

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unrelated part

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-01-14 08:32:51 +00:00
Yingwen 4b3bd7317b feat: add per-partition convert, result cache metrics (#7539)
* fix: show convert cost in explain analyze verbose

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: increase puffin metadata cache metric

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: add result cache hit/miss to filter metrics

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: print flat format in debug

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: update sqlness test

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: make scan cost contains part/reader build cost

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: collect divider cost

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: remove unused field in ScannerMetrics

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: collect metadata read bytes

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: collect read metrics in get_parquet_meta_data

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2026-01-13 09:17:09 +00:00
Weny Xu 567d3e66e9 feat: integrate repartition procedure into DdlManager (#7548)
* feat: add repartition procedure factory support to DdlManager

- Introduce RepartitionProcedureFactory trait for creating and registering
  repartition procedures
- Implement DefaultRepartitionProcedureFactory for metasrv with full support
- Implement StandaloneRepartitionProcedureFactory for standalone (unsupported)
- Add procedure loader registration for RepartitionProcedure and
  RepartitionGroupProcedure
- Add helper methods to TableMetadataAllocator for allocator access
- Add error types for repartition procedure operations
- Update DdlManager to accept and use RepartitionProcedureFactoryRef

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: integrate repartition procedure into DdlManager

- Add submit_repartition_task() to handle repartition from alter table
- Route Repartition operations in submit_alter_table_task() to repartition factory
- Refactor: rename submit_procedure() to execute_procedure_and_wait()
- Make all DDL operations wait for completion by default
- Add submit_procedure() for fire-and-forget submissions
- Add CreateRepartitionProcedure error type
- Add placeholder Repartition handling in grpc-expr (unsupported)
- Update greptime-proto dependency

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: implement ALTER TABLE REPARTITION procedure submission

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(repartition): handle central region in apply staging manifest

- Introduce ApplyStagingManifestInstructions struct to organize instructions
- Add special handling for central region when applying staging manifests
- Transition state from UpdateMetadata to RepartitionEnd after applying staging manifests
- Remove next_state() method in RepartitionStart and inline state transitions
- Improve logging and expression serialization in DDL statement executor
- Move repartition tests from standalone to distributed test suite

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-01-09 08:37:21 +00:00
Ruihang Xia fce1687fa7 fix: incorrect timestamp index inference (#7530)
* add sqlness case, but can't reproduce

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reproduction

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix wildcard rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-01-07 11:18:25 +00:00
discord9 ac6d68aa2d fix: simp expr recursively (#7523)
* fix: simp expr recursively

Signed-off-by: discord9 <discord9@163.com>

* test: some simple constant folding case

Signed-off-by: discord9 <discord9@163.com>

* fix: literal ts cast to UTC

Signed-off-by: discord9 <discord9@163.com>

* fix: patch merge scan batch col tz instead

Signed-off-by: discord9 <discord9@163.com>

* test: fix

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-01-07 09:22:26 +00:00
564cc0c750 feat: table/column/flow COMMENT (#7060)
* initial impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* avoid unimplemented panic

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* validate flow

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix table column comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* table level comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify table info serde

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* don't txn

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove empty trait

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* wip: procedure

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* grpc support

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* try from pb struct

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* doc comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* check unchanged fast case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tune errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix merge error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use try_as_raw_value

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
2025-12-10 15:08:47 +00:00
Ruihang Xia f2288a86b0 perf: treat DISTINCT as comm/part-comm (#7348)
* perf: treat DISTINCT as comm/part-comm

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sqlness tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-12-04 11:14:00 +00:00
dennis zhuangandCopilot 1f91422bae feat!: improve mysql/pg compatibility (#7315)
* feat(mysql): add SHOW WARNINGS support and return warnings for unsupported SET variables

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* feat(function): add MySQL IF() function and PostgreSQL description functions for connector compatibility

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: show tables for mysql

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: partitions table in information_schema and add starrocks external catalog compatibility

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: async udf

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: set warnings

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* feat: impl pg_my_temp_schema and make description functions simple

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: add test for issue 7313

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* feat: apply suggestions

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: partition_expression and partition_description

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: test

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: unit tests

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: saerch_path only works for pg

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* feat: improve warnings processing

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: warnings while writing affected rows and refactor

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: improve ShobjDescriptionFunction signature

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: array_to_boolean

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-01 20:41:14 +00:00
Ruihang Xia c054c13e48 perf: avoid unnecessary merge sort (#7274)
* perf: avoid unnecessary merge sort

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fantastic if chain

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* more comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-11-21 09:02:25 +00:00
discord9 8153068b89 chore: bump main branch version to 1.0.0-beta.1 (#7191)
* chore: bump main branch version to 1.0.0-beta.1

Signed-off-by: discord9 <discord9@163.com>

* rename beta.1 to beta1

Signed-off-by: discord9 <discord9@163.com>

* again

Signed-off-by: discord9 <discord9@163.com>

* test: correct redact version

Signed-off-by: discord9 <discord9@163.com>

* chore

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-11-11 14:52:03 +00:00
discord9 68247fc9b1 fix: count_state use stat to eval&predicate w/out region (#7116)
* fix: count_state use stat to eval

Signed-off-by: discord9 <discord9@163.com>

* cleanup

Signed-off-by: discord9 <discord9@163.com>

* fix: use predicate without region

Signed-off-by: discord9 <discord9@163.com>

* test: diverge standalone/dist impl

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-10-27 02:14:45 +00:00
zyy17andYingwen a0e6bcbeb3 feat: add cpu_usage_millicores and memory_usage_bytes in information_schema.cluster_info table. (#7051)
* refactor: add `hostname` in cluster_info table

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: update information schema result

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* feat: enable zstd for bulk memtable encoded parts (#7045)

feat: enable zstd in bulk memtable

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: add `get_total_cpu_millicores()` / `get_total_cpu_cores()` / `get_total_memory_bytes()` / `get_total_memory_readable()` in common-stat

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* feat: add `cpu_usage_millicores` and `memory_usage_bytes` in `information_schema.cluster_info` table

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* fix: compile warning and integration test failed

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* fix: integration test failed

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add `ResourceStat`

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: apply code review comments

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: update greptime-proto

Signed-off-by: zyy17 <zyylsxm@gmail.com>

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
2025-10-24 03:12:45 +00:00
LFCandNing Sun 2f637a262e chore: update datafusion to 50 (#7076)
* chore: update datafusion to 50

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* fix: update datafusion_pg_catalog import

* chore: fix toml format

* chore: fix toml format again

* fix nextest

Signed-off-by: luofucong <luofc@foxmail.com>

* fix sqlness

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: switch datafusion-orc to upstream tag

* fix sqlness

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
2025-10-23 07:18:36 +00:00
zyy17 cf1b8392af refactor!: unify the API of getting total cpu and memory (#7049)
* refactor: add `get_total_cpu_millicores()` / `get_total_cpu_cores()` / `get_total_memory_bytes()` / `get_total_memory_readable()` in common-stat

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* tests: update sqlness test cases

Signed-off-by: zyy17 <zyylsxm@gmail.com>

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
2025-10-16 12:41:34 +00:00
zyy17 6ab4672866 refactor: add peer_hostname field in information_schema.cluster_info table (#7050)
* refactor: add `hostname` in cluster_info table

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: update information schema result

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: apply code review comments

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: update greptime-proto

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: add the compatibility for old proto

Signed-off-by: zyy17 <zyylsxm@gmail.com>

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
2025-10-16 06:02:47 +00:00
discord9 aca8b690d1 fix: step aggr merge phase not order nor filter (#6998)
* fix: not order

Signed-off-by: discord9 <discord9@163.com>

* test: redacted

Signed-off-by: discord9 <discord9@163.com>

* feat: fix up state wrapper

Signed-off-by: discord9 <discord9@163.com>

* df last_value state not as promised!

Signed-off-by: discord9 <discord9@163.com>

* fix?: could fix better

Signed-off-by: discord9 <discord9@163.com>

* test: unstable result

Signed-off-by: discord9 <discord9@163.com>

* fix: work around by fixing state

Signed-off-by: discord9 <discord9@163.com>

* chore: after rebase fix

Signed-off-by: discord9 <discord9@163.com>

* chore: finish some todo

Signed-off-by: discord9 <discord9@163.com>

* chore: per copilot

Signed-off-by: discord9 <discord9@163.com>

* refactor: not fix but just notify mismatch

Signed-off-by: discord9 <discord9@163.com>

* chore: warn -> debug state mismatch

Signed-off-by: discord9 <discord9@163.com>

* chore: refine error msg

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness add last_value date_bin test

Signed-off-by: discord9 <discord9@163.com>

* ?: substrait order by decode failure

Signed-off-by: discord9 <discord9@163.com>

* unit test reproduce that

Signed-off-by: discord9 <discord9@163.com>

* feat: support state wrapper's order serde in substrait

Signed-off-by: discord9 <discord9@163.com>

* refactor: stuff

Signed-off-by: discord9 <discord9@163.com>

* test: standalone/distributed different exec

Signed-off-by: discord9 <discord9@163.com>

* fmt

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* refactor: closure

Signed-off-by: discord9 <discord9@163.com>

* test: first value order by

Signed-off-by: discord9 <discord9@163.com>

* refactor: per cr

Signed-off-by: discord9 <discord9@163.com>

* feat: ScanHint last_value last row selector

Signed-off-by: discord9 <discord9@163.com>

* docs: per cr

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-09-26 09:12:45 +00:00
discord9 238ed003df fix: group by expr not as column in step aggr (#7008)
* fix: group by expr not as column

Signed-off-by: discord9 <discord9@163.com>

* test: dist analyzer date_bin

Signed-off-by: discord9 <discord9@163.com>

* ???fix wip

Signed-off-by: discord9 <discord9@163.com>

* fix: deduce using correct input fields

Signed-off-by: discord9 <discord9@163.com>

* refactor: clearer wrapper

Signed-off-by: discord9 <discord9@163.com>

* chore: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* chore: rm todo

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-09-24 06:57:01 +00:00
discord9 85c1a91bae feat: support SubqueryAlias pushdown (#6963)
* wip enforce dist requirement rewriter

Signed-off-by: discord9 <discord9@163.com>

* feat: enforce dist req

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness result

Signed-off-by: discord9 <discord9@163.com>

* fix: double projection

Signed-off-by: discord9 <discord9@163.com>

* test: fix sqlness

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

* docs: use btree map

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness explain&comment

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-09-16 13:27:35 +00:00
Weny Xu 16febbd4c2 feat: add CPU, memory and node status info to cluster_info (#6897)
* feat: add CPU and memory info to `cluster_info`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add `node_status` to `cluster_info` table

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: update sqlness

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-09-08 08:59:34 +00:00
discord9 6a3791ab31 fix(flow): promql auto create table (#6867)
* fix: non aggr prom ql auto create table

Signed-off-by: discord9 <discord9@163.com>

* feat: val column use any name

Signed-off-by: discord9 <discord9@163.com>

* feat: check if it's tql src table

Signed-off-by: discord9 <discord9@163.com>

* test: check sink table is tql-able

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness redacted

Signed-off-by: discord9 <discord9@163.com>

* fix: sql also handle no aggr case

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-09-02 08:54:16 +00:00