Commit Graph
16 Commits
Author SHA1 Message Date
Yingwen 76924c2d36 feat(mito2): introduce two-phase metric series scans (#8826)
* feat(mito2): add two-phase series scan

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: regenerate configuration reference

Signed-off-by: evenyag <realevenyag@gmail.com>

* test(sqlness): update series scan explain results

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: update config API expectation

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): bound two-phase series discovery

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): avoid candidate distribution deadlock

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore(mito2): remove obsolete dead code allowances

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): share series scan memory pool

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2026-08-14 06:32:17 +00:00
jeremyhi 764c93bf43 perf(query): choose bounded CTE as hash join build side (#8807)
* fix(query): choose bounded CTE as hash join build side

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* perf(query): remove join estimate cap

* test(compat): accept repartition in analyze plan

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-08-13 10:51:53 +00:00
Ruihang Xia 0479c05092 perf: preserve dictionary-encoded query labels (#8541)
* perf: preserve dictionary-encoded query labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(client): skip dictionary Flight batches

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(servers): decode dictionary labels in HTTP output

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test(mito2): support dictionary tags in series scans

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(servers): preserve dictionary child nulls in SQL

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(promql): compare dictionary tags by logical nulls

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): keep dictionary tags within query paths

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor(query): scope PK dictionary encoding to reads

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): preserve pushdown for dictionary labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): handle dictionary labels in query operators

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): complete label type matching

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): align dictionary query schemas

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): decode incompatible OR labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): preserve dictionary partition pruning

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: handle dictionary query edge cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: import dictionary downcast macro

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-07-17 13:57:10 +00:00
discord9 b30765e5e9 fix: preserve distributed topk merge ordering (#8432)
* fix: preserve distributed topk for scalar latest

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve scalar latest ordering across merge scan

Signed-off-by: discord9 <discord9@163.com>

* fix: carry merge scan ordering from planner rewrite

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan partition ordering gate

Signed-off-by: discord9 <discord9@163.com>

* fix: require adjacent merge sort for ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: update ordering sqlness plans

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve distributed topk merge ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover latest per series queries

Signed-off-by: discord9 <discord9@163.com>

* test: update distributed merge sort sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering partition gate

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan over-partition ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover distributed latest with low parallelism

Signed-off-by: discord9 <discord9@163.com>

* fix: keep distributed merge sort opaque

Signed-off-by: discord9 <discord9@163.com>

* test: assert distributed scalar latest merge sort

Signed-off-by: discord9 <discord9@163.com>

* test: clarify merge scan ordering helper

Signed-off-by: discord9 <discord9@163.com>

* fix: complete merge sort exec delegation

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort limit pushdown

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer opacity

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve merge sort child topk

Signed-off-by: discord9 <discord9@163.com>

* test: update order by topk plan

Signed-off-by: discord9 <discord9@163.com>

* fix: recognize merge sort global fetch

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-07-14 12:53:07 +00:00
discord9 9cf071808a fix(query): run optimizer rules before MergeScan (#8339)
* fix(query): push down join filters before MergeScan

Signed-off-by: discord9 <discord9@163.com>

* fix(query): run optimizer before MergeScan pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): narrow pre-MergeScan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): refine pre-MergeScan optimizer prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): satisfy predicate extractor clippy

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover pre-MergeScan optimizer edges

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover set comparison prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): guard remote scan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve subquery planning errors

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve usable scan predicates

Signed-off-by: discord9 <discord9@163.com>

* fix(query): simplify scan predicate extraction

Signed-off-by: discord9 <discord9@163.com>

* fix(query): keep scan filter extraction scoped

Signed-off-by: discord9 <discord9@163.com>

* docs(query): explain pre-MergeScan optimizer

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-06-23 12:15:40 +00:00
discord9 30e895abbe fix: prom cast to f64 (#7840)
* fix: cast to f64

Signed-off-by: discord9 <discord9@163.com>

* test: div case

Signed-off-by: discord9 <discord9@163.com>

* test: int test

Signed-off-by: discord9 <discord9@163.com>

* chore: sqlness update

Signed-off-by: discord9 <discord9@163.com>

* chore: test

Signed-off-by: discord9 <discord9@163.com>

* chore: update test

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-03-24 06:24:52 +00:00
b2074e3863 chore: upgrade DataFusion family, again (#7578)
* chore: upgrade DataFusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: switch to released version of datafusion-pg-catalog

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
2026-03-03 07:36:39 +00:00
fys 1aa80d9363 fix: incorrect-tql-explain result (#7675) 2026-02-11 02:30:15 +00:00
Ruihang Xia 4ac73a7e9e feat: adapt prometheus 3.x selector and lookback behavior (#7688)
* feat: adapt prometheus 3.x selector and lookback behavior

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove commit hash from comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-02-10 04:00:45 +00:00
Ruihang Xia 170f94fc08 feat: enable pruning for manipulate plans (#7565)
* feat: enable pruning for manipulate plans

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* apply to other plans and add sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix scalar manipulate and histogram fold for missing some columns

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* don't drop every columns

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unrelated part

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-01-14 08:32:51 +00:00
Ruihang Xia c054c13e48 perf: avoid unnecessary merge sort (#7274)
* perf: avoid unnecessary merge sort

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fantastic if chain

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* more comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-11-21 09:02:25 +00:00
discord9 238ed003df fix: group by expr not as column in step aggr (#7008)
* fix: group by expr not as column

Signed-off-by: discord9 <discord9@163.com>

* test: dist analyzer date_bin

Signed-off-by: discord9 <discord9@163.com>

* ???fix wip

Signed-off-by: discord9 <discord9@163.com>

* fix: deduce using correct input fields

Signed-off-by: discord9 <discord9@163.com>

* refactor: clearer wrapper

Signed-off-by: discord9 <discord9@163.com>

* chore: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* chore: rm todo

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-09-24 06:57:01 +00:00
Ruihang Xia 288f69a30f fix: plan disorder from upgrading datafusion (#6787)
* fix: plan disorder from upgrading datafusion

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-08-23 12:29:47 +00:00
LFC f9d2a89a0c chore: update datafusion family (#6675)
* chore: update datafusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* use official otel-arrow-rust

Signed-off-by: luofucong <luofc@foxmail.com>

* rebase

Signed-off-by: luofucong <luofc@foxmail.com>

* use the official orc-rust

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* remove the empty lines

Signed-off-by: luofucong <luofc@foxmail.com>

* try following PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
2025-08-15 12:41:49 +00:00
discord9 875207d26c feat: register all aggregate function to auto step aggr fn (#6596)
* feat: support generic aggr push down

Signed-off-by: discord9 <discord9@163.com>

* typo

Signed-off-by: discord9 <discord9@163.com>

* fix: type ck in merge wrapper

Signed-off-by: discord9 <discord9@163.com>

* test: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* feat: support all registried aggr func

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-08-05 11:37:45 +00:00
discord9 8d71213ea6 fix: aggr group by all partition cols use partial commutative (#6534)
* fix: aggr group by all partition cols use partial commutative

Signed-off-by: discord9 <discord9@163.com>

* test: bugged case

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness fix

Signed-off-by: discord9 <discord9@163.com>

* test: more redacted

Signed-off-by: discord9 <discord9@163.com>

* more cases

Signed-off-by: discord9 <discord9@163.com>

* even more test cases

Signed-off-by: discord9 <discord9@163.com>

* join testcase

Signed-off-by: discord9 <discord9@163.com>

* fix: column requirement added in correct location

Signed-off-by: discord9 <discord9@163.com>

* fix test

Signed-off-by: discord9 <discord9@163.com>

* chore: clippy

Signed-off-by: discord9 <discord9@163.com>

* track col reqs per stack

Signed-off-by: discord9 <discord9@163.com>

* fix: continue

Signed-off-by: discord9 <discord9@163.com>

* chore: clippy

Signed-off-by: discord9 <discord9@163.com>

* refactor: test mod

Signed-off-by: discord9 <discord9@163.com>

* test utils

Signed-off-by: discord9 <discord9@163.com>

* test: better test

Signed-off-by: discord9 <discord9@163.com>

* more testcases

Signed-off-by: discord9 <discord9@163.com>

* test limit push down

Signed-off-by: discord9 <discord9@163.com>

* more testcases

Signed-off-by: discord9 <discord9@163.com>

* more testcase

Signed-off-by: discord9 <discord9@163.com>

* more test

Signed-off-by: discord9 <discord9@163.com>

* chore: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: update commnets

Signed-off-by: discord9 <discord9@163.com>

* fix: check col reqs from bottom to upper

Signed-off-by: discord9 <discord9@163.com>

* chore: more comment

Signed-off-by: discord9 <discord9@163.com>

* docs: more todo

Signed-off-by: discord9 <discord9@163.com>

* chore: comments

Signed-off-by: discord9 <discord9@163.com>

* test: a new failing test that should be fixed

Signed-off-by: discord9 <discord9@163.com>

* fix: part col alias tracking

Signed-off-by: discord9 <discord9@163.com>

* chore: unused

Signed-off-by: discord9 <discord9@163.com>

* chore: clippy

Signed-off-by: discord9 <discord9@163.com>

* docs: comment

Signed-off-by: discord9 <discord9@163.com>

* mroe testcase

Signed-off-by: discord9 <discord9@163.com>

* more testcase for step/part aggr combine

Signed-off-by: discord9 <discord9@163.com>

* FIXME: a new bug

Signed-off-by: discord9 <discord9@163.com>

* literally unfixable

Signed-off-by: discord9 <discord9@163.com>

* chore: remove some debug print

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-07-23 09:05:34 +00:00