mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-10-05 11:35:35 +00:00
discord9andNing Sun
94d7e2c7fc
* feat!: upgrade DataFusion dependencies to 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * refactor: migrate DataFusion 55 APIs Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: preserve table function planning behavior Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: preserve PostgreSQL query compatibility Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: preserve distributed execution plan behavior Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: cover DataFusion 55 behavior regressions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: update DataFusion 55 SQLness expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: complete DataFusion 55 test API migration Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: address DataFusion 55 CI regressions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: address remaining DataFusion 55 regressions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: adapt latest base code to DataFusion 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: normalize environment-specific DataFusion 55 plans Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: align final DataFusion 55 expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: isolate DataFusion 55 regression cases Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: preserve empty result schema in timestamp widening Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: preserve JSON source column order Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * chore: use released DataFusion 55 integrations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: adapt latest execution plan mock to DataFusion 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: pin DataFusion recursive schema and date repairs Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(promql): align dictionary temporality match keys Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: retain Greptime DataFusion fork behaviors on version 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: restore ordinary function error expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: refresh distributed count compatibility plan Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(query): adapt last-row cast hint to DataFusion 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: refresh instant last-row empty results for Arrow 59 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * style: simplify DataFusion expression visitor imports Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: restore sorting and PostgreSQL column-order assertions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(function): restore primitive numeric coercion signatures Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * refactor(function): share geo integer signature types Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: cover timestamp widening overflow boundaries Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: fix decimal coercion regression imports Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(function): preserve scalar count_hash NULL state semantics Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: simplify decimal clamp case type inference Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: retain historical count_hash wrapper result Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: restore timestamp widening equality and IN pruning Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: carry upstream aggregate dynamic filter correctness fix Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: carry upstream null and predicate simplification fixes Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: restore baseline JSON ordering expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: restore histogram JSON ordering expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: refresh empty PromQL range result schemas Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: align native timestamp plan with DF55 decimal display Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: refresh native timestamp SQLness results for DF55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: regenerate NULL sample empty result headers for DF55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: use DF55 child replacement API in timestamp regressions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: expose pushed scan dynamic filters to DF55 producers Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: encode string-backed PostgreSQL OID aliases in binary results Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: verify REGPROC binary and text over PostgreSQL protocol Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: register real PostgreSQL catalogs in server fixtures Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: complete DF55 expression inventories for custom query plans Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: correct RangeSelect expression fixture and column identities Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * ci: wait for Kafka WAL helper deployment rollout Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: update custom storage empty result headers Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: require exact row counts in scan statistics Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: suppress deprecated partition_statistics warning in test Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Co-authored-by: Ning Sun <sunng@protonmail.com>
163 lines
11 KiB
Plaintext
163 lines
11 KiB
Plaintext
-- Set-comparison subqueries (`ANY`/`ALL`) must be rewritten before
|
|
-- PushDownFilter. Otherwise the set-comparison subquery can be pushed into
|
|
-- TableScan.partial_filters, which is not a valid remote scan filter.
|
|
CREATE TABLE sc_t (
|
|
ts TIMESTAMP(3) TIME INDEX,
|
|
v INT,
|
|
PRIMARY KEY (v)
|
|
) ENGINE = mito;
|
|
|
|
Affected Rows: 0
|
|
|
|
CREATE TABLE sc_s (
|
|
ts TIMESTAMP(3) TIME INDEX,
|
|
v INT
|
|
) ENGINE = mito;
|
|
|
|
Affected Rows: 0
|
|
|
|
INSERT INTO sc_t VALUES
|
|
('2024-01-30 00:00:00', 1),
|
|
('2024-01-30 01:00:00', 6),
|
|
('2024-01-30 02:00:00', 10);
|
|
|
|
Affected Rows: 3
|
|
|
|
INSERT INTO sc_s VALUES
|
|
('2024-01-30 00:00:00', 5),
|
|
('2024-01-30 01:00:00', NULL);
|
|
|
|
Affected Rows: 2
|
|
|
|
ADMIN FLUSH_TABLE('sc_t');
|
|
|
|
+---------------------------+
|
|
| ADMIN FLUSH_TABLE('sc_t') |
|
|
+---------------------------+
|
|
| 0 |
|
|
+---------------------------+
|
|
|
|
ADMIN FLUSH_TABLE('sc_s');
|
|
|
|
+---------------------------+
|
|
| ADMIN FLUSH_TABLE('sc_s') |
|
|
+---------------------------+
|
|
| 0 |
|
|
+---------------------------+
|
|
|
|
-- SQLNESS REPLACE region=\d+\(\d+,\s+\d+\) region=REDACTED
|
|
-- SQLNESS REPLACE (peers.*) REDACTED
|
|
EXPLAIN SELECT v FROM sc_t WHERE v > ANY(SELECT v FROM sc_s) ORDER BY v;
|
|
|
|
+---------------+---------------------------------------------------------------------------------------------------------------+
|
|
| plan_type | plan |
|
|
+---------------+---------------------------------------------------------------------------------------------------------------+
|
|
| logical_plan | Sort: sc_t.v ASC NULLS LAST |
|
|
| | Projection: sc_t.v |
|
|
| | Filter: __correlated_sq_1.mark OR __correlated_sq_2.mark AND NOT __correlated_sq_1.mark AND Boolean(NULL) |
|
|
| | LeftMark Join: Filter: sc_t.v > __correlated_sq_2.v IS NULL |
|
|
| | Filter: __correlated_sq_1.mark OR NOT __correlated_sq_1.mark AND Boolean(NULL) |
|
|
| | LeftMark Join: Filter: sc_t.v > __correlated_sq_1.v IS TRUE |
|
|
| | Projection: sc_t.v |
|
|
| | MergeScan [is_placeholder=false, remote_input=[ |
|
|
| | TableScan: sc_t |
|
|
| | ]] |
|
|
| | MergeScan [is_placeholder=false, remote_input=[ |
|
|
| | Projection: __correlated_sq_1.v |
|
|
| | SubqueryAlias: __correlated_sq_1 |
|
|
| | Projection: sc_s.v |
|
|
| | TableScan: sc_s |
|
|
| | ]] |
|
|
| | MergeScan [is_placeholder=false, remote_input=[ |
|
|
| | Projection: __correlated_sq_2.v |
|
|
| | SubqueryAlias: __correlated_sq_2 |
|
|
| | Projection: sc_s.v |
|
|
| | TableScan: sc_s |
|
|
| | ]] |
|
|
| physical_plan | SortPreservingMergeExec: [v@0 ASC NULLS LAST] |
|
|
| | SortExec: expr=[v@0 ASC NULLS LAST], preserve_partitioning=[true] |
|
|
| | FilterExec: mark@1 OR mark@2 AND NOT mark@1 AND NULL, projection=[v@0] |
|
|
| | NestedLoopJoinExec: join_type=LeftMark, filter=v@0 > v@1 IS NULL |
|
|
| | CoalescePartitionsExec |
|
|
| | FilterExec: mark@1 OR NOT mark@1 AND NULL |
|
|
| | NestedLoopJoinExec: join_type=LeftMark, filter=(v@0 > v@1) IS NOT DISTINCT FROM true |
|
|
| | ProjectionExec: expr=[v@1 as v] |
|
|
| | MergeScanExec: REDACTED
|
|
| | RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1 |
|
|
| | MergeScanExec: REDACTED
|
|
| | RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1 |
|
|
| | MergeScanExec: REDACTED
|
|
| | |
|
|
+---------------+---------------------------------------------------------------------------------------------------------------+
|
|
|
|
SELECT v FROM sc_t WHERE v > ANY(SELECT v FROM sc_s) ORDER BY v;
|
|
|
|
+----+
|
|
| v |
|
|
+----+
|
|
| 6 |
|
|
| 10 |
|
|
+----+
|
|
|
|
-- SQLNESS REPLACE region=\d+\(\d+,\s+\d+\) region=REDACTED
|
|
-- SQLNESS REPLACE (peers.*) REDACTED
|
|
EXPLAIN SELECT v FROM sc_t WHERE v != ALL(SELECT v FROM sc_s) ORDER BY v;
|
|
|
|
+---------------+----------------------------------------------------------------------------------------------------+
|
|
| plan_type | plan |
|
|
+---------------+----------------------------------------------------------------------------------------------------+
|
|
| logical_plan | Sort: sc_t.v ASC NULLS LAST |
|
|
| | Projection: sc_t.v |
|
|
| | Filter: __correlated_sq_2.mark AND Boolean(NULL) OR NOT __correlated_sq_2.mark |
|
|
| | LeftMark Join: Filter: sc_t.v != __correlated_sq_2.v IS NULL |
|
|
| | Projection: sc_t.v |
|
|
| | Filter: NOT __correlated_sq_1.mark |
|
|
| | LeftMark Join: Filter: sc_t.v != __correlated_sq_1.v IS FALSE |
|
|
| | Projection: sc_t.v |
|
|
| | MergeScan [is_placeholder=false, remote_input=[ |
|
|
| | TableScan: sc_t |
|
|
| | ]] |
|
|
| | MergeScan [is_placeholder=false, remote_input=[ |
|
|
| | Projection: __correlated_sq_1.v |
|
|
| | SubqueryAlias: __correlated_sq_1 |
|
|
| | Projection: sc_s.v |
|
|
| | TableScan: sc_s |
|
|
| | ]] |
|
|
| | MergeScan [is_placeholder=false, remote_input=[ |
|
|
| | Projection: __correlated_sq_2.v |
|
|
| | SubqueryAlias: __correlated_sq_2 |
|
|
| | Projection: sc_s.v |
|
|
| | TableScan: sc_s |
|
|
| | ]] |
|
|
| physical_plan | SortPreservingMergeExec: [v@0 ASC NULLS LAST] |
|
|
| | SortExec: expr=[v@0 ASC NULLS LAST], preserve_partitioning=[true] |
|
|
| | FilterExec: mark@1 AND NULL OR NOT mark@1, projection=[v@0] |
|
|
| | NestedLoopJoinExec: join_type=LeftMark, filter=v@0 != v@1 IS NULL |
|
|
| | CoalescePartitionsExec |
|
|
| | FilterExec: NOT mark@1, projection=[v@0] |
|
|
| | NestedLoopJoinExec: join_type=LeftMark, filter=(v@0 != v@1) IS NOT DISTINCT FROM false |
|
|
| | ProjectionExec: expr=[v@1 as v] |
|
|
| | MergeScanExec: REDACTED
|
|
| | RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1 |
|
|
| | MergeScanExec: REDACTED
|
|
| | RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1 |
|
|
| | MergeScanExec: REDACTED
|
|
| | |
|
|
+---------------+----------------------------------------------------------------------------------------------------+
|
|
|
|
SELECT v FROM sc_t WHERE v != ALL(SELECT v FROM sc_s) ORDER BY v;
|
|
|
|
+---+
|
|
| v |
|
|
+---+
|
|
+---+
|
|
|
|
DROP TABLE sc_t;
|
|
|
|
Affected Rows: 0
|
|
|
|
DROP TABLE sc_s;
|
|
|
|
Affected Rows: 0
|
|
|