rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-06 13:22:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat!: Bump datafusion, prost, hyper, tonic, tower, axum (#5417)

Browse Source 
* change dep

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: adapt to arrow's interval array

* chore: fix compile errors in datatypes crate

* chore: fix api crate compiler errors

* chore: fix compiler errors in common-grpc

* chore: fix common-datasource errors

* chore: fix deprecated code in common-datasource

* fix promql and physical plan related

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* wip: upgrading network deps

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* block on updating `sqlparser`

* upgrade sqlparser

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* adapt new df's trait requirements

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: fix compiler errors in mito2

* chore: fix common-function crate errors

* chore: fix catalog errors

* change import path

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: fix some errors in query crate

* chore: fix some errors in query crate

* aggr expr and some other tiny fixes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: fix expr related errors in query crate

* chore: fix query serializer and admin command

* chore: fix grpc services

* feat: axum serve

* chore: fix http server

* remove handle_error handler
* refactor timeout layer
* serve axum

* chore: fix flow aggr functions

* chore: fix flow

* feat: fix errors in meta-srv

* boxed()
* use TokioIo

* feat!: Remove script crate and python feature (#5321)

* feat: exclude script crate

* chore: simplify feature

* feat: remove the script crate

* chore: remove python feature and some comments

* chore: fix warning

* chore: fix servers tests compiler errors

* feat: fix tests-integration errors

* chore: fix unused

* test: fix catalog test

* chore: fix compiler errors for crates using common-meta

testing feature is enabled when check with --workspace

* test: use display for logical plan test

* test: implement rewrite for ScanHintRule

* fix: http server build panic

* test: fix mito test

* fix: sql parser type alias error

* test: fix TestClient not listen

* test: some flow tests

* test(flow): more fix

* fix: test_otlp_logs

* test: fix promql test that using deprecated method fun()

* fix: sql type replace supports Int8 ~ Int64, UInt8 ~ UInt64

* test: fix infer schema test case

* test: fix tests related to plan display

* chore: fix last flow test

* test: fix function format related assertion

* test: use larger port range for tests

* fix: test_otlp_traces

* fix: test_otlp_metrics

* fix range query and dist plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: flow handle distinct use deprecated field

* fix: can't pass Join plan expressions to LogicalPlan::with_new_exprs

* test: fix deserialize test

* test: reduce split key case num

* tests: lower case aggr func name

* test: fix some sqlness tests

* tests: more sqlness fix

* tests: fixed sqlness test

* commit non-bug changes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: make our udf correct

* fix: implement empty methods of ContextProvider for DfContextProviderAdapter

* test: update sqlness test result

* chore: remove unused

* fix: provide alias name for AggregateExprBuilder in range plan

* test: update range query result

* fix: implement missing ContextProvider methods for DfContextProviderAdapter

* test: update timestamps, cte result

* fix: supports empty projection in mito

* test: update comment for cte test

* fix: support projection for numbers

* test: update test cases after projection fix

* fix: fix range select first_value/last_value

* fix: handle CAST and time index conflict

* fix: handle order by correctly in range first_value/last_value

* test: update sqlness result

* test: update view test result

* test: update decimal test

wait for https://github.com/apache/datafusion/pull/14126 to fix this

* feat: remove redundant physical optimization

todo(ruihang): Check if we can remove this.

* test: update sqlness test result

* chore: range select default sort use nulls_first = false

* test: update filter push down test result

* test: comment deciaml test to avoid different panic message

* test: update some distributed test result

* test: update test for distributed count and filter push down

* test: update subqueries test

* fix: SessionState may overwrite our UDFs

* chore: fix compiler errors after merging main

* fix: fix elasticsearch and dashboard router panic

* chore: fix common-functions tests

* chore: update sqlness result

* test: fix id keyword and update sqlness result

* test: fix flow_null test

* fix: enlarge thread size in debug mode to avoid overflow

* chore: fix warnings in common-function

* chore: fix warning in flow

* chore: fix warnings in query crate

* chore: remove unused warnings

* chore: fix deprecated warnings for parquet

* chore: fix deprecated warning in servers crate

* style: fix clippy

* test: enlarge mito cache tttl test ttl time

* chore: fix typo

* style: fmt toml

* refactor: reimplement PartialOrd for RangeSelect

* chore: remove script crate files introduced by merge

* fix: return error if sql option is not kv

* chore: do not use ..default::default()

* chore: per review

* chore: update error message in BuildAdminFunctionArgsSnafu

Co-authored-by: jeremyhi <jiachun_feng@proton.me>

* refactor: typed precision

* update sqlness view case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: flow per review

* chore: add example in comment

* chore: warn if parquet stats of timestamp is not INT64

* style: add a newline before derive to make the comment more clear

* test: update sqlness result

* fix: flow from substrait

* chore: change update_range_context log to debug level

* chore: move axum-extra axum-macros to workspace

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: luofucong <luofc@foxmail.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: shuiyisong <xixing.sys@gmail.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
This commit is contained in:
Yingwen
2025-01-23 14:15:40 +08:00
committed by GitHub
parent 3ed085459c
commit 35b635f639
293 changed files with 4067 additions and 3225 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
tests/cases/distributed/explain/analyze.result
View File

@@ -19,6 +19,14 @@ VALUES
Affected Rows: 3
SELECT count(*) FROM system_metrics;
+----------+
| count(*) |
+----------+
| 3 |
+----------+
-- SQLNESS REPLACE (metrics.*) REDACTED
-- SQLNESS REPLACE (RoundRobinBatch.*) REDACTED
-- SQLNESS REPLACE (-+) -
@@ -32,9 +40,10 @@ explain analyze SELECT count(*) FROM system_metrics;
+-+-+-+
| 0_| 0_|_MergeScanExec: REDACTED
|_|_|_|
| 1_| 0_|_AggregateExec: mode=Final, gby=[], aggr=[COUNT(system_REDACTED
| 1_| 0_|_ProjectionExec: expr=[count(system_REDACTED
|_|_|_AggregateExec: mode=Final, gby=[], aggr=[count(system_REDACTED
|_|_|_CoalescePartitionsExec REDACTED
|_|_|_AggregateExec: mode=Partial, gby=[], aggr=[COUNT(system_REDACTED
|_|_|_AggregateExec: mode=Partial, gby=[], aggr=[count(system_REDACTED
|_|_|_RepartitionExec: partitioning=REDACTED
|_|_|_SeqScan: region=REDACTED, partition_count=1 (1 memtable ranges, 0 file 0 ranges) REDACTED
|_|_|_|

2
tests/cases/distributed/explain/analyze.sql
View File

@@ -15,6 +15,8 @@ VALUES
("host2", "idc_a", 80.0, 70.3, 90.0, 1667446797450),
("host1", "idc_b", 50.0, 66.7, 40.6, 1667446797450);
SELECT count(*) FROM system_metrics;
-- SQLNESS REPLACE (metrics.*) REDACTED
-- SQLNESS REPLACE (RoundRobinBatch.*) REDACTED
-- SQLNESS REPLACE (-+) -

15
tests/cases/distributed/explain/analyze_append_table_count.result
View File

@@ -17,6 +17,19 @@ WITH(
Affected Rows: 0
INSERT INTO `test_table` (`bytes`, `http_version`, `ip`, `method`, `path`, `status`, `user`, `timestamp`)
VALUES (1024, 'HTTP/1.1', '192.168.1.1', 'GET', '/index.html', 200, 'user1', 1667446797450);
Affected Rows: 1
SELECT count(*) FROM test_table;
+----------+
| count(*) |
+----------+
| 1 |
+----------+
-- SQLNESS REPLACE (metrics.*) REDACTED
-- SQLNESS REPLACE (RoundRobinBatch.*) REDACTED
-- SQLNESS REPLACE (-+) -
@@ -30,7 +43,7 @@ EXPLAIN ANALYZE SELECT count(*) FROM test_table;
+-+-+-+
| 0_| 0_|_MergeScanExec: REDACTED
|_|_|_|
| 1_| 0_|_ProjectionExec: expr=[0 as COUNT(test_table.timestamp)] REDACTED
| 1_| 0_|_ProjectionExec: expr=[1 as count(*)] REDACTED
|_|_|_common_recordbatch::adapter::MetricCollector REDACTED
|_|_|_|
|_|_| Total rows: 1_|

5
tests/cases/distributed/explain/analyze_append_table_count.sql
View File

@@ -16,6 +16,11 @@ WITH(
append_mode = 'true'
);
INSERT INTO `test_table` (`bytes`, `http_version`, `ip`, `method`, `path`, `status`, `user`, `timestamp`)
VALUES (1024, 'HTTP/1.1', '192.168.1.1', 'GET', '/index.html', 200, 'user1', 1667446797450);
SELECT count(*) FROM test_table;
-- SQLNESS REPLACE (metrics.*) REDACTED
-- SQLNESS REPLACE (RoundRobinBatch.*) REDACTED
-- SQLNESS REPLACE (-+) -

6
tests/cases/distributed/explain/join_10_tables.result
View File

@@ -73,8 +73,7 @@ limit 1;
+-+-+
| plan_type_| plan_|
+-+-+
| logical_plan_| Limit: skip=0, fetch=1_|
|_|_Sort: t_1.ts DESC NULLS FIRST, fetch=1_|
| logical_plan_| Sort: t_1.ts DESC NULLS FIRST, fetch=1_|
|_|_Inner Join: t_9.ts = t_10.ts, t_9.vin = t_10.vin_|
|_|_Inner Join: t_8.ts = t_9.ts, t_8.vin = t_9.vin_|
|_|_Inner Join: t_7.ts = t_8.ts, t_7.vin = t_8.vin_|
@@ -96,8 +95,7 @@ limit 1;
|_|_MergeScan [is_placeholder=false]_|
|_|_MergeScan [is_placeholder=false]_|
|_|_MergeScan [is_placeholder=false]_|
| physical_plan | GlobalLimitExec: skip=0, fetch=1_|
|_|_SortPreservingMergeExec: [ts@0 DESC], fetch=1_|
| physical_plan | SortPreservingMergeExec: [ts@0 DESC], fetch=1_|
|_|_SortExec: TopK(fetch=1), expr=[ts@0 DESC], preserve_partitioning=[true]_|
|_|_CoalesceBatchesExec: target_batch_size=8192_|
|_|_REDACTED

4
tests/cases/distributed/explain/order_by.result
View File

@@ -61,8 +61,8 @@ EXPLAIN SELECT DISTINCT a, b FROM test ORDER BY a, b;
| logical_plan_| Sort: test.a ASC NULLS LAST, test.b ASC NULLS LAST_|
|_|_Aggregate: groupBy=[[test.a, test.b]], aggr=[[]]_|
|_|_MergeScan [is_placeholder=false]_|
| physical_plan | SortPreservingMergeExec: [a@0 ASC NULLS LAST,b@1 ASC NULLS LAST]_|
|_|_SortExec: expr=[a@0 ASC NULLS LAST,b@1 ASC NULLS LAST], preserve_partitioning=[true] |
| physical_plan | SortPreservingMergeExec: [a@0 ASC NULLS LAST, b@1 ASC NULLS LAST]_|
|_|_SortExec: expr=[a@0 ASC NULLS LAST, b@1 ASC NULLS LAST], preserve_partitioning=[true] |
|_|_AggregateExec: mode=FinalPartitioned, gby=[a@0 as a, b@1 as b], aggr=[]_|
|_|_CoalesceBatchesExec: target_batch_size=8192_|
|_|_RepartitionExec: partitioning=REDACTED

31
tests/cases/distributed/explain/subqueries.result
View File

@@ -82,7 +82,7 @@ order by t.i desc;
+-+-+
| logical_plan_| Sort: t.i DESC NULLS FIRST_|
|_|_SubqueryAlias: t_|
|_|_CrossJoin:_|
|_|_Cross Join:_|
|_|_Filter: integers.i IS NOT NULL_|
|_|_Projection: integers.i_|
|_|_MergeScan [is_placeholder=false]_|
@@ -101,19 +101,26 @@ order by t.i desc;
|_|_|
+-+-+
INSERT INTO other SELECT i, 2 FROM integers WHERE i=(SELECT MAX(i) FROM integers);
Affected Rows: 0
-- Explain physical plan for DML is not supported because it looks up the table name in a way that is
-- different from normal queries. It also requires the table provider to implement the `insert_into()` method.
EXPLAIN INSERT INTO other SELECT i, 2 FROM integers WHERE i=(SELECT MAX(i) FROM integers);
+--------------+-------------------------------------------------------------------+
| plan_type | plan |
+--------------+-------------------------------------------------------------------+
| logical_plan | Dml: op=[Insert Into] table=[other] |
| | Projection: integers.i AS i, TimestampMillisecond(2, None) AS j |
| | Inner Join: integers.i = __scalar_sq_1.MAX(integers.i) |
| | Projection: integers.i |
| | MergeScan [is_placeholder=false] |
| | SubqueryAlias: __scalar_sq_1 |
| | MergeScan [is_placeholder=false] |
+--------------+-------------------------------------------------------------------+
+---------------------+-------------------------------------------------------------------+
| plan_type | plan |
+---------------------+-------------------------------------------------------------------+
| logical_plan | Dml: op=[Insert Into] table=[other] |
| | Projection: integers.i AS i, TimestampMillisecond(2, None) AS j |
| | Inner Join: integers.i = __scalar_sq_1.max(integers.i) |
| | Projection: integers.i |
| | MergeScan [is_placeholder=false] |
| | SubqueryAlias: __scalar_sq_1 |
| | MergeScan [is_placeholder=false] |
| physical_plan_error | Error during planning: failed to resolve catalog: datafusion |
+---------------------+-------------------------------------------------------------------+
drop table other;

4
tests/cases/distributed/explain/subqueries.sql
View File

@@ -28,6 +28,10 @@ from (
where t.i is not null
order by t.i desc;
INSERT INTO other SELECT i, 2 FROM integers WHERE i=(SELECT MAX(i) FROM integers);
-- Explain physical plan for DML is not supported because it looks up the table name in a way that is
-- different from normal queries. It also requires the table provider to implement the `insert_into()` method.
EXPLAIN INSERT INTO other SELECT i, 2 FROM integers WHERE i=(SELECT MAX(i) FROM integers);
drop table other;