Commit Graph

940 Commits

Author SHA1 Message Date
Lei, HUANG
9e4887f29f fix: disable dashboard (#1494) 2023-04-27 22:55:15 +08:00
yuanbohan
cca34aa914 chore: upgrade promql-parser version (#1484) 2023-04-27 13:10:15 +00:00
Ruihang Xia
0ac50632aa feat: use server time if it's not specified (#1480)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-27 20:54:26 +08:00
Yingwen
b1f7ad097a test: Fix s3 region in test (#1493) 2023-04-27 12:25:20 +00:00
Weny Xu
a77a4a4bd1 fix: add s3 region info (#1492)
fix: add region info
2023-04-27 19:13:01 +08:00
Weny Xu
47f1cbaaed fix: add s3 region info (#1486) 2023-04-27 17:35:34 +08:00
Yingwen
8e3c3cbc40 build: Download assets to cargo output dir (#1476)
* build: Download assets to cargo output dir

Also remove the output from the build script and only print the output
on failure

* chore: Update src/servers/build.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* build: replace pushd by cd

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-27 17:09:10 +08:00
Vanish
9f0efc748d feat: make log level and destination configurable from config files (#1444)
* feat: implement load_options.

* refactor: build by ConfigOptions.

* refactor: init_global_logging by LoggingOptions.

* chore: make clippy happy.

* refactor: use TopLevelOptions push top level options to subcommand.

* test: test TopLevelOptions.

* refactor: push Options in Box.

* refactor: push Options in Box.

* refactor: use let-else and Options.
2023-04-27 15:30:04 +08:00
Ruihang Xia
939a51aea9 feat: adopt REPLACE interceptor and quit all processes on exit (#1478)
* bump version and update test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* quit all processes on drop

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update tests/runner/src/env.rs

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
2023-04-27 07:16:41 +00:00
Weny Xu
bf35620904 refactor: refactor BufferedWriter (#1439)
* feat: implement ApproximateBufWriter

* refactor: refactor BufferedWriter

* refactor: remove ApproximateBufWriter

* fix: fix losing pending writes issue

* chore: fmt

* chore: remove unused import

* chore: rename method name

* feat: return written row count

* chore: apply suggestions from CR

* fix: fix counting the bytes_written twice issue
2023-04-27 14:45:33 +08:00
Weny Xu
09f55e3cd8 chore: remove info log (#1483) 2023-04-27 14:05:22 +08:00
dennis zhuang
b88d8e5b82 feat: bump opendal to 0.33 (#1479) 2023-04-27 12:13:18 +08:00
Weny Xu
a709a5c842 feat: support to create parquet format external table (#1463)
* feat: support parquet format external table

* Update src/file-table-engine/src/error.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-26 16:45:37 +08:00
Lei, HUANG
fb9978e95d refactor: catalog (#1454)
* wip

* add schema_async

* remove CatalogList

* remove catalog provider and schema provider

* fix

* fix: rename table

* fix: sqlness

* fix: ignore tonic error metadata

* fix: table engine name

* feat: rename catalog_async to catalog

* respect engine name in table regional value when deregistering tables

* fix: CR
2023-04-26 08:36:40 +00:00
discord9
ef4e473e6d fix: recompile&register scripts as UDF on reboot (#1421)
* fixme: recompile somewhere else

* feat: re-compile&re-register all scripts in table

* fix: allow empty scripts table

* chore: add non-blocking somewhere

* chore: PR advices

* chore: more PR advices

* style: remove useless join

* style: remove redunent code

* refactor: use `bg` runtime instead

* style: cargo fmt
2023-04-26 16:30:58 +08:00
Ning Sun
1a245f35b9 feat: improve metrics and log level (#1470)
* refactor: tune log and metrics for meta/frontend

* feat: add panic counter
2023-04-26 13:13:40 +08:00
dennis zhuang
8d8a480dc1 fix: object store caching bug, #1466 (#1467)
* fix: object store caching bug, #1466

* fix: forgot to add S3WithCache tests
2023-04-25 21:48:51 +08:00
Lei, HUANG
197c34bc17 fix: grpc client keepalive (#1461)
fix: grpc keepalive
2023-04-25 20:07:57 +08:00
Ruihang Xia
4d9afee8ef chore(deps): update substrait dep in client (#1453)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-25 16:21:59 +08:00
Weny Xu
7f14d40798 test: add tests for external table (#1460) 2023-04-25 15:14:46 +08:00
Yingwen
eb50cee601 feat: Switch to the procedure framework (#1448)
* feat: Remove create_mock_sql_handler()

create_to_request() and alter_to_request() don't need `&self`, so
we don't need to mock the sql handler to test them

* feat: Enable procedure manager by default

* docs: Update config example

* test: Enable procedure framework in all tests

* refactor(datanode): rename methods using procedure

* test(catalog): Fix temp dir drops before test finishes

* tests: Enable procedure framework in sqlness

* test: Fix sqlness standalone rename test

* fix: Drop procedure allows table not in engine

* test: Change rename table test

* fix: add options to table meta when creating table by procedure

* test: adjust error message in schema test case

* test: Fix test_sql_api error message
2023-04-25 12:04:02 +08:00
Lei, HUANG
92c0808766 fix: frontend opt should respect http addr in config file when no com… (#1456)
* fix: frontend opt should respect http addr in config file when no command options is given

* refactor: command line options should be Option<bool>

* fix: ci
2023-04-25 03:43:42 +00:00
Ruihang Xia
f9ea6b63bf feat: impl instant query and add tests (#1452)
* feat: impl instant query and add tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-25 11:08:14 +08:00
fys
2287db7ff7 fix: execute sql query in another catalog (#1457) 2023-04-25 10:30:35 +08:00
shuiyisong
69acf32914 chore: add len() to Bytes and StringBytes (#1455)
* chore: add `len()` to Bytes and StringBytes

* chore: add `len()` to Bytes and StringBytes
2023-04-25 10:18:41 +08:00
Ruihang Xia
b9db2cfd83 fix: support restart sqlness in distributed mode (#1443)
* fix: support restart sqlness in distributed mode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move alter_table case to common dir

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* is_standalone flag

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update tests/runner/src/env.rs

Co-authored-by: LFC <bayinamine@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
2023-04-24 19:36:12 +08:00
JeremyHi
6d247f73fd fix: add log on leader stepdown (#1450) 2023-04-24 19:16:57 +08:00
Ruihang Xia
2cf828da3c feat: implement Prometheus-compatible API in gRPC (#1449)
* update greptime-proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove duplicate delete enum

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl handler and service

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-24 18:03:48 +08:00
Weny Xu
f2167663b2 feat: support to create external table (#1372)
* feat: support to create external table

* chore: apply suggestions from CR

* test: add create external table without ts type

* chore: apply suggestions from CR

* fix: fix import typo

* refactor: move consts to table crate

* chore: apply suggestions from CR

* refactor: rename create_table_schema
2023-04-24 14:43:12 +08:00
LFC
17daf4cdff feat: support "delete" in distributed mode (#1441)
* feat: support "delete" in distributed mode

* fix: resolve PR comments
2023-04-24 12:07:50 +08:00
shuiyisong
7c6754d03e feat: meter write request (#1447)
* chore: add write meter

* chore: update meter macro

* chore: update meter framework url to https
2023-04-24 11:42:06 +08:00
zyy17
e64fea3a15 ci: upgrade nightly release tag from v0.2.0 to v0.3.0 (#1446) 2023-04-24 11:04:39 +08:00
Weny Xu
22b5a94d02 feat: support creating the physical plan for JSON and CSV files (#1424)
* feat: support creating the physical plan for JSON and CSV files

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* refactor(file-table-engine): use datasource Format instead
2023-04-24 10:17:11 +08:00
Weny Xu
d374859e24 refactor: replace Copy Format with datasource Format (#1435)
* refactor: replace Copy Format with datasource Format

* chore: apply suggestions from CR

* chore: apply suggestions from CR
2023-04-23 08:31:54 +00:00
Ning Sun
c5dba29f9e refactor: remove redundant plugins argument (#1436) 2023-04-23 12:39:46 +08:00
Hao
9f442dedf9 chore: fix some typo and add deriv to plan in promql (#1438) 2023-04-23 12:21:25 +08:00
Ruihang Xia
5d77ed00bb test: add basic cases for distributed TQL (#1437)
* test: add basic cases for distributed TQL

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* drop table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-23 03:34:42 +00:00
Zheming Li
c75845c570 fix: wrong next column in manifest (#1440)
Signed-off-by: Zheming Li <nkdudu@126.com>
2023-04-23 11:25:38 +08:00
Yingwen
1ee9ad4ca1 feat: manage multiple engine procedure in the engine manager (#1434)
* feat(table): Add engine procedure to engine manager

* feat(datanode): Get engine procedure from engine manager

* feat(table-procedure): Add source error to SubprocedureFailed

* test: Enable procedure in tests and pass all tests

* style(table-procedure): Fix clippy
2023-04-23 10:04:09 +08:00
Weny Xu
f2cc912c87 feat: implement ParquetFileReaderFactory (#1423)
* feat: implement ParquetFileReaderFactory

* refactor: use LazyParquetFileReader instead

* chore: apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
2023-04-21 13:40:58 +08:00
dennis zhuang
2a9f482bc7 feat: show create table (#1336)
* temp commit

* feat: impl Display for CreateTable statement

* feat: impl show create table for standalone

* fix: forgot show.rs

* feat: clean code

* fix: typo

* feat: impl show create table for distributed

* test: add show create table sqlness test

* fix: typo

* fix: sqlness tests

* feat: render partition rules for distributed table

* Update src/sql/src/statements.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/sql/src/statements.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/sql/src/statements.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/sql/src/statements/create.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: by CR comments

* fix: compile error

* fix: missing column comments and extra table options

* test: add show create table test

* test: add show create table test

* chore: timestamp precision

* fix: test

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
2023-04-21 11:37:16 +08:00
Weny Xu
d5e4662181 refactor: refactor stmt_options_to_table_options (#1403)
refactor: move stmt_options_to_table_options to query crate
2023-04-21 11:08:01 +08:00
Yingwen
9cd2cf630d feat: procedures for file table engine (#1417)
* refactor: Add table_ref() to requests as their methods

* feat: Add CreateImmutableFileTable

* feat: Add DropImmutableFileTable

* feat: Implement TableEngineProcedure for ImmutableFileTableEngine

* feat: Add common-procedure-test crate

* refactor: mito engine use common-procedure-test to test procedures

* test: Add test for create and drop table

* chore: Address review comments
2023-04-20 18:52:44 +08:00
Ruihang Xia
7152a1b79e feat: expose output_ordering on scan plan (#1425)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-20 17:58:48 +08:00
fys
f2cfd8e608 refactor: default catalog and schema are created at Metasrv (#1391)
* refactor: default catalog and schema are created at Metasrv

* fix: unit test

* fix: add license

* simplify the meta mock

* cr
2023-04-20 17:58:37 +08:00
ZonaHe
e8cd2f0e48 feat: update dashboard to v0.2.3 (#1430)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-04-20 17:51:11 +08:00
Yingwen
830367b8f4 feat: Drop table by procedure (#1401)
* feat: Add drop table procedure

* feat: support dropping table by procedure on datanode

* test: Add test for DropTableProcedure

* test: Test drop table by procedure

* chore: update comments

* fix: Make on_remove_from_catalog idempotent
2023-04-20 15:57:56 +08:00
Ruihang Xia
37678e2e02 ci: enable test on release (#1428)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-20 12:06:20 +08:00
Ruihang Xia
b6647af2e3 test: add integration case to check dashboard path (#1422)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-20 11:17:01 +08:00
ZonaHe
d2c90b4c59 feat: update dashboard to v0.2.2 (#1426)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
v0.2.0-nightly-20230420
2023-04-19 19:09:17 +08:00