Commit Graph

980 Commits

Author SHA1 Message Date
dennis zhuang
aa0c5b888c docs: update readme (#1549)
* docs: update readme

* Update README.md

Co-authored-by: Ning Sun <classicning@gmail.com>

* chore: cr comments

* chore: cr comments

---------

Co-authored-by: Ning Sun <classicning@gmail.com>
2023-05-10 14:36:07 +08:00
Weny Xu
fbb7db42aa chore: unify code styling (#1523) v0.3.0-nightly-20230510 2023-05-10 11:10:39 +08:00
Ning Sun
a1587595d9 feat: add information_schema as exception of cross schema check (#1551)
* feat: add information_schema as a cross-schema query exception

* fix: resolve lint issue
2023-05-10 10:55:00 +08:00
Weny Xu
abd5a8ecbb chore(datasource): make CompressionType follow the style of the guide (#1522) 2023-05-10 10:50:24 +08:00
Ruihang Xia
4ddab8e982 build: change release CI to only run test on linux (#1548)
* disable all linux release

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split linux and macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* correct job name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add missing build job

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* run build-macos first

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable unstable test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable test on macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* re-enable test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* do not dependent on build-macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-05-10 10:49:14 +08:00
Yingwen
1833e487a4 refactor: remove unnecessary async from RepeatedTask::start (#1545)
* refactor: relax RepeatedTask requirements

Some refactor:
- Remove async from start()
- Cancel task in drop
- Allow TaskFunction::call taking &mut self
- Make start/stop concurrent safe

* test(log-store): Fix log store tests (start multiple times)
2023-05-09 21:03:15 +08:00
ZonaHe
c93b5743e8 feat: update dashboard to v0.2.4 (#1553)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-05-09 20:56:20 +08:00
Weny Xu
550c494d25 fix: Copy from must follow the order of table fields issue (#1521)
* fix: Copy from must follow the order of table fields issue

* chore: apply suggestion from CR
2023-05-09 17:46:16 +08:00
Yingwen
2ab0e42d6f feat: clean procedure's state after it is done (#1543)
* feat(common-procedure): pub(crate) use proc_path

* feat(common-procedure): Implement delete_procedure

* feat(common-procedure): Clean procedure after it is finished

* chore(common-procedure): put path_string in front of try_stream

* test(common-procedure): Test cleaning up procedures

* feat(common-procedure): Clean procedure states in recover()

* feat(common-procedure): Use VecDeque for finished procedures
2023-05-09 11:44:50 +08:00
JeremyHi
05e6ca1e14 fix: the latest number of regions (#1546)
* fix: the latest number of regions

* fix: unit test
2023-05-09 10:11:26 +08:00
localhost
b9661818f2 chore: remove useless Option type in plugins (#1544)
Co-authored-by: paomian <qtang@greptime.com>
2023-05-08 21:54:24 +08:00
localhost
f86390345c chore: remove useless Option type in plugins (#1544)
Co-authored-by: paomian <qtang@greptime.com>
2023-05-08 21:53:45 +08:00
localhost
7191bb9652 chore: remove useless Option type in plugins (#1544)
Co-authored-by: paomian <qtang@greptime.com>
2023-05-08 21:52:12 +08:00
localhost
34c7f78861 chore: add configurator to http server (#1488)
* chore: add configurator params to start server fun

* chore: update plugins type

---------

Co-authored-by: paomian <qtang@greptime.com>
2023-05-08 10:55:03 +00:00
JeremyHi
610651fa8f feat: meta metrics (#1538)
* chore: from_etcd_kv (better name)

* feat: kv request metric

* feat: router metric

* feat: connections metric
2023-05-08 17:50:21 +08:00
fys
c48067f88d fix: no active datanode when frontend start (#1533)
* fix: no active datanode when frontend start

* chore: add log when can not get stat_val
2023-05-08 15:02:07 +08:00
Ning Sun
ec1b95c250 docs: add play section (#1528)
* docs: add play section

* Update README.md

Co-authored-by: xiaomin tang <xtang@users.noreply.github.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: xiaomin tang <xtang@users.noreply.github.com>
2023-05-08 14:26:22 +08:00
gitccl
fbf1ddd006 feat: open catalogs and schemas in parallel (#1527)
* feat: open catalogs and schemas in parallel

* fix: code review
2023-05-08 10:34:30 +08:00
Ning Sun
d679cfcb53 feat: add semantic_type to information_schema.columns (#1530) 2023-05-06 15:48:37 +08:00
discord9
2c82ded975 feat: table metrics (#1469)
* feat: Statistic

* add todo

* fmt: cargo fmt

* feat: some simple impl for MemTable

* chore: a try on adding statistics

* Update src/table/src/stats.rs

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* docs: fix typo

* newlines unnecessary

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-05-06 14:59:49 +08:00
Ruihang Xia
d4f3f617e4 chore(toolchain): update rust-toolchain to 2023-05-03 (#1524)
* chore(toolchain): update rust-toolchain to 2023-05-03

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update workflow yaml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-05-06 11:34:09 +08:00
Ruihang Xia
6fe117d7d5 fix: vector and matrix in Prometheus use different field (#1520)
* fix empty tag

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix result type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* make it work

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-05-05 15:54:26 +08:00
Ning Sun
b0ab641602 feat: add catalog/schema/table count as catalog metrics (#1499)
* feat: add catalog/schema/table count as catalog metrics

* test: add integration tests for catalog metrics
2023-05-05 05:54:12 +00:00
Huaijin
224ec9bd25 fix: wrong max_table_id log in remote catalog manager (#1516)
* fix: wrong max_table_id log in remote catalog manager

* chore: update link in CONTRIBUTING.md

* chore: add a new const MAX_SYS_TABLE_ID
2023-05-05 03:39:45 +00:00
Niwaka
d86b3386dc fix: incorrect show create table output (#1514)
* fix: incorrect show create table output

* feat: change CreateTable's Display if table is external

* feat: change CreateTable's Display if table is external
2023-05-05 11:29:09 +08:00
Lei, HUANG
c8301feed7 fix: respect MySQL timestamp format (#1510) 2023-05-04 18:57:38 +08:00
dennis zhuang
b1920c41a4 fix: object store cache bug (#1482)
* feat: use streaming read instead of reading whole file

* feat: enable atomic writing for object store file caching

* fix: recover existing keys from local cache

* test: recovering keys from local file cache for LruCachePolicy

* Update src/datanode/src/instance.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: cr comments

* feat: md5 hash caching path

* fix: test

* fix: read cache

* Update src/object-store/src/cache_policy.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
2023-05-04 18:25:40 +08:00
Weny Xu
c471007edd feat: support to copy table from/to CSV and JSON format file (#1475)
* refactor: refactor copy from executor

* feat: support to copy from CSV and JSON format files

* feat: support to copy table to the CSV and JSON format file

* test: add tests copy from/to

* chore: apply suggestions from CR
2023-05-04 17:20:28 +08:00
Yingwen
2818f466d3 feat: Log error in GreptimeRequestHandler (#1507)
* feat(common-error): Add should_log_error

* feat(servers): log error in grpc handler
2023-05-04 15:48:38 +08:00
JeremyHi
d7a906e0bd feat: metasrv mailbox (#1481)
* refactor: id first in pusher_key

* feat: is_acceptable for multi roles

* feat: mailbox

* fix: channel for mailbox

* feat: impl mailbox via heartbeat

* chore: add unit test for mailbox

* chore: by cr

* chore: typo

* chore: refactor the mailbox API

* chore: br cr

* chore: check timeout interval to 10ms

* chore: add response header
2023-05-04 15:42:43 +08:00
Ning Sun
6e1bb9e458 feat: add support for information_schema.columns (#1500)
* feat: add support for information_schema.columns

* feat: remove information_schema from its view

* Update src/catalog/src/information_schema.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* fix: error on table data type

* test: correct sqlness test for information schema

* test: add information_schema.columns sqlness tests

---------

Co-authored-by: LFC <bayinamine@gmail.com>
2023-05-04 14:29:38 +08:00
Ning Sun
494ad570c5 feat: update pgwire to 0.14 (#1504) 2023-05-04 14:24:26 +08:00
Vanish
12d59e6341 chore: remove redundant code. (#1502) 2023-05-04 14:20:26 +08:00
Yingwen
479ef9d379 fix: checkpoint GC task also deletes the file with the last version (#1491)
* test(storage): use assert_eq to check scan result

* feat(storage): Add more info to manifest log

* feat: Avoid error log when unable to delete

* fix: The manifest gc task should delete files <= last_version

* feat(storage): Don't log if the error kind is not found

* feat: Add keep_last_checkpoint option
2023-05-04 14:18:38 +08:00
Niwaka
93ffe1ff33 feat: improve and distinguish different errors for IllegalInsertData (#1503)
* feat: improve and distinguish different errors for IllegalInsertData

* feat: change error code for UnexpectedValuesLength and ColumnAlreadyExists

* chore: improve readability of error message
2023-05-04 12:36:24 +08:00
Niwaka
d461328238 fix: insert distributed table if partition column has default value (#1498)
* fix: insert distributed table if partition column has default value

* Address review

* address review

* address review

* chore: introduce assert_columns

---------

Co-authored-by: WenyXu <wenymedia@gmail.com>
2023-05-02 20:50:02 +08:00
Vanish
6aae5b7286 feat: prevent sensitive information (key, password, secrets etc.) from being printed in plain (#1501)
* feat: add secret type

* chore: replace key, password, secrets with secret type.

* chore: use secrecy

* chore: remove redundant file

* style: taplo fmt
2023-05-01 20:54:54 +08:00
Ning Sun
7dbac89000 feat: add metrics for protocol interfaces (#1495)
* feat: add metrics for various interfaces

* feat: add db label for protocols

* feat: add postgres protocol metrics

* feat: add metrics for grpcs apis

* feat: add auth failure counter for mysql/pg

* fix: add db label to grpc prometheus interface

* Apply suggestions from code review

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* feat: add error code for auth failure counter

* fix: use schema as dbname when catalog is default

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
2023-04-28 23:42:35 +08:00
Yingwen
0b0b5a10da feat: Remove store from procedure config (#1489)
* feat(procedure): Add key prefix

* feat: Remove store config from ProcedureConfig

* refactor(procedure): Address review comments

Add proc_path! macro and rename KEY_PREFIX to PROC_PATH

* docs: Update procedure config examples
2023-04-28 22:12:57 +08:00
Yingwen
51be35a7b1 feat(mito): Combine the original and procedure's implementation (#1468)
* fix(mito): Add metrics to mito DDL procedure

* feat(mito): Use procedure's implementation to create table

* feat(mito): Use procedure's implementation to alter table

* feat(mito): Use procedure's implementation to drop table

* style(mito): Fix clippy

* test(mito): Fix tests

* feat(mito): Add TableCreator

* feat(mito): update alter table procedure

* fix(mito): alter procedure create alter op first

* feat(mito): Combine alter table code

* fix(mito): Fix deadlock

* feat(mito): Simplify drop table procedure
2023-04-28 11:48:52 +08:00
Lei, HUANG
9e4887f29f fix: disable dashboard (#1494) 2023-04-27 22:55:15 +08:00
yuanbohan
cca34aa914 chore: upgrade promql-parser version (#1484) 2023-04-27 13:10:15 +00:00
Ruihang Xia
0ac50632aa feat: use server time if it's not specified (#1480)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-27 20:54:26 +08:00
Yingwen
b1f7ad097a test: Fix s3 region in test (#1493) 2023-04-27 12:25:20 +00:00
Weny Xu
a77a4a4bd1 fix: add s3 region info (#1492)
fix: add region info
2023-04-27 19:13:01 +08:00
Weny Xu
47f1cbaaed fix: add s3 region info (#1486) 2023-04-27 17:35:34 +08:00
Yingwen
8e3c3cbc40 build: Download assets to cargo output dir (#1476)
* build: Download assets to cargo output dir

Also remove the output from the build script and only print the output
on failure

* chore: Update src/servers/build.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* build: replace pushd by cd

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-27 17:09:10 +08:00
Vanish
9f0efc748d feat: make log level and destination configurable from config files (#1444)
* feat: implement load_options.

* refactor: build by ConfigOptions.

* refactor: init_global_logging by LoggingOptions.

* chore: make clippy happy.

* refactor: use TopLevelOptions push top level options to subcommand.

* test: test TopLevelOptions.

* refactor: push Options in Box.

* refactor: push Options in Box.

* refactor: use let-else and Options.
2023-04-27 15:30:04 +08:00
Ruihang Xia
939a51aea9 feat: adopt REPLACE interceptor and quit all processes on exit (#1478)
* bump version and update test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* quit all processes on drop

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update tests/runner/src/env.rs

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
2023-04-27 07:16:41 +00:00
Weny Xu
bf35620904 refactor: refactor BufferedWriter (#1439)
* feat: implement ApproximateBufWriter

* refactor: refactor BufferedWriter

* refactor: remove ApproximateBufWriter

* fix: fix losing pending writes issue

* chore: fmt

* chore: remove unused import

* chore: rename method name

* feat: return written row count

* chore: apply suggestions from CR

* fix: fix counting the bytes_written twice issue
2023-04-27 14:45:33 +08:00