Commit Graph

2423 Commits

Author SHA1 Message Date
Zhenchi
63681f0e4d refactor(table): remove unused table requests (#3603)
* refactor(table): remove unused requests

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* update comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: clippy

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: compile

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2024-03-28 11:31:14 +00:00
Ruihang Xia
06a90527a3 fix: adjust status code to http error code map (#3601)
* fix: adjust status code to http error code map

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update integration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2024-03-28 08:54:51 +00:00
niebayes
d5ba2fcf9d test: add more integration test for kafka wal (#3190)
* test: add integration tests for kafka wal

* chore: rebase main

* chore: unify naming convention for wal config

* chore: add register loaders switch

* chore: alter tables by adding a new column

* chore: move rand to dev-dependencies

* chore: update Cargo.lock
2024-03-28 06:55:18 +00:00
dennis zhuang
e3b37ee2c9 fix: canonicalize catalog and schema names (#3600) 2024-03-28 06:40:15 +00:00
dennis zhuang
5d7ce08358 feat: adds metric engine to information_schema engines table (#3599)
* feat: adds metric engine to information_schema engines table

* fix: support value for metric engine
2024-03-28 06:37:34 +00:00
JeremyHi
92a8e863de chore: do not reply for broadcast msg (#3595) 2024-03-27 11:39:23 +00:00
JeremyHi
9428cb8e7c feat: remove support for logical tables in the create table procedure (#3592)
* feat: Remove support for logical tables in the create table procedure

* chore: remove the redandent table ids alloc

* chore: minor fix
2024-03-27 10:03:42 +00:00
Weny Xu
5addb7d75a test: add tests for drop databases (#3594)
* refactor: minimize visibility of drop database steps

* feat: implement as_any

* refactor: move common functions to test_util

* test: add tests for drop databases

* fix: fix deteling physical table route unexpectedly
2024-03-27 09:18:37 +00:00
Weny Xu
623c930736 refactor: refactor drop table executor (#3589)
* refactor: refactor drop table executor

* chore: apply suggestions from CR
2024-03-27 06:29:54 +00:00
JeremyHi
5fa01e7a96 feat: create regions persist true (#3590)
* feat: change open-region-step's status persist as true

* feat: avoid cloning

* fix: fix unit test
2024-03-27 06:26:58 +00:00
Yingwen
922b1a9b66 feat: Implement append mode for a region (#3558)
* feat: add dedup option to merge reader

* test: test merger

* feat: append mode option

* feat: implement append mode for regions

* feat: only allow put under append mode

* feat: always create builder

* test: test append mode

* style: fix clippy

* test: trigger compaction

* chore: fix compiler errors
2024-03-27 03:21:22 +00:00
shuiyisong
653697f1d5 chore: add back core dependency (#3588) 2024-03-26 19:53:22 -07:00
JohnsonLee
83643eb195 feat: Support printing postgresql's bytea data type in its "hex" and "escape" format (#3567)
* feat: support set variable statement of session

* feat: support printing postgresql's bytea data type in its "hex" and "escape" format in ugly way

* refactor: add 'SessionConfigValue' type and unify the name

* doc: add license header

* refactor: confine coupling with 'sql::ast::Value' in SessionConfigValue

* refactor: move all bytea wrapper into bytea.rs

* fix: remove unused import in context.rs and postgres.rs

* refactor: rename 'set_configuration_parameter' to 'set_session_config'

rename 'set_configuration_parameter' in statement_.rs to 'set_session_config'

* refactor: use mod to organize options via macro

* refactor: re-model the session config value with static type

* test: add integration test

* refactor: move the encode bytea by format type logic into encoder

refactor: use Arc<DashMap> instead of DashMap in QueryContext

refactor: use Arc<DashMap> instead of DashMap in QueryContext

    Avoid expensive clone

refactor: use unreachable!() instead of unimplemented!()

refactor: move the encode bytea by format type logic into encoder

test: add binary format integration test case

* test: add ut for byte related type

* doc: remove TODO of bytea_output

* refactor: simplify the implementation with simple struct instead of complex typing

* fix: typo of 'Available'

* fix compile

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
2024-03-27 01:54:41 +00:00
tison
d83279567b feat(auth): watch file user provider (#3566)
* feat(auth): watch file user provider

Signed-off-by: tison <wander4096@gmail.com>

* impl

Signed-off-by: tison <wander4096@gmail.com>

* use debouncer

Signed-off-by: tison <wander4096@gmail.com>

* add test

Signed-off-by: tison <wander4096@gmail.com>

* clippy

Signed-off-by: tison <wander4096@gmail.com>

* add path for FileWatch snafu

Signed-off-by: tison <wander4096@gmail.com>

* Apply comments

Signed-off-by: tison <wander4096@gmail.com>

* fix compile

Signed-off-by: tison <wander4096@gmail.com>

* drop notify-debouncer-full dep

Signed-off-by: tison <wander4096@gmail.com>

* empty to allow all

Signed-off-by: tison <wander4096@gmail.com>

* more test and log

Signed-off-by: tison <wander4096@gmail.com>

* relax the wait period

Signed-off-by: tison <wander4096@gmail.com>

* avoid sleep

Signed-off-by: tison <wander4096@gmail.com>

* Revert "avoid sleep"

This reverts commit d7a0be1dea.

* avoid sleep

Signed-off-by: tison <wander4096@gmail.com>

* cargo fmt

Signed-off-by: tison <wander4096@gmail.com>

* tidy dep

Signed-off-by: tison <wander4096@gmail.com>

* adjust

Signed-off-by: tison <wander4096@gmail.com>

* try be stable on CI

Signed-off-by: tison <wander4096@gmail.com>

* deugging

Signed-off-by: tison <wander4096@gmail.com>

* debugging

Signed-off-by: tison <wander4096@gmail.com>

* watch on the dir

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-03-27 01:19:18 +00:00
tison
150454b1fd chore: Delete CODE_OF_CONDUCT.md (#3578)
Leverage GitHub's feature to reuse `GreptimeTeam/.github` content.

This depends on https://github.com/GreptimeTeam/.github/pull/5.
2024-03-26 09:43:05 -07:00
JeremyHi
58c7858cd4 feat: update physical table schema on alter logical tables (#3585)
* feat: update physical table schema on alter

* feat: alter logical table in sql path

* feat: invalidate cache step1

* feat: invalidate cache step2

* feat: invalidate cache step3

* feat: invalidate cache step4

* fix: failed ut

* fix: standalone cache invalidator

* feat: log the count of already finished

* feat: re-invalidate cache

* chore: by comment

* chore: Update src/common/meta/src/ddl/create_logical_tables.rs

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
2024-03-26 14:29:53 +00:00
dimbtp
dd18d8c97b build(deps): remove some unused dependencies (#3582)
* build(deps): remove some unused dependencies

* add `arc-swap` dependency back
2024-03-26 12:48:28 +00:00
Lei, HUANG
175929426a feat: support time range in copy table (#3583)
* feat: support specifying time range in copy table statement

* chore: update sqlness results

* fix: sqlness
2024-03-26 11:24:28 +00:00
Ruihang Xia
8f9676aad2 fix: incorrect version info in (#3586)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2024-03-26 09:31:01 +00:00
Ruihang Xia
74565151e9 fix: update pk_cache in compat reader (#3576)
* fix: update pk_cache in compat reader

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update document

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add more sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* avoid mysterious bug

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2024-03-26 08:31:00 +00:00
Ruihang Xia
83c1b485ea chore: limit OpenDAL's feature gates (#3584)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2024-03-26 07:54:06 +00:00
JeremyHi
c2dd1136fe feat: batch alter logical tables (#3569)
* feat: add unit test for alter logical tables

* Update src/common/meta/src/ddl/alter_table.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* feat: add some comments

* chore: add debug_assert_eq

* chore: fix some nits

* chore: remove the method batch_get_table_routes

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2024-03-26 07:07:23 +00:00
tison
7c1c6e8b8c refactor: try upgrade regex-automata (#3575)
* refactor: try upgrade regex-automata

Signed-off-by: tison <wander4096@gmail.com>

* try fix

Signed-off-by: tison <wander4096@gmail.com>

* always check match with next_eoi_state

Signed-off-by: tison <wander4096@gmail.com>

* add a guard to prevent over moving the state

Signed-off-by: tison <wander4096@gmail.com>

* tidy

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-03-26 04:28:14 +00:00
Yingwen
62d8bbb10c ci: use single commit on the deployment branch (#3580) 2024-03-25 21:04:57 -07:00
Weny Xu
bf14d33962 feat: implement the drop database procedure (#3541)
* refactor: remove Sync trait of Procedure

* refactor: remove unnecessary async

* feat: implement the drop database procedure

* refactor: refactor DdlManager register_loaders

* feat: register the DropDatabaseProcedureLoader

* chore: fmt toml

* feat: support to submit DropDatabaseTask

* feat: support drop database stmt

* fix: empty the tables stream

* fix: ensure the factory always exists

* test: update sqlness results

* chore: correct comments

* test: update sqlness results

* test: update sqlness results

* chore: apply suggestions from CR

* chore: apply suggestions from CR
2024-03-25 06:12:47 +00:00
tison
0f1747b80d chore: retain original headers (#3572)
Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
2024-03-25 03:53:51 +00:00
Ruihang Xia
992c7ec71b feat: update physical table's schema on creating logical table (#3570)
* feat: update physical table's schema on creating logical table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove debug code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tweak ut const

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* invalid physical table cache

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2024-03-25 03:19:30 +00:00
x³u³
2ad0b24efa fix: set http response chartset to utf-8 when using table format (#3571) 2024-03-25 03:13:01 +00:00
Ruihang Xia
2b2fd80bf4 feat: return new added columns in region server's extension response (#3533)
* feat: adapt the new proto response

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update interfaces

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* write columns to extension

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use physical column's schema

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort logical columns by name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* format code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* return physical table's column

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/common/meta/src/datanode_manager.rs

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* implement sort column logic

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* proxy create table procedure to create logical table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add unit test for sort_columns

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
v0.8.0-nightly-20240325
2024-03-23 09:31:16 +00:00
x³u³
24886b9530 test: add a parameter type mismatch test case to sql integration test (#3568) 2024-03-22 17:43:20 +00:00
tison
8345f1753c chore: avoid confusing TryFrom (#3565)
Signed-off-by: tison <wander4096@gmail.com>
2024-03-22 11:16:36 +00:00
tison
3420a010e6 refactor: reduce one clone by carefully pass ready boundary (#3543)
* refactor: reduce one clone by carefully pass ready boundary

Signed-off-by: tison <wander4096@gmail.com>

* defensive handle None

Signed-off-by: tison <wander4096@gmail.com>

* tidy code a bit

Signed-off-by: tison <wander4096@gmail.com>

* except batch exist

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-03-22 04:46:17 +00:00
discord9
9f020aa414 fix(flow): Arrange get range with batch unaligned (#3552)
* fix: Arrange get range with batch unaligned

* chore: per review

* refactor: sort at apply_updates
2024-03-22 04:08:37 +00:00
tison
c9ac72e7f8 ci: use a PAT to list all writers (#3559)
Signed-off-by: tison <wander4096@gmail.com>
2024-03-21 20:25:01 -07:00
Lei, HUANG
86fb9d8ac7 refactor: remove redudant PromStoreProtocolHandler::write (#3553)
refactor: remove redudant PromStoreProtocolHandler::write API and rename PromStoreProtocolHandler::write_fast to write
2024-03-22 02:09:00 +00:00
Lei, HUANG
1f0fc40287 fix: performance degradation caused by config change (#3556) 2024-03-21 12:23:52 +00:00
tison
8b7a5aaa4a refactor: handle error for http format (#3548)
* refactor: handle error for http format

Signed-off-by: tison <wander4096@gmail.com>

* finish format handling

Signed-off-by: tison <wander4096@gmail.com>

* simplify auth error

Signed-off-by: tison <wander4096@gmail.com>

* fix

Signed-off-by: tison <wander4096@gmail.com>

* clippy format

Signed-off-by: tison <wander4096@gmail.com>

* no longer set greptime-db-format on influxdb error

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-03-21 07:29:11 +00:00
Weny Xu
856a4e1e4f refactor: refactor CacheInvalidator (#3550)
* refactor: refactor InvalidateCache Instruction

* refactor: refactor CacheInvalidator
2024-03-20 10:18:28 +00:00
Yingwen
39b69f1e3b refactor!: Renames the new memtable to PartitionTreeMemtable (#3547)
* refactor: rename mod merge_tree to partition_tree

* refactor: rename merge_tree

* refactor: change merge tree comment

* refactor: rename merge tree struct

* refactor: memtable options
2024-03-20 06:40:41 +00:00
tison
bbcdb28b7c chore: fix comment in fetch-dashboard-assets.sh (#3546) 2024-03-20 06:18:14 +00:00
YCCD
6377982501 feat: Able to pretty print sql query result in http output (#3539)
* feat: Able to pretty print sql query result in http output

* fix: add some tests

* fix: add some space, delete fn into_payload, and impl Display for TableResponse
2024-03-20 03:25:17 +00:00
Lei, HUANG
ddbcff68dd feat: support append-only mode in time-series memtable (#3540)
* feat: support append-only mode in time-series memtable

* fix: rename sort_and_dedup to sort
2024-03-19 20:37:54 +00:00
WU Jingdi
5b315c2d40 feat: support multi params in promql range function macro (#3464)
feat: support multi params in promql range function
2024-03-19 20:36:51 +00:00
Ruihang Xia
9816d2a08b fix: clone data instead of moving it - homemade future is dangerous (#3542)
* fix: clone data instead of moving it - homemade future is dangerous

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add comment

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2024-03-19 13:46:55 +00:00
Ning Sun
a99d6eb3f9 feat: update pgwire to 0.20 for improved performance (#3538) 2024-03-19 10:11:05 +00:00
discord9
2c115bc22a feat(flow): shared in-memory state for dataflow operator (#3508)
* feat: Arrangement shared state

* feat: arrange&tests

* docs: detailed&tests for get

* chore: license

* refactor: opt out ts expr&tests: internal ts

* docs: remove some TODOs

* feat: use smallvec size of 2

* refactor: per review

* chore: per review

* chore: per review

* chore: remove reduant clone

* feat: return max expire time&docs: more explain cur expire config
2024-03-19 10:03:05 +00:00
Yingwen
641592644d feat: support per table memtable options (#3524)
* feat: add memtable builder to region

* refactor: rename memtable_builder in worker to default_memtable_builder

* fix: return error instead of using default compaction options

Support deserializing memtable and compaction options from the option
map

* feat: optional memtable options

* feat: add MemtableBuilderProvider to create builders

* feat: change default memtable and skip deserializing dedup

* chore: update test and comment

* chore: test invalid type

* feat: metric engine use new memtable manually

* feat: expose more memtable configs

* feat: add memtable options to valid option list

* test: add test

* test: sqlness test

* chore: serde workspace

* chore: remove comments
2024-03-19 08:50:10 +00:00
Weny Xu
fa0f3555d4 refactor: introduce the DropTableExecutor (#3534)
* refactor: introduce the DropTableExecutor

* fix: register the dropping regions

* test: add tests for on_prepare

* chroe: add TODO comment
2024-03-19 08:29:12 +00:00
ZonaHe
3cad844acd feat: update dashboard to v0.4.9 (#3531)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2024-03-19 03:18:42 +00:00
JeremyHi
cf25cf984b chore: avoid unnecessary cloning (#3537) 2024-03-18 13:24:13 +00:00