rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-24 08:50:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,427 Commits 131 Branches 253 Tags
f49cd0ca181d1267e1a1a7598529139f34be2137
Commit Graph

679 Commits

Author SHA1 Message Date
JeremyHi
f49cd0ca18 refactor: cache invalidator (#3611) 
* chore: remove some alias

* refactor: cache invalidator
 2024-03-29 07:33:51 +00:00
Ruihang Xia
77cc7216af feat: support 2+2 and /status/buildinfo (#3604) 
* feat: implement buildinfo endpoint

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor prom result struct

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add more integration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* format toml file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/servers/src/http/prometheus_resp.rs

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-03-29 06:31:39 +00:00
niebayes
d5ba2fcf9d test: add more integration test for kafka wal (#3190) 
* test: add integration tests for kafka wal

* chore: rebase main

* chore: unify naming convention for wal config

* chore: add register loaders switch

* chore: alter tables by adding a new column

* chore: move rand to dev-dependencies

* chore: update Cargo.lock
 2024-03-28 06:55:18 +00:00
dennis zhuang
e3b37ee2c9 fix: canonicalize catalog and schema names (#3600) 2024-03-28 06:40:15 +00:00
JeremyHi
92a8e863de chore: do not reply for broadcast msg (#3595) 2024-03-27 11:39:23 +00:00
JeremyHi
9428cb8e7c feat: remove support for logical tables in the create table procedure (#3592) 
* feat: Remove support for logical tables in the create table procedure

* chore: remove the redandent table ids alloc

* chore: minor fix
 2024-03-27 10:03:42 +00:00
Weny Xu
5addb7d75a test: add tests for drop databases (#3594) 
* refactor: minimize visibility of drop database steps

* feat: implement as_any

* refactor: move common functions to test_util

* test: add tests for drop databases

* fix: fix deteling physical table route unexpectedly
 2024-03-27 09:18:37 +00:00
Weny Xu
623c930736 refactor: refactor drop table executor (#3589) 
* refactor: refactor drop table executor

* chore: apply suggestions from CR
 2024-03-27 06:29:54 +00:00
JeremyHi
5fa01e7a96 feat: create regions persist true (#3590) 
* feat: change open-region-step's status persist as true

* feat: avoid cloning

* fix: fix unit test
 2024-03-27 06:26:58 +00:00
JeremyHi
58c7858cd4 feat: update physical table schema on alter logical tables (#3585) 
* feat: update physical table schema on alter

* feat: alter logical table in sql path

* feat: invalidate cache step1

* feat: invalidate cache step2

* feat: invalidate cache step3

* feat: invalidate cache step4

* fix: failed ut

* fix: standalone cache invalidator

* feat: log the count of already finished

* feat: re-invalidate cache

* chore: by comment

* chore: Update src/common/meta/src/ddl/create_logical_tables.rs

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-03-26 14:29:53 +00:00
dimbtp
dd18d8c97b build(deps): remove some unused dependencies (#3582) 
* build(deps): remove some unused dependencies

* add `arc-swap` dependency back
 2024-03-26 12:48:28 +00:00
JeremyHi
c2dd1136fe feat: batch alter logical tables (#3569) 
* feat: add unit test for alter logical tables

* Update src/common/meta/src/ddl/alter_table.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* feat: add some comments

* chore: add debug_assert_eq

* chore: fix some nits

* chore: remove the method batch_get_table_routes

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-03-26 07:07:23 +00:00
Weny Xu
bf14d33962 feat: implement the drop database procedure (#3541) 
* refactor: remove Sync trait of Procedure

* refactor: remove unnecessary async

* feat: implement the drop database procedure

* refactor: refactor DdlManager register_loaders

* feat: register the DropDatabaseProcedureLoader

* chore: fmt toml

* feat: support to submit DropDatabaseTask

* feat: support drop database stmt

* fix: empty the tables stream

* fix: ensure the factory always exists

* test: update sqlness results

* chore: correct comments

* test: update sqlness results

* test: update sqlness results

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-03-25 06:12:47 +00:00
tison
0f1747b80d chore: retain original headers (#3572) 
Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-03-25 03:53:51 +00:00
Ruihang Xia
992c7ec71b feat: update physical table's schema on creating logical table (#3570) 
* feat: update physical table's schema on creating logical table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove debug code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tweak ut const

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* invalid physical table cache

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-03-25 03:19:30 +00:00
Ruihang Xia
2b2fd80bf4 feat: return new added columns in region server's extension response (#3533) 
* feat: adapt the new proto response

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update interfaces

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* write columns to extension

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use physical column's schema

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort logical columns by name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* format code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* return physical table's column

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/common/meta/src/datanode_manager.rs

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* implement sort column logic

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* proxy create table procedure to create logical table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add unit test for sort_columns

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2024-03-23 09:31:16 +00:00
Weny Xu
856a4e1e4f refactor: refactor CacheInvalidator (#3550) 
* refactor: refactor InvalidateCache Instruction

* refactor: refactor CacheInvalidator
 2024-03-20 10:18:28 +00:00
WU Jingdi
5b315c2d40 feat: support multi params in promql range function macro (#3464) 
feat: support multi params in promql range function
 2024-03-19 20:36:51 +00:00
Weny Xu
fa0f3555d4 refactor: introduce the DropTableExecutor (#3534) 
* refactor: introduce the DropTableExecutor

* fix: register the dropping regions

* test: add tests for on_prepare

* chroe: add TODO comment
 2024-03-19 08:29:12 +00:00
shuiyisong
3acd5bfad0 chore: http header with metrics (#3536) 
* chore: bring write cost to output

* chore: add write cost to greptimev1result

* chore: add metrics to influxdb write resp header

* chore: add metrics to prom store

* chore: add metrics to otlp

* chore: add debug log

* fix: prom remote read with output

* fix: prom queries don't output metrics header

* chore: extract header value

* chore: refactor code

* chore: fix cr issue
 2024-03-18 11:21:19 +00:00
Weny Xu
343525dab8 refactor: remove removed-prefixed keys (#3535) 2024-03-18 11:07:30 +00:00
tison
0afac58e4d feat(metasrv): implement maintenance (#3527) 
* feat(metasrv): implement maintenance

Signed-off-by: tison <wander4096@gmail.com>

* fixup and test

Signed-off-by: tison <wander4096@gmail.com>

* Add coauthors

Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: xifyang <595482900@qq.com>

* tidy code

Signed-off-by: tison <wander4096@gmail.com>

* Apply suggestions from code review

Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>

* always read kv_backend maintenance state

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: xifyang <595482900@qq.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
 2024-03-18 09:41:14 +00:00
Weny Xu
a29e7ebb7d feat: acquire all locks in procedure (#3514) 
* feat: acquire catalog and schema lock in region failover

* chore: remove unused code

* feat!: acquire catalog and schema lock in region migration

* feat: acquire catalog and schema lock in create table
 2024-03-14 11:41:23 +00:00
LFC
e3c82568e5 fix: correctly generate sequences when the value is pre-existed (#3502) 2024-03-14 06:55:12 +00:00
Zhenchi
904484b525 fix(common-time): allow building nanos timestamp from parts split from i64::MIN (#3493) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-03-13 02:46:00 +00:00
Yingwen
9aa8f756ab fix: allow passing extra table options (#3484) 
* fix: do not check options in parser

* test: fix tests

* test: fix sqlness

* test: add sqlness test

* chore: log options

* chore: must specify compaction type

* feat: validate option key

* feat: add option key validation back
 2024-03-12 07:03:52 +00:00
tison
1255c1fc9e feat: to_timezone function (#3470) 
* feat: to_timezone function

Signed-off-by: tison <wander4096@gmail.com>

* impl Function for ToTimezoneFunction

Signed-off-by: tison <wander4096@gmail.com>

* add test

Signed-off-by: tison <wander4096@gmail.com>

* Add original authors

Co-authored-by: parkma99 <park-ma@hotmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

* address comments

Signed-off-by: tison <wander4096@gmail.com>

* add issue link

Signed-off-by: tison <wander4096@gmail.com>

* code refactor

Signed-off-by: tison <wander4096@gmail.com>

* further tidy

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: parkma99 <park-ma@hotmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-03-12 01:46:19 +00:00
JeremyHi
da098f5568 fix: make max-txn-ops limit valid (#3481) 2024-03-11 09:27:51 +00:00
shuiyisong
aa953dcc34 fix: impl RecordBatchStream method explicitly (#3482) 
fix: impl RecordBatchStream method explicitly
 2024-03-11 09:07:10 +00:00
Ruihang Xia
d8939eb891 feat: clamp function (#3465) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add unit tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* a little type exercise

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-03-11 03:26:10 +00:00
shuiyisong
0bb949787c refactor: introduce new Output with OutputMeta (#3466) 
* refactor: introduce new output struct

* chore: add helper function

* chore: update comment

* chore: update commit

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: rename according to cr

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-03-11 02:24:09 +00:00
gcmutator
21ff3620be chore: remove repetitive words (#3469) 
remove repetitive words

Signed-off-by: gcmutator <329964069@qq.com>
 2024-03-09 04:18:47 +00:00
JeremyHi
352bd7b6fd feat: max-txn-ops option (#3458) 
* feat: max-txn-ops limit

* chore: by comment
 2024-03-08 02:34:40 +00:00
WU Jingdi
53f2a5846c feat: support tracing rule sampler (#3405) 
* feat: support tracing rule sampler

* chore: simplify code
 2024-03-05 15:40:02 +08:00
JeremyHi
2d975e4f22 feat: tableref cache (#3420) 
* feat: tableref cache

* chore: minor refactor

* chore: avoid to string

* chore: change log level

* feat: add metrics for prometheus remote write decode
 2024-03-02 07:37:31 +00:00
JohnsonLee
a50025269f feat: Support automatic DNS lookup for kafka bootstrap servers (#3379) 
* feat: Support automatic DNS lookup for kafka bootstrap servers

* Revert "feat: Support automatic DNS lookup for kafka bootstrap servers"

This reverts commit 5baed7b01d.

* feat: Support automatic DNS lookup for Kafka broker

* fix: resolve broker endpoint in client manager

* fix: apply clippy lints

* refactor: slimplify the code with clippy hint

* refactor: move resolve_broker_endpoint to common/wal/src/lib.rs

* test: add mock test for resolver_broker_endpoint

* refactor: accept niebayes's advice

* refactor: rename EndpointIpNotFound to EndpointIPV4NotFound

* refactor: remove mock test and simplify the implementation

* docs: add comments about test_vallid_host_ipv6

* Apply suggestions from code review

Co-authored-by: niebayes <niebayes@gmail.com>

* move more common code

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
Co-authored-by: niebayes <niebayes@gmail.com>
 2024-02-29 07:29:20 +00:00
tison
dc205a2c5d feat: enable ArrowFlight compression (#3403) 
* feat: enable ArrowFlight compression

Signed-off-by: tison <wander4096@gmail.com>

* turn on features

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-02-28 08:55:44 +00:00
Weny Xu
a8cbec824c refactor: refactor TableRouteManager (#3392) 
* feat: introduce TableRouteStorage

* refactor: remove get & batch_get in TableRouteManager

* refactor: move txn related fn to TableRouteStorage

* chore: apply suggestions from CR

* chore(codecov): ingore tests-integration dir
 2024-02-28 06:18:09 +00:00
dennis zhuang
4b36c285f1 feat: flush or compact table and region functions (#3363) 
* feat: adds Requester to process table flush and compaction request

* feat: admin_fn macros for administration functions

* test: add query result

* feat: impl flush_region, flush_table, compact_region, and flush_region functions

* docs: add Arguments to admin_fn macro

* chore: apply suggestion

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: apply suggestion

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: group_requests_by_peer and adds log

* Update src/common/macro/src/admin_fn.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* feat: adds todo for spawan thread

* feat: rebase with main

---------

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-02-27 08:57:38 +00:00
Weny Xu
7453d9779d fix: throw errors instead of panic (#3391) 
* fix: throw errors instead of panic

* chore: apply suggestions from CR
 2024-02-27 03:46:12 +00:00
dimbtp
b1e290f959 fix: range fix in modulo function tests (#3389) 
fix: range fix for modulo tests
 2024-02-26 15:50:23 +00:00
Kould
578dd8f87a feat: add isnull function (#3360) 
* code fmt

* feat: add isnull function

* feat: add isnull function
 2024-02-22 12:41:25 +00:00
Weny Xu
1dc4fec662 refactor: allocate table ids in the procedure (#3293) 
* refactor: refactor the create logical tables

* test(create_logical_tables): add tests for on_prepare

* test(create_logical_tables): add tests for on_create_metadata

* refactor: rename to create_logical_tables_metadata

* chore: fmt toml

* chore: apply suggestions from CR
 2024-02-22 10:53:28 +00:00
dennis zhuang
564fe3beca feat: impl migrate_region and procedure_state SQL function (#3325) 
* fix: logical region can't find region routes

* feat: fetch partitions info in batch

* refactor: rename batch functions

* refactor: rename DdlTaskExecutor to ProcedureExecutor

* feat: impl migrate_region and query_procedure_state for ProcedureExecutor

* feat: adds SQL function procedure_state and finish migrate_region impl

* fix: constant vector

* feat: unit tests for migrate_region and procedure_state

* test: test region migration by SQL

* fix: compile error after rebeasing

* fix: clippy warnings

* feat: ensure procedure_state and migrate_region can be only called under greptime catalog

* fix: license header
 2024-02-22 02:37:11 +00:00
SteveLauC
e9a2b0a9ee chore: use workspace-wide lints (#3352) 
* chore: use workspace-wide lints

* respond to review
 2024-02-22 01:01:10 +00:00
tison
4c07606da6 refactor: put together HTTP headers (#3337) 
* refactor: put together HTTP headers

Signed-off-by: tison <wander4096@gmail.com>

* do refactor

Signed-off-by: tison <wander4096@gmail.com>

* drop dirty commit

Signed-off-by: tison <wander4096@gmail.com>

* reduce changeset

Signed-off-by: tison <wander4096@gmail.com>

* fixup compilations

Signed-off-by: tison <wander4096@gmail.com>

* tidy files

Signed-off-by: tison <wander4096@gmail.com>

* drop common-api

Signed-off-by: tison <wander4096@gmail.com>

* fmt

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-02-21 09:51:10 +00:00
Weny Xu
41656c8635 refactor: allocate table id in the procedure (#3271) 
* refactor: replace TableMetadataManager with TableNameManager

* refactor: allocate table id in the procedure

* refactor: refactor client logical of handling retries

* feat(test_util): add TestCreateTableExprBuilder

* feat(test_util): add MockDatanodeManager

* feat(test_util): add new_ddl_context

* feat(test_util): add build_raw_table_info_from_expr

* feat(test_util): add MockDatanodeManager::new

* feat(procedure): add downcast_output_ref to Status

* test(create_table): add tests for CreateTableProcedure on_prepare

* refactor(ddl): rename handle_operate_region_error to add_peer_context_if_need

* test(create_table): add tests for CreateTableProcedure on_datanode_create_regions

* test(create_table): add tests for CreateTableProcedure on_create_metadata

* refactor(meta): use CreateTableExprBuilder

* feat(create_table): ensure number of partitions is greater than 0

* refactor: rename to add_peer_context_if_needed

* feat: add context for panic

* refactor: simplify the should_retry

* refactor: use Option<&T> instead of &Option<T>

* refactor: move downcast_output_ref under cfg(test)

* chore: fmt toml
 2024-02-21 04:38:46 +00:00
Ruihang Xia
b1f54d8a03 fix: disable ansi contorl char when stdout is redirected (#3332) 
* fix: disable ansi contorl char when stdout is redirected

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* don't touch file logging layer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable ansi for two file layers

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Co-authored-by: LFC <bayinamine@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
 2024-02-20 06:42:56 +00:00
shuiyisong
bf5e1905cd refactor: bring metrics to http output (#3247) 
* refactor: bring metrics to http output

* chore: remove unwrap

* chore: make walk plan accumulate

* chore: change field name and comment

* chore: add metrics to http resp header

* chore: move PrometheusJsonResponse to a separate file and impl IntoResponse

* chore: put metrics in prometheus resp header too
 2024-02-20 03:25:18 +00:00
JeremyHi
6668d6b042 fix: split write metadata request (#3311) 
* feat: add txn_helper

* fix: split the create metadata requests to avoid exceeding the txn limit

* fix: add license header

* chore: some improve
 2024-02-19 07:33:09 +00:00