rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2025-12-22 22:20:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,995 Commits 104 Branches 223 Tags
main
Commit Graph

84 Commits

Author SHA1 Message Date
Weny Xu
4d66bd96b8 feat: make distributed time constants and client timeouts configurable (#7433) 
Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-12-19 02:23:20 +00:00
fys
c9a7b1fd68 docs(config): clarify store_addrs format (#7279) 2025-11-21 22:26:52 +00:00
WaterWhisperer
de9ae6066f refactor: remove export_metrics and related configuration (#7236) 
Signed-off-by: WaterWhisperer <waterwhisperer24@qq.com>
 2025-11-17 02:32:22 +00:00
Ruihang Xia
f65dcd12cc feat: refine failure detector (#7005) 
* feat: refine failure detector

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert back default value

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert change of test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-09-24 01:43:22 +00:00
Weny Xu
dbb76483e8 chore: disable stats persistence by default (#6900) 
* chore: disable stats persistence by default

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix clippy

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-09-03 12:39:43 +00:00
Logic
cbcfdf9d65 feat: add optional schema for Postgres metadata tables (#6764) 
* feat(meta): add optional schema for Postgres metadata tables

- Add `schema` option to specify a custom schema for metadata tables
- Update `PgStore` and `PgElection` to support optional schema
- Modify SQL templates to use schema when provided
- Add tests for schema support in Postgres backend

Signed-off-by: Logic <zqr10159@dromara.org>

* refactor(meta): remove unused `create_schema_statement` and simplify `PgSqlTemplateFactory`

- Remove `create_schema_statement` from `PgSqlTemplateSet` struct
- Simplify `PgSqlTemplateFactory` by removing `new` method and merging it with `with_schema`
- Update related tests to reflect these changes

Signed-off-by: Logic <zqr10159@dromara.org>

* refactor(meta-srv): remove unused imports

- Remove unused import of BoxedError from common_error::ext- Remove unused import of TlsOption from servers::tls

Signed-off-by: Logic <zqr10159@dromara.org>

* build(meta): update Postgres version and add error handling imports

- Update Postgres version to 17 in docker-compose.yml
- Add BoxedError import for error handling in meta-srv

Signed-off-by: Logic <zqr10159@dromara.org>

* feat(postgres): add support for optional schema in PgElection and related components

Signed-off-by: Logic <zqr10159@dromara.org>

* feat(postgres): add support for optional schema in PgElection and related components

Signed-off-by: Logic <zqr10159@dromara.org>

* fix(develop): update Postgres schema commands to specify host

Signed-off-by: Logic <zqr10159@dromara.org>

* refactor(postgres): simplify plugin options handling and update SQL examples

Signed-off-by: Logic <zqr10159@dromara.org>

* refactor(postgres): simplify plugin options handling and update SQL examples

Signed-off-by: Logic <zqr10159@dromara.org>

* fix(postgres): update meta_election_lock_id description for optional schema support

Signed-off-by: Logic <zqr10159@dromara.org>

* fix(postgres): add health check and fallback wait for Postgres in CI setup

* fix(postgres): update Docker setup for Postgres and add support for Postgres 15

* fix(postgres): remove redundant Postgres setup step in CI configuration

* Update tests-integration/fixtures/postgres/init.sql

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* Update .github/workflows/develop.yml

* Update tests-integration/fixtures/docker-compose.yml

* Update src/common/meta/src/kv_backend/rds/postgres.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* Update src/common/meta/src/kv_backend/rds/postgres.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* Update src/common/meta/src/kv_backend/rds/postgres.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* Update src/common/meta/src/kv_backend/rds/postgres.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* fix: Refactor PostgreSQL backend to support optional schema in PgStore and related SQL templates

* feat: Update PostgreSQL configuration and add PG15 specific integration tests

* feat: Update PostgreSQL configuration and add PG15 specific integration tests

* refactor(postgres): update test schemas from 'greptime_schema' to 'test_schema'

* Update .github/workflows/develop.yml

* refactor: minor factor

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix unit test

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: Logic <zqr10159@dromara.org>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2025-08-28 09:24:14 +00:00
Weny Xu
f441598247 fix: correct config doc (#6836) 
fix: fix typo

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-08-28 06:26:25 +00:00
Yan Tingwang
32a3ef36f9 feat(metasrv): support tls for etcd client (#6818) 
* add TLS support for etcd client connections~

Signed-off-by: codephage2020 <tingwangyan2020@163.com>

* locate correct certs

Signed-off-by: codephage2020 <tingwangyan2020@163.com>

* Updated certs

Signed-off-by: codephage2020 <tingwangyan2020@163.com>

* Updated CI

Signed-off-by: codephage2020 <tingwangyan2020@163.com>

* Updated CI

Signed-off-by: codephage2020 <tingwangyan2020@163.com>

* Update docker-compose.yml

* tests for TLS client creation

Signed-off-by: codephage2020 <tingwangyan2020@163.com>

* modify tests

Signed-off-by: codephage2020 <tingwangyan2020@163.com>

---------

Signed-off-by: codephage2020 <tingwangyan2020@163.com>
 2025-08-27 07:41:05 +00:00
Weny Xu
566a647ec7 feat: add replay checkpoint to reduce overhead for remote WAL (#6816) 
* feat: introduce `TopicRegionValue`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: persist region replay checkpoint

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: introduce checkpoint

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: udpate config.md

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: minor refactor

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: send open region instructions with reply checkpoint

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: use usize

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add topic name pattern

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: enable wal prune by default

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-08-27 07:24:33 +00:00
Ning Sun
cdc168e753 feat: support for custom headers in otel exporter (#6773) 
* feat: support for custom headers in otel exporter

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: remove wrapping option

Signed-off-by: Ning Sun <sunning@greptime.com>

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
 2025-08-21 13:10:13 +00:00
Weny Xu
896d72191e feat: introduce PersistStatsHandler (#6777) 
* feat: add `Inserter` trait and impl

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: import items

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: introduce `PersistStatsHandler`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: disable persisting stats in sqlness

Signed-off-by: WenyXu <wenymedia@gmail.com>

* reset channel manager

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: avoid to collect

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: remove insert options

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: use `write_bytes` instead of `write_bytes_per_sec`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: compute write bytes delta

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* Update src/meta-srv/src/handler/persist_stats_handler.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2025-08-21 09:34:58 +00:00
Ning Sun
5a4036cc66 feat: update opentelemetry family (#6762) 
* feat: update opentelemetry family

Signed-off-by: Ning Sun <sunning@greptime.com>

* doc: update doc samples

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: toml format

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: update default otel enpoint

Signed-off-by: Ning Sun <sunning@greptime.com>

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
 2025-08-19 09:23:50 +00:00
Weny Xu
021ad09c21 refactor: simplify WAL pruning procedure and introduce region flush trigger (#6741) 
* chore: add logs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: update wal config for metasrv

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: introduce region flush trigger

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: debug assert

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: log level

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: simplify wal prune procedure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: upgrade rskafka

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: always flush inactive regions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: refactor flush trigger

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: remove unused code

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: typo

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: update unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add metrics

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: rename

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-08-14 14:15:30 +00:00
zyy17
777da35b0d refactor: unify the event recorder (#6689) 
* refactor: unify the event recorder

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add `table_name()` in `Event` trait

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: add `slow_query_options` in `Instance`

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add `EventHandlerOptions` and `options()` in `EventHandler` trait

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: add `aggregate_events_by_type()` and support log mode of slow query

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: polish the code

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* fix: clippy errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: support to set ttl by using extension of query context

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: refine the configs fields

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* fix: sqlness test errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: use `Duration` type instead of `String` for ttl fields

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: use pre-allocation for building RowInsertRequests

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: fix clippy errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: code review

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: fix integration errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: polish code for `group_events_by_type()` and `build_row_inserts_request()`, also add the unit tests

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: refine comments

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-08-12 18:26:12 +00:00
zyy17
cc35bab5e4 feat: record the migration events in metasrv (#6579) 
* feat: collect procedure event

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* feat: collect region migration events

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* test: add integration test

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: fix docs error

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: fix integration test error

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: change status code for errors

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add `event()` in Procedure

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: improve trait design

1. Add `user_metadata()` in `Procedure` trait;

2. Add `Eventable` trait;

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: polish the code

Signed-off-by: zyy17 <zyylsxm@gmail.com>

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-08-06 03:30:33 +00:00
Ruihang Xia
572e29b158 feat: support tls for pg backend (#6611) 
* load tls

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl tls

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* pass options

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement require mode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* default to prefer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update example config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* adjust example config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle client cert and key properly

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement verify_ca and verify_full

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update integration test for config api

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change config name and default mode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-08-04 00:41:08 +00:00
Yingwen
9527e0df2f feat: HTTP API to activate/deactive heap prof (activate by default) (#6593) 
* feat: add HTTP API to activate/deactivate heap profiling

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: add HTTP API to get profiling status

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: enable heap prof by default

Signed-off-by: evenyag <realevenyag@gmail.com>

* build: add "prof:true,prof_active:false" as default env to dockerfiles

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: activate heap profiling after log initialization

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: add memory options to control whether to activate profiling

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: update docs

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: fmt toml

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix config test

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: usage of new api

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: log profile after version

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: update how to docs

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: fix how to docs

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2025-08-01 03:24:56 +00:00
Weny Xu
5fcb97724f chore: correct typo in configuration (#6411) 
Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-06-27 07:55:13 +00:00
Weny Xu
24019334ee feat: implement automatic region failure detector registrations (#6370) 
* feat: implement automatic region failure detector registrations

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: remove unused error

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add more tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add `region_failure_detector_initialization_delay` option

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update config.md

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update config.md

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-06-24 06:12:12 +00:00
zyy17
a49edc6ca6 refactor: add otlp_export_protocol config to support export trace data through gRPC and HTTP protocol (#6357) 
* refactor: support http traces

* refactor: add `otlp_export_protocol` config to support export trace data through gRPC and HTTP protocol

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* Update src/common/telemetry/src/logging.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-06-23 02:56:12 +00:00
jeremyhi
12648f388a feat: refactor grpc options of metasrv (#6275) 
* feat: refactor grpc options of metasrv

* fix: unit test

* feat: config update
 2025-06-10 01:35:57 +00:00
dennis zhuang
0311db3089 fix: export metrics settings in sample config (#6170) 
* fix: export metrics settings in sample config

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: forgot update docs

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: test loading example configs

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2025-06-09 10:53:01 +00:00
zyy17
fdf32a8f46 refactor: respect data_home as root data home directory (#6050) 
* refactor: initialize logging dir by using data_home

* chore: remove tail '/' for dir name
 2025-06-09 02:31:21 +00:00
jeremyhi
3104d49434 chore: example of http config in metasrv (#6218) 
* chore: example of http config in metasrv

* docs: make config-docs
 2025-05-30 03:27:54 +00:00
zyy17
14b655ea57 refactor: add SlowQueryRecorder to record slow query in system table and refactor slow query options (#6008) 
* refactor: add common-slow-query crate

* refactor: refine the naming

* chore: fix clippy

* chore: fix typo

* chore: sperate SlowQueryOptions From Logging

* chore: fix clippy

* chore: fix ci

* chore: refine the code

* chore: update config example

* refactor: use drop() to end the slow query timer

* refactor: move common-slow-query to frontend crate

* chore: polish some code

* refactor: code review

* refactor: add promql_range/promql_step/promql_start/promql_end fields in slow_queries

* refactor: add build_slow_query_logger()

* refactor: turn on slow query on frontend by default
 2025-05-15 04:18:48 +00:00
Weny Xu
e30753fc31 feat: allow forced region failover for local WAL (#5972) 
* feat: allow forced region failover for local WAL

* chore: upgrade config.md

* chore: apply suggestions from CR
 2025-04-24 08:11:45 +00:00
Yuhan Wang
e817a65d75 feat: enable submitting wal prune procedure periodically (#5867) 
* feat: enable submitting wal prune procedure periodically

* chore: fix and add options

* test: add unit test

* test: fix unit test

* test: enable active_wal_pruning in test

* test: update default config

* chore: update config name

* refactor: use semaphore to control the number of prune process

* refactor: use split client for wal prune manager and topic creator

* chore: add configs

* chore: apply review comments

* fix: use tracker properly

* fix: use guard to track semaphore

* test: update unit tests

* chore: update config name

* chore: use prunable_entry_id

* refactor: semaphore to only limit the process of submitting

* chore: remove legacy sort

* chore: better configs

* fix: update config.md

* chore: respect fmt

* test: update unit tests

* chore: use interval_at

* fix: fix unit test

* test: fix unit test

* test: fix unit test

* chore: apply review comments

* docs: update config docs
 2025-04-18 16:02:33 +00:00
Weny Xu
ac0f9ab575 refactor: remove backoff config (#5808) 
* refactor: remove backoff config

* chore: update config.md

* fix: correct backoff config

* chore: change deadline to 120s
 2025-04-07 07:22:22 +00:00
Weny Xu
d3a60d8821 feat: add limit for the number of running procedures (#5793) 
* refactor: remove unused `messages`

* feat: introduce running procedure num limit

* feat: update config

* chore: apply suggestions from CR

* feat: impl `status_code` for `log-store` crate
 2025-03-31 06:14:21 +00:00
Ning Sun
dc76571166 feat: move default data path from /tmp to current directory (#5719) 2025-03-16 09:57:46 +00:00
Lei, HUANG
765d1277ee fix(metasrv): clean expired nodes in memory (#5592) 
* fix/frontend-node-state: Refactor NodeInfoKey and Context Handling in Meta Server

 • Removed unused cluster_id from NodeInfoKey struct.
 • Updated HeartbeatHandlerGroup to return Context alongside HeartbeatResponse.
 • Added current_node_info to Context for tracking node information.
 • Implemented on_node_disconnect in Context to handle node disconnection events, specifically for Frontend roles.
 • Adjusted register_pusher function to return PusherId directly.
 • Updated tests to accommodate changes in Context structure.

* fix/frontend-node-state: Refactor Heartbeat Handler Context Management

Refactored the HeartbeatHandlerGroup::handle method to use a mutable reference for Context instead of passing it by value. This change simplifies the
context management by eliminating the need to return the context with the response. Updated the Metasrv implementation to align with this new context
handling approach, improving code clarity and reducing unnecessary context cloning.

* revert: clean cluster info on disconnect

* fix/frontend-node-state: Add Frontend Expiry Listener and Update NodeInfoKey Conversion

 • Introduced FrontendExpiryListener to manage the expiration of frontend nodes, including its integration with leadership change notifications.
 • Modified NodeInfoKey conversion to use references, enhancing efficiency and consistency across the codebase.
 • Updated collect_cluster_info_handler and metasrv to incorporate the new listener and conversion changes.
 • Added frontend_expiry module to the project structure for better organization and maintainability.

* chore: add config for node expiry

* add some doc

* fix: clippy

* fix/frontend-node-state:
 ### Refactor Node Expiry Handling
 - **Configuration Update**: Removed `node_expiry_tick` from `metasrv.example.toml` and `MetasrvOptions` in `metasrv.rs`.
 - **Module Renaming**: Renamed `frontend_expiry.rs` to `node_expiry_listener.rs` and updated references in `lib.rs`.
 - **Code Refactoring**: Replaced `FrontendExpiryListener` with `NodeExpiryListener` in `node_expiry_listener.rs` and `metasrv.rs`, removing the tick     interval and adjusting logic to use a fixed 60-second interval for node expiry checks.

* fix/frontend-node-state:
 Improve logging in `node_expiry_listener.rs`

 - Enhanced warning message to include peer information when an unrecognized node info key is encountered in `node_expiry_listener.rs`.

* docs: update config docs

* fix/frontend-node-state:
 **Refactor Context Handling in Heartbeat Services**

 - Updated `HeartbeatHandlerGroup` in `handler.rs` to pass `Context` by value instead of by mutable reference, allowing for more flexible context
 management.
 - Modified `Metasrv` implementation in `heartbeat.rs` to clone `Context` when passing to `handle` method, ensuring thread safety and consistency in
 asynchronous operations.
 2025-02-27 06:16:36 +00:00
jeremyhi
fac22575aa refactor!: unify the option names across all components (#5457) 
* refactor: rename grpc options

* refactor: make the arg clearly

* chore: comments on server_addr

* chore: fix test

* chore: remove the store_addr alias

* refactor: cli option rpc_server_addr

* chore: keep store-addr alias

* chore: by comment
 2025-02-06 06:37:14 +00:00
Yohan Wal
3ed085459c feat(remote-wal): introduce TopicRegionManager (#5407) 
* feat: add manager to map region to topic

* chore: add a delete

* chore: rename keys

* chore: update config file

* fix: fix unit test

* chore: change prefix

* chore: clean up

* chore: follow review comments

* chore: follow review comments

* chore: follow review comments

* chore: follow review comments

* chore: follow review comments
 2025-01-22 06:06:27 +00:00
Yohan Wal
751fa4ede9 feat(config): make table name of pg backend configurable (#5244) 
* feat: configurable table name

* fix: election sql

* feat: configurable lock_id

* chore: update config file

* perf: useless allocation

* perf: useless allocation

* chore: remove unused type hint

* Apply suggestions from code review

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* chore: update config file

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2025-01-08 14:55:35 +00:00
Weny Xu
05f115e047 fix(pg_backend): correct set idle_in_transaction_session_timeout statement (#5304) 
* fix(metasrv): correct `backend` field configuration

* refactor!: added `#[serde(rename_all = "snake_case")]` macro to the `BackendImpl` enum

* fix(metasrv): correct `set idle_in_transaction_session_timeout` statement

* build: enable `pg_backend` by default
 2025-01-06 12:07:35 +00:00
dennis zhuang
2c4ac76754 feat: adjust WAL purge default configurations (#5107) 
* feat: adjust WAL purge default configurations

* fix: config

* feat: change raft engine file_size default to 128Mib
 2024-12-11 08:08:05 +00:00
Weny Xu
2b699e735c chore: correct example config file (#5105) 
* chore: correct example config file

* fix: fix unit test
 2024-12-06 03:14:08 +00:00
Ruihang Xia
9712295177 fix(config): update tracing section headers in example TOML files (#4898) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-10-30 08:31:31 +00:00
Kaifeng Zheng
a283e13da7 feat: set max log files to 720 by default, info log only (#4787) 
* feat: set max log files to 720 by default, info log only

* expose max_log_files in tomls

* include dir info when panicing, limit max_log_files of err_log to 30, and that of slow_queries to opt.max_log_files

* fix clippy

* update config.md

* update expected config str

* limit err_log max files size to `max_log_files` too, include err info when panicing, put `max_l_f` in right position

* fix typos

* chore: config

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-10-04 18:05:40 +00:00
zyy17
e39a9e6feb feat: add StatementStatistics for slow query logging implementation (#4719) 
* feat: log slow query

* feat: log slow query for sql

* refactor: add slow query logging options

* ci: fix errors

* feat: add StatementStatistics

* chore: revert modification of servers crate

* docs: update config docs

* fix: clippy errors
 2024-09-30 03:26:50 +00:00
zyy17
aa03d3b11c docs: use docs comment prefix and bump toml2docs version (#4711) 2024-09-11 07:49:23 +00:00
zyy17
8453df1392 refactor: make init_global_logging() clean and add log_format (#4657) 
refactor: refine the code logic of init_global_logging and add json output format
 2024-09-04 03:04:51 +00:00
dennis zhuang
841e66c810 fix: config api and export metrics default database (#4633) 2024-08-28 14:28:49 +00:00
Weny Xu
3297d5f657 feat: allow skipping topic creation (#4616) 
* feat: introduce `create_topics` opt

* feat: allow skipping topic creation

* chore: refine docs

* chore: apply suggestions from CR
 2024-08-26 08:34:27 +00:00
Lanqing Yang
ff04109ee6 docs: add example configs introduced by pg_kvbackend (#4573) 
chore: add example configs that introduced after pg_kvbackend
 2024-08-22 01:52:02 +00:00
Weny Xu
2e2eacf3b2 feat: add SASL and TLS config for Kafka client (#4536) 
* feat: add SASL and TLS config

* feat: add SASL/PLAIN and TLS config for Kafka client

* chore: use `ring`

* feat: support SASL SCRAM-SHA-256 and SCRAM-SHA-512

* fix: correct unit test

* test: add integration test

* chore: apply suggestions from CR

* refactor: introduce `KafkaConnectionConfig`

* chore: refine toml examples

* docs: add missing fields

* chore: refine examples

* feat: allow no server ca cert

* chore: refine examples

* chore: fix clippy

* feat: load system ca certs

* chore: fmt toml

* chore: unpin version

* Update src/common/wal/src/error.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-08-12 12:27:11 +00:00
Ning Sun
9532ffb954 fix: configuration example for selector (#4532) 
* fix: configuration example for selector

* docs: update config docs

* test: update unit tests for configuration in meta
 2024-08-09 09:51:05 +00:00
Weny Xu
a25d9f736f chore: set default otlp_endpoint (#4508) 
* chore: set default `otlp_endpoint`

* fix: fix ci
 2024-08-06 06:48:14 +00:00
Ruihang Xia
7daf24c47f feat: remove dedicated runtime for grpc, mysql and pg protocols (#4436) 
* feat: remove dedicated runtime for grpc, mysql and pg protocols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove other runtimes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* spawn compact task into compact_runtime

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refine naming

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/servers/tests/mysql/mysql_server_test.rs

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* turnoff fuzz test matrix fail fast option

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: update rt config for ci tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-07-30 06:17:58 +00:00
Weny Xu
89b86c87a2 chore: add docs for config file (#4432) 2024-07-25 08:11:10 +00:00