Weny Xu
864cc117b3
fix: append noop entry when auto topic creation is disabled ( #6092 )
...
* feat: improve topic management and add stale records cleanup
* fix: fix unit tests
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2025-05-16 11:26:47 +00:00
Weny Xu
1f62c3b545
fix: table metadata collection ( #6102 )
...
fix: fix collect metadata
2025-05-14 12:19:54 +00:00
Weny Xu
29540b55ee
feat(meta): add pusher deregister signal to mailbox receiver ( #6072 )
2025-05-13 08:04:43 +00:00
Weny Xu
8dca448baf
feat: add datanode workloads support ( #6055 )
...
* feat: add datanode workload type support
* refactor: enhance datanode lease filtering with mode conditions
* chore: update config.md
* fix: fix clippy
* chore: apply suggestions from CR
* feat: add feature gate
* fix: fmt and clippy
* refactor: minor refactor
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* refactor: minior refactor
* test: fix unit test
2025-05-09 10:16:21 +00:00
LFC
4b5ab75312
refactor: remove some async in ServerHandlers ( #6057 )
...
* refactor: remove some async in ServerHandlers
* address PR comments
2025-05-07 03:57:16 +00:00
Weny Xu
df31f0b9ec
fix: improve region migration error handling and optimize leader downgrade with lease check ( #6026 )
...
* fix(meta): improve region migration error handling and lease management
* chore: refine comments
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* feat: consume opening_region_guard
2025-05-07 00:54:35 +00:00
Weny Xu
8726bf9f7a
feat(meta): enhance region lease handling with operating status ( #6027 )
...
* feat(meta): enhance region lease handling with operating status
* test: adjust flush metadata region test
2025-04-30 13:00:34 +00:00
Weny Xu
0bf07d7f91
feat: optimize region migration concurrency with fine-grained table lock ( #6023 )
...
* feat(procedure): add dynamic key locking mechanism
* feat: optimize region migration concurrency with fine-grained table lock
* chore: apply suggestions from CR
2025-04-30 06:45:03 +00:00
Weny Xu
8338aa14d3
feat: enhance maintenance mode API and handling ( #6022 )
...
* feat(meta): enhance maintenance mode API and handling
* chore: apply suggestions from CR
2025-04-29 11:39:28 +00:00
Weny Xu
a3ae2d7b52
feat: flush leader region before downgrading ( #5995 )
...
* feat: flush leader region before downgrading
* test: add unit tests
* chore: apply suggestions from CR
2025-04-29 03:28:00 +00:00
jeremyhi
133f404547
fix: sanitize_connection_string ( #6012 )
2025-04-28 13:56:26 +00:00
Weny Xu
13ac4d5048
fix: only consider the datanode that reports the failure ( #6004 )
...
* fix: only consider the datanode that reports the failure
* chore: fix clippy
2025-04-28 06:08:02 +00:00
Weny Xu
5ed0a095b6
feat: introduce RegionStatAwareSelector trait ( #5990 )
...
* feat: introduce `RegionStatAwareSelector`
* feat: exclude all failed datanodes
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2025-04-27 11:22:39 +00:00
shuiyisong
3c943be189
chore: update rust toolchain ( #5818 )
...
* chore: update nightly version
* chore: sort lint lines
* chore: minor fix
* chore: update nix
* chore: update toolchain to 2024-04-14
* chore: update toolchain to 2024-04-15
* chore: remove unnecessory test
* chore: do not assert oid in sqlness test
* chore: fix margin issue
* chore: fix cr issues
* chore: fix cr issues
---------
Co-authored-by: Ning Sun <sunning@greptime.com >
2025-04-27 09:02:36 +00:00
Weny Xu
e30753fc31
feat: allow forced region failover for local WAL ( #5972 )
...
* feat: allow forced region failover for local WAL
* chore: upgrade config.md
* chore: apply suggestions from CR
2025-04-24 08:11:45 +00:00
Weny Xu
ff3a46b1d0
feat: improve observability of region migration procedure ( #5967 )
...
* feat: improve observability of region migration procedure
* chore: apply suggestions from CR
* chore: observe non-zero value
2025-04-24 04:00:14 +00:00
Weny Xu
a533ac2555
feat: enhance selector with node exclusion support ( #5966 )
2025-04-24 02:27:27 +00:00
Weny Xu
f3d000f6ec
feat: track region failover attempts and adjust timeout ( #5952 )
2025-04-23 18:19:18 +00:00
discord9
9557b76224
fix: try prune one less ( #5965 )
...
* try prune one less
* test: also not add one
* ci: use longer fuzz time
* revert fuzz time&per review
* chore: no (
* docs: add explain to offset used in delete records
* test: fix test_procedure_execution
2025-04-23 16:57:54 +00:00
LFC
71db79c8d6
feat: node excluder ( #5964 )
...
* feat: node excluder
* fix ci
* fix ci
2025-04-23 10:48:46 +00:00
Weny Xu
0f77135ef9
feat: add exclude_peer_ids to SelectorOptions ( #5949 )
...
* feat: add `exclude_peer_ids` to `SelectorOptions`
* chore: apply suggestions from CR
* fix: clippy
2025-04-22 07:49:22 +00:00
Weny Xu
0a4594c9e2
fix: remove obsolete failover detectors after region leader change ( #5944 )
...
* fix: remove obsolete failover detectors after region leader change
* chore: apply suggestions from CR
* fix: fix unit tests
* fix: fix unit test
* fix: failover logic
2025-04-22 06:15:47 +00:00
jeremyhi
60e4607b64
chore: better buckets for heartbeat stat size histogram ( #5945 )
...
chore: better buckets for METRIC_META_HEARTBEAT_STAT_MEMORY_SIZE
2025-04-21 16:12:27 +00:00
Weny Xu
7a8e1bc3f9
feat: support building metasrv with selector from plugins ( #5942 )
...
* chore: expose selector
* feat: use f64
* chore: expose selector::common
* feat: build metasrv with selector from plugins
2025-04-21 10:59:24 +00:00
Yuhan Wang
e817a65d75
feat: enable submitting wal prune procedure periodically ( #5867 )
...
* feat: enable submitting wal prune procedure periodically
* chore: fix and add options
* test: add unit test
* test: fix unit test
* test: enable active_wal_pruning in test
* test: update default config
* chore: update config name
* refactor: use semaphore to control the number of prune process
* refactor: use split client for wal prune manager and topic creator
* chore: add configs
* chore: apply review comments
* fix: use tracker properly
* fix: use guard to track semaphore
* test: update unit tests
* chore: update config name
* chore: use prunable_entry_id
* refactor: semaphore to only limit the process of submitting
* chore: remove legacy sort
* chore: better configs
* fix: update config.md
* chore: respect fmt
* test: update unit tests
* chore: use interval_at
* fix: fix unit test
* test: fix unit test
* test: fix unit test
* chore: apply review comments
* docs: update config docs
2025-04-18 16:02:33 +00:00
Weny Xu
1e394af583
feat: prevent migrating a leader region to a peer that already has a region follower ( #5923 )
...
* feat: prevent migrating a leader region to a peer that already has a region follower
* chore: refine err msg
2025-04-18 11:13:01 +00:00
jeremyhi
4d38d8aa1e
chore: add heartbeat metrics ( #5929 )
2025-04-18 07:22:12 +00:00
Weny Xu
5a36fa5e18
fix: alway rejects write while downgrading region ( #5842 )
...
* fix: alway rejects write while downgrading region
* chore: apply suggestions from CR
2025-04-11 06:42:41 +00:00
Ruihang Xia
c26e165887
refactor: check and fix super import ( #5846 )
...
* refactor: check and fix super import
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add to makefile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change dir
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2025-04-08 11:48:52 +00:00
Yuhan Wang
6e6e335a81
feat(remote-wal): send flush request when pruning remote wal ( #5825 )
...
* feat: update minimum entry id in kvbackend
* fix: persist before delete
* chore: apply comments
* feat: add flush region in wal prune procedure
* fix: cherry-pick error
* chore: fmt
* chore: drop rx to avoid block by response
* chore: update comments
* chore: apply review comments
* test: fix unit test
* feat: add option not to flush region during wal prune
* test: fix unit test
* fix: delete at minimum replay entry id + 1
* fix: cas
* chore: add comments
* chore: apply review comments
* chore: apply review comments
* chore: fix error msg
* chore: apply review comments
* fix: idempotent cas
* refactor: use a one-way sender
* chore: better err msg
* chore: fix unit test
* chore: apply review comments
* chore: apply review comments
* chore: replace send oneway
2025-04-07 14:05:18 +00:00
Weny Xu
981d51785b
fix: throw errors instead of ignoring ( #5792 )
...
* fix: throw errors instead of ignoring
* fix: fix unit tests
* refactor: remove schema version check
* fix: fix clippy
* chore: remove unused error
* refactor: remove schema version check
* feat: handle mutliple results
* feat: introduce consistency guard
* fix: release consistency guard on datanode operation completion
* test: add tests
* chore: remove schema version
* refactor: rename
* test: add more tests
* chore: print all error
* tests: query table after alteration
* log ignored request
* refine fuzz test
* chore: fix clippy and log mailbox message
* chore: close prepared statement after execution
* chore: add comment
* chore: remove log
* chore: rename to `ConsistencyPoison`
* chore: remove unused error
* fix: fix unit tests
* chore: apply suggestions from CR
2025-04-07 13:51:00 +00:00
Weny Xu
917510ffd0
feat: introduce poison mechanism for procedure ( #5822 )
...
* feat: introduce poison for procedure
* tests: add unit tests
* refactor: minor refactor
* fix: unit tests
* chore: fix unit tests
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: update comments
* chore: introduce `ProcedureStatus::Poisoned`
* chore: upgrade greptime-proto to `2be0f`
* chore: apply suggestions from CR
2025-04-07 08:25:13 +00:00
Yuhan Wang
f73b61e767
feat(remote-wal): add remote wal prune procedure ( #5714 )
...
* feat: add remote wal prune procedure
* feat: add retry logic and remove rollback
* chore: simplify the logic
* fix: remove REMOTE_WAL_LOCK
* fix: use in-memory kv
* perf: O(n) judgement
* chore: add single write lock
* test: add unit test
* chore: remove unused function
* chore: update comments
* chore: apply comments
* chore: apply comments
2025-04-03 08:11:51 +00:00
Weny Xu
dbb79c9671
feat: introduce CollectLeaderRegionHandler ( #5811 )
...
* feat: introduce `CollectLeaderRegionHandler`
* feat: add to default handler group
* fix: correct unit test
* chore: rename
2025-04-02 04:47:00 +00:00
Weny Xu
68ce796771
chore: expose modules ( #5810 )
2025-04-01 05:33:20 +00:00
Weny Xu
d701c18150
feat: introduce CustomizedRegionLeaseRenewer ( #5762 )
...
* feat: add manifest_version to `GrantedRegion`
* chore: upgrade proto
* chore: apply review suggestions
* chore: apply suggestions from CR
* feat: introduce `CustomizedRegionLeaseRenewerRef`
* chore: upgrade to `103948`
2025-03-31 13:25:05 +00:00
Weny Xu
d3a60d8821
feat: add limit for the number of running procedures ( #5793 )
...
* refactor: remove unused `messages`
* feat: introduce running procedure num limit
* feat: update config
* chore: apply suggestions from CR
* feat: impl `status_code` for `log-store` crate
2025-03-31 06:14:21 +00:00
Weny Xu
41aee1f1b7
feat: implement sync_region for mito engine ( #5765 )
...
* chore: upgrade proto to `2d52b`
* feat: add `SyncRegion` to `WorkerRequest`
* feat: impl `sync_region` for `Engine` trait
* test: add tests
* chore: fmt code
* chore: upgrade proto
* chore: unify `RegionLeaderState` and `RegionFollowerState`
* chore: check immutable memtable
* chore: fix clippy
* chore: apply suggestions from CR
2025-03-31 03:53:47 +00:00
Weny Xu
7e1ba49d3d
refactor: remove useless region follower legacy code ( #5795 )
2025-03-28 08:10:30 +00:00
Weny Xu
c2ba7fb16c
refactor: remove useless region follower legacy code ( #5787 )
...
chore: remove region follower procedure
2025-03-27 11:50:29 +00:00
Weny Xu
9b7b012620
feat: impl show region ( #5782 )
...
* fix: fix region follower procedure
* feat: add table related info to region peers table and follower regions
* feat: impl show region
* chore: apply suggestions from CR
2025-03-27 10:41:44 +00:00
fys
2b2ea5bf72
chore: upgrade some dependencies ( #5777 )
...
* chore: upgrade some dependencies
* chore: upgrade some dependencies
* fix: cr
* fix: ci
* fix: test
* fix: cargo fmt
2025-03-27 02:48:44 +00:00
Yuhan Wang
a36901a653
chore: ut and some fix ( #5752 )
...
* chore: ut and some fix
* fix: remove NOWAIT
* refactor: use param for meta lease ttl
* chore: feature gate
* chore: add comments
* chore: apply comments
* fix: advice by claude 3.7 sonnet
* chore: apply comments
2025-03-24 09:05:06 +00:00
Yohan Wal
a0739a96e4
fix: wrap table name with `` ( #5748 )
...
* fix: wrap table name with quotes
* fix: minor fix
2025-03-20 09:38:54 +00:00
Yohan Wal
1dc4a196bf
feat: add mysql election logic ( #5694 )
...
* feat: add mysql election
* feat: add mysql election
* chore: fix deps
* chore: fix deps
* fix: duplicate container
* fix: duplicate setup for sqlness
* fix: call once
* fix: do not use NOWAIT for mysql 5.7
* chore: apply comments
* fix: no parallel sqlness for mysql
* chore: comments and minor revert
* chore: apply comments
* chore: apply comments
* chore: add to table name
* ci: use 2 metasrv to detect election bugs
* refactor: better election logic
* chore: apply comments
* chore: apply comments
* feat: version check before startup
2025-03-19 11:31:18 +00:00
Wenbin
83642dab87
feat: remove duplicated peer definition ( #5728 )
...
* remove duplicate peer
* fix
2025-03-18 11:30:25 +00:00
Ning Sun
dc76571166
feat: move default data path from /tmp to current directory ( #5719 )
2025-03-16 09:57:46 +00:00
AntiTopQuark
b8070adc3a
feat: enhancement information_schema.flows ( #5623 )
...
* feat: enhancement information_schema.flows
* feat: enhancement information_schema.flows
* u
* u
* u
* u
* u
* u
* u
* u
* u
* update
* update
* update
* delete unused code
* u
* u
* Update src/flow/src/adapter/worker.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* Update src/common/meta/src/key/flow/flow_state.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* Update src/common/meta/src/key/flow/flow_info.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* Update src/common/meta/src/key/flow/flow_state.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* Update src/common/meta/src/key/flow/flow_info.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* u
* u
* u
* u
* u
* u
* chore: fix sqlness
* chore: update proto
* fix: remove date time
* fix: update result of information_schema test
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
Co-authored-by: discord9 <discord9@163.com >
2025-03-11 15:08:10 +00:00
jeremyhi
1d87bd2d43
feat: alter region follower ( #5676 )
...
* feat: add region follower manager
* feat: add region procudure
* refactor: make add, remove follower procedure look nice
* feat: add region follower procedure
* chore: undo some chane, possibly made by AI
* feat: on prepare cheking
* feat: on update metadata
* feat: on broadcast
* chore: unit test
* feat: add remove follower operation
* feat: add or remove region follower procedure
* chore: ut
* chore: rename
* chore: by comment
* chore: by comment
---------
Co-authored-by: jeremy <jeremy@greptime.local >
2025-03-11 08:44:50 +00:00
Lin Yihai
2cbf51d0be
refactor!: Remove Value::DateTime and ValueRef::DateTime. ( #5616 )
...
* refactor: Remove Value::DateTime and ValueRef::DateTime
* fix: don't panic if arrow cast field.
* fix: map `ColumnDataType::Datetime` to `ConcreteDataType::timestamp_microsecond_datatype`
* fix: Map `ValueData::DatetimeValue` correctly.
* refactor: Replace `datetime` with `timestamp_micro_second`
2025-03-11 07:03:27 +00:00
