mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-11-18 00:34:11 +00:00
* fix(log-store): close raft-engine engines on shutdown raft-engine preallocates the active log file and only trims that tail when the Engine is dropped. GreptimeDB never dropped its engines before exiting, so every start after a clean shutdown truncated the tail as a broken batch and logged "Truncating log file due to broken batch". RaftEngineBackend::close and RaftEngineLogStore::stop now release the engine; standalone closes the metadata backend last, and the datanode stops the raft-engine WAL after the region server. Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * refactor: address review on raft-engine shutdown - Add ClosableKvBackend to common-meta; the standalone metadata backend creator returns one and Instance closes it explicitly instead of downcasting to RaftEngineBackend, so wrappers forward the close. - Record a failed metadata backend close as a shutdown error. - Sync and drop the engine on a blocking thread. - Only check tail trimming when raft-engine preallocated, and check reopening after close/stop on every platform. Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * fix: release raft-engine engines on every shutdown path - Stop the raft-engine WAL when the datanode build fails after opening it. - Purge and sync tasks hold the engine weakly, so dropping the last backend or log store reference (as the CLI tools do) closes the engine. - Fix needless borrows reported by clippy. Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * fix: drop unused ClosableKvBackend import Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * fix: cancel log store readers on stop; address review - RaftEngineLogStore tracks the tasks feeding read streams and cancels and joins them on stop, so a retained, unconsumed stream no longer keeps the engine open past shutdown. - Tests: stop with a retained read stream; datanode shutdown with a failing region engine and after a normal shutdown reopens the raft-engine WAL; standalone stop with failing steps reopens the metadata directory. - entries_to_batch no longer takes self; rename opened to get_open_engine, log_files_len to log_files_size and PREALLOCATED_LEN to PREALLOCATION_THRESHOLD; mod declarations before imports; rename the standalone shutdown closure to record_shutdown_error. Model: claude-opus-5-5 Tokens: 15.8k input, 148k output, 2.9M cache write, 103M cache read Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * fix: update Cargo.lock for tokio-util's rt feature The rt feature makes tokio-util depend on futures-util, which the lock file has to record for `cargo check --locked`. Model: claude-opus-5-5 Tokens: 15.9k input, 162k output, 3.0M cache write, 124M cache read Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * refactor: close the raft-engine WAL by ownership instead of a lock Per review, RaftEngineLogStore holds the engine as a plain Arc<Engine> again: - MitoEngine drops its Wal raw entry reader once its workers are stopped, so it no longer holds the log store after stop. - RaftEngineLogStore::close(self) stops the tasks and readers and closes the engine. Datanode::shutdown takes the log store after stopping the region server and closes it if it holds the last reference; otherwise it logs a warning and only stops it. - A failed build stops the region server it created before closing the log store, so the engines release it there too. Model: claude-opus-5-5 Effort: medium Tokens: 78, 19161, 107293, 4916570 del commit Cost: 2.23 USD del commit Session: 82696960-1984-4432-bcf1-33bfece8b526 Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * test: drop the index argument from LogStore::read in the raft-engine tests #9563 removed it from the trait. Model: claude-opus-5-5 Effort: medium Tokens: 16, 3715, 6600, 1706794 del commit Cost: 0.47 USD del commit Session: 82696960-1984-4432-bcf1-33bfece8b526 Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * fix: end a cancelled WAL read with an error; test a written region - A raft-engine read cancelled by stop now ends its stream with an IllegalState error instead of an end of stream, so a replay cannot take a truncated WAL as fully read. The reader flags the cancel instead of sending the error, since it is usually blocked on a full channel at that point. - Datanode test: a raft-engine datanode with a created region and writes shuts down cleanly and its WAL reopens, which fails if anything still holds the log store. - Move the doc comment of is_stopped back onto it. Model: claude-opus-5-5 Effort: medium Tokens: 42, 10316, 407027, 4108529 del commit Cost: 4.28 USD del commit Session: 82696960-1984-4432-bcf1-33bfece8b526 Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> * fix(log-store): drop the no-op sync before closing the raft-engine engine Per review: on raft-engine 0.4.2 `Engine::sync` writes an empty batch, and `Engine::write` returns before the fsync, so the sync in `close_engine` read as a final flush that never ran. The active log file is trimmed and synced by `LogFileWriter::close` when the last reference is dropped, so `close_engine` now only drops it on a blocking thread. Model: claude-opus-5-5 Effort: xhigh Tokens: 120, 33613, 182918, 7511982 del commit Cost: 3.64 USD del commit Session: 57ca77a8-f6fe-4fa2-a55a-dda31e5c541f Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com> --------- Signed-off-by: Silvio <62994417+AbstractBike@users.noreply.github.com>