mirror of
https://github.com/neondatabase/neon.git
synced 2026-01-16 18:02:56 +00:00
9fb77d6cdd
In - https://github.com/neondatabase/neon/pull/10993#issuecomment-2690428336 I added infinite retries for buffered writer flush IOs, primarily to gracefully handle ENOSPC but more generally so that the buffered writer is not left in a state where reads from the surrounding InMemoryLayer cause panics. However, I didn't add cancellation sensitivity, which is concerning because then there is no way to detach a timeline/tenant that is encountering the write IO errors. That’s a legitimate scenario in the case of some edge case bug. See the #10993 description for details. This PR - first makes flush loop infallible, enabled by infinite retries - then adds sensitivity to `Timeline::cancel` to the flush loop, thereby making it fallible in one specific way again - finally fixes the InMemoryLayer/EphemeralFile/BufferedWriter amalgamate to remain read-available after flush loop is cancelled. The support for read-availability after cancellation is necessary so that reads from the InMemoryLayer that are already queued up behind the RwLock that wraps the BufferedWriter won't panic because of the `mutable=None` that we leave behind in case the flush loop gets cancelled. # Alternatives One might think that we can only ship the change for read-availability if flush encounters an error, without the infinite retrying and/or cancellation sensitivity complexity. The problem with that is that read-availability sounds good but is really quite useless, because we cannot ingest new WAL without a writable InMemoryLayer. Thus, very soon after we transition to read-only mode, reads from compute are going to wait anyway, but on `wait_lsn` instead of the RwLock, because ingest isn't progressing. Thus, having the infinite flush retries still makes more sense because they're just "slowness" to the user, whereas wait_lsn is hard errors.
41 lines
1.4 KiB
Rust
41 lines
1.4 KiB
Rust
use tokio::sync::mpsc;
|
|
|
|
/// A bi-directional channel.
|
|
pub struct Duplex<S, R> {
|
|
pub tx: mpsc::Sender<S>,
|
|
pub rx: mpsc::Receiver<R>,
|
|
}
|
|
|
|
/// Creates a bi-directional channel.
|
|
///
|
|
/// The channel will buffer up to the provided number of messages. Once the buffer is full,
|
|
/// attempts to send new messages will wait until a message is received from the channel.
|
|
/// The provided buffer capacity must be at least 1.
|
|
pub fn channel<A: Send, B: Send>(buffer: usize) -> (Duplex<A, B>, Duplex<B, A>) {
|
|
let (tx_a, rx_a) = mpsc::channel::<A>(buffer);
|
|
let (tx_b, rx_b) = mpsc::channel::<B>(buffer);
|
|
|
|
(Duplex { tx: tx_a, rx: rx_b }, Duplex { tx: tx_b, rx: rx_a })
|
|
}
|
|
|
|
impl<S: Send, R: Send> Duplex<S, R> {
|
|
/// Sends a value, waiting until there is capacity.
|
|
///
|
|
/// A successful send occurs when it is determined that the other end of the channel has not hung up already.
|
|
pub async fn send(&self, x: S) -> Result<(), mpsc::error::SendError<S>> {
|
|
self.tx.send(x).await
|
|
}
|
|
|
|
pub fn try_send(&self, x: S) -> Result<(), mpsc::error::TrySendError<S>> {
|
|
self.tx.try_send(x)
|
|
}
|
|
|
|
/// Receives the next value for this receiver.
|
|
///
|
|
/// This method returns `None` if the channel has been closed and there are
|
|
/// no remaining messages in the channel's buffer.
|
|
pub async fn recv(&mut self) -> Option<R> {
|
|
self.rx.recv().await
|
|
}
|
|
}
|