Skip to main content

spawn_compact_stream

mito2::read::stream

Function spawn_compact_stream 

Source 
pub(crate) fn spawn_compact_stream(
    stream: BoxedRecordBatchStream,
    capacity: usize,
    semaphore: Option<Arc<Semaphore>>,
) -> BoxedRecordBatchStream
Expand description

Polls stream on its own compaction runtime task, buffering up to capacity batches ahead of the consumer. While polling, the task holds a permit of semaphore if one is given.

The channel also closes if the task panics, so the returned stream checks how the task ended before reporting the end of the input. Dropping the returned stream aborts the task.