Module batch_size 

Utilities for choosing an execution batch size from source statistics.

Constants

TARGET_BATCH_BYTES 🔒

Approximate decoded bytes to produce in one batch.

Functions

estimate_batch_size

Estimates a batch size from (num_rows, estimated_bytes) pairs.

estimate_row_width 🔒

Returns the ceiling of the estimated bytes per row.

largest_row_group_statistic 🔒

Returns (num_rows, uncompressed_bytes) of the largest row group in a Parquet file, which bounds the decoded size of a batch read from it.