Struct DirtyTimeWindows 

pub struct DirtyTimeWindows {
    windows: BTreeMap<Timestamp, Option<Timestamp>>,
    max_filter_num_per_query: usize,
    time_window_merge_threshold: usize,
}

For keep recording of dirty time windows, which is time window that have new data inserted since last query.

Fields

windows: BTreeMap<Timestamp, Option<Timestamp>>

windows’s start -> end and non-overlapping end is exclusive(and optional)

max_filter_num_per_query: usize

Maximum number of filters allowed in a single query

time_window_merge_threshold: usize

Time window merge distance

Implementations

impl DirtyTimeWindows

pub fn new( max_filter_num_per_query: usize, time_window_merge_threshold: usize, ) -> Self

impl DirtyTimeWindows

pub const MERGE_DIST: i32 = 3

Time window merge distance

TODO(discord9): make those configurable

pub fn add_lower_bounds( &mut self, lower_bounds: impl Iterator<Item = Timestamp>, )

Add lower bounds to the dirty time windows. Upper bounds are ignored.

Arguments

• lower_bounds - An iterator of lower bounds to be added.

pub fn window_size(&self) -> Duration

pub fn add_window(&mut self, start: Timestamp, end: Option<Timestamp>)

pub fn add_windows(&mut self, time_ranges: Vec<(Timestamp, Timestamp)>)

pub fn add_dirty_windows(&mut self, dirty_windows: &DirtyTimeWindows)

Add all dirty markers from another dirty-window set.

fn add_or_merge_window(&mut self, start: Timestamp, end: Option<Timestamp>)

fn union_window_end( current_end: Option<Timestamp>, incoming_end: Option<Timestamp>, ) -> Option<Timestamp>

pub fn clean(&mut self)

Clean all dirty time windows, useful when can’t found time window expr

pub fn set_dirty(&mut self)

Set windows to be dirty, only useful for full aggr without time window to mark some new data is inserted

pub fn len(&self) -> usize

Number of dirty windows.

pub fn is_empty(&self) -> bool

pub fn effective_count(&self, window_size: &Duration) -> usize

Get the effective count of time windows, which is the number of time windows that can be used for query, compute from total time window range divided by window_size.

pub fn gen_filter_exprs( &mut self, col_name: &str, expire_lower_bound: Option<Timestamp>, window_size: Duration, window_cnt: usize, flow_id: u64, task_ctx: Option<&BatchingTask>, ) -> Result<Option<FilterExprInfo>, Error>

Generate all filter expressions consuming all time windows

there is two limits:

• shouldn’t return a too long time range(<=window_size * window_cnt), so that the query can be executed in a reasonable time
• shouldn’t return too many time range exprs, so that the query can be parsed properly instead of causing parser to overflow

fn align_time_window( &self, start: Timestamp, end: Option<Timestamp>, time_window_expr: &TimeWindowExpr, ) -> Result<(Timestamp, Option<Timestamp>), Error>

pub fn merge_dirty_time_windows( &mut self, window_size: Duration, expire_lower_bound: Option<Timestamp>, ) -> Result<(), Error>

Merge time windows that overlaps or get too close

TODO(discord9): not merge and prefer to send smaller time windows? how?

Trait Implementations

impl Clone for DirtyTimeWindows

fn clone(&self) -> DirtyTimeWindows

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DirtyTimeWindows

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DirtyTimeWindows

fn default() -> Self

Returns the “default value” for a type.