Module utils

Expand description

some utils for helping with batching mode

Structs§

AddFilterRewriter: Find out the Filter Node corresponding to innermost(deepest) WHERE and add a new filter expr to it
ColumnMatcherRewriter: Optionally add to the final select columns like update_at if the sink table has such column (which doesn’t necessary need to have exact name just need to be a extra timestamp column) and __ts_placeholder(this column need to have exact this name and be a timestamp) with values like now() and 0
FindGroupByFinalName: Helper to find the innermost group by expr in schema, return None if no group by expr
IncrementalAggregateAnalysis: Analysis result for an incremental aggregate plan.
IncrementalAggregateMergeColumn: Describes how one aggregate output field should be merged with the corresponding existing field in the sink table.
OutputProjectionInfo 🔒

analyze_incremental_aggregate_plan
build_left_join_merge_expr 🔒
check_input_plan_shape 🔒
collect_output_projection_info 🔒
df_plan_to_sql
extract_incremental_aggregate 🔒
find_column_names 🔒: Recursively find all Expr::Column names inside an expression tree. Only recurses into wrappers that are merge-transparent. Non-transparent wrappers (e.g., ScalarFunction, Negative, Cast) are intentionally not recursed into since their merge semantics would be incorrect.
find_group_key_names 🔒
find_uncovered_output_fields 🔒
find_unsupported_group_key_projection_outputs 🔒
gen_plan_with_matching_schema 🔒: Generate a plan that matches the schema of the sink table from given sql by alias and adding auto columns
get_table_info_df_schema
has_aggregate 🔒
has_grouping_set 🔒
merge_op_for_aggregate_expr 🔒
peel_subquery_aliases 🔒
qualified_col 🔒
qualified_column 🔒
resolve_aggregate_output_field_name 🔒
rewrite_incremental_aggregate_with_sink_merge: Rewrites one incremental aggregate delta plan by left-joining it with the existing sink-table state and projecting merged aggregate outputs.
sql_to_df_plan: Convert sql to datafusion logical plan Also support TQL (but only Eval not Explain or Analyze)
unqualified_col 🔒