* feat(mito2): support independent TWCS trigger_file_num for active and inactive windows
Split the single TWCS trigger_file_num into per-window-state thresholds:
the active window keeps the existing trigger (default 4, legacy
compaction.twcs.trigger_file_num stays a compatible alias), while
inactive windows use a new trigger (default 2). Inactive windows
additionally fall back from balanced L0-only/L1-only candidates to a
progress-making unbalanced mixed candidate so historical windows can
converge; the active window retains the row/byte balance guards.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): bound inactive TWCS window convergence by rewrite budget
Inactive windows that cannot compact within one level previously either
stayed stuck (a threshold-qualified but unbalanced level returned no
candidate without trying any fallback) or fell back to a mixed merge
with no balance checks at all, which could rewrite a huge compacted file
to absorb tiny fresh files.
Inactive windows now converge progressively: threshold-qualified
balanced picks, sub-threshold balanced single-level picks, a mixed merge
whose total rewrite must fit in the output file budget, and finally an
L0-only merge without balance checks. Windows that qualify for none of
these are left uncompacted, bounding write amplification.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(mito2): cover TWCS window trigger options in alter_table_options sqlness case
Exercise SET/UNSET of compaction.twcs.active_window.trigger_file_num and
compaction.twcs.inactive_window.trigger_file_num end to end, including
that setting the canonical active key removes the legacy
compaction.twcs.trigger_file_num alias from the table options.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): derive TWCS active window from the max-sequence file
The active window was determined by the max event-time window among
level-0 files. Between an L0 compaction removing its inputs and the next
flush landing, level 0 is empty, so the active window transiently became
None and every window fell back to the inactive rules - triggering
full-window convergence merges during ongoing ingestion whose outputs
are then superseded by new data.
Flush and compaction outputs both inherit the max input sequence, so the
file with the highest sequence across all levels always tracks the most
recent write. Use its window as the active window, falling back to the
previous L0-based rule when no file carries a sequence (legacy files).
The new helper deliberately computes window keys with the
assign_to_windows convention (truncate to seconds, then align up),
because the result is compared against window keys produced there; the
older ceil-based helper is kept unchanged for the legacy fallback path.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat(mito2): add active-window L1 compaction safety trigger
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(compat): cover TWCS active window options
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): align TWCS window trigger validation
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): resolve database TWCS trigger aliases
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): prioritize newer compaction windows
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): repick serial compaction outputs
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat(mito2): configure inactive-window L1 trigger
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat(mito2): prioritize TWCS compaction candidates
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(meta): preserve TWCS trigger downgrade compatibility
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(sql): distinguish invalid database option values
Separate database option key and value validation so recognized keys report the invalid value and its constraint. Add parser coverage for invalid, valid, and unknown options.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(meta): explain TWCS legacy key compatibility
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): clarify active window trigger field
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(options): normalize TWCS trigger aliases
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): ignore ineligible files for active window
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): mark explicit TWCS options as overrides
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): normalize zero compaction output size
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(options): validate database TWCS trigger values
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* style(store-api): collapse TWCS alias conflict condition
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Lei, HUANG
dennis zhuang
WaterWhisperer
Yohan WalandWenyXu