* refactor(mito2): split compaction module into scheduler/reader submodules
Extract the compaction scheduler lifecycle (scheduler, status, phases,
execution, SST reservations, pending requests) and its tests out of
compaction.rs into compaction/scheduler.rs and compaction/scheduler_test.rs.
Split the remaining helpers by responsibility:
- estimate_compaction_bytes/refresh_picker_output move to scheduler.rs,
the only call site
- get_expired_ssts moves to picker.rs, shared by the TWCS and window
pickers
- CompactionSstReaderBuilder/time_range_to_predicate/ts_to_lit move to
the new compaction/reader.rs
The root compaction.rs keeps the shared output types and
find_dynamic_options, and re-exports the moved types so existing call
paths stay unchanged. Pure code motion, no behavior change.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): separate compaction scheduling from execution details
Turn compaction/scheduler.rs into a directory module to make the
scheduling flow easier to review:
- scheduler.rs keeps the pure scheduling core: the CompactionScheduler
state machine, scheduling entry points, termination chaining, DDL
coordination and region lifecycle events
- scheduler/planning.rs holds the execution-facing parts: background
planning dispatch, picker invocation, plan acceptance, remote/local
submission and memory estimation
- scheduler/state.rs holds the per-region lifecycle types:
CompactionStatus, ActiveCompaction, CompactionPhase, CompactingFiles,
LocalCompactionState, CompactionExecution and PendingCompaction
Child modules keep access to the scheduler's private methods, so the
split is pure code motion with minimal visibility changes (pub(super)
only where the parent module or tests reach into child items).
No behavior change.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): use absolute scheduler imports
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(mito2): document compaction scheduler modules
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(agents): add per-crate guides, architecture invariants, and generated-files list
Add agent/contributor navigation docs modeled on the AGENTS.md convention:
- Per-crate AGENTS.md for hot crates (mito2, metric-engine, flow, frontend,
meta-srv): module map, read/write paths, change-coupling points, test
commands, and gotchas.
- .agents/architecture-invariants.md: repo-wide rules that clippy and the
style guide do not cover (format compatibility, crate layering, async
runtimes, error handling, experimental gating, the DataFusion fork).
- .agents/generated-files.md: tool-generated artifacts that must not be
hand-edited (sqlness .result, config.md, dashboards, build.rs output, proto).
- Anchor the .gitignore CLAUDE.md/AGENTS.md rules to the repo root so per-crate
AGENTS.md files are tracked while root-level personal config stays ignored.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: update crate AGENTS.md and fix config.md path
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs(agents): fix DataFusion patch layout and SQL query lifecycle order
Address review feedback on #8346:
- architecture-invariants: the DataFusion sub-crates pin an exact crates.io
version in [workspace.dependencies] and are redirected to the fork rev in
[patch.crates-io]; the two sections hold different forms, not the same rev.
- frontend: the SQL query lifecycle runs the pre_parsing/post_parsing
interceptors around parsing, before the per-statement permission check.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Lei, HUANG
dennis zhuang