Weny Xu
d86741d0ad
* feat(operator): retry failed writes with refreshed partitions Signed-off-by: WenyXu <wenymedia@gmail.com> * style(operator): simplify retry test type references Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(operator): clarify write request rebuilding names Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(operator): encapsulate write retry attempts Signed-off-by: WenyXu <wenymedia@gmail.com> * style(operator): group inserter configuration methods Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(operator): share routed peers and separate retry decisions Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(operator): avoid temporary write routing allocations Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>
2.8 KiB
operator — Agent & Contributor Guide
Navigation map for src/operator. Repo-wide invariants:
.agents/architecture-invariants.md.
operator connects frontend statements and writes to query, catalog, metasrv,
and datanodes. It owns statement dispatch, DDL task conversion, and region-level
insert/delete routing.
Module map
| Area | Path | Entry point |
|---|---|---|
| Statements | src/operator/src/statement.rs, src/operator/src/statement/ |
StatementExecutor and per-statement handlers |
| Inserts | src/operator/src/insert.rs, src/operator/src/bulk_insert.rs |
Insert, auto-create/alter, routing, dispatch |
| Deletes | src/operator/src/delete.rs |
Delete conversion and dispatch |
| Write dispatch | src/operator/src/write_dispatcher.rs |
Shared insert/delete/bulk dispatch, bounded retries, route refresh and write request rebuilding |
| Request conversion | src/operator/src/req_convert.rs, src/operator/src/req_convert/ |
Table/row/column requests to region requests |
| Region requests | src/operator/src/region_req_factory.rs |
Region-level DDL/DML request construction |
| Procedures | src/operator/src/procedure.rs |
Metasrv procedure administration |
| Flow | src/operator/src/flow.rs, src/operator/src/statement/ddl.rs |
Flow requests and internal mode selection |
Change coupling
- New statement kinds usually touch parser/AST handling, frontend permission and interception, executor dispatch, and sqlness cases.
- DDL task changes require matching
common-meta, metasrv, recovery, and compatibility updates. - Insert/delete conversion changes must cover both request formats, partition routing, and schema-on-write behavior.
- Flow creation changes must stay aligned with
src/flow/AGENTS.mdandcommon-metaflow metadata.
Testing
cargo nextest run -p operator
cargo sqlness bare -t <case>
Check standalone and distributed routing when changing region dispatch.
Write retries operate on prepared region requests in write_dispatcher; keep
admission and Flow side effects outside retries, and preserve successful regions.
Write admission uses fallible write_meter! calls in Inserter::do_request
and handle_bulk_insert, before flow side effects and datanode dispatch. Bulk
records carry row counts and the request channel with zero WCU. Validate both
the default meter-macros/noop build and active metering when changing this path.
Finite requests split internally use admit_write / admit_row_insert_batches
before dispatch. Their database-scoped QueryContext admission prevents a second
row debit while do_request still records WCU for the actual writes.
Async table-batcher outputs carry storage completions in OutputMeta; keep
them when combining submissions so callers can distinguish admission from success.