Files
greptimedb/src/frontend/AGENTS.md
T
Weny Xu 372f33540a feat(frontend): backport write retries to release/5e02efa9 (#9569)
* feat(client): support region request reuse and route refresh hints (#9540)

* feat(client): support shared region requests without deep cloning

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: propagate route refresh retry hints

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style: preserve local error variant imports in retry hints

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): distinguish staging rejection from write backpressure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): preserve legacy retry hints in instruction replies

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(client): inline retry hint compatibility cases

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(datanode): align batch-open retry hint assertion

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style: use imported types in region request tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 684c8ff291)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): centralize write partitioning and dispatch (#9541)

* refactor(operator): centralize write partitioning and dispatch

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): preserve write dispatch behavior during refactor

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): import write dispatch error paths

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): align dispatch imports and test utilities

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): use imported types in partition tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit a2d28e6060)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(operator): retry failed writes with refreshed partitions (#9542)

* feat(operator): retry failed writes with refreshed partitions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): simplify retry test type references

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): clarify write request rebuilding names

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): encapsulate write retry attempts

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): group inserter configuration methods

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): share routed peers and separate retry decisions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): avoid temporary write routing allocations

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit d86741d0ad)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(frontend): retry transient distributed write failures (#9543)

* refactor(operator): route shared requests through NodeManager

Remove the concrete NodeClients requester API and use the existing node manager for owned and shared requests. Preserve the original write hint while the legacy error adapter remains in place.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): preserve retry policy through error hints

Propagate client errors through External and let DDL read retry hints while preserving cleanup and terminal-child policies. Region transport failures retain the legacy Retryable classification, which now also participates in bounded write retries.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): restore sparse partition tags when rebuilding writes

Decode retained primary keys when refreshed partition expressions need different tags. Keep the partitioner and multi-region sparse write regression tests with the fix.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): stop detached write retries when callers cancel

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(frontend): configure retries across distributed write paths

Wire the shared dispatcher through frontend and logical batching, preserving partition columns and request headers. Keep standalone retries disabled and include configuration, batching, and storage regression coverage.

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 863eebdecb)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(mito2): format index error imports

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-10-14 14:02:28 +08:00

4.1 KiB

frontend — Agent & Contributor Guide

Navigation aid for src/frontend. Keep it short and point to code. Paths are relative to the repo root.

Repo-wide rules that apply here: .agents/architecture-invariants.md.

What this crate does

Frontend is the request entry point and orchestration layer. It accepts multi-protocol requests (gRPC, HTTP, MySQL, PostgreSQL, InfluxDB, OTLP, Jaeger, Prometheus, OpenTSDB), checks permissions, parses/plans SQL, and dispatches: reads go to the query engine (query crate), writes go to the inserter/deleter (operator crate).

Boundary with servers: the servers crate implements the wire protocols and network I/O; frontend provides the business logic by implementing handler traits (SqlQueryHandler, GrpcQueryHandler, InfluxdbLineProtocolHandler, ...). In standalone mode the standalone crate bridges the frontend to an embedded datanode RegionServer; in distributed mode the frontend talks to remote datanodes via operator/client.

Module map

Module Path Purpose
instance src/frontend/src/instance.rs Instance: the core handler; implements SqlQueryHandler, PrometheusHandler, etc.
instance/builder src/frontend/src/instance/builder.rs FrontendBuilder assembles Instance from its dependencies
instance/grpc src/frontend/src/instance/grpc.rs GrpcQueryHandler: insert/delete/query/promql over gRPC
instance/region_query src/frontend/src/instance/region_query.rs Routes distributed region reads to datanodes
instance/* src/frontend/src/instance/ Per-protocol handlers (influxdb.rs, promql.rs, otlp/, jaeger.rs, logs.rs, prom_store.rs, ...)
frontend src/frontend/src/frontend.rs Frontend lifecycle wrapper (FrontendOptions, start/shutdown)
server src/frontend/src/server.rs Services: builds and wires the protocol servers
heartbeat src/frontend/src/heartbeat.rs Heartbeat to metasrv; handles suspend / cache invalidation
service_config src/frontend/src/service_config/ Per-protocol option structs

Request lifecycles

  • SQL query (instance.rs): do_query_inner handles parsing, interceptors, permission checks, timeout/cancellation, and delegates planning/execution to StatementExecutor. Distributed scans enter through region_query.rs.
  • Insert (instance/grpc.rs): handle_inserts / handle_row_inserts → check_permission → operator's Inserter (schema validation, optional auto-create, partition routing) → local RegionServer (standalone) or RPC to datanodes (distributed). FrontendBuilder configures the operator write dispatcher with frontend retry options for distributed inserts, deletes, and bulk writes.

Public surface

  • Instance (instance.rs) — the business-logic container.
  • Frontend (frontend.rs) — lifecycle wrapper around Instance + servers + heartbeat.
  • Created from cmd: src/cmd/src/frontend.rs (distributed) and src/cmd/src/standalone.rs (standalone, with embedded datanode).

When you change X, also touch Y

  • servers handler traits: a new/changed protocol handler requires the matching impl here.
  • operator Inserter/Deleter or query QueryEngine API: update the call sites in instance.rs / instance/grpc.rs.
  • session::QueryContext: new context fields thread through most handlers.
  • sql statements: new statement kinds need handling in query_statement.

Testing

cargo nextest run -p frontend

Gotchas

  • Keep the frontend/servers split straight: wire format and network live in servers; permissions, planning, and routing live here.
  • Standalone vs distributed diverge in datanode access (the standalone crate's local RegionServer adapter vs NodeClients RPC), MetaClient usage, and whether heartbeat matters. The builder defaults local invalidation to its layered cache registry; callers may supply a custom invalidator.

Maintenance contract

Update this file when you add a protocol handler, change the query/insert lifecycle, or change how Instance is constructed or wired to servers.