greptimedb/src/frontend/AGENTS.md

# frontend — Agent & Contributor Guide

Navigation aid for `src/frontend`. Keep it short and point to code. Paths are
relative to the repo root.

Repo-wide rules that apply here: [`.agents/architecture-invariants.md`](../../.agents/architecture-invariants.md).

## What this crate does

Frontend is the request entry point and orchestration layer. It accepts
multi-protocol requests (gRPC, HTTP, MySQL, PostgreSQL, InfluxDB, OTLP, Jaeger,
Prometheus, OpenTSDB), checks permissions, parses/plans SQL, and dispatches:
reads go to the query engine (`query` crate), writes go to the inserter/deleter
(`operator` crate).

Boundary with `servers`: the `servers` crate implements the wire protocols and
network I/O; `frontend` provides the business logic by implementing handler
traits (`SqlQueryHandler`, `GrpcQueryHandler`, `InfluxdbLineProtocolHandler`,
...). In standalone mode the `standalone` crate bridges the frontend to an
embedded datanode `RegionServer`; in distributed mode the frontend talks to
remote datanodes via `operator`/`client`.

## Module map

| Module | Path | Purpose |
| --- | --- | --- |
| `instance` | `src/frontend/src/instance.rs` | `Instance`: the core handler; implements `SqlQueryHandler`, `PrometheusHandler`, etc. |
| `instance/builder` | `src/frontend/src/instance/builder.rs` | `FrontendBuilder` assembles `Instance` from its dependencies |
| `instance/grpc` | `src/frontend/src/instance/grpc.rs` | `GrpcQueryHandler`: insert/delete/query/promql over gRPC |
| `instance/region_query` | `src/frontend/src/instance/region_query.rs` | Routes distributed region reads to datanodes |
| `instance/*` | `src/frontend/src/instance/` | Per-protocol handlers (`influxdb.rs`, `promql.rs`, `otlp/`, `jaeger.rs`, `logs.rs`, `prom_store.rs`, ...) |
| `frontend` | `src/frontend/src/frontend.rs` | `Frontend` lifecycle wrapper (`FrontendOptions`, start/shutdown) |
| `server` | `src/frontend/src/server.rs` | `Services`: builds and wires the protocol servers |
| `heartbeat` | `src/frontend/src/heartbeat.rs` | Heartbeat to metasrv; handles suspend / cache invalidation |
| `service_config` | `src/frontend/src/service_config/` | Per-protocol option structs |

Shared frontend cache counters live in `src/common/frontend/src/metrics.rs`,
including caches owned by pipeline and protocol handlers.

## Request lifecycles

- **SQL query** (`instance.rs`): `do_query_inner` handles parsing, interceptors,
  permission checks, timeout/cancellation, and delegates planning/execution to
  `StatementExecutor`. Distributed scans enter through `region_query.rs`.
- **Logical-table batch DDL** (`instance.rs`): dedicated SQL handler → bounded
  parse/interception → all-member permissions → operator CREATE preparation →
  one logical-table batch procedure. Keep pre-submission validation side-effect free.
- **Insert** (`instance/grpc.rs`): `handle_inserts` / `handle_row_inserts` →
  `check_permission` → `operator`'s `Inserter` (schema validation, optional
  auto-create, partition routing, meter admission) → local `RegionServer`
  (standalone) or RPC to datanodes (distributed). `FrontendBuilder` configures
  the operator write dispatcher with frontend retry options for distributed
  inserts, deletes, and bulk writes. Arrow bulk inserts pass the
  request channel to `Inserter` and check meter admission for each nonempty batch.
- Finite ingestion requests split internally admit their total rows per database
  before dispatch (`operator::insert::admit_write` / `admit_row_insert_batches`).
  The returned context covers chunks and derived writes while preserving WCU
  accounting and the original protocol channel.
- Internal gRPC listeners mark requests with `Channel::Internal` in middleware
  (`server.rs`), including requests handled by Enterprise Flight wrappers.

- **Flight bulk insert** (`instance/grpc.rs`): initializes on the first batch after
  the lazy schema handshake; checks permissions and reconciles missing columns
  through `Inserter` once per stream, then reuses the refreshed table.

- **OTLP trace lookup tables** (`instance/otlp/trace_ingest.rs`): `Instance` owns
  a shared cache of confirmed service/operation writes, which bypass batching.
  `otlp.trace_aux_cache_size` sets its estimated byte budget at startup; zero disables it.
  Async main writes retain completion results; a frontend task writes auxiliary
  rows only for confirmed chunks before populating the cache.
  Cold async trace requests reserve a shared auxiliary slot before main writes,
  bounded separately by the table batcher's `max_inflight_requests`; confirmed
  cache hits bypass this admission. The slot lasts through auxiliary completion.
  These tables are ingestion-managed; manual mutation requires restarting
  serving frontends.

- **OTLP metrics** (`instance/otlp.rs`): authorize all emitted destinations before
  schema preflight can reject points or expose destination schemas.

- **Logical-table batching** (`instance/logical_batcher.rs`): `Services` initializes
  one shared batcher for opted-in HTTP Prom and OTLP metric-engine
  writes. Downstream routers can initialize it when enabling replacement endpoints.
  `Instance::handle_otlp_metric_row_inserts` shares OTLP eligibility and dispatch
  between converters and falls back for incompatible tables.
  The schema adapter holds a weak instance reference to avoid an ownership cycle.

- **Table batching** (`instance/builder.rs`): protocol entry points opt in through
  `QueryContext`. The primary inserter prepares eligible ordinary-table writes
  for `servers::batcher::table::TablePendingRowsBatcher`. A separate execution-only
  inserter, with no batcher attached, sends the prepared bulk writes to datanodes,
  avoiding recursive batching. The batcher handles successful-write Flow notifications.

## Public surface

- `Instance` (`instance.rs`) — the business-logic container.
- `Frontend` (`frontend.rs`) — lifecycle wrapper around `Instance` + servers + heartbeat.
- Created from `cmd`: `src/cmd/src/frontend.rs` (distributed) and
  `src/cmd/src/standalone.rs` (standalone, with embedded datanode).

## When you change X, also touch Y

- **`servers` handler traits**: a new/changed protocol handler requires the
  matching `impl` here.
- **`operator` Inserter/Deleter or `query` QueryEngine API**: update the call
  sites in `instance.rs` / `instance/grpc.rs`.
- **`session::QueryContext`**: new context fields thread through most handlers.
- **`sql` statements**: new statement kinds need handling in `query_statement`.

## Testing

```bash
cargo nextest run -p frontend
```

## Gotchas

- Keep the frontend/servers split straight: wire format and network live in
  `servers`; permissions, planning, and routing live here.
- Standalone vs distributed diverge in datanode access (the `standalone` crate's
  local `RegionServer` adapter vs `NodeClients` RPC), MetaClient usage, and
  whether heartbeat matters. The builder defaults local invalidation to its
  layered cache registry; callers may supply a custom invalidator.

## Maintenance contract

Update this file when you add a protocol handler, change the query/insert
lifecycle, or change how `Instance` is constructed or wired to `servers`.