greptimedb/src/meta-srv/AGENTS.md

meta-srv — Agent & Contributor Guide

Navigation aid for src/meta-srv. Keep it short and point to code. Paths are relative to the repo root.

Repo-wide rules that apply here: .agents/architecture-invariants.md.

What this crate does

Metasrv is the metadata and coordination service for distributed mode: it persists cluster/table/region metadata, runs leader election, drives the heartbeat loop that tracks cluster topology, and orchestrates distributed procedures (DDL, region migration, repartition). Data models, the KV backend abstraction, election, key encoding, and the DDL manager live in common-meta; meta-srv implements the server, state machine, and control logic on top.

Module map

Module	Path	Purpose
bootstrap	src/meta-srv/src/bootstrap.rs	Wires up KV backend, election, gRPC services, procedure manager
metasrv	src/meta-srv/src/metasrv.rs, src/meta-srv/src/metasrv/builder.rs	Metasrv struct, MetasrvOptions, MetasrvBuilder, startup
state	src/meta-srv/src/state.rs	Leader/Follower state machine and transitions
handler	src/meta-srv/src/handler/	Heartbeat handler chain (HeartbeatHandler): leader check, region lease, stats, mailbox
service	src/meta-srv/src/service/	gRPC services (heartbeat, procedure, cluster, store) and HTTP admin
procedure	src/meta-srv/src/procedure/	Distributed procedures: region_migration/, repartition.rs, wal_prune/
region	src/meta-srv/src/region/	Region supervisor, lease keeper, failover triggers
discovery	src/meta-srv/src/discovery/	Node discovery and lease-backed node info
pubsub	src/meta-srv/src/pubsub/	Heartbeat topic publish/subscribe support
gc	src/meta-srv/src/gc/	Metadata-driven garbage-collection procedures and scheduling
selector	src/meta-srv/src/selector/	Region placement strategies (round-robin / load-based / lease-based)
peer	src/meta-srv/src/peer.rs	Peer allocation through selectors
cluster	src/meta-srv/src/cluster.rs	MetaPeerClient: internal RPC with leader fallback
cache_invalidator	src/meta-srv/src/cache_invalidator.rs	Pushes cache-invalidation to frontends/datanodes
key	src/meta-srv/src/key/	Metasrv-side KV key encoding

Core flows

• Heartbeat: service/heartbeat.rs receives datanode/frontend heartbeats and runs the handler chain in handler/; responses carry mailbox messages (DDL results, cache invalidation). Node information is refreshed in memory on each heartbeat. Datanode statistics collection and flownode remapping share handler/utils.rs for per-node address persistence. Datanode address updates retain the statistics handler's per-node ordering. Address epochs are confirmed only after a successful save; cache invalidation remains best effort.
• DDL: service/procedure.rs (ddl) is leader-only and hands off to common-meta's DdlManager, executed via common-procedure.
• Region migration: procedure/region_migration/manager.rs plus per-step files; triggered by failover or an explicit command.
• Leader election: via common-meta's election; transitions in state.rs.

Public surface

Clients use the meta-client crate (MetaClient). On the server side the entry points are the gRPC services under src/meta-srv/src/service/ and the HTTP admin API under src/meta-srv/src/service/admin/.

When you change X, also touch Y

• Heartbeat interval / leases: timing constants live in common-meta (distributed_time_constants); changing them affects lease and failover timing in handler/region_lease_handler.rs and region/supervisor.rs.
• Procedures: state must stay persisted in the KV backend and execute() must be idempotent for crash recovery.
• Cache invalidation: new metadata types need matching invalidation in cache_invalidator.rs and the heartbeat publish handler.
• Affinity: common-meta/affinity.rs defines requirements, matching, batch resolution, and the query allocation-context provider. The selector factory optionally supplies a resolver shared by DDL allocation and the region supervisor; without one, allocation skips source lookup. DDL configurators can install a query provider before procedure loaders are registered. Repartition and failover resolve source requirements before selection. Each selector allows one unconstrained retry on candidate shortage when its context permits it. Failover always permits this retry and continues without affinity when source resolution fails or returns no requirements; DDL/repartition retain their own policy. Candidate discovery errors still propagate. Procedure state is unchanged.
• Selector stats fields: keep selector/ implementations in sync with the stats they consume.

Testing

cargo nextest run -p meta-srv
# mock election / KV backend:
cargo nextest run -p meta-srv --features mock

Helpers: src/meta-srv/src/mocks.rs, src/meta-srv/src/test_util.rs, src/meta-srv/src/procedure/test_util.rs.

Gotchas

• Most mutating operations are leader-only — guard them (see the leader check in handler/ and service/procedure.rs). Non-leaders return "not leader".
• In-memory state is not durable; anything needed after a leader change must be persisted to the KV backend. The leader uses a cached KV backend that resets on leader change.
• Region lease renewal timing and the supervisor's check interval must be coordinated, or a region can appear active on two datanodes.

Maintenance contract

Update this file when you add a procedure, change the heartbeat handler chain, change leader-only boundaries, or alter the metasrv ↔ common-meta split.