mirror of
https://github.com/neondatabase/neon.git
synced 2026-01-04 12:02:55 +00:00
28 lines
1.2 KiB
Markdown
28 lines
1.2 KiB
Markdown
# Storage broker
|
|
|
|
Storage broker targets two issues:
|
|
- Allowing safekeepers and pageservers learn which nodes also hold their
|
|
timelines, and timeline statuses there.
|
|
- Avoiding O(n^2) connections between storage nodes while doing so.
|
|
|
|
This is used
|
|
- By pageservers to determine the most advanced and alive safekeeper to pull WAL from.
|
|
- By safekeepers to synchronize on the timeline: advance
|
|
`remote_consistent_lsn`, `backup_lsn`, choose who offloads WAL to s3.
|
|
|
|
Technically, it is a simple stateless pub-sub message broker based on tonic
|
|
(grpc) making multiplexing easy. Since it is stateless, fault tolerance can be
|
|
provided by k8s; there is no built in replication support, though it is not hard
|
|
to add.
|
|
|
|
Currently, the only message is `SafekeeperTimelineInfo`. Each safekeeper, for
|
|
each active timeline, once in a while pushes timeline status to the broker.
|
|
Other nodes subscribe and receive this info, using it per above.
|
|
|
|
Broker serves /metrics on the same port as grpc service.
|
|
|
|
grpcurl can be used to check which values are currently being pushed:
|
|
```
|
|
grpcurl -proto broker/proto/broker.proto -d '{"all":{}}' -plaintext localhost:50051 storage_broker.BrokerService/SubscribeSafekeeperInfo
|
|
```
|