mirror of
https://github.com/neondatabase/neon.git
synced 2026-06-03 05:20:38 +00:00
cdb08f0362
It is similar to XLogReader, but when either requested segment is missing locally or requested LSN is before basebackup_lsn NeonWALReader asynchronously fetches WAL from one of safekeepers. Patch includes walproposer switch to NeonWALReader, splitting wouldn't make much sense as it is hard to test otherwise. This finally removes risk of pg_wal explosion (as well as slow start time) when one safekeeper is lagging, at the same time allowing to recover it. In the future reader should also be used by logical walsender for similar reasons (currently we download the tail on compute start synchronously). The main test is test_lagging_sk. However, I also run it manually a lot varying MAX_SEND_SIZE on both sides (on safekeeper and on walproposer), testing various fragmentations (one side having small buffer, another, both), which brought up https://github.com/neondatabase/neon/issues/6055 closes https://github.com/neondatabase/neon/issues/1012
neon extension consists of several parts:
shared preload library neon.so
-
implements storage manager API and network communications with remote page server.
-
walproposer: implements broadcast protocol between postgres and WAL safekeepers.
-
control plane connector: Captures updates to roles/databases using ProcessUtility_hook and sends them to the control ProcessUtility_hook.
-
remote extension server: Request compute_ctl to download extension files.
-
file_cache: Local file cache is used to temporary store relations pages in local file system for better performance.
-
relsize_cache: Relation size cache for better neon performance.
SQL functions in neon--*.sql
Utility functions to expose neon specific information to user and metrics collection. This extension is created in all databases in the cluster by default.