rust/neon
1
0
Fork 0
mirror of https://github.com/neondatabase/neon.git synced 2026-01-05 20:42:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
9bf7664049c63f7d8e087f11eebe45006beb75ba
neon/libs/vm_monitor
History
Em Sharnoff 9bf7664049 vm-monitor: Remove spammy log line (#6284) 
During a previous incident, we noticed that this particular line can be
repeatedly logged every 100ms if the memory usage continues is
persistently high enough to warrant upscaling.

Per the added comment: Ideally we'd still like to include this log line,
because it's useful information, but the simple way to include it
produces far too many log lines, and the more complex ways to
deduplicate the log lines while still including the information are
probably not worth the effort right now.
2024-01-08 21:12:39 -08:00
..
src
vm-monitor: Remove spammy log line (#6284)
2024-01-08 21:12:39 -08:00
Cargo.toml
vm-monitor: Remove dependency on workspace_hack (#5752)
2023-11-07 09:41:20 -08:00
README.md
vm-monitor: Switch from memory.high to polling memory.stat (#5524)
2023-10-17 15:30:40 -07:00

README.md

vm-monitor

The vm-monitor (or just monitor) is a core component of the autoscaling system, along with the autoscale-scheduler and the autoscaler-agents. The monitor has two primary roles: 1) notifying agents when immediate upscaling is necessary due to memory conditions and 2) managing Postgres' file cache and a cgroup to carry out upscaling and downscaling decisions.

More on scaling

We scale CPU and memory using NeonVM, our in-house QEMU tool for use with Kubernetes. To control thresholds for receiving memory usage notifications, we start Postgres in the neon-postgres cgroup and set its memory.{max,high}.

Structure

The vm-monitor is loosely comprised of a few systems. These are:

  • the server: this is just a simple axum server that accepts requests and upgrades them to websocket connections. The server only allows one connection at a time. This means that upon receiving a new connection, the server will terminate and old one if it exists.
  • the filecache: a struct that allows communication with the Postgres file cache. On startup, we connect to the filecache and hold on to the connection for the entire monitor lifetime.
  • the cgroup watcher: the CgroupWatcher polls the neon-postgres cgroup's memory usage and sends rolling aggregates to the runner.
  • the runner: the runner marries the filecache and cgroup watcher together, communicating with the agent throught the Dispatcher, and then calling filecache and cgroup watcher functions as needed to upscale and downscale