Files
orca/cloud/dev
Jinwoo Hong 9f10f415f5 feat(relay-infra): dynamic NAT ports and alerts for the 2026-09-04 stall signals (#18693)
* feat(relay-infra): dynamic NAT ports and alerts for the 2026-09-04 stall signals

Relay cells reach Cloud SQL's public IP through Cloud NAT. The static default
of 64 ports per VM filled on 2026-09-04 and every cell's proxy dial timed out
at once, which read as a fleet-wide SQL stall against a healthy database.
Switch both regional NATs to dynamic port allocation (64..4096 per VM).

Add the three alerts that would have paged inside the first ten minutes:
- Cloud SQL WAL-triggered checkpoint loop (log metric on
  "checkpoint starting: wal", > 3 in 5 min)
- Cloud SQL disk utilization > 70%
- Cloud NAT OUT_OF_RESOURCES packet drops on the relay gateways

No workflow applies google_monitoring_* or the NAT resources today; every
relay workflow is target-scoped to cells. Apply is a reviewed targeted plan
(see PR body).

* test(cloud): register the three new relay alert policies in the root partition fixture
2026-09-04 16:51:42 -04:00
..