Jinwoo Hong 50b07c0e8f fix(relay-ops): keep one stalled director instance from tripping the reserve watch's 503 rule (#31484) ...

* fix(relay-ops): keep one stalled director instance from tripping the reserve watch's 503 rule

The reserve watch's director503 rule counted fleet-wide non-drain 503s. On 10-21 07:55Z one
director instance's database calls stalled for 3 minutes; its 503s alone carried the fleet over
the line and all four watched cells flipped back to db.

Over the fleet line the rule now also needs at least 2 director instances each above a fifth of
it for the same 3 minutes, counted from Cloud Run's request log by labels.instanceId less each
instance's scheduled 503s. Above max(10x background, 200) for 3 minutes it trips with no split.

* fix(relay-ops): size the instance share to the directors seen and count platform 503s

* fix(relay-ops): size the instance share per judged minute and refuse a missing count

2026-10-21 11:55:24 -04:00

..

contracts

infra(relay): raise asia-east2 cell pools to 16 and record the measured connection ceiling (#21163)

2026-09-17 02:00:17 -04:00

fixtures/terraform-root-partition

fix(relay): move director traffic in two steps so a deploy does not land on one instance (#31051)

2026-10-20 04:13:17 -04:00

scripts

fix(relay-ops): keep one stalled director instance from tripping the reserve watch's 503 rule (#31484)

2026-10-21 11:55:24 -04:00