mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 16:02:32 +00:00
Jinwoo Hong
a3c1d32995
The incident monitor froze three healthy 15-minute production gates on 2026-09-05 because asia-east2 cells are judged against a bar calibrated for us-central1. A cell's /ready fetches the auth JWKS and runs SELECT 1 against Cloud SQL, both in us-central1, so from the US GitHub runner the asia-east2 round trip measures p50 0.88 s / max 2.7 s against 0.08-0.5 s for us-central1 cells. Give cell.<id>.latency_ms a per-region threshold (us-central1 2000, asia-east2 4000) carried on IncidentCellExpectation from the tfvars region. Director and auth latency rules keep the flat 2000 bar, and hard faults are still caught by the .health/.ready equal-1 checks and the probe's 8 s fetch timeout. Also name the signal and its observed/threshold in the live preflight failure message, keeping the source/code tokens other tooling matches on.