starred/orca
1
0
Fork 0
mirror of https://github.com/stablyai/orca.git synced 2026-09-29 16:02:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(cloud): c27 crash loop; per-hour die counts

Browse Source
This commit is contained in:
Jinwoo-H
2026-09-04 07:43:00 -04:00
parent ad0bea58fb
commit ca7e1c290b
1 changed files with 3 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
cloud/docs/relay-reconnect-2026-09-findings.md
+3 -1
View File
@@ -390,7 +390,9 @@ SQL CPU 0.84 -> 0.78 in the preceding minutes, director concurrency 18–22 (idl
09:39 (c21, c24), 09:55 (c16, c8), 09:59 (c20), 10:05 (c8, c20), 10:19 (c16 stalled, no crash), then a 58-min
lull, 11:04 (c9; c28 died 13x in 4 min, autoheal recreate 11:09Z, its 3rd recreate today), 11:17 (c10, c28
again, c22, c23, c14 x9 looping; 23 dies in ~90 s; fleet 13.3k -> 10.8k), 11:31 (c14, c23, c25, c15, c24, c19), 11:34 (c20, c26, c29 x4, c14, c27 x3, c25; fleet 13.1k -> 10.3k).
Three cascades in 17 min. Every cell that has died today is on 5aedbca5; c7 (85bf6799, 5 h) has not.
Three cascades in 17 min. 11:38–11:42 c27 crash-looped 11x (autoheal recreate pending), c29 recreating.
Every cell that has died today is on 5aedbca5; c7 (85bf6799, 5 h) has not. Cell dies per hour today:
01Z 5, 02Z 7, 03Z 4, 04Z 7, 05Z 4, 06Z 9, 07Z 11, 08Z 30, 09Z 26, 10Z 2, 11Z 68+ (to 11:42).
Director concurrency pinned at 85 for 09:32–09:33; 503s 4,141 and 4,396 per minute. 09:39: c21, c24
(2,870 503s). Crashes per instance 08:10–09:40Z: c28 x14, c27 x5, c23 x5, c22 x4, c14 x4, c13/c20 x3,
then c26/c9/c24/c16/c8 x2. Mean gap between cascades since 08:40: ~12 min. Every 15-min gate attempt