Files
orca/src/shared/terminal-webgl-diagnostics.ts
T
NeilandOrca ab1c37889a fix(crash-reporting): stop fit-retry bursts from erasing the pre-crash trail (#10729)
* fix(crash-reporting): stop fit-retry bursts from erasing the pre-crash trail

Windows renderer OOM F0BKR84AHEH (0xE0000008) arrived with a 30-entry
breadcrumb ring in which two `terminal_safe_fit_retry_exhausted` bursts
consumed 26-90% of the slots. Every hidden pane is `display:none` -> 0x0 ->
unmeasurable, so one post-reload reattach wave exhausts the retry budget once
per mounted pane inside ~60ms.

The bursts were also uninterpretable: `pane.id` restarts at 1 per PaneManager
and there is one manager per tab, so 34 identical `paneId: 1` crumbs cannot
distinguish one pane looping from 34 panes firing once. Coalesce the crumb by
name and carry the live-pane census on the payload instead, so the count
survives without costing 34 ring slots.

Same treatment for WebGL diagnostics, which were worse off: context-loss and
atlas-reset crumbs only reached a DevTools-only ring (`window.n()`), so a
renderer that dies takes them with it. That bundle had three GPU-process deaths
in the 65s before the renderer OOM and zero WebGL evidence - absence of
instrumentation, not absence of the event. Mirror them into the crash report,
coalesced per kind so a routine atlas reset cannot mask a context loss.

Evidence-only: no behavior, rendering, or lifecycle path changes.

Co-authored-by: Orca <help@stably.ai>

* perf(pane-manager): count panes without materializing public views

The census runs on the crash path; getPanes() allocates a full ManagedPane
projection per pane just to read .length.

Co-authored-by: Orca <help@stably.ai>

* test(crash-reporting): pin the fit-retry burst against the 30-entry ring

Reproduces the F0BKR84AHEH ring loss directly: 10 pre-crash crumbs plus a
34-crumb per-pane burst. Uncoalesced, the burst takes all 30 slots and zero
pre-crash crumbs survive; coalesced, it takes one slot, all 10 survive, and
the pane count rides on the payload instead of on the crumb multiplicity.

Co-authored-by: Orca <help@stably.ai>

* fix(crash-reporting): name the WebGL census the same as the fit-retry census

The context-loss crumb spread getLivePaneCensus() raw, so one ring described
one measurement two ways: managers/panes here, livePanes/livePaneManagers on
the fit crumb. Spreading also meant renaming the census return keys would
silently reshape the crumb. Name the fields at the call site and pin them.

Co-authored-by: Orca <help@stably.ai>

* fix(crash-reporting): keep a hot coalesce key from being the first LRU eviction

The suppression path returned before the delete-then-set that re-anchors
recency, so a key hit continuously never moved from its original insertion
slot and became the first eviction candidate — the inverse of the LRU's
stated intent.

`renderer_error` keys carry message+stack identity, so one noisy render loop
mints unbounded distinct keys. Within a single 30s window that churn evicted
the `terminal_safe_fit_retry_exhausted` key mid-burst, un-suppressing it and
re-arming the exact ring flush the coalescing exists to prevent.

Re-anchor position only; `recordedAt` is left alone so the suppression window
still expires on schedule rather than renewing on every hit.

Found while adversarially probing the LRU claim in #10729's own description,
which asserted these keys "cannot evict live keys".

* fix(crash-reporting): report the newest census of a coalesced burst

The suppression path wrote nothing to the ring, so a coalesced burst froze
its FIRST event. Panes mount progressively, so pane 1 exhausting alone
legitimately measures livePanes: 1 -- and the 33 later crumbs, each carrying
a truer census, were dropped. A 34-pane wave was recorded as `livePanes: 1`
with no count: the exact "one pane looping" misread that coalescing by name
was introduced to prevent.

The existing burst test missed this because it fed a constant census on
every crumb, making frozen-first and newest-wins indistinguishable.

Stash the newest payload and fold it into the ring entry the key already
owns: still one slot, now reading livePanes: 34 + suppressedSinceLast: 33.
Resolution is deferred to snapshot time -- sanitizing per suppressed hit of
a 1459/min crash loop measured 2194 ns/op vs 185 ns/op deferred.

Two follow-on defects fixed alongside: an expiring key dropped its pending
payload (it loses its only handle on the ring entry), and resolving the
re-emitting key's own old slot double-counted a burst.

---------

Co-authored-by: Orca <help@stably.ai>
2026-07-26 12:55:45 -07:00

30 lines
1.0 KiB
TypeScript

/**
* Lib-safe sink for WebGL renderer breadcrumbs (context loss/restore, atlas
* resets). The renderer breadcrumb ring lives in the components layer
* (terminal-freeze-breadcrumbs.ts), which may import from lib/pane-manager but
* not the reverse. This indirection lets lib-layer WebGL code record a crumb
* without a backward import: components registers the recorder at startup;
* until then (and in non-renderer contexts) recording is a silent no-op.
*/
/** Crash-report breadcrumb name these crumbs are mirrored under. */
export const TERMINAL_WEBGL_DIAGNOSTIC_BREADCRUMB = 'terminal_webgl_diagnostic'
export type WebglDiagnosticRecorder = (
kind: string,
detail?: Record<string, string | number | boolean | null>
) => void
let recorder: WebglDiagnosticRecorder | null = null
export function setTerminalWebglDiagnosticRecorder(next: WebglDiagnosticRecorder | null): void {
recorder = next
}
export function recordTerminalWebglDiagnostic(
kind: string,
detail?: Record<string, string | number | boolean | null>
): void {
recorder?.(kind, detail)
}