Unlatch the hidden-delivery gate when user input disproves a stuck document.visibilityState

macOS occlusion tracking can wedge document.visibilityState at 'hidden'
after display sleep and never fire another visibilitychange. The hidden-
delivery gate then keeps dropping renderer-bound bytes for panes the user
is looking at (field snapshot 2026-07-06, v1.4.124-rc.2.perf: 78MB dropped
across 2 pane-level-visible ptys with a fully healthy transport), and every
recovery path (window focus, system-resume relay, backlog recovery) re-ran
syncHiddenRendererPtyDelivery only to recompute the same stale predicate —
nothing could ever clear the gate. The user sees a frozen terminal; typing
echo is dropped in main; only a reload recovers.

Real user input while the document claims hidden is a physical
contradiction: keystrokes and clicks only reach a focused, on-screen
window. stale-document-visibility.ts latches that proof, runs each pane's
existing visibilitychange resync (gate unhide + hidden-output snapshot
restore), and hands authority back to the occlusion tracker on the next
genuine visibilitychange. No timers; the failure bias is safe — a wrong
latch can only restore pre-gate delivery cost, never drop bytes. Hot path
unchanged: the foreground predicate still returns on the same single
comparison while the document is visible.

tests/e2e/terminal-stuck-occlusion-recovery.spec.ts pins the wedge
(visibilityState pinned hidden -> output dropped, not painted; the
hiddenDeliveryGatedVisiblePtyCount field discriminator reads >0) and the
recovery (one Shift keypress repaints the missed output from the main-owned
snapshot, no reload, while visibilityState still reads hidden). Negative
control verified: the spec fails without this fix. Typing-latency perf
gate passes; terminal-pane unit suites 366/366.

Co-authored-by: Orca <help@stably.ai>
This commit is contained in:
Jinwoo-H
2026-07-06 17:39:20 -04:00
co-authored by Orca
parent 7046c61832
commit fd5c0f643f
6 changed files with 462 additions and 15 deletions
+40 -9
View File
@@ -3,7 +3,7 @@
Agent-facing map of every optimization on this branch: what it does, why it exists,
where it lives, and the invariants you must not break when adding to it. The
chronological evidence trail (benchmarks, retractions, A/B protocols) is in
`notes/terminal-performance-initiative.md`; this doc is the *current-state* view.
`notes/terminal-performance-initiative.md`; this doc is the _current-state_ view.
**Context**: Orca's terminal was ~300× slower than Terminal.app under agent load
(DSR-under-load p50 134ms, p99 292ms on v1.4.91; agent-TUI throughput 2.0 MB/s).
@@ -21,13 +21,14 @@ shell → pty → daemon (persistence, headless model) → unix socket
Main is on the hot path for every byte (unlike VS Code's ptyHost→renderer
MessagePort). The daemon owns sessions so they survive app restarts; it also runs
a headless xterm emulator per pty — the *model* — which is the source of truth
for screen contents. The renderer terminal is a *view* that can be discarded and
a headless xterm emulator per pty — the _model_ — which is the source of truth
for screen contents. The renderer terminal is a _view_ that can be discarded and
rebuilt from model snapshots.
## Optimization inventory
### 1. Renderer parse-path fixes (the original 16× on agent TUIs)
- **Parse-clocked scheduler drains** (`pane-terminal-output-scheduler.ts`): drain
cadence follows xterm's actual parse completion instead of fixed timers, so the
queue never outruns the parser.
@@ -40,8 +41,10 @@ rebuilt from model snapshots.
per-chunk cost.
### 2. term-speed-2 chain (model/view contract — the architecture)
Revived from ~38 never-merged branches; kill-switched, default ON. Docs:
`docs/reference/terminal-model-view-contract.md`.
- **Hidden view parking**: hidden tabs tear down their xterm view entirely
(memory: parked panes cost ~0).
- **Hidden delivery gate** (main): renderer-bound bytes for hidden ptys are
@@ -55,6 +58,7 @@ Revived from ~38 never-merged branches; kill-switched, default ON. Docs:
reconciliation drops duplicates already covered by the snapshot baseline.
### 3. Batching & scheduling cadence
- **Batch windows 8ms → 2ms** in both `daemon-stream-data-batcher.ts`
(`STREAM_DATA_BATCH_INTERVAL_MS`) and `ipc/pty.ts` (`PTY_BATCH_INTERVAL_MS`).
At 9% utilization there is no queue — latency was literally the sum of fixed
@@ -67,9 +71,11 @@ Revived from ~38 never-merged branches; kill-switched, default ON. Docs:
instead of queuing macrotask-per-keystroke.
### 4. Backpressure (the correctness spine — read before touching delivery)
Three cooperating layers, innermost first:
- **ACK at parse-drain** (`deliverPtyDataWithDeferredAck`, scheduler
`ackCredit`): the renderer credits a chunk when xterm has *parsed* it (or the
`ackCredit`): the renderer credits a chunk when xterm has _parsed_ it (or the
chunk is legitimately discarded), not when IPC delivered it.
**INVARIANT: every delivered chunk credits exactly once — parsed or
discarded.** Every scheduler/pty-connection discard path (backlog replacement,
@@ -92,15 +98,32 @@ Three cooperating layers, innermost first:
push listeners + pull restore markers through the modelRestoreNeeded router.
E2e blackhole harness: `__terminalDeliveryWatchdog`,
`terminal-push-delivery-loss-recovery.spec.ts`.
- **Stale-visibility proof for the hidden gate** (`stale-document-visibility.ts`
and the `shouldWritePtyOutputForeground` fallthrough in pty-connection.ts):
recovers the field-confirmed wedge where macOS occlusion tracking pins
`document.visibilityState` at `'hidden'` after display sleep and never fires
another visibilitychange (v1.4.124-rc.2.perf snapshot: 78MB hidden-gate
dropped across 2 pane-level-visible ptys, transport healthy). Real user
input (keydown/pointerdown/window focus) while the document claims hidden is
a physical contradiction — it latches an override, runs each pane's existing
visibilitychange resync (gate unhide + hidden-output restore), and a genuine
visibilitychange hands authority back. No timers; recovery is purely
event-proven, and the failure bias is safe (a wrong override only restores
pre-gate delivery cost, never drops bytes). Hot-path cost: zero when
visible (same single comparison); one property read per user-interaction
event. E2e: `terminal-stuck-occlusion-recovery.spec.ts` (pins both the
freeze repro and the keystroke recovery, plus the
`hiddenDeliveryGatedVisiblePtyCount` field discriminator).
- **Producer flow control** (protocol v19 `pausePty`/`resumePty`, 256KB pause /
32KB resume watermarks, keyed off **pendingData only** — never renderer
counters; kill switch `PRODUCER_FLOW_CONTROL_ENABLED`, ipc/pty.ts): when main's
buffer grows, the *shell* blocks. For main-hosted ptys pause is synchronous
buffer grows, the _shell_ blocks. For main-hosted ptys pause is synchronous
(drops impossible); for daemon ptys the pause notify has ~20-30ms socket
latency, so wire-speed bursts can still cross the 2MB cap (known follow-up:
daemon-side self-pacing watermark).
### 5. Flood resilience (why bulk output can't wedge or lie anymore)
- **Restore-loop cut** (pty-connection.ts): the hidden-output-restore loop
abandons immediately when a foreground pane's live-chunk queue overflows
(3-iteration hard cap), and a 2s flood-suppression window stops main's own
@@ -110,20 +133,23 @@ Three cooperating layers, innermost first:
drop re-arms restore) that caused multi-second renderer stalls.
- **Query survival**: if the 2MB cap ever drops bulk output, embedded terminal
queries are extracted (`terminal-reply-query-extraction.ts`) and answered by
*synthesizing replies on the input path* (CPR from live buffer, DA1 canned,
_synthesizing replies on the input path_ (CPR from live buffer, DA1 canned,
OSC via direct responder) — probes and TUIs never hang on a dropped reply.
- Drops are downstream of the model: the daemon ingests every byte, so the
post-flood repaint restores complete, correct content.
### 6. Wake/sleep recovery
- powerMonitor resume → `system:resumed` IPC → renderer wake recovery (fixes
WebGL-latch blank-after-sleep that DOM focus/visibilitychange missed).
- Cumulative ACKs make the historical "lost ACKs across suspend pin the global
window forever" wedge (BMW user bug) structurally impossible.
### 7. Snapshot fidelity (the garble fixes — all fuzz-pinned)
Reveal-from-snapshot multiplied exposure of serializer defects ~1000×. Five bugs
found by differential fuzzing; four fixed, one tolerated:
- **B: SGR intensity ordering** — upstream `@xterm/addon-serialize` emitted
`1;22` (22 clears the bold 1 just set). Patched via pnpm patch
(`config/patches/@xterm__addon-serialize@*.patch`): clear-before-set for the
@@ -143,6 +169,7 @@ found by differential fuzzing; four fixed, one tolerated:
`bufferHasSerializeHostileWrappedRow`, the only remaining tolerance.
## Correctness infrastructure (run these before merging delivery/restore changes)
- `headless-emulator-fidelity.fuzz.test.ts` — differential: HeadlessEmulator vs
reference xterm, seeded TUI streams. `FUZZ_ITERATIONS=2000` for deep,
`FUZZ_SEED=n` to replay.
@@ -158,8 +185,9 @@ found by differential fuzzing; four fixed, one tolerated:
(pane-manager / terminal-pane suites).
## Benchmarking protocol (hard-won rules)
- Rig: `tools/benchmarks/terminal-pipeline-bench.mjs` — DSR idle + DSR under
1MB/s agent-TUI load + DSR-fenced throughput on 4 fixtures. Run *inside* the
1MB/s agent-TUI load + DSR-fenced throughput on 4 fixtures. Run _inside_ the
terminal under test.
- **Bench at 10MB** (`--size-mb 10`). The ACK-at-parse bug shipped because dev
benches used 3MB and never tripped the cap.
@@ -170,8 +198,9 @@ found by differential fuzzing; four fixed, one tolerated:
- Packaged builds are truth; dev has ~2× overhead.
## Release mechanics
- Perf RCs: `release-cut.yml` workflow_dispatch, `kind=rc ref=orca-performance
version_suffix=perf` → tags like `v1.4.122-rc.1.perf`. The suffix sorts above
version_suffix=perf` → tags like `v1.4.122-rc.1.perf`. The suffix sorts above
its base rc.N but below rc.N+1 (never hijacks the RC channel). The rc counter
(`release-rc-history.mjs`), telemetry identity classifier, and build guard are
all suffix-aware — a suffixed rc classifies as `rc`.
@@ -205,6 +234,7 @@ Conflict pattern, established over ~6 syncs:
before the next RC cut.
## Known limits / next levers (in rough priority order)
1. Daemon self-pacing: daemon-hosted ptys can cross the 2MB cap for ~20-30ms at
wire speed before `pausePty` bites. Fix: daemon enforces its own watermark
locally (VS Code does this server-side for remotes).
@@ -218,12 +248,13 @@ Conflict pattern, established over ~6 syncs:
is open against main separately.
## Guardrails for future agents
- The chunk-credit invariant (§4) is the load-bearing one. If you add ANY path
that receives, defers, drops, or splits pty data in the renderer, prove it
credits exactly once. The credit-invariant unit tests are the gate.
- Flow control keys off `pendingData` only. Do not couple it to renderer
counters; the two layers compose because they are independent.
- Snapshot changes must keep seq semantics: a snapshot covers *exactly* bytes
- Snapshot changes must keep seq semantics: a snapshot covers _exactly_ bytes
≤ its seq (Bug E made this true; don't regress it). Chunks after restore are
reconciled by seq — off-by-N re-triggers duplicate-drop garble.
- Hidden panes must receive nothing (delivery gate) but side-effects and query
@@ -8652,12 +8652,21 @@ describe('connectPanePty', () => {
})
it('does not apply stale background Codex query chunks after hidden snapshot restore', async () => {
const visibilityChangeHandler: { current: (() => void) | null } = { current: null }
// Fire-all like a real event target: the pane resync handler and the
// stale-visibility trust handler both listen for visibilitychange.
const visibilityChangeListeners: (() => void)[] = []
const visibilityChangeHandler = {
current: (): void => {
for (const listener of visibilityChangeListeners) {
listener()
}
}
}
;(globalThis as { document?: Document }).document = {
visibilityState: 'visible',
addEventListener: vi.fn((type: string, listener: EventListenerOrEventListenerObject) => {
if (type === 'visibilitychange') {
visibilityChangeHandler.current = listener as () => void
visibilityChangeListeners.push(listener as () => void)
}
}),
removeEventListener: vi.fn()
@@ -9959,14 +9968,23 @@ describe('connectPanePty', () => {
it('keeps recovery pending when hidden output arrives during an in-flight snapshot', async () => {
let visibilityState: DocumentVisibilityState = 'visible'
const visibilityChangeHandler: { current: (() => void) | null } = { current: null }
// Fire-all like a real event target: the pane resync handler and the
// stale-visibility trust handler both listen for visibilitychange.
const visibilityChangeListeners: (() => void)[] = []
const visibilityChangeHandler = {
current: (): void => {
for (const listener of visibilityChangeListeners) {
listener()
}
}
}
;(globalThis as { document?: Document }).document = {
get visibilityState() {
return visibilityState
},
addEventListener: vi.fn((type: string, listener: EventListenerOrEventListenerObject) => {
if (type === 'visibilitychange') {
visibilityChangeHandler.current = listener as () => void
visibilityChangeListeners.push(listener as () => void)
}
}),
removeEventListener: vi.fn()
@@ -49,6 +49,10 @@ import { isPtyLocked } from '@/lib/pane-manager/mobile-driver-state'
import { reconcilePtySizeAcrossFrames, type PtySizeReconcileHandle } from './pty-size-reconcile'
import { createPtySizeReassertion } from './pty-size-reassertion'
import { isPaneReplaying, replayIntoTerminal, replayIntoTerminalAsync } from './replay-guard'
import {
isDocumentVisibilityProvenStale,
registerStaleDocumentVisibilityRecovery
} from './stale-document-visibility'
import {
nativeWindowsRewriteNeedsFollowupRenderRefresh,
terminalOutputPrefersRenderRefresh,
@@ -766,7 +770,13 @@ function shouldWritePtyOutputForeground(isPaneVisible: boolean): boolean {
// Why: Electron can keep visible panes mounted while the whole app is
// backgrounded. Treat hidden documents like background tabs so Chromium
// timer throttling cannot pin terminal writes on the renderer foreground path.
return document.visibilityState === 'visible'
if (document.visibilityState === 'visible') {
return true
}
// Why: macOS occlusion tracking can wedge visibilityState at 'hidden' after
// display sleep; proven-stale means real user input contradicted it, so the
// hidden-delivery gate must not keep dropping a watched pane's bytes.
return isDocumentVisibilityProvenStale()
}
function containsSynchronizedOutputStart(data: string): boolean {
@@ -5582,8 +5592,17 @@ export function connectPanePty(
}
}
document.addEventListener('visibilitychange', onDocumentVisibilityChange)
unregisterDocumentVisibilityRecovery = () =>
// Why: when user input proves visibilityState is wedged at 'hidden'
// (stale macOS occlusion), run the same resync — no visibilitychange
// will ever fire in that state, and the gate would drop watched bytes
// forever.
const unregisterStaleVisibilityRecovery = registerStaleDocumentVisibilityRecovery(
onDocumentVisibilityChange
)
unregisterDocumentVisibilityRecovery = () => {
document.removeEventListener('visibilitychange', onDocumentVisibilityChange)
unregisterStaleVisibilityRecovery()
}
}
const dataCallback = (data: string, meta?: PtyDataMeta): void => {
@@ -0,0 +1,157 @@
// Why: pins the stale-occlusion proof — user input while
// document.visibilityState claims 'hidden' must latch the override and run
// each pane's recovery exactly once, and a genuine visibilitychange must hand
// authority back to the occlusion tracker.
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import type * as StaleDocumentVisibility from './stale-document-visibility'
type Handler = () => void
type EventTargetStub = {
addEventListener: (type: string, handler: Handler, options?: unknown) => void
removeEventListener: (type: string, handler: Handler, options?: unknown) => void
emit: (type: string) => void
listenerCount: (type: string) => number
}
function createEventTargetStub(): EventTargetStub {
const listeners = new Map<string, Set<Handler>>()
return {
addEventListener(type, handler) {
const set = listeners.get(type) ?? new Set()
set.add(handler)
listeners.set(type, set)
},
removeEventListener(type, handler) {
listeners.get(type)?.delete(handler)
},
emit(type) {
for (const handler of listeners.get(type) ?? []) {
handler()
}
},
listenerCount(type) {
return listeners.get(type)?.size ?? 0
}
}
}
type DocumentStub = EventTargetStub & { visibilityState: string }
describe('stale document visibility', () => {
const originalDocument = (globalThis as { document?: unknown }).document
const originalWindow = (globalThis as { window?: unknown }).window
let documentStub: DocumentStub
let windowStub: EventTargetStub
let warnSpy: ReturnType<typeof vi.spyOn>
let mod: typeof StaleDocumentVisibility
beforeEach(async () => {
vi.resetModules()
documentStub = { ...createEventTargetStub(), visibilityState: 'visible' }
windowStub = createEventTargetStub()
;(globalThis as { document: unknown }).document = documentStub
;(globalThis as { window: unknown }).window = windowStub
warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {})
mod = await import('./stale-document-visibility')
})
afterEach(() => {
mod.resetStaleDocumentVisibilityForTesting()
warnSpy.mockRestore()
;(globalThis as { document: unknown }).document = originalDocument
;(globalThis as { window: unknown }).window = originalWindow
})
it('latches the override and runs recovery once when input arrives while hidden', () => {
const recovery = vi.fn()
mod.registerStaleDocumentVisibilityRecovery(recovery)
documentStub.visibilityState = 'hidden'
documentStub.emit('keydown')
expect(mod.isDocumentVisibilityProvenStale()).toBe(true)
expect(recovery).toHaveBeenCalledTimes(1)
expect(warnSpy).toHaveBeenCalledTimes(1)
// Further input during the same stuck episode must not re-run recovery.
documentStub.emit('keydown')
documentStub.emit('pointerdown')
expect(recovery).toHaveBeenCalledTimes(1)
})
it('does not latch while the document is genuinely visible', () => {
const recovery = vi.fn()
mod.registerStaleDocumentVisibilityRecovery(recovery)
documentStub.emit('keydown')
documentStub.emit('pointerdown')
windowStub.emit('focus')
expect(mod.isDocumentVisibilityProvenStale()).toBe(false)
expect(recovery).not.toHaveBeenCalled()
})
it('treats pointerdown and window focus as staleness proof too', () => {
const recovery = vi.fn()
mod.registerStaleDocumentVisibilityRecovery(recovery)
documentStub.visibilityState = 'hidden'
documentStub.emit('pointerdown')
expect(mod.isDocumentVisibilityProvenStale()).toBe(true)
mod.resetStaleDocumentVisibilityForTesting()
mod.registerStaleDocumentVisibilityRecovery(recovery)
documentStub.visibilityState = 'hidden'
windowStub.emit('focus')
expect(mod.isDocumentVisibilityProvenStale()).toBe(true)
})
it('hands authority back to the occlusion tracker on a genuine visibilitychange', () => {
const recovery = vi.fn()
mod.registerStaleDocumentVisibilityRecovery(recovery)
documentStub.visibilityState = 'hidden'
documentStub.emit('keydown')
expect(mod.isDocumentVisibilityProvenStale()).toBe(true)
documentStub.visibilityState = 'visible'
documentStub.emit('visibilitychange')
expect(mod.isDocumentVisibilityProvenStale()).toBe(false)
// A fresh wedge after trust was restored must latch (and recover) again.
documentStub.visibilityState = 'hidden'
documentStub.emit('keydown')
expect(mod.isDocumentVisibilityProvenStale()).toBe(true)
expect(recovery).toHaveBeenCalledTimes(2)
})
it('keeps running other panes when one recovery listener throws', () => {
const throwing = vi.fn(() => {
throw new Error('pane exploded')
})
const healthy = vi.fn()
mod.registerStaleDocumentVisibilityRecovery(throwing)
mod.registerStaleDocumentVisibilityRecovery(healthy)
documentStub.visibilityState = 'hidden'
documentStub.emit('keydown')
expect(throwing).toHaveBeenCalledTimes(1)
expect(healthy).toHaveBeenCalledTimes(1)
})
it('removes global listeners when the last pane unregisters', () => {
const unregisterA = mod.registerStaleDocumentVisibilityRecovery(vi.fn())
const unregisterB = mod.registerStaleDocumentVisibilityRecovery(vi.fn())
expect(documentStub.listenerCount('keydown')).toBe(1)
unregisterA()
expect(documentStub.listenerCount('keydown')).toBe(1)
unregisterB()
expect(documentStub.listenerCount('keydown')).toBe(0)
expect(documentStub.listenerCount('pointerdown')).toBe(0)
expect(documentStub.listenerCount('visibilitychange')).toBe(0)
expect(windowStub.listenerCount('focus')).toBe(0)
// Re-registering must reinstall for a later pane.
mod.registerStaleDocumentVisibilityRecovery(vi.fn())
expect(documentStub.listenerCount('keydown')).toBe(1)
})
})
@@ -0,0 +1,100 @@
// Chromium's macOS window-occlusion tracker can wedge document.visibilityState
// at 'hidden' after display sleep, never firing another visibilitychange. Real
// user input reaching this window while the document claims hidden is a
// physical contradiction — keystrokes and clicks only reach a focused,
// on-screen window. This module latches that proof so terminal delivery can
// treat the document as visible until the next genuine visibilitychange
// restores trust in the occlusion tracker. Without it, the hidden-delivery
// gate stays latched for panes the user is looking at and main drops their
// bytes indefinitely (field: 78MB dropped on 2 visible ptys, v1.4.124-rc.2.perf).
type StaleVisibilityRecoveryListener = () => void
const recoveryListeners = new Set<StaleVisibilityRecoveryListener>()
let visibilityProvenStale = false
let globalListenersInstalled = false
export function isDocumentVisibilityProvenStale(): boolean {
return visibilityProvenStale
}
function onUserInteractionWithDocument(): void {
if (visibilityProvenStale || document.visibilityState !== 'hidden') {
return
}
visibilityProvenStale = true
console.warn(
'[terminal] user input arrived while document.visibilityState is hidden — treating occlusion state as stale and re-syncing terminal delivery',
{ recoveryListenerCount: recoveryListeners.size }
)
for (const listener of recoveryListeners) {
try {
listener()
} catch {
// Why: one pane's recovery failure must not starve the other panes'.
}
}
}
function onDocumentVisibilityChange(): void {
// A genuine visibilitychange means the occlusion tracker is reporting
// again — hand authority back to document.visibilityState.
visibilityProvenStale = false
}
function installGlobalListeners(): void {
if (
globalListenersInstalled ||
typeof document === 'undefined' ||
typeof window === 'undefined' ||
typeof document.addEventListener !== 'function'
) {
return
}
globalListenersInstalled = true
// Capture phase so no stopPropagation in the app can hide the proof; the
// handler is a single property read when visibility is healthy.
document.addEventListener('keydown', onUserInteractionWithDocument, {
capture: true,
passive: true
})
document.addEventListener('pointerdown', onUserInteractionWithDocument, {
capture: true,
passive: true
})
window.addEventListener('focus', onUserInteractionWithDocument)
document.addEventListener('visibilitychange', onDocumentVisibilityChange)
}
function removeGlobalListeners(): void {
if (!globalListenersInstalled) {
return
}
globalListenersInstalled = false
document.removeEventListener('keydown', onUserInteractionWithDocument, { capture: true })
document.removeEventListener('pointerdown', onUserInteractionWithDocument, { capture: true })
window.removeEventListener('focus', onUserInteractionWithDocument)
document.removeEventListener('visibilitychange', onDocumentVisibilityChange)
}
// The listener runs when staleness is first proven; register the same handler
// used for document visibilitychange so recovery reuses the pane's existing
// gate-resync + hidden-output-restore path.
export function registerStaleDocumentVisibilityRecovery(
listener: StaleVisibilityRecoveryListener
): () => void {
installGlobalListeners()
recoveryListeners.add(listener)
return () => {
recoveryListeners.delete(listener)
if (recoveryListeners.size === 0) {
removeGlobalListeners()
}
}
}
export function resetStaleDocumentVisibilityForTesting(): void {
visibilityProvenStale = false
recoveryListeners.clear()
removeGlobalListeners()
}
@@ -0,0 +1,122 @@
/**
* Repro + recovery for the stuck-occlusion freeze (field snapshot,
* v1.4.124-rc.2.perf, 2026-07-06): macOS occlusion tracking wedges
* document.visibilityState at 'hidden' after display sleep and never fires
* another visibilitychange. The hidden-delivery gate then marks the pane the
* user is looking at as hidden, and main drops its renderer-bound bytes
* indefinitely (78MB dropped across 2 visible ptys in the field) — a frozen
* terminal with a perfectly healthy transport.
*
* The test emulates the wedge exactly as Chromium produces it: visibilityState
* pinned to 'hidden' with one final visibilitychange, then silence. Recovery
* must come from the staleness proof — a real keystroke while the document
* claims hidden — which unlatches the gate and repaints the missed output from
* the main-owned snapshot, WITHOUT a reload and WITHOUT any visibilitychange.
*/
import type { Page } from '@stablyai/playwright-test'
import { test, expect } from './helpers/orca-app'
import { waitForSessionReady, waitForActiveWorktree, ensureTerminalVisible } from './helpers/store'
import {
waitForActiveTerminalManager,
waitForActivePanePtyId,
execInTerminal,
getTerminalContent
} from './helpers/terminal'
type DeliverySnapshot = {
hiddenDeliveryGatedPtyCount: number
hiddenDeliveryGatedVisiblePtyCount: number
hiddenDeliveryDroppedChars: number
}
async function getDeliverySnapshot(page: Page): Promise<DeliverySnapshot> {
return page.evaluate(async () => {
const snapshot = await window.api.pty.getRendererDeliveryDebugSnapshot()
return {
hiddenDeliveryGatedPtyCount: snapshot.hiddenDeliveryGatedPtyCount,
hiddenDeliveryGatedVisiblePtyCount: snapshot.hiddenDeliveryGatedVisiblePtyCount,
hiddenDeliveryDroppedChars: snapshot.hiddenDeliveryDroppedChars
}
})
}
test.describe('terminal stuck-occlusion recovery', () => {
test.afterEach(async ({ orcaPage }) => {
// Drop the instance shadow so the prototype getter (real state) rules
// again, and fire one genuine visibilitychange to restore tracker trust.
await orcaPage.evaluate(() => {
delete (document as { visibilityState?: string }).visibilityState
document.dispatchEvent(new Event('visibilitychange'))
})
})
test('a keystroke unlatches the hidden-delivery gate wedged by stale visibilityState', async ({
orcaPage
}) => {
test.setTimeout(120_000)
await waitForSessionReady(orcaPage)
await waitForActiveWorktree(orcaPage)
await ensureTerminalVisible(orcaPage)
await waitForActiveTerminalManager(orcaPage)
const ptyId = await waitForActivePanePtyId(orcaPage)
// Live baseline: foreground delivery works. The $((…)) arithmetic keeps
// the asserted string out of the typed command's local echo.
await execInTerminal(orcaPage, ptyId, 'echo live-before-$((41+1))')
await expect
.poll(async () => getTerminalContent(orcaPage), { timeout: 15_000 })
.toContain('live-before-42')
// Emulate the Chromium occlusion wedge: visibilityState pins at 'hidden',
// one last visibilitychange fires, then the tracker goes silent forever.
await orcaPage.evaluate(() => {
Object.defineProperty(document, 'visibilityState', {
get: () => 'hidden',
configurable: true
})
document.dispatchEvent(new Event('visibilitychange'))
})
// The visible pane's pty gets marked hidden in main — the field state:
// gate holding a pty that main's own visibility set says is visible.
await expect
.poll(async () => (await getDeliverySnapshot(orcaPage)).hiddenDeliveryGatedPtyCount, {
timeout: 15_000
})
.toBeGreaterThan(0)
expect(
(await getDeliverySnapshot(orcaPage)).hiddenDeliveryGatedVisiblePtyCount
).toBeGreaterThan(0)
// The freeze repro: output produced now is dropped by main, not painted.
const droppedBefore = (await getDeliverySnapshot(orcaPage)).hiddenDeliveryDroppedChars
await execInTerminal(orcaPage, ptyId, 'echo occluded-$((70+8))')
await expect
.poll(async () => (await getDeliverySnapshot(orcaPage)).hiddenDeliveryDroppedChars, {
timeout: 15_000
})
.toBeGreaterThan(droppedBefore)
expect(await getTerminalContent(orcaPage)).not.toContain('occluded-78')
// The staleness proof: one real keystroke while the document claims
// hidden. No visibilitychange fires — recovery must ride the proof alone.
await orcaPage.keyboard.press('Shift')
// Gate unlatches and the missed output repaints from the main-owned
// snapshot — no reload, visibilityState still reads 'hidden'.
await expect
.poll(async () => getTerminalContent(orcaPage), { timeout: 30_000 })
.toContain('occluded-78')
await expect
.poll(async () => (await getDeliverySnapshot(orcaPage)).hiddenDeliveryGatedVisiblePtyCount, {
timeout: 15_000
})
.toBe(0)
// Live delivery continues under the override.
await execInTerminal(orcaPage, ptyId, 'echo live-after-$((200+56))')
await expect
.poll(async () => getTerminalContent(orcaPage), { timeout: 15_000 })
.toContain('live-after-256')
})
})