From 665b3d372e681731d31c35c129c2e511e28cd414 Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Mon, 31 Aug 2026 04:08:17 -0700 Subject: [PATCH] docs(reliability): record schema-v2 split benchmark --- config/reliability-gates.jsonc | 82 +++++++++++++++++++++++++++------- 1 file changed, 66 insertions(+), 16 deletions(-) diff --git a/config/reliability-gates.jsonc b/config/reliability-gates.jsonc index 862562948f8..3054f52025d 100644 --- a/config/reliability-gates.jsonc +++ b/config/reliability-gates.jsonc @@ -5735,20 +5735,24 @@ "providers": ["local", "local-daemon", "ssh", "wsl", "remote-runtime"], "coveredPlatforms": ["macos"], "coveredProviders": ["local", "remote-runtime"], - "coverageNotes": "Deterministic renderer contracts hold CWD resolution behind an explicit promise, require the new pane to be created synchronously, and prove that close cancels the pending connection. A module-level stable-pane-key handoff preserves the exact CWD promise and bounded pre-connect input across whole-tab remounts, including a tab rehome between worktree buckets; stale owners are fenced, concrete PTY bind and definitive spawn failure clear the record, and explicit pane close discards it. Detach contracts reject cwd-pending and cwd-resolved deferred splits before PTY bind without mutation, carry resolved cwd for other unbound panes, and preserve persisted or live PTY handoff. Local IPC transport contracts exercise the real bounded pre-connect buffer and one live input FIFO across seeded and newly typed ordinary, acknowledged, and immediate writes, concurrent flushes, in-flight teardown, late spawn success or failure, attach failure, failed spawn, same-id reuse, stale-spawn retirement ownership, destroy, and mutable recovery metadata. The handoff registry is capped at 64 records for 15 seconds and shares the existing 1,024-entry/conservative UTF-16 input ceilings. A mocked direct-SSH authority-rotation contract proves a rejected stale spawn releases its deferred-CWD fence. The committed headful Electron benchmark drives the real platform shortcut in a visible BrowserWindow and document for 3 warmups followed by 20 measured cold-CWD cycles, requiring a distinct child PTY, first echo, and observed child pty:exit before the next cycle. Remote-runtime coverage proves delegation remains host-owned; its host-delegated split path does not consume the local pre-connect input options, so remote-runtime input-remount replay and physical local-daemon, SSH, WSL, Linux, Windows, and folder-workspace latency journeys remain gaps.", + "coverageNotes": "Deterministic renderer contracts hold CWD resolution behind an explicit promise, require the new pane to be created synchronously, and prove that close cancels the pending connection. A module-level stable-pane-key handoff preserves the exact CWD promise and bounded pre-connect input across whole-tab remounts, including a tab rehome between worktree buckets; stale owners are fenced, concrete PTY bind and definitive spawn failure clear the record, and explicit pane close discards it. Detach contracts reject cwd-pending and cwd-resolved deferred splits before PTY bind without mutation, carry resolved cwd for other unbound panes, and preserve persisted or live PTY handoff. Local IPC transport contracts exercise the real bounded pre-connect buffer and one live input FIFO across seeded and newly typed ordinary, acknowledged, and immediate writes, concurrent flushes, in-flight teardown, late spawn success or failure, attach failure, failed spawn, same-id reuse, stale-spawn retirement ownership, destroy, and mutable recovery metadata. The handoff registry is capped at 64 records for 15 seconds and shares the existing 1,024-entry/conservative UTF-16 input ceilings. A mocked direct-SSH authority-rotation contract proves a rejected stale spawn releases its deferred-CWD fence. The schema-v2 headful Electron benchmark records exact revision identity and attributes CWD request/settlement, PTY spawn request/result, bind, fixture unlock request/IPC write, fixture readiness, input, and first echo across 3 warmups and 20 measured cold-CWD cycles, requiring a distinct child PTY and observed child pty:exit before the next cycle. Remote-runtime coverage proves delegation remains host-owned; its host-delegated split path does not consume the local pre-connect input options, so remote-runtime input-remount replay and physical local-daemon, SSH, WSL, Linux, Windows, and folder-workspace latency journeys remain gaps.", "motivatingLinks": ["https://github.com/stablyai/orca/commit/572ed1a8882"], "invariant": "A terminal split creates and activates its renderer pane before an inherited-CWD lookup settles, starts its PTY only after the resolved directory is available, and cannot be externally detached while that deferred spawn remains unbound. A whole-tab remount or worktree rehome preserves the same stable pane's CWD promise and admitted local pre-connect bytes in order until a successor binds or the intent is definitively abandoned; stale owners cannot append or clear the successor's record. Other unbound panes preserve resolved cwd as startupCwd when detached. Bounded pre-connect input and later live local input share byte order, and teardown settles acknowledged writes without creating or rebinding a stale PTY. A disconnected or detached pending connect cannot bind its late fresh spawn, report its late failure through current callbacks, or ID-retire a newer same-ID owner; rejecting a stale direct-SSH spawn also releases the matching deferred-CWD fence. Natural exit cannot deliver queued work into a reused PTY id. Bound and remote-runtime splits remain owned by their execution host.", - "oracle": "Hold CWD resolution behind a controllable promise, invoke the production split path, and require manager.splitPane plus split telemetry before resolving it. Before PTY bind, require both cwd-pending and cwd-resolved deferred detach attempts to return null without layout, pane, tab, ownership, or focus mutation; separately require resolved cwd on an allowed unbound detach and unchanged persisted/live PTY adoption. Exercise the stable-pane handoff registry through repeated remounts and a worktree rehome, requiring the identical CWD promise, ordered ordinary/acknowledged/immediate seed replay, stale-owner fencing, 64-record/15-second bounds, and discard on bind, failure, or explicit close. Rotate a mocked direct-SSH authority while its delayed spawn is in flight, reject and disconnect the stale PTY claim, then require exactly one deferred-CWD cleanup when the delayed connect settles. At the local IPC PTY boundary, require zero connect calls while pending, the resolved CWD in spawn and local recovery metadata, one shared FIFO across seeded/new pre-connect and live ordinary/acknowledged/immediate input, prompt predecessor acknowledged-promise settlement on teardown, retirement of an unowned late fresh spawn, preservation of a newer same-ID owner, and zero stale delivery after failure, close, destroy, detach, natural exit, or same-id reuse. Capture callback exceptions must not change admission results. In visible Electron, press the real split shortcut for 3 warmup cycles, then after a cold inherited-CWD interval for each of 20 measured cycles, and require complete focus, distinct-PTY bind, immediate-input echo, pane-count, and child-exit observations for every cycle; a timed-out, missing-event, or cleanup-aborted run must publish no headline latency and fail.", + "oracle": "Hold CWD resolution behind a controllable promise, invoke the production split path, and require manager.splitPane plus split telemetry before resolving it. Before PTY bind, require both cwd-pending and cwd-resolved deferred detach attempts to return null without layout, pane, tab, ownership, or focus mutation; separately require resolved cwd on an allowed unbound detach and unchanged persisted/live PTY adoption. Exercise the stable-pane handoff registry through repeated remounts and a worktree rehome, requiring the identical CWD promise, ordered ordinary/acknowledged/immediate seed replay, stale-owner fencing, 64-record/15-second bounds, and discard on bind, failure, or explicit close. Rotate a mocked direct-SSH authority while its delayed spawn is in flight, reject and disconnect the stale PTY claim, then require exactly one deferred-CWD cleanup when the delayed connect settles. At the local IPC PTY boundary, require zero connect calls while pending, the resolved CWD in spawn and local recovery metadata, one shared FIFO across seeded/new pre-connect and live ordinary/acknowledged/immediate input, prompt predecessor acknowledged-promise settlement on teardown, retirement of an unowned late fresh spawn, preservation of a newer same-ID owner, and zero stale delivery after failure, close, destroy, detach, natural exit, or same-id reuse. Capture callback exceptions must not change admission results. In visible Electron, press the real split shortcut for 3 warmup cycles, then after a cold inherited-CWD interval for each of 20 measured cycles, require an exact clean revision identity, complete focus/CWD/spawn/bind/fixture/input/echo attribution, distinct child PTYs, pane count, and child exits for every cycle; a timed-out, missing-event, or cleanup-aborted run must publish no headline latency and fail.", "commands": [ - "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/terminal-pane/terminal-pane-split-with-inherited-cwd.test.ts src/renderer/src/lib/pane-manager/pane-split-close.test.ts src/renderer/src/components/terminal-pane/resolve-split-cwd.test.ts src/renderer/src/components/terminal-pane/pty-connection-split-cwd-resolution.test.ts src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts src/renderer/src/components/terminal-pane/pty-preconnect-input-buffer.test.ts src/renderer/src/components/terminal-pane/pty-transport-input-write.test.ts src/renderer/src/components/terminal-pane/deferred-split-pane-handoff.test.ts tests/e2e/terminal-split-activation-latency-artifact.unit.test.ts --reporter=dot", + "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/terminal-pane/terminal-pane-split-with-inherited-cwd.test.ts src/renderer/src/lib/pane-manager/pane-split-close.test.ts src/renderer/src/components/terminal-pane/resolve-split-cwd.test.ts src/renderer/src/components/terminal-pane/pty-connection-split-cwd-resolution.test.ts src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts src/renderer/src/components/terminal-pane/pty-preconnect-input-buffer.test.ts src/renderer/src/components/terminal-pane/pty-transport-input-write.test.ts src/renderer/src/components/terminal-pane/deferred-split-pane-handoff.test.ts tests/e2e/terminal-split-activation-latency-artifact.unit.test.ts tests/e2e/terminal-split-activation-latency-main-probe.ts tests/e2e/terminal-split-activation-latency-phases.ts tests/e2e/terminal-split-activation-latency-report.unit.test.ts --reporter=dot", // Historical evidence record retained so its seven-file command remains auditable. "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/terminal-pane/terminal-pane-split-with-inherited-cwd.test.ts src/renderer/src/lib/pane-manager/pane-split-close.test.ts src/renderer/src/components/terminal-pane/resolve-split-cwd.test.ts src/renderer/src/components/terminal-pane/pty-connection-split-cwd-resolution.test.ts src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts src/renderer/src/components/terminal-pane/pty-preconnect-input-buffer.test.ts src/renderer/src/components/terminal-pane/pty-transport-input-write.test.ts --reporter=dot", - // Historical evidence records only; these labels do not verify the checkout or harness. + // Historical evidence record retained so its nine-file command remains auditable. + "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/terminal-pane/terminal-pane-split-with-inherited-cwd.test.ts src/renderer/src/lib/pane-manager/pane-split-close.test.ts src/renderer/src/components/terminal-pane/resolve-split-cwd.test.ts src/renderer/src/components/terminal-pane/pty-connection-split-cwd-resolution.test.ts src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts src/renderer/src/components/terminal-pane/pty-preconnect-input-buffer.test.ts src/renderer/src/components/terminal-pane/pty-transport-input-write.test.ts src/renderer/src/components/terminal-pane/deferred-split-pane-handoff.test.ts tests/e2e/terminal-split-activation-latency-artifact.unit.test.ts --reporter=dot", + // Historical schema-v1 evidence records only; their labels do not verify the checkout or harness. "ORCA_TERMINAL_SPLIT_LATENCY_BENCH=1 ORCA_TERMINAL_SPLIT_LATENCY_LABEL=baseline-df14d1a2983d8339e788d0e521f1c4affd9c6d5f-headful-run1 ORCA_TERMINAL_SPLIT_LATENCY_OUTPUT=/private/tmp/orca-terminal-split-activation-baseline-df14d1a2983d8339e788d0e521f1c4affd9c6d5f-headful-run1.json pnpm exec playwright test tests/e2e/terminal-split-activation-latency.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1", "ORCA_TERMINAL_SPLIT_LATENCY_BENCH=1 ORCA_TERMINAL_SPLIT_LATENCY_LABEL=candidate-d453ffcdb704764daced1b2917fddee7224389f0-headful-run2 ORCA_TERMINAL_SPLIT_LATENCY_OUTPUT=/private/tmp/orca-terminal-split-activation-candidate-d453ffcdb704764daced1b2917fddee7224389f0-headful-run2.json pnpm exec playwright test tests/e2e/terminal-split-activation-latency.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1", - // Exact-HEAD evidence records use the committed benchmark harness; they remain opt-in audit commands. + // Exact-HEAD schema-v1 evidence records use the committed harness but predate revision metadata. "ORCA_TERMINAL_SPLIT_LATENCY_BENCH=1 ORCA_TERMINAL_SPLIT_LATENCY_LABEL=candidate-962faacec8c-headful-current ORCA_TERMINAL_SPLIT_LATENCY_OUTPUT=/private/tmp/orca-terminal-split-activation-962faacec8c-headful.json pnpm exec playwright test tests/e2e/terminal-split-activation-latency.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1", - "ORCA_TERMINAL_SPLIT_LATENCY_BENCH=1 ORCA_TERMINAL_SPLIT_LATENCY_LABEL=candidate-962faacec8c-headful-run2 ORCA_TERMINAL_SPLIT_LATENCY_OUTPUT=/private/tmp/orca-terminal-split-activation-962faacec8c-headful-run2.json pnpm exec playwright test tests/e2e/terminal-split-activation-latency.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1" + "ORCA_TERMINAL_SPLIT_LATENCY_BENCH=1 ORCA_TERMINAL_SPLIT_LATENCY_LABEL=candidate-962faacec8c-headful-run2 ORCA_TERMINAL_SPLIT_LATENCY_OUTPUT=/private/tmp/orca-terminal-split-activation-962faacec8c-headful-run2.json pnpm exec playwright test tests/e2e/terminal-split-activation-latency.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1", + // Schema-v2 evidence embeds the exact checkout identity and clean/dirty state. + "ORCA_TERMINAL_SPLIT_LATENCY_BENCH=1 ORCA_TERMINAL_SPLIT_LATENCY_LABEL=candidate-073e6c7b0eb-headful-clean ORCA_TERMINAL_SPLIT_LATENCY_OUTPUT=/private/tmp/orca-terminal-split-activation-073e6c7b0eb-headful-clean.json pnpm exec playwright test tests/e2e/terminal-split-activation-latency.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1" ], "testFiles": [ "src/renderer/src/components/terminal-pane/terminal-pane-split-with-inherited-cwd.test.ts", @@ -5760,6 +5764,9 @@ "src/renderer/src/components/terminal-pane/pty-transport-input-write.test.ts", "src/renderer/src/components/terminal-pane/deferred-split-pane-handoff.test.ts", "tests/e2e/terminal-split-activation-latency-artifact.unit.test.ts", + "tests/e2e/terminal-split-activation-latency-main-probe.ts", + "tests/e2e/terminal-split-activation-latency-phases.ts", + "tests/e2e/terminal-split-activation-latency-report.unit.test.ts", "tests/e2e/terminal-split-activation-latency.spec.ts" ], "assertionRefs": [ @@ -5845,14 +5852,39 @@ "fails when the benchmark artifact path cannot be written" ] }, + { + "file": "tests/e2e/terminal-split-activation-latency-main-probe.ts", + "assertions": [ + "attributes CWD and PTY spawn request/settlement events to the source and child PTYs", + "captures the fixture unlock carriage return on both ordinary and acknowledged IPC channels", + "restores the intercepted IPC handlers and listener when the probe is disposed" + ] + }, + { + "file": "tests/e2e/terminal-split-activation-latency-phases.ts", + "assertions": [ + "merges main-process events by operation and PTY identity without cross-cycle attribution", + "requires every activation, fixture, input, echo, pane, PTY, and cleanup observation for success", + "reports each attributed phase distribution with non-negative cross-clock durations" + ] + }, + { + "file": "tests/e2e/terminal-split-activation-latency-report.unit.test.ts", + "assertions": [ + "attributes main-process phases to the matching source and child PTYs", + "embeds schema-v2 revision identity and summarizes the attributed phases", + "invalidates a sample when the actual fixture-unlock IPC write is missing" + ] + }, { "file": "tests/e2e/terminal-split-activation-latency.spec.ts", "assertions": [ "requires a visible BrowserWindow and visible document before sampling", + "records schema-v2 revision identity plus attributed CWD, spawn, bind, fixture-ready, input, and echo phases", "records 3 warmups, then 20 measured real-shortcut cycles after cold inherited-CWD intervals", "requires every split to focus, bind a PTY distinct from its source, and echo immediate input", "observes each closed child PTY exit before starting the next cycle", - "publishes headline latency for a fully successful 3-warmup/20-measured run" + "publishes headline latency only for a fully successful 3-warmup/20-measured run" ] } ], @@ -5875,6 +5907,15 @@ "durationSeconds": 44.43, "summary": "Nine focused files and 96 tests passed. Vitest reported 37.78 seconds and measured wall time was 44.43 seconds; the run adds stable-pane CWD/input handoff, repeated-remount stale-owner fencing, bounded 64-record/15-second retention, seeded-input caps, ordered ordinary/acknowledged/immediate replay, predecessor acknowledged-promise settlement, capture-callback failure containment, and benchmark-artifact write-failure coverage to the existing split, detach, CWD, and local transport contracts." }, + { + "date": "2026-08-31", + "runner": "local", + "platform": "macos", + "result": "passed", + "command": "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/terminal-pane/terminal-pane-split-with-inherited-cwd.test.ts src/renderer/src/lib/pane-manager/pane-split-close.test.ts src/renderer/src/components/terminal-pane/resolve-split-cwd.test.ts src/renderer/src/components/terminal-pane/pty-connection-split-cwd-resolution.test.ts src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts src/renderer/src/components/terminal-pane/pty-preconnect-input-buffer.test.ts src/renderer/src/components/terminal-pane/pty-transport-input-write.test.ts src/renderer/src/components/terminal-pane/deferred-split-pane-handoff.test.ts tests/e2e/terminal-split-activation-latency-artifact.unit.test.ts tests/e2e/terminal-split-activation-latency-main-probe.ts tests/e2e/terminal-split-activation-latency-phases.ts tests/e2e/terminal-split-activation-latency-report.unit.test.ts --reporter=dot", + "durationSeconds": 4.14, + "summary": "The updated twelve-path focused command passed 99 tests (10 runnable test files plus 2 benchmark support modules), including schema-v2 main-process phase attribution, report revision identity, fixture IPC-write validation, stable-pane handoff, ordered pre-connect/live input, cleanup, detach, failure, and same-ID ownership contracts." + }, { "date": "2026-08-30", "runner": "local", @@ -5910,24 +5951,33 @@ "command": "ORCA_TERMINAL_SPLIT_LATENCY_BENCH=1 ORCA_TERMINAL_SPLIT_LATENCY_LABEL=candidate-962faacec8c-headful-run2 ORCA_TERMINAL_SPLIT_LATENCY_OUTPUT=/private/tmp/orca-terminal-split-activation-962faacec8c-headful-run2.json pnpm exec playwright test tests/e2e/terminal-split-activation-latency.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1", "durationSeconds": 72, "summary": "At exact HEAD 962faacec8c, the visible BrowserWindow and document completed 3/3 warmups followed by 20/20 measured cold-CWD cycles with every event present, distinct child PTYs, and observed child exits. Shortcut-to-focus p50/p95/max was 12.8/14.0/14.3 ms, PTY bind was 236.5/654.0/687.3 ms, and first echo was 566.8/1191.5/1219.7 ms. Artifact SHA-256: 4f66b93e5c936ade05f880010f4ec027385d5c89893430169af91c7fdfbe1d06." + }, + { + "date": "2026-08-31", + "runner": "local", + "platform": "macos", + "result": "passed", + "command": "ORCA_TERMINAL_SPLIT_LATENCY_BENCH=1 ORCA_TERMINAL_SPLIT_LATENCY_LABEL=candidate-073e6c7b0eb-headful-clean ORCA_TERMINAL_SPLIT_LATENCY_OUTPUT=/private/tmp/orca-terminal-split-activation-073e6c7b0eb-headful-clean.json pnpm exec playwright test tests/e2e/terminal-split-activation-latency.spec.ts --config tests/playwright.config.ts --project electron-headful --workers=1", + "durationSeconds": 87.6, + "summary": "At exact clean HEAD 073e6c7b0eb1c0ccbb115db1528b641901997c73, the schema-v2 artifact records dirty=false, a visible BrowserWindow/document, and 3/3 warmups plus 20/20 measured cycles with every missing-event counter at zero, distinct child PTYs, and observed child exits. Measured focus p50/p95/max was 12.0/13.6/14.7 ms; attributed CWD lookup was 40/97/103 ms, CWD-settle to spawn request 1/4/4 ms, spawn request to result 48/120/122 ms, and spawn result to bind 2.1/2.5/2.5 ms. Fixture unlock request to IPC write was 0.5/0.8/0.9 ms, IPC write to fixture-ready parse was 35.1/87.4/141.8 ms, and input to first echo was 1.3/3.5/3.9 ms. Total shortcut-to-bind was 113.5/161.3/162.7 ms and shortcut-to-first-echo was 158.5/240.7/291.2 ms. Artifact SHA-256: 1e7ff9e658b717056273d64ecdf662cc6b5776bb6dae1827ed213b7647e4a5fb." } ], - "evidenceProcedure": "Run the benchmark spec in a visible macOS Electron project from the primary worktree, complete 3 warmups followed by 20 measured cold-CWD cycles, save the JSON report, and record its SHA-256. The older baseline/candidate records are historical audit records whose labels do not verify the checkout; the two 2026-08-31 exact-HEAD records use the committed final harness. A paired rerun of the baseline and final product revisions with one committed harness remains required before promotion.", + "evidenceProcedure": "Run the benchmark spec in a visible macOS Electron project from a clean primary worktree, complete 3 warmups followed by 20 measured cold-CWD cycles, require zero missing events and successful cleanup, save the schema-v2 JSON report, and record its SHA-256 plus embedded revision identity. The four older records are schema-v1 historical audit records whose labels do not verify the checkout or include phase attribution; the clean schema-v2 record is the current candidate evidence. A paired baseline/final rerun with one revision-verifying harness remains required before promotion or a readiness comparison claim.", "runtimeBudget": { "p95Seconds": 240, - "scope": "the full listed gate command set: one current nine-file focused unit invocation (eight renderer/local-transport files plus one benchmark-artifact contract), one historical seven-file unit invocation, and four opt-in 3-warmup/20-measured visible Electron benchmark invocations (two historical records plus two exact-HEAD records)" + "scope": "the full listed gate command set: one current twelve-path focused invocation (ten runnable test files plus two benchmark support modules), one historical nine-file unit invocation, one historical seven-file unit invocation, and five opt-in 3-warmup/20-measured visible Electron benchmark invocations (four schema-v1 historical records plus one clean schema-v2 record)" }, "flakeHistory": { "status": "not-started", - "evidence": "The focused promise-barrier, remount-handoff, and benchmark-artifact suite now passes locally in 96 tests; the visible benchmark passes for the historical pair and for two exact-HEAD runs. Routed CI and soak history have not started. The focused unit command runs the checkout from which it is invoked. Historical benchmark labels do not verify the product checkout; the exact-HEAD records use the committed final harness." + "evidence": "The focused promise-barrier, remount-handoff, benchmark-artifact, and schema-v2 attribution suite passes locally in 99 tests; the clean visible benchmark passes 3/3 warmups and 20/20 measured cycles with zero missing events. Its first attempt hit a transient warmup cleanup-dialog click timeout, then the exact command passed on retry without a launch, profile, or port workaround. Routed CI and soak history have not started. Historical benchmark labels do not verify the product checkout or embed revision identity." }, "redGreenEvidence": { "status": "partial", - "evidence": "Before the production seam landed, the split assertion failed with zero manager calls while CWD was pending, and the transport assertion rejected the first pre-connect input. Before the detach fence, a deferred split could be removed after CWD resolved but before PTY bind, dropping its pre-connect input. Before the single-flight hardening, the concurrent-flush oracle delivered ordinary input before the earlier acknowledged write and clear left the in-flight promise pending. Before stale-spawn ownership fencing, the combined deferred-connect and newer same-ID attach fixture called kill on the current PTY. An intentional one-line revert of deferred-CWD cleanup in the stale direct-SSH claim branch failed its focused callback assertion with zero calls instead of one; restoring it passed the prior focused tests. The new remount-handoff, transport, and artifact-write regressions are green in the 96-test nine-file run, but isolated intentional-revert evidence for each cleanup branch remains outstanding." + "evidence": "Before the production seam landed, the split assertion failed with zero manager calls while CWD was pending, and the transport assertion rejected the first pre-connect input. Before the detach fence, a deferred split could be removed after CWD resolved but before PTY bind, dropping its pre-connect input. Before the single-flight hardening, the concurrent-flush oracle delivered ordinary input before the earlier acknowledged write and clear left the in-flight promise pending. Before stale-spawn ownership fencing, the combined deferred-connect and newer same-ID attach fixture called kill on the current PTY. An intentional one-line revert of deferred-CWD cleanup in the stale direct-SSH claim branch failed its focused callback assertion with zero calls instead of one; restoring it passed the prior focused tests. The remount-handoff, transport, artifact-write, and schema-v2 attribution regressions are green in the 99-test twelve-path run, but isolated intentional-revert evidence for each cleanup branch remains outstanding." }, "performanceBudget": { "required": true, - "evidence": "Pane creation and focus add no timer, polling, provider inventory, or subprocess work. CWD resolution remains one existing bounded request off the visible activation path, and detach admission adds only bounded map and record lookups. The remount handoff adds one module-level map lookup per pane lifecycle, a 64-record cap, and a 15-second expiry; it retains no unbounded payload. Pre-connect input, including an in-flight acknowledged write and remount seed replay, is capped at 1,024 entries and a conservative UTF-16 ceiling derived from the existing terminal-input byte limit, drains through one worker in order, and clears on teardown or failed connect. In same-mode visible BrowserWindow runs with 3 warmups and n=20 measured cold-CWD cycles, baseline df14d1a2983d8339e788d0e521f1c4affd9c6d5f versus candidate d453ffcdb704764daced1b2917fddee7224389f0 changed shortcut-to-focus p50/p95/max from 65.7/88.7/102.6 ms to 12.8/14.4/16.5 ms. PTY bind changed from 122.8/154.0/159.7 ms to 177.0/316.1/333.3 ms, and first echo changed from 203.8/264.2/332.7 ms to 268.6/627.4/710.7 ms, so shell readiness regressed in this pair and remains diagnostic rather than part of the activation claim. The n=20 empirical p95 values are descriptive, are not a distribution guarantee, and are not CI-enforced." + "evidence": "Pane creation and focus add no timer, polling, provider inventory, or subprocess work. CWD resolution remains one existing bounded request off the visible activation path, and detach admission adds only bounded map and record lookups. The remount handoff adds one module-level map lookup per pane lifecycle, a 64-record cap, and a 15-second expiry; it retains no unbounded payload. Pre-connect input, including an in-flight acknowledged write and remount seed replay, is capped at 1,024 entries and a conservative UTF-16 ceiling derived from the existing terminal-input byte limit, drains through one worker in order, and clears on teardown or failed connect. The historical schema-v1 same-mode pair recorded shortcut-to-focus p50/p95/max changing from 65.7/88.7/102.6 ms to 12.8/14.4/16.5 ms; those labels do not embed revision identity, so the comparison is directional evidence only. The clean schema-v2 candidate attributes focus at 12.0/13.6/14.7 ms while CWD lookup takes 40/97/103 ms and spawn request-to-result takes 48/120/122 ms, demonstrating that provider/process startup follows activation rather than blocking it. In that clean run, shortcut-to-bind is 113.5/161.3/162.7 ms, fixture IPC-write-to-ready is 35.1/87.4/141.8 ms, and input-to-echo is 1.3/3.5/3.9 ms; these readiness phases are diagnostic, one-host descriptive measurements, and no clean schema-v2 baseline exists to support a readiness improvement or regression claim. The n=20 empirical p95 values are descriptive, are not a distribution guarantee, and are not CI-enforced." }, "promotionCriteria": [ "Record complete red/green evidence for close, remount/rehome handoff, mixed-input ordering, metadata, and failure cleanup.", @@ -5936,12 +5986,12 @@ "Collect physical local-daemon, SSH or WSL plus Linux, Windows, and folder-workspace evidence before claiming provider-complete coverage." ], "knownGaps": [ - "The visible benchmark pair ran on one Apple-silicon macOS host with a synthetic POSIX echo shell and a git-backed workspace; its n=20 empirical p95 is descriptive and not CI-enforced.", + "The clean schema-v2 candidate run and the historical schema-v1 comparison records ran on one Apple-silicon macOS host with a synthetic POSIX echo shell and a git-backed workspace; their n=20 empirical p95 values are descriptive and not CI-enforced.", "No physical local-daemon, SSH, WSL, Linux, Windows, or folder-workspace latency journey has run; the synthetic fixture is currently skipped on Windows because it requires a POSIX shell.", "Remote-runtime split creation remains host-delegated and its transport does not consume the local pre-connect seed/capture options; CWD handoff is covered, but remote-runtime pre-connect input replay has no implementation or evidence.", - "The stored artifact pair predates the final harness cleanup/reporting and exit-generation hardening; immutable hashes preserve artifact identity, while operator labels record only the claimed product revision. Both product revisions should be rerun with one committed harness before promotion.", - "No forced-failure visible benchmark artifact has been recorded; the focused artifact-write contract now verifies that an unwritable output path fails, while invalid-report construction remains unverified by a full visible run.", - "PTY-bind and first-echo readiness regressed in the recorded candidate pair and remain diagnostic rather than blocking the activation-ordering claim." + "The four stored schema-v1 artifacts predate the final harness attribution/reporting and do not embed revision identity; the clean schema-v2 candidate artifact is revision-verified, but both product revisions still need a paired schema-v2 rerun with one committed harness before promotion.", + "No forced-failure visible benchmark artifact has been recorded; the focused artifact-write and missing-event report contracts verify local failure handling, while failure-report serialization remains unverified by a full visible run.", + "No clean schema-v2 baseline phase artifact exists, so the attributed CWD, spawn, fixture-ready, bind, and echo timings diagnose where time is spent but do not establish a shell-readiness improvement or regression." ], "demotionRule": "Keep experimental or demote if pane activation waits on CWD, a deferred split can detach before PTY bind, a remount or rehome loses its stable CWD/input handoff, stale owners mutate a successor record, detached cwd is lost, input reorders or remains pending after cleanup, a closed pane can spawn, stale retirement kills a newer same-ID owner, remote-runtime delegation creates a competing local pane, or the focused suite flakes without an identified product or harness cause." },