starred/orca
1
0
Fork 0
mirror of https://github.com/stablyai/orca.git synced 2026-09-23 00:02:29 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: bound direct SSH reconnect fan-out and recovery (#11003)

Browse Source 
* docs: design for direct SSH reconnect fan-out

Capture the implementation-ready plan for host-qualified, epoch-fenced
SSH reconnect recovery after two rounds of multi-model LLM counsel review.

* docs: reconcile SSH reconnect fan-out design

* docs: close reconnect design consistency gaps

* feat: implement bounded direct SSH reconnect recovery

* fix: bound direct SSH retry settlement

* fix: harden direct SSH reconnect authority

* fix: preserve split SSH retry ownership

* fix: preserve SSH split continuation authority

* docs: record final SSH reconnect validation

* fix: preserve SSH authority through retained and detached state

* fix: retain SSH authority across delayed split mounts

* fix: close SSH authority recovery gaps

* fix: fence stale SSH transport replacement

* fix: serialize SSH target teardown

* fix: settle SSH teardown failures before reconnect

* fix: retire failed SSH reset sessions

* test: reconcile current main E2E contracts

* fix: close direct SSH reconnect review gaps

* fix: fence stale SSH reconnect side effects

* fix: close final SSH reconnect lifecycle gaps

* test: stabilize current-main reliability gates

* test: prove plugin navigation containment

* test: make plugin navigation oracle authoritative

* test: make plugin navigation oracle deterministic

* ci: allow sharded e2e suite to finish

* test: wait for runtime pane publication

* test: classify pane readiness by error code

* test: select close persistence terminal by tab identity

* docs: mark reconnect implementation validated

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
This commit is contained in:
Jinjing
2026-07-28 12:33:17 -07:00
committed by GitHub
co-authored by OrcaWin
parent 6fc05df985
commit a40183389b
125 changed files with 20200 additions and 1228 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/e2e.yml
+1 -1
View File
@@ -82,7 +82,7 @@ jobs:
name: e2e ${{ matrix.shard_name }}
needs: build
runs-on: ubuntu-latest
timeout-minutes: 20
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
config/reliability-gates.jsonc
+398 -24
View File
@@ -253,7 +253,11 @@
"surfaces": [
"terminal input availability",
"worktree discovery",
"Source Control status refresh"
"Source Control status refresh",
"direct SSH detected-worktree scheduling",
"direct SSH reconnect telemetry",
"direct SSH host catalog authority",
"direct SSH generation-scope rollover"
],
"platforms": [
"macos",
@@ -271,14 +275,17 @@
"local",
"ssh"
],
"coverageNotes": "Local deterministic evidence covers git-common classification, desktop watcher debounce counts, non-overlapping poller semantics, macOS native-watch fallback, preload cleanup, and Source Control active-visible repo filtering. Linux/Windows are covered at the shared poller layer by forcing the non-darwin path; live platform runs remain gaps.",
"coverageNotes": "Local deterministic evidence covers git-common classification, desktop watcher debounce counts, non-overlapping poller semantics, macOS native-watch fallback, preload cleanup, Source Control active-visible repo filtering, the direct SSH five-slot fair scheduler, timeout barrier, aggregate privacy schema, coordinator-to-renderer telemetry wiring, host-catalog provenance rejection, and process generation-scope rollover across sibling targets. A macOS Electron client completed a direct SSH disconnect/reconnect against an ephemeral Linux Docker target with exact host/authority hydration and remote proof-file verification. Linux/Windows desktop clients, multi-target live fanout, paired-client, and WSL runs remain gaps.",
"motivatingLinks": [
"https://github.com/stablyai/orca/pull/7086"
],
"invariant": "Index-only Git activity below the common Git directory must not emit worktrees:changed, invalidate worktree caches, or trigger fetchWorktrees fanout; structural add/remove/HEAD/gitdir/locked/config.worktree changes must still refresh worktrees and nudge Source Control; external head moves (commit, amend, reset) must reach background worktree rows through spawn-free metadata reads, never through structural fanout.",
"oracle": "Classify exact git-common paths as structural, status-only, or ignored; count notifications from debounced watcher events; force the Linux/Windows poll path to emit allowlisted leaf events, detect linked HEAD rewrites independent of entry-directory mtime, and surface in-place index rewrites via the backstop re-stat; diff head identities from metadata-file reads and notify only real head moves; assert Source Control subscribes to both structural and status-only signals with active-repo and visibility filters.",
"invariant": "Index-only Git activity below the common Git directory must not emit worktrees:changed, invalidate worktree caches, or trigger fetchWorktrees fanout; structural add/remove/HEAD/gitdir/locked/config.worktree changes must still refresh worktrees and nudge Source Control; external head moves (commit, amend, reset) must reach background worktree rows through spawn-free metadata reads, never through structural fanout. Direct SSH reconnect discovery must stay host- and authority-qualified, reject contradictory main-catalog provenance without returning rows, admit at most five locally unsettled provider calls, retain a retrying timeout barrier, and emit one identifier-free aggregate product event per target operation. A process generation-scope rollover revokes every direct SSH target and old-scope provider request, not only the target whose counter exhausted.",
"oracle": "Classify exact git-common paths as structural, status-only, or ignored; count notifications from debounced watcher events; force the Linux/Windows poll path to emit allowlisted leaf events, detect linked HEAD rewrites independent of entry-directory mtime, and surface in-place index rewrites via the backstop re-stat; diff head identities from metadata-file reads and notify only real head moves; assert Source Control subscribes to both structural and status-only signals with active-repo and visibility filters. For direct SSH, reject catalog rows whose explicit and legacy host provenance contradict, roll one exhausted target into a fresh process generation scope while invalidating sibling target tokens, count locally unsettled attempts and round-robin admissions, keep lineage blocked through the first timeout retry, distinguish timeout/rejection/cancel/stale results, and reject telemetry properties carrying target, repo, host, path, label, user, request, lease, terminal, or raw-error data.",
"commands": [
"pnpm exec vitest run --config config/vitest.config.ts src/main/ipc/worktree-base-directory-event-filter.test.ts src/main/ipc/worktree-base-directory-watcher.test.ts src/main/ipc/worktree-base-directory-poller.test.ts src/main/ipc/worktree-head-identity-reader.test.ts src/renderer/src/hooks/worktree-head-identity-apply.test.ts src/renderer/src/components/right-sidebar/git-status-push-signal-refresh.test.ts src/renderer/src/hooks/useIpcEvents.test.ts"
"pnpm exec vitest run --config config/vitest.config.ts src/main/ipc/worktree-base-directory-event-filter.test.ts src/main/ipc/worktree-base-directory-watcher.test.ts src/main/ipc/worktree-base-directory-poller.test.ts src/main/ipc/worktree-head-identity-reader.test.ts src/renderer/src/hooks/worktree-head-identity-apply.test.ts src/renderer/src/components/right-sidebar/git-status-push-signal-refresh.test.ts src/renderer/src/hooks/useIpcEvents.test.ts",
"pnpm exec vitest run --config config/vitest.config.ts src/main/ipc/repos-remote.test.ts src/main/ssh/ssh-connection-generation.test.ts src/main/ssh/ssh-provider-authority.test.ts --reporter=dot",
"pnpm exec vitest run --config config/vitest.config.ts src/shared/direct-ssh-reconnect-telemetry-schema.test.ts src/renderer/src/lib/direct-ssh-reconnect-product-telemetry.test.ts src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler.test.ts src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts src/renderer/src/hooks/useIpcEvents.test.ts --reporter=dot",
"ORCA_E2E_SSH_DOCKER=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/ssh-docker-relay-perf.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1"
],
"testFiles": [
"src/main/ipc/worktree-base-directory-event-filter.test.ts",
@@ -287,7 +294,15 @@
"src/main/ipc/worktree-head-identity-reader.test.ts",
"src/renderer/src/hooks/worktree-head-identity-apply.test.ts",
"src/renderer/src/components/right-sidebar/git-status-push-signal-refresh.test.ts",
"src/renderer/src/hooks/useIpcEvents.test.ts"
"src/renderer/src/hooks/useIpcEvents.test.ts",
"src/main/ipc/repos-remote.test.ts",
"src/main/ssh/ssh-connection-generation.test.ts",
"src/main/ssh/ssh-provider-authority.test.ts",
"src/shared/direct-ssh-reconnect-telemetry-schema.test.ts",
"src/renderer/src/lib/direct-ssh-reconnect-product-telemetry.test.ts",
"src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler.test.ts",
"src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts",
"tests/e2e/ssh-docker-relay-perf.spec.ts"
],
"assertionRefs": [
{
@@ -343,7 +358,66 @@
{
"file": "src/renderer/src/hooks/useIpcEvents.test.ts",
"assertions": [
"renderer preload API fixtures include the status-metadata and head-identity subscription contracts"
"renderer preload API fixtures include the status-metadata and head-identity subscription contracts",
"direct SSH coordinator telemetry is wired through the fail-soft product adapter"
]
},
{
"file": "src/main/ipc/repos-remote.test.ts",
"assertions": [
"a host-qualified catalog rejects contradictory executionHostId and connectionId provenance without returning rows",
"local, sibling SSH, and runtime rows remain excluded from the exact direct SSH catalog"
]
},
{
"file": "src/main/ssh/ssh-connection-generation.test.ts",
"assertions": [
"one exhausted target rolls the process generation scope and invalidates every sibling target token",
"old-scope mutation expectations fail while the new-scope authority continues rotating"
]
},
{
"file": "src/main/ssh/ssh-provider-authority.test.ts",
"assertions": [
"generation-scope rollover invalidates every target authority before abort callbacks run",
"every registered old-scope provider request aborts exactly once"
]
},
{
"file": "src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler.test.ts",
"assertions": [
"coordinator-owned locally unsettled provider work never exceeds five and target lanes round-robin",
"the first timeout remains retrying and reports queue wait separately from provider execution",
"cancel debt admits at most two replacements and terminally distinguishes budget exhaustion"
]
},
{
"file": "src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts",
"assertions": [
"lineage and token creation remain blocked until a timed-out repo retry settles",
"exact overlapping preparation emits one aggregate with a join count",
"telemetry callback failure cannot affect reconnect completion"
]
},
{
"file": "src/shared/direct-ssh-reconnect-telemetry-schema.test.ts",
"assertions": [
"timeout, rejection, cancellation, and stale outcomes have independent fields",
"target, repo, host, path, label, user, request, lease, terminal, and raw-error fields are rejected"
]
},
{
"file": "src/renderer/src/lib/direct-ssh-reconnect-product-telemetry.test.ts",
"assertions": [
"one coordinator aggregate maps to one typed product event with queue and provider percentiles",
"adapter failure is swallowed before it can reach recovery"
]
},
{
"file": "tests/e2e/ssh-docker-relay-perf.spec.ts",
"assertions": [
"repo and worktree hydration use the exact direct SSH host and complete provider authority",
"terminal input remains live after disconnect/reconnect and writes a proof file visible inside the Linux target"
]
}
],
@@ -356,23 +430,50 @@
"result": "passed",
"durationSeconds": 3.12,
"summary": "7 files and 130 tests passed locally, adding head-identity emit-on-change without structural fanout, reflog status triggers, config.worktree structural classification, the in-place index backstop, and the spawn-free head reader with symref traversal rejection and hex-object-id output validation."
},
{
"date": "2026-07-27",
"runner": "local",
"platform": "macos",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/main/ipc/repos-remote.test.ts src/main/ssh/ssh-connection-generation.test.ts src/main/ssh/ssh-provider-authority.test.ts --reporter=dot",
"result": "passed",
"durationSeconds": 1.61,
"summary": "Three main-process catalog and authority files passed with 118 tests, including contradictory catalog provenance rejection and all-target generation-scope revocation."
},
{
"date": "2026-07-27",
"runner": "local",
"platform": "macos",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/shared/direct-ssh-reconnect-telemetry-schema.test.ts src/renderer/src/lib/direct-ssh-reconnect-product-telemetry.test.ts src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler.test.ts src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts src/renderer/src/hooks/useIpcEvents.test.ts --reporter=dot",
"result": "passed",
"durationSeconds": 3.45,
"summary": "Five focused direct SSH scheduler, coordinator, telemetry, and hook-wiring files passed with 130 tests."
},
{
"date": "2026-07-27",
"runner": "local",
"platform": "macos",
"command": "ORCA_E2E_SSH_DOCKER=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/ssh-docker-relay-perf.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1",
"result": "passed",
"durationSeconds": 66,
"summary": "Four Electron Docker SSH tests passed: two typing/performance paths, one concurrent file/Git load path, and exact-authority disconnect/reconnect with a container-visible remote proof file."
}
],
"runtimeBudget": {
"p95Seconds": 10,
"scope": "focused main/preload/renderer unit and polling tests"
"p95Seconds": 15,
"scope": "focused main/preload/renderer polling and direct SSH scheduler/telemetry tests"
},
"flakeHistory": {
"status": "unknown",
"evidence": "New experimental deterministic gate with one local macOS run; needs CI soak before promotion."
"evidence": "Three deterministic local macOS runs cover the original watcher lane, main catalog/authority lane, and direct SSH scheduler/telemetry lane; CI soak is still unavailable."
},
"redGreenEvidence": {
"status": "partial",
"evidence": "The watcher count assertions fail against the old single-signal classifier because linked index events call notifyWorktreesChanged. Saved CI red/green artifacts are still needed before blocking promotion."
"evidence": "The watcher count assertions fail against the old single-signal classifier because linked index events call notifyWorktreesChanged. The direct SSH tests encode failures for unbounded admission, early lineage release, merged timeout/rejection/cancel/stale results, identifier-bearing telemetry, duplicate joined events, and telemetry exceptions, but no intentional-break artifact was run or claimed; saved red/green artifacts are still needed before blocking promotion."
},
"performanceBudget": {
"required": true,
"evidence": "Index-only bursts produce zero structural notifications, so renderer fetchWorktrees and detected-worktree cache invalidation are not reached. The non-darwin poller stays bounded and non-overlapping, compares HEAD/gitdir/locked signatures every tick, and gates linked index inspection behind the entry-directory signature. A 30-second live Electron run with 2,000 external linked-status calls delivered 50 ordered input chunks and recorded zero Orca-owned git worktree spawns across six diagnostic windows; a locked create/delete positive control still caused structural refreshes."
"evidence": "Index-only bursts produce zero structural notifications, so renderer fetchWorktrees and detected-worktree cache invalidation are not reached. The non-darwin poller stays bounded and non-overlapping. Direct SSH coordinator-owned detected-worktree work is capped at five locally unsettled calls with a two-call late-work allowance; terminal finalization precedes provider discovery, and queue wait and provider execution are reported separately. A 30-second live Electron run with 2,000 external linked-status calls delivered 50 ordered input chunks and recorded zero Orca-owned git worktree spawns across six diagnostic windows; no equivalent live direct SSH fanout benchmark is claimed."
},
"promotionCriteria": [
"Run in soak for at least 100 consecutive passes or 14 days across required CI platforms.",
@@ -383,7 +484,10 @@
"The live Electron diagnostic and screenshot evidence must remain attached to the motivating PR for durable review.",
"Linux and Windows are forced through the shared non-darwin poller in unit tests but are not live-tested here.",
"Git loose ref watching remains outside this incident fix by design.",
"SSH watches classify head-move triggers but skip the metadata-read identity diff; remote background-worktree heads still wait on a structural event or activation."
"SSH watches classify head-move triggers but skip the metadata-read identity diff; remote background-worktree heads still wait on a structural event or activation.",
"The Docker/Linux journey covers one direct SSH target; a live multi-target fanout and large-catalog benchmark remains missing.",
"Paired web clients intentionally do not run the desktop direct SSH coordinator, and paired-close non-interference lacks a new live run.",
"WSL direct SSH fanout remains an explicit live-test gap rather than inferred coverage."
],
"demotionRule": "Keep experimental or demote if the focused gate flakes without a product or harness bug, if index-only churn can emit worktrees:changed, or if structural add/remove/HEAD/lock changes fail to converge."
},
@@ -3567,6 +3671,10 @@
"layer": "provider-contract",
"surfaces": [
"SSH deferred restore",
"direct SSH reconnect finalization",
"direct SSH folder workspace reattach",
"direct SSH split-pane retry ownership",
"same-authority terminal correction",
"remote-runtime mirror polling",
"remote-runtime network recovery",
"terminal create idempotency",
@@ -3591,7 +3699,7 @@
"ssh",
"remote-runtime"
],
"coverageNotes": "Deterministic renderer coverage proves startup publishes the state returned by ssh.connect, stale cleanup cannot unregister a replacement runtime terminal, and a mounted remote-runtime terminal survives repeated transport partitions without changing PTY identity. Client/server heartbeat tests cover timer suspension, socket generations fence stale callbacks, cold restored-terminal attachment retries, cached pixels remain unhealthy until authoritative replay, automatic retries stop after one minute, manual reconnect preserves the PTY, and pane closure releases recovery UI state. Capability-gated create retries adopt a provider-owned PTY by stable terminal identity after an unknown outcome or runtime-process restart, stop retrying after one minute without a fatal error, and remain manually retryable without accepting stale create completions. A macOS Electron journey covers live SSH restore; a Windows remote-runtime smoke covers reachability and PTY round-trip. A live partition journey using patched Mac and Windows builds remains a gap.",
"coverageNotes": "Deterministic renderer coverage proves startup publishes the state returned by ssh.connect, retained native and runtime SSH payloads are admitted through production routes only with valid complete authority, stale cleanup cannot unregister a replacement runtime terminal, direct SSH Git and folder panes clear and retry by exact authority, one authority chain stops after two automatic attempts even when each timeout exceeds the rolling window, rejected acknowledgements mutate no store maps, and one shared exact attempt admits every concurrent split-pane spawn and reattach while preserving the first PTY as the tab fallback. A later sibling failure rotates the tab once, stale callbacks from the prior attempt mutate no state, split remount activity suppression is counted per leaf, primary PTY exit promotes a bound survivor or preserves an empty continuation gap for a late sibling, and primary, non-primary, or null-PTY detach preserves exact authority on both resulting tabs. Intentional pane disposal cancels its settlement timer without breaking StrictMode remount timeout ownership. Target snapshot hydration/reconnect preserves sibling SSH/local/WSL/runtime state, and a mounted remote-runtime terminal survives repeated transport partitions without changing PTY identity. Direct SSH coordinator tests cover immediate terminal finalization, hydration correction, damping, bounded retry, and telemetry non-interference. Client/server heartbeat tests cover timer suspension, socket generations fence stale callbacks, cold restored-terminal attachment retries, cached pixels remain unhealthy until authoritative replay, automatic retries stop after one minute, manual reconnect preserves the PTY, and pane closure releases recovery UI state. Current macOS Electron journeys against an ephemeral Linux Docker SSH target cover exact-authority repo/worktree hydration, live terminal recovery after disconnect/reconnect, and eager six-terminal remount after renderer reload. A Windows remote-runtime smoke covers reachability and PTY round-trip. Multi-target live fanout, paired-close, WSL, and patched live partition journeys remain gaps.",
"motivatingLinks": [
"https://github.com/stablyai/orca/pull/6951",
"https://github.com/stablyai/orca/pull/6955",
@@ -3599,14 +3707,18 @@
"https://github.com/stablyai/orca/pull/7009",
"https://github.com/stablyai/orca/pull/8597"
],
"invariant": "SSH, WSL, and remote-runtime restore paths must treat provider listing failures and unknown liveness as unknown, not dead, while still avoiding duplicate spawn and clearing expired relay leases exactly once. Every restored remote terminal must preserve its provider PTY identity. After a recoverable partition the same authenticated runtime must reattach the same PTY, reject detached input, apply the latest viewport, and report healthy only after authoritative replay. Automatic PTY recovery stops after one bounded minute without a fatal terminal error; a manual reconnect starts a newly fenced epoch against the same PTY, and closed panes retain no recovery UI state. One capability-gated terminal-create mutation must produce at most one host PTY across an unknown response outcome, remain manually retryable after cutoff, and never let a stale completion replace a newer pane lifecycle.",
"oracle": "Deterministic tests cover bounded stale-handle replacement, suspended heartbeat clocks, cold and established subscription failure, ten partition/recovery cycles, automatic-recovery cutoff, and manual reconnect. They assert one unsubscribe per epoch, observable recovery phases, stable PTY identity, resumed snapshot/output/input, no healthy state before replay, no retry or input after cutoff, a new manual epoch against the same PTY, quiet recovery UI with an explicit Reconnect action, pane-close state cleanup, one stable create mutation id, one-minute create-retry cutoff, old-runtime no-retry behavior, authenticated client/worktree isolation, cross-process PTY adoption without rerunning startup, unavailable or legacy-incomplete inventory fail-closed behavior, and bounded in-flight coordination. Existing count tests prove concurrent panes share one in-flight inventory request per runtime/worktree and accepted-snapshot listeners are identity-scoped and released after rebind.",
"invariant": "SSH, WSL, and remote-runtime restore paths must treat provider listing failures and unknown liveness as unknown, not dead, while still avoiding duplicate spawn and clearing expired relay leases exactly once. Direct SSH reconnect must atomically clear only exact-target live PTY bindings, preserve relay identity, retry Git and folder panes without paired close or provider shutdown, and allow at most two automatic attempts in one authority chain even when each settlement exceeds the rolling window. A rejected acknowledgement mutates no store map. A successful exact split-pane spawn or reattach must retain that attempt as shared live authority until sibling leaves settle; the first success cannot consume sibling authority, a sibling failure can start at most one second tab-wide attempt, and prior-attempt callbacks become inert after rotation. Once the retry budget is exhausted, a failure cannot start attempt three or revoke attempt-two authority from siblings that may still settle. Primary PTY exit must promote a bound survivor or preserve exact authority through an empty activation gap, and split detach must project that authority to both resulting tabs. Hydrated PTY hints cannot supersede a current exact-attempt owner, and target snapshot hydration/reconnect cannot reset sibling SSH, local, WSL, or runtime-owned state. Every restored remote terminal must preserve its provider PTY identity. After a recoverable partition the same authenticated runtime must reattach the same PTY, reject detached input, apply the latest viewport, and report healthy only after authoritative replay. Automatic PTY recovery stops after one bounded minute without a fatal terminal error; a manual reconnect starts a newly fenced epoch against the same PTY, and closed panes retain no recovery UI state. One capability-gated terminal-create mutation must produce at most one host PTY across an unknown response outcome, remain manually retryable after cutoff, and never let a stale completion replace a newer pane lifecycle.",
"oracle": "Deterministic tests cover bounded stale-handle replacement, suspended heartbeat clocks, cold and established subscription failure, ten partition/recovery cycles, automatic-recovery cutoff, manual reconnect, and exact direct SSH binding recovery. They assert one atomic store publication clears only exact-target PTY indexes, null-PTY activation remains unchanged, relay identity survives, Git and folder panes retry symmetrically, another target/local/WSL/runtime panes remain byte-identical through target snapshot hydration and reconnect, only an accepted exact failure or timeout starts the second attempt, two 31-second timeouts cannot start a third settlement-triggered attempt, rejected stale/mismatched acknowledgements preserve every store map, and concurrent split-pane spawn and reattach callbacks both commit under the same attempt ID after the first success replaces pending state with live shared authority. A sibling failure revokes that shared authority and starts exactly one second attempt; duplicate failures and late first-attempt PTY callbacks preserve the second attempt and every state map. Attempt-two failure retains continuation authority for later siblings, primary exit promotes a bound survivor or preserves the lease until a late sibling binds, and primary plus non-primary detach retain exact authority and history on both resulting tabs. Both remount callbacks consume split-count activity suppression, intentional dispose emits no failure/timeout, and a same-attempt StrictMode remount still owns one timeout. Hydration clears an untrusted PTY hint without clearing its current pending owner, healthy current-authority bindings suppress correction, hydration finalizes once, and reconnect emits no paired close lifecycle. Tests also assert one unsubscribe per remote-runtime epoch, observable recovery phases, stable PTY identity, resumed snapshot/output/input, no healthy state before replay, no retry or input after cutoff, a new manual epoch against the same PTY, quiet recovery UI with an explicit Reconnect action, pane-close state cleanup, one stable create mutation id, old-runtime no-retry behavior, cross-process PTY adoption, and bounded in-flight coordination.",
"commands": [
"pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/startup/ssh-startup-reconnect.test.ts src/renderer/src/lib/resolved-worktree-execution-host.test.ts src/renderer/src/components/terminal/background-terminal-worktree-mount.test.ts src/renderer/src/runtime/sync-runtime-graph-scheduling.test.ts src/renderer/src/components/terminal-pane/use-terminal-pane-lifecycle.test.ts src/renderer/src/components/terminal-pane/pty-connection.test.ts src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.test.ts src/renderer/src/runtime/remote-runtime-session-tabs-inflight.test.ts src/renderer/src/runtime/web-session-terminal-handle-events.test.ts src/renderer/src/store/slices/terminal-pty-identity-replacement.test.ts",
"pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/terminal-pane/pty-transport.test.ts",
"pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.test.ts src/renderer/src/components/terminal-pane/remote-runtime-pty-recovery-state.test.ts src/renderer/src/components/terminal-pane/TerminalRemoteRuntimeReconnectBanner.test.tsx src/renderer/src/components/terminal-pane/terminal-remote-runtime-recovery-ui-state.test.ts src/shared/remote-runtime-socket-liveness.test.ts src/shared/remote-runtime-shared-control-connection.test.ts src/shared/remote-runtime-shared-control-socket-generation.test.ts src/shared/remote-runtime-client-error-classification.test.ts src/main/runtime/rpc/remote-runtime-server-heartbeat.test.ts src/main/runtime/rpc/methods/terminal-create-idempotency.test.ts src/main/runtime/orca-runtime-terminal-create-idempotency.test.ts",
"pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/store/slices/direct-ssh-terminal-retry.test.ts src/renderer/src/store/slices/direct-ssh-pane-detach-ledger.test.ts src/renderer/src/store/slices/direct-ssh-terminal-recovery.test.ts src/renderer/src/store/slices/direct-ssh-terminal-workspace-scope.test.ts src/renderer/src/store/slices/terminals-hydration.test.ts src/renderer/src/store/slices/repos-ssh-host-reconciliation.test.ts src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts src/renderer/src/hooks/direct-ssh-host-hydration.test.ts src/renderer/src/hooks/direct-ssh-state-routing.test.ts src/renderer/src/hooks/remote-workspace-target-sync.test.ts src/renderer/src/components/terminal-pane/pty-connection.test.ts src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts --reporter=dot",
"pnpm exec vitest run --config config/vitest.config.ts src/main/ipc/repos-remote.test.ts src/main/ipc/ssh.test.ts src/main/ipc/worktrees.test.ts src/main/runtime/public-ssh-state.test.ts src/main/ssh/ssh-connection-manager.test.ts src/main/ssh/ssh-connection.test.ts src/main/ssh/ssh-provider-authority.test.ts src/preload/ssh-authority-forwarding.test.ts src/renderer/src/runtime/runtime-client-events.test.ts src/renderer/src/runtime/runtime-environment-ssh-state.test.ts src/shared/ssh-retained-payload-admission.test.ts src/shared/ssh-types.test.ts --reporter=dot",
"pnpm exec electron-vite build --mode e2e",
"SKIP_BUILD=1 pnpm exec playwright test tests/e2e/terminal-cold-activation-deferral.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1",
"ORCA_E2E_SSH_DOCKER=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/ssh-cold-activation-restore.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1"
"ORCA_E2E_SSH_DOCKER=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/ssh-cold-activation-restore.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1",
"ORCA_E2E_SSH_DOCKER=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/ssh-docker-relay-perf.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1"
],
"testFiles": [
"src/renderer/src/startup/ssh-startup-reconnect.test.ts",
@@ -3615,6 +3727,7 @@
"src/renderer/src/runtime/sync-runtime-graph-scheduling.test.ts",
"src/renderer/src/components/terminal-pane/use-terminal-pane-lifecycle.test.ts",
"src/renderer/src/components/terminal-pane/pty-connection.test.ts",
"src/renderer/src/components/terminal-pane/pty-transport.test.ts",
"src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.test.ts",
"src/renderer/src/components/terminal-pane/remote-runtime-pty-recovery-state.test.ts",
"src/renderer/src/components/terminal-pane/TerminalRemoteRuntimeReconnectBanner.test.tsx",
@@ -3629,8 +3742,32 @@
"src/main/runtime/rpc/remote-runtime-server-heartbeat.test.ts",
"src/main/runtime/rpc/methods/terminal-create-idempotency.test.ts",
"src/main/runtime/orca-runtime-terminal-create-idempotency.test.ts",
"src/renderer/src/store/slices/direct-ssh-terminal-retry.test.ts",
"src/renderer/src/store/slices/direct-ssh-pane-detach-ledger.test.ts",
"src/renderer/src/store/slices/direct-ssh-terminal-recovery.test.ts",
"src/renderer/src/store/slices/direct-ssh-terminal-workspace-scope.test.ts",
"src/renderer/src/store/slices/terminals-hydration.test.ts",
"src/renderer/src/store/slices/repos-ssh-host-reconciliation.test.ts",
"src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts",
"src/renderer/src/hooks/direct-ssh-host-hydration.test.ts",
"src/renderer/src/hooks/direct-ssh-state-routing.test.ts",
"src/renderer/src/hooks/remote-workspace-target-sync.test.ts",
"src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts",
"src/main/ipc/repos-remote.test.ts",
"src/main/ipc/ssh.test.ts",
"src/main/ipc/worktrees.test.ts",
"src/main/runtime/public-ssh-state.test.ts",
"src/main/ssh/ssh-connection-manager.test.ts",
"src/main/ssh/ssh-connection.test.ts",
"src/main/ssh/ssh-provider-authority.test.ts",
"src/preload/ssh-authority-forwarding.test.ts",
"src/renderer/src/runtime/runtime-client-events.test.ts",
"src/renderer/src/runtime/runtime-environment-ssh-state.test.ts",
"src/shared/ssh-retained-payload-admission.test.ts",
"src/shared/ssh-types.test.ts",
"tests/e2e/terminal-cold-activation-deferral.spec.ts",
"tests/e2e/ssh-cold-activation-restore.spec.ts"
"tests/e2e/ssh-cold-activation-restore.spec.ts",
"tests/e2e/ssh-docker-relay-perf.spec.ts"
],
"assertionRefs": [
{
@@ -3720,6 +3857,189 @@
"snapshot-first replacement still migrates stale PTY-indexed state"
]
},
{
"file": "src/renderer/src/store/slices/direct-ssh-terminal-recovery.test.ts",
"assertions": [
"one atomic patch clears exact-target live PTY indexes while preserving relay identity and null-PTY activation",
"another SSH target, local, WSL, floating, and runtime-owned terminal state remains unchanged"
]
},
{
"file": "src/renderer/src/store/slices/direct-ssh-terminal-workspace-scope.test.ts",
"assertions": [
"Git and folder workspaces resolve only from consistent exact-target provenance",
"ambiguous, contradictory, mixed, and runtime-owned folders fail closed"
]
},
{
"file": "src/renderer/src/store/slices/direct-ssh-terminal-retry.test.ts",
"assertions": [
"one authority chain permits at most two automatic attempts even when both timeouts exceed the rolling thirty-second window",
"rejected stale-authority, stale-attempt, or pre-commit success acknowledgements mutate none of the tab, PTY-index, pending, history, or live-binding maps",
"both split-pane siblings bind under one exact attempt while the first PTY remains the tab fallback",
"a sibling failure starts one second tab-wide attempt and stale first-attempt callbacks preserve it",
"an exhausted attempt retains sibling continuation authority and promotes a surviving primary PTY without attempt three",
"primary exit before sibling commit preserves the exact continuation lease and accepts the late sibling"
]
},
{
"file": "src/renderer/src/store/slices/direct-ssh-pane-detach-ledger.test.ts",
"assertions": [
"primary and non-primary split detach preserve exact live authority and retry history on both resulting tabs",
"detaching the only bound split while its sibling is still spawning preserves the source continuation lease until that sibling binds",
"detaching during a null-PTY continuation gap projects the exact lease to both pending tabs",
"a pending-only all-null detach preserves the exact lease on both tabs before either leaf binds",
"same-authority invalidation and correction leave both detached live PTYs unchanged"
]
},
{
"file": "src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts",
"assertions": [
"a detached null-PTY leaf remains marked for pending activation before ownership transfer"
]
},
{
"file": "src/main/ipc/repos-remote.test.ts",
"assertions": [
"host-qualified repo catalogs require one consistent execution host and complete current SSH authority",
"contradictory, partial, mismatched, stale, and runtime-owned catalog requests fail closed"
]
},
{
"file": "src/main/ipc/ssh.test.ts",
"assertions": [
"concurrent same-authority connects share one provider attempt",
"authority rotation starts stale transport cancellation before teardown, concurrent fresh callers share one replacement, and stale completion cannot clobber the fresh session",
"same-turn disconnect and forward-teardown failures across removal, reset, and terminate keep replacement connects and metadata mutation behind complete target cleanup"
]
},
{
"file": "src/main/ssh/ssh-connection-manager.test.ts",
"assertions": [
"disconnect invalidates a pending transport attempt immediately so late rejection or resolution cannot remove the replacement"
]
},
{
"file": "src/main/ssh/ssh-connection.test.ts",
"assertions": [
"late ssh2 ready and startup error events after disconnect cannot resurrect or overwrite disconnected state"
]
},
{
"file": "src/main/ipc/worktrees.test.ts",
"assertions": [
"host-qualified worktree reads reject malformed or contradictory repo executionHostId/connectionId provenance before provider access and after provider awaits without durable mutations",
"host-qualified lineage excludes other SSH and runtime owners and rejects ambiguous or contradictory provenance",
"one lineage request snapshots repo, folder, and group catalogs once and memoizes repeated owner resolution"
]
},
{
"file": "src/main/runtime/public-ssh-state.test.ts",
"assertions": [
"public SSH state preserves the complete provider epoch and connection generation pair"
]
},
{
"file": "src/main/ssh/ssh-provider-authority.test.ts",
"assertions": [
"provider epoch and connection generation rotate as one exact authority pair",
"provider resolution rejects stale or incomplete authority",
"unknown-target currency probes reject without allocating provider authority state"
]
},
{
"file": "src/preload/ssh-authority-forwarding.test.ts",
"assertions": [
"full authority crosses Electron IPC without loss",
"partial authority becomes unknown for bounded reconciliation and malformed full authority is dropped",
"variable-form host-qualified worktree requests retain fail-closed outcomes in their return type"
]
},
{
"file": "src/shared/ssh-retained-payload-admission.test.ts",
"assertions": [
"retained connection states reject partial or malformed authority",
"only partial compatibility authority can normalize to unknown for bounded direct-SSH reconciliation",
"shared direct SSH authority admission requires bounded identifiers and a non-negative safe generation"
]
},
{
"file": "src/renderer/src/runtime/runtime-client-events.test.ts",
"assertions": [
"retained runtime snapshots and live events preserve the full pair",
"partial runtime authority is rejected before it reaches environment state"
]
},
{
"file": "src/renderer/src/runtime/runtime-environment-ssh-state.test.ts",
"assertions": [
"runtime-owned SSH state remains isolated by environment and rejects partial retained authority",
"in-flight hydration cannot resurrect disconnected or removed runtime environments"
]
},
{
"file": "src/shared/ssh-types.test.ts",
"assertions": [
"SSH connection state carries the provider epoch and connection generation authority pair"
]
},
{
"file": "src/renderer/src/store/slices/terminals-hydration.test.ts",
"assertions": [
"target-scoped hydration and reconnect preserve sibling SSH and runtime tabs, PTY indexes, runtime ownership, and active selection",
"authoritative target-tab deletion prunes only that tab's retry, live-binding, and retry-history ledgers",
"a snapshot PTY from another SSH host is rejected from the target scope"
]
},
{
"file": "src/renderer/src/store/slices/repos-ssh-host-reconciliation.test.ts",
"assertions": [
"a provider result becomes stale when same-ID repo ownership turns malformed or contradictory during the await"
]
},
{
"file": "src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts",
"assertions": [
"terminal invalidation and retry run synchronously before provider preparation",
"hydrated terminal finalization and same-authority correction are current-authority fenced",
"rapid authority rotation keeps immediate terminal checks while damping full preparation"
]
},
{
"file": "src/renderer/src/hooks/direct-ssh-host-hydration.test.ts",
"assertions": [
"exact-host catalog and lineage hydration preserves sibling SSH, local, runtime, ambiguous, and contradictory rows"
]
},
{
"file": "src/renderer/src/hooks/remote-workspace-target-sync.test.ts",
"assertions": [
"snapshot hydration preserves newer local recovery and keeps imported PTY ids retryable until exact-attempt transport acknowledgement",
"stale operation tokens cannot apply an older snapshot over current authority",
"target snapshot projection and persisted-terminal reconnect are host-qualified and preserve sibling SSH, local, WSL, and runtime state"
]
},
{
"file": "src/renderer/src/components/terminal-pane/pty-connection.test.ts",
"assertions": [
"StrictMode remounts join only the same direct SSH retry attempt",
"authority rotation starts a new spawn and rejects then retires a late obsolete-authority fresh PTY",
"late stale rebind and reattach completions, including lease replacement during asynchronous SSH preparation, callback errors, rejected promises, session-expired, empty, and launch-metadata outcomes, cannot clear current state, start replacement recovery, publish errors, or publish metadata",
"both concurrent split-pane spawns commit through the same exact retry attempt",
"both concurrent split-pane reattaches commit through the same exact retry attempt",
"a sibling mounted after first success captures the retained live lease",
"authority rotation rejects and retires a delayed sibling spawned from a retained live lease",
"intentional pane disposal cancels retry settlement while a same-attempt StrictMode remount retains one timeout"
]
},
{
"file": "src/renderer/src/components/terminal-pane/pty-transport.test.ts",
"assertions": [
"admission rejection precedes buffered final-frame and exit publication",
"abandoning an obsolete reattach drops its data, replay, write-unavailable, and exit handlers without killing the durable PTY",
"a rejected or destroyed fresh session fallback settles retirement before it can publish handlers, shutdown refusal is reported as unknown, and reattach remains non-destructive"
]
},
{
"file": "tests/e2e/ssh-cold-activation-restore.spec.ts",
"assertions": [
@@ -3727,6 +4047,13 @@
"all six SSH managers mount eagerly and none is parked",
"restored terminal input reaches a proof file on the Linux SSH host"
]
},
{
"file": "tests/e2e/ssh-docker-relay-perf.spec.ts",
"assertions": [
"repo and worktree setup fails closed unless the exact direct SSH host and complete authority are returned",
"a reconnected terminal accepts input and writes a proof file visible inside the Linux SSH target"
]
}
],
"evidenceRuns": [
@@ -3756,23 +4083,68 @@
"result": "passed",
"durationSeconds": 5,
"summary": "Eleven fault-injection and recovery-UI files and 127 tests passed, covering suspended heartbeat clocks, stale socket, PTY, and create generations, canonical pre-ready close recovery with one replacement subscription, cold and repeated PTY reattachment, authoritative health, bounded PTY and terminal-create recovery, post-probe timeout clipping, manually retryable create cutoff, accurate capability-probe failures, same-PTY manual reconnect, pane-state cleanup, fatal error deduplication, stable create identity, cross-process PTY adoption, and fail-closed legacy inventory."
},
{
"date": "2026-07-28",
"runner": "local",
"platform": "macos",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/store/slices/direct-ssh-terminal-retry.test.ts src/renderer/src/store/slices/direct-ssh-pane-detach-ledger.test.ts src/renderer/src/store/slices/direct-ssh-terminal-recovery.test.ts src/renderer/src/store/slices/direct-ssh-terminal-workspace-scope.test.ts src/renderer/src/store/slices/terminals-hydration.test.ts src/renderer/src/store/slices/repos-ssh-host-reconciliation.test.ts src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts src/renderer/src/hooks/direct-ssh-host-hydration.test.ts src/renderer/src/hooks/direct-ssh-state-routing.test.ts src/renderer/src/hooks/remote-workspace-target-sync.test.ts src/renderer/src/components/terminal-pane/pty-connection.test.ts src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts --reporter=dot",
"result": "passed",
"durationSeconds": 15.8,
"summary": "Twelve direct SSH files and 647 tests passed, including exact lease revalidation after asynchronous SSH preparation, primary-exit continuation gaps, pending-only and live null-PTY two-sided split-detach authority, delayed post-success sibling admission, stale-authority provider retirement, late ownership-provenance rejection, and deleted-tab ledger pruning."
},
{
"date": "2026-07-28",
"runner": "local",
"platform": "macos",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/terminal-pane/pty-transport.test.ts",
"result": "passed",
"durationSeconds": 1.48,
"summary": "All 90 transport tests passed, including pre-publication admission rejection, handler-complete non-destructive detach for obsolete SSH reattach transports, settled retirement of rejected or destroyed fresh fallbacks, and reported shutdown refusal."
},
{
"date": "2026-07-28",
"runner": "local",
"platform": "macos",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/main/ipc/repos-remote.test.ts src/main/ipc/ssh.test.ts src/main/ipc/worktrees.test.ts src/main/runtime/public-ssh-state.test.ts src/main/ssh/ssh-connection-manager.test.ts src/main/ssh/ssh-connection.test.ts src/main/ssh/ssh-provider-authority.test.ts src/preload/ssh-authority-forwarding.test.ts src/renderer/src/runtime/runtime-client-events.test.ts src/renderer/src/runtime/runtime-environment-ssh-state.test.ts src/shared/ssh-retained-payload-admission.test.ts src/shared/ssh-types.test.ts --reporter=dot",
"result": "passed",
"durationSeconds": 2.55,
"summary": "Twelve main, preload, runtime, and shared authority files and 513 tests passed, including fail-closed pre/post-await repo ownership provenance, production retained-payload admission, fenced stale-transport replacement, failure-safe target lifecycle barriers, and real ssh2 late-ready/error rejection."
},
{
"date": "2026-07-28",
"runner": "local",
"platform": "macos",
"command": "ORCA_E2E_SSH_DOCKER=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/ssh-docker-relay-perf.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1",
"result": "passed",
"durationSeconds": 51.4,
"summary": "Four Electron Docker SSH tests passed on the final implementation, including exact-authority repo/worktree hydration, two concurrent immutable file streams under Git churn, live terminal input before and after disconnect/reconnect, and an independent container-visible remote proof file."
},
{
"date": "2026-07-28",
"runner": "local",
"platform": "macos",
"command": "ORCA_E2E_SSH_DOCKER=1 SKIP_BUILD=1 pnpm exec playwright test tests/e2e/ssh-cold-activation-restore.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1",
"result": "passed",
"durationSeconds": 12.5,
"summary": "One Electron journey passed on the final implementation after exact-authority hydration; six restored SSH terminal managers remounted after renderer reload and remote input reached the Linux target."
}
],
"runtimeBudget": {
"p95Seconds": 45,
"scope": "provider contract plus optional SSH soak"
"p95Seconds": 150,
"scope": "all configured provider-contract, build, Electron, Docker SSH reconnect, and six-terminal cold-restore commands"
},
"flakeHistory": {
"status": "unknown",
"evidence": "Focused renderer contracts and one clean Docker/Linux SSH Electron run pass locally; the live journey needs CI soak history before promotion."
"evidence": "Focused renderer contracts, one current 627-test direct SSH run, and current Docker/Linux reconnect and six-terminal cold-restore journeys pass locally. One pressure run entered reconnect while waiting for its seventh marker; later runs delivered all markers but exposed an independent disappearing-second-file fixture race. The corrected two-reader single-file load passed the complete 4/4 suite while retaining concurrent stream pressure. Live multi-target fanout still needs CI soak history before promotion."
},
"redGreenEvidence": {
"status": "partial",
"evidence": "The remote-runtime fault tests failed before the recovery changes by leaving a cold restored subscription detached, reporting connected before authoritative replay, delivering a fatal setup error twice, and allowing an unknown create outcome to spawn again after process-local state was lost. The bounded-recovery tests additionally failed before the policy change because PTY and terminal-create recovery remained active after one minute and a retry reused the stale epoch. Final review tests failed before lifecycle fencing because create cutoff emitted a fatal error with no manual path and a delayed create completion replaced a newer cross-runtime attachment. The fixed tests pass with resumed snapshot/output/input, one-minute cutoffs, new manual epochs, capability re-probing, stale-create rejection, and cross-process provider PTY adoption without rerunning startup. Existing SSH and stale-handle reattach coverage remains green. Needs WSL, a patched live remote-runtime partition journey, and saved intentional-break artifacts before promotion."
"evidence": "The remote-runtime fault tests failed before the recovery changes by leaving a cold restored subscription detached, reporting connected before authoritative replay, delivering a fatal setup error twice, and allowing an unknown create outcome to spawn again after process-local state was lost. On exact pre-fix HEAD 939719443, the split-pane store, fresh-spawn, and reattach oracles failed because the first success removed pending authority and the sibling could not bind; the same three oracles pass after live bindings retain the exact attempt ID. On exact committed HEAD d501f2e96, primary-exit-before-sibling and primary/non-primary split-detach oracles failed because live authority was deleted during the empty gap or transferred to only one resulting tab; all three pass after continuation-gap preservation and two-sided detach projection. On exact committed HEAD d44ea382b with test-only oracles, a pending-only all-null detach deleted the attempt ledger, a sibling mounted after first success committed without its lease, and that sibling could bind after authority rotation; all three pass after pending-only projection and retained-live-lease capture with provider retirement. On exact committed HEAD 9fa84dacf, the production-manager in-progress oracle rejected the fresh authority and the forward-removal barrier delayed stale transport cancellation; both pass after replacement starts cancellation immediately, shares concurrent fresh callers, and waits for teardown before connecting. On exact committed HEAD e5ba9a9e5, overlapping disconnect allowed a replacement connect before forward teardown completed and delayed transport cancellation behind that barrier; disconnect, removal, and terminate now share a target lifecycle barrier, start transport cancellation immediately, retain captured-session identity, and admit the replacement only after cleanup. On exact committed HEAD 9a29e7a81, a rejected forward teardown short-circuited the lifecycle while transport disconnect was pending, removal left its captured relay session alive, same-turn connect escaped admission, and reset remained outside the target barrier; the exact failure oracles pass after both cleanup branches settle, captured sessions always retire, admission is authority-fenced, and reset shares the barrier. On exact committed HEAD 47c7198f2, reset's remaining bespoke forward teardown could still reject after authority rotation but before captured-session retirement; reset now uses the same hardened session teardown and a failed reset remains cleanly retryable. Other direct SSH tests encode red conditions for non-atomic binding clear, cross-target retry, folder omission, duplicate same-authority attempts, hydration overwrite, terminal finalization delayed behind provider work, obsolete-authority pending-spawn adoption, Git lineage namespace mismatch, and snapshot PTY hint promotion without exact-attempt acknowledgement. Existing SSH and stale-handle reattach coverage remains green. Needs WSL, paired-close, and a patched live remote-runtime partition journey before promotion."
},
"performanceBudget": {
"required": true,
"evidence": "Remote-runtime recovery allocates at most one backoff timer and one one-minute deadline per detached pane, then stops all PTY retry work until explicit user action; regular PTYs and initial creates do not poll inventory. Each unknown-outcome create attempt performs one bounded provider inventory scan, coordinated per authenticated client/worktree mutation, and the renderer stops issuing attempts after one minute. Timers, accepted-snapshot listeners, stale streams, and pane UI entries are released on health, cutoff, rebind, removal, detach, or destroy; ten-cycle tests prove one unsubscribe per epoch and cutoff tests prove request counts stay fixed for five additional minutes. Client and server liveness each use one interval per socket/transport. At most 4,096 create promises are retained only while in flight, and capacity rejection happens before spawning. Common terminal input/output paths add only constant-time state checks; recovery UI updates only on deduplicated phase transitions."
"evidence": "Direct SSH terminal invalidation and retry each use one exact-target store publication and execute before provider discovery; another target's five occupied provider slots cannot delay terminal finalization. Each split-pane completion or delayed mount adds constant-time pending/live lease lookups and no provider listing, polling, subprocess, cross-tab scan, or new fanout; two mounted leaves still perform exactly their two existing provider operations. The scheduler caps locally unsettled detected-worktree work at five with a two-call late-work allowance. Remote-runtime recovery allocates at most one backoff timer and one one-minute deadline per detached pane, then stops all PTY retry work until explicit user action. Timers, accepted-snapshot listeners, stale streams, and pane UI entries are released on health, cutoff, rebind, removal, detach, or destroy; ten-cycle tests prove one unsubscribe per epoch. Common terminal input/output paths add only constant-time state checks. No live large-terminal-map direct SSH timing is claimed."
},
"promotionCriteria": [
"Use deterministic fake providers for failure and unknown-liveness cases.",
@@ -3782,7 +4154,9 @@
"knownGaps": [
"Current command covers store wake-hint metadata, main-process SSH provider failure semantics, provider attach/expired-attach behavior, and renderer deferred SSH reconnect/transient-failure/expired-relay fallback with mocked transports.",
"The live SSH journey is environment-dependent and currently runs from a macOS Electron client against a Linux Docker host.",
"WSL restore remains inferred rather than directly covered.",
"Current Docker/Linux journeys prove one target's reconnect and cold-restore paths; live multi-target fanout, folder-workspace reconnect, and large-terminal-map timing remain untested.",
"No paired-client close/non-interference journey was run for direct SSH reconnect; paired web clients intentionally remain outside coordinator ownership.",
"WSL restore and direct SSH/WSL isolation remain inferred rather than directly covered.",
"Linux and Windows desktop-client partition journeys using patched builds are not yet collected; the Windows smoke proves current reachability and PTY round-trip only.",
"Terminal-create recovery depends on providers authoritatively listing live terminal handles and worktree ownership; older runtimes do not advertise the capability and are never retried after an unknown outcome."
],
docs/reference/plans/2026-07-27-ssh-reconnect-fanout.md
+1009
View File
File diff suppressed because it is too large Load Diff
examples/plugins/hostile-panel/panel.html
+10
View File
@@ -31,6 +31,12 @@
results.appendChild(li)
document.title = 'probes:' + results.children.length
}
function markNavigationInvocation(name) {
var marker = document.createElement('meta')
marker.dataset.navigationProbeInvoked = name
marker.content = 'true'
document.head.appendChild(marker)
}
// Probe 1: fetch() exfiltration — must be blocked by connect-src 'none'.
var cookieValue = 'x'
@@ -73,6 +79,7 @@
// Navigation probes are opt-in so a failed attempt cannot erase the
// network and bridge evidence before the harness observes it.
document.getElementById('navigate-top').addEventListener('click', function () {
markNavigationInvocation('top-navigation')
try {
window.top.location.href = 'https://example.com/'
setTimeout(function () {
@@ -84,6 +91,7 @@
})
document.getElementById('navigate-self').addEventListener('click', function () {
markNavigationInvocation('self-navigation')
try {
window.location.href = 'https://example.com/self-navigation'
setTimeout(function () {
@@ -95,6 +103,7 @@
})
document.getElementById('navigate-anchor-form').addEventListener('click', function () {
markNavigationInvocation('anchor-form-navigation')
var anchor = document.createElement('a')
anchor.href = 'https://example.com/anchor-navigation'
anchor.textContent = 'Navigation probe'
@@ -111,6 +120,7 @@
})
document.getElementById('navigate-meta').addEventListener('click', function () {
markNavigationInvocation('meta-refresh-navigation')
var refresh = document.createElement('meta')
refresh.httpEquiv = 'refresh'
refresh.content = '0;url=https://example.com/meta-refresh'
src/main/ipc/repos-remote.test.ts
+129
View File
@@ -149,9 +149,16 @@ vi.mock('./ssh', () => ({
import { registerRepoHandlers } from './repos'
import { clearSubmodulePathsCacheForTests, listSubmodulePaths } from '../git/status'
import { toSshExecutionHostId } from '../../shared/execution-host'
import {
getSshProviderAuthority,
resetSshProviderAuthorities,
rotateSshProviderAuthority
} from '../ssh/ssh-provider-authority'
beforeEach(() => {
clearGitCapabilityStateForTests()
resetSshProviderAuthorities()
})
describe('projectGroups IPC validation', () => {
@@ -210,6 +217,128 @@ describe('projectGroups IPC validation', () => {
expect(mockStore.createProjectGroup).not.toHaveBeenCalled()
})
it('returns an immutable repo catalog for exactly one execution host', async () => {
const localRepo = {
id: 'duplicate',
path: '/local/repo',
displayName: 'local',
badgeColor: '#000',
addedAt: 0
}
const sshRepo = {
id: 'duplicate',
path: '/remote/repo',
displayName: 'remote',
badgeColor: '#000',
addedAt: 0,
connectionId: 'conn-1'
}
const runtimeRepo = {
id: 'duplicate',
path: '/runtime/repo',
displayName: 'runtime',
badgeColor: '#000',
addedAt: 0,
executionHostId: 'runtime:environment-a'
}
mockStore.getRepos.mockReturnValue([localRepo, sshRepo, runtimeRepo])
await expect(
handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: toSshExecutionHostId('conn-1'),
expectedAuthority: getSshProviderAuthority('conn-1')
})
).resolves.toMatchObject({
authoritative: true,
authority: {
kind: 'direct-ssh',
executionHostId: 'ssh:conn-1',
targetId: 'conn-1'
},
repos: [sshRepo]
})
const local = await handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: 'local'
})
expect(local).toMatchObject({ authoritative: true, repos: [localRepo] })
expect((local as { repos: object[] }).repos[0]).not.toBe(localRepo)
})
it('rejects repo catalogs whose execution host contradicts their SSH connection', async () => {
const baseRepo = {
id: 'repo-1',
path: '/repo',
displayName: 'repo',
badgeColor: '#000',
addedAt: 0
}
mockStore.getRepos.mockReturnValue([
{
...baseRepo,
connectionId: 'conn-1',
executionHostId: 'local'
}
])
await expect(
handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: 'local'
})
).resolves.toMatchObject({ authoritative: false, reason: 'rejected' })
await expect(
handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: toSshExecutionHostId('conn-1'),
expectedAuthority: getSshProviderAuthority('conn-1')
})
).resolves.toMatchObject({ authoritative: false, reason: 'rejected' })
mockStore.getRepos.mockReturnValue([
{
...baseRepo,
connectionId: 'conn-1',
executionHostId: toSshExecutionHostId('conn-2')
}
])
await expect(
handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: toSshExecutionHostId('conn-1'),
expectedAuthority: getSshProviderAuthority('conn-1')
})
).resolves.toMatchObject({ authoritative: false, reason: 'rejected' })
})
it('rejects runtime, partial, mismatched, and stale catalog authority', async () => {
await expect(
handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: 'runtime:environment-a'
})
).resolves.toMatchObject({ authoritative: false, reason: 'rejected' })
await expect(
handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: toSshExecutionHostId('conn-1')
})
).resolves.toMatchObject({ authoritative: false, reason: 'rejected' })
await expect(
handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: toSshExecutionHostId('conn-1'),
expectedAuthority: {
...getSshProviderAuthority('other-target'),
targetId: 'other-target'
}
})
).resolves.toMatchObject({ authoritative: false, reason: 'rejected' })
const expectedAuthority = getSshProviderAuthority('conn-1')
const pending = handlers.get('repos:listForExecutionHost')!(null, {
executionHostId: toSshExecutionHostId('conn-1'),
expectedAuthority
})
rotateSshProviderAuthority('conn-1')
await expect(pending).resolves.toMatchObject({ authoritative: false, reason: 'stale' })
})
it('rejects malformed local project group update arguments before persistence', () => {
expect(() =>
handlers.get('projectGroups:update')!(null, {
src/main/ipc/repos.ts
+110
View File
@@ -83,11 +83,16 @@ import { track } from '../telemetry/client'
import { scheduleCurrentWorktreeBaseDirectoryWatcherSync } from './worktree-base-directory-watcher'
import { getCohortAtEmit } from '../telemetry/cohort-classifier'
import type { RepoMethod } from '../../shared/telemetry-events'
import type {
HostRepoCatalogSnapshot,
ListReposForExecutionHostArgs
} from '../../shared/host-repo-catalog-contract'
import { detectRepoIconAndUpstream } from '../repo-icon-autodetect'
import { enrichMissingRepoGitRemoteIdentities } from '../repo-git-remote-identity-enrichment'
import { getProjectHostSetupForRepo } from '../../shared/project-host-setup-projection'
import {
getRepoExecutionHostId,
LOCAL_EXECUTION_HOST_ID,
normalizeExecutionHostId,
parseExecutionHostId,
type ExecutionHostId
@@ -101,6 +106,8 @@ import {
import { getGitCloneFailureMessage } from '../../shared/git-clone-failure-message'
import { prepareLocalWorktreeRootForRepo } from '../worktree-root-preparation'
import { runWithGitReadCacheInvalidation } from '../git/status'
import { isAdmissibleDirectSshAuthority } from '../../shared/ssh-retained-payload-admission'
import { isCurrentSshProviderAuthority } from '../ssh/ssh-provider-authority'
// Why: `method` is the IPC entry point the user took, not what they added (never path/URL/name); repos:create → 'folder_picker'.
// Why: `isGitRepo` is a non-identifying git-vs-folder signal from the caller's detection; pass undefined when unknown, never default false.
@@ -119,6 +126,102 @@ function emitRepoAdded(method: RepoMethod, alreadyExisted: boolean, isGitRepo?:
track('repo_added', props)
}
function hasValidCatalogSshAuthority(
args: ListReposForExecutionHostArgs
): args is Extract<ListReposForExecutionHostArgs, { expectedAuthority: unknown }> {
if (!('expectedAuthority' in args)) {
return false
}
return isAdmissibleDirectSshAuthority(args.expectedAuthority)
}
function repoHostContradictsConnection(repo: Repo): boolean {
if (!repo.executionHostId || !repo.connectionId) {
return false
}
const explicitHost = parseExecutionHostId(repo.executionHostId)
return explicitHost?.kind !== 'ssh' || explicitHost.targetId !== repo.connectionId
}
function getConsistentRepoCatalogForHost(
repos: readonly Repo[],
host: NonNullable<ReturnType<typeof parseExecutionHostId>>
): Repo[] | null {
const hasContradiction = repos.some(
(repo) =>
repoHostContradictsConnection(repo) &&
(getRepoExecutionHostId(repo) === host.id ||
(host.kind === 'ssh' && repo.connectionId === host.targetId))
)
return hasContradiction ? null : repos.filter((repo) => getRepoExecutionHostId(repo) === host.id)
}
async function listReposForExecutionHost(
store: Store,
args: ListReposForExecutionHostArgs
): Promise<HostRepoCatalogSnapshot> {
const parsedHost = parseExecutionHostId(args?.executionHostId)
const rejected = (
reason: Extract<HostRepoCatalogSnapshot, { authoritative: false }>['reason']
): HostRepoCatalogSnapshot => ({
authoritative: false,
executionHostId: args.executionHostId,
reason
})
if (!parsedHost || parsedHost.kind === 'runtime') {
return rejected('rejected')
}
if (parsedHost.kind === 'local') {
if ('expectedAuthority' in args) {
return rejected('rejected')
}
const repos = getConsistentRepoCatalogForHost(store.getRepos(), parsedHost)
if (!repos) {
return rejected('rejected')
}
return {
authoritative: true,
authority: { kind: 'local', executionHostId: LOCAL_EXECUTION_HOST_ID },
repos: structuredClone(repos)
}
}
if (
!hasValidCatalogSshAuthority(args) ||
args.expectedAuthority.targetId !== parsedHost.targetId
) {
return rejected('rejected')
}
const authority = { ...args.expectedAuthority }
if (!isCurrentSshProviderAuthority(authority)) {
return rejected('stale')
}
const provider = getSshGitProvider(parsedHost.targetId)
if (!provider) {
return rejected('unavailable')
}
const matchingRepos = getConsistentRepoCatalogForHost(store.getRepos(), parsedHost)
if (!matchingRepos) {
return rejected('rejected')
}
const repos = structuredClone(matchingRepos)
await Promise.resolve()
if (
getSshGitProvider(parsedHost.targetId) !== provider ||
!isCurrentSshProviderAuthority(authority)
) {
return rejected('stale')
}
return {
authoritative: true,
authority: {
kind: 'direct-ssh',
executionHostId: parsedHost.id,
...authority
},
repos
}
}
function buildProjectHostSetupResult(store: Store, repo: Repo): ProjectHostSetupResult {
const setup = getProjectHostSetupForRepo(store.getProjectHostSetups(), repo)
const project = store.getProjects().find((entry) => entry.id === setup.projectId)
@@ -1119,6 +1222,7 @@ async function runNestedRepoScanForIpc(
export function registerRepoHandlers(mainWindow: BrowserWindow, store: Store): void {
// Remove previously registered handlers so we can re-register on macOS app re-activation (new window).
ipcMain.removeHandler('repos:list')
ipcMain.removeHandler('repos:listForExecutionHost')
ipcMain.removeHandler('repos:add')
ipcMain.removeHandler('repos:remove')
ipcMain.removeHandler('repos:removeForHost')
@@ -1175,6 +1279,12 @@ export function registerRepoHandlers(mainWindow: BrowserWindow, store: Store): v
return store.getRepos()
})
ipcMain.handle(
'repos:listForExecutionHost',
(_event, args: ListReposForExecutionHostArgs): Promise<HostRepoCatalogSnapshot> =>
listReposForExecutionHost(store, args)
)
ipcMain.handle('projects:list', () => {
enrichMissingRepoGitRemoteIdentities(store, {
onChanged: () => notifyReposChanged(mainWindow)
src/main/ipc/ssh.test.ts
+479 -42
View File
@@ -15,6 +15,7 @@ const {
mockPtyProvider,
mockFsProvider,
mockGitProvider,
mockRegisterSshGitProvider,
mockPortForwardManager,
mockPortScannerCallbacks,
mockNextConnectionManagers,
@@ -68,6 +69,7 @@ const {
},
mockFsProvider: {},
mockGitProvider: {},
mockRegisterSshGitProvider: vi.fn(),
mockPortForwardManager: {
addForward: vi.fn(),
updateForward: vi.fn(),
@@ -182,7 +184,7 @@ vi.mock('../providers/ssh-git-provider', () => ({
}))
vi.mock('../providers/ssh-git-dispatch', () => ({
registerSshGitProvider: vi.fn(),
registerSshGitProvider: mockRegisterSshGitProvider,
unregisterSshGitProvider: vi.fn()
}))
@@ -335,6 +337,7 @@ describe('SSH IPC handlers', () => {
mockPtyProvider.onReplay.mockReset()
mockPtyProvider.attachForReconnect.mockReset().mockResolvedValue({})
mockPtyProvider.shutdown.mockReset()
mockRegisterSshGitProvider.mockReset()
mockPortForwardManager.addForward.mockReset()
mockPortForwardManager.updateForward.mockReset()
mockPortForwardManager.removeForward.mockReset()
@@ -504,6 +507,45 @@ describe('SSH IPC handlers', () => {
expect(mockConnectionManager.connect).toHaveBeenCalledWith(target)
})
it('registers the provider before broadcasting connected authority', async () => {
const target: SshTarget = {
id: 'ssh-1',
label: 'Server',
host: 'example.com',
port: 22,
username: 'deploy'
}
mockSshStore.getTarget.mockReturnValue(target)
mockConnectionManager.connect.mockResolvedValue({})
mockConnectionManager.getState.mockReturnValue({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0
})
await handlers.get('ssh:connect')!(null, { targetId: 'ssh-1' })
const connectedIndex = mockWindow.webContents.send.mock.calls.findIndex(
([channel, payload]) =>
channel === 'ssh:state-changed' &&
(payload as { state: SshConnectionState }).state.status === 'connected'
)
expect(connectedIndex).toBeGreaterThanOrEqual(0)
expect(mockRegisterSshGitProvider.mock.invocationCallOrder[0]).toBeLessThan(
mockWindow.webContents.send.mock.invocationCallOrder[connectedIndex]
)
expect(mockWindow.webContents.send.mock.calls[connectedIndex]?.[1]).toEqual({
targetId: 'ssh-1',
state: expect.objectContaining({
targetId: 'ssh-1',
status: 'connected',
providerEpoch: expect.any(String),
connectionGeneration: 1
})
})
})
it('ssh:connect exposes the detected remote platform in public state', async () => {
const target: SshTarget = {
id: 'ssh-1',
@@ -539,6 +581,7 @@ describe('SSH IPC handlers', () => {
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: expect.any(String),
connectionGeneration: 1,
remotePlatform: 'win32'
})
@@ -549,6 +592,7 @@ describe('SSH IPC handlers', () => {
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: expect.any(String),
connectionGeneration: 1,
supportsFolderDownload: true,
remotePlatform: 'win32'
@@ -584,45 +628,46 @@ describe('SSH IPC handlers', () => {
onDispose?.('connection_lost')
expect(mockWindow.webContents.send).toHaveBeenCalledWith('ssh:state-changed', {
targetId: 'ssh-1',
state: {
targetId: 'ssh-1',
status: 'reconnecting',
error: 'Relay channel lost. Reconnecting...',
reconnectAttempt: 1,
connectionGeneration: 1
}
})
expect(handlers.get('ssh:getState')!(null, { targetId: 'ssh-1' })).toEqual({
const reconnectingState = handlers.get('ssh:getState')!(null, {
targetId: 'ssh-1'
}) as SshConnectionState
expect(reconnectingState).toEqual({
targetId: 'ssh-1',
status: 'reconnecting',
error: 'Relay channel lost. Reconnecting...',
reconnectAttempt: 1,
connectionGeneration: 1
providerEpoch: expect.any(String),
connectionGeneration: 2
})
expect(mockWindow.webContents.send).toHaveBeenCalledWith('ssh:state-changed', {
targetId: 'ssh-1',
state: reconnectingState
})
await vi.advanceTimersByTimeAsync(500)
expect(mockWindow.webContents.send).toHaveBeenCalledWith('ssh:state-changed', {
targetId: 'ssh-1',
state: {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
connectionGeneration: 1,
supportsFolderDownload: true
}
})
expect(handlers.get('ssh:getState')!(null, { targetId: 'ssh-1' })).toEqual({
const connectedState = handlers.get('ssh:getState')!(null, {
targetId: 'ssh-1'
}) as SshConnectionState
expect(connectedState).toEqual({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
connectionGeneration: 1
providerEpoch: reconnectingState.providerEpoch,
connectionGeneration: reconnectingState.connectionGeneration
})
expect(() => assertSshMutationExpectation('ssh-1', 'ssh-1', 1)).not.toThrow()
expect(mockWindow.webContents.send).toHaveBeenCalledWith('ssh:state-changed', {
targetId: 'ssh-1',
state: {
...connectedState,
supportsFolderDownload: true
}
})
expect(() => assertSshMutationExpectation('ssh-1', 'ssh-1', 1)).toThrow(
'SSH connection changed; refresh and try again'
)
expect(() => assertSshMutationExpectation('ssh-1', 'ssh-1', 2)).not.toThrow()
} finally {
vi.useRealTimers()
}
@@ -677,6 +722,7 @@ describe('SSH IPC handlers', () => {
status: 'reconnecting',
error: 'Relay channel reconnecting...',
reconnectAttempt: 0,
providerEpoch: expect.any(String),
connectionGeneration: 2
})
expect(() => assertSshMutationExpectation('ssh-1', 'ssh-1', stagedGeneration)).toThrow(
@@ -866,7 +912,8 @@ describe('SSH IPC handlers', () => {
status: 'reconnecting',
error: 'Relay channel lost. Reconnecting...',
reconnectAttempt: 1,
connectionGeneration: 1
providerEpoch: expect.any(String),
connectionGeneration: 2
})
mockDeployAndLaunchRelay.mockClear()
@@ -877,7 +924,8 @@ describe('SSH IPC handlers', () => {
status: 'connected',
error: null,
reconnectAttempt: 0,
connectionGeneration: 2
providerEpoch: expect.any(String),
connectionGeneration: 3
})
expect(mockPortForwardManager.removeAllForwards).toHaveBeenCalledWith('ssh-1')
@@ -887,7 +935,8 @@ describe('SSH IPC handlers', () => {
status: 'connected',
error: null,
reconnectAttempt: 0,
connectionGeneration: 2
providerEpoch: expect.any(String),
connectionGeneration: 3
})
} finally {
vi.useRealTimers()
@@ -918,7 +967,7 @@ describe('SSH IPC handlers', () => {
try {
await handlers.get('ssh:connect')!(null, { targetId: 'ssh-1' })
for (const delayMs of relayReconnectDelaysMs) {
for (const [index, delayMs] of relayReconnectDelaysMs.entries()) {
useSlowRelayLaunchOnce(relayLostStabilizedMs + 1)
getLatestRelayDisposeCallback()('connection_lost')
await vi.advanceTimersByTimeAsync(delayMs + relayLostStabilizedMs + 1)
@@ -927,7 +976,8 @@ describe('SSH IPC handlers', () => {
status: 'connected',
error: null,
reconnectAttempt: 0,
connectionGeneration: 1
providerEpoch: expect.any(String),
connectionGeneration: index + 2
})
}
@@ -938,7 +988,8 @@ describe('SSH IPC handlers', () => {
status: 'error',
error: 'Relay channel kept dropping. Click Reconnect on the SSH target before retrying.',
reconnectAttempt: 0,
connectionGeneration: 1
providerEpoch: expect.any(String),
connectionGeneration: relayReconnectDelaysMs.length + 2
})
} finally {
vi.useRealTimers()
@@ -976,7 +1027,8 @@ describe('SSH IPC handlers', () => {
status: 'connected',
error: null,
reconnectAttempt: 0,
connectionGeneration: 1
providerEpoch: expect.any(String),
connectionGeneration: 2
})
await vi.advanceTimersByTimeAsync(relayLostStabilizedMs + 1)
@@ -988,7 +1040,8 @@ describe('SSH IPC handlers', () => {
status: 'connected',
error: null,
reconnectAttempt: 0,
connectionGeneration: 1
providerEpoch: expect.any(String),
connectionGeneration: 2
})
expect(mockPortForwardManager.removeAllForwards).not.toHaveBeenCalled()
expect(mockDeployAndLaunchRelay).not.toHaveBeenCalled()
@@ -1195,6 +1248,7 @@ describe('SSH IPC handlers', () => {
await expect(handlers.get('ssh:connect')!(null, { targetId: 'ssh-1' })).resolves.toEqual({
...connectedState,
providerEpoch: expect.any(String),
connectionGeneration: 1
})
expect(mockDeployAndLaunchRelay).not.toHaveBeenCalled()
@@ -1396,6 +1450,7 @@ describe('SSH IPC handlers', () => {
status: 'error',
error: 'network down',
reconnectAttempt: 0,
providerEpoch: expect.any(String),
connectionGeneration: 1
}
})
@@ -1451,6 +1506,30 @@ describe('SSH IPC handlers', () => {
expect(mockConnectionManager.disconnect).toHaveBeenCalledWith('ssh-1')
})
it('lets a same-turn disconnect invalidate connect before transport admission', async () => {
const target: SshTarget = {
id: 'ssh-1',
label: 'Server',
host: 'example.com',
port: 22,
username: 'deploy'
}
mockSshStore.getTarget.mockReturnValue(target)
mockConnectionManager.connect.mockResolvedValue({})
mockConnectionManager.disconnect.mockResolvedValue(undefined)
const connect = handlers.get('ssh:connect')!(null, {
targetId: 'ssh-1'
}) as Promise<SshConnectionState>
const disconnect = handlers.get('ssh:disconnect')!(null, {
targetId: 'ssh-1'
}) as Promise<void>
await disconnect
await expect(connect).rejects.toThrow('SSH connection attempt was cancelled')
expect(mockConnectionManager.connect).not.toHaveBeenCalled()
})
it('invalidates a pending connect when disconnect wins and allows a fresh connect', async () => {
const target: SshTarget = {
id: 'ssh-1',
@@ -1462,11 +1541,30 @@ describe('SSH IPC handlers', () => {
const staleConn = {}
const freshConn = {}
let resolveStaleConnect!: (connection: unknown) => void
let resolveForwardRemoval!: () => void
let transportConnectPending = false
mockSshStore.getTarget.mockReturnValue(target)
mockConnectionManager.connect.mockReturnValueOnce(
new Promise((resolve) => {
resolveStaleConnect = resolve
mockConnectionManager.connect
.mockReturnValueOnce(
new Promise((resolve) => {
transportConnectPending = true
resolveStaleConnect = resolve
})
)
.mockImplementationOnce(async () => {
if (transportConnectPending) {
throw new Error('Connection to Server is already in progress')
}
return freshConn
})
mockConnectionManager.disconnect.mockImplementationOnce(async () => {
transportConnectPending = false
})
mockPortForwardManager.removeAllForwards.mockImplementationOnce(
() =>
new Promise<void>((resolve) => {
resolveForwardRemoval = resolve
})
)
mockConnectionManager.getState.mockReturnValue({
targetId: 'ssh-1',
@@ -1480,17 +1578,224 @@ describe('SSH IPC handlers', () => {
}) as Promise<SshConnectionState>
await vi.waitFor(() => expect(mockConnectionManager.connect).toHaveBeenCalledTimes(1))
await handlers.get('ssh:disconnect')!(null, { targetId: 'ssh-1' })
mockConnectionManager.connect.mockResolvedValueOnce(freshConn)
const disconnect = handlers.get('ssh:disconnect')!(null, {
targetId: 'ssh-1'
}) as Promise<void>
await vi.waitFor(() => expect(mockConnectionManager.disconnect).toHaveBeenCalledWith('ssh-1'))
const freshConnect = handlers.get('ssh:connect')!(null, {
targetId: 'ssh-1'
}) as Promise<SshConnectionState>
resolveStaleConnect(staleConn)
await expect(staleConnect).rejects.toThrow('SSH connection attempt was cancelled')
expect(mockConnectionManager.connect).toHaveBeenCalledTimes(1)
resolveForwardRemoval()
await disconnect
await vi.waitFor(() => expect(mockConnectionManager.connect).toHaveBeenCalledTimes(2))
resolveStaleConnect(staleConn)
await expect(freshConnect).resolves.toMatchObject({ targetId: 'ssh-1', status: 'connected' })
expect(mockDeployAndLaunchRelay).toHaveBeenCalledTimes(1)
})
it('keeps reconnect behind transport disconnect when forward teardown fails', async () => {
const target: SshTarget = {
id: 'ssh-1',
label: 'Server',
host: 'example.com',
port: 22,
username: 'deploy'
}
let resolveTransportDisconnect!: () => void
let transportDisconnectPending = false
mockSshStore.getTarget.mockReturnValue(target)
mockConnectionManager.connect.mockResolvedValueOnce({}).mockImplementationOnce(async () => {
if (transportDisconnectPending) {
throw new Error('Connection to Server is already in progress')
}
return {}
})
mockConnectionManager.disconnect.mockImplementationOnce(
() =>
new Promise<void>((resolve) => {
transportDisconnectPending = true
resolveTransportDisconnect = () => {
transportDisconnectPending = false
resolve()
}
})
)
mockConnectionManager.getState.mockReturnValue({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0
})
await handlers.get('ssh:connect')!(null, { targetId: 'ssh-1' })
mockPortForwardManager.removeAllForwards.mockRejectedValueOnce(
new Error('forward teardown failed')
)
const disconnect = handlers.get('ssh:disconnect')!(null, {
targetId: 'ssh-1'
}) as Promise<void>
const disconnectSettled = vi.fn()
void disconnect.then(disconnectSettled, disconnectSettled)
await vi.waitFor(() =>
expect(mockPortForwardManager.removeAllForwards).toHaveBeenCalledWith('ssh-1')
)
const reconnect = handlers.get('ssh:connect')!(null, {
targetId: 'ssh-1'
}) as Promise<SshConnectionState>
const reconnectResult = reconnect.then(
(state) => ({ ok: true as const, state }),
(error: unknown) => ({ ok: false as const, error })
)
await Promise.resolve()
expect(disconnectSettled).not.toHaveBeenCalled()
expect(mockConnectionManager.connect).toHaveBeenCalledTimes(1)
resolveTransportDisconnect()
await expect(disconnect).rejects.toThrow('forward teardown failed')
await expect(reconnectResult).resolves.toMatchObject({
ok: true,
state: { targetId: 'ssh-1', status: 'connected' }
})
expect(mockConnectionManager.connect).toHaveBeenCalledTimes(2)
expect(mockMux.dispose).toHaveBeenCalledWith('connection_lost')
})
it('retires a removed target session after forward teardown fails', async () => {
const target: SshTarget = {
id: 'ssh-1',
label: 'Server',
host: 'example.com',
port: 22,
username: 'deploy'
}
let resolveTransportDisconnect!: () => void
mockSshStore.getTarget.mockReturnValue(target)
mockConnectionManager.connect.mockResolvedValue({})
mockConnectionManager.disconnect.mockImplementationOnce(
() =>
new Promise<void>((resolve) => {
resolveTransportDisconnect = resolve
})
)
mockConnectionManager.getState.mockReturnValue({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0
})
await handlers.get('ssh:connect')!(null, { targetId: 'ssh-1' })
mockPortForwardManager.removeAllForwards.mockRejectedValueOnce(
new Error('forward teardown failed')
)
const removal = handlers.get('ssh:removeTarget')!(null, {
id: 'ssh-1'
}) as Promise<void>
await vi.waitFor(() =>
expect(mockPortForwardManager.removeAllForwards).toHaveBeenCalledWith('ssh-1')
)
await Promise.resolve()
expect(mockSshStore.removeTarget).not.toHaveBeenCalled()
resolveTransportDisconnect()
await removal
expect(mockMux.dispose).toHaveBeenCalledWith('shutdown')
expect(mockStore.removeSshRemotePtyLeases).toHaveBeenCalledWith('ssh-1')
expect(mockSshStore.removeTarget).toHaveBeenCalledWith('ssh-1')
})
it('replaces a stale shared connect after authority rotates without disconnect', async () => {
const target: SshTarget = {
id: 'ssh-1',
label: 'Server',
host: 'example.com',
port: 22,
username: 'deploy'
}
let resolveStaleConnect!: (connection: unknown) => void
let resolveForwardRemoval!: () => void
let resolveTransportDisconnect!: () => void
let transportConnectPending = false
mockSshStore.getTarget.mockReturnValue(target)
mockSshStore.addTarget.mockReturnValue(target)
mockConnectionManager.connect
.mockReturnValueOnce(
new Promise((resolve) => {
transportConnectPending = true
resolveStaleConnect = resolve
})
)
.mockImplementationOnce(async () => {
if (transportConnectPending) {
throw new Error('Connection to Server is already in progress')
}
return {}
})
mockConnectionManager.disconnect.mockImplementationOnce(
() =>
new Promise<void>((resolve) => {
resolveTransportDisconnect = () => {
transportConnectPending = false
resolve()
}
})
)
mockPortForwardManager.removeAllForwards.mockImplementationOnce(
() =>
new Promise<void>((resolve) => {
resolveForwardRemoval = resolve
})
)
mockConnectionManager.getState.mockReturnValue({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0
})
const staleConnect = handlers.get('ssh:connect')!(null, {
targetId: 'ssh-1'
}) as Promise<SshConnectionState>
const sharedStaleConnect = handlers.get('ssh:connect')!(null, {
targetId: 'ssh-1'
}) as Promise<SshConnectionState>
await vi.waitFor(() => expect(mockConnectionManager.connect).toHaveBeenCalledTimes(1))
mockSshStore.lastRepoReadoptions = [
{ oldTargetId: 'ssh-1', newTargetId: 'ssh-new', repoIds: ['repo-1'] }
]
await handlers.get('ssh:addTarget')!(null, { target })
const freshConnect = handlers.get('ssh:connect')!(null, {
targetId: 'ssh-1'
}) as Promise<SshConnectionState>
await vi.waitFor(() =>
expect(mockPortForwardManager.removeAllForwards).toHaveBeenCalledWith('ssh-1')
)
await vi.waitFor(() => expect(mockConnectionManager.disconnect).toHaveBeenCalledWith('ssh-1'))
const sharedFreshConnect = handlers.get('ssh:connect')!(null, {
targetId: 'ssh-1'
}) as Promise<SshConnectionState>
expect(mockConnectionManager.connect).toHaveBeenCalledTimes(1)
resolveForwardRemoval()
resolveTransportDisconnect()
await vi.waitFor(() => expect(mockConnectionManager.connect).toHaveBeenCalledTimes(2))
resolveStaleConnect({})
await expect(staleConnect).rejects.toThrow('SSH connection attempt was cancelled')
await expect(sharedStaleConnect).rejects.toThrow('SSH connection attempt was cancelled')
await expect(freshConnect).resolves.toMatchObject({ targetId: 'ssh-1', status: 'connected' })
await expect(sharedFreshConnect).resolves.toMatchObject({
targetId: 'ssh-1',
status: 'connected'
})
expect(mockDeployAndLaunchRelay).toHaveBeenCalledTimes(1)
})
@@ -1568,6 +1873,61 @@ describe('SSH IPC handlers', () => {
expect(mockStore.markSshRemotePtyLease).toHaveBeenCalledWith('ssh-1', 'pty-lease', 'terminated')
})
it('keeps reconnect behind the complete terminate-sessions lifecycle', async () => {
const target: SshTarget = {
id: 'ssh-1',
label: 'Server',
host: 'example.com',
port: 22,
username: 'deploy'
}
let resolveShutdown!: () => void
let resolveForwardRemoval!: () => void
mockSshStore.getTarget.mockReturnValue(target)
mockConnectionManager.connect.mockResolvedValue({})
mockConnectionManager.getState.mockReturnValue({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0
})
mockStore.getSshRemotePtyLeases.mockReturnValue([
{ targetId: 'ssh-1', ptyId: 'pty-1', state: 'detached' }
])
vi.mocked(getSshPtyProvider).mockReturnValue(mockPtyProvider as never)
vi.mocked(getPtyIdsForConnection).mockReturnValue([])
mockPtyProvider.shutdown.mockReturnValueOnce(
new Promise<void>((resolve) => {
resolveShutdown = resolve
})
)
mockPortForwardManager.removeAllForwards.mockImplementationOnce(
() =>
new Promise<void>((resolve) => {
resolveForwardRemoval = resolve
})
)
await handlers.get('ssh:connect')!(null, { targetId: 'ssh-1' })
const terminate = handlers.get('ssh:terminateSessions')!(null, {
targetId: 'ssh-1'
}) as Promise<void>
await vi.waitFor(() => expect(mockPtyProvider.shutdown).toHaveBeenCalledOnce())
const reconnect = handlers.get('ssh:connect')!(null, {
targetId: 'ssh-1'
}) as Promise<SshConnectionState>
expect(mockConnectionManager.connect).toHaveBeenCalledTimes(1)
resolveShutdown()
await vi.waitFor(() => expect(mockConnectionManager.disconnect).toHaveBeenCalledWith('ssh-1'))
expect(mockConnectionManager.connect).toHaveBeenCalledTimes(1)
resolveForwardRemoval()
await terminate
await vi.waitFor(() => expect(mockConnectionManager.connect).toHaveBeenCalledTimes(2))
await expect(reconnect).resolves.toMatchObject({ targetId: 'ssh-1', status: 'connected' })
})
it('ssh:terminateSessions ignores expired leases when disconnected', async () => {
mockStore.getSshRemotePtyLeases.mockReturnValue([
{ targetId: 'ssh-1', ptyId: 'pty-expired', state: 'expired' }
@@ -1641,6 +2001,43 @@ describe('SSH IPC handlers', () => {
expect(mockConnectionManager.disconnect).toHaveBeenCalledWith('ssh-1')
})
it('retires the captured session when reset forward teardown fails', async () => {
const target: SshTarget = {
id: 'ssh-1',
label: 'Server',
host: 'example.com',
port: 22,
username: 'deploy'
}
const conn = {}
mockSshStore.getTarget.mockReturnValue(target)
mockConnectionManager.connect.mockResolvedValue(conn)
mockConnectionManager.getConnection.mockReturnValue(conn)
mockConnectionManager.getState.mockReturnValue({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0
})
await handlers.get('ssh:connect')!(null, { targetId: 'ssh-1' })
mockPortForwardManager.removeAllForwards.mockRejectedValueOnce(
new Error('forward teardown failed')
)
await expect(handlers.get('ssh:resetRelay')!(null, { targetId: 'ssh-1' })).rejects.toThrow(
'forward teardown failed'
)
expect(mockMux.dispose).toHaveBeenCalledWith('connection_lost')
expect(mockForceStopRelayForTarget).not.toHaveBeenCalled()
await handlers.get('ssh:resetRelay')!(null, { targetId: 'ssh-1' })
expect(mockPortForwardManager.removeAllForwards).toHaveBeenCalledTimes(1)
expect(mockForceStopRelayForTarget).toHaveBeenCalledWith(conn, 'ssh-1')
expect(mockConnectionManager.disconnect).toHaveBeenCalledWith('ssh-1')
})
it('ssh:resetRelay waits for an in-flight connect before tearing down the session', async () => {
const target: SshTarget = {
id: 'ssh-1',
@@ -1776,6 +2173,42 @@ describe('SSH IPC handlers', () => {
expect(mockConnectionManager.disconnect).toHaveBeenCalledWith('ssh-1')
})
it('keeps removal behind an in-flight relay reset', async () => {
const target: SshTarget = {
id: 'ssh-1',
label: 'Server',
host: 'example.com',
port: 22,
username: 'deploy'
}
let resolveForceStop!: () => void
mockSshStore.getTarget.mockReturnValue(target)
mockConnectionManager.getConnection.mockReturnValue({})
mockConnectionManager.disconnect.mockResolvedValue(undefined)
mockForceStopRelayForTarget.mockReturnValueOnce(
new Promise<void>((resolve) => {
resolveForceStop = resolve
})
)
const reset = handlers.get('ssh:resetRelay')!(null, {
targetId: 'ssh-1'
}) as Promise<void>
await vi.waitFor(() => expect(mockForceStopRelayForTarget).toHaveBeenCalledOnce())
const removal = handlers.get('ssh:removeTarget')!(null, {
id: 'ssh-1'
}) as Promise<void>
await Promise.resolve()
expect(mockSshStore.removeTarget).not.toHaveBeenCalled()
resolveForceStop()
await reset
await removal
expect(mockConnectionManager.disconnect).toHaveBeenCalledTimes(2)
expect(mockSshStore.removeTarget).toHaveBeenCalledWith('ssh-1')
})
it('reconnects on system resume when the relay liveness probe fails', async () => {
const target: SshTarget = {
id: 'ssh-1',
@@ -1974,6 +2407,10 @@ describe('SSH IPC handlers', () => {
mockConnectionManager.getState.mockReturnValue(state)
const result = await handlers.get('ssh:getState')!(null, { targetId: 'ssh-1' })
expect(result).toEqual({ ...state, connectionGeneration: 0 })
expect(result).toEqual({
...state,
providerEpoch: expect.any(String),
connectionGeneration: 0
})
})
})
src/main/ipc/ssh.ts
+233 -127
View File
@@ -15,7 +15,8 @@ import type {
SshRepoReadoption,
SshTarget,
SshConnectionStatus,
SshConnectionState
SshConnectionState,
DirectSshAuthority
} from '../../shared/ssh-types'
import { SSH_TERMINATE_RECONNECT_REQUIRED } from '../../shared/constants'
import { isRuntimeOwnedSshTargetId } from '../../shared/execution-host'
@@ -40,11 +41,15 @@ import {
} from './pty'
import type { OrcaRuntimeService } from '../runtime/orca-runtime'
import {
advanceSshConnectionGeneration,
getSshConnectionGeneration,
initializeSshConnectionGenerationSession,
resetSshConnectionGenerations
} from '../ssh/ssh-connection-generation'
import {
getSshProviderAuthority,
isCurrentSshProviderAuthority,
resetSshProviderAuthorities,
rotateSshProviderAuthority
} from '../ssh/ssh-provider-authority'
let sshStore: SshConnectionStore | null = null
let connectionManager: SshConnectionManager | null = null
@@ -108,34 +113,35 @@ export function listRegisteredRemovedSshTargetLabels(): Record<string, string> {
export async function disconnectRegisteredSshTarget(targetId: string): Promise<void> {
invalidateConnectAttempt(targetId)
if (!connectionManager) {
return
}
await detachActiveSshSession(targetId)
await connectionManager.disconnect(targetId)
await runTargetLifecycle(targetId, () =>
teardownSshTargetTransport(targetId, (session) => session.detach())
)
}
export async function removeRegisteredSshTarget(targetId: string): Promise<void> {
if (!sshStore) {
return
}
const store = sshStore
invalidateConnectAttempt(targetId)
// Why: removal is destructive; dispose so remote PTYs cannot reattach to a deleted target.
await disposeActiveSshSession(targetId)
try {
await connectionManager?.disconnect(targetId)
} catch (err) {
// Why: a failed disconnect must not block metadata removal, else the target lingers in the store with uncleaned leases.
console.warn(
`[ssh] Failed to disconnect removed target ${targetId}: ${err instanceof Error ? err.message : String(err)}`
)
}
persistedStore?.removeSshRemotePtyLeases(targetId)
sshStore.removeTarget(targetId)
await runTargetLifecycle(targetId, async () => {
try {
// Why: removal is destructive; dispose so remote PTYs cannot reattach to a deleted target.
await teardownSshTargetTransport(targetId, (session) => session.dispose())
} catch (err) {
// Why: a failed disconnect must not block metadata removal, else the target lingers in the store with uncleaned leases.
console.warn(
`[ssh] Failed to disconnect removed target ${targetId}: ${err instanceof Error ? err.message : String(err)}`
)
}
persistedStore?.removeSshRemotePtyLeases(targetId)
store.removeTarget(targetId)
})
}
// One session per SSH target owns the whole relay lifecycle (mux, providers, abort controller, state machine).
const activeSessions = new Map<string, SshRelaySession>()
const targetLifecycleInFlight = new Map<string, Promise<void>>()
export function getActiveSshAiVaultHostInfo(targetId: string): SshRelayAiVaultHostInfo | null {
if (isRuntimeOwnedSshTargetId(targetId)) {
@@ -154,12 +160,61 @@ export function getActiveSshAiVaultHostInfos(): SshRelayAiVaultHostInfo[] {
})
}
async function detachActiveSshSession(targetId: string): Promise<void> {
await teardownActiveSshSession(targetId, (session) => session.detach())
function runTargetLifecycle(targetId: string, operation: () => Promise<void>): Promise<void> {
const prior = targetLifecycleInFlight.get(targetId)
const operationPromise = (async () => {
if (prior) {
await prior.catch(() => undefined)
}
await operation()
})()
let trackedPromise!: Promise<void>
trackedPromise = operationPromise.finally(() => {
if (targetLifecycleInFlight.get(targetId) === trackedPromise) {
targetLifecycleInFlight.delete(targetId)
}
})
targetLifecycleInFlight.set(targetId, trackedPromise)
return trackedPromise
}
async function disposeActiveSshSession(targetId: string): Promise<void> {
await teardownActiveSshSession(targetId, (session) => session.dispose())
async function awaitTargetLifecycle(targetId: string): Promise<void> {
while (true) {
const lifecycle = targetLifecycleInFlight.get(targetId)
if (!lifecycle) {
return
}
await lifecycle.catch(() => undefined)
}
}
async function teardownSshTargetTransport(
targetId: string,
teardown: (session: SshRelaySession) => void
): Promise<void> {
let transportDisconnect: Promise<{ ok: true } | { ok: false; error: unknown }>
try {
transportDisconnect = Promise.resolve(connectionManager?.disconnect(targetId)).then(
() => ({ ok: true }) as const,
(error: unknown) => ({ ok: false, error }) as const
)
} catch (error) {
transportDisconnect = Promise.resolve({ ok: false, error })
}
const sessionTeardown = teardownActiveSshSession(targetId, teardown).then(
() => ({ ok: true }) as const,
(error: unknown) => ({ ok: false, error }) as const
)
const [disconnectResult, teardownResult] = await Promise.all([
transportDisconnect,
sessionTeardown
])
if (!teardownResult.ok) {
throw teardownResult.error
}
if (!disconnectResult.ok) {
throw disconnectResult.error
}
}
async function teardownActiveSshSession(
@@ -170,12 +225,26 @@ async function teardownActiveSshSession(
if (!session) {
return
}
// Why: await port teardown so local listeners are released before disconnect/remove completes, else an immediate reconnect hits EADDRINUSE.
await portForwardManager?.removeAllForwards(targetId)
teardown(session)
activeSessions.delete(targetId)
clearRelayLostBackoff(targetId)
clearRelayStateOverride(targetId)
let teardownError: { error: unknown } | null = null
try {
// Why: await port teardown so local listeners are released before disconnect/remove completes, else an immediate reconnect hits EADDRINUSE.
await portForwardManager?.removeAllForwards(targetId)
} catch (error) {
teardownError = { error }
}
try {
teardown(session)
} catch (error) {
teardownError ??= { error }
}
if (activeSessions.get(targetId) === session) {
activeSessions.delete(targetId)
clearRelayLostBackoff(targetId)
clearRelayStateOverride(targetId)
}
if (teardownError) {
throw teardownError.error
}
}
function relayGracePeriodForTarget(target: SshTarget | null | undefined): number | undefined {
@@ -185,25 +254,22 @@ function relayGracePeriodForTarget(target: SshTarget | null | undefined): number
// Why: tabs must share one connect, while a disconnect must invalidate that
// attempt so its late continuation cannot clobber a replacement.
type ConnectAttempt = {
generation: number
authority: DirectSshAuthority
promise: Promise<SshConnectionState>
}
const connectInFlight = new Map<string, ConnectAttempt>()
const pendingTransportReconnects = new Set<string>()
function currentConnectGeneration(targetId: string): number {
return getSshConnectionGeneration(targetId)
}
function invalidateConnectAttempt(targetId: string): void {
advanceSshConnectionGeneration(targetId)
rotateSshProviderAuthority(targetId)
pendingTransportReconnects.delete(targetId)
connectInFlight.delete(targetId)
credentialRequestedForTarget.delete(targetId)
}
function isCurrentConnectAttempt(targetId: string, generation: number): boolean {
return currentConnectGeneration(targetId) === generation
function isCurrentConnectAttempt(targetId: string, authority: DirectSshAuthority): boolean {
return authority.targetId === targetId && isCurrentSshProviderAuthority(authority)
}
function connectCancelledError(): Error {
@@ -262,9 +328,12 @@ function broadcastSshState(
function withSshRemotePlatform(targetId: string, state: SshConnectionState): SshConnectionState {
const remotePlatform = activeSessions.get(targetId)?.getHostPlatform()?.os
const authority = getSshProviderAuthority(targetId)
return {
...state,
connectionGeneration: currentConnectGeneration(targetId),
targetId,
providerEpoch: authority.providerEpoch,
connectionGeneration: authority.connectionGeneration,
...(remotePlatform ? { remotePlatform } : {})
}
}
@@ -517,10 +586,12 @@ function createSshConnectionCallbacks(): SshConnectionCallbacks {
// Why: an SSH reconnect must re-deploy the relay and rebuild providers; the guard below fires only for real reconnects, not an explicit connect's 'deploying'.
const session = activeSessions.get(targetId)
const sessionState = session?.getState()
if (
const transportReconnectStarted =
state.status === 'reconnecting' &&
(sessionState === 'ready' || sessionState === 'reconnecting')
) {
(sessionState === 'ready' || sessionState === 'reconnecting') &&
!pendingTransportReconnects.has(targetId)
if (transportReconnectStarted) {
rotateSshProviderAuthority(targetId)
pendingTransportReconnects.add(targetId)
} else if (
state.status === 'disconnected' ||
@@ -532,10 +603,6 @@ function createSshConnectionCallbacks(): SshConnectionCallbacks {
}
const completedTransportReconnect =
state.status === 'connected' && pendingTransportReconnects.delete(targetId)
if (completedTransportReconnect) {
// Why: staged mutations from the replaced SSH transport must fail even if its relay session disappeared before recovery completed.
advanceSshConnectionGeneration(targetId)
}
const shouldReconnectRelay =
session !== undefined &&
completedTransportReconnect &&
@@ -613,6 +680,9 @@ function broadcastDetectedPortsFromCurrentWindow(
function configureRelaySessionCallbacks(session: SshRelaySession): void {
session.setOnTerminalRelayError((tid, err) => {
clearRelayLostBackoff(tid)
if (activeSessions.get(tid)?.getState() !== 'deploying') {
rotateSshProviderAuthority(tid)
}
console.warn(
`[ssh] Terminal relay error for ${tid}: ${err.message}; skipping reconnect backoff.`
)
@@ -643,6 +713,7 @@ function configureRelaySessionCallbacks(session: SshRelaySession): void {
if (state.reconnectTimer) {
return
}
rotateSshProviderAuthority(tid)
if (state.attempts >= RELAY_LOST_MAX_ATTEMPTS) {
console.warn(
`[ssh] Relay channel for ${tid} kept dying across ${state.attempts} attempts; giving up. User must reconnect manually.`
@@ -781,6 +852,11 @@ export function registerSshHandlers(
}
const repoReadoptions = sshStore.lastRepoReadoptions
sshStore.lastRepoReadoptions = []
for (const targetId of new Set(
repoReadoptions.flatMap(({ oldTargetId, newTargetId }) => [oldTargetId, newTargetId])
)) {
rotateSshProviderAuthority(targetId)
}
const win = getCurrentMainWindow()
if (win && !win.isDestroyed()) {
win.webContents.send('repos:changed')
@@ -828,7 +904,9 @@ export function registerSshHandlers(
appendFileSync(e2eProbePath, `${JSON.stringify(targetId)}\n`)
throw new Error('e2e_forbidden_local_ssh_connect')
}
const observedGeneration = currentConnectGeneration(targetId)
// Why: fence callers that entered before a same-turn disconnect/reset but resume after its cleanup.
const admissionAuthority = getSshProviderAuthority(targetId)
await awaitTargetLifecycle(targetId)
const reset = resetRelayInFlight.get(targetId)
if (reset) {
await reset
@@ -836,16 +914,29 @@ export function registerSshHandlers(
// Why: serialize concurrent ssh:connect for the same target; interleaved connects otherwise leak the first session.
const existing = connectInFlight.get(targetId)
let replacePendingTransport = false
if (existing) {
return existing.promise
if (isCurrentConnectAttempt(targetId, existing.authority)) {
return existing.promise
}
}
if (currentConnectGeneration(targetId) !== observedGeneration) {
if (!isCurrentConnectAttempt(targetId, admissionAuthority)) {
throw connectCancelledError()
}
const observedAuthority = admissionAuthority
if (existing) {
if (connectInFlight.get(targetId) === existing) {
connectInFlight.delete(targetId)
replacePendingTransport = true
}
}
if (!isCurrentSshProviderAuthority(observedAuthority)) {
throw connectCancelledError()
}
pendingTransportReconnects.delete(targetId)
const promise = doConnect(targetId)
const attempt = { generation: currentConnectGeneration(targetId), promise }
const promise = doConnect(targetId, replacePendingTransport)
const attempt = { authority: getSshProviderAuthority(targetId), promise }
connectInFlight.set(targetId, attempt)
try {
return await promise
@@ -863,7 +954,10 @@ export function registerSshHandlers(
return connectTarget(args.targetId)
})
async function doConnect(targetId: string): Promise<SshConnectionState> {
async function doConnect(
targetId: string,
replacePendingTransport = false
): Promise<SshConnectionState> {
const target = sshStore!.getTarget(targetId)
if (!target) {
throw new Error(`SSH target "${targetId}" not found`)
@@ -886,20 +980,38 @@ export function registerSshHandlers(
return getPublicSshState(targetId)!
}
const generation = advanceSshConnectionGeneration(targetId)
const authority = rotateSshProviderAuthority(targetId)
clearRelayStateOverride(targetId)
const pendingTransportDisconnect = replacePendingTransport
? connectionManager!.disconnect(targetId).then(
() => ({ ok: true }) as const,
(error: unknown) => ({ ok: false, error }) as const
)
: null
let conn
// Why: tear down any existing session first to avoid leaking its multiplexer, providers, and timers (double-connect / reconnect-after-error).
if (existingSession) {
// Why: await port teardown before disposing, else the new session's restorePortForwards can hit EADDRINUSE on not-yet-released ports.
await portForwardManager!.removeAllForwards(targetId)
if (!isCurrentConnectAttempt(targetId, generation)) {
if (!isCurrentConnectAttempt(targetId, authority)) {
throw connectCancelledError()
}
existingSession.detach()
activeSessions.delete(targetId)
clearRelayLostBackoff(targetId)
clearRelayStateOverride(targetId)
if (activeSessions.get(targetId) === existingSession) {
activeSessions.delete(targetId)
clearRelayLostBackoff(targetId)
clearRelayStateOverride(targetId)
}
}
if (pendingTransportDisconnect) {
const disconnectResult = await pendingTransportDisconnect
if (!disconnectResult.ok) {
throw disconnectResult.error
}
if (!isCurrentConnectAttempt(targetId, authority)) {
throw connectCancelledError()
}
}
// Why: create the session early so onStateChange sees it in 'deploying' and skips reconnect logic.
@@ -914,7 +1026,7 @@ export function registerSshHandlers(
configureRelaySessionCallbacks(session)
activeSessions.set(targetId, session)
const ownsSession = (): boolean =>
isCurrentConnectAttempt(targetId, generation) && activeSessions.get(targetId) === session
isCurrentConnectAttempt(targetId, authority) && activeSessions.get(targetId) === session
try {
conn = await connectionManager!.connect(target)
@@ -988,66 +1100,60 @@ export function registerSshHandlers(
ipcMain.handle('ssh:terminateSessions', async (_event, args: { targetId: string }) => {
invalidateConnectAttempt(args.targetId)
const session = activeSessions.get(args.targetId)
const provider = getSshPtyProvider(args.targetId)
const leasedIds = persistedStore!
.getSshRemotePtyLeases(args.targetId)
.filter((lease) => lease.state !== 'terminated' && lease.state !== 'expired')
.map((lease) => lease.ptyId)
const ptyIdsByRelayId = new Map<string, string>()
for (const ptyId of getPtyIdsForConnection(args.targetId)) {
const relayPtyId = toRelaySshPtyId(args.targetId, ptyId)
ptyIdsByRelayId.set(relayPtyId, toAppSshPtyId(args.targetId, ptyId))
}
for (const ptyId of leasedIds) {
const relayPtyId = toRelaySshPtyId(args.targetId, ptyId)
ptyIdsByRelayId.set(
relayPtyId,
ptyIdsByRelayId.get(relayPtyId) ?? toAppSshPtyId(args.targetId, ptyId)
)
}
const ptyIds = Array.from(ptyIdsByRelayId, ([relayPtyId, appPtyId]) => ({
relayPtyId,
appPtyId
}))
if (ptyIds.length > 0 && !provider) {
throw new Error(
`${SSH_TERMINATE_RECONNECT_REQUIRED}: SSH relay is not connected; reconnect before terminating remote sessions.`
)
}
const shutdownResults = provider
? await Promise.allSettled(
ptyIds.map(({ appPtyId }) =>
provider.shutdown(appPtyId, { immediate: true, keepHistory: false })
)
)
: []
const shutdownFailures: string[] = []
for (const [index, result] of shutdownResults.entries()) {
const { appPtyId, relayPtyId } = ptyIds[index]
if (result.status !== 'fulfilled' && !isSshPtyNotFoundError(result.reason)) {
shutdownFailures.push(
`${relayPtyId}: ${result.reason instanceof Error ? result.reason.message : String(result.reason)}`
)
continue
await runTargetLifecycle(args.targetId, async () => {
const provider = getSshPtyProvider(args.targetId)
const leasedIds = persistedStore!
.getSshRemotePtyLeases(args.targetId)
.filter((lease) => lease.state !== 'terminated' && lease.state !== 'expired')
.map((lease) => lease.ptyId)
const ptyIdsByRelayId = new Map<string, string>()
for (const ptyId of getPtyIdsForConnection(args.targetId)) {
const relayPtyId = toRelaySshPtyId(args.targetId, ptyId)
ptyIdsByRelayId.set(relayPtyId, toAppSshPtyId(args.targetId, ptyId))
}
clearProviderPtyState(appPtyId)
deletePtyOwnership(appPtyId)
persistedStore!.markSshRemotePtyLease(args.targetId, relayPtyId, 'terminated')
}
if (shutdownFailures.length > 0) {
// Why: a failed relay shutdown can leave the remote process alive in the grace window; keep the lease/session so the user can retry.
throw new Error(`Failed to terminate SSH host sessions: ${shutdownFailures.join('; ')}`)
}
if (session) {
await portForwardManager!.removeAllForwards(args.targetId)
session.dispose()
activeSessions.delete(args.targetId)
clearRelayLostBackoff(args.targetId)
clearRelayStateOverride(args.targetId)
}
await connectionManager!.disconnect(args.targetId)
for (const ptyId of leasedIds) {
const relayPtyId = toRelaySshPtyId(args.targetId, ptyId)
ptyIdsByRelayId.set(
relayPtyId,
ptyIdsByRelayId.get(relayPtyId) ?? toAppSshPtyId(args.targetId, ptyId)
)
}
const ptyIds = Array.from(ptyIdsByRelayId, ([relayPtyId, appPtyId]) => ({
relayPtyId,
appPtyId
}))
if (ptyIds.length > 0 && !provider) {
throw new Error(
`${SSH_TERMINATE_RECONNECT_REQUIRED}: SSH relay is not connected; reconnect before terminating remote sessions.`
)
}
const shutdownResults = provider
? await Promise.allSettled(
ptyIds.map(({ appPtyId }) =>
provider.shutdown(appPtyId, { immediate: true, keepHistory: false })
)
)
: []
const shutdownFailures: string[] = []
for (const [index, result] of shutdownResults.entries()) {
const { appPtyId, relayPtyId } = ptyIds[index]
if (result.status !== 'fulfilled' && !isSshPtyNotFoundError(result.reason)) {
shutdownFailures.push(
`${relayPtyId}: ${result.reason instanceof Error ? result.reason.message : String(result.reason)}`
)
continue
}
clearProviderPtyState(appPtyId)
deletePtyOwnership(appPtyId)
persistedStore!.markSshRemotePtyLease(args.targetId, relayPtyId, 'terminated')
}
if (shutdownFailures.length > 0) {
// Why: a failed relay shutdown can leave the remote process alive in the grace window; keep the lease/session so the user can retry.
throw new Error(`Failed to terminate SSH host sessions: ${shutdownFailures.join('; ')}`)
}
await teardownSshTargetTransport(args.targetId, (session) => session.dispose())
})
})
async function doResetRelay(targetId: string, target: SshTarget): Promise<void> {
@@ -1061,13 +1167,11 @@ export function registerSshHandlers(
}
}
rotateSshProviderAuthority(targetId)
const session = activeSessions.get(targetId)
if (session) {
await portForwardManager!.removeAllForwards(targetId)
// Why: detach() not dispose() — reset has its own stale-lease semantics below that dispose()'s clean-termination recording would hide.
session.detach()
activeSessions.delete(targetId)
clearRelayLostBackoff(targetId)
await teardownActiveSshSession(targetId, (capturedSession) => capturedSession.detach())
}
const existingConn = connectionManager!.getConnection(targetId)
@@ -1106,13 +1210,13 @@ export function registerSshHandlers(
}
let resetPromise: Promise<void>
resetPromise = Promise.resolve()
.then(() => doResetRelay(args.targetId, target))
.finally(() => {
if (resetRelayInFlight.get(args.targetId) === resetPromise) {
resetRelayInFlight.delete(args.targetId)
}
})
resetPromise = runTargetLifecycle(args.targetId, () =>
doResetRelay(args.targetId, target)
).finally(() => {
if (resetRelayInFlight.get(args.targetId) === resetPromise) {
resetRelayInFlight.delete(args.targetId)
}
})
resetRelayInFlight.set(args.targetId, resetPromise)
return resetPromise
})
@@ -1300,8 +1404,10 @@ export async function resetSshHandlerStateForTests(): Promise<void> {
}
relayStateOverrides.clear()
connectInFlight.clear()
targetLifecycleInFlight.clear()
pendingTransportReconnects.clear()
resetSshConnectionGenerations()
resetSshProviderAuthorities()
resetRelayInFlight.clear()
testingTargets.clear()
credentialRequestedForTarget.clear()
src/main/ipc/worktrees.test.ts
+992 -11
View File
File diff suppressed because it is too large Load Diff
src/main/ipc/worktrees.ts
+853 -68
View File
File diff suppressed because it is too large Load Diff
src/main/runtime/public-ssh-state.test.ts
+21
View File
@@ -0,0 +1,21 @@
import { describe, expect, it } from 'vitest'
import type { SshConnectionState, SshProviderEpoch } from '../../shared/ssh-types'
import { getPublicSshState } from './public-ssh-state'
describe('public SSH state', () => {
it('preserves the complete provider authority pair', () => {
const state: SshConnectionState = {
targetId: 'ssh-a',
status: 'error',
error: 'private detail',
reconnectAttempt: 0,
providerEpoch: 'provider-a' as SshProviderEpoch,
connectionGeneration: 3
}
expect(getPublicSshState(state)).toEqual({
...state,
error: 'SSH connection unavailable'
})
})
})
src/main/ssh/ssh-connection-manager.test.ts
+23
View File
@@ -71,4 +71,27 @@ describe('SshConnectionManager', () => {
expect(mockState.instances).toHaveLength(2)
expect(manager.getConnection(target.id)).toBe(secondConnection)
})
it('keeps the replacement when the disconnected attempt resolves late', async () => {
let resolveFirst!: () => void
mockState.connectResults.push(
new Promise<void>((resolve) => {
resolveFirst = resolve
}),
Promise.resolve()
)
const manager = new SshConnectionManager({
onStateChange: vi.fn()
})
const firstConnect = manager.connect(target)
await manager.disconnect(target.id)
const replacement = await manager.connect(target)
resolveFirst()
await expect(firstConnect).resolves.not.toBe(replacement)
expect(mockState.instances[0].disconnect).toHaveBeenCalledOnce()
expect(await manager.connect(target)).toBe(replacement)
expect(manager.getConnection(target.id)).toBe(replacement)
})
})
src/main/ssh/ssh-connection.test.ts
+44
View File
@@ -458,6 +458,50 @@ describe('SshConnection', () => {
expect(conn.getState().status).toBe('disconnected')
})
it('rejects late ssh2 ready after disconnect without resurrecting the connection', async () => {
const callbacks = createCallbacks()
const conn = new SshConnection(createTarget(), callbacks)
const connectResult = conn.connect().catch((error: Error) => error)
for (let i = 0; i < 5 && clientInstances.length === 0; i++) {
await Promise.resolve()
}
expect(clientInstances).toHaveLength(1)
await conn.disconnect()
await expect(connectResult).resolves.toMatchObject({
message: 'SSH connection attempt was cancelled'
})
expect(conn.getState()).toMatchObject({ status: 'disconnected', error: null })
expect(callbacks.onStateChange).not.toHaveBeenCalledWith(
'target-1',
expect.objectContaining({ status: 'connected' })
)
})
it('keeps disconnected state when ssh2 reports a late startup error', async () => {
connectBehavior = 'error'
connectErrorMessage = 'Connection lost before handshake'
const callbacks = createCallbacks()
const conn = new SshConnection(createTarget(), callbacks)
const connectResult = conn.connect().catch((error: Error) => error)
for (let i = 0; i < 5 && clientInstances.length === 0; i++) {
await Promise.resolve()
}
expect(clientInstances).toHaveLength(1)
await conn.disconnect()
await expect(connectResult).resolves.toMatchObject({
message: 'Connection lost before handshake'
})
expect(conn.getState()).toMatchObject({ status: 'disconnected', error: null })
expect(callbacks.onStateChange).not.toHaveBeenCalledWith(
'target-1',
expect.objectContaining({ status: 'error' })
)
})
it('getTarget returns a copy of the target', () => {
const target = createTarget()
const conn = new SshConnection(target, createCallbacks())
src/main/ssh/ssh-provider-authority.test.ts
+170
View File
@@ -0,0 +1,170 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
import {
assertSshMutationExpectation,
resetSshConnectionGenerations,
setSshConnectionGeneration
} from './ssh-connection-generation'
import {
getSshProviderAuthority,
isCurrentSshProviderAuthority,
registerSshProviderRequestAbort,
resetSshProviderAuthorities,
rotateSshProviderAuthority
} from './ssh-provider-authority'
describe('SSH provider authority', () => {
beforeEach(() => {
resetSshConnectionGenerations()
resetSshProviderAuthorities()
})
it('rotates provider epoch and connection generation atomically', () => {
const initial = getSshProviderAuthority('ssh-a')
const rotated = rotateSshProviderAuthority('ssh-a')
expect(rotated).toEqual({
targetId: 'ssh-a',
providerEpoch: expect.any(String),
connectionGeneration: initial.connectionGeneration + 1
})
expect(rotated.providerEpoch).not.toBe(initial.providerEpoch)
expect(getSshProviderAuthority('ssh-a')).toEqual(rotated)
expect(() =>
assertSshMutationExpectation('ssh-a', 'ssh-a', initial.connectionGeneration)
).toThrow('SSH connection changed; refresh and try again')
expect(() =>
assertSshMutationExpectation('ssh-a', 'ssh-a', rotated.connectionGeneration)
).not.toThrow()
})
it('rejects a stale authority by full-pair equality', () => {
const stale = getSshProviderAuthority('ssh-a')
rotateSshProviderAuthority('ssh-a')
expect(isCurrentSshProviderAuthority(stale)).toBe(false)
expect(isCurrentSshProviderAuthority(getSshProviderAuthority('ssh-a'))).toBe(true)
})
it('checks unknown authority without allocating provider state', () => {
const sequence = (authority: ReturnType<typeof getSshProviderAuthority>): number =>
Number.parseInt(authority.providerEpoch.split('-').at(-1) ?? '', 36)
const before = getSshProviderAuthority('before-probe')
expect(
isCurrentSshProviderAuthority({
targetId: 'unknown-target',
providerEpoch: 'untrusted-epoch' as typeof before.providerEpoch,
connectionGeneration: 0
})
).toBe(false)
const after = getSshProviderAuthority('after-probe')
expect(sequence(after) - sequence(before)).toBe(1)
})
it('aborts every old-authority provider request once with target isolation', () => {
const authorityA = getSshProviderAuthority('ssh-a')
const authorityB = getSshProviderAuthority('ssh-b')
const controllersA = [new AbortController(), new AbortController()]
const controllerB = new AbortController()
const abortsA = controllersA.map(() => vi.fn())
const abortB = vi.fn()
controllersA.forEach((controller, index) => {
controller.signal.addEventListener('abort', abortsA[index])
registerSshProviderRequestAbort(authorityA, controller)
})
controllerB.signal.addEventListener('abort', abortB)
registerSshProviderRequestAbort(authorityB, controllerB)
rotateSshProviderAuthority('ssh-a')
rotateSshProviderAuthority('ssh-a')
expect(abortsA[0]).toHaveBeenCalledOnce()
expect(abortsA[1]).toHaveBeenCalledOnce()
expect(abortB).not.toHaveBeenCalled()
rotateSshProviderAuthority('ssh-b')
expect(abortB).toHaveBeenCalledOnce()
})
it('revokes every target when one target rolls the generation session scope', () => {
setSshConnectionGeneration('ssh-a', 2 ** 13 - 1)
const authorityA = getSshProviderAuthority('ssh-a')
const authorityB = getSshProviderAuthority('ssh-b')
const controllerA = new AbortController()
const controllerB = new AbortController()
const abortA = vi.spyOn(controllerA, 'abort')
const abortB = vi.spyOn(controllerB, 'abort')
let oldAuthoritiesWereCurrent = true
controllerA.signal.addEventListener('abort', () => {
oldAuthoritiesWereCurrent =
isCurrentSshProviderAuthority(authorityA) || isCurrentSshProviderAuthority(authorityB)
})
registerSshProviderRequestAbort(authorityA, controllerA)
registerSshProviderRequestAbort(authorityB, controllerB)
const rotated = rotateSshProviderAuthority('ssh-a')
expect(rotated.connectionGeneration).toBe(2 ** 13 + 1)
expect(abortA).toHaveBeenCalledOnce()
expect(abortB).toHaveBeenCalledOnce()
expect(oldAuthoritiesWereCurrent).toBe(false)
expect(isCurrentSshProviderAuthority(authorityA)).toBe(false)
expect(isCurrentSshProviderAuthority(authorityB)).toBe(false)
})
it('rejects old-authority registration reentered from an abort callback', () => {
const oldAuthority = getSshProviderAuthority('ssh-a')
const controllerA = new AbortController()
const controllerB = new AbortController()
const abortA = vi.fn()
let oldAuthorityWasCurrent = true
controllerA.signal.addEventListener('abort', () => {
abortA()
oldAuthorityWasCurrent = isCurrentSshProviderAuthority(oldAuthority)
registerSshProviderRequestAbort(oldAuthority, controllerB)
})
registerSshProviderRequestAbort(oldAuthority, controllerA)
rotateSshProviderAuthority('ssh-a')
rotateSshProviderAuthority('ssh-a')
expect(abortA).toHaveBeenCalledOnce()
expect(oldAuthorityWasCurrent).toBe(false)
expect(controllerB.signal.aborted).toBe(false)
})
it('removes settled and explicitly aborted registrations before later rotation', () => {
const authority = getSshProviderAuthority('ssh-a')
const settled = new AbortController()
const explicitlyAborted = new AbortController()
const settledAbort = vi.spyOn(settled, 'abort')
const explicitAbort = vi.spyOn(explicitlyAborted, 'abort')
const removeSettled = registerSshProviderRequestAbort(authority, settled)
registerSshProviderRequestAbort(authority, explicitlyAborted)
removeSettled()
explicitlyAborted.abort()
rotateSshProviderAuthority('ssh-a')
expect(settledAbort).not.toHaveBeenCalled()
expect(explicitAbort).toHaveBeenCalledOnce()
})
it('aborts and clears registrations during reset isolation', () => {
const oldAuthority = getSshProviderAuthority('ssh-a')
const oldController = new AbortController()
const oldAbort = vi.spyOn(oldController, 'abort')
registerSshProviderRequestAbort(oldAuthority, oldController)
resetSshProviderAuthorities()
const newAuthority = getSshProviderAuthority('ssh-a')
const newController = new AbortController()
const newAbort = vi.spyOn(newController, 'abort')
registerSshProviderRequestAbort(newAuthority, newController)
rotateSshProviderAuthority('ssh-a')
expect(oldAbort).toHaveBeenCalledOnce()
expect(newAbort).toHaveBeenCalledOnce()
})
})
src/main/ssh/ssh-provider-authority.ts
+148
View File
@@ -0,0 +1,148 @@
import { randomBytes } from 'node:crypto'
import type { DirectSshAuthority, SshProviderEpoch } from '../../shared/ssh-types'
import {
advanceSshConnectionGeneration,
getSshConnectionGeneration
} from './ssh-connection-generation'
const authorityByTarget = new Map<string, DirectSshAuthority>()
type ProviderRequestAbortRegistration = {
authority: DirectSshAuthority
controller: AbortController
onAbort: () => void
}
const providerRequestAbortsByTarget = new Map<string, Set<ProviderRequestAbortRegistration>>()
let providerEpochSequence = 0
function issueSshProviderEpoch(): SshProviderEpoch {
if (providerEpochSequence >= Number.MAX_SAFE_INTEGER) {
throw new Error('SSH provider epoch sequence exhausted')
}
providerEpochSequence += 1
return `${randomBytes(12).toString('hex')}-${providerEpochSequence.toString(36)}` as SshProviderEpoch
}
function createAuthority(targetId: string, connectionGeneration: number): DirectSshAuthority {
return {
targetId,
providerEpoch: issueSshProviderEpoch(),
connectionGeneration
}
}
function authoritiesEqual(left: DirectSshAuthority, right: DirectSshAuthority): boolean {
return (
left.targetId === right.targetId &&
left.providerEpoch === right.providerEpoch &&
left.connectionGeneration === right.connectionGeneration
)
}
function removeProviderRequestAbort(registration: ProviderRequestAbortRegistration): void {
registration.controller.signal.removeEventListener('abort', registration.onAbort)
const registrations = providerRequestAbortsByTarget.get(registration.authority.targetId)
if (!registrations?.delete(registration)) {
return
}
if (registrations.size === 0) {
providerRequestAbortsByTarget.delete(registration.authority.targetId)
}
}
function abortProviderRequestsForAuthority(authority: DirectSshAuthority): void {
const registrations = providerRequestAbortsByTarget.get(authority.targetId)
if (!registrations) {
return
}
const matching = [...registrations].filter((registration) =>
authoritiesEqual(registration.authority, authority)
)
for (const registration of matching) {
removeProviderRequestAbort(registration)
}
for (const registration of matching) {
if (!registration.controller.signal.aborted) {
registration.controller.abort()
}
}
}
function abortAllProviderRequests(): void {
const registrations = [...providerRequestAbortsByTarget.values()].flatMap((entries) => [
...entries
])
providerRequestAbortsByTarget.clear()
for (const registration of registrations) {
registration.controller.signal.removeEventListener('abort', registration.onAbort)
}
for (const registration of registrations) {
if (!registration.controller.signal.aborted) {
registration.controller.abort()
}
}
}
export function registerSshProviderRequestAbort(
authority: DirectSshAuthority,
controller: AbortController
): () => void {
if (controller.signal.aborted || !isCurrentSshProviderAuthority(authority)) {
return () => {}
}
const capturedAuthority = { ...authority }
const registration: ProviderRequestAbortRegistration = {
authority: capturedAuthority,
controller,
onAbort: () => removeProviderRequestAbort(registration)
}
const registrations =
providerRequestAbortsByTarget.get(authority.targetId) ??
new Set<ProviderRequestAbortRegistration>()
registrations.add(registration)
providerRequestAbortsByTarget.set(authority.targetId, registrations)
controller.signal.addEventListener('abort', registration.onAbort, { once: true })
return () => removeProviderRequestAbort(registration)
}
export function getSshProviderAuthority(targetId: string): DirectSshAuthority {
const generation = getSshConnectionGeneration(targetId)
const current = authorityByTarget.get(targetId)
if (current?.connectionGeneration === generation) {
return current
}
const authority = createAuthority(targetId, generation)
authorityByTarget.set(targetId, authority)
return authority
}
export function rotateSshProviderAuthority(targetId: string): DirectSshAuthority {
const previous = authorityByTarget.get(targetId)
const previousGeneration = getSshConnectionGeneration(targetId)
const nextGeneration = advanceSshConnectionGeneration(targetId)
const authority = createAuthority(targetId, nextGeneration)
if (nextGeneration !== previousGeneration + 1) {
authorityByTarget.clear()
authorityByTarget.set(targetId, authority)
abortAllProviderRequests()
return authority
}
authorityByTarget.set(targetId, authority)
if (previous) {
abortProviderRequestsForAuthority(previous)
}
return authority
}
export function isCurrentSshProviderAuthority(authority: DirectSshAuthority): boolean {
const current = authorityByTarget.get(authority.targetId)
return (
current !== undefined &&
current.connectionGeneration === getSshConnectionGeneration(authority.targetId) &&
authoritiesEqual(current, authority)
)
}
export function resetSshProviderAuthorities(): void {
abortAllProviderRequests()
authorityByTarget.clear()
}
src/preload/api-types.ts
+23 -1
View File
@@ -26,6 +26,20 @@ import type {
} from '../shared/local-log-tail-types'
import type { ReadClipboardTextOptions } from '../shared/clipboard-text'
import type { AppIdentity } from '../shared/app-identity'
import type {
HostQualifiedDetectedWorktreeResult,
LegacyDetectedWorktreeRequest,
ListDetectedWorktreesArgs,
ProviderRequestId
} from '../shared/detected-worktree-provider-contract'
import type {
HostRepoCatalogSnapshot,
ListReposForExecutionHostArgs
} from '../shared/host-repo-catalog-contract'
import type {
HostLineageSnapshot,
ListDesktopLineageForHostArgs
} from '../shared/host-lineage-contract'
import type {
WriteTerminalRenderDesyncEvidenceArgs,
WriteTerminalRenderDesyncEvidenceResult
@@ -1157,6 +1171,7 @@ export type PreloadApi = {
}
repos: {
list: () => Promise<Repo[]>
listForExecutionHost?: (args: ListReposForExecutionHostArgs) => Promise<HostRepoCatalogSnapshot>
// Why: error union matches the IPC handler's return shape; renderer callers branch on `'error' in result`.
add: (args: {
path: string
@@ -1342,7 +1357,13 @@ export type PreloadApi = {
}
worktrees: {
list: (args: { repoId: string }) => Promise<Worktree[]>
listDetected: (args: { repoId: string }) => Promise<DetectedWorktreeListResult>
listDetected: {
(
args: ListDetectedWorktreesArgs
): Promise<HostQualifiedDetectedWorktreeResult | DetectedWorktreeListResult>
(args: LegacyDetectedWorktreeRequest): Promise<DetectedWorktreeListResult>
}
cancelListDetected?: (args: { providerRequestId: ProviderRequestId }) => Promise<void>
listAll: () => Promise<Worktree[]>
create: (args: CreateWorktreeArgs) => Promise<CreateWorktreeResult>
/** Two-phase progress for a background `create`, correlated by `creationId`. The remote/runtime
@@ -1392,6 +1413,7 @@ export type PreloadApi = {
lineage: Record<string, WorktreeLineage>
workspaceLineage?: Record<string, WorkspaceLineage>
}>
listLineageForHost?: (args: ListDesktopLineageForHostArgs) => Promise<HostLineageSnapshot>
updateLineage: (args: {
worktreeId: string
parentWorktreeId?: string
src/preload/index.ts
+48 -13
View File
@@ -30,6 +30,18 @@ import type {
PortForwardEntry,
EnrichedDetectedPort
} from '../shared/ssh-types'
import {
admitSshConnectionStateForAuthorityReconciliation,
admitSshDetectedPorts
} from '../shared/ssh-retained-payload-admission'
import type {
HostRepoCatalogSnapshot,
ListReposForExecutionHostArgs
} from '../shared/host-repo-catalog-contract'
import type {
HostLineageSnapshot,
ListDesktopLineageForHostArgs
} from '../shared/host-lineage-contract'
import type {
PluginPanelActionOutcome,
PluginPanelEntry
@@ -591,6 +603,9 @@ const api = {
repos: {
list: () => ipcRenderer.invoke('repos:list'),
listForExecutionHost: (args: ListReposForExecutionHostArgs): Promise<HostRepoCatalogSnapshot> =>
ipcRenderer.invoke('repos:listForExecutionHost', args),
add: (args) => ipcRenderer.invoke('repos:add', args),
addRemote: (args) => ipcRenderer.invoke('repos:addRemote', args),
@@ -724,6 +739,8 @@ const api = {
listDetected: (args) => ipcRenderer.invoke('worktrees:listDetected', args),
cancelListDetected: (args) => ipcRenderer.invoke('worktrees:cancelListDetected', args),
listAll: () => ipcRenderer.invoke('worktrees:listAll'),
create: (args) => ipcRenderer.invoke('worktrees:create', args),
@@ -756,6 +773,9 @@ const api = {
listLineage: () => ipcRenderer.invoke('worktrees:listLineage'),
listLineageForHost: (args: ListDesktopLineageForHostArgs): Promise<HostLineageSnapshot> =>
ipcRenderer.invoke('worktrees:listLineageForHost', args),
updateLineage: (args) => ipcRenderer.invoke('worktrees:updateLineage', args),
persistSortOrder: (args) => ipcRenderer.invoke('worktrees:persistSortOrder', args),
@@ -4298,8 +4318,10 @@ const api = {
importConfig: (args?: { reAdopt?: boolean }): Promise<SshConfigImportResult> =>
ipcRenderer.invoke('ssh:importConfig', args),
connect: (args: { targetId: string }): Promise<SshConnectionState | null> =>
ipcRenderer.invoke('ssh:connect', args),
connect: async (args: { targetId: string }): Promise<SshConnectionState | null> => {
const state: unknown = await ipcRenderer.invoke('ssh:connect', args)
return state ? admitSshConnectionStateForAuthorityReconciliation(state, args.targetId) : null
},
disconnect: (args: { targetId: string }): Promise<void> =>
ipcRenderer.invoke('ssh:disconnect', args),
@@ -4310,24 +4332,37 @@ const api = {
resetRelay: (args: { targetId: string }): Promise<void> =>
ipcRenderer.invoke('ssh:resetRelay', args),
getState: (args: { targetId: string }): Promise<SshConnectionState | null> =>
ipcRenderer.invoke('ssh:getState', args),
getState: async (args: { targetId: string }): Promise<SshConnectionState | null> => {
const state: unknown = await ipcRenderer.invoke('ssh:getState', args)
return state ? admitSshConnectionStateForAuthorityReconciliation(state, args.targetId) : null
},
needsPassphrasePrompt: (args: { targetId: string }): Promise<boolean> =>
ipcRenderer.invoke('ssh:needsPassphrasePrompt', args),
testConnection: (args: {
testConnection: async (args: {
targetId: string
}): Promise<{ success: boolean; error?: string; state?: SshConnectionState }> =>
ipcRenderer.invoke('ssh:testConnection', args),
}): Promise<{ success: boolean; error?: string; state?: SshConnectionState }> => {
const result: { success: boolean; error?: string; state?: unknown } =
await ipcRenderer.invoke('ssh:testConnection', args)
const state = result.state
? admitSshConnectionStateForAuthorityReconciliation(result.state, args.targetId)
: null
return { ...result, ...(state ? { state } : { state: undefined }) }
},
onStateChanged: (
callback: (data: { targetId: string; state: SshConnectionState }) => void
): (() => void) => {
const listener = (
_event: Electron.IpcRendererEvent,
data: { targetId: string; state: SshConnectionState }
) => callback(data)
data: { targetId: string; state: unknown }
): void => {
const state = admitSshConnectionStateForAuthorityReconciliation(data.state, data.targetId)
if (state) {
callback({ targetId: data.targetId, state })
}
}
ipcRenderer.on('ssh:state-changed', listener)
return () => ipcRenderer.removeListener('ssh:state-changed', listener)
},
@@ -4355,8 +4390,8 @@ const api = {
listPortForwards: (args?: { targetId?: string }): Promise<PortForwardEntry[]> =>
ipcRenderer.invoke('ssh:listPortForwards', args),
listDetectedPorts: (args: { targetId: string }): Promise<EnrichedDetectedPort[]> =>
ipcRenderer.invoke('ssh:listDetectedPorts', args),
listDetectedPorts: async (args: { targetId: string }): Promise<EnrichedDetectedPort[]> =>
admitSshDetectedPorts(await ipcRenderer.invoke('ssh:listDetectedPorts', args)),
onPortForwardsChanged: (
callback: (data: { targetId: string; forwards: PortForwardEntry[] }) => void
@@ -4374,8 +4409,8 @@ const api = {
): (() => void) => {
const handler = (
_event: Electron.IpcRendererEvent,
data: { targetId: string; ports: EnrichedDetectedPort[] }
) => callback(data)
data: { targetId: string; ports: unknown }
) => callback({ targetId: data.targetId, ports: admitSshDetectedPorts(data.ports) })
ipcRenderer.on('ssh:detected-ports-changed', handler)
return () => ipcRenderer.removeListener('ssh:detected-ports-changed', handler)
},
src/preload/ssh-authority-forwarding.test.ts
+164
View File
@@ -0,0 +1,164 @@
import { afterEach, beforeEach, describe, expect, expectTypeOf, it, vi } from 'vitest'
import type { PreloadApi } from './api-types'
import type { SshConnectionState, SshProviderEpoch } from '../shared/ssh-types'
import type {
HostQualifiedDetectedWorktreeResult,
ListDetectedWorktreesArgs
} from '../shared/detected-worktree-provider-contract'
import type { DetectedWorktreeListResult } from '../shared/types'
function listDetectedVariableTypeProbe(api: PreloadApi, args: ListDetectedWorktreesArgs) {
return api.worktrees.listDetected(args)
}
const { exposeInMainWorld, invoke, on, removeListener, send, sendSync } = vi.hoisted(() => ({
exposeInMainWorld: vi.fn(),
invoke: vi.fn(),
on: vi.fn(),
removeListener: vi.fn(),
send: vi.fn(),
sendSync: vi.fn()
}))
vi.mock('electron', () => ({
contextBridge: { exposeInMainWorld },
ipcRenderer: { invoke, on, removeListener, send, sendSync },
webFrame: {
getZoomFactor: vi.fn(() => 1),
setZoomFactor: vi.fn(),
setVisualZoomLevelLimits: vi.fn()
},
webUtils: { getPathForFile: vi.fn(() => '') }
}))
vi.mock('@electron-toolkit/preload', () => ({ electronAPI: {} }))
describe('native preload SSH authority forwarding', () => {
const originalContextIsolated = Object.getOwnPropertyDescriptor(process, 'contextIsolated')
beforeEach(() => {
vi.resetModules()
exposeInMainWorld.mockReset()
invoke.mockReset()
on.mockReset()
removeListener.mockReset()
send.mockReset()
sendSync.mockReset()
Object.defineProperty(process, 'contextIsolated', { configurable: true, value: true })
vi.stubGlobal('window', {
addEventListener: vi.fn(),
dispatchEvent: vi.fn(),
removeEventListener: vi.fn()
})
vi.stubGlobal('document', { addEventListener: vi.fn() })
})
afterEach(() => {
vi.unstubAllGlobals()
if (originalContextIsolated) {
Object.defineProperty(process, 'contextIsolated', originalContextIsolated)
} else {
Reflect.deleteProperty(process, 'contextIsolated')
}
})
it('retains host-qualified outcomes for variable-form detected-worktree requests', () => {
expectTypeOf(listDetectedVariableTypeProbe).returns.toEqualTypeOf<
Promise<HostQualifiedDetectedWorktreeResult | DetectedWorktreeListResult>
>()
})
it('forwards full-pair get and push states without cloning away authority', async () => {
const state: SshConnectionState = {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'native-provider-epoch' as SshProviderEpoch,
connectionGeneration: 29
}
invoke.mockResolvedValueOnce(state)
await import('./index')
const api = exposeInMainWorld.mock.calls.find(([name]) => name === 'api')?.[1] as PreloadApi
await expect(api.ssh.getState({ targetId: 'ssh-1' })).resolves.toEqual(state)
expect(invoke).toHaveBeenCalledWith('ssh:getState', { targetId: 'ssh-1' })
const onStateChanged = vi.fn()
api.ssh.onStateChanged(onStateChanged)
const listener = on.mock.calls.find(([channel]) => channel === 'ssh:state-changed')?.[1] as (
event: unknown,
data: { targetId: string; state: SshConnectionState }
) => void
listener({}, { targetId: 'ssh-1', state })
expect(onStateChanged).toHaveBeenCalledWith({ targetId: 'ssh-1', state })
expect(onStateChanged.mock.calls[0]?.[0].state).toEqual(state)
})
it('normalizes partial compatibility authority to unknown for reconciliation', async () => {
const partialState = {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'partial-provider-epoch'
} as SshConnectionState
invoke.mockResolvedValueOnce(partialState)
await import('./index')
const api = exposeInMainWorld.mock.calls.find(([name]) => name === 'api')?.[1] as PreloadApi
const returned = await api.ssh.getState({ targetId: 'ssh-1' })
expect(returned).toEqual({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: null
})
const onStateChanged = vi.fn()
api.ssh.onStateChanged(onStateChanged)
const listener = on.mock.calls.find(([channel]) => channel === 'ssh:state-changed')?.[1] as (
event: unknown,
data: { targetId: string; state: SshConnectionState }
) => void
listener({}, { targetId: 'ssh-1', state: partialState })
expect(onStateChanged.mock.calls[0]?.[0].state).toEqual({
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: null
})
})
it('drops malformed full authority before it reaches the renderer', async () => {
await import('./index')
const api = exposeInMainWorld.mock.calls.find(([name]) => name === 'api')?.[1] as PreloadApi
const onStateChanged = vi.fn()
api.ssh.onStateChanged(onStateChanged)
const listener = on.mock.calls.find(([channel]) => channel === 'ssh:state-changed')?.[1] as (
event: unknown,
data: { targetId: string; state: SshConnectionState }
) => void
listener(
{},
{
targetId: 'ssh-1',
state: {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: '' as SshProviderEpoch,
connectionGeneration: 29
}
}
)
expect(onStateChanged).not.toHaveBeenCalled()
})
})
src/renderer/src/App.tsx
+5 -2
View File
@@ -407,8 +407,11 @@ function applyRemoteWorkspacePatchStatus(
message:
result.message ??
(result.reason === 'stale-revision'
? 'Workspace changed on another device'
: 'Remote workspace sync unavailable')
? translate(
'auto.hooks.useIpcEvents.workspaceChangedOnAnotherDevice',
'Workspace changed on another device'
)
: translate('auto.hooks.useIpcEvents.2fe88c2e06', 'Remote workspace sync unavailable'))
})
}
src/renderer/src/components/right-sidebar/plugin-panel-bridge-host.test.ts
+56 -1
View File
@@ -1,6 +1,9 @@
import { describe, expect, it, vi } from 'vitest'
import type { PluginPanelActionOutcome } from '../../../../shared/plugins/plugin-panel-bridge'
import type { PanelMessageBudget } from '../../../../shared/plugins/plugin-panel-message-budget'
import {
createPanelMessageBudget,
type PanelMessageBudget
} from '../../../../shared/plugins/plugin-panel-message-budget'
import { createPanelBridgeMessageHandler } from './plugin-panel-bridge-host'
type FakePanelWindow = Window & { postMessage: ReturnType<typeof vi.fn> }
@@ -105,6 +108,58 @@ describe('createPanelBridgeMessageHandler', () => {
)
})
it('refuses an oversized request without relaying it', () => {
const panelWindow = createFakePanelWindow()
const { handler, callPanelAction } = createHandler(panelWindow)
handler(
messageEvent(
{
...VALID_DATA,
params: { padding: 'x'.repeat(128 * 1024) }
},
panelWindow
)
)
expect(callPanelAction).not.toHaveBeenCalled()
expect(panelWindow.postMessage).toHaveBeenCalledWith(
expect.objectContaining({
type: 'orca-panel-action-result',
requestId: 'req-1',
ok: false,
errorCode: 'invalid_request'
}),
'*'
)
})
it('refuses a valid request after malformed and pong traffic exhaust the budget', () => {
const panelWindow = createFakePanelWindow()
const callPanelAction = vi.fn()
const handler = createPanelBridgeMessageHandler({
sessionToken: SESSION_TOKEN,
getPanelWindow: () => panelWindow,
callPanelAction,
budget: createPanelMessageBudget({ maxMessages: 2 })
})
handler(messageEvent({ type: 'invalid-hostile-message' }, panelWindow))
handler(messageEvent({ type: 'orca-panel-pong', pingId: 7 }, panelWindow))
handler(messageEvent(VALID_DATA, panelWindow))
expect(callPanelAction).not.toHaveBeenCalled()
expect(panelWindow.postMessage).toHaveBeenCalledWith(
expect.objectContaining({
type: 'orca-panel-action-result',
requestId: 'req-1',
ok: false,
errorCode: 'rate_limited'
}),
'*'
)
})
it('relays a denial outcome (missing manifest permission) back to the panel', async () => {
const panelWindow = createFakePanelWindow()
const { handler } = createHandler(panelWindow, {
src/renderer/src/components/terminal-pane/pty-connection-types.ts
+7 -1
View File
@@ -14,6 +14,7 @@ import type {
import type { TerminalKittyKeyboardModeTracker } from '../../../../shared/terminal-kitty-keyboard-mode-tracker'
import type { PtyTransportRecoveryState } from './pty-transport-types'
import type { SessionOptionValue } from '../../../../shared/native-chat-session-options'
import type { DirectSshPaneRetryAttemptId } from '@/store/slices/direct-ssh-terminal-recovery'
export type PtyConnectionDeps = {
tabId: string
@@ -70,7 +71,12 @@ export type PtyConnectionDeps = {
updateTabTitle: (tabId: string, title: string) => void
setRuntimePaneTitle: (tabId: string, paneId: number, title: string) => void
clearRuntimePaneTitle: (tabId: string, paneId: number) => void
updateTabPtyId: (tabId: string, ptyId: string, replacedPtyId?: string) => void
updateTabPtyId: (
tabId: string,
ptyId: string,
replacedPtyId?: string,
directSshRetryAttemptId?: DirectSshPaneRetryAttemptId
) => void
markWorktreeUnread: (worktreeId: string) => void
markTerminalTabUnread: (tabId: string) => void
markTerminalPaneUnread: (paneKey: string) => void
src/renderer/src/components/terminal-pane/pty-connection.test.ts
+1283 -3
View File
File diff suppressed because it is too large Load Diff
src/renderer/src/components/terminal-pane/pty-connection.ts
+223 -11
View File
@@ -294,9 +294,13 @@ import {
registerTerminalSideEffectFactConsumer
} from './terminal-side-effect-facts-handler'
import { isRendererHiddenPtyDeliveryGateEnabled } from './terminal-hidden-delivery-gate'
import type { DirectSshPaneRetryAttempt } from '@/store/slices/direct-ssh-terminal-recovery'
import { directSshAuthoritiesEqual } from '@/store/slices/direct-ssh-terminal-authority-ledger'
const pendingSpawnByPaneKey = new Map<string, Promise<string | null>>()
const SSH_SESSION_EXPIRED_ERROR = 'SSH_SESSION_EXPIRED'
// Why: relay requests expire at 30s; leave one second for their fallback before re-arming locally.
const DIRECT_SSH_PANE_RETRY_SETTLEMENT_TIMEOUT_MS = 31_000
const REMOTE_PTY_ID_PREFIX = 'remote:'
const PTY_CONNECT_DIAG_LIMIT = 200
const COMMAND_CODE_OUTPUT_DONE_SETTLE_MS = 1500
@@ -1262,7 +1266,6 @@ export function connectPanePty(
const clearRegisteredStartupLaunchConfig = (): void => {
useAppStore.getState().clearAgentLaunchConfig(cacheKey)
}
const pendingSpawnKey = cacheKey
const neutralTerminalTitle = (): string => {
const state = useAppStore.getState()
const tab = (state.tabsByWorktree[deps.worktreeId] ?? []).find(
@@ -2839,8 +2842,18 @@ export function connectPanePty(
deps.syncPanePtyLayoutBinding(pane.id, ptyId)
notifyCodexPaneBoundForStaleSweep(ptyId)
const tabPtyIds = useAppStore.getState().ptyIdsByTabId?.[deps.tabId] ?? []
if (options.updateTabPtyId !== 'if-missing' || !tabPtyIds.includes(ptyId)) {
if (options.replacePtyId) {
const directSshRetryAttemptId =
capturedDirectSshRetryPtyAccepted && directSshRetryAttempt
? directSshRetryAttempt.attemptId
: undefined
if (
directSshRetryAttemptId ||
options.updateTabPtyId !== 'if-missing' ||
!tabPtyIds.includes(ptyId)
) {
if (directSshRetryAttemptId) {
deps.updateTabPtyId(deps.tabId, ptyId, options.replacePtyId, directSshRetryAttemptId)
} else if (options.replacePtyId) {
deps.updateTabPtyId(deps.tabId, ptyId, options.replacePtyId)
} else {
deps.updateTabPtyId(deps.tabId, ptyId)
@@ -2876,6 +2889,15 @@ export function connectPanePty(
}
const onPtySpawn = (ptyId: string): void => {
if (!claimCapturedDirectSshRetryPty(ptyId)) {
// Why: this callback proves a fresh process was created, so rejecting its obsolete lease must also retire it.
queueMicrotask(() => {
if (transport.getPtyId() === ptyId) {
transport.disconnect()
}
})
return
}
// Why: record that this exact PTY was freshly spawned (not reattached), so a
// newborn shell that dies before any interaction (e.g. failing direnv on a
// just-created worktree) can be kept visible rather than tearing down the
@@ -2886,6 +2908,9 @@ export function connectPanePty(
bindActivePanePty(ptyId, { seedInitialAgentStatus: true })
}
const onPtyRebind = (ptyId: string, replacedPtyId: string): void => {
if (!canAdoptCapturedDirectSshRetryPty(ptyId)) {
return
}
// Why: provider handle rotation keeps the existing pane/session generation;
// replace its stale store identity without fresh-spawn exit semantics.
bindActivePanePty(ptyId, { replacePtyId: replacedPtyId })
@@ -3274,6 +3299,126 @@ export function connectPanePty(
!terminalOwnerUnresolved && !connectionOwnerHydrating && runtimeEnvironmentId === null
? (worktreeConnectionId ?? null)
: null
type DirectSshRetryLease = Pick<
DirectSshPaneRetryAttempt,
'attemptId' | 'authority' | 'tabGeneration'
>
const directSshRetryAttempt: DirectSshRetryLease | undefined = (() => {
const pendingAttempt = state.directSshPaneRetryByTabId?.[deps.tabId]
const liveBinding = state.directSshLivePtyBindingByTabId?.[deps.tabId]
const attempt =
pendingAttempt?.authority.targetId === connectionId &&
pendingAttempt.tabGeneration === (tab?.generation ?? 0)
? pendingAttempt
: liveBinding?.authority.targetId === connectionId &&
liveBinding.tabGeneration === (tab?.generation ?? 0)
? liveBinding
: undefined
return attempt
})()
const pendingSpawnKey = directSshRetryAttempt
? JSON.stringify([cacheKey, directSshRetryAttempt.attemptId])
: cacheKey
let capturedDirectSshRetryPtyAccepted = false
let directSshPaneRetrySettlementCancelled = false
const directSshPaneRetrySettlementTimers = new Set<ReturnType<typeof setTimeout>>()
const directSshPaneRetryTimedPromises = new WeakSet<object>()
const capturedDirectSshRetryLeaseMatches = (): boolean => {
if (!directSshRetryAttempt) {
return true
}
const currentState = useAppStore.getState()
const currentConnection = currentState.sshConnectionStates.get(
directSshRetryAttempt.authority.targetId
)
const currentTab = (currentState.tabsByWorktree[deps.worktreeId] ?? []).find(
(candidate) => candidate.id === deps.tabId
)
if (
currentConnection?.providerEpoch !== directSshRetryAttempt.authority.providerEpoch ||
currentConnection.connectionGeneration !==
directSshRetryAttempt.authority.connectionGeneration ||
(currentTab?.generation ?? 0) !== directSshRetryAttempt.tabGeneration
) {
return false
}
const pendingAttempt = currentState.directSshPaneRetryByTabId?.[deps.tabId]
const pendingMatches =
pendingAttempt?.attemptId === directSshRetryAttempt.attemptId &&
directSshAuthoritiesEqual(pendingAttempt.authority, directSshRetryAttempt.authority) &&
pendingAttempt.tabGeneration === directSshRetryAttempt.tabGeneration
const liveBinding = currentState.directSshLivePtyBindingByTabId?.[deps.tabId]
const liveBindingMatchesAttempt =
liveBinding?.attemptId === directSshRetryAttempt.attemptId &&
directSshAuthoritiesEqual(liveBinding.authority, directSshRetryAttempt.authority) &&
liveBinding.tabGeneration === directSshRetryAttempt.tabGeneration
return pendingMatches || liveBindingMatchesAttempt
}
const capturedDirectSshRetryStateMatches = (ptyId: string): boolean => {
if (!directSshRetryAttempt) {
return true
}
const currentConnection = useAppStore
.getState()
.sshConnectionStates.get(directSshRetryAttempt.authority.targetId)
return (
parseAppSshPtyId(ptyId)?.connectionId === directSshRetryAttempt.authority.targetId &&
currentConnection?.status === 'connected' &&
capturedDirectSshRetryLeaseMatches()
)
}
const claimCapturedDirectSshRetryPty = (ptyId: string): boolean => {
if (!capturedDirectSshRetryStateMatches(ptyId)) {
return false
}
capturedDirectSshRetryPtyAccepted = directSshRetryAttempt !== undefined
return true
}
const canAdoptCapturedDirectSshRetryPty = (ptyId: string): boolean => {
const canAdopt = capturedDirectSshRetryStateMatches(ptyId)
if (canAdopt && directSshRetryAttempt) {
capturedDirectSshRetryPtyAccepted = true
}
return canAdopt
}
const settleDirectSshPaneRetryAttempt = (
attempt: DirectSshRetryLease | undefined,
status: 'failed' | 'timed-out'
): void => {
if (!attempt) {
return
}
useAppStore.getState().settleDirectSshPaneRetry?.({
status,
tabId: deps.tabId,
attemptId: attempt.attemptId,
authority: attempt.authority,
tabGeneration: attempt.tabGeneration
})
}
const armDirectSshPaneRetryTimeout = (
promise: Promise<unknown>,
attempt: DirectSshRetryLease | undefined
): void => {
if (!attempt || disposed || directSshPaneRetryTimedPromises.has(promise)) {
return
}
directSshPaneRetryTimedPromises.add(promise)
const timer = setTimeout(() => {
directSshPaneRetrySettlementTimers.delete(timer)
if (directSshPaneRetrySettlementCancelled) {
return
}
settleDirectSshPaneRetryAttempt(attempt, 'timed-out')
}, DIRECT_SSH_PANE_RETRY_SETTLEMENT_TIMEOUT_MS)
directSshPaneRetrySettlementTimers.add(timer)
void promise
.finally(() => {
directSshPaneRetrySettlementTimers.delete(timer)
clearTimeout(timer)
})
.catch(() => {})
}
const shellOverride = tab?.shellOverride
// Why: a serve/remote-runtime pane has no SSH connectionId and a Linux cwd, so
// the native-Windows ConPTY heuristic misfires on a Windows client and wrongly
@@ -4911,6 +5056,9 @@ export function connectPanePty(
: typeof spawnedPtyId === 'string'
? spawnedPtyId
: transport.getPtyId()
if (resolvedPtyId && !claimCapturedDirectSshRetryPty(resolvedPtyId)) {
return null
}
if (spawnedPtyId && typeof spawnedPtyId === 'object' && 'id' in spawnedPtyId) {
registerEffectiveLaunchConfig(spawnedPtyId.launchConfig, {
...(coldRestoreOverride ? { launchToken: coldRestoreOverride.launchToken } : {}),
@@ -4997,6 +5145,18 @@ export function connectPanePty(
pendingSpawnByPaneKey.delete(pendingSpawnKey)
}
})
armDirectSshPaneRetryTimeout(trackedPromise, directSshRetryAttempt)
void trackedPromise.then((spawnedPtyId) => {
if (spawnedPtyId) {
return
}
queueMicrotask(() => {
if (disposed || transport.getPtyId() || pendingSpawnByPaneKey.has(pendingSpawnKey)) {
return
}
settleDirectSshPaneRetryAttempt(directSshRetryAttempt, 'failed')
})
})
// Why: split panes in the same tab can spawn concurrently. Key by pane
// as well as tab so a remount cannot attach to a sibling setup pane's PTY.
pendingSpawnByPaneKey.set(pendingSpawnKey, trackedPromise)
@@ -7319,6 +7479,16 @@ export function connectPanePty(
}
}
const isCapturedDirectSshReattachCurrent = (ptyId: string): boolean =>
!directSshRetryAttempt || capturedDirectSshRetryStateMatches(ptyId)
const rejectObsoleteDirectSshReattach = (ptyId: string | null | undefined): boolean => {
if (!directSshRetryAttempt || (ptyId && claimCapturedDirectSshRetryPty(ptyId))) {
return false
}
transport.detach?.({ preserveExitObserver: false })
return true
}
const handleReattachResult = async (
result: PtyConnectResult | string | void,
staleSessionId?: string | null,
@@ -7339,6 +7509,13 @@ export function connectPanePty(
return true
}
const retryPtyId =
connectResult?.id ??
(typeof result === 'string' ? result : (staleSessionId ?? transport.getPtyId()))
if (rejectObsoleteDirectSshReattach(retryPtyId)) {
// Why: an obsolete reattach must stop consuming frames without killing the durable PTY a newer lease may adopt.
return false
}
const ptyId =
connectResult?.id ?? (typeof result === 'string' ? result : transport.getPtyId())
if (!ptyId) {
@@ -7425,7 +7602,11 @@ export function connectPanePty(
syncHiddenRendererPtyDelivery()
deps.syncPanePtyLayoutBinding(pane.id, ptyId)
notifyCodexPaneBoundForStaleSweep(ptyId)
deps.updateTabPtyId(deps.tabId, ptyId)
if (capturedDirectSshRetryPtyAccepted && directSshRetryAttempt) {
deps.updateTabPtyId(deps.tabId, ptyId, undefined, directSshRetryAttempt.attemptId)
} else {
deps.updateTabPtyId(deps.tabId, ptyId)
}
agentCompletionCoordinator.startProcessTracking()
sampleVisiblePaneForegroundAgent()
@@ -7678,7 +7859,7 @@ export function connectPanePty(
console.warn('[pty-connection] needsPassphrasePrompt probe failed:', err)
// Why: on probe failure fall through to auto-connect rather than stranding the tab — a stuck tab is worse than a surprising prompt.
}
if (disposed) {
if (disposed || !capturedDirectSshRetryLeaseMatches()) {
return
}
if (needsPrompt) {
@@ -7743,7 +7924,7 @@ export function connectPanePty(
finish(currentOutcome)
}
})
if (disposed) {
if (disposed || !capturedDirectSshRetryLeaseMatches()) {
return
}
if (outcome === 'cancelled') {
@@ -7758,11 +7939,11 @@ export function connectPanePty(
// Why: wait for the shared SSH connection (multiple panes/tabs may need it) before PTY reattach, rather than returning early when it's in-flight.
const connectResult = await waitForSshConnection(connectionId)
if (!connectResult.connected) {
reportError(`SSH connection failed: ${connectResult.error}`)
if (disposed || !capturedDirectSshRetryLeaseMatches()) {
return
}
if (disposed) {
if (!connectResult.connected) {
reportError(`SSH connection failed: ${connectResult.error}`)
return
}
useAppStore.getState().removeDeferredSshReconnectTarget(connectionId)
@@ -7790,6 +7971,9 @@ export function connectPanePty(
expiredReattachError = true
return
}
if (!isCapturedDirectSshReattachCurrent(pendingSessionId)) {
return
}
reportError(message)
})
beginReattachLiveDataDeferral(outputCallbacks.generation)
@@ -7814,6 +7998,7 @@ export function connectPanePty(
: {}),
...(coldRestoreStartup?.agent ? { launchAgent: coldRestoreStartup.agent } : {}),
...(shouldDeclareHiddenAtSpawn() ? { initiallyHidden: true } : {}),
...(directSshRetryAttempt ? { admitPtyId: claimCapturedDirectSshRetryPty } : {}),
callbacks: outputCallbacks.callbacks
})
void Promise.resolve(reattachPromise)
@@ -7821,7 +8006,7 @@ export function connectPanePty(
.finally(() => {
transportConnectInFlightSince = null
})
void Promise.resolve(reattachPromise)
const trackedReattachPromise = Promise.resolve(reattachPromise)
.then(async (result) => {
if (outputCallbacks.generation !== transportStreamGeneration) {
finishReattachLiveDataDeferral(false, outputCallbacks.generation)
@@ -7849,6 +8034,9 @@ export function connectPanePty(
if (disposed) {
return
}
if (rejectObsoleteDirectSshReattach(pendingSessionId)) {
return
}
deps.clearExitedPanePtyLayoutBinding(pane.id, pendingSessionId)
deps.clearTabPtyId(deps.tabId, pendingSessionId)
startFreshColdRestoreAgentResume(coldRestoreStartup, {
@@ -7892,6 +8080,9 @@ export function connectPanePty(
if (disposed || outputCallbacks.generation !== transportStreamGeneration) {
return
}
if (rejectObsoleteDirectSshReattach(pendingSessionId)) {
return
}
if (isSshSessionExpiredError(err)) {
deps.clearExitedPanePtyLayoutBinding(pane.id, pendingSessionId)
deps.clearTabPtyId(deps.tabId, pendingSessionId)
@@ -7904,6 +8095,7 @@ export function connectPanePty(
forceBlankRestoredViewport: true
})
})
armDirectSshPaneRetryTimeout(trackedReattachPromise, directSshRetryAttempt)
} else {
startFreshColdRestoreAgentResume()
}
@@ -8013,6 +8205,9 @@ export function connectPanePty(
expiredReattachError = true
return
}
if (!isCapturedDirectSshReattachCurrent(deferredReattachSessionId)) {
return
}
reportError(message)
})
beginReattachLiveDataDeferral(outputCallbacks.generation)
@@ -8035,6 +8230,7 @@ export function connectPanePty(
...(coldRestoreStartup?.launchToken ? { launchToken: coldRestoreStartup.launchToken } : {}),
...(coldRestoreStartup?.agent ? { launchAgent: coldRestoreStartup.agent } : {}),
...(shouldDeclareHiddenAtSpawn() ? { initiallyHidden: true } : {}),
...(directSshRetryAttempt ? { admitPtyId: claimCapturedDirectSshRetryPty } : {}),
callbacks: outputCallbacks.callbacks
})
@@ -8043,7 +8239,7 @@ export function connectPanePty(
.finally(() => {
transportConnectInFlightSince = null
})
void Promise.resolve(reattachPromise)
const trackedReattachPromise = Promise.resolve(reattachPromise)
.then(async (result) => {
if (outputCallbacks.generation !== transportStreamGeneration) {
finishReattachLiveDataDeferral(false, outputCallbacks.generation)
@@ -8062,6 +8258,9 @@ export function connectPanePty(
if (disposed) {
return
}
if (rejectObsoleteDirectSshReattach(deferredReattachSessionId)) {
return
}
deps.clearExitedPanePtyLayoutBinding(pane.id, deferredReattachSessionId)
deps.clearTabPtyId(deps.tabId, deferredReattachSessionId)
startFreshColdRestoreAgentResume(coldRestoreStartup, {
@@ -8105,6 +8304,9 @@ export function connectPanePty(
if (outputCallbacks.generation !== transportStreamGeneration) {
return
}
if (rejectObsoleteDirectSshReattach(deferredReattachSessionId)) {
return
}
warnTerminalLifecycleAnomaly('restored PTY reattach threw', {
tabId: deps.tabId,
worktreeId: deps.worktreeId,
@@ -8126,6 +8328,7 @@ export function connectPanePty(
forceBlankRestoredViewport: true
})
})
armDirectSshPaneRetryTimeout(trackedReattachPromise, directSshRetryAttempt)
} else if (detachedRemoteLeafPtyId || detachedLivePtyId || eagerLivePtyId) {
// Why: mirrored web-leaf panes must attach to their exact remote PTY, not spawn a replacement host tab.
// eagerLivePtyId covers a still-live background PTY (e.g. an automation agent) with a live eager buffer to adopt.
@@ -8162,6 +8365,7 @@ export function connectPanePty(
const pendingSpawn = pendingSpawnByPaneKey.get(pendingSpawnKey)
if (pendingSpawn) {
recordPtyConnectDiagnostic(`pane=${pane.id} -> PENDING SPAWN`)
armDirectSshPaneRetryTimeout(pendingSpawn, directSshRetryAttempt)
void pendingSpawn
.then((spawnedPtyId) => {
if (disposed) {
@@ -8185,6 +8389,9 @@ export function connectPanePty(
}
return
}
if (!canAdoptCapturedDirectSshRetryPty(spawnedPtyId)) {
return
}
clearPaneMode2031State()
clearHiddenOutputRestoreState()
const outputCallbacks = captureTransportOutputCallbacks(reportError)
@@ -8366,6 +8573,11 @@ export function connectPanePty(
reconcileIfSessionMissing,
dispose() {
disposed = true
directSshPaneRetrySettlementCancelled = true
for (const timer of directSshPaneRetrySettlementTimers) {
clearTimeout(timer)
}
directSshPaneRetrySettlementTimers.clear()
// Why: a stalled xterm replay may never reach its finally; release live-frame credit when this renderer no longer owns the stream.
for (const chunk of deferredReattachLiveData ?? []) {
chunk.ackCredit?.()
src/renderer/src/components/terminal-pane/pty-transport-types.ts
+3 -1
View File
@@ -119,6 +119,8 @@ export type PtyTransport = {
launchToken?: string
launchAgent?: TuiAgent
startupCommandDelivery?: StartupCommandDelivery
/** Reject a stale restored identity before this transport can publish global PTY handlers. */
admitPtyId?: (ptyId: string) => boolean
callbacks: PtyCallbacks
}) => void | Promise<void | string | PtyConnectResult>
attach: (options: {
@@ -170,7 +172,7 @@ export type PtyTransport = {
resetCrossChunkParserState?: () => void
serializeBuffer?: (opts?: { scrollbackRows?: number }) => Promise<PtyBufferSnapshot | null>
preserve?: () => void
detach?: () => void
detach?: (options?: { preserveExitObserver?: boolean }) => void
destroy?: () => void | Promise<void>
}
src/renderer/src/components/terminal-pane/pty-transport.test.ts
+205 -3
View File
@@ -277,6 +277,109 @@ describe('createIpcPtyTransport', () => {
expect(exitSeenByNewPane).toHaveBeenCalledWith(0)
})
it('rejects a stale reattach before it can replace newer PTY handlers', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const spawn = window.api.pty.spawn as unknown as ReturnType<typeof vi.fn>
let resolveStale!: (value: { id: string; isReattach: boolean }) => void
spawn.mockReturnValueOnce(
new Promise((resolve) => {
resolveStale = resolve
})
)
const staleData = vi.fn()
const staleExit = vi.fn()
const stalePane = createIpcPtyTransport({})
const staleConnect = stalePane.connect({
url: '',
sessionId: 'pty-1',
admitPtyId: () => false,
callbacks: { onData: staleData, onExit: staleExit }
})
const currentData = vi.fn()
const currentExit = vi.fn()
const currentPane = createIpcPtyTransport({})
currentPane.attach({
existingPtyId: 'pty-1',
callbacks: { onData: currentData, onExit: currentExit }
})
resolveStale({ id: 'pty-1', isReattach: true })
await staleConnect
onData?.({ id: 'pty-1', data: 'current output' })
onExit?.({ id: 'pty-1', code: 0 })
expect(currentData).toHaveBeenCalledWith('current output')
expect(currentExit).toHaveBeenCalledWith(0)
expect(staleData).not.toHaveBeenCalled()
expect(staleExit).not.toHaveBeenCalled()
expect(window.api.pty.kill).not.toHaveBeenCalled()
})
it('retires a rejected fresh fallback before it can publish PTY handlers', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const spawn = window.api.pty.spawn as unknown as ReturnType<typeof vi.fn>
const kill = window.api.pty.kill as unknown as ReturnType<typeof vi.fn>
const onPtySpawn = vi.fn()
const onDataCallback = vi.fn()
const onExitCallback = vi.fn()
spawn.mockResolvedValueOnce({ id: 'pty-fresh-fallback', sessionExpired: true })
const transport = createIpcPtyTransport({ onPtySpawn })
const result = await transport.connect({
url: '',
sessionId: 'pty-missing',
admitPtyId: () => false,
callbacks: { onData: onDataCallback, onExit: onExitCallback }
})
onData?.({ id: 'pty-fresh-fallback', data: 'orphaned output' })
onExit?.({ id: 'pty-fresh-fallback', code: 0 })
expect(result).toEqual({ id: 'pty-fresh-fallback', sessionExpired: true })
expect(kill).toHaveBeenCalledExactlyOnceWith('pty-fresh-fallback')
expect(onPtySpawn).not.toHaveBeenCalled()
expect(onDataCallback).not.toHaveBeenCalled()
expect(onExitCallback).not.toHaveBeenCalled()
expect(transport.getPtyId()).toBeNull()
expect(transport.isConnected()).toBe(false)
})
it('surfaces rejected fresh fallback retirement without publishing PTY handlers', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const spawn = window.api.pty.spawn as unknown as ReturnType<typeof vi.fn>
const kill = window.api.pty.kill as unknown as ReturnType<typeof vi.fn>
const onPtySpawn = vi.fn()
const onDataCallback = vi.fn()
const onExitCallback = vi.fn()
const onErrorCallback = vi.fn()
const retirementError = new Error('provider shutdown refused')
spawn.mockResolvedValueOnce({ id: 'pty-fresh-fallback', sessionExpired: true })
kill.mockRejectedValueOnce(retirementError)
const transport = createIpcPtyTransport({ onPtySpawn })
await expect(
transport.connect({
url: '',
sessionId: 'pty-missing',
admitPtyId: () => false,
callbacks: {
onData: onDataCallback,
onExit: onExitCallback,
onError: onErrorCallback
}
})
).resolves.toBeUndefined()
onData?.({ id: 'pty-fresh-fallback', data: 'orphaned output' })
onExit?.({ id: 'pty-fresh-fallback', code: 0 })
expect(kill).toHaveBeenCalledExactlyOnceWith('pty-fresh-fallback')
expect(onPtySpawn).not.toHaveBeenCalled()
expect(onDataCallback).not.toHaveBeenCalled()
expect(onExitCallback).not.toHaveBeenCalled()
expect(onErrorCallback).toHaveBeenCalledExactlyOnceWith(retirementError.message)
expect(transport.getPtyId()).toBeNull()
expect(transport.isConnected()).toBe(false)
})
it('buffers data across a normal detach-then-attach gap and drains it to the next pane', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const receivedByNewPane = vi.fn()
@@ -463,6 +566,38 @@ describe('createIpcPtyTransport', () => {
expect(transport.isConnected()).toBe(false)
})
it('rejects a buffered dead-session exit before publishing its final frame', async () => {
const { bufferPreHandlerPtyData, bufferPreHandlerPtyExit, clearPreHandlerPtyState } =
await import('./pty-pre-handler-buffer')
const { createIpcPtyTransport } = await import('./pty-transport')
const spawn = window.api.pty.spawn as unknown as ReturnType<typeof vi.fn>
const onDataCallback = vi.fn()
const onExitCallback = vi.fn()
const onDisconnect = vi.fn()
const onPtyExit = vi.fn()
const sessionId = 'stale-dead-parked-session'
bufferPreHandlerPtyData(sessionId, 'stale final output')
bufferPreHandlerPtyExit(sessionId, 17)
const transport = createIpcPtyTransport({ onPtyExit })
const result = await transport.connect({
url: '',
sessionId,
admitPtyId: () => false,
callbacks: { onData: onDataCallback, onExit: onExitCallback, onDisconnect }
})
expect(result).toEqual({ id: sessionId })
expect(spawn).not.toHaveBeenCalled()
expect(onDataCallback).not.toHaveBeenCalled()
expect(onExitCallback).not.toHaveBeenCalled()
expect(onDisconnect).not.toHaveBeenCalled()
expect(onPtyExit).not.toHaveBeenCalled()
expect(window.api.pty.kill).not.toHaveBeenCalled()
expect(transport.isConnected()).toBe(false)
clearPreHandlerPtyState(sessionId)
})
it('returns startup cwd fallback metadata to the connection layer', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const spawn = window.api.pty.spawn as unknown as ReturnType<typeof vi.fn>
@@ -1527,8 +1662,10 @@ describe('createIpcPtyTransport', () => {
it('does not kill a pre-existing session when a reattach resolves after destroy', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const spawnControls: { resolve: ((value: { id: string }) => void) | null } = { resolve: null }
const spawnPromise = new Promise<{ id: string }>((resolve) => {
const spawnControls: {
resolve: ((value: { id: string; isReattach: true }) => void) | null
} = { resolve: null }
const spawnPromise = new Promise<{ id: string; isReattach: true }>((resolve) => {
spawnControls.resolve = resolve
})
const spawnMock = vi.fn().mockReturnValue(spawnPromise)
@@ -1563,12 +1700,41 @@ describe('createIpcPtyTransport', () => {
if (!spawnControls.resolve) {
throw new Error('Expected spawn resolver to be captured')
}
spawnControls.resolve({ id: 'pty-preexisting' })
spawnControls.resolve({ id: 'pty-preexisting', isReattach: true })
await connectPromise
expect(killMock).not.toHaveBeenCalledWith('pty-preexisting')
})
it('kills a fresh session fallback that resolves after the transport was destroyed', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const spawnControls: {
resolve: ((value: { id: string; sessionExpired: true }) => void) | null
} = { resolve: null }
const spawnPromise = new Promise<{ id: string; sessionExpired: true }>((resolve) => {
spawnControls.resolve = resolve
})
const spawn = window.api.pty.spawn as unknown as ReturnType<typeof vi.fn>
const kill = window.api.pty.kill as unknown as ReturnType<typeof vi.fn>
spawn.mockReturnValueOnce(spawnPromise)
const transport = createIpcPtyTransport({})
const connectPromise = transport.connect({
url: '',
sessionId: 'pty-missing',
callbacks: {}
})
transport.destroy?.()
if (!spawnControls.resolve) {
throw new Error('Expected spawn resolver to be captured')
}
spawnControls.resolve({ id: 'pty-fresh-fallback', sessionExpired: true })
await connectPromise
expect(kill).toHaveBeenCalledExactlyOnceWith('pty-fresh-fallback')
expect(transport.getPtyId()).toBeNull()
})
it('kills a PTY that finishes spawning after the transport was destroyed', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const spawnControls: { resolve: ((value: { id: string }) => void) | null } = { resolve: null }
@@ -2025,6 +2191,42 @@ describe('createIpcPtyTransport', () => {
expect(onPtyExit).toHaveBeenCalledWith('pty-detached')
expect(transport.getPtyId()).toBeNull()
})
it('drops the exit observer when abandoning an obsolete reattach without killing it', async () => {
const { createIpcPtyTransport } = await import('./pty-transport')
const onPtyExit = vi.fn()
const kill = window.api.pty.kill as unknown as ReturnType<typeof vi.fn>
const transport = createIpcPtyTransport({ onPtyExit })
const onDataCallback = vi.fn()
const onReplayData = vi.fn()
const onWriteUnavailableCallback = vi.fn()
const onExitCallback = vi.fn()
const onDisconnect = vi.fn()
transport.attach({
existingPtyId: 'pty-obsolete',
callbacks: {
onData: onDataCallback,
onReplayData,
onWriteUnavailable: onWriteUnavailableCallback,
onExit: onExitCallback,
onDisconnect
}
})
transport.detach?.({ preserveExitObserver: false })
onData?.({ id: 'pty-obsolete', data: 'stale data' })
onReplay?.({ id: 'pty-obsolete', data: 'stale replay' })
onWriteUnavailable?.({ id: 'pty-obsolete' })
onExit?.({ id: 'pty-obsolete', code: 0 })
expect(onDataCallback).not.toHaveBeenCalled()
expect(onReplayData).not.toHaveBeenCalled()
expect(onWriteUnavailableCallback).not.toHaveBeenCalled()
expect(onExitCallback).not.toHaveBeenCalled()
expect(onDisconnect).not.toHaveBeenCalled()
expect(onPtyExit).not.toHaveBeenCalled()
expect(kill).not.toHaveBeenCalled()
})
})
describe('createRemoteRuntimePtyTransport', () => {
src/renderer/src/components/terminal-pane/pty-transport.ts
+21 -5
View File
@@ -696,6 +696,9 @@ export function createIpcPtyTransport(opts: IpcPtyTransportOptions = {}): PtyTra
}
if (options.sessionId && hasPreHandlerPtyExit(options.sessionId)) {
if (options.admitPtyId && !options.admitPtyId(options.sessionId)) {
return { id: options.sessionId } satisfies PtyConnectResult
}
// Why: deliver the exited parked session's buffered final frame/exit before spawn, so the dead incarnation can't orphan a fresh shell reusing its id.
ptyId = options.sessionId
connected = true
@@ -761,15 +764,24 @@ export function createIpcPtyTransport(opts: IpcPtyTransportOptions = {}): PtyTra
const resultLaunchAgent = isTuiAgent(spawnResult.launchAgent)
? spawnResult.launchAgent
: undefined
const retireFreshSpawn = async (): Promise<void> => {
if (!spawnResult.isReattach && !spawnResult.coldRestore) {
await window.api.pty.kill(spawnResult.id)
}
}
// Why: on destroy mid-connect, kill only a fresh spawn — killing a reattached session (owned by the tab lifecycle) loses a live shell.
if (destroyed) {
if (!options.sessionId) {
window.api.pty.kill(spawnResult.id)
}
await retireFreshSpawn()
return
}
if (options.admitPtyId && !options.admitPtyId(spawnResult.id)) {
// Why: a rejected session-expired fallback has no owner to retire its newly created process.
await retireFreshSpawn()
return spawnResult
}
ptyId = spawnResult.id
connected = true
@@ -933,12 +945,16 @@ export function createIpcPtyTransport(opts: IpcPtyTransportOptions = {}): PtyTra
}
},
detach() {
detach(options) {
clearAccumulatedState()
inputWriteQueue.clear()
if (ptyId) {
// Why: on remount keep the exit observer alive so a shell dying in the gap still clears stale tab/leaf bindings before reattach.
unregisterPtyDataAndStatusHandlers(ptyId)
if (options?.preserveExitObserver === false) {
unregisterPtyHandlers(ptyId)
} else {
unregisterPtyDataAndStatusHandlers(ptyId)
}
}
connected = false
ptyId = null
src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.test.ts
+35
View File
@@ -416,4 +416,39 @@ describe('detachTerminalPaneToTab', () => {
ptyIdsByLeafId: { [LEAF_2]: 'remote:env-2@@terminal-9' }
})
})
it('keeps a detached null-PTY leaf eligible to finish its pending activation', () => {
const store = createStore({
root: {
type: 'split',
direction: 'vertical',
first: { type: 'leaf', leafId: LEAF_1 },
second: { type: 'leaf', leafId: LEAF_2 }
},
activeLeafId: LEAF_2,
expandedLeafId: null
})
const manager = {
getPanes: vi.fn(() => [{ id: 1 }, { id: 2 }]),
getLeafId: vi.fn(() => LEAF_2),
detachPaneForExternalMove: vi.fn(() => true)
}
const result = detachTerminalPaneToTab({
getStore: () => store,
manager,
persistLayoutSnapshot: vi.fn(),
sourcePaneId: 2,
sourceTabId: SOURCE_TAB_ID,
targetGroupId: TARGET_GROUP_ID,
worktreeId: WORKTREE_ID
})
expect(result?.ptyId).toBeNull()
expect(store.createTab).toHaveBeenCalledWith(WORKTREE_ID, TARGET_GROUP_ID, 'powershell.exe', {
activate: true,
pendingActivationSpawn: true,
recordInteraction: true
})
})
})
src/renderer/src/components/terminal-pane/terminal-pane-tab-detach.ts
+1
View File
@@ -285,6 +285,7 @@ export function detachTerminalPaneToTab(args: {
const tab = latestStore.createTab(args.worktreeId, args.targetGroupId, sourceShellOverride, {
activate: true,
initialPtyId: ptyId ?? undefined,
...(!ptyId ? { pendingActivationSpawn: true } : {}),
recordInteraction: true
})
const afterCreateStore = args.getStore()
src/renderer/src/components/terminal-pane/use-terminal-pane-lifecycle.ts
+7 -1
View File
@@ -25,6 +25,7 @@ import { buildWindowsPtyCompatibilityOptions } from '@/lib/pane-manager/windows-
import { buildTerminalKeyboardProtocolOptions } from '@/lib/pane-manager/terminal-keyboard-protocol'
import { resolvePaneKeyboardProtocolAgent } from './terminal-keyboard-protocol-pane-agent'
import { useAppStore } from '@/store'
import type { DirectSshPaneRetryAttemptId } from '@/store/slices/direct-ssh-terminal-recovery'
import {
createFilePathLinkProvider,
getTerminalFileOpenHint,
@@ -276,7 +277,12 @@ type UseTerminalPaneLifecycleDeps = {
updateTabTitle: (tabId: string, title: string) => void
setRuntimePaneTitle: (tabId: string, paneId: number, title: string) => void
clearRuntimePaneTitle: (tabId: string, paneId: number) => void
updateTabPtyId: (tabId: string, ptyId: string, replacedPtyId?: string) => void
updateTabPtyId: (
tabId: string,
ptyId: string,
replacedPtyId?: string,
directSshRetryAttemptId?: DirectSshPaneRetryAttemptId
) => void
markWorktreeUnread: (worktreeId: string) => void
markTerminalTabUnread: (tabId: string) => void
markTerminalPaneUnread: (paneKey: string) => void
src/renderer/src/env.d.ts
Vendored
+1
View File
@@ -79,6 +79,7 @@ declare global {
// oxlint-disable-next-line typescript-eslint/consistent-type-definitions -- declaration merging requires interface
interface ImportMetaEnv {
readonly VITE_DIRECT_SSH_RECONNECT_COORDINATOR?: string
readonly VITE_EXPOSE_STORE?: boolean
}
src/renderer/src/hooks/direct-ssh-host-hydration-scope.ts
+20
View File
@@ -0,0 +1,20 @@
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import { resolveDirectSshTargetScope } from '../lib/direct-ssh-target-scope'
import type { AppState } from '../store/types'
export function directSshHostHydrationScope(
state: AppState,
authority: DirectSshAuthority,
catalogRevision: number
) {
return resolveDirectSshTargetScope({
targetId: authority.targetId,
catalogRevision,
repos: state.repos,
worktreesByRepo: state.worktreesByRepo,
detectedWorktreesByRepo: state.detectedWorktreesByRepo,
folderWorkspaces: state.folderWorkspaces,
projectGroups: state.projectGroups,
restoredRuntimeHostIdByWorkspaceSessionKey: state.restoredRuntimeHostIdByWorkspaceSessionKey
})
}
src/renderer/src/hooks/direct-ssh-host-hydration-telemetry.ts
+17
View File
@@ -0,0 +1,17 @@
import type { DirectSshTargetScope } from '../lib/direct-ssh-target-scope'
import type { DirectSshPreparationInputTelemetry } from './direct-ssh-reconnect-coordinator'
export function directSshHostHydrationTelemetry(
scope: DirectSshTargetScope,
catalogOutcome: DirectSshPreparationInputTelemetry['catalogOutcome'],
catalogDurationMs: number
): DirectSshPreparationInputTelemetry {
return {
catalogOutcome,
catalogDurationMs,
gitWorktreeCount: scope.gitWorktreeIds.size,
folderWorkspaceCount: Math.max(0, scope.terminalWorkspaceKeys.size - scope.gitWorktreeIds.size),
ambiguousOwnerCount: scope.ambiguousOwnerCount,
contradictoryOwnerCount: scope.contradictoryOwnerCount
}
}
src/renderer/src/hooks/direct-ssh-host-hydration.test.ts
+390
View File
@@ -0,0 +1,390 @@
import { createStore } from 'zustand/vanilla'
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { HostLineageSnapshot } from '../../../shared/host-lineage-contract'
import type { HostRepoCatalogSnapshot } from '../../../shared/host-repo-catalog-contract'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../shared/ssh-types'
import type { Repo } from '../../../shared/types'
import { folderWorkspaceKey, worktreeWorkspaceKey } from '../../../shared/workspace-scope'
import type { AppState } from '../store/types'
import { createDirectSshHostHydration } from './direct-ssh-host-hydration'
function authority(targetId = 'target-a', epoch = 'epoch-a'): DirectSshAuthority {
return {
targetId,
providerEpoch: epoch as SshProviderEpoch,
connectionGeneration: 1
}
}
function repo(id: string, targetId: string | null): Repo {
return {
id,
path: targetId ? `/${targetId}/${id}` : `/local/${id}`,
projectGroupId: null,
connectionId: targetId,
executionHostId: targetId ? (`ssh:${targetId}` as const) : ('local' as const)
} as Repo
}
function state(overrides: Record<string, unknown> = {}): AppState {
return {
repos: [],
worktreesByRepo: {},
detectedWorktreesByRepo: {},
folderWorkspaces: [],
projectGroups: [],
restoredRuntimeHostIdByWorkspaceSessionKey: {},
worktreeLineageById: {},
workspaceLineageByChildKey: {},
...overrides
} as unknown as AppState
}
function hostSnapshot(
owner: DirectSshAuthority,
repos: ReturnType<typeof repo>[]
): HostRepoCatalogSnapshot {
return {
authoritative: true as const,
authority: {
kind: 'direct-ssh' as const,
executionHostId: `ssh:${owner.targetId}` as const,
...owner
},
repos
}
}
afterEach(() => {
vi.useRealTimers()
})
describe('createDirectSshHostHydration', () => {
it('replaces only the exact SSH host catalog when repo IDs collide', async () => {
const owner = authority()
const store = createStore<AppState>(() =>
state({
repos: [repo('shared', null), repo('shared', 'target-a'), repo('shared', 'target-b')]
})
)
const hydration = createDirectSshHostHydration({
store,
listRepos: vi.fn(async () => hostSnapshot(owner, [repo('shared', 'target-a')])),
listLineage: vi.fn(),
isCurrentAuthority: () => true
})
const input = await hydration.capturePreparationInput(owner, 'reconnect')
expect(input?.repoRefs).toEqual([{ repoId: 'shared', executionHostId: 'ssh:target-a' }])
expect(store.getState().repos).toEqual([
repo('shared', null),
repo('shared', 'target-b'),
repo('shared', 'target-a')
])
})
it('rejects a mismatched host response without publishing', async () => {
const owner = authority()
const store = createStore<AppState>(() => state({ repos: [repo('cached', 'target-a')] }))
const before = store.getState().repos
let publications = 0
store.subscribe(() => {
publications += 1
})
const hydration = createDirectSshHostHydration({
store,
listRepos: vi.fn(async () =>
hostSnapshot(authority('target-b', 'epoch-b'), [repo('wrong', 'target-b')])
),
listLineage: vi.fn(),
isCurrentAuthority: () => true
})
await expect(hydration.capturePreparationInput(owner, 'wake-refresh')).resolves.toBeNull()
expect(store.getState().repos).toBe(before)
expect(publications).toBe(0)
})
it('settles a non-cooperative catalog read at five seconds and clears its timer', async () => {
vi.useFakeTimers()
const owner = authority()
const store = createStore<AppState>(() => state({ repos: [repo('cached', 'target-a')] }))
let resolveLate!: (value: HostRepoCatalogSnapshot) => void
const lateSnapshot = new Promise<HostRepoCatalogSnapshot>((resolve) => {
resolveLate = resolve
})
let publications = 0
store.subscribe(() => {
publications += 1
})
const hydration = createDirectSshHostHydration({
store,
listRepos: vi.fn(() => lateSnapshot),
listLineage: vi.fn(),
isCurrentAuthority: () => true
})
const pending = hydration.capturePreparationInput(owner, 'wake-refresh')
let settled = false
void pending.then(() => {
settled = true
})
await vi.advanceTimersByTimeAsync(4_999)
expect(settled).toBe(false)
await vi.advanceTimersByTimeAsync(1)
await expect(pending).resolves.toMatchObject({
catalogRevision: 0,
repoRefs: [{ repoId: 'cached', executionHostId: 'ssh:target-a' }]
})
expect(vi.getTimerCount()).toBe(0)
resolveLate(hostSnapshot(owner, [repo('late', 'target-a')]))
await Promise.resolve()
await Promise.resolve()
expect(store.getState().repos).toEqual([repo('cached', 'target-a')])
expect(publications).toBe(0)
})
it('discards a catalog reply after authority changes with zero mutation', async () => {
const owner = authority()
const store = createStore<AppState>(() => state({ repos: [repo('cached', 'target-a')] }))
let current = true
let resolve!: (value: ReturnType<typeof hostSnapshot>) => void
const pendingSnapshot = new Promise<HostRepoCatalogSnapshot>((settle) => {
resolve = settle
})
let publications = 0
store.subscribe(() => {
publications += 1
})
const hydration = createDirectSshHostHydration({
store,
listRepos: () => pendingSnapshot,
listLineage: vi.fn(),
isCurrentAuthority: () => current
})
const pending = hydration.capturePreparationInput(owner, 'reconnect')
current = false
resolve(hostSnapshot(owner, [repo('new', 'target-a')]))
await expect(pending).resolves.toBeNull()
expect(store.getState().repos).toEqual([repo('cached', 'target-a')])
expect(publications).toBe(0)
})
it('settles pending host reads and clears their timers on stop', async () => {
vi.useFakeTimers()
const owner = authority()
const store = createStore<AppState>(() => state())
const hydration = createDirectSshHostHydration({
store,
listRepos: () => new Promise<HostRepoCatalogSnapshot>(() => {}),
listLineage: vi.fn(),
isCurrentAuthority: () => true
})
const pending = hydration.capturePreparationInput(owner, 'wake-refresh')
await Promise.resolve()
expect(vi.getTimerCount()).toBe(1)
hydration.stop()
await expect(pending).resolves.toBeNull()
expect(vi.getTimerCount()).toBe(0)
})
it('replaces only exact-host Git and folder lineage across both key namespaces', async () => {
const owner = authority()
const store = createStore<AppState>(() =>
state({
repos: [
repo('a', 'target-a'),
repo('b', 'target-b'),
repo('ambiguous', 'target-a'),
repo('ambiguous', 'target-a'),
repo('contradictory', 'target-a')
],
worktreesByRepo: {
a: [{ id: 'a::/work', repoId: 'a', hostId: 'ssh:target-a' }],
b: [{ id: 'b::/work', repoId: 'b', hostId: 'ssh:target-b' }],
ambiguous: [
{
id: 'ambiguous::/work',
repoId: 'ambiguous',
hostId: 'ssh:target-a'
}
],
contradictory: [
{
id: 'contradictory::/work',
repoId: 'contradictory',
hostId: 'ssh:target-b'
}
]
},
projectGroups: [
{
id: 'group-a',
parentGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
},
{
id: 'group-b',
parentGroupId: null,
connectionId: 'target-b',
executionHostId: 'ssh:target-b'
}
],
folderWorkspaces: [
{
id: 'folder-a',
projectGroupId: 'group-a',
folderPath: '/target-a/folder',
connectionId: 'target-a'
},
{
id: 'folder-b',
projectGroupId: 'group-b',
folderPath: '/target-b/folder',
connectionId: 'target-b'
}
],
worktreeLineageById: {
'a::/work': { parentWorktreeId: 'stale-a' },
'b::/work': { parentWorktreeId: 'keep-b' },
'ambiguous::/work': { parentWorktreeId: 'keep-ambiguous' },
'contradictory::/work': { parentWorktreeId: 'keep-contradictory' }
},
workspaceLineageByChildKey: {
[worktreeWorkspaceKey('a::/work')]: { parentWorkspaceKey: 'stale-a-workspace' },
[worktreeWorkspaceKey('b::/work')]: { parentWorkspaceKey: 'keep-b-workspace' },
[worktreeWorkspaceKey('ambiguous::/work')]: {
parentWorkspaceKey: 'keep-ambiguous-workspace'
},
[worktreeWorkspaceKey('contradictory::/work')]: {
parentWorkspaceKey: 'keep-contradictory-workspace'
},
[folderWorkspaceKey('folder-a')]: { parentWorkspaceKey: 'stale-folder' },
[folderWorkspaceKey('folder-b')]: { parentWorkspaceKey: 'keep-folder-b' }
}
})
)
const hydration = createDirectSshHostHydration({
store,
listRepos: vi.fn(),
listLineage: vi.fn(
async () =>
({
authoritative: true,
authority: {
kind: 'direct-ssh',
executionHostId: 'ssh:target-a',
...owner
},
worktreeLineageById: {
'a::/work': { parentWorktreeId: 'fresh-a' },
'b::/work': { parentWorktreeId: 'foreign-overwrite' },
'ambiguous::/work': { parentWorktreeId: 'ambiguous-overwrite' },
'contradictory::/work': { parentWorktreeId: 'contradictory-overwrite' }
},
workspaceLineageByChildKey: {
'a::/work': { parentWorkspaceKey: 'raw-terminal-key-must-not-enter-lineage' },
[worktreeWorkspaceKey('a::/work')]: {
parentWorkspaceKey: 'fresh-a-workspace'
},
[worktreeWorkspaceKey('b::/work')]: {
parentWorkspaceKey: 'foreign-workspace-overwrite'
},
[worktreeWorkspaceKey('ambiguous::/work')]: {
parentWorkspaceKey: 'ambiguous-workspace-overwrite'
},
[worktreeWorkspaceKey('contradictory::/work')]: {
parentWorkspaceKey: 'contradictory-workspace-overwrite'
},
[folderWorkspaceKey('folder-a')]: { parentWorkspaceKey: 'fresh-folder' },
[folderWorkspaceKey('folder-b')]: {
parentWorkspaceKey: 'foreign-folder-overwrite'
}
}
}) as unknown as HostLineageSnapshot
),
isCurrentAuthority: () => true
})
await expect(
hydration.readHostScopedLineage({
...owner,
catalogRevision: 0,
repoRefs: [{ repoId: 'a', executionHostId: 'ssh:target-a' }],
authorityRequirement: 'required',
reason: 'reconnect'
})
).resolves.toBe('complete')
expect(store.getState().worktreeLineageById).toEqual({
'a::/work': { parentWorktreeId: 'fresh-a' },
'b::/work': { parentWorktreeId: 'keep-b' },
'ambiguous::/work': { parentWorktreeId: 'keep-ambiguous' },
'contradictory::/work': { parentWorktreeId: 'keep-contradictory' }
})
expect(store.getState().workspaceLineageByChildKey).toEqual({
[worktreeWorkspaceKey('b::/work')]: { parentWorkspaceKey: 'keep-b-workspace' },
[worktreeWorkspaceKey('ambiguous::/work')]: {
parentWorkspaceKey: 'keep-ambiguous-workspace'
},
[worktreeWorkspaceKey('contradictory::/work')]: {
parentWorkspaceKey: 'keep-contradictory-workspace'
},
[folderWorkspaceKey('folder-b')]: { parentWorkspaceKey: 'keep-folder-b' },
[worktreeWorkspaceKey('a::/work')]: { parentWorkspaceKey: 'fresh-a-workspace' },
[folderWorkspaceKey('folder-a')]: { parentWorkspaceKey: 'fresh-folder' }
})
})
it('rejects lineage captured before a newer same-authority catalog revision', async () => {
const owner = authority()
const store = createStore<AppState>(() =>
state({
repos: [repo('a', 'target-a')],
worktreesByRepo: {
a: [{ id: 'a::/work', repoId: 'a', hostId: 'ssh:target-a' }]
},
worktreeLineageById: {
'a::/work': { parentWorktreeId: 'keep-current' }
}
})
)
let resolveLineage!: (value: HostLineageSnapshot) => void
const pendingLineage = new Promise<HostLineageSnapshot>((resolve) => {
resolveLineage = resolve
})
const hydration = createDirectSshHostHydration({
store,
listRepos: vi.fn(async () => hostSnapshot(owner, [repo('a', 'target-a')])),
listLineage: () => pendingLineage,
isCurrentAuthority: () => true
})
const firstInput = await hydration.capturePreparationInput(owner, 'reconnect')
if (!firstInput) {
throw new Error('Expected first preparation input')
}
const pending = hydration.readHostScopedLineage(firstInput)
await hydration.capturePreparationInput(owner, 'wake-refresh')
const beforeLateLineage = store.getState().worktreeLineageById
resolveLineage({
authoritative: true,
authority: {
kind: 'direct-ssh',
executionHostId: 'ssh:target-a',
...owner
},
worktreeLineageById: {
'a::/work': { parentWorktreeId: 'stale-overwrite' }
},
workspaceLineageByChildKey: {}
} as unknown as HostLineageSnapshot)
await expect(pending).resolves.toBe('stale')
expect(store.getState().worktreeLineageById).toBe(beforeLateLineage)
})
})
src/renderer/src/hooks/direct-ssh-host-hydration.ts
+311
View File
@@ -0,0 +1,311 @@
import type { StoreApi } from 'zustand'
import { getRepoExecutionHostId, toSshExecutionHostId } from '../../../shared/execution-host'
import type { HostLineageSnapshot } from '../../../shared/host-lineage-contract'
import type { HostRepoCatalogSnapshot } from '../../../shared/host-repo-catalog-contract'
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import { isWorkspaceKey } from '../../../shared/workspace-scope'
import type { AppState } from '../store/types'
import type {
DirectSshLineageOutcome,
DirectSshPreparationInput,
DirectSshPreparationReason
} from './direct-ssh-reconnect-coordinator'
import { directSshHostHydrationTelemetry } from './direct-ssh-host-hydration-telemetry'
import { directSshHostHydrationScope } from './direct-ssh-host-hydration-scope'
import { directSshAuthoritiesEqual } from './direct-ssh-reconnect-tokens'
export const DIRECT_SSH_HOST_READ_TIMEOUT_MS = 5_000
type HostReadTimer = unknown
export type DirectSshHostHydrationDeps = {
store: Pick<StoreApi<AppState>, 'getState' | 'setState'>
listRepos: (authority: DirectSshAuthority) => Promise<HostRepoCatalogSnapshot>
listLineage: (authority: DirectSshAuthority) => Promise<HostLineageSnapshot>
isCurrentAuthority: (authority: DirectSshAuthority) => boolean
setTimer?: (callback: () => void, delayMs: number) => HostReadTimer
clearTimer?: (timer: HostReadTimer) => void
}
export type DirectSshHostHydration = {
capturePreparationInput: (
authority: DirectSshAuthority,
reason: DirectSshPreparationReason,
snapshotRevision?: number
) => Promise<DirectSshPreparationInput | null>
readHostScopedLineage: (input: DirectSshPreparationInput) => Promise<DirectSshLineageOutcome>
isPreparationTokenCurrent: (token: {
authority: DirectSshAuthority
catalogRevision: number
repoFingerprint: string
}) => boolean
stop: () => void
}
type BoundedResult<T> = { status: 'complete'; value: T } | { status: 'timed-out' | 'unavailable' }
function isAuthoritativeHost(
authority: HostRepoCatalogSnapshot | HostLineageSnapshot,
expected: DirectSshAuthority
): authority is Extract<typeof authority, { authoritative: true }> {
if (!authority.authoritative || authority.authority.kind !== 'direct-ssh') {
return false
}
return (
authority.authority.executionHostId === toSshExecutionHostId(expected.targetId) &&
directSshAuthoritiesEqual(authority.authority, expected)
)
}
function mergeExactHostCatalog(state: AppState, snapshot: HostRepoCatalogSnapshot): AppState {
if (!snapshot.authoritative || snapshot.authority.kind !== 'direct-ssh') {
return state
}
const hostId = snapshot.authority.executionHostId
return {
...state,
repos: [
...state.repos.filter((repo) => getRepoExecutionHostId(repo) !== hostId),
...snapshot.repos
]
}
}
function mergeExactHostLineage(
state: AppState,
snapshot: Extract<HostLineageSnapshot, { authoritative: true }>,
authority: DirectSshAuthority,
catalogRevision: number
): AppState {
const scope = directSshHostHydrationScope(state, authority, catalogRevision)
const worktreeLineageById = Object.fromEntries(
Object.entries(state.worktreeLineageById).filter(
([worktreeId]) => !scope.gitWorktreeIds.has(worktreeId)
)
)
const workspaceLineageByChildKey = Object.fromEntries(
Object.entries(state.workspaceLineageByChildKey).filter(
([childKey]) => !isWorkspaceKey(childKey) || !scope.lineageWorkspaceKeys.has(childKey)
)
)
const incomingWorktreeLineage = Object.fromEntries(
Object.entries(snapshot.worktreeLineageById).filter(([worktreeId]) =>
scope.gitWorktreeIds.has(worktreeId)
)
)
const incomingWorkspaceLineage = Object.fromEntries(
Object.entries(snapshot.workspaceLineageByChildKey).filter(
([workspaceKey]) =>
isWorkspaceKey(workspaceKey) && scope.lineageWorkspaceKeys.has(workspaceKey)
)
)
return {
...state,
worktreeLineageById: {
...worktreeLineageById,
...incomingWorktreeLineage
},
workspaceLineageByChildKey: {
...workspaceLineageByChildKey,
...incomingWorkspaceLineage
}
}
}
export function createDirectSshHostHydration(
deps: DirectSshHostHydrationDeps
): DirectSshHostHydration {
const setTimer: NonNullable<DirectSshHostHydrationDeps['setTimer']> =
deps.setTimer ?? ((callback, delayMs) => setTimeout(callback, delayMs))
const clearTimer: NonNullable<DirectSshHostHydrationDeps['clearTimer']> =
deps.clearTimer ?? ((timer) => clearTimeout(timer as ReturnType<typeof setTimeout>))
const catalogRevisionByTarget = new Map<string, number>()
const catalogInFlight = new Map<string, Promise<'complete' | 'degraded' | 'stale'>>()
const pendingDeadlines = new Set<{ timer: HostReadTimer; settle: () => void }>()
let stopped = false
const bounded = async <T>(operation: Promise<T>): Promise<BoundedResult<T>> => {
let pendingDeadline: { timer: HostReadTimer; settle: () => void } | undefined
const deadline = new Promise<BoundedResult<T>>((resolve) => {
const settle = (): void => resolve({ status: 'timed-out' })
const timer = setTimer(settle, DIRECT_SSH_HOST_READ_TIMEOUT_MS)
pendingDeadline = { timer, settle }
pendingDeadlines.add(pendingDeadline)
})
try {
return await Promise.race([
operation.then<BoundedResult<T>, BoundedResult<T>>(
(value) => ({ status: 'complete', value }),
() => ({ status: 'unavailable' })
),
deadline
])
} finally {
if (pendingDeadline) {
clearTimer(pendingDeadline.timer)
pendingDeadlines.delete(pendingDeadline)
}
}
}
const refreshCatalog = (
authority: DirectSshAuthority
): Promise<'complete' | 'degraded' | 'stale'> => {
const key = JSON.stringify([
authority.targetId,
authority.providerEpoch,
authority.connectionGeneration
])
const existing = catalogInFlight.get(key)
if (existing) {
return existing
}
const operation = (async (): Promise<'complete' | 'degraded' | 'stale'> => {
const boundedResult = await bounded(Promise.resolve().then(() => deps.listRepos(authority)))
if (stopped || !deps.isCurrentAuthority(authority)) {
return 'stale'
}
if (boundedResult.status !== 'complete') {
return 'degraded'
}
const snapshot = boundedResult.value
if (!snapshot.authoritative) {
return snapshot.reason === 'stale' || snapshot.reason === 'rejected' ? 'stale' : 'degraded'
}
if (
!isAuthoritativeHost(snapshot, authority) ||
snapshot.repos.some(
(repo) =>
getRepoExecutionHostId(repo) !== toSshExecutionHostId(authority.targetId) ||
repo.connectionId !== authority.targetId
)
) {
return 'stale'
}
let admitted = false
deps.store.setState((state) => {
if (stopped || !deps.isCurrentAuthority(authority)) {
return state
}
admitted = true
return mergeExactHostCatalog(state, snapshot)
})
if (!admitted) {
return 'stale'
}
catalogRevisionByTarget.set(
authority.targetId,
(catalogRevisionByTarget.get(authority.targetId) ?? 0) + 1
)
return 'complete'
})().finally(() => {
if (catalogInFlight.get(key) === operation) {
catalogInFlight.delete(key)
}
})
catalogInFlight.set(key, operation)
return operation
}
const capturePreparationInput = async (
authority: DirectSshAuthority,
reason: DirectSshPreparationReason,
snapshotRevision?: number
): Promise<DirectSshPreparationInput | null> => {
const catalogStartedAt = Date.now()
const catalogOutcome = await refreshCatalog(authority)
const catalogDurationMs = Math.max(0, Date.now() - catalogStartedAt)
if (catalogOutcome === 'stale' || stopped || !deps.isCurrentAuthority(authority)) {
return null
}
const catalogRevision = catalogRevisionByTarget.get(authority.targetId) ?? 0
const scope = directSshHostHydrationScope(deps.store.getState(), authority, catalogRevision)
return {
...authority,
catalogRevision,
repoRefs: scope.gitRepos,
authorityRequirement: 'required',
...(snapshotRevision === undefined ? {} : { snapshotRevision }),
reason,
telemetry: directSshHostHydrationTelemetry(scope, catalogOutcome, catalogDurationMs)
}
}
const readHostScopedLineage = async (
input: DirectSshPreparationInput
): Promise<DirectSshLineageOutcome> => {
const authority = {
targetId: input.targetId,
providerEpoch: input.providerEpoch,
connectionGeneration: input.connectionGeneration
}
const boundedResult = await bounded(Promise.resolve().then(() => deps.listLineage(authority)))
if (
stopped ||
!deps.isCurrentAuthority(authority) ||
(catalogRevisionByTarget.get(authority.targetId) ?? 0) !== input.catalogRevision
) {
return 'stale'
}
if (boundedResult.status !== 'complete') {
return 'degraded'
}
const snapshot = boundedResult.value
if (!snapshot.authoritative) {
return snapshot.reason === 'stale' || snapshot.reason === 'rejected' ? 'stale' : 'degraded'
}
if (!isAuthoritativeHost(snapshot, authority)) {
return 'stale'
}
let admitted = false
deps.store.setState((state) => {
if (
stopped ||
!deps.isCurrentAuthority(authority) ||
(catalogRevisionByTarget.get(authority.targetId) ?? 0) !== input.catalogRevision
) {
return state
}
admitted = true
return mergeExactHostLineage(state, snapshot, authority, input.catalogRevision)
})
return admitted ? 'complete' : 'stale'
}
return {
capturePreparationInput,
readHostScopedLineage,
isPreparationTokenCurrent: (token) => {
if (
stopped ||
!deps.isCurrentAuthority(token.authority) ||
(catalogRevisionByTarget.get(token.authority.targetId) ?? 0) !== token.catalogRevision
) {
return false
}
const scope = directSshHostHydrationScope(
deps.store.getState(),
token.authority,
token.catalogRevision
)
const sortedRepos = [...scope.gitRepos].sort((left, right) => {
const leftKey = `${left.executionHostId}\0${left.repoId}`
const rightKey = `${right.executionHostId}\0${right.repoId}`
return leftKey < rightKey ? -1 : leftKey > rightKey ? 1 : 0
})
return (
JSON.stringify(sortedRepos.map((repo) => [repo.executionHostId, repo.repoId])) ===
token.repoFingerprint
)
},
stop: () => {
stopped = true
for (const deadline of pendingDeadlines) {
clearTimer(deadline.timer)
deadline.settle()
}
pendingDeadlines.clear()
catalogInFlight.clear()
}
}
}
src/renderer/src/hooks/direct-ssh-reconnect-coordinator-outcomes.ts
+66
View File
@@ -0,0 +1,66 @@
import type {
DirectSshPreparationMetrics,
DirectSshPreparationOutcome,
DirectSshReconnectOutcome,
DirectSshRepoOutcomeCounts
} from './direct-ssh-reconnect-coordinator-types'
export function createEmptyDirectSshRepoOutcomeCounts(): DirectSshRepoOutcomeCounts {
return {
complete: 0,
'non-authoritative': 0,
'timed-out': 0,
'cancel-budget-exhausted': 0,
canceled: 0,
stale: 0,
rejected: 0
}
}
export function createEmptyDirectSshPreparationMetrics(): DirectSshPreparationMetrics {
return {
queueWaitDurationsMs: [],
providerExecutionDurationsMs: [],
timeoutRetryCount: 0,
locallySettledWaiterCount: 0,
cancelDebtCount: 0,
replacementAdmissionDelayedCount: 0,
schedulerOverlappingJoinCount: 0,
peakLocallyUnsettled: 0,
estimatedLateWorkAllowanceCount: 0,
lineageDurationMs: 0
}
}
export function combineDirectSshReconnectOutcome(
prepared: DirectSshPreparationOutcome,
staleBindingsCleared: number,
retriedTerminals: number,
correctedTerminals: number
): DirectSshReconnectOutcome {
return {
...prepared,
staleBindingsCleared,
retriedTerminals,
correctedTerminals,
stabilizing: false
}
}
export function createTerminalOnlyDirectSshReconnectOutcome(
status: DirectSshReconnectOutcome['status'],
staleBindingsCleared = 0,
retriedTerminals = 0
): DirectSshReconnectOutcome {
return {
status,
token: null,
repoOutcomes: createEmptyDirectSshRepoOutcomeCounts(),
lineageOutcome: 'not-started',
metrics: createEmptyDirectSshPreparationMetrics(),
staleBindingsCleared,
retriedTerminals,
correctedTerminals: 0,
stabilizing: status === 'stabilizing'
}
}
src/renderer/src/hooks/direct-ssh-reconnect-coordinator-stabilization.ts
+25
View File
@@ -0,0 +1,25 @@
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import type { DirectSshReconnectTimer } from './direct-ssh-reconnect-coordinator-types'
export type DirectSshReconnectTargetState = {
authority: DirectSshAuthority
installedAt: number
dampUntil: number | null
timer: DirectSshReconnectTimer | null
}
export function createDirectSshReconnectTargetState(
authority: DirectSshAuthority,
previous: DirectSshReconnectTargetState | undefined,
installedAt: number,
stabilizationMs: number
): DirectSshReconnectTargetState {
const rotatedRapidly =
previous !== undefined && installedAt - previous.installedAt < stabilizationMs
return {
authority,
installedAt,
dampUntil: rotatedRapidly ? installedAt + stabilizationMs : null,
timer: null
}
}
src/renderer/src/hooks/direct-ssh-reconnect-coordinator-telemetry.ts
+118
View File
@@ -0,0 +1,118 @@
import type {
DirectSshCoordinatorTelemetry,
DirectSshPreparationInput,
DirectSshPreparationOutcome,
DirectSshPreparationReason,
DirectSshReconnectOutcome
} from './direct-ssh-reconnect-coordinator-types'
import { createTerminalOnlyDirectSshReconnectOutcome } from './direct-ssh-reconnect-coordinator-outcomes'
type TelemetryOutcome = DirectSshPreparationOutcome | DirectSshReconnectOutcome
type TelemetryOptions = {
terminalFinalizationDurationMs?: number
staleBindingsCleared?: number
retriedTerminals?: number
correctedTerminals?: number
damped?: boolean
authorityRotationCount?: number
catalogOutcome?: DirectSshCoordinatorTelemetry['catalogOutcome']
catalogDurationMs?: number
}
type DirectSshCoordinatorTelemetryReporter = {
report: (
mode: DirectSshCoordinatorTelemetry['mode'],
input: DirectSshPreparationInput,
outcome: TelemetryOutcome,
startedAt: number,
options?: TelemetryOptions
) => void
reportWithoutInput: (
mode: DirectSshCoordinatorTelemetry['mode'],
reason: DirectSshPreparationReason,
outcome: TelemetryOutcome,
startedAt: number,
options?: TelemetryOptions
) => void
}
export function createDirectSshCoordinatorTelemetryReporter(args: {
onTelemetry?: (event: DirectSshCoordinatorTelemetry) => void
now: () => number
}): DirectSshCoordinatorTelemetryReporter {
const emit = (event: DirectSshCoordinatorTelemetry): void => {
try {
args.onTelemetry?.(event)
} catch {
// Recovery cannot depend on diagnostics.
}
}
const report = (
mode: DirectSshCoordinatorTelemetry['mode'],
input: DirectSshPreparationInput,
outcome: TelemetryOutcome,
startedAt: number,
options: TelemetryOptions = {}
): void => {
const inputMetrics = input.telemetry
const outcomeMetrics =
outcome.metrics ?? createTerminalOnlyDirectSshReconnectOutcome('stale').metrics!
emit({
mode,
reason: input.reason,
outcome: outcome.status,
durationMs: Math.max(0, args.now() - startedAt),
staleBindingsCleared: options.staleBindingsCleared ?? 0,
retriedTerminals: options.retriedTerminals ?? 0,
correctedTerminals: options.correctedTerminals ?? 0,
terminalFinalizationDurationMs: options.terminalFinalizationDurationMs ?? 0,
catalogOutcome: options.catalogOutcome ?? inputMetrics?.catalogOutcome ?? 'complete',
catalogDurationMs: options.catalogDurationMs ?? inputMetrics?.catalogDurationMs ?? 0,
gitWorktreeCount: inputMetrics?.gitWorktreeCount ?? 0,
folderWorkspaceCount: inputMetrics?.folderWorkspaceCount ?? 0,
ambiguousOwnerCount: inputMetrics?.ambiguousOwnerCount ?? 0,
contradictoryOwnerCount: inputMetrics?.contradictoryOwnerCount ?? 0,
repoOutcomes: { ...outcome.repoOutcomes },
lineageOutcome: outcome.lineageOutcome,
queueWaitDurationsMs: [...outcomeMetrics.queueWaitDurationsMs],
providerExecutionDurationsMs: [...outcomeMetrics.providerExecutionDurationsMs],
timeoutRetryCount: outcomeMetrics.timeoutRetryCount,
locallySettledWaiterCount: outcomeMetrics.locallySettledWaiterCount,
cancelDebtCount: outcomeMetrics.cancelDebtCount,
replacementAdmissionDelayedCount: outcomeMetrics.replacementAdmissionDelayedCount,
overlappingJoinCount: outcomeMetrics.schedulerOverlappingJoinCount,
peakLocallyUnsettled: outcomeMetrics.peakLocallyUnsettled,
estimatedLateWorkAllowanceCount: outcomeMetrics.estimatedLateWorkAllowanceCount,
authorityRotationCount: options.authorityRotationCount ?? 0,
damped: options.damped ?? false
})
}
const reportWithoutInput: DirectSshCoordinatorTelemetryReporter['reportWithoutInput'] = (
mode,
reason,
outcome,
startedAt,
options = {}
) => {
report(
mode,
{
targetId: '',
providerEpoch: '' as DirectSshPreparationInput['providerEpoch'],
connectionGeneration: 0,
catalogRevision: 0,
repoRefs: [],
authorityRequirement: 'required',
reason
},
outcome,
startedAt,
options
)
}
return { report, reportWithoutInput }
}
src/renderer/src/hooks/direct-ssh-reconnect-coordinator-types.ts
+152
View File
@@ -0,0 +1,152 @@
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import type { DirectSshGitRepoRef } from '../lib/direct-ssh-target-scope-types'
import type {
DirectSshWorktreeRefreshScheduler,
DirectSshWorktreeRefreshTerminalStatus
} from './direct-ssh-worktree-refresh-scheduler'
export type DirectSshPreparationReason =
| 'reconnect'
| 'initial-hydration'
| 'workspace-snapshot'
| 'wake-refresh'
export type DirectSshAuthorityRequirement = 'required' | 'allow-metadata-fallback'
export type DirectSshPreparationInput = DirectSshAuthority & {
catalogRevision: number
repoRefs: readonly DirectSshGitRepoRef[]
authorityRequirement: DirectSshAuthorityRequirement
snapshotRevision?: number
reason: DirectSshPreparationReason
telemetry?: DirectSshPreparationInputTelemetry
}
export type DirectSshPreparationInputTelemetry = {
catalogOutcome: 'complete' | 'degraded'
catalogDurationMs: number
gitWorktreeCount: number
folderWorkspaceCount: number
ambiguousOwnerCount: number
contradictoryOwnerCount: number
}
export type DirectSshPreparationToken = {
authority: DirectSshAuthority
catalogRevision: number
repoFingerprint: string
authorityRequirement: DirectSshAuthorityRequirement
snapshotRevision: number | null
outcome: 'complete' | 'degraded'
}
export type DirectSshSnapshotApplyToken = DirectSshPreparationToken & {
snapshotRevision: number
}
export type DirectSshLineageOutcome = 'complete' | 'degraded' | 'canceled' | 'stale'
export type DirectSshRepoOutcomeCounts = Record<DirectSshWorktreeRefreshTerminalStatus, number>
export type DirectSshPreparationOutcome = {
status: 'complete' | 'degraded' | 'canceled' | 'stale' | 'stopped'
token: DirectSshPreparationToken | null
repoOutcomes: DirectSshRepoOutcomeCounts
lineageOutcome: DirectSshLineageOutcome | 'not-started'
metrics?: DirectSshPreparationMetrics
}
export type DirectSshPreparationMetrics = {
queueWaitDurationsMs: readonly number[]
providerExecutionDurationsMs: readonly number[]
timeoutRetryCount: number
locallySettledWaiterCount: number
cancelDebtCount: number
replacementAdmissionDelayedCount: number
schedulerOverlappingJoinCount: number
peakLocallyUnsettled: number
estimatedLateWorkAllowanceCount: number
lineageDurationMs: number
}
export type DirectSshReconnectOutcome = Omit<DirectSshPreparationOutcome, 'status'> & {
status: DirectSshPreparationOutcome['status'] | 'stabilizing'
staleBindingsCleared: number
retriedTerminals: number
correctedTerminals: number
stabilizing: boolean
}
export type DirectSshCorrectionReason =
| 'preparation-complete'
| 'wake-refresh'
| 'workspace-hydrated'
| 'snapshot-applied'
export type DirectSshCoordinatorTelemetry = {
mode: 'reconnect' | 'prepare-only'
reason: DirectSshPreparationReason
outcome: DirectSshReconnectOutcome['status']
durationMs: number
staleBindingsCleared: number
retriedTerminals: number
correctedTerminals: number
terminalFinalizationDurationMs: number
catalogOutcome: 'complete' | 'degraded' | 'stale'
catalogDurationMs: number
gitWorktreeCount: number
folderWorkspaceCount: number
ambiguousOwnerCount: number
contradictoryOwnerCount: number
repoOutcomes: DirectSshRepoOutcomeCounts
lineageOutcome: DirectSshPreparationOutcome['lineageOutcome']
queueWaitDurationsMs: readonly number[]
providerExecutionDurationsMs: readonly number[]
timeoutRetryCount: number
locallySettledWaiterCount: number
cancelDebtCount: number
replacementAdmissionDelayedCount: number
overlappingJoinCount: number
peakLocallyUnsettled: number
estimatedLateWorkAllowanceCount: number
authorityRotationCount: number
damped: boolean
}
export type DirectSshReconnectTimer = unknown
export type DirectSshReconnectCoordinatorDeps = {
scheduler: DirectSshWorktreeRefreshScheduler
isCurrentConnectedAuthority: (authority: DirectSshAuthority) => boolean
capturePreparationInput: (
authority: DirectSshAuthority,
reason: DirectSshPreparationReason
) => Promise<DirectSshPreparationInput | null>
readHostScopedLineage: (input: DirectSshPreparationInput) => Promise<DirectSshLineageOutcome>
invalidateStaleTerminalBindings: (authority: DirectSshAuthority) => number
retryTargetPanes: (authority: DirectSshAuthority) => number
finalizeHydratedTerminalPanes: (authority: DirectSshAuthority) => number
correctUnboundTerminalPanes: (
authority: DirectSshAuthority,
reason: DirectSshCorrectionReason
) => number
syncRemoteWorkspaceAfterConnect: (token: DirectSshPreparationToken) => void | Promise<void>
onTelemetry?: (event: DirectSshCoordinatorTelemetry) => void
now?: () => number
setTimer?: (callback: () => void, delayMs: number) => DirectSshReconnectTimer
clearTimer?: (timer: DirectSshReconnectTimer) => void
stabilizationMs?: number
}
export type DirectSshReconnectCoordinator = {
requestReconnect: (authority: DirectSshAuthority) => Promise<DirectSshReconnectOutcome>
prepareOnly: (input: DirectSshPreparationInput) => Promise<DirectSshPreparationOutcome>
finalizeHydratedTerminals: (authority: DirectSshAuthority) => number
correctUnboundTerminals: (
authority: DirectSshAuthority,
reason: DirectSshCorrectionReason
) => number
replaceAuthority: (authority: DirectSshAuthority) => void
invalidate: (targetId: string) => void
stop: () => void
}
src/renderer/src/hooks/direct-ssh-reconnect-coordinator.test.ts
+566
View File
@@ -0,0 +1,566 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import type {
HostQualifiedDetectedWorktreeResult,
ProviderRequestId
} from '../../../shared/detected-worktree-provider-contract'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../shared/ssh-types'
import {
createDirectSshWorktreeRefreshScheduler,
type DirectSshWorktreeRefreshKey,
type DirectSshWorktreeRefreshLease,
type DirectSshWorktreeRefreshOutcome,
type DirectSshWorktreeRefreshScheduler,
type WaiterLeaseId
} from './direct-ssh-worktree-refresh-scheduler'
import {
admitDirectSshSnapshotApplyToken,
buildDirectSshSnapshotApplyToken,
createDirectSshReconnectCoordinator,
type DirectSshLineageOutcome,
type DirectSshPreparationInput,
type DirectSshReconnectCoordinatorDeps
} from './direct-ssh-reconnect-coordinator'
type Deferred<T> = {
promise: Promise<T>
resolve: (value: T) => void
reject: (error: unknown) => void
}
function deferred<T>(): Deferred<T> {
let resolve!: (value: T) => void
let reject!: (error: unknown) => void
const promise = new Promise<T>((resolvePromise, rejectPromise) => {
resolve = resolvePromise
reject = rejectPromise
})
return { promise, resolve, reject }
}
function epoch(value: string): SshProviderEpoch {
return value as SshProviderEpoch
}
function authority(targetId: string, epochId = `epoch-${targetId}`): DirectSshAuthority {
return {
targetId,
providerEpoch: epoch(epochId),
connectionGeneration: 1
}
}
function preparationInput(
owner: DirectSshAuthority,
repoIds: readonly string[] = ['repo'],
overrides: Partial<DirectSshPreparationInput> = {}
): DirectSshPreparationInput {
return {
...owner,
catalogRevision: 1,
repoRefs: repoIds.map((repoId) => ({
repoId,
executionHostId: `ssh:${owner.targetId}`
})),
authorityRequirement: 'required',
reason: 'workspace-snapshot',
...overrides
}
}
type FakeLease = {
key: DirectSshWorktreeRefreshKey
deferred: Deferred<DirectSshWorktreeRefreshOutcome>
lease: DirectSshWorktreeRefreshLease
}
function createFakeScheduler(): {
scheduler: DirectSshWorktreeRefreshScheduler
leases: FakeLease[]
complete: (index: number, status?: DirectSshWorktreeRefreshOutcome['status']) => void
} {
const leases: FakeLease[] = []
const request = vi.fn((key: DirectSshWorktreeRefreshKey) => {
const leaseDeferred = deferred<DirectSshWorktreeRefreshOutcome>()
let released = false
const lease: DirectSshWorktreeRefreshLease = {
waiterLeaseId: `lease-${leases.length + 1}` as WaiterLeaseId,
result: leaseDeferred.promise,
release: vi.fn(() => {
if (!released) {
released = true
leaseDeferred.resolve({ status: 'canceled' })
}
})
}
leases.push({ key, deferred: leaseDeferred, lease })
return lease
})
const scheduler: DirectSshWorktreeRefreshScheduler = {
request,
invalidateAuthority: vi.fn(),
invalidateTarget: vi.fn(),
disposeProvider: vi.fn(),
getSnapshot: vi.fn(() => ({
locallyUnsettled: 0,
queued: 0,
retrying: 0,
logicalTasks: leases.length,
waiters: leases.length,
cancelDebtByAuthority: new Map()
})),
stop: vi.fn()
}
return {
scheduler,
leases,
complete: (index, status = 'complete') => leases[index].deferred.resolve({ status })
}
}
function createCoordinatorHarness(fakeScheduler = createFakeScheduler()) {
const current = new Map<string, DirectSshAuthority>()
const events: string[] = []
const capturePreparationInput = vi.fn(
async (owner: DirectSshAuthority, reason: DirectSshPreparationInput['reason']) => {
events.push(`capture:${owner.targetId}`)
return preparationInput(owner, ['repo'], { reason })
}
)
const deps: DirectSshReconnectCoordinatorDeps = {
scheduler: fakeScheduler.scheduler,
isCurrentConnectedAuthority: (owner) => {
const active = current.get(owner.targetId)
return (
active?.providerEpoch === owner.providerEpoch &&
active.connectionGeneration === owner.connectionGeneration
)
},
capturePreparationInput,
readHostScopedLineage: vi.fn(async (): Promise<DirectSshLineageOutcome> => 'complete'),
invalidateStaleTerminalBindings: vi.fn((owner) => {
events.push(`invalidate:${owner.targetId}`)
return 1
}),
retryTargetPanes: vi.fn((owner) => {
events.push(`retry:${owner.targetId}`)
return 2
}),
finalizeHydratedTerminalPanes: vi.fn(() => 1),
correctUnboundTerminalPanes: vi.fn(() => 1),
syncRemoteWorkspaceAfterConnect: vi.fn(),
onTelemetry: vi.fn()
}
const coordinator = createDirectSshReconnectCoordinator(deps)
return { coordinator, deps, current, events, ...fakeScheduler }
}
async function flush(): Promise<void> {
await Promise.resolve()
await Promise.resolve()
await Promise.resolve()
}
afterEach(() => {
vi.useRealTimers()
})
describe('createDirectSshReconnectCoordinator', () => {
it('runs terminal invalidation and retry synchronously before preparation awaits', () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
void harness.coordinator.requestReconnect(owner)
expect(harness.events).toEqual(['invalidate:target-a', 'retry:target-a', 'capture:target-a'])
expect(harness.leases).toHaveLength(0)
})
it('prepares the first observed authority immediately', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
const pending = harness.coordinator.requestReconnect(owner)
await flush()
expect(harness.deps.capturePreparationInput).toHaveBeenCalledWith(owner, 'reconnect')
expect(harness.leases).toHaveLength(1)
harness.complete(0)
await expect(pending).resolves.toMatchObject({ status: 'complete', stabilizing: false })
})
it('keeps target B independent while target A preparation is blocked', async () => {
const harness = createCoordinatorHarness()
const targetA = authority('target-a')
const targetB = authority('target-b')
harness.current.set(targetA.targetId, targetA)
harness.current.set(targetB.targetId, targetB)
const pendingA = harness.coordinator.requestReconnect(targetA)
await flush()
const pendingB = harness.coordinator.requestReconnect(targetB)
await flush()
expect(harness.leases).toHaveLength(2)
harness.complete(1)
await expect(pendingB).resolves.toMatchObject({ status: 'complete' })
expect(harness.deps.syncRemoteWorkspaceAfterConnect).toHaveBeenCalledTimes(1)
let targetASettled = false
void pendingA.then(() => {
targetASettled = true
})
await flush()
expect(targetASettled).toBe(false)
harness.coordinator.stop()
})
it('treats exact-equal authority replacement as a true timing no-op', async () => {
vi.useFakeTimers()
vi.setSystemTime(0)
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.coordinator.replaceAuthority(owner)
vi.setSystemTime(4_000)
harness.coordinator.replaceAuthority({ ...owner })
expect(harness.scheduler.disposeProvider).not.toHaveBeenCalled()
expect(harness.scheduler.invalidateTarget).not.toHaveBeenCalled()
vi.setSystemTime(5_001)
const next = authority('target-a', 'next')
harness.current.set(next.targetId, next)
const pending = harness.coordinator.requestReconnect(next)
await flush()
expect(harness.deps.capturePreparationInput).toHaveBeenCalledWith(next, 'reconnect')
harness.complete(0)
await expect(pending).resolves.toMatchObject({ status: 'complete', stabilizing: false })
})
it('cancels obsolete work on changed authority without ordering epochs', async () => {
const harness = createCoordinatorHarness()
const oldOwner = authority('target-a', 'z-old')
const newOwner = authority('target-a', 'a-new')
harness.current.set(oldOwner.targetId, oldOwner)
harness.coordinator.replaceAuthority(oldOwner)
const pending = harness.coordinator.prepareOnly(preparationInput(oldOwner))
await flush()
harness.current.set(newOwner.targetId, newOwner)
harness.coordinator.replaceAuthority(newOwner)
await expect(pending).resolves.toMatchObject({ status: 'stale', token: null })
expect(harness.leases[0].lease.release).toHaveBeenCalledWith('invalidated')
expect(harness.scheduler.disposeProvider).toHaveBeenCalledWith(oldOwner)
})
it('settles invalidation locally while a late lineage read remains pending', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
const lineage = deferred<'complete'>()
vi.mocked(harness.deps.readHostScopedLineage).mockReturnValue(lineage.promise)
harness.current.set(owner.targetId, owner)
harness.coordinator.replaceAuthority(owner)
const pending = harness.coordinator.prepareOnly(preparationInput(owner))
harness.complete(0)
await flush()
expect(harness.deps.readHostScopedLineage).toHaveBeenCalledOnce()
harness.coordinator.invalidate(owner.targetId)
await expect(pending).resolves.toMatchObject({ status: 'stale', token: null })
lineage.resolve('complete')
await flush()
expect(harness.deps.syncRemoteWorkspaceAfterConnect).not.toHaveBeenCalled()
})
it('joins exact overlapping preparation and reruns after completion', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
harness.coordinator.replaceAuthority(owner)
const input = preparationInput(owner)
const first = harness.coordinator.prepareOnly(input)
const joined = harness.coordinator.prepareOnly({
...input,
repoRefs: input.repoRefs.toReversed()
})
expect(joined).toBe(first)
expect(harness.leases).toHaveLength(1)
harness.complete(0)
await expect(first).resolves.toMatchObject({ status: 'complete' })
const later = harness.coordinator.prepareOnly(input)
expect(later).not.toBe(first)
expect(harness.leases).toHaveLength(2)
harness.coordinator.stop()
})
it('keeps prepare-only isolated from terminal retry and reconnect sync', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
harness.coordinator.replaceAuthority(owner)
const pending = harness.coordinator.prepareOnly(preparationInput(owner))
harness.complete(0, 'non-authoritative')
await expect(pending).resolves.toMatchObject({
status: 'degraded',
token: { outcome: 'degraded' }
})
expect(harness.deps.invalidateStaleTerminalBindings).not.toHaveBeenCalled()
expect(harness.deps.retryTargetPanes).not.toHaveBeenCalled()
expect(harness.deps.correctUnboundTerminalPanes).not.toHaveBeenCalled()
expect(harness.deps.syncRemoteWorkspaceAfterConnect).not.toHaveBeenCalled()
})
it('rejects repo refs for another SSH host before scheduler admission', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
harness.coordinator.replaceAuthority(owner)
await expect(
harness.coordinator.prepareOnly(
preparationInput(owner, ['repo'], {
repoRefs: [{ repoId: 'repo', executionHostId: 'ssh:target-b' }]
})
)
).resolves.toMatchObject({ status: 'stale', token: null })
expect(harness.leases).toHaveLength(0)
expect(harness.deps.readHostScopedLineage).not.toHaveBeenCalled()
})
it('holds lineage and token creation through the scheduler timeout retry barrier', async () => {
const attempts: {
id: ProviderRequestId
deferred: Deferred<HostQualifiedDetectedWorktreeResult>
key: DirectSshWorktreeRefreshKey
}[] = []
const scheduler = createDirectSshWorktreeRefreshScheduler({
startAttempt: (attemptKey) => {
const result = deferred<HostQualifiedDetectedWorktreeResult>()
const id = `provider-${attempts.length + 1}` as ProviderRequestId
attempts.push({ id, deferred: result, key: attemptKey })
return { providerRequestId: id, result: result.promise, cancel: vi.fn() }
}
})
const harness = createCoordinatorHarness({
scheduler,
leases: [],
complete: () => {}
})
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
harness.coordinator.replaceAuthority(owner)
const pending = harness.coordinator.prepareOnly(preparationInput(owner))
attempts[0].deferred.resolve({
status: 'timed-out',
providerRequestId: attempts[0].id,
executionHostId: attempts[0].key.executionHostId
})
await flush()
expect(attempts).toHaveLength(2)
expect(harness.deps.readHostScopedLineage).not.toHaveBeenCalled()
attempts[1].deferred.resolve({
status: 'complete',
providerRequestId: attempts[1].id,
repoId: 'repo',
authority: {
kind: 'direct-ssh',
executionHostId: attempts[1].key.executionHostId,
...owner
},
result: { repoId: 'repo', authoritative: true, source: 'git', worktrees: [] }
})
await expect(pending).resolves.toMatchObject({ status: 'complete', token: {} })
expect(harness.deps.readHostScopedLineage).toHaveBeenCalledOnce()
})
it('starts lineage only after every repo is terminal and returns a fenced token', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
harness.coordinator.replaceAuthority(owner)
const pending = harness.coordinator.prepareOnly(
preparationInput(owner, ['repo-b', 'repo-a'], {
catalogRevision: 7,
snapshotRevision: 11
})
)
expect(harness.leases.map((lease) => lease.key.repoId)).toEqual(['repo-a', 'repo-b'])
harness.complete(0)
await flush()
expect(harness.deps.readHostScopedLineage).not.toHaveBeenCalled()
harness.complete(1)
await expect(pending).resolves.toMatchObject({
status: 'complete',
token: {
authority: owner,
catalogRevision: 7,
snapshotRevision: 11,
outcome: 'complete'
}
})
expect(harness.deps.readHostScopedLineage).toHaveBeenCalledOnce()
})
it('requires exact snapshot revision and authority admission', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
harness.coordinator.replaceAuthority(owner)
const pending = harness.coordinator.prepareOnly(
preparationInput(owner, [], { snapshotRevision: 5 })
)
const prepared = await pending
const token = prepared.token!
expect(buildDirectSshSnapshotApplyToken(token, 6)).toBeNull()
const exact = buildDirectSshSnapshotApplyToken(token, 5)!
expect(admitDirectSshSnapshotApplyToken(exact, owner, 5)).toBe(true)
expect(admitDirectSshSnapshotApplyToken(exact, owner, 6)).toBe(false)
expect(admitDirectSshSnapshotApplyToken(exact, authority('target-a', 'other'), 5)).toBe(false)
})
it('damps rapid rotations to the latest authority while terminal checks stay immediate', async () => {
vi.useFakeTimers()
vi.setSystemTime(0)
const harness = createCoordinatorHarness()
const initial = authority('target-a', 'initial')
const middle = authority('target-a', 'middle')
const flapping = authority('target-a', 'flapping')
const latest = authority('target-a', 'latest')
harness.current.set(initial.targetId, initial)
harness.coordinator.replaceAuthority(initial)
vi.setSystemTime(5_001)
harness.current.set(middle.targetId, middle)
const stableReplacement = harness.coordinator.requestReconnect(middle)
await flush()
harness.complete(0)
await expect(stableReplacement).resolves.toMatchObject({
status: 'complete',
stabilizing: false
})
vi.setSystemTime(6_000)
harness.current.set(flapping.targetId, flapping)
await expect(harness.coordinator.requestReconnect(flapping)).resolves.toMatchObject({
status: 'stabilizing'
})
vi.setSystemTime(7_000)
harness.current.set(latest.targetId, latest)
await expect(harness.coordinator.requestReconnect(latest)).resolves.toMatchObject({
status: 'stabilizing'
})
expect(harness.deps.retryTargetPanes).toHaveBeenCalledTimes(3)
expect(harness.deps.capturePreparationInput).toHaveBeenCalledTimes(1)
await vi.advanceTimersByTimeAsync(4_999)
expect(harness.deps.capturePreparationInput).toHaveBeenCalledTimes(1)
await vi.advanceTimersByTimeAsync(1)
expect(harness.deps.capturePreparationInput).toHaveBeenCalledTimes(2)
expect(harness.deps.capturePreparationInput).toHaveBeenCalledWith(latest, 'reconnect')
expect(harness.deps.capturePreparationInput).not.toHaveBeenCalledWith(flapping, 'reconnect')
harness.complete(1)
})
it('corrects only current authority and cleans pending work and timers on stop', async () => {
vi.useFakeTimers()
vi.setSystemTime(0)
const harness = createCoordinatorHarness()
const initial = authority('target-a', 'initial')
const latest = authority('target-a', 'latest')
harness.current.set(initial.targetId, initial)
harness.coordinator.replaceAuthority(initial)
harness.current.set(latest.targetId, latest)
void harness.coordinator.requestReconnect(latest)
expect(harness.coordinator.correctUnboundTerminals(latest, 'wake-refresh')).toBe(1)
expect(harness.coordinator.finalizeHydratedTerminals(initial)).toBe(0)
harness.coordinator.stop()
await vi.advanceTimersByTimeAsync(5_000)
expect(harness.deps.capturePreparationInput).not.toHaveBeenCalled()
expect(harness.scheduler.stop).toHaveBeenCalledOnce()
expect(harness.coordinator.correctUnboundTerminals(latest, 'wake-refresh')).toBe(0)
})
it('emits one aggregate for joined work with scheduler and scope metrics', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
harness.coordinator.replaceAuthority(owner)
const input = preparationInput(owner, ['repo'], {
telemetry: {
catalogOutcome: 'degraded',
catalogDurationMs: 12,
gitWorktreeCount: 3,
folderWorkspaceCount: 2,
ambiguousOwnerCount: 1,
contradictoryOwnerCount: 1
}
})
const first = harness.coordinator.prepareOnly(input)
const joined = harness.coordinator.prepareOnly(input)
harness.leases[0].deferred.resolve({
status: 'complete',
metrics: {
queueWaitDurationsMs: [8],
providerExecutionDurationsMs: [13],
timeoutRetryCount: 1,
locallySettledWaiterCount: 1,
cancelDebtCount: 1,
replacementAdmissionDelayedCount: 1,
overlappingJoinCount: 2,
peakLocallyUnsettled: 4,
estimatedLateWorkAllowanceCount: 1
}
})
await Promise.all([first, joined])
expect(harness.deps.onTelemetry).toHaveBeenCalledOnce()
expect(harness.deps.onTelemetry).toHaveBeenCalledWith(
expect.objectContaining({
mode: 'prepare-only',
catalogOutcome: 'degraded',
catalogDurationMs: 12,
gitWorktreeCount: 3,
folderWorkspaceCount: 2,
queueWaitDurationsMs: [8],
providerExecutionDurationsMs: [13],
timeoutRetryCount: 1,
cancelDebtCount: 1,
replacementAdmissionDelayedCount: 1,
overlappingJoinCount: 3,
peakLocallyUnsettled: 4
})
)
})
it('does not let a telemetry callback failure affect recovery', async () => {
const harness = createCoordinatorHarness()
const owner = authority('target-a')
harness.current.set(owner.targetId, owner)
vi.mocked(harness.deps.onTelemetry!).mockImplementation(() => {
throw new Error('telemetry unavailable')
})
const pending = harness.coordinator.requestReconnect(owner)
await flush()
harness.complete(0)
await expect(pending).resolves.toMatchObject({ status: 'complete' })
})
})
src/renderer/src/hooks/direct-ssh-reconnect-coordinator.ts
+312
View File
@@ -0,0 +1,312 @@
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import {
createDirectSshPreparationCoordinator,
type DirectSshPreparationCoordinator
} from './direct-ssh-reconnect-preparation'
import {
combineDirectSshReconnectOutcome,
createEmptyDirectSshRepoOutcomeCounts,
createTerminalOnlyDirectSshReconnectOutcome
} from './direct-ssh-reconnect-coordinator-outcomes'
import {
createDirectSshReconnectTargetState,
type DirectSshReconnectTargetState
} from './direct-ssh-reconnect-coordinator-stabilization'
import { createDirectSshCoordinatorTelemetryReporter } from './direct-ssh-reconnect-coordinator-telemetry'
import type {
DirectSshCorrectionReason,
DirectSshPreparationInput,
DirectSshPreparationOutcome,
DirectSshPreparationReason,
DirectSshReconnectCoordinator,
DirectSshReconnectCoordinatorDeps,
DirectSshReconnectOutcome,
DirectSshReconnectTimer
} from './direct-ssh-reconnect-coordinator-types'
import {
directSshAuthoritiesEqual,
isDirectSshPreparationInputHostConsistent,
normalizeDirectSshPreparationInput
} from './direct-ssh-reconnect-tokens'
export type * from './direct-ssh-reconnect-coordinator-types'
export {
admitDirectSshSnapshotApplyToken,
buildDirectSshSnapshotApplyToken
} from './direct-ssh-reconnect-tokens'
export const DIRECT_SSH_RELAY_STABILIZATION_MS = 5_000
export function createDirectSshReconnectCoordinator(
deps: DirectSshReconnectCoordinatorDeps
): DirectSshReconnectCoordinator {
const now = deps.now ?? Date.now
const setTimer =
deps.setTimer ?? ((callback: () => void, delayMs: number) => setTimeout(callback, delayMs))
const clearTimer =
deps.clearTimer ??
((timer: DirectSshReconnectTimer) => clearTimeout(timer as ReturnType<typeof setTimeout>))
const stabilizationMs = deps.stabilizationMs ?? DIRECT_SSH_RELAY_STABILIZATION_MS
const targets = new Map<string, DirectSshReconnectTargetState>()
let stopped = false
const isCurrent = (authority: DirectSshAuthority): boolean => {
const state = targets.get(authority.targetId)
return (
!stopped &&
directSshAuthoritiesEqual(state?.authority, authority) &&
deps.isCurrentConnectedAuthority(authority)
)
}
const preparation: DirectSshPreparationCoordinator = createDirectSshPreparationCoordinator({
scheduler: deps.scheduler,
isCurrentAuthority: isCurrent,
readLineage: deps.readHostScopedLineage,
now
})
const telemetry = createDirectSshCoordinatorTelemetryReporter({
onTelemetry: deps.onTelemetry,
now
})
const replaceAuthority = (authority: DirectSshAuthority): boolean => {
if (stopped) {
return false
}
const previous = targets.get(authority.targetId)
if (directSshAuthoritiesEqual(previous?.authority, authority)) {
return false
}
if (previous) {
if (previous.timer) {
clearTimer(previous.timer)
}
preparation.invalidateAuthority(previous.authority)
deps.scheduler.disposeProvider(previous.authority)
}
const installedAt = now()
targets.set(
authority.targetId,
createDirectSshReconnectTargetState(authority, previous, installedAt, stabilizationMs)
)
return previous !== undefined
}
const captureInput = async (
authority: DirectSshAuthority,
reason: DirectSshPreparationReason
): Promise<DirectSshPreparationInput | null> => {
const captured = await deps.capturePreparationInput(authority, reason)
if (
!captured ||
!directSshAuthoritiesEqual(captured, authority) ||
!isDirectSshPreparationInputHostConsistent(captured) ||
!isCurrent(authority)
) {
return null
}
return normalizeDirectSshPreparationInput({ ...captured, reason })
}
const startSync = (token: NonNullable<DirectSshPreparationOutcome['token']>): void => {
try {
void Promise.resolve(deps.syncRemoteWorkspaceAfterConnect(token)).catch(() => {})
} catch {
// Sync failure is reported by its owning boundary.
}
}
const runPreparedReconnect = async (
authority: DirectSshAuthority,
staleBindingsCleared: number,
retriedTerminals: number,
damped: boolean,
operationStartedAt = now(),
terminalFinalizationDurationMs = 0,
authorityRotationCount = 0
): Promise<DirectSshReconnectOutcome> => {
const input = await captureInput(authority, 'reconnect')
if (!input) {
const outcome = createTerminalOnlyDirectSshReconnectOutcome(
'stale',
staleBindingsCleared,
retriedTerminals
)
telemetry.reportWithoutInput('reconnect', 'reconnect', outcome, operationStartedAt, {
staleBindingsCleared,
retriedTerminals,
terminalFinalizationDurationMs,
catalogOutcome: 'stale',
catalogDurationMs: Math.max(0, now() - operationStartedAt),
authorityRotationCount,
damped
})
return outcome
}
const acquired = preparation.acquire(input)
const prepared = await acquired.promise
let correctedTerminals = 0
if (prepared.token && isCurrent(authority)) {
correctedTerminals = deps.correctUnboundTerminalPanes(authority, 'preparation-complete')
if (isCurrent(authority)) {
startSync(prepared.token)
}
}
const outcome = combineDirectSshReconnectOutcome(
prepared,
staleBindingsCleared,
retriedTerminals,
correctedTerminals
)
if (!acquired.joined) {
telemetry.report('reconnect', input, prepared, operationStartedAt, {
terminalFinalizationDurationMs,
staleBindingsCleared,
retriedTerminals,
correctedTerminals,
damped,
authorityRotationCount
})
}
return outcome
}
const runDelayedPreparation = async (authority: DirectSshAuthority): Promise<void> => {
if (!isCurrent(authority)) {
return
}
await runPreparedReconnect(authority, 0, 0, true, now(), 0, 1)
}
const scheduleLatestPreparation = (state: DirectSshReconnectTargetState): void => {
if (state.timer || state.dampUntil === null) {
return
}
const delayMs = Math.max(0, state.dampUntil - now())
state.timer = setTimer(() => {
state.timer = null
state.dampUntil = null
void runDelayedPreparation(state.authority)
}, delayMs)
}
const requestReconnect = async (
authority: DirectSshAuthority
): Promise<DirectSshReconnectOutcome> => {
const startedAt = now()
if (stopped || !deps.isCurrentConnectedAuthority(authority)) {
const outcome = createTerminalOnlyDirectSshReconnectOutcome(stopped ? 'stopped' : 'stale')
telemetry.reportWithoutInput('reconnect', 'reconnect', outcome, startedAt, {
catalogOutcome: 'degraded'
})
return outcome
}
const rotated = replaceAuthority(authority)
if (!isCurrent(authority)) {
const outcome = createTerminalOnlyDirectSshReconnectOutcome('stale')
telemetry.reportWithoutInput('reconnect', 'reconnect', outcome, startedAt, {
authorityRotationCount: rotated ? 1 : 0
})
return outcome
}
const terminalStartedAt = now()
const staleBindingsCleared = deps.invalidateStaleTerminalBindings(authority)
const retriedTerminals = deps.retryTargetPanes(authority)
const terminalFinalizationDurationMs = Math.max(0, now() - terminalStartedAt)
const state = targets.get(authority.targetId)!
if (state.dampUntil !== null && now() < state.dampUntil) {
scheduleLatestPreparation(state)
const outcome = createTerminalOnlyDirectSshReconnectOutcome(
'stabilizing',
staleBindingsCleared,
retriedTerminals
)
telemetry.reportWithoutInput('reconnect', 'reconnect', outcome, startedAt, {
staleBindingsCleared,
retriedTerminals,
terminalFinalizationDurationMs,
catalogOutcome: 'degraded',
authorityRotationCount: rotated ? 1 : 0,
damped: true
})
return outcome
}
return runPreparedReconnect(
authority,
staleBindingsCleared,
retriedTerminals,
false,
startedAt,
terminalFinalizationDurationMs,
rotated ? 1 : 0
)
}
const prepareOnly = (
rawInput: DirectSshPreparationInput
): Promise<DirectSshPreparationOutcome> => {
const input = normalizeDirectSshPreparationInput(rawInput)
if (stopped || !isCurrent(input) || !isDirectSshPreparationInputHostConsistent(input)) {
const outcome: DirectSshPreparationOutcome = {
status: stopped ? 'stopped' : 'stale',
token: null,
repoOutcomes: createEmptyDirectSshRepoOutcomeCounts(),
lineageOutcome: 'not-started',
metrics: createTerminalOnlyDirectSshReconnectOutcome('stale').metrics
}
telemetry.report('prepare-only', input, outcome, now())
return Promise.resolve(outcome)
}
const startedAt = now()
const acquired = preparation.acquire(input)
if (!acquired.joined) {
void acquired.promise.then((outcome) => {
telemetry.report('prepare-only', input, outcome, startedAt)
})
}
return acquired.promise
}
const finalizeHydratedTerminals = (authority: DirectSshAuthority): number =>
isCurrent(authority) ? deps.finalizeHydratedTerminalPanes(authority) : 0
const correctUnboundTerminals = (
authority: DirectSshAuthority,
reason: DirectSshCorrectionReason
): number => (isCurrent(authority) ? deps.correctUnboundTerminalPanes(authority, reason) : 0)
const invalidate = (targetId: string): void => {
const state = targets.get(targetId)
if (state?.timer) {
clearTimer(state.timer)
}
preparation.invalidateTarget(targetId)
deps.scheduler.invalidateTarget(targetId)
targets.delete(targetId)
}
const stop = (): void => {
if (stopped) {
return
}
stopped = true
for (const state of targets.values()) {
if (state.timer) {
clearTimer(state.timer)
}
}
preparation.stop()
deps.scheduler.stop()
targets.clear()
}
return {
requestReconnect,
prepareOnly,
finalizeHydratedTerminals,
correctUnboundTerminals,
replaceAuthority,
invalidate,
stop
}
}
src/renderer/src/hooks/direct-ssh-reconnect-preparation-metrics.ts
+39
View File
@@ -0,0 +1,39 @@
import type { DirectSshWorktreeRefreshOutcome } from './direct-ssh-worktree-refresh-scheduler'
import type { DirectSshPreparationMetrics } from './direct-ssh-reconnect-coordinator-types'
import { createEmptyDirectSshPreparationMetrics } from './direct-ssh-reconnect-coordinator-outcomes'
export function aggregateDirectSshPreparationMetrics(
outcomes: readonly DirectSshWorktreeRefreshOutcome[],
joinCount: number
): DirectSshPreparationMetrics {
const aggregate = createEmptyDirectSshPreparationMetrics()
for (const outcome of outcomes) {
const metrics = outcome.metrics
if (!metrics) {
continue
}
aggregate.queueWaitDurationsMs = [
...aggregate.queueWaitDurationsMs,
...metrics.queueWaitDurationsMs
]
aggregate.providerExecutionDurationsMs = [
...aggregate.providerExecutionDurationsMs,
...metrics.providerExecutionDurationsMs
]
aggregate.timeoutRetryCount += metrics.timeoutRetryCount
aggregate.locallySettledWaiterCount += metrics.locallySettledWaiterCount
aggregate.cancelDebtCount += metrics.cancelDebtCount
aggregate.replacementAdmissionDelayedCount += metrics.replacementAdmissionDelayedCount
aggregate.schedulerOverlappingJoinCount += metrics.overlappingJoinCount
aggregate.peakLocallyUnsettled = Math.max(
aggregate.peakLocallyUnsettled,
metrics.peakLocallyUnsettled
)
aggregate.estimatedLateWorkAllowanceCount = Math.max(
aggregate.estimatedLateWorkAllowanceCount,
metrics.estimatedLateWorkAllowanceCount
)
}
aggregate.schedulerOverlappingJoinCount += joinCount
return aggregate
}
src/renderer/src/hooks/direct-ssh-reconnect-preparation.ts
+291
View File
@@ -0,0 +1,291 @@
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import type {
DirectSshWorktreeRefreshLease,
DirectSshWorktreeRefreshOutcome,
DirectSshWorktreeRefreshScheduler,
DirectSshWorktreeRefreshTerminalStatus
} from './direct-ssh-worktree-refresh-scheduler'
import type {
DirectSshLineageOutcome,
DirectSshPreparationInput,
DirectSshPreparationOutcome,
DirectSshPreparationToken,
DirectSshRepoOutcomeCounts
} from './direct-ssh-reconnect-coordinator-types'
import {
createEmptyDirectSshPreparationMetrics,
createEmptyDirectSshRepoOutcomeCounts
} from './direct-ssh-reconnect-coordinator-outcomes'
import { aggregateDirectSshPreparationMetrics } from './direct-ssh-reconnect-preparation-metrics'
import {
directSshAuthoritiesEqual,
directSshPreparationOperationKey,
directSshRepoFingerprint,
normalizeDirectSshPreparationInput
} from './direct-ssh-reconnect-tokens'
type PreparationOperation = {
key: string
input: DirectSshPreparationInput
leases: DirectSshWorktreeRefreshLease[]
joinCount: number
invalidatedAs: 'stale' | 'stopped' | null
settleInvalidation: (reason: 'stale' | 'stopped') => void
invalidation: Promise<'stale' | 'stopped'>
promise: Promise<DirectSshPreparationOutcome>
}
export type DirectSshPreparationCoordinator = {
acquire: (input: DirectSshPreparationInput) => {
promise: Promise<DirectSshPreparationOutcome>
joined: boolean
}
invalidateAuthority: (authority: DirectSshAuthority) => void
invalidateTarget: (targetId: string) => void
stop: () => void
}
type PreparationCoordinatorDeps = {
scheduler: DirectSshWorktreeRefreshScheduler
isCurrentAuthority: (authority: DirectSshAuthority) => boolean
readLineage: (input: DirectSshPreparationInput) => Promise<DirectSshLineageOutcome>
now?: () => number
}
const TERMINAL_STATUSES: readonly DirectSshWorktreeRefreshTerminalStatus[] = [
'complete',
'non-authoritative',
'timed-out',
'cancel-budget-exhausted',
'canceled',
'stale',
'rejected'
]
function countRepoOutcomes(
outcomes: readonly DirectSshWorktreeRefreshOutcome[]
): DirectSshRepoOutcomeCounts {
const counts = createEmptyDirectSshRepoOutcomeCounts()
for (const outcome of outcomes) {
counts[outcome.status]++
}
return counts
}
function awaitRepoLeases(
leases: readonly DirectSshWorktreeRefreshLease[]
): Promise<DirectSshWorktreeRefreshOutcome[]> {
if (leases.length === 0) {
return Promise.resolve([])
}
return new Promise((resolve) => {
const outcomes: DirectSshWorktreeRefreshOutcome[] = []
let remaining = leases.length
leases.forEach((lease, index) => {
void lease.result.then(
(outcome) => {
outcomes[index] = outcome
remaining--
if (remaining === 0) {
resolve(outcomes)
}
},
() => {
outcomes[index] = { status: 'rejected' }
remaining--
if (remaining === 0) {
resolve(outcomes)
}
}
)
})
})
}
function terminalPreparationOutcome(
status: 'canceled' | 'stale' | 'stopped',
repoOutcomes: DirectSshRepoOutcomeCounts,
metrics = createEmptyDirectSshPreparationMetrics()
): DirectSshPreparationOutcome {
return {
status,
token: null,
repoOutcomes,
lineageOutcome: 'not-started',
metrics
}
}
function buildPreparationToken(
input: DirectSshPreparationInput,
outcome: 'complete' | 'degraded'
): DirectSshPreparationToken {
return {
authority: {
targetId: input.targetId,
providerEpoch: input.providerEpoch,
connectionGeneration: input.connectionGeneration
},
catalogRevision: input.catalogRevision,
repoFingerprint: directSshRepoFingerprint(input),
authorityRequirement: input.authorityRequirement,
snapshotRevision: input.snapshotRevision ?? null,
outcome
}
}
function hasOutcome(
counts: DirectSshRepoOutcomeCounts,
statuses: readonly DirectSshWorktreeRefreshTerminalStatus[]
): boolean {
return statuses.some((status) => counts[status] > 0)
}
export function createDirectSshPreparationCoordinator(
deps: PreparationCoordinatorDeps
): DirectSshPreparationCoordinator {
const now = deps.now ?? Date.now
const operations = new Map<string, PreparationOperation>()
let stopped = false
const run = async (operation: PreparationOperation): Promise<DirectSshPreparationOutcome> => {
const { input } = operation
operation.leases = input.repoRefs.map((repo) =>
deps.scheduler.request({
targetId: input.targetId,
providerEpoch: input.providerEpoch,
connectionGeneration: input.connectionGeneration,
repoId: repo.repoId,
executionHostId: repo.executionHostId,
catalogRevision: input.catalogRevision,
authorityRequirement: input.authorityRequirement
})
)
const outcomes = await awaitRepoLeases(operation.leases)
const repoOutcomes = countRepoOutcomes(outcomes)
const metrics = aggregateDirectSshPreparationMetrics(outcomes, operation.joinCount)
if (operation.invalidatedAs) {
return terminalPreparationOutcome(operation.invalidatedAs, repoOutcomes, metrics)
}
if (!deps.isCurrentAuthority(input)) {
return terminalPreparationOutcome('stale', repoOutcomes, metrics)
}
if (hasOutcome(repoOutcomes, ['canceled', 'stale'])) {
const status = repoOutcomes.stale > 0 ? 'stale' : 'canceled'
return terminalPreparationOutcome(status, repoOutcomes, metrics)
}
let lineageOutcome: DirectSshLineageOutcome
const lineageStartedAt = now()
try {
lineageOutcome = await deps.readLineage(input)
} catch {
lineageOutcome = 'degraded'
}
metrics.lineageDurationMs = Math.max(0, now() - lineageStartedAt)
if (operation.invalidatedAs) {
return terminalPreparationOutcome(operation.invalidatedAs, repoOutcomes, metrics)
}
if (!deps.isCurrentAuthority(input) || lineageOutcome === 'stale') {
return terminalPreparationOutcome('stale', repoOutcomes, metrics)
}
if (lineageOutcome === 'canceled') {
return terminalPreparationOutcome('canceled', repoOutcomes, metrics)
}
const degraded =
lineageOutcome === 'degraded' ||
TERMINAL_STATUSES.some((status) => status !== 'complete' && repoOutcomes[status] > 0)
const status = degraded ? 'degraded' : 'complete'
return {
status,
token: buildPreparationToken(input, status),
repoOutcomes,
lineageOutcome,
metrics
}
}
const acquire = (
rawInput: DirectSshPreparationInput
): { promise: Promise<DirectSshPreparationOutcome>; joined: boolean } => {
const input = normalizeDirectSshPreparationInput(rawInput)
const key = directSshPreparationOperationKey(input)
const current = operations.get(key)
if (current) {
current.joinCount++
return { promise: current.promise, joined: true }
}
if (stopped) {
return {
promise: Promise.resolve(
terminalPreparationOutcome('stopped', createEmptyDirectSshRepoOutcomeCounts())
),
joined: false
}
}
const operation: PreparationOperation = {
key,
input,
leases: [],
joinCount: 0,
invalidatedAs: null,
settleInvalidation: () => {},
invalidation: Promise.resolve('stopped'),
promise: Promise.resolve(
terminalPreparationOutcome('stopped', createEmptyDirectSshRepoOutcomeCounts())
)
}
operation.invalidation = new Promise((resolve) => {
operation.settleInvalidation = resolve
})
operations.set(key, operation)
operation.promise = Promise.race([
run(operation),
operation.invalidation.then((reason) =>
terminalPreparationOutcome(reason, createEmptyDirectSshRepoOutcomeCounts())
)
]).finally(() => {
if (operations.get(key) === operation) {
operations.delete(key)
}
})
return { promise: operation.promise, joined: false }
}
const invalidateMatching = (
predicate: (operation: PreparationOperation) => boolean,
reason: 'stale' | 'stopped'
): void => {
for (const operation of operations.values()) {
if (!predicate(operation)) {
continue
}
operation.invalidatedAs = reason
operation.settleInvalidation(reason)
for (const lease of operation.leases) {
lease.release(reason === 'stopped' ? 'stopped' : 'invalidated')
}
}
}
const invalidateAuthority = (authority: DirectSshAuthority): void => {
invalidateMatching(
(operation) => directSshAuthoritiesEqual(operation.input, authority),
'stale'
)
}
const invalidateTarget = (targetId: string): void => {
invalidateMatching((operation) => operation.input.targetId === targetId, 'stale')
}
const stop = (): void => {
if (stopped) {
return
}
stopped = true
invalidateMatching(() => true, 'stopped')
}
return { acquire, invalidateAuthority, invalidateTarget, stop }
}
src/renderer/src/hooks/direct-ssh-reconnect-rollout.test.ts
+42
View File
@@ -0,0 +1,42 @@
import { describe, expect, it, vi } from 'vitest'
import {
DIRECT_SSH_RECONNECT_SESSION_ROUTE_KEY,
isDirectSshReconnectCoordinatorRoutingEnabled,
resolveDirectSshReconnectCoordinatorRouting
} from './direct-ssh-reconnect-rollout'
describe('direct SSH reconnect rollout', () => {
it('defaults coordinator routing on', () => {
expect(resolveDirectSshReconnectCoordinatorRouting({})).toBe(true)
expect(resolveDirectSshReconnectCoordinatorRouting({ buildValue: 'true' })).toBe(true)
})
it('supports build and session disable routes without session re-enablement', () => {
expect(
resolveDirectSshReconnectCoordinatorRouting({
buildValue: 'false',
sessionValue: 'true'
})
).toBe(false)
expect(
resolveDirectSshReconnectCoordinatorRouting({
buildValue: 'true',
sessionValue: 'false'
})
).toBe(false)
})
it('reads the current session route and fails open when storage is unavailable', () => {
const getItem = vi.fn(() => 'false')
vi.stubGlobal('sessionStorage', { getItem })
expect(isDirectSshReconnectCoordinatorRoutingEnabled()).toBe(false)
expect(getItem).toHaveBeenCalledWith(DIRECT_SSH_RECONNECT_SESSION_ROUTE_KEY)
getItem.mockImplementation(() => {
throw new Error('blocked')
})
expect(isDirectSshReconnectCoordinatorRoutingEnabled()).toBe(true)
vi.unstubAllGlobals()
})
})
src/renderer/src/hooks/direct-ssh-reconnect-rollout.ts
+25
View File
@@ -0,0 +1,25 @@
export const DIRECT_SSH_RECONNECT_SESSION_ROUTE_KEY = 'orca.directSshReconnectCoordinator.enabled'
export function resolveDirectSshReconnectCoordinatorRouting(args: {
buildValue?: string
sessionValue?: string | null
}): boolean {
if (args.buildValue?.trim().toLowerCase() === 'false') {
return false
}
return args.sessionValue?.trim().toLowerCase() !== 'false'
}
export function isDirectSshReconnectCoordinatorRoutingEnabled(): boolean {
let sessionValue: string | null = null
try {
sessionValue =
globalThis.sessionStorage?.getItem(DIRECT_SSH_RECONNECT_SESSION_ROUTE_KEY) ?? null
} catch {
// Storage can be unavailable in privacy-restricted renderer sessions.
}
return resolveDirectSshReconnectCoordinatorRouting({
buildValue: import.meta.env.VITE_DIRECT_SSH_RECONNECT_COORDINATOR,
sessionValue
})
}
src/renderer/src/hooks/direct-ssh-reconnect-tokens.test.ts
+27
View File
@@ -0,0 +1,27 @@
import { describe, expect, it } from 'vitest'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../shared/ssh-types'
import { directSshAuthoritiesEqual } from './direct-ssh-reconnect-tokens'
const authority: DirectSshAuthority = {
targetId: 'target-a',
providerEpoch: 'epoch-a' as SshProviderEpoch,
connectionGeneration: 1
}
describe('directSshAuthoritiesEqual', () => {
it('fails closed when either authority is missing', () => {
expect(directSshAuthoritiesEqual(null, null)).toBe(false)
expect(directSshAuthoritiesEqual(authority, null)).toBe(false)
expect(directSshAuthoritiesEqual(undefined, authority)).toBe(false)
})
it('requires the complete authority tuple', () => {
expect(directSshAuthoritiesEqual(authority, { ...authority })).toBe(true)
expect(
directSshAuthoritiesEqual(authority, {
...authority,
connectionGeneration: authority.connectionGeneration + 1
})
).toBe(false)
})
})
src/renderer/src/hooks/direct-ssh-reconnect-tokens.ts
+83
View File
@@ -0,0 +1,83 @@
import { toSshExecutionHostId } from '../../../shared/execution-host'
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import type {
DirectSshPreparationInput,
DirectSshPreparationToken,
DirectSshSnapshotApplyToken
} from './direct-ssh-reconnect-coordinator-types'
export function directSshAuthoritiesEqual(
left: DirectSshAuthority | null | undefined,
right: DirectSshAuthority | null | undefined
): boolean {
if (!left || !right) {
return false
}
return (
left.targetId === right.targetId &&
left.providerEpoch === right.providerEpoch &&
left.connectionGeneration === right.connectionGeneration
)
}
export function normalizeDirectSshPreparationInput(
input: DirectSshPreparationInput
): DirectSshPreparationInput {
return {
...input,
repoRefs: [...input.repoRefs].sort(
(left, right) =>
compareText(left.executionHostId, right.executionHostId) ||
compareText(left.repoId, right.repoId)
)
}
}
function compareText(left: string, right: string): number {
return left < right ? -1 : left > right ? 1 : 0
}
export function isDirectSshPreparationInputHostConsistent(
input: DirectSshPreparationInput
): boolean {
const expectedHost = toSshExecutionHostId(input.targetId)
return input.repoRefs.every((repo) => repo.executionHostId === expectedHost)
}
export function directSshRepoFingerprint(input: DirectSshPreparationInput): string {
return JSON.stringify(input.repoRefs.map((repo) => [repo.executionHostId, repo.repoId]))
}
export function directSshPreparationOperationKey(input: DirectSshPreparationInput): string {
return JSON.stringify([
input.targetId,
input.providerEpoch,
input.connectionGeneration,
input.catalogRevision,
directSshRepoFingerprint(input),
input.authorityRequirement,
input.snapshotRevision ?? null,
input.reason
])
}
export function buildDirectSshSnapshotApplyToken(
token: DirectSshPreparationToken,
snapshotRevision: number
): DirectSshSnapshotApplyToken | null {
if (token.snapshotRevision !== null && token.snapshotRevision !== snapshotRevision) {
return null
}
return { ...token, snapshotRevision }
}
export function admitDirectSshSnapshotApplyToken(
token: DirectSshSnapshotApplyToken,
authority: DirectSshAuthority,
snapshotRevision: number
): boolean {
return (
token.snapshotRevision === snapshotRevision &&
directSshAuthoritiesEqual(token.authority, authority)
)
}
src/renderer/src/hooks/direct-ssh-runtime-wake-isolation.test.tsx
+184
View File
@@ -0,0 +1,184 @@
// @vitest-environment happy-dom
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { renderHook } from '@testing-library/react'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../shared/ssh-types'
import { createTestStore, makeTab, makeWorktree } from '@/store/slices/store-test-helpers'
import { useRemoteRuntimeRecoveryTriggers } from '@/runtime/use-remote-runtime-recovery-triggers'
import { registerDirectSshWakeRouting } from './direct-ssh-state-routing'
const { retryAllRemoteRuntimePtyRecoveriesNowMock } = vi.hoisted(() => ({
retryAllRemoteRuntimePtyRecoveriesNowMock: vi.fn()
}))
vi.mock('@/components/terminal-pane/remote-runtime-pty-recovery-state', () => ({
retryAllRemoteRuntimePtyRecoveriesNow: retryAllRemoteRuntimePtyRecoveriesNowMock
}))
function authority(): DirectSshAuthority {
return {
targetId: 'direct-target',
providerEpoch: 'direct-epoch' as SshProviderEpoch,
connectionGeneration: 7
}
}
describe('runtime and direct SSH wake isolation', () => {
const retryConnectionsNow = vi.fn(() => Promise.resolve())
const resumeCallbacks = new Set<() => void>()
const onSystemResumed = vi.fn((callback: () => void) => {
resumeCallbacks.add(callback)
return () => resumeCallbacks.delete(callback)
})
beforeEach(() => {
retryConnectionsNow.mockClear()
retryAllRemoteRuntimePtyRecoveriesNowMock.mockClear()
onSystemResumed.mockClear()
resumeCallbacks.clear()
;(window as unknown as { api: unknown }).api = {
runtimeEnvironments: { retryConnectionsNow },
ui: { onSystemResumed }
}
})
afterEach(() => {
delete (window as unknown as { api?: unknown }).api
})
it('advances both runtime backoffs and one exact direct wake without rebumping a healthy pane', () => {
const currentAuthority = authority()
const directWorktreeId = 'repo-direct::/work/direct'
const runtimeWorktreeId = 'repo-runtime::/work/runtime'
const directPtyId = 'ssh:direct-target@@pty-live'
const store = createTestStore()
store.setState({
repos: [
{
id: 'repo-direct',
path: '/work/direct',
displayName: 'direct',
badgeColor: '#000',
addedAt: 1,
connectionId: 'direct-target',
executionHostId: 'ssh:direct-target'
},
{
id: 'repo-runtime',
path: '/work/runtime',
displayName: 'runtime',
badgeColor: '#000',
addedAt: 1,
executionHostId: 'runtime:env-1'
}
],
worktreesByRepo: {
'repo-direct': [
makeWorktree({
id: directWorktreeId,
repoId: 'repo-direct',
path: '/work/direct',
hostId: 'ssh:direct-target'
})
],
'repo-runtime': [
makeWorktree({
id: runtimeWorktreeId,
repoId: 'repo-runtime',
path: '/work/runtime',
hostId: 'ssh:runtime-owned-target',
runtimeOwnerEnvironmentId: 'env-1'
})
]
},
tabsByWorktree: {
[directWorktreeId]: [
makeTab({
id: 'tab-direct',
worktreeId: directWorktreeId,
ptyId: directPtyId,
generation: 0
})
],
[runtimeWorktreeId]: [
makeTab({ id: 'tab-runtime', worktreeId: runtimeWorktreeId, ptyId: null })
]
},
ptyIdsByTabId: { 'tab-direct': [directPtyId], 'tab-runtime': [] },
directSshLivePtyBindingByTabId: {
'tab-direct': {
attemptId: 'direct-attempt' as never,
authority: currentAuthority,
tabGeneration: 0,
ptyId: directPtyId
}
},
sshConnectionStates: new Map([
[
'direct-target',
{
targetId: 'direct-target',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: currentAuthority.providerEpoch,
connectionGeneration: currentAuthority.connectionGeneration
}
]
]),
sshStateByEnvironment: new Map([
[
'env-1',
{
connectionStates: new Map([
[
'runtime-owned-target',
{
targetId: 'runtime-owned-target',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'runtime-epoch' as SshProviderEpoch,
connectionGeneration: 3
}
]
]),
targetLabels: new Map([['runtime-owned-target', 'runtime target']]),
removedTargetLabels: new Map(),
targetsHydrated: true
}
]
])
})
const wakePreparation = vi.fn()
const correctedCounts: number[] = []
const unregisterDirectWake = registerDirectSshWakeRouting({
getConnectionStates: () => store.getState().sshConnectionStates,
wakeAuthority: (nextAuthority) => {
correctedCounts.push(store.getState().retryDirectSshTargetPanes(nextAuthority, 1_000))
wakePreparation(nextAuthority)
},
onSystemResumed
})
const { unmount } = renderHook(() => useRemoteRuntimeRecoveryTriggers())
window.dispatchEvent(new Event('online'))
expect(retryConnectionsNow).toHaveBeenCalledTimes(1)
expect(retryAllRemoteRuntimePtyRecoveriesNowMock).toHaveBeenCalledTimes(1)
expect(wakePreparation).toHaveBeenCalledOnce()
expect(wakePreparation).toHaveBeenCalledWith(currentAuthority)
expect(correctedCounts).toEqual([0])
expect(store.getState().tabsByWorktree[directWorktreeId][0]).toMatchObject({
generation: 0,
ptyId: directPtyId
})
expect(store.getState().tabsByWorktree[runtimeWorktreeId][0].generation ?? 0).toBe(0)
expect(store.getState().directSshPaneRetryByTabId).toEqual({})
unregisterDirectWake()
unmount()
window.dispatchEvent(new Event('online'))
expect(retryConnectionsNow).toHaveBeenCalledTimes(1)
expect(wakePreparation).toHaveBeenCalledTimes(1)
})
})
src/renderer/src/hooks/direct-ssh-state-routing.test.ts
+200
View File
@@ -0,0 +1,200 @@
// @vitest-environment happy-dom
import { describe, expect, it, vi } from 'vitest'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../shared/ssh-types'
import {
directSshAuthorityFromConnectionState,
registerDirectSshWakeRouting,
routeDirectSshConnectedState
} from './direct-ssh-state-routing'
function authority(epoch = 'epoch-a'): DirectSshAuthority {
return {
targetId: 'target-a',
providerEpoch: epoch as SshProviderEpoch,
connectionGeneration: 1
}
}
function harness() {
return {
coordinatorRoutingEnabled: true,
coordinator: {
requestReconnect: vi.fn(async () => ({ status: 'complete' })),
correctUnboundTerminals: vi.fn(() => 0),
replaceAuthority: vi.fn()
},
invalidateStaleTerminalBindings: vi.fn(() => 0),
retryTargetPanes: vi.fn(() => 0),
prepareAndSync: vi.fn(),
rememberReconnectAuthority: vi.fn()
}
}
describe('routeDirectSshConnectedState', () => {
it('keeps initial hydration preparation-only', () => {
const deps = harness()
const current = authority()
expect(
routeDirectSshConnectedState(deps, {
authority: current,
previousAuthority: null,
origin: 'initial-hydration'
})
).toBe('initial-hydration')
expect(deps.prepareAndSync).toHaveBeenCalledWith(current, 'initial-hydration')
expect(deps.coordinator.requestReconnect).not.toHaveBeenCalled()
expect(deps.coordinator.correctUnboundTerminals).not.toHaveBeenCalled()
})
it('requests reconnect only when the complete pushed authority changes', () => {
const deps = harness()
const previous = authority('epoch-old')
const current = authority('epoch-new')
expect(
routeDirectSshConnectedState(deps, {
authority: current,
previousAuthority: previous,
origin: 'push'
})
).toBe('changed-authority')
expect(deps.rememberReconnectAuthority).toHaveBeenCalledWith(current)
expect(deps.coordinator.requestReconnect).toHaveBeenCalledWith(current)
expect(deps.coordinator.replaceAuthority).not.toHaveBeenCalled()
expect(deps.prepareAndSync).not.toHaveBeenCalled()
})
it('runs the disabled fallback terminal actions synchronously before bounded preparation', () => {
const order: string[] = []
const deps = harness()
deps.coordinatorRoutingEnabled = false
deps.coordinator.replaceAuthority.mockImplementation(() => order.push('replace'))
deps.invalidateStaleTerminalBindings.mockImplementation(() => {
order.push('invalidate')
return 1
})
deps.retryTargetPanes.mockImplementation(() => {
order.push('retry')
return 1
})
deps.prepareAndSync.mockImplementation(() => {
order.push('prepare')
return new Promise(() => {})
})
const previous = authority('epoch-old')
const current = authority('epoch-new')
expect(
routeDirectSshConnectedState(deps, {
authority: current,
previousAuthority: previous,
origin: 'push'
})
).toBe('changed-authority-fallback')
expect(order).toEqual(['replace', 'invalidate', 'retry', 'prepare'])
expect(deps.coordinator.requestReconnect).not.toHaveBeenCalled()
expect(deps.prepareAndSync).toHaveBeenCalledWith(current, 'reconnect', {
authorityAlreadyReplaced: true
})
expect(deps.rememberReconnectAuthority).toHaveBeenCalledWith(current)
})
it('routes an exact-equal push through correction and fresh preparation', () => {
const deps = harness()
const current = authority()
expect(
routeDirectSshConnectedState(deps, {
authority: current,
previousAuthority: { ...current },
origin: 'push'
})
).toBe('same-authority-wake')
expect(deps.coordinator.correctUnboundTerminals).toHaveBeenCalledWith(current, 'wake-refresh')
expect(deps.prepareAndSync).toHaveBeenCalledWith(current, 'wake-refresh')
expect(deps.coordinator.requestReconnect).not.toHaveBeenCalled()
})
})
describe('registerDirectSshWakeRouting', () => {
it('wakes only complete connected direct authorities and removes both listeners', () => {
const wakeAuthority = vi.fn()
let resume: (() => void) | undefined
const unsubscribeSystemResumed = vi.fn()
const stop = registerDirectSshWakeRouting({
getConnectionStates: () =>
new Map([
[
'target-a',
{
...authority(),
status: 'connected' as const,
error: null,
reconnectAttempt: 0
}
],
[
'target-partial',
{
targetId: 'target-partial',
status: 'connected' as const,
error: null,
reconnectAttempt: 0,
providerEpoch: 'partial' as SshProviderEpoch
}
],
[
'target-disconnected',
{
...authority('disconnected'),
targetId: 'target-disconnected',
status: 'disconnected' as const,
error: null,
reconnectAttempt: 0
}
]
]),
wakeAuthority,
onSystemResumed: (callback) => {
resume = callback
return unsubscribeSystemResumed
}
})
window.dispatchEvent(new Event('online'))
resume?.()
expect(wakeAuthority).toHaveBeenCalledTimes(2)
expect(wakeAuthority).toHaveBeenNthCalledWith(1, authority())
expect(wakeAuthority).toHaveBeenNthCalledWith(2, authority())
stop()
stop()
window.dispatchEvent(new Event('online'))
resume?.()
expect(unsubscribeSystemResumed).toHaveBeenCalledOnce()
expect(wakeAuthority).toHaveBeenCalledTimes(2)
})
it('rejects mismatched keys and malformed partial authority', () => {
expect(
directSshAuthorityFromConnectionState('other-target', {
...authority(),
status: 'connected',
error: null,
reconnectAttempt: 0
})
).toBeNull()
expect(
directSshAuthorityFromConnectionState('target-a', {
targetId: 'target-a',
status: 'connected',
error: null,
reconnectAttempt: 0,
connectionGeneration: 1
})
).toBeNull()
})
})
src/renderer/src/hooks/direct-ssh-state-routing.ts
+110
View File
@@ -0,0 +1,110 @@
import type { DirectSshAuthority, SshConnectionState } from '../../../shared/ssh-types'
import { directSshAuthoritiesEqual } from './direct-ssh-reconnect-tokens'
export type DirectSshConnectedStateOrigin = 'push' | 'initial-hydration'
export type DirectSshConnectedStateRoutingDeps = {
coordinator: {
requestReconnect: (authority: DirectSshAuthority) => Promise<unknown>
correctUnboundTerminals: (authority: DirectSshAuthority, reason: 'wake-refresh') => number
replaceAuthority: (authority: DirectSshAuthority) => void
}
coordinatorRoutingEnabled: boolean
invalidateStaleTerminalBindings: (authority: DirectSshAuthority) => number
retryTargetPanes: (authority: DirectSshAuthority) => number
prepareAndSync: (
authority: DirectSshAuthority,
reason: 'reconnect' | 'initial-hydration' | 'wake-refresh',
options?: { authorityAlreadyReplaced?: boolean }
) => void | Promise<void>
rememberReconnectAuthority: (authority: DirectSshAuthority | null) => void
}
export type DirectSshConnectedStateRoute =
| 'initial-hydration'
| 'changed-authority'
| 'changed-authority-fallback'
| 'same-authority-wake'
export function routeDirectSshConnectedState(
deps: DirectSshConnectedStateRoutingDeps,
input: {
authority: DirectSshAuthority
previousAuthority: DirectSshAuthority | null
origin: DirectSshConnectedStateOrigin
}
): DirectSshConnectedStateRoute {
const { authority } = input
if (input.origin === 'initial-hydration') {
deps.rememberReconnectAuthority(null)
void deps.prepareAndSync(authority, 'initial-hydration')
return 'initial-hydration'
}
if (!directSshAuthoritiesEqual(input.previousAuthority, authority)) {
deps.rememberReconnectAuthority(authority)
if (deps.coordinatorRoutingEnabled) {
void deps.coordinator.requestReconnect(authority)
return 'changed-authority'
}
deps.coordinator.replaceAuthority(authority)
deps.invalidateStaleTerminalBindings(authority)
deps.retryTargetPanes(authority)
void deps.prepareAndSync(authority, 'reconnect', { authorityAlreadyReplaced: true })
return 'changed-authority-fallback'
}
deps.coordinator.correctUnboundTerminals(authority, 'wake-refresh')
void deps.prepareAndSync(authority, 'wake-refresh')
return 'same-authority-wake'
}
export function directSshAuthorityFromConnectionState(
targetId: string,
state: SshConnectionState
): DirectSshAuthority | null {
if (
state.targetId !== targetId ||
state.status !== 'connected' ||
!state.providerEpoch ||
state.connectionGeneration === undefined
) {
return null
}
return {
targetId,
providerEpoch: state.providerEpoch,
connectionGeneration: state.connectionGeneration
}
}
export function registerDirectSshWakeRouting(deps: {
getConnectionStates: () => Iterable<readonly [string, SshConnectionState]>
wakeAuthority: (authority: DirectSshAuthority) => void
onSystemResumed?: (callback: () => void) => () => void
}): () => void {
let stopped = false
const wake = (): void => {
if (stopped) {
return
}
for (const [targetId, state] of deps.getConnectionStates()) {
const authority = directSshAuthorityFromConnectionState(targetId, state)
if (authority) {
deps.wakeAuthority(authority)
}
}
}
if (typeof window.addEventListener === 'function') {
window.addEventListener('online', wake)
}
const unsubscribeSystemResumed = deps.onSystemResumed?.(wake)
return () => {
if (stopped) {
return
}
stopped = true
if (typeof window.removeEventListener === 'function') {
window.removeEventListener('online', wake)
}
unsubscribeSystemResumed?.()
}
}
src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler-cancellation.ts
+16
View File
@@ -0,0 +1,16 @@
import type {
DirectSshWorktreeRefreshAttempt,
DirectSshWorktreeRefreshReleaseReason
} from './direct-ssh-worktree-refresh-scheduler-types'
export function cancelDirectSshWorktreeRefreshAttempt(
attempt: DirectSshWorktreeRefreshAttempt,
reason: DirectSshWorktreeRefreshReleaseReason
): boolean {
try {
const outcome = attempt.cancel(reason)
return outcome !== false && outcome !== 'retained' && outcome !== 'already-settled'
} catch {
return true
}
}
src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler-metrics.ts
+95
View File
@@ -0,0 +1,95 @@
import type {
DirectSshWorktreeRefreshLogicalTask,
DirectSshWorktreeRefreshMetrics,
DirectSshWorktreeRefreshSchedulerSnapshot
} from './direct-ssh-worktree-refresh-scheduler-types'
export function createDirectSshWorktreeRefreshMetrics(): DirectSshWorktreeRefreshMetrics {
return {
queueWaitDurationsMs: [],
providerExecutionDurationsMs: [],
timeoutRetryCount: 0,
locallySettledWaiterCount: 0,
cancelDebtCount: 0,
replacementAdmissionDelayedCount: 0,
overlappingJoinCount: 0,
peakLocallyUnsettled: 0,
estimatedLateWorkAllowanceCount: 0
}
}
export function adjustDirectSshAuthorityUnsettled(
unsettledByAuthority: Map<string, number>,
authorityId: string,
delta: number
): void {
const next = (unsettledByAuthority.get(authorityId) ?? 0) + delta
if (next > 0) {
unsettledByAuthority.set(authorityId, next)
} else {
unsettledByAuthority.delete(authorityId)
}
}
export function copyDirectSshWorktreeRefreshMetrics(
metrics: DirectSshWorktreeRefreshMetrics
): DirectSshWorktreeRefreshMetrics {
return {
...metrics,
queueWaitDurationsMs: [...metrics.queueWaitDurationsMs],
providerExecutionDurationsMs: [...metrics.providerExecutionDurationsMs]
}
}
export function recordDirectSshCancelDebt(
metrics: DirectSshWorktreeRefreshMetrics,
lateWorkLimit: number
): void {
metrics.cancelDebtCount++
metrics.estimatedLateWorkAllowanceCount = Math.min(
lateWorkLimit,
metrics.estimatedLateWorkAllowanceCount + 1
)
}
export function recordDirectSshQueueAdmission(
metrics: DirectSshWorktreeRefreshMetrics,
queueWaitDurationMs: number,
locallyUnsettled: number
): void {
metrics.queueWaitDurationsMs = [...metrics.queueWaitDurationsMs, queueWaitDurationMs]
metrics.peakLocallyUnsettled = Math.max(metrics.peakLocallyUnsettled, locallyUnsettled)
}
export function recordDirectSshProviderExecution(
metrics: DirectSshWorktreeRefreshMetrics,
providerExecutionDurationMs: number
): void {
metrics.providerExecutionDurationsMs = [
...metrics.providerExecutionDurationsMs,
providerExecutionDurationMs
]
}
export function directSshWorktreeRefreshSchedulerSnapshot(
tasks: ReadonlyMap<string, DirectSshWorktreeRefreshLogicalTask>,
locallyUnsettled: number,
cancelDebtByAuthority: ReadonlyMap<string, number>
): DirectSshWorktreeRefreshSchedulerSnapshot {
let queued = 0
let retrying = 0
let waiters = 0
for (const task of tasks.values()) {
queued += task.state === 'queued' ? 1 : 0
retrying += task.state === 'retrying' ? 1 : 0
waiters += task.waiters.size
}
return {
locallyUnsettled,
queued,
retrying,
logicalTasks: tasks.size,
waiters,
cancelDebtByAuthority: new Map(cancelDebtByAuthority)
}
}
src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler-types.ts
+128
View File
@@ -0,0 +1,128 @@
import type {
HostQualifiedDetectedWorktreeResult,
ProviderRequestId,
SshExecutionHostId
} from '../../../shared/detected-worktree-provider-contract'
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import type {
DetectedWorktreeRefreshReleaseOutcome,
WaiterLeaseId
} from '../store/slices/detected-worktree-refresh-leases'
export type { WaiterLeaseId } from '../store/slices/detected-worktree-refresh-leases'
export type DirectSshWorktreeRefreshAuthorityRequirement = 'required' | 'allow-metadata-fallback'
export type DirectSshWorktreeRefreshKey = DirectSshAuthority & {
repoId: string
executionHostId: SshExecutionHostId
catalogRevision: number
authorityRequirement: DirectSshWorktreeRefreshAuthorityRequirement
}
export type DirectSshWorktreeRefreshTerminalStatus =
| 'complete'
| 'non-authoritative'
| 'timed-out'
| 'cancel-budget-exhausted'
| 'canceled'
| 'stale'
| 'rejected'
export type DirectSshWorktreeRefreshOutcome = {
status: DirectSshWorktreeRefreshTerminalStatus
providerRequestId?: ProviderRequestId
providerResult?: HostQualifiedDetectedWorktreeResult
metrics?: DirectSshWorktreeRefreshMetrics
}
export type DirectSshWorktreeRefreshMetrics = {
queueWaitDurationsMs: readonly number[]
providerExecutionDurationsMs: readonly number[]
timeoutRetryCount: number
locallySettledWaiterCount: number
cancelDebtCount: number
replacementAdmissionDelayedCount: number
overlappingJoinCount: number
peakLocallyUnsettled: number
estimatedLateWorkAllowanceCount: number
}
export type DirectSshWorktreeRefreshReleaseReason = 'superseded' | 'invalidated' | 'stopped'
export type DirectSshWorktreeRefreshLease = {
waiterLeaseId: WaiterLeaseId
result: Promise<DirectSshWorktreeRefreshOutcome>
release: (reason: DirectSshWorktreeRefreshReleaseReason) => void
}
export type DirectSshWorktreeRefreshAttempt = {
providerRequestId: ProviderRequestId
result: Promise<HostQualifiedDetectedWorktreeResult>
cancel: (
reason: DirectSshWorktreeRefreshReleaseReason
) => boolean | void | DetectedWorktreeRefreshReleaseOutcome
}
export type DirectSshWorktreeRefreshSchedulerDeps = {
startAttempt: (key: DirectSshWorktreeRefreshKey) => DirectSshWorktreeRefreshAttempt
createWaiterLeaseId?: () => WaiterLeaseId
onUnexpectedError?: (error: unknown) => void
now?: () => number
}
export type DirectSshWorktreeRefreshSchedulerSnapshot = {
locallyUnsettled: number
queued: number
retrying: number
logicalTasks: number
waiters: number
cancelDebtByAuthority: ReadonlyMap<string, number>
}
export type DirectSshWorktreeRefreshScheduler = {
request: (key: DirectSshWorktreeRefreshKey) => DirectSshWorktreeRefreshLease
invalidateAuthority: (authority: DirectSshAuthority) => void
invalidateTarget: (targetId: string) => void
disposeProvider: (authority: DirectSshAuthority) => void
getSnapshot: () => DirectSshWorktreeRefreshSchedulerSnapshot
stop: () => void
}
export type DirectSshWorktreeRefreshWaiter = {
resolve: (outcome: DirectSshWorktreeRefreshOutcome) => void
}
export type DirectSshWorktreeRefreshLogicalTask = {
key: DirectSshWorktreeRefreshKey
keyId: string
authorityId: string
state: 'queued' | 'running' | 'retrying' | 'terminal'
attemptCount: number
attempt: DirectSshWorktreeRefreshAttempt | null
attemptCanceled: boolean
attemptStartedAt: number | null
queuedAt: number
metrics: DirectSshWorktreeRefreshMetrics
waiters: Map<WaiterLeaseId, DirectSshWorktreeRefreshWaiter>
}
export function directSshProviderAuthorityKey(authority: DirectSshAuthority): string {
return JSON.stringify([
authority.targetId,
authority.providerEpoch,
authority.connectionGeneration
])
}
export function directSshWorktreeRefreshKey(key: DirectSshWorktreeRefreshKey): string {
return JSON.stringify([
key.targetId,
key.repoId,
key.executionHostId,
key.providerEpoch,
key.connectionGeneration,
key.catalogRevision,
key.authorityRequirement
])
}
src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler.test.ts
+435
View File
@@ -0,0 +1,435 @@
import { describe, expect, it, vi } from 'vitest'
import type {
HostQualifiedDetectedWorktreeResult,
ProviderRequestId
} from '../../../shared/detected-worktree-provider-contract'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../shared/ssh-types'
import {
createDirectSshWorktreeRefreshScheduler,
DIRECT_SSH_WORKTREE_SCAN_CONCURRENCY,
type DirectSshWorktreeRefreshAttempt,
type DirectSshWorktreeRefreshKey,
type DirectSshWorktreeRefreshReleaseReason
} from './direct-ssh-worktree-refresh-scheduler'
type Deferred<T> = {
promise: Promise<T>
resolve: (value: T) => void
reject: (error: unknown) => void
}
type ControlledAttempt = DirectSshWorktreeRefreshAttempt & {
key: DirectSshWorktreeRefreshKey
deferred: Deferred<HostQualifiedDetectedWorktreeResult>
cancel: ReturnType<typeof vi.fn<DirectSshWorktreeRefreshAttempt['cancel']>>
}
function deferred<T>(): Deferred<T> {
let resolve!: (value: T) => void
let reject!: (error: unknown) => void
const promise = new Promise<T>((resolvePromise, rejectPromise) => {
resolve = resolvePromise
reject = rejectPromise
})
return { promise, resolve, reject }
}
function epoch(value: string): SshProviderEpoch {
return value as SshProviderEpoch
}
function requestId(value: string): ProviderRequestId {
return value as ProviderRequestId
}
function key(
targetId: string,
repoId: string,
overrides: Partial<DirectSshWorktreeRefreshKey> = {}
): DirectSshWorktreeRefreshKey {
return {
targetId,
repoId,
executionHostId: `ssh:${targetId}`,
providerEpoch: epoch(`epoch-${targetId}`),
connectionGeneration: 1,
catalogRevision: 1,
authorityRequirement: 'required',
...overrides
}
}
function terminalResult(
attempt: ControlledAttempt,
status: 'timed-out' | 'canceled' | 'stale' | 'rejected'
): HostQualifiedDetectedWorktreeResult {
return {
status,
providerRequestId: attempt.providerRequestId,
executionHostId: attempt.key.executionHostId
}
}
function completeResult(
attempt: ControlledAttempt,
status: 'complete' | 'non-authoritative' = 'complete'
): HostQualifiedDetectedWorktreeResult {
return {
status,
providerRequestId: attempt.providerRequestId,
repoId: attempt.key.repoId,
authority: {
kind: 'direct-ssh',
executionHostId: attempt.key.executionHostId,
targetId: attempt.key.targetId,
providerEpoch: attempt.key.providerEpoch,
connectionGeneration: attempt.key.connectionGeneration
},
result: {
repoId: attempt.key.repoId,
authoritative: status === 'complete',
source: status === 'complete' ? 'git' : 'metadata-fallback',
worktrees: []
}
}
}
function createHarness(cancelOutcome?: 'cancel-failed', now?: () => number) {
const attempts: ControlledAttempt[] = []
const onUnexpectedError = vi.fn()
const startAttempt = vi.fn((attemptKey: DirectSshWorktreeRefreshKey) => {
const providerDeferred = deferred<HostQualifiedDetectedWorktreeResult>()
const waiterDeferred = deferred<HostQualifiedDetectedWorktreeResult>()
const providerRequestId = requestId(`provider-${attempts.length + 1}`)
const attempt: ControlledAttempt = {
key: attemptKey,
providerRequestId,
result: waiterDeferred.promise,
deferred: providerDeferred,
cancel: vi.fn((_reason: DirectSshWorktreeRefreshReleaseReason) => {
waiterDeferred.resolve({
providerRequestId,
executionHostId: attemptKey.executionHostId,
status: 'canceled'
})
return cancelOutcome
})
}
void providerDeferred.promise.then(waiterDeferred.resolve, waiterDeferred.reject)
attempts.push(attempt)
return attempt
})
const scheduler = createDirectSshWorktreeRefreshScheduler({
startAttempt,
onUnexpectedError,
now
})
return { scheduler, attempts, startAttempt, onUnexpectedError }
}
async function flushAttempt(): Promise<void> {
await Promise.resolve()
await Promise.resolve()
await Promise.resolve()
}
describe('createDirectSshWorktreeRefreshScheduler', () => {
it('admits an idle singleton immediately and cleans it up on completion', async () => {
const { scheduler, attempts, startAttempt } = createHarness()
const lease = scheduler.request(key('target-a', 'repo-a'))
expect(startAttempt).toHaveBeenCalledTimes(1)
expect(scheduler.getSnapshot().locallyUnsettled).toBe(1)
attempts[0].deferred.resolve(completeResult(attempts[0]))
await expect(lease.result).resolves.toMatchObject({ status: 'complete' })
await flushAttempt()
expect(scheduler.getSnapshot()).toMatchObject({
locallyUnsettled: 0,
logicalTasks: 0,
waiters: 0
})
})
it('never admits more than five locally unsettled attempts', async () => {
const { scheduler, attempts } = createHarness()
const leases = Array.from({ length: 6 }, (_, index) =>
scheduler.request(key('target-a', `repo-${index}`))
)
expect(attempts).toHaveLength(DIRECT_SSH_WORKTREE_SCAN_CONCURRENCY)
expect(scheduler.getSnapshot().queued).toBe(1)
attempts[0].deferred.resolve(completeResult(attempts[0]))
await leases[0].result
await flushAttempt()
expect(attempts).toHaveLength(6)
expect(scheduler.getSnapshot().locallyUnsettled).toBe(5)
scheduler.stop()
})
it('round-robins target lanes after at most one earlier lane admission', async () => {
const { scheduler, attempts } = createHarness()
const leases = Array.from({ length: 7 }, (_, index) =>
scheduler.request(key('target-a', `repo-a-${index}`))
)
const targetB = scheduler.request(key('target-b', 'repo-b'))
attempts[0].deferred.resolve(completeResult(attempts[0]))
await leases[0].result
await flushAttempt()
expect(attempts[5].key.targetId).toBe('target-a')
attempts[1].deferred.resolve(completeResult(attempts[1]))
await leases[1].result
await flushAttempt()
expect(attempts[6].key.targetId).toBe('target-b')
scheduler.stop()
await expect(targetB.result).resolves.toMatchObject({ status: 'canceled' })
})
it('joins only exact keys and separates authority, revision, and requirement', () => {
const { scheduler, attempts } = createHarness()
const base = key('target-a', 'repo')
const first = scheduler.request(base)
const joined = scheduler.request({ ...base })
scheduler.request({ ...base, providerEpoch: epoch('epoch-next') })
scheduler.request({ ...base, connectionGeneration: 2 })
scheduler.request({ ...base, catalogRevision: 2 })
scheduler.request({ ...base, authorityRequirement: 'allow-metadata-fallback' })
expect(first.waiterLeaseId).not.toBe(joined.waiterLeaseId)
expect(attempts).toHaveLength(5)
expect(scheduler.getSnapshot()).toMatchObject({ logicalTasks: 5, waiters: 6 })
scheduler.stop()
})
it('keeps the logical barrier pending through one timeout and uses a fresh request ID', async () => {
const { scheduler, attempts } = createHarness()
const lease = scheduler.request(key('target-a', 'repo'))
let settled = false
void lease.result.then(() => {
settled = true
})
attempts[0].deferred.resolve(terminalResult(attempts[0], 'timed-out'))
await flushAttempt()
expect(settled).toBe(false)
expect(attempts).toHaveLength(2)
expect(attempts[1].providerRequestId).not.toBe(attempts[0].providerRequestId)
attempts[1].deferred.resolve(completeResult(attempts[1]))
await expect(lease.result).resolves.toMatchObject({
status: 'complete',
providerRequestId: attempts[1].providerRequestId
})
})
it('requeues a timed-out retry at its target lane tail', async () => {
const { scheduler, attempts } = createHarness()
const retrying = scheduler.request(key('target-a', 'retrying'))
const blockers = Array.from({ length: 4 }, (_, index) =>
scheduler.request(key('target-a', `blocker-${index}`))
)
const targetB = scheduler.request(key('target-b', 'target-b'))
attempts[0].deferred.resolve(terminalResult(attempts[0], 'timed-out'))
await flushAttempt()
expect(attempts[5].key.targetId).toBe('target-b')
attempts[1].deferred.resolve(completeResult(attempts[1]))
await blockers[0].result
await flushAttempt()
expect(attempts[6].key.repoId).toBe('retrying')
attempts[6].deferred.resolve(completeResult(attempts[6]))
await expect(retrying.result).resolves.toMatchObject({ status: 'complete' })
scheduler.stop()
await expect(targetB.result).resolves.toMatchObject({ status: 'canceled' })
})
it('makes a second timeout terminal', async () => {
const { scheduler, attempts } = createHarness()
const lease = scheduler.request(key('target-a', 'repo'))
attempts[0].deferred.resolve(terminalResult(attempts[0], 'timed-out'))
await flushAttempt()
attempts[1].deferred.resolve(terminalResult(attempts[1], 'timed-out'))
await expect(lease.result).resolves.toMatchObject({
status: 'timed-out',
providerRequestId: attempts[1].providerRequestId
})
})
it('invalidates queued and current work without reporting cancellation errors', async () => {
const { scheduler, attempts, onUnexpectedError } = createHarness()
const leases = Array.from({ length: 6 }, (_, index) =>
scheduler.request(key('target-a', `repo-${index}`))
)
scheduler.invalidateTarget('target-a')
await expect(Promise.all(leases.map((lease) => lease.result))).resolves.toEqual(
Array.from({ length: 6 }, () => expect.objectContaining({ status: 'stale' }))
)
expect(attempts.map((attempt) => attempt.cancel.mock.calls.length)).toEqual([1, 1, 1, 1, 1])
expect(scheduler.getSnapshot()).toMatchObject({ logicalTasks: 0, waiters: 0 })
for (const attempt of attempts) {
attempt.deferred.reject(new Error('expected provider cancellation'))
}
await flushAttempt()
expect(onUnexpectedError).not.toHaveBeenCalled()
expect(scheduler.getSnapshot().locallyUnsettled).toBe(0)
})
it('bounds five failed cancellations to two replacements while old calls remain pending', async () => {
const { scheduler, attempts } = createHarness('cancel-failed')
const authority: DirectSshAuthority = {
targetId: 'target-a',
providerEpoch: epoch('epoch-target-a'),
connectionGeneration: 1
}
const initial = Array.from({ length: 5 }, (_, index) =>
scheduler.request(key('target-a', `old-${index}`))
)
scheduler.invalidateAuthority(authority)
await Promise.all(initial.map((lease) => lease.result))
await flushAttempt()
expect(scheduler.getSnapshot()).toMatchObject({
locallyUnsettled: 0,
queued: 0,
logicalTasks: 0
})
const replacements = Array.from({ length: 3 }, (_, index) =>
scheduler.request(key('target-a', `replacement-${index}`))
)
expect(attempts).toHaveLength(7)
expect(scheduler.getSnapshot().locallyUnsettled).toBe(2)
await expect(replacements[2].result).resolves.toMatchObject({
status: 'cancel-budget-exhausted'
})
attempts[5].deferred.resolve(completeResult(attempts[5]))
attempts[6].deferred.resolve(completeResult(attempts[6]))
await expect(
Promise.all(replacements.slice(0, 2).map((lease) => lease.result))
).resolves.toEqual(
Array.from({ length: 2 }, () => expect.objectContaining({ status: 'complete' }))
)
await flushAttempt()
expect(scheduler.getSnapshot().locallyUnsettled).toBe(0)
scheduler.disposeProvider(authority)
for (const attempt of attempts.slice(0, 5)) {
attempt.deferred.reject(new Error('late canceled provider'))
}
await flushAttempt()
expect([...scheduler.getSnapshot().cancelDebtByAuthority.values()]).toEqual([])
const afterDisposal = scheduler.request(key('target-a', 'after-disposal'))
expect(attempts).toHaveLength(8)
attempts[7].deferred.resolve(completeResult(attempts[7]))
await expect(afterDisposal.result).resolves.toMatchObject({ status: 'complete' })
})
it('settles one released waiter without canceling a joined provider', async () => {
const { scheduler, attempts } = createHarness()
const refreshKey = key('target-a', 'repo')
const released = scheduler.request(refreshKey)
const retained = scheduler.request(refreshKey)
released.release('superseded')
await expect(released.result).resolves.toMatchObject({ status: 'canceled' })
expect(attempts[0].cancel).not.toHaveBeenCalled()
expect(scheduler.getSnapshot().waiters).toBe(1)
attempts[0].deferred.resolve(completeResult(attempts[0]))
await expect(retained.result).resolves.toMatchObject({ status: 'complete' })
released.release('stopped')
await flushAttempt()
expect(scheduler.getSnapshot()).toMatchObject({ logicalTasks: 0, waiters: 0 })
})
it('does not charge cancel debt when a shared provider invocation stays active', async () => {
const { scheduler, attempts } = createHarness()
const lease = scheduler.request(key('target-a', 'repo'))
attempts[0].cancel.mockReturnValue(false)
lease.release('superseded')
await expect(lease.result).resolves.toMatchObject({ status: 'canceled' })
expect(attempts[0].cancel).toHaveBeenCalledOnce()
expect([...scheduler.getSnapshot().cancelDebtByAuthority.values()]).toEqual([])
attempts[0].deferred.resolve(completeResult(attempts[0]))
await flushAttempt()
})
it('stops current and queued waiters and rejects later requests without starting', async () => {
const { scheduler, attempts, startAttempt } = createHarness()
const current = scheduler.request(key('target-a', 'current'))
scheduler.stop()
await expect(current.result).resolves.toMatchObject({ status: 'canceled' })
expect(attempts[0].cancel).toHaveBeenCalledOnce()
const afterStop = scheduler.request(key('target-a', 'after-stop'))
await expect(afterStop.result).resolves.toMatchObject({ status: 'canceled' })
expect(startAttempt).toHaveBeenCalledOnce()
expect(scheduler.getSnapshot()).toMatchObject({ logicalTasks: 0, waiters: 0 })
attempts[0].deferred.reject(new Error('stopped'))
await flushAttempt()
})
it('surfaces non-authoritative and rejected terminal outcomes', async () => {
const { scheduler, attempts } = createHarness()
const metadata = scheduler.request(key('target-a', 'metadata'))
const rejected = scheduler.request(key('target-a', 'rejected'))
attempts[0].deferred.resolve(completeResult(attempts[0], 'non-authoritative'))
attempts[1].deferred.resolve(terminalResult(attempts[1], 'rejected'))
await expect(metadata.result).resolves.toMatchObject({ status: 'non-authoritative' })
await expect(rejected.result).resolves.toMatchObject({ status: 'rejected' })
})
it('reports queue wait, provider execution, retry, joins, and scoped peak metrics', async () => {
let now = 0
const { scheduler, attempts } = createHarness(undefined, () => now)
const leases = Array.from({ length: 6 }, (_, index) =>
scheduler.request(key('target-a', `repo-${index}`))
)
scheduler.request(key('target-a', 'repo-5'))
now = 25
attempts[0].deferred.resolve(completeResult(attempts[0]))
await leases[0].result
await flushAttempt()
expect(attempts).toHaveLength(6)
now = 100
attempts[5].deferred.resolve(terminalResult(attempts[5], 'timed-out'))
await flushAttempt()
now = 140
attempts[6].deferred.resolve(completeResult(attempts[6]))
await expect(leases[5].result).resolves.toMatchObject({
status: 'complete',
metrics: {
queueWaitDurationsMs: [25, 0],
providerExecutionDurationsMs: [75, 40],
timeoutRetryCount: 1,
cancelDebtCount: 1,
overlappingJoinCount: 1,
peakLocallyUnsettled: 5
}
})
scheduler.stop()
})
})
src/renderer/src/hooks/direct-ssh-worktree-refresh-scheduler.ts
+316
View File
@@ -0,0 +1,316 @@
import type {
HostQualifiedDetectedWorktreeResult,
ProviderRequestId
} from '../../../shared/detected-worktree-provider-contract'
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import type {
DirectSshWorktreeRefreshAttempt,
DirectSshWorktreeRefreshKey,
DirectSshWorktreeRefreshLease,
DirectSshWorktreeRefreshLogicalTask as LogicalTask,
DirectSshWorktreeRefreshOutcome,
DirectSshWorktreeRefreshReleaseReason,
DirectSshWorktreeRefreshScheduler,
DirectSshWorktreeRefreshSchedulerDeps,
WaiterLeaseId
} from './direct-ssh-worktree-refresh-scheduler-types'
import { cancelDirectSshWorktreeRefreshAttempt } from './direct-ssh-worktree-refresh-scheduler-cancellation'
import {
adjustDirectSshAuthorityUnsettled,
copyDirectSshWorktreeRefreshMetrics,
createDirectSshWorktreeRefreshMetrics,
directSshWorktreeRefreshSchedulerSnapshot,
recordDirectSshCancelDebt,
recordDirectSshProviderExecution,
recordDirectSshQueueAdmission
} from './direct-ssh-worktree-refresh-scheduler-metrics'
import { DirectSshWorktreeRefreshTargetQueue } from './direct-ssh-worktree-refresh-target-queue'
import {
directSshProviderAuthorityKey,
directSshWorktreeRefreshKey
} from './direct-ssh-worktree-refresh-scheduler-types'
export type * from './direct-ssh-worktree-refresh-scheduler-types'
export const DIRECT_SSH_WORKTREE_SCAN_CONCURRENCY = 5
export const DIRECT_SSH_PROVIDER_START_BUDGET = 7
export function createDirectSshWorktreeRefreshScheduler(
deps: DirectSshWorktreeRefreshSchedulerDeps
): DirectSshWorktreeRefreshScheduler {
const now = deps.now ?? Date.now
const tasksByKey = new Map<string, LogicalTask>()
const targetQueue = new DirectSshWorktreeRefreshTargetQueue()
const unsettledByAuthority = new Map<string, number>()
const cancelDebtByAuthority = new Map<string, number>()
let locallyUnsettled = 0
let leaseSequence = 0
let stopped = false
const createLeaseId = (): WaiterLeaseId =>
deps.createWaiterLeaseId?.() ?? (`direct-ssh-waiter-${++leaseSequence}` as WaiterLeaseId)
const metricsFor = (
task: LogicalTask,
locallySettledWaiterCount = task.metrics.locallySettledWaiterCount
): DirectSshWorktreeRefreshOutcome['metrics'] => {
const metrics = copyDirectSshWorktreeRefreshMetrics(task.metrics)
metrics.locallySettledWaiterCount = locallySettledWaiterCount
return metrics
}
const finishTask = (task: LogicalTask, outcome: DirectSshWorktreeRefreshOutcome): void => {
if (task.state === 'terminal') {
return
}
task.state = 'terminal'
task.attempt = null
tasksByKey.delete(task.keyId)
task.metrics.locallySettledWaiterCount += task.waiters.size
const settledOutcome = { ...outcome, metrics: metricsFor(task) }
for (const waiter of task.waiters.values()) {
waiter.resolve(settledOutcome)
}
task.waiters.clear()
}
const addCancelDebt = (task: LogicalTask): void => {
recordDirectSshCancelDebt(
task.metrics,
DIRECT_SSH_PROVIDER_START_BUDGET - DIRECT_SSH_WORKTREE_SCAN_CONCURRENCY
)
cancelDebtByAuthority.set(
task.authorityId,
(cancelDebtByAuthority.get(task.authorityId) ?? 0) + 1
)
}
const cancelAttempt = (
task: LogicalTask,
reason: DirectSshWorktreeRefreshReleaseReason
): void => {
if (!task.attempt || task.attemptCanceled) {
return
}
task.attemptCanceled = true
if (cancelDirectSshWorktreeRefreshAttempt(task.attempt, reason)) {
addCancelDebt(task)
}
}
const canStart = (task: LogicalTask): boolean => {
const unsettled = unsettledByAuthority.get(task.authorityId) ?? 0
const debt = cancelDebtByAuthority.get(task.authorityId) ?? 0
return unsettled + debt + 1 <= DIRECT_SSH_PROVIDER_START_BUDGET
}
const handleAttemptResult = (
task: LogicalTask,
requestId: ProviderRequestId,
result: HostQualifiedDetectedWorktreeResult
): void => {
if (task.state === 'terminal' || task.attempt?.providerRequestId !== requestId) {
return
}
if (result.providerRequestId !== requestId) {
finishTask(task, { status: 'rejected', providerRequestId: requestId })
return
}
if (result.status === 'timed-out') {
addCancelDebt(task)
if (task.attemptCount === 1) {
task.metrics.timeoutRetryCount++
task.attempt = null
task.attemptCanceled = true
targetQueue.enqueue(task, true, now())
return
}
}
const status =
result.status === 'authority-unknown' || result.status === 'ambiguous-owner'
? 'non-authoritative'
: result.status
finishTask(task, {
status,
providerRequestId: result.providerRequestId,
providerResult: result
})
}
function drain(): void {
while (!stopped && locallyUnsettled < DIRECT_SSH_WORKTREE_SCAN_CONCURRENCY) {
const task = targetQueue.takeNext()
if (!task) {
return
}
if (!canStart(task)) {
task.metrics.replacementAdmissionDelayedCount++
finishTask(task, { status: 'cancel-budget-exhausted' })
continue
}
task.state = 'running'
task.attemptCount++
locallyUnsettled++
recordDirectSshQueueAdmission(
task.metrics,
Math.max(0, now() - task.queuedAt),
locallyUnsettled
)
adjustDirectSshAuthorityUnsettled(unsettledByAuthority, task.authorityId, 1)
let attempt: DirectSshWorktreeRefreshAttempt
try {
attempt = deps.startAttempt(task.key)
task.attempt = attempt
task.attemptCanceled = false
task.attemptStartedAt = now()
} catch (error) {
locallyUnsettled--
adjustDirectSshAuthorityUnsettled(unsettledByAuthority, task.authorityId, -1)
deps.onUnexpectedError?.(error)
finishTask(task, { status: 'rejected' })
continue
}
void attempt.result
.then((result) => {
if (task.attemptStartedAt !== null) {
recordDirectSshProviderExecution(
task.metrics,
Math.max(0, now() - task.attemptStartedAt)
)
task.attemptStartedAt = null
}
handleAttemptResult(task, attempt.providerRequestId, result)
})
.catch((error) => {
if (task.attemptStartedAt !== null) {
recordDirectSshProviderExecution(
task.metrics,
Math.max(0, now() - task.attemptStartedAt)
)
task.attemptStartedAt = null
}
if (task.state !== 'terminal' && !task.attemptCanceled) {
deps.onUnexpectedError?.(error)
finishTask(task, {
status: 'rejected',
providerRequestId: attempt.providerRequestId
})
}
})
.finally(() => {
locallyUnsettled--
adjustDirectSshAuthorityUnsettled(unsettledByAuthority, task.authorityId, -1)
drain()
})
}
}
const releaseWaiter = (
task: LogicalTask,
waiterLeaseId: WaiterLeaseId,
reason: DirectSshWorktreeRefreshReleaseReason
): void => {
const waiter = task.waiters.get(waiterLeaseId)
if (!waiter) {
return
}
task.waiters.delete(waiterLeaseId)
task.metrics.locallySettledWaiterCount++
waiter.resolve({ status: 'canceled', metrics: metricsFor(task) })
if (task.waiters.size > 0) {
return
}
cancelAttempt(task, reason)
finishTask(task, { status: 'canceled' })
}
const request = (key: DirectSshWorktreeRefreshKey): DirectSshWorktreeRefreshLease => {
const keyId = directSshWorktreeRefreshKey(key)
let task = tasksByKey.get(keyId)
if (!task) {
const requestedAt = now()
task = {
key,
keyId,
authorityId: directSshProviderAuthorityKey(key),
state: 'queued',
attemptCount: 0,
attempt: null,
attemptCanceled: false,
attemptStartedAt: null,
queuedAt: requestedAt,
metrics: createDirectSshWorktreeRefreshMetrics(),
waiters: new Map()
}
tasksByKey.set(keyId, task)
targetQueue.enqueue(task, false, now())
} else {
task.metrics.overlappingJoinCount++
}
const waiterLeaseId = createLeaseId()
let resolve!: (outcome: DirectSshWorktreeRefreshOutcome) => void
const result = new Promise<DirectSshWorktreeRefreshOutcome>((settle) => {
resolve = settle
})
task.waiters.set(waiterLeaseId, { resolve })
if (stopped) {
releaseWaiter(task, waiterLeaseId, 'stopped')
targetQueue.clear()
} else {
drain()
}
return {
waiterLeaseId,
result,
release: (reason) => releaseWaiter(task!, waiterLeaseId, reason)
}
}
const invalidateMatching = (predicate: (task: LogicalTask) => boolean): void => {
for (const task of tasksByKey.values()) {
if (!predicate(task)) {
continue
}
cancelAttempt(task, 'invalidated')
finishTask(task, { status: 'stale', providerRequestId: task.attempt?.providerRequestId })
}
drain()
}
const invalidateAuthority = (authority: DirectSshAuthority): void => {
const id = directSshProviderAuthorityKey(authority)
invalidateMatching((task) => task.authorityId === id)
}
const invalidateTarget = (targetId: string): void => {
invalidateMatching((task) => task.key.targetId === targetId)
}
const disposeProvider = (authority: DirectSshAuthority): void => {
invalidateAuthority(authority)
cancelDebtByAuthority.delete(directSshProviderAuthorityKey(authority))
}
const getSnapshot = () =>
directSshWorktreeRefreshSchedulerSnapshot(tasksByKey, locallyUnsettled, cancelDebtByAuthority)
const stop = (): void => {
if (stopped) {
return
}
stopped = true
for (const task of tasksByKey.values()) {
cancelAttempt(task, 'stopped')
finishTask(task, { status: 'canceled', providerRequestId: task.attempt?.providerRequestId })
}
targetQueue.clear()
}
return {
request,
invalidateAuthority,
invalidateTarget,
disposeProvider,
getSnapshot,
stop
}
}
src/renderer/src/hooks/direct-ssh-worktree-refresh-target-queue.ts
+44
View File
@@ -0,0 +1,44 @@
import type { DirectSshWorktreeRefreshLogicalTask } from './direct-ssh-worktree-refresh-scheduler-types'
export class DirectSshWorktreeRefreshTargetQueue {
private readonly queuedByTarget = new Map<string, DirectSshWorktreeRefreshLogicalTask[]>()
private readonly targetOrder: string[] = []
enqueue(task: DirectSshWorktreeRefreshLogicalTask, retrying: boolean, now: number): void {
task.state = retrying ? 'retrying' : 'queued'
task.queuedAt = now
const lane = this.queuedByTarget.get(task.key.targetId)
if (lane) {
lane.push(task)
return
}
this.queuedByTarget.set(task.key.targetId, [task])
this.targetOrder.push(task.key.targetId)
}
takeNext(): DirectSshWorktreeRefreshLogicalTask | null {
while (this.targetOrder.length > 0) {
const targetId = this.targetOrder.shift()!
const lane = this.queuedByTarget.get(targetId)
const task = lane?.shift()
if (!lane || !task) {
this.queuedByTarget.delete(targetId)
continue
}
if (lane.length > 0) {
this.targetOrder.push(targetId)
} else {
this.queuedByTarget.delete(targetId)
}
if (task.state !== 'terminal' && task.waiters.size > 0) {
return task
}
}
return null
}
clear(): void {
this.queuedByTarget.clear()
this.targetOrder.length = 0
}
}
src/renderer/src/hooks/remote-workspace-session-merge.ts
+133
View File
@@ -0,0 +1,133 @@
import type { TerminalTab, WorkspaceSessionState } from '../../../shared/types'
import { worktreeWorkspaceKey } from '../../../shared/workspace-scope'
import { splitWorktreeId } from '../../../shared/worktree-id'
import type { AppState } from '../store/types'
function preserveNewerLocalTerminalFields(remote: TerminalTab, local: TerminalTab): TerminalTab {
const preserved = {
...remote,
generation: local.generation,
ptyId: local.ptyId
}
return local.pendingActivationSpawn
? { ...preserved, pendingActivationSpawn: local.pendingActivationSpawn }
: preserved
}
export function mergeDirectSshRemoteWorkspaceSession(
current: WorkspaceSessionState,
remote: WorkspaceSessionState,
replaceWorktreeIds: ReadonlySet<string>,
liveTabsByWorktree: AppState['tabsByWorktree'],
preserveLocalTerminalTabIds: ReadonlySet<string>
): WorkspaceSessionState {
const currentTabsById = new Map(
[...replaceWorktreeIds]
.flatMap((worktreeId) => liveTabsByWorktree[worktreeId] ?? [])
.map((tab) => [tab.id, tab])
)
const locallyPreservedTabIds = new Set<string>()
const tabsByWorktree = Object.fromEntries(
Object.entries(remote.tabsByWorktree).map(([worktreeId, tabs]) => [
worktreeId,
tabs.map((tab) => {
const local = currentTabsById.get(tab.id)
if (
!local ||
((local.generation ?? 0) <= (tab.generation ?? 0) &&
!local.pendingActivationSpawn &&
!preserveLocalTerminalTabIds.has(tab.id))
) {
return tab
}
locallyPreservedTabIds.add(tab.id)
return preserveNewerLocalTerminalFields(tab, local)
})
])
)
const remoteTabIds = new Set(
Object.values(tabsByWorktree).flatMap((tabs) => tabs.map((tab) => tab.id))
)
const replacedTabIds = new Set([
...remoteTabIds,
...Object.entries(current.tabsByWorktree)
.filter(([worktreeId]) => replaceWorktreeIds.has(worktreeId))
.flatMap(([, tabs]) => tabs.map((tab) => tab.id))
])
const omitTargetWorktrees = <T>(record: Record<string, T> | undefined): Record<string, T> =>
Object.fromEntries(
Object.entries(record ?? {}).filter(([worktreeId]) => !replaceWorktreeIds.has(worktreeId))
)
const terminalLayoutsByTabId = {
...Object.fromEntries(
Object.entries(current.terminalLayoutsByTabId).filter(
([tabId]) => !replacedTabIds.has(tabId) || locallyPreservedTabIds.has(tabId)
)
),
...Object.fromEntries(
Object.entries(remote.terminalLayoutsByTabId).filter(
([tabId]) => !locallyPreservedTabIds.has(tabId)
)
)
}
const activeOutsideTarget =
current.activeWorktreeId != null && !replaceWorktreeIds.has(current.activeWorktreeId)
return {
...current,
activeRepoId: activeOutsideTarget ? current.activeRepoId : remote.activeRepoId,
activeWorktreeId: activeOutsideTarget ? current.activeWorktreeId : remote.activeWorktreeId,
activeWorkspaceKey: activeOutsideTarget
? current.activeWorkspaceKey
: remote.activeWorktreeId
? worktreeWorkspaceKey(remote.activeWorktreeId)
: null,
activeTabId: activeOutsideTarget ? current.activeTabId : remote.activeTabId,
tabsByWorktree: {
...omitTargetWorktrees(current.tabsByWorktree),
...tabsByWorktree
},
terminalLayoutsByTabId,
activeWorktreeIdsOnShutdown: [
...(current.activeWorktreeIdsOnShutdown ?? []).filter((id) => !replaceWorktreeIds.has(id)),
...(remote.activeWorktreeIdsOnShutdown ?? [])
],
activeTabIdByWorktree: {
...omitTargetWorktrees(current.activeTabIdByWorktree),
...remote.activeTabIdByWorktree
},
remoteSessionIdsByTabId: {
...Object.fromEntries(
Object.entries(current.remoteSessionIdsByTabId ?? {}).filter(
([tabId]) => !replacedTabIds.has(tabId) || locallyPreservedTabIds.has(tabId)
)
),
...Object.fromEntries(
Object.entries(remote.remoteSessionIdsByTabId ?? {}).filter(
([tabId]) => !locallyPreservedTabIds.has(tabId)
)
)
},
lastVisitedAtByWorktreeId: {
...omitTargetWorktrees(current.lastVisitedAtByWorktreeId),
...remote.lastVisitedAtByWorktreeId
},
defaultTerminalTabsAppliedByWorktreeId: {
...omitTargetWorktrees(current.defaultTerminalTabsAppliedByWorktreeId),
...remote.defaultTerminalTabsAppliedByWorktreeId
}
}
}
export function uniqueWorktreeIdByPath(
worktreeIds: ReadonlySet<string>
): (worktreePath: string) => string | null {
const byPath = new Map<string, string | null>()
for (const worktreeId of worktreeIds) {
const path = splitWorktreeId(worktreeId)?.worktreePath
if (!path) {
continue
}
byPath.set(path, byPath.has(path) ? null : worktreeId)
}
return (worktreePath) => byPath.get(worktreePath) ?? null
}
src/renderer/src/hooks/remote-workspace-snapshot-apply.ts
+171
View File
@@ -0,0 +1,171 @@
import type { StoreApi } from 'zustand'
import type { RemoteWorkspaceSnapshot } from '../../../shared/remote-workspace-types'
import { importRemoteWorkspaceSession } from '../../../shared/remote-workspace-session-projection'
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import { translate } from '@/i18n/i18n'
import { buildWorkspaceSessionPayload } from '../lib/workspace-session'
import { resolveDirectSshTargetScope } from '../lib/direct-ssh-target-scope'
import type { AppState } from '../store/types'
import {
admitDirectSshSnapshotApplyToken,
type DirectSshPreparationToken,
type DirectSshSnapshotApplyToken
} from './direct-ssh-reconnect-coordinator'
import { directSshAuthoritiesEqual } from './direct-ssh-reconnect-tokens'
import {
mergeDirectSshRemoteWorkspaceSession,
uniqueWorktreeIdByPath
} from './remote-workspace-session-merge'
const REMOTE_WORKSPACE_SNAPSHOT_WRITE_SUPPRESS_MS = 1_000
const SNAPSHOT_TERMINAL_RECONNECT_TIMEOUT_MS = 30_000
let snapshotApplyDepth = 0
let snapshotWriteSuppressUntil = 0
export function isDirectSshRemoteWorkspaceApplyInProgress(): boolean {
return snapshotApplyDepth > 0 || Date.now() < snapshotWriteSuppressUntil
}
type RemoteWorkspaceSnapshotApplyInput = {
store: Pick<StoreApi<AppState>, 'getState'>
snapshot: RemoteWorkspaceSnapshot
token: DirectSshSnapshotApplyToken
arrival: number
isArrivalCurrent: (targetId: string, arrival: number) => boolean
isPreparationTokenCurrent: (token: DirectSshPreparationToken) => boolean
waitForWorkspaceSessionReady: () => Promise<boolean>
finalizeHydratedTerminals: (authority: DirectSshAuthority) => number
}
function exactTargetWorktreeIds(state: AppState, authority: DirectSshAuthority): Set<string> {
return resolveDirectSshTargetScope({
targetId: authority.targetId,
catalogRevision: 0,
repos: state.repos,
worktreesByRepo: state.worktreesByRepo,
detectedWorktreesByRepo: state.detectedWorktreesByRepo,
folderWorkspaces: state.folderWorkspaces,
projectGroups: state.projectGroups,
restoredRuntimeHostIdByWorkspaceSessionKey: state.restoredRuntimeHostIdByWorkspaceSessionKey
}).gitWorktreeIds
}
function currentRecoveryTabIds(
state: AppState,
authority: DirectSshAuthority,
worktreeIds: ReadonlySet<string>
): Set<string> {
const targetTabIds = new Set(
[...worktreeIds].flatMap((worktreeId) =>
(state.tabsByWorktree[worktreeId] ?? []).map((tab) => tab.id)
)
)
return new Set(
[
...Object.entries(state.directSshPaneRetryByTabId),
...Object.entries(state.directSshLivePtyBindingByTabId)
]
.filter(
([tabId, entry]) =>
targetTabIds.has(tabId) && directSshAuthoritiesEqual(entry.authority, authority)
)
.map(([tabId]) => tabId)
)
}
export async function applyDirectSshRemoteWorkspaceSnapshot({
store,
snapshot,
token,
arrival,
isArrivalCurrent,
isPreparationTokenCurrent,
waitForWorkspaceSessionReady,
finalizeHydratedTerminals
}: RemoteWorkspaceSnapshotApplyInput): Promise<void> {
const { authority } = token
if (!isArrivalCurrent(authority.targetId, arrival)) {
return
}
if (
!isPreparationTokenCurrent(token) ||
!admitDirectSshSnapshotApplyToken(token, authority, snapshot.revision)
) {
return
}
if (!(await waitForWorkspaceSessionReady())) {
if (isArrivalCurrent(authority.targetId, arrival) && isPreparationTokenCurrent(token)) {
store.getState().setRemoteWorkspaceSyncStatus(authority.targetId, {
phase: 'error',
direction: 'pull',
message: translate(
'auto.hooks.useIpcEvents.88214a785b',
'Workspace sync waited for local session hydration and timed out'
)
})
}
return
}
const state = store.getState()
const worktreeIds = exactTargetWorktreeIds(state, authority)
const remoteSession = importRemoteWorkspaceSession(snapshot.session, {
resolveWorktreeId: uniqueWorktreeIdByPath(worktreeIds)
})
const merged = mergeDirectSshRemoteWorkspaceSession(
buildWorkspaceSessionPayload(state),
remoteSession,
worktreeIds,
state.tabsByWorktree,
currentRecoveryTabIds(state, authority, worktreeIds)
)
if (!isArrivalCurrent(authority.targetId, arrival) || !isPreparationTokenCurrent(token)) {
return
}
snapshotApplyDepth += 1
try {
const currentStore = store.getState()
const replaceWorkspaceKeys = [...worktreeIds]
currentStore.hydrateWorkspaceSession(merged, {
directSshAuthority: authority,
replaceWorkspaceKeys
})
currentStore.hydrateTabsSession(merged, { replaceWorkspaceKeys })
// Why: direct SSH snapshots project terminal state only; global editor/browser hydration would reset unrelated hosts.
currentStore.markRemoteWorkspaceHydrated(authority.targetId)
currentStore.setRemoteWorkspaceSyncStatus(authority.targetId, {
phase: 'synced',
direction: 'pull',
revision: snapshot.revision,
updatedAt: snapshot.updatedAt,
lastSyncedAt: Date.now(),
message: translate('auto.hooks.useIpcEvents.4f78ba5885', 'Workspace synced')
})
const reconnectAbort = new AbortController()
let reconnectTimer: ReturnType<typeof setTimeout> | null = null
await Promise.race([
Promise.resolve()
.then(() =>
store.getState().reconnectPersistedTerminals(reconnectAbort.signal, {
directSshAuthority: authority,
workspaceKeys: replaceWorkspaceKeys
})
)
.catch(() => {}),
new Promise<void>((resolve) => {
reconnectTimer = setTimeout(() => {
reconnectAbort.abort()
resolve()
}, SNAPSHOT_TERMINAL_RECONNECT_TIMEOUT_MS)
})
])
if (reconnectTimer) {
clearTimeout(reconnectTimer)
}
if (isArrivalCurrent(authority.targetId, arrival) && isPreparationTokenCurrent(token)) {
finalizeHydratedTerminals(authority)
}
} finally {
snapshotWriteSuppressUntil = Date.now() + REMOTE_WORKSPACE_SNAPSHOT_WRITE_SUPPRESS_MS
snapshotApplyDepth -= 1
}
}
src/renderer/src/hooks/remote-workspace-target-sync.test.ts
+557
View File
@@ -0,0 +1,557 @@
import { describe, expect, it, vi } from 'vitest'
import type {
RemoteWorkspacePatchResult,
RemoteWorkspaceSnapshot
} from '../../../shared/remote-workspace-types'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../shared/ssh-types'
import { i18n } from '@/i18n/i18n'
import { PSEUDO_LOCALIZATION_LOCALE } from '@/i18n/pseudo-localization'
import type { AppState } from '../store/types'
import type {
DirectSshPreparationInput,
DirectSshPreparationToken
} from './direct-ssh-reconnect-coordinator'
import { createRemoteWorkspaceTargetSync } from './remote-workspace-target-sync'
type Deferred<T> = {
promise: Promise<T>
resolve: (value: T) => void
}
function deferred<T>(): Deferred<T> {
let resolve!: (value: T) => void
const promise = new Promise<T>((settle) => {
resolve = settle
})
return { promise, resolve }
}
const owner: DirectSshAuthority = {
targetId: 'target-a',
providerEpoch: 'epoch-a' as SshProviderEpoch,
connectionGeneration: 1
}
function token(snapshotRevision: number | null = null): DirectSshPreparationToken {
return {
authority: owner,
catalogRevision: 1,
repoFingerprint: JSON.stringify([['ssh:target-a', 'repo-a']]),
authorityRequirement: 'required',
snapshotRevision,
outcome: 'complete'
}
}
function snapshot(
revision: number,
tabsByWorktreePath: RemoteWorkspaceSnapshot['session']['tabsByWorktreePath'] = {}
): RemoteWorkspaceSnapshot {
return {
namespace: 'workspace',
revision,
updatedAt: revision,
schemaVersion: 1,
session: {
activeWorktreePath: null,
activeTabId: null,
tabsByWorktreePath,
terminalLayoutsByTabId: {}
}
}
}
function repo(id = 'repo-a') {
return {
id,
path: `/remote/${id}`,
projectGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
}
}
function worktree(id = 'repo-a::/remote/work') {
return {
id,
repoId: id.slice(0, id.indexOf('::')),
hostId: 'ssh:target-a'
}
}
function appState(overrides: Record<string, unknown> = {}): AppState {
return {
workspaceSessionReady: true,
repos: [repo()],
worktreesByRepo: { 'repo-a': [worktree()] },
detectedWorktreesByRepo: {},
folderWorkspaces: [],
projectGroups: [],
restoredRuntimeHostIdByWorkspaceSessionKey: {},
activeRepoId: null,
activeWorkspaceKey: null,
activeWorktreeId: null,
activeTabId: null,
tabsByWorktree: {},
ptyIdsByTabId: {},
lastKnownRelayPtyIdByTabId: {},
directSshPaneRetryByTabId: {},
directSshLivePtyBindingByTabId: {},
terminalLayoutsByTabId: {},
activeTabIdByWorktree: {},
openFiles: [],
editorDrafts: {},
markdownFrontmatterVisible: {},
activeFileIdByWorktree: {},
activeTabTypeByWorktree: {},
browserTabsByWorktree: {},
browserPagesByWorkspace: {},
activeBrowserTabIdByWorktree: {},
browserUrlHistory: [],
unifiedTabsByWorktree: {},
groupsByWorktree: {},
layoutByWorktree: {},
activeGroupIdByWorktree: {},
sshConnectionStates: new Map(),
lastVisitedAtByWorktreeId: {},
defaultTerminalTabsAppliedByWorktreeId: {},
hydrateWorkspaceSession: vi.fn(),
hydrateTabsSession: vi.fn(),
hydrateEditorSession: vi.fn(),
hydrateBrowserSession: vi.fn(),
markRemoteWorkspaceHydrated: vi.fn(),
setRemoteWorkspaceSyncStatus: vi.fn(),
reconnectPersistedTerminals: vi.fn(async () => {}),
...overrides
} as unknown as AppState
}
function createHarness(
state: AppState,
get: (args: { targetId: string }) => Promise<RemoteWorkspaceSnapshot | null>,
patchResult: RemoteWorkspacePatchResult = { ok: true, snapshot: snapshot(1) }
) {
const setForConnectedTargets = vi.fn(async () => [
{
targetId: owner.targetId,
result: patchResult
}
])
let current = true
const capturePreparationInput = vi.fn(
async (
authority: DirectSshAuthority,
reason: 'workspace-snapshot',
snapshotRevision: number
): Promise<DirectSshPreparationInput> => ({
...authority,
catalogRevision: 1,
repoRefs: [{ repoId: 'repo-a', executionHostId: 'ssh:target-a' }],
authorityRequirement: 'required',
reason,
snapshotRevision
})
)
const prepareOnly = vi.fn(async (input: DirectSshPreparationInput) => ({
status: 'complete' as const,
token: token(input.snapshotRevision ?? null),
repoOutcomes: {
complete: 1,
'non-authoritative': 0,
'timed-out': 0,
'cancel-budget-exhausted': 0,
canceled: 0,
stale: 0,
rejected: 0
},
lineageOutcome: 'complete' as const
}))
const finalizeHydratedTerminals = vi.fn(() => 1)
const sync = createRemoteWorkspaceTargetSync({
store: { getState: () => state },
remoteWorkspace: { get, setForConnectedTargets },
getCurrentAuthority: () => (current ? owner : null),
isPreparationTokenCurrent: () => current,
capturePreparationInput,
prepareOnly,
finalizeHydratedTerminals
})
return {
sync,
setForConnectedTargets,
capturePreparationInput,
prepareOnly,
finalizeHydratedTerminals,
makeStale: () => {
current = false
}
}
}
async function flush(): Promise<void> {
await Promise.resolve()
await Promise.resolve()
await Promise.resolve()
}
describe('createRemoteWorkspaceTargetSync', () => {
it('captures local tabs before get when deciding a revision-zero upload', async () => {
const state = appState({
tabsByWorktree: {
'repo-a::/remote/work': [{ id: 'tab-a', worktreeId: 'repo-a::/remote/work', ptyId: null }]
}
})
const pendingGet = deferred<RemoteWorkspaceSnapshot | null>()
const harness = createHarness(state, () => pendingGet.promise)
const pending = harness.sync.syncAfterConnect(token())
await flush()
state.tabsByWorktree = {}
pendingGet.resolve(snapshot(0))
await pending
expect(harness.setForConnectedTargets).toHaveBeenCalledOnce()
expect(harness.setForConnectedTargets).toHaveBeenCalledWith(
expect.objectContaining({ hydratedTargetIds: ['target-a'] })
)
})
it.each([
['stale-revision', 'Workspace changed on another device'],
['unavailable', 'Remote workspace sync unavailable']
] as const)('localizes the %s upload fallback', async (reason, message) => {
const previousLanguage = i18n.language
await i18n.changeLanguage(PSEUDO_LOCALIZATION_LOCALE)
try {
const state = appState({
tabsByWorktree: {
'repo-a::/remote/work': [{ id: 'tab-a', worktreeId: 'repo-a::/remote/work', ptyId: null }]
}
})
const harness = createHarness(state, async () => snapshot(0), {
ok: false,
reason
})
await harness.sync.syncAfterConnect(token())
expect(state.setRemoteWorkspaceSyncStatus).toHaveBeenLastCalledWith(
'target-a',
expect.objectContaining({ message: `[${message}]` })
)
} finally {
await i18n.changeLanguage(previousLanguage)
}
})
it('publishes nothing from a snapshot response after its authority turns stale', async () => {
const state = appState()
const pendingGet = deferred<RemoteWorkspaceSnapshot | null>()
const harness = createHarness(state, () => pendingGet.promise)
const pending = harness.sync.syncAfterConnect(token())
await flush()
harness.makeStale()
pendingGet.resolve(snapshot(2))
await pending
expect(state.hydrateTabsSession).not.toHaveBeenCalled()
expect(state.markRemoteWorkspaceHydrated).not.toHaveBeenCalled()
expect(state.setRemoteWorkspaceSyncStatus).toHaveBeenCalledTimes(1)
expect(state.setRemoteWorkspaceSyncStatus).toHaveBeenCalledWith('target-a', {
phase: 'pulling',
direction: 'pull'
})
})
it('prepares an unsolicited snapshot once and preserves newer local terminal fields', async () => {
const calls: string[] = []
const state = appState({
tabsByWorktree: {
'repo-a::/remote/work': [
{
id: 'stable-tab',
worktreeId: 'repo-a::/remote/work',
ptyId: 'local-pty',
generation: 7,
pendingActivationSpawn: { requestedAt: 10 }
}
]
},
hydrateTabsSession: vi.fn((session) => {
calls.push('hydrate')
expect(session.tabsByWorktree['repo-a::/remote/work'][0]).toMatchObject({
id: 'stable-tab',
ptyId: 'local-pty',
generation: 7,
pendingActivationSpawn: { requestedAt: 10 }
})
}),
reconnectPersistedTerminals: vi.fn(async () => {
calls.push('reconnect')
})
})
const harness = createHarness(state, async () => null)
harness.finalizeHydratedTerminals.mockImplementation(() => {
calls.push('finalize')
return 1
})
const incoming = snapshot(3, {
'/remote/work': [
{
id: 'stable-tab',
worktreePath: '/remote/work',
ptyId: 'remote-pty',
generation: 99
} as RemoteWorkspaceSnapshot['session']['tabsByWorktreePath'][string][number]
]
})
await harness.sync.applyUnsolicitedSnapshot('target-a', incoming)
expect(harness.capturePreparationInput).toHaveBeenCalledOnce()
expect(harness.prepareOnly).toHaveBeenCalledOnce()
expect(calls).toEqual(['hydrate', 'reconnect', 'finalize'])
expect(state.hydrateWorkspaceSession).toHaveBeenCalledWith(
expect.any(Object),
expect.objectContaining({
directSshAuthority: owner,
replaceWorkspaceKeys: ['repo-a::/remote/work']
})
)
expect(state.hydrateTabsSession).toHaveBeenCalledWith(expect.any(Object), {
replaceWorkspaceKeys: ['repo-a::/remote/work']
})
expect(state.hydrateEditorSession).not.toHaveBeenCalled()
expect(state.hydrateBrowserSession).not.toHaveBeenCalled()
})
it('preserves a higher local generation from an older remote snapshot', async () => {
const hydrateTabsSession = vi.fn()
const state = appState({
tabsByWorktree: {
'repo-a::/remote/work': [
{
id: 'stable-tab',
worktreeId: 'repo-a::/remote/work',
ptyId: 'local-pty',
generation: 7
}
]
},
hydrateTabsSession
})
const harness = createHarness(state, async () => null)
const incoming = snapshot(5, {
'/remote/work': [
{
id: 'stable-tab',
worktreePath: '/remote/work',
ptyId: 'old-remote-pty',
generation: 1
} as RemoteWorkspaceSnapshot['session']['tabsByWorktreePath'][string][number]
]
})
await harness.sync.applyUnsolicitedSnapshot('target-a', incoming)
expect(
hydrateTabsSession.mock.calls[0][0].tabsByWorktree['repo-a::/remote/work'][0]
).toMatchObject({ generation: 7, ptyId: 'local-pty' })
})
it('admits a genuinely newer remote generation without local recovery state', async () => {
const hydrateTabsSession = vi.fn()
const state = appState({
tabsByWorktree: {
'repo-a::/remote/work': [
{
id: 'stable-tab',
worktreeId: 'repo-a::/remote/work',
ptyId: 'local-pty',
generation: 1
}
]
},
hydrateTabsSession
})
const harness = createHarness(state, async () => null)
const incoming = snapshot(6, {
'/remote/work': [
{
id: 'stable-tab',
worktreePath: '/remote/work',
ptyId: 'new-remote-pty',
generation: 8
} as RemoteWorkspaceSnapshot['session']['tabsByWorktreePath'][string][number]
]
})
await harness.sync.applyUnsolicitedSnapshot('target-a', incoming)
expect(
hydrateTabsSession.mock.calls[0][0].tabsByWorktree['repo-a::/remote/work'][0]
).toMatchObject({ generation: 8, ptyId: 'new-remote-pty' })
})
it('does not preserve recovery evidence from another authority', async () => {
const hydrateTabsSession = vi.fn()
const state = appState({
tabsByWorktree: {
'repo-a::/remote/work': [
{
id: 'stable-tab',
worktreeId: 'repo-a::/remote/work',
ptyId: 'local-pty',
generation: 1
}
]
},
directSshLivePtyBindingByTabId: {
'stable-tab': {
authority: {
targetId: 'target-b',
providerEpoch: 'epoch-b' as SshProviderEpoch,
connectionGeneration: 2
},
tabGeneration: 1,
ptyId: 'local-pty'
}
},
hydrateTabsSession
})
const harness = createHarness(state, async () => null)
const incoming = snapshot(7, {
'/remote/work': [
{
id: 'stable-tab',
worktreePath: '/remote/work',
ptyId: 'new-remote-pty',
generation: 8
} as RemoteWorkspaceSnapshot['session']['tabsByWorktreePath'][string][number]
]
})
await harness.sync.applyUnsolicitedSnapshot('target-a', incoming)
expect(
hydrateTabsSession.mock.calls[0][0].tabsByWorktree['repo-a::/remote/work'][0]
).toMatchObject({ generation: 8, ptyId: 'new-remote-pty' })
})
it('does not finalize an older snapshot superseded during terminal reattach', async () => {
const firstReattach = deferred<void>()
let reattachCount = 0
const state = appState({
reconnectPersistedTerminals: vi.fn(() => {
reattachCount += 1
return reattachCount === 1 ? firstReattach.promise : Promise.resolve()
})
})
const harness = createHarness(state, async () => null)
const first = harness.sync.applyUnsolicitedSnapshot('target-a', snapshot(7))
await flush()
expect(state.reconnectPersistedTerminals).toHaveBeenCalledOnce()
const second = harness.sync.applyUnsolicitedSnapshot('target-a', snapshot(8))
await second
expect(harness.finalizeHydratedTerminals).toHaveBeenCalledOnce()
firstReattach.resolve()
await first
expect(harness.finalizeHydratedTerminals).toHaveBeenCalledOnce()
})
it('keeps staged snapshot PTYs retryable until pane transport acknowledgment', async () => {
const calls: string[] = []
const recordLiveBindings = vi.fn(() => {
calls.push('record')
return 1
})
const state = appState({
reconnectPersistedTerminals: vi.fn(async () => {
calls.push('reconnect')
}),
recordDirectSshTargetLivePtyBindings: recordLiveBindings
})
const harness = createHarness(state, async () => null)
harness.finalizeHydratedTerminals.mockImplementation(() => {
calls.push('finalize')
return 0
})
await harness.sync.applyUnsolicitedSnapshot('target-a', snapshot(9))
expect(calls).toEqual(['reconnect', 'finalize'])
expect(recordLiveBindings).not.toHaveBeenCalled()
})
it('re-arms after a snapshot terminal reconnect failure', async () => {
const state = appState({
reconnectPersistedTerminals: vi.fn(async () => {
throw new Error('reattach failed')
})
})
const harness = createHarness(state, async () => null)
await harness.sync.applyUnsolicitedSnapshot('target-a', snapshot(10))
expect(harness.finalizeHydratedTerminals).toHaveBeenCalledOnce()
})
it('times out snapshot terminal reconnect and fences its late result', async () => {
vi.useFakeTimers()
const pendingReattach = deferred<void>()
let reconnectSignal: AbortSignal | undefined
const state = appState({
reconnectPersistedTerminals: vi.fn((signal?: AbortSignal) => {
reconnectSignal = signal
return pendingReattach.promise
})
})
const harness = createHarness(state, async () => null)
const pending = harness.sync.applyUnsolicitedSnapshot('target-a', snapshot(11))
await vi.advanceTimersByTimeAsync(30_000)
await pending
expect(reconnectSignal?.aborted).toBe(true)
expect(harness.finalizeHydratedTerminals).toHaveBeenCalledOnce()
pendingReattach.resolve()
await flush()
expect(harness.finalizeHydratedTerminals).toHaveBeenCalledOnce()
vi.useRealTimers()
})
it('fails closed on duplicate target paths and keeps folder workspaces out of projection', async () => {
const hydrateTabsSession = vi.fn()
const state = appState({
repos: [repo('repo-a'), repo('repo-b')],
worktreesByRepo: {
'repo-a': [worktree('repo-a::/same')],
'repo-b': [worktree('repo-b::/same')]
},
tabsByWorktree: {
'folder:folder-a': [{ id: 'folder-tab', worktreeId: 'folder:folder-a', ptyId: null }]
},
hydrateTabsSession
})
const harness = createHarness(state, async () => null)
const incoming = snapshot(4, {
'/same': [
{
id: 'ambiguous',
worktreePath: '/same',
ptyId: null
} as RemoteWorkspaceSnapshot['session']['tabsByWorktreePath'][string][number]
]
})
await harness.sync.applyUnsolicitedSnapshot('target-a', incoming)
const merged = hydrateTabsSession.mock.calls[0][0]
expect(merged.tabsByWorktree).toEqual({
'folder:folder-a': [{ id: 'folder-tab', worktreeId: 'folder:folder-a', ptyId: null }]
})
})
})
src/renderer/src/hooks/remote-workspace-target-sync.ts
+256
View File
@@ -0,0 +1,256 @@
import type { StoreApi } from 'zustand'
import type {
RemoteWorkspacePatchResult,
RemoteWorkspaceSnapshot
} from '../../../shared/remote-workspace-types'
import type { WorkspaceSessionState } from '../../../shared/types'
import type { DirectSshAuthority } from '../../../shared/ssh-types'
import { translate } from '@/i18n/i18n'
import { buildWorkspaceSessionPayload } from '../lib/workspace-session'
import type { AppState } from '../store/types'
import type {
DirectSshPreparationInput,
DirectSshPreparationOutcome,
DirectSshPreparationToken
} from './direct-ssh-reconnect-coordinator'
import { buildDirectSshSnapshotApplyToken } from './direct-ssh-reconnect-coordinator'
import { resolveDirectSshTargetScope } from '../lib/direct-ssh-target-scope'
import { applyDirectSshRemoteWorkspaceSnapshot } from './remote-workspace-snapshot-apply'
export { isDirectSshRemoteWorkspaceApplyInProgress } from './remote-workspace-snapshot-apply'
const WORKSPACE_HYDRATION_TIMEOUT_MS = 10_000
type RemoteWorkspaceApi = {
get: (args: { targetId: string }) => Promise<RemoteWorkspaceSnapshot | null>
setForConnectedTargets: (args: {
session?: WorkspaceSessionState
hydratedTargetIds?: string[]
}) => Promise<{ targetId: string; result: RemoteWorkspacePatchResult }[]>
}
export type RemoteWorkspaceTargetSyncDeps = {
store: Pick<StoreApi<AppState>, 'getState'>
remoteWorkspace: RemoteWorkspaceApi
getCurrentAuthority: (targetId: string) => DirectSshAuthority | null
isPreparationTokenCurrent: (token: DirectSshPreparationToken) => boolean
capturePreparationInput: (
authority: DirectSshAuthority,
reason: 'workspace-snapshot',
snapshotRevision: number
) => Promise<DirectSshPreparationInput | null>
prepareOnly: (input: DirectSshPreparationInput) => Promise<DirectSshPreparationOutcome>
finalizeHydratedTerminals: (authority: DirectSshAuthority) => number
}
export type RemoteWorkspaceTargetSync = {
syncAfterConnect: (token: DirectSshPreparationToken) => Promise<void>
applyUnsolicitedSnapshot: (targetId: string, snapshot: RemoteWorkspaceSnapshot) => Promise<void>
stop: () => void
}
function exactTargetWorktreeIds(state: AppState, authority: DirectSshAuthority): Set<string> {
return resolveDirectSshTargetScope({
targetId: authority.targetId,
catalogRevision: 0,
repos: state.repos,
worktreesByRepo: state.worktreesByRepo,
detectedWorktreesByRepo: state.detectedWorktreesByRepo,
folderWorkspaces: state.folderWorkspaces,
projectGroups: state.projectGroups,
restoredRuntimeHostIdByWorkspaceSessionKey: state.restoredRuntimeHostIdByWorkspaceSessionKey
}).gitWorktreeIds
}
function applyPatchStatus(
store: AppState,
targetId: string,
result: RemoteWorkspacePatchResult | undefined
): void {
if (!result) {
store.setRemoteWorkspaceSyncStatus(targetId, {
phase: 'offline',
direction: 'push',
lastSyncedAt: Date.now(),
message: translate('auto.hooks.useIpcEvents.2fe88c2e06', 'Remote workspace sync unavailable')
})
} else if (result.ok) {
store.setRemoteWorkspaceSyncStatus(targetId, {
phase: 'synced',
direction: 'push',
revision: result.snapshot.revision,
updatedAt: result.snapshot.updatedAt,
lastSyncedAt: Date.now(),
message: translate('auto.hooks.useIpcEvents.f8aaf2bde3', 'Workspace uploaded')
})
} else {
store.setRemoteWorkspaceSyncStatus(targetId, {
phase: result.reason === 'stale-revision' ? 'conflict' : 'offline',
direction: 'push',
revision: result.snapshot?.revision,
updatedAt: result.snapshot?.updatedAt,
lastSyncedAt: Date.now(),
message:
result.message ??
(result.reason === 'stale-revision'
? translate(
'auto.hooks.useIpcEvents.workspaceChangedOnAnotherDevice',
'Workspace changed on another device'
)
: translate('auto.hooks.useIpcEvents.2fe88c2e06', 'Remote workspace sync unavailable'))
})
}
}
export function createRemoteWorkspaceTargetSync(
deps: RemoteWorkspaceTargetSyncDeps
): RemoteWorkspaceTargetSync {
const arrivalByTarget = new Map<string, number>()
let stopped = false
const beginArrival = (targetId: string): number => {
const arrival = (arrivalByTarget.get(targetId) ?? 0) + 1
arrivalByTarget.set(targetId, arrival)
return arrival
}
const isArrivalCurrent = (targetId: string, arrival: number): boolean =>
!stopped && arrivalByTarget.get(targetId) === arrival
const waitForWorkspaceSessionReady = async (): Promise<boolean> => {
const deadline = Date.now() + WORKSPACE_HYDRATION_TIMEOUT_MS
while (!stopped && Date.now() < deadline) {
if (deps.store.getState().workspaceSessionReady) {
return true
}
await new Promise((resolve) => setTimeout(resolve, 100))
}
return !stopped && deps.store.getState().workspaceSessionReady
}
const syncAfterConnect = async (token: DirectSshPreparationToken): Promise<void> => {
const { authority } = token
const arrival = beginArrival(authority.targetId)
const workspaceReady = await waitForWorkspaceSessionReady()
if (!isArrivalCurrent(authority.targetId, arrival) || !deps.isPreparationTokenCurrent(token)) {
return
}
if (!workspaceReady) {
deps.store.getState().setRemoteWorkspaceSyncStatus(authority.targetId, {
phase: 'error',
direction: 'pull',
message: translate(
'auto.hooks.useIpcEvents.88214a785b',
'Workspace sync waited for local session hydration and timed out'
)
})
return
}
const stateBeforeGet = deps.store.getState()
const worktreeIds = exactTargetWorktreeIds(stateBeforeGet, authority)
const hasLocalTabs = [...worktreeIds].some(
(worktreeId) => (stateBeforeGet.tabsByWorktree[worktreeId] ?? []).length > 0
)
stateBeforeGet.setRemoteWorkspaceSyncStatus(authority.targetId, {
phase: 'pulling',
direction: 'pull'
})
const snapshot = await deps.remoteWorkspace.get({ targetId: authority.targetId })
if (!isArrivalCurrent(authority.targetId, arrival) || !deps.isPreparationTokenCurrent(token)) {
return
}
if (!snapshot) {
deps.store.getState().setRemoteWorkspaceSyncStatus(authority.targetId, {
phase: 'offline',
direction: 'pull',
message: translate(
'auto.hooks.useIpcEvents.2fe88c2e06',
'Remote workspace sync unavailable'
)
})
return
}
if (snapshot.revision > 0) {
const applyToken = buildDirectSshSnapshotApplyToken(token, snapshot.revision)
if (applyToken) {
await applyDirectSshRemoteWorkspaceSnapshot({
store: deps.store,
snapshot,
token: applyToken,
arrival,
isArrivalCurrent,
isPreparationTokenCurrent: deps.isPreparationTokenCurrent,
waitForWorkspaceSessionReady,
finalizeHydratedTerminals: deps.finalizeHydratedTerminals
})
}
return
}
deps.store.getState().markRemoteWorkspaceHydrated(authority.targetId)
if (!hasLocalTabs) {
deps.store.getState().setRemoteWorkspaceSyncStatus(authority.targetId, {
phase: 'idle',
revision: snapshot.revision,
updatedAt: snapshot.updatedAt,
message: translate('auto.hooks.useIpcEvents.2ec42e1c52', 'No remote workspace yet')
})
return
}
if (!deps.isPreparationTokenCurrent(token)) {
return
}
const results = await deps.remoteWorkspace.setForConnectedTargets({
session: buildWorkspaceSessionPayload(deps.store.getState()),
hydratedTargetIds: [authority.targetId]
})
if (!deps.isPreparationTokenCurrent(token)) {
return
}
const result = results.find((entry) => entry.targetId === authority.targetId)?.result
applyPatchStatus(deps.store.getState(), authority.targetId, result)
}
const applyUnsolicitedSnapshot = async (
targetId: string,
snapshot: RemoteWorkspaceSnapshot
): Promise<void> => {
const arrival = beginArrival(targetId)
const authority = deps.getCurrentAuthority(targetId)
if (!authority) {
return
}
const input = await deps.capturePreparationInput(
authority,
'workspace-snapshot',
snapshot.revision
)
if (!input || !isArrivalCurrent(targetId, arrival)) {
return
}
const prepared = await deps.prepareOnly(input)
if (!prepared.token || !isArrivalCurrent(targetId, arrival)) {
return
}
const applyToken = buildDirectSshSnapshotApplyToken(prepared.token, snapshot.revision)
if (applyToken) {
await applyDirectSshRemoteWorkspaceSnapshot({
store: deps.store,
snapshot,
token: applyToken,
arrival,
isArrivalCurrent,
isPreparationTokenCurrent: deps.isPreparationTokenCurrent,
waitForWorkspaceSessionReady,
finalizeHydratedTerminals: deps.finalizeHydratedTerminals
})
}
}
return {
syncAfterConnect,
applyUnsolicitedSnapshot,
stop: () => {
stopped = true
arrivalByTarget.clear()
}
}
}
src/renderer/src/hooks/useIpcEvents.test.ts
+320 -6
View File
@@ -1426,7 +1426,7 @@ describe('useIpcEvents updater integration', () => {
})
it('clears stale remote PTYs when an SSH connection fully disconnects', async () => {
const clearTabPtyId = vi.fn()
const clearDirectSshTargetPtyBindings = vi.fn(() => 1)
const setSshConnectionState = vi.fn()
const setSshTargetsMetadata = vi.fn()
const clearRemovedSshTargetState = vi.fn()
@@ -1478,7 +1478,7 @@ describe('useIpcEvents updater integration', () => {
removeSshCredentialRequest: vi.fn(),
clearRemoteDetectedAgents: vi.fn(),
clearRemovedSshTargetState,
clearTabPtyId,
clearDirectSshTargetPtyBindings,
repos: [{ id: 'repo-1', connectionId: 'conn-1' }],
worktreesByRepo: {
'repo-1': [{ id: 'wt-1', repoId: 'repo-1' }]
@@ -1663,8 +1663,8 @@ describe('useIpcEvents updater integration', () => {
'conn-1',
expect.objectContaining({ status: 'disconnected' })
)
expect(clearTabPtyId).toHaveBeenCalledWith('tab-1')
expect(clearTabPtyId).not.toHaveBeenCalledWith('tab-2')
expect(clearDirectSshTargetPtyBindings).toHaveBeenCalledOnce()
expect(clearDirectSshTargetPtyBindings).toHaveBeenCalledWith('conn-1')
expect(storeState.clearRemoteDetectedAgents).toHaveBeenCalledWith('conn-1')
setSshConnectionState.mockClear()
@@ -4805,6 +4805,8 @@ describe('useIpcEvents agent status snapshot integration', () => {
drop?: (paneKey: string) => void
remoteWorkspace?: Record<string, unknown>
runtime?: Record<string, unknown>
ssh?: Record<string, unknown>
ui?: Record<string, unknown>
}): Record<string, unknown> {
return {
api: {
@@ -4864,7 +4866,8 @@ describe('useIpcEvents agent status snapshot integration', () => {
onFullscreenChanged: () => () => {},
onTerminalZoom: () => () => {},
getZoomLevel: () => 0,
set: vi.fn()
set: vi.fn(),
...args.ui
},
settings: { onChanged: () => () => {} },
updater: {
@@ -4901,7 +4904,8 @@ describe('useIpcEvents agent status snapshot integration', () => {
onCredentialRequest: () => () => {},
onCredentialResolved: () => () => {},
onPortForwardsChanged: () => () => {},
onDetectedPortsChanged: () => () => {}
onDetectedPortsChanged: () => () => {},
...args.ssh
},
agentStatus: {
onSet: args.onSet,
@@ -4947,11 +4951,321 @@ describe('useIpcEvents agent status snapshot integration', () => {
vi.doMock('@/lib/zoom-events', () => ({ dispatchZoomLevelChanged: vi.fn() }))
}
function buildSshAuthorityReconciliationHarness(args: {
partialAuthority: { providerEpoch?: string; connectionGeneration?: number }
latestAuthority: { providerEpoch: string; connectionGeneration: number }
}): {
emitPartialState: () => void
getState: ReturnType<typeof vi.fn>
requestReconnect: ReturnType<typeof vi.fn>
setSshConnectionState: ReturnType<typeof vi.fn>
storedState: () => Record<string, unknown> | undefined
} {
const targetId = 'target-reconciliation'
const baseState = {
targetId,
status: 'connected' as const,
error: null,
reconnectAttempt: 0
}
const partialState = { ...baseState, ...args.partialAuthority }
const latestState = { ...baseState, ...args.latestAuthority }
const sshConnectionStates = new Map<string, Record<string, unknown>>()
let sshStateListener: ((data: { targetId: string; state: unknown }) => void) | undefined
const getState = vi.fn(() => Promise.resolve(latestState))
const requestReconnect = vi.fn(async () => ({ status: 'complete' }))
const setSshConnectionState = vi.fn((nextTargetId: string, state: Record<string, unknown>) => {
sshConnectionStates.set(nextTargetId, state)
})
const storeState = buildStoreState({
sshTargetLabels: new Map([[targetId, 'Reconciliation Target']]),
sshConnectionStates,
setSshConnectionState,
invalidateStaleDirectSshTargetPtyBindings: vi.fn(() => 0),
retryDirectSshTargetPanes: vi.fn(() => 0),
setSshTargetsMetadata: vi.fn(),
setRemovedSshTargetLabels: vi.fn(),
setRemoteWorkspaceSyncStatus: vi.fn(),
clearRemoteDetectedAgents: vi.fn(),
clearDirectSshTargetPtyBindings: vi.fn(),
clearRemovedSshTargetState: vi.fn()
})
const coordinator = {
requestReconnect,
replaceAuthority: vi.fn(),
prepareOnly: vi.fn(),
correctUnboundTerminals: vi.fn(() => 0),
finalizeHydratedTerminals: vi.fn(() => 0),
invalidate: vi.fn(),
stop: vi.fn()
}
stubReactSyncEffect()
stubAuxiliaryModules()
vi.doMock('../store', () => ({
useAppStore: {
subscribe: vi.fn(() => () => {}),
getState: () => storeState
}
}))
vi.doMock('./direct-ssh-reconnect-rollout', () => ({
isDirectSshReconnectCoordinatorRoutingEnabled: () => true
}))
vi.doMock('./direct-ssh-worktree-refresh-scheduler', () => ({
createDirectSshWorktreeRefreshScheduler: () => ({
stop: vi.fn(),
disposeProvider: vi.fn()
})
}))
vi.doMock('./direct-ssh-host-hydration', () => ({
createDirectSshHostHydration: () => ({
capturePreparationInput: vi.fn(),
readHostScopedLineage: vi.fn(),
isPreparationTokenCurrent: vi.fn(() => true),
stop: vi.fn()
})
}))
vi.doMock('./direct-ssh-reconnect-coordinator', () => ({
createDirectSshReconnectCoordinator: () => coordinator
}))
vi.doMock('@/lib/direct-ssh-reconnect-product-telemetry', () => ({
createDirectSshReconnectProductTelemetryAdapter: vi.fn()
}))
vi.stubGlobal(
'window',
buildWindowApi({
onSet: () => () => {},
ssh: {
getState,
onStateChanged: (listener: (data: { targetId: string; state: unknown }) => void) => {
sshStateListener = listener
return () => {}
}
}
})
)
return {
emitPartialState: () => {
if (!sshStateListener) {
throw new Error('Expected SSH state listener')
}
sshStateListener({ targetId, state: partialState })
},
getState,
requestReconnect,
setSshConnectionState,
storedState: () => sshConnectionStates.get(targetId)
}
}
beforeEach(() => {
vi.resetModules()
vi.unstubAllGlobals()
})
it.each([
{ partialAuthority: { providerEpoch: 'epoch-current' } },
{ partialAuthority: { connectionGeneration: 7 } }
])(
'fills a same-watermark partial SSH authority and routes it once',
async ({ partialAuthority }) => {
const harness = buildSshAuthorityReconciliationHarness({
partialAuthority,
latestAuthority: {
providerEpoch: 'epoch-current',
connectionGeneration: 7
}
})
const { useIpcEvents } = await import('./useIpcEvents')
useIpcEvents()
harness.emitPartialState()
await vi.waitFor(() => {
expect(harness.requestReconnect).toHaveBeenCalledOnce()
})
expect(harness.getState).toHaveBeenCalledOnce()
expect(harness.setSshConnectionState).toHaveBeenCalledTimes(2)
expect(harness.storedState()).toEqual({
targetId: 'target-reconciliation',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'epoch-current',
connectionGeneration: 7
})
}
)
it.each([
{ partialAuthority: { providerEpoch: 'epoch-conflict' } },
{ partialAuthority: { connectionGeneration: 6 } }
])(
'rejects a reconciliation reply that conflicts with present authority',
async ({ partialAuthority }) => {
const harness = buildSshAuthorityReconciliationHarness({
partialAuthority,
latestAuthority: {
providerEpoch: 'epoch-current',
connectionGeneration: 7
}
})
const { useIpcEvents } = await import('./useIpcEvents')
useIpcEvents()
harness.emitPartialState()
await vi.waitFor(() => {
expect(harness.getState).toHaveBeenCalledOnce()
})
await Promise.resolve()
expect(harness.requestReconnect).not.toHaveBeenCalled()
expect(harness.setSshConnectionState).toHaveBeenCalledOnce()
expect(harness.storedState()).toEqual(
expect.objectContaining({
targetId: 'target-reconciliation',
...partialAuthority
})
)
}
)
it.each([
{ enabled: true, expectedRoute: ['request'] },
{
enabled: false,
expectedRoute: ['replace', 'invalidate', 'retry', 'capture', 'prepare']
}
])(
'routes a changed direct SSH authority through the enabled=$enabled path',
async ({ enabled, expectedRoute }) => {
const order: string[] = []
let sshStateListener: ((data: { targetId: string; state: unknown }) => void) | undefined
const oldState = {
targetId: 'target-a',
status: 'connected' as const,
error: null,
reconnectAttempt: 0,
providerEpoch: 'epoch-old',
connectionGeneration: 1
}
const nextState = {
...oldState,
providerEpoch: 'epoch-new',
connectionGeneration: 2
}
const storeState = buildStoreState({
sshTargetLabels: new Map([['target-a', 'Target A']]),
sshConnectionStates: new Map([['target-a', oldState]]),
setSshConnectionState: (targetId: string, state: unknown) => {
;(storeState.sshConnectionStates as Map<string, unknown>).set(targetId, state)
},
invalidateStaleDirectSshTargetPtyBindings: () => {
order.push('invalidate')
return 1
},
retryDirectSshTargetPanes: () => {
order.push('retry')
return 1
},
setSshTargetsMetadata: vi.fn(),
setRemovedSshTargetLabels: vi.fn(),
setRemoteWorkspaceSyncStatus: vi.fn(),
clearRemoteDetectedAgents: vi.fn(),
clearDirectSshTargetPtyBindings: vi.fn(),
clearRemovedSshTargetState: vi.fn()
})
const coordinator = {
requestReconnect: vi.fn(async () => {
order.push('request')
return { status: 'complete' }
}),
replaceAuthority: vi.fn(() => {
order.push('replace')
}),
prepareOnly: vi.fn(async () => {
order.push('prepare')
return { token: null }
}),
correctUnboundTerminals: vi.fn(() => 0),
finalizeHydratedTerminals: vi.fn(() => 0),
invalidate: vi.fn(),
stop: vi.fn()
}
const capturePreparationInput = vi.fn(async (authority, reason) => {
order.push('capture')
return {
...authority,
reason,
catalogRevision: 1,
repoRefs: [],
authorityRequirement: 'required'
}
})
stubReactSyncEffect()
stubAuxiliaryModules()
vi.doMock('../store', () => ({
useAppStore: {
subscribe: vi.fn(() => () => {}),
getState: () => storeState
}
}))
vi.doMock('./direct-ssh-reconnect-rollout', () => ({
isDirectSshReconnectCoordinatorRoutingEnabled: () => enabled
}))
vi.doMock('./direct-ssh-worktree-refresh-scheduler', () => ({
createDirectSshWorktreeRefreshScheduler: () => ({
stop: vi.fn(),
disposeProvider: vi.fn()
})
}))
vi.doMock('./direct-ssh-host-hydration', () => ({
createDirectSshHostHydration: () => ({
capturePreparationInput,
readHostScopedLineage: vi.fn(),
isPreparationTokenCurrent: vi.fn(() => true),
stop: vi.fn()
})
}))
vi.doMock('./direct-ssh-reconnect-coordinator', () => ({
createDirectSshReconnectCoordinator: () => coordinator
}))
vi.doMock('@/lib/direct-ssh-reconnect-product-telemetry', () => ({
createDirectSshReconnectProductTelemetryAdapter: vi.fn()
}))
vi.stubGlobal(
'window',
buildWindowApi({
onSet: () => () => {},
ssh: {
onStateChanged: (listener: (data: { targetId: string; state: unknown }) => void) => {
sshStateListener = listener
return () => {}
}
}
})
)
const { useIpcEvents } = await import('./useIpcEvents')
useIpcEvents()
sshStateListener?.({ targetId: 'target-a', state: nextState })
await Promise.resolve()
await Promise.resolve()
expect(order).toEqual(expectedRoute)
if (enabled) {
expect(coordinator.requestReconnect).toHaveBeenCalledOnce()
expect(coordinator.prepareOnly).not.toHaveBeenCalled()
} else {
expect(coordinator.requestReconnect).not.toHaveBeenCalled()
expect(coordinator.replaceAuthority).toHaveBeenCalledOnce()
expect(coordinator.prepareOnly).toHaveBeenCalledOnce()
}
}
)
it('caps pending mobile state events while startup hydration is unresolved', async () => {
const setFitOverride = vi.fn()
const hydrateOverrides = vi.fn()
src/renderer/src/hooks/useIpcEvents.ts
+391 -384
View File
@@ -2,7 +2,6 @@
import { useEffect } from 'react'
import { toast } from 'sonner'
import { useAppStore } from '../store'
import { shouldRetryPaneSpawnOnSshReconnect } from './ssh-reconnect-pane-retry'
import { getTabIdsAwaitingHostHydrationRemount } from '@/lib/parked-terminal-host-hydration'
import { applyWorktreeHeadIdentities } from './worktree-head-identity-apply'
import { getWorktreeMapFromState, getRepoMapFromState } from '@/store/selectors'
@@ -27,22 +26,17 @@ import { nextEditorFontZoomLevel, computeEditorFontSize } from '@/lib/editor-fon
import type {
TerminalLayoutSnapshot,
TerminalPaneLayoutNode,
UpdateStatus,
WorkspaceSessionState
UpdateStatus
} from '../../../shared/types'
import type {
RemoteWorkspacePatchResult,
RemoteWorkspaceSnapshot
} from '../../../shared/remote-workspace-types'
import type { RateLimitState } from '../../../shared/rate-limit-types'
import type { SshConnectionState } from '../../../shared/ssh-types'
import type { DirectSshAuthority, SshConnectionState } from '../../../shared/ssh-types'
import { toSshExecutionHostId } from '../../../shared/execution-host'
import { isWslHookRelayConnectionId } from '../../../shared/wsl-hook-relay-contract'
import type {
RuntimeBrowserDriverState,
RuntimeTerminalPresentation,
RuntimeTerminalDriverState
} from '../../../shared/runtime-types'
import { importRemoteWorkspaceSession } from '../../../shared/remote-workspace-session-projection'
import { zoomLevelToPercent } from '@/components/settings/SettingsConstants'
import { stepUIZoomLevel } from '../../../shared/ui-zoom-level'
import { dispatchZoomLevelChanged } from '@/lib/zoom-events'
@@ -143,6 +137,27 @@ import { closeTerminalTab } from '@/components/terminal/terminal-tab-actions'
import { initialAgentTabViewModeProps } from '@/lib/native-chat-initial-view-mode'
import { getConnectionIdFromState } from '@/lib/connection-context'
import { isNativeChatTranscriptLocalReadable } from '@/lib/native-chat-transcript-readability'
import { acquireDirectSshDetectedWorktreeRefresh } from '@/store/slices/worktrees'
import { createDirectSshWorktreeRefreshScheduler } from './direct-ssh-worktree-refresh-scheduler'
import {
createDirectSshReconnectCoordinator,
type DirectSshPreparationInput,
type DirectSshPreparationReason
} from './direct-ssh-reconnect-coordinator'
import { directSshAuthoritiesEqual } from './direct-ssh-reconnect-tokens'
import { createDirectSshHostHydration } from './direct-ssh-host-hydration'
import { createDirectSshReconnectProductTelemetryAdapter } from '@/lib/direct-ssh-reconnect-product-telemetry'
import {
createRemoteWorkspaceTargetSync,
isDirectSshRemoteWorkspaceApplyInProgress,
type RemoteWorkspaceTargetSync
} from './remote-workspace-target-sync'
import {
registerDirectSshWakeRouting,
routeDirectSshConnectedState,
type DirectSshConnectedStateOrigin
} from './direct-ssh-state-routing'
import { isDirectSshReconnectCoordinatorRoutingEnabled } from './direct-ssh-reconnect-rollout'
function getShortcutPlatform(): NodeJS.Platform {
if (navigator.userAgent.includes('Mac')) {
@@ -254,9 +269,6 @@ const MAX_PENDING_MOBILE_STATE_EVENTS = 300
// Why: a rename's event burst lags the on-disk move; shield both ids from the deletion diff for a grace window.
const WORKTREE_RENAME_PURGE_GRACE_MS = 20_000
const recentlyRenamedWorktreeIdExpiry = new Map<string, number>()
let remoteWorkspaceSnapshotApplyDepth = 0
let remoteWorkspaceSnapshotWriteSuppressUntil = 0
const REMOTE_WORKSPACE_SNAPSHOT_WRITE_SUPPRESS_MS = 1000
function isAgentStatusForRecentlyClosedTab(
store: Pick<AppState, 'recentlyClosedAgentStatusTabIds' | 'recentlyRetiredAgentStatusPaneKeys'>,
@@ -415,262 +427,7 @@ function activateExistingLeafInLayout(
}
export function isRemoteWorkspaceSnapshotApplyInProgress(): boolean {
return (
remoteWorkspaceSnapshotApplyDepth > 0 || Date.now() < remoteWorkspaceSnapshotWriteSuppressUntil
)
}
async function waitForWorkspaceSessionReady(): Promise<boolean> {
for (let attempt = 0; attempt < 100; attempt += 1) {
if (useAppStore.getState().workspaceSessionReady) {
return true
}
await new Promise((resolve) => window.setTimeout(resolve, 100))
}
return useAppStore.getState().workspaceSessionReady
}
async function prepareRemoteWorkspaceTarget(targetId: string): Promise<boolean> {
if (!(await waitForWorkspaceSessionReady())) {
return false
}
const store = useAppStore.getState()
let repos = store.repos.filter((repo) => repo.connectionId === targetId)
if (repos.length === 0) {
await store.fetchRepos()
repos = useAppStore.getState().repos.filter((repo) => repo.connectionId === targetId)
}
await Promise.all(repos.map((repo) => useAppStore.getState().fetchWorktrees(repo.id)))
await useAppStore.getState().fetchWorktreeLineage()
return true
}
function targetRepoIds(targetId: string): Set<string> {
return new Set(
useAppStore
.getState()
.repos.filter((repo) => repo.connectionId === targetId)
.map((repo) => repo.id)
)
}
function targetWorktreeIds(targetId: string): Set<string> {
const repoIds = targetRepoIds(targetId)
return new Set(
Object.values(useAppStore.getState().worktreesByRepo)
.flat()
.filter((worktree) => repoIds.has(worktree.repoId))
.map((worktree) => worktree.id)
)
}
function mergeRemoteWorkspaceSession(
current: WorkspaceSessionState,
remote: WorkspaceSessionState,
targetId: string
): WorkspaceSessionState {
const replaceWorktreeIds = targetWorktreeIds(targetId)
const remoteTabIds = new Set(
Object.values(remote.tabsByWorktree)
.flat()
.map((tab) => tab.id)
)
const replacedTabIds = new Set([
...remoteTabIds,
...Object.entries(current.tabsByWorktree)
.filter(([worktreeId]) => replaceWorktreeIds.has(worktreeId))
.flatMap(([, tabs]) => tabs.map((tab) => tab.id))
])
const omitTargetWorktrees = <T>(record: Record<string, T> | undefined): Record<string, T> =>
Object.fromEntries(
Object.entries(record ?? {}).filter(([worktreeId]) => !replaceWorktreeIds.has(worktreeId))
)
return {
...current,
activeRepoId:
remote.activeRepoId ??
(current.activeWorktreeId && replaceWorktreeIds.has(current.activeWorktreeId)
? null
: current.activeRepoId),
activeWorktreeId:
remote.activeWorktreeId ??
(current.activeWorktreeId && replaceWorktreeIds.has(current.activeWorktreeId)
? null
: current.activeWorktreeId),
activeTabId:
remote.activeTabId ??
(current.activeTabId && replacedTabIds.has(current.activeTabId) ? null : current.activeTabId),
tabsByWorktree: {
...omitTargetWorktrees(current.tabsByWorktree),
...remote.tabsByWorktree
},
terminalLayoutsByTabId: {
...Object.fromEntries(
Object.entries(current.terminalLayoutsByTabId).filter(
([tabId]) => !replacedTabIds.has(tabId)
)
),
...remote.terminalLayoutsByTabId
},
activeWorktreeIdsOnShutdown: [
...(current.activeWorktreeIdsOnShutdown ?? []).filter((id) => !replaceWorktreeIds.has(id)),
...(remote.activeWorktreeIdsOnShutdown ?? [])
],
activeTabIdByWorktree: {
...omitTargetWorktrees(current.activeTabIdByWorktree),
...remote.activeTabIdByWorktree
},
remoteSessionIdsByTabId: {
...Object.fromEntries(
Object.entries(current.remoteSessionIdsByTabId ?? {}).filter(
([tabId]) => !replacedTabIds.has(tabId)
)
),
...remote.remoteSessionIdsByTabId
},
lastVisitedAtByWorktreeId: {
...omitTargetWorktrees(current.lastVisitedAtByWorktreeId),
...remote.lastVisitedAtByWorktreeId
}
}
}
async function applyRemoteWorkspaceSnapshot(
targetId: string,
snapshot: RemoteWorkspaceSnapshot
): Promise<void> {
if (!(await prepareRemoteWorkspaceTarget(targetId))) {
throw new Error('Workspace sync waited for local session hydration and timed out')
}
const worktreeIds = targetWorktreeIds(targetId)
const localByPath = new Map(
Array.from(worktreeIds).map((worktreeId) => {
const separator = worktreeId.indexOf('::')
return [separator === -1 ? worktreeId : worktreeId.slice(separator + 2), worktreeId] as const
})
)
const remoteSession = importRemoteWorkspaceSession(snapshot.session, {
resolveWorktreeId: (worktreePath) => localByPath.get(worktreePath) ?? null
})
const current = buildWorkspaceSessionPayload(useAppStore.getState())
const merged = mergeRemoteWorkspaceSession(current, remoteSession, targetId)
const store = useAppStore.getState()
remoteWorkspaceSnapshotApplyDepth += 1
try {
store.hydrateWorkspaceSession(merged)
store.hydrateTabsSession(merged)
store.hydrateEditorSession(merged)
store.hydrateBrowserSession(merged)
store.markRemoteWorkspaceHydrated(targetId)
store.setRemoteWorkspaceSyncStatus(targetId, {
phase: 'synced',
direction: 'pull',
revision: snapshot.revision,
updatedAt: snapshot.updatedAt,
lastSyncedAt: Date.now(),
message: translate('auto.hooks.useIpcEvents.4f78ba5885', 'Workspace synced')
})
await useAppStore.getState().reconnectPersistedTerminals()
} finally {
// Why: reattach updates pty ids/titles after hydration; they came from the snapshot, don't echo back as a new revision.
remoteWorkspaceSnapshotWriteSuppressUntil =
Date.now() + REMOTE_WORKSPACE_SNAPSHOT_WRITE_SUPPRESS_MS
remoteWorkspaceSnapshotApplyDepth -= 1
}
}
async function syncRemoteWorkspaceAfterConnect(targetId: string): Promise<void> {
const store = useAppStore.getState()
if (!(await prepareRemoteWorkspaceTarget(targetId))) {
store.setRemoteWorkspaceSyncStatus(targetId, {
phase: 'error',
direction: 'pull',
message: translate(
'auto.hooks.useIpcEvents.88214a785b',
'Workspace sync waited for local session hydration and timed out'
)
})
return
}
store.setRemoteWorkspaceSyncStatus(targetId, { phase: 'pulling', direction: 'pull' })
const worktreeIds = targetWorktreeIds(targetId)
const hasLocalTabs = Array.from(worktreeIds).some(
(worktreeId) => (useAppStore.getState().tabsByWorktree[worktreeId] ?? []).length > 0
)
const snapshot = await window.api.remoteWorkspace.get({ targetId })
if (!snapshot) {
useAppStore.getState().setRemoteWorkspaceSyncStatus(targetId, {
phase: 'offline',
direction: 'pull',
message: translate('auto.hooks.useIpcEvents.2fe88c2e06', 'Remote workspace sync unavailable')
})
return
}
if (snapshot.revision > 0) {
await applyRemoteWorkspaceSnapshot(targetId, snapshot)
return
}
useAppStore.getState().markRemoteWorkspaceHydrated(targetId)
if (hasLocalTabs) {
// Why: read the relay before publishing local tabs, or a reconnect can overwrite a newer snapshot with stale local state.
const session = buildWorkspaceSessionPayload(useAppStore.getState())
const results = await window.api.remoteWorkspace.setForConnectedTargets({
session,
hydratedTargetIds: [targetId]
})
const result = results.find((entry) => entry.targetId === targetId)?.result
applyRemoteWorkspacePatchStatus(targetId, result)
if (result?.ok) {
useAppStore.getState().markRemoteWorkspaceHydrated(targetId)
}
return
}
useAppStore.getState().setRemoteWorkspaceSyncStatus(targetId, {
phase: 'idle',
revision: snapshot.revision,
updatedAt: snapshot.updatedAt,
message: translate('auto.hooks.useIpcEvents.2ec42e1c52', 'No remote workspace yet')
})
}
function applyRemoteWorkspacePatchStatus(
targetId: string,
result: RemoteWorkspacePatchResult | undefined
): void {
if (!result) {
useAppStore.getState().setRemoteWorkspaceSyncStatus(targetId, {
phase: 'offline',
direction: 'push',
lastSyncedAt: Date.now(),
message: translate('auto.hooks.useIpcEvents.2fe88c2e06', 'Remote workspace sync unavailable')
})
return
}
if (result.ok) {
useAppStore.getState().setRemoteWorkspaceSyncStatus(targetId, {
phase: 'synced',
direction: 'push',
revision: result.snapshot.revision,
updatedAt: result.snapshot.updatedAt,
lastSyncedAt: Date.now(),
message: translate('auto.hooks.useIpcEvents.f8aaf2bde3', 'Workspace uploaded')
})
return
}
useAppStore.getState().setRemoteWorkspaceSyncStatus(targetId, {
phase: result.reason === 'stale-revision' ? 'conflict' : 'offline',
direction: 'push',
revision: result.snapshot?.revision,
updatedAt: result.snapshot?.updatedAt,
lastSyncedAt: Date.now(),
message:
result.message ??
(result.reason === 'stale-revision'
? 'Workspace changed on another device'
: 'Remote workspace sync unavailable')
})
return isDirectSshRemoteWorkspaceApplyInProgress()
}
type BrowserSessionTabTarget =
@@ -850,6 +607,149 @@ function getWorktreeRuntimeEnvironmentId(worktreeId: string | null | undefined):
export function useIpcEvents(): void {
useEffect(() => {
const unsubs: (() => void)[] = []
const reconnectAuthorityByTarget = new Map<string, DirectSshAuthority>()
const authorityReconciliationDeadlines = new Set<{
timer: ReturnType<typeof setTimeout>
settle: () => void
}>()
let directSshEffectStopped = false
const currentDirectSshAuthority = (targetId: string): DirectSshAuthority | null => {
const state = useAppStore.getState().sshConnectionStates?.get(targetId)
if (
state?.status !== 'connected' ||
state.targetId !== targetId ||
!state.providerEpoch ||
state.connectionGeneration === undefined
) {
return null
}
return {
targetId,
providerEpoch: state.providerEpoch,
connectionGeneration: state.connectionGeneration
}
}
const scheduler = createDirectSshWorktreeRefreshScheduler({
startAttempt: (key) => {
const acquired = acquireDirectSshDetectedWorktreeRefresh(useAppStore, {
repoId: key.repoId,
executionHostId: key.executionHostId,
authority: {
targetId: key.targetId,
providerEpoch: key.providerEpoch,
connectionGeneration: key.connectionGeneration
},
requireAuthoritative: key.authorityRequirement === 'required'
})
return {
providerRequestId: acquired.providerRequestId,
result: acquired.result.then((result) => acquired.merge(result)),
cancel: acquired.release
}
}
})
const hostHydration = createDirectSshHostHydration({
store: useAppStore,
isCurrentAuthority: (authority) =>
directSshAuthoritiesEqual(currentDirectSshAuthority(authority.targetId), authority),
listRepos: (authority) => {
const executionHostId = toSshExecutionHostId(authority.targetId)
return (
window.api.repos.listForExecutionHost?.({
executionHostId,
expectedAuthority: authority
}) ??
Promise.resolve({
authoritative: false,
executionHostId,
reason: 'unavailable' as const
})
)
},
listLineage: (authority) => {
const executionHostId = toSshExecutionHostId(authority.targetId)
return (
window.api.worktrees.listLineageForHost?.({
executionHostId,
expectedAuthority: authority
}) ??
Promise.resolve({
authoritative: false,
executionHostId,
reason: 'unavailable' as const
})
)
}
})
type DirectSshTerminalActions = Partial<
Pick<AppState, 'invalidateStaleDirectSshTargetPtyBindings' | 'retryDirectSshTargetPanes'>
>
const directSshTerminalActions = (): DirectSshTerminalActions =>
useAppStore.getState() as DirectSshTerminalActions
let remoteWorkspaceTargetSync: RemoteWorkspaceTargetSync | null = null
const reconnectCoordinator = createDirectSshReconnectCoordinator({
scheduler,
isCurrentConnectedAuthority: (authority) =>
directSshAuthoritiesEqual(currentDirectSshAuthority(authority.targetId), authority),
capturePreparationInput: hostHydration.capturePreparationInput,
readHostScopedLineage: hostHydration.readHostScopedLineage,
invalidateStaleTerminalBindings: (authority) =>
directSshTerminalActions().invalidateStaleDirectSshTargetPtyBindings?.(authority) ?? 0,
retryTargetPanes: (authority) =>
directSshTerminalActions().retryDirectSshTargetPanes?.(authority) ?? 0,
finalizeHydratedTerminalPanes: (authority) =>
directSshTerminalActions().retryDirectSshTargetPanes?.(authority) ?? 0,
correctUnboundTerminalPanes: (authority) =>
directSshTerminalActions().retryDirectSshTargetPanes?.(authority) ?? 0,
syncRemoteWorkspaceAfterConnect: (token) =>
remoteWorkspaceTargetSync?.syncAfterConnect(token),
onTelemetry: createDirectSshReconnectProductTelemetryAdapter()
})
const remoteWorkspaceApi = window.api.remoteWorkspace
if (remoteWorkspaceApi) {
remoteWorkspaceTargetSync = createRemoteWorkspaceTargetSync({
store: useAppStore,
remoteWorkspace: remoteWorkspaceApi,
getCurrentAuthority: currentDirectSshAuthority,
isPreparationTokenCurrent: hostHydration.isPreparationTokenCurrent,
capturePreparationInput: (authority, reason, snapshotRevision) =>
hostHydration.capturePreparationInput(authority, reason, snapshotRevision),
prepareOnly: reconnectCoordinator.prepareOnly,
finalizeHydratedTerminals: (authority) =>
directSshAuthoritiesEqual(reconnectAuthorityByTarget.get(authority.targetId), authority)
? reconnectCoordinator.finalizeHydratedTerminals(authority)
: 0
})
}
const prepareAndSyncDirectSshTarget = async (
authority: DirectSshAuthority,
reason: DirectSshPreparationReason,
options?: { authorityAlreadyReplaced?: boolean }
): Promise<void> => {
try {
if (!options?.authorityAlreadyReplaced) {
reconnectCoordinator.replaceAuthority(authority)
}
const input: DirectSshPreparationInput | null = await hostHydration.capturePreparationInput(
authority,
reason
)
if (!input) {
return
}
const prepared = await reconnectCoordinator.prepareOnly(input)
if (prepared.token && hostHydration.isPreparationTokenCurrent(prepared.token)) {
await remoteWorkspaceTargetSync?.syncAfterConnect(prepared.token)
}
} catch (error) {
if (directSshAuthoritiesEqual(currentDirectSshAuthority(authority.targetId), authority)) {
useAppStore.getState().setRemoteWorkspaceSyncStatus(authority.targetId, {
phase: 'error',
message: error instanceof Error ? error.message : 'Workspace sync failed'
})
}
}
}
const backgroundSleepingAgentWakeDispatcher = createBackgroundSleepingAgentWakeDispatcher()
unsubs.push(backgroundSleepingAgentWakeDispatcher.dispose)
type PendingAgentStatusEvent = {
@@ -1091,40 +991,38 @@ export function useIpcEvents(): void {
let reachableRuntimeEnvironmentKey = buildRuntimeClientEventEnvironmentKey(
reachableRuntimeEnvironmentIds
)
unsubs.push(
useAppStore.subscribe(() => {
const nextEnvironmentIds = getRuntimeClientEventEnvironmentIds()
const nextKey = buildRuntimeClientEventEnvironmentKey(nextEnvironmentIds)
const nextReachableEnvironmentIds = getReachableRuntimeEnvironmentIds()
const nextReachableKey = buildRuntimeClientEventEnvironmentKey(nextReachableEnvironmentIds)
if (
nextKey === runtimeClientEventEnvironmentKey &&
nextReachableKey === reachableRuntimeEnvironmentKey
) {
return
}
for (const environmentId of getRuntimeProjectRefreshEnvironmentIds({
previousDesired: runtimeClientEventEnvironmentIds,
nextDesired: nextEnvironmentIds,
previousReachable: reachableRuntimeEnvironmentIds,
nextReachable: nextReachableEnvironmentIds
})) {
runtimeProjectRefreshScheduler.request(environmentId)
}
for (const environmentId of getNewlyDisconnectedRuntimeEnvironmentIds(
reachableRuntimeEnvironmentIds,
nextReachableEnvironmentIds
)) {
// No-op when the environment has no SSH bucket (e.g. web client).
useAppStore.getState().markEnvironmentSshStateStale(environmentId)
}
runtimeClientEventEnvironmentIds = nextEnvironmentIds
runtimeClientEventEnvironmentKey = nextKey
reachableRuntimeEnvironmentIds = nextReachableEnvironmentIds
reachableRuntimeEnvironmentKey = nextReachableKey
runtimeClientEventsSync.sync()
})
)
const unsubscribeRuntimeEnvironmentStore = useAppStore.subscribe(() => {
const nextEnvironmentIds = getRuntimeClientEventEnvironmentIds()
const nextKey = buildRuntimeClientEventEnvironmentKey(nextEnvironmentIds)
const nextReachableEnvironmentIds = getReachableRuntimeEnvironmentIds()
const nextReachableKey = buildRuntimeClientEventEnvironmentKey(nextReachableEnvironmentIds)
if (
nextKey === runtimeClientEventEnvironmentKey &&
nextReachableKey === reachableRuntimeEnvironmentKey
) {
return
}
for (const environmentId of getRuntimeProjectRefreshEnvironmentIds({
previousDesired: runtimeClientEventEnvironmentIds,
nextDesired: nextEnvironmentIds,
previousReachable: reachableRuntimeEnvironmentIds,
nextReachable: nextReachableEnvironmentIds
})) {
runtimeProjectRefreshScheduler.request(environmentId)
}
for (const environmentId of getNewlyDisconnectedRuntimeEnvironmentIds(
reachableRuntimeEnvironmentIds,
nextReachableEnvironmentIds
)) {
// No-op when the environment has no SSH bucket (e.g. web client).
useAppStore.getState().markEnvironmentSshStateStale(environmentId)
}
runtimeClientEventEnvironmentIds = nextEnvironmentIds
runtimeClientEventEnvironmentKey = nextKey
reachableRuntimeEnvironmentIds = nextReachableEnvironmentIds
reachableRuntimeEnvironmentKey = nextReachableKey
runtimeClientEventsSync.sync()
})
unsubs.push(runtimeClientEventsSync.stop)
unsubs.push(runtimeProjectRefreshScheduler.stop)
@@ -2664,40 +2562,60 @@ export function useIpcEvents(): void {
unsubs.push(unsubscribeWorkspaceSpaceProgress)
}
const sshStateWatermarkByTargetId = new Map<string, number>()
let applySshConnectionStateChange!: (
targetId: string,
state: SshConnectionState,
origin: DirectSshConnectedStateOrigin
) => void
// Why: hydrate initial SSH state for all targets so worktree cards show correct connect state on launch.
void (async () => {
try {
const targets = await window.api.ssh.listTargets()
if (directSshEffectStopped) {
return
}
useAppStore.getState().setSshTargetsMetadata(targets)
// Why: ghost-host UI (removed target still referenced by a workspace) shows a tombstone name instead of the raw id.
try {
const removedLabels = await window.api.ssh.listRemovedTargetLabels()
if (directSshEffectStopped) {
return
}
useAppStore.getState().setRemovedSshTargetLabels(removedLabels)
} catch {
// Best-effort — a missing map just falls back to the raw target id.
}
for (const target of targets) {
const hydrationWatermark = sshStateWatermarkByTargetId.get(target.id) ?? 0
const state = await window.api.ssh.getState({ targetId: target.id })
if (state) {
useAppStore.getState().setSshConnectionState(target.id, state as SshConnectionState)
if (
!directSshEffectStopped &&
state &&
(sshStateWatermarkByTargetId.get(target.id) ?? 0) === hydrationWatermark
) {
applySshConnectionStateChange(
target.id,
state as SshConnectionState,
'initial-hydration'
)
// Why: ports arrive only via push events; on reattach to a live session fetch snapshots or the Ports panel shows empty.
if ((state as SshConnectionState).status === 'connected') {
const authority = currentDirectSshAuthority(target.id)
const [forwards, detected] = await Promise.all([
window.api.ssh.listPortForwards({ targetId: target.id }),
window.api.ssh.listDetectedPorts({ targetId: target.id })
])
// Why: if the session disconnected while awaiting the snapshot, applying it would resurrect a dead session's ports.
const currentState = useAppStore.getState().sshConnectionStates.get(target.id)
if (currentState?.status === 'connected') {
if (
!directSshEffectStopped &&
authority &&
directSshAuthoritiesEqual(currentDirectSshAuthority(target.id), authority)
) {
useAppStore.getState().setPortForwards(target.id, forwards)
useAppStore.getState().setDetectedPorts(target.id, detected)
}
void syncRemoteWorkspaceAfterConnect(target.id).catch((err) => {
useAppStore.getState().setRemoteWorkspaceSyncStatus(target.id, {
phase: 'error',
message: err instanceof Error ? err.message : 'Workspace sync failed'
})
})
}
}
}
@@ -2730,12 +2648,75 @@ export function useIpcEvents(): void {
})
)
const applySshConnectionStateChange = (targetId: string, state: SshConnectionState): void => {
const reconcileSshAuthority = (
targetId: string,
initiatingState: SshConnectionState,
origin: DirectSshConnectedStateOrigin,
watermark: number
): void => {
let pendingDeadline: { timer: ReturnType<typeof setTimeout>; settle: () => void } | undefined
const deadline = new Promise<null>((resolve) => {
const settle = (): void => resolve(null)
const timer = setTimeout(settle, 5_000)
pendingDeadline = { timer, settle }
authorityReconciliationDeadlines.add(pendingDeadline)
})
void Promise.race([window.api.ssh.getState({ targetId }).catch(() => null), deadline])
.then((latest) => {
if (
directSshEffectStopped ||
latest?.targetId !== targetId ||
!latest?.providerEpoch ||
latest.connectionGeneration === undefined ||
sshStateWatermarkByTargetId.get(targetId) !== watermark
) {
return
}
const current = useAppStore.getState().sshConnectionStates?.get(targetId)
if (
current?.status !== initiatingState.status ||
latest.status !== initiatingState.status ||
current.providerEpoch !== initiatingState.providerEpoch ||
current.connectionGeneration !== initiatingState.connectionGeneration ||
(current.providerEpoch !== undefined &&
current.providerEpoch !== null &&
current.providerEpoch !== latest.providerEpoch) ||
(current.connectionGeneration !== undefined &&
current.connectionGeneration !== latest.connectionGeneration)
) {
return
}
applySshConnectionStateChange(
targetId,
{
...current,
providerEpoch: latest.providerEpoch,
connectionGeneration: latest.connectionGeneration
},
origin
)
})
.catch(() => undefined)
.finally(() => {
if (pendingDeadline) {
clearTimeout(pendingDeadline.timer)
authorityReconciliationDeadlines.delete(pendingDeadline)
}
})
}
applySshConnectionStateChange = (
targetId: string,
state: SshConnectionState,
origin: DirectSshConnectedStateOrigin
): void => {
const store = useAppStore.getState()
const previous = store.sshConnectionStates?.get(targetId)
store.setSshConnectionState(targetId, state)
const remoteRepos = store.repos.filter((r) => r.connectionId === targetId)
if (['disconnected', 'auth-failed', 'reconnection-failed', 'error'].includes(state.status)) {
reconnectAuthorityByTarget.delete(targetId)
reconnectCoordinator.invalidate(targetId)
// Why: remote agent list is tied to a live relay; clear on disconnect so reconnect re-detects against the new relay.
store.clearRemoteDetectedAgents(targetId)
@@ -2743,62 +2724,54 @@ export function useIpcEvents(): void {
store.clearPortForwards(targetId)
store.setDetectedPorts(targetId, [])
// Why: SSH teardown fires no per-PTY exit events; clear stale PTY ids so reconnect remounts rather than reattach a dead PTY.
const remoteWorktreeIds = new Set(
Object.values(store.worktreesByRepo)
.flat()
.filter((w) => remoteRepos.some((r) => r.id === w.repoId))
.map((w) => w.id)
// SSH teardown has no per-PTY exits; clear only exact-target bindings in one store publication.
store.clearDirectSshTargetPtyBindings(targetId)
return
}
if (state.status !== 'connected') {
return
}
const authority = currentDirectSshAuthority(targetId)
if (!authority) {
reconcileSshAuthority(
targetId,
state,
origin,
sshStateWatermarkByTargetId.get(targetId) ?? 0
)
for (const worktreeId of remoteWorktreeIds) {
const tabs = useAppStore.getState().tabsByWorktree[worktreeId] ?? []
for (const tab of tabs) {
if (tab.ptyId) {
useAppStore.getState().clearTabPtyId(tab.id)
return
}
const previousAuthority =
previous?.status === 'connected' &&
previous.providerEpoch &&
previous.connectionGeneration !== undefined
? {
targetId,
providerEpoch: previous.providerEpoch,
connectionGeneration: previous.connectionGeneration
}
: null
routeDirectSshConnectedState(
{
coordinator: reconnectCoordinator,
coordinatorRoutingEnabled: isDirectSshReconnectCoordinatorRoutingEnabled(),
invalidateStaleTerminalBindings: (nextAuthority) =>
directSshTerminalActions().invalidateStaleDirectSshTargetPtyBindings?.(nextAuthority) ??
0,
retryTargetPanes: (nextAuthority) =>
directSshTerminalActions().retryDirectSshTargetPanes?.(nextAuthority) ?? 0,
prepareAndSync: prepareAndSyncDirectSshTarget,
rememberReconnectAuthority: (nextAuthority) => {
if (nextAuthority) {
reconnectAuthorityByTarget.set(targetId, nextAuthority)
} else {
reconnectAuthorityByTarget.delete(targetId)
}
}
}
}
if (state.status === 'connected') {
void Promise.all(remoteRepos.map((r) => store.fetchWorktrees(r.id))).then(async () => {
await useAppStore.getState().fetchWorktreeLineage()
// Why: panes that never spawned (no PTY provider at cold start) or whose deferred reattach never ran sit inert.
// Bumping generation remounts TerminalPane so the deferred-connect gate reattaches or spawns fresh now that the provider exists.
const freshStore = useAppStore.getState()
const remoteRepoIds = new Set(remoteRepos.map((r) => r.id))
const worktreeIds = Object.values(freshStore.worktreesByRepo)
.flat()
.filter((w) => remoteRepoIds.has(w.repoId))
.map((w) => w.id)
for (const worktreeId of worktreeIds) {
const tabs = freshStore.tabsByWorktree[worktreeId] ?? []
const needsRetry = (t: { id: string; ptyId?: string | null }): boolean =>
shouldRetryPaneSpawnOnSshReconnect({
targetId,
tabPtyId: t.ptyId,
deferredSessionId: freshStore.deferredSshSessionIdsByTabId[t.id]
})
if (tabs.some(needsRetry)) {
useAppStore.setState((s) => ({
tabsByWorktree: {
...s.tabsByWorktree,
[worktreeId]: (s.tabsByWorktree[worktreeId] ?? []).map((t) =>
needsRetry(t) ? { ...t, generation: (t.generation ?? 0) + 1 } : t
)
}
}))
}
}
void syncRemoteWorkspaceAfterConnect(targetId).catch((err) => {
useAppStore.getState().setRemoteWorkspaceSyncStatus(targetId, {
phase: 'error',
message: err instanceof Error ? err.message : 'Workspace sync failed'
})
})
})
}
},
{ authority, previousAuthority, origin }
)
}
let sshTargetStateEventId = 0
@@ -2808,6 +2781,10 @@ export function useIpcEvents(): void {
const store = useAppStore.getState()
const state = data.state as SshConnectionState
const stateEventId = ++sshTargetStateEventId
sshStateWatermarkByTargetId.set(
data.targetId,
(sshStateWatermarkByTargetId.get(data.targetId) ?? 0) + 1
)
latestSshTargetStateEventByTargetId.set(data.targetId, stateEventId)
if (!store.sshTargetLabels.has(data.targetId)) {
// Why: unknown target id could be a post-boot add or a removed target racing disconnect; confirm with main first.
@@ -2820,6 +2797,9 @@ export function useIpcEvents(): void {
return
}
latestSshTargetStateEventByTargetId.delete(data.targetId)
if (directSshEffectStopped) {
return
}
const latestStore = useAppStore.getState()
if (!targets.some((target) => target.id === data.targetId)) {
// Why: state events can race after target removal; absence from main's target list means deletion, not a new target.
@@ -2827,22 +2807,37 @@ export function useIpcEvents(): void {
return
}
latestStore.setSshTargetsMetadata(targets)
applySshConnectionStateChange(data.targetId, state)
applySshConnectionStateChange(data.targetId, state, 'push')
})
.catch(() => {
if (latestSshTargetStateEventByTargetId.get(data.targetId) === stateEventId) {
if (
!directSshEffectStopped &&
latestSshTargetStateEventByTargetId.get(data.targetId) === stateEventId
) {
latestSshTargetStateEventByTargetId.delete(data.targetId)
applySshConnectionStateChange(data.targetId, state)
applySshConnectionStateChange(data.targetId, state, 'push')
}
})
return
}
latestSshTargetStateEventByTargetId.delete(data.targetId)
applySshConnectionStateChange(data.targetId, state)
applySshConnectionStateChange(data.targetId, state, 'push')
}
unsubs.push(window.api.ssh.onStateChanged(handleSshStateChangedEvent))
unsubs.push(
registerDirectSshWakeRouting({
getConnectionStates: () => useAppStore.getState().sshConnectionStates ?? [],
wakeAuthority: (authority) => {
reconnectCoordinator.correctUnboundTerminals(authority, 'wake-refresh')
void prepareAndSyncDirectSshTarget(authority, 'wake-refresh')
},
...(typeof window.api.ui.onSystemResumed === 'function'
? { onSystemResumed: (callback: () => void) => window.api.ui.onSystemResumed(callback) }
: {})
})
)
let remoteWorkspaceClientId: string | null = null
let remoteWorkspaceClientIdPromise: Promise<string | null> | null = null
@@ -2873,13 +2868,15 @@ export function useIpcEvents(): void {
if (event.sourceClientId && clientId && event.sourceClientId === clientId) {
return
}
await applyRemoteWorkspaceSnapshot(event.targetId, event.snapshot).catch((err) => {
useAppStore.getState().setRemoteWorkspaceSyncStatus(event.targetId, {
phase: 'error',
revision: event.snapshot.revision,
message: err instanceof Error ? err.message : 'Failed to apply remote workspace'
await remoteWorkspaceTargetSync
?.applyUnsolicitedSnapshot(event.targetId, event.snapshot)
.catch((err) => {
useAppStore.getState().setRemoteWorkspaceSyncStatus(event.targetId, {
phase: 'error',
revision: event.snapshot.revision,
message: err instanceof Error ? err.message : 'Failed to apply remote workspace'
})
})
})
})()
})
)
@@ -3318,13 +3315,11 @@ export function useIpcEvents(): void {
// Why: main hook server is the durable source of truth; pull the snapshot only after tabs are ready so early startup pushes can be ignored, not buffered.
requestAgentStatusSnapshotIfReady()
unsubs.push(
useAppStore.subscribe((state, previousState) => {
requestAgentStatusSnapshotIfReady()
flushPendingAgentStatuses()
syncAgentHookCompletionNotificationsForStoreUpdate(state, previousState)
})
)
const unsubscribeAgentStatusStore = useAppStore.subscribe((state, previousState) => {
requestAgentStatusSnapshotIfReady()
flushPendingAgentStatuses()
syncAgentHookCompletionNotificationsForStoreUpdate(state, previousState)
})
let mobileStateHydrated = isRuntimeEnvironmentActive()
type PendingMobileStateEvent =
@@ -3452,7 +3447,19 @@ export function useIpcEvents(): void {
pendingAgentStatusEvents.length = 0
mobileStateHydrationDisposed = true
pendingMobileStateEvents.length = 0
unsubscribeRuntimeEnvironmentStore()
unsubscribeAgentStatusStore()
unsubs.forEach((fn) => fn())
directSshEffectStopped = true
for (const deadline of authorityReconciliationDeadlines) {
clearTimeout(deadline.timer)
deadline.settle()
}
authorityReconciliationDeadlines.clear()
remoteWorkspaceTargetSync?.stop()
hostHydration.stop()
reconnectCoordinator.stop()
reconnectAuthorityByTarget.clear()
resetAgentHookCompletionNotificationCoordinators()
}
}, [])
src/renderer/src/i18n/locales/en.json
+1
View File
@@ -702,6 +702,7 @@
"60428567b4": "Local terminal reveal is unavailable while a remote runtime is active",
"f8aaf2bde3": "Workspace uploaded",
"2fe88c2e06": "Remote workspace sync unavailable",
"workspaceChangedOnAnotherDevice": "Workspace changed on another device",
"2ec42e1c52": "No remote workspace yet",
"88214a785b": "Workspace sync waited for local session hydration and timed out",
"4f78ba5885": "Workspace synced",
src/renderer/src/i18n/locales/es.json
+1
View File
@@ -679,6 +679,7 @@
"60428567b4": "Mostrar terminales locales no está disponible mientras haya un host remoto activo",
"f8aaf2bde3": "Espacio de trabajo subido",
"2fe88c2e06": "La sincronización del espacio de trabajo remoto no está disponible",
"workspaceChangedOnAnotherDevice": "El espacio de trabajo cambió en otro dispositivo",
"2ec42e1c52": "Aún no hay espacio de trabajo remoto",
"88214a785b": "La sincronización del espacio de trabajo esperó a que la sesión local terminara de cargarse y agotó el tiempo de espera",
"4f78ba5885": "Espacio de trabajo sincronizado",
src/renderer/src/i18n/locales/ja.json
+1
View File
@@ -679,6 +679,7 @@
"60428567b4": "リモート ランタイムがアクティブな間はローカル terminal の公開は利用できません",
"f8aaf2bde3": "ワークスペースがアップロードされました",
"2fe88c2e06": "リモートワークスペースの同期は利用できません",
"workspaceChangedOnAnotherDevice": "ワークスペースが別のデバイスで変更されました",
"2ec42e1c52": "リモートワークスペースはまだありません",
"88214a785b": "ワークスペースの同期はローカル セッションのハイドレーションを待機し、タイムアウトになりました",
"4f78ba5885": "ワークスペースが同期されました",
src/renderer/src/i18n/locales/ko.json
+1
View File
@@ -679,6 +679,7 @@
"60428567b4": "원격 런타임이 활성화된 동안에는 로컬 terminal 표시를 사용할 수 없습니다.",
"f8aaf2bde3": "워크스페이스가 업로드됨",
"2fe88c2e06": "원격 워크스페이스 동기화를 사용할 수 없습니다.",
"workspaceChangedOnAnotherDevice": "다른 기기에서 워크스페이스가 변경되었습니다.",
"2ec42e1c52": "아직 원격 워크스페이스가 없습니다.",
"88214a785b": "워크스페이스 동기화가 로컬 세션 하이드레이션을 기다렸다가 시간 초과되었습니다.",
"4f78ba5885": "워크스페이스가 동기화되었습니다.",
src/renderer/src/i18n/locales/zh.json
+1
View File
@@ -679,6 +679,7 @@
"60428567b4": "当远程运行时处于活动状态时，本地终端显示不可用",
"f8aaf2bde3": "工作区已上传",
"2fe88c2e06": "远程工作区同步不可用",
"workspaceChangedOnAnotherDevice": "工作区已在另一台设备上更改",
"2ec42e1c52": "还没有远程工作区",
"88214a785b": "工作区同步等待本地会话恢复(hydration)并超时",
"4f78ba5885": "工作区已同步",
src/renderer/src/lib/direct-ssh-reconnect-product-telemetry-types.ts
+8
View File
@@ -0,0 +1,8 @@
import type { EventProps } from '../../../shared/telemetry-events'
export type DirectSshReconnectProductProps = EventProps<'direct_ssh_reconnect_operation'>
export type DirectSshReconnectProductSink = (
name: 'direct_ssh_reconnect_operation',
props: DirectSshReconnectProductProps
) => void
src/renderer/src/lib/direct-ssh-reconnect-product-telemetry.test.ts
+106
View File
@@ -0,0 +1,106 @@
import { describe, expect, it, vi } from 'vitest'
import type { DirectSshCoordinatorTelemetry } from '../hooks/direct-ssh-reconnect-coordinator'
import {
createDirectSshReconnectProductTelemetryAdapter,
toDirectSshReconnectProductProps
} from './direct-ssh-reconnect-product-telemetry'
function coordinatorEvent(
overrides: Partial<DirectSshCoordinatorTelemetry> = {}
): DirectSshCoordinatorTelemetry {
return {
mode: 'prepare-only',
reason: 'workspace-snapshot',
outcome: 'degraded',
durationMs: 900,
staleBindingsCleared: 1,
retriedTerminals: 2,
correctedTerminals: 3,
terminalFinalizationDurationMs: 4,
catalogOutcome: 'complete',
catalogDurationMs: 20,
gitWorktreeCount: 5,
folderWorkspaceCount: 2,
ambiguousOwnerCount: 1,
contradictoryOwnerCount: 1,
repoOutcomes: {
complete: 2,
'non-authoritative': 1,
'timed-out': 1,
'cancel-budget-exhausted': 1,
canceled: 1,
stale: 1,
rejected: 1
},
lineageOutcome: 'degraded',
queueWaitDurationsMs: [30, 10, 20, 40],
providerExecutionDurationsMs: [400, 100, 300, 200],
timeoutRetryCount: 2,
locallySettledWaiterCount: 4,
cancelDebtCount: 2,
replacementAdmissionDelayedCount: 1,
overlappingJoinCount: 3,
peakLocallyUnsettled: 4,
estimatedLateWorkAllowanceCount: 2,
authorityRotationCount: 1,
damped: true,
...overrides
}
}
describe('direct SSH reconnect product telemetry', () => {
it('maps one coordinator aggregate to privacy-safe product fields', () => {
const props = toDirectSshReconnectProductProps(coordinatorEvent())
expect(props).toMatchObject({
mode: 'prepare_only',
reason: 'workspace_snapshot',
outcome: 'degraded',
repo_retrying_count: 2,
repo_timed_out_count: 1,
repo_rejected_count: 1,
repo_canceled_count: 1,
repo_stale_count: 1,
lineage_degraded_count: 1,
queue_wait_sample_count: 4,
queue_wait_duration_ms_p50: 20,
queue_wait_duration_ms_p95: 40,
provider_execution_sample_count: 4,
provider_execution_duration_ms_p50: 200,
provider_execution_duration_ms_p99: 400,
cancel_debt_count: 2,
replacement_admission_delayed_count: 1,
overlapping_join_count: 3,
timeout_retry_count: 2,
damped_preparation_count: 1
})
expect(Object.keys(props).join(' ')).not.toMatch(
/target_id|repo_id|host|path|label|user|request_id|lease_id|terminal_id|error/
)
expect(props).not.toHaveProperty('terminal_correction_failed_count')
expect(props).not.toHaveProperty('terminal_correction_rearmed_count')
expect(props).not.toHaveProperty('concurrent_non_coordinator_call_count')
expect(props).not.toHaveProperty('arrival_order_reply_discarded_count')
})
it('emits exactly one typed event per adapter call', () => {
const sink = vi.fn()
const emit = createDirectSshReconnectProductTelemetryAdapter(sink)
emit(coordinatorEvent())
expect(sink).toHaveBeenCalledOnce()
expect(sink).toHaveBeenCalledWith(
'direct_ssh_reconnect_operation',
expect.objectContaining({ total_duration_ms: 900 })
)
})
it('does not let telemetry failures escape into recovery', () => {
const emit = createDirectSshReconnectProductTelemetryAdapter(() => {
throw new Error('telemetry unavailable')
})
expect(() => emit(coordinatorEvent())).not.toThrow()
})
})
src/renderer/src/lib/direct-ssh-reconnect-product-telemetry.ts
+127
View File
@@ -0,0 +1,127 @@
import type { DirectSshCoordinatorTelemetry } from '../hooks/direct-ssh-reconnect-coordinator'
import { track } from './telemetry'
import type {
DirectSshReconnectProductProps,
DirectSshReconnectProductSink
} from './direct-ssh-reconnect-product-telemetry-types'
const MAX_COUNT = 1_000_000
const MAX_DURATION_MS = 86_400_000
function boundedInteger(value: number, maximum = MAX_COUNT): number {
return Math.min(maximum, Math.max(0, Math.round(Number.isFinite(value) ? value : 0)))
}
function durationDistribution(values: readonly number[]): {
count: number
p50: number
p95: number
p99: number
max: number
} {
const sorted = values
.map((value) => boundedInteger(value, MAX_DURATION_MS))
.toSorted((a, b) => a - b)
const percentile = (quantile: number): number =>
sorted.length === 0 ? 0 : sorted[Math.max(0, Math.ceil(sorted.length * quantile) - 1)]
return {
count: boundedInteger(sorted.length),
p50: percentile(0.5),
p95: percentile(0.95),
p99: percentile(0.99),
max: sorted.at(-1) ?? 0
}
}
function lineageCounts(
outcome: DirectSshCoordinatorTelemetry['lineageOutcome']
): Pick<
DirectSshReconnectProductProps,
| 'lineage_complete_count'
| 'lineage_degraded_count'
| 'lineage_canceled_count'
| 'lineage_stale_count'
| 'lineage_not_started_count'
> {
return {
lineage_complete_count: outcome === 'complete' ? 1 : 0,
lineage_degraded_count: outcome === 'degraded' ? 1 : 0,
lineage_canceled_count: outcome === 'canceled' ? 1 : 0,
lineage_stale_count: outcome === 'stale' ? 1 : 0,
lineage_not_started_count: outcome === 'not-started' ? 1 : 0
}
}
export function toDirectSshReconnectProductProps(
event: DirectSshCoordinatorTelemetry
): DirectSshReconnectProductProps {
const queueWait = durationDistribution(event.queueWaitDurationsMs)
const providerExecution = durationDistribution(event.providerExecutionDurationsMs)
return {
mode: event.mode === 'prepare-only' ? 'prepare_only' : 'reconnect',
reason: event.reason.replaceAll('-', '_') as DirectSshReconnectProductProps['reason'],
outcome: event.outcome,
terminal_retried_count: boundedInteger(event.retriedTerminals),
terminal_stale_binding_cleared_count: boundedInteger(event.staleBindingsCleared),
terminal_correction_succeeded_count: boundedInteger(event.correctedTerminals),
catalog_complete_count: event.catalogOutcome === 'complete' ? 1 : 0,
catalog_degraded_count: event.catalogOutcome === 'degraded' ? 1 : 0,
catalog_stale_count: event.catalogOutcome === 'stale' ? 1 : 0,
repo_complete_count: boundedInteger(event.repoOutcomes.complete),
repo_non_authoritative_count: boundedInteger(event.repoOutcomes['non-authoritative']),
// One first-timeout transition per repo makes retrying repos equal timeout retries.
repo_retrying_count: boundedInteger(event.timeoutRetryCount),
repo_timed_out_count: boundedInteger(event.repoOutcomes['timed-out']),
repo_cancel_budget_exhausted_count: boundedInteger(
event.repoOutcomes['cancel-budget-exhausted']
),
repo_canceled_count: boundedInteger(event.repoOutcomes.canceled),
repo_stale_count: boundedInteger(event.repoOutcomes.stale),
repo_rejected_count: boundedInteger(event.repoOutcomes.rejected),
...lineageCounts(event.lineageOutcome),
git_worktree_count: boundedInteger(event.gitWorktreeCount),
folder_workspace_count: boundedInteger(event.folderWorkspaceCount),
ambiguous_owner_count: boundedInteger(event.ambiguousOwnerCount),
contradictory_owner_count: boundedInteger(event.contradictoryOwnerCount),
total_duration_ms: boundedInteger(event.durationMs, MAX_DURATION_MS),
terminal_finalization_duration_ms: boundedInteger(
event.terminalFinalizationDurationMs,
MAX_DURATION_MS
),
catalog_duration_ms: boundedInteger(event.catalogDurationMs, MAX_DURATION_MS),
queue_wait_sample_count: queueWait.count,
queue_wait_duration_ms_p50: queueWait.p50,
queue_wait_duration_ms_p95: queueWait.p95,
queue_wait_duration_ms_p99: queueWait.p99,
queue_wait_duration_ms_max: queueWait.max,
provider_execution_sample_count: providerExecution.count,
provider_execution_duration_ms_p50: providerExecution.p50,
provider_execution_duration_ms_p95: providerExecution.p95,
provider_execution_duration_ms_p99: providerExecution.p99,
provider_execution_duration_ms_max: providerExecution.max,
timeout_retry_count: boundedInteger(event.timeoutRetryCount),
locally_settled_waiter_count: boundedInteger(event.locallySettledWaiterCount),
cancel_debt_count: boundedInteger(event.cancelDebtCount),
replacement_admission_delayed_count: boundedInteger(event.replacementAdmissionDelayedCount),
overlapping_join_count: boundedInteger(event.overlappingJoinCount),
coordinator_owned_direct_ssh_detected_worktree_concurrency_peak: boundedInteger(
event.peakLocallyUnsettled,
5
),
estimated_late_work_allowance_count: boundedInteger(event.estimatedLateWorkAllowanceCount, 2),
authority_rotation_count: boundedInteger(event.authorityRotationCount),
damped_preparation_count: event.damped ? 1 : 0
}
}
export function createDirectSshReconnectProductTelemetryAdapter(
sink: DirectSshReconnectProductSink = track
): (event: DirectSshCoordinatorTelemetry) => void {
return (event) => {
try {
sink('direct_ssh_reconnect_operation', toDirectSshReconnectProductProps(event))
} catch {
// Recovery cannot depend on product telemetry.
}
}
}
src/renderer/src/lib/direct-ssh-target-owner-index.ts
+11
View File
@@ -0,0 +1,11 @@
export function indexDirectSshOwnerRows<T extends { id: string }>(
rows: readonly T[]
): Map<string, T[]> {
const rowsById = new Map<string, T[]>()
for (const row of rows) {
const owners = rowsById.get(row.id) ?? []
owners.push(row)
rowsById.set(row.id, owners)
}
return rowsById
}
src/renderer/src/lib/direct-ssh-target-scope-types.ts
+52
View File
@@ -0,0 +1,52 @@
import type {
FolderWorkspace,
ProjectGroup,
Repo,
WorkspaceKey,
Worktree
} from '../../../shared/types'
export type DirectSshGitRepoRef = {
repoId: string
executionHostId: `ssh:${string}`
}
export type DirectSshTargetScope = {
catalogRevision: number
gitRepos: DirectSshGitRepoRef[]
gitWorktreeIds: Set<string>
terminalWorkspaceKeys: Set<string>
lineageWorkspaceKeys: Set<WorkspaceKey>
ambiguousOwnerCount: number
contradictoryOwnerCount: number
}
export type DirectSshRepoOwner = Pick<
Repo,
'id' | 'path' | 'projectGroupId' | 'connectionId' | 'executionHostId'
>
export type DirectSshWorktreeOwner = Pick<
Worktree,
'id' | 'repoId' | 'hostId' | 'runtimeOwnerEnvironmentId'
>
export type DirectSshFolderOwner = Pick<
FolderWorkspace,
'id' | 'projectGroupId' | 'folderPath' | 'connectionId'
>
export type DirectSshGroupOwner = Pick<
ProjectGroup,
'id' | 'parentGroupId' | 'connectionId' | 'executionHostId'
>
export type DirectSshTargetScopeInput = {
targetId: string
catalogRevision: number
repos: readonly DirectSshRepoOwner[]
worktreesByRepo?: Readonly<Record<string, readonly DirectSshWorktreeOwner[]>>
detectedWorktreesByRepo?: Readonly<
Record<string, { worktrees: readonly DirectSshWorktreeOwner[] }>
>
folderWorkspaces?: readonly DirectSshFolderOwner[]
projectGroups?: readonly DirectSshGroupOwner[]
restoredRuntimeHostIdByWorkspaceSessionKey?: Readonly<Record<string, string | null | undefined>>
}
src/renderer/src/lib/direct-ssh-target-scope.test.ts
+287
View File
@@ -0,0 +1,287 @@
import { describe, expect, it } from 'vitest'
import { folderWorkspaceKey, worktreeWorkspaceKey } from '../../../shared/workspace-scope'
import {
resolveDirectSshTargetScope,
type DirectSshTargetScopeInput
} from './direct-ssh-target-scope'
const baseInput: DirectSshTargetScopeInput = {
targetId: 'target-a',
catalogRevision: 17,
repos: []
}
describe('resolveDirectSshTargetScope', () => {
it('resolves duplicate repo IDs by exact SSH host instead of first wins', () => {
const scope = resolveDirectSshTargetScope({
...baseInput,
repos: [
{
id: 'shared',
path: '/local/shared',
projectGroupId: null,
connectionId: null,
executionHostId: 'local'
},
{
id: 'shared',
path: '/remote/shared',
projectGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
}
],
worktreesByRepo: {
shared: [
{
id: 'shared::remote',
repoId: 'shared',
hostId: 'ssh:target-a'
}
]
}
})
expect(scope.catalogRevision).toBe(17)
expect(scope.gitRepos).toEqual([{ repoId: 'shared', executionHostId: 'ssh:target-a' }])
expect(scope.gitWorktreeIds).toEqual(new Set(['shared::remote']))
expect(scope.terminalWorkspaceKeys).toEqual(new Set(['shared::remote']))
expect(scope.lineageWorkspaceKeys).toEqual(new Set([worktreeWorkspaceKey('shared::remote')]))
expect(scope.ambiguousOwnerCount).toBe(0)
expect(scope.contradictoryOwnerCount).toBe(0)
})
it('keeps duplicate same-host repo owners ambiguous', () => {
const duplicate = {
id: 'duplicate',
path: '/remote/duplicate',
projectGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a' as const
}
const scope = resolveDirectSshTargetScope({
...baseInput,
repos: [duplicate, { ...duplicate }],
worktreesByRepo: {
duplicate: [
{
id: 'duplicate::worktree',
repoId: 'duplicate',
hostId: 'ssh:target-a'
}
]
}
})
expect(scope.gitRepos).toEqual([])
expect(scope.gitWorktreeIds.size).toBe(0)
expect(scope.terminalWorkspaceKeys.size).toBe(0)
expect(scope.ambiguousOwnerCount).toBe(2)
})
it('gives folder workspaces the same exact target isolation as Git worktrees', () => {
const scope = resolveDirectSshTargetScope({
...baseInput,
repos: [
{
id: 'remote-repo',
path: '/srv/project/repo',
projectGroupId: 'remote-group',
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
}
],
worktreesByRepo: {
'remote-repo': [
{
id: 'remote-repo::worktree',
repoId: 'remote-repo',
hostId: 'ssh:target-a'
}
]
},
projectGroups: [
{
id: 'remote-group',
parentGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
}
],
folderWorkspaces: [
{
id: 'remote-folder',
projectGroupId: 'remote-group',
folderPath: '/srv/project',
connectionId: 'target-a'
}
]
})
expect(scope.gitWorktreeIds).toEqual(new Set(['remote-repo::worktree']))
expect(scope.terminalWorkspaceKeys).toEqual(
new Set(['remote-repo::worktree', 'folder:remote-folder'])
)
expect(scope.lineageWorkspaceKeys).toEqual(
new Set([worktreeWorkspaceKey('remote-repo::worktree'), folderWorkspaceKey('remote-folder')])
)
})
it('keeps runtime-owned SSH work isolated from the direct target', () => {
const scope = resolveDirectSshTargetScope({
...baseInput,
repos: [
{
id: 'runtime-ssh',
path: '/runtime/repo',
projectGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
}
],
worktreesByRepo: {
'runtime-ssh': [
{
id: 'runtime-ssh::worktree',
repoId: 'runtime-ssh',
hostId: 'ssh:target-a',
runtimeOwnerEnvironmentId: 'hub-a'
}
]
}
})
expect(scope.gitWorktreeIds.size).toBe(0)
expect(scope.terminalWorkspaceKeys.size).toBe(0)
expect(scope.contradictoryOwnerCount).toBe(1)
})
it('rejects explicit worktree and exact repo ownership contradictions', () => {
const scope = resolveDirectSshTargetScope({
...baseInput,
repos: [
{
id: 'repo',
path: '/target-a/repo',
projectGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
}
],
worktreesByRepo: {
repo: [{ id: 'repo::wrong-host', repoId: 'repo', hostId: 'ssh:target-b' }]
}
})
expect(scope.gitRepos).toEqual([{ repoId: 'repo', executionHostId: 'ssh:target-a' }])
expect(scope.gitWorktreeIds.size).toBe(0)
expect(scope.contradictoryOwnerCount).toBe(1)
})
it('excludes ambiguous legacy and duplicate same-host folder owners', () => {
const scope = resolveDirectSshTargetScope({
...baseInput,
repos: [],
worktreesByRepo: {
legacy: [{ id: 'legacy::worktree', repoId: 'legacy' }],
unresolved: [
{
id: 'unresolved::worktree',
repoId: 'unresolved',
hostId: 'runtime:unresolved-owner'
}
]
},
projectGroups: [
{
id: 'group',
parentGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
}
],
folderWorkspaces: [
{
id: 'duplicate-folder',
projectGroupId: 'group',
folderPath: '/srv/project',
connectionId: 'target-a'
},
{
id: 'duplicate-folder',
projectGroupId: 'group',
folderPath: '/srv/project',
connectionId: 'target-a'
}
]
})
expect(scope.gitWorktreeIds.size).toBe(0)
expect(scope.terminalWorkspaceKeys.size).toBe(0)
expect(scope.ambiguousOwnerCount).toBe(3)
expect(scope.contradictoryOwnerCount).toBe(0)
})
it('rejects conflicting folder workspace, group, repo, and restored provenance', () => {
const scope = resolveDirectSshTargetScope({
...baseInput,
repos: [
{
id: 'repo-b',
path: '/srv/project/repo',
projectGroupId: 'group',
connectionId: 'target-b',
executionHostId: 'ssh:target-b'
}
],
projectGroups: [
{
id: 'group',
parentGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
}
],
folderWorkspaces: [
{
id: 'mixed-folder',
projectGroupId: 'group',
folderPath: '/srv/project',
connectionId: 'target-a'
}
],
restoredRuntimeHostIdByWorkspaceSessionKey: {
'folder:mixed-folder': 'runtime:hub-a'
}
})
expect(scope.terminalWorkspaceKeys.size).toBe(0)
expect(scope.contradictoryOwnerCount).toBe(1)
})
it('does not infer an effective folder connection from an SSH host stamp alone', () => {
const scope = resolveDirectSshTargetScope({
...baseInput,
repos: [],
projectGroups: [
{
id: 'legacy-group',
parentGroupId: null,
connectionId: null,
executionHostId: 'ssh:target-a'
}
],
folderWorkspaces: [
{
id: 'legacy-folder',
projectGroupId: 'legacy-group',
folderPath: '/srv/legacy',
connectionId: null
}
]
})
expect(scope.terminalWorkspaceKeys.size).toBe(0)
expect(scope.ambiguousOwnerCount).toBe(1)
})
})
src/renderer/src/lib/direct-ssh-target-scope.ts
+310
View File
@@ -0,0 +1,310 @@
import {
getRepoExecutionHostId,
parseExecutionHostId,
toRuntimeExecutionHostId,
toSshExecutionHostId,
type ExecutionHostId
} from '../../../shared/execution-host'
import { isPathInsideOrEqual } from '../../../shared/cross-platform-path'
import { getProjectGroupSubtreeIds } from '../../../shared/project-groups'
import { folderWorkspaceKey, worktreeWorkspaceKey } from '../../../shared/workspace-scope'
import type {
DirectSshFolderOwner as FolderOwner,
DirectSshGitRepoRef,
DirectSshGroupOwner as GroupOwner,
DirectSshRepoOwner as RepoOwner,
DirectSshTargetScope,
DirectSshTargetScopeInput,
DirectSshWorktreeOwner as WorktreeOwner
} from './direct-ssh-target-scope-types'
import { indexDirectSshOwnerRows } from './direct-ssh-target-owner-index'
export type {
DirectSshGitRepoRef,
DirectSshTargetScope,
DirectSshTargetScopeInput
} from './direct-ssh-target-scope-types'
type HostEvidence = {
hosts: Set<ExecutionHostId>
ambiguous: boolean
contradictory: boolean
}
function newHostEvidence(): HostEvidence {
return { hosts: new Set(), ambiguous: false, contradictory: false }
}
function addHostEvidence(evidence: HostEvidence, rawHostId: string | null | undefined): void {
if (!rawHostId?.trim()) {
return
}
const host = parseExecutionHostId(rawHostId)
if (!host) {
evidence.ambiguous = true
return
}
if (host.kind === 'runtime' && host.environmentId === 'unresolved-owner') {
evidence.ambiguous = true
return
}
evidence.hosts.add(host.id)
}
function resolveRepoEvidence(repo: RepoOwner): HostEvidence {
const evidence = newHostEvidence()
addHostEvidence(evidence, repo.executionHostId)
if (repo.connectionId?.trim()) {
evidence.hosts.add(toSshExecutionHostId(repo.connectionId.trim()))
}
evidence.hosts =
evidence.hosts.size === 0 && !evidence.ambiguous
? new Set([getRepoExecutionHostId(repo)])
: evidence.hosts
evidence.contradictory = evidence.hosts.size > 1
return evidence
}
function collectWorktreeRows(input: DirectSshTargetScopeInput): Map<string, WorktreeOwner[]> {
return indexDirectSshOwnerRows([
...Object.values(input.worktreesByRepo ?? {}).flat(),
...Object.values(input.detectedWorktreesByRepo ?? {}).flatMap((result) => result.worktrees)
])
}
function addRepoDerivedEvidence(
evidence: HostEvidence,
repoRows: readonly RepoOwner[],
explicitHosts: ReadonlySet<ExecutionHostId>
): void {
const repoHosts = new Set<ExecutionHostId>()
const repoHostCounts = new Map<ExecutionHostId, number>()
let hasInvalidRepo = false
for (const repo of repoRows) {
const repoEvidence = resolveRepoEvidence(repo)
hasInvalidRepo ||= repoEvidence.ambiguous
evidence.contradictory ||= repoEvidence.contradictory
for (const host of repoEvidence.hosts) {
repoHosts.add(host)
repoHostCounts.set(host, (repoHostCounts.get(host) ?? 0) + 1)
}
}
if (explicitHosts.size > 0) {
const exactHosts = [...explicitHosts].filter((host) => repoHosts.has(host))
if (exactHosts.length > 0) {
for (const host of exactHosts) {
evidence.hosts.add(host)
evidence.ambiguous ||= (repoHostCounts.get(host) ?? 0) > 1
}
} else if (repoHosts.size === 1) {
evidence.hosts.add([...repoHosts][0])
} else if (repoHosts.size > 1) {
evidence.ambiguous = true
}
} else if (repoHosts.size === 1) {
const repoHost = [...repoHosts][0]
evidence.hosts.add(repoHost)
evidence.ambiguous ||= (repoHostCounts.get(repoHost) ?? 0) > 1
} else if (repoHosts.size > 1) {
evidence.ambiguous = true
}
evidence.ambiguous ||= hasInvalidRepo
}
function resolveWorktreeEvidence(
input: DirectSshTargetScopeInput,
rows: readonly WorktreeOwner[],
repoRowsById: ReadonlyMap<string, readonly RepoOwner[]>
): HostEvidence {
const evidence = newHostEvidence()
const repoIds = new Set(rows.map((row) => row.repoId))
evidence.ambiguous ||= repoIds.size !== 1
const explicitHosts = new Set<ExecutionHostId>()
for (const row of rows) {
const parsedHost = parseExecutionHostId(row.hostId)
if (row.hostId?.trim() && !parsedHost) {
evidence.ambiguous = true
} else if (parsedHost?.kind === 'runtime' && parsedHost.environmentId === 'unresolved-owner') {
evidence.ambiguous = true
} else if (parsedHost) {
explicitHosts.add(parsedHost.id)
evidence.hosts.add(parsedHost.id)
}
const runtimeOwner = row.runtimeOwnerEnvironmentId?.trim()
if (runtimeOwner) {
evidence.hosts.add(toRuntimeExecutionHostId(runtimeOwner))
}
}
for (const repoId of repoIds) {
const repoRows = repoRowsById.get(repoId)
if (repoRows) {
addRepoDerivedEvidence(evidence, repoRows, explicitHosts)
} else if (explicitHosts.size === 0) {
evidence.ambiguous = true
}
}
const restored = input.restoredRuntimeHostIdByWorkspaceSessionKey
addHostEvidence(evidence, restored?.[rows[0].id])
addHostEvidence(evidence, restored?.[worktreeWorkspaceKey(rows[0].id)])
evidence.contradictory ||= evidence.hosts.size > 1
return evidence
}
function getFolderCandidateRepos(
folder: FolderOwner,
groups: readonly GroupOwner[],
repos: readonly RepoOwner[],
scopeConnectionId: string | null
): RepoOwner[] {
const groupIds = getProjectGroupSubtreeIds(groups, folder.projectGroupId)
const groupRepos = repos.filter(
(repo) => typeof repo.projectGroupId === 'string' && groupIds.has(repo.projectGroupId)
)
const pathRepos = repos.filter(
(repo) =>
!(typeof repo.projectGroupId === 'string' && groupIds.has(repo.projectGroupId)) &&
isPathInsideOrEqual(folder.folderPath, repo.path)
)
if (scopeConnectionId) {
return [
...groupRepos,
...pathRepos.filter((repo) => (repo.connectionId ?? null) === scopeConnectionId)
]
}
if (groupRepos.length === 0) {
return pathRepos
}
const groupConnections = new Set(groupRepos.map((repo) => repo.connectionId ?? null))
return [
...groupRepos,
...pathRepos.filter((repo) => groupConnections.has(repo.connectionId ?? null))
]
}
function resolveFolderEvidence(
input: DirectSshTargetScopeInput,
folder: FolderOwner,
group: GroupOwner | undefined
): HostEvidence {
const evidence = newHostEvidence()
const folderConnection = folder.connectionId?.trim() || null
const groupConnection = group?.connectionId?.trim() || null
if (folderConnection) {
evidence.hosts.add(toSshExecutionHostId(folderConnection))
}
if (groupConnection) {
evidence.hosts.add(toSshExecutionHostId(groupConnection))
}
addHostEvidence(evidence, group?.executionHostId)
addHostEvidence(
evidence,
input.restoredRuntimeHostIdByWorkspaceSessionKey?.[folderWorkspaceKey(folder.id)]
)
const scopeConnection = folderConnection ?? groupConnection
const candidateRepos = getFolderCandidateRepos(
folder,
input.projectGroups ?? [],
input.repos,
scopeConnection
)
const repoOwnerKeys = new Set<string>()
for (const repo of candidateRepos) {
const repoEvidence = resolveRepoEvidence(repo)
evidence.ambiguous ||= repoEvidence.ambiguous
evidence.contradictory ||= repoEvidence.contradictory
for (const host of repoEvidence.hosts) {
const ownerKey = JSON.stringify([repo.id, host])
evidence.ambiguous ||= repoOwnerKeys.has(ownerKey)
repoOwnerKeys.add(ownerKey)
evidence.hosts.add(host)
}
}
const hasSshOwner = [...evidence.hosts].some(
(hostId) => parseExecutionHostId(hostId)?.kind === 'ssh'
)
const hasConnectionOwner =
Boolean(scopeConnection) || candidateRepos.some((repo) => Boolean(repo.connectionId?.trim()))
evidence.ambiguous ||= !group || (hasSshOwner && !hasConnectionOwner)
evidence.contradictory ||= evidence.hosts.size > 1
return evidence
}
export function resolveDirectSshTargetScope(
input: DirectSshTargetScopeInput
): DirectSshTargetScope {
const expectedHost = toSshExecutionHostId(input.targetId)
const repoRowsById = indexDirectSshOwnerRows(input.repos)
const gitRepos: DirectSshGitRepoRef[] = []
let ambiguousOwnerCount = 0
let contradictoryOwnerCount = 0
for (const [repoId, rows] of repoRowsById) {
const matchingRows = rows.filter((repo) => {
const evidence = resolveRepoEvidence(repo)
if (evidence.contradictory) {
contradictoryOwnerCount++
return false
}
if (evidence.ambiguous) {
ambiguousOwnerCount++
return false
}
return evidence.hosts.has(expectedHost)
})
if (matchingRows.length === 1) {
gitRepos.push({ repoId, executionHostId: expectedHost })
} else if (matchingRows.length > 1) {
ambiguousOwnerCount++
}
}
const gitWorktreeIds = new Set<string>()
const terminalWorkspaceKeys = new Set<string>()
const lineageWorkspaceKeys = new Set<ReturnType<typeof worktreeWorkspaceKey>>()
for (const [worktreeId, rows] of collectWorktreeRows(input)) {
const evidence = resolveWorktreeEvidence(input, rows, repoRowsById)
if (evidence.contradictory) {
contradictoryOwnerCount++
} else if (evidence.ambiguous || evidence.hosts.size === 0) {
ambiguousOwnerCount++
} else if (evidence.hosts.has(expectedHost)) {
gitWorktreeIds.add(worktreeId)
terminalWorkspaceKeys.add(worktreeId)
lineageWorkspaceKeys.add(worktreeWorkspaceKey(worktreeId))
}
}
const folderRowsById = indexDirectSshOwnerRows(input.folderWorkspaces ?? [])
const groupRowsById = indexDirectSshOwnerRows(input.projectGroups ?? [])
for (const [folderId, rows] of folderRowsById) {
if (rows.length !== 1) {
ambiguousOwnerCount++
continue
}
const groupRows = groupRowsById.get(rows[0].projectGroupId) ?? []
if (groupRows.length > 1) {
ambiguousOwnerCount++
continue
}
const evidence = resolveFolderEvidence(input, rows[0], groupRows[0])
if (evidence.contradictory) {
contradictoryOwnerCount++
} else if (evidence.ambiguous || evidence.hosts.size === 0) {
ambiguousOwnerCount++
} else if (evidence.hosts.has(expectedHost)) {
const workspaceKey = folderWorkspaceKey(folderId)
terminalWorkspaceKeys.add(workspaceKey)
lineageWorkspaceKeys.add(workspaceKey)
}
}
return {
catalogRevision: input.catalogRevision,
gitRepos,
gitWorktreeIds,
terminalWorkspaceKeys,
lineageWorkspaceKeys,
ambiguousOwnerCount,
contradictoryOwnerCount
}
}
src/renderer/src/lib/workspace-session-hydration-keys.ts
+1
View File
@@ -6,6 +6,7 @@ import { getRepoIdFromWorktreeId } from '../../../shared/worktree-id'
export type WorkspaceSessionHydrationOptions = {
additionalValidWorkspaceKeys?: readonly WorkspaceKey[]
replaceWorkspaceKeys?: readonly string[]
}
// Worktree-keyed fields carrying restorable chrome — a repo appears here only if it has live
src/renderer/src/runtime/runtime-client-events.test.ts
+72 -5
View File
@@ -1,4 +1,5 @@
import { describe, expect, it, vi } from 'vitest'
import type { SshProviderEpoch } from '../../../shared/ssh-types'
import { subscribeRuntimeClientEvents } from './runtime-client-events'
import { replaceRuntimeEnvironmentRevisions } from './runtime-environment-revision'
@@ -137,7 +138,7 @@ describe('subscribeRuntimeClientEvents', () => {
])
})
it('applies the redacted SSH snapshot from the ready frame', async () => {
it('preserves full SSH authority in retained snapshots and live client events', async () => {
let capturedOnResponse: ((response: unknown) => void) | undefined
const subscribe = vi.fn(async (_args, nextCallbacks) => {
capturedOnResponse = (nextCallbacks as { onResponse: (response: unknown) => void }).onResponse
@@ -160,9 +161,11 @@ describe('subscribeRuntimeClientEvents', () => {
targetId: 'ssh-1',
state: {
targetId: 'ssh-1',
status: 'disconnected',
status: 'connected',
error: null,
reconnectAttempt: 0
reconnectAttempt: 0,
providerEpoch: 'snapshot-provider-epoch' as SshProviderEpoch,
connectionGeneration: 17
}
}
]
@@ -175,10 +178,74 @@ describe('subscribeRuntimeClientEvents', () => {
targetId: 'ssh-1',
state: {
targetId: 'ssh-1',
status: 'disconnected',
status: 'connected',
error: null,
reconnectAttempt: 0
reconnectAttempt: 0,
providerEpoch: 'snapshot-provider-epoch',
connectionGeneration: 17
}
})
capturedOnResponse({
ok: true,
result: {
type: 'sshStateChanged',
targetId: 'ssh-1',
state: {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'live-provider-epoch' as SshProviderEpoch,
connectionGeneration: 18
}
}
})
expect(onEvent).toHaveBeenLastCalledWith({
type: 'sshStateChanged',
targetId: 'ssh-1',
state: {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'live-provider-epoch',
connectionGeneration: 18
}
})
})
it('rejects a partial runtime authority instead of retaining it', async () => {
let capturedOnResponse: ((response: unknown) => void) | undefined
const subscribe = vi.fn(async (_args, nextCallbacks) => {
capturedOnResponse = (nextCallbacks as { onResponse: (response: unknown) => void }).onResponse
return { subscriptionId: 'sub-1', unsubscribe: vi.fn() }
})
const onEvent = vi.fn()
const onError = vi.fn()
vi.stubGlobal('window', { api: { runtimeEnvironments: { subscribe } } })
await subscribeRuntimeClientEvents('env-1', onEvent, onError)
if (!capturedOnResponse) {
throw new Error('Expected subscription callbacks')
}
capturedOnResponse({
ok: true,
result: {
type: 'sshStateChanged',
targetId: 'ssh-1',
state: {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'partial-provider-epoch'
}
}
})
expect(onEvent).not.toHaveBeenCalled()
expect(onError).toHaveBeenCalledWith(expect.objectContaining({ message: expect.any(String) }))
})
})
src/renderer/src/runtime/runtime-client-events.ts
+16 -1
View File
@@ -4,6 +4,7 @@ import type {
} from '../../../shared/runtime-client-events'
import type { RuntimeRpcResponse } from '../../../shared/runtime-rpc-envelope'
import { isRuntimeSubscriptionReplayResponse } from '../../../shared/runtime-subscription-replay'
import { admitSshConnectionState } from '../../../shared/ssh-retained-payload-admission'
import { getRuntimeEnvironmentRevision } from './runtime-environment-revision'
export type RuntimeClientEventSubscription = {
@@ -53,13 +54,27 @@ function handleRuntimeClientEventResponse(
const message = response.result as RuntimeClientEventStreamMessage
if (message.type === 'ready') {
for (const sshState of message.snapshot?.sshStates ?? []) {
onEvent({ type: 'sshStateChanged', ...sshState })
const state = admitSshConnectionState(sshState.state, sshState.targetId)
if (state) {
onEvent({ type: 'sshStateChanged', targetId: sshState.targetId, state })
} else {
onError(new Error('Invalid retained SSH connection state'))
}
}
return
}
if (message.type === 'end') {
return
}
if (message.type === 'sshStateChanged') {
const state = admitSshConnectionState(message.state, message.targetId)
if (state) {
onEvent({ type: 'sshStateChanged', targetId: message.targetId, state })
} else {
onError(new Error('Invalid retained SSH connection state'))
}
return
}
if (isRuntimeClientEvent(message)) {
onEvent(message)
}
src/renderer/src/runtime/runtime-environment-ssh-state.test.ts
+44 -4
View File
@@ -1,5 +1,5 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
import type { SshConnectionState } from '../../../shared/ssh-types'
import type { SshConnectionState, SshProviderEpoch } from '../../../shared/ssh-types'
import { useAppStore } from '@/store'
import {
applyRuntimeEnvironmentSshStateChanged,
@@ -20,7 +20,14 @@ function connState(
targetId: string,
status: SshConnectionState['status'] = 'connected'
): SshConnectionState {
return { targetId, status, error: null, reconnectAttempt: 0 }
return {
targetId,
status,
error: null,
reconnectAttempt: 0,
providerEpoch: `${targetId}-provider-epoch` as SshProviderEpoch,
connectionGeneration: 7
}
}
type RpcResponses = {
@@ -84,6 +91,10 @@ describe('hydrateRuntimeEnvironmentSshState', () => {
expect(bucket?.targetLabels.get('ssh-1')).toBe('devbox')
expect(bucket?.removedTargetLabels.get('ssh-old')).toBe('retired box')
expect(bucket?.connectionStates.get('ssh-1')?.status).toBe('connected')
expect(bucket?.connectionStates.get('ssh-1')).toMatchObject({
providerEpoch: 'ssh-1-provider-epoch',
connectionGeneration: 7
})
// ssh-2 had no live state: absent, so reads fall back to 'disconnected'.
expect(bucket?.connectionStates.has('ssh-2')).toBe(false)
// Local maps stay untouched.
@@ -219,13 +230,36 @@ describe('applyRuntimeEnvironmentSshStateChanged', () => {
applyRuntimeEnvironmentSshStateChanged(envId, 'ssh-1', connState('ssh-1', 'disconnected'))
expect(
useAppStore.getState().sshStateByEnvironment.get(envId)?.connectionStates.get('ssh-1')?.status
).toBe('disconnected')
useAppStore.getState().sshStateByEnvironment.get(envId)?.connectionStates.get('ssh-1')
).toMatchObject({
status: 'disconnected',
providerEpoch: 'ssh-1-provider-epoch',
connectionGeneration: 7
})
expect(callRuntimeRpcMock).not.toHaveBeenCalled()
// Local map untouched.
expect(useAppStore.getState().sshConnectionStates.size).toBe(0)
})
it('rejects partial authority before retaining a runtime-owned state', () => {
const envId = nextEnvId()
useAppStore
.getState()
.setEnvironmentSshTargetsMetadata(envId, [{ id: 'ssh-1', label: 'devbox' }])
applyRuntimeEnvironmentSshStateChanged(envId, 'ssh-1', {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'partial-provider-epoch' as SshProviderEpoch
})
expect(
useAppStore.getState().sshStateByEnvironment.get(envId)?.connectionStates.has('ssh-1')
).toBe(false)
})
it('does not touch another environment bucket or local state (no cross-pollution)', () => {
const envA = nextEnvId()
const envB = nextEnvId()
@@ -300,6 +334,12 @@ describe('connectRuntimeEnvironmentSshTarget', () => {
expect(
useAppStore.getState().sshStateByEnvironment.get(envId)?.connectionStates.get('ssh-1')?.status
).toBe('connected')
expect(
useAppStore.getState().sshStateByEnvironment.get(envId)?.connectionStates.get('ssh-1')
).toMatchObject({
providerEpoch: 'ssh-1-provider-epoch',
connectionGeneration: 7
})
expect(useAppStore.getState().sshConnectionStates.size).toBe(0)
})
src/renderer/src/runtime/runtime-environment-ssh-state.ts
+13 -6
View File
@@ -2,6 +2,7 @@ import { useAppStore } from '@/store'
import type { SshConnectionState, SshTargetSummary } from '../../../shared/ssh-types'
import { callRuntimeRpc } from './runtime-rpc-client'
import { getEnvironmentSshStateGeneration } from '@/store/slices/runtime-environment-ssh'
import { admitSshConnectionState } from '../../../shared/ssh-retained-payload-admission'
/**
* Mirrors a remote Orca server's own SSH targets into that environment's
@@ -71,10 +72,11 @@ async function fetchEnvironmentSshConnectionStates(
{ targetId: target.id },
{ timeoutMs: SSH_RPC_TIMEOUT_MS }
)
if (state) {
const admittedState = state ? admitSshConnectionState(state, target.id) : null
if (admittedState) {
useAppStore
.getState()
.setEnvironmentSshConnectionState(environmentId, target.id, state, generation)
.setEnvironmentSshConnectionState(environmentId, target.id, admittedState, generation)
}
} catch {
// Why: a timeout or unsupported RPC is not authoritative evidence that the HUB's SSH link disconnected.
@@ -157,10 +159,14 @@ export function applyRuntimeEnvironmentSshStateChanged(
if (generation !== getEnvironmentSshStateGeneration(environmentId)) {
return
}
const admittedState = admitSshConnectionState(state, targetId)
if (!admittedState) {
return
}
const store = useAppStore.getState()
const bucket = store.sshStateByEnvironment.get(environmentId)
if (bucket?.targetsHydrated && bucket.targetLabels.has(targetId)) {
store.setEnvironmentSshConnectionState(environmentId, targetId, state, generation)
store.setEnvironmentSshConnectionState(environmentId, targetId, admittedState, generation)
return
}
void hydrateRuntimeEnvironmentSshState(environmentId, { force: true }).catch(() => {})
@@ -180,12 +186,13 @@ export async function connectRuntimeEnvironmentSshTarget(
{ targetId },
{ timeoutMs: 60_000 }
)
if (state) {
const admittedState = state ? admitSshConnectionState(state, targetId) : null
if (admittedState) {
useAppStore
.getState()
.setEnvironmentSshConnectionState(environmentId, targetId, state, generation)
.setEnvironmentSshConnectionState(environmentId, targetId, admittedState, generation)
}
return state
return admittedState
}
/** Resyncs the environment's target metadata after a failed connect so a
src/renderer/src/startup/ssh-startup-reconnect.test.ts
+29 -1
View File
@@ -1,5 +1,5 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { SshConnectionState } from '../../../shared/ssh-types'
import type { SshConnectionState, SshProviderEpoch } from '../../../shared/ssh-types'
import { reconnectSshTargetForRendererStartup } from './ssh-startup-reconnect'
const connectedState: SshConnectionState = {
@@ -7,6 +7,8 @@ const connectedState: SshConnectionState = {
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'startup-provider-epoch' as SshProviderEpoch,
connectionGeneration: 41,
remotePlatform: 'linux'
}
@@ -27,6 +29,32 @@ describe('reconnectSshTargetForRendererStartup', () => {
expect(result).toEqual({ timedOut: false })
expect(publishState).toHaveBeenCalledWith('ssh-1', connectedState)
expect(publishState.mock.calls[0]?.[1]).toMatchObject({
providerEpoch: 'startup-provider-epoch',
connectionGeneration: 41
})
})
it('does not synthesize the missing half of a partial startup authority', async () => {
const publishState = vi.fn()
const partialState = {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'partial-provider-epoch' as SshProviderEpoch
} satisfies SshConnectionState
await reconnectSshTargetForRendererStartup({
targetId: 'ssh-1',
timeoutMs: 1_000,
connect: vi.fn().mockResolvedValue(partialState),
publishState,
onFailure: vi.fn()
})
expect(publishState).toHaveBeenCalledWith('ssh-1', partialState)
expect(publishState.mock.calls[0]?.[1]).not.toHaveProperty('connectionGeneration')
})
it('marks a stalled connect as deferred without publishing stale state', async () => {
src/renderer/src/store/slices/bulk-worktree-purge-terminal-maps-leak.test.ts
+55
View File
@@ -49,6 +49,8 @@ globalThis.window = { api: mockApi }
import { createTestStore, seedStore, makeWorktree, makeTab } from './store-test-helpers'
import { parseRemoteRuntimePtyId } from '@/runtime/runtime-terminal-stream'
import type { SshProviderEpoch } from '../../../../shared/ssh-types'
import type { DirectSshPaneRetryAttemptId } from './direct-ssh-terminal-recovery'
const WT1 = 'repo1::/path/wt1'
const WT2 = 'repo1::/path/wt2'
@@ -62,6 +64,11 @@ const REMOTE_PTY2_SAME_HANDLE = 'remote:env-2@@terminal-wt1'
const PTY2 = 'pty-wt2'
function seedMaps(store: ReturnType<typeof createTestStore>): void {
const authority = {
targetId: 'target-a',
providerEpoch: 'epoch-a' as SshProviderEpoch,
connectionGeneration: 1
}
seedStore(store, {
worktreesByRepo: {
repo1: [
@@ -133,6 +140,32 @@ function seedMaps(store: ReturnType<typeof createTestStore>): void {
[PTY1_SPLIT]: true,
[REMOTE_PTY1]: true,
[PTY2]: true
},
directSshPaneRetryByTabId: {
[TAB1]: {
attemptId: 'attempt-1' as DirectSshPaneRetryAttemptId,
authority,
tabGeneration: 0,
startedAt: 1
}
},
directSshLivePtyBindingByTabId: {
[TAB1]: {
attemptId: 'attempt-1' as DirectSshPaneRetryAttemptId,
authority,
tabGeneration: 0,
ptyId: PTY1
},
[TAB2]: {
attemptId: 'attempt-2' as DirectSshPaneRetryAttemptId,
authority,
tabGeneration: 0,
ptyId: PTY2
}
},
directSshPaneRetryHistoryByTabId: {
[TAB1]: { authority, attemptedAt: [1] },
[TAB2]: { authority, attemptedAt: [2] }
}
})
}
@@ -163,6 +196,9 @@ describe('bulk worktree purge evicts the per-tab/per-pty terminal maps it previo
expect(s.pendingCodexPaneRestartIds[PTY1]).toBeUndefined()
expect(s.pendingCodexPaneRestartIds[PTY1_SPLIT]).toBeUndefined()
expect(s.pendingCodexPaneRestartIds[REMOTE_PTY1]).toBeUndefined()
expect(s.directSshPaneRetryByTabId[TAB1]).toBeUndefined()
expect(s.directSshLivePtyBindingByTabId[TAB1]).toBeUndefined()
expect(s.directSshPaneRetryHistoryByTabId[TAB1]).toBeUndefined()
// Surviving worktree's tab/pty: every entry retained (no over-eviction).
expect(s.lastKnownRelayPtyIdByTabId[TAB2]).toBe(PTY2)
@@ -176,6 +212,8 @@ describe('bulk worktree purge evicts the per-tab/per-pty terminal maps it previo
expect(s.migrationUnsupportedByPtyId[PTY2]).toBeDefined()
expect(s.suppressedPtyExitIds[PTY2]).toBe(true)
expect(s.pendingCodexPaneRestartIds[PTY2]).toBe(true)
expect(s.directSshLivePtyBindingByTabId[TAB2]).toBeDefined()
expect(s.directSshPaneRetryHistoryByTabId[TAB2]).toBeDefined()
})
it('keeps environment-scoped remote guard identities independent', () => {
@@ -244,4 +282,21 @@ describe('bulk worktree purge evicts the per-tab/per-pty terminal maps it previo
// exact identities without parsing aliases or scanning surviving terminals.
expect(parseRemotePtyId).not.toHaveBeenCalled()
})
it('preserves direct SSH ledgers when a surviving worktree reuses the tab id', () => {
const store = createTestStore()
seedMaps(store)
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WT2]: [...state.tabsByWorktree[WT2], makeTab({ id: TAB1, worktreeId: WT2, ptyId: PTY2 })]
}
}))
store.getState().purgeWorktreeTerminalState([WT1])
expect(store.getState().directSshPaneRetryByTabId[TAB1]).toBeDefined()
expect(store.getState().directSshLivePtyBindingByTabId[TAB1]).toBeDefined()
expect(store.getState().directSshPaneRetryHistoryByTabId[TAB1]).toBeDefined()
})
})
src/renderer/src/store/slices/detected-worktree-refresh-leases.test.ts
+271
View File
@@ -0,0 +1,271 @@
import { describe, expect, it, vi } from 'vitest'
import type {
HostQualifiedDetectedWorktreeResult,
ListDetectedWorktreesArgs
} from '../../../../shared/detected-worktree-provider-contract'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../../shared/ssh-types'
import {
createDetectedWorktreeRefreshLeaseRegistry,
type DetectedWorktreeRefreshProviderInput
} from './detected-worktree-refresh-leases'
type ControlledPromise<T> = {
promise: Promise<T>
resolve(value: T): void
reject(error: unknown): void
}
function controlledPromise<T>(): ControlledPromise<T> {
let resolve!: (value: T) => void
let reject!: (error: unknown) => void
const promise = new Promise<T>((resolvePromise, rejectPromise) => {
resolve = resolvePromise
reject = rejectPromise
})
return { promise, resolve, reject }
}
function authority(providerEpoch: string, connectionGeneration = 1): DirectSshAuthority {
return {
targetId: 'ssh-a',
providerEpoch: providerEpoch as SshProviderEpoch,
connectionGeneration
}
}
function directInput(
expectedAuthority = authority('epoch-a')
): DetectedWorktreeRefreshProviderInput {
return {
repoId: 'repo-a',
executionHostId: 'ssh:ssh-a',
expectedAuthority
}
}
function completeResult(request: ListDetectedWorktreesArgs): HostQualifiedDetectedWorktreeResult {
if (!('expectedAuthority' in request)) {
throw new Error('Expected direct SSH request')
}
return {
status: 'complete',
providerRequestId: request.providerRequestId,
repoId: request.repoId,
authority: {
kind: 'direct-ssh',
executionHostId: request.executionHostId,
...request.expectedAuthority
},
result: {
repoId: request.repoId,
authoritative: true,
source: 'git',
worktrees: []
}
}
}
function createHarness() {
const pending: ControlledPromise<HostQualifiedDetectedWorktreeResult>[] = []
const starts: ListDetectedWorktreesArgs[] = []
const cancelProviderRequest = vi.fn()
const registry = createDetectedWorktreeRefreshLeaseRegistry({
startProviderRequest: (request) => {
starts.push(request)
const provider = controlledPromise<HostQualifiedDetectedWorktreeResult>()
pending.push(provider)
return provider.promise
},
cancelProviderRequest
})
return { cancelProviderRequest, pending, registry, starts }
}
describe('detected worktree refresh leases', () => {
it('joins exact inputs with distinct waiter and shared provider identities', () => {
const { registry, starts } = createHarness()
const first = registry.acquire('repo-a:ssh:ssh-a', directInput())
const second = registry.acquire('repo-a:ssh:ssh-a', directInput())
expect(starts).toHaveLength(1)
expect(first.providerRequestId).toBe(second.providerRequestId)
expect(first.waiterLeaseId).not.toBe(second.waiterLeaseId)
expect(first.waiterLeaseId).not.toBe(first.providerRequestId)
expect(starts[0]).not.toHaveProperty('waiterLeaseId')
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 1,
waiterLeases: 2
})
})
it('settles only the released waiter while another lease remains live', async () => {
const { cancelProviderRequest, pending, registry, starts } = createHarness()
const first = registry.acquire('repo-a:ssh:ssh-a', directInput())
const second = registry.acquire('repo-a:ssh:ssh-a', directInput())
expect(first.release('superseded')).toBe('retained')
await expect(first.result).resolves.toEqual({
providerRequestId: first.providerRequestId,
executionHostId: 'ssh:ssh-a',
status: 'canceled'
})
expect(cancelProviderRequest).not.toHaveBeenCalled()
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 1,
waiterLeases: 1
})
const providerResult = completeResult(starts[0])
pending[0].resolve(providerResult)
await expect(second.result).resolves.toEqual(providerResult)
await expect(first.result).resolves.toMatchObject({ status: 'canceled' })
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 0,
waiterLeases: 0
})
})
it('cancels once on last release and waits for local provider settlement', async () => {
const { cancelProviderRequest, pending, registry, starts } = createHarness()
const input = directInput()
const lease = registry.acquire('repo-a:ssh:ssh-a', input)
expect(lease.release('invalidated')).toBe('cancel-started')
expect(lease.release('stopped')).toBe('already-settled')
await expect(lease.result).resolves.toMatchObject({
providerRequestId: lease.providerRequestId,
status: 'canceled'
})
expect(cancelProviderRequest).toHaveBeenCalledTimes(1)
expect(cancelProviderRequest).toHaveBeenCalledWith(starts[0], 'invalidated')
expect(cancelProviderRequest.mock.calls[0]?.[0]).not.toHaveProperty('waiterLeaseId')
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 1,
waiterLeases: 0
})
pending[0].resolve(completeResult(starts[0]))
await pending[0].promise
await Promise.resolve()
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 0,
waiterLeases: 0
})
})
it('reports no cancellation start when the invocation throws before send', async () => {
const pending = controlledPromise<HostQualifiedDetectedWorktreeResult>()
let sends = 0
const invokeCancellation = vi.fn(() => {
throw new Error('before send')
})
const startProviderRequest = vi.fn(() => pending.promise)
const registry = createDetectedWorktreeRefreshLeaseRegistry({
startProviderRequest,
cancelProviderRequest: () => {
invokeCancellation()
sends++
}
})
const lease = registry.acquire('repo-a:ssh:ssh-a', directInput())
expect(lease.release('invalidated')).toBe('cancel-failed')
expect(invokeCancellation).toHaveBeenCalledOnce()
expect(sends).toBe(0)
await expect(lease.result).resolves.toMatchObject({ status: 'canceled' })
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 1,
waiterLeases: 0
})
const replacement = registry.acquire('repo-a:ssh:ssh-a', directInput())
expect(replacement.providerRequestId).not.toBe(lease.providerRequestId)
expect(startProviderRequest).toHaveBeenCalledTimes(2)
pending.resolve({
providerRequestId: lease.providerRequestId,
executionHostId: 'ssh:ssh-a',
status: 'canceled'
})
await pending.promise
await Promise.resolve()
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 0,
waiterLeases: 0
})
})
it('keeps asynchronous cancellation fire-and-forget', async () => {
const pending = controlledPromise<HostQualifiedDetectedWorktreeResult>()
const registry = createDetectedWorktreeRefreshLeaseRegistry({
startProviderRequest: () => pending.promise,
cancelProviderRequest: () => Promise.reject(new Error('async cancellation failure'))
})
const lease = registry.acquire('repo-a:ssh:ssh-a', directInput())
expect(lease.release('invalidated')).toBe('cancel-started')
await expect(lease.result).resolves.toMatchObject({ status: 'canceled' })
await Promise.resolve()
pending.resolve({
providerRequestId: lease.providerRequestId,
executionHostId: 'ssh:ssh-a',
status: 'canceled'
})
await pending.promise
await Promise.resolve()
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 0,
waiterLeases: 0
})
})
it('does not join incompatible full-authority inputs under one public key', () => {
const { registry, starts } = createHarness()
const first = registry.acquire('repo-a:ssh:ssh-a', directInput(authority('epoch-a', 1)))
const second = registry.acquire('repo-a:ssh:ssh-a', directInput(authority('epoch-b', 1)))
const third = registry.acquire('repo-a:ssh:ssh-a', directInput(authority('epoch-a', 2)))
expect(starts).toHaveLength(3)
expect(
new Set([first.providerRequestId, second.providerRequestId, third.providerRequestId]).size
).toBe(3)
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 3,
waiterLeases: 3
})
})
it('removes settled invocations so a later acquire starts fresh', async () => {
const { pending, registry, starts } = createHarness()
const first = registry.acquire('repo-a:ssh:ssh-a', directInput())
const firstResult = completeResult(starts[0])
pending[0].resolve(firstResult)
await expect(first.result).resolves.toEqual(firstResult)
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 0,
waiterLeases: 0
})
const second = registry.acquire('repo-a:ssh:ssh-a', directInput())
expect(starts).toHaveLength(2)
expect(second.providerRequestId).not.toBe(first.providerRequestId)
expect(second.waiterLeaseId).not.toBe(first.waiterLeaseId)
})
it('cleans rejected provider invocations without retaining waiters', async () => {
const { pending, registry } = createHarness()
const lease = registry.acquire('repo-a:ssh:ssh-a', directInput())
const error = new Error('provider failed')
pending[0].reject(error)
await expect(lease.result).rejects.toBe(error)
expect(registry.getActiveCounts()).toEqual({
providerInvocations: 0,
waiterLeases: 0
})
})
})
src/renderer/src/store/slices/detected-worktree-refresh-leases.ts
+265
View File
@@ -0,0 +1,265 @@
import type {
DirectSshDetectedWorktreeRequest,
HostQualifiedDetectedWorktreeResult,
ListDetectedWorktreesArgs,
LocalDetectedWorktreeRequest,
ProviderRequestId
} from '../../../../shared/detected-worktree-provider-contract'
export type WaiterLeaseId = string & {
readonly __waiterLeaseId: unique symbol
}
export type DetectedWorktreeRefreshReleaseReason = 'superseded' | 'invalidated' | 'stopped'
export type DetectedWorktreeRefreshReleaseOutcome =
| 'retained'
| 'cancel-started'
| 'cancel-failed'
| 'already-settled'
export type DetectedWorktreeRefreshLease = {
waiterLeaseId: WaiterLeaseId
providerRequestId: ProviderRequestId
result: Promise<HostQualifiedDetectedWorktreeResult>
release(reason: DetectedWorktreeRefreshReleaseReason): DetectedWorktreeRefreshReleaseOutcome
}
export type DetectedWorktreeRefreshProviderInput =
| Omit<LocalDetectedWorktreeRequest, 'providerRequestId'>
| Omit<DirectSshDetectedWorktreeRequest, 'providerRequestId'>
export type DetectedWorktreeRefreshLeaseRegistryOptions = {
startProviderRequest(
request: ListDetectedWorktreesArgs
): Promise<HostQualifiedDetectedWorktreeResult>
cancelProviderRequest(
request: ListDetectedWorktreesArgs,
reason: DetectedWorktreeRefreshReleaseReason
): void | Promise<void>
}
type LeaseWaiter = {
settled: boolean
resolve(result: HostQualifiedDetectedWorktreeResult): void
reject(error: unknown): void
}
type ProviderInvocation = {
publicKey: string
request: ListDetectedWorktreesArgs
waiters: Map<WaiterLeaseId, LeaseWaiter>
settled: boolean
cancelAttempted: boolean
}
export type DetectedWorktreeRefreshLeaseRegistry = {
acquire(
publicKey: string,
input: DetectedWorktreeRefreshProviderInput
): DetectedWorktreeRefreshLease
getActiveCounts(): { providerInvocations: number; waiterLeases: number }
}
let providerRequestSequence = 0
let waiterLeaseSequence = 0
function mintProviderRequestId(): ProviderRequestId {
providerRequestSequence = nextIdentitySequence(providerRequestSequence)
return `detected-worktree-provider-${providerRequestSequence}` as ProviderRequestId
}
function mintWaiterLeaseId(): WaiterLeaseId {
waiterLeaseSequence = nextIdentitySequence(waiterLeaseSequence)
return `detected-worktree-waiter-${waiterLeaseSequence}` as WaiterLeaseId
}
function nextIdentitySequence(current: number): number {
if (current >= Number.MAX_SAFE_INTEGER) {
throw new Error('Detected worktree refresh identity sequence exhausted')
}
return current + 1
}
function requestsAreCompatible(
request: ListDetectedWorktreesArgs,
input: DetectedWorktreeRefreshProviderInput
): boolean {
if (request.repoId !== input.repoId || request.executionHostId !== input.executionHostId) {
return false
}
const requestAuthority = 'expectedAuthority' in request ? request.expectedAuthority : undefined
const inputAuthority = 'expectedAuthority' in input ? input.expectedAuthority : undefined
if (!requestAuthority || !inputAuthority) {
return requestAuthority === inputAuthority
}
return (
requestAuthority.targetId === inputAuthority.targetId &&
requestAuthority.providerEpoch === inputAuthority.providerEpoch &&
requestAuthority.connectionGeneration === inputAuthority.connectionGeneration
)
}
function canceledResult(request: ListDetectedWorktreesArgs): HostQualifiedDetectedWorktreeResult {
return {
providerRequestId: request.providerRequestId,
executionHostId: request.executionHostId,
status: 'canceled'
}
}
function providerRequestWithId(
input: DetectedWorktreeRefreshProviderInput,
providerRequestId: ProviderRequestId
): ListDetectedWorktreesArgs {
if ('expectedAuthority' in input) {
return {
...input,
expectedAuthority: { ...input.expectedAuthority },
providerRequestId
}
}
return { ...input, providerRequestId }
}
export function createDetectedWorktreeRefreshLeaseRegistry(
options: DetectedWorktreeRefreshLeaseRegistryOptions
): DetectedWorktreeRefreshLeaseRegistry {
const invocationsByPublicKey = new Map<string, Set<ProviderInvocation>>()
const removeInvocation = (invocation: ProviderInvocation): void => {
const bucket = invocationsByPublicKey.get(invocation.publicKey)
if (!bucket) {
return
}
bucket.delete(invocation)
if (bucket.size === 0) {
invocationsByPublicKey.delete(invocation.publicKey)
}
}
const settleProviderInvocation = (
invocation: ProviderInvocation,
settlement:
| { status: 'fulfilled'; result: HostQualifiedDetectedWorktreeResult }
| { status: 'rejected'; error: unknown }
): void => {
if (invocation.settled) {
return
}
invocation.settled = true
removeInvocation(invocation)
for (const waiter of invocation.waiters.values()) {
if (waiter.settled) {
continue
}
waiter.settled = true
if (settlement.status === 'fulfilled') {
waiter.resolve(settlement.result)
} else {
waiter.reject(settlement.error)
}
}
invocation.waiters.clear()
}
const startInvocation = (
publicKey: string,
input: DetectedWorktreeRefreshProviderInput
): ProviderInvocation => {
const request = providerRequestWithId(input, mintProviderRequestId())
const invocation: ProviderInvocation = {
publicKey,
request,
waiters: new Map(),
settled: false,
cancelAttempted: false
}
const bucket = invocationsByPublicKey.get(publicKey) ?? new Set()
bucket.add(invocation)
invocationsByPublicKey.set(publicKey, bucket)
let result: Promise<HostQualifiedDetectedWorktreeResult>
try {
result = options.startProviderRequest(request)
} catch (error) {
result = Promise.reject(error)
}
void result.then(
(providerResult) =>
settleProviderInvocation(invocation, {
status: 'fulfilled',
result: providerResult
}),
(error) => settleProviderInvocation(invocation, { status: 'rejected', error })
)
return invocation
}
const acquire = (
publicKey: string,
input: DetectedWorktreeRefreshProviderInput
): DetectedWorktreeRefreshLease => {
const invocation =
[...(invocationsByPublicKey.get(publicKey) ?? [])].find(
(candidate) =>
!candidate.settled &&
!candidate.cancelAttempted &&
requestsAreCompatible(candidate.request, input)
) ?? startInvocation(publicKey, input)
const waiterLeaseId = mintWaiterLeaseId()
let resolveResult!: (result: HostQualifiedDetectedWorktreeResult) => void
let rejectResult!: (error: unknown) => void
const result = new Promise<HostQualifiedDetectedWorktreeResult>((resolve, reject) => {
resolveResult = resolve
rejectResult = reject
})
const waiter: LeaseWaiter = {
settled: false,
resolve: resolveResult,
reject: rejectResult
}
invocation.waiters.set(waiterLeaseId, waiter)
return {
waiterLeaseId,
providerRequestId: invocation.request.providerRequestId,
result,
release: (reason) => {
if (waiter.settled) {
return 'already-settled'
}
waiter.settled = true
invocation.waiters.delete(waiterLeaseId)
waiter.resolve(canceledResult(invocation.request))
if (invocation.settled || invocation.waiters.size > 0 || invocation.cancelAttempted) {
return 'retained'
}
invocation.cancelAttempted = true
let cancellation: void | Promise<void>
try {
cancellation = options.cancelProviderRequest(invocation.request, reason)
} catch {
return 'cancel-failed'
}
void Promise.resolve(cancellation).catch(() => undefined)
return 'cancel-started'
}
}
}
return {
acquire,
getActiveCounts: () => {
let providerInvocations = 0
let waiterLeases = 0
for (const bucket of invocationsByPublicKey.values()) {
providerInvocations += bucket.size
for (const invocation of bucket) {
waiterLeases += invocation.waiters.size
}
}
return { providerInvocations, waiterLeases }
}
}
}
src/renderer/src/store/slices/direct-ssh-pane-detach-ledger.test.ts
+275
View File
@@ -0,0 +1,275 @@
import { describe, expect, it } from 'vitest'
import type { SshProviderEpoch } from '../../../../shared/ssh-types'
import type { DirectSshPaneRetryAttemptId } from './direct-ssh-terminal-recovery'
import { createTestStore, makeTab, makeWorktree, seedStore } from './store-test-helpers'
const WORKTREE_ID = 'repo1::/path/wt1'
const SOURCE_TAB_ID = 'tab-source'
const TARGET_TAB_ID = 'tab-target'
const PRIMARY_PTY_ID = 'ssh:target-a@@pty-primary'
const SIBLING_PTY_ID = 'ssh:target-a@@pty-sibling'
const SURVIVOR_LEAF_ID = '11111111-1111-4111-8111-111111111111'
const DETACHED_LEAF_ID = '22222222-2222-4222-8222-222222222222'
const authority = {
targetId: 'target-a',
providerEpoch: 'epoch-a' as SshProviderEpoch,
connectionGeneration: 1
}
function createSplitDetachStore() {
const store = createTestStore()
seedStore(store, {
repos: [
{
id: 'repo1',
path: '/path',
displayName: 'repo1',
badgeColor: '#000',
addedAt: 1,
connectionId: authority.targetId,
executionHostId: 'ssh:target-a'
}
],
worktreesByRepo: {
repo1: [
makeWorktree({
id: WORKTREE_ID,
repoId: 'repo1',
path: '/path/wt1',
hostId: 'ssh:target-a'
})
]
},
tabsByWorktree: {
[WORKTREE_ID]: [
makeTab({ id: SOURCE_TAB_ID, worktreeId: WORKTREE_ID, ptyId: PRIMARY_PTY_ID }),
makeTab({ id: TARGET_TAB_ID, worktreeId: WORKTREE_ID, ptyId: null })
]
},
ptyIdsByTabId: {
[SOURCE_TAB_ID]: [PRIMARY_PTY_ID, SIBLING_PTY_ID],
[TARGET_TAB_ID]: []
},
directSshLivePtyBindingByTabId: {
[SOURCE_TAB_ID]: {
attemptId: 'split-detach' as DirectSshPaneRetryAttemptId,
authority,
tabGeneration: 0,
ptyId: PRIMARY_PTY_ID
}
},
directSshPaneRetryHistoryByTabId: {
[SOURCE_TAB_ID]: { authority, attemptedAt: [1] }
},
sshConnectionStates: new Map([
[
authority.targetId,
{
targetId: authority.targetId,
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: authority.providerEpoch,
connectionGeneration: authority.connectionGeneration
}
]
])
})
return store
}
describe('direct SSH split-pane detach ledger', () => {
it.each([
{
label: 'primary',
detachedPtyId: PRIMARY_PTY_ID,
survivorPtyId: SIBLING_PTY_ID
},
{
label: 'non-primary sibling',
detachedPtyId: SIBLING_PTY_ID,
survivorPtyId: PRIMARY_PTY_ID
}
])(
'preserves exact authority on both sides of a $label detach',
({ detachedPtyId, survivorPtyId }) => {
const store = createSplitDetachStore()
store.getState().syncPaneDetachPtyOwnership({
detachedLeafId: DETACHED_LEAF_ID,
detachedPtyId,
sourceLayout: {
root: { type: 'leaf', leafId: SURVIVOR_LEAF_ID },
activeLeafId: SURVIVOR_LEAF_ID,
expandedLeafId: null,
ptyIdsByLeafId: { [SURVIVOR_LEAF_ID]: survivorPtyId }
},
sourceTabId: SOURCE_TAB_ID,
targetTabId: TARGET_TAB_ID
})
expect(store.getState().directSshLivePtyBindingByTabId[SOURCE_TAB_ID]).toMatchObject({
attemptId: 'split-detach',
ptyId: survivorPtyId
})
expect(store.getState().directSshLivePtyBindingByTabId[TARGET_TAB_ID]).toMatchObject({
attemptId: 'split-detach',
ptyId: detachedPtyId
})
expect(store.getState().directSshPaneRetryHistoryByTabId[SOURCE_TAB_ID]).toEqual({
authority,
attemptedAt: [1]
})
expect(store.getState().directSshPaneRetryHistoryByTabId[TARGET_TAB_ID]).toEqual({
authority,
attemptedAt: [1]
})
expect(store.getState().invalidateStaleDirectSshTargetPtyBindings(authority)).toBe(0)
expect(store.getState().retryDirectSshTargetPanes(authority, 2)).toBe(0)
}
)
it('preserves source continuation when its sibling has not bound yet', () => {
const store = createSplitDetachStore()
const attemptId = 'split-detach' as DirectSshPaneRetryAttemptId
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: state.tabsByWorktree[WORKTREE_ID].map((tab) =>
tab.id === SOURCE_TAB_ID ? { ...tab, pendingActivationSpawn: true } : tab
)
},
ptyIdsByTabId: {
...state.ptyIdsByTabId,
[SOURCE_TAB_ID]: [PRIMARY_PTY_ID]
}
}))
store.getState().syncPaneDetachPtyOwnership({
detachedLeafId: DETACHED_LEAF_ID,
detachedPtyId: PRIMARY_PTY_ID,
sourceLayout: {
root: { type: 'leaf', leafId: SURVIVOR_LEAF_ID },
activeLeafId: SURVIVOR_LEAF_ID,
expandedLeafId: null,
ptyIdsByLeafId: {}
},
sourceTabId: SOURCE_TAB_ID,
targetTabId: TARGET_TAB_ID
})
expect(store.getState().directSshLivePtyBindingByTabId[SOURCE_TAB_ID]).toMatchObject({
attemptId
})
expect(store.getState().tabsByWorktree[WORKTREE_ID][0]).toMatchObject({
ptyId: null,
pendingActivationSpawn: true
})
store.getState().updateTabPtyId(SOURCE_TAB_ID, SIBLING_PTY_ID, undefined, attemptId)
expect(store.getState().directSshLivePtyBindingByTabId[SOURCE_TAB_ID]).toMatchObject({
attemptId,
ptyId: SIBLING_PTY_ID
})
expect(store.getState().ptyIdsByTabId[SOURCE_TAB_ID]).toEqual([SIBLING_PTY_ID])
})
it('projects an empty continuation gap to both unbound detach sides', () => {
const store = createSplitDetachStore()
const attemptId = 'split-detach' as DirectSshPaneRetryAttemptId
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: state.tabsByWorktree[WORKTREE_ID].map((tab) => ({
...tab,
ptyId: null,
pendingActivationSpawn: true
}))
},
ptyIdsByTabId: {
...state.ptyIdsByTabId,
[SOURCE_TAB_ID]: []
}
}))
store.getState().syncPaneDetachPtyOwnership({
detachedLeafId: DETACHED_LEAF_ID,
detachedPtyId: null,
sourceLayout: {
root: { type: 'leaf', leafId: SURVIVOR_LEAF_ID },
activeLeafId: SURVIVOR_LEAF_ID,
expandedLeafId: null,
ptyIdsByLeafId: {}
},
sourceTabId: SOURCE_TAB_ID,
targetTabId: TARGET_TAB_ID
})
expect(store.getState().directSshLivePtyBindingByTabId[SOURCE_TAB_ID]).toMatchObject({
attemptId
})
expect(store.getState().directSshLivePtyBindingByTabId[TARGET_TAB_ID]).toMatchObject({
attemptId
})
store.getState().updateTabPtyId(SOURCE_TAB_ID, SIBLING_PTY_ID, undefined, attemptId)
store.getState().updateTabPtyId(TARGET_TAB_ID, PRIMARY_PTY_ID, undefined, attemptId)
expect(store.getState().ptyIdsByTabId[SOURCE_TAB_ID]).toEqual([SIBLING_PTY_ID])
expect(store.getState().ptyIdsByTabId[TARGET_TAB_ID]).toEqual([PRIMARY_PTY_ID])
})
it('preserves a pending-only lease when both detach sides are still unbound', () => {
const store = createSplitDetachStore()
const attemptId = 'split-pending-only' as DirectSshPaneRetryAttemptId
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: state.tabsByWorktree[WORKTREE_ID].map((tab) => ({
...tab,
ptyId: null,
pendingActivationSpawn: true
}))
},
ptyIdsByTabId: {
...state.ptyIdsByTabId,
[SOURCE_TAB_ID]: []
},
directSshLivePtyBindingByTabId: {},
directSshPaneRetryByTabId: {
[SOURCE_TAB_ID]: {
attemptId,
authority,
tabGeneration: 0,
startedAt: 1
}
}
}))
store.getState().syncPaneDetachPtyOwnership({
detachedLeafId: DETACHED_LEAF_ID,
detachedPtyId: null,
sourceLayout: {
root: { type: 'leaf', leafId: SURVIVOR_LEAF_ID },
activeLeafId: SURVIVOR_LEAF_ID,
expandedLeafId: null,
ptyIdsByLeafId: {}
},
sourceTabId: SOURCE_TAB_ID,
targetTabId: TARGET_TAB_ID
})
expect(store.getState().directSshPaneRetryByTabId).toMatchObject({
[SOURCE_TAB_ID]: { attemptId, authority, tabGeneration: 0 },
[TARGET_TAB_ID]: { attemptId, authority, tabGeneration: 0 }
})
store.getState().updateTabPtyId(SOURCE_TAB_ID, SIBLING_PTY_ID, undefined, attemptId)
store.getState().updateTabPtyId(TARGET_TAB_ID, PRIMARY_PTY_ID, undefined, attemptId)
expect(store.getState().ptyIdsByTabId[SOURCE_TAB_ID]).toEqual([SIBLING_PTY_ID])
expect(store.getState().ptyIdsByTabId[TARGET_TAB_ID]).toEqual([PRIMARY_PTY_ID])
})
})
src/renderer/src/store/slices/direct-ssh-pane-retry-ledger.ts
+164
View File
@@ -0,0 +1,164 @@
import type { DirectSshAuthority } from '../../../../shared/ssh-types'
import type { TerminalLayoutSnapshot } from '../../../../shared/types'
import { shouldRetryPaneSpawnOnSshReconnect } from '../../hooks/ssh-reconnect-pane-retry'
import {
directSshAuthoritiesEqual,
liveBindingMatches,
pruneObsoleteAuthorityState
} from './direct-ssh-terminal-authority-ledger'
import { invalidateStaleDirectSshTerminalBindings } from './direct-ssh-terminal-recovery'
import { getTerminalActivationSpawnSuppression } from './terminal-activation-spawn-suppression'
import type {
DirectSshPaneRetryAttemptId,
DirectSshTerminalBindingState,
DirectSshTerminalRetryResult
} from './direct-ssh-terminal-recovery-types'
const AUTOMATIC_RETRY_LIMIT = 2
type DirectSshTerminalRetryOptions = {
qualifiedTabIds?: ReadonlySet<string>
}
function createAttemptId(
authority: DirectSshAuthority,
tabId: string,
tabGeneration: number,
now: number
): DirectSshPaneRetryAttemptId {
return JSON.stringify([
authority.targetId,
authority.providerEpoch,
authority.connectionGeneration,
tabId,
tabGeneration,
now
]) as DirectSshPaneRetryAttemptId
}
export function retryDirectSshTerminalPanes(
state: DirectSshTerminalBindingState & {
deferredSshSessionIdsByTabId: Record<string, string>
terminalLayoutsByTabId?: Record<string, TerminalLayoutSnapshot>
},
terminalWorkspaceKeys: ReadonlySet<string>,
authority: DirectSshAuthority,
now: number,
options: DirectSshTerminalRetryOptions = {}
): DirectSshTerminalRetryResult {
const authorityState = pruneObsoleteAuthorityState(state, authority)
const invalidated = invalidateStaleDirectSshTerminalBindings(
{ ...state, ...authorityState },
terminalWorkspaceKeys,
authority,
options.qualifiedTabIds
)
const working = { ...state, ...authorityState, ...invalidated.patch }
let tabsByWorktree = working.tabsByWorktree
let pending = working.directSshPaneRetryByTabId
let history = working.directSshPaneRetryHistoryByTabId
let retriedCount = 0
for (const workspaceKey of terminalWorkspaceKeys) {
const tabs = working.tabsByWorktree[workspaceKey] ?? []
let nextTabs = tabs
for (const [index, tab] of tabs.entries()) {
if (options.qualifiedTabIds && !options.qualifiedTabIds.has(tab.id)) {
continue
}
const currentPending = pending[tab.id]
if (
currentPending &&
directSshAuthoritiesEqual(currentPending.authority, authority) &&
currentPending.tabGeneration === (tab.generation ?? 0)
) {
continue
}
if (
liveBindingMatches(tab, working.directSshLivePtyBindingByTabId[tab.id], authority) ||
!shouldRetryPaneSpawnOnSshReconnect({
targetId: authority.targetId,
tabPtyId: tab.ptyId,
deferredSessionId: state.deferredSshSessionIdsByTabId[tab.id]
})
) {
continue
}
const previousHistory = history[tab.id]
const sameAuthorityAttempts =
previousHistory && directSshAuthoritiesEqual(previousHistory.authority, authority)
? previousHistory.attemptedAt
: []
// Why: a 31s PTY timeout outlives the old rolling window; authority rotation is the reset boundary for this automatic chain.
const recentAttempts = sameAuthorityAttempts
if (recentAttempts.length >= AUTOMATIC_RETRY_LIMIT) {
continue
}
const tabGeneration = (tab.generation ?? 0) + 1
if (nextTabs === tabs) {
nextTabs = [...tabs]
}
nextTabs[index] = {
...tab,
generation: tabGeneration,
pendingActivationSpawn: getTerminalActivationSpawnSuppression(
state.terminalLayoutsByTabId?.[tab.id]
)
}
if (pending === working.directSshPaneRetryByTabId) {
pending = { ...working.directSshPaneRetryByTabId }
}
pending[tab.id] = {
attemptId: createAttemptId(authority, tab.id, tabGeneration, now),
authority,
tabGeneration,
startedAt: now
}
if (history === working.directSshPaneRetryHistoryByTabId) {
history = { ...working.directSshPaneRetryHistoryByTabId }
}
history[tab.id] = { authority, attemptedAt: [...recentAttempts, now] }
retriedCount += 1
}
if (nextTabs !== tabs) {
if (tabsByWorktree === working.tabsByWorktree) {
tabsByWorktree = { ...working.tabsByWorktree }
}
tabsByWorktree[workspaceKey] = nextTabs
}
}
const recoveryChanged =
Object.keys(authorityState).some(
(key) =>
authorityState[key as keyof typeof authorityState] !==
state[key as keyof typeof authorityState]
) || invalidated.patch != null
if (retriedCount === 0 && !recoveryChanged) {
return { retriedCount, patch: null }
}
return {
retriedCount,
patch: {
...authorityState,
...invalidated.patch,
tabsByWorktree,
directSshPaneRetryByTabId: pending,
directSshPaneRetryHistoryByTabId: history
}
}
}
export function retrySettledDirectSshTerminalPane(
state: DirectSshTerminalBindingState & {
deferredSshSessionIdsByTabId: Record<string, string>
},
terminalWorkspaceKeys: ReadonlySet<string>,
authority: DirectSshAuthority,
tabId: string,
now: number
): DirectSshTerminalRetryResult {
return retryDirectSshTerminalPanes(state, terminalWorkspaceKeys, authority, now, {
qualifiedTabIds: new Set([tabId])
})
}
src/renderer/src/store/slices/direct-ssh-terminal-authority-ledger.ts
+252
View File
@@ -0,0 +1,252 @@
import type { DirectSshAuthority } from '../../../../shared/ssh-types'
import { parseAppSshPtyId } from '../../../../shared/ssh-pty-id'
import type { TerminalTab } from '../../../../shared/types'
import type {
DirectSshLivePtyBinding,
DirectSshPaneRetryResult,
DirectSshTerminalBindingState
} from './direct-ssh-terminal-recovery-types'
export function directSshAuthoritiesEqual(
left: DirectSshAuthority,
right: DirectSshAuthority
): boolean {
return (
left.targetId === right.targetId &&
left.providerEpoch === right.providerEpoch &&
left.connectionGeneration === right.connectionGeneration
)
}
export function withoutTabIds<T>(
source: Record<string, T>,
tabIds: ReadonlySet<string>
): Record<string, T> {
let next = source
for (const tabId of tabIds) {
if (!(tabId in next)) {
continue
}
if (next === source) {
next = { ...source }
}
delete next[tabId]
}
return next
}
export function pruneObsoleteAuthorityState(
state: DirectSshTerminalBindingState,
authority: DirectSshAuthority
): Pick<
DirectSshTerminalBindingState,
| 'directSshPaneRetryByTabId'
| 'directSshLivePtyBindingByTabId'
| 'directSshPaneRetryHistoryByTabId'
> {
const prune = <T extends { authority: DirectSshAuthority }>(
source: Record<string, T>
): Record<string, T> => {
const obsoleteIds = Object.entries(source)
.filter(
([, value]) =>
value.authority.targetId === authority.targetId &&
!directSshAuthoritiesEqual(value.authority, authority)
)
.map(([tabId]) => tabId)
return withoutTabIds(source, new Set(obsoleteIds))
}
return {
directSshPaneRetryByTabId: prune(state.directSshPaneRetryByTabId),
directSshLivePtyBindingByTabId: prune(state.directSshLivePtyBindingByTabId),
directSshPaneRetryHistoryByTabId: prune(state.directSshPaneRetryHistoryByTabId)
}
}
export function liveBindingMatches(
tab: TerminalTab,
binding: DirectSshLivePtyBinding | undefined,
authority: DirectSshAuthority
): boolean {
return Boolean(
binding &&
directSshAuthoritiesEqual(binding.authority, authority) &&
binding.tabGeneration === (tab.generation ?? 0) &&
(binding.ptyId === tab.ptyId || (tab.ptyId == null && Boolean(tab.pendingActivationSpawn)))
)
}
export function settleDirectSshPaneRetryState(
state: DirectSshTerminalBindingState,
result: DirectSshPaneRetryResult
): Partial<DirectSshTerminalBindingState> | null {
const pending = state.directSshPaneRetryByTabId[result.tabId]
const live = state.directSshLivePtyBindingByTabId[result.tabId]
const pendingMatches = Boolean(
pending &&
pending.attemptId === result.attemptId &&
directSshAuthoritiesEqual(pending.authority, result.authority) &&
pending.tabGeneration === result.tabGeneration
)
const liveMatches = Boolean(
live &&
live.attemptId === result.attemptId &&
directSshAuthoritiesEqual(live.authority, result.authority) &&
live.tabGeneration === result.tabGeneration
)
if (!pendingMatches && !liveMatches) {
return null
}
const tabIds = new Set([result.tabId])
const nextPending = pendingMatches
? withoutTabIds(state.directSshPaneRetryByTabId, tabIds)
: state.directSshPaneRetryByTabId
if (result.status !== 'success') {
return {
directSshPaneRetryByTabId: nextPending,
directSshLivePtyBindingByTabId: liveMatches
? withoutTabIds(state.directSshLivePtyBindingByTabId, tabIds)
: state.directSshLivePtyBindingByTabId
}
}
const tab = Object.values(state.tabsByWorktree)
.flat()
.find((candidate) => candidate.id === result.tabId)
if (
!tab ||
(tab.generation ?? 0) !== result.tabGeneration ||
!tab.ptyId ||
!(state.ptyIdsByTabId[result.tabId] ?? []).includes(result.ptyId)
) {
return null
}
return {
directSshPaneRetryByTabId: nextPending,
directSshLivePtyBindingByTabId: {
...state.directSshLivePtyBindingByTabId,
[result.tabId]: {
attemptId: result.attemptId,
authority: result.authority,
tabGeneration: result.tabGeneration,
ptyId: liveMatches && live ? live.ptyId : tab.ptyId
}
}
}
}
export function transferDirectSshPaneDetachLedger(
state: DirectSshTerminalBindingState,
args: {
detachedPtyId: string | null
sourcePtyId: string | null
sourceTabId: string
targetTabId: string
isAuthorityCurrent: (authority: DirectSshAuthority) => boolean
}
): Pick<
DirectSshTerminalBindingState,
| 'directSshPaneRetryByTabId'
| 'directSshLivePtyBindingByTabId'
| 'directSshPaneRetryHistoryByTabId'
> {
const tabIds = new Set([args.sourceTabId, args.targetTabId])
let directSshPaneRetryByTabId = withoutTabIds(state.directSshPaneRetryByTabId, tabIds)
let directSshLivePtyBindingByTabId = withoutTabIds(state.directSshLivePtyBindingByTabId, tabIds)
let directSshPaneRetryHistoryByTabId = withoutTabIds(
state.directSshPaneRetryHistoryByTabId,
tabIds
)
const tabs = Object.values(state.tabsByWorktree).flat()
const sourceTab = tabs.find((tab) => tab.id === args.sourceTabId)
const targetTab = tabs.find((tab) => tab.id === args.targetTabId)
const live = state.directSshLivePtyBindingByTabId[args.sourceTabId]
const pending = state.directSshPaneRetryByTabId[args.sourceTabId]
const sourceHasPendingContinuation = Boolean(
sourceTab && !args.sourcePtyId && sourceTab.pendingActivationSpawn
)
const targetHasPendingContinuation = Boolean(
targetTab && !args.detachedPtyId && targetTab.pendingActivationSpawn
)
const hasEmptyPendingContinuation = Boolean(
pending &&
sourceHasPendingContinuation &&
targetHasPendingContinuation &&
pending.tabGeneration === (sourceTab?.generation ?? 0) &&
args.isAuthorityCurrent(pending.authority)
)
const liveLease =
live && sourceTab && liveBindingMatches(sourceTab, live, live.authority) ? live : null
const pendingLease =
pending &&
sourceTab &&
pending.tabGeneration === (sourceTab.generation ?? 0) &&
([args.detachedPtyId, args.sourcePtyId].some(
(ptyId) => parseAppSshPtyId(ptyId ?? '')?.connectionId === pending.authority.targetId
) ||
hasEmptyPendingContinuation)
? pending
: null
const sourceLease = liveLease ?? pendingLease
const authority = sourceLease?.authority
if (
sourceLease &&
authority &&
targetTab &&
(args.detachedPtyId || targetHasPendingContinuation) &&
args.isAuthorityCurrent(authority)
) {
const nextLiveBindings = { ...directSshLivePtyBindingByTabId }
if (
sourceTab &&
((args.sourcePtyId &&
parseAppSshPtyId(args.sourcePtyId)?.connectionId === authority.targetId) ||
sourceHasPendingContinuation)
) {
if (liveLease) {
nextLiveBindings[args.sourceTabId] = {
attemptId: sourceLease.attemptId,
authority,
tabGeneration: sourceTab.generation ?? 0,
ptyId: args.sourcePtyId ?? liveLease.ptyId
}
} else if (pendingLease) {
directSshPaneRetryByTabId = {
...directSshPaneRetryByTabId,
[args.sourceTabId]: pendingLease
}
}
}
if (args.detachedPtyId || liveLease) {
nextLiveBindings[args.targetTabId] = {
attemptId: sourceLease.attemptId,
authority,
tabGeneration: targetTab.generation ?? 0,
ptyId: args.detachedPtyId ?? liveLease!.ptyId
}
} else if (pendingLease) {
directSshPaneRetryByTabId = {
...directSshPaneRetryByTabId,
[args.targetTabId]: {
...pendingLease,
tabGeneration: targetTab.generation ?? 0
}
}
}
directSshLivePtyBindingByTabId = nextLiveBindings
const history = state.directSshPaneRetryHistoryByTabId[args.sourceTabId]
if (history && directSshAuthoritiesEqual(history.authority, authority)) {
directSshPaneRetryHistoryByTabId = {
...directSshPaneRetryHistoryByTabId,
...(args.sourcePtyId || sourceHasPendingContinuation
? { [args.sourceTabId]: history }
: {}),
[args.targetTabId]: history
}
}
}
return {
directSshPaneRetryByTabId,
directSshLivePtyBindingByTabId,
directSshPaneRetryHistoryByTabId
}
}
src/renderer/src/store/slices/direct-ssh-terminal-recovery-types.ts
+64
View File
@@ -0,0 +1,64 @@
import type { DirectSshAuthority } from '../../../../shared/ssh-types'
import type { TerminalTab } from '../../../../shared/types'
import type { CodexRestartNotice } from './terminals'
export type DirectSshPaneRetryAttemptId = string & {
readonly __directSshPaneRetryAttemptId: unique symbol
}
export type DirectSshPaneRetryAttempt = {
attemptId: DirectSshPaneRetryAttemptId
authority: DirectSshAuthority
tabGeneration: number
startedAt: number
}
export type DirectSshLivePtyBinding = {
attemptId: DirectSshPaneRetryAttemptId
authority: DirectSshAuthority
tabGeneration: number
ptyId: string
}
export type DirectSshPaneRetryHistory = {
authority: DirectSshAuthority
attemptedAt: number[]
}
export type DirectSshPaneRetryResult =
| {
status: 'success'
tabId: string
attemptId: DirectSshPaneRetryAttemptId
authority: DirectSshAuthority
tabGeneration: number
ptyId: string
}
| {
status: 'failed' | 'timed-out' | 'superseded'
tabId: string
attemptId: DirectSshPaneRetryAttemptId
authority: DirectSshAuthority
tabGeneration: number
}
export type DirectSshTerminalBindingState = {
tabsByWorktree: Record<string, TerminalTab[]>
ptyIdsByTabId: Record<string, string[]>
lastKnownRelayPtyIdByTabId: Record<string, string>
pendingCodexPaneRestartIds: Record<string, true>
codexRestartNoticeByPtyId: Record<string, CodexRestartNotice>
directSshPaneRetryByTabId: Record<string, DirectSshPaneRetryAttempt>
directSshLivePtyBindingByTabId: Record<string, DirectSshLivePtyBinding>
directSshPaneRetryHistoryByTabId: Record<string, DirectSshPaneRetryHistory>
}
export type DirectSshTerminalBindingClearResult = {
clearedCount: number
patch: Partial<DirectSshTerminalBindingState> | null
}
export type DirectSshTerminalRetryResult = {
retriedCount: number
patch: Partial<DirectSshTerminalBindingState> | null
}
src/renderer/src/store/slices/direct-ssh-terminal-recovery.test.ts
+203
View File
@@ -0,0 +1,203 @@
import { describe, expect, it } from 'vitest'
import type { TerminalTab } from '../../../../shared/types'
import {
clearDirectSshTerminalBindings,
type DirectSshTerminalBindingState
} from './direct-ssh-terminal-recovery'
import { createTestStore, makeWorktree } from './store-test-helpers'
function makeTab(id: string, ptyId: string | null, pendingActivationSpawn?: true): TerminalTab {
return {
id,
ptyId,
worktreeId: 'repo::/work',
title: id,
customTitle: null,
color: null,
sortOrder: 0,
createdAt: 1,
...(pendingActivationSpawn ? { pendingActivationSpawn } : {})
}
}
function makeState(): DirectSshTerminalBindingState {
return {
tabsByWorktree: {
'repo::/ssh-work': [
makeTab('ssh-live', 'ssh-target@@pty-1', true),
makeTab('ssh-unbound', null, true)
],
'folder::ssh-work': [makeTab('folder-live', 'ssh-target@@pty-2')],
'repo::/other-host': [makeTab('other-live', 'ssh-other@@pty-3', true)]
},
ptyIdsByTabId: {
'ssh-live': ['ssh-target@@pty-1', 'ssh-target@@pty-split'],
'ssh-unbound': [],
'folder-live': ['ssh-target@@pty-2'],
'other-live': ['ssh-other@@pty-3']
},
lastKnownRelayPtyIdByTabId: {
'ssh-live': 'ssh-target@@pty-1',
'folder-live': 'ssh-target@@pty-2',
'other-live': 'ssh-other@@pty-3'
},
pendingCodexPaneRestartIds: {
'ssh-target@@pty-1': true,
'ssh-target@@pty-split': true,
'ssh-other@@pty-3': true
},
codexRestartNoticeByPtyId: {
'ssh-target@@pty-1': {
previousAccountLabel: 'old',
nextAccountLabel: 'new'
},
'ssh-other@@pty-3': {
previousAccountLabel: 'old',
nextAccountLabel: 'new'
}
},
directSshPaneRetryByTabId: {},
directSshLivePtyBindingByTabId: {},
directSshPaneRetryHistoryByTabId: {}
}
}
describe('clearDirectSshTerminalBindings', () => {
it('clears exact Git and folder workspace bindings in one projection', () => {
const state = makeState()
const result = clearDirectSshTerminalBindings(
state,
new Set(['repo::/ssh-work', 'folder::ssh-work'])
)
expect(result.clearedCount).toBe(2)
expect(result.patch?.tabsByWorktree?.['repo::/ssh-work']).toEqual([
expect.objectContaining({ id: 'ssh-live', ptyId: null }),
state.tabsByWorktree['repo::/ssh-work'][1]
])
expect(result.patch?.tabsByWorktree?.['repo::/ssh-work'][0]).not.toHaveProperty(
'pendingActivationSpawn'
)
expect(result.patch?.tabsByWorktree?.['repo::/ssh-work'][1]).toBe(
state.tabsByWorktree['repo::/ssh-work'][1]
)
expect(result.patch?.tabsByWorktree?.['folder::ssh-work'][0].ptyId).toBeNull()
expect(result.patch?.ptyIdsByTabId).toMatchObject({
'ssh-live': [],
'folder-live': [],
'other-live': ['ssh-other@@pty-3']
})
})
it('preserves relay reattach ids and every other host', () => {
const state = makeState()
const result = clearDirectSshTerminalBindings(state, new Set(['repo::/ssh-work']))
expect(result.patch).not.toHaveProperty('lastKnownRelayPtyIdByTabId')
expect(state.lastKnownRelayPtyIdByTabId['ssh-live']).toBe('ssh-target@@pty-1')
expect(result.patch?.tabsByWorktree?.['repo::/other-host']).toBe(
state.tabsByWorktree['repo::/other-host']
)
expect(result.patch?.pendingCodexPaneRestartIds).toEqual({
'ssh-other@@pty-3': true
})
expect(result.patch?.codexRestartNoticeByPtyId).toEqual({
'ssh-other@@pty-3': {
previousAccountLabel: 'old',
nextAccountLabel: 'new'
}
})
})
it('re-arms reconnectable disconnects without erasing retry history', () => {
const state = makeState()
const authority = {
targetId: 'target',
providerEpoch: 'epoch-1',
connectionGeneration: 1
}
state.directSshPaneRetryByTabId['ssh-live'] = {
attemptId: 'attempt-1',
authority,
tabGeneration: 1,
startedAt: 10
} as never
state.directSshLivePtyBindingByTabId['folder-live'] = {
authority,
tabGeneration: 0,
ptyId: 'ssh-target@@pty-2'
} as never
state.directSshPaneRetryHistoryByTabId['ssh-live'] = {
authority,
attemptedAt: [10]
} as never
const result = clearDirectSshTerminalBindings(
state,
new Set(['repo::/ssh-work', 'folder::ssh-work'])
)
expect(result.patch?.directSshPaneRetryByTabId).toEqual({})
expect(result.patch?.directSshLivePtyBindingByTabId).toEqual({})
expect(result.patch).not.toHaveProperty('directSshPaneRetryHistoryByTabId')
expect(state.directSshPaneRetryHistoryByTabId['ssh-live']?.attemptedAt).toEqual([10])
})
it('clears an exact target through one Zustand publication without activity changes', () => {
const store = createTestStore()
const worktreeId = 'repo-ssh::/work/demo'
const tab = { ...makeTab('ssh-live', 'ssh-target@@pty-1', true), worktreeId }
store.setState({
repos: [
{
id: 'repo-ssh',
path: '/work/demo',
displayName: 'demo',
badgeColor: '#000',
addedAt: 1,
connectionId: 'target'
}
],
worktreesByRepo: {
'repo-ssh': [
makeWorktree({
id: worktreeId,
repoId: 'repo-ssh',
path: '/work/demo',
hostId: 'ssh:target'
})
]
},
tabsByWorktree: { [worktreeId]: [tab] },
ptyIdsByTabId: { [tab.id]: ['ssh-target@@pty-1'] },
lastKnownRelayPtyIdByTabId: { [tab.id]: 'ssh-target@@pty-1' },
sortEpoch: 9
})
let publications = 0
const unsubscribe = store.subscribe(() => {
publications += 1
})
expect(store.getState().clearDirectSshTargetPtyBindings('target')).toBe(1)
unsubscribe()
const state = store.getState()
expect(publications).toBe(1)
expect(state.tabsByWorktree[worktreeId][0]).toMatchObject({ ptyId: null })
expect(state.ptyIdsByTabId[tab.id]).toEqual([])
expect(state.lastKnownRelayPtyIdByTabId[tab.id]).toBe('ssh-target@@pty-1')
expect(state.sortEpoch).toBe(9)
})
it('leaves unbound tabs byte-identical and repeats as a no-op', () => {
const state = makeState()
const first = clearDirectSshTerminalBindings(state, new Set(['repo::/ssh-work']))
const afterFirst = { ...state, ...first.patch }
const second = clearDirectSshTerminalBindings(afterFirst, new Set(['repo::/ssh-work']))
expect(first.patch?.tabsByWorktree?.['repo::/ssh-work'][1]).toBe(
state.tabsByWorktree['repo::/ssh-work'][1]
)
expect(second).toEqual({ clearedCount: 0, patch: null })
})
})
src/renderer/src/store/slices/direct-ssh-terminal-recovery.ts
+173
View File
@@ -0,0 +1,173 @@
import type { DirectSshAuthority } from '../../../../shared/ssh-types'
import type { TerminalTab } from '../../../../shared/types'
import type { CodexRestartNotice } from './terminals'
import {
directSshAuthoritiesEqual,
liveBindingMatches,
pruneObsoleteAuthorityState,
withoutTabIds
} from './direct-ssh-terminal-authority-ledger'
import type {
DirectSshTerminalBindingClearResult,
DirectSshTerminalBindingState
} from './direct-ssh-terminal-recovery-types'
export type {
DirectSshLivePtyBinding,
DirectSshPaneRetryAttempt,
DirectSshPaneRetryAttemptId,
DirectSshPaneRetryHistory,
DirectSshPaneRetryResult,
DirectSshTerminalBindingState
} from './direct-ssh-terminal-recovery-types'
function clearPtyBinding(
state: DirectSshTerminalBindingState,
tab: TerminalTab,
next: {
ptyIdsByTabId: Record<string, string[]>
pendingCodexPaneRestartIds: Record<string, true>
codexRestartNoticeByPtyId: Record<string, CodexRestartNotice>
}
): void {
const clearedPtyIds = new Set([tab.ptyId, ...(state.ptyIdsByTabId[tab.id] ?? [])])
next.ptyIdsByTabId[tab.id] = []
for (const ptyId of clearedPtyIds) {
if (!ptyId) {
continue
}
delete next.pendingCodexPaneRestartIds[ptyId]
delete next.codexRestartNoticeByPtyId[ptyId]
}
}
export function clearDirectSshTerminalBindings(
state: DirectSshTerminalBindingState,
terminalWorkspaceKeys: ReadonlySet<string>,
qualifiedTabIds?: ReadonlySet<string>
): DirectSshTerminalBindingClearResult {
let tabsByWorktree = state.tabsByWorktree
const ptyIdsByTabId = { ...state.ptyIdsByTabId }
const pendingCodexPaneRestartIds = { ...state.pendingCodexPaneRestartIds }
const codexRestartNoticeByPtyId = { ...state.codexRestartNoticeByPtyId }
const scopedTabIds = new Set<string>()
let clearedCount = 0
for (const workspaceKey of terminalWorkspaceKeys) {
const tabs = state.tabsByWorktree[workspaceKey]
if (!tabs) {
continue
}
let nextTabs = tabs
for (const [index, tab] of tabs.entries()) {
if (qualifiedTabIds && !qualifiedTabIds.has(tab.id)) {
continue
}
scopedTabIds.add(tab.id)
if (tab.ptyId == null) {
continue
}
const { pendingActivationSpawn: _pendingActivationSpawn, ...tabWithoutActivationSpawn } = tab
void _pendingActivationSpawn
if (nextTabs === tabs) {
nextTabs = [...tabs]
}
nextTabs[index] = { ...tabWithoutActivationSpawn, ptyId: null }
clearPtyBinding(state, tab, {
ptyIdsByTabId,
pendingCodexPaneRestartIds,
codexRestartNoticeByPtyId
})
clearedCount += 1
}
if (nextTabs !== tabs) {
if (tabsByWorktree === state.tabsByWorktree) {
tabsByWorktree = { ...state.tabsByWorktree }
}
tabsByWorktree[workspaceKey] = nextTabs
}
}
const directSshPaneRetryByTabId = withoutTabIds(state.directSshPaneRetryByTabId, scopedTabIds)
const directSshLivePtyBindingByTabId = withoutTabIds(
state.directSshLivePtyBindingByTabId,
scopedTabIds
)
const recoveryChanged =
directSshPaneRetryByTabId !== state.directSshPaneRetryByTabId ||
directSshLivePtyBindingByTabId !== state.directSshLivePtyBindingByTabId
if (clearedCount === 0 && !recoveryChanged) {
return { clearedCount, patch: null }
}
return {
clearedCount,
patch: {
tabsByWorktree,
ptyIdsByTabId,
pendingCodexPaneRestartIds,
codexRestartNoticeByPtyId,
directSshPaneRetryByTabId,
directSshLivePtyBindingByTabId
}
}
}
export function invalidateStaleDirectSshTerminalBindings(
state: DirectSshTerminalBindingState,
terminalWorkspaceKeys: ReadonlySet<string>,
authority: DirectSshAuthority,
qualifiedTabIds?: ReadonlySet<string>
): DirectSshTerminalBindingClearResult {
const authorityState = pruneObsoleteAuthorityState(state, authority)
const staleTabIds = new Set<string>()
const preservedPendingByTabId: DirectSshTerminalBindingState['directSshPaneRetryByTabId'] = {}
for (const workspaceKey of terminalWorkspaceKeys) {
for (const tab of state.tabsByWorktree[workspaceKey] ?? []) {
if (qualifiedTabIds && !qualifiedTabIds.has(tab.id)) {
continue
}
const liveBinding = authorityState.directSshLivePtyBindingByTabId[tab.id]
const pending = authorityState.directSshPaneRetryByTabId[tab.id]
const hasCurrentLiveBinding = liveBindingMatches(tab, liveBinding, authority)
const hasCurrentPending =
pending != null &&
directSshAuthoritiesEqual(pending.authority, authority) &&
pending.tabGeneration === (tab.generation ?? 0)
if (
(tab.ptyId != null && !hasCurrentLiveBinding) ||
(liveBinding != null && !hasCurrentLiveBinding) ||
(pending != null && !hasCurrentPending)
) {
staleTabIds.add(tab.id)
if (hasCurrentPending) {
preservedPendingByTabId[tab.id] = pending
}
}
}
}
const cleared = clearDirectSshTerminalBindings(
{ ...state, ...authorityState },
terminalWorkspaceKeys,
staleTabIds
)
const authorityChanged = Object.keys(authorityState).some(
(key) =>
authorityState[key as keyof typeof authorityState] !==
state[key as keyof typeof authorityState]
)
if (!cleared.patch && !authorityChanged) {
return cleared
}
const preservedPending =
cleared.patch && Object.keys(preservedPendingByTabId).length > 0
? {
directSshPaneRetryByTabId: {
...cleared.patch.directSshPaneRetryByTabId,
...preservedPendingByTabId
}
}
: {}
return {
clearedCount: cleared.clearedCount,
patch: { ...authorityState, ...cleared.patch, ...preservedPending }
}
}
src/renderer/src/store/slices/direct-ssh-terminal-retry.test.ts
+647
View File
@@ -0,0 +1,647 @@
import { describe, expect, it } from 'vitest'
import type { DirectSshAuthority, SshProviderEpoch } from '../../../../shared/ssh-types'
import type { DirectSshPaneRetryAttemptId } from './direct-ssh-terminal-recovery'
import { createTestStore, makeTab, makeWorktree } from './store-test-helpers'
const WORKTREE_ID = 'repo-ssh::/work/demo'
const TAB_ID = 'tab-ssh'
function authority(epoch = 'epoch-1', generation = 1): DirectSshAuthority {
return {
targetId: 'target',
providerEpoch: epoch as SshProviderEpoch,
connectionGeneration: generation
}
}
function seedStore(ptyId: string | null = null) {
const store = createTestStore()
const currentAuthority = authority()
store.setState({
repos: [
{
id: 'repo-ssh',
path: '/work/demo',
displayName: 'demo',
badgeColor: '#000',
addedAt: 1,
connectionId: 'target',
executionHostId: 'ssh:target'
}
],
worktreesByRepo: {
'repo-ssh': [
makeWorktree({
id: WORKTREE_ID,
repoId: 'repo-ssh',
path: '/work/demo',
hostId: 'ssh:target'
})
]
},
tabsByWorktree: {
[WORKTREE_ID]: [makeTab({ id: TAB_ID, worktreeId: WORKTREE_ID, ptyId })]
},
ptyIdsByTabId: { [TAB_ID]: ptyId ? [ptyId] : [] },
lastKnownRelayPtyIdByTabId: ptyId ? { [TAB_ID]: ptyId } : {},
sshConnectionStates: new Map([
[
'target',
{
targetId: 'target',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: currentAuthority.providerEpoch,
connectionGeneration: currentAuthority.connectionGeneration
}
]
])
})
return store
}
describe('direct SSH terminal retry ledger', () => {
it('invalidates a non-null binding without current-authority evidence atomically', () => {
const ptyId = 'ssh:target@@pty-old'
const store = seedStore(ptyId)
let publications = 0
const unsubscribe = store.subscribe(() => {
publications += 1
})
expect(store.getState().invalidateStaleDirectSshTargetPtyBindings(authority())).toBe(1)
unsubscribe()
expect(publications).toBe(1)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].ptyId).toBeNull()
expect(store.getState().ptyIdsByTabId[TAB_ID]).toEqual([])
expect(store.getState().lastKnownRelayPtyIdByTabId[TAB_ID]).toBe(ptyId)
})
it('preserves a healthy current-authority sibling in the same workspace', () => {
const store = seedStore('ssh:target@@pty-stale')
const sibling = makeTab({
id: 'tab-healthy',
worktreeId: WORKTREE_ID,
ptyId: 'ssh:target@@pty-healthy'
})
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: [...state.tabsByWorktree[WORKTREE_ID], sibling]
},
ptyIdsByTabId: {
...state.ptyIdsByTabId,
'tab-healthy': ['ssh:target@@pty-healthy']
},
directSshLivePtyBindingByTabId: {
'tab-healthy': {
attemptId: 'healthy-attempt' as DirectSshPaneRetryAttemptId,
authority: authority(),
tabGeneration: 0,
ptyId: 'ssh:target@@pty-healthy'
}
}
}))
expect(store.getState().invalidateStaleDirectSshTargetPtyBindings(authority())).toBe(1)
const tabs = store.getState().tabsByWorktree[WORKTREE_ID]
expect(tabs.find((tab) => tab.id === TAB_ID)?.ptyId).toBeNull()
expect(tabs.find((tab) => tab.id === 'tab-healthy')).toBe(sibling)
expect(store.getState().ptyIdsByTabId['tab-healthy']).toEqual(['ssh:target@@pty-healthy'])
expect(store.getState().directSshLivePtyBindingByTabId['tab-healthy']).toBeDefined()
})
it('keeps one pending attempt and acknowledges success after the live commit', () => {
const store = seedStore()
let publications = 0
const unsubscribe = store.subscribe(() => {
publications += 1
})
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_000)).toBe(1)
unsubscribe()
expect(publications).toBe(1)
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_001)).toBe(0)
const retried = store.getState().tabsByWorktree[WORKTREE_ID][0]
expect(retried).toMatchObject({ generation: 1, pendingActivationSpawn: true })
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toMatchObject({
tabGeneration: 1,
startedAt: 1_000
})
const attempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
const pendingBeforeStaleSettlement = store.getState().directSshPaneRetryByTabId
store.getState().settleDirectSshPaneRetry({
status: 'failed',
tabId: TAB_ID,
attemptId: attempt.attemptId,
authority: attempt.authority,
tabGeneration: attempt.tabGeneration - 1
})
expect(store.getState().directSshPaneRetryByTabId).toBe(pendingBeforeStaleSettlement)
store.getState().settleDirectSshPaneRetry(
{
status: 'failed',
tabId: TAB_ID,
attemptId: attempt.attemptId,
authority: authority('stale-epoch', 0),
tabGeneration: attempt.tabGeneration
},
1_001
)
expect(store.getState().directSshPaneRetryByTabId).toBe(pendingBeforeStaleSettlement)
store.getState().settleDirectSshPaneRetry({
status: 'success',
tabId: TAB_ID,
attemptId: attempt.attemptId,
authority: attempt.authority,
tabGeneration: attempt.tabGeneration,
ptyId: 'ssh:target@@pty-new'
})
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBeDefined()
store
.getState()
.updateTabPtyId(
TAB_ID,
'ssh:target@@pty-new',
undefined,
'wrong-attempt' as DirectSshPaneRetryAttemptId
)
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBeDefined()
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].ptyId).toBeNull()
expect(store.getState().ptyIdsByTabId[TAB_ID]).toEqual([])
expect(store.getState().lastKnownRelayPtyIdByTabId[TAB_ID]).toBeUndefined()
store.getState().updateTabPtyId(TAB_ID, 'ssh:target@@pty-new', undefined, attempt.attemptId)
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toMatchObject({
tabGeneration: 1,
ptyId: 'ssh:target@@pty-new'
})
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_002)).toBe(0)
})
it('re-arms one failure and exhausts the authority chain after two attempts', () => {
const store = seedStore()
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_000)).toBe(1)
const firstAttempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
store.getState().settleDirectSshPaneRetry(
{
status: 'failed',
tabId: TAB_ID,
attemptId: firstAttempt.attemptId,
authority: firstAttempt.authority,
tabGeneration: firstAttempt.tabGeneration
},
2_000
)
const secondAttempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
expect(secondAttempt.tabGeneration).toBe(2)
store.getState().settleDirectSshPaneRetry(
{
status: 'timed-out',
tabId: TAB_ID,
attemptId: secondAttempt.attemptId,
authority: secondAttempt.authority,
tabGeneration: secondAttempt.tabGeneration
},
3_000
)
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBe(secondAttempt)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].generation).toBe(2)
expect(store.getState().retryDirectSshTargetPanes(authority(), 30_999)).toBe(0)
expect(store.getState().retryDirectSshTargetPanes(authority(), 31_000)).toBe(0)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].generation).toBe(2)
})
it('keeps one split-pane attempt valid until every sibling settles', () => {
const store = seedStore()
store.setState({
terminalLayoutsByTabId: {
[TAB_ID]: {
root: {
type: 'split',
direction: 'horizontal',
first: { type: 'leaf', leafId: '11111111-1111-4111-8111-111111111111' },
second: { type: 'leaf', leafId: '22222222-2222-4222-8222-222222222222' }
},
activeLeafId: '11111111-1111-4111-8111-111111111111',
expandedLeafId: null,
ptyIdsByLeafId: {}
}
}
})
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_000)).toBe(1)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].pendingActivationSpawn).toBe(2)
const firstAttempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
const firstPtyId = 'ssh:target@@pty-first'
const siblingPtyId = 'ssh:target@@pty-sibling'
store.getState().updateTabPtyId(TAB_ID, firstPtyId, undefined, firstAttempt.attemptId)
store.getState().updateTabPtyId(TAB_ID, siblingPtyId, undefined, firstAttempt.attemptId)
expect(store.getState().ptyIdsByTabId[TAB_ID]).toEqual([firstPtyId, siblingPtyId])
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].ptyId).toBe(firstPtyId)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].pendingActivationSpawn).toBeUndefined()
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toMatchObject({
attemptId: firstAttempt.attemptId,
ptyId: firstPtyId
})
store.getState().settleDirectSshPaneRetry(
{
status: 'failed',
tabId: TAB_ID,
attemptId: firstAttempt.attemptId,
authority: firstAttempt.authority,
tabGeneration: firstAttempt.tabGeneration
},
2_000
)
const secondAttempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
expect(secondAttempt.tabGeneration).toBe(2)
expect(store.getState().ptyIdsByTabId[TAB_ID]).toEqual([])
const beforeStaleCallback = store.getState()
store
.getState()
.updateTabPtyId(TAB_ID, 'ssh:target@@pty-stale', undefined, firstAttempt.attemptId)
store.getState().settleDirectSshPaneRetry(
{
status: 'failed',
tabId: TAB_ID,
attemptId: firstAttempt.attemptId,
authority: firstAttempt.authority,
tabGeneration: firstAttempt.tabGeneration
},
2_001
)
expect(store.getState().tabsByWorktree).toBe(beforeStaleCallback.tabsByWorktree)
expect(store.getState().ptyIdsByTabId).toBe(beforeStaleCallback.ptyIdsByTabId)
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBe(secondAttempt)
expect(store.getState().directSshPaneRetryHistoryByTabId[TAB_ID].attemptedAt).toEqual([
1_000, 2_000
])
const secondFirstPtyId = 'ssh:target@@pty-second-first'
const secondSiblingPtyId = 'ssh:target@@pty-second-sibling'
store.getState().updateTabPtyId(TAB_ID, secondFirstPtyId, undefined, secondAttempt.attemptId)
store.getState().settleDirectSshPaneRetry(
{
status: 'failed',
tabId: TAB_ID,
attemptId: secondAttempt.attemptId,
authority: secondAttempt.authority,
tabGeneration: secondAttempt.tabGeneration
},
3_000
)
store.getState().updateTabPtyId(TAB_ID, secondSiblingPtyId, undefined, secondAttempt.attemptId)
expect(store.getState().ptyIdsByTabId[TAB_ID]).toEqual([secondFirstPtyId, secondSiblingPtyId])
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().retryDirectSshTargetPanes(authority(), 3_001)).toBe(0)
})
it('promotes a surviving split PTY without revoking its exact retry lease', () => {
const store = seedStore()
store.setState({ activeWorktreeId: null })
store.getState().retryDirectSshTargetPanes(authority(), 1_000)
const attempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
const firstPtyId = 'ssh:target@@pty-first'
const siblingPtyId = 'ssh:target@@pty-sibling'
store.getState().updateTabPtyId(TAB_ID, firstPtyId, undefined, attempt.attemptId)
store.getState().updateTabPtyId(TAB_ID, siblingPtyId, undefined, attempt.attemptId)
store.getState().clearTabPtyId(TAB_ID, firstPtyId)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].ptyId).toBe(siblingPtyId)
expect(store.getState().ptyIdsByTabId[TAB_ID]).toEqual([siblingPtyId])
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toMatchObject({
attemptId: attempt.attemptId,
ptyId: siblingPtyId
})
expect(store.getState().retryDirectSshTargetPanes(authority(), 2_000)).toBe(0)
})
it('keeps exact retry authority through a primary-exit gap before a sibling binds', () => {
const store = seedStore()
store.setState({
activeWorktreeId: null,
terminalLayoutsByTabId: {
[TAB_ID]: {
root: {
type: 'split',
direction: 'horizontal',
first: { type: 'leaf', leafId: '11111111-1111-4111-8111-111111111111' },
second: { type: 'leaf', leafId: '22222222-2222-4222-8222-222222222222' }
},
activeLeafId: '11111111-1111-4111-8111-111111111111',
expandedLeafId: null,
ptyIdsByLeafId: {}
}
}
})
store.getState().retryDirectSshTargetPanes(authority(), 1_000)
const attempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
const firstPtyId = 'ssh:target@@pty-first'
const siblingPtyId = 'ssh:target@@pty-sibling'
store.getState().updateTabPtyId(TAB_ID, firstPtyId, undefined, attempt.attemptId)
store.getState().clearTabPtyId(TAB_ID, firstPtyId)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0]).toMatchObject({
ptyId: null,
pendingActivationSpawn: true
})
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toMatchObject({
attemptId: attempt.attemptId,
ptyId: firstPtyId
})
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_001)).toBe(0)
store.getState().updateTabPtyId(TAB_ID, siblingPtyId, undefined, attempt.attemptId)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].ptyId).toBe(siblingPtyId)
expect(store.getState().ptyIdsByTabId[TAB_ID]).toEqual([siblingPtyId])
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toMatchObject({
attemptId: attempt.attemptId,
ptyId: siblingPtyId
})
})
it('does not turn two thirty-one-second timeouts into an unbounded retry chain', () => {
const store = seedStore()
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_000)).toBe(1)
const firstAttempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
store.getState().settleDirectSshPaneRetry(
{
status: 'timed-out',
tabId: TAB_ID,
attemptId: firstAttempt.attemptId,
authority: firstAttempt.authority,
tabGeneration: firstAttempt.tabGeneration
},
32_000
)
const secondAttempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
expect(secondAttempt.tabGeneration).toBe(2)
store.getState().settleDirectSshPaneRetry(
{
status: 'timed-out',
tabId: TAB_ID,
attemptId: secondAttempt.attemptId,
authority: secondAttempt.authority,
tabGeneration: secondAttempt.tabGeneration
},
63_000
)
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBe(secondAttempt)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].generation).toBe(2)
expect(store.getState().retryDirectSshTargetPanes(authority(), 63_000)).toBe(0)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].generation).toBe(2)
})
it('re-arms only the exact failed tab', () => {
const store = seedStore()
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_000)).toBe(1)
const firstAttempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
const sibling = makeTab({
id: 'tab-unrelated',
worktreeId: WORKTREE_ID,
ptyId: null
})
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: [...state.tabsByWorktree[WORKTREE_ID], sibling]
},
ptyIdsByTabId: { ...state.ptyIdsByTabId, [sibling.id]: [] }
}))
store.getState().settleDirectSshPaneRetry(
{
status: 'failed',
tabId: TAB_ID,
attemptId: firstAttempt.attemptId,
authority: firstAttempt.authority,
tabGeneration: firstAttempt.tabGeneration
},
2_000
)
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]?.tabGeneration).toBe(2)
expect(store.getState().directSshPaneRetryByTabId[sibling.id]).toBeUndefined()
expect(
store.getState().tabsByWorktree[WORKTREE_ID].find((tab) => tab.id === sibling.id)
?.generation ?? 0
).toBe(0)
})
it('re-arms a pending attempt when its bound SSH PTY exits', () => {
const store = seedStore()
store.getState().retryDirectSshTargetPanes(authority(), 1_000)
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: state.tabsByWorktree[WORKTREE_ID].map((tab) => ({
...tab,
ptyId: 'ssh:target@@pty-failed'
}))
},
ptyIdsByTabId: {
...state.ptyIdsByTabId,
[TAB_ID]: ['ssh:target@@pty-failed']
}
}))
store.getState().clearTabPtyId(TAB_ID, 'ssh:target@@pty-failed')
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().retryDirectSshTargetPanes(authority(), 2_000)).toBe(1)
})
it('clearing a successful binding re-arms it without discarding relay identity', () => {
const store = seedStore()
store.getState().retryDirectSshTargetPanes(authority(), 1_000)
const attempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
store.getState().updateTabPtyId(TAB_ID, 'ssh:target@@pty-new', undefined, attempt.attemptId)
store.getState().clearTabPtyId(TAB_ID)
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().lastKnownRelayPtyIdByTabId[TAB_ID]).toBe('ssh:target@@pty-new')
expect(store.getState().retryDirectSshTargetPanes(authority(), 2_000)).toBe(1)
})
it('removes successful evidence when a snapshot overwrites the live binding', () => {
const store = seedStore()
store.getState().retryDirectSshTargetPanes(authority(), 1_000)
const attempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
store.getState().updateTabPtyId(TAB_ID, 'ssh:target@@pty-new', undefined, attempt.attemptId)
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: state.tabsByWorktree[WORKTREE_ID].map((tab) => ({
...tab,
ptyId: null
}))
},
ptyIdsByTabId: { ...state.ptyIdsByTabId, [TAB_ID]: [] }
}))
expect(store.getState().retryDirectSshTargetPanes(authority(), 2_000)).toBe(1)
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBeDefined()
})
it('keeps snapshot-restored PTY hints eligible for corrective retry', () => {
const store = seedStore()
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: state.tabsByWorktree[WORKTREE_ID].map((tab) => ({
...tab,
ptyId: 'ssh:target@@pty-snapshot'
}))
},
ptyIdsByTabId: {
...state.ptyIdsByTabId,
[TAB_ID]: ['ssh:target@@pty-snapshot']
}
}))
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_000)).toBe(1)
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toMatchObject({
authority: authority(),
tabGeneration: 1
})
})
it('clears a snapshot hint without superseding its current pending attempt', () => {
const store = seedStore()
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_000)).toBe(1)
const pending = store.getState().directSshPaneRetryByTabId[TAB_ID]
store.setState((state) => ({
tabsByWorktree: {
...state.tabsByWorktree,
[WORKTREE_ID]: state.tabsByWorktree[WORKTREE_ID].map((tab) => ({
...tab,
ptyId: 'ssh:target@@pty-snapshot'
}))
},
ptyIdsByTabId: {
...state.ptyIdsByTabId,
[TAB_ID]: ['ssh:target@@pty-snapshot']
}
}))
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_001)).toBe(0)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0]).toMatchObject({
generation: 1,
ptyId: null
})
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBe(pending)
expect(store.getState().directSshPaneRetryHistoryByTabId[TAB_ID].attemptedAt).toEqual([1_000])
})
it('rotates obsolete pending, success, and retry history by exact authority', () => {
const store = seedStore()
store.getState().retryDirectSshTargetPanes(authority(), 1_000)
const attempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
store.getState().updateTabPtyId(TAB_ID, 'ssh:target@@pty-new', undefined, attempt.attemptId)
const nextAuthority = authority('epoch-2', 2)
store.setState({
sshConnectionStates: new Map([
[
'target',
{
targetId: 'target',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: nextAuthority.providerEpoch,
connectionGeneration: nextAuthority.connectionGeneration
}
]
])
})
expect(store.getState().retryDirectSshTargetPanes(nextAuthority, 1_001)).toBe(1)
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]?.authority).toEqual(nextAuthority)
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().directSshPaneRetryHistoryByTabId[TAB_ID]).toEqual({
authority: nextAuthority,
attemptedAt: [1_001]
})
})
it('fails closed when the store no longer names the exact authority', () => {
const store = seedStore()
const stale = authority('stale', 0)
const before = store.getState()
expect(store.getState().retryDirectSshTargetPanes(stale, 1_000)).toBe(0)
expect(store.getState().invalidateStaleDirectSshTargetPtyBindings(stale)).toBe(0)
expect(store.getState().tabsByWorktree).toBe(before.tabsByWorktree)
expect(store.getState().directSshPaneRetryByTabId).toBe(before.directSshPaneRetryByTabId)
})
it('retries a stale-catalog tab from retained SSH ownership exactly once', () => {
const store = seedStore()
store.setState({
repos: [],
worktreesByRepo: {},
lastKnownRelayPtyIdByTabId: { [TAB_ID]: 'ssh:target@@pty-retained' },
tabsByWorktree: {
[WORKTREE_ID]: [makeTab({ id: TAB_ID, worktreeId: WORKTREE_ID, ptyId: null })],
local: [makeTab({ id: 'tab-local', worktreeId: 'local', ptyId: null })]
}
})
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_000)).toBe(1)
expect(store.getState().retryDirectSshTargetPanes(authority(), 1_001)).toBe(0)
expect(store.getState().tabsByWorktree[WORKTREE_ID][0].generation).toBe(1)
expect(store.getState().tabsByWorktree.local[0].generation ?? 0).toBe(0)
})
it('removes all direct SSH retry ledgers when a tab closes', () => {
const store = seedStore()
store.getState().retryDirectSshTargetPanes(authority(), 1_000)
const attempt = store.getState().directSshPaneRetryByTabId[TAB_ID]
store.getState().updateTabPtyId(TAB_ID, 'ssh:target@@pty-live', undefined, attempt.attemptId)
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toBeDefined()
store.setState((state) => ({
directSshPaneRetryByTabId: {
...state.directSshPaneRetryByTabId,
[TAB_ID]: {
attemptId: 'pending-close' as DirectSshPaneRetryAttemptId,
authority: authority(),
tabGeneration: 1,
startedAt: 2_000
}
}
}))
store.getState().closeTab(TAB_ID, { reason: 'pty-exit' })
expect(store.getState().directSshPaneRetryByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().directSshLivePtyBindingByTabId[TAB_ID]).toBeUndefined()
expect(store.getState().directSshPaneRetryHistoryByTabId[TAB_ID]).toBeUndefined()
})
})
src/renderer/src/store/slices/direct-ssh-terminal-workspace-scope.test.ts
+260
View File
@@ -0,0 +1,260 @@
import { describe, expect, it } from 'vitest'
import type { TerminalTab } from '../../../../shared/types'
import { resolveDirectSshTerminalWorkspaceKeys } from './direct-ssh-terminal-workspace-scope'
function tab(worktreeId: string, ptyId: string): TerminalTab {
return {
id: `tab-${worktreeId}`,
worktreeId,
ptyId,
title: worktreeId,
customTitle: null,
color: null,
sortOrder: 0,
createdAt: 1
}
}
describe('resolveDirectSshTerminalWorkspaceKeys', () => {
it('uses a parsed live target PTY when the catalog row is stale', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{ targetId: 'target-a', catalogRevision: 1, repos: [] },
{ 'stale-worktree': [tab('stale-worktree', 'ssh:target-a@@pty-1')] }
)
).toEqual(new Set(['stale-worktree']))
})
it('uses a retained target PTY after disconnect clears the live binding', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{ targetId: 'target-a', catalogRevision: 1, repos: [] },
{ 'stale-worktree': [tab('stale-worktree', '')] },
{ 'tab-stale-worktree': 'ssh:target-a@@pty-1' }
)
).toEqual(new Set(['stale-worktree']))
})
it('refuses retained PTY fallback when explicit ownership names another host', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{
targetId: 'target-a',
catalogRevision: 1,
repos: [],
worktreesByRepo: {
repo: [
{
id: 'contradictory-worktree',
repoId: 'repo',
hostId: 'ssh:target-b'
}
]
}
},
{ 'contradictory-worktree': [tab('contradictory-worktree', '')] },
{ 'tab-contradictory-worktree': 'ssh:target-a@@pty-1' }
)
).toEqual(new Set())
})
it('refuses retained PTY fallback when a repo id is owned by two SSH hosts', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{
targetId: 'target-a',
catalogRevision: 1,
repos: [
{
id: 'shared',
path: '/srv/a/shared',
projectGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:target-a'
},
{
id: 'shared',
path: '/srv/b/shared',
projectGroupId: null,
connectionId: 'target-b',
executionHostId: 'ssh:target-b'
}
],
worktreesByRepo: {
shared: [{ id: 'shared::/work', repoId: 'shared' }]
}
},
{ 'shared::/work': [tab('shared::/work', '')] },
{ 'tab-shared::/work': 'ssh:target-a@@pty-retained' }
)
).toEqual(new Set())
})
it('refuses a retained target when the current PTY names another SSH host', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{ targetId: 'target-a', catalogRevision: 1, repos: [] },
{
'stale-worktree': [
tab('stale-worktree', 'ssh:target-b@@pty-live'),
tab('stale-worktree', '')
]
},
{ 'tab-stale-worktree': 'ssh:target-a@@pty-retained' }
)
).toEqual(new Set())
})
it('refuses parsed PTY fallback when explicit worktree ownership names another host', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{
targetId: 'target-a',
catalogRevision: 1,
repos: [],
worktreesByRepo: {
repo: [
{
id: 'contradictory-worktree',
repoId: 'repo',
hostId: 'ssh:target-b'
}
]
}
},
{
'contradictory-worktree': [tab('contradictory-worktree', 'ssh:target-a@@pty-1')]
}
)
).toEqual(new Set())
})
it('refuses parsed PTY fallback when explicit host provenance is malformed', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{
targetId: 'target-a',
catalogRevision: 1,
repos: [],
worktreesByRepo: {
repo: [
{
id: 'malformed-worktree',
repoId: 'repo',
hostId: 'ssh:%'
}
]
}
},
{
'malformed-worktree': [tab('malformed-worktree', 'ssh:target-a@@pty-1')]
}
)
).toEqual(new Set())
})
it('refuses folder PTY fallback when project host provenance is malformed', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{
targetId: 'target-a',
catalogRevision: 1,
repos: [],
folderWorkspaces: [
{
id: 'folder-a',
projectGroupId: 'group-a',
folderPath: '/srv/project',
connectionId: 'target-a'
}
],
projectGroups: [
{
id: 'group-a',
parentGroupId: null,
connectionId: 'target-a',
executionHostId: 'ssh:%'
}
]
},
{
'folder:folder-a': [tab('folder:folder-a', 'ssh:target-a@@pty-1')]
}
)
).toEqual(new Set())
})
it('refuses fallback when explicit worktree and repo-derived ownership disagree', () => {
expect(
resolveDirectSshTerminalWorkspaceKeys(
{
targetId: 'target-a',
catalogRevision: 1,
repos: [
{
id: 'repo',
path: '/srv/repo',
projectGroupId: null,
connectionId: 'target-b',
executionHostId: 'ssh:target-b'
}
],
worktreesByRepo: {
repo: [
{
id: 'contradictory-worktree',
repoId: 'repo',
hostId: 'ssh:target-a'
}
]
}
},
{
'contradictory-worktree': [tab('contradictory-worktree', 'ssh:target-a@@pty-1')]
}
)
).toEqual(new Set())
})
it('refuses runtime-owned and contradictory folder provenance', () => {
const keys = resolveDirectSshTerminalWorkspaceKeys(
{
targetId: 'target-a',
catalogRevision: 1,
repos: [],
worktreesByRepo: {
repo: [
{
id: 'runtime-worktree',
repoId: 'repo',
hostId: 'ssh:target-a',
runtimeOwnerEnvironmentId: 'runtime-1'
}
]
},
folderWorkspaces: [
{
id: 'folder-b',
projectGroupId: 'group-b',
folderPath: '/srv/project',
connectionId: 'target-b'
}
],
projectGroups: [
{
id: 'group-b',
parentGroupId: null,
connectionId: 'target-b',
executionHostId: 'ssh:target-b'
}
]
},
{
'runtime-worktree': [tab('runtime-worktree', 'ssh:target-a@@pty-1')],
'folder:folder-b': [tab('folder:folder-b', 'ssh:target-a@@pty-2')]
}
)
expect(keys).toEqual(new Set())
})
})
src/renderer/src/store/slices/direct-ssh-terminal-workspace-scope.ts
+130
View File
@@ -0,0 +1,130 @@
import {
getRepoExecutionHostId,
parseExecutionHostId,
toSshExecutionHostId
} from '../../../../shared/execution-host'
import { isPathInsideOrEqual } from '../../../../shared/cross-platform-path'
import { getProjectGroupSubtreeIds } from '../../../../shared/project-groups'
import { parseAppSshPtyId } from '../../../../shared/ssh-pty-id'
import type { TerminalTab } from '../../../../shared/types'
import { folderWorkspaceKey, worktreeWorkspaceKey } from '../../../../shared/workspace-scope'
import {
resolveDirectSshTargetScope,
type DirectSshTargetScopeInput
} from '../../lib/direct-ssh-target-scope'
function isExplicitContradictoryHost(
rawHostId: string | null | undefined,
expectedHostId: string
): boolean {
if (!rawHostId?.trim()) {
return false
}
const parsed = parseExecutionHostId(rawHostId)
return parsed == null || parsed.id !== expectedHostId
}
function worktreeHasContradictoryOwner(
input: DirectSshTargetScopeInput,
worktreeId: string
): boolean {
const rows = [
...Object.values(input.worktreesByRepo ?? {}).flat(),
...Object.values(input.detectedWorktreesByRepo ?? {}).flatMap((entry) => entry.worktrees)
].filter((worktree) => worktree.id === worktreeId)
const expectedHostId = toSshExecutionHostId(input.targetId)
for (const row of rows) {
if (
row.runtimeOwnerEnvironmentId?.trim() ||
isExplicitContradictoryHost(row.hostId, expectedHostId)
) {
return true
}
const repos = input.repos.filter((repo) => repo.id === row.repoId)
if (repos.some((repo) => getRepoExecutionHostId(repo) !== expectedHostId)) {
return true
}
}
const restored = input.restoredRuntimeHostIdByWorkspaceSessionKey
return (
isExplicitContradictoryHost(restored?.[worktreeId], expectedHostId) ||
isExplicitContradictoryHost(restored?.[worktreeWorkspaceKey(worktreeId)], expectedHostId)
)
}
function folderHasContradictoryOwner(
input: DirectSshTargetScopeInput,
folderWorkspaceId: string
): boolean {
const expectedHostId = toSshExecutionHostId(input.targetId)
const folders = (input.folderWorkspaces ?? []).filter((folder) => folder.id === folderWorkspaceId)
for (const folder of folders) {
if (folder.connectionId?.trim() && folder.connectionId.trim() !== input.targetId) {
return true
}
const groups = (input.projectGroups ?? []).filter((group) => group.id === folder.projectGroupId)
if (
groups.some(
(group) =>
(group.connectionId?.trim() && group.connectionId.trim() !== input.targetId) ||
isExplicitContradictoryHost(group.executionHostId, expectedHostId)
)
) {
return true
}
const groupIds = getProjectGroupSubtreeIds(input.projectGroups ?? [], folder.projectGroupId)
const candidateRepos = input.repos.filter(
(repo) =>
(repo.projectGroupId != null && groupIds.has(repo.projectGroupId)) ||
isPathInsideOrEqual(folder.folderPath, repo.path)
)
if (candidateRepos.some((repo) => getRepoExecutionHostId(repo) !== expectedHostId)) {
return true
}
}
return isExplicitContradictoryHost(
input.restoredRuntimeHostIdByWorkspaceSessionKey?.[folderWorkspaceKey(folderWorkspaceId)],
expectedHostId
)
}
function workspaceHasContradictoryOwner(
input: DirectSshTargetScopeInput,
workspaceKey: string
): boolean {
if (workspaceKey.startsWith('folder:')) {
return folderHasContradictoryOwner(input, workspaceKey.slice('folder:'.length))
}
return worktreeHasContradictoryOwner(input, workspaceKey)
}
export function resolveDirectSshTerminalWorkspaceKeys(
input: DirectSshTargetScopeInput,
tabsByWorktree: Readonly<Record<string, readonly TerminalTab[]>>,
lastKnownRelayPtyIdByTabId: Readonly<Record<string, string>> = {}
): Set<string> {
const keys = new Set(resolveDirectSshTargetScope(input).terminalWorkspaceKeys)
for (const [workspaceKey, tabs] of Object.entries(tabsByWorktree)) {
const liveSshTargets = new Set(
tabs
.map((tab) => parseAppSshPtyId(tab.ptyId ?? '')?.connectionId)
.filter((targetId): targetId is string => Boolean(targetId))
)
const retainedSshTargets = new Set(
tabs
.map((tab) => parseAppSshPtyId(lastKnownRelayPtyIdByTabId[tab.id] ?? '')?.connectionId)
.filter((targetId): targetId is string => Boolean(targetId))
)
const ptyTargets = liveSshTargets.size > 0 ? liveSshTargets : retainedSshTargets
if (
keys.has(workspaceKey) ||
workspaceHasContradictoryOwner(input, workspaceKey) ||
ptyTargets.size !== 1 ||
!ptyTargets.has(input.targetId)
) {
continue
}
keys.add(workspaceKey)
}
return keys
}
src/renderer/src/store/slices/repos-ssh-host-reconciliation.test.ts
+130 -23
View File
@@ -1,4 +1,13 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
import type {
HostQualifiedDetectedWorktreeResult,
ListDetectedWorktreesArgs
} from '../../../../shared/detected-worktree-provider-contract'
import type {
DirectSshAuthority,
SshConnectionState,
SshProviderEpoch
} from '../../../../shared/ssh-types'
import type { Project, ProjectHostSetup, Repo, Worktree } from '../../../../shared/types'
import { createTestStore } from './store-test-helpers'
@@ -59,6 +68,52 @@ function directSshWorktree(targetId: string, displayName = 'main'): Worktree {
}
}
function directSshAuthority(targetId: string): DirectSshAuthority {
return {
targetId,
providerEpoch: `epoch-${targetId}` as SshProviderEpoch,
connectionGeneration: 1
}
}
function connectedSshState(authority: DirectSshAuthority): SshConnectionState {
return {
targetId: authority.targetId,
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: authority.providerEpoch,
connectionGeneration: authority.connectionGeneration
}
}
function qualifiedWorktreeResult(
request: ListDetectedWorktreesArgs,
worktree: Worktree
): HostQualifiedDetectedWorktreeResult {
if (!('expectedAuthority' in request)) {
throw new Error('Expected a direct SSH provider request')
}
return {
status: 'complete',
providerRequestId: request.providerRequestId,
repoId: request.repoId,
authority: {
kind: 'direct-ssh',
executionHostId: request.executionHostId,
...request.expectedAuthority
},
result: {
repoId: request.repoId,
authoritative: true,
source: 'git',
worktrees: [
{ ...worktree, ownership: 'orca-managed', selectedCheckout: false, visible: true }
]
}
}
}
const project: Project = {
id: projectId,
displayName: 'Re-adopted repo',
@@ -188,17 +243,23 @@ describe('SSH repo host reconciliation', () => {
it('rejects an old-host worktree response that resolves after re-adoption', async () => {
const staleWorktree = directSshWorktree('ssh-old', 'stale')
let resolveOldWorktrees!: (value: unknown) => void
const oldWorktrees = new Promise((resolve) => {
const oldAuthority = directSshAuthority('ssh-old')
let providerRequest!: ListDetectedWorktreesArgs
let resolveOldWorktrees!: (value: HostQualifiedDetectedWorktreeResult) => void
const oldWorktrees = new Promise<HostQualifiedDetectedWorktreeResult>((resolve) => {
resolveOldWorktrees = resolve
})
worktreesListDetected.mockReturnValueOnce(oldWorktrees)
worktreesListDetected.mockImplementationOnce((request: ListDetectedWorktreesArgs) => {
providerRequest = request
return oldWorktrees
})
reposList.mockResolvedValue([directSshRepo('ssh-new')])
projectsList.mockResolvedValue([project])
setupsList.mockResolvedValue([directSshSetup('ssh-new')])
const store = createTestStore()
store.setState({
repos: [directSshRepo('ssh-old')],
sshConnectionStates: new Map([['ssh-old', connectedSshState(oldAuthority)]]),
worktreesByRepo: { [repoId]: [staleWorktree] },
detectedWorktreesByRepo: {
[repoId]: {
@@ -214,17 +275,20 @@ describe('SSH repo host reconciliation', () => {
const staleFetch = store.getState().fetchWorktrees(repoId)
await vi.waitFor(() => expect(worktreesListDetected).toHaveBeenCalled())
expect(worktreesListDetected).toHaveBeenCalledOnce()
expect(providerRequest).toMatchObject({
repoId,
executionHostId: 'ssh:ssh-old',
expectedAuthority: oldAuthority
})
store.getState().recordSshRepoReadoptions([readoption])
await store.getState().fetchReposForAllHosts({ remoteHosts: 'skip' })
resolveOldWorktrees({
repoId,
authoritative: true,
source: 'git',
worktrees: [
{ ...staleWorktree, ownership: 'orca-managed', selectedCheckout: false, visible: true }
]
const newAuthority = directSshAuthority('ssh-new')
store.setState({
sshConnectionStates: new Map([['ssh-new', connectedSshState(newAuthority)]])
})
await staleFetch
resolveOldWorktrees(qualifiedWorktreeResult(providerRequest, staleWorktree))
await expect(staleFetch).resolves.toBe(false)
expect(store.getState().worktreesByRepo[repoId]).toEqual([
directSshWorktree('ssh-new', 'stale')
@@ -241,26 +305,69 @@ describe('SSH repo host reconciliation', () => {
it('rejects a worktree response after its final repo owner is removed', async () => {
const staleWorktree = directSshWorktree('ssh-old', 'stale')
let resolveOldWorktrees!: (value: unknown) => void
const oldWorktrees = new Promise((resolve) => {
const oldAuthority = directSshAuthority('ssh-old')
let providerRequest!: ListDetectedWorktreesArgs
let resolveOldWorktrees!: (value: HostQualifiedDetectedWorktreeResult) => void
const oldWorktrees = new Promise<HostQualifiedDetectedWorktreeResult>((resolve) => {
resolveOldWorktrees = resolve
})
worktreesListDetected.mockReturnValueOnce(oldWorktrees)
worktreesListDetected.mockImplementationOnce((request: ListDetectedWorktreesArgs) => {
providerRequest = request
return oldWorktrees
})
const store = createTestStore()
store.setState({ repos: [directSshRepo('ssh-old')] })
store.setState({
repos: [directSshRepo('ssh-old')],
sshConnectionStates: new Map([['ssh-old', connectedSshState(oldAuthority)]])
})
const staleFetch = store.getState().fetchWorktrees(repoId)
await vi.waitFor(() => expect(worktreesListDetected).toHaveBeenCalled())
store.setState({ repos: [], worktreesByRepo: {}, detectedWorktreesByRepo: {} })
resolveOldWorktrees({
expect(worktreesListDetected).toHaveBeenCalledOnce()
expect(providerRequest).toMatchObject({
repoId,
authoritative: true,
source: 'git',
worktrees: [
{ ...staleWorktree, ownership: 'orca-managed', selectedCheckout: false, visible: true }
]
executionHostId: 'ssh:ssh-old',
expectedAuthority: oldAuthority
})
await staleFetch
store.setState({ repos: [], worktreesByRepo: {}, detectedWorktreesByRepo: {} })
resolveOldWorktrees(qualifiedWorktreeResult(providerRequest, staleWorktree))
await expect(staleFetch).resolves.toBe(false)
expect(store.getState().worktreesByRepo[repoId]).toBeUndefined()
expect(store.getState().detectedWorktreesByRepo[repoId]).toBeUndefined()
})
it.each([
['contradictory', { ...directSshRepo('ssh-old'), connectionId: 'ssh-new' }],
[
'malformed',
{ ...directSshRepo('ssh-old'), executionHostId: 'ssh:%' as Repo['executionHostId'] }
],
['empty explicit host', { ...directSshRepo('ssh-old'), executionHostId: '' }],
['empty connection', { ...directSshRepo('ssh-old'), connectionId: ' ' }]
])('rejects a late worktree response when repo owner provenance becomes %s', async (_, owner) => {
const staleWorktree = directSshWorktree('ssh-old', 'stale')
const oldAuthority = directSshAuthority('ssh-old')
let providerRequest!: ListDetectedWorktreesArgs
let resolveOldWorktrees!: (value: HostQualifiedDetectedWorktreeResult) => void
const oldWorktrees = new Promise<HostQualifiedDetectedWorktreeResult>((resolve) => {
resolveOldWorktrees = resolve
})
worktreesListDetected.mockImplementationOnce((request: ListDetectedWorktreesArgs) => {
providerRequest = request
return oldWorktrees
})
const store = createTestStore()
store.setState({
repos: [directSshRepo('ssh-old')],
sshConnectionStates: new Map([['ssh-old', connectedSshState(oldAuthority)]])
})
const staleFetch = store.getState().fetchWorktrees(repoId)
await vi.waitFor(() => expect(worktreesListDetected).toHaveBeenCalledOnce())
store.setState({ repos: [owner as Repo] })
resolveOldWorktrees(qualifiedWorktreeResult(providerRequest, staleWorktree))
await expect(staleFetch).resolves.toBe(false)
expect(store.getState().worktreesByRepo[repoId]).toBeUndefined()
expect(store.getState().detectedWorktreesByRepo[repoId]).toBeUndefined()
src/renderer/src/store/slices/runtime-environment-ssh.test.ts
+41 -4
View File
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest'
import type { SshConnectionState } from '../../../../shared/ssh-types'
import type { SshConnectionState, SshProviderEpoch } from '../../../../shared/ssh-types'
import type { RuntimeStatus } from '../../../../shared/runtime-types'
import { createTestStore } from './store-test-helpers'
import {
@@ -15,7 +15,14 @@ function connState(
targetId: string,
status: SshConnectionState['status'] = 'connected'
): SshConnectionState {
return { targetId, status, error: null, reconnectAttempt: 0 }
return {
targetId,
status,
error: null,
reconnectAttempt: 0,
providerEpoch: `${targetId}-provider-epoch` as SshProviderEpoch,
connectionGeneration: 7
}
}
function markReachable(store: ReturnType<typeof createTestStore>, environmentId: string): void {
@@ -48,6 +55,10 @@ describe('runtime-environment-ssh slice', () => {
// Environment A's bucket: populated.
const bucketA = state.sshStateByEnvironment.get(ENV_A)
expect(bucketA?.connectionStates.get('ssh-a')?.status).toBe('connected')
expect(bucketA?.connectionStates.get('ssh-a')).toMatchObject({
providerEpoch: 'ssh-a-provider-epoch',
connectionGeneration: 7
})
expect(bucketA?.targetLabels.get('ssh-a')).toBe('a-box')
expect(bucketA?.removedTargetLabels.get('ssh-dead')).toBe('old box')
expect(bucketA?.targetsHydrated).toBe(true)
@@ -80,7 +91,7 @@ describe('runtime-environment-ssh slice', () => {
expect(store.getState().sshStateByEnvironment.get(ENV_A)).toBe(bucketBefore)
})
it('publishes an authoritative connection generation change in the owning bucket', () => {
it('publishes a generation-only authority change without dropping the provider epoch', () => {
const store = createTestStore()
store.getState().setEnvironmentSshConnectionState(ENV_A, 'ssh-a', {
...connState('ssh-a'),
@@ -95,7 +106,33 @@ describe('runtime-environment-ssh slice', () => {
const bucketAfter = store.getState().sshStateByEnvironment.get(ENV_A)
expect(bucketAfter).not.toBe(bucketBefore)
expect(bucketAfter?.connectionStates.get('ssh-a')?.connectionGeneration).toBe(2)
expect(bucketAfter?.connectionStates.get('ssh-a')).toMatchObject({
providerEpoch: 'ssh-a-provider-epoch',
connectionGeneration: 2
})
})
it('publishes an epoch-only authority change without dropping the connection generation', () => {
const store = createTestStore()
store.getState().setEnvironmentSshConnectionState(ENV_A, 'ssh-a', {
...connState('ssh-a'),
providerEpoch: 'provider-a' as never,
connectionGeneration: 1
})
const bucketBefore = store.getState().sshStateByEnvironment.get(ENV_A)
store.getState().setEnvironmentSshConnectionState(ENV_A, 'ssh-a', {
...connState('ssh-a'),
providerEpoch: 'provider-b' as never,
connectionGeneration: 1
})
const bucketAfter = store.getState().sshStateByEnvironment.get(ENV_A)
expect(bucketAfter).not.toBe(bucketBefore)
expect(bucketAfter?.connectionStates.get('ssh-a')).toMatchObject({
providerEpoch: 'provider-b',
connectionGeneration: 1
})
})
it('flips the hydrated flag on the first fetch of an empty target list', () => {
src/renderer/src/store/slices/ssh-target-cleanup.ts
+50 -9
View File
@@ -1,6 +1,7 @@
import type { AppState } from '../types'
import type { SshConnectionState, SshTarget } from '../../../../shared/ssh-types'
import { parseAppSshPtyId } from '../../../../shared/ssh-pty-id'
import { resolveDirectSshTargetScope } from '../../lib/direct-ssh-target-scope'
export function sshConnectionStatesEqual(
a: SshConnectionState | undefined,
@@ -11,6 +12,7 @@ export function sshConnectionStatesEqual(
a?.status === b.status &&
a?.error === b.error &&
a?.reconnectAttempt === b.reconnectAttempt &&
a?.providerEpoch === b.providerEpoch &&
a?.connectionGeneration === b.connectionGeneration &&
a?.supportsFolderDownload === b.supportsFolderDownload &&
a?.remotePlatform === b.remotePlatform
@@ -28,15 +30,20 @@ export function sshTargetLabelsEqual(
}
function collectSshTargetTerminalTabIds(state: AppState, targetId: string): Set<string> {
const repoIds = new Set(
state.repos.filter((repo) => repo.connectionId === targetId).map((repo) => repo.id)
)
const targetWorktreeIds = resolveDirectSshTargetScope({
targetId,
catalogRevision: 0,
repos: state.repos,
worktreesByRepo: state.worktreesByRepo,
detectedWorktreesByRepo: state.detectedWorktreesByRepo,
restoredRuntimeHostIdByWorkspaceSessionKey: state.restoredRuntimeHostIdByWorkspaceSessionKey
}).gitWorktreeIds
const tabIds = new Set<string>()
for (const [repoId, worktrees] of Object.entries(state.worktreesByRepo)) {
if (!repoIds.has(repoId)) {
continue
}
for (const worktrees of Object.values(state.worktreesByRepo)) {
for (const worktree of worktrees) {
if (!targetWorktreeIds.has(worktree.id)) {
continue
}
for (const tab of state.tabsByWorktree[worktree.id] ?? []) {
tabIds.add(tab.id)
}
@@ -80,6 +87,19 @@ function omitRemovedSshTargetTabSessions(
return { next, removed }
}
function omitRemovedSshTargetRecovery<T extends { authority: { targetId: string } }>(
entries: Record<string, T>,
targetId: string,
targetTabIds: ReadonlySet<string>
): { next: Record<string, T>; removed: boolean } {
const next = Object.fromEntries(
Object.entries(entries).filter(
([tabId, entry]) => !targetTabIds.has(tabId) && entry.authority.targetId !== targetId
)
)
return { next, removed: Object.keys(next).length !== Object.keys(entries).length }
}
function clearSshTargetTabPtyState(
state: AppState,
targetId: string,
@@ -160,6 +180,21 @@ export function buildRemovedSshTargetCleanupPatch(
// sweep now reads it as liveness, so clear it here too (#9911).
const { next: nextPendingReconnect, removed: removedPendingReconnect } =
omitRemovedSshTargetTabSessions(state.pendingReconnectPtyIdByTabId, targetId, targetTabIds)
const { next: nextPaneRetries, removed: removedPaneRetries } = omitRemovedSshTargetRecovery(
state.directSshPaneRetryByTabId,
targetId,
targetTabIds
)
const { next: nextLiveBindings, removed: removedLiveBindings } = omitRemovedSshTargetRecovery(
state.directSshLivePtyBindingByTabId,
targetId,
targetTabIds
)
const { next: nextRetryHistory, removed: removedRetryHistory } = omitRemovedSshTargetRecovery(
state.directSshPaneRetryHistoryByTabId,
targetId,
targetTabIds
)
const nextDeferredTargets = state.deferredSshReconnectTargets.filter((id) => id !== targetId)
const nextTransientClearedConnections = {
@@ -210,7 +245,10 @@ export function buildRemovedSshTargetCleanupPatch(
removedCredentialRequest ||
removedDeferredTarget ||
removedDeferredSession ||
removedPendingReconnect
removedPendingReconnect ||
removedPaneRetries ||
removedLiveBindings ||
removedRetryHistory
if (!changed) {
return null
}
@@ -237,6 +275,9 @@ export function buildRemovedSshTargetCleanupPatch(
...(removedCredentialRequest ? { sshCredentialQueue: nextCredentialQueue } : {}),
...(removedDeferredTarget ? { deferredSshReconnectTargets: nextDeferredTargets } : {}),
...(removedDeferredSession ? { deferredSshSessionIdsByTabId: nextDeferredSessions } : {}),
...(removedPendingReconnect ? { pendingReconnectPtyIdByTabId: nextPendingReconnect } : {})
...(removedPendingReconnect ? { pendingReconnectPtyIdByTabId: nextPendingReconnect } : {}),
...(removedPaneRetries ? { directSshPaneRetryByTabId: nextPaneRetries } : {}),
...(removedLiveBindings ? { directSshLivePtyBindingByTabId: nextLiveBindings } : {}),
...(removedRetryHistory ? { directSshPaneRetryHistoryByTabId: nextRetryHistory } : {})
}
}
src/renderer/src/store/slices/ssh.test.ts
+181 -1
View File
@@ -1,5 +1,6 @@
import { describe, expect, it } from 'vitest'
import { toAppSshPtyId } from '../../../../shared/ssh-pty-id'
import type { SshProviderEpoch } from '../../../../shared/ssh-types'
import { createTestStore, makeTab, makeWorktree, TEST_REPO } from './store-test-helpers'
describe('createSshSlice', () => {
@@ -12,6 +13,16 @@ describe('createSshSlice', () => {
const removedPtyId = toAppSshPtyId(targetId, 'pty-live')
const staleLastKnownPtyId = toAppSshPtyId(targetId, 'pty-last-known')
const otherPtyId = toAppSshPtyId(otherTargetId, 'pty-other')
const removedAuthority = {
targetId,
providerEpoch: 'removed-epoch' as SshProviderEpoch,
connectionGeneration: 1
} as const
const otherAuthority = {
targetId: otherTargetId,
providerEpoch: 'other-epoch' as SshProviderEpoch,
connectionGeneration: 2
} as const
store.setState({
repos: [
@@ -114,7 +125,44 @@ describe('createSshSlice', () => {
'tab-ssh': toAppSshPtyId(targetId, 'pty-1'),
'tab-stale-encoded': toAppSshPtyId(targetId, 'pty-9'),
'tab-other': toAppSshPtyId(otherTargetId, 'pty-2')
}
},
directSshPaneRetryByTabId: {
'tab-ssh': {
attemptId: 'removed-attempt',
authority: removedAuthority,
tabGeneration: 1,
startedAt: 10
},
'tab-stale-retry': {
attemptId: 'stale-removed-attempt',
authority: removedAuthority,
tabGeneration: 2,
startedAt: 11
},
'tab-other': {
attemptId: 'other-attempt',
authority: otherAuthority,
tabGeneration: 3,
startedAt: 12
}
} as never,
directSshLivePtyBindingByTabId: {
'tab-ssh': {
authority: removedAuthority,
tabGeneration: 1,
ptyId: removedPtyId
},
'tab-other': {
authority: otherAuthority,
tabGeneration: 3,
ptyId: otherPtyId
}
} as never,
directSshPaneRetryHistoryByTabId: {
'tab-ssh': { authority: removedAuthority, attemptedAt: [10] },
'tab-stale-retry': { authority: removedAuthority, attemptedAt: [11] },
'tab-other': { authority: otherAuthority, attemptedAt: [12] }
} as never
})
store.getState().clearRemovedSshTargetState(targetId)
@@ -139,6 +187,12 @@ describe('createSshSlice', () => {
expect(state.pendingReconnectPtyIdByTabId).toEqual({
'tab-other': toAppSshPtyId(otherTargetId, 'pty-2')
})
expect(Object.keys(state.directSshPaneRetryByTabId)).toEqual(['tab-other'])
expect(Object.keys(state.directSshLivePtyBindingByTabId)).toEqual(['tab-other'])
expect(Object.keys(state.directSshPaneRetryHistoryByTabId)).toEqual(['tab-other'])
expect(state.directSshPaneRetryByTabId['tab-other']?.authority.targetId).toBe(otherTargetId)
expect(state.directSshLivePtyBindingByTabId['tab-other']?.ptyId).toBe(otherPtyId)
expect(state.directSshPaneRetryHistoryByTabId['tab-other']?.attemptedAt).toEqual([12])
expect(state.tabsByWorktree[worktreeId][0]).toMatchObject({ id: 'tab-ssh', ptyId: null })
expect('pendingActivationSpawn' in state.tabsByWorktree[worktreeId][0]).toBe(false)
expect(state.ptyIdsByTabId['tab-ssh']).toEqual([])
@@ -163,6 +217,107 @@ describe('createSshSlice', () => {
})
})
it("preserves another SSH target's terminal ledgers when repo ids collide", () => {
const store = createTestStore()
const removedTargetId = 'target-a'
const survivingTargetId = 'target-b'
const removedWorktreeId = 'shared::/srv/a/work'
const survivingWorktreeId = 'shared::/srv/b/work'
const removedPtyId = toAppSshPtyId(removedTargetId, 'pty-a')
const survivingPtyId = toAppSshPtyId(survivingTargetId, 'pty-b')
const removedAuthority = {
targetId: removedTargetId,
providerEpoch: 'epoch-a' as SshProviderEpoch,
connectionGeneration: 1
}
const survivingAuthority = {
targetId: survivingTargetId,
providerEpoch: 'epoch-b' as SshProviderEpoch,
connectionGeneration: 2
}
const survivingAttempt = {
attemptId: 'attempt-b',
authority: survivingAuthority,
tabGeneration: 4,
startedAt: 20
}
const survivingBinding = {
authority: survivingAuthority,
tabGeneration: 4,
ptyId: survivingPtyId
}
const survivingHistory = { authority: survivingAuthority, attemptedAt: [20] }
store.setState({
repos: [
{
...TEST_REPO,
id: 'shared',
connectionId: removedTargetId,
executionHostId: 'ssh:target-a'
},
{
...TEST_REPO,
id: 'shared',
connectionId: survivingTargetId,
executionHostId: 'ssh:target-b'
}
],
worktreesByRepo: {
shared: [
makeWorktree({
id: removedWorktreeId,
repoId: 'shared',
hostId: 'ssh:target-a'
}),
makeWorktree({
id: survivingWorktreeId,
repoId: 'shared',
hostId: 'ssh:target-b'
})
]
},
tabsByWorktree: {
[removedWorktreeId]: [
makeTab({ id: 'tab-a', worktreeId: removedWorktreeId, ptyId: removedPtyId })
],
[survivingWorktreeId]: [
makeTab({ id: 'tab-b', worktreeId: survivingWorktreeId, ptyId: survivingPtyId })
]
},
ptyIdsByTabId: { 'tab-a': [removedPtyId], 'tab-b': [survivingPtyId] },
lastKnownRelayPtyIdByTabId: { 'tab-a': removedPtyId, 'tab-b': survivingPtyId },
directSshPaneRetryByTabId: {
'tab-a': {
attemptId: 'attempt-a',
authority: removedAuthority,
tabGeneration: 3,
startedAt: 10
},
'tab-b': survivingAttempt
} as never,
directSshLivePtyBindingByTabId: {
'tab-a': { authority: removedAuthority, tabGeneration: 3, ptyId: removedPtyId },
'tab-b': survivingBinding
} as never,
directSshPaneRetryHistoryByTabId: {
'tab-a': { authority: removedAuthority, attemptedAt: [10] },
'tab-b': survivingHistory
} as never
})
store.getState().clearRemovedSshTargetState(removedTargetId)
const state = store.getState()
expect(state.tabsByWorktree[removedWorktreeId][0]?.ptyId).toBeNull()
expect(state.tabsByWorktree[survivingWorktreeId][0]?.ptyId).toBe(survivingPtyId)
expect(state.ptyIdsByTabId['tab-b']).toEqual([survivingPtyId])
expect(state.lastKnownRelayPtyIdByTabId['tab-b']).toBe(survivingPtyId)
expect(state.directSshPaneRetryByTabId['tab-b']).toBe(survivingAttempt)
expect(state.directSshLivePtyBindingByTabId['tab-b']).toBe(survivingBinding)
expect(state.directSshPaneRetryHistoryByTabId['tab-b']).toBe(survivingHistory)
})
it('keeps SSH target label references stable when refreshed metadata is unchanged', () => {
const store = createTestStore()
const labels = new Map([['ssh-1', 'Remote']])
@@ -233,6 +388,31 @@ describe('createSshSlice', () => {
expect(store.getState().sshConnectionStates.get('ssh-1')?.connectionGeneration).toBe(2)
})
it('publishes an authoritative SSH provider epoch change', () => {
const store = createTestStore()
store.getState().setSshConnectionState('ssh-1', {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'provider-a' as never,
connectionGeneration: 1
})
const previousState = store.getState()
store.getState().setSshConnectionState('ssh-1', {
targetId: 'ssh-1',
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: 'provider-b' as never,
connectionGeneration: 1
})
expect(store.getState()).not.toBe(previousState)
expect(store.getState().sshConnectionStates.get('ssh-1')?.providerEpoch).toBe('provider-b')
})
it('publishes a connected-state folder capability change', () => {
const store = createTestStore()
store.getState().setSshConnectionState('ssh-1', {
src/renderer/src/store/slices/store-cascades.test.ts
+136
View File
@@ -3,6 +3,8 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'
import { buildWorktreeComparator } from '@/components/sidebar/smart-sort'
import type * as AgentStatusModule from '@/lib/agent-status'
import { getDefaultSettings } from '../../../../shared/constants'
import type { SshProviderEpoch } from '../../../../shared/ssh-types'
import type { DirectSshPaneRetryAttemptId } from './direct-ssh-terminal-recovery'
import { createCompatibleRuntimeStatusResponseIfNeeded } from '../../runtime/runtime-compatibility-test-fixture'
import {
clearRuntimeCompatibilityCacheForTests,
@@ -1302,6 +1304,140 @@ describe('setActiveWorktree', () => {
)
})
it('moves current direct SSH binding evidence when detaching a live pane', () => {
const store = createTestStore()
const wt = 'repo1::/path/wt1'
const sourceTabId = 'tab-source'
const targetTabId = 'tab-target'
const authority = {
targetId: 'target-a',
providerEpoch: 'epoch-a' as SshProviderEpoch,
connectionGeneration: 1
}
const detachedPtyId = 'ssh:target-a@@pty-detached'
seedStore(store, {
tabsByWorktree: {
[wt]: [
makeTab({ id: sourceTabId, worktreeId: wt, ptyId: detachedPtyId }),
makeTab({ id: targetTabId, worktreeId: wt, ptyId: null })
]
},
ptyIdsByTabId: { [sourceTabId]: [detachedPtyId], [targetTabId]: [] },
directSshLivePtyBindingByTabId: {
[sourceTabId]: {
attemptId: 'live-detach' as DirectSshPaneRetryAttemptId,
authority,
tabGeneration: 0,
ptyId: detachedPtyId
}
},
directSshPaneRetryHistoryByTabId: {
[sourceTabId]: { authority, attemptedAt: [1] }
},
sshConnectionStates: new Map([
[
authority.targetId,
{
targetId: authority.targetId,
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: authority.providerEpoch,
connectionGeneration: authority.connectionGeneration
}
]
])
})
store.getState().syncPaneDetachPtyOwnership({
detachedLeafId: '11111111-1111-4111-8111-111111111111',
detachedPtyId,
sourceLayout: makeLayout(),
sourceTabId,
targetTabId
})
const state = store.getState()
expect(state.directSshPaneRetryByTabId[sourceTabId]).toBeUndefined()
expect(state.directSshLivePtyBindingByTabId[sourceTabId]).toBeUndefined()
expect(state.directSshPaneRetryHistoryByTabId[sourceTabId]).toBeUndefined()
expect(state.directSshLivePtyBindingByTabId[targetTabId]).toEqual({
attemptId: 'live-detach',
authority,
tabGeneration: 0,
ptyId: detachedPtyId
})
expect(state.directSshPaneRetryHistoryByTabId[targetTabId]).toEqual({
authority,
attemptedAt: [1]
})
})
it('rearms a current pending SSH detach as live destination evidence', () => {
const store = createTestStore()
const wt = 'repo1::/path/wt1'
const sourceTabId = 'tab-source'
const targetTabId = 'tab-target'
const authority = {
targetId: 'target-a',
providerEpoch: 'epoch-a' as SshProviderEpoch,
connectionGeneration: 1
}
const detachedPtyId = 'ssh:target-a@@pty-detached'
seedStore(store, {
tabsByWorktree: {
[wt]: [
makeTab({ id: sourceTabId, worktreeId: wt, ptyId: detachedPtyId, generation: 2 }),
makeTab({ id: targetTabId, worktreeId: wt, ptyId: null })
]
},
ptyIdsByTabId: { [sourceTabId]: [detachedPtyId], [targetTabId]: [] },
directSshPaneRetryByTabId: {
[sourceTabId]: {
attemptId: 'pending-detach' as DirectSshPaneRetryAttemptId,
authority,
tabGeneration: 2,
startedAt: 1
}
},
sshConnectionStates: new Map([
[
authority.targetId,
{
targetId: authority.targetId,
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: authority.providerEpoch,
connectionGeneration: authority.connectionGeneration
}
]
])
})
store.getState().syncPaneDetachPtyOwnership({
detachedLeafId: '11111111-1111-4111-8111-111111111111',
detachedPtyId,
sourceLayout: makeLayout(),
sourceTabId,
targetTabId
})
const state = store.getState()
expect(state.directSshPaneRetryByTabId[sourceTabId]).toBeUndefined()
expect(state.directSshLivePtyBindingByTabId[sourceTabId]).toBeUndefined()
expect(state.directSshPaneRetryHistoryByTabId[sourceTabId]).toBeUndefined()
expect(state.directSshLivePtyBindingByTabId[targetTabId]).toEqual({
attemptId: 'pending-detach',
authority,
tabGeneration: 0,
ptyId: detachedPtyId
})
expect(state.tabsByWorktree[wt].find((tab) => tab.id === targetTabId)?.ptyId).toBe(
detachedPtyId
)
})
// Regression for #9911: a split SSH tab's single relay slot points at the
// last-bound pane; when it exits, clearTabPtyId must promote a surviving pane
// instead of clearing, or a later relay-drop bulk-clear leaves the survivor
src/renderer/src/store/slices/tabs.test.ts
+159 -1
View File
@@ -104,7 +104,13 @@ const mockApi = {
// @ts-expect-error -- mock
globalThis.window = { api: mockApi }
import { createTestStore, makeOpenFile, makeTabGroup, makeUnifiedTab } from './store-test-helpers'
import {
createTestStore,
makeOpenFile,
makeTabGroup,
makeUnifiedTab,
makeWorktree
} from './store-test-helpers'
const WT = 'repo1::/tmp/feature'
@@ -1783,6 +1789,158 @@ describe('TabsSlice', () => {
expect(store.getState().unifiedTabsByWorktree).toEqual({})
})
it('replaces only explicitly scoped worktree tab chrome', () => {
const siblingWorktreeId = 'repo2::/tmp/sibling'
const targetGroup = makeTabGroup({
id: 'group-target',
worktreeId: WT,
activeTabId: 'target-old',
tabOrder: ['target-old']
})
const siblingGroup = makeTabGroup({
id: 'group-sibling',
worktreeId: siblingWorktreeId,
activeTabId: 'sibling-tab',
tabOrder: ['sibling-tab']
})
const siblingTabs = [
makeUnifiedTab({
id: 'sibling-tab',
worktreeId: siblingWorktreeId,
groupId: siblingGroup.id
})
]
const siblingGroups = [siblingGroup]
store.setState({
worktreesByRepo: {
repo1: [makeWorktree({ id: WT, repoId: 'repo1' })],
repo2: [makeWorktree({ id: siblingWorktreeId, repoId: 'repo2' })]
},
unifiedTabsByWorktree: {
[WT]: [makeUnifiedTab({ id: 'target-old', worktreeId: WT, groupId: targetGroup.id })],
[siblingWorktreeId]: siblingTabs
},
groupsByWorktree: {
[WT]: [targetGroup],
[siblingWorktreeId]: siblingGroups
},
activeGroupIdByWorktree: {
[WT]: targetGroup.id,
[siblingWorktreeId]: siblingGroup.id
},
layoutByWorktree: {
[WT]: { type: 'leaf', groupId: targetGroup.id },
[siblingWorktreeId]: { type: 'leaf', groupId: siblingGroup.id }
}
})
const targetNew = makeUnifiedTab({
id: 'target-new',
worktreeId: WT,
groupId: targetGroup.id,
label: 'Remote target'
})
store.getState().hydrateTabsSession(
{
activeRepoId: 'repo1',
activeWorktreeId: WT,
activeTabId: targetNew.id,
tabsByWorktree: {},
terminalLayoutsByTabId: {},
unifiedTabs: {
[WT]: [targetNew],
[siblingWorktreeId]: [
makeUnifiedTab({
id: 'sibling-replaced',
worktreeId: siblingWorktreeId,
groupId: siblingGroup.id
})
]
},
tabGroups: {
[WT]: [{ ...targetGroup, activeTabId: targetNew.id, tabOrder: [targetNew.id] }],
[siblingWorktreeId]: [
{ ...siblingGroup, activeTabId: 'sibling-replaced', tabOrder: ['sibling-replaced'] }
]
}
},
{ replaceWorkspaceKeys: [WT] }
)
expect(store.getState().unifiedTabsByWorktree[WT]).toEqual([targetNew])
expect(store.getState().unifiedTabsByWorktree[siblingWorktreeId]).toBe(siblingTabs)
expect(store.getState().groupsByWorktree[siblingWorktreeId]).toBe(siblingGroups)
})
it('deletes omitted target chrome while preserving sibling references', () => {
const siblingWorktreeId = 'repo2::/tmp/sibling'
const targetGroup = makeTabGroup({
id: 'group-target',
worktreeId: WT,
activeTabId: 'target-tab',
tabOrder: ['target-tab']
})
const siblingGroup = makeTabGroup({
id: 'group-sibling',
worktreeId: siblingWorktreeId,
activeTabId: 'sibling-tab',
tabOrder: ['sibling-tab']
})
const siblingTabs = [
makeUnifiedTab({
id: 'sibling-tab',
worktreeId: siblingWorktreeId,
groupId: siblingGroup.id
})
]
const siblingGroups = [siblingGroup]
const siblingLayout = { type: 'leaf' as const, groupId: siblingGroup.id }
store.setState({
worktreesByRepo: {
repo1: [makeWorktree({ id: WT, repoId: 'repo1' })],
repo2: [makeWorktree({ id: siblingWorktreeId, repoId: 'repo2' })]
},
unifiedTabsByWorktree: {
[WT]: [makeUnifiedTab({ id: 'target-tab', worktreeId: WT, groupId: targetGroup.id })],
[siblingWorktreeId]: siblingTabs
},
groupsByWorktree: {
[WT]: [targetGroup],
[siblingWorktreeId]: siblingGroups
},
activeGroupIdByWorktree: {
[WT]: targetGroup.id,
[siblingWorktreeId]: siblingGroup.id
},
layoutByWorktree: {
[WT]: { type: 'leaf', groupId: targetGroup.id },
[siblingWorktreeId]: siblingLayout
}
})
store.getState().hydrateTabsSession(
{
activeRepoId: 'repo1',
activeWorktreeId: WT,
activeTabId: null,
tabsByWorktree: {},
terminalLayoutsByTabId: {},
unifiedTabs: {},
tabGroups: {}
},
{ replaceWorkspaceKeys: [WT] }
)
const state = store.getState()
expect(state.unifiedTabsByWorktree).not.toHaveProperty(WT)
expect(state.groupsByWorktree).not.toHaveProperty(WT)
expect(state.activeGroupIdByWorktree).not.toHaveProperty(WT)
expect(state.layoutByWorktree).not.toHaveProperty(WT)
expect(state.unifiedTabsByWorktree[siblingWorktreeId]).toBe(siblingTabs)
expect(state.groupsByWorktree[siblingWorktreeId]).toBe(siblingGroups)
expect(state.layoutByWorktree[siblingWorktreeId]).toBe(siblingLayout)
})
})
// ─── Cross-content-type neighbor selection ────────────────────────
src/renderer/src/store/slices/tabs.ts
+39 -1
View File
@@ -47,6 +47,17 @@ import {
export type TabSplitDirection = 'left' | 'right' | 'up' | 'down'
function replaceWorkspaceRecordKeys<T>(
current: Record<string, T>,
hydrated: Record<string, T>,
workspaceKeys: ReadonlySet<string>
): Record<string, T> {
return {
...Object.fromEntries(Object.entries(current).filter(([key]) => !workspaceKeys.has(key))),
...Object.fromEntries(Object.entries(hydrated).filter(([key]) => workspaceKeys.has(key)))
}
}
export type TabsSlice = {
unifiedTabsByWorktree: Record<string, Tab[]>
// Why: id of the tab whose inline title editor should open; shortcut (tab.rename) sets it, the tab clears it on consume.
@@ -2011,6 +2022,33 @@ export const createTabsSlice: StateCreator<AppState, [], [], TabsSlice> = (set,
validWorktreeIds.add(folderWorkspaceKey(workspace.id))
}
addAdditionalValidWorkspaceKeys(validWorktreeIds, options)
set(buildHydratedTabState(session, validWorktreeIds))
const hydrated = buildHydratedTabState(session, validWorktreeIds)
if (!options?.replaceWorkspaceKeys) {
set(hydrated)
return
}
const replaceWorkspaceKeys = new Set(options.replaceWorkspaceKeys)
set((current) => ({
unifiedTabsByWorktree: replaceWorkspaceRecordKeys(
current.unifiedTabsByWorktree,
hydrated.unifiedTabsByWorktree,
replaceWorkspaceKeys
),
groupsByWorktree: replaceWorkspaceRecordKeys(
current.groupsByWorktree,
hydrated.groupsByWorktree,
replaceWorkspaceKeys
),
activeGroupIdByWorktree: replaceWorkspaceRecordKeys(
current.activeGroupIdByWorktree,
hydrated.activeGroupIdByWorktree,
replaceWorkspaceKeys
),
layoutByWorktree: replaceWorkspaceRecordKeys(
current.layoutByWorktree,
hydrated.layoutByWorktree,
replaceWorkspaceKeys
)
}))
}
})
src/renderer/src/store/slices/terminal-activation-spawn-suppression.ts
+22
View File
@@ -0,0 +1,22 @@
import type { TerminalLayoutSnapshot, TerminalPaneLayoutNode } from '../../../../shared/types'
function countTerminalLayoutLeaves(node: TerminalPaneLayoutNode | null | undefined): number {
if (!node) {
return 0
}
if (node.type === 'leaf') {
return 1
}
return countTerminalLayoutLeaves(node.first) + countTerminalLayoutLeaves(node.second)
}
export function getTerminalActivationSpawnSuppression(
layout: TerminalLayoutSnapshot | undefined
): true | number {
const paneCount = Math.max(
1,
countTerminalLayoutLeaves(layout?.root),
Object.keys(layout?.ptyIdsByLeafId ?? {}).length
)
return paneCount === 1 ? true : paneCount
}
src/renderer/src/store/slices/terminals-hydration.test.ts
+152
View File
@@ -92,6 +92,8 @@ const mockApi = {
globalThis.window = { api: mockApi }
import type { WorkspaceSessionState } from '../../../../shared/types'
import type { SshProviderEpoch } from '../../../../shared/ssh-types'
import type { DirectSshPaneRetryAttemptId } from './direct-ssh-terminal-recovery-types'
import {
FLOATING_TERMINAL_WORKTREE_ID,
getDefaultWorkspaceSession
@@ -710,6 +712,156 @@ describe('hydrateWorkspaceSession', () => {
expect(store.getState().worktreeNavHistoryIndex).toBe(1)
expect(canGoBackWorktreeHistory(store.getState())).toBe(true)
})
it('hydrates and reconnects one SSH target without mutating unrelated state', async () => {
const store = createTestStore()
const targetWorktreeId = 'repo-a::/target'
const siblingWorktreeId = 'repo-b::/sibling'
const localWorktreeId = 'repo-local::/local'
const runtimeWorktreeId = 'repo-runtime::/runtime'
const folderWorktreeId = folderWorkspaceKey('folder-1')
const targetTab = makeTab({ id: 'tab-target', worktreeId: targetWorktreeId, ptyId: null })
const deletedTargetTab = makeTab({ id: 'tab-target-deleted', worktreeId: targetWorktreeId })
const siblingTab = makeTab({
id: 'tab-sibling',
worktreeId: siblingWorktreeId,
ptyId: 'ssh:target-b@@pty-b'
})
const runtimeTab = makeTab({
id: 'tab-runtime',
worktreeId: runtimeWorktreeId,
ptyId: 'runtime:env@@pty-runtime'
})
const localTab = makeTab({ id: 'tab-local', worktreeId: localWorktreeId, ptyId: null })
const folderTab = makeTab({ id: 'tab-folder', worktreeId: folderWorktreeId, ptyId: null })
const siblingTabs = [siblingTab]
const runtimeTabs = [runtimeTab]
const [localTabs, folderTabs] = [[localTab], [folderTab]]
const runtimeOwners = { [runtimeWorktreeId]: 'runtime:env' as const }
const authority = {
targetId: 'target-a',
providerEpoch: 'epoch-a' as SshProviderEpoch,
connectionGeneration: 7
}
const ledgerTabs = [targetTab, deletedTargetTab, siblingTab]
const retry = {
attemptId: 'attempt' as DirectSshPaneRetryAttemptId,
authority,
tabGeneration: 0,
startedAt: 0
}
const retryByTabId = Object.fromEntries(ledgerTabs.map((tab) => [tab.id, retry]))
const liveBinding = { ...retry, ptyId: 'ssh:target-a@@pty-ledger' }
const liveByTabId = Object.fromEntries(ledgerTabs.map((tab) => [tab.id, liveBinding]))
const historyByTabId = Object.fromEntries(
ledgerTabs.map((tab) => [tab.id, { authority, attemptedAt: [0] }])
)
seedStore(store, {
workspaceSessionReady: true,
repos: [
{ ...TEST_REPO, id: 'repo-a', connectionId: 'target-a' },
{ ...TEST_REPO, id: 'repo-b', connectionId: 'target-b' }
],
worktreesByRepo: {
'repo-a': [makeWorktree({ id: targetWorktreeId, repoId: 'repo-a', path: '/target' })],
'repo-b': [makeWorktree({ id: siblingWorktreeId, repoId: 'repo-b', path: '/sibling' })]
},
tabsByWorktree: {
[targetWorktreeId]: [targetTab, deletedTargetTab],
[siblingWorktreeId]: siblingTabs,
[localWorktreeId]: localTabs,
[runtimeWorktreeId]: runtimeTabs,
[folderWorktreeId]: folderTabs
},
ptyIdsByTabId: {
[targetTab.id]: [],
[siblingTab.id]: [siblingTab.ptyId!],
[runtimeTab.id]: [runtimeTab.ptyId!]
},
activeRepoId: 'repo-b',
activeWorktreeId: siblingWorktreeId,
activeTabId: siblingTab.id,
restoredRuntimeHostIdByWorkspaceSessionKey: runtimeOwners,
directSshPaneRetryByTabId: retryByTabId,
directSshLivePtyBindingByTabId: liveByTabId,
directSshPaneRetryHistoryByTabId: historyByTabId,
sshConnectionStates: new Map([
[
authority.targetId,
{
targetId: authority.targetId,
status: 'connected',
error: null,
reconnectAttempt: 0,
providerEpoch: authority.providerEpoch,
connectionGeneration: authority.connectionGeneration
}
]
])
})
const session: WorkspaceSessionState = {
...getDefaultWorkspaceSession(),
activeRepoId: 'repo-a',
activeWorktreeId: targetWorktreeId,
activeTabId: targetTab.id,
tabsByWorktree: {
[targetWorktreeId]: [
{ ...targetTab, ptyId: 'ssh:target-a@@pty-a' },
makeTab({
id: 'tab-wrong-host',
worktreeId: targetWorktreeId,
ptyId: 'ssh:target-b@@pty-wrong'
})
],
[siblingWorktreeId]: siblingTabs,
[runtimeWorktreeId]: runtimeTabs
},
activeWorktreeIdsOnShutdown: [targetWorktreeId],
terminalLayoutsByTabId: {}
}
store.getState().hydrateWorkspaceSession(session, {
directSshAuthority: authority,
replaceWorkspaceKeys: [targetWorktreeId]
})
expect(store.getState().tabsByWorktree[siblingWorktreeId]).toBe(siblingTabs)
expect(store.getState().tabsByWorktree[localWorktreeId]).toBe(localTabs)
expect(store.getState().tabsByWorktree[runtimeWorktreeId]).toBe(runtimeTabs)
expect(store.getState().tabsByWorktree[folderWorktreeId]).toBe(folderTabs)
expect(store.getState().ptyIdsByTabId[siblingTab.id]).toEqual([siblingTab.ptyId])
expect(store.getState().ptyIdsByTabId[runtimeTab.id]).toEqual([runtimeTab.ptyId])
expect(store.getState().restoredRuntimeHostIdByWorkspaceSessionKey).toBe(runtimeOwners)
expect(store.getState().activeWorktreeId).toBe(siblingWorktreeId)
expect(store.getState().pendingReconnectPtyIdByTabId).toEqual({
[targetTab.id]: 'ssh:target-a@@pty-a'
})
const retainedLedgerTabIds = ledgerTabs
.filter((tab) => tab.id !== deletedTargetTab.id)
.map((tab) => tab.id)
.sort()
expect(Object.keys(store.getState().directSshPaneRetryByTabId).sort()).toEqual(
retainedLedgerTabIds
)
expect(Object.keys(store.getState().directSshLivePtyBindingByTabId).sort()).toEqual(
retainedLedgerTabIds
)
expect(Object.keys(store.getState().directSshPaneRetryHistoryByTabId).sort()).toEqual(
retainedLedgerTabIds
)
await store.getState().reconnectPersistedTerminals(undefined, {
directSshAuthority: authority,
workspaceKeys: [targetWorktreeId]
})
const targetTabs = store.getState().tabsByWorktree[targetWorktreeId]
expect(targetTabs.find((tab) => tab.id === targetTab.id)?.ptyId).toBe('ssh:target-a@@pty-a')
expect(targetTabs.find((tab) => tab.id === 'tab-wrong-host')?.ptyId).toBeNull()
expect(store.getState().tabsByWorktree[siblingWorktreeId]).toBe(siblingTabs)
expect(store.getState().tabsByWorktree[runtimeWorktreeId]).toBe(runtimeTabs)
expect(store.getState().workspaceSessionReady).toBe(true)
})
})
describe('hydrationSucceeded flag (issue #1158)', () => {

Some files were not shown because too many files have changed in this diff Show More