mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 08:02:28 +00:00
7fad71e44842e7de293db3300f4a0eefb6040412
573
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Neilandrayim
|
453237cc57 |
fix(terminal): render the row tail the IME preedit overlay covers (#15014)
* fix(terminal): render the covered row tail inside the IME preedit overlay Closes #12545. Composing mid-line hid the character at the cursor for the whole composition. The preedit overlay is an opaque box anchored to the cursor cell, and nothing reaches the pty while composing, so those cells still held their characters — the box simply covered them. `CompositionHelper` now draws the rest of the row after the preedit inside the view, so the composition reads as inserted text pushing the tail right. Four details come with it: - The view is start-anchored while it carries a tail, so the preedit stays put and the pushed tail clips at the right edge; alone, `rtl` still keeps a long preedit's end in view. - It is themed from `options.theme` instead of the stock `#000`/`#FFF`, with any alpha dropped — the view masks the cells it draws over, so a see-through background would re-expose the very characters the tail stands in for. - The helper textarea syncs to the preedit's own bounds, so IME candidate dialogs anchor to the composing text rather than past the rendered tail. - A TUI can repaint the row under an open composition, so `updateCompositionElements` — which already runs on every render — re-reads the remainder and re-renders on change. A string compare adds no layout read. The tail is read with an explicit end column: the cacheable form of `translateToString` arms the line string cache's self-renewing idle-clear timer, and the composition path must own no timers. Geometry is not the cause. Two mature reference terminal implementations compose marked text into the grid rather than into a floating box, and both still blank the cells under it — one of them literally substitutes the marked characters into the row's character array before rasterizing. Moving off the overlay would not have fixed this report; rendering the covered tail is what does. The e2e arm asserts the invariant an opaque overlay owes the grid: it must render every committed cell its bounding rect covers. That is measured from the real rect against the real cell grid, so it fails on the unfixed build with `covers "하" / renders "가"`. Known limitation: the rendered tail is plain-styled while composing (theme foreground on theme background, no per-cell colors); colors return on commit. This is inherent to the overlay, and drawing the preedit into the cell renderer instead would be a far larger change. Co-authored-by: rayim <rayim@fxy.global> * test(e2e): assert the occlusion invariant, not the runner's cell width CI covered four columns where this machine covers two — 34.4px over an 8.43px grid against 12.3px over an 8px grid — so pinning the covered text verbatim pinned the font metrics rather than the behaviour. Assert instead that every committed cell the overlay covers appears in what it draws, which is the actual invariant and holds at any cell width. Still fails against main: covers "하" / renders "가". * fix(terminal): keep the rendered tail's spacing on the grid The composition view is white-space: nowrap, which collapses runs of spaces exactly like normal — it only suppresses wrapping. So a committed tail carrying padding drew its trailing glyph cells left of where the grid has them: measured in Chromium with xterm's own rule, twenty spaces plus a border rendered two cells wide instead of twenty-one. The visible case is Orca's most common IME context — composing inside an agent TUI input box, where the row is a prompt, padding, then a real border glyph the trim cannot drop. A stray border appeared a cell after the preedit while the real one stayed put. xterm sets white-space: pre on its grid rows for this reason; the view was only nowrap-safe while it held preedit text alone. The existing fixtures are all space-free, and the e2e invariant is that the overlay renders everything it covers — collapsing makes it cover less, so both stayed green. Pinned with a padded-row fixture. --------- Co-authored-by: rayim <rayim@fxy.global> |
||
 Jinwoo Hong
|
77ef6bb9ee | fix(terminal): verify agent prompt submission (#14962) | ||
|
Jinwoo Hong
|
fa9b20cb41 | feat(skills): reland private bundle sharing safely (#14934) | ||
|
Neil
|
9f3a912c1e |
fix(terminal): type Option-composed ASCII instead of reporting it as a chord (#14743)
* fix(terminal): preserve Option-composed ASCII input * fix(terminal): preserve Option keyboard protocol semantics * fix(terminal): complete Option keyboard event encoding * fix(terminal): harden Option input encoding * fix(terminal): close keyboard protocol fallback gaps * test(terminal): prove Option-composed ASCII reaches the pty end to end The Option-compose fix had unit coverage only. This drives a live Electron pane whose kitty flags are armed by the application's own CSI > 1 u and asserts the bytes at the pty boundary: composed `@` and Shift-layer `\` arrive as text, configured Option-as-Alt still reports the layout-resolved chord, and a non-ASCII glyph still reaches the app as its alt hotkey. Restoring the pre-fix policy fails exactly the two composed-text scenarios. Also records the ASCII rule's rationale where the rule lives, not only in a test comment. * refactor(terminal): drop the unread Option layers from the layout snapshot The native helper computed an Option and Option+Shift character for every key, shipped both over IPC, validated them in the parser and cached them in the renderer — but no production caller ever asked for them. Only the base and Shift layers are read, and Shift is the one the web layout map cannot supply, which is why the helper exists at all. Removing them halves the helper's UCKeyTranslate work per key and drops the option parameter that six signatures were threading through for nobody. |
||
 Jinjing
|
763b1febeb |
Revert "feat(skills): add private bundle sharing (#14401)" (#14913)
This reverts commit
|
||
 Jinwoo HongandE2E Test
|
757fae28d7 |
feat(skills): add private bundle sharing (#14401)
Co-authored-by: E2E Test <e2e@test.local> |
||
|
Neil
|
fd1dba9db9 |
fix(daemon): validate spawn cwd asynchronously so one dead share cannot freeze every terminal (#14848)
* fix(daemon): validate spawn cwd asynchronously so one dead share cannot freeze every terminal createOrAttach validated the working directory synchronously on the daemon's only thread. Measured on Windows 11 + Ubuntu-24.04: existsSync on an unreachable UNC share 21,022 ms wsl.exe probe, cold distro 1,266 ms wsl.exe probe, warm distro 59 ms existsSync/statSync on healthy \\wsl.localhost 4 ms / 1 ms A single unreachable share therefore blocks the whole RPC loop past the client's 30s request ceiling, so every other terminal stalls behind it and reports `DaemonProtocolError: Request createOrAttach timed out after 30000ms`. The main process already validates asynchronously and passes prevalidatedCwd (ipc/pty.ts); the daemon never got the same treatment. Add validateWorkingDirectoryAsync (one stat, not exists-then-stat, so an unreachable share is not paid for twice) and await it from the daemon spawn preflights. spawnSubprocess now returns SubprocessHandle | Promise<...>, which existing sync stubs still satisfy. Deliberately not bounding the stat with a timeout: the 30s ceiling comes from blocking the shared loop, not from the duration. A timeout cannot tell "slow share" from "gone share", so it would fail spawns that succeed today at 3-8s on a cold VPN mount, and trade an accurate "working directory does not exist" for a guess. The new await opened a race: it sits between the "already exists?" check and the sessions.set that publishes the session, so two concurrent creates for one session id both spawned. Gate creation per session id; distinct ids still spawn in parallel. STA-4470 * fix(daemon): fence async spawn lifecycle |
||
|
Jinwoo Hong
|
d2ffe1f362 | fix(terminal): settle CLI prompts for Claude and Codex (#14608) | ||
|
Jinwoo Hong
|
2eb3e11327 | fix(terminal): make close and handles incarnation-stable (STA-4327) (#14590) | ||
|
Jinjing
|
2b10767d9d |
fix(e2e): unblock golden file-link hover and Windows worktree activate (#14720)
* fix(e2e): unblock golden file-link hover and Windows worktree activate Mac/Windows tmp paths wrap across xterm rows, so locateLink never found the full absolute path. Print ./package.json instead. createGoldenWorktree used os.tmpdir() (Windows 8.3 RUNNER~1) while Git listed the long path, so activateGoldenWorktree never matched. Realpath after worktree add and compare on the Node side. * fix(e2e): handle realpath failures in golden worktree creation Ensure half-built worktrees and branches are rolled back when realpathSync fails, preventing leaks into later test runs. Extract error handling into rollbackGoldenWorktree() for consistent cleanup. |
||
|
Neil
|
9367169888 |
refactor(tests): split every oversized test file off the max-lines suppression list (#14728)
* refactor(tests): split oversized test files off the max-lines suppression list Every `*.test.ts`/`*.spec.ts` that carried an `eslint/oxlint-disable max-lines` directive is now split into focused, behavior-scoped suites that fit the 800-line test budget, with shared setup extracted into co-located `*-test-harness.ts` / `*-test-fixtures.ts` modules (300-line budget). 83 files became ~930; the largest output is 797 effective lines. `orca-runtime.test.ts` is intentionally untouched. Test bodies were moved by scripted line-range slicing rather than retyped, so assertions are byte-identical. The only permitted body edits were mechanical rebinding where a shared value moved into a harness (e.g. `tmpHome` -> `homes.tmpHome`). Registries that enumerate test files were updated in lockstep: - config/max-lines-baseline.txt: pruned 341 -> 258 entries (all 83 removed). - config/reliability-gates.jsonc: 33 gates repointed at the split files, with assertionRefs split per file where a gate's coverage now spans several. - .github/workflows/pr.yml: the real-zsh lane now lists the 4 split files that actually exercise zsh, so they keep running in the dedicated shell lane. Also renamed agent-hooks `server-test-fixtures.ts` to `server.test-fixtures.ts` so the global-fetch call-site audit keeps skipping it, and added `.js` extensions to the CLI suites' dynamic harness imports (node16 resolution) to unbreak `build:cli`. Verification: full suite 52,449 passing vs 52,448 at baseline with zero assertions lost; `pnpm lint`, `pnpm typecheck`, and `pnpm build:cli` all exit 0; the terminal-pane e2e spec runs 31/31 headless. * refactor(tests): split hook-idle arbitration suite that oxfmt pushed over budget The pre-commit oxfmt pass reflowed pty-connection-hook-idle-arbitration.test.ts to 811 effective lines, 11 over the test budget. Split the hook-completion side effect and replacement-agent veto cases into their own suite; both files now sit well under the cap and the 15 tests are unchanged. * test: port upstream test changes into the split files after rebase Rebasing onto main surfaced 27 tests that main had added to files this branch deleted, plus edits to tests that had already moved. Taking the deletion side of those modify/delete conflicts would have dropped that coverage silently, so each upstream change is ported into the split file that now owns the behavior — for example main's six orchestration mailbox tests land across orchestration-runs, -send, and -check. Also repoints `orchestration.notification-mailbox-consistency`, a gate main added after this branch's gate remap, at those same three split files, and re-prunes the max-lines baseline against main's (257 entries). Verified: all 27 upstream test titles present; full suite 52,761 passing with the only diff vs baseline being 12 tests main itself removed and 3 that moved from skipped to passing; lint and typecheck exit 0. * fix(test): flush pending continuations before tearing down terminal test globals CI shard 5/16 failed on both Node 24 and 26 with `ReferenceError: window is not defined` from pty-connection.ts, surfacing through pty-connection-daemon-snapshot-replay.test.ts. The reattach/settle chains `await` a real promise and then touch `window.api`. Under fake timers those continuations cannot run, so they only become schedulable once restoreTerminalTestGlobals() switches back to real timers — which previously happened immediately before `delete globalThis.window`, so a late continuation threw and failed the whole file. Flush async ticks in that window instead. This is latent in the source rather than new: the pre-split 25k-line file kept running other tests after these, which gave the chains time to settle before teardown. Splitting the file moved teardown directly behind them. * fix(test): keep an inert window after terminal test teardown instead of deleting it The async-tick flush was not enough: the reattach/settle chain can resolve after teardown regardless of how long we drain, so CI shard 5/16 still failed with `ReferenceError: window is not defined` from pty-connection.ts. A real renderer never loses `window`, so deleting it was the artificial part. Swap in an inert proxy whose properties resolve to callables and whose calls resolve to undefined, making a late `window.api.pty.*` call a harmless no-op. The next test replaces it wholesale via installTerminalTestGlobals(), and no test asserts that `window` is absent. |
||
 Brennan Benson
|
66dfdc456f |
feat(computer-use): support macOS middle click and stop the silent left-click fallback (#14721)
* feat(computer-use): support macOS middle click and gate the AX click path `--mouse-button middle` already validated end-to-end through the CLI, the zod schema, and the provider validator, and both the Windows and Linux providers honored it. Only the macOS provider rejected it outright with "middle-click is not yet supported", so the flag was a dead end on the one platform that has no fallback. Two changes: - Add `.middle` to the macOS button mapping. macOS has no dedicated middle event family, so it rides `otherMouseDown`/`otherMouseUp` with the button number carried by `mouseButton: .center`; that constructor argument is honored for exactly the `otherMouse*` types, so no extra field write is needed. - Validate the requested button before the accessibility fast path, and skip that path for buttons it cannot express. Previously the raw string was read unvalidated, and `performClickAction` only special-cased `right`, so `click --mouse-button middle --element-index N` (no modifiers, count 1) fell through to `AXPress` — a left click — and reported success with `path: "accessibility"`. Any unrecognized button string did the same. This matches guards the Windows and Linux providers already had. The button enum moves into `OrcaComputerUseMacOSCore` so it is unit-testable; `main.swift` keeps only the CoreGraphics mapping. Also documents `--mouse-button` in the computer-use skill guide, which never mentioned the flag, so agents on Windows and Linux had no way to discover it. * test(computer-use): cover macOS middle click in the real-desktop e2e suite * test(computer-use): prove macOS middle-click delivery |
||
|
Neil
|
a663f1bd00 |
fix(terminal): hold a cursor chord until the composing syllable commits (#14730)
* fix(terminal): hold a cursor chord until the composing syllable commits The composed glyph reaches the pty from the composition session-end handler, which runs after the chord's keydown. Only Enter was held for that, so every other chord went straight out on the transport and overtook the text it was typed after: with 가나 on the line, typing 가나다 and pressing Cmd+Left left 다가나, the composing 다 landed at the cursor's destination. Defer any sendInput chord while a composition is live or its session has not yet flushed. Korean 2-Set shows the shape most clearly — the platform replays the chord unmarked after keyup, so isComposing is already false while the session is still pending. No fallback timer on this path. A newline arriving late still arrives, which is what that timer is for; a chord arriving mid-preedit is the corruption the wait exists to prevent, and a conversion can hold its candidate window open for seconds. Dropping the chord costs one keypress, firing early costs a line. Pane commands are unaffected: they are not sendInput actions. Fixes #12871 * test(e2e): pin the composing-chord order at the pty The unit coverage asserts the handler's ordering against a synthetic transport. This asserts it where it is actually observable: the committed glyph and the chord reach the pty by two different routes, and only their merged order is visible to the shell. Verified to discriminate — against keyboard-handlers.ts from main the same spec reads 01 eb8ba4 0a, the chord ahead of the syllable, which is the reported corruption byte-for-byte. * refactor(terminal): add the composing-chord deferral without touching the Enter path Nesting the new branch inside the Enter condition re-indented the whole Enter block, which is the kind of diff that can silently change it. Keeping them as sibling conditions leaves the Enter path out of the diff entirely. * test(e2e): pin the renderer to macOS for the Cmd+Left chord Cmd+Left resolves to \x01 only under the macOS branch of the shortcut policy, so on a Linux shard the chord produced no byte and the spec passed by measuring nothing — it failed in CI for that reason, not for the behaviour under test. Pinning the platform is the established pattern for these specs, and expectImePlatformPolicy fails loudly if the override does not take. |
||
 Jinwoo HongandJinwoo-H
|
1d8eaa81c7 |
fix(orchestration): preserve mailbox delivery identity (#13717)
* test(orchestration): reproduce mailbox pointer mismatch * fix(orchestration): align pointers with actionable mailboxes * fix(orchestration): harden pointer reservation lifecycle * fix(orchestration): bound mailbox reconciliation * fix(orchestration): make pointer staging restart-safe * test(orchestration): expect scoped dispatch index * fix(orchestration): guard skewed inbox indexes * refactor(orchestration): extract mailbox notification lifecycle * fix(orchestration): settle mailbox pointer writes * fix(orchestration): bound mailbox recovery work * fix(orchestration): fence inactive mailbox snapshots * test(orchestration): cover mailbox notification boundary * test(orchestration): pin STA-4325 delivery identity * fix(orchestration): preserve mailbox delivery identity * fix(orchestration): harden mailbox settlement * test(orchestration): make mailbox gates self-contained * fix(orchestration): preserve paged mailbox ownership --------- Co-authored-by: Jinwoo-H <Jinwoo-H@users.noreply.github.com> |
||
|
Jinwoo Hong
|
500b72d8ef |
fix(vm): harden provisioned root ownership and cleanup (#14477)
* fix(vm): verify provisioned root ownership * test(vm): retry transient removal menu * test(vm): stabilize provisioned root teardown * fix(vm): clarify recipe-owned cleanup * fix(vm): pin provisioned root source commit * fix(vm): make runtime cleanup user-cancellable |
||
 NeilandOrca
|
41ba29d79a |
test(e2e): headless preedit-geometry coverage for Korean and CJK terminal input (#14500)
* test(e2e): headless preedit-geometry coverage for Korean and CJK terminal input Both IME defects that shipped and were reverted passed a suite of ~3000 IME assertions, because every one of them checked bytes reaching the pty and a preedit rendered into a hidden overlay satisfies all of them while the user composes blind. The one arm that asserted real geometry was headful-gated and macOS-only, so it never ran in CI. Drives composition through CDP Input.imeSetComposition rather than a native input source, which removes the accessibility grant, the system input source and the visible window that forced that gate, so this runs in the ordinary electron-headless project. The load-bearing assertion is the composition overlay's real bounding rect. Verified to have teeth: with max-width 0 and overflow hidden injected, the active class, the textContent, display block and checkVisibility all still pass, and only the rect assertion fails. * test(e2e): restore the CDP composition drivers the preedit specs need The trimmed copy on main kept only the key-dispatch helpers, so the composition drivers the geometry specs import were missing. Adds them back: setImeComposition, commitImeText, dispatchImeProcessKey, composeHangulSyllable and dispatchResumedCompositionUpdate. The shared helpers are unchanged. Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Orca <help@stably.ai> |
||
  Langning ZhangandJinwoo-H
|
92b6ffd17d |
Terminate renderer graph reload generations and contain disposed-frame notifications (#14070)
* fix(runtime): terminate renderer graph reload generations * fix(runtime): harden renderer reload teardown * fix(runtime): fence renderer graph publication ownership * test(runtime): register renderer graph reload gate * test(runtime): record live reload validation * fix(runtime): ignore cancelled renderer navigations * chore: preserve main formatting during branch sync * chore: satisfy changed-code quality gate * fix(runtime): restore cancelled renderer reloads * fix(runtime): preserve committed reload fencing * test(runtime): prove cancelled reload timeout * docs(reliability): record reload cancellation oracle --------- Co-authored-by: Jinwoo-H <jinwoo0825@gmail.com> |
||
 dngur6344
|
78ca45e2ae |
fix(renderer): remove duplicate git history tooltips (#14453)
* fix(renderer): remove duplicate git history tooltips * test(renderer): harden tooltip regression coverage |
||
|
Jinjing
|
d03eb3218e |
Revert "fix(renderer): stop git history hover from showing two tooltips (#14468)" (#14485)
This reverts commit
|
||
|
Jinjing
|
b3efd48c7f |
test(e2e): stabilize terminal failure coverage (#14466)
* test(e2e): stabilize terminal failure coverage * docs(e2e): classify failed-run findings * rm report |
||
|
Jinjing
|
2e91a74deb |
fix(renderer): stop git history hover from showing two tooltips (#14468)
Remove native title attributes from commit rows, subjects, and ref badges so only the managed tooltip appears. |
||
|
Neil
|
77f23b013f |
refactor(shared): drop the shared/types barrel and import from the real modules (#14447)
#14397 split `shared/types.ts` into 46 per-domain modules but kept the path as a re-export barrel so the import sites did not have to change. This removes the barrel: every consumer now imports from the module that actually declares the type, and `src/shared/types.ts` is deleted. Barrels hide where a type lives, make every consumer look like it depends on the whole domain, and let an unrelated edit invalidate a module that ~2,000 files transitively import. 2,323 import declarations across 2,321 files. Rewritten mechanically: each specifier was resolved to an absolute path via the TypeScript AST and recomputed, rather than string-substituted, so alias forms (`@/../../shared/ types`) and per-specifier `type` modifiers survive. Four cases the mechanical pass had to handle, each found by a gate rather than by reading the diff: - Modules inside `src/shared` import the barrel as `./types`, not `shared/types`. A pre-filter on the latter string skipped 176 of them and left imports dangling at a deleted file, which surfaced as confusing `Property 'x' is optional in type 'Repo' but required in Pick<Repo, ...>` errors rather than "module not found". - The barrel RENAMED one type on the way through (`WorkspaceSource as WorkspaceCreateTelemetrySource`), so the original name in the owning module has to be re-aliased at each consumer. - Three test files put `;(globalThis as ...)` on the line after the import. TypeScript parses that `;` as the import statement's terminator, so replacing through `statement.getEnd()` deletes it and breaks ASI. The rewrite now stops at the module specifier. - A file that already imported directly from a module got a SECOND import from it, because the barrel re-exported those same names — which trips `import/no-duplicates` under `--deny-warnings`. A post-pass merges declarations sharing a specifier and type-only-ness; the `import type` plus `import` pair from one module is left alone, since that form is allowed. Splitting one barrel import into several genuinely adds lines, which pushed `terminal-layout-pty-ownership.ts` to 301 counted lines: its 107-character import must wrap, and neither local type collapses onto one line (101 and 116 characters). Rather than contort a type declaration to fit a line budget, `collectLeafIds` and `pruneLeaves` move to `terminal-pane-layout-tree.ts` — they are pure structural operations on the layout tree and independent of PTY ownership. `visible-worktrees.ts` similarly loses its own mini-barrel re-export of `isDefaultBranchWorkspace`, with the four real consumers repointed at the declaring module. No `max-lines` bypass added. Verified: cold `tsc --noEmit` green on node, cli, and web (buildinfo deleted first — these projects are `composite: true` and reuse stale caches); the full `pnpm lint` green, not just bare oxlint — the narrower local check is what let the duplicate imports reach CI; max-lines ratchet OK at 344. |
||
|
Jinwoo Hong
|
cd6114ab7e | fix(browser): acknowledge paired tab before navigation (#14402) | ||
|
Neil
|
583ab1601b |
refactor(shared): group worktree, github, and linear modules into folders (#14437)
`src/shared` is a flat directory of ~1,150 entries. The worktree, github, and
linear domains accounted for 71 of them, so finding the module you wanted meant
scanning a wall of same-prefixed filenames.
Move each domain into its own folder and drop the now-redundant prefix:
src/shared/github-pr-types.ts -> src/shared/github/pull-request-types.ts
src/shared/worktree-id.ts -> src/shared/worktree/id.ts
src/shared/linear-links.ts -> src/shared/linear/links.ts
This follows the existing `network/` and `new-workspace/` convention in the
same directory, which also drop the prefix inside the folder.
Whole clusters move, including tests. Foldering only part of a domain would be
worse than flat: a reader would have to check both `github/` and the flat
directory, and `github-auth-types.ts` / `github-project-types.ts` are type
modules that belong with the rest. No files with these prefixes remain flat.
Import specifiers were rewritten by resolving each one to an absolute path and
recomputing it, not by string substitution, so the `@/../../shared/...` alias
forms are handled correctly. 501 specifiers across 298 files.
Two things `tsc` cannot catch, handled explicitly:
- `github-project-types.ts` carries its own `max-lines` bypass, so its baseline
entry is REPOINTED to the new path rather than pruned. Pruning would drop the
bypass and then flag the new path as a fresh violation. Ratchet stays at 345.
- `mobile/` is outside `pnpm typecheck` and cannot be typechecked here
(`mobile/node_modules` is empty). Instead every relative specifier in the repo
was resolved against the filesystem: 174 unresolved before this change and 174
after — identical, so nothing broke in mobile either.
The pinned `tests/e2e/.cross-version-checkouts` fixtures are deliberately NOT
rewritten; they are a snapshot of an older release and still reference the old
paths.
Verified: cold `tsc --noEmit` green on node, cli, and web (buildinfo deleted
first — these projects are `composite: true` and reuse stale caches).
|
||
|
Neil
|
eb22e497bb | Revert "fix(ssh): reapply the reattach-identity work and stop the fallback fence stranding moved panes" (#14395) | ||
|
Brennan Benson
|
d16092e503 |
fix(updater): recover renderer shutdown checkpoint (#14373)
* fix(updater): recover renderer shutdown checkpoint * test(updater): cover checkpoint recovery in Electron * fix(updater): keep staging failures blocking |
||
|
Neil
|
6a0c8fa541 |
fix(ssh): reapply the reattach-identity work and stop the fallback fence stranding moved panes (#14384)
* Reapply #13326 and #13928 (un-revert #14361) Restores the SSH reattach-identity and daemon-occupancy fixes. Reverting them reintroduced their P0s, filed as STA-4224, STA-4225, STA-4227, STA-4230, STA-4232, STA-4233 and STA-4234 against #14361. The tab loss that motivated the revert is fixed in the commits that follow, so this reapplication is not a straight redo. * fix(relay): stop the fallback attach fence refusing a pane that moved tabs The primary fence was moved to the shell's own incarnation precisely because paneKey/tabId froze the pane's LOCATION at spawn and refused panes that had merely moved. The fallback that older clients fall into kept the old rule, so the correction never reached it — the same 'the rule exists, but this path does not ask it' leak this work has hit repeatedly. A refusal here is not recoverable: an identity mismatch never grounds a respawn, so the pane keeps a live shell it can no longer reach and renders blank. Narrowed to paneKey, which is the identity; the tab is a location. Restoring the tabId comparison reddens the new test. |
||
|
Jinwoo Hong
|
77b37d85e2 |
feat(vm): create workspaces from provisioned SSH roots (#14359)
* feat(vm): use recipe-provisioned SSH roots * fix(vm): preserve ordinary create failure timing * test(vm): prepare provisioned root SSH fixture * ci(vm): enable SSH setup for provisioned root E2E |
||
|
Jinwoo Hong
|
a5a998ea77 | fix(vm): make hidden SSH cleanup retryable (#14351) | ||
 Dong dahaoandJinwoo-H
|
d243137e35 |
fix(orchestration): resolve explicit worker worktrees directly (#14275)
* fix(orchestration): resolve explicit worker worktrees directly * fix(orchestration): share worker workspace resolution * fix(runtime): reject cross-host path ambiguity * fix(orchestration): share federated workspace resolution * refactor(runtime): share worktree host identity * test(orchestration): align worker lifecycle fixtures --------- Co-authored-by: Jinwoo-H <jinwoo0825@gmail.com> |
||
|
Neil
|
11cd2b4310 |
revert(ssh): back out #13326 and #13928 — reconnect loses every tab (#14361)
* Revert "fix(daemon): stop killing live coding agents when the daemon can't report its sessions (#13928)" This reverts commit |
||
|
Jinwoo Hong
|
f0c1503326 | test(renderer): cover Agent Dashboard status replay fanout (#14036) | ||
|
Jinjing
|
ca9777093b |
test(e2e): stabilize release triage coverage (#14328)
* test(e2e): stabilize release triage coverage * test(e2e): avoid post-selection remount race * test(e2e): wait for New tab search focus * test(e2e): focus New tab search without pointer input * test(e2e): reopen voice settings after device change |
||
|
Jinjing
|
70abf5cacc |
test: add golden e2e tests for agent TUI launch and shell recovery (#14258)
* test: add golden e2e tests for agent TUI launch and shell recovery
Add test fixtures and E2E tests to verify agent TUI functionality:
- Stub agent implementation supports cross-platform execution (Unix/Windows)
- Test verifies multiline composer with Shift+Enter support in agent TUI
- Test verifies clean shell resumes after agent exit without state leakage
* test: add golden e2e tests for agent TUI launch and shell recovery
Add agent TUI launch and shell-recovery tests to the golden (release-blocking)
E2E suite, covering agent initialization and shell availability after agent
exit. Improve escape sequence handling in the stub agent to prevent stray key
reports from contaminating test output. Add terminal input readiness checks to
ensure commands execute reliably before verification.
* test: coerce golden stub stdin chunks for type-aware lint
Node types the stdin data event as string | Buffer even after
setEncoding('utf8'), so restrict-plus-operands failed CI.
* test: fix golden stub agent Windows batch files and add Ctrl+C support
- Store batch files with CRLF to avoid Windows 512-byte parser boundary bug
- Handle Ctrl+C (0x03) in raw mode as alternative to Ctrl+D (0x04)
- Update release notes documenting golden test skip behavior on older tags
* Remove Windows batch file gitattributes workaround
The -text whitespace=cr-at-eol rule preventing CRLF conversion for
.cmd files is no longer needed. Allow batch files to use normalized
line endings.
|
||
|
Jinjing
|
b8d6b21dfa |
test(e2e): add golden E2E tests for workspace session management (#14304)
* test(e2e): add golden E2E tests for workspace session management - Restore exact file and terminal state after quit/relaunch - Verify terminal file link activation and external edit detection - Test worktree creation and switching with isolated terminals - Isolate test repo paths between concurrent CI runs with UUIDs * Add platform-aware marker echo command utility - Create splitMarkerEchoCommand() to generate shell commands that safely echo test markers across Windows and Unix platforms - Split markers into prefix/suffix fragments so output assertions prove execution, not just shell echo-back - Consolidate SORTABLE_TAB export and improve tab bar locator logic - Refactor terminal link helpers to extract client point calculation |
||
|
Jinjing
|
e84fb46eaf |
test: add golden E2E tests for source control workflows (#14260)
* test: add golden E2E tests for source control workflows - Tests core source control interactions: file edit/save, commit staging, and diff viewing - Integrated into CI/CD pipelines for Linux, macOS, and Windows - Includes helper utilities for test setup and worktree management * test(e2e): verify golden commit author and fix test flakiness - Configure git author name/email at worktree level during setup - Verify commits are made with correct author details in assertions - Add explicit timeouts to file visibility waits and git status polling - Fix test ordering to seed edits after source control is open - Simplify git status refresh logic to rely on automatic updates * Add rollback to createGoldenWorktree on setup failure Cleanup callbacks only register after setup succeeds. When a config command fails, the half-built worktree and branch leak into later test runs, causing flakiness. Now we roll back immediately and re-throw the setup error. * test(e2e): match explorer rows after the git status badge appears The golden file-save spec used an exact /^README.md$/ filter. After save, the explorer row text becomes "README.md M", so reopen clicked nothing. * test: strengthen golden worktree setup verification - Track working directory in git call inspection to verify correct execution context - Verify user.name/email config applies to worktree-specific settings, not repo - Add exhaustive setup call sequence assertions to catch setup/rollback leaks |
||
|
Jinjing
|
e7b85266f5 |
Add golden E2E tests for fresh terminal and shell commands (#14302)
* test(e2e): add golden tests for fresh terminal and shell commands Adds regression tests for terminal initialization in fresh profiles and shell command execution to the golden test suite, integrated across Linux, macOS, and Windows CI. * test(e2e): bracket shell command output between markers The echoed command can wrap or be clipped by the buffer tail. Bracket output between begin and end markers to reliably identify real output, and strip ANSI escape sequences that interfere with parsing. |
||
|
Jinjing
|
1d2b8c7f42 |
test(e2e): stabilize triaged release failures (#14242)
* test(e2e): stabilize triaged release failures * rm file |
||
|
NeilandOrca
|
4c5f818187 |
refactor(skills): remove the unreachable Skills page and the file count it rendered (#14259)
Co-authored-by: Orca <help@stably.ai> |
||
|
NeilandOrca
|
2e8cf589de |
fix(daemon): stop killing live coding agents when the daemon can't report its sessions (#13928)
* fix(daemon): stop killing a wedged daemon that still owns live agent PTYs
A daemon too busy to answer listSessions was indistinguishable from a dead
one: getAliveDaemonSessionCount() returns null ("could not verify"), the
preserve gate required `!== null && > 0`, so the run fell through to
killStaleDaemon() and every running coding agent died with it. The sibling
replace branches all preserve on null; this one alone collapsed "can't tell"
into "empty", which src/main/daemon/AGENTS.md already forbids.
Give the decision an out-of-band second opinion. inspectDaemonPtyOwnership()
reads the OS process table — never the daemon socket, which is exactly what
failed — and reports whether the daemon's own process still has live PTY
descendants. Under preserveWhenOwningLivePtys, that evidence vetoes the
signal and the launcher adopts the daemon in degraded mode instead.
The veto is opt-in so it cannot make a daemon unkillable: only the
failed_health_check path enables it. Manage Sessions -> Restart still kills.
Only positive evidence preserves, so a wedged daemon with nothing to lose is
still replaced (#8689).
Also stop replacing silently: the verdict now prints on the post-kill truth,
which stays quiet on a cold start because nothing was killed.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): survive preserving a daemon too wedged to be adopted
Adversarial review found the veto's own success path could not complete
against the daemon it exists to protect. Preserving routed through
holdDaemonAdoptionLease(), which opens a hello — the exact operation a
wedged daemon cannot answer — so it threw, aborted initDaemonPtyProvider,
and left no spawner. restartDaemon() throws without one, so the user lost
the documented Manage Sessions -> Restart remedy on top of having no
daemon: strictly worse than the data loss being fixed.
A still-listening endpoint means wedged, not gone, so keep a lease-free
handle instead. The lease only cancels the adoption watchdog, which never
fires on a daemon that owns sessions. Degraded mode likewise tolerates a
lease and a session discovery it cannot complete.
Three more from the same review:
- The veto keyed on reason === 'failed_health_check', but a daemon that
answered listSessions with 0 lands in that same branch and must stay
replaceable. Key on liveSessionCount === null, which is what the option
actually documents.
- Zombies are not evidence of live work. A wedged daemon cannot reap, so
its exited agents linger as <defunct> and would read as "still running"
— a false positive correlated with the wedge itself. Enumerate through
the process table's stat column and exclude them; sample twice so a
resolver probe or health-check shell cannot masquerade as an agent.
- Restore the "did anything answer?" log guard alongside the confirmed
kill, so a daemon that self-retires before the kill is still announced.
Co-authored-by: Orca <help@stably.ai>
* test(daemon): kill the mutations that let the PTY veto ship as a no-op
Mutation testing found four survivors — changes that break the fix while
every test stays green:
- Swapping the POSIX reader to the 500ms-cached one passed. It is not just
a staleness hazard: inside the TTL both sampling attempts receive the same
array, collapsing the two-sample confirmation to one. Pin the fresh reader.
- Replacing killStaleDaemon's default inspector with one that never reports
live PTYs — the veto disabled in production — passed, because every veto
test injects the hook. Exercise the real seam.
- Adding the veto to cleanupDaemonForProtocol passed, which is verbatim the
failure its own doc warns about: a user-initiated restart of a daemon
owning live PTYs would refuse, then throw. Pin that call's arity.
- The ppid-cycle fixture put the cycle outside the daemon's subtree, so the
walk never entered it and deleting the visited guard passed.
Also bound the Windows enumeration, which had no budget of its own: two CIM
queries with a wmic fallback can stall the launch path for tens of seconds.
Blind is a safe answer there; hanging is not.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): require session-leader evidence and stop preserving daemons that can never be adopted
Round-three review found the veto too eager in three ways, each of which
traded the original data loss for a whole-session degrade or a permanently
daemon-less app.
Evidence was "any non-zombie descendant", justified by re-sampling to weed
out transients. The two samples are taken back to back — one ps fork apart —
so nothing transient is ever weeded out, and a hung helper (often the very
reason the daemon is wedged) reads as an agent. Use the structural signal
instead: a PTY child is a session leader, because forkpty calls setsid, and
no helper the daemon forks ever is. Re-sampling now only retries blindness.
A 'rejected' daemon answered and refused the handshake, so it can never be
adopted; preserving it repeated the same failed adoption on every launch,
forever. And the veto read the process table, not the socket, so it could
fire on a daemon whose endpoint was already gone — adoption then threw,
init aborted, and the app was left with no spawner and no working Restart.
Gate on both: only preserve what could still be reached.
Also: releasing the launcher's temporary lease after the permanent lease
failed reopened the adoption gap that ordering exists to close, and the
tolerance added to discoverDaemonSessions was dead code — nothing on that
path rejects.
Co-authored-by: Orca <help@stably.ai>
* refactor(daemon): decide occupancy before the kill, not inside it
Three review rounds each found a new failure state in the previous shape,
which was the design telling us something. The safety rule — never destroy
running work — was replicated across the launcher's branches instead of
being decided once, and the last round added it to one more branch behind a
boolean. Policy had been put inside a mechanism: killStaleDaemon grew an
input flag to disable its new veto, an output back-channel to report it, and
a caller-side re-derivation of the classification the flag had lost. One
structural error, one symptom per layer it crossed.
Name the question instead. resolveDaemonOccupancy answers occupied | empty |
unknown, asking the daemon first (authoritative both ways) and falling back
to the process table only to RAISE the answer to occupied. OS evidence can
prove work exists; it can never prove absence, so it never licenses a kill.
The launcher now decides before it destroys anything, so killStaleDaemon
goes back to being only "make this pid go away" — no options, nothing for a
future caller to forget to disable, and Manage Sessions -> Restart cannot be
vetoed because there is no veto left to hit.
Holding is a real outcome now. A daemon that owns live terminals but cannot
answer a handshake gets mode 'held': no adoption attempt, no lease, no fork
beside it. That deletes the lease-free-handle fallback, the try/catch around
preserve, and the tolerated-lease branch in init that existed only because
the correct outcome had no representation.
Two defects this removes outright:
- The endpoint check used a local boolean probe that returns false on
timeout, so under load — and unconditionally on Windows named pipes, where
a busy server answers ERROR_PIPE_BUSY — the guard disabled itself in
exactly the conditions it was written for. Use the canonical three-valued
probe, whose own docs say absence of proof is not proof of death.
- A daemon that answered and refused the handshake was killed with its
agents. It is now held like any other occupied daemon.
Also folds the replacement verdict onto pendingReplacement, retiring a pair
of mutable launcher locals whose only job was moving one warning past the
kill.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): make the occupancy residual total
The module's contract is that 'unknown' is where every unanswerable question
lands, but a throwing dependency escaped instead — routing a failed
observation into the launch path rather than onto the safe residual. Latent
today because both real implementations swallow their own failures, which is
exactly the kind of thing that stops being true during a refactor.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): stop counting the daemon's own probe PTYs as hosted work
Round-four review found the evidence proving the wrong thing. The filter
excluded the daemon's plain subprocesses on the grounds that only a PTY child
is a session leader — but the daemon opens PTYs for its own health probe and
conpty warmup, and forkpty makes those session leaders too. The comment's own
premise refuted its exclusion list. A daemon hosting zero user terminals could
be held on the strength of its stuck probe child, and since the held daemon
also had no sessions, dropping our authenticated pair let it retire and take
the very state we were protecting. Exclude them by exact command, on both
platforms.
Two more from the same review:
- pty-spawn-unhealthy is only reachable after a successful hello, so that
daemon is adoptable. It was routed to 'held' — which never adopts — purely
because the count had come from the process table. Check it first; hold now
requires an unreachable daemon.
- The grace loop rescanned the process table every pass, though it is waiting
for IPC and the table cannot change its answer in five seconds. Ask the
daemon during the wait and read the table once, after. raiseOccupancy-
WithProcessEvidence makes that split explicit, and can only ever raise.
Bound the wait by wall clock too: the retry count alone never bounded it, and
startup fails open at 60s by abandoning the daemon provider outright, which
would trade a wedged daemon for no daemon and a Restart that throws.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): hold a hello-rejected daemon instead of adopting one that refused us
Found while reviewing why a mutation looked equivalent. Gating the hold on
health === 'unreachable' left 'rejected' — a daemon that answered and refused
the handshake — falling through to preserveDaemon(), whose adoption opens the
very hello it just refused. That throws, and the throw costs the app its
daemon and its Restart remedy. Killing it instead is no better: it can still
be hosting running agents.
Neither of those daemons can complete a handshake, so neither may be adopted,
and both must be held. Gate on that rather than on one of its two causes.
Adds regression tests for the round-four fixes: the self-spawned probe
exclusion is exact-match on both platforms, an adoptable pty-spawn-unhealthy
daemon is never routed to a mode that cannot adopt, evidence can only raise a
verdict, and the grace budget stays under the startup fail-open cap.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): decide adoptability from what the daemon can do now, in one place
Round five found the same question — can this daemon complete a hello right
now? — answered in three places with three different conclusions, because
'held' had been added as a fifth branch rather than as the classification the
other branches route through. Two of those answers were wrong, and both ended
in no daemon at all, which is the outcome 'held' exists to prevent.
- A daemon whose adoption hello had just failed was handed back tagged
'degraded-new-pty-fallback'. Init skips the lease only for 'held', so it
reopened the same connection, threw, and aborted startup — leaving the
agents alive but unreachable and Manage Sessions -> Restart throwing.
- The pty-spawn-unhealthy arm ran first and claimed a successful hello proved
adoptability, but that reading is from before the grace window. A daemon
that answered at t=0 and went silent through thirty seconds of retries took
that arm and threw the same way. Ask whether it is answering now, first.
The budget was a comment with a Date.now() beside it: one occupancy probe
could cost 50s, because the client's default is a 5s hello per connection
step plus a 30s request timeout. Bound the probe explicitly, start the clock
before the first one, and size the window so the whole path — health check,
pid verification, loop overshoot and process-table read — fits under the
startup fail-open with room to spare.
Also folds the four sibling branches onto the same occupancy resolution.
getAliveDaemonSessionCount was byte-identical to countLiveSessionsOverIpc, so
one concept had two implementations and only one of them had been fixed.
The Windows probe exclusions could never match: the warmup spawns COMSPEC, an
absolute path, against an exact-equality test on 'cmd.exe /c exit'. Compare
the program by basename and keep the argv tail exact.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): stop the local fallback answering for sessions it does not own
Closing a held daemon's pane reported success while the agent kept running.
Unrouted ids resolve to the in-process fallback, whose shutdown returns
silently for an id it has never heard of and whose write and resize are
no-ops — and while a daemon is held nothing ever enumerates its sessions, so
every one of them is unrouted. The pane vanished, the orphan outlived the
app, and typing into a stuck terminal disappeared without a word.
Only attach was fenced against that route. Extend the same rule to the
operations that change or feed a session: route to the fallback only when it
genuinely owns the pty, and otherwise say the session cannot be reached.
The error type is load-bearing. pty:kill treats "Session not found" as proof
the pty is already gone and synthesizes an exit, so reusing that error would
have reproduced the lie one layer down. TerminalSessionOwnerUnverifiedError
means "still there, we cannot reach its host", which is reported as a failed
close and keeps ownership for a retry.
Co-authored-by: Orca <help@stably.ai>
* test(daemon): pin that a held session cannot be closed by a provider that never had it
Covers the held-daemon routing fence, including the coupling that is
invisible from the routing file: the thrown error must not match pty:kill's
already-gone predicate, or the close is swallowed into a synthesized exit and
the orphan is hidden again. A rename would otherwise reintroduce the bug
silently.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): bound the POSIX evidence read and re-verify the pid it describes
Round-six follow-ups, none destructive.
The process-table read had a deadline on Windows but not on POSIX, where the
shared reader's ps timeout does not cover queueing behind an in-flight scan —
so the one step that runs after the grace window could still outlast it.
The pid handed to that read was verified before the grace window, which is
long enough for the daemon to die and its pid to be recycled onto a shell
with children. Verify it where it is used instead, and only when there is
still something to raise: the common case now skips the identity probe
altogether, which also takes a few seconds off the worst-case launch.
Two renderer call sites killed PTYs without handling rejection. That was
harmless while an unreachable session was answered by a silent no-op; now
that it honestly rejects, pane teardown and repo removal would log an
unhandled rejection every time — exactly when the daemon is already sick.
Deliberately not taken from that review: giving the endpoint-occupied catch
the same held fallback as the failed-health path. That path arrives with
occupancy unknown or empty, so holding there would swallow a real launch
failure to protect nothing.
Splits the repro script, which had grown past the line limit, into the
sequence it proves and the two things it proves it with: process-table
inspection, and the static assertions on the launcher's hold decision.
Co-authored-by: Orca <help@stably.ai>
* test(daemon): hold the classification budget to the whole path, not one term
The previous assertion compared the grace window to the fail-open cap, which
passed while the real path ran to roughly twice the cap — a single probe cost
50s against a 5s assumption, and the terms on either side of the loop were
never counted at all.
Sum the declared budgets instead: health check, grace window, the one probe
that always runs past a ceiling tested at loop entry, and the evidence read
on both platforms. Raising any of them now has to face this, and the spare
time the kill ladder and fork still need afterwards is stated rather than
assumed.
Lives outside the launcher's own spec because that file mocks daemon-health,
which would shadow the constants being held to account.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): do not read a stranded login wrapper as a hosted terminal
Raised by a colleague's handoff on the same-day reports. macOS wraps every
terminal in /usr/bin/login for TCC attribution, and #13764 shows the wrapper
can outlive the shell it wrapped — leaving a session leader that hosts
nothing. One affected host had accumulated enough of them to reach swap
pressure.
That is exactly the evidence this change treats as proof of live work, so a
daemon whose sessions had all ended would have been held indefinitely on the
strength of the corpses, on precisely the hosts where the problem is worst.
Same class as the daemon's own probe PTYs: a session leader is necessary
evidence, not sufficient. A wrapper still doing its job has the shell it
exec'd beneath it.
Co-authored-by: Orca <help@stably.ai>
* test(daemon): pin the daemon's PTY spawn sites so the exclusion list cannot silently rot
The ownership evidence discounts the PTYs the daemon opens for itself, and that
list is only safe while it is complete — a self-spawned PTY nobody excluded
reads as user work and holds a daemon that owns nothing. The list grew one
reviewer at a time, which is the wrong mechanism for a correctness invariant.
Pin the input rather than the list. The daemon has exactly three PTY spawn
sites: the user's terminal, the spawn health probe, and the Windows conpty
warmup. A fourth now fails this test until someone decides which side it
belongs on.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): stop the evidence going blind on the host it exists to protect
Readiness review, section 04. Every agent pane already drives the shared
process-table reader on its own cadence, so the uncached read queues behind
them — and the host with the most agents to lose is the one likeliest to blow
the deadline on queueing alone. Both attempts return unknown and the daemon is
killed anyway, which is the original bug wearing the fix as a costume.
Fall back to the TTL-cached table, which on that host is always warm for
exactly the reason the uncached read is always queued. A table a few hundred
milliseconds old still answers whether this daemon has children, and the
failure directions are not symmetric: over-holding costs one degraded launch
that self-heals, under-counting ends running agents.
The same review found the launch budget still overran the 60s startup
fail-open — by ~7s on Windows — and that the test guarding it under-counted
the path it was written to bound, for the second time. It omitted the identity
probe before the evidence read and the endpoint check that ends the grace
loop. Both are now summed, the headroom requirement covers the kill ladder and
fork that follow a replace verdict, and the grace window and Windows probe
deadline are sized to fit.
Not taken from that review: reusing the verified pid inside killStaleDaemon to
drop the duplicate probe. That second verification is what fences the signal to
this incarnation, and a seconds-old result is exactly the pid-reuse hazard it
exists to prevent.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): confirm emptiness before it authorizes a kill
Readiness review, sections 02/03/05/06 — no P0 or P1 in any of them. These are
the P2s worth taking.
The important one: on macOS a terminal contributes exactly one session leader,
the login wrapper, because the shell it forks is in the same session and shows
S+ rather than Ss. I had assumed the shell counted too. It does not — so a
wrapper that looks childless in a single snapshot makes its whole terminal
invisible, and that snapshot cannot tell a wrapper whose shell has gone from
one whose shell has not yet appeared. Emptiness is the answer that authorizes a
kill, so it now costs a second read; 'owns-live-ptys' still needs none. The
fixtures said Ss where a real shell says S+, which is why the tests never
noticed.
Also from that review: a fabricated row was cast to ProcessTableRow to reuse a
command-only predicate, which is sound only while that predicate reads nothing
else — narrowed to Pick<'command'> so the compiler keeps it honest. The
pty:signal listener relied on its provider staying async to convert a routing
refusal into a rejection; it is an ipcMain.on listener with nothing above it, so
it now catches synchronously too. And the repro's teardown signalled remembered
pids a minute after phase 1 waited for them to die — re-verify by tag first,
since signalling a recycled pid is the mistake the script exists to study.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): stop guessing at Windows occupancy instead of guessing better
The readiness review found the Windows branch reading a wedged daemon's
orphaned conpty hosts as live terminals. ClosePseudoConsole only runs on the
daemon's own JS thread, so a daemon too wedged to answer is also too wedged to
reap them, and they accumulate exactly when this code runs. A wedged, empty
Windows daemon would then be held forever — #8689 re-opened, and a regression
from main rather than a missing protection.
The tempting fix is another exclusion. That would be the sixth revision to what
counts as a live PTY, each one added because a reviewer found something that
looks like a session and is not, and each one trading safety for availability
in a fix whose entire purpose is the opposite trade. The list is the problem.
POSIX has a real signal: forkpty makes a hosted terminal a session leader, which
nothing the daemon forks for itself ever is. Windows has no equivalent, so its
branch could only ever count descendants and subtract guesses. Delete it and
answer 'unknown' — Windows keeps exactly the behaviour it has on main, and the
protection is claimed only where it can be justified.
Also stops a blind confirming read from upgrading an unconfirmed emptiness into
a verdict. Emptiness is what authorizes a kill; a read that saw nothing
corroborates nothing.
Co-authored-by: Orca <help@stably.ai>
* refactor(daemon): clear the debris the Windows removal left behind
Behaviour-preserving. Windows now abstains once at the entry point rather than
twice inside a retry loop that had nothing to retry, which also retires the
platform check further down that could no longer be false. The self-spawn
matcher kept backslash splitting and .exe stripping for a branch that no longer
exists, and the launcher's grace loop repeated its own IPC call and stacked two
explanations above the wrong statement.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): stop the budget cut spending Windows' only protection
Round seven found the one thing this PR must never do: kill a session that main
would have kept.
Main's grace loop probed with a non-shared 5s connect budget, so a wedged
daemon got roughly a minute to come back. Bounding the probes and adding a
wall clock cut that to about twelve seconds — a good trade on POSIX, where a
daemon that outlasts the window is still protected by process-table evidence,
and a bad one on Windows, which has no such evidence and now has nothing else.
A Windows daemon wedged for half a minute while hosting agents was adopted by
main and is killed by this branch. The fail-open cannot rescue it either:
ensureRunning() is not abortable, so the launcher runs to completion.
Size the window per platform instead, against what each actually spends:
Windows pays no evidence read and no identity probe to feed one, so it can
afford far more grace, and grace is worth more where it is the only thing
there. Both numbers come from the budget test rather than taste.
Three more from the same review:
- The evidence read applied its deadline twice, once to the fresh table and
again to the cached fallback, so an attempt could cost double what the launch
budget was told. Share one deadline across both.
- Two tests described protection the code no longer delivers: one asserted ~60s
of grace the wall clock had already retired, the other passed only because its
mocked probes are free and would fail against real ones. Say what the code
actually promises, and freeze the clock where the point is retry depth.
- The self-spawned PTY inventory promised more than it inspects. It sees direct
node-pty calls in one directory; the macOS login-session probe reaches a PTY
through expect(1) and is caught by the stranded-wrapper filter instead. Scope
the claim, since that indirection is the shape the next escape will take.
Co-authored-by: Orca <help@stably.ai>
* refactor(daemon): spend the launch budget against a clock instead of a sum
Round eight found the fourth term missing from the hand-written budget — the
launcher's own adoption connect, which runs before the health check on the
non-shared five-second path. The three before it were an identity probe, an
endpoint probe, and an evidence deadline applied twice. Every one of them
passed the test meant to catch exactly that, because the test could only check
the terms someone had remembered to add.
So stop summing. The classification now runs against a deadline and stops when
it expires, and the test asserts only that the deadline leaves room for the
kill ladder and the fork that follow it. A budget that has to be remembered is
a budget that will be wrong; this one cannot be, because nothing has to be
counted.
That also retires the platform-split grace window, which existed to hand
Windows more of a sum nobody could total correctly.
The same review found the regression it was compensating for was never the
window. main gave each probe up to fifty seconds — five per connection step,
thirty for the request — where this branch gave eight for both together. A
daemon whose handshake needs more than four seconds therefore answered none of
the probes, however many it got, and on Windows nothing else can speak for it.
Splitting the two budgets fixes the case the window never could: connecting
stays tight, because a daemon that cannot handshake is wedged and worth
re-asking cheaply, while a daemon that did handshake is demonstrably alive and
its count is worth waiting for.
Also stops a Date.now spy leaking out of a failed test and freezing the clock
for the rest of the file.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): make the classification clock actually bound the work it names
The clock introduced in the previous commit gated the probes but not the two
steps after them. The identity re-check and the process-table read ran on their
own deadlines, outside the ceiling, so the launcher could still spend its whole
budget on probes and then take another ten seconds — the same overrun the sum
used to produce, arrived at from the other end.
Hold that time back from every probe instead. A probe is only started when the
clock can still fund a handshake after the reserve, and its budget is what
remains minus the reserve, so no probe can eat it however long the daemon takes
to answer. Worst case is now the ceiling by construction rather than by
addition.
The reserve has a test asserting it is large enough for what it covers, which
failed on its first run and caught that ten seconds was not.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): ask the wedged daemon a question it can actually answer
Round nine found the re-verification was stricter than the check that triaged
the daemon onto this path. The launcher gets here because a three-second health
check — one socket, one hello — timed out. It then re-asked with two sockets
and two hellos inside four shared seconds, and repeated that identical question
up to twelve times. A daemon that consistently needs five seconds fails every
one of them, so the retries could only ever agree with the check that sent it
here. main re-asked with five seconds per connection step and thirty for the
answer, and kept the sessions this branch destroyed.
Retries and patience solve different problems. Keep the cheap probes, which
catch a daemon that recovers on its own, then spend what is left of the clock
on one tolerant ask — the only question that can disagree with the triage. It
is skipped when the endpoint is provably gone, since a cold start arrives here
too and has nothing to wait for.
Two more from the same review. The clock claimed to cover the launcher's own
adoption connect and started after it, so the fourth term that went missing
from the sum was still uncounted; it now starts above that connect and bounds
it. And Windows was holding back twelve seconds for an identity check and a
process-table read it never performs — the reserve is zero where the steps it
reserves for do not run.
Retuned the ceiling to leave the kill ladder and fork real margin rather than
half a second, with the packaged-Windows host copy named as what the margin is
for.
Co-authored-by: Orca <help@stably.ai>
* fix(daemon): ask the tolerant question while the clock can still fund the answer
The launcher asked the cheap question first and the patient one last. That is
backwards. This path is only reached because a 3s health check timed out, so
every 4s probe re-asks on a stricter budget than the one that triaged the daemon
here — it can only ever agree. The one ask that could disagree ran last, by which
point the clock could fund its handshake but not its answer, and a daemon that
answered in 12s was read as dead and replaced along with its agents.
Three changes, one idea: never make an ask you cannot afford to hear out.
- The patient ask goes first, with every millisecond the answer does not need.
- Cheap retries follow it, and stop once the clock cannot fund both halves.
Funded to knock but not to listen is not an ask.
- The adoption connect is capped. It is not a classification step — it acquires
a lease preserveDaemon() re-establishes anyway — but on a daemon that accepts
the socket and never completes hello it would spend the entire classification
budget, leaving nothing for the probes that protect that daemon's sessions.
The test double now forwards the connect budget instead of dropping it, so what
the launcher was willing to wait for is observable at all.
* fix(daemon): stop three follow-ups rotting where review already found them
A truncated paste now says so. A routing throw partway through a paste was not a
PtyWriteUnavailableError, so no pty:writeUnavailable reached the renderer and the
pane never re-attached — the remaining chunks simply vanished with nothing to
attribute the gap to. It stays deliberately distinct from SessionNotFoundError,
which isPtyAlreadyGoneError matches and synthesizes into an exit the session never
had; a test now pins both directions so neither drifts.
The launch-budget spec no longer fails on Windows. The evidence reserve is zero
there by design — neither guarded step runs without a session-leader signal — but
the assertion demanded eleven seconds of it unconditionally, so `pnpm test` broke
on any Windows dev machine. PR CI never saw it: only the WSL boundary spec runs on
windows-2022. The identity ceiling it reserves against is imported now instead of
being a 3_000 someone would have to remember to change.
And the grace-retry comment described the design that preceded the clock: ~5s
probes, ~60s of grace, a number worth raising. The clock binds first and usually
permits far fewer, so raising it alone buys nothing.
* fix(daemon): stop killing a daemon we merely failed to observe
Ten review rounds each found a different band where this branch replaced a daemon
that origin/main would have kept, and every fix bought a new one. The reason is
arithmetic, not carelessness: matching the old tolerance for a single probe costs
about 25s, the classification clock has 15s to give, and funding the difference
puts startup past the 60s fail-open once the kill ladder and the fork are paid.
No assignment of those numbers is safe.
So the residual stops being lethal. daemon-occupancy.ts always said it — "'unknown'
is the residual, and it is not permission" — while daemon-init.ts fell through from
unknown to killStaleDaemon. That fall-through is what made every millisecond of
budget a correctness parameter. Now an unclassifiable daemon is held in degraded
mode: existing terminals keep working, fresh ones run locally, and being wrong
costs a degraded session instead of somebody's agent.
Two exclusions, both about never holding something unrecoverable. A proven-dead
endpoint is a cold start or a corpse, and holding one would hand every first launch
a provider pointed at no daemon. 'rejected' answered and refused, so it can never be
adopted and its sessions can never be reattached.
The cost is real and deliberate: a wedged-but-empty daemon is no longer replaced at
launch, so #8689 degrades to "restart it from Manage Sessions". Which only works if
the user knows — and degraded mode was computed, plumbed through preload, and read
by nothing. It now renders where the Restart button already lives, and says what it
actually costs: new terminals close when you quit, and restarting ends whatever the
host is still holding.
Rejected on the way here: a background reclassifier (a permanently wedged daemon
never answers, and recovery is already handled by degraded-daemon-fresh-spawn-
routing.ts), letting accumulated process-table reads license a kill (its errors are
systematic, so more samples agree rather than converge), and sidelining the daemon
onto a renamed socket (verified working at the syscall level, then abandoned: the
daemon's own endpoint-ownership watch reads the moved entry as lost and retires
itself, precisely when it recovers).
* test(daemon): pin the hold that no longer depends on a clock
The repro proved the launcher holds a daemon it can see is occupied. The protection
that now matters most is the one for a daemon it cannot see at all, and nothing
asserted it. Adds the unknown-hold to the static assertions: that it exists, that it
excludes 'rejected' and a proven-dead endpoint, and that every killStaleDaemon call
site in the file is downstream of it.
Verified by mutation — dropping either exclusion fails the assertion.
* fix(daemon): repair what round eleven found, including a fix that did nothing
The patient ask was not patient. With twelve seconds reserved for the evidence
read, `max(CONNECT, probeBudget - REQUEST)` resolved to exactly CONNECT — the
tolerant ask got the cheap ask's four seconds, and the grace loop's gate needed
19s of a budget that only ever held 11s, so it never ran at all. Both shipped
green because mocked probes consume no wall clock, so the budget never binds in
a test. Two arithmetic tests now compute against realistic elapsed time, which is
where the defect actually lived.
The reservation was backwards anyway. Evidence can only raise 'unknown' to an
uncounted 'occupied', and both now hold the daemon, so the read changes a log
line and nothing else — while starving the one probe whose counted answer still
reaches preserveDaemon() and full daemon mode. It is opportunistic now: if the
probes spent the clock, it is skipped and the verdict stays 'unknown', which
holds exactly as an evidence-raised 'occupied' would have.
Recovery was a one-way flip on a two-way condition. Once a health check promoted
fresh spawns back to the daemon, nothing ever demoted them, so a daemon that
wedged again cost a hello timeout plus a full launcher re-classification for
every new terminal, for the rest of the session. A failed spawn now routes back
and re-arms the cooldown. The class had no tests at all; it has six.
The notice was wrong twice. It claimed terminals already open keep working — in
held mode discovery runs over the same IPC the daemon is failing, so its sessions
are never routed and attach refuses rather than answering on its behalf. They are
running, but unreachable. And it named half the cost of Restart: runRestartDaemon
shuts down the local fallback sessions too, so the terminals it had just called
safe die as well. Also fixed: the amber-on-amber body text failed AA at 3.94:1,
the keys were in a namespace no sibling uses, and the scale did not match the
notice it renders beside.
The banner armed a destructive button with state that never refreshed. It now
refetches on focus, like the sibling notice that solved this first.
Also: the launch mode type in the test harness omitted 'held', so no test could
describe the launch the hold produces; held mode's routing through the degraded
provider was unpinned, and deleting it left every test green; and pty:signal kept
a try/catch for a synchronous throw that async routing cannot produce.
* fix(daemon): stop offering a remedy that cannot work, and say why two modules stay
The degraded warning told the user to restart the daemon. When something other
than an Orca daemon holds the endpoint, restarting clears nothing: killStaleDaemon
only kills a process whose identity matches the pid record, and a foreign holder
matches none, so the next launch is identical. The message now says the daemon is
unreachable rather than asserting what it owns, and names the second remedy. The
notice says "usually clears this" for the same reason.
That case is also now written down as a known residual: an endpoint that accepts
connections but never speaks the protocol reads as an incumbent on every launch,
so it stays degraded with no auto-recovery, where before it was replaced.
The rest is comments, because three separate deletion proposals landed on this
code in one review cycle and each was a regression. The evidence read is not
redundant with the unknown hold: the occupied branch has no proven-dead check and
the unknown hold does, so it is the only thing between a kill and a daemon whose
socket entry vanished while it still hosts agents. Its children scan is not
redundant with pid verification either — a verified-live pid alone would also hold
a childless daemon, which is the one #8689 case still safe to replace. And grace
retries are worth more since 'unknown' stopped killing, not less: a counted
'occupied' reaches full adoption where the alternative is a degraded hold.
Each now says which case dies if it is removed. Reviewers reaching for the delete
key three times in a row is the code failing to explain itself, not excess.
* docs(daemon): record what a budget raise would owe before it is safe
The classification budget serves two verdicts with opposite time-costs. Reaching
"don't kill" slowly is free — the daemon survives however long it took. Reaching
'empty' slowly is not, because the kill ladder and the fork still have to fit
before the fail-open. At 34s that case cannot arise; at 44s it can, and an overrun
there is the worst branch on offer: daemon killed, replacement forked then
discarded, no provider installed, Restart broken.
So the raise is not a number change, it is a number change plus a guard: hold
rather than replace when the headroom left cannot fund the ladder and the fork.
Safe precisely because that path has proven the daemon empty, so holding costs no
agents. Written down next to the warning against tuning the budget, because the
next person to want a bigger number will read that warning and need this one.
Also recorded: the launcher closure has no access to the startup abort signal, so
the cheap version of that guard is not available without threading it through the
spawner.
* fix(daemon): delete a retry loop that could never run, at any budget
Round twelve proved the loop unreachable by algebra rather than by tracing:
remaining = B - E - max(CONNECT, (B - E) - REQUEST) = REQUEST,
whenever B - E > CONNECT + REQUEST
The patient connect takes every millisecond the answer does not need, so what
survives it is always exactly OCCUPANCY_REQUEST_BUDGET_MS — and the gate wanted
CONNECT + REQUEST. That holds for every ceiling, which also settles the raise I
had been holding open: at 44s the remainder is still exactly the request budget,
so ten more seconds of worst-case startup would have funded zero retries. Funding
one honestly needs ~71s against a 60s fail-open.
So WEDGED_DAEMON_GRACE_RETRIES = 11 documented patience the launcher did not have,
and no number could give it. Deleted, with the derivation left where the loop was
so the next person does not re-derive it from scratch.
Little is lost. A 4s retry cannot reach a daemon needing longer than 4s to answer,
which is the entire wedge population, while the one patient ask waits ~12s. The
only case retries caught and this does not is a daemon recovering within seconds
of being asked — and DegradedDaemonFreshSpawnRouter.recover() already returns it to
full daemon service on the next spawn, off the startup clock.
The budget tests could not have caught any of this: they recompute the expression
from imported constants and never execute the launcher, so collapsing the patient
connect back to the cheap constant — the round-eleven defect exactly — left all
1475 green. There is now a test that watches the launcher spend it, verified by
mutation, and the arithmetic ones say plainly that they are not the guard.
Also corrected: the evidence-read comment claimed the read only affects a log line.
It decides the verdict wherever the unknown hold declines to — it has neither the
proven-dead check nor the rejected check — so it is what holds a daemon whose socket
vanished, and what holds a hello-rejected daemon still hosting agents.
* docs(daemon): cost the deferred guard honestly, and say why it is unreachable
Two corrections to the note, both of which change what it tells the next person.
The reason the guard's case cannot arise at 34s is structural, not a lucky
margin: an `empty` verdict means the daemon answered, so it resolved fast by
construction, and proven-dead means nothing is listening, so the probe and the
ladder both short-circuit. The path that actually spends the budget is the wedge
that never answers — and that one now ends in a hold, paying neither the ladder
nor the fork. Long path and expensive tail are disjoint. Raising the budget is
precisely what re-couples them, by extending how late an `empty` may arrive.
And the guard was costed as a signature change through DaemonSpawner, which is
wrong. createOutOfProcessLauncher is a factory called where `signal` is already in
scope; a third parameter closed over there leaves the launcher's call signature
untouched. Overstating the price invites skipping the guard rather than paying it.
Also recorded: two terms this budget does not bound at all — the healthy branch,
which never consults the clock and still ends in a cleanup and a fork, and the
unbounded daemon-host copy on packaged Windows.
* docs(daemon): correct an overstated claim about the deleted retry loop
The deletion was justified as "the loop could never run, at any budget." That is
true of three wedge shapes and false of a fourth: a connect that fails fast leaves
the budget nearly whole, and while refused and missing endpoints are caught by the
proven-dead guard, the EPERM/EMFILE class reads 'unknown' and would have passed
the gate.
The deletion still stands — retrying an fd-exhausted or permission-denied connect
fails identically the second time, and recover() restores full daemon service on
the next spawn once the condition clears — but a comment that overstates its own
reach is how the next person concludes the reasoning was never checked.
* test(daemon): restore two guards a range deletion swallowed
Deleting the obsolete grace-loop test took out the two tests either side of it —
the ones pinning that the hold declines a proven-dead endpoint and a rejected
daemon. Both exclusions went unpinned in the same commit that removed the loop,
and the suite stayed green, because nothing else covers either path.
Found by mutation rather than by reading: removing `health !== 'rejected'` from
the hold left all 1472 passing. The pre-existing rejected test does not cover it —
its second client answers listSessions, so occupancy resolves to 'empty' and the
replace path is reached without the exclusion ever being consulted. The restored
test keeps the daemon unreachable so the verdict stays 'unknown', which is the
only state where the exclusion decides anything.
Also pins the evidence gate, the other survivor: the threshold must cover an
identity ps plus two ownership probes, or a read started at the last moment the
gate allows finishes past the ceiling the kill ladder and fork are sized against.
Mutation results now: patient connect collapsed -> caught; evidence gate -> caught;
hold removed -> caught; proven-dead exclusion -> caught; rejected exclusion ->
caught; fresh-spawn revert -> caught.
* fix(settings): make the degraded copy the copy users actually see
Two user-facing fixes were no-ops. translate() resolves from en.json, and the
catalog only ever gained the string it was first synced with — the sync script adds
missing keys and never updates changed defaults, which the extraction gate reports
as "inline defaults differ" and then passes anyway. So editing the inline default
changed the source and nothing else. Caught by rendering the component and reading
what came out, not by reading the diff.
What was stale in the catalog, and is now corrected there:
The notice promised the panes reconnect on their own once the host recovers. They
do not. TerminalErrorToast already tells the user "Reopen this pane to retry",
because nothing re-attaches a pane whose owner could not be verified — the session
is left untouched, which is the point, but recovery is a user action. Third claim
of mine in this PR that was stronger than the code.
And "Restarting the host clears this" still overstated the foreign-endpoint case,
where killStaleDaemon matches no pid record and clears nothing.
Also removes the components.settings.DaemonDegradedNotice.* namespace, left behind
when the keys were renamed to the auto.* convention every sibling uses. It was dead
weight carrying the oldest copy of all three strings.
* docs(daemon): 'held' no longer means what its type said it meant
The mode was introduced for a daemon that demonstrably owns live terminals and
cannot answer a handshake. It is now also what an unclassifiable daemon gets, where
the whole point is that we could not establish what it owns. A type whose comment
asserts the one fact the branch could not determine is the same overclaim this PR
has been correcting elsewhere.
Also un-exports ENDPOINT_PROBE_TIMEOUT_MS: it was widened for a test that no longer
references it, and nothing outside the module reads it.
* fix(daemon): stop a lost spawn reply from shadowing a live agent
`!mapped` was standing in for "this is a fresh spawn," and it is not the same
question. The mapping is only recorded after a reply arrives, so a spawn that names
a session and then loses its reply — the daemon created it, the answer timed out —
is indistinguishable from a genuinely new one. Demoting there sent the retry to the
fallback, which answers with a fresh local shell under the same id while the agent
keeps running on the daemon. The pane binds to the shell; the agent is orphaned.
That is the symptom this PR exists to remove, arriving through a door the PR opened
itself. Reachability today looks nil — the only sessionId-bearing spawn in ipc/pty.ts
carries attachOnly, which routes elsewhere — but the guard was unsound rather than
merely unused, and "no caller does that yet" is not a property anyone maintains.
Now an identified session pins to the provider that may already own it, and only an
anonymous spawn moves the shared route. Anonymous spawns are what demotion was for:
nothing can shadow them, and they are the ones paying a hello timeout plus a full
re-classification per terminal.
Found by GPT-5.6-Sol reviewing this file in isolation. Two tests added; reverting to
the old guard fails one.
* docs(daemon): name the three paths that can still kill a daemon
Adversarial review found all three; none is a regression against the pre-hold
behaviour, and none should be closed by weakening the evidence rules.
'unknown' plus a proven-dead endpoint still kills when process evidence cannot
answer. The probe proves the directory entry is gone, not the process — a socket
entry can vanish while the daemon still hosts agents. Evidence covers that on POSIX
because it runs for any 'unknown' rather than only a live endpoint, so the gap is
the blind cases: clock spent, pid unverifiable, ps unreadable. It is not reachable
on Windows at all, where a named pipe vanishes with its process, so a dead endpoint
there implies no agents to lose.
'unknown' plus 'rejected' still kills, and the reviewer is right that inability to
adopt is not inability to preserve — those agents keep running, unreachable. Killing
stays the choice because a daemon that can never be adopted and is never replaced
leaves the app permanently degraded with no route back, but that is a judgement, not
a proof, and it is now written as one.
And the verdict is not atomic with the kill: an 'empty' answer can go stale if
another instance creates a session first. Pre-existing, and narrowed rather than
widened here — the window now opens only after the daemon has reported zero sessions
itself.
* docs(daemon): record the TOCTOU fix that was built, tested and reverted
shutdownIfIdle is the right instrument and the daemon already implements it
atomically: sole authenticated client, nothing in flight, zero sessions, listener
closed before the acknowledgement. Asking it immediately before the kill closes the
window that a re-read of listSessions can only move.
It is not landing here. Gating every empty-verdict replacement on a new round trip
means every failure of that round trip has to mean hold, which trades a rare race
for a common failure mode and makes #8689 worse whenever the call is merely slow.
It also flipped two endpoint-identity tests from rejecting to resolving, and an
unexplained behaviour change is not something to merge at commit forty-two of a
change whose whole subject is unintended consequences.
Written down with the mechanism intact so the next person starts from a working
design rather than rediscovering it.
* fix(repro): restore the 91 lines a bad deletion took out of the repro
Removing readSourceConstant matched a docblock far earlier in the file and deleted
everything between, taking the imports and three functions with it. The script
still parsed, still linted, and still passed `node --check` — it failed only when
run, with `existsSync is not defined`, which is why it went unnoticed for two
commits. The only end-to-end proof in this PR had been dead that whole time.
Restored from before the deletion and the intended edit re-applied by itself. Now
runs green: phase 1 kills a wedged daemon with real agents attached and confirms
they die; phase 2 puts the identical wedge through the decision and shows the
daemon unsignalled, both agents alive, and everything back with sessions intact on
SIGCONT; phase 3 asserts the launcher holds — now including the unknown-hold, which
is the branch this PR turns on.
Lesson worth keeping: a syntax check is not a test. Running it is.
* fix(daemon): stop the emptiness confirmation reading the same snapshot twice
The second sample exists because one snapshot cannot tell a login(1) wrapper whose
shell has not appeared yet from a wrapper whose shell has gone. But when the fresh
read misses its deadline — the busy host this evidence exists to protect — both
attempts fell through to the same TTL-cached table. Two agreeing samples, one
observation, and the window being excluded is shorter than the cache.
The confirming read is now denied the cached fallback. If it cannot get a fresh
table it answers 'unknown', which holds. The first sample keeps the fallback,
because there the cache protects the answer worth protecting: a stale table still
shows that a daemon has children, and going blind there is what got them killed.
Also records three limits of this evidence that review surfaced and that no code
change should paper over — a reparented orphan outside the descendant tree, the
Windows abstention, and an argv match that cannot establish executable identity.
Each only fails to raise a verdict, so each costs a hold not taken rather than a
kill licensed.
* fix(daemon): stop discarding Linux terminals to solve a macOS problem
The stranded-login(1) exclusion ran on every POSIX host. Orca only wraps terminals
in login(1) for TCC attribution on macOS, so off darwin the pattern can only match
a user's own login — and one still prompting for credentials has no child yet,
which is precisely the shape the exclusion throws away. A Linux daemon hosting that
terminal read as childless, and a childless daemon is one nothing protects from the
endpoint-dead path. Now scoped to darwin, where the problem it solves lives.
And a count that is not a count is no longer read as emptiness. `counted > 0` maps
NaN, -1 and 1.5 to 'empty', which is the single verdict that licenses a kill; the
listSessions dep is injectable, so reaching it never required asking a daemon
anything. Non-integers and negatives now resolve to 'unknown'.
Both mutation-verified. Noting for whoever runs the next mutation pass: the first
attempt at the login mutation silently failed to apply because the pattern had been
reflowed by the formatter, and a mutation that does not apply looks exactly like a
test suite that caught it. Assert the pattern matched.
* docs(daemon): bound what the degraded owner check actually covers
Review confirmed the five destructive operations are guarded and that the error
taxonomy holds — TerminalSessionOwnerUnverifiedError cannot be reclassified as a
gone session anywhere in production, so no fake exit is ever synthesized from it.
It also found six methods that still route raw, and they are worth naming rather
than leaving for the next reader to rediscover: flow control, background state,
buffer clear, startup authority and the per-session queries. For an unresolved
daemon id those reach the fallback silently. None can destroy a session, which is
why they are not being changed at this point in this branch, but a buffer clear
that reports success while the daemon's history survives is a real lie.
Recorded with the reason not to fix them casually: acknowledgeDataEvent is invoked
directly from an ipcMain.on listener and setPtyBackgrounded from a synchronous
callback, so a throwing owner check added there without changing the call sites
turns a silent misroute into an escaping exception.
* fix(daemon): restore the demotion my shadow fix made unreachable
The review is right. Gating demotion on the absence of a sessionId assumed fresh
spawns are anonymous, and they are not: every production fresh spawn mints an id
before reaching the provider (ipc/pty.ts assigns spawnOptions.sessionId on all
three paths). So the branch only ever ran in tests, and a daemon that recovered and
wedged again kept every later terminal pointed at itself — each paying a hello
timeout plus a full launcher re-classification, each failing anyway. That is the
cost the demotion existed to remove, reintroduced while fixing something else.
The mistake was treating one condition as two questions. Pinning protects THIS id,
which the daemon may already have created before losing the reply, so a retry must
never be answered locally under the same name. Demoting protects the NEXT terminal,
which is a different session and cannot be shadowed by this one. They are
independent, and both now happen.
`attachOnly` is the honest discriminator for the second: an attach that names a
session never reaches this router, so anything arriving here without it is a fresh
terminal whatever id it carries.
Both halves mutation-verified separately — restoring the sessionId guard fails
three tests, dropping the pin fails two — including a test shaped like what
ipc/pty.ts actually sends, which is what the old test suite never had.
* fix(repro): stop the script doing the exact thing it exists to warn about
Both findings are right, and the first is pointed: this script demonstrates that
signalling a pid you have not re-verified can kill someone else's work, and its own
teardown did that twice.
The staged daemon is killed on purpose in phase 1. Once Node reaps that child its
pid is free for reuse, and teardown signalled the remembered number anyway; it now
signals only while the child object still reports no exit.
The markers were half-verified. Each proves its own identity by tag before being
signalled, but its session leader was killed on a number remembered from staging a
minute earlier — and the leader is the one pid here that can be recycled while its
child lives on under a new parent. The leader is now re-read from the live marker
and signalled only when the marker still claims it.
Second finding: isRealUserDaemon matched a hardcoded macOS userData path, so on
Linux a real daemon could never be recognised — the assertion that this run harmed
nothing was inert on the platform where nobody would notice. Now matched per
platform, with a loose fallback rather than a silent false.
Repro re-run end to end: all three phases pass, pre-existing daemons still running,
real userData daemon untouched.
* fix(daemon): only demote and only pin when the failure earns it
Both guards in the fresh-spawn catch were too broad, and narrowing them is the one
thing worth keeping from the restart-ownership branch.
Demotion fired on any spawn failure. A rejected cwd or a bad profile says nothing
about whether the daemon is reachable, and costing the whole session its daemon
persistence over one of those degrades terminals the daemon would have served fine.
It now requires the failure to look like an unreachable daemon.
Pinning fired on any failure too. The pin exists for a request that was sent and
whose answer was lost — that is the only shape that can hide a session the daemon
already created. A failure that never reached it created nothing, so pinning that
id would strand later attempts on a host holding nothing of theirs. It now requires
an error that could have been dispatched.
The distinction is sharper than the code it replaces: a hello timeout demotes but
does not pin, because a handshake that never completed cannot have created a
session. The old code pinned it anyway.
Test doubles now raise DaemonProtocolError rather than plain Errors, which is what
the client actually produces and what these predicates are written against. Both
narrowings mutation-verified.
* test(daemon): prove the recovery the degraded notice promises
The readiness review flagged one claim it could neither confirm nor refute: the
notice tells the user "reopening a pane retries, and works once it does", and
nothing pinned that. It mattered because held mode is exactly the case where no
route was ever recorded — discovery ran over the same IPC the daemon was failing —
so recovery cannot come from a cached route. It has to come from the next attach
re-inventorying a provider whose failure cooldown has expired.
It does. While wedged the resolver refuses rather than letting the fallback answer
with a fresh shell, and once the daemon answers again the same attach reattaches
the original session. Verified by mutation: with the daemon left wedged, the test
fails.
Fourth user-facing claim in this branch checked against the code rather than
assumed. The previous three were wrong.
---------
Co-authored-by: Orca <help@stably.ai>
|
||
|
NeilandOrca
|
3ab8b6a117 |
fix(ssh): stop SSH reconnect from multiplying terminals and resuming agents twice (STA-3077) (#13326)
* fix(ssh): stop reconnect from grafting panes and stacking remote leases Reconnecting an SSH-backed workspace added terminal panes the user never opened, and the remote host accumulated shells nobody was using — one report went from 2 to 19 to 20 relay PTYs across three reconnects (STA-3077). Two root causes, both in the store. Reattach could create UI. `persistPtyBinding` has four creating branches — mint a tab, mint a root leaf, split the root and graft a leaf, mint a layout. All four are load-bearing for `pty:spawn`, which can beat the renderer's debounced layout writer, but none of them is appropriate on reattach, where the pane either already exists or is gone for good. Add `mayCreate`, defaulting true so the spawn path is untouched; every creating branch already sets `terminalMembershipChanged`, so refusing is a check rather than a new code path. Lease identity had no pane key. `upsertSshRemotePtyLease` matched on `(targetId, ptyId)` alone, so a pane that re-leased under a new relay id left its predecessor live with nothing to retire it, and the next reattach fanned out over both. One pane now keeps at most one live lease. Superseded leases are marked `expired` rather than terminated: losing a lease is not proof the shell died, so the remote process is deliberately left running. Tests assert observable behavior rather than mechanism, so they stay valid under any implementation that fixes this. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record the terminal session behavior contract Properties stated as observable behavior rather than mechanism, so an oracle written against them survives a change of implementation. Records the weaker, correct form of the timer rule — a timer may never be the sole cause of a destructive action — because recovery budgets and scratch-file age gates are correct code that an absolute ban would condemn. Also notes which mechanisms are deliberately not required, so each has to earn its place rather than arrive with an architecture. Co-authored-by: Orca <help@stably.ai> * fix(ssh): heal duplicate pane leases that predate pane-keyed supersession Pane-keyed supersession stops new duplicates, but it does nothing for installs that already carry the ones STA-3077 accumulated — the report behind this reached 20 live leases across a handful of panes, and every reconnect fanned out over all of them. Retire the stale duplicates once per reattach pass, keeping the newest lease for each pane under a total order so two hosts resolve a tie the same way. As with supersession, retired leases are marked `expired` rather than terminated: their remote shells are deliberately left running, because a lease we chose not to revive is not evidence the shell died. The relay-session store stubs gain the new method. Note the gap this leaves open: those shells keep running and are no longer reachable from the app, so the "accumulates unused shells" half of the report needs a visible recovery surface rather than a silent kill. Co-authored-by: Orca <help@stably.ai> * fix(terminal): stop respawning a shell that is still running A pane that failed to reattach spawned a fresh shell. Because the restored session id came along, the replacement resumed the same agent session, and two processes appended to one transcript — reported repeatedly, up to five concurrent resumes of a single session. Two defects fed it. The relay reported a source that merely needed re-establishing as `SSH_SESSION_EXPIRED`. The shell was still running; only its output source was gone. Give that outcome its own error so it stops reading as "the session no longer exists". The reattach failure handler then treated every error as proof of death. It checked for expiry and, in the else branch, took the identical action — so the check bought nothing and a transport fault, a timed-out call, or a wedged relay all respawned. Respawn now requires proof: an explicit host expiry or a not-found PTY. Anything else, including an error we have never seen before, is unresolved, leaves the shell running, and keeps the binding for a later reattach. Two existing tests asserted the old behavior. One threw a bare error as scaffolding to reach the spawn-adoption door; it now throws proof, which is what it meant. The other pinned the expiry mapping itself, and now asserts the outcome fails closed *without* being reported as expiry. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record what makes a retention bound safe Shortening a grace period is the wrong lever. Measuring process time and gating reclamation on an independent observation are what make one safe, and they are what deployed systems actually do. Also records that lifecycle belongs in the attach reply rather than a delivered event — that is what removes the need for a durable per-consumer cursor to guarantee an exit is never lost. Co-authored-by: Orca <help@stably.ai> * test(terminal): assert the empty-failure case without an empty Error A thrown empty value exercises the same property — a failure carrying no usable message is not proof the session is gone — and does not trip the empty-error-message lint. Co-authored-by: Orca <help@stably.ai> * fix(ssh): let the durable pane binding outrank recency when retiring leases Choosing the newest lease for a pane is wrong whenever a newer lease exists that no pane is bound to: it retires the lease the pane is actually attached to, detaching a live terminal instead of healing it. Two changes. Arbitration now prefers the lease matching the pane's durable binding, across both the SSH-target and local partitions, falling back to recency only when no binding names either candidate. And supersession at upsert time now defers rather than expiring a bound predecessor. When a lease arrives for a pane that is still bound to a different PTY, the binding has not caught up yet, so both stay live and reattach arbitrates once the binding is available. Co-authored-by: Orca <help@stably.ai> * fix(ssh): roll back a lease retirement whose durable write fails `flush()` logs and swallows write errors, so a failed write left these leases retired in memory while disk still called them attached — and the pane bindings scrubbed alongside them stayed scrubbed. Use `flushOrThrow` and restore both the lease states and the affected session partitions when it throws, reporting nothing retired. Co-authored-by: Orca <help@stably.ai> * test(ssh): prove pane and remote PTY cardinality across reconnects Counts the shells the relay actually hosts, on the container, rather than inferring them from app state — that is the census the report was based on. Asserts the PIDs are unchanged, not merely the count, so a kill-and-respawn cannot pass. Every pane streams before the transport is severed: an idle pane sends no recovery checkpoint, so only a live source comes back needing re-establishment, which is the outcome that used to read as expiry. Co-authored-by: Orca <help@stably.ai> * fix(ssh): actually pass mayCreate:false from the reattach binding write The `mayCreate` guard was correct and had no production caller, so the reattach path still went through the creating branches and grafted panes back. `restoreReattachedPtyRuntime` is that call site — RC3 in the original diagnosis — and it now refuses to create. Binding moves ahead of runtime registration, because registering first would surface a pane the user never opened before the refusal landed. A refusal leaves the remote shell running and reattachable; a *thrown* write stays unknown and still registers, so a failed disk write cannot detach a live pane. Adds an oracle over the call site itself. The store-level tests all passed while the fix was inert, because they called the store directly — only pinning the wiring catches that. Co-authored-by: Orca <help@stably.ai> * fix(terminal): apply the respawn-requires-proof rule to both reattach paths connectPanePty has two near-verbatim reattach blocks — one keyed on the deferred SSH session, one on the restored session — and only the second was fixed. The first still checked for expiry and then respawned unconditionally anyway, so a transport fault there resumed the same agent session a second time. Also keep the wire token out of the pane. The main-process bridge only special-cases expiry, so a source-restore failure crossed IPC as raw `SSH_SOURCE_RESTORE_REQUIRED: <id>` text and surfaced to the user. It correctly does not respawn; it just should not read like that. Co-authored-by: Orca <help@stably.ai> * test(ssh): state plainly that the reconnect spec is a forward guard It was run against an unfixed tree and passed, so it does not prove the STA-3077 fixes and should not be read as if it does. A clean severed transport does not reproduce the field conditions — accumulated duplicate leases, or a source returning needing re-establishment. It keeps its place as a forward guard: it counts the shells the relay actually hosts and pins their PIDs, so a later change that grafts a pane or respawns a shell fails here. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record that a guard must be pinned at its call site A refusal that exists and is never passed is indistinguishable from no refusal, and store-level tests cannot tell the difference — they call the store directly. Learned from `mayCreate`, which was correct and had no production caller for several commits. Co-authored-by: Orca <help@stably.ai> * fix(ssh): park one PTY's exhausted delivery recovery instead of dropping the channel A per-PTY recovery budget running out disposed the whole relay channel, so one PTY that could not re-prove its delivery aborted every in-flight filesystem and git request on that host and stalled every sibling pane. A retry count is not proof of anything, and it certainly is not proof about the other sessions sharing the channel. Exhaustion now parks that PTY's delivery. The remote shell keeps running, its lease stands, and the next relay open reattaches it with a fresh delivery generation — the parked state is cleared on teardown and the generation changes on reconnect, so a reconnect recovers it. The consecutive-attempt ceiling goes away entirely; the per-generation one is what bounds the retry cost, and the second ceiling only existed to reach the channel drop sooner. Tradeoff worth stating: the failing pane used to self-heal within seconds because the forced reconnect wiped all rejection state, and it now stays frozen until the next relay open. That is a worse outcome for that one pane and a much better one for every other session on the host, and reconnecting is user-reachable. Co-authored-by: Orca <help@stably.ai> * fix(pty): let liveness say unknown instead of forcing it to say dead `IPtyProvider.hasPty` returned a boolean, so a provider whose inventory was empty for reasons that have nothing to do with the session — socket down, cache never hydrated, provider generation just constructed — had no way to say so and answered "absent". Its own siblings already knew better: `probePtyLiveness` and the runtime's `PtyController.hasPty` were both already `boolean | null`, with consumers branching on null correctly. The lie was injected at exactly one interface. Now three-valued, and each provider answers unknown where it cannot prove absence: the daemon adapter off-socket, the SSH provider before a completed listing, the router when any adapter cannot answer, and the degraded provider rather than fabricating a verdict. `terminal_gone` requires unanimous proven absence. Also fixes a real cold-start bug this surfaced: `pty:hasPty` never awaited the daemon-swap startup promise, though the sibling `probePtyLiveness` bridge already did, so before the swap the local provider answered an authoritative false for every daemon-owned id. Net +27 production lines. The plan behind this predicted -92 on the strength of deleting the renderer's dead-session reconcile path; that code is live (`pty-connection.ts` imports it), so nothing was deleted. Expressing a third value where there were two costs lines, and a deletion that is not real is not worth manufacturing. Co-authored-by: Orca <help@stably.ai> * docs(terminal): track the terminal-session correctness handoff package The package was untracked under a gitignored `docs/**`, with the un-ignore rules living only in an uncommitted .gitignore edit — a single `git clean -xdf` would have destroyed the authoritative plan. The 814-path construction snapshot is now pushed as `nwparker/react185-authority-snapshot` too; it had no remote ref. Co-authored-by: Orca <help@stably.ai> * test(ssh): make the reconnect settle window actually wait The settle poll reused a matcher the assertion 15 lines above had already satisfied, and Playwright's poll engine probes immediately and returns as soon as the matcher passes — so it observed the same state twice and elapsed 0ms. A shell grafted a second or two after reattach reported ready slipped through into the next cycle. Reviewer was right on #13111. Test-only; no production change. Co-authored-by: Orca <help@stably.ai> * test(ssh): census both durable session partitions on reconnect Adds a second reconnect scenario and a helper that reads pane records from the local partition as well as the ssh host partition. That split matters: the reattach binding call passes no hostId, so a grafted pane lands in the LOCAL partition and an oracle reading only the host partition passes whether or not the guard is present. Both tests remain forward guards. The second one was reported as discriminating and did not reproduce: with `mayCreate: false` removed from the call site and the app rebuilt, both still passed. Its induction races `pty:kill` against a severed transport, so when the kill lands the lease is cleaned up and there is nothing left to graft. The handoff README is corrected to say so rather than claim a journey. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record the user decision relaxing G6 G6 becomes minimise-and-justify rather than strictly net-negative. The deletion budget the plan assumed does not exist: an entrypoint-rooted import graph found 51 of 53 candidate files reachable and instantiated on live paths, leaving 263 deletable LOC against roughly +1,021 to offset. Correctness may still not be traded for line count. Co-authored-by: Orca <help@stably.ai> * test(terminal): add discriminating oracles for restart, daemon, skew and namespaces Six parallel streams, each required to fail with its guard removed rather than merely pass. Local restart proves the OS process itself survives, by reading `ps -o lstart=` for the shell's own pid. That matters: with the quit path made destructive, the tab, leaf and pty ids all came back byte-identical while the shell underneath was a new process — every existing restart spec would have stayed green. Two separate guards were removed to redden it, and the second reddens only the stale-operation case. Daemon restart discriminates by reverting three-valued `hasPty`; version skew now covers publication semantics and confirms the new `SSH_SOURCE_RESTORE_REQUIRED` token mutates nothing on an old client; two-host isolation censuses both containers. Deletes `src/relay/pty-source-replay-index.ts` — 201 production lines with no importer outside its own test, verified against an entrypoint-rooted import graph rather than a name grep. Five namespace tests are skipped, not passing: they reproduce a defect still live on main where folder-workspace ids compare equal with the instance suffix stripped. PR #12474 fixes it; they are its oracle. Co-authored-by: Orca <help@stably.ai> * test(ssh): induce the reattach graft deterministically instead of racing a kill The previous induction closed a pane while the transport was severed and relied on `pty:kill` FAILING so the lease outlived the pane record. It does not fail: with the provider already torn down, `pty:kill` takes its tombstone branch and marks the lease terminated, and `reattachKnownPtys` filters terminated leases out of the fan-out — so the reconnect never visited the PTY the test was about. It passed on both trees. Seed the precondition instead. Spawn a real remote PTY on a leaf that never becomes a pane, then roll the host partition back to its pre-spawn snapshot, leaving a live lease and a live remote shell that no durable pane owns. No failure races a success. Adds a vacuity guard that is independent of the tree under test: the lease's own `lastAttachedAt` must advance, proving the fan-out actually visited this lease before the pane census is trusted. Verified on this machine under an isolated TMPDIR, since the e2e harness keys its seeded-repo pointer on a machine-global tmpdir path: guard present passes, guard removed fails with the phantom leaf grafted into the local partition, guard restored passes. Co-authored-by: Orca <help@stably.ai> * docs(terminal): propose one authoritative binding identity Every defect this program has touched is the same defect: identity compared with the wrong key, or not compared at all. Lease keyed without the pane, reattach using a creating write, folder-workspace ids compared with the instance suffix stripped, local mutating IPC carrying only an id, a live shell classified as expired, liveness unable to say unknown. Proposal: one branded binding type built from fields that already exist and are already persisted, constructible only from an authoritative source, carried by mutating operations, compared by one shared function. Makes a wrong-key comparison a type error rather than the next incident. Under adversarial review, including against the open issue corpus. Not accepted. Co-authored-by: Orca <help@stably.ai> * fix(pty): refuse mutating operations aimed at a superseded PTY `pty:write`, `pty:writeAccepted` and `pty:resize` accepted any id. The renderer queues input, so a keystroke buffered before a reattach landed on whatever PTY had since taken the pane — and a resize reshaped the successor's shell. Main already tracks `ptyPaneKey` and `paneKeyPtyId` in lock-step, so their disagreement is proof the caller's id was superseded. No wire change, no renderer change, nothing added to the input payload. An id with no recorded pane stays permitted: unowned and orphaned PTYs are unknown, not stale, and unknown never authorizes refusing an explicit operation. That is also what keeps orphan cleanup working — those ids have no pane by construction. The tests pin the CALL SITES, not the predicate. A capability that exists and is never called is indistinguishable from no capability, which is exactly how `mayCreate` sat inert here for several commits with every test green. Co-authored-by: Orca <help@stably.ai> * fix(pty): fence signals at a superseded PTY, and pin why kill is exempt A signal means "interrupt my pane", so delivering one to a PTY the pane has already replaced is a misdirected interrupt. Fence it with the same lock-step proof used for write and resize. `pty:kill` stays deliberately unfenced and a test now pins that: a superseded PTY is orphaned, and reclaiming it is exactly what the orphan-cleanup callers ask for. Refusing there would break the operation that reclaims leaked shells — the opposite of the intent. The fence sits at the IPC boundary, above `tryGetProviderForPty`, so it covers local, daemon and SSH rather than the local path alone. Co-authored-by: Orca <help@stably.ai> * test(terminal): poll the pane binding read so a slower host cannot flake it `readPaneBinding` took a single unpolled read of a DOM dataset attribute immediately after a renderer reload, while its sibling helper polls the same data for 15s. On a native Linux host both tests failed every run with 'No bound terminal pane is mounted' while the app was demonstrably healthy — the screenshot showed the terminal restored with a live prompt and the boot PID echoed. The assertion is unchanged; it is only awaited. Nothing is weakened. Found by running this spec on native Linux rather than assuming macOS behaviour generalises. Co-authored-by: Orca <help@stably.ai> * test(terminal): make the restart identity spec run on Windows too Both probes were POSIX-only and unconditional: `echo ...=\$\$` for the shell's own pid, and `ps -o lstart=` for its start time. Running the spec on a real Windows host proved it dies before reaching either guard, so Journey 1's Windows half was unprovable rather than merely unproven. PowerShell exposes the same two facts as `$PID` and `Get-Process` StartTime. The start time still matters on both platforms for the same reason: a PID alone cannot separate a survivor from a reused number. Still green on macOS. The Windows path is written from the host probe and has not itself been executed end to end — that is the next thing to run there, not a claim being made here. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record the fence's real gap and what peer designs taught Marks the client-constructed binding proposal as rejected with the three false claims that sank it, and records what shipped instead. States the shipped fence's actual limitation rather than leaving it implied: it compares a binding, not an incarnation, so a respawn under a reused ptyId passes. The obvious remedy is wrong here — the agent-create id is deterministic by design so a replayed create stays idempotent, and randomising it would trade this narrow gap for a duplicate-spawn bug. Also records the ranked lessons from four comparable agent IDEs, chiefly that a typed end-reason at end time is what stops a user quit from looking like a resume candidate. Co-authored-by: Orca <help@stably.ai> * docs(terminal): promote Journey 1 to proven on all three platforms The oracle now runs natively on macOS, Linux and Windows, and its discrimination was watched on each: a mutation reddens it, a restore greens it. On Linux and Windows both mutations were run, and the second reddens only the stale-operation test — so the journey's two clauses are proved independently rather than jointly. Windows is the new evidence. The PowerShell branches added blind at ebffb85a848 executed correctly on their first run: `$PID` expanded to real integers, which also proves the pane shell there is PowerShell-family rather than Git Bash, and `Get-Process StartTime` returned kernel start times 5.4s apart — so a recycled pid could not have passed as a survivor. First journey promoted in this program. The other twelve are unchanged, and the residual limit on "every stale exact operation" is recorded rather than glossed. Co-authored-by: Orca <help@stably.ai> * test(terminal): add discriminating oracles for the daemon, skew and multi-host journeys Daemon: replaces a spec that modelled only a client restart and never crossed the daemon boundary, whose successor generation owned nothing so "the live successor is neither killed nor replaced" was vacuous. The PTY leader is now a real login shell reporting `$$` back through the production write path, resolved to a kernel start time. Two mutations each redden exactly one of the three clauses, on macOS and Linux: reverting three-valued `hasPty` reddens only the unknown-not-dead clause; widening the sole-provider fallback reddens only the stale generation clause. Skew: reverting the restore-required publication to expiry reddens 4 of 5 new tests while the legacy control stays green — the regression this branch fixed is now caught if reintroduced. Multi-host: restoring `mux.dispose('connection_lost')` reddens sibling isolation on one host. It does NOT redden across hosts, and that is recorded rather than glossed: a mux belongs to one relay session per target, so its dispose cannot cross a host boundary. Journey 4's cross-host clause rests on isolation-by-construction, not on a mutation. No production code changes. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record journey evidence that falls short of promotion Four journeys now have discriminating oracles but none meets its full stated scope, and each shortfall is named rather than rounded up. Journey 2 is one WSL run from promotion. Journey 12's tests are in-process, so they do not close the live-skew gap the original ledger named. Journey 4's cross-host clause cannot be proven by mutation at all — a mux is per target, so its dispose cannot cross hosts, and the cross-host test stayed green under the mutation that reddens siblings. Journey 13 measured one dimension of ten, on lifted predicates rather than through real IPC. Co-authored-by: Orca <help@stably.ai> * docs(terminal): promote Journey 2 to proven on macOS, Linux and physical WSL The oracle runs on every environment the journey names, and is clause-selective on all three: reverting three-valued `hasPty` reddens only the unknown-not-dead clause, and widening the sole-provider fallback reddens only the stale-generation clause. Selectivity in WSL was established rather than assumed. The spec runs serially, so a red first test reports the others as "did not run" — they were re-run alone under the same mutation and stayed green. Also records that an Orca WSL-mode terminal now starts on that host at all, which it could not before: the distro had no provisioned default Unix user, so every interactive launch blocked on first-run setup. One diagnosis from the WSL run is corrected here rather than repeated: the unrelated `local-pty-shell-ready` failure was attributed to bash 5.3.9, but macOS runs the same bash version and passes 67/67. The trigger is environmental to that distro, and the underlying defect is that the spec pins an absolute count of OSC markers it does not own. Co-authored-by: Orca <help@stably.ai> * docs(terminal): correct the WSL provider-suite diagnosis The WSL run blamed bash 5.3.9 for the unrelated `local-pty-shell-ready` failure. macOS runs the same bash version and passes 67/67, so the version is not the cause — the trigger is environmental to that distro, and the underlying defect is that the spec asserts an absolute count of OSC markers it does not own. Co-authored-by: Orca <help@stably.ai> * test(runtime): unskip the workspace-namespace oracles now their fix has merged These five reproduced a defect that was live on main: folder-workspace ids were compared with the instance suffix stripped, so two workspaces sharing a directory read as the same namespace. They were committed skipped, pointing at the PR that fixes it. That PR is merged, and they pass. Verified they still bite: restoring the suffix-stripping comparison reddens exactly these five and leaves the other four green. An oracle written before its fix, held skipped, and confirmed against the fix after the merge — rather than deleted and rewritten from the answer. Co-authored-by: Orca <help@stably.ai> * test(ssh): add MaxSessions, lazy-discovery and paired-skew oracles Three journeys attempted; none promoted, and the reasons are recorded in the ledger rather than rounded up. MaxSessions=1 against real OpenSSH, with the cap read back from `sshd -T` rather than assumed, and remote pids read on the container two independent ways that must agree, each carrying its kernel start time. Two disjoint mutations discriminate — one reddens only the reconnect clause, the other only the two restart clauses. But the disconnect clause is a forward guard: four separate guard removals left it green, so nothing shipped is load-bearing for it. Lazy discovery samples sshd's own accept log and live session census across a 22s window with the in-use host as a positive control. No mutation reddens its third clause alone — the real cross-host lease scoping is load-bearing, but removing it breaks the sibling host during setup, so the failure carries no clause information. The paired-runtime skew spec pairs two real processes at different versions and refuses to run rather than degrade into a same-version pairing that would look green and prove nothing. No production code changes. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record why the duplicate-resume fix was not built I recommended adding a typed end-reason so a user quit stops looking like a resume candidate, then went to implement it and stopped. `SleepingAgentSessionRecord` already carries three fields that each exist to stop something resuming that should not have — `origin`, `restoreOnTabOpenOnly`, and `automaticResumeBlockedBy` — each traceable to its own incident, consulted at 22 non-test sites. A fourth predicate, however well typed, is the fifth containment cycle. The designs without this bug do not have a better flag; they resume only on an explicit action, into a new terminal id, and make two agents in one terminal unrepresentable in the schema. The first of those is a product decision about whether automatic resume stays a feature, so it is the user's call rather than mine. Co-authored-by: Orca <help@stably.ai> * docs(terminal): reconcile G6 with the recorded decision and assess its clauses G6's body still demanded strictly-negative production LOC after the user relaxed it to minimise-and-justify, so the gate had two conflicting pass conditions and no single truth value. Its body now points at that decision. Assessed the remaining clauses against the branch rather than assuming. Two fail structurally: more than one identity comparison and mutation admission path still exist, and `terminal-input-quarantine.ts` is still reachable from two production files. Records why the quarantine is not subsumed by the superseded-PTY fence, which I had assumed and checked. The fence refuses writes aimed at a stale ptyId; the quarantine guards the user's next keystrokes landing on the successor under its current, correct id — a case the fence never sees. Removing it needs the recovery path to surface a different shell as unresolved, not a deletion. Co-authored-by: Orca <help@stably.ai> * docs(terminal): the input quarantine is load-bearing, not superseded G6 lists "no superseded quarantine remains reachable" and this module was assumed to be one. Disabling its single call site reproduces the hazard it exists for — `cho hi; rm -rf x` reaching the shell — so deleting it without a replacement re-opens command execution. The replacement was costed by building it rather than estimated: +26 production LOC to thread the incarnation, ~+33 complete, and the cross-remount state it needs outlives the destroyed pane so it becomes a module about the size of the one deleted. Floor is roughly +140 to delete 88, and it would add a second identity comparison to a gate already failing for having more than one. The decisive part is that the route is not uniformly available: remote runtime results carry no incarnation, old hosts cannot be made to publish one, and mixed versions are the normal state. A paired client reads unknown, which this program's own rule says is not proof — so either every remote reattach surfaces unresolved, or a fallback is needed and the only correct fallback is this module. Whether to amend the clause or accept something weaker on remote hosts is a user decision, so the clause verdict is left as failing rather than quietly reclassified. Co-authored-by: Orca <help@stably.ai> * refactor(runtime): collapse duplicate identity comparisons G6 requires one identity comparison; five implementations existed across two concepts. Worktree-namespace identity had two: `runtimeWorktreeIdsEqual` and `runtimeWorktreeIdentityKey` independently re-derived repoId plus normalized path. Equality now derives from the key, so the comparison and the sleep / mutation-queue keying cannot drift into two different rules — which is exactly how the suffix-stripping bug reached production once. Pane identity had three byte-identical leaf-UUID comparisons, in orchestration `db.ts`, `lifecycle-reconciliation.ts`, and `orchestration-legacy-process-identity.ts`. One copy moved to `stable-pane-id.ts`, which already owns `PaneKey`, `parsePaneKey` and `makePaneKey` and which all three already imported. No new module, no branded type, no parallel comparison. Net -14 production lines. The namespace oracle still bites: restoring the filesystem parser inside the identity key reddens exactly its five cases. The raw counts are not the actionable set, and the classification is worth recording: of 409 non-test `worktreeId` comparisons, 71 are typeof guards and 81 are sentinel tag checks. Most of the remainder are renderer predicates over store rows where both operands are the same main-minted id, so normalizing there would widen equality rather than correct it. Co-authored-by: Orca <help@stably.ai> * refactor(terminal): finish a half-done fixture move and audit the rest `xterm-bypass-event-fixture.ts` and `__fixtures__/xterm-bypass-event.ts` were byte-identical apart from an import path. The `__fixtures__` copy had zero importers and the live copy compiled as production — someone started the move and left both. Dead copy deleted, live one moved, its three test importers updated. Audited the wider G6 clause by importer rather than filename: 32 test-only files, roughly 3,300 LOC, currently compile as production; 4 of the 36 candidates have real production importers and are correctly placed. The list is recorded in the goalposts. Those 32 are almost all older than this program and outside the terminal surface, so sweeping them belongs in its own change rather than inside a terminal PR. The clause stays failing, with the remaining files named. Co-authored-by: Orca <help@stably.ai> * docs(terminal): the fixture clause already holds where it matters Checked what the build emits rather than reasoning from file paths. None of the 32 test-only fixtures appears in `out/` — Rollup drops them because no production entrypoint reaches them. On "compiles into the shipped product", this clause holds today. On the other reading it cannot be closed by moving files at all: both production tsconfigs use bare `include` globs with no `exclude`, so a `__tests__/` directory matches exactly like any other path, as does every `*.test.ts` in the repo. Relocating 32 fixtures would remove nothing from typecheck scope. A sweep was started and stopped once this was verified, rather than landing 32 moves across areas this program does not own for no gain. If the intent is that typecheck scope should exclude test code, that is a repo-wide tsconfig change with a different owner. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add plain-language design and test overviews Two reviewable documents with diagrams, written so someone with no prior context can follow what breaks, why, and what changed. The design overview explains the five things stacked behind one terminal rectangle, the 2 -> 19 -> 20 report, the three root causes, and the rule underneath all of them: unknown is not dead. The test overview explains why a green test proves nothing on its own, the four-step mutation proof we adopted, and — the part worth reviewing hardest — an honest account of what could not be proven and why, including the properties that are true by construction and therefore have no guard to remove. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add a self-contained visual report of the design and its evidence Pre-renders every diagram to inline SVG in both themes so the report opens offline and stays sharp when zoomed. States the gate/journey score and the retractions alongside the fixes, so the unproven half is as visible as the proven half. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record the finalized two-plane architecture decision Adopts the data-plane proposal and adds the control-plane track it does not cover: re-key ownership by pane, split orphan inventory out, then delete the compensating code. Records that the host-authority alternative was refuted and that the shipped keystroke fence is inert on the reattach path. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add the design brief the review counsel works from Separates verified code facts from unverified leads so reviewers attack the design rather than a reconstruction of it, and records which simpler alternatives were already refuted and why. Co-authored-by: Orca <help@stably.ai> * docs(terminal): report the design counsel's outcome and the live respawn bug it found Three review rounds across two models replaced the two-record split with one leaf-keyed record, deleted attach-time pane identity, and made orphans a connect-time projection. Records that a shipped gesture still turns a healthy remote shell into a duplicate agent resume, and that the renderer classifier in that chain treats an error-message shape as proof of death. Co-authored-by: Orca <help@stably.ai> * docs(terminal): correct the report — the respawn proof gate guards a minority path A final review traced every auto-respawn route. The primary one converts the reattach failure into a boolean before any classifier sees it, so the shipped proof gate never runs there. Records that two of the six shipped changes are narrower than claimed, and why their tests could not have caught it. Co-authored-by: Orca <help@stably.ai> * docs(terminal): explain the landed design on its own terms One leaf-keyed ownership record, orphans computed at connect, and replacement shells only on positive proof — with the shipping order and the one product trade the design asks the owner to accept. Co-authored-by: Orca <help@stably.ai> * docs(terminal): rewrite the design explainer in plain English The first version assumed the reader knew the codebase. Reframed around two bugs, two fixes and one decision, with the jargon replaced by pane / program / note / helper and a five-word glossary for what could not be avoided. Co-authored-by: Orca <help@stably.ai> * fix(ssh): stop reading an identity mismatch as a dead shell The relay reports a pane-identity mismatch by saying the pty was not found, but it found it — comparing identity is how it noticed. Publishing that as expiry made the renderer clear the binding and cold-restore with agent resume, so a live shell gained a second agent on one transcript. Reachable today by detaching a pane into a new tab, which changes the tab the relay froze at spawn. Mismatch now carries its own token and the classifier refuses it as proof. Genuine absence still expires, so a shell that really went away is not stranded. The three failure tokens move to src/shared: main published them and the renderer decided respawn on them, from two copies that had drifted apart. Co-authored-by: Orca <help@stably.ai> * fix(ssh): stop sending pane identity on reattach The relay froze pane identity at spawn, so moving a pane to another tab made it refuse a live shell — and refuse by saying 'not found'. The comparison is presence-guarded, so not sending the fields disarms it on every relay version including ones already installed on hosts: no wire change, no redeploy. Nothing is lost. It existed to catch a relay restart recycling pty-N for a new shell, and in exactly that case pane and tab both still match, so it accepted the wrong shell anyway. The incarnation the attach returns is what distinguishes those, and it already crosses the wire. Removes the whole client-side apparatus: the expected-identity type, its per-lease derivation, its map, and the parameter threaded through four layers. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add tracked goalposts for the new design Each goalpost is a behaviour with an oracle and the mutation that must redden it, so 'proven' cannot be claimed from a green test. Records the anti-inert rule as a first-class goalpost, since three guards in this program passed their tests while sitting off the route production takes. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record that the recovery grant is dead code, deleting a design step The lease stores a relay-native pty id and the caller passes the app form, with a raw equality comparison between them, so the 30s grant cannot fire for a real SSH pane. The death rule that existed to referee it is deleted rather than built, and the dead path itself becomes a removal. Co-authored-by: Orca <help@stably.ai> * docs(terminal): keep the full design detail in the repo It only existed in an ephemeral job directory, so the plain-English explainer had no durable source for its specifics — record shape, death rule, reattach algorithm, migration order and the 25 oracles. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add a resume prompt for a clean session Points at the goalposts as the contract, names the three goalposts whose oracles are already written and red, and carries the process rules that were learned the expensive way — prove guards reachable, verify mutations land, commit per step, and never let a subagent write production files in a shared worktree. Co-authored-by: Orca <help@stably.ai> * test(ssh): add the failing oracles for goalposts S3, S4 and S5 Intentionally RED: 14 clauses that fail against current behaviour and go green under the changes named in new-design-goalposts.md. The branch is held unmerged, so red here means unimplemented, not broken. Each was verified to fail for the right reason and to flip green under the identified fix, which was then reverted. Each pins the producer as well as the consumer, so no clause can pass vacuously if its route is ever severed — the failure mode that let three earlier guards ship inert. Co-authored-by: Orca <help@stably.ai> * fix(ssh): stop fabricating an exit when a reattach fails A failed attach never proves the shell exited. The relay answers not-found for a pane-identity mismatch and for any id it merely cannot hand back, so treating it as death sent the pane a synthetic `pty:exit { code: -1 }`, cleared provider state, deleted ownership and expired the lease — four claims about a process we know nothing about, on a shell that is usually still running. Collapse every failure into the non-destructive branch that already existed a few lines above (`restoreRequired = 'reattachAttemptsExhausted'` + wakeRecovery). A branch collapse, not a new mechanism: goalpost S3. Two tests pinned the deleted premise and are INVERTED rather than patched, so the new intent stays covered: - ssh-relay-orphan-abandon-paths: "retires the lease without a kill when the relay proves the PTY is gone" -> "leaves the shell running when the relay only reports the PTY as not found". Its comment claimed attach verifies liveness before answering not-found; it does not. - ssh-relay-session: "invalidates and broadcasts remote PTYs that cannot reattach" -> "leaves an unreattachable remote PTY alone while its sibling reattaches". Also repairs two clauses left red by |
||
|
Neil
|
54e01e6df8 | test(wire): select stable desktop release baselines (#14156) | ||
|
Jinjing
|
7c93aed6dc |
Fix fsync of read-only files on POSIX (#14235)
* fix(files): fsync read-only files on POSIX * test(e2e): add golden E2E tests for POSIX profile index fsync Validates that profile index files are properly persisted on POSIX systems, including with restrictive umask settings. These are release-blocking golden tests for Linux and macOS. * test(terminal): wait for fish child ownership before stdin write Fish 4.8 withdraws DECSET 2031 before spawning the child, so the shell-contracts harness could send hello into an intermediate prompt and hang waiting for CHILD-READ. Wait for the child's CHILD-READY marker and answer split DA1/CPR/OSC queries across chunk boundaries. * test(e2e): verify profile index persists to disk with restrictive umask Strengthen the POSIX fsync test to verify the rebuilt index is actually written to disk and has correct permissions under a restrictive umask, not just cached in memory. |
||
|
Jinjing
|
939acf1c2b |
Fix remote server browser tab fallback behavior (#14194)
* fix(browser): require runtime browser capability * fix(browser): reuse capability verdict and exclude floating terminals Cache the runtime browser capability check across multiple remote port opens instead of checking for each port independently. Prevent floating-terminal workspaces from routing to remote browsers. * fix(browser): pin remote pane link opens to owning runtime Remote pane link opens must route to the pane's runtime, never the client. Fails if the workspace has moved to a different host while the pane still displays a remote page, preventing silent misrouting to a dev server. * trim comments |
||
|
NeilandOrca
|
c61f29639c |
fix(workspaces): recognize pasted Linear issue URLs (#14190)
* fix(workspaces): recognize pasted Linear issue URLs * fix(workspaces): resolve legacy Linear URLs and keep typed-name fallback Saved API-key Linear workspaces often omit organizationUrlKey, so pasted issue URLs never called fetchLinearIssue. Probe those unknown-org workspaces and accept only a matching issue URL. Keep "Use … as workspace name" visible in Smart Entry when a Linear URL owns the results. sourceIntent still focuses the issue row. Co-authored-by: Orca <help@stably.ai> * fix(workspaces): jump to existing worktrees from pasted task URLs Cmd+J now treats GitHub, GitLab, Jira, and Linear issue/PR URLs as decisive search, so pasting one lists already-linked worktrees first and keeps a create preview underneath. GitHub/GitLab/Jira still hand the raw URL to the composer for cross-project detection. Co-authored-by: Orca <help@stably.ai> * fix(workspaces): resolve GitHub issue titles in Cmd+J URL paste Pasted GitHub issue/PR URLs now fetch the title for the create preview, same as Linear. Existing linked worktrees stay selected first so Enter jumps; create still hands the raw URL to the composer. Co-authored-by: Orca <help@stably.ai> * fix(workspaces): attach resolved GitHub items from Cmd+J create Pasting a GitHub issue/PR URL into Cmd+J already previewed the title, but Enter still opened the composer with the raw URL. Await the in-flight lookup and hand the linked work item through, matching Linear and Task page create. Also fix CI type, lint, and focus-routing source checks. Co-authored-by: Orca <help@stably.ai> * fix(workspaces): add Cmd+J task-URL locale keys Unblock PR CI localization catalog checks and make the Linear lookup-miss e2e wait out the resolving state before advancing to the agent field. Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Orca <help@stably.ai> |
||
|
Neil
|
602f0cbe63 |
fix(sidebar): stabilize downward worktree card dragging
Fix downward worktree card dragging with virtualization-safe global indices and stable preview geometry. Add unit and Electron regression coverage. |
||
|
Jinwoo Hong
|
9a10561258 | fix(terminal): retain SSH startup delivery through reconnect (#14161) | ||
|
Jinwoo Hong
|
de729d6067 | Fix paired web creation failure handling (STA-4024, STA-4025, STA-4063) (#14100) | ||
 m4air
|
b115f8d256 |
Add Windows golden E2E test for fresh-startup regression
Windows terminal rendering golden is flaky on CI runners. Re-enable Windows in the golden E2E gate with a scoped test for the fresh-profile startup regression from #14130. Terminal rendering continues on Linux and macOS; Windows runs fresh-startup only. |
||
|
Jinwoo Hong
|
e8350f1990 | fix(browser): route remote terminal links to owning host (#14117) |