Commit Graph
4 Commits
Author SHA1 Message Date
Neil e42c60e8a3 fix(ssh): resolve a pane's binding from the target partition, not the stale local copy (#18546)
One SSH pane accumulated one extra reattachable lease per relay restart (2, 3, 4,
5, 6 across five), and every one of them costs a `pty.attach` round trip on every
later connect, forever. Nothing prunes `sshRemotePtyLeases`, so the fan-out only
grows.

`supersedeSiblingLeasesForPane` is fenced on the PTY the pane is durably bound to,
and `durablyBoundPtyIdForPane` read `state.workspaceSession` (local) before
`workspaceSessionsByHostId['ssh:<target>']`. But `persistPtyBinding(binding, hostId)`
updates ONLY the host partition:

  AFTER-PERSIST  local= ssh:t@@pty2:old:1   host= ssh:t@@pty2:new:1

So for the length of a reconnect the local copy still names the predecessor, the
fence resolved to it, supersession took an already-`expired` lease as its winner,
and returned having marked nothing. Both partitions agree again once the renderer
republishes its layout, which is why the settled store looks consistent and hid
this.

Read both partitions as an ordered list, target's own first, and test the fence by
membership rather than by equality with whichever was read first. Pick the winner
preferring a lease this client still has a route to, since the stale partition
names an expired one. Never retire a lease that is both bound and live, so a
partition disagreement can't strand a running remote process.

Superseded predecessors stay `expired` and are never `terminated`: losing a lease
is not evidence the shell died (docs/reference/ssh-execution-boundary.md). A pane
with no binding is skipped rather than pruned, so a genuine orphan stays askable.

Also re-runs supersession from the binding side after each spawn commit's binding
write, so the lease/binding order at a call site no longer decides, and reconciles
every pane for a target immediately before `reattachKnownPtys` reads the set it
feeds to `pty.attach` — that repairs stores which already accumulated these rows.

The guard suite could not catch this: every assertion bound the pane BEFORE
upserting the lease, an order no caller uses. Rewritten to the spawn commits' real
order (lease, then binding, then the binding-side trigger); it fails 8 assertions
without this change. Added a suite that drives the real `persistPtyIpcSpawnCommit`
rather than the store primitives, including the exact stale-partition state written
by production's own binding writer.

Verified on the Docker SSH lane: five `relay.js` SIGKILLs with recovery between
each, reattachable leases flat at one per pane.

Note: this bounds the reattach SET, not the store. `sshRemotePtyLeases` still has
no cap or TTL and rows still accumulate; pruning is left alone deliberately, since
an `expired` row without `supersededBy` is a genuine orphan and must not be dropped
on age.
2026-09-03 16:47:32 -07:00
NeilandBrennan Benson fbe94ceff6 fix: close readiness gaps found by merged-change audit (#17159)
* fix(ssh): fence stale kills and retired pane replay

* fix(ssh): support cancellable interactive authentication

* fix(ssh): await remote catalog before snapshot adoption

* fix(pty): contain Windows ConPTY input failures

* fix(power): avoid redundant macOS display blocking

* perf(editor): narrow markdown override subscriptions

* fix(quick-open): close directory handles after reads

* refactor(linux): remove unused proc socket scanner

* fix(usage): apply flat Sonnet 4.6 pricing

* ci: prime Node next native test cache

* docs(skills): resolve snapshot cleanup data path

* fix(ssh): recover install locks after host reboot

* test(ssh): recognize boot-aware install locks

* test(ssh): prove previous-boot lock recovery live

* test(wire): pin pre-metadata release coverage

* fix(terminal): preserve remote tab ownership through recovery races

* test(runtime): fence replaced terminal handles in agent guard

* fix(ssh): preserve remote snapshot authority across polls

* fix(pty): contain late ConPTY output EPIPE

* test(pty): register Windows exit watcher before kill

* fix: close SSH and tab readiness race gaps

* fix(tabs): retain headless order and placeholder titles

* fix(build): avoid parallel electron-vite config race

* test(windows): avoid MSYS temp path rewriting

* test(windows): avoid killing exited PTY

* fix(pty): avoid late ConPTY input teardown race

* fix(terminal): sync reconnect error ownership after commit

* fix(runtime): use canonical worktree identity comparison

* test(ssh): assert complete cold-hydration baseline

* test(windows): invoke quoted retention fixture via PowerShell

* test(windows): read ConPTY grid through mode con

* fix(terminal): publish PTY replacements atomically

* fix(terminal): infer stale identity on reattach

* fix(terminal): fence stale pane PTY callbacks

* fix(terminal): fence stale pane binds after rebind

* fix(terminal): reject stale pane transport callbacks

* fix(terminal): fence mirrored reattach spawn callbacks

* fix(terminal): replace stale pane PTYs on remount

* fix(ci): size the Windows launcher-compile test budget from measurement

`native-smoke (windows-latest)` fails ~4.5% of runs on
`preserves a multiline argument through the compiled remote launcher`
with "Test timed out in 15000ms" — on unrelated PRs, for reasons that
have nothing to do with them. Across 176 sampled attempts it is the only
red that job produced, and it hit seven different PRs in two days:
#16900, #16904, #16915, #16955 (twice), #16979, #17014, #17085.

The test is six process creations: powershell.exe forks csc.exe, then
the freshly compiled orca.exe forks node.exe, twice. Hosted Windows
runners periodically slow process creation down, and this test amplifies
that far harder than anything else in the job. Comparing the 80 attempts
where it ran under 3s against the 12 where it ran over 12s, its own
median goes 2198ms -> 15917ms (7.2x) while the same file's
powershell-only test moves 556 -> 686ms (1.2x), the cmd.exe and Git Bash
process tests in the neighbouring file move 1.4x, and the other 35 files
put together move 1.5x.

Measured across those 176 attempts: 1881ms to 35438ms, p50 4264ms,
correlation +0.881 with the job's total Vitest duration. 8 of 176 (4.5%)
exceeded the 15s cap; 2 of 176 (1.1%) also exceeded the shared 30s
testTimeout, so deleting the override and inheriting the config is not
enough on its own. 60s clears all 176 with 1.7x headroom on the worst.

This is slow, not hung. Every body here is synchronous spawnSync, so
Vitest cannot interrupt one — the timer fires only after the body
returns and the reported duration is real elapsed time. That is why a
failure reads `× ... 22464ms` under `Test timed out in 15000ms`. The
work finished; the stopwatch was short. Seven reruns at one identical
head measured 2053 / 4680 / 5551 / 8732 / 13506 / 14868 / 21937ms — the
last of those would have been red on code that had not changed.

The 15s came from #8897, which raised this test off Vitest's built-in 5s
default because the job then ran bare `pnpm vitest run`. #8909 landed
3h27m later and pointed the job at config/vitest.config.ts, which is the
real fix for that. The constant stayed behind and has been the binding
budget ever since.

* fix(terminal): fence stale remount reattach ownership

* fix(terminal): reconcile mounted pane identity after replacement

* fix(terminal): fence stale reattach fallback ownership

* fix(terminal): fence deferred SSH reattach ownership

* fix(terminal): fence stale split pane ownership callbacks

* fix(terminal): keep stale spawns from consuming startup

---------

Co-authored-by: Brennan Benson <79079362+brennanb2025@users.noreply.github.com>
2026-08-31 08:17:40 -07:00
Brennan Benson 6ba6d58cd2 fix(orchestration): route @agent messages by resolved identity, not terminal title (#16237)
* feat(agent-status): add the pane agent identity resolver

Four ladders answer "which agent is in this pane" independently — the tab icon, the
open-tab/search occupant, the sidebar title rows, and the sidebar hook-row fallback — and they
disagree. Two consult the terminal title before the launch record, so a string Orca parsed
outranks a fact Orca owns.

resolvePaneAgentIdentity is the single ranked answer. Two rules, one of which is not an ordering:

1. Evidence is ranked by how directly it observes the process; a display title is last.
2. Each observation carries the runId of the agent run it describes. Evidence from a superseded
   run is INELIGIBLE, not merely outranked.

Rule 2 is the part reordering could never supply. A completed hook naming A plus a title naming
B is either a bug (hook right, title stale) or a legitimate pane reclaim (title right) —
identical signals, opposite correct answers. Run ids make them different facts: in the bug both
belong to the current run; in the reclaim the hook belongs to a previous one. That pair ships as
a test asserting the two produce opposite answers from the same evidence.

Missing run ids are treated as eligible. Absence means "this peer does not publish them", not
"this is stale", so an old host's rows are never blanked. Sibling evidence is opt-in so
pane-scoped consumers cannot inherit another pane's agent.

No consumer imports this yet; each migrates separately with its own evidence.

Verified non-vacuous: reversing the authority order fails 10 of 18 assertions and removing the
run filter fails 3.

* fix(agent-status): close three resolver contract holes found in review

**Duplicate evidence of one source resolved by array order.** `eligible.find(...)` returned the
first match, so two live hooks naming different agents were settled by input position — the exact
property this resolver exists to remove. The original order-independence test only used DISTINCT
sources, so it never exercised it. Conflicting same-class evidence now returns null with
`ambiguousAt`, and does NOT fall through to a weaker source: letting a title answer whenever two
hooks disagree is worse than saying nothing.

**A bare numeric runId collided across authority restarts.** `incarnation` is a total order only
within one `authorityId` (agent-status-observation.ts states this), and the id is regenerated per
authority instance, so a restarted host counting from its own floor would report `1` and match an
unrelated live run 1. The run key now carries its authority, and evidence from a DIFFERENT
authority is treated as incomparable — kept, like an absent key — rather than as stale.

**Title stayed reachable by consumers that authorize writes.** Ranking it last makes misuse
unlikely; `minimumSource` makes it impossible. An action consumer passes `'launch'` and weaker
evidence is dropped before ranking, so routing or delivery cannot name a target from a parsed
string even by reordering its inputs. Display surfaces omit it and are unaffected.

Also restores the generic agent-vocabulary parameter, which lives on the routing branch and was
lost when this branch was rebased.

Each fix is mutation-verified: first-match restored fails 3, ignoring authority fails 1, dropping
the floor fails 2. The authority test was itself vacuous on the first attempt — both sides used
`incarnation: 1`, so a resolver ignoring authority still passed on the numeric compare. It now uses
differing incarnations.

The remaining review finding, that `process > launch` has no freshness bound, is NOT fixed here:
it needs an observation timestamp the evidence type does not yet carry. Recorded rather than
silently dropped.

* fix(orchestration): route @agent messages by resolved identity, not terminal title

`@claude` picked its recipients with `buildAgentNameRe('claude').test(title)`, so any pane whose
TITLE contained the word received Claude's messages. Terminal titles carry task text, and people
describe agent work in them, so this is the ordinary case rather than a contrived one: the
recorded title "Switch Claude and Codex off the load balancer… - grok" is a Grok pane that
received both @claude and @codex. Misdelivered instructions, not a cosmetic slip.

The cause is that `RuntimeTerminalSummary` carried no identity at all — `title` was the only
identity-ish field on it, so routing by title was the only option available. Fix the input:

- `RuntimeTerminalSummary.agentIdentity?: TuiAgent` — optional, host-resolved from launch and
  foreground-process evidence the host owns, with the title ranked last and contributing only
  when the evidence parser finds an unambiguous name. A title that merely mentions an agent
  yields no evidence, which is the whole point.
- `resolvePublishedPaneAgentIdentity` in `src/shared` rather than inside the runtime class, so
  the decision is testable without a runtime and so routing, delivery and the UI cannot drift.
- Groups match `agentIdentity`; the title matcher and its bespoke Cursor predicate are deleted.

Unknown fails closed. `agentIdentity` is absent when the host predates the field or had no
evidence beyond the title, and delivery is an action: not delivering is visible and recoverable
(the sender sees no recipients), while delivering to the wrong agent is neither. The optional
field is additive, so an old client simply ignores it (wire rule 1).

This is also the first real caller of the evidence parser and the identity resolver.

Tests: 27 in groups, 8 for the publisher, 3 RPC fan-out cases updated to the new contract. The
`@cursor`-must-not-match-"text cursor blink" hazard is now excluded structurally instead of by a
per-agent predicate.

Verified non-vacuous by mutation: swapping the process/title ranks fails 2 publisher assertions,
and reverting groups to title matching fails 15 of 27. One earlier mutation silently failed to
apply after formatting reflowed the block — the file was checked before trusting the result.

* perf(runtime): reuse terminal title during summary build

* fix(orchestration): refuse title evidence when publishing identity for routing

Rebuilt on current main so this carries the hardened parser from #16148 and the corrected
resolver from #16157 (authority-scoped run keys, no order-dependent duplicate resolution).

Applies the resolver's new `minimumSource` floor at the publisher. What this publishes authorizes
an action — routing decides which real agent pane receives a message — so ranking title last is
not enough; the floor removes it from consideration entirely, and no amount of reordering by a
caller can bring it back.

The trade, stated because it is a real capability loss: a hook-less agent over SSH that Orca did
not launch, and whose foreground process the host cannot read, is no longer addressable by @agent.
Accepted because a message delivered into the wrong agent's prompt is unrecoverable while an
undelivered one is visible — the sender sees zero recipients. Whether real panes actually carry
launch/foreground evidence is the open question, and is what live validation must answer.

* fix(pty): preserve agent identity on daemon reattach

* fix(runtime): retire stale pane agent identity

* chore: normalize runtime types formatting

* fix(agent-status): identify a pane from its own hook, not from how it was started

Two defects, one cause: identity was inferred from the outside instead of read from the agent.

**Hook evidence was never plumbed in.** The publisher considered `process`, `launch` and `title`
and contained zero hook references — while the resolver ranks `live-hook` first. The top rung of
the ladder was never connected.

That made identity depend on Orca having launched the agent. Most agents are started by typing
`claude` or `codex` at a shell, which leaves no launch record. On macOS the foreground process
still names them, so the gap was invisible. On WSL the Windows host reads the foreground process
as `wsl.exe` — the distro wrapper, not the agent inside it — so those panes had no signal at all
and became unaddressable by `@agent`.

A hook is the agent reporting itself, so it survives both: no launch record needed, and no
dependency on reading a process across the WSL boundary.

**`launch` outranked `completed-hook`.** Ranking is now by TENSE rather than by how authoritative
a source sounds:

    present: live-hook > process
    past:    completed-hook > launch > sleeping-session > sibling > title

A launch record is an event, not a state — it stays true after the agent exits, which is why a
pane reused after closing its agent kept reading as the old one. A completed hook at least proves
the agent actually ran in that pane; a launch record only proves Orca tried to start one.

Neither rank was covered: all 392 existing tests passed unchanged after reordering. Mutation now
fails 2 on the old order and 4 with hook evidence removed.

Known remaining gap, deliberately not papered over: a hand-started WSL agent with no managed hooks
has no identity signal at all. Restoring a title guess there would reinstate the misdelivery this
PR exists to prevent.

* fix(orchestration): restore title as the last resort, not a forbidden source

An earlier revision passed `minimumSource: 'launch'` so routing could not see a title at any rank,
reasoning that a display string must never authorize a write. That conflated the evidence parser
with the raw substring match it replaced.

`buildAgentNameRe('claude').test(title)` was the misdelivery. `collectAgentTitleEvidence` returns
null on exactly those shapes: "Review the Claude session-history fix" on a Codex pane yields
nothing, and "Switch Claude and Codex off the load balancer… - grok" yields grok from its owner
suffix. Ranking title last is therefore sufficient; refusing it is not necessary.

Refusing it had a real cost. An agent a user starts by hand inside an Orca WSL terminal has no
launch record, no readable foreground process (the Windows host sees `wsl.exe`, not the agent in
the distro), and — until managed Codex hooks install there — no hook either. An unambiguous title
was the only thing left, and dropping it made that pane unaddressable by @agent where the previous
code could reach it. That is a regression, and most agents are started that way.

End-to-end coverage added at the routing layer with title allowed: @claude still does not reach a
Codex pane whose task text names Claude, @codex still does not reach a Grok pane whose task text
names Codex, and a pane identified only by an unambiguous title is reachable again.

* revert(agent-status): keep launch above completed-hook until run keys exist

Reverts the tense-based reorder from this branch. The reasoning behind it was sound as far as it
went — a launch record is a past event, not an observation, which is why a reused pane kept reading
as its previous agent — but it fixed one staleness by opening a worse one.

A completed hook is past tense too, and without an agent-run key it never expires at all. Ranking
it above `launch` lets a stale hook from a previous agent outrank the launch record Orca stamped
for the process running NOW. pane-agent-owner.ts already says this in its own comment: "Ranking
launch/live-hook above the completed/sleeping records keeps a genuine pane on its real agent and
stops a stale record from hijacking it."

The reorder belongs with authority-scoped run generation, which is what makes any past-tense
evidence expire. It is staged in the migration plan rather than shipped here.

What this branch keeps: hook evidence feeding pane identity (so an agent a user starts by hand is
identified from its own report rather than needing a launch record), and title restored as a
genuine last resort behind the evidence parser.

* fix(runtime): guard the pane key so terminal.list survives a non-UUID leaf

`makePaneKey` throws on a leaf id that is not a UUID. The hook-evidence lookup called it unguarded
inside `buildTerminalSummary`, so a single such leaf took down `terminal.list` for the whole list
rather than degrading that one pane — 136 tests across 5 files, and the native code-quality gate
tripped separately on a duplicate test title.

Both were mine, and both were caught by CI rather than by me: I ran the focused suites before
pushing instead of the affected directories.

* fix(runtime): declare published terminal agent identity

* fix(runtime): demote completed hook identity evidence
2026-08-27 15:53:05 -07:00
Jinjing 3bc13f7b8c Split monolithic PTY IPC module into organized submodules (#15172)
* rm unused files

* remove unused files

* Refactor PTY IPC and add host environment paths

- Split PTY handlers out of inline baseline checks
- Rename local PTY shell provider for clarity
- Pass userDataPath and resourcesPath to host environment

* Establish PTY daemon identity before first await in spawn flow

Move identity setup, session ID minting, and hidden delivery state to
the beginning of preflight, ensuring these complete synchronously
before any awaited operations. Defer async operations like folder
workspace validation; add liveness tracking for SSH provider failures.
Refactor pane spawn reservation to prevent concurrent spawns from
creating duplicate providers.

* Add incarnationId tracking throughout PTY exit lifecycle

Track PTY incarnation IDs in exit messages sent to renderer, and add cause tracking for exit events. This enables proper lifecycle state management when PTYs can be respawned or have multiple concurrent instances. Also adds deadline support to process listing operations and stop-request tracking for better shutdown observability.

* Use fake timers in SFTP namespace tests for deterministic abort handling

Tests now use `vi.useFakeTimers()` to control time during abort scenarios,
advancing timers explicitly instead of waiting on real async delays. Ensures
more reliable test execution without flakiness from timing-dependent behavior.

* Fix PTY spawn lifecycle: handle concurrent races and cleanup abandoned a

Properly release Agent Teams leader handles when spawns are abandoned or fail,
restore provisional PTY sizes on reattachment, and settle concurrent spawn races
for the same pane. Add validation guards for destroyed renderers and improve
handler re-registration to reset delivery state before bridging a new window.

* Move PTY cleanup to localized error boundaries

Restore provisional PTY size when build-options fails and guard pre-allocated handle registration. This ensures cleanup happens at the point of error, not deferred to the general catch block.

* Replace Promise.resolve() with vi.waitFor in PTY claim test

Wait explicitly for the providerSpawn call to be made using vi.waitFor()
instead of relying on event-loop yielding. This makes the test more
deterministic and reduces flakiness from timing assumptions.

* Redact PTY IDs in pending data drop diagnostics

Prevent workspace paths embedded in session IDs from leaking through
diagnostic logs by using redactPtyIdForDiagnostics.

* Mark PTY exit events as observed by provider

Exit handlers now receive `providerExitObserved: true` to
distinguish definitive provider-witnessed exits from inferred
state changes. Preserves optional exit cause when present.

* Add defensive input validation to PTY IPC handlers

Validate that IPC arguments are present and the correct type before
passing them to handler logic. Uses optional chaining and type checks
to safely handle malformed requests from the renderer process.

* Replace direct Electron imports with PTY host bindings

Abstract app, ipcMain, and powerMonitor access through getter functions
to support multiple host environments and improve testability.

* Defend against transient PTY setup failures with state cleanup

Host-env setup failures now trigger cleanup of runtime-allocated PTY state. Cached PTY geometry is preserved after transient reattach failures but cleared when the provider reports the PTY exited before the spawn reply—preventing stale geometry from corrupting future operations. Error handling now distinguishes expired SSH sessions and early-exit conditions to preserve geometry appropriately.
2026-08-24 15:30:16 -07:00