mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 08:02:28 +00:00
1da1bdc01c3cc25218cd0a47687dcdee210dfa5e
221
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Jinwoo HongandE2E Test
|
757fae28d7 |
feat(skills): add private bundle sharing (#14401)
Co-authored-by: E2E Test <e2e@test.local> |
||
 Neil
|
eb22e497bb | Revert "fix(ssh): reapply the reattach-identity work and stop the fallback fence stranding moved panes" (#14395) | ||
|
Neil
|
6a0c8fa541 |
fix(ssh): reapply the reattach-identity work and stop the fallback fence stranding moved panes (#14384)
* Reapply #13326 and #13928 (un-revert #14361) Restores the SSH reattach-identity and daemon-occupancy fixes. Reverting them reintroduced their P0s, filed as STA-4224, STA-4225, STA-4227, STA-4230, STA-4232, STA-4233 and STA-4234 against #14361. The tab loss that motivated the revert is fixed in the commits that follow, so this reapplication is not a straight redo. * fix(relay): stop the fallback attach fence refusing a pane that moved tabs The primary fence was moved to the shell's own incarnation precisely because paneKey/tabId froze the pane's LOCATION at spawn and refused panes that had merely moved. The fallback that older clients fall into kept the old rule, so the correction never reached it — the same 'the rule exists, but this path does not ask it' leak this work has hit repeatedly. A refusal here is not recoverable: an identity mismatch never grounds a respawn, so the pane keeps a live shell it can no longer reach and renders blank. Narrowed to paneKey, which is the identity; the tab is a location. Restoring the tabId comparison reddens the new test. |
||
|
Jinwoo Hong
|
ff8dda81e8 |
fix(serve): exit cleanly after headless Linux signals (#14334)
* fix(serve): keep owned Xvfb alive through Electron teardown * test(serve): gate packaged signal shutdown * test: harden headless shutdown lifecycle gate * fix(serve): isolate Xvfb from foreground signals * docs(serve): preserve Xvfb during systemd stop * test(serve): pin shutdown policy to owned Xvfb unit * test(serve): harden shutdown gate portability * test(serve): bound systemd unit parsing |
||
|
Neil
|
11cd2b4310 |
revert(ssh): back out #13326 and #13928 — reconnect loses every tab (#14361)
* Revert "fix(daemon): stop killing live coding agents when the daemon can't report its sessions (#13928)" This reverts commit |
||
 NeilandOrca
|
3ab8b6a117 |
fix(ssh): stop SSH reconnect from multiplying terminals and resuming agents twice (STA-3077) (#13326)
* fix(ssh): stop reconnect from grafting panes and stacking remote leases Reconnecting an SSH-backed workspace added terminal panes the user never opened, and the remote host accumulated shells nobody was using — one report went from 2 to 19 to 20 relay PTYs across three reconnects (STA-3077). Two root causes, both in the store. Reattach could create UI. `persistPtyBinding` has four creating branches — mint a tab, mint a root leaf, split the root and graft a leaf, mint a layout. All four are load-bearing for `pty:spawn`, which can beat the renderer's debounced layout writer, but none of them is appropriate on reattach, where the pane either already exists or is gone for good. Add `mayCreate`, defaulting true so the spawn path is untouched; every creating branch already sets `terminalMembershipChanged`, so refusing is a check rather than a new code path. Lease identity had no pane key. `upsertSshRemotePtyLease` matched on `(targetId, ptyId)` alone, so a pane that re-leased under a new relay id left its predecessor live with nothing to retire it, and the next reattach fanned out over both. One pane now keeps at most one live lease. Superseded leases are marked `expired` rather than terminated: losing a lease is not proof the shell died, so the remote process is deliberately left running. Tests assert observable behavior rather than mechanism, so they stay valid under any implementation that fixes this. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record the terminal session behavior contract Properties stated as observable behavior rather than mechanism, so an oracle written against them survives a change of implementation. Records the weaker, correct form of the timer rule — a timer may never be the sole cause of a destructive action — because recovery budgets and scratch-file age gates are correct code that an absolute ban would condemn. Also notes which mechanisms are deliberately not required, so each has to earn its place rather than arrive with an architecture. Co-authored-by: Orca <help@stably.ai> * fix(ssh): heal duplicate pane leases that predate pane-keyed supersession Pane-keyed supersession stops new duplicates, but it does nothing for installs that already carry the ones STA-3077 accumulated — the report behind this reached 20 live leases across a handful of panes, and every reconnect fanned out over all of them. Retire the stale duplicates once per reattach pass, keeping the newest lease for each pane under a total order so two hosts resolve a tie the same way. As with supersession, retired leases are marked `expired` rather than terminated: their remote shells are deliberately left running, because a lease we chose not to revive is not evidence the shell died. The relay-session store stubs gain the new method. Note the gap this leaves open: those shells keep running and are no longer reachable from the app, so the "accumulates unused shells" half of the report needs a visible recovery surface rather than a silent kill. Co-authored-by: Orca <help@stably.ai> * fix(terminal): stop respawning a shell that is still running A pane that failed to reattach spawned a fresh shell. Because the restored session id came along, the replacement resumed the same agent session, and two processes appended to one transcript — reported repeatedly, up to five concurrent resumes of a single session. Two defects fed it. The relay reported a source that merely needed re-establishing as `SSH_SESSION_EXPIRED`. The shell was still running; only its output source was gone. Give that outcome its own error so it stops reading as "the session no longer exists". The reattach failure handler then treated every error as proof of death. It checked for expiry and, in the else branch, took the identical action — so the check bought nothing and a transport fault, a timed-out call, or a wedged relay all respawned. Respawn now requires proof: an explicit host expiry or a not-found PTY. Anything else, including an error we have never seen before, is unresolved, leaves the shell running, and keeps the binding for a later reattach. Two existing tests asserted the old behavior. One threw a bare error as scaffolding to reach the spawn-adoption door; it now throws proof, which is what it meant. The other pinned the expiry mapping itself, and now asserts the outcome fails closed *without* being reported as expiry. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record what makes a retention bound safe Shortening a grace period is the wrong lever. Measuring process time and gating reclamation on an independent observation are what make one safe, and they are what deployed systems actually do. Also records that lifecycle belongs in the attach reply rather than a delivered event — that is what removes the need for a durable per-consumer cursor to guarantee an exit is never lost. Co-authored-by: Orca <help@stably.ai> * test(terminal): assert the empty-failure case without an empty Error A thrown empty value exercises the same property — a failure carrying no usable message is not proof the session is gone — and does not trip the empty-error-message lint. Co-authored-by: Orca <help@stably.ai> * fix(ssh): let the durable pane binding outrank recency when retiring leases Choosing the newest lease for a pane is wrong whenever a newer lease exists that no pane is bound to: it retires the lease the pane is actually attached to, detaching a live terminal instead of healing it. Two changes. Arbitration now prefers the lease matching the pane's durable binding, across both the SSH-target and local partitions, falling back to recency only when no binding names either candidate. And supersession at upsert time now defers rather than expiring a bound predecessor. When a lease arrives for a pane that is still bound to a different PTY, the binding has not caught up yet, so both stay live and reattach arbitrates once the binding is available. Co-authored-by: Orca <help@stably.ai> * fix(ssh): roll back a lease retirement whose durable write fails `flush()` logs and swallows write errors, so a failed write left these leases retired in memory while disk still called them attached — and the pane bindings scrubbed alongside them stayed scrubbed. Use `flushOrThrow` and restore both the lease states and the affected session partitions when it throws, reporting nothing retired. Co-authored-by: Orca <help@stably.ai> * test(ssh): prove pane and remote PTY cardinality across reconnects Counts the shells the relay actually hosts, on the container, rather than inferring them from app state — that is the census the report was based on. Asserts the PIDs are unchanged, not merely the count, so a kill-and-respawn cannot pass. Every pane streams before the transport is severed: an idle pane sends no recovery checkpoint, so only a live source comes back needing re-establishment, which is the outcome that used to read as expiry. Co-authored-by: Orca <help@stably.ai> * fix(ssh): actually pass mayCreate:false from the reattach binding write The `mayCreate` guard was correct and had no production caller, so the reattach path still went through the creating branches and grafted panes back. `restoreReattachedPtyRuntime` is that call site — RC3 in the original diagnosis — and it now refuses to create. Binding moves ahead of runtime registration, because registering first would surface a pane the user never opened before the refusal landed. A refusal leaves the remote shell running and reattachable; a *thrown* write stays unknown and still registers, so a failed disk write cannot detach a live pane. Adds an oracle over the call site itself. The store-level tests all passed while the fix was inert, because they called the store directly — only pinning the wiring catches that. Co-authored-by: Orca <help@stably.ai> * fix(terminal): apply the respawn-requires-proof rule to both reattach paths connectPanePty has two near-verbatim reattach blocks — one keyed on the deferred SSH session, one on the restored session — and only the second was fixed. The first still checked for expiry and then respawned unconditionally anyway, so a transport fault there resumed the same agent session a second time. Also keep the wire token out of the pane. The main-process bridge only special-cases expiry, so a source-restore failure crossed IPC as raw `SSH_SOURCE_RESTORE_REQUIRED: <id>` text and surfaced to the user. It correctly does not respawn; it just should not read like that. Co-authored-by: Orca <help@stably.ai> * test(ssh): state plainly that the reconnect spec is a forward guard It was run against an unfixed tree and passed, so it does not prove the STA-3077 fixes and should not be read as if it does. A clean severed transport does not reproduce the field conditions — accumulated duplicate leases, or a source returning needing re-establishment. It keeps its place as a forward guard: it counts the shells the relay actually hosts and pins their PIDs, so a later change that grafts a pane or respawns a shell fails here. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record that a guard must be pinned at its call site A refusal that exists and is never passed is indistinguishable from no refusal, and store-level tests cannot tell the difference — they call the store directly. Learned from `mayCreate`, which was correct and had no production caller for several commits. Co-authored-by: Orca <help@stably.ai> * fix(ssh): park one PTY's exhausted delivery recovery instead of dropping the channel A per-PTY recovery budget running out disposed the whole relay channel, so one PTY that could not re-prove its delivery aborted every in-flight filesystem and git request on that host and stalled every sibling pane. A retry count is not proof of anything, and it certainly is not proof about the other sessions sharing the channel. Exhaustion now parks that PTY's delivery. The remote shell keeps running, its lease stands, and the next relay open reattaches it with a fresh delivery generation — the parked state is cleared on teardown and the generation changes on reconnect, so a reconnect recovers it. The consecutive-attempt ceiling goes away entirely; the per-generation one is what bounds the retry cost, and the second ceiling only existed to reach the channel drop sooner. Tradeoff worth stating: the failing pane used to self-heal within seconds because the forced reconnect wiped all rejection state, and it now stays frozen until the next relay open. That is a worse outcome for that one pane and a much better one for every other session on the host, and reconnecting is user-reachable. Co-authored-by: Orca <help@stably.ai> * fix(pty): let liveness say unknown instead of forcing it to say dead `IPtyProvider.hasPty` returned a boolean, so a provider whose inventory was empty for reasons that have nothing to do with the session — socket down, cache never hydrated, provider generation just constructed — had no way to say so and answered "absent". Its own siblings already knew better: `probePtyLiveness` and the runtime's `PtyController.hasPty` were both already `boolean | null`, with consumers branching on null correctly. The lie was injected at exactly one interface. Now three-valued, and each provider answers unknown where it cannot prove absence: the daemon adapter off-socket, the SSH provider before a completed listing, the router when any adapter cannot answer, and the degraded provider rather than fabricating a verdict. `terminal_gone` requires unanimous proven absence. Also fixes a real cold-start bug this surfaced: `pty:hasPty` never awaited the daemon-swap startup promise, though the sibling `probePtyLiveness` bridge already did, so before the swap the local provider answered an authoritative false for every daemon-owned id. Net +27 production lines. The plan behind this predicted -92 on the strength of deleting the renderer's dead-session reconcile path; that code is live (`pty-connection.ts` imports it), so nothing was deleted. Expressing a third value where there were two costs lines, and a deletion that is not real is not worth manufacturing. Co-authored-by: Orca <help@stably.ai> * docs(terminal): track the terminal-session correctness handoff package The package was untracked under a gitignored `docs/**`, with the un-ignore rules living only in an uncommitted .gitignore edit — a single `git clean -xdf` would have destroyed the authoritative plan. The 814-path construction snapshot is now pushed as `nwparker/react185-authority-snapshot` too; it had no remote ref. Co-authored-by: Orca <help@stably.ai> * test(ssh): make the reconnect settle window actually wait The settle poll reused a matcher the assertion 15 lines above had already satisfied, and Playwright's poll engine probes immediately and returns as soon as the matcher passes — so it observed the same state twice and elapsed 0ms. A shell grafted a second or two after reattach reported ready slipped through into the next cycle. Reviewer was right on #13111. Test-only; no production change. Co-authored-by: Orca <help@stably.ai> * test(ssh): census both durable session partitions on reconnect Adds a second reconnect scenario and a helper that reads pane records from the local partition as well as the ssh host partition. That split matters: the reattach binding call passes no hostId, so a grafted pane lands in the LOCAL partition and an oracle reading only the host partition passes whether or not the guard is present. Both tests remain forward guards. The second one was reported as discriminating and did not reproduce: with `mayCreate: false` removed from the call site and the app rebuilt, both still passed. Its induction races `pty:kill` against a severed transport, so when the kill lands the lease is cleaned up and there is nothing left to graft. The handoff README is corrected to say so rather than claim a journey. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record the user decision relaxing G6 G6 becomes minimise-and-justify rather than strictly net-negative. The deletion budget the plan assumed does not exist: an entrypoint-rooted import graph found 51 of 53 candidate files reachable and instantiated on live paths, leaving 263 deletable LOC against roughly +1,021 to offset. Correctness may still not be traded for line count. Co-authored-by: Orca <help@stably.ai> * test(terminal): add discriminating oracles for restart, daemon, skew and namespaces Six parallel streams, each required to fail with its guard removed rather than merely pass. Local restart proves the OS process itself survives, by reading `ps -o lstart=` for the shell's own pid. That matters: with the quit path made destructive, the tab, leaf and pty ids all came back byte-identical while the shell underneath was a new process — every existing restart spec would have stayed green. Two separate guards were removed to redden it, and the second reddens only the stale-operation case. Daemon restart discriminates by reverting three-valued `hasPty`; version skew now covers publication semantics and confirms the new `SSH_SOURCE_RESTORE_REQUIRED` token mutates nothing on an old client; two-host isolation censuses both containers. Deletes `src/relay/pty-source-replay-index.ts` — 201 production lines with no importer outside its own test, verified against an entrypoint-rooted import graph rather than a name grep. Five namespace tests are skipped, not passing: they reproduce a defect still live on main where folder-workspace ids compare equal with the instance suffix stripped. PR #12474 fixes it; they are its oracle. Co-authored-by: Orca <help@stably.ai> * test(ssh): induce the reattach graft deterministically instead of racing a kill The previous induction closed a pane while the transport was severed and relied on `pty:kill` FAILING so the lease outlived the pane record. It does not fail: with the provider already torn down, `pty:kill` takes its tombstone branch and marks the lease terminated, and `reattachKnownPtys` filters terminated leases out of the fan-out — so the reconnect never visited the PTY the test was about. It passed on both trees. Seed the precondition instead. Spawn a real remote PTY on a leaf that never becomes a pane, then roll the host partition back to its pre-spawn snapshot, leaving a live lease and a live remote shell that no durable pane owns. No failure races a success. Adds a vacuity guard that is independent of the tree under test: the lease's own `lastAttachedAt` must advance, proving the fan-out actually visited this lease before the pane census is trusted. Verified on this machine under an isolated TMPDIR, since the e2e harness keys its seeded-repo pointer on a machine-global tmpdir path: guard present passes, guard removed fails with the phantom leaf grafted into the local partition, guard restored passes. Co-authored-by: Orca <help@stably.ai> * docs(terminal): propose one authoritative binding identity Every defect this program has touched is the same defect: identity compared with the wrong key, or not compared at all. Lease keyed without the pane, reattach using a creating write, folder-workspace ids compared with the instance suffix stripped, local mutating IPC carrying only an id, a live shell classified as expired, liveness unable to say unknown. Proposal: one branded binding type built from fields that already exist and are already persisted, constructible only from an authoritative source, carried by mutating operations, compared by one shared function. Makes a wrong-key comparison a type error rather than the next incident. Under adversarial review, including against the open issue corpus. Not accepted. Co-authored-by: Orca <help@stably.ai> * fix(pty): refuse mutating operations aimed at a superseded PTY `pty:write`, `pty:writeAccepted` and `pty:resize` accepted any id. The renderer queues input, so a keystroke buffered before a reattach landed on whatever PTY had since taken the pane — and a resize reshaped the successor's shell. Main already tracks `ptyPaneKey` and `paneKeyPtyId` in lock-step, so their disagreement is proof the caller's id was superseded. No wire change, no renderer change, nothing added to the input payload. An id with no recorded pane stays permitted: unowned and orphaned PTYs are unknown, not stale, and unknown never authorizes refusing an explicit operation. That is also what keeps orphan cleanup working — those ids have no pane by construction. The tests pin the CALL SITES, not the predicate. A capability that exists and is never called is indistinguishable from no capability, which is exactly how `mayCreate` sat inert here for several commits with every test green. Co-authored-by: Orca <help@stably.ai> * fix(pty): fence signals at a superseded PTY, and pin why kill is exempt A signal means "interrupt my pane", so delivering one to a PTY the pane has already replaced is a misdirected interrupt. Fence it with the same lock-step proof used for write and resize. `pty:kill` stays deliberately unfenced and a test now pins that: a superseded PTY is orphaned, and reclaiming it is exactly what the orphan-cleanup callers ask for. Refusing there would break the operation that reclaims leaked shells — the opposite of the intent. The fence sits at the IPC boundary, above `tryGetProviderForPty`, so it covers local, daemon and SSH rather than the local path alone. Co-authored-by: Orca <help@stably.ai> * test(terminal): poll the pane binding read so a slower host cannot flake it `readPaneBinding` took a single unpolled read of a DOM dataset attribute immediately after a renderer reload, while its sibling helper polls the same data for 15s. On a native Linux host both tests failed every run with 'No bound terminal pane is mounted' while the app was demonstrably healthy — the screenshot showed the terminal restored with a live prompt and the boot PID echoed. The assertion is unchanged; it is only awaited. Nothing is weakened. Found by running this spec on native Linux rather than assuming macOS behaviour generalises. Co-authored-by: Orca <help@stably.ai> * test(terminal): make the restart identity spec run on Windows too Both probes were POSIX-only and unconditional: `echo ...=\$\$` for the shell's own pid, and `ps -o lstart=` for its start time. Running the spec on a real Windows host proved it dies before reaching either guard, so Journey 1's Windows half was unprovable rather than merely unproven. PowerShell exposes the same two facts as `$PID` and `Get-Process` StartTime. The start time still matters on both platforms for the same reason: a PID alone cannot separate a survivor from a reused number. Still green on macOS. The Windows path is written from the host probe and has not itself been executed end to end — that is the next thing to run there, not a claim being made here. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record the fence's real gap and what peer designs taught Marks the client-constructed binding proposal as rejected with the three false claims that sank it, and records what shipped instead. States the shipped fence's actual limitation rather than leaving it implied: it compares a binding, not an incarnation, so a respawn under a reused ptyId passes. The obvious remedy is wrong here — the agent-create id is deterministic by design so a replayed create stays idempotent, and randomising it would trade this narrow gap for a duplicate-spawn bug. Also records the ranked lessons from four comparable agent IDEs, chiefly that a typed end-reason at end time is what stops a user quit from looking like a resume candidate. Co-authored-by: Orca <help@stably.ai> * docs(terminal): promote Journey 1 to proven on all three platforms The oracle now runs natively on macOS, Linux and Windows, and its discrimination was watched on each: a mutation reddens it, a restore greens it. On Linux and Windows both mutations were run, and the second reddens only the stale-operation test — so the journey's two clauses are proved independently rather than jointly. Windows is the new evidence. The PowerShell branches added blind at ebffb85a848 executed correctly on their first run: `$PID` expanded to real integers, which also proves the pane shell there is PowerShell-family rather than Git Bash, and `Get-Process StartTime` returned kernel start times 5.4s apart — so a recycled pid could not have passed as a survivor. First journey promoted in this program. The other twelve are unchanged, and the residual limit on "every stale exact operation" is recorded rather than glossed. Co-authored-by: Orca <help@stably.ai> * test(terminal): add discriminating oracles for the daemon, skew and multi-host journeys Daemon: replaces a spec that modelled only a client restart and never crossed the daemon boundary, whose successor generation owned nothing so "the live successor is neither killed nor replaced" was vacuous. The PTY leader is now a real login shell reporting `$$` back through the production write path, resolved to a kernel start time. Two mutations each redden exactly one of the three clauses, on macOS and Linux: reverting three-valued `hasPty` reddens only the unknown-not-dead clause; widening the sole-provider fallback reddens only the stale generation clause. Skew: reverting the restore-required publication to expiry reddens 4 of 5 new tests while the legacy control stays green — the regression this branch fixed is now caught if reintroduced. Multi-host: restoring `mux.dispose('connection_lost')` reddens sibling isolation on one host. It does NOT redden across hosts, and that is recorded rather than glossed: a mux belongs to one relay session per target, so its dispose cannot cross a host boundary. Journey 4's cross-host clause rests on isolation-by-construction, not on a mutation. No production code changes. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record journey evidence that falls short of promotion Four journeys now have discriminating oracles but none meets its full stated scope, and each shortfall is named rather than rounded up. Journey 2 is one WSL run from promotion. Journey 12's tests are in-process, so they do not close the live-skew gap the original ledger named. Journey 4's cross-host clause cannot be proven by mutation at all — a mux is per target, so its dispose cannot cross hosts, and the cross-host test stayed green under the mutation that reddens siblings. Journey 13 measured one dimension of ten, on lifted predicates rather than through real IPC. Co-authored-by: Orca <help@stably.ai> * docs(terminal): promote Journey 2 to proven on macOS, Linux and physical WSL The oracle runs on every environment the journey names, and is clause-selective on all three: reverting three-valued `hasPty` reddens only the unknown-not-dead clause, and widening the sole-provider fallback reddens only the stale-generation clause. Selectivity in WSL was established rather than assumed. The spec runs serially, so a red first test reports the others as "did not run" — they were re-run alone under the same mutation and stayed green. Also records that an Orca WSL-mode terminal now starts on that host at all, which it could not before: the distro had no provisioned default Unix user, so every interactive launch blocked on first-run setup. One diagnosis from the WSL run is corrected here rather than repeated: the unrelated `local-pty-shell-ready` failure was attributed to bash 5.3.9, but macOS runs the same bash version and passes 67/67. The trigger is environmental to that distro, and the underlying defect is that the spec pins an absolute count of OSC markers it does not own. Co-authored-by: Orca <help@stably.ai> * docs(terminal): correct the WSL provider-suite diagnosis The WSL run blamed bash 5.3.9 for the unrelated `local-pty-shell-ready` failure. macOS runs the same bash version and passes 67/67, so the version is not the cause — the trigger is environmental to that distro, and the underlying defect is that the spec asserts an absolute count of OSC markers it does not own. Co-authored-by: Orca <help@stably.ai> * test(runtime): unskip the workspace-namespace oracles now their fix has merged These five reproduced a defect that was live on main: folder-workspace ids were compared with the instance suffix stripped, so two workspaces sharing a directory read as the same namespace. They were committed skipped, pointing at the PR that fixes it. That PR is merged, and they pass. Verified they still bite: restoring the suffix-stripping comparison reddens exactly these five and leaves the other four green. An oracle written before its fix, held skipped, and confirmed against the fix after the merge — rather than deleted and rewritten from the answer. Co-authored-by: Orca <help@stably.ai> * test(ssh): add MaxSessions, lazy-discovery and paired-skew oracles Three journeys attempted; none promoted, and the reasons are recorded in the ledger rather than rounded up. MaxSessions=1 against real OpenSSH, with the cap read back from `sshd -T` rather than assumed, and remote pids read on the container two independent ways that must agree, each carrying its kernel start time. Two disjoint mutations discriminate — one reddens only the reconnect clause, the other only the two restart clauses. But the disconnect clause is a forward guard: four separate guard removals left it green, so nothing shipped is load-bearing for it. Lazy discovery samples sshd's own accept log and live session census across a 22s window with the in-use host as a positive control. No mutation reddens its third clause alone — the real cross-host lease scoping is load-bearing, but removing it breaks the sibling host during setup, so the failure carries no clause information. The paired-runtime skew spec pairs two real processes at different versions and refuses to run rather than degrade into a same-version pairing that would look green and prove nothing. No production code changes. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record why the duplicate-resume fix was not built I recommended adding a typed end-reason so a user quit stops looking like a resume candidate, then went to implement it and stopped. `SleepingAgentSessionRecord` already carries three fields that each exist to stop something resuming that should not have — `origin`, `restoreOnTabOpenOnly`, and `automaticResumeBlockedBy` — each traceable to its own incident, consulted at 22 non-test sites. A fourth predicate, however well typed, is the fifth containment cycle. The designs without this bug do not have a better flag; they resume only on an explicit action, into a new terminal id, and make two agents in one terminal unrepresentable in the schema. The first of those is a product decision about whether automatic resume stays a feature, so it is the user's call rather than mine. Co-authored-by: Orca <help@stably.ai> * docs(terminal): reconcile G6 with the recorded decision and assess its clauses G6's body still demanded strictly-negative production LOC after the user relaxed it to minimise-and-justify, so the gate had two conflicting pass conditions and no single truth value. Its body now points at that decision. Assessed the remaining clauses against the branch rather than assuming. Two fail structurally: more than one identity comparison and mutation admission path still exist, and `terminal-input-quarantine.ts` is still reachable from two production files. Records why the quarantine is not subsumed by the superseded-PTY fence, which I had assumed and checked. The fence refuses writes aimed at a stale ptyId; the quarantine guards the user's next keystrokes landing on the successor under its current, correct id — a case the fence never sees. Removing it needs the recovery path to surface a different shell as unresolved, not a deletion. Co-authored-by: Orca <help@stably.ai> * docs(terminal): the input quarantine is load-bearing, not superseded G6 lists "no superseded quarantine remains reachable" and this module was assumed to be one. Disabling its single call site reproduces the hazard it exists for — `cho hi; rm -rf x` reaching the shell — so deleting it without a replacement re-opens command execution. The replacement was costed by building it rather than estimated: +26 production LOC to thread the incarnation, ~+33 complete, and the cross-remount state it needs outlives the destroyed pane so it becomes a module about the size of the one deleted. Floor is roughly +140 to delete 88, and it would add a second identity comparison to a gate already failing for having more than one. The decisive part is that the route is not uniformly available: remote runtime results carry no incarnation, old hosts cannot be made to publish one, and mixed versions are the normal state. A paired client reads unknown, which this program's own rule says is not proof — so either every remote reattach surfaces unresolved, or a fallback is needed and the only correct fallback is this module. Whether to amend the clause or accept something weaker on remote hosts is a user decision, so the clause verdict is left as failing rather than quietly reclassified. Co-authored-by: Orca <help@stably.ai> * refactor(runtime): collapse duplicate identity comparisons G6 requires one identity comparison; five implementations existed across two concepts. Worktree-namespace identity had two: `runtimeWorktreeIdsEqual` and `runtimeWorktreeIdentityKey` independently re-derived repoId plus normalized path. Equality now derives from the key, so the comparison and the sleep / mutation-queue keying cannot drift into two different rules — which is exactly how the suffix-stripping bug reached production once. Pane identity had three byte-identical leaf-UUID comparisons, in orchestration `db.ts`, `lifecycle-reconciliation.ts`, and `orchestration-legacy-process-identity.ts`. One copy moved to `stable-pane-id.ts`, which already owns `PaneKey`, `parsePaneKey` and `makePaneKey` and which all three already imported. No new module, no branded type, no parallel comparison. Net -14 production lines. The namespace oracle still bites: restoring the filesystem parser inside the identity key reddens exactly its five cases. The raw counts are not the actionable set, and the classification is worth recording: of 409 non-test `worktreeId` comparisons, 71 are typeof guards and 81 are sentinel tag checks. Most of the remainder are renderer predicates over store rows where both operands are the same main-minted id, so normalizing there would widen equality rather than correct it. Co-authored-by: Orca <help@stably.ai> * refactor(terminal): finish a half-done fixture move and audit the rest `xterm-bypass-event-fixture.ts` and `__fixtures__/xterm-bypass-event.ts` were byte-identical apart from an import path. The `__fixtures__` copy had zero importers and the live copy compiled as production — someone started the move and left both. Dead copy deleted, live one moved, its three test importers updated. Audited the wider G6 clause by importer rather than filename: 32 test-only files, roughly 3,300 LOC, currently compile as production; 4 of the 36 candidates have real production importers and are correctly placed. The list is recorded in the goalposts. Those 32 are almost all older than this program and outside the terminal surface, so sweeping them belongs in its own change rather than inside a terminal PR. The clause stays failing, with the remaining files named. Co-authored-by: Orca <help@stably.ai> * docs(terminal): the fixture clause already holds where it matters Checked what the build emits rather than reasoning from file paths. None of the 32 test-only fixtures appears in `out/` — Rollup drops them because no production entrypoint reaches them. On "compiles into the shipped product", this clause holds today. On the other reading it cannot be closed by moving files at all: both production tsconfigs use bare `include` globs with no `exclude`, so a `__tests__/` directory matches exactly like any other path, as does every `*.test.ts` in the repo. Relocating 32 fixtures would remove nothing from typecheck scope. A sweep was started and stopped once this was verified, rather than landing 32 moves across areas this program does not own for no gain. If the intent is that typecheck scope should exclude test code, that is a repo-wide tsconfig change with a different owner. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add plain-language design and test overviews Two reviewable documents with diagrams, written so someone with no prior context can follow what breaks, why, and what changed. The design overview explains the five things stacked behind one terminal rectangle, the 2 -> 19 -> 20 report, the three root causes, and the rule underneath all of them: unknown is not dead. The test overview explains why a green test proves nothing on its own, the four-step mutation proof we adopted, and — the part worth reviewing hardest — an honest account of what could not be proven and why, including the properties that are true by construction and therefore have no guard to remove. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add a self-contained visual report of the design and its evidence Pre-renders every diagram to inline SVG in both themes so the report opens offline and stays sharp when zoomed. States the gate/journey score and the retractions alongside the fixes, so the unproven half is as visible as the proven half. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record the finalized two-plane architecture decision Adopts the data-plane proposal and adds the control-plane track it does not cover: re-key ownership by pane, split orphan inventory out, then delete the compensating code. Records that the host-authority alternative was refuted and that the shipped keystroke fence is inert on the reattach path. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add the design brief the review counsel works from Separates verified code facts from unverified leads so reviewers attack the design rather than a reconstruction of it, and records which simpler alternatives were already refuted and why. Co-authored-by: Orca <help@stably.ai> * docs(terminal): report the design counsel's outcome and the live respawn bug it found Three review rounds across two models replaced the two-record split with one leaf-keyed record, deleted attach-time pane identity, and made orphans a connect-time projection. Records that a shipped gesture still turns a healthy remote shell into a duplicate agent resume, and that the renderer classifier in that chain treats an error-message shape as proof of death. Co-authored-by: Orca <help@stably.ai> * docs(terminal): correct the report — the respawn proof gate guards a minority path A final review traced every auto-respawn route. The primary one converts the reattach failure into a boolean before any classifier sees it, so the shipped proof gate never runs there. Records that two of the six shipped changes are narrower than claimed, and why their tests could not have caught it. Co-authored-by: Orca <help@stably.ai> * docs(terminal): explain the landed design on its own terms One leaf-keyed ownership record, orphans computed at connect, and replacement shells only on positive proof — with the shipping order and the one product trade the design asks the owner to accept. Co-authored-by: Orca <help@stably.ai> * docs(terminal): rewrite the design explainer in plain English The first version assumed the reader knew the codebase. Reframed around two bugs, two fixes and one decision, with the jargon replaced by pane / program / note / helper and a five-word glossary for what could not be avoided. Co-authored-by: Orca <help@stably.ai> * fix(ssh): stop reading an identity mismatch as a dead shell The relay reports a pane-identity mismatch by saying the pty was not found, but it found it — comparing identity is how it noticed. Publishing that as expiry made the renderer clear the binding and cold-restore with agent resume, so a live shell gained a second agent on one transcript. Reachable today by detaching a pane into a new tab, which changes the tab the relay froze at spawn. Mismatch now carries its own token and the classifier refuses it as proof. Genuine absence still expires, so a shell that really went away is not stranded. The three failure tokens move to src/shared: main published them and the renderer decided respawn on them, from two copies that had drifted apart. Co-authored-by: Orca <help@stably.ai> * fix(ssh): stop sending pane identity on reattach The relay froze pane identity at spawn, so moving a pane to another tab made it refuse a live shell — and refuse by saying 'not found'. The comparison is presence-guarded, so not sending the fields disarms it on every relay version including ones already installed on hosts: no wire change, no redeploy. Nothing is lost. It existed to catch a relay restart recycling pty-N for a new shell, and in exactly that case pane and tab both still match, so it accepted the wrong shell anyway. The incarnation the attach returns is what distinguishes those, and it already crosses the wire. Removes the whole client-side apparatus: the expected-identity type, its per-lease derivation, its map, and the parameter threaded through four layers. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add tracked goalposts for the new design Each goalpost is a behaviour with an oracle and the mutation that must redden it, so 'proven' cannot be claimed from a green test. Records the anti-inert rule as a first-class goalpost, since three guards in this program passed their tests while sitting off the route production takes. Co-authored-by: Orca <help@stably.ai> * docs(terminal): record that the recovery grant is dead code, deleting a design step The lease stores a relay-native pty id and the caller passes the app form, with a raw equality comparison between them, so the 30s grant cannot fire for a real SSH pane. The death rule that existed to referee it is deleted rather than built, and the dead path itself becomes a removal. Co-authored-by: Orca <help@stably.ai> * docs(terminal): keep the full design detail in the repo It only existed in an ephemeral job directory, so the plain-English explainer had no durable source for its specifics — record shape, death rule, reattach algorithm, migration order and the 25 oracles. Co-authored-by: Orca <help@stably.ai> * docs(terminal): add a resume prompt for a clean session Points at the goalposts as the contract, names the three goalposts whose oracles are already written and red, and carries the process rules that were learned the expensive way — prove guards reachable, verify mutations land, commit per step, and never let a subagent write production files in a shared worktree. Co-authored-by: Orca <help@stably.ai> * test(ssh): add the failing oracles for goalposts S3, S4 and S5 Intentionally RED: 14 clauses that fail against current behaviour and go green under the changes named in new-design-goalposts.md. The branch is held unmerged, so red here means unimplemented, not broken. Each was verified to fail for the right reason and to flip green under the identified fix, which was then reverted. Each pins the producer as well as the consumer, so no clause can pass vacuously if its route is ever severed — the failure mode that let three earlier guards ship inert. Co-authored-by: Orca <help@stably.ai> * fix(ssh): stop fabricating an exit when a reattach fails A failed attach never proves the shell exited. The relay answers not-found for a pane-identity mismatch and for any id it merely cannot hand back, so treating it as death sent the pane a synthetic `pty:exit { code: -1 }`, cleared provider state, deleted ownership and expired the lease — four claims about a process we know nothing about, on a shell that is usually still running. Collapse every failure into the non-destructive branch that already existed a few lines above (`restoreRequired = 'reattachAttemptsExhausted'` + wakeRecovery). A branch collapse, not a new mechanism: goalpost S3. Two tests pinned the deleted premise and are INVERTED rather than patched, so the new intent stays covered: - ssh-relay-orphan-abandon-paths: "retires the lease without a kill when the relay proves the PTY is gone" -> "leaves the shell running when the relay only reports the PTY as not found". Its comment claimed attach verifies liveness before answering not-found; it does not. - ssh-relay-session: "invalidates and broadcasts remote PTYs that cannot reattach" -> "leaves an unreattachable remote PTY alone while its sibling reattaches". Also repairs two clauses left red by |
||
|
Neil
|
501337454c |
perf(runtime): stop rescanning every repo every 30s with a Git-admin fingerprint (#14207)
The main-process worktree resolution cache expired on wall-clock time: the whole-fleet snapshot has a 1s TTL, so any poller faster than 1Hz recomputed it, and every 30s the per-repo scan cache expired and shelled out `git worktree list` for every registered repo. A production trace recorded 4,272 `git worktree` invocations over 3h27m across 10 repos. In-Orca mutations are already event-driven, so the 30s TTL existed only to discover changes made outside Orca. Before re-running an expired scan for a local, non-WSL repo, read a cheap subprocess-free Git-admin fingerprint (admin dir entries, per-checkout HEAD and its ref tip, gitdir/locked/existence per entry, packed-refs and reftable stamps). If it matches the fingerprint captured at the cached scan's start, extend the cache without spawning Git. A real scan still runs every 5 minutes so anything the probe cannot see still reconciles. Measured: 600 -> 60 `git worktree list` spawns on the reported workload (10 idle repos, 1Hz polling, 30 simulated minutes), and main-thread event-loop stall of 2.69ms -> 0.01ms per refresh. External worktree add/remove/move/lock/checkout/commit discovery stays bounded at 30s. SSH repos, WSL-routed repos, folder workspaces, agent-scratch repos, and any repo whose layout the probe cannot read keep today's behaviour exactly. Design: docs/reference/worktree-scan-fingerprint.md |
||
|
Neil
|
9c091cf77e |
chore(perf): add renderer agent-status benchmark harness (#13905)
Splits run-idle-cpu-benchmark.mjs into a scale fixture, an in-page timing probe, and process sampling, and records a measured origin/main baseline so the agent-status batching slice has an auditable before. The agent-status write workload is not included: it needs setAgentStatuses, so it lands with the store slice. |
||
 Brennan Benson
|
4dcb410982 | docs: remove restored shell validation report (#13759) | ||
|
Brennan Benson
|
7250fde148 |
fix(codex): repair hook trust before restored shells reconnect (#13745)
* fix(codex): reconcile hooks for restored shells * docs: move restored shell validation report * perf(codex): gate retained pane inventory |
||
|
NeilandOrca
|
17cfc968cf |
Revert the terminal IME composition-ownership change (#13282)
* Revert "test(ime): restore coverage the composition-ownership change removed (#13168)" This reverts commit |
||
|
NeilandOrca
|
17b3dff3c4 |
refactor(terminal): return IME composition ownership to xterm (#13128)
* fix(terminal): return IME composition ownership to xterm * fix(mobile): derive terminal input from native replacement ranges * test(mobile): record iOS Japanese IME traces * fix(mobile): preserve native IME replacement ranges * fix(xterm): flush queued application input after IME commit * test(terminal): pin Korean intermediate commit * test: pin Windows IME shortcut ownership * test: replay IBus number candidate commit * fix: preserve native macOS input-method punctuation * refactor(terminal): remove stale mac focus override * fix(mobile): preserve soft keyboard deletion ranges * fix: keep IME-owned palette chords in renderer * fix: stop carried IME shortcuts at renderer owner * fix: preserve carried IME shortcut dispatch * fix: narrow main-owned shortcut actions * test(mobile): pin Japanese IME replacement traces * test(terminal): retain paired native IME trace * fix(chat): preserve browser IME composition ownership * fix(chat): retain macOS IME confirm gesture * fix(chat): expire unmatched IME confirm carry * fix(chat): isolate IME confirmation expiry * fix(chat): retain active IME confirmation * refactor(terminal): remove dead composition handler * feat(ime): add shared Enter-ownership seams for CJK composition The confirming Enter of a CJK composition arrives as two keydowns and the orderings differ by platform: Windows/Linux redispatch the unmarked Enter/13 before keyup, macOS delivers keyup first. A guard reading only isComposing or keyCode 229 misses the redispatch, so surfaces submitted on a confirm. Adds useImeEnterGestureOwnership (carry token, next-frame expiry), a shared ImeEnterGuardedForm for native implicit submission, and the cmdk seam covering 18 CommandInput surfaces at one site. A chorded Enter arms the carry but is never swallowed — the reverse would eat a user's deliberate Cmd/Ctrl+Enter. Both failure modes are pinned by ime-enter-gesture-ownership-contract.test.ts. Co-authored-by: Orca <help@stably.ai> * refactor(terminal): consolidate native input listeners and parked-screen owner Extracts the shared native-input listener installer and renames the parked-screen detector for what it actually does, replacing per-call-site duplication. The listener installer keeps a forgetOptionKeyLocationOnBlur flag so per-window semantics are preserved rather than flattened. Net deletion; no behaviour change intended. Co-authored-by: Orca <help@stably.ai> * test(terminal): pin recorded IME shapes as regression tests Nine regression tests built from hashed affected-platform captures, each with a paired ordinary negative and a discriminating mutation verified to take the file from all-passing to exactly one failure. Covers the Windows MS-Korean Shift family (#12179, #11878, #12151, #11946, #12152) and the Korean TUI line-break rows (STA-3237, STA-3222, STA-3129). STA-3237 pins the empirical 3-Shift / 2-active-composition / 2-newline ratio the device run established — the third Shift produces nothing because Space has already committed. That ratio is not derivable from a static capture. Co-authored-by: Orca <help@stably.ai> * fix(ime): guard Enter-commit surfaces against CJK confirm Applies the Enter-ownership guards across the surfaces whose Enter commits something: publishes, clones, pairs, installs, posts, or persists. Tiered deliberately rather than uniformly. Irreversible and remote-effect sites take the carry token, which also blocks the unmarked redispatch. Locally reversible sites take the oracle check with a one-line comment naming the residual, because a spurious commit there costs one undo. Three numeric fields are left unguarded with the reason in-code: Chromium blanks number inputs at compositionstart, so a confirm-Enter only ever reaches an empty-draft reset. Measured with a CDP probe rather than assumed — a guard that cannot fire is noise. Co-authored-by: Orca <help@stably.ai> * test(ime): teeth-check the Enter guards on every guarded surface One suite per guarded surface, each verified by deleting the guard and confirming the test fails. A green guard test without that check is unverified, not verified. Two shapes pass vacuously in happy-dom and are avoided here: native implicit form submission never fires, and blur() is inert on an unfocused element. Both made "the commit did not happen" assertions pass with the guard removed, so the suites assert the guard's contract directly instead. Co-authored-by: Orca <help@stably.ai> * fix(mobile): keep iOS Korean commits whole through the live-input path iOS Korean reports isComposing: false on every event, so it bypasses the composition guard entirely. The strict owner rejected UIKit's transformed post-change field and sent only the leading jamo — the reported symptom. Prefers the authoritative same-event field text over the predicted text when the supplied operation cannot produce it. Generic: no Korean special-case, no locale classifier, no normalization. Adds the RN-target-keyed submit carry alongside it. Co-authored-by: Orca <help@stably.ai> * test(e2e): make IME capture harnesses fail loudly instead of silently Four instruments recorded silence as success, so a void run scored as a clean one: - readTerminalImeBoundaryTrace returned an empty trace when the probe never installed, making every "nothing leaked" negative pass vacuously - summarizeLatencies([]) returned a perfect zero distribution that passed all three latency thresholds - the macOS Vietnamese spec pinned an input-source ID that does not exist, and failed as though the operator had chosen the wrong source - the expectedLineCount=1 prefix property was undocumented and one edit from silently downgrading a PTY assertion Input sources now resolve by enumeration and name the near-matches on failure. Co-authored-by: Orca <help@stably.ai> * test(terminal): cover Cangjie cancellation and fix a cross-namespace assertion Adds #11951's recorded Cangjie cancel shape to the existing cancellation suite, which covered Pinyin and Sogou but not Cangjie. One keystroke then Backspace arriving as deleteContentBackward with data: null, so the stale preedit is the only thing a fallback could replay. Verified against the historical pre-6cd944c62b3 bundle: the positive fails with ['尸'] where [] is expected, while the ordinary negative stays green. Also fixes the Vietnamese spec, which asserted a TIS-space input-source ID against getKeyboardInputSourceId(). Those two Orca APIs report the same source in different namespaces — TIS nests it under VietnameseIM, the app API does not. The resolver stays as an installation precondition; the assertion matches the leaf. Co-authored-by: Orca <help@stably.ai> * test(e2e): add a real-IME macOS arm for the Korean chord commit The existing korean-ime-terminal-shift-enter-commit spec synthesizes composition over CDP: Input.imeSetComposition sets the preedit directly and Input.insertText performs the commit. Asserting the IME produced events you injected yourself is circular, so that spec cannot certify real-IME behaviour. This arm selects 2-Set Korean via TIS, reads it back live, and injects through System Events key codes, so the OS owns the preedit, the commit instant, and isComposing. PTY byte expectations are preserved verbatim. Covers 2 of the original 4 cases by design. The other two are the Windows/Linux redispatch-before-keyup ordering, which macOS cannot produce and which cannot be selected -- the OS decides it. Reintroducing synthesis to "restore coverage" would reintroduce the circularity. Co-authored-by: Orca <help@stably.ai> * test(e2e): assert the macOS chord arm at the PTY boundary, not the renderer The byte expectations were transcribed from korean-ime-terminal-shift-enter-commit :364/:383, which assert against onData -- a renderer boundary where the terminator is CR. This spec reads the PTY child, where the tty has already converted CR to LF. Names both forms per row rather than swapping the constant, so the conversion reads as evidence that the capture reached past the renderer, as #11936 and #11951 record. Ctrl+Enter's CSI-u sequence is unaffected and is identical at both boundaries. Co-authored-by: Orca <help@stably.ai> * test(e2e): measure composer-to-onData latency and stop dropping IME keystrokes Two defects in the echo latency probe. It hooked onWriteParsed and onRender but never onData, so it measured key->parse->render echo rather than the composer-vs-onData delta the latency rows need. Adds a third hook feeding its own sample set. And `event.key.length !== 1` silently dropped IME keystrokes: Pinyin and Cangjie keydowns arrive as key:'Process' (length 7). Replayed over the captured corpus, the old filter accepted 580 of 4137 Chinese IME keydowns -- it was discarding 80% of them. The new filter matches the shape the owner itself branches on. Attribution charges each onData to the latest keydown rather than a FIFO head, because composing jamo emit no onData at all and a queue would credit a whole composition to its first keystroke. The consumer now asserts sample count before any percentile, so a zero-sample run cannot render as a flawless distribution. Co-authored-by: Orca <help@stably.ai> * test(terminal): pin the WSL shifted-jamo newline shape for #11919 In Korean 2-set, Shift types ordinary letters -- the double consonants and the compound vowels. Each such keystroke reaches Chromium as key='Process', keyCode=229, shiftKey=true. The v1.4.163 classifier matched exactly that pattern with no code guard, so it called those keystrokes Enter, rewrote them to a synthetic Shift+Enter, and injected a newline into the middle of the word -- with no Enter key pressed. That is why the reporters said "no modifier key pressed": they had not chorded Shift+Enter, but they had pressed Shift, to type the double consonant. Asserts the row's own recorded capture: 40 immediate keydowns, exactly 3 of them Shift-carrying inside a single syllable, and an onData stream with one newline per Enter press and none mid-word. Two ordinary negatives keep it from being a blanket mute -- the same session's non-IME keydowns still reach shortcut policy, and an ordinary Shift+Enter still resolves through the real policy. Co-authored-by: Orca <help@stably.ai> * test(terminal): pin the composition commit lag that made Korean type one behind macOS Korean 2-Set commits syllable N only when the first jamo of N+1 arrives, so compositionend and compositionstart land in the same task. A composition-start handler cancelled the pending finalizer that was the only path to triggerDataEvent and ended the session without emitting bytes, so every committed syllable reached onData exactly one syllable late and the backlog cleared only at a Space or Enter. Types continuously with no Enter and no Space -- either would flush the backlog and hide it -- and samples onData at every syllable boundary. Paired with a length-matched ASCII arm that stays green throughout, so the positive is a fact about composition rather than about timing in general. Bisected to a single call site across five builds: pristine, 1.4.155 and 1.4.162 pass, 1.4.163 fails, removing the one call repairs it, restoring it fails identically. That window is exactly the reporter's "started immediately after updating". Co-authored-by: Orca <help@stably.ai> * test(mobile): cover the send-queue abort that silently drops queued keystrokes One failed send in use-terminal-live-input-commit aborts every keystroke queued behind it, with the error swallowed by .catch(() => false). The existing test resolves(true) on every send, so the failure branch was uncovered. Four arms: the abort itself, an ordinary negative on the healthy path, a throwing sender, and a liveness control proving the queue recovers once the chain settles. Deleting the abort takes 4 passed to 3 failed, with the ordinary negative correctly surviving. Scope is stated in the docblock: this is a transport send-queue abort, reachable only via a real disconnect or RPC error. REQUEST_TIMEOUT_MS is 30s, so latency alone cannot reach the branch — consistent with #7094's symptom class, not proven to be its cause. * test(terminal): pin that daemon snapshot/restore cannot disturb a composition Two independent reporters attributed broken Korean composition to the always-on PTY daemon repainting terminal state over the preedit. The attribution is wrong on ancestry — the daemon shipped three months before the version both call good — but the boundary was never actually tested. Runs the real applyMainBufferSnapshot choreography against a live composition, including the full 2J/3J/H wipe plus the resize and alt-screen branches. textarea.value, selectionStart/End, compositionView.textContent and .active all survive byte-identical, and interleaving a restore between every jamo of 문제 still commits 문제 at onData. Also pins that the uncommitted preedit is absent from the captured snapshot: it lives in the textarea, never the buffer, so a restore has nothing stale to echo back. Injecting one textarea.value = '' into the restore fails exactly the three restore-boundary tests. * test(terminal): pin that Cmd tears down a composition where Ctrl and Shift do not xterm's composition keydown exempts only keyCode 16/17/18 (Shift/Ctrl/Alt) plus 20/229. macOS Meta — 91/93/224 — is absent, so a Cmd press mid-composition takes _finalizeComposition(false): the overlay goes dark and never recovers, because compositionstart is not re-fired. The user composes the rest of the word blind. Linux and Windows users press Ctrl and are exempt. xterm already has a Meta-aware modifier predicate in wasModifierKeyOnlyEvent, so this is an internal inconsistency rather than a deliberate choice. Owns no reported row and is version-neutral: 5/5 on both 1.4.162 and 1.4.163. The branch is unexercised in all 328 recorded traces, so this is a hazard pin, not a regression guard. Only the teardown is asserted; the likely duplicated commit needs a compositionend the IME kept alive across the Cmd, which no capture contains. Deleting the exemption fails exactly the three paired negatives; adding Meta to it fails exactly the two Cmd arms. * test(native-chat): characterize preedit loss when a question card replaces the composer An AskUserQuestion card fully replaces the composer by design, but the in-flight composition goes with it: the composer unmounts before compositionend reaches it, so the preedit is never committed to the draft. The committed text survives only because the draft is cached and restored via defaultValue. Node identity changes, value 'abc' is preserved, the 가 is gone. Drives the real NativeChatView -> SessionGate -> InteractiveCard -> questionActive swap -> Composer -> ComposerField, flipped by writing the same store field an AskUserQuestion hook event writes. Flipping questionActive to false fails exactly this test and nothing else across 639 native-chat tests, so the path was entirely unguarded. CHARACTERIZATION TEST: it asserts the loss. Fixing the defect — committing the preedit before the swap, or keeping the composer mounted — will make this file fail. Update the expectations to the new contract rather than working around them. Owns no reported row. #12118/STA-3219 flicker is keyed to token counters, which provably do not remount, and a question card arrives once per question. * test(terminal): pin the duplicated commit when Meta interrupts a composition _finalizeComposition(false) sends textarea.value.substring(start, end) but cannot clear the IME-owned textarea, so a later compositionend re-sends the same range. Meta reaches that path because CompositionHelper exempts only Shift/Ctrl/Alt; xterm's own wasModifierKeyOnlyEvent covers Meta four ways, so the omission is an internal inconsistency rather than a choice. Companion to the modifier-exemption guard, which deliberately pins only the overlay teardown. This pins the data consequence. HAZARD PIN: owns no reported row. The trigger is unverified on hardware — no capture in the corpus contains a Meta-during-composition gesture, and whether macOS keeps the composition alive across it is unmeasured. The duplication follows from the code given that sequence; whether users reach the sequence is the open half. An earlier premise that Space (keyCode 32) reaches this path was refuted by a corpus scan: 0 of 731 evidence files carry a keyCode-32 Space while composing, against 171 at 229, and 229 returns early. * test(terminal): characterize the syllable lost when the textarea blurs mid-composition CoreBrowserTerminal._handleTextAreaBlur clears the helper textarea unconditionally — "Text can safely be removed on blur" — while CompositionHelper._finalizeComposition reads the committed text back out of that same value from a deferred timeout. By the time it runs the value is empty, the substring is '', and triggerDataEvent never sees the syllable. xterm checks composition state in _syncTextArea and omits the same check here. Six cases. Blurring mid-composition loses the syllable in every ordering, including compositionend-before-blur, which is Chromium's real order — so it is not an ordering artifact. A bare textarea.blur() with no Orca code loses it too, which places the owner upstream: Orca's unguarded release on outside pointerdown is one trigger, not the cause. Committing 한 then blurring mid-가 yields ['한'] where ['한','가'] is correct: one syllable gone, surrounding text intact. Teeth checked by inverting — adding an Orca-side composition guard flips exactly the three cases that route through the release path and leaves the bare-blur and no-blur cases green, which is the scope split: a fix in regular-terminal-focus-ownership alone would not close this. HAZARD PIN, but unlike the others this one has a real production injector — clicking outside the terminal mid-composition. Owns no reported row. The shape matches #9738's report; the injector does not, and a shape match with a mismatched injector is not an owner. * test(terminal): say which arm the STA-3237 fixture came from The recorded keydowns are wave 4's A-shift-unmarked-only — the arm that emits no PTY bytes. Nothing in the file said so, so two readers concluded the row's events fail the owner's predicate and that STA-3237 and STA-3222 were different defects. They share an owner; the arm that fires is Process/229+Shift, absent from this bubble-phase trace because the owner claims it in the capture phase. Also corrects "code-blind": the v1.4.163 policy emits \x1b\r only for a shift-only key:'Enter', and a jamo keydown reaches that branch solely via the isTerminalImeProcessEnter rewrite. The mock is deliberately wider so the ownership guard stays under test if that rewrite moves. Comments only — no assertion, fixture value, or mock behaviour changed. * test(e2e): track the input-source selector the macOS specs shell out to Five tracked macOS IME specs ran `swift .tmp/select-input-source.swift`, a file that is gitignored and existed only on one machine. Anyone else checking out the repo — or the same machine after .tmp is cleaned — could not run them, and they are the capture drivers for the macOS rows that are blocked waiting for exactly those runs. Moves it to tests/e2e/ beside its callers. The chord spec now resolves it from __dirname rather than reaching two levels up into .tmp. * test(terminal): pin the CJK repaint decision against the reporter's own output #12164 comment 1 and #5921 report agent output with double-width glyphs rendering duplicated character-by-character while ASCII in the same line stays clean. No IME, no composition, no keystroke — the user never types the CJK. Segmenting all three verbatim samples into maximal same-risk-class runs gives 33 runs and zero violations of "this run is corrupted iff the production detector flags it": 17 wide runs all corrupted, 16 narrow runs all byte-identical. The paired negative is co-located in the same line rather than in a separate run — the reporter supplied it without knowing. Doubling is asserted as present, not uniform: 자바스크립트 and 시스템 each leave a jamo undoubled, which is a repaint-region boundary artifact rather than a per-character transform. The discriminating arm is in the test rather than a source mutation: |
||
|
Jinwoo Hong
|
06780260c0 |
test(remote-runtime): run an old client and an old server against current code (#12682)
Mixed versions are the normal state of the remote-server feature: users update clients and servers independently. Until now nothing tested that. Every cross-version claim was made by code reading plus unit tests with hand-written old/new shapes — enough to catch design problems, not enough to catch a real skew regression. This runs the REAL protocol implementations from two builds against each other in one process: the actual host methods and RPC dispatcher on one side, the actual renderer multiplexer on the other, with a transport that reproduces the production asymmetry — each side decodes with its OWN codec and drops frames whose opcode it does not know. A frame survives only if the RECEIVING build understands it, which is what makes this level sufficient without launching two apps. The old side is a genuine checkout extracted from the release tag; the extracted client was confirmed to lack a symbol that exists only on main. Journey: subscribe, first snapshot, input reaching the process, live output, hide/reveal snapshot, transport drop, resubscribe, input landing again — across old->new, new->old, and a current/current control. Every step ends on an observed-state barrier; no sleeps. The oracle asserts the recorded step list, the exact 16-frame named sequence, negotiated capabilities, the exact input the host wrote to the PTY, rendered content, and zero decoder-rejected frames. A host method the stub lacks is recorded by name and asserted empty, so a harness gap cannot masquerade as a wire break. Detection is proven per violation shape, and it attributes each to the correct side: an unnegotiated opcode goes red only where a decoder would reject it, a removed published field goes red only where an old client consumes it, and a legal additive field stays green in all three pairings so the harness will not cry wolf on safe changes. It also documents the three compatibility rules in docs/reference/remote-wire-compatibility.md, linked from AGENTS.md, since they previously existed only as folklore — notably that "decoders reject unknown opcodes" is true for the desktop decoder but NOT for mobile, which silently drops them. Deliberately scoped: terminal stream only. The session-tab sync channel is not covered, nor agent-session publications, file/Git RPCs, mobile E2EE framing, or the relay transport. Two version points, so a regression introduced and reverted between them is invisible. CI selection was verified rather than assumed — `vitest list` confirms 0 matches under the shard's exclude and 4 under the dedicated job — because a lane silently running zero tests is precisely how a host-side defect escaped CI earlier in this series. Closes STA-3469. |
||
|
NeilandOrca
|
d8e5944b60 |
Stop a duplicate headless orca serve from crash-looping and exhausting AppImage FUSE mounts (#12212)
* fix(startup): stop a duplicate headless serve from crash-looping and leaking AppImage mounts A second Orca launch that loses the single-instance lock called app.quit() before `ready`. That quit is deferred, so the doomed process kept booting into Chromium's Linux display initialization, failed with "Missing X server or $DISPLAY", and died with SIGSEGV. systemd read that as a crash and restarted it forever; each restart re-mounted the AppImage and left the squashfuse mount behind, until the host hit the 1000-mount FUSE ceiling and every later launch failed. The lock-losing launch now calls app.exit(3), which terminates synchronously before any display init. Exit code 3 is a stable "another process already owns this userData profile" contract, and the documented systemd unit uses RestartPreventExitStatus=3 plus a real StartLimitIntervalSec/StartLimitBurst window so a permanently failing launch can no longer retry unbounded. Second-instance argv is now forwarded to the owner, and a duplicate `orca serve` no longer asks the live headless server to open a desktop window. Desktop activation for ordinary launches and macOS dock re-activation is unchanged. Closes #11935 * docs(headless): clear the start limit before the scripted service starts StartLimitIntervalSec=300/StartLimitBurst=5 rate-limits operator starts too, so after a crash-loop trips the burst systemd refuses a plain `systemctl start` for the rest of the window. The Upgrade and Roll back scripts run under `set -euo pipefail`, so that refusal aborted the rollback mid-flight and left the server down on the exact recovery path the doc prescribes. Both scripts (and their EXIT-trap recoveries) now run `systemctl reset-failed` first, the unit reference explains the interaction, and the crash-loop bullet points at it for manual starts. Co-authored-by: Orca <help@stably.ai> * test(startup): reproduce the #11935 duplicate-serve crash loop under real Electron The committed coverage for #11935 was source-text greps, so nothing gated the mechanism the fix rests on: pre-`ready` `app.quit()` is deferred, which is why the lock-losing headless `orca serve` kept booting into Linux display init. This runs two real Electron processes against one disposable profile. The duplicate executes the lock-loss gate's own `app.*` statement, lifted out of `src/main/index.ts`, so reverting to `app.quit()` fails the test. It also feeds the owner's real forwarded argv through `shouldActivateDesktopForSecondInstance`. Also record why the activation predicate matches `--serve` and not the `serve` subcommand: an AppImage launched as `orca serve` exits at the CLI redirect before requesting the lock. * test(startup): wait for the owner process to exit before removing its profile Windows holds the profile's handles for a beat after SIGKILL, so an immediate rmSync can fail with EBUSY/EPERM. Co-authored-by: Orca <help@stably.ai> * test(startup): pass the fixture marker path by env, not argv Chromium reorders argv and the duplicate's argv is itself under test, so a trailing positional was the wrong channel for it. Co-authored-by: Orca <help@stably.ai> * test(startup): only the activation case waits on the owner notification The exit-contract cases assert on the duplicate's own already-terminated process, so they should not block on cross-process delivery. Co-authored-by: Orca <help@stably.ai> * test(startup): drop the staged lock race, keep the real-Electron gate contract CI proved the two-process form cannot work on a display-less Linux runner: Chromium's ProcessSingleton needs the browser IO thread, which needs `ready`, which needs a display. The pre-`ready` owner looked stale and the duplicate took the lock (`expected [ 'DUPLICATE_WON_LOCK' ] to include 'DUPLICATE_LOST_LOCK'`). Lock acquisition and argv forwarding are already covered in single-instance-lock.test.ts. What only a real process can settle is what the loser does next, so that is all this file now runs -- display-independent. Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Orca <help@stably.ai> |
||
  OrcaWinandOrcaWin
|
ed7849eb7b |
fix(worktrees): stop silently switching existing Windows setup scripts to Git Bash (#12406)
* fix(worktrees): stop silently switching existing Windows setup scripts to Git Bash #6967 derived the Windows setup-runner shell from `terminalWindowsShell`. On upgrade, any Windows user whose terminal preference resolved to Git Bash had their existing `orca.yaml` setup script (and issue command) handed to bash instead of cmd.exe. Scripts authored against the cmd runner — `copy`, `xcopy`, `set VAR=value`, `if errorlevel 1`, `%VAR%`, backslash paths — broke with no migration and no warning, and the failure looked like Orca broke the project. The conflation is also wrong in the steady state: a terminal preference is per-user, so two people on the same repo got different interpreters for the same orca.yaml and no project could write a setup script that worked for all of its Windows contributors. The interpreter is now a property of the script, declared the standard way: a leading `#!` line. Native Windows keeps the historical `.cmd` runner unless the script declares a POSIX shell, so no existing script changes behavior. `resolveSetupRunnerShell` keeps its role as the feasibility gate — a bash runner still requires the terminal to resolve to Git Bash, since the launch command is typed into that shell and uses MSYS `/c/...` paths. `buildWindowsRunnerScript` now drops a leading `#!` line rather than `call`ing it, so a declared-bash script that falls back to cmd (Git Bash missing) fails on a real setup line instead of aborting on errorlevel at line one. WSL worktrees, POSIX platforms, and SSH hosts are untouched. * fix(worktrees): keep the cmd setup runner launchable from a Git Bash pane Adversarial review of this PR found that pinning the runner format per script reopened issue #6896 one layer down. - `WorktreeSetupLaunch.shell` had been redefined to mean "the format the runner file was written in". `resolveSetupRunnerCommand` consumes it as "the shell that types the launch command", so a Git Bash terminal with a batch setup script produced `cmd.exe /c "C:\...\setup-runner.cmd"` typed into a bash pane, where MSYS rewrites the `/c` switch into a drive path: cmd opens interactively and setup never runs. `shell` is the terminal's family again; the runner file's .cmd/.sh extension carries the format, and a batch runner launched from a POSIX pane reuses the existing PowerShell ProcessStartInfo launcher. - The cmd runner dropped a leading `#!` line and ran the rest as batch, so a bash script reaching cmd (PowerShell/cmd terminal, or any SSH-to-Windows host) got its interpreter-agnostic prefix executed before failing mid-way. It now prints why and exits 1 without running anything. - A `#!` line's option flags were discarded: `#!/usr/bin/env -S bash -euo pipefail` lost pipefail because the runner is launched as `bash <path>`. The generated posix runner now replays declared flags via `set` and drops the duplicate interpreter line. - Docs cover the per-user setup command in repository hook settings, which goes through the same `#!` rule, and describe what the `#!` line does and does not select. Tests: composed launch command for a POSIX pane + cmd runner (hooks, shared runner command, setup sequencing gate, observed-setup signal), the cmd runner's shebang refusal, and shebang flag replay. Each fails with the source reverted. * fix(worktrees): replay only real `set` flags and keep the gate in the pane's shell Two round-2 review findings: - `#!/bin/bash -l` replayed `set -l`, which exits 2 and aborted the runner under its own `set -e` before a single setup line ran (all platforms). Only the flags `set` documents are replayed now; a bare `-o` with no option name is dropped instead of dumping the shell-option table. - The wait-for-setup gate picked its language from the runner file, so a batch runner launched from a Git Bash pane got the PowerShell gate while the agent startup command was already POSIX-quoted — `Invoke-Expression` cannot parse `'\''`. The gate now follows the pane; the runner still launches through the ProcessStartInfo launcher, never through bash. --------- Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
 Jinjing
|
96c954f3be |
chore: remove force-added design docs from docs/ (#11891)
Keep only the durable docs already allowlisted for tracking (STYLEGUIDE, assets, localized readme, and reference compatibility guides). Drop feature design notes, plans, and repro artifacts that were force-added past the existing docs ignore rules. |
||
  Sebastián Castañoandscastanoh21
|
676ef7fab8 |
feat(cli): add orca skills install and orca skills update for headless skill setup (#9201)
Adds `orca skills install` and `orca skills update` so skills can be set up without the GUI — SSH hosts, containers, CI. Previously `orca skills` had only `list` and `get`, so there was no headless path. **Agent targeting is scoped explicitly rather than delegated to detection.** The `skills` CLI decides which agents to install into, and with `-y` and zero detected agents it takes `targetAgents = validAgents` — all ~75. That is not a corner case for a headless CLI: a fresh SSH box or container with no agent installed is the normal starting state. Measured on a bare host, the unscoped command created **52 top-level agent directories and 54 junctions** (one real payload in `~/.agents/skills`, the rest links) on Windows, and 52/53 on macOS. The CLI now passes `--agent` derived from Orca's own detection, mapped to the `skills` key namespace, plus `universal`. Supplying `--agent` makes `runAdd` use it directly and never call `detectInstalledAgents()`, so the fan-out branch is unreachable. On a bare host it now refuses with `No coding agent detected on this host` and exit 1, creating nothing. Same command with scoping: **1 directory, 0 junctions.** `universal` alone would under-install — Claude Code is not in that set, and 19 of 28 mapped keys write agent-private homes `universal` never touches. `--agent '*'` is the bug itself. The mapping is hedged three ways: `null` for any agent whose key could not be confirmed, `satisfies Record<TuiAgent, …>` so a new Orca agent is a compile error, and a test pinning every mapped key against the CLI's own valid list. Fixed during review — two holes that each restored the full fan-out through a different door: - `--agent ','` trimmed to nothing, which skipped the refusal *and* emitted no `--agent`. - `--agent -y` passed an emptiness check, and the vendor CLI silently drops `-`-leading values, re-emptying its list. The real invariant is argument *shape*, not emptiness, and it is now enforced at the choke point in `buildAgentFeatureSkillInstallArgs`, so no caller can emit `-y` without a usable target. `*` remains allowed — asking for every agent explicitly is a choice, not an accident. Verified with 51 hostile inputs through the built binary, each recorded argv replayed through the vendor's own parser. Also fixed: the `ORCA_CLI_CWD` refusal now runs before target resolution (it was quoting the wrong host's agent list), and `--dry-run` is refused in a forwarded shell rather than printing a command naming the wrong machine. Validated on a real Windows host across PowerShell 7, PowerShell 5.1, cmd.exe and Git Bash: `.cmd` shims route through `cmd.exe` and `.exe` shims spawn directly (proved with instrumented shims, not inferred), the ENOENT path produces an actionable error rather than a silent failure, and `skills update` genuinely restores a corrupted skill byte-for-byte. Known, not addressed here — both upstream behaviours this only forwards: a partial install failure exits 0, and "no installed skills found" exits 0. Both are invisible to the headless callers this feature exists for. Co-authored-by: scastanoh21 <scastanoh21@gmail.com> |
||
|
OrcaWinandOrcaWin
|
dde72f85de |
fix(windows): separate updater from orchestration migration (#11405)
* fix(windows): separate updater from orchestration migration * fix(terminal): attest adopted reveal identity --------- Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
|
OrcaWinandOrcaWin
|
363e478909 |
fix(orchestration): preserve active workers across updates (#11271)
* fix(orchestration): preserve active workers across updates * test(ssh): model absent legacy adoption * test(orchestration): align compatibility contracts * fix(windows): escape updater PowerShell booleans * fix(windows): restore stock uninstall process check * fix(orchestration): keep recovery off renderer startup barrier * fix(orchestration): harden legacy recovery migration * fix(orchestration): close recovery review gaps * fix(orchestration): complete legacy worker cutover recovery * fix(orchestration): preserve legacy workers across updates --------- Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
|
NeilandOrca
|
3a80fbe162 |
Revert terminal rendering changes from #10692, #10794, #10871, and #10907 (#11338)
* Revert "fix(terminal): avoid flash while restoring parked terminals (#10871)" This reverts commit |
||
|
JinjingandOrcaWin
|
a40183389b |
feat: bound direct SSH reconnect fan-out and recovery (#11003)
* docs: design for direct SSH reconnect fan-out Capture the implementation-ready plan for host-qualified, epoch-fenced SSH reconnect recovery after two rounds of multi-model LLM counsel review. * docs: reconcile SSH reconnect fan-out design * docs: close reconnect design consistency gaps * feat: implement bounded direct SSH reconnect recovery * fix: bound direct SSH retry settlement * fix: harden direct SSH reconnect authority * fix: preserve split SSH retry ownership * fix: preserve SSH split continuation authority * docs: record final SSH reconnect validation * fix: preserve SSH authority through retained and detached state * fix: retain SSH authority across delayed split mounts * fix: close SSH authority recovery gaps * fix: fence stale SSH transport replacement * fix: serialize SSH target teardown * fix: settle SSH teardown failures before reconnect * fix: retire failed SSH reset sessions * test: reconcile current main E2E contracts * fix: close direct SSH reconnect review gaps * fix: fence stale SSH reconnect side effects * fix: close final SSH reconnect lifecycle gaps * test: stabilize current-main reliability gates * test: prove plugin navigation containment * test: make plugin navigation oracle authoritative * test: make plugin navigation oracle deterministic * ci: allow sharded e2e suite to finish * test: wait for runtime pane publication * test: classify pane readiness by error code * test: select close persistence terminal by tab identity * docs: mark reconnect implementation validated --------- Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
|
NeilandOrca
|
97cb32c1cc |
fix(terminal): release an abandoned synchronized-output frame on reveal (STA-2694) (#10907)
* fix(terminal): release an abandoned synchronized-output frame on reveal Alt-screen agent TUIs (OpenCode/OpenTUI, Codex, grok) bracket every repaint in `?2026h … ?2026l`. Hiding a pane mid-bracket — which a worktree switch or cold-park lands on routinely, since these brackets are written many times a second — leaves xterm's `decPrivateModes.synchronizedOutput` latched. RenderService.refreshRows checks that latch *before* rendering, so while it holds, every repaint Orca owns is a no-op: the forced render-pause repaint, the plain `refresh()` fallback, and the shared glyph-atlas rebuild all render zero rows while the xterm buffer is perfectly correct. Release the latch at the two reveal repaint entry points so those repaints actually paint. Also adds an OpenCode-shaped alt-screen e2e fixture and spec. The existing inline-TUI convergence spec covers the normal-buffer shape (live block glued to the bottom, history scrolling into scrollback); this covers the full-screen alternate-buffer shape, where nothing scrolls and so no row ever self-heals through the scroll path. Scope note: xterm arms a 1s watchdog that clears this latch on its own, so this closes a bounded window rather than the whole STA-2694 report. The e2e spec passes with and without the production change for that reason; the unit tests are what pin the behavior. Refs STA-2694. * fix(terminal): clear the render model on the plain-refocus repaint path `schedulePaneRevealPresent` — the atlas-preserving path a plain window refocus takes — only called `terminal.refresh()`. xterm's renderers are diff-based: `_updateModel` early-continues on any cell whose code/fg/bg/ext still match the cached model, so a refresh repaints nothing for a pane whose buffer never changed. When an occluded window loses its canvas contents while that model stays populated, the refresh skips exactly the cells that went stale and the pane keeps compositing pre-hide pixels — until a window resize reallocates the model, which is the repair users find by hand. Clear the model first (`RenderService.clear()` → renderer `clear()` → `_clearModel(true)`) so the refresh becomes a guaranteed full repaint. That drops cached cells and glyph vertices but NOT the texture atlas, which is shared by every same-config terminal and whose mid-stream wipe re-arms xterm's page-merge garble race (xterm.js #4480) — the reason this path is atlas-preserving in the first place. Also covers the DOM-renderer fallback in `resetWebglTextureAtlas`: `clearTextureAtlas()` is what invalidated the model on the WebGL path, so a pane without an addon had nothing invalidate it and hit the same skip. Scope note: the e2e spec guards buffer/geometry convergence across the hide/reveal boundaries and adds idle-agent and headful desktop-hide cases, but it cannot observe a stale canvas — both oracles built for that (canvas-vs-buffer ink sampling, screenshot-vs-forced-repaint) were proven blind by injecting the defect, and the spec header documents why. The unit tests pin the ordering and the atlas-preservation invariant. Refs STA-2694. Co-authored-by: Orca <help@stably.ai> * docs(terminal): hand off the STA-2694 reveal-artifact investigation Records both fixed defects with their xterm mechanisms, the reveal/wake call graph, why every e2e oracle for a stale canvas was proven blind, how to arm the in-app render-desync sentinel on real hardware, and the one unverified lead (dimension staleness) that would explain why a window resize specifically is the repair users find. Refs STA-2694. Co-authored-by: Orca <help@stably.ai> * Revert "fix(terminal): clear the render model on the plain-refocus repaint path" This reverts commit |
||
|
Neil
|
6b16c20796 | fix(memory): clarify Resource Manager accounting (#10821) | ||
|
Neil
|
a0944cc129 |
fix(linux): restore Ubuntu 20.04 launch — pin node-pty glibc symbols + add glibc/libstdc++ packaging gate (#9902) (#10019)
* fix(linux): restore Ubuntu 20.04 launch by pinning node-pty glibc symbols (#9902) The bundled node-pty pty.node is compiled from source in release CI on ubuntu-latest (glibc 2.39). glibc's 2.32-2.34 libpthread/libutil merge relocated openpty/forkpty (GLIBC_2.34) and pthread_sigmask (GLIBC_2.32) into libc under new symbol versions, so the from-source build bound to versions absent on Ubuntu 20.04 (glibc 2.31). The main process imports node-pty at startup, so the app crashed on launch. pty.node is the sole blocker (Electron needs GLIBC_2.25; other native modules <= 2.17). - Patch node-pty: a .symver shim pins the 3 symbols to their pre-merge version (GLIBC_2.2.5 x64 / GLIBC_2.17 arm64), and Linux-only ldflags force libutil.so.1/libpthread.so.0 back into DT_NEEDED. Guarded to Linux; macOS/Windows untouched. - Add a packaging gate (verify-linux-glibc-floor.cjs, afterPack): reads each bundled native binary's objdump -p version needs and fails the Linux build if any strong GLIBC_/GLIBCXX_/CXXABI_ node exceeds stock Ubuntu 20.04 (glibc 2.31 / GLIBCXX_3.4.28 / CXXABI_1.3.12). Catches GLIBC_ABI_DT_RELR, rejects GLIBC_PRIVATE, skips weak needs, fail-closed. - Docs + tests; the lazy sherpa-onnx speech prebuilt (GLIBCXX_3.4.29, never loaded at launch) is a documented libstdc++-floor exemption. * fix(linux): assert DT_NEEDED provider deps in the glibc-floor gate Harden the packaging gate (flagged in adversarial re-eval): the version-floor check alone can false-pass if the patch's forced `-l:libutil.so.1` ever silently drops — the pinned openpty@GLIBC_2.2.5 still resolves from libc's compat alias at build time, but fails to load on Ubuntu 20.04 where openpty/forkpty live only in libutil. The gate now also asserts that any binary importing openpty/forkpty keeps libutil.so.1 in DT_NEEDED. Validated on a real symver-pinned .so with libutil dropped (now fails) vs. present (passes). Documents the recommended real-host smoke-test follow-up. |
||
|
OrcaWin
|
b232df732b | fix(terminal): make remote agent sessions host-authoritative (#9687) | ||
|
OrcaWin
|
f9f3cd2fbe | fix(terminal): prevent reconnect from killing live daemon sessions (#9804) | ||
|
OrcaWin
|
34c160442f | Fix headless Linux serve pairing readiness (#9785) | ||
|
OrcaWin
|
cc44acaaa3 | Fix Windows ConPTY OSC color reply leaks at the PTY owner (#9651) | ||
  
|
aad34cbb32 |
docs(headless-server): add upgrade SOP for orca serve on Linux (#9575)
* docs(headless-server): add upgrade SOP for orca serve on Linux The headless Linux guide covered install/run/systemd but had no upgrade section, leaving operators to guess how to move to a new AppImage without losing state. Add an "Upgrade" section documenting the manual SOP (serve mode never auto-updates) and one troubleshooting bullet: - State lives under the service user's ~/.config (orca + Orca dirs), independent of /opt/orca, and orca-data.json is forward-migrated on load, so a forward upgrade is safe. - Replace the binary with an atomic same-filesystem rename (download to .new, verify, mv) — never curl -o over the FUSE-mounted live binary. - Back up the whole .config before upgrading, because rollback is NOT binary-only safe: an older build strips newer orca-data.json fields it doesn't recognize, and the .bak.* ring is corruption-recovery, not a pre-upgrade copy. - Note there is no headless version command; track the release tag instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(headless-server): harden the orca serve upgrade/rollback runbook Address CodeRabbit review on #9575: - Fail closed: run the upgrade block under `set -euo pipefail`, remove any stale `.new` file before download, and gate the atomic `mv` on an explicit ELF check so a failed/partial/non-ELF download can never be promoted. - Keep /opt/orca/VERSION tied to the installed binary: a single `TAG` variable drives both the download URL and the recorded VERSION, saved as VERSION.prev on upgrade and restored on rollback so the audit file never drifts. - Crash-loop troubleshooting now points to Roll back first (restores the pre-upgrade orca-data.json) instead of re-running Upgrade. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(headless): harden server upgrade SOP --------- Co-authored-by: fanyunqian.1 <fanyunqian.1@bytedance.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
|
Jinwoo Hong
|
c0f0810dd9 |
Fix native Windows PTY startup query handling (#9500)
* Fix native Windows PTY startup query handling * Fix daemon boot smoke protocol lookup * Fix Windows daemon repro protocol lookup |
||
|
Jinwoo HongandOrca
|
7adda25b0a |
fix(daemon): retire empty current-generation daemons (#9277)
* fix(daemon): retire empty current-generation daemons Co-authored-by: Orca <help@stably.ai> * fix(daemon): retire empty daemons on disconnect Co-authored-by: Orca <help@stably.ai> * test(daemon): authenticate Windows lifecycle harness Co-authored-by: Orca <help@stably.ai> * test(daemon): assert remaining shutdown budget Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Orca <help@stably.ai> |
||
|
Jinjing
|
1def694e80 |
Add native chat skill picker with host-aware discovery (#9480)
* Add native chat skill and command picker with host-aware discovery Adds a unified, keyboard-first skill and command picker to native chat that: - Uses agent-native invocation syntax (slash for Claude/OpenClaude/Grok, dollar for Codex) - Discovers skills only on the pane's execution host (local, WSL, SSH-unavailable, or runtime) - Groups or separates commands and skills per agent configuration - Deduplicates by canonical path but preserves visibility through all contributing roots - Handles IME composition, loading states, and errors without claiming PTY-level control - Records picker telemetry (open, item accepted, send classification, discovery outcomes) - Extends shared agent profiles to define per-agent skill grammars and source ownership * Remove obsolete reference and design documentation Clean up stale design specs, implementation plans, and investigation notes from docs/reference/. These documents predate the current implementation and are no longer actively maintained or referenced by the codebase. * Extract shared skill discovery utilities and add skill invocation envelo - Move skill comparison and source classification to shared module for native/WSL reuse - Extract display text sanitization to prevent control/zero-width character spoofing - Add native-chat command envelope parser and surfacer for skill invocations - Extend discovery timeout backstop to account for WSL metadata read sequence * Localize skill picker UI for Spanish, Japanese, Korean, Chinese Translate skill picker UI strings including commands, skills, loading states, error messages, and scope labels for the new skill picker feature across four language locales. * Fix skill picker bugs and improve code robustness - Fix i18n plural handling: rename `count` to `sourceCount` to prevent unintended plural-key resolution in localized strings - Fix skill discovery array mutations: copy `root.providers` to prevent bugs during dedup merge - Fix image attachments being silently dropped when message text starts with /skill or agent prefix - Extract `quoteBashString` utility for WSL command code reuse across builders - Add line-separator safety characters (0x2028/0x2029) to skill display filter - Remove stale doc reference links and clarify inline comments * Add reference docs for git compatibility and headless Linux server setup Track previously untracked operational guides in `docs/reference/` that explain Git binary compatibility requirements across host types and how to run `orca serve` on headless Linux. Update AGENTS.md and README.md to link to these references. |
||
|
NeilandJinjing
|
6e91ca6c0e |
fix(browser): local HTTPS Try HTTPS + cert proceed (#8454) (#9104)
Co-authored-by: Jinjing <6427696+AmethystLiang@users.noreply.github.com> |
||
|
Jinwoo HongandOrca
|
319ae4e9ea |
fix(terminal): make whole-tab close durable (#8958)
Co-authored-by: Orca <help@stably.ai> |
||
 
|
6e2a4a824d |
fix(worktrees): stop surfacing prunable git worktrees as live workspaces (#8409)
* fix(worktrees): stop surfacing prunable git worktrees as live workspaces A worktree still registered in git but whose directory was deleted (git's `prunable` state) was enumerated as a normal workspace, producing repeated pty:spawn DaemonProtocolError / fs:readDir ENOENT loops and a blank pane. - Parse the `prunable` porcelain field (Git >= 2.36) in both the main and relay worktree-list parsers. - For Git < 2.36 (no `prunable` field), probe each linked worktree path for existence on the fallback line-block path, skipping locked registrations to mirror git's own prunable rules. - Omit prunable worktrees from the detected-workspace enumeration only; removal/cleanup flows keep seeing them. - Extend the real-binary compatibility contract with the 2.36 `prunable` boundary. Fixes #8389 Claude-Session: https://claude.ai/code/session_018Rg1Bpq4GGwmz613hq6RSD * fix(worktrees): pin the prunable/locked porcelain annotations to their real Git 2.31 boundary The prunable and locked annotations landed in Git 2.31, five releases before `worktree list -z` (2.36); only -z defines the capability fallback boundary. Correct the compatibility contract so a future matrix entry in the 2.31-2.35 range passes, and reword the fallback comments: on 2.31-2.35 the annotations still parse and the existence probe is a backstop; only Git <2.31 relies on it outright. * fix(worktrees): omit prunable registrations from the Space scan A prunable registration has no directory to size or reclaim, so Space rendered it as a dead "Missing" row whose checkbox stayed disabled with no prune/remove affordance (reported on macOS after a reboot cleared /private/tmp under 16 registrations). Skip prunable entries in the scan, matching the workspace enumeration; removal flows list worktrees separately and still see them. --------- Co-authored-by: kaynan <kaynan.camargo@terceiro-sky.com.br> Co-authored-by: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> |
||
|
Brennan Benson
|
40d0159926 |
fix(terminal): kill agent descendant processes on session teardown (STA-1800) (#8706)
* fix(terminal): kill agent descendant processes on session teardown (STA-1800) Agent CLIs spawn tool children in detached process groups that PTY SIGHUP can never reach. Killing an agent session (tab close, retire, sleep) left those children running as orphans — eight orphaned git processes burned ~8 cores for up to 11.5h under the agents-running keep-awake and drained a battery to 8%. New pty-descendant-termination module: snapshot the ppid tree BEFORE signalling (a dead root's descendants reparent to pid 1 and become unfindable), SIGTERM the root group and every descendant, then after a 2s grace SIGKILL survivors gated on a pid+start-time identity re-check so a recycled pid is never signalled. Snapshot is bounded and never rejects; failures degrade to today's shell-only kill. Wired for agent sessions only (plain terminals keep nohup semantics) at all three POSIX kill sites: local provider shutdown, daemon TerminalHost immediate kill (the pty:kill path — force-kill bypassed Session.kill entirely), and daemon Session graceful kill. Verified live in the built app: an agent pane with a detached-pgid child; the child survived on the unwired build (three control runs) and dies within ~5s with the fix. Windows ConPTY and SSH-hosted PTYs keep the previous foreground-tree contract (documented follow-ups). * fix(terminal): harden descendant teardown * fix(terminal): require fresh process snapshots * fix(terminal): close descendant teardown races * refactor(terminal): preserve teardown line budget * fix(terminal): keep descendant teardown fresh and identity-safe * docs(reliability): record integrated descendant E2E * fix(terminal): bound descendant teardown work * fix(terminal): share descendant snapshot indexes * docs(reliability): record descendant review evidence * revert: remove speculative descendant hardening |
||
  
|
0302ae86b8 |
feat(ssh): support Kerberos/GSSAPI hosts via the system OpenSSH transport (#7507)
* feat(ssh): support Kerberos/GSSAPI hosts via the system OpenSSH transport ssh2 has no gssapi-with-mic support, and adding it would mean forking its protocol layer plus packaging the kerberos native module for three platforms. Instead, route GSSAPI hosts through the existing system-OpenSSH transport, which delegates Kerberos (tickets, SSPI on Windows) to the platform ssh binary. Two tiers, because RHEL-family distros enable GSSAPIAuthentication globally in /etc/ssh/ssh_config and ssh -G therefore reports it for every host: - Targets whose ~/.ssh/config Host block explicitly sets GSSAPIAuthentication yes (imported as target.gssapiAuthentication) try system ssh first, falling through to ssh2 so key auth and credential prompts still work when no ticket is available. - When ssh2 exhausts key/agent auth and the ssh -G-resolved config enables GSSAPI, retry over system ssh before prompting for credentials, so Kerberos-only hosts on distro-default configs connect without a password prompt. Hosts where keys work never leave the ssh2 path. Manual targets flagged for GSSAPI pass -o GSSAPIAuthentication=yes explicitly since they bypass ssh_config. Both tiers work headless (no credential callbacks required). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ssh): harden GSSAPI transport selection (review fixes for PR #7507) Review fixes on top of the Kerberos/GSSAPI feature branch (s546126/kerberos-ssh): - HIGH: reset useSystemSshTransport on the ssh2 fall-through. doSystemSshProbe sets the flag before spawnSystemSshCommand, which throws synchronously when no system ssh binary is on PATH (outside the probe try/catch). The proactive fall-through previously reset only 2 of 3 transport fields, so exec/sftp kept routing through the failed transport - breaking GSSAPI on Windows-with-Git-ssh and headless Linux. - MEDIUM: throw a cancellation error (not the stale ssh2 authError) when a disconnect supersedes the reactive probe mid-flight, and guard connect()'s catch on disposed, so a deliberate disconnect is not overwritten with auth-failed. - MEDIUM: skip the encrypted-key passphrase prompt when the GSSAPI fallback applies, so a Kerberos ticket is tried before prompting; the general prompt still fires if the probe fails. Adds 3 mutation-verified regression tests and hardens two existing tests to assert the probe actually ran. Not connected to any PR remote. Co-authored-by: Orca <help@stably.ai> * fix(ssh): isolate GSSAPI system transport Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: s546126 <268420947+s546126@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Neil <4138956+nwparker@users.noreply.github.com> Co-authored-by: Jinwoo-H <jinwoo0825@gmail.com> Co-authored-by: Orca <help@stably.ai> |
||
|
OrcaWin
|
a8a8040589 |
fix(agent-hooks): make Windows cmd hook launcher directly spawnable (#8430 regression) (#8737)
Codex, Antigravity, and Devin launch their agent-hook `command` as a program (argv[0]), not through cmd.exe. PR #8430 changed wrapWindowsCmdHookCommand to emit an `if exist "path\." (drain) else if exist "path" (call "path") else (drain)` compound whose argv[0] is the cmd builtin `if` — unspawnable — so every Codex/Antigravity/Devin hook (SessionStart, UserPromptSubmit, Stop, ...) failed with "hook exited with code 1" on Windows starting in v1.4.138. Revert the cmd-safe fast path to the bare, directly-spawnable .cmd path (the proven pre-#8430 form). A cmd-builtin drain and direct-spawnability are mutually exclusive, and nesting cmd.exe /d /c breaks large-payload draining; the missing-script stdin drain stays on the encoded-PowerShell fallback (used for spaced/non-ASCII paths). Upgrades self-heal on first launch: startup install() unconditionally rewrites the command and Codex trust entry, sweeping the old compound form. Add a platform-independent regression guard (launcher must resolve to a real file, never a cmd-builtin fragment), update the lifecycle test + docs, and fix a stale Devin comment. |
||
|
Brennan Benson
|
36cd8a3347 |
fix(terminal): retire sessions when tabs close (#8628)
* fix(terminal): retire sessions when tabs close * fix(terminal): close remaining session lifecycle gaps * fix(terminal): close review-discovered lifecycle gaps * fix(terminal): revalidate bulk session retirement * fix(terminal): harden retirement review edges * fix(agent): reverify restored pane authority * test: make terminal retirement gate portable on Windows * test: use POSIX join in Linux PATH assertion * test: keep simulated Linux PATH host-consistent |
||
|
Kaynan Sampaio de CamargoandJinjing
|
c3ab805d12 |
fix(agent-hooks): drain stdin before hook script early exits so agents never hit EPIPE (#8430)
* Fix hook scripts to drain stdin before any early-exit path Generated agent hook scripts and missing-script launchers could exit successfully before consuming the payload written to their stdin, leaving the writer with a broken pipe (EPIPE/ERROR_BROKEN_PIPE) once the reader closed early. Capture stdin (or drain it via a shared epilogue/fast-path guard) before any whole-script success exit across all POSIX, batch, PowerShell, and Git Bash launcher variants, and add a cross-agent lifecycle test suite plus a live Electron verification script to guard the contract going forward. * Harden hook scripts against unreadable managed scripts and add a Claude/ - Extend the POSIX launcher guard to also require `[ -r ]`, not just `-f`/`-x`, so an executable-but-unreadable managed script still drains stdin instead of erroring or silently misbehaving. - Add a verifier case (`verifyClaudeDevinSkip`) that spins up a local HTTP server and confirms the Claude hook never forwards a request that Devin already imported, catching accidental double-forwarding. - Update installer-utils tests and stdin-lifecycle docs to match the new readable-file guard and the added verification case. * Fix hook-launcher verification to derive script paths from the installed Extract the quoted path from the launcher's `if [ -f '...'` clause instead of reconstructing it via join(home, ...), so missing/failing-script test cases can't silently fall through to the real script if the install layout changes. --------- Co-authored-by: Jinjing <6427696+AmethystLiang@users.noreply.github.com> |
||
|
Jinwoo Hong
|
7b12d38b17 | perf(terminal): tune cold-park keep-warm so common rotation never remounts (#8262) | ||
|
Jinwoo HongandOrca
|
25ecf2eea2 |
fix: reconcile SSH repo rows after host re-add (#8201)
Co-authored-by: Orca <help@stably.ai> |
||
|
NeilandOrca
|
fd6805a299 |
Fix mobile terminal query reply authority (#8227)
* Fix mobile terminal query reply authority * fix(terminal): harden mobile query reply handoffs * fix(terminal): exclude passive mobile query responders * fix(terminal): gate mobile query replies on host capability Older hosts strip terminal.send's inputKind (zod drops unknown keys), so a forwarded xterm reply would land as ordinary floor-taking shell input. Hosts now advertise terminal.query-reply-input.v1 via status.get and mobile drops replies unless the host advertises it (pre-fix behavior). Also documents the bounded desktop-to-mobile handoff double-reply residual. Co-authored-by: Orca <help@stably.ai> * fix(terminal): advance snapshot seq across recovery snapshots The pending-overflow recovery loop trims buffered output against recovery.seq while query replay and boundary strips kept using the initial snapshot seq. Unreachable under today's control flow (no await separates the initial-overflow consume from the loop), but the stale seq would silently drop covered query replies if that ordering ever changes. Track the seq that actually covered the buffered chunks. Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Orca <help@stably.ai> |
||
|
Neil
|
533992bdda |
fix(git): cache unsupported capabilities per host (#8109)
* fix(git): cache unsupported capabilities per host Old Git worktree, ref-search, and merge-tree fallbacks retried unsupported flags on recurring operations, flooding subprocess traces. Centralize capability probing per native, WSL, and SSH execution host, coalesce concurrent probes, and retry periodically for in-place Git upgrades. * fix(git): recognize real old-Git merge-tree rejection * test(git): enforce real binary compatibility matrix * fix(ci): preserve Git compatibility test ownership * fix(git): retain supported capability state |
||
|
e84a8ddec9 |
Terminal performance initiative: pipeline fixes + term-speed-2 revival + PTY flow control (integration branch) (#7214)
* Skip legacy hidden skip grammar assertions
* Fix hidden TUI snapshot test setup
* Fix sleep wake history test contract
* Fix hidden delivery startup gate helper
* Fix hidden Latin skip branch predicate
* Fix hidden synchronized split-boundary replay
* Stabilize remote runtime mixed subscription test
* Keep hidden startup query parser active during window
* Stabilize raw emoji golden restore width
* Stabilize raw emoji golden fixture completion
* Keep terminals responsive under agent output load
* Add frozen-terminal repro harness and silent-drop regression tests
Investigation harness for the frozen-terminal reports (Discord
#performance, issue #2836): pane shows content, shell alive, daemon
output.log flat while typing.
- e2e: renderer crash -> auto-reload recovery and three restart/restore
shapes (live daemon, SIGSTOP-wedged daemon, daemon killed between
launches), each probing input at both drop layers. Post-crash phases
drive the renderer from the main process because a crashed target
severs Playwright's CDP session even though the app recovers.
- e2e helpers: layer-discriminating probes (direct pty.write vs
transport input, plus pty:listSessions ownership-rebuild revival).
- unit repro: vendored xterm 6.1.0-beta.287 WriteBuffer permanently
wedges when a sync throw escapes a write-completion callback or a
custom parser handler (xterm-write-buffer-stall.repro.test.ts).
- unit repros for both silent input-drop layers: main drops writes for
a live PTY once ptyOwnership loses the id (revived by listSessions),
and the renderer transport stays unbound after a failed connect.
- pty.test.ts: unregister every leaked SSH provider id in afterEach so
module-level provider state cannot leak across tests.
Co-authored-by: Orca <help@stably.ai>
* Harden xterm write pipeline against sync-throw wedge that freezes panes
A synchronous exception escaping xterm's WriteBuffer loop permanently
wedges that terminal: _innerWrite has no try/catch around the parse
action or the write-completion callback, the tail re-schedule never
runs, and write() only re-arms on an empty buffer. The pane stops
rendering and, if a replay was in flight, the replay guard latches and
pty-connection's onData silently eats every keystroke — matching the
field reports (Discord #performance, issue #2836: content visible,
shell alive, daemon output.log flat). Both vectors verified against
vendored xterm 6.1.0-beta.287 in xterm-write-buffer-stall.repro.test.ts.
Three layers of defense:
- Guard every write-completion callback Orca hands xterm at the two
choke points (writeForegroundTerminalChunk, writeBackgroundTerminalChunk),
with settle and onParsed guarded separately so a WebGL/renderer
failure during viewport settle cannot starve the replay-guard release.
- Guard all throwing-capable custom parser handlers (DA1, OSC 10/11,
CSI ?h/?l mode reports, OSC 52 clipboard, OSC 7 cwd), degrading a
throw to "not handled" — same escape class as
terminal-link-provider-guard.ts.
- Replay-guard watchdog: each engagement releases exactly once, from
xterm's completion or a 10s watchdog, so a lost completion (wedged
pipeline, disposed-terminal race) cannot latch the guard on a live
pane; replayIntoTerminalAsync resolves on either path so restore
chains cannot hang. Force-releases record a crash breadcrumb.
All guard trips record rate-capped crash breadcrumbs, so the next field
occurrence names the throwing stack instead of failing silently.
Co-authored-by: Orca <help@stably.ai>
* Cap unbounded terminal output buffers in main and the foreground queue
Field evidence (Discord #performance / #2836): renderer memory climbs to
~1.5 GB and terminals freeze; a force reload does not help until memory
recovers. Two unbounded buffers matched that shape:
- Main-process pendingData grew by string concatenation without bound
while the renderer could not receive (frozen, starved, mid-reload) —
main-heap bloat a renderer reload cannot clear. Now capped at 2 MB per
PTY: past the cap the buffered bytes are dropped and the entry stays
O(1) until the renderer ACKs again, then a droppedOutput sentinel is
delivered and the pane repaints from the authoritative main-owned
buffer snapshot (existing hidden-output restore path) instead of
continuing a stream with a silent gap.
- The renderer output scheduler capped only hidden-pane backlogs; the
foreground path could queue a visible pane's flood without bound when
the drain could not keep up. The 2 MB cap now applies to every
foreground enqueue branch too, with a foreground-specific skip notice.
Verified: new main-side cap test (starve → flood → sentinel → normal
flow resumes), renderer sentinel-to-snapshot-restore test, two
foreground scheduler cap tests; full pty/terminal-pane/pane-manager
suites (1981 tests) and typecheck pass.
Co-authored-by: Orca <help@stably.ai>
* Make replay-guard stall release probe-certified instead of time-based
The previous stall watchdog blindly released the input guard after 10s.
If a replay were genuinely still parsing on a starved machine, that
early release could leak xterm's auto-replies into the shell — and into
agent TUIs, where a leaked ESC reads as the user pressing Escape.
Replace the blind release with a probe: when a completion looks
overdue, enqueue an empty write behind the replay. xterm parses writes
in order, so every outcome is provably safe:
- probe parses after the replay completion ran: normal release already
happened; probe is a no-op.
- probe parses but the replay completion never ran: all replay bytes
have parsed, no further auto-replies can exist — the completion was
genuinely lost. Release + breadcrumb.
- probe never parses (bounded wait): the pipeline is wedged, and a dead
parser can never emit auto-replies, so releasing cannot leak input.
Release + breadcrumb naming the pane as needing recovery.
While the probe is pending — a slow-but-alive replay — the guard now
HOLDS instead of releasing early; that case is pinned by a regression
test.
Co-authored-by: Orca <help@stably.ai>
* Scale output backlog caps with the scrollback setting and breadcrumb drops
The 2 MB pending-output caps were flat, which risked dropping lines a
50k-row scrollback user would have retained. Both caps (main pendingData
and the renderer output queue) now derive from one shared policy:
max(2 MB, scrollbackRows x 120 chars) — 2 MB at the 5k default, 6 MB at
the 50k max. The main side reads the setting live via getSettings; the
renderer scheduler is configured where the terminal lifecycle already
reads the scrollback setting.
Every drop now records a rate-limited crash breadcrumb with dropped and
cap sizes (terminal_output_backlog_dropped in the renderer,
terminal_pending_output_dropped in main — no pty ids, session ids can
embed workspace paths). Field drop frequency and size decide whether the
cap constants need raising, replacing theory with data (#2836, #7017).
Backlog skip notices are now cap-agnostic since the limit varies.
Co-authored-by: Orca <help@stably.ai>
* Extract breadcrumb recording into a collection-safe leaf module
Playwright loads spec imports at collection time, and e2e specs import
terminal-module constants (e.g. terminal-attention.spec.ts pulls
POST_REPLAY_MODE_RESET from layout-serialization, whose chain reaches
replay-guard). The breadcrumb import added to the terminal modules made
that chain reach crash-diagnostics.ts, whose top-level import.meta.hot
and webview-registry import crash Playwright's transform
("ReferenceError: exports is not defined in ES module scope") — every
e2e shard failed at collection before running a single test.
Move recordRendererCrashBreadcrumb into crash-breadcrumb-recorder.ts
(type-only imports, no import.meta) and point the terminal modules and
their test mocks at it; crash-diagnostics re-exports for existing
callers. Full e2e suite collects again (262 tests / 94 files); unit
suites, typecheck, lint green. No runtime behavior change.
Co-authored-by: Orca <help@stably.ai>
* Add cross-terminal pipeline benchmark (DSR-fenced throughput + latency probe)
Run inside any terminal (Orca pane, iTerm2, Ghostty, Terminal.app, VS Code)
to measure its full byte path. DSR round-trip latency at idle and under a
paced agent-TUI load, plus fenced throughput over four deterministic
fixtures. The DSR fence forces 'all bytes parsed' before the clock stops so
xterm.js-class ingest queues can't flatter the result.
First piece of the terminal performance initiative's measurement rig.
Co-authored-by: Orca <help@stably.ai>
* Add terminal performance initiative plan
Working plan for the orca-performance branch: verified architecture
findings, workstreams (baselines, #7153 validation, term-speed-2 revival
with merge-scout numbers, stall fixes, flow control, rig extensions,
utilityProcess router, telemetry), benchmark protocol, sequencing, and
baseline-relative success criteria.
Co-authored-by: Orca <help@stably.ai>
* Add cross-terminal baseline results (Orca 1.4.91 prod vs Terminal.app vs Ghostty)
Headline: Orca DSR latency under 1MB/s agent-TUI load is p50 134ms / p99 292ms
vs 0.45ms (Terminal.app) and 0.21ms (Ghostty). Idle latency is fine (0.69ms
p50) — the problem is queueing under load, not the pipeline hop. agent-tui
fenced throughput: Orca 2.0 MB/s vs Terminal.app 37 MB/s, Ghostty 78 MB/s.
Co-authored-by: Orca <help@stably.ai>
* Add pipeline-loss decomposition benches (headless xterm + daemon ingest)
Both isolate layers of the 51x agent-tui gap found in baseline-jul02:
bare @xterm/headless parses agent-tui at 103 MB/s and daemon Session
ingest (emulator + pending-output recording + fanout) at 103 MB/s —
on the byte stream the full Orca pipeline delivers at 2.0 MB/s.
Parser and daemon are exonerated; the loss is in main per-chunk
processing, delivery/ACK pacing, or renderer layers above xterm.
Co-authored-by: Orca <help@stably.ai>
* Record baseline + decomposition findings in initiative plan
Co-authored-by: Orca <help@stably.ai>
* Add dev-build orca-performance bench result (confounded: dev mode, 282-col window, 3MB fixtures)
DSR under load p50 161ms — the #7139/#7150 branch does not move the
under-load latency class. Expected in hindsight: DSR replies are ordered
within the output stream, so the metric measures output-queue depth;
cooperative drain paces input responsiveness but cannot reorder the queue.
Shrinking the queue itself (producer flow control, task 6) and raising
agent-tui throughput (task 9) are the levers for this number.
Co-authored-by: Orca <help@stably.ai>
* Record dev-build #7153 check in findings log
Co-authored-by: Orca <help@stably.ai>
* Parse-clock high-priority terminal drains instead of fixed-nap dripping
Attribution (task #9): the drain loop wrote at most 2x16KB then slept
4/16ms regardless of parse speed — an isolation bench (new
pane-terminal-output-scheduler-throughput.bench.test.ts) measures that
drip at 1.9 MB/s background / 27 MB/s foreground against xterm's
~103 MB/s parse rate, matching the baseline-jul02 end-to-end numbers
(agent-tui 2.0 MB/s in prod 1.4.91).
Fix: high-priority (visible-pane) drains now re-arm on xterm's
parse-completion callback and carry 8 writes per tick; the isolation
ceiling rises 27 -> 117.6 MB/s (parse-limited). Background cadence is
deliberately unchanged (2 MB/s drip protects the focused pane; hidden
delivery is term-speed-2's job). DRAIN_TIME_BUDGET_MS still bounds
per-tick work, preserving #7139's cooperative-drain intent.
Validation: 621 scheduler/guard/pty tests green, typecheck clean.
Co-authored-by: Orca <help@stably.ai>
* Record task #9 attribution + parse-clock fix in findings log
Co-authored-by: Orca <help@stably.ai>
* Findings: 51x loss attributed to O(tail) retained-tail redraw path in main onPtyData
Co-authored-by: Orca <help@stably.ai>
* Window the retained-tail redraw path to the cursor's reach
Attribution (findings log 2026-07-03): main's onPtyData consumed ~93% of
the event loop under an agent-TUI flood, and the dominant term was
appendNormalizedToMultilineTailBuffer + finalizeRetainedTerminalRows
materializing ~2x tail-length row objects plus a per-row trailing-space
regex on every chunk — 0.888ms/chunk at the 2,000-line cap, on every
Claude-Code-shaped frame (cursor-up + erase-below).
The multiline algorithm now runs on a suffix window sized by the chunk's
maximum upward cursor excursion (plus the inherited redraw cursor and a
safety margin); the untouched prefix is shared by reference with a cheap
last-char trailing-space check to match the reference trim. Pathological
full-height cursor-ups fall back to the unwindowed implementation, which
is kept verbatim and exported as the reference for the 500-case
differential fuzz (retained-tail-redraw-window.equivalence.test.ts).
Micro-bench at a full 2,000-line tail: 0.888 -> 0.073 ms/chunk (12x).
1,415 runtime tests green, typecheck clean.
Co-authored-by: Orca <help@stably.ai>
* Add dev bench results: parse-clock and windowed-tail fixes
Co-authored-by: Orca <help@stably.ai>
* Record windowed-tail partial win + next-cycle recipe in findings log
Co-authored-by: Orca <help@stably.ai>
* Findings: remaining whale is the per-chunk blocked-reason check (~85% of onPtyData post-fix)
Co-authored-by: Orca <help@stably.ai>
* Throttle the terminal wait-blocked check off the PTY hot path
Post-windowed-tail attribution (findings log 2026-07-03): the blocked-
reason complex — two full-tail buildTerminalWaitText builds plus
toLowerCase and multi-pattern scans per chunk, existing only to stamp
waitBlockedAt — consumed ~85% of onPtyData's remaining cost (~700-790ms/s
under an agent-TUI flood).
The check now runs at a 50ms cadence over coalesced chunks (PTY chunk
boundaries are arbitrary, so coalescing preserves semantics), with a
trailing-edge timer so burst-final state is always evaluated, and an
immediate bypass when the incoming chunk (plus a 31-char split carry)
contains a prompt keyword — so actionable-prompt stamping stays
per-chunk-immediate while keyword-free flood frames skip the complex
entirely. Previous wait text is cached per pty instead of rebuilt, and
state is cleared at both pty teardown sites.
1,415 runtime tests green (including the cross-chunk prompt test, which
exercises the keyword bypass), typecheck and lint clean.
Co-authored-by: Orca <help@stably.ai>
* Findings + results: three stacked fixes unlock the pipeline (agent-tui 16x, DSR-load p50 161->18.8ms in dev)
Co-authored-by: Orca <help@stably.ai>
* Add producer flow-control design to initiative plan
Co-authored-by: Orca <help@stably.ai>
* Findings: revival branch green but perf-gated — daemon Session ingest regressed 103->40-48 MB/s (chain emulator restructure); merge blocked until blockedfix parity
Co-authored-by: Orca <help@stably.ai>
* Pre-filter daemon OSC/mouse scanners for introducer-free chunks
Skips the scan-tail copy and full-chunk walks when a chunk cannot contain
an OSC or private-mode sequence (single native includes() checks), with
split-sequence correctness preserved via explicit tail retention. Strictly
positive micro-optimization on the daemon per-chunk path; 641 daemon tests
green (1 pre-existing WSL failure unrelated).
Co-authored-by: Orca <help@stably.ai>
* Retract confounded daemon conviction; mandate load-controlled A/B protocol for the revival merge gate
Co-authored-by: Orca <help@stably.ai>
* Record A/B gate pass in findings log; add A/B result JSONs
Co-authored-by: Orca <help@stably.ai>
* Add producer-side PTY flow control (watermarks + protocol v19)
Main now pauses the actual PTY when a pane's renderer-pending backlog
crosses the 256KB high watermark and resumes once it drains below the
32KB low watermark (wide hysteresis band so a draining queue cannot flap
pause/resume per flush slice). node-pty pause() stops the pty fd read, so
the kernel/ConPTY buffer fills and a flooding shell blocks on write —
flood-induced buffered lag becomes shell blocking instead of unbounded
main-process buffering (terminal-performance-initiative §5).
Transport: new fire-and-forget pausePty/resumePty daemon notifications
(protocol v19; 18 added to PREVIOUS_DAEMON_PROTOCOL_VERSIONS), routed
DaemonServer -> TerminalHost -> Session -> subprocess pause()/resume().
LocalPtyProvider pauses node-pty directly. Router/degraded providers
forward; IPtyProvider gains optional pauseProducer/resumeProducer.
Safety invariants:
- Lost-resume failsafe: daemon Session auto-resumes 5s after a pause with
no matching resume; main re-asserts the pause at most once per 5s while
still above the high watermark, so a lost resume can never wedge a shell
and a sustained flood stays throttled.
- Resume on every teardown path: Session kill/exit/dispose/detach; main
releases on pty exit and on window-destroyed bookkeeping wipes; the
adapter owes paused sessions a resumePty on the next connect after a
socket drop.
- Providers without support (SSH relay, legacy protocol <= v18) no-op
silently, and the scrollback-scaled pending-output cap still bounds
main memory when pause is unavailable.
- Kill switch: PRODUCER_FLOW_CONTROL_ENABLED in ipc/pty.ts flips the
whole mechanism off in one line.
daemon-errors.ts is split out of types.ts to stay under the max-lines cap.
Tests: watermark transitions/hysteresis/re-assert (controller unit),
lost-resume failsafe + resume-on-kill/exit/dispose/detach (session),
notification routing + v18 gating + reconnect owed-resume (adapter),
direct pause/resume (local provider), and a flood test asserting pause
fires once, pending stays bounded at HIGH + one chunk, and resume fires
once after drain (ipc/pty).
Co-authored-by: Orca <help@stably.ai>
* Findings: flow control merged; definition-of-done accounting; prod verification re-scoped to packaged RC
Co-authored-by: Orca <help@stably.ai>
* Fix stray brace from revival merge in long-table-scroll-restore e2e spec (broke e2e transform in CI)
Co-authored-by: Orca <help@stably.ai>
* Prod verdict: v1.4.121-rc.0 bench — DSR-load p50 134->18.6ms (7.2x), agent-tui 2.0->11.2 MB/s, idle at Terminal.app parity; pipeline now cadence-bound
Co-authored-by: Orca <help@stably.ai>
* Recover terminal output delivery after system sleep
Root cause: main gates every pty:data send on a global + per-PTY
in-flight counter that only renderer ACKs decrement. If ACKs are lost
across a system suspend, the counters pin at the cap and every PTY —
old and newly created — is silently gated forever while output piles up
in pendingData. A focus-preserving display wake also fires no renderer
focus/visibilitychange events, so terminal wake recovery (and the WebGL
context-loss latch clear) never runs. Only a renderer reload recovered.
Three fixes:
- ACK-stall watchdog (src/main/ipc/pty.ts): if sends stay gate-blocked
for 10s with zero ACK progress while the renderer webContents is
alive, warn once, reset the in-flight delivery counters, and flush
held pendingData. Armed lazily on the first gate-blocked send and
disarmed by every ACK, so it can never fire under healthy heavy load.
- Renderer lifecycle reset now also zeroes the in-flight counters — a
reload/navigation destroys the renderer dispatcher, so outstanding
ACKs can never arrive and stale counters would gate the new renderer.
- System-resume wake IPC: main relays powerMonitor 'resume' as
system:resumed to live windows (plus forceRepaint); preload exposes
ui.onSystemResumed; the terminal wake-recovery hook runs the same
recovery path as window focus/visibilitychange.
Co-authored-by: Orca <help@stably.ai>
* VS Code head-to-head: Orca beats/ties 5 of 6 metrics (16x idle, 5x styles-stress, better p99); load p50 gap attributed to ACK window + timer-clamped drain cadence
Co-authored-by: Orca <help@stably.ai>
* Schedule zero-delay terminal drains via MessageChannel
Chromium clamps nested setTimeout(0) to ~4ms, stacking dead gaps onto
every parse-clocked drain tick; the explicit 4ms high-priority re-arm
interval added more. A posted message is still a macrotask — input and
paint are serviced between posts — so cooperative yielding survives
without the clamp. Generation-tokened cancellation; vitest keeps the
timer path (fake timers can't advance channel posts) plus a real-timer
smoke test for the channel path. Standing-queue target: VS Code's ~7ms
class (measured us 18.6ms, them 7.18ms, same rig).
Co-authored-by: Orca <help@stably.ai>
* Cut daemon and main PTY batch windows 8ms -> 2ms
At 9% pipeline utilization the DSR-under-load latency is fixed batching
windows, not queue depth (proved by the MessageChannel drain lever
moving nothing). Both hops charged an expected half-window per chunk;
2ms keeps burst coalescing at negligible IPC overhead (~500 msgs/s
worst case vs MB/s payloads).
Co-authored-by: Orca <help@stably.ai>
* Findings + tests: batch windows were the DSR-load gap (19->8.0ms dev); timing tests updated to 2ms windows
Co-authored-by: Orca <help@stably.ai>
* Fix PR CI and guard resume relay during shutdown
Co-authored-by: Orca <help@stably.ai>
* Chain e2e specs 6/6 green — gate x drain validation debt paid
Co-authored-by: Orca <help@stably.ai>
* Replace ack-stall watchdog with cumulative ACKs + solicited delivery resync
Design review: the 10s blind-reset watchdog decided correctness from a
wall-clock threshold. Rework piece 1 into a deterministic two-part design
(pieces 2 and 3 — lifecycle-reset counter zeroing and powerMonitor wake
IPC — are unchanged):
- Cumulative ACKs (TCP-style): the renderer dispatcher now tracks a
monotonic per-pty total of processed chars (terminal-pty-ack-gate) and
sends it on every ACK alongside the legacy per-chunk delta. Main keeps
per-pty sentChars/ackedChars and max-merges received totals — idempotent
and reorder-tolerant, so a lost ACK self-heals when any later ACK
arrives instead of becoming permanent in-flight debt. Provider
(SSH/daemon) backpressure is credited only the derived delta, clamped,
never negative. Main tolerates both payload shapes keyed by field
presence (dev hot-reload can mix renderer/main versions); totals reset
on pty exit and renderer lifecycle reset on both sides.
- Solicited resync (replaces the blind reset): when new pty data arrives
while that pty's delivery is fully gated and no probe is outstanding,
main sends pty:requestDeliveryResync; the renderer replies with its
cumulative totals and main reconciles via max-merge, then flushes held
pendingData. Event-triggered, verified-state recovery — no wall-clock
threshold decides correctness. The only timer is a 5s request/response
hygiene timeout that clears the outstanding flag and logs one
diagnostic warn per silent streak; it never mutates counters (a
renderer that cannot answer has dead IPC — reload is the only cure).
The 10s corrective watchdog is deleted.
Co-authored-by: Orca <help@stably.ai>
* Starting point: prior agent's garble differential fuzz harness
Three files recovered (were untracked) from a prior agent killed by API
outages, plus a trivial curly-brace lint fix in the op dispatcher so the
pre-commit hook passes:
- src/shared/agent-tui-ansi-fuzz-stream.ts (seeded agent-TUI byte-stream gen)
- src/shared/terminal-restore-parity-fixture.ts (renderer-parity fixture)
- src/main/daemon/headless-emulator-fidelity.fuzz.test.ts (suite 1: differential
HeadlessEmulator vs @xterm/headless reference on identical bytes)
Co-authored-by: Orca <help@stably.ai>
* Suite 1 findings: two new serialize round-trip bugs (B bold-loss, C cursor)
Scanned seeds 1..2000. Beyond the pre-documented serialize wrap-null-cell bug
(A, 27 seeds, tolerated), the fuzz surfaced two NEW real @xterm/addon-serialize
0.15.0-beta.287 round-trip defects, both of which garble a revealed hidden pane:
- Bug B (seeds 435, 770, 1321): serializing a dim cell followed by a bold-only
cell emits \x1b[1;22m; SGR 22 clears bold too, so restored bold is lost.
Minimal repro: '\x1b[2mA\x1b[22m\x1b[1mB' -> restored 'B' loses bold.
- Bug C (seeds 454, 1696): a final content row filled to the right margin leaves
xterm wrap-pending; the serializer's relative cursor restore lands one column
short. Minimal repro: '0123456789\x1b[3;5H' at cols=10 -> cursor x=3 not x=4.
Both isolated to pure serializer replay (no Orca preamble), confirming upstream.
Parity fixture verified faithful to the renderer pane's buffer options. Each is
pinned as a standalone it.skip repro; full evidence + classification in
notes/garble-fuzz-divergences.md. Seed 113 (handoff's DECSC/DECRC case) does not
diverge on the current harness. No production code changed.
Co-authored-by: Orca <help@stably.ai>
* Add perf prerelease update check modifier
Co-authored-by: Orca <help@stably.ai>
* Suite 2: hidden-reveal seq-reconciliation fuzz + two new snapshot bugs (D, E)
Property-tests the reveal seq-reconciliation byte-stitch (getChunkDataAfterSnapshot
/ reconcileChunkAgainstRestoredSnapshot in pty-connection.ts), mirrored exactly:
N=200 seeded hide/reveal scenarios with a rich agent-TUI hidden prefix snapshot
and an append-only racing tail, chunked with seq/rawLength meta, seq-domain
restarts, unmetered chunks and droppedOutput markers. Asserts snapshot-at-S +
reconciled tail == snapshot-of-everything (seq-neutral) and == always-visible
(end-to-end). Runtime ~5s at 200; FUZZ_ITERATIONS override documented.
Two NEW real snapshot-limitation garbles found while building it, both distinct
from suite 1's serialize bugs and pinned as standalone it.skip repros:
- Bug D: the DECSC saved-cursor register is not serialized. A hidden TUI that
saves the cursor (ESC 7 / CSI s) and restores it on reveal (ESC 8 / CSI u)
lands the restore at home. Repro: 'AB\x1b7\x1b[4;10HCD' + '\x1b8X' -> 'XB' vs 'ABX'.
- Bug E: a snapshot taken mid-escape-sequence (a PTY read split an escape) drops
the partial sequence (it's parser state, not buffer), so the tail's
continuation renders literal. Repro: 'AB\x1b[3' + 'mCD' -> 'ABmCD' vs 'ABCD'.
Fired on ~24% of the corpus (tolerated + counted via prefixEndsMidSequence).
The append-only-tail design isolates seq reconciliation from these and the Bug C
cursor cascade. Full evidence + fix directions in notes. No production changes.
Co-authored-by: Orca <help@stably.ai>
* Suite 3: 25-cycle park/reveal drift e2e test
Extends terminal-hidden-view-parking.spec.ts with a deterministic 25-cycle
park->reveal test on a static rich alt-screen TUI frame (box drawing, SGR
colors, wide CJK/emoji). Baselines against the frame after the first snapshot
restore (so both sides pass through identical machinery — the alt-screen restore
correctly drops normal-buffer scrollback, which is contract not garble), then
asserts every subsequent reveal reproduces it byte-for-byte with no accumulated
drift and no hidden-skip banner. Exercises the real renderer teardown +
HeadlessEmulator snapshot restore + PTY reattach path the fuzz suites model in
isolation. Passes in ~29s (electron-headless, workers=1).
Co-authored-by: Orca <help@stably.ai>
* Fix two serialize round-trip bugs garbling hidden-terminal snapshot restore
BUG B (addon patch): @xterm/addon-serialize's SGR diff emitted bold/dim set
params before the shared intensity reset 22, so "1;22" wiped a freshly set
bold and a bare "22" dropped a still-set bold/dim. Patched via pnpm
patchedDependencies (config/patches) to diff bold+dim as one intensity
group with the clearing 22 emitted first. Other flag pairs (4/24, 3/23,
7/27, ...) have dedicated resets and were verified unaffected.
BUG C (Orca-side hardening): the addon restores the cursor with relative
moves computed from where it assumes replay leaves the cursor; a final row
filled exactly to the right margin leaves replay wrap-pending and the
restore lands one column short. New shared
serializeWithAbsoluteCursor appends an absolute CUP from the source
terminal's authoritative cursor at every restore/replay serialize site
(daemon/runtime HeadlessEmulator.getSnapshot, renderer mobile snapshot
serializer, shutdown layout capture). It skips empty snapshots and
wrap-pending sources so it never changes already-correct behavior.
Round-trip repros + non-regression coverage in
src/main/daemon/terminal-snapshot-serialize-roundtrip.test.ts (verified
failing with the fixes stashed). buildRehydrateSequences extracted to its
own module to keep headless-emulator.ts under the max-lines budget.
Co-authored-by: Orca <help@stably.ai>
* Gates: tolerate+count Bugs B/C in deep mode; drop inverse from reconciliation tail
- Fidelity suite: add snapshotHasSelfCancellingBoldReset (Bug B) and
isMarginWrapPendingCursorOffByOne (Bug C) predicates so the corpus tolerates +
counts them like Bug A. FUZZ_ITERATIONS=2000 is now green (~113s) and fails
only on genuinely new divergences; each tolerance keeps its <50% degeneracy
guard. Default 300 unchanged (~17s).
- Reconciliation suite: drop SGR 7 (inverse) from the append-only tail. Inverse
marks trailing blanks with an inverse-fg the serializer round-trips slightly
differently by capture depth — a Bug-B-class serialize nuance, not seq
reconciliation. FUZZ_ITERATIONS=1000 is now green; default 200 unchanged.
- Notes updated: every bug class is both pinned (skipped repro) and tolerated in
its corpus; combined default runtime ~19s.
Regex uses String.fromCharCode(27) to stay oxlint no-control-regex clean.
Co-authored-by: Orca <help@stably.ai>
* Keep RC update checks off perf prereleases
Co-authored-by: Orca <help@stably.ai>
* Fix snapshot DECSC register loss (Bug D) and mid-escape boundary drop (Bug E)
Bug D: the serialized screen cannot carry the VT100 DECSC saved-cursor
register, so a hidden ESC 7 followed by a post-reveal ESC 8 restored to
home and clobbered live cells. The snapshot epilogue now re-saves at the
source's saved position before the final absolute CUP
(readSavedCursorRegister + serializeWithAbsoluteCursor; the active
buffer's own register, so alt screens carry theirs). Position-only by
design; never-saved terminals are left untouched.
Bug E: a PTY read ending mid-escape leaves the sequence in the emulator's
parser, so serialize dropped it and the racing tail's continuation bytes
rendered literally after reveal (~24% of the fuzz corpus). The emulator
now tracks the unparsed trailing partial at ingest
(terminal-partial-escape-tail.ts, committed post-parse like the mouse
mirror) and ships it as TerminalSnapshot.pendingEscapeTailAnsi.
applyMainBufferSnapshot writes it LAST, after POST_REPLAY reset — any
later ESC would abort the dangling sequence. Seq accounting is unchanged:
the tail is a suffix of bytes the snapshot seq already counts, so
reconcile slicing needs no adjustment.
Fuzz suites: unskip the Bug B/C repros (fixed on this branch) and the new
D/E repros; remove the B/C/E tolerance predicates so regressions fail
loudly. Only Bug A (upstream wrap null-cell) stays tolerated + counted.
Green at FUZZ_ITERATIONS=2000 (fidelity) and 1000 (reconciliation).
Co-authored-by: Orca <help@stably.ai>
* Count suffixed RC tags (rc.N.perf) in the shared rc counter — second suffixed cut collided with the first
Co-authored-by: Orca <help@stably.ai>
* Classify suffixed rc tags (rc.N.perf) as rc telemetry identity in release builds
The build-identity guard only knew vX.Y.Z and vX.Y.Z-rc.N, so suffixed
perf RCs cut fine but every platform build refused the tag and the
releases published empty.
Co-authored-by: Orca <help@stably.ai>
* Cut the hidden-restore flood feedback loop (A) + query carve-out on drops (B)
(A) Under a foreground flood, the hidden-output-restore loop re-fetched
snapshots endlessly: each synchronous applyMainBufferSnapshot starved ACK
processing, main pinned at the in-flight cap, dropped at the pending cap,
and every droppedOutput/modelRestoreNeeded marker re-armed another
restore until the flood ended (rc.7.perf DSR timeouts).
- Restore loop: a foreground live-chunk queue overflow now abandons the
restore immediately (the stream is outrunning snapshot fetch+replay),
with a 3-iteration hard cap + lifecycle warn as backstop.
- Re-arm gate: drop markers/sentinels and reconcile seq-gaps on a visible
pane during its own in-flight/just-abandoned restore no longer re-arm;
live bytes write through and ONE deferred repaint (2s after the last
backpressure signal) heals the gap. Hidden-pane gate semantics are
unchanged.
- Query salvage: discarding queued restore bytes (overflow/refetch) now
extracts DSR/CPR/DA/OSC-color queries and replays them to xterm so
replies still flow.
(B) Main-side: dropOversizedPendingPtyData carves reply-eliciting query
sequences out of the dropped buffer (and out of post-drop latched data,
bounded) and ships them on the droppedOutput sentinel, so DSR probes
survive bulk drops. Query scanning moved to
src/shared/terminal-reply-query-extraction.ts, shared verbatim with the
renderer's hidden-startup query extraction.
Co-authored-by: Orca <help@stably.ai>
* ACK terminal output at parse-drain, not dispatcher enqueue (C)
The renderer credited main's per-PTY in-flight window the moment a
pty:data chunk entered the dispatcher, so the 512KB window meant "bytes
received", never "bytes parsed". Under flood the renderer write queue
grew unbounded behind instant ACKs; main saw no backpressure, crossed
the pending cap, and bulk-dropped output (rc.7.perf DSR timeouts).
Crediting is now parse-deferred: each delivery carries a fire-once
credit (deliverPtyDataWithDeferredAck); the pane's first scheduler write
claims it (writeTerminalOutput.ackCredit) and the output scheduler fires
it when the bytes are consumed — after terminal.write in the
parse-clocked drain, or on ANY discard path (backlog cap replacement,
discardTerminalOutput, disposed-terminal drops, flush recovery).
Deliveries that never reach the scheduler (reconcile drops, restore
queueing, pre-mount eager buffer) settle at handler return, so the
invariant holds: every delivered chunk credits exactly once, parsed or
discarded. E2E ack-gate hold/release and delivery-resync semantics are
unchanged (all crediting still routes through ackPtyData).
Main-side equilibrium: with ACKs at parse cadence, in-flight becomes
true backpressure — pendingData stays near the 256KB producer-pause
watermark, far under the >=2MB drop cap, so bulk floods block the shell
(node-pty pause) instead of dropping.
Co-authored-by: Orca <help@stably.ai>
* Synthesize salvaged query replies directly instead of replaying into xterm
The 10MB dev bench proved the write-back salvage insufficient: a
pending-cap drop always triggers a snapshot restore, whose replay guard
swallows xterm auto-replies and whose discardTerminalOutput races away
still-queued query writes — the salvaged DSR died both ways and the
fence still timed out.
Salvage now answers directly on the input path (immune to both): CPR
(CSI 6n) from the live buffer via transport.sendInput, DA1 with the
renderer's canned response, OSC color probes via the existing direct
responder. Rare queries (DECRQM, DA2) keep the best-effort xterm
replay.
Co-authored-by: Orca <help@stably.ai>
* Untrack branch-added bench result JSONs (20 files); keep numbers in the findings log
Files stay on disk; main's 7 pre-existing results are untouched.
Co-authored-by: Orca <help@stably.ai>
* Branch guide: document merge-not-rebase sync strategy and conflict pattern
Co-authored-by: Orca <help@stably.ai>
* Merge origin/main (#7316 tab-strip click-vs-drag fix); adapt #7290 recovery-reload tests to this branch's dual did-finish-load listeners
The three tests grabbed the FIRST did-finish-load listener; on this branch
the renderer delivery-gate reset registers before the orphan sweep, so the
sweep tests exercised the wrong handler (one failing, two vacuously green).
They now fire all listeners like a real reload.
Co-authored-by: Orca <help@stably.ai>
* Fix branch CI lint: split pane-interaction functions out of artificial-opencode-terminal-load.spec (815>800 lines), modernize perf-html-report script
No max-lines disable per repo rules; extracted to
artificial-opencode-pane-interactions.ts. toReversed() and
import.meta.filename replace reverse()/fileURLToPath.
Co-authored-by: Orca <help@stably.ai>
* Fix Windows update-relaunch killing the live terminal daemon
On a Windows update relaunch the daemon can be wedged past every RPC
budget (final checkpoint flush + installer/AV disk pressure), so the 3s
health check AND the 5s session-list hello both time out while sessions
are still alive - and the launcher failed closed, killing the daemon and
every terminal session it owned.
- Adopt an unresponsive daemon whose pipe still accepts a raw
connection; a new rejected health state keeps replacing daemons that
answered and refused the handshake (never adoptable).
- Give Windows pid files a real startedAtMs (daemon self-reports it in
the ready IPC message) and verify it via CIM CreationDate piggybacked
on the existing command-line query, so the pid-recycling guard is no
longer inert on win32.
- Only delete legacy daemon pid/token files when the pid-file process is
provably dead; deleting a live daemon''s token made its sessions
permanently unadoptable after a protocol bump.
- Capture agent resume records every 60s in the renderer (skipping
unchanged records) so hard kills still leave a fresh resume record.
* Heal blank terminals when main→renderer push delivery dies (renderer-pull delivery watchdog)
Field evidence (v1.4.121-rc.0 debug snapshot, 2026-07-06): a wedged window
held 530,115 un-ACKed in-flight chars — one PTY pinned at the 512KiB per-PTY
high water plus a fresh terminal's 245-char prompt that was sent and never
consumed — while the user ran the snapshot over invoke from that same window.
Main→renderer push delivery (pty:data and every sibling channel) was dead;
renderer→main→renderer invoke was alive. Upstream precedent for
one-directional IPC death: electron#37067 (suspected Mojo pipe disconnect,
stalled as need-info). Every terminal goes blank, new terminals are born
blank, and only a renderer reload recovered.
The existing recovery layers cover the OTHER variants of this bug family and
structurally cannot reach this one:
- The xterm write-pipeline sync-throw guards, output-buffer caps, and
probe-certified replay-guard release (#7150 family) run only after bytes
arrive in the renderer — here they never do. (The pending cap did work as
designed in the field: ~2.1MB pendingDroppedChars, bounded main heap.)
- Cumulative ACKs self-heal lost ACK messages and the solicited delivery
resync reconciles verified totals (4647df86a; #7260 on main) — but the
resync probe, the powerMonitor wake relay, and the droppedOutput restore
markers all ride main→renderer push, the direction that is dead. The
probe's unanswered path deliberately only logs.
This adds the missing lane, renderer-initiated and ridden entirely over
invoke — the direction the field snapshot proved alive:
- terminal-delivery-watchdog.ts: 15s heartbeat, free while output flows.
Hot-path cost is one Map upsert per received chunk; a tick does no IPC
unless the terminal plane was silent for the whole interval and a PTY
still expects delivery. Two consecutive silent ticks with main reporting
ACK-starved in-flight confirm the wedge; heals are one-shot per 60s
cooldown so a persisting wedge cannot repaint-storm.
- pty:reportRendererDeliveryState (invoke): always max-merges the renderer's
cumulative processed totals (a free extra repair lane for the lost-ACK
variant); with heal:true — and only after main has itself seen ≥10s of ACK
silence — writes off bytes the renderer provably never received
(received ≤ acked < sent; a received-but-unparsed backpressure window is
never written off), drops that PTY's pendingData (snapshot covers
everything ≤ markerSeq, hidden-drop parity), credits provider flow
control, and returns restore markers in the reply.
- The renderer re-attaches all push listeners (cures a detached-listener
variant outright; a safe no-op against a dead channel) and routes the
pulled markers through the existing pty:modelRestoreNeeded machinery —
panes repaint from the main-owned buffer snapshot with zero push
delivery involved.
- Field discrimination built in: the heal warn logs
ipcRenderer.listenerCount('pty:data') (listener detached vs channel dead)
with the full delivery snapshot, so the next occurrence names the root
cause without asking the user to run anything in a console.
Repro harness: the exposeStore-gated __terminalDeliveryWatchdog hook
blackholes pty:data ahead of the dispatcher — the field failure in
miniature (no receive count, no ACK credit, no dispatch).
terminal-push-delivery-loss-recovery.spec.ts proves the wedged output
repaints while the blackhole is still engaged and live flow resumes after
release, with no reload. Unit suites pin the watchdog state machine
(zero IPC under flow, two-tick confirm, cooldown), the dispatcher reattach
seam, and the main-side write-off semantics.
Perf: nothing added to main's send/flush path; the renderer data path gains
one integer/Map update per chunk; idle cost is one ~100-byte invoke per 15s
only during total terminal silence. Terminal perf e2e suite (typing
latency, redraw freeze, output scheduler, hidden TUI restore, artificial
opencode load) passes on this change; no watchdog activity occurs under
ack-gate pressure scenarios because receive-progress gates the heartbeat.
Co-authored-by: Orca <help@stably.ai>
* Expose the hidden-yet-visible delivery-gate contradiction in the debug snapshot
The v1.4.124-rc.2.perf blank-terminal field snapshot showed a different
state than the v1.4.121 transport wedge: no delivery gating at all
(ackGatedFlushSkipCount 0, in-flight 38KB, far under every cap) but TWO
ptys hidden-delivery-gated with 78MB dropped as hidden. The aggregate
counters cannot say whether the pane the user was staring at was one of
the gated ones — the one number that separates "normal background
dropping" from "main is starving a visible pane because the reveal
unmark never fired".
Add hiddenDeliveryGatedVisiblePtyCount / hiddenDeliveryGatedActivePtyCount
(overlap of the gate's hidden set with the renderer's visible/active
reports — a contradiction that must be zero) to the delivery debug
snapshot, and a once-per-minute warn when hidden-gated bytes are dropped
for a pty the renderer reports visible or active, with the full snapshot
attached. Zero cost outside the debug read and the already-dropping path.
Co-authored-by: Orca <help@stably.ai>
* Unlatch the hidden-delivery gate when user input disproves a stuck document.visibilityState
macOS occlusion tracking can wedge document.visibilityState at 'hidden'
after display sleep and never fire another visibilitychange. The hidden-
delivery gate then keeps dropping renderer-bound bytes for panes the user
is looking at (field snapshot 2026-07-06, v1.4.124-rc.2.perf: 78MB dropped
across 2 pane-level-visible ptys with a fully healthy transport), and every
recovery path (window focus, system-resume relay, backlog recovery) re-ran
syncHiddenRendererPtyDelivery only to recompute the same stale predicate —
nothing could ever clear the gate. The user sees a frozen terminal; typing
echo is dropped in main; only a reload recovers.
Real user input while the document claims hidden is a physical
contradiction: keystrokes and clicks only reach a focused, on-screen
window. stale-document-visibility.ts latches that proof, runs each pane's
existing visibilitychange resync (gate unhide + hidden-output snapshot
restore), and hands authority back to the occlusion tracker on the next
genuine visibilitychange. No timers; the failure bias is safe — a wrong
latch can only restore pre-gate delivery cost, never drop bytes. Hot path
unchanged: the foreground predicate still returns on the same single
comparison while the document is visible.
tests/e2e/terminal-stuck-occlusion-recovery.spec.ts pins the wedge
(visibilityState pinned hidden -> output dropped, not painted; the
hiddenDeliveryGatedVisiblePtyCount field discriminator reads >0) and the
recovery (one Shift keypress repaints the missed output from the main-owned
snapshot, no reload, while visibilityState still reads hidden). Negative
control verified: the spec fails without this fix. Typing-latency perf
gate passes; terminal-pane unit suites 366/366.
Co-authored-by: Orca <help@stably.ai>
* Add a one-paste terminal freeze report: __orcaTerminalFreezeReport()
Every field report of the frozen-terminal family so far has needed
follow-up asks (console output, main logs, second snapshots) because each
capture showed one process's counters at one instant. This makes a single
DevTools command sufficient: `await window.__orcaTerminalFreezeReport()`
returns renderer state (document.visibilityState + the stale-visibility
override, pty:data listener count, delivery-watchdog totals), main's debug
snapshot extended with a per-pty delivery table (sent/acked/pending, hidden
vs visible-set membership, last send/ACK ages, window focus flags, power
suspend/resume ages, app version), and bounded breadcrumb rings from BOTH
processes recording the transitions that matter: gate marks/unmarks,
visibilitychange and stale-visibility latches, watchdog stalls and heals,
restore markers, heal write-offs, and renderer lifecycle resets (so "user
already reloaded" is visible in the history).
Costs stay off the data path: breadcrumbs record only rare transitions into
a 100-entry ring with same-kind coalescing (a flood costs one slot per
second); the per-pty table is built only when the snapshot is read; the
per-send bookkeeping adds one Date.now() to existing accounting writes. Pty
ids are redacted to their `@@` suffix because daemon session ids embed
worktree paths. The report assembles over invoke IPC — the direction proven
alive in every observed wedge — and a failing invoke is captured as data
instead of sinking the report.
The stuck-occlusion e2e now also pins the report end-to-end: after the
wedge + keystroke recovery, the report must carry the stale-visibility
latch and gate transitions in the renderer ring, gate-mark/unmark in main's
ring, and a populated per-pty table. Suites: pty.test.ts 258, terminal-pane
1705, shared ring 5; typing-latency perf gate passes.
Co-authored-by: Orca <help@stably.ai>
* perf(daemon): keep-tail thin hidden panes' stream so agent floods never bury typing (STA multi-workspace lag)
Hidden panes are exempt from pendingData flow control (main gate-drops
their bytes after ingestion), so N background agents ran unbounded ahead
on the one shared daemon->main stream socket (measured 192MB user-space
backlog) and visible-pane echo waited FIFO behind it — typing appeared
seconds late whenever several agents burst on a loaded machine
(8x512KB/s + 12 CPU spinners: p50 293ms fix-off; 12x1MB/s: 6.1s).
Mechanism (replaces producer pacing — no reveal catch-up, ever):
- Shallow socket write gate (128KB) + per-session fairness bypass bounds
echo latency by construction; kernel-flush refill sentinel keeps held
bulk draining at full speed (drain-only refill capped at ~8MB/s).
- Backgrounded sessions' queued output is keep-tail dropped (newest
512KB kept, in-order dataGap replaces the middle); a ~2MB GLOBAL
budget shrinks per-session keep-tails (floor 64KB) so a worktree
switch never waits behind the aggregate. Reply-eliciting query bytes
(DSR/DA/OSC probes) are salvaged from dropped spans.
- Notifications are structurally lossless: the daemon runs the same
shared scanners main uses (bell/OSC 133/pr-link/2031) over every byte
BEFORE drop decisions and relays facts in byte order; ordered
background markers hand scan authority back and forth, seeded with the
emulator's partial escape tail so a sequence split across the handoff
neither phantom-fires nor goes missing. Titles/agent-status stay
main-side (kept-tail convergent).
- Main: background = hidden AND no remote view subscriber (a live
mobile/web view is never thinned); on dataGap main resets cross-chunk
parse carries, drops the headless mobile mirror, and reuses the
hidden-drop model-restore marker.
Wire: three new stream events, tolerated within protocol v19 (old mains
ignore unknown events; old daemons never see the trigger). Kill
switches: ORCA_DAEMON_BACKGROUND_STREAM_DROP=0,
ORCA_DAEMON_SHALLOW_SOCKET_GATE=0.
A/B (pnpm bench:multi-workspace-typing): 8x512KB/s + 12 CPU workers
p50 293ms/p90 647ms -> 15/21ms (= baseline); 12x1MB/s 6,146ms -> 20ms;
light loads unchanged; zero missing echoes. Latin hidden-restore e2e
green (probe-verified aggregate-drain root cause). New deterministic
repro harness: tests/e2e/terminal-multi-workspace-typing-latency.spec.ts
+ CPU pressure workers.
Co-authored-by: Orca <help@stably.ai>
* diag(terminal): breadcrumb WebGL context-loss/atlas + wake triggers into freeze report
Silent instrumentation (memory ring only, no new console lines) so the next
post-wake garble report attributes itself. Adds:
- shared/terminal-webgl-diagnostics.ts: lib-safe sink so pane-webgl-renderer
(lib) can record without importing the components-layer ring; wired to the
ring in terminal-freeze-breadcrumbs.
- webgl-context-loss crumb at onContextLoss, webgl-atlas-reset crumb at the
atlas registry reset — the pair that distinguishes 'atlas corrupted' from
'missed repaint'.
- wake-recovery:<source> crumb (focus/visibilitychange/system-resumed) with the
clearGlyphAtlases decision; source in the kind so distinct triggers don't
coalesce.
- per-pane WebGL state (getAllPaneRenderingDiagnostics) in the freeze report.
Gates: typecheck 0 errors; terminal suites 328 files pass; oxlint clean.
Co-authored-by: Orca <help@stably.ai>
* fix(lint): use Number.parseInt/parseFloat in terminal-view-attributes
oxlint unicorn(prefer-number-properties) flagged 24 global parseInt/parseFloat
calls in the terminal-view-attributes feature (
|
||
|
Jinwoo HongandOrca
|
9bed9bbd34 |
perf(ssh): bound relay bulk-stream backlog so PTY echo is not head-of-line blocked (#7601)
Co-authored-by: Orca <help@stably.ai> |
||
|
5b5ce6cef8 |
Harden reliability gate policy and register merged terminal regression tests as gates (#7295)
Co-authored-by: Orca <help@stably.ai> Co-authored-by: brennanb2025 <delta-eng@stably.ai> |
||
|
Brennan BensonandOrca
|
1282f5c2da |
Add reliability gate manifest (#7001)
Co-authored-by: Orca <help@stably.ai> |
||
|
Jinwoo HongandOrca
|
8afa84af82 |
Fix terminal switch input lag from daemon session listing (#7002)
Co-authored-by: Orca <help@stably.ai> |
||
 
|
9694830764 |
Document headless Linux server setup (#6802)
* docs: add headless Linux server guide * docs: add ldd/appimage-extract tip for diagnosing missing libraries Salvaged from #6817 before closing it as a duplicate. Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Jinwoo-H <jinwoo@stably.ai> Co-authored-by: Orca <help@stably.ai> |