mirror of
https://github.com/stablyai/orca.git
synced 2026-09-21 16:02:20 +00:00
2b19f21adab5907697ef76ce429bafcd2cfea9ec
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Neil
|
a5d6114baf |
fix(ssh): stop pane adoption certifying a death from the relay's not-found union (#18531)
* fix(ssh): stop pane adoption certifying a death from the relay's not-found union
`attachStablePaneOwner` was the last reader that synthesised a runtime exit
from a reattach refusal, and it published code `0` — which
`orca-runtime-on-pty-exit` records as `rememberPtyLivenessVerdict(exited)`, a
death certificate whose only legitimate writer is a host-delivered exit frame.
The refusal it acted on is a union. `pty.attach` answers `PTY "<id>" not found`
both for a pid the relay probed with `isProcessAlive` and for an id its session
map simply never had — which, because ids carry a per-start mint epoch, is every
id minted before a relay restart, checked against nothing. So a relay restart
plus a reconnect certified a shell that was still running under the old daemon's
orphaned process tree, retired the pane binding, and cold-started a second agent
onto the same transcript. The sibling `handlePtyReattachFailure` has always
refused to certify from that union; this path did not.
- The relay marks the one refusal it backed with a liveness check
(`PTY_ATTACH_PROVEN_EXITED_MARKER`). The marker is additive, so an unmarked
answer — including an older relay's — stays ambiguous, which is the safe
direction.
- The client mints that half as `SshPtyProvenExitedOnRelayError`, a subclass so
every existing `isSshPtyAbsentFromRelayError` consumer is unchanged.
- Pane adoption publishes `UNVERIFIED_PROCESS_EXIT_CODE` (-1), the sentinel its
sibling publishes, and passes `hostExitConfirmed` only for evidence that
observed the process: the marked relay refusal, or `SessionNotFoundError` from
the registry that owns the PTY. The ambiguous half now records `unverifiable`
instead of `exited`.
- The gone-branch keys on the error type rather than the bare `PTY ".+" not
found` text, so an untyped string can no longer authorise abandoning a
binding — the discriminator `pty-connect-limits.ts` already documented.
Refs docs/reference/ssh-execution-boundary.md
* test(pty): make the pane-adoption fixtures throw what real providers throw
These four fixtures rejected with bare `new Error('Session not found: ...')` and
`new Error('PTY "..." not found')`. No provider produces either untyped:
`local-pty-spawn` and `decodeDaemonResponseError` both mint
`SessionNotFoundError`, and the SSH reattach path types the relay's wire text
before any pane sees it. Fixtures that skip the type were the reason a
message-shaped gate looked adequate.
The exit-code expectations move with it: the pane path now publishes the -1
stop sentinel plus `hostExitConfirmed`, so a certificate follows the evidence
rather than a synthesized zero.
|
||
|
Neil
|
f35015d0c8 |
fix(ssh): measure pane idleness in the unit the sweep's kill operates on (#18415)
The orphan-relay-PTY sweep authorizes `pty.shutdown { immediate: true }`, which
runs `forceKillPosixPtyProcessGroups`: collect every process group on the pane's
tty, then `killpg` each one. The blast radius is therefore (groups on the tty) x
(members of those groups, wherever they are). The idleness evidence measured only
the first factor, so three shapes read as idle and were SIGKILLed:
- with job control off (`set +m`) a background job keeps the SHELL's pgid, so the
tty carries exactly one process group and that group is running the user's build;
- a child that drops the controlling terminal (`ioctl(TIOCNOTTY)` without `setsid`)
keeps the pgid, reports `tpgid == -1`, and never appears in `ps -t <tty>`;
- a double-forked grandchild keeps the pgid and tty but reparents to pid 1, so the
`ppid` walk cannot reach it and the named-process backstop never fires.
`shellOwnsEveryTtyProcessGroup` now also requires the shell's own process group to
hold no other member anywhere in the table, indexed in the same single pass. A
pids-per-tty set would catch the first and third but not the second, which is why
the count is pgid-wide rather than tty-scoped. The wire field keeps its tty-shaped
name: the value only ever became stricter, so an old client skips more, never less.
Second, unrelated-in-mechanism but same file family: `foregroundSkipReason` summed
`capturedAgeMs + evidenceAgeSinceListingMs` without validating either. A non-numeric
`capturedAgeMs` makes the sum `NaN`, and `NaN > 5000` is false, so a malformed record
PASSED the freshness gate and proceeded toward the stop — the one place in the file
that defaulted toward kill. Nothing validated it on this path
(`mapSshPtyProcessList` checks the ownership fields and spreads the rest through;
`PtyProcessListAdmission` is not on the sweep path). It now runs
`isForegroundProcessEvidence` and fails closed.
Verified on real Linux, not only in mocks: a container drives `bash -i` on a real
pty, builds each construction, runs the real publisher and planner, and then calls
the real `forceKillPosixPtyProcessGroups`. Before, all three published
`shellOwnsEveryTtyProcessGroup: true`, planned SWEEP, and the planted pid was gone
after the signal. After, all three skip and survive, and an idle shell is still
reclaimed.
Residuals are written down at the predicate and in ssh-execution-boundary.md: the
capture is a snapshot (bounded by the evidence-age budget, not removed), and a
process the host's own `ps` cannot enumerate stays unobservable while `killpg`
still reaches it.
|
||
|
Neil
|
946627f2ce |
fix(runtime): route runtime filesystem commands by resolved execution host (#18325)
`ResolvedRuntimeFileTarget` carried `connectionId?: string` and no host id, so `undefined` spelled three different answers at once — "runtime: host", "unresolved" and "genuinely local". Its sole resolver read `store.getRepo(worktree.repoId)?.connectionId` and never looked at `worktree.hostId`, which outranks every repo row, so one arbitrarily chosen row decided the execution host for ~30 filesystem dispatches. This is #18307's defect in the same file family; it was deliberately left out of that PR rather than doubling an already-36-site diff. The target now carries `executionHostId: ExecutionHostId` (never null, never optional), resolved through `resolveWorktreeHostRouting` — the same adapter #18307 added — and dispatched through #18296's `resolveFilesystemRouteForHost`. Dispatch sites call `requireRuntimeFileProvider`, where `null` means exactly one thing: the host is `local` and the read happens here. Four answers that used to collapse into one: - `ssh:x` with a rival row on `ssh:y` — routes to x. Previously the first row won. - `local` with a surviving `connectionId` — a row contradicting itself; no SSH connection is handed out. - `runtime:<env>` — throws `ExecutionHostNotDispatchableError`. Its repo row's connection names a target in the *server's* namespace; reading it here reaches a same-named target on this client. - rival rows disagreeing with no worktree host — `worktree_execution_host_unresolved`, matching the launch and Git paths rather than guessing a row. Two further reads stop degrading. `assertRuntimeFileMutationExpectation` recomputed the host from `connectionId`, so a client's host expectation could pass against a host the workspace never named; it now compares the resolved host. And the cross-workspace terminal tap coalesced `knownWorkspaceTarget?.connectionId ?? connectionId`, so a sibling workspace resolved as `local` inherited the origin worktree's SSH target and statted a local path on the remote box; a non-optional host id replaces rather than coalesces. An unreachable SSH host still throws `SSH_FILESYSTEM_PROVIDER_UNAVAILABLE_MESSAGE`; loss of contact is never evidence of locality (docs/reference/ssh-execution-boundary.md). Quick-open listing and path search keep degrading to empty for an unreachable host — that is a false negative, not a local answer — and now do so only for a host that really is remote. The whole `runtime-file-commands-*` family carries `@ts-nocheck` from a mechanical class split, so removing the field could not raise the compile errors that made #18307 safe. `runtime-file-command-target.ts` is deliberately checked, and a ratchet test stands in for the errors the family cannot produce. No wire change: `ResolvedRuntimeFileTarget` is main-process internal, and the SSH watcher-release and grant keys are byte-identical to before. |
||
|
Neil
|
d5750648c2 |
fix(runtime): route runtime Git by resolved execution host, not repo connectionId (#18307)
`RuntimeGitTarget` carried `connectionId?: string` and no host id, so `undefined` spelled three different answers at once — "runtime: host", "unresolved", and "genuinely local". Its sole resolver read `store.getRepo(worktree.repoId)?.connectionId` and never looked at `worktree.hostId`, which outranks every repo row, so one arbitrarily chosen row decided the execution host for 36 downstream dispatches. The target now carries `executionHostId: ExecutionHostId` (never null, never optional), resolved through the shared rule that landed with #17909/#17919 and dispatched through the host-keyed routes from #18296. Dispatch sites call `requireRuntimeGitProvider`, where `null` means exactly one thing: the host is `local` and the command runs here as free functions. Four answers that used to collapse into one: - `ssh:x` with a rival row on `ssh:y` — routes to x. Previously the first row won, which is the reproduced cross-host leak. - `local` with a surviving `connectionId` — a row contradicting itself; no SSH connection is handed out. - `runtime:<env>` — throws `ExecutionHostNotDispatchableError`. Its repo row's connection names a target in the *server's* namespace; dialling it here reaches a same-named target on this client. - rival rows disagreeing with no worktree host — `worktree_execution_host_unresolved`, matching the launch path rather than guessing a row. An unreachable SSH host still throws `SSH_GIT_PROVIDER_UNAVAILABLE_MESSAGE`; loss of contact is never evidence of locality (docs/reference/ssh-execution-boundary.md). `resolveWorktreeLaunchHost` keeps its exact signature and now delegates to `resolveWorktreeHostRouting`, the same resolution answering "which host is this on" rather than "what may this client dial" — the git target needs the first question because `local` and `runtime:` are two different non-SSH answers. No wire change: `RuntimeGitTarget` is main-process internal, and the SSH and local model-discovery host keys are byte-identical to before. `RuntimeFileTarget` has the same defect in ~30 filesystem dispatches and is deliberately left for a follow-up. |
||
|
Neil
|
f37d2fec97 |
fix(linux): land the reviewed Linux packaging stack on main (#18100)
* fix(linux): give the CLI one entrypoint by extracting the AppImage once
* refactor(linux): trim AppImage CLI registration seams
* test(cli): assert registration lock serialization
* fix(linux): fence AppImage terminal shim mounts
* fix(linux): accept extracted AppImage runtimes with APPDIR only
* docs(linux): make headless AppImage extraction runnable
* refactor(linux): import bundled launcher directly
* fix(linux): reclaim superseded AppImage payloads and packaged symlinks
Pruning removed 3215 of 3216 files from a superseded generation and always
stranded resources/app.asar, leaking ~105 MB per version update. Electron's
asar shim reports a *.asar file as a directory, so the recursive remove tried
to rmdir a real file and failed with ENOTEMPTY; the .catch(() => {}) hid it.
Reproduced end to end on Ubuntu 24.04: 519M -> 623M across one update, and
519M again once the payload is actually reclaimed.
removeExtractedAppImagePayload holds process.noAsar for the removal, counted
so overlapping removals cannot hand the shim back early, and the prune site
now warns with the path instead of swallowing the rejection. All three
removal sites use it -- staging cleanup and displaced roots leaked the same
way.
Also reclaim symlinks left by a packaged deb/rpm install, which the
extracted-cache-only rule turned into a hard conflict on a deb -> AppImage
migration, and name the remedy in the conflict error.
* fix(linux): bound the CLI registration lock wait
`retries: 1000` caps the attempt count, not elapsed time, so at up to 1s per
attempt an IPC-driven registration could hang ~16 minutes against a wedged
holder with no feedback.
A legitimate holder is bounded by the extraction timeout, so wait that plus
slack and then fail with a message naming the lock file, rather than hanging.
`maxRetryTime` is forwarded verbatim to the `retry` package by proper-lockfile.
* fix(linux): stop re-extracting the AppImage on inode metadata churn
The extracted-payload cache key hashed ctime alongside dev/ino/size/mtime.
ctime moves on any inode metadata write -- `chmod +x`, which every AppImage
user is told to run, plus `chown`, an ACL or SELinux relabel, and a backup
restore -- none of which alter a byte of the payload.
Measured on Ubuntu 24.04: `chmod +x` leaves dev, ino, size and mtime
identical and moves ctime alone, so the key changed and the next launch paid
a full ~519 MB re-extraction and a multi-second stall to rebuild a payload it
already had, then pruned the old generation.
Key on content identity instead. An in-place content change moves mtime and
almost always size; a replacement moves the inode. The existing
replace-in-place test still passes.
* fix(linux): stop CLI commands from falling through to Chromium startup
* refactor(cli): remove redundant command membership check
* test(cli): cover command-named project selectors
* fix(cli): redirect the open-url command before startup
* test(linux): cover AUR serve wrapper flags
* fix(linux): tighten CLI launch detection
* fix(linux): respect CLI flag value boundaries
* fix(linux): strip injected Chromium switches from CLI args
* fix(linux): report a missing display instead of dying in uv_close
* refactor(linux): read display locks without a preflight race
* fix(linux): preserve unverified external displays
* chore: format reliability gate manifest
* test(packaging): split runtime resource checks
* fix(linux): fail serve when no display is available
* fix(linux): do not treat a lockless X socket as a dead display
An X server writes its lock beside its socket and both survive a crash
(verified against Xvfb under SIGKILL), so a socket with no lock was never
left by a crashed server. It is an endpoint published from elsewhere: a
container bind-mounting only /tmp/.X11-unix, WSLg, or a foreign PID
namespace. Declaring those dead made the desktop gate exit(1) on displays
that work, with no workaround, and the serve gate refuse to start.
Liveness now splits by ownership. A foreign DISPLAY trusts a lockless
socket; Orca's own :99 does not, because removeStaleDisplayArtifacts
unlinks the lock before the socket and so manufactures that state itself --
adopting it would resurrect the orphan-socket bug and stop the cleanup from
self-healing. The stale-lock rejection is unchanged.
Also correct four doc statements this behaviour falsified.
* fix(linux): fail closed when a stale socket blocks the Xvfb rebind
Readiness only checked that /tmp/.X11-unix/X99 exists. A stale socket we
could not unlink still exists after our own Xvfb refused to bind, so Orca set
DISPLAY to a dead server and Chromium died in Ozone init.
Measured on Ubuntu 24.04 against the pre-fix build: with a leftover :99
socket and no lock, serve exits 139 (SIGSEGV), the socket inode is unchanged
before and after, and no lock is recreated -- it neither cleaned up nor
respawned. To a user that is a crash, not a misconfiguration.
This is reachable in the documented topology, where orca-xvfb.service has no
User= and runs as root while serve runs as User=orca: /tmp is sticky, so the
orca uid cannot unlink a root-owned socket, rmSync fails, and Xvfb exits with
the display already active.
Readiness now requires the display to actually be live -- our socket plus a
lock naming a running process -- so the same state reports an unusable
display and exits 1 with the existing diagnosis.
* fix(linux): recognise abstract X sockets and inherited Wayland fds
Two display setups this gate could not prove were refused outright, and on the
desktop path that is app.exit(1) with no workaround.
An X server may bind only the abstract namespace (`@/tmp/.X11-unix/X0`), which
leaves no filesystem socket to stat. Abstract addresses are kernel-owned and
vanish the moment the owner exits, so an entry in /proc/net/unix is proof of a
live server -- no lock file needed and no stale entry possible. Verified on
Ubuntu 24.04, where 139 such addresses were present.
WAYLAND_SOCKET is an already-connected fd handed over by the compositor, so
there is no path to stat and WAYLAND_DISPLAY may be unset entirely. Its
presence is the display.
Both are consulted only after the filesystem-socket check fails, so no
existing verdict changes.
* fix(linux): never treat Orca's own display number as a foreign endpoint
Recognising a lockless X socket as live is correct for an endpoint published
from elsewhere -- a container bind mount, WSLg -- because an X server writes
its lock beside its socket and both survive a crash. It is wrong for
VIRTUAL_DISPLAY_NUMBER, because Orca's own teardown unlinks the lock before
the socket and so manufactures that exact state.
The managed branch was already strict, but a caller that sets DISPLAY=:99
explicitly takes the foreign path and skipped it, accepting a dead display
left by Orca's own interrupted cleanup. Route the managed number through the
strict probe on both paths.
Found by an adversarial audit of the asymmetry introduced earlier in this
branch; the documented systemd topology is unaffected because its Xvfb writes
a real lock.
* test(linux): add a packaged-artifact contract for the CLI launch paths
* test(linux): avoid buffered serve readiness detection
* test(linux): signal AppImage serve owner directly
* test(linux): tolerate readiness timeout boundary
* test(linux): add startup margin to shutdown oracle
* ci(linux): give package contracts timeout headroom
* fix(ci): route all Linux packaging contract changes
* test(linux): poll shutdown readiness without tail leaks
* test(linux): bound shutdown cleanup grace
* test(linux): assert on CLI output, not the harness's own control lines
run-cli-case.sh echoes `RESULT status=N case=<name>`, and the two cases named
*-skills asserted `expectOutput: 'skills'`. That substring was satisfied by
the case name in the harness's own line, so 2 of 8 cases asserted nothing
about the command -- gutting `skills` entirely would still have gone green.
Control lines are now excluded before matching, and both cases assert the
rendered help header, which only real help output produces. Verified on an
Ubuntu 24.04 host: 8/8 still pass against a stack-tip AppImage.
Also register the gate in reliability-gates.jsonc, which #15085 added a CI
Docker gate without. Red/green is recorded from a stock release AppImage
failing 4 of 8, three of them at status 133 (SIGTRAP).
* fix(linux): require static AppImage runtimes (#17319)
* test(linux): reject a wrong-architecture native binary at packaging time
Cross-building the arm64 slice on an x64 host silently packed an x86-64
`pty.node` -- the rebuild logged "Forcing native rebuild for linux-arm64" and
shipped the host's binary anyway. Every gate here inspects symbol versions,
which are perfectly valid on the wrong architecture, so nothing noticed.
Observed on a Raspberry Pi 5: the packaged app loaded, then failed with
"Failed to load native module: pty.node", and the launch contract reported
3 of 8 cases crashed rather than naming the cause. Swapping in the aarch64
`pty.node` took the same build to 8/8.
Compare ELF `e_machine` against the slice being packaged and fail with the
offending path. Checked before the glibc pass, because a wrong-architecture
binary's symbol versions are valid but meaningless and would send the reader
down the wrong path.
Release CI builds arm64 on a native runner, so this guards local and future
cross-builds rather than a shipped artifact.
* test(linux): judge per-arch vendored binaries against their own path
The first CI run of the architecture gate failed the x64 package job on
`@parcel/watcher-linux-arm64-glibc/watcher.node`. That binary is arm64 on
purpose: the package ships every architecture and its loader picks the match,
so its presence in an x64 build is correct.
Judge a binary against the architecture its own path names, falling back to
the slice when the path names none. That keeps the case this gate exists for
-- `bin/linux-arm64-*/node-pty.node` holding an x86-64 binary, which is what
shipped to a Raspberry Pi 5 -- while letting multi-arch dependencies through.
Dry-run over the real dependency tree flags nothing for either target arch.
* fix(linux): move deb/rpm update installation outside Orca (#17318)
* fix(linux): complete deb/rpm package metadata
* fix(linux): preserve CLI link during package upgrades
* docs(linux): document local RPM build prerequisites
* fix(linux): move deb/rpm update installation outside Orca
* fix(updater): preserve Linux recovery across stale events
* fix(updater): fence stale downloaded events by active target
* fix(updater): preserve active Linux package recovery
* test(linux): keep workflow order assertion in scope
* test(updater): assert stale recovery stays silent
* fix(updater): preserve Linux package recovery after checks
* refactor(updater): keep Linux marker message with status
* fix(linux): describe the right manual update path for deb/rpm hosts
A remote host installed from .deb or .rpm now reports
manual-service-update-required, and the guidance told the operator to
"update through the service manager that starts this server" -- which is
correct for unsupported-headless-serve but wrong for a package install,
where nothing about the remedy involves the service manager.
Say both, keyed on how the host was installed.
* docs(linux): document orcad update restart safety
* docs(linux): scope restart census omissions
* docs(linux): use absolute service CLI launcher
* fix(serve): validate in-process serve options before startup (#17683)
* fix(linux): stop offering updates a distro-managed install cannot apply (#17918)
Closes #17702.
The resources/package-type marker is authoritative but never checked against
the host, so any repackager that unpacks Orca's .deb -- AUR, Nix, a container
rebuild -- inherits `deb` verbatim. Install feasibility was then computed
after a ~165 MB download, so those users got check -> download -> a card
promising an install command -> a dead end.
Validate the marker against the host: a deb/rpm marker with no matching
package manager in the trusted directories means a package manager owns this
install. This reuses the exact lists and resolver that
buildLinuxPackageInstallCommand already loops over, so a false positive is
impossible by construction -- any host flagged here would have failed with
no-package-manager after the download anyway. The gate only moves that
verdict earlier. Verified across Debian 12, Ubuntu 24.04, Arch, Fedora 40 and
openSUSE Leap: no false positive on a real deb host, correct on every
repackaging host.
The release is still reported, because the user does want to know 1.4.194
exists and to update through their distro; only the download path is closed.
`externallyManaged` is an additive optional field on the existing `available`
status, so older paired clients decode it unchanged. downloadUpdate() refuses
authoritatively, since main owns this verdict rather than the card, and
unwinds any pinned-build state first -- a Linux pinned jump resolves to
'release', and stranding isPinnedBuildActive would silently kill every
background check for the rest of the process.
Note the fix the issue suggests cannot work: electron-updater builds a
PacmanUpdater whose doDownloadUpdate looks for a .pacman asset Orca does not
publish, then dereferences undefined.
* style(cli): restore prettier wrapping on install error copy
* test(linux): re-pin the child-process ratchets and the batch-shim allowlist after the merge
|
||
|
Neil
|
6d61305a96 |
docs(ssh): record that an app update strands relay-backed terminals as unverifiable (#17972)
* docs(ssh): record that an app update strands relay-backed terminals The boundary doc listed two ways remote work can stop and omitted the third outcome, where the work does not stop but becomes permanently unreachable (#13852). Names the mechanism, keeps it in the live/unverifiable/exited vocabulary, and contrasts it with the daemon's protocol-versioned endpoint. * docs(ssh): cite the boundary's sources by symbol so the refs cannot rot |
||
 Brennan Benson
|
5ca747dad0 |
docs(ssh): state the SSH execution boundary and pin the liveness vocabulary (#14971)
* docs(ssh): state the SSH execution boundary and pin the liveness vocabulary Nothing under docs/ described how work splits between the client and an SSH host, so agents and humans inferred it from error strings and got it wrong: loss of contact was repeatedly reported as process death, which orphaned live remote agents and cold-started duplicates over the same worktree. Pins the vocabulary to the incumbent live/unverifiable/exited verdict from unstopped-pty-verification so no synonym is introduced, records the one real discriminator (all of a host's terminals drop together on link loss; one alone means process exit), and lists the outstanding gaps with citations. Tracked via the docs allow-list and linked from AGENTS.md, per the convention in .gitignore. * docs(ssh): cite the live restoreRequired site after it moved The throw now lives in reattachSshPtySessionForSpawn; ssh-pty-provider.ts no longer contains it. Caught by the worker fixing it, against a newer main than the audit ran on. * docs(ssh): require host evidence for liveness verdicts * docs(ssh): keep boundary references stable * docs(ssh): fence liveness evidence to its host identity * docs(ssh): state replay and environment boundaries precisely * docs(ssh): correct replay and platform boundary claims * docs(ssh): describe headless runtime continuity accurately * docs(ssh): distinguish authority from client metadata * docs(ssh): describe pending fixes accurately * docs(ssh): date the gap list and name the PR that closes each entry The Known gaps section was accurate when written and becomes actively misleading as its fixes land: it told a reader to go fix restoreRequired, the missing unverifiable verdict, and the absent terminal-list host field, three things now addressed by #14974, #14977 and #14973. Mark the section as dated, require verification against current code before acting on any entry, name the PR per entry, and move landed items out. Also correct the two body claims that the landed fixes invalidated. The rules above are durable; only this section rots. * docs(ssh): make the boundary doc a durable ruleset, not an incident record The Known gaps section was 18 of 93 lines enumerating specific defects from one investigation, several already fixed by sibling PRs in the same batch. A reference doc that needs a 'this section rots' warning is telling you the section belongs somewhere else; those entries belong in issues. Replace the six-row table of currently-lying signals with the method that outlands any particular bug: ask whether the owning host produced the signal, whether every PTY on the target went quiet together, whether the termination event matches the current incarnation and generation, and whether a returned status is actually a claim. Same for artifacts - state what ls-remote and a PR head each do and do not prove, rather than listing which command is currently wrong. Nothing here goes stale when the open fixes land. |