mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 16:02:32 +00:00
main
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  OrcaWinandm4air
|
ba742a86bb |
fix(linux): release orphaned processes when their owner exits (#22247)
* fix(linux): release orphaned processes when their owner exits * fix(linux): handle inhibitor errors until streams close --------- Co-authored-by: m4air <m4air@m4airs-MacBook-Air.local> |
||
  
|
88f2f01061 |
fix(daemon): escape the terminal daemon into its own systemd scope so a service restart no longer kills every live PTY (#19430)
* fix(daemon): escape the terminal daemon into its own systemd scope so a service restart no longer kills every live PTY Root cause: daemon-launched-child.ts forks the detached terminal daemon with detached: true, which escapes the POSIX process group (setsid) but never the systemd cgroup. Every PTY the daemon owns is itself an undetached direct child of the daemon (native-pty-spawn.ts). Under a combined systemd unit (Type=simple, KillMode=mixed, per docs/reference/headless-linux-server.md), a systemctl restart/stop SIGKILLs every process still in the cgroup at the stop timeout -- the daemon and every live terminal -- even though the codebase already has a fully-built adoption/reattachment path for a surviving daemon (orcad-entry.ts's refreshRestoredOrchestrationAuthority + reconcileLegacyWorkerTerminals, gated on daemonOwnsFreshPersistentPtys()). That path never fires today because the daemon never survives long enough. Fix: when systemd is actually supervising the process and the OS user has a reachable systemd --user manager (isDurableDaemonScopeSupported(), Linux only), launch the daemon via systemd-run --user --scope so it lands in a cgroup that is a sibling of the service unit's cgroup, not a descendant of it. A systemctl restart of the combined unit then never reaches it. Any failure of the scoped launch (no reachable bus, D-Bus policy rejection, etc.) falls back transparently to the existing plain fork() launch, so every platform/environment without this capability is unaffected. The daemon self-detects its own resulting cgroup scope via /proc/self/cgroup (detectOwnCgroupScopeUnit()) rather than trusting the launcher's intent, and publishes it as cgroupUnit in its pid record and orcad's health/readiness payload (health.terminalDaemon.cgroupUnit), so a running deployment can be observed to confirm the fix actually engaged. No new session registry is added: the existing daemon pid-record + adoption protocol (publishDaemonPidFile, daemon-pid-record-quarantine.ts's dead-record reclaim, refreshRestoredOrchestrationAuthority) already implements durable, crash-safe reattachment for a surviving daemon -- it was simply never exercised against a full unit restart before now. Proven via a systemd-in-Docker recovery test: a live PTY session's shell process, its daemon, and the daemon's cgroup scope were all confirmed unchanged across a real systemctl restart of a Type=simple/KillMode=mixed unit, while the main process pid changed (confirming the unit actually restarted) and the new process's health payload recognized the surviving daemon as adopted and live. A fresh write into the same PTY post-restart reached the same running shell. Ordinary terminal create/work/release and the #18789/#18790 worker-release reap-fix regression tests are unaffected. Fixes stablyai/orca#19408 * fix(daemon): probe the real per-UID XDG_RUNTIME_DIR before trusting the process's own env isDurableDaemonScopeSupported()/buildDurableDaemonScopeCommand() trusted the current process's own XDG_RUNTIME_DIR env var first, falling back to /run/user/<uid> only when that var was unset entirely. On mtl-02, orca-serve@factory.service's RuntimeDirectory= hardening directive makes systemd export XDG_RUNTIME_DIR=/run/orca_serve/factory into the unit's process -- a private scratch dir that shares the env var's name but has nothing to do with the user session bus. /proc/<pid>/environ on that host confirmed exactly that path plus DBUS_SESSION_BUS_ADDRESS=disabled:, while the real bus was reachable the whole time at /run/user/985 (confirmed via systemctl --user is-system-running with that dir exported by hand). The probe treated the hardened override as authoritative, found no bus socket there, and reported unsupported on every launch -- so the cgroup-escape fix from #19408/#19430 never actually engaged on real hardware, even though tonight's factory deployment picked it up. Fix: resolveUserRuntimeDir() now always tries the conventional /run/user/<uid> path first (computed independently via getuid(), never trusted from env), checking for a genuinely connectable bus socket via statSync(...).isSocket() rather than a bare existsSync. It falls back to the process's own XDG_RUNTIME_DIR only when that canonical path has no reachable bus -- covering hosts that legitimately have no /run/user/<uid> at all but do have a working bus wherever their own environment points. buildDurableDaemonScopeCommand() now explicitly sets XDG_RUNTIME_DIR to whichever path this resolution picked, rather than inheriting the spread env's (possibly hardened-wrong) value. Both isDurableDaemonScopeSupported() and buildDurableDaemonScopeCommand() gained an injectable canonicalRuntimeDir parameter (defaulting to the real computed path) so tests can exercise the hardened-override scenario deterministically with a real, connectable AF_UNIX socket fixture instead of the live host's actual runtime directory. Docker's stock jrei/systemd-ubuntu test container never had this hardening directive, so this gap was structurally invisible to the container-based verification in #19430 -- only caught against real mtl-02 hardware. * fix(daemon): report the daemon's own pid over the ready handshake, not systemd-run's The launcher used to infer the daemon's identity pid from the immediate spawned child (`child.pid`). On the durable-scope path that child is `systemd-run --user --scope`, not the daemon, so the launcher was asserting an identity it had no authority over. `DaemonReadyIdentity` now carries a required `pid` populated from `process.pid` inside the daemon itself, and `daemon-launched-child.ts` takes `launchedIdentity.pid` from that self-report. Both sides of the `holdDaemonAdoptionLease` pid comparison therefore originate inside the daemon process, which is the idiom this branch already uses for cgroup membership (`detectOwnCgroupScopeUnit` reads `/proc/self/cgroup` rather than trusting what the launcher intended). Note on the reported consequence: `systemd-run --scope` registers its *own* pid on the transient scope unit and then `execvpe()`s the target command -- same pid, no intermediate process -- so adoption did not in fact fail on systemd >= 206 (verified against systemd 255.4-1ubuntu8.17 and current main, `src/run/run.c` `start_transient_scope()`). The fix stands on its own merits: it removes a silent dependency on that exec-vs-fork implementation detail, which a `systemd-run` shim earlier in PATH or any future systemd change would have broken with no diagnostic. `terminateLaunchedDaemonChild` was audited and deliberately left on `child.pid`: for the same execve-preserves-pid reason that pid is either still systemd-run mid-scope-setup (killing it correctly aborts the launch) or already the daemon, so it targets the right process either way. Regression coverage: `daemon-launched-child-identity.test.ts` pins the identity source, and `daemon-ready-identity.test.ts` gains pid-validation cases. Ready-message fixtures across the `daemon-init-*` suites were updated for the now-mandatory field. Addresses: https://github.com/stablyai/orca/pull/19430#discussion_r3953722704 https://github.com/stablyai/orca/pull/19430#discussion_r3954346518 * test(daemon): assert cgroupUnit in the pid-file parse contract `parseDaemonPidFile` returns `cgroupUnit` on every branch as of the durable-scope commit on this branch, but five exhaustive `toEqual` assertions in daemon-health.test.ts still described the pre-scope shape, so they failed on the branch independently of any later change. Adds the field to those expectations. Deliberately not relaxed to `toMatchObject`: asserting the full parsed shape is what makes these tests catch a field silently dropped from the pid-file contract. * refactor(daemon): resolve the canonical user runtime dir at one point The per-UID path cannot change for a live process, so compute it once into a module const instead of threading the same default call through three signatures, and drop the try/catch around a getuid() that cannot throw once it exists. Trims the module prose to the non-obvious facts and corrects the pid-file record comment: an unscoped daemon writes null; only records no daemon wrote are absent. * test(daemon): clean up the cgroup-scope fixtures and assert a verdict The cgroup fixture tracked only the file it wrote, leaking one temp dir per case. Drains both fixture lists with splice so the pop-may-be-undefined guards go away, and replaces a not-throw/typeof-boolean pair with the verdict it was circling: no resolvable runtime dir means unsupported. * refactor(daemon): share the detached child options across both launch paths cwd, detached and stdio were repeated in the fork and systemd-run branches, which left the two comments explaining them hovering over the env block instead. Names them once so each branch carries only its own delta. * refactor(daemon): validate the ready pid like every other field typeof-first narrows the value, so the two 'as number' casts the isSafeInteger check needed disappear and the pid guard reads like the startedAtMs guard below it. * fix(daemon): don't retry the launch unscoped after losing the endpoint race A scoped attempt that lost the endpoint to another daemon was retried unscoped: a second doomed fork, a misleading 'cgroup-scope launch failed' warning, and the same DaemonEndpointUnavailableError the caller was already going to adopt on. Rethrows it instead, since no launch mode can win a race that is already lost. Also drops a private alias for DaemonChildSpawnOptions and the two 'as number' casts on child.pid in the startup-failure cleanup. * fix(daemon): unlink the pid record by the pid the daemon published The record holds the daemon's self-reported pid, so match on that rather than on the immediate child's, which is the systemd-run wrapper's until it execs. * fix(daemon): route the scope launch through the child-process chokepoint The two files this PR added imported `node:child_process` directly, which `child-process-import-boundary.test.ts` fails on deterministically: the offender count went 155 -> 157 against a pin of exactly 155. Raising the pin or listing the files is what that test explicitly forbids, and the allowlist's own note says a split "moved the import, it did not add one" -- so the fix is to get both new files off the module and put the count back at 155. - `daemon-cgroup-scope.ts`: the `systemd-run --version` probe now uses `runProcessSync` instead of `execFileSync`, so it gets the shared spawn decisions. Kept synchronous deliberately: `launchDaemonChild` attaches the readiness listener in the same tick it is called, and an await before the spawn moves the child past that tick. A non-zero exit is data rather than a throw here, so the verdict now checks `code === 0 && !timedOut`. - `daemon-launched-child-spawn.ts`: the scoped launch uses `spawnProcess`, and the long-standing unscoped launch keeps `fork` semantics through a new `forkProcess`. - `src/shared/child-process/fork-process.ts`: the fork arm of the chokepoint. `spawnProcess` cannot express a Node child with an IPC channel started from a module path under an overridden `execPath`, and the existing launch tests are written against `fork`'s contract, so a spawn rewrite would have changed module resolution, `execPath` and `execArgv` at once. It passes `windowsHide: true` -- the flag every other call site in that directory sets, reachable via an assertion because `ForkOptions` omits it -- which keeps `windows-console-visibility.test.ts` at its pin of 65 too. Both ratchets pass with both pins and both allowlists untouched. Docs: `orcad-operations.md` and `headless-linux-server.md` still described the limitation this PR removes as permanent. Both now describe the durable-scope survival path and its preconditions (systemd as PID 1, a reachable user bus / `loginctl enable-linger`, `systemd-run` on PATH), and scope the old text to the unscoped-fallback case, pointing at `health.terminalDaemon.cgroupUnit` as the way to tell the two apart on a running host. * fix(daemon): seal the cgroup capability probe from the host and correct KillMode=mixed docs The capability probe consulted the host's own /run/systemd/system marker and spawned the real systemd-run binary, so the hermetic unit tests could only pass on a systemd host (and fail closed otherwise, even with faked bus sockets). - Thread systemdBootPath and runVersionProbe as test seams through isDurableDaemonScopeSupported, defaulting to the real boot marker and systemd-run --version probe in production. - Narrow the injected probe to the ProcessResult slice it consumes. - Cover: no-systemd-boot, non-zero probe exit, and probe-timeout cases. - Correct KillMode=mixed semantics in the docs: the cgroup-wide SIGKILL fires the instant the main process exits, not after TimeoutStopSec; document the Docker-container caveat and add KillMode=mixed to the multi-service template. * fix(daemon): satisfy assertion checks in scoped launch * fix(daemon): satisfy anti-slop and console guards * test(serve): update shutdown docs assertions for daemon scope * fix(daemon): migrate adopted legacy scopes * docs: qualify restart safety by daemon scope * docs(daemon): qualify Upgrade restart prose with durable scope caveat Align the Upgrade section in docs/reference/headless-linux-server.md with the earlier preservation section and docs/reference/orcad-operations.md: a service restart terminates live processes only when running under the unscoped fallback, and stops should be treated as destructive unless health.terminalDaemon.cgroupUnit names an orca-daemon-*.scope. Update the shutdown workflow test assertion in config/scripts/headless-serve-shutdown-workflow.test.mjs to match. * fix(daemon): harden legacy scope migration --------- Co-authored-by: Lesley Murfin <260182349+LesleyMurfin@users.noreply.github.com> Co-authored-by: m4air <m4air@Mac.localdomain> |
||
 Neil
|
f37d2fec97 |
fix(linux): land the reviewed Linux packaging stack on main (#18100)
* fix(linux): give the CLI one entrypoint by extracting the AppImage once
* refactor(linux): trim AppImage CLI registration seams
* test(cli): assert registration lock serialization
* fix(linux): fence AppImage terminal shim mounts
* fix(linux): accept extracted AppImage runtimes with APPDIR only
* docs(linux): make headless AppImage extraction runnable
* refactor(linux): import bundled launcher directly
* fix(linux): reclaim superseded AppImage payloads and packaged symlinks
Pruning removed 3215 of 3216 files from a superseded generation and always
stranded resources/app.asar, leaking ~105 MB per version update. Electron's
asar shim reports a *.asar file as a directory, so the recursive remove tried
to rmdir a real file and failed with ENOTEMPTY; the .catch(() => {}) hid it.
Reproduced end to end on Ubuntu 24.04: 519M -> 623M across one update, and
519M again once the payload is actually reclaimed.
removeExtractedAppImagePayload holds process.noAsar for the removal, counted
so overlapping removals cannot hand the shim back early, and the prune site
now warns with the path instead of swallowing the rejection. All three
removal sites use it -- staging cleanup and displaced roots leaked the same
way.
Also reclaim symlinks left by a packaged deb/rpm install, which the
extracted-cache-only rule turned into a hard conflict on a deb -> AppImage
migration, and name the remedy in the conflict error.
* fix(linux): bound the CLI registration lock wait
`retries: 1000` caps the attempt count, not elapsed time, so at up to 1s per
attempt an IPC-driven registration could hang ~16 minutes against a wedged
holder with no feedback.
A legitimate holder is bounded by the extraction timeout, so wait that plus
slack and then fail with a message naming the lock file, rather than hanging.
`maxRetryTime` is forwarded verbatim to the `retry` package by proper-lockfile.
* fix(linux): stop re-extracting the AppImage on inode metadata churn
The extracted-payload cache key hashed ctime alongside dev/ino/size/mtime.
ctime moves on any inode metadata write -- `chmod +x`, which every AppImage
user is told to run, plus `chown`, an ACL or SELinux relabel, and a backup
restore -- none of which alter a byte of the payload.
Measured on Ubuntu 24.04: `chmod +x` leaves dev, ino, size and mtime
identical and moves ctime alone, so the key changed and the next launch paid
a full ~519 MB re-extraction and a multi-second stall to rebuild a payload it
already had, then pruned the old generation.
Key on content identity instead. An in-place content change moves mtime and
almost always size; a replacement moves the inode. The existing
replace-in-place test still passes.
* fix(linux): stop CLI commands from falling through to Chromium startup
* refactor(cli): remove redundant command membership check
* test(cli): cover command-named project selectors
* fix(cli): redirect the open-url command before startup
* test(linux): cover AUR serve wrapper flags
* fix(linux): tighten CLI launch detection
* fix(linux): respect CLI flag value boundaries
* fix(linux): strip injected Chromium switches from CLI args
* fix(linux): report a missing display instead of dying in uv_close
* refactor(linux): read display locks without a preflight race
* fix(linux): preserve unverified external displays
* chore: format reliability gate manifest
* test(packaging): split runtime resource checks
* fix(linux): fail serve when no display is available
* fix(linux): do not treat a lockless X socket as a dead display
An X server writes its lock beside its socket and both survive a crash
(verified against Xvfb under SIGKILL), so a socket with no lock was never
left by a crashed server. It is an endpoint published from elsewhere: a
container bind-mounting only /tmp/.X11-unix, WSLg, or a foreign PID
namespace. Declaring those dead made the desktop gate exit(1) on displays
that work, with no workaround, and the serve gate refuse to start.
Liveness now splits by ownership. A foreign DISPLAY trusts a lockless
socket; Orca's own :99 does not, because removeStaleDisplayArtifacts
unlinks the lock before the socket and so manufactures that state itself --
adopting it would resurrect the orphan-socket bug and stop the cleanup from
self-healing. The stale-lock rejection is unchanged.
Also correct four doc statements this behaviour falsified.
* fix(linux): fail closed when a stale socket blocks the Xvfb rebind
Readiness only checked that /tmp/.X11-unix/X99 exists. A stale socket we
could not unlink still exists after our own Xvfb refused to bind, so Orca set
DISPLAY to a dead server and Chromium died in Ozone init.
Measured on Ubuntu 24.04 against the pre-fix build: with a leftover :99
socket and no lock, serve exits 139 (SIGSEGV), the socket inode is unchanged
before and after, and no lock is recreated -- it neither cleaned up nor
respawned. To a user that is a crash, not a misconfiguration.
This is reachable in the documented topology, where orca-xvfb.service has no
User= and runs as root while serve runs as User=orca: /tmp is sticky, so the
orca uid cannot unlink a root-owned socket, rmSync fails, and Xvfb exits with
the display already active.
Readiness now requires the display to actually be live -- our socket plus a
lock naming a running process -- so the same state reports an unusable
display and exits 1 with the existing diagnosis.
* fix(linux): recognise abstract X sockets and inherited Wayland fds
Two display setups this gate could not prove were refused outright, and on the
desktop path that is app.exit(1) with no workaround.
An X server may bind only the abstract namespace (`@/tmp/.X11-unix/X0`), which
leaves no filesystem socket to stat. Abstract addresses are kernel-owned and
vanish the moment the owner exits, so an entry in /proc/net/unix is proof of a
live server -- no lock file needed and no stale entry possible. Verified on
Ubuntu 24.04, where 139 such addresses were present.
WAYLAND_SOCKET is an already-connected fd handed over by the compositor, so
there is no path to stat and WAYLAND_DISPLAY may be unset entirely. Its
presence is the display.
Both are consulted only after the filesystem-socket check fails, so no
existing verdict changes.
* fix(linux): never treat Orca's own display number as a foreign endpoint
Recognising a lockless X socket as live is correct for an endpoint published
from elsewhere -- a container bind mount, WSLg -- because an X server writes
its lock beside its socket and both survive a crash. It is wrong for
VIRTUAL_DISPLAY_NUMBER, because Orca's own teardown unlinks the lock before
the socket and so manufactures that exact state.
The managed branch was already strict, but a caller that sets DISPLAY=:99
explicitly takes the foreign path and skipped it, accepting a dead display
left by Orca's own interrupted cleanup. Route the managed number through the
strict probe on both paths.
Found by an adversarial audit of the asymmetry introduced earlier in this
branch; the documented systemd topology is unaffected because its Xvfb writes
a real lock.
* test(linux): add a packaged-artifact contract for the CLI launch paths
* test(linux): avoid buffered serve readiness detection
* test(linux): signal AppImage serve owner directly
* test(linux): tolerate readiness timeout boundary
* test(linux): add startup margin to shutdown oracle
* ci(linux): give package contracts timeout headroom
* fix(ci): route all Linux packaging contract changes
* test(linux): poll shutdown readiness without tail leaks
* test(linux): bound shutdown cleanup grace
* test(linux): assert on CLI output, not the harness's own control lines
run-cli-case.sh echoes `RESULT status=N case=<name>`, and the two cases named
*-skills asserted `expectOutput: 'skills'`. That substring was satisfied by
the case name in the harness's own line, so 2 of 8 cases asserted nothing
about the command -- gutting `skills` entirely would still have gone green.
Control lines are now excluded before matching, and both cases assert the
rendered help header, which only real help output produces. Verified on an
Ubuntu 24.04 host: 8/8 still pass against a stack-tip AppImage.
Also register the gate in reliability-gates.jsonc, which #15085 added a CI
Docker gate without. Red/green is recorded from a stock release AppImage
failing 4 of 8, three of them at status 133 (SIGTRAP).
* fix(linux): require static AppImage runtimes (#17319)
* test(linux): reject a wrong-architecture native binary at packaging time
Cross-building the arm64 slice on an x64 host silently packed an x86-64
`pty.node` -- the rebuild logged "Forcing native rebuild for linux-arm64" and
shipped the host's binary anyway. Every gate here inspects symbol versions,
which are perfectly valid on the wrong architecture, so nothing noticed.
Observed on a Raspberry Pi 5: the packaged app loaded, then failed with
"Failed to load native module: pty.node", and the launch contract reported
3 of 8 cases crashed rather than naming the cause. Swapping in the aarch64
`pty.node` took the same build to 8/8.
Compare ELF `e_machine` against the slice being packaged and fail with the
offending path. Checked before the glibc pass, because a wrong-architecture
binary's symbol versions are valid but meaningless and would send the reader
down the wrong path.
Release CI builds arm64 on a native runner, so this guards local and future
cross-builds rather than a shipped artifact.
* test(linux): judge per-arch vendored binaries against their own path
The first CI run of the architecture gate failed the x64 package job on
`@parcel/watcher-linux-arm64-glibc/watcher.node`. That binary is arm64 on
purpose: the package ships every architecture and its loader picks the match,
so its presence in an x64 build is correct.
Judge a binary against the architecture its own path names, falling back to
the slice when the path names none. That keeps the case this gate exists for
-- `bin/linux-arm64-*/node-pty.node` holding an x86-64 binary, which is what
shipped to a Raspberry Pi 5 -- while letting multi-arch dependencies through.
Dry-run over the real dependency tree flags nothing for either target arch.
* fix(linux): move deb/rpm update installation outside Orca (#17318)
* fix(linux): complete deb/rpm package metadata
* fix(linux): preserve CLI link during package upgrades
* docs(linux): document local RPM build prerequisites
* fix(linux): move deb/rpm update installation outside Orca
* fix(updater): preserve Linux recovery across stale events
* fix(updater): fence stale downloaded events by active target
* fix(updater): preserve active Linux package recovery
* test(linux): keep workflow order assertion in scope
* test(updater): assert stale recovery stays silent
* fix(updater): preserve Linux package recovery after checks
* refactor(updater): keep Linux marker message with status
* fix(linux): describe the right manual update path for deb/rpm hosts
A remote host installed from .deb or .rpm now reports
manual-service-update-required, and the guidance told the operator to
"update through the service manager that starts this server" -- which is
correct for unsupported-headless-serve but wrong for a package install,
where nothing about the remedy involves the service manager.
Say both, keyed on how the host was installed.
* docs(linux): document orcad update restart safety
* docs(linux): scope restart census omissions
* docs(linux): use absolute service CLI launcher
* fix(serve): validate in-process serve options before startup (#17683)
* fix(linux): stop offering updates a distro-managed install cannot apply (#17918)
Closes #17702.
The resources/package-type marker is authoritative but never checked against
the host, so any repackager that unpacks Orca's .deb -- AUR, Nix, a container
rebuild -- inherits `deb` verbatim. Install feasibility was then computed
after a ~165 MB download, so those users got check -> download -> a card
promising an install command -> a dead end.
Validate the marker against the host: a deb/rpm marker with no matching
package manager in the trusted directories means a package manager owns this
install. This reuses the exact lists and resolver that
buildLinuxPackageInstallCommand already loops over, so a false positive is
impossible by construction -- any host flagged here would have failed with
no-package-manager after the download anyway. The gate only moves that
verdict earlier. Verified across Debian 12, Ubuntu 24.04, Arch, Fedora 40 and
openSUSE Leap: no false positive on a real deb host, correct on every
repackaging host.
The release is still reported, because the user does want to know 1.4.194
exists and to update through their distro; only the download path is closed.
`externallyManaged` is an additive optional field on the existing `available`
status, so older paired clients decode it unchanged. downloadUpdate() refuses
authoritatively, since main owns this verdict rather than the card, and
unwinds any pinned-build state first -- a Linux pinned jump resolves to
'release', and stranding isPinnedBuildActive would silently kill every
background check for the rest of the process.
Note the fix the issue suggests cannot work: electron-updater builds a
PacmanUpdater whose doDownloadUpdate looks for a .pacman asset Orca does not
publish, then dereferences undefined.
* style(cli): restore prettier wrapping on install error copy
* test(linux): re-pin the child-process ratchets and the batch-shim allowlist after the merge
|
||
 Jinjing
|
c4b39295c1 |
style: format codebase (#16935)
* style: format codebase * style: format codebase * refactor: extract skill install dialog footer and content Extract footer and content sections from SkillInstallDialog and SkillInstallManagementDialog into separate components for improved maintainability and clarity of component responsibilities. |
||
|
Neil
|
5631aa00dd |
feat(orcad): items 2–7 — degradation, natives, daemon, ops, deploy (#16398)
* fix(ports): stop joining an undefined resourcesPath on a non-Electron host `resolveWorkerEntryPath` branched on `isPackaged` alone and joined `process.resourcesPath`. orcad reports `isPackaged` true — correctly, it is a production build, and ~15 consumers read it that way to gate HTTPS-only skill downloads and the real CLI name — but `process.resourcesPath` is Electron-only and `undefined` under plain Node. So the packaged branch threw `TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string` where a clean "worker unavailable" was the honest outcome. The type said `resourcesPath: string`, which is how it went unnoticed; it is now `string | undefined`, so the compiler carries the fact. A host with no Electron resources tree has no asar to look in, so it falls back to the module directory and lets the caller report a missing worker. Found by the item 1 agent while auditing the same `isPackaged` defect class in the watcher. Verified in both directions: reverting the guard reproduces the TypeError. * feat(orcad): prove node-pty loads before anything requires it Of the two ways node-pty fails, only one is catchable. A missing module throws MODULE_NOT_FOUND. A module built against the wrong libc or Node ABI is refused by the dynamic loader, and in the worst case takes the process down before any handler exists — that is #9902, which crashed the desktop app on Ubuntu 20.04 before a window appeared. There was no libc or ABI precondition anywhere in the tree. So orcad now proves the load in a CHILD process, from main.ts, before anything requires node-pty. Whatever the child does — throw, abort, die on a signal — is data rather than our own death, and the operator gets a sentence naming the host's libc, Node ABI and prebuild slot plus the command to run. Proven-unloadable exits 78 (EX_CONFIG), so a supervisor does not restart an unequippable host forever. A probe that never answered is unverifiable, not blocked: refusing to boot on an inconclusive signal would take down hosts that work. The child dlopens the file node-pty would have chosen, before requiring the package. node-pty's loader walks several directories and rethrows only the LAST error, so a refused binary reads as "Cannot find module ./prebuilds/..." — which sends the operator to install a module that is already there. It also reports through stdout: node echoes the whole -e source above a stack trace, and matching tokens against stderr made the probe's own source text answer for the verdict. Verdicts reach clients as a terminal_unavailable degradation alongside the existing browser_unavailable one, through the same cause-registry shape. degradations[].code is now an open vocabulary; clients already render only `message`. Prebuilds are compiled from PATCHED sources — the patch IS the glibc-floor fix, so an upstream tarball reproduces #9902 — into linux-{x64,arm64}-{glibc,musl} and darwin-{x64,arm64} slots. libc is in the slot name because node-pty's loader falls back to prebuilds/<platform>-<arch> and cannot tell glibc from musl. orcad installs the matching slot at boot, so a host with no compiler serves terminals. The relay's five pure toolchain-diagnosis functions moved to a transport-free module so the Node bundle can reuse them without dragging ssh2 in behind them; the relay keeps its API by re-export. macOS gets `xcode-select --install` rather than the cross-distro apt/dnf/pacman/apk menu, every line of which is wrong there. * test(orcad): pin the node-pty precondition to ground truth, not a prepared host CI's test shard runs `vitest` directly, so `ensure-native-runtime --runtime=node` never prepares node-pty for the Node ABI — `degraded` is the correct verdict there, and asserting 'ok' encoded an environment the shard does not have. Asserting whatever it returned would be vacuous, so the expectation is now derived from an independent require() of node-pty. Verified it still bites: forcing the precondition to always report 'ok' fails the suite. * feat(orcad): run the terminal daemon, and the ops contract around it orcad declared `canRecoverPersistentLocalPtys: () => false` because it did not run the terminal daemon, so every restart, update and rollback SIGKILLed every running terminal — on the host whose selling point is that work survives the client going away. That is the one property `ssh-execution-boundary.md` recommends the peer model for. Item 4 — the daemon: - Port the launch path off electron: `daemon-init.ts`, `daemon-host-relocation.ts` and `observability/logs-directory.ts` now read the `AppEnvironment` port. Relocation additionally asks whether the app root is an asar archive rather than whether the build is packaged, so a Node host answering `isPackaged() === true` no longer walks into an Electron-only NSIS-escape path (same precedent as `parcel-watcher-entry-path.ts`). - `build-orcad.mjs` emits `daemon-entry.js` beside `orcad.js`, scans the forked children's metafiles for electron/node:sqlite, and load-checks the child under plain Node. - orcad spawns and adopts the daemon; shutdown disconnects and never kills it. `canRecoverPersistentLocalPtys` now reads the live provider and is false under degraded routing, where fresh terminals would die with the process. Item 3 — the ops contract (docs/reference/orcad-operations.md): - Bind policy: `--bind`, default loopback, pinned so neither `orca serve`'s wide default nor the connected-device widen can override it, and so a paired client cannot rebind the listener from outside. - Instance lock on the data root before profile load, scoped to the runtime role so it never refuses a restart that a live daemon makes worthwhile. - Supervision: exit codes a supervisor can act on (78 = do not retry), second-signal escalation, a shutdown deadline, and crash-loop containment on daemon respawn. - Health in the readiness payload: build hash, Node ABI, and a PTY self-test that spans both processes — the daemon spawns a real PTY in its own process and the verdict crosses its socket. Both bundle load-checks now assert on exit codes: these bundles are minified onto one line, so Node's uncaught-exception report echoes every string literal in the bundle and the previous message match passed against a bundle that never loaded. * feat(orcad): deploy, activate and roll back a versioned orcad install Plan items 6 and 7 from docs/design/shipping-orcad.html. Install reuses the relay's transaction verbatim — per-version lock, staged SFTP write, .install-complete sentinel, stale-lock recovery — under a parameterized namespace, so orcad-<v>/ sits beside relay-<v>/ permanently (§06). Parameterizing GC is the trap that creates: each model now collects only its own directories, enforced twice (prefix-scoped remote listing plus a local ownership re-check), and a client picks its model from how the host is registered, never from what it finds on disk. Activation is separate from installation, because a versioned directory selects nothing. A candidate is launched, publishes orca_server_ready, and only becomes active if its cross-process health payload passes: right build hash, listening, daemon live, PTY self-test green. A rejected candidate is stopped and the incumbent restarted, so a careful deploy cannot cause the outage it was being careful about. Update and rollback are shaped by the daemon. An update restarts orcad, the daemon outlives it, and the surviving daemon was forked from the outgoing bundle — so live terminals defer the update rather than proceed, and GC pins the active version, the rollback target and the live daemon's bundle. Orca's persisted state carries no schema version, so rollback restores a pre-activation snapshot rather than trusting backward-readability; the point past which it is unsafe is the first terminal created after activation, which the snapshot cannot describe and the surviving daemon still owns. Running the generated shell for real found two bugs the text assertions missed: tar members re-quoted inside a shell variable captured nothing, and kill -0 reports a zombie as alive. * test(orcad): assert the precondition is self-consistent, not environment-shaped The real-host case cannot predict a status: CI's shard runs vitest directly, so node-pty is never built for the Node ABI and 'degraded' is correct there, while a prepared checkout gives 'ok'. The previous attempt used require('node-pty') as ground truth, which resolves the JS wrapper while the native binding loads lazily — it proved strictly less than the precondition checks, and failed CI for exactly that reason. What is invariant on a host with node-pty installed: never 'blocked', and never a degraded verdict carrying an unestablished reason. The injected-input tests keep the logic coverage. * fix(orcad): drop an eslint-disable the rule no longer needs * test(orcad): separate slot placement from the load verdict Both remaining CI failures were the same shape: tests reaching into node_modules for a pty.node that only exists after `ensure-native-runtime --runtime=node`, which CI's shard never runs because it invokes vitest directly. Slot *placement* is the logic worth checking on every host, so it now uses a synthetic payload and asserts the verdict stays honest about not loading. The three assertions that genuinely need a Node-ABI binding are gated on it existing. Verified: breaking slot installation fails both placement tests; with the real pty.node hidden the file is 17 passed / 3 skipped instead of ENOENT. * test(orcad): gate the load-dependent cases on a real load, not on the file existing CI ships a pty.node built for Electron's ABI, so existsSync was true while require still failed — the gate ran exactly the tests that host can never satisfy. It now probes the binding in a child process, so a bad one cannot take the runner down. The self-consistency assertion also allowed too little: 'blocked' is the honest verdict for a corrupt binding, alongside 'ok' on a prepared host and 'degraded' on an unprepared one. What stays invariant is that anything other than 'ok' names an established cause, so a terminal is never declined for a reason nobody worked out. Verified against all three host states: prepared (19 passed), unprepared, and a corrupt binding (17 passed / 3 skipped, no failures). * test(orcad): gate on the whole premise — binding AND spawn-helper CI has a loadable pty.node but no spawn-helper, and a slot without the helper is legitimately 'degraded'. So the previous gate let a test run whose premise ('a complete slot yields ok') that host cannot satisfy. Verified in both states: with the helper present 19 pass; with it removed the load-dependent cases skip (17 passed / 3 skipped) instead of failing. * fix(orcad): preserve degradation types after rebase |