* ci(daemon): gate PRs on daemon protocol crossing from the newest release Lands daemon-protocol-facts.mjs from the Windows update diagnostic branch with a stricter parser, and adds check-daemon-protocol-crossing.mjs (rule R1): the working tree must attach the newest release tag's daemon. Rollback crossing is reported only. Runs in the cross-version-wire job, which already has full tags; tag selection moves to config/scripts/stable-release-tags.mjs so both use one rule. * feat(persistence): run profile backups in the worker whenever its entry is bundled * refactor(orcad): make profile and native preflight runtime-neutral The profile preflight parser now takes the expected runtime identity from the caller (shipped callers pass the pinned Bun identity), and the native preflight is renamed to orcad-runtime-native-preflight with neutral wording. * feat(runtime): pin the Node 24.21.0 server runtime with an offline CI check Add src/shared/node-runtime-pin.ts (NODE_RUNTIME_PIN, SERVER_TARGETS, NODE_RUNTIME_ASSETS for all 8 server targets plus the headers tarball), generated by config/scripts/update-node-runtime-pin.mjs from the nodejs.org and unofficial-builds SHASUMS. check-node-runtime-pin.mjs verifies, with no network, that the pin tracks the locked Electron, matches engines.node's major, and covers exactly SERVER_TARGETS; it runs in the static analysis job. ORCAD_BUN_TARGETS consumers now read SERVER_TARGETS so there is one target list; orcad's Bun runtime and build output are unchanged. * test(persistence): skip plain-Node backup selection tests in the Bun profile suite * fix(runtime): reject a pinned archive that belongs to another target * ci(daemon): fail PRs that swap a runtime launcher and bump the daemon protocol D7.1 R3: hosting orcad or the daemon on another runtime is not a protocol change, so one PR must not do both. The launcher file list lives in the check script; the allow-runtime-launcher-protocol-bump label overrides it. * feat(orcad): select pinned-Node slots by a .runtime-node marker D7.1 R5: a Node slot names its shared runtimes/node-<sha256>/node through .runtime-node instead of .build-target, so Bun-era clients read it as a legacy slot rather than exiting 78 on a missing bun-runtime. Nothing builds the marker yet. * fix(runtime): load the Node pin without the typeless-module warning check-node-runtime-pin.mjs now requires the pin and takes nodeDistArchiveName from its own module, so it no longer loads the update script's build graph. * fix(orcad): resolve Node slots to the design's runtimes/node-<sha>/bin/node layout * feat(orcad): 8-slot node-pty prebuilds against the pinned Node headers at N-API 8 - build-orcad-prebuilds.mjs adds win32-x64/arm64 (conpty.node, the vendored conpty.dll/OpenConsole.exe, upstream's N-API conpty_console_list.node), compiles in a scratch copy against the hash-verified pinned headers (node.lib pinned per Windows arch) with NAPI_VERSION=8, rejects post-8 node_api_* imports, and writes a schema 2 manifest with per-file sha256, N-API level and the glibc need. - --require-slots [slots] verifies files against hashes; --smoke loads the slot under the pinned Node and spawns a PTY; --print-slot names the host slot. - The slot installer gates on N-API, libc, arch, glibc and file hashes instead of the exact NODE_MODULE_VERSION, and installs nested files (conpty/). - bun-profile-tests.yml builds, verifies and smokes each runner's slot. * fix(orcad): scope node-pty's glibc .symver pins to glibc on musl prebuild slots musl's unversioned libc cannot satisfy openpty@GLIBC_* references at link time, so the Alpine slot compile would fail. Pin the staged pty.cc guard to __GLIBC__ and assert both musl transforms against the installed patch. * feat(orcad): run orcad on the pinned Node instead of Bun A packaged orcad slot now references the pinned Node 24.21.0 by its executableSha256 (`.runtime-node`, `.server-target`) instead of carrying bun-runtime, and ships node-pty from the slot's prebuild, only its own ripgrep, and no Windows Bun PTY gate. The runtime lives beside the slots at runtimes/node-<sha>/bin/node (node.exe on Windows, upstream name). - build:orcad (build-orcad-node.mjs) builds the host slot's prebuild when missing and places the pinned runtime; the template is schema 3 with per-target files. - handoffToBundledOrcad() resolves the slot's runtime reference and checks process.versions.node against the pin; a host Node >= 18 still hands off. Startup preflight keys on running as that runtime; callers expect 'node'. - orcad and its daemon use node-pty (ConPTY + windows-pty-job on Windows); the Bun PTY sources, gate entry and canUseBunPty branches are removed. - SSH deploy uploads the official archive once per pin, extracts and hash-checks it on the host, and self-tests it before publishing. Bun slots stay launchable for rollback; Node slots never use host Node. - The runtime materializer is generic over pinned assets; the Bun wrapper remains only for the OpenCode vault reader (design Phase 2). - Cross-runtime test: a profile DB written by Bun 1.4.2 (WAL left by SIGKILL) opens and backs up under the pinned Node, and the reverse. No daemon PROTOCOL_VERSION change (design D7.1 R3). * docs(ci): name the headless lanes after the pinned Node, drop Bun shard timings Design D10: ci-demand-rollout.md and ci-runner-efficiency.md follow the bun-profile-tests.yml -> node-server-tests.yml rename; shard timings drop the deleted Bun PTY tests and follow the renamed ones. * chore(ci): count the runtime archive download as a runtime launcher path * fix(orcad): pin the macOS C++ standard for node-pty prebuilds The official Node headers' config.gypi sets clang: 0, so common.gypi skips its gnu++20 xcode_settings and Apple clang 15 (macos-14 runners) compiles node-addon-api as C++98. * fix(orcad): resolve the preflight's slot through realpath, as the handoff does A symlinked orcad.js handed off to its real slot's pinned Node, but the startup and profile preflights read the symlink's directory, found no runtime marker there, and silently skipped the readiness check. * refactor(ssh): drop materializeCachedNodeRuntime, which nothing calls Deploys upload the verified official archive (design D5); no client path needs an extracted Node executable cached by digest. * test(orcad): gate the Bun-to-Node upgrade and Node-to-Bun rollback with live terminals Design D7.1 R1/R3/R4 and D7.2. The last Bun orcad and this checkout's Node slot are installed side by side under ~/.orca-remote, launched and stopped with the client's own deploy commands, and share one data root. Each direction proves the incoming orcad adopts the outgoing runtime's daemon (same PID, same shell, output continues), opens its profile database and backs it up with its own shipped worker, and that GC keeps the slot the live daemon was forked from. The node-server Linux lanes provide Bun 1.4.2 and build that Bun orcad from main, and run with --cross-runtime. --artifact and --cross-runtime now make their tests fail on a missing input instead of skipping. * ci(node-server): pin node:24.21.0-alpine by its multi-arch index digest * test(ssh): name the runtime archive fixture after its role * test(node-server): load node-pty from the packaged slot in artifact runs The node-server lane installs dependencies without building node-pty, and Linux has no upstream prebuild, so the real-PTY failed-I/O teardown test (picked up by the pty-subprocess selector) could not load pty.node. In --artifact runs, alias node-pty to out/orcad's shipped slot so the test exercises the addon orcad actually runs under the pinned Node. * fix(orcad): let the Windows profile preflight exit after its PTY probe On Windows, node-pty keeps the conout worker thread and pseudoconsole alive until kill(), even after the shell exits. The PTY health probe never killed a cleanly exited probe, so the packaged preflight printed its readiness line and then hung until the build's 30s timeout, reported with an empty stderr. - The probe kills its PTY on Windows after exit and uses the bundled ConPTY the daemon spawns with. - The preflight exits once stdout is flushed; its owner reads to EOF. - Preflight failures now report code, signal, timeout, stdout and stderr. * test(node-server): load the slot's node-pty in the real-PTY test, not by alias A vite alias redirected only ESM imports of node-pty; windows-pty-job and local-pty-utils resolve it through require, so Windows loaded two conpty.node copies and the Git Bash job-membership proof read an empty job. The failed-I/O teardown test now loads node-pty through a fixture that picks the packaged slot in artifact lanes. The pty-subprocess selector was a prefix that also pulled in its POSIX-host sibling unit tests, which pr.yml runs and which were never qualified on Windows. Select the directory plus the two sibling files that belong here. --------- Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
10 KiB
Linux glibc Compatibility
Orca's Linux builds target stock Ubuntu 20.04 and newer — glibc 2.31 and
libstdc++ GLIBCXX_3.4.28 (also Debian 11, RHEL 9), on both x64 and arm64.
Packaging enforces this floor automatically; keep it in mind when adding or
upgrading native dependencies. (The optional speech feature is the one
exception — see below.)
Local package build prerequisites
pnpm run build:linux produces AppImage, deb, and RPM artifacts. The RPM target
requires rpmbuild on PATH; install rpm on Ubuntu/Debian, rpm-build on
Fedora/RHEL, or rpm through Homebrew on macOS, then verify it with
rpmbuild --version before packaging. Cross-host builds have the same
requirement.
Why this needs attention
A native module (.node) links against the glibc of the machine that compiled
it. Our release CI compiles node-pty from source on GitHub's ubuntu-latest
runner, whose glibc rises over time as the image is bumped. A binary compiled on
a newer glibc can reference symbol versions that do not exist on an older target,
and the dynamic loader then refuses to load it:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by .../pty.node)
Because the Orca main process loads node-pty at startup, that failure crashes the whole app before a window appears — this is exactly what shipped in v1.4.150 and broke launch on Ubuntu 20.04 (#9902).
The specific trap is glibc's 2.32–2.34 "libpthread/libutil merge", which moved several long-stable functions into libc under brand-new symbol versions:
| Symbol | New version | node-pty use |
|---|---|---|
pthread_sigmask |
GLIBC_2.32 |
reset child signal mask |
openpty |
GLIBC_2.34 |
allocate the pty |
forkpty |
GLIBC_2.34 |
fork the shell |
Electron itself (glibc 2.25) and the other bundled native modules
(sherpa-onnx, @parcel/watcher, both prebuilt on old glibc) stay well under
the floor, so node-pty was the sole blocker.
How we keep the floor
1. Pin the relocated symbols (the fix).
config/patches/node-pty@1.1.0.patch
adds a .symver shim in src/unix/pty.cc that binds openpty, forkpty, and
pthread_sigmask to their pre-merge version node — GLIBC_2.2.5 on x64,
GLIBC_2.17 on arm64 (each architecture's baseline glibc). glibc still ships
those as compatibility aliases, so the reference resolves on both new build hosts
and old targets.
The catch: gcc defaults to --as-needed and, since the pinned symbols now
resolve from libc's compat aliases at build time, it drops libutil/libpthread
from DT_NEEDED. On the target those libraries are where the symbols actually
live, so the patch's binding.gyp ldflags force
-Wl,--no-as-needed,-l:libutil.so.1,-l:libpthread.so.0 back into DT_NEEDED.
The shim is guarded by #if defined(__linux__); macOS and Windows are untouched.
2. Gate packaging (the regression guard).
config/scripts/verify-linux-glibc-floor.cjs
runs in the electron-builder afterPack hook for Linux. It reads every bundled
native binary's version needs (objdump -p "Version References" — the
authoritative load-time list, which also captures symbol-less markers like
GLIBC_ABI_DT_RELR) and fails the build if any strong GLIBC_/GLIBCXX_/
CXXABI_ node is newer than stock Ubuntu 20.04 provides, naming the file and the
offending node. Weak needs are ignored (the loader tolerates them). It also
asserts the flip side of the .symver fix: any binary that imports
openpty/forkpty must keep libutil.so.1 in DT_NEEDED — otherwise the
pinned openpty@GLIBC_2.2.5 resolves from libc's compat alias at build time (so
the version check passes) yet fails to load on 20.04, where those functions live
only in libutil. A future runner bump, a new native dependency, or a dropped
ldflag therefore fails the release build instead of shipping a Linux app that
crashes on launch.
The gate is a static invariant, not an integration test. The load path was verified by hand for this fix (real Ubuntu 20.04, x64 + arm64:
requirenode-pty and spawn a shell). A CI smoke test that loads the packagedpty.nodein a glibc-2.31 container and spawns a shell is the recommended follow-up — it would make the load path self-verifying and stay valid even if the build ever moves to an old-glibc sysroot.
The one carve-out is the sherpa-onnx speech prebuilt, which already requires
GLIBCXX_3.4.29 (GCC 11). It loads lazily in the speech worker
(src/main/speech/stt-worker.ts), never at app launch, so it is exempt from the
libstdc++ floor — its glibc needs are still checked. Speech-to-text therefore
needs a host with libstdc++ from GCC 11+ (Ubuntu 21.10 / 22.04 LTS or newer); the
app itself still launches on stock 20.04.
3. Check before loading, on hosts that ship without a compiler (orcad).
The two gates above protect the packaged desktop app, where the binary is built and
verified by the same pipeline. orcad is deployed to hosts Orca never built on, so it
adds a runtime precondition
(src/main/orcad/node-pty-precondition.ts),
run from main.ts before anything requires node-pty. It loads the addon in a child
process, so a binary the loader refuses — or one that aborts outright — is data rather
than this process's death, and the operator gets a sentence naming the host's libc, its
Node ABI, its prebuild slot and the command to run. A proven-unloadable binary exits 78
(EX_CONFIG) instead of reaching the require; a probe that never answered is reported
as unverifiable and boots anyway, because a silent probe is not evidence. Whatever it
finds is published in status.get's degradations[] under terminal_unavailable.
4. Ship the binary, built from patched sources.
config/scripts/build-orcad-prebuilds.mjs
(pnpm run build:orcad-prebuilds, before build:orcad, which copies its target's slot into
the package's node_modules/node-pty/build/Release) compiles node-pty for the current
host against the pinned Node's hash-verified headers at N-API 8, and files it under
out/orcad-prebuilds/<slot>/, where a slot is linux-{x64,arm64}-{glibc,musl},
darwin-{x64,arm64} or win32-{x64,arm64}. Its manifest.json records each file's
sha256, the N-API level and, for glibc slots, the highest GLIBC_ version the binary
needs; the loader checks N-API, libc, arch and that glibc version before it installs a
slot. glibc slots must pass the Ubuntu 20.04 floor gate above; musl slots skip it, since
they never meet Ubuntu's libraries. libc is part of the slot name
because node-pty's own loader falls back to prebuilds/<platform>-<arch> and cannot tell
glibc from musl — a glibc binary parked there is loaded on Alpine and dies at dlopen.
The script refuses to compile a tree where config/patches/node-pty@1.1.0.patch is not
applied: without the patch the prebuilt is a #9902 crash shipped as an artifact rather
than a first-connect error. CI runs it once per slot inside the matching container
(--slot= forces the label), merges the trees, and --require-slots fails a release with
a hole in the matrix; --require-slots <slot> checks one slot's files against their
hashes and --smoke loads it under the pinned Node and spawns a PTY
(.github/workflows/node-server-tests.yml runs both on every slot's runner).
Adding or upgrading a native dependency
-
Prefer packages that ship prebuilt binaries compiled against an old toolchain (manylinux /
glibc 2.17-class), like@parcel/watcher. -
For a module we compile from source, if the gate flags it, either pin the offending symbols the way node-pty does, or build it in an old-glibc container.
-
To check locally on a Linux host, list what a binary requires (skipping the weak
0x02-flagged needs the loader tolerates):objdump -p path/to/module.node | sed -n '/Version References/,/^$/p'No strong
GLIBC_node may exceed2.31, and noGLIBCXX_/CXXABI_node may exceed3.4.28/1.3.12— what stock Ubuntu 20.04 ships.
Runtime floor: the environ race below glibc 2.41 (Electron ≥ 43.7.0)
Separate from the build floor above, one glibc runtime bug constrains which
Electron we may ship. Before glibc 2.41, setenv/unsetenv reallocate the
environ array and free the old one, so a concurrent getenv() on another
thread reads freed memory. Ubuntu 20.04–24.04 (2.31–2.39) are all below that
line, so every Linux target we support is exposed.
Electron 43.5.0 made that latent race reachable on every launch: it started
setting GDK_GL=disable around gtk_init() and unsetting it right after, while
in the same change moving FontConfig warm-up onto a thread-pool thread that runs
concurrently and calls getenv() constantly
(electron#53070). The result
is a browser-process use-after-free about a second into startup — no window, no
GPU child involved, and the corruption surfaces wherever the next allocation
lands, which is why reports name unrelated frames (gtk_widget_realize,
libxcb-dri3, FontConfig/expat). Orca 1.4.199/1.4.200 shipped that runtime and
died on launch on Ubuntu + NVIDIA/X11
(#20081).
Electron 43.7.0 fixes it by overriding setenv/unsetenv/putenv/clearenv
so a published environ is never freed, deferring to glibc on 2.41+
(electron#53491, backported
to 42/43/44/45). Do not downgrade Electron below 43.7.0, or move to another
line, without confirming that backport is in the target release —
config/scripts/electron-runtime-floor.test.ts fails the suite if the pin drops
below the floor. Orca itself writes process.env during early startup
(patchPackagedProcessPath, configureOrcaUserDataPathEnv,
hydrate-shell-path), so it is a first-class trigger, not just a bystander.