Commit Graph
25 Commits
Author SHA1 Message Date
OrcaWinandOrcaWin 1864543ade ci: warm GitHub-hosted runner caches and drop the serve-switch installer cache (#34576)
* ci: warm hosted runner caches and drop the serve-switch installer cache

* ci: seed only restored caches from hosted warm legs

* docs(ci): hosted dev builds and no larger runners

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
2026-10-27 15:33:07 -07:00
OrcaWinandOrcaWin 61ad1ba166 ci: move routine checks and unsigned builds to Blacksmith (#32913)
* ci: move routine checks and unsigned builds to Blacksmith

* ci: retain GitHub Mac images for isolated SSH fixtures

* ci: keep shared terminal perf preparation on Blacksmith

* test: allow encrypted slow-link fixture to finish on ARM CI

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
2026-10-23 13:29:01 -07:00
Neil 4fc1855bda build: share one pnpm virtual store across worktrees (off in CI and on Windows) (#30254)
* build: share one pnpm virtual store across worktrees, off under CI

Each worktree carried a full node_modules; the global virtual store links
one slot per package version instead. pnpm keeps an explicit setting on
under CI, so a pnpmfile hook turns it off there: CI packages from,
caches and verifies a project-local node_modules.

* fix(build): keep shared electron and windows-process-tree dirs untouched under GVS

Every worktree runs Electron from the one shared electron package, so the
per-repo dist cache must not swap that dist and a foreign-arch install must
not land in it. The relay addon builder now repairs and compiles a private
copy instead of the shared windows-process-tree dir.

* fix(build): accept CLI runtime imports reached through an in-artifact store link

Under the global virtual store node_modules/<pkg> realpaths into a store
outside the repo. The check still rejects anything not reached through a
node_modules link inside the artifact, such as a stray ~/node_modules.

* docs: explain the global virtual store layout and the ~/node_modules hazard

* test: compare CLI closure paths with POSIX separators so the suite passes on Windows

* build: keep Windows on the normal pnpm layout unless ORCA_PNPM_GLOBAL_VIRTUAL_STORE=1

Store links are true symlinks on Windows, and installs without admin or
Developer Mode are unverified; CI still forces the normal layout first.

* fix(build): close the remaining shared-slot gaps under the global virtual store

- verify-skills-cli-runtime resolves each import from the importer's realpath,
  as Node does, so an artifact-local copy can't mask a ~/node_modules load.
- A concurrent sibling install of the shared Electron dist is kept, not swapped out.
- rebuild:node goes through ensure-native-runtime, which builds a shared
  node-pty beside the slot instead of node-gyp cleaning its build/ dir.

* fix(build): keep pnpm rebuild:node a forced rebuild

It replaced pnpm rebuild node-pty, which always rebuilt; without --force it
only rebuilt a node-pty that failed to load.

* docs: note that the store switches must stay set, since pnpm run relinks

* build: move the pnpmfile under config/ to keep the repo root allowlisted

pnpm-workspace.yaml points pnpm at it; the lockfile checksum is unchanged
because the content is.

* fix(build): enable the global virtual store from the pnpmfile, not the workspace file

pnpm exec derives NODE_PATH from the file setting, so even with the store
forced off under CI every child process got the root node_modules on its
NODE_PATH; a Windows rebuild fixture then resolved the real
@orca/windows-registry it deliberately omits. The hook now sets the store
both ways, and ORCA_PNPM_GLOBAL_VIRTUAL_STORE=0/1 is the one switch.

* test(pty): keep F3's concurrent spawns on the node-pty mock

Two concurrent first import('node-pty') calls race vitest's mock registry
and the loser loads the real addon under the mocked linux platform. It
passed only where a platform-agnostic build/Release happened to exist,
so it failed on every shared-store worktree.

* ci: fail PRs whose packages need a dependency they never declared

A new dependency_isolation job installs with the global virtual store
forced on (ORCA_PNPM_FORCE_GLOBAL_VIRTUAL_STORE_IN_CI), HOME and the store
in an empty temp dir, then runs pnpm tc and the undeclared-dependency test.
Postinstall now warns when a node_modules above the shared store, such as
~/node_modules, would mask the same class of bug locally.

* ci: set the isolation job's HOME and store through GITHUB_ENV

Job-level env cannot read the runner context, which made pr.yml invalid.
2026-10-18 20:36:24 -07:00
OrcaWinandOrcaWin fc584d425b ci: skip checks for draft pull requests [skip ci] (#29484)
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
2026-10-15 23:18:13 -07:00
Neil e554ef2a78 ci: cancel obsolete PR checks even when new checks are skipped (#29364)
* ci: cancel superseded PR checks even when replacement checks are filtered out

* test(ci): verify cleanup keys against every automatic PR workflow
2026-10-15 20:12:36 -07:00
Neil 8df0f67c82 ci: skip redundant PR platform warmers (#28917) 2026-10-14 18:48:13 -07:00
Neil f23fa8ca76 Prepare Windows natives with scoped compiler settings and measure servicing (#28881)
* Scope Windows native compiler setup and measure ARM inbox preparation

* Add a controlled Windows native preparation comparison

* Test native preparation through an injected process runner

* Guard reviewed Windows native batch runners
2026-10-14 18:18:24 -07:00
Neil 6607c54984 perf(ci): cut redundant tests and reuse expensive setup (#28669)
* perf(ci): reduce test overhead and reuse expensive build work

* test(ci): keep worker startup outside failure-case deadlines

* test: isolate push fixture advisory locks

* test(ci): preserve fixed drain retry allowance

* Align CI contracts with current triggers and guide content

* Stabilize consolidated reveal scenarios and retain behavioral CI contracts

* Fix upstream lint blockers while retaining behavior and assertions

* Classify global Jira preferences in the WSL migration inventory

* Align Jira test fixtures with startup and build safety checks

* Include package-test routing dependency in CI checkouts and fixtures

* Keep native stream classes intact in transcript dependency tests
2026-10-14 16:14:27 -07:00
OrcaWinandOrca Worker 4559c250b3 fix(windows): restrict secure files in-process instead of spawning icacls (#28135)
* fix(windows): restrict secure files in-process instead of spawning icacls

Every secure write on Windows spawned icacls.exe five times on the calling
thread (the Electron main thread for most stores): verify, /reset,
/inheritance:r /grant:r, verify, verify. Process creation under endpoint
security costs ~0.6 s each, so each write froze the UI for seconds (#26027).
The /reset + /inheritance:r pair is also what behavioural EDR scores as
ransomware tradecraft (#25583).

Add native/windows-security (@orca/windows-security): a small N-API addon
over advapi32 that reads and writes DACLs in-process. It exposes the token
user SID, a DACL read as SDDL, a DACL write (sync and on the thread pool),
a create-with-DACL file write, and an additive grant. Policy and
verification stay in TypeScript.

- writeSecureFile creates each temp file already carrying the protected
  user/SYSTEM/Administrators DACL (CreateFileW + SECURITY_ATTRIBUTES), reads
  it back once, and renames it. No ACL is modified on the write path and no
  process starts. Before: 5-7 spawns, 94-138 ms per write. After: 0 spawns,
  ~0.5 ms.
- A secure directory is verified once per process. It is rewritten only
  when verification fails, on the thread pool, because the write walks
  every descendant (~2 s on a 29k-entry userData).
- The verifier follows OpenSSH's rule: only the user, SYSTEM and
  Administrators may be granted anything, and the path must be protected
  (or, for a file, inherit from a secure parent). DACLs written by the
  icacls-era hardening pass unchanged, so there is no migration.
- If the addon cannot load, the write still happens and reports
  unrestricted, logged once. There is no icacls or PowerShell fallback.
- grantDirAcl, the startup userData grant, and the install-dir probe and
  package repair use the same addon, so whoami.exe and icacls.exe are no
  longer spawned anywhere at runtime.
- orcad's Windows slots ship the addon beside their bundles (built per
  arch, PE-machine checked). Its data-root privacy check fails closed, so
  without the addon a Windows orcad would not start.

Fixes #26027
Fixes #25583

* refactor(secure-file): harden existing Windows files through the sync lane

The read path called restrictWindowsPathSync directly and recorded success
itself; the existing sync lane already does both. One call site fewer.

* refactor(windows-security): one load, one rule, no fallthrough (review #28135)

- The descriptor parser is anchored, so a conditional ACE (its body holds
  parentheses) or anything else it cannot read fails closed instead of
  verifying the ACEs before it.
- findSecureDaclViolation is one flat rule: protected, allow rules only for
  the user, SYSTEM and Administrators (OpenSSH's ACE rule), and full
  control for the user. The inherits-from-a-secure-parent exception is gone:
  it accepted exactly what loosening the parent reaches.
- The addon and the user SID load once, behind one wrapper that adds the
  extended-length prefix to every path and runs async DACL writes to one
  path in order, so a startup grant cannot land after a re-lock and restore
  the broader DACL.
- With the addon loaded, a failed born-restricted create throws with the
  matching Node error code instead of falling back to a plain write.
- A grant leaves a NULL DACL untouched instead of turning 'everyone' into
  'only this SID'. WriteFile runs once and fails on a short write.
- The artifact create-intent store writes through writeSecureFile and drops
  its own temp writer, directory LRU and synchronous directory re-lock.
- Test seams the global fake makes redundant are gone.

* fix(packaging): fail a Windows build whose workspace addon binary is missing or foreign-arch

The @orca/windows-* workspace links exist on every host, so their presence
proved nothing: a missing or wrong-architecture .node only surfaced at
runtime, as unrestricted secure files or an orcad that refuses to start.
beforePack now reads each addon's PE machine against the target arch.

* refactor(windows-security): prune what the native ACL left dead (review #28135)

- Sync-spawn ratchet: hardenSecurePath and the sync-flag exemption it alone
  used are gone, and the advice and baseline header no longer say secure
  writers run icacls.
- The install-dir probe reads ACEs through parseSddlDacl, so an ACE it
  cannot parse fails the whole reading closed instead of being skipped; AC
  is a resolved alias there now.
- The packaging check reads the addon binary through node_modules, the copy
  that ships.
- No winCode-to-errno table: nothing reads .code off a failed secure write.
- The load memo is not mutated; secure-path keeps its own log-once flag.
- grantInstallDirPackageAcl is a plain function, and secure-file restricts
  a Windows directory and a POSIX path directly instead of through one
  two-platform helper.

* fix(windows-security): resolve LA in real-ACL tests; tear a file whose close fails

- The GitHub Windows runner is the built-in Administrator (RID 500), whose
  SID reads back as LA. Production already resolves it through
  localDomainSidOf; the win32 test helpers parsed without it. One shared
  readDaclOf resolves aliases for every real-ACL suite, and the verifier's
  unit test covers an LA-aliased directory and LA as a stranger.
- createFileWithDacl treats a failed CloseHandle like a failed write (lost
  data on a network volume) and deletes the torn file.
- Drop a duplicate test reset and a typo.

* fix(windows-security): load only from inside the app; read DACLs as structure (review #28135)

- Loader: orcad's staged binary first, then the workspace package only if
  its real path is inside the app (process.resourcesPath, or this
  checkout). A bare specifier otherwise walks every parent up to
  C:\node_modules, which any local user may create, and orcad holds
  credentials.
- The addon reads a DACL as { protected, aces: [{ type, flags, mask, sid }] }
  via GetAce, with a SID only for plain allow/deny ACEs and null for a NULL
  DACL. findSecureDaclViolation and the install-dir probe check that
  structure, so conditional, callback and object ACEs fail closed. The SDDL
  parser, its alias table, LA/localDomainSid resolution and getDaclSddl are
  gone; SDDL is only the write format.
- Install-dir tree pass: files flagless as before, but a directory only when
  its own DACL is protected (inheritably). Re-granting inheriting
  directories re-propagated each subtree (3.2 s for 1,093 directories). The
  root is no longer granted, or counted, twice. The two passes are plain
  functions behind a { root, tree } seam.
- One namespacing (path.win32.toNamespacedPath) at the addon boundary, also
  the per-path ordering key; the comment states the real sync-grant
  invariant.
- restrictWindowsDirectory(dir): Promise<boolean> replaces the callback
  lane; the reporter is guarded once, so the 'settle' stage is gone.
- Sync-spawn ratchet: the secure writers no longer spawn or block, so they
  leave the gate (109 -> 61 uses).
- Errors name the Win32 call and code only; a failed
  GetSecurityDescriptorControl fails the write instead of defaulting to
  unprotected.

* test(secure-file): assert the durable POSIX write's outcome, not the old chmod order

writeSecureFile now chmods the temp file to 0o600 before fsyncing it, so
the fsync also covers the mode change and the file is owner-writable when
opened. Assert what matters under umask 0o200: the write succeeds, the
temp file is fsynced read-only, and the published file is exactly 0o600.

* fix(windows-security): namespace paths with the host's toNamespacedPath

path.win32.toNamespacedPath rewrote POSIX temp paths (/tmp/x -> \tmp\x) in the
Linux runs of the Windows-logic unit tests, so the fake addon wrote to the wrong
place. The addon only runs on Windows, where the host function is the win32 one;
elsewhere it is the identity. The long-path assertion runs on Windows only.

* test(windows-security): assert the install-dir long-path prefix on Windows only

The prefix comes from the host's toNamespacedPath, which is the identity off
Windows; the Linux run of this suite expected the win32 escape.

* fix(windows-user-data-acl): grant the root even when its children cannot be listed (review #28135)

A readdir failure on userData aborted the whole grant, so the root grant
that could repair a DACL refusing the listing was never attempted. The
listing failure is now recorded, the root is still granted, and the
marker stays withheld so the next launch retries.

* test(windows-security): wait for relock propagation, accept Entra ID SIDs (review #28135)

- The large-relock test waited only for the parent to turn protected,
  which happens before propagation reaches the children, then read a child.
  It now waits on the parent and both ends of the child range.
- The token-user test rejected S-1-12-1 (Entra ID) accounts, failing the
  suite on Entra-joined dev machines.

---------

Co-authored-by: Orca Worker <orca-worker@localhost>
2026-10-13 01:56:55 -07:00
Neil 845b723258 [CI] Warm only Linux for compiler recipe changes (#28081)
* [CI] Warm only Linux for compiler recipe changes

* [CI] Follow cache warmer callers in policy contracts
2026-10-12 16:34:58 -07:00
Neil e02e2bb1d9 [CI] Advance transcript clocks and skip test-only computer checks (#28060) 2026-10-12 13:46:53 -07:00
Neil ac156c0302 perf(ci): overlap Bun typechecking and skip redundant alias scans (#27382)
* perf(ci): skip unnecessary alias scans and compare preflight compilers

* perf(ci): overlap measured Bun typechecking with static checks

* perf(ci): use measured Bun lint runtime and strengthen compiler parity checks
2026-10-10 22:48:25 -07:00
Neil 316822f2eb ci: skip cache warming for cache-test-only changes (#26399) 2026-10-07 19:46:10 -07:00
Neil 6e6f651380 Avoid repeated pnpm archive downloads in cache producers (#24927)
* Let measured cache producers keep stores without downloading them

* Check that restore-only callers do not publish a producer path

* Enable the measured producer mode and record hosted comparisons
2026-10-02 20:37:54 -07:00
Neil a824fb74ab Reuse the headless detector compiler without installing full dependencies (#24895)
* Reuse the headless detector compiler without full dependency setup

* Keep optional compiler-cache saves from failing cache warming
2026-10-02 16:56:08 -07:00
Neil 61836f6026 Reduce scheduled CI cache warming to every six hours (#24881)
* Reduce scheduled CI cache warming to every six hours

* Document cache warmer recovery interval and measured tradeoff
2026-10-02 14:35:06 -07:00
Neil b94c75c4bd Reuse pnpm verification records in Alpine CI (#24817)
* ci: reuse pnpm verification records in Alpine builders

* ci: qualify consumers of the verification restore action

* ci: match Linux verification cache archive paths
2026-10-02 13:54:04 -07:00
Neil efbf651c7b Reduce CI setup costs and fixture failures (#24537)
* Let scheduled CI warmers wait and measure WebRTC startup

* Measure a smaller daemon shutdown fixture image

* Counterbalance WebRTC startup and verify retained fixture files

* Record CI fixture measurements and remove temporary pilots

* Clarify fixture build dependency cleanup evidence

* Make coalesced snapshot fixture delivery deterministic

* test: type the PTY write delay observer
2026-10-02 02:46:02 -07:00
Neil 8ff6296bc7 Speed up serializer checks and keep native caches stable (#24476)
* Reuse serializer oracle cells and isolate native cache policy

* Preserve native cache post-save paths and record hosted oracle gain

* Record native cache reuse and separate cancel-test startup budget
2026-10-01 21:43:56 -07:00
Neil 197ea3a3b3 Free PR CI capacity by avoiding repeated setup and real-time test waits (#24355)
* Reduce repeated PR setup and transcript timing waits; add hosted comparisons

* Align parallelism contract with Node-only external rebuild toolchain

* Record hosted coverage and launch package, store, and cancellation comparisons

* Apply hosted Windows setup savings and remove measured test waits

* Keep measured PR package gains and remove completed comparison jobs

* Report measured test counts with precise units
2026-10-01 11:51:43 -07:00
Neil d05080175b perf(ci): stop duplicating shared Linux download caches (#23578)
* perf(ci): pilot pnpm verification record caching on Linux

* test(ci): review pnpm verification record in mobile cache audit

* perf(ci): share Electron downloads and clean closed PR caches

* test(ci): retain cache ordering checks for restore-only consumers

* perf(ci): limit archive sharing rollout to primed Linux hosts
2026-09-28 01:58:46 -07:00
OrcaWinandm4air 47cebbf5d2 ci: use ARM unit runners, overlap web builds, and reuse verifier fixtures (#23376)
* test: reuse isolated mobile bundle fixtures for verifier checks

* ci: run PR unit shards on ARM and overlap independent web builds

* docs: record controlled CI overlap and runner measurements

* test: observe WebRTC packets with the host clock

* ci: isolate Windows installer CIM probe from native test load

* docs: record native probe scheduling validation

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
2026-09-27 01:06:54 -07:00
OrcaWinandm4air 3eb1adec20 ci: reuse fixture setup and scope localization extraction (#23291)
Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
2026-09-26 18:46:26 -07:00
OrcaWinandm4air 7a4f83336b ci: shard SSH Docker coverage and warm Windows native caches (#23286)
Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
2026-09-26 17:47:57 -07:00
OrcaWinandm4air 38bcdf76ac perf(ci): reduce queue pressure without paid runners (#23053)
* ci: measure complete unit file costs for shard balancing

* perf(ci): reduce repeated PR setup and capture complete shard timings

* perf(ci): seed reusable main-branch native and typecheck caches

* fix(ci): stop superseded unit workflows from resisting cancellation

* perf(ci): reuse bundle fixtures and share the baseline Git build

* ci: record hosted gains and refresh main hook parity

* ci: retain default workers after performance-budget regression

* docs: record hosted mobile timing flake

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
2026-09-25 22:48:28 -07:00