build: share one pnpm virtual store across worktrees (off in CI and on Windows) (#30254)

* build: share one pnpm virtual store across worktrees, off under CI

Each worktree carried a full node_modules; the global virtual store links
one slot per package version instead. pnpm keeps an explicit setting on
under CI, so a pnpmfile hook turns it off there: CI packages from,
caches and verifies a project-local node_modules.

* fix(build): keep shared electron and windows-process-tree dirs untouched under GVS

Every worktree runs Electron from the one shared electron package, so the
per-repo dist cache must not swap that dist and a foreign-arch install must
not land in it. The relay addon builder now repairs and compiles a private
copy instead of the shared windows-process-tree dir.

* fix(build): accept CLI runtime imports reached through an in-artifact store link

Under the global virtual store node_modules/<pkg> realpaths into a store
outside the repo. The check still rejects anything not reached through a
node_modules link inside the artifact, such as a stray ~/node_modules.

* docs: explain the global virtual store layout and the ~/node_modules hazard

* test: compare CLI closure paths with POSIX separators so the suite passes on Windows

* build: keep Windows on the normal pnpm layout unless ORCA_PNPM_GLOBAL_VIRTUAL_STORE=1

Store links are true symlinks on Windows, and installs without admin or
Developer Mode are unverified; CI still forces the normal layout first.

* fix(build): close the remaining shared-slot gaps under the global virtual store

- verify-skills-cli-runtime resolves each import from the importer's realpath,
  as Node does, so an artifact-local copy can't mask a ~/node_modules load.
- A concurrent sibling install of the shared Electron dist is kept, not swapped out.
- rebuild:node goes through ensure-native-runtime, which builds a shared
  node-pty beside the slot instead of node-gyp cleaning its build/ dir.

* fix(build): keep pnpm rebuild:node a forced rebuild

It replaced pnpm rebuild node-pty, which always rebuilt; without --force it
only rebuilt a node-pty that failed to load.

* docs: note that the store switches must stay set, since pnpm run relinks

* build: move the pnpmfile under config/ to keep the repo root allowlisted

pnpm-workspace.yaml points pnpm at it; the lockfile checksum is unchanged
because the content is.

* fix(build): enable the global virtual store from the pnpmfile, not the workspace file

pnpm exec derives NODE_PATH from the file setting, so even with the store
forced off under CI every child process got the root node_modules on its
NODE_PATH; a Windows rebuild fixture then resolved the real
@orca/windows-registry it deliberately omits. The hook now sets the store
both ways, and ORCA_PNPM_GLOBAL_VIRTUAL_STORE=0/1 is the one switch.

* test(pty): keep F3's concurrent spawns on the node-pty mock

Two concurrent first import('node-pty') calls race vitest's mock registry
and the loser loads the real addon under the mocked linux platform. It
passed only where a platform-agnostic build/Release happened to exist,
so it failed on every shared-store worktree.

* ci: fail PRs whose packages need a dependency they never declared

A new dependency_isolation job installs with the global virtual store
forced on (ORCA_PNPM_FORCE_GLOBAL_VIRTUAL_STORE_IN_CI), HOME and the store
in an empty temp dir, then runs pnpm tc and the undeclared-dependency test.
Postinstall now warns when a node_modules above the shared store, such as
~/node_modules, would mask the same class of bug locally.

* ci: set the isolation job's HOME and store through GITHUB_ENV

Job-level env cannot read the runner context, which made pr.yml invalid.
This commit is contained in:
Neil
2026-10-18 20:36:24 -07:00
committed by GitHub
parent df0c44571b
commit 4fc1855bda
31 changed files with 503 additions and 27 deletions
+18
View File
@@ -37,6 +37,24 @@ In PowerShell, use `$env:ORCA_DEV_MAIN_SOURCEMAPS='0'` before `pnpm dev`.
Remove the variable when debugging the main process with source maps. Renderer and
preload settings are unaffected; production builds always keep their hidden main maps.
On macOS and Linux, installs use pnpm's global virtual store: each package version is one slot
under `~/pnpm/store` that every worktree links to, instead of a full `node_modules` per worktree.
An existing worktree converts on its next `pnpm install`, and an older branch without the setting
converts back the same way. CI always uses the normal layout. Windows does too unless you set
`ORCA_PNPM_GLOBAL_VIRTUAL_STORE=1`, because its store links are real symlinks, which may need
administrator rights or Developer Mode. Things to know:
- Keep `~/node_modules` (and any other `node_modules` above the store) out of your home
directory. Node resolves upward from the store slot, so a stray one there supplies
dependencies the package never declared and hides missing-dependency bugs; `pnpm install` warns
when it finds one.
- Native addons are never rebuilt in place in a shared slot: each CPU gets its own build
beside it (see [`native-build-entry.cjs`](../config/scripts/native-build-entry.cjs)), so
cross-arch packaging is safe for other worktrees.
- `ORCA_PNPM_GLOBAL_VIRTUAL_STORE=1` opts in on Windows and `=0` opts out anywhere. Keep it set
in your shell: `pnpm run` re-checks the install and relinks a worktree whose layout no longer
matches.
Ordinary installs include native optional dependencies for the current OS and CPU only.
Before a cross-architecture build (including `pnpm build:mac`, which produces both x64 and
arm64 artifacts by default), run `pnpm install:release` to add the other CPU's variants.
@@ -14,7 +14,7 @@ runs:
- id: identity
shell: bash
env:
COMPILER_POLICY_HASH: ${{ hashFiles('package.json', 'pnpm-lock.yaml', 'pnpm-workspace.yaml', '.npmrc', '.pnpmfile.cjs', 'config/patches/**', '.github/actions/install-node-dependencies/**', '.github/actions/restore-pnpm-verification/**', 'config/scripts/headless-detector-compiler-cache.mjs', '.github/actions/prepare-headless-compiler/action.yml') }}
COMPILER_POLICY_HASH: ${{ hashFiles('package.json', 'pnpm-lock.yaml', 'pnpm-workspace.yaml', '.npmrc', 'config/pnpmfile.cjs', 'config/patches/**', '.github/actions/install-node-dependencies/**', '.github/actions/restore-pnpm-verification/**', 'config/scripts/headless-detector-compiler-cache.mjs', '.github/actions/prepare-headless-compiler/action.yml') }}
run: node config/scripts/headless-detector-compiler-cache.mjs identity
- id: cache
uses: actions/cache/restore@v5
@@ -54,7 +54,7 @@ runs:
env:
NATIVE_RUNTIME: ${{ inputs.native-runtime }}
NODE_VERSION: ${{ inputs.node-version }}
NATIVE_SOURCE_HASH: ${{ hashFiles('pnpm-lock.yaml', 'pnpm-workspace.yaml', '.npmrc', '.pnpmfile.cjs', '.github/actions/prepare-native-runtime/action.yml', 'config/scripts/ensure-native-runtime.mjs', 'config/scripts/initialize-windows-native-toolchain.mjs', 'config/scripts/capture-windows-native-environment.cmd', 'config/scripts/rebuild-native-deps.mjs', 'config/scripts/node-pty-job-ownership.cjs', 'config/scripts/windows-pe-machine.cjs', 'config/scripts/windows-process-tree-gyp-rebuild.mjs', 'config/scripts/windows-process-tree-creation-time.cjs', 'config/scripts/install-electron-package-binary.mjs', 'config/scripts/electron-platform-path.mjs', 'config/scripts/zip-extractor-command.mjs', 'src/shared/zip-extractor-command.ts', 'config/scripts/shared-electron-dist-cache.mjs', 'config/scripts/space-sharing-copy.mjs', 'config/patches/node-pty@1.1.0.patch', 'config/patches/@vscode__windows-process-tree@0.8.0.patch', 'native/windows-registry/src/addon.cc', 'native/windows-registry/binding.gyp', 'native/windows-registry/package.json', 'native/windows-registry/index.js', 'native/windows-security/src/addon.cc', 'native/windows-security/binding.gyp', 'native/windows-security/package.json', 'native/windows-security/index.js') }}
NATIVE_SOURCE_HASH: ${{ hashFiles('pnpm-lock.yaml', 'pnpm-workspace.yaml', '.npmrc', 'config/pnpmfile.cjs', '.github/actions/prepare-native-runtime/action.yml', 'config/scripts/ensure-native-runtime.mjs', 'config/scripts/initialize-windows-native-toolchain.mjs', 'config/scripts/capture-windows-native-environment.cmd', 'config/scripts/rebuild-native-deps.mjs', 'config/scripts/node-pty-job-ownership.cjs', 'config/scripts/windows-pe-machine.cjs', 'config/scripts/windows-process-tree-gyp-rebuild.mjs', 'config/scripts/windows-process-tree-creation-time.cjs', 'config/scripts/install-electron-package-binary.mjs', 'config/scripts/electron-platform-path.mjs', 'config/scripts/zip-extractor-command.mjs', 'src/shared/zip-extractor-command.ts', 'config/scripts/shared-electron-dist-cache.mjs', 'config/scripts/space-sharing-copy.mjs', 'config/patches/node-pty@1.1.0.patch', 'config/patches/@vscode__windows-process-tree@0.8.0.patch', 'native/windows-registry/src/addon.cc', 'native/windows-registry/binding.gyp', 'native/windows-registry/package.json', 'native/windows-registry/index.js', 'native/windows-security/src/addon.cc', 'native/windows-security/binding.gyp', 'native/windows-security/package.json', 'native/windows-security/index.js') }}
run: |
scope="${ImageOS:-$RUNNER_OS}"
if [ -r /etc/os-release ]; then
@@ -20,7 +20,7 @@ on:
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- '.npmrc'
- '.pnpmfile.cjs'
- 'config/pnpmfile.cjs'
- 'config/scripts/ensure-native-runtime.mjs'
- 'config/scripts/initialize-windows-native-toolchain.mjs'
- 'config/scripts/capture-windows-native-environment.cmd'
+1 -1
View File
@@ -29,7 +29,7 @@ on:
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- '.npmrc'
- '.pnpmfile.cjs'
- 'config/pnpmfile.cjs'
- 'config/scripts/ensure-native-runtime.mjs'
- 'config/scripts/initialize-windows-native-toolchain.mjs'
- 'config/scripts/capture-windows-native-environment.cmd'
+1 -1
View File
@@ -14,7 +14,7 @@ on:
- 'pnpm-workspace.yaml'
- 'tsconfig.json'
- '.npmrc'
- '.pnpmfile.cjs'
- 'config/pnpmfile.cjs'
- '.github/actions/install-node-dependencies/**'
- '.github/actions/restore-pnpm-verification/**'
- '.github/actions/prepare-headless-compiler/**'
+65
View File
@@ -53,6 +53,7 @@ jobs:
package: ${{ steps.readiness.outputs.reused != 'true' && steps.filter.outputs.package }}
package_windows: ${{ steps.readiness.outputs.reused != 'true' && steps.filter.outputs.package_windows }}
layout_oracle: ${{ steps.readiness.outputs.reused != 'true' && steps.filter.outputs.layout_oracle }}
dependency_isolation: ${{ steps.readiness.outputs.reused != 'true' && steps.filter.outputs.dependency_isolation }}
e2e_should_run: ${{ steps.e2e_filter.outputs.should_run }}
test_files: ${{ steps.e2e_filter.outputs.test_files }}
e2e_run_changed: ${{ steps.e2e_filter.outputs.e2e_run_changed }}
@@ -1557,6 +1558,66 @@ jobs:
with:
ref: ${{ github.event.pull_request.head.sha }}
# Why: pnpm's global virtual store links each package to only its declared dependencies, which
# is how developers install. Hoisting hides an undeclared one everywhere else, so install that way
# here with HOME and the store in an empty temp dir, where no stray node_modules can mask it.
dependency_isolation:
name: dependency isolation under the global virtual store
needs: [code_paths]
if: >-
!cancelled() &&
needs.code_paths.result == 'success' &&
needs.code_paths.outputs.dependency_isolation == 'true'
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 30
env:
ORCA_PNPM_FORCE_GLOBAL_VIRTUAL_STORE_IN_CI: '1'
steps:
# Job-wide (via GITHUB_ENV, since job env cannot read runner.temp) so every pnpm command,
# including pnpm run's own install check, keeps this layout.
- name: Point HOME and the pnpm store at empty temp dirs
run: |
mkdir -p "$RUNNER_TEMP/empty-home"
{
echo "HOME=$RUNNER_TEMP/empty-home"
echo "pnpm_config_store_dir=$RUNNER_TEMP/isolated-store"
} >> "$GITHUB_ENV"
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version-file: package.json
package-manager-cache: false
- name: Setup pnpm
uses: pnpm/setup@v2
with:
install: false
- name: Install with every package isolated in its own store slot
run: |
set -euo pipefail
pnpm install --frozen-lockfile --ignore-scripts
if [ -d node_modules/bun ]; then
pnpm rebuild bun
fi
- name: Typecheck and check undeclared dependency resolution
run: |
set -euo pipefail
pnpm tc
pnpm exec vitest run --config config/vitest.config.ts config/scripts/undeclared-dependency-resolution.test.ts
# Pointless if any pnpm command above fell back to the hoisted layout or another store.
case "$(realpath node_modules/zod)" in
"$(realpath "$pnpm_config_store_dir")"/*) ;;
*) echo "node_modules/zod is not in the isolated store"; exit 1 ;;
esac
verify:
name: PR checks passed
# Publish an explicit result after every required dependency settles.
@@ -1575,6 +1636,7 @@ jobs:
- package
- package_windows
- layout_oracle
- dependency_isolation
# Report settled results without competing with the GitHub-hosted test queue.
runs-on: blacksmith-2vcpu-ubuntu-2204
timeout-minutes: 5
@@ -1618,6 +1680,8 @@ jobs:
PACKAGE_WINDOWS_SHOULD_RUN: ${{ needs.code_paths.outputs.package_windows }}
LAYOUT_ORACLE: ${{ needs.layout_oracle.result }}
LAYOUT_ORACLE_SHOULD_RUN: ${{ needs.code_paths.outputs.layout_oracle }}
DEPENDENCY_ISOLATION: ${{ needs.dependency_isolation.result }}
DEPENDENCY_ISOLATION_SHOULD_RUN: ${{ needs.code_paths.outputs.dependency_isolation }}
run: |
if [ "$CODE_PATHS" != "success" ]; then
exit 1
@@ -1660,4 +1724,5 @@ jobs:
check_job package "$PACKAGE" "$PACKAGE_SHOULD_RUN"
check_job package_windows "$PACKAGE_WINDOWS" "$PACKAGE_WINDOWS_SHOULD_RUN"
check_job layout_oracle "$LAYOUT_ORACLE" "$LAYOUT_ORACLE_SHOULD_RUN"
check_job dependency_isolation "$DEPENDENCY_ISOLATION" "$DEPENDENCY_ISOLATION_SHOULD_RUN"
exit "$failed"
+44
View File
@@ -0,0 +1,44 @@
'use strict'
// Self-contained on purpose: scratch installs copy this file alone, beside the lockfile that pins its checksum.
const GLOBAL_VIRTUAL_STORE_ENV = 'ORCA_PNPM_GLOBAL_VIRTUAL_STORE'
// CI-only: the dependency-isolation job installs with the store on so undeclared deps fail PRs.
const FORCE_IN_CI_ENV = 'ORCA_PNPM_FORCE_GLOBAL_VIRTUAL_STORE_IN_CI'
function isCi(env) {
return (Boolean(env.CI) && env.CI !== 'false' && env.CI !== '0') || env.GITHUB_ACTIONS === 'true'
}
/**
* Whether this install uses pnpm's global virtual store: one store slot per package version,
* shared by every worktree, instead of a full node_modules each.
*
* Off under CI, which packages from, caches, and verifies a project-local node_modules, unless
* `ORCA_PNPM_FORCE_GLOBAL_VIRTUAL_STORE_IN_CI=1` (the dependency-isolation job).
* Windows needs `ORCA_PNPM_GLOBAL_VIRTUAL_STORE=1` because its store links are true symlinks,
* unverified for users without admin or Developer Mode; `=0` opts out anywhere else.
*/
function usesGlobalVirtualStore({ env = process.env, platform = process.platform } = {}) {
if (isCi(env)) {
return env[FORCE_IN_CI_ENV] === '1'
}
const choice = env[GLOBAL_VIRTUAL_STORE_ENV]
if (choice === '0' || choice === '1') {
return choice === '1'
}
return platform !== 'win32'
}
module.exports = {
hooks: {
updateConfig(config) {
// Why here and not pnpm-workspace.yaml: pnpm exec derives NODE_PATH from the file setting,
// which put the root node_modules on every CI child's NODE_PATH even with the store off.
config.enableGlobalVirtualStore = usesGlobalVirtualStore()
return config
}
},
FORCE_IN_CI_ENV,
GLOBAL_VIRTUAL_STORE_ENV,
usesGlobalVirtualStore
}
@@ -19,9 +19,21 @@
* node config/scripts/build-windows-process-tree-addon.mjs --arch=arm64
*/
import { execFileSync } from 'node:child_process'
import { copyFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'
import {
copyFileSync,
cpSync,
existsSync,
mkdirSync,
mkdtempSync,
readFileSync,
realpathSync,
rmSync,
writeFileSync
} from 'node:fs'
import { createRequire } from 'node:module'
import { join, resolve } from 'node:path'
import { tmpdir } from 'node:os'
import { join, relative, resolve } from 'node:path'
import { isSharedPackageDir } from './native-build-entry.cjs'
import { canReusePreparedAddon } from './windows-process-tree-prepared-addon.mjs'
import { ORCAD_WINDOWS_PROCESS_TREE_FILENAME } from '../../src/shared/orcad-artifacts.ts'
import {
@@ -30,7 +42,7 @@ import {
nodeGypRebuildInvocation,
stageWindowsProcessTreeNodeAddonApiHeaders,
windowsProcessTreeAddonHasBreakawayLauncher,
WINDOWS_PROCESS_TREE_PACKAGE_DIR as PACKAGE_DIR
WINDOWS_PROCESS_TREE_PACKAGE_DIR as INSTALLED_PACKAGE_DIR
} from './windows-process-tree-gyp-rebuild.mjs'
const { PE_MACHINE, describePeMachine, readPeMachine } = createRequire(import.meta.url)(
@@ -39,6 +51,21 @@ const { PE_MACHINE, describePeMachine, readPeMachine } = createRequire(import.me
const ROOT = resolve(import.meta.dirname, '..', '..')
const SUPPORTED_ARCHES = ['x64', 'arm64']
// Why: under pnpm's global virtual store the installed dir is every worktree's, so the source
// repairs and the build below run in a private copy instead.
const SCRATCH_DIR =
existsSync(INSTALLED_PACKAGE_DIR) && isSharedPackageDir(INSTALLED_PACKAGE_DIR, ROOT)
? mkdtempSync(join(tmpdir(), 'orca-windows-process-tree-'))
: null
const PACKAGE_DIR = SCRATCH_DIR === null ? INSTALLED_PACKAGE_DIR : join(SCRATCH_DIR, 'package')
if (SCRATCH_DIR !== null) {
const physicalDir = realpathSync(INSTALLED_PACKAGE_DIR)
cpSync(physicalDir, PACKAGE_DIR, {
recursive: true,
dereference: true,
filter: (path) => relative(physicalDir, path) !== 'build'
})
}
function parseArgs(argv) {
const arch = argv.find((a) => a.startsWith('--arch='))?.slice('--arch='.length) ?? process.arch
@@ -409,7 +436,7 @@ function applyWindowsProcessTreeBuildFixes() {
writeFileSync(processPath, processCc)
}
const repairedCreationTime = repairCreationTimeSources()
stageWindowsProcessTreeNodeAddonApiHeaders(PACKAGE_DIR)
stageWindowsProcessTreeNodeAddonApiHeaders(PACKAGE_DIR, INSTALLED_PACKAGE_DIR)
const repairedCommandLine = ensureWindowsProcessTreeCommandLinePatch(PACKAGE_DIR)
const repaired =
bindingGyp !== originalBinding ||
@@ -447,7 +474,7 @@ function main() {
) {
console.log(`[windows-process-tree] reusing the prepared ${arch} addon`)
} else {
const gyp = nodeGypRebuildInvocation(arch)
const gyp = nodeGypRebuildInvocation(arch, PACKAGE_DIR)
console.log(`[windows-process-tree] building ${arch} from ${gyp.cwd}`)
execFileSync(process.execPath, gyp.args, { cwd: gyp.cwd, stdio: 'inherit' })
}
@@ -489,5 +516,9 @@ try {
main()
} catch (error) {
console.error(`[windows-process-tree] ${error instanceof Error ? error.message : String(error)}`)
process.exit(1)
process.exitCode = 1
} finally {
if (SCRATCH_DIR !== null) {
rmSync(SCRATCH_DIR, { recursive: true, force: true })
}
}
@@ -32,6 +32,8 @@ const manifests = [
'package.json',
'pnpm-lock.yaml',
'pnpm-workspace.yaml',
// The lockfile pins its checksum, so a frozen install fails without it.
'config/pnpmfile.cjs',
...workspacePackageManifests(repository)
]
const pnpm = resolvePnpmCliInvocation()
+1 -1
View File
@@ -60,7 +60,7 @@ export async function moduleCachePolicyHash(root = ROOT) {
'pnpm-lock.yaml',
'pnpm-workspace.yaml',
'.npmrc',
'.pnpmfile.cjs',
'config/pnpmfile.cjs',
'config/.bun-version',
'config/scripts/ci-unit-module-cache.mjs',
'config/scripts/ci-unit-module-cache-contents.mjs',
+7 -3
View File
@@ -33,6 +33,8 @@ const { assertWindowsProcessTreeCreationTime } = require('./windows-process-tree
const scriptPath = import.meta.filename
const projectDir = resolve(import.meta.dirname, '../..')
const runtime = readRuntimeArg()
// Why: `pnpm rebuild:node` must rebuild even a loadable node-pty; a shared slot still reuses its usable entry.
const forceRebuild = process.argv.includes('--force')
const NATIVE_MODULES = [
'node-pty',
@@ -80,12 +82,14 @@ function readRuntimeArg() {
function ensureNodeRuntime() {
const initial = runNodeCheck()
const patchedNodePtyRebuildReason = getPatchedNodePtyRebuildReason()
if (initial.ok && !patchedNodePtyRebuildReason) {
if (initial.ok && !patchedNodePtyRebuildReason && !forceRebuild) {
return
}
if (patchedNodePtyRebuildReason) {
console.warn(`[native-runtime] ${patchedNodePtyRebuildReason}`)
if (patchedNodePtyRebuildReason || forceRebuild) {
console.warn(
`[native-runtime] ${patchedNodePtyRebuildReason ?? 'Forced rebuild of node-pty for Node.'}`
)
if (!initial.ok) {
printCheckError(initial)
}
@@ -17,6 +17,7 @@ import { platform as osPlatform, tmpdir } from 'node:os'
import { join, resolve } from 'node:path'
import { getElectronPlatformPath } from './electron-platform-path.mjs'
import { getZipExtractorCommand } from './zip-extractor-command.mjs'
import { isSharedPackageDir } from './native-build-entry.cjs'
import {
shareElectronDistFromCache,
hasAdoptedSharedElectronDist,
@@ -77,6 +78,8 @@ async function main() {
return
}
assertForeignTargetLeavesSharedElectronAlone()
// Why: PR tests run under system Node after native modules are rebuilt for
// Node. Install only Electron's npm package binary here; do not run the full
// Electron native-module rebuild path, which would undo the Node ABI rebuild.
@@ -92,6 +95,25 @@ async function main() {
}
}
/**
* node_modules/electron/dist exists to run Electron on this machine; packaging downloads its own
* Electron per target. Under pnpm's global virtual store that dir is shared by every worktree, so a
* foreign-target install would hand them all an Electron they cannot run.
*/
function assertForeignTargetLeavesSharedElectronAlone() {
if (
(targetPlatform === osPlatform() && targetArch === process.arch) ||
!isSharedPackageDir(electronPackageDir, projectDir)
) {
return
}
throw new Error(
`Refusing to install a ${targetPlatform}-${targetArch} Electron into ${electronPackageDir}, ` +
'which every worktree shares through the global virtual store. Unset ELECTRON_INSTALL_ARCH ' +
'/ npm_config_arch, or install with ORCA_PNPM_GLOBAL_VIRTUAL_STORE=0.'
)
}
function electronPackageIsUsable() {
try {
const installedPlatformPath = readFileSync(resolve(electronPackageDir, 'path.txt'), 'utf8')
@@ -179,6 +201,10 @@ async function installElectronPackageBinary(sharedEntry) {
process.exit(1)
}
// Why: a worktree sharing this slot may have installed it during the download; never swap out a dist it runs.
if (isSharedPackageDir(electronPackageDir, projectDir) && electronPackageIsUsable()) {
return
}
moveExtractedElectronDist(extractDir, electronDistDir)
if (sharedEntry !== null) {
publishElectronDistForSiblingWorktrees(sharedEntry, electronDistDir)
@@ -7,7 +7,7 @@ const TOOLCHAIN_FILES = new Set([
'pnpm-workspace.yaml',
'tsconfig.json',
'.npmrc',
'.pnpmfile.cjs',
'config/pnpmfile.cjs',
'.github/workflows/pr.yml'
])
@@ -29,7 +29,7 @@ it.each([
'tsconfig.json',
'config/tsconfig.web.json',
'.npmrc',
'.pnpmfile.cjs',
'config/pnpmfile.cjs',
'.github/workflows/pr.yml',
'.github/actions/install-node-dependencies/action.yml'
])('retains extraction when an input changes: %s', (path) => {
+1 -1
View File
@@ -34,7 +34,7 @@ const ALWAYS_FILES = new Set([
'pnpm-lock.yaml',
'pnpm-workspace.yaml',
'.npmrc',
'.pnpmfile.cjs',
'config/pnpmfile.cjs',
'tsconfig.json',
'.github/workflows/node-server-tests.yml',
'config/scripts/node-server-change-scope.mjs',
@@ -63,6 +63,8 @@ describe('patched dependencies', () => {
'package.json',
'pnpm-lock.yaml',
'pnpm-workspace.yaml',
// The lockfile pins its checksum, so a frozen install fails without it.
'config/pnpmfile.cjs',
...workspacePackageManifests(PROJECT_DIR)
]) {
mkdirSync(resolve(scratch, file, '..'), { recursive: true })
@@ -0,0 +1,42 @@
import { createRequire } from 'node:module'
import { describe, expect, it } from 'vitest'
const require = createRequire(import.meta.url)
const {
FORCE_IN_CI_ENV,
GLOBAL_VIRTUAL_STORE_ENV: STORE_ENV,
usesGlobalVirtualStore,
hooks
} = require('../pnpmfile.cjs')
describe('usesGlobalVirtualStore', () => {
const cases = [
// [env, platform, expected]
[{}, 'darwin', true],
[{}, 'linux', true],
[{}, 'win32', false],
[{ [STORE_ENV]: '1' }, 'win32', true],
[{ [STORE_ENV]: '0' }, 'win32', false],
[{ [STORE_ENV]: '0' }, 'darwin', false],
[{ [STORE_ENV]: 'yes' }, 'linux', true],
[{ CI: 'true' }, 'darwin', false],
[{ CI: '1' }, 'linux', false],
[{ GITHUB_ACTIONS: 'true' }, 'linux', false],
[{ CI: 'true', [STORE_ENV]: '1' }, 'win32', false],
[{ CI: 'true', [FORCE_IN_CI_ENV]: '1' }, 'linux', true],
[{ GITHUB_ACTIONS: 'true', [FORCE_IN_CI_ENV]: '1' }, 'win32', true],
[{ [FORCE_IN_CI_ENV]: '1', [STORE_ENV]: '0' }, 'darwin', false],
[{ CI: 'false' }, 'darwin', true],
[{ CI: '0' }, 'linux', true]
]
it.each(cases)('env %j on %s -> %s', (env, platform, expected) => {
expect(usesGlobalVirtualStore({ env, platform })).toBe(expected)
})
})
describe('config/pnpmfile.cjs updateConfig', () => {
it('sets the store from the policy, so pnpm-workspace.yaml never has to', () => {
const config = hooks.updateConfig({})
expect(config.enableGlobalVirtualStore).toBe(usesGlobalVirtualStore())
})
})
+17 -2
View File
@@ -33,7 +33,8 @@ export const PR_CHECK_JOBS = [
'managed_hook_node18',
'package',
'package_windows',
'layout_oracle'
'layout_oracle',
'dependency_isolation'
]
const ALWAYS_ON_CODE_JOBS = new Set(['static_analysis', 'typecheck', 'test'])
@@ -158,6 +159,18 @@ const ORCAD_BROWSER_PREFIXES = [
// The page bundle the desktop packages: the builder and verifier, the manifest writer and the
// packaging guard they share, the entry, the route tree it mounts, the mobile source those routes
// import, and the shell policy the render check runs the page under.
// What decides which deps each package sees; package.json and the lockfile already force every job.
const DEPENDENCY_ISOLATION_PREFIXES = [
'pnpm-workspace.yaml',
'config/pnpmfile.cjs',
'config/scripts/undeclared-dependency-resolution',
'config/build-plugins/markdown-parser-exports',
'config/tsconfig',
'src/packages/',
'native/windows-registry/package.json',
'native/windows-security/package.json'
]
const MOBILE_WEB_APP_PREFIXES = [
'config/scripts/build-mobile-web-app',
'config/scripts/run-mobile-web-app-checks',
@@ -294,7 +307,7 @@ const NATIVE_CACHE_FILES = new Set([
'.github/actions/prepare-native-runtime/action.yml',
'pnpm-workspace.yaml',
'.npmrc',
'.pnpmfile.cjs',
'config/pnpmfile.cjs',
'config/scripts/ensure-native-runtime.mjs',
'config/scripts/initialize-windows-native-toolchain.mjs',
'config/scripts/capture-windows-native-environment.cmd',
@@ -553,6 +566,8 @@ function jobDetector(job) {
return (files) => files.some(isWindowsPackagePath)
case 'layout_oracle':
return (files) => files.some(isLayoutOracleSource)
case 'dependency_isolation':
return (files) => files.some((file) => matchesPrefix(file, DEPENDENCY_ISOLATION_PREFIXES))
default:
return () => true
}
@@ -203,6 +203,15 @@ describe('per-job path classification', () => {
})
})
it('runs dependency isolation only when what packages may resolve changes', () => {
expectClassification(['config/scripts/undeclared-dependency-resolution.test.ts'], {
dependency_isolation: true
})
expect(classifyPrJobs(['pnpm-workspace.yaml']).dependency_isolation).toBe(true)
expect(classifyPrJobs(['config/pnpmfile.cjs']).dependency_isolation).toBe(true)
expect(classifyPrJobs(['src/renderer/src/App.tsx']).dependency_isolation).toBe(false)
})
it('runs native package jobs only for the platform that ships the changed native', () => {
expectClassification(['native/windows-cli-launcher/src/main.rs'], { package_windows: true })
expectClassification(['native/computer-use-linux/runtime.py'], { package: true })
+2
View File
@@ -31,6 +31,7 @@ import {
windowsProcessTreeAddonPath
} from './windows-process-tree-gyp-rebuild.mjs'
import { disableMsbuildFileTrackingOnWindows } from './msbuild-file-tracking.mjs'
import { warnAboutNodeModulesAboveSharedStore } from './store-ancestor-node-modules.mjs'
import {
isSharedNativePackage,
nodePtyBuildArtifactPaths,
@@ -105,6 +106,7 @@ const forceRebuild =
process.env.ORCA_FORCE_NATIVE_REBUILD === '1' || cliOptions.force || isCrossHostRebuild
let modulesToRebuild = onlyModules
warnAboutNodeModulesAboveSharedStore(projectDir)
ensureElectronPackageInstalled()
restoreNodePtyWindowsConptyRuntime()
@@ -4,6 +4,7 @@ import { existsSync, lstatSync, mkdirSync, readFileSync, rmSync } from 'node:fs'
import path from 'node:path'
import { makeTreeReadOnly, shareTree } from './space-sharing-copy.mjs'
import { publishSharedTree, sharedTreeStagingPath } from './shared-tree-publish.mjs'
import { isSharedPackageDir } from './native-build-entry.cjs'
const IDENTITY_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]*$/
// Sibling of path.txt, never inside dist: an install replaces dist wholesale.
@@ -24,6 +25,11 @@ export function resolveSharedElectronDistEntry(options) {
if (![version, targetPlatform, targetArch].every((part) => IDENTITY_PATTERN.test(part ?? ''))) {
return null
}
// Under pnpm's global virtual store every worktree already shares one electron package dir;
// adopting a cache entry would swap the dist out from under worktrees running it.
if ((options.isSharedPackageDir ?? isSharedPackageDir)(options.electronPackageDir, repoRoot)) {
return null
}
let gitCommonDir
try {
gitCommonDir = resolveGitCommonDir(repoRoot, options.execFile ?? execFileSync)
@@ -82,6 +82,12 @@ describe('resolveSharedElectronDistEntry', () => {
}
})
it('declines when the global virtual store already shares the electron package', () => {
expect(
resolveSharedElectronDistEntry({ ...baseOptions, isSharedPackageDir: () => true })
).toBeNull()
})
it('declines on CI, where every job gets a fresh checkout', () => {
expect(resolveSharedElectronDistEntry({ ...baseOptions, env: { CI: '1' } })).toBeNull()
expect(resolveSharedElectronDistEntry({ ...baseOptions, env: { CI: 'true' } })).toBeNull()
@@ -0,0 +1,46 @@
import { existsSync, realpathSync } from 'node:fs'
import { basename, dirname, join } from 'node:path'
import { isSharedPackageDir } from './native-build-entry.cjs'
/**
* node_modules dirs above the global virtual store's `links` root, given one store slot's path.
*
* Node resolves upward from a slot, so a stray `~/node_modules` (or `/tmp/node_modules`) above
* the store hands every package dependencies it never declared and hides exactly the bugs the
* isolated layout exists to surface.
*/
export function nodeModulesAboveStore(slotPackageDir, exists = existsSync) {
let storeRoot = slotPackageDir
while (basename(storeRoot) !== 'links') {
const parent = dirname(storeRoot)
if (parent === storeRoot) {
return []
}
storeRoot = parent
}
const found = []
for (let dir = dirname(storeRoot); ; dir = dirname(dir)) {
if (exists(join(dir, 'node_modules'))) {
found.push(join(dir, 'node_modules'))
}
if (dirname(dir) === dir) {
return found
}
}
}
/** Warns, never fails: the install itself is fine, only undeclared-dependency checks are blinded. */
export function warnAboutNodeModulesAboveSharedStore(projectDir, probePackage = 'zod') {
const linked = join(projectDir, 'node_modules', probePackage)
if (!existsSync(linked) || !isSharedPackageDir(linked, projectDir)) {
return []
}
const found = nodeModulesAboveStore(realpathSync(linked))
for (const dir of found) {
console.warn(
`[gvs] ${dir} sits above the shared pnpm store, so packages there resolve dependencies ` +
'they never declared and undeclared-dependency bugs stay hidden. Move or remove it.'
)
}
return found
}
@@ -0,0 +1,49 @@
import { join } from 'node:path'
import { describe, expect, it } from 'vitest'
import { nodeModulesAboveStore } from './store-ancestor-node-modules.mjs'
const slot = join(
'/',
'home',
'dev',
'pnpm',
'store',
'v11',
'links',
'@',
'zod',
'4.6.5',
'h',
'node_modules',
'zod'
)
describe('nodeModulesAboveStore', () => {
it('reports node_modules above the store root, not the slot-local ones', () => {
const present = new Set([
join('/', 'home', 'dev', 'node_modules'),
join(
'/',
'home',
'dev',
'pnpm',
'store',
'v11',
'links',
'@',
'zod',
'4.6.5',
'h',
'node_modules'
)
])
expect(nodeModulesAboveStore(slot, (path) => present.has(path))).toEqual([
join('/', 'home', 'dev', 'node_modules')
])
})
it('reports nothing for a clean store or a path outside any store', () => {
expect(nodeModulesAboveStore(slot, () => false)).toEqual([])
expect(nodeModulesAboveStore(join('/', 'repo', 'node_modules', 'zod'), () => true)).toEqual([])
})
})
+46 -4
View File
@@ -53,9 +53,8 @@ function isOptionalPackageImport(artifactRoot, importer, specifier) {
if (specifier.startsWith('.') || isAbsolute(specifier)) {
return false
}
const segments = specifier.split('/')
const packageName = specifier.startsWith('@') ? segments.slice(0, 2).join('/') : segments[0]
let directory = realpathSync(dirname(importer))
const packageName = bareSpecifierPackage(specifier)
let directory = dirname(importer)
while (!isOutsideRoot(artifactRoot, directory)) {
const packageJson = join(directory, 'package.json')
@@ -78,13 +77,52 @@ function isOptionalPackageImport(artifactRoot, importer, specifier) {
return false
}
function bareSpecifierPackage(specifier) {
const segments = specifier.split('/')
return specifier.startsWith('@') ? segments.slice(0, 2).join('/') : segments[0]
}
/**
* The in-root path of a package import whose realpath leaves the root.
*
* Under pnpm's global virtual store `<root>/node_modules/<pkg>` links into a store shared by every
* worktree, so Node's realpath lands outside the root even though the artifact does carry the
* dependency. Accept it only when a node_modules link inside the root really leads there.
*/
function linkedPathInsideRoot(artifactRoot, importer, specifier, resolved) {
const isBare = !specifier.startsWith('.') && !isAbsolute(specifier)
let directory = dirname(importer)
while (!isOutsideRoot(artifactRoot, directory)) {
// A bare import reaches the store through node_modules/<pkg>; a relative one through the
// linked package dir it sits in.
const candidates = isBare
? [join(directory, 'node_modules', bareSpecifierPackage(specifier)), directory]
: [directory]
for (const candidate of candidates) {
if (!existsSync(candidate)) {
continue
}
const physical = realpathSync(candidate)
if (physical !== candidate && !isOutsideRoot(physical, resolved)) {
return join(candidate, relative(physical, resolved))
}
}
if (directory === artifactRoot) {
break
}
directory = dirname(directory)
}
return null
}
function resolveRuntimeImport(outDir, artifactRoot, importer, specifier) {
if (BUILTINS.has(specifier) || isBuiltin(specifier)) {
return null
}
let resolved
try {
resolved = createRequire(importer).resolve(specifier)
// Why physical: Node resolves from the realpath, so a store-linked importer must not see the artifact's own node_modules.
resolved = createRequire(realpathSync(importer)).resolve(specifier)
} catch (error) {
if (isOptionalPackageImport(artifactRoot, importer, specifier)) {
return null
@@ -96,6 +134,10 @@ function resolveRuntimeImport(outDir, artifactRoot, importer, specifier) {
)
}
if (isOutsideRoot(artifactRoot, resolved)) {
const linked = linkedPathInsideRoot(artifactRoot, importer, specifier, resolved)
if (linked !== null) {
return linked
}
throw new Error(
`[verify-skills-cli-runtime] import "${specifier}" from ` +
`${artifactPath(outDir, importer)} resolved outside ${artifactRoot}: ${resolved}`
@@ -1,4 +1,4 @@
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { mkdir, mkdtemp, readFile, rm, symlink, writeFile } from 'node:fs/promises'
import { realpathSync } from 'node:fs'
import { createRequire } from 'node:module'
import { tmpdir } from 'node:os'
@@ -65,7 +65,7 @@ describe('skills CLI runtime closure', () => {
expect(
collectRuntimeClosure(root)
.map((file) => relative(realpathSync(root), file))
.map((file) => relative(realpathSync(root), file).split('\\').join('/'))
.sort()
).toEqual(['cli/handlers/skills.js', 'cli/index.js', 'shared/first.js', 'shared/second.js'])
} finally {
@@ -139,6 +139,56 @@ describe('skills CLI runtime closure', () => {
}
})
it('follows an in-artifact node_modules link into a shared store', async () => {
const root = await mkdtemp(join(tmpdir(), 'orca-skills-cli-closure-'))
try {
const artifactRoot = join(root, 'artifact')
const outDir = join(artifactRoot, 'out')
// pnpm's global virtual store: the artifact links to a slot that lives outside it.
const slotDir = join(root, 'store', 'linked-package')
await writeSkillsCliFixture(outDir, "require('linked-package')\n")
await mkdir(slotDir, { recursive: true })
await writeFile(join(slotDir, 'package.json'), JSON.stringify({ main: 'index.js' }), 'utf8')
await writeFile(join(slotDir, 'index.js'), "require('./inner')\n", 'utf8')
await writeFile(join(slotDir, 'inner.js'), '', 'utf8')
await mkdir(join(artifactRoot, 'node_modules'), { recursive: true })
await symlink(slotDir, join(artifactRoot, 'node_modules', 'linked-package'), 'junction')
const closure = collectRuntimeClosure(outDir, artifactRoot).map((file) =>
relative(realpathSync(artifactRoot), file).split('\\').join('/')
)
expect(closure).toContain('node_modules/linked-package/inner.js')
} finally {
await rm(root, { recursive: true, force: true })
}
})
it("resolves a linked package's imports from its store slot, as Node does", async () => {
const root = await mkdtemp(join(tmpdir(), 'orca-skills-cli-closure-'))
try {
const artifactRoot = join(root, 'artifact')
const outDir = join(artifactRoot, 'out')
const slotDir = join(root, 'store', 'linked-package')
await writeSkillsCliFixture(outDir, "require('linked-package')\n")
await mkdir(slotDir, { recursive: true })
await writeFile(join(slotDir, 'package.json'), JSON.stringify({ main: 'index.js' }), 'utf8')
await writeFile(join(slotDir, 'index.js'), "require('dep')\n", 'utf8')
// Node reaches the ancestor copy (like ~/node_modules) from the slot, not the artifact's own.
for (const dir of [
join(root, 'node_modules', 'dep'),
join(artifactRoot, 'node_modules', 'dep')
]) {
await mkdir(dir, { recursive: true })
await writeFile(join(dir, 'index.js'), '', 'utf8')
}
await symlink(slotDir, join(artifactRoot, 'node_modules', 'linked-package'), 'junction')
expect(() => collectRuntimeClosure(outDir, artifactRoot)).toThrow(/resolved outside/)
} finally {
await rm(root, { recursive: true, force: true })
}
})
it('rejects package dependencies resolved outside the artifact', async () => {
const root = await mkdtemp(join(tmpdir(), 'orca-skills-cli-closure-'))
try {
+1 -1
View File
@@ -123,7 +123,7 @@
"postinstall": "pnpm run build:packages && node config/scripts/rebuild-native-deps.mjs",
"rebuild:electron": "node config/scripts/rebuild-native-deps.mjs",
"reclaim-space": "node config/scripts/reclaim-worktree-space.mjs",
"rebuild:node": "pnpm rebuild node-pty",
"rebuild:node": "node config/scripts/ensure-native-runtime.mjs --runtime=node --force",
"build:unpack": "pnpm run build && pnpm run ensure:electron-runtime && electron-builder --config config/electron-builder.config.cjs --dir",
"build:win": "pnpm run build:desktop && pnpm run ensure:electron-runtime && electron-builder --config config/electron-builder.config.cjs --win",
"build:icons": "bash resources/icon-source/generate.sh",
+2
View File
@@ -169,6 +169,8 @@ overrides:
packageExtensionsChecksum: sha256-TrarseUCmMKP0lVjkfyLP2TAvkGZMZzvzPxjKY48nrY=
pnpmfileChecksum: sha256-568iOYVj6ZdABzEY52uLAgKwLTmlyqdEBiaV2dJCY74=
patchedDependencies:
'@vscode/windows-process-tree@0.8.0': a09f2e8b2b4e73ef20c317af91f9e06c09abf62fa2c891d99b16f53047f596fd
'@xterm/addon-image@0.10.0-beta.301': f088c9207dac42844a542b9da8078be52351cee60585725ef189b9fd30f33f34
+4
View File
@@ -79,6 +79,10 @@ minimumReleaseAgeExclude:
- oxlint@1.88.0
shamefullyHoist: true
# Enables pnpm's global virtual store outside CI; under config/ because the repo root only admits
# allowlisted entries.
pnpmfile: config/pnpmfile.cjs
# Orca always launches the user's own resolved Claude CLI via
# pathToClaudeCodeExecutable, so the SDK's bundled ~95 MB-per-platform CLI
# binaries must never be installed. Excluding them is what makes the path
@@ -1,5 +1,6 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
import type * as MacosTccLoginShell from './macos-tcc-login-shell'
import type { ShellSpawnParams } from './local-pty-utils'
const {
existsSyncMock,
@@ -53,6 +54,16 @@ vi.mock('node-pty', () => ({
spawn: spawnMock
}))
// Why: F3's two concurrent first `import('node-pty')` calls race vitest's mock registry, and the
// loser loads the real addon under the mocked platform; hand the provider the mock directly.
vi.mock('./local-pty-runtime-spawn', async () => {
const { spawnShellWithFallback } = await import('./local-pty-utils')
return {
loadLocalPtyRuntimeSpawn: async () => (params: Omit<ShellSpawnParams, 'ptySpawn'>) =>
spawnShellWithFallback({ ...params, ptySpawn: spawnMock })
}
})
vi.mock('./macos-tcc-login-shell', async (importOriginal) => ({
...(await importOriginal<typeof MacosTccLoginShell>()),
prepareMacosTccLoginShell: prepareMacosTccLoginShellMock