Files
orca/config/scripts/node-server-change-scope.mjs
T
Neil 4fc1855bda build: share one pnpm virtual store across worktrees (off in CI and on Windows) (#30254)
* build: share one pnpm virtual store across worktrees, off under CI

Each worktree carried a full node_modules; the global virtual store links
one slot per package version instead. pnpm keeps an explicit setting on
under CI, so a pnpmfile hook turns it off there: CI packages from,
caches and verifies a project-local node_modules.

* fix(build): keep shared electron and windows-process-tree dirs untouched under GVS

Every worktree runs Electron from the one shared electron package, so the
per-repo dist cache must not swap that dist and a foreign-arch install must
not land in it. The relay addon builder now repairs and compiles a private
copy instead of the shared windows-process-tree dir.

* fix(build): accept CLI runtime imports reached through an in-artifact store link

Under the global virtual store node_modules/<pkg> realpaths into a store
outside the repo. The check still rejects anything not reached through a
node_modules link inside the artifact, such as a stray ~/node_modules.

* docs: explain the global virtual store layout and the ~/node_modules hazard

* test: compare CLI closure paths with POSIX separators so the suite passes on Windows

* build: keep Windows on the normal pnpm layout unless ORCA_PNPM_GLOBAL_VIRTUAL_STORE=1

Store links are true symlinks on Windows, and installs without admin or
Developer Mode are unverified; CI still forces the normal layout first.

* fix(build): close the remaining shared-slot gaps under the global virtual store

- verify-skills-cli-runtime resolves each import from the importer's realpath,
  as Node does, so an artifact-local copy can't mask a ~/node_modules load.
- A concurrent sibling install of the shared Electron dist is kept, not swapped out.
- rebuild:node goes through ensure-native-runtime, which builds a shared
  node-pty beside the slot instead of node-gyp cleaning its build/ dir.

* fix(build): keep pnpm rebuild:node a forced rebuild

It replaced pnpm rebuild node-pty, which always rebuilt; without --force it
only rebuilt a node-pty that failed to load.

* docs: note that the store switches must stay set, since pnpm run relinks

* build: move the pnpmfile under config/ to keep the repo root allowlisted

pnpm-workspace.yaml points pnpm at it; the lockfile checksum is unchanged
because the content is.

* fix(build): enable the global virtual store from the pnpmfile, not the workspace file

pnpm exec derives NODE_PATH from the file setting, so even with the store
forced off under CI every child process got the root node_modules on its
NODE_PATH; a Windows rebuild fixture then resolved the real
@orca/windows-registry it deliberately omits. The hook now sets the store
both ways, and ORCA_PNPM_GLOBAL_VIRTUAL_STORE=0/1 is the one switch.

* test(pty): keep F3's concurrent spawns on the node-pty mock

Two concurrent first import('node-pty') calls race vitest's mock registry
and the loser loads the real addon under the mocked linux platform. It
passed only where a platform-agnostic build/Release happened to exist,
so it failed on every shared-store worktree.

* ci: fail PRs whose packages need a dependency they never declared

A new dependency_isolation job installs with the global virtual store
forced on (ORCA_PNPM_FORCE_GLOBAL_VIRTUAL_STORE_IN_CI), HOME and the store
in an empty temp dir, then runs pnpm tc and the undeclared-dependency test.
Postinstall now warns when a node_modules above the shared store, such as
~/node_modules, would mask the same class of bug locally.

* ci: set the isolation job's HOME and store through GITHUB_ENV

Job-level env cannot read the runner context, which made pr.yml invalid.
2026-10-18 20:36:24 -07:00

245 lines
8.8 KiB
JavaScript

import { appendFileSync, globSync, readFileSync } from 'node:fs'
import { resolve } from 'node:path'
import { pathToFileURL } from 'node:url'
import { nodeServerTestPaths } from './node-server-test-paths.mjs'
import { nodeServerQualification } from './node-server-qualification.mjs'
import { classifyNativeCompatChanges } from './node-server-native-compat-scope.mjs'
const ROOT = resolve(import.meta.dirname, '../..')
const BUILD_SCRIPTS = [
'config/scripts/build-orcad-node.mjs',
'config/scripts/server-build-target.mjs',
'config/scripts/pinned-node-downloads.mjs',
'config/scripts/build-orcad.mjs',
'config/scripts/build-orcad-prebuilds.mjs',
'config/scripts/orcad-windows-prebuild-cache.mjs',
'config/scripts/orcad-prebuild-smoke-child.cjs',
'config/scripts/build-windows-process-tree-addon.mjs',
'config/scripts/build-windows-cli-launcher.mjs',
'config/scripts/build-windows-security-addon.mjs',
'config/scripts/run-node-server-tests.mjs',
'config/vitest.config.ts',
'config/scripts/vitest-dom-setup.ts',
'config/scripts/happy-dom-offscreen-canvas.ts',
'config/scripts/happy-dom-mutation-observer-retention.ts',
'config/scripts/vitest-host-ports-setup.ts',
'config/scripts/vitest-real-agent-home-write-guard.ts'
]
const NODE_SERVER_POLICY_TESTS = new Set([
'config/scripts/node-server-change-scope.test.mjs',
'config/scripts/node-server-qualification.test.mjs'
])
const ALWAYS_FILES = new Set([
'package.json',
'pnpm-lock.yaml',
'pnpm-workspace.yaml',
'.npmrc',
'config/pnpmfile.cjs',
'tsconfig.json',
'.github/workflows/node-server-tests.yml',
'config/scripts/node-server-change-scope.mjs',
'config/scripts/node-server-native-compat-scope.mjs',
...NODE_SERVER_POLICY_TESTS,
'config/scripts/workspace-source-exports.mjs',
'config/scripts/workspace-source-exports.test.mjs',
'config/scripts/headless-detector-compiler-cache.mjs',
'config/scripts/initialize-windows-native-toolchain.mjs',
'config/scripts/capture-windows-native-environment.cmd',
'config/scripts/node-server-qualification.mjs',
// The source graph sees process-host source, not the build that emits the dist servers load.
'src/packages/process-host/.gitignore',
'src/packages/process-host/scripts/build-dist.mjs',
'src/packages/process-host/tsconfig.json'
])
const ALWAYS_PREFIXES = [
'.github/actions/install-node-dependencies/',
'.github/actions/restore-pnpm-verification/',
'.github/actions/prepare-headless-compiler/',
'.github/actions/prepare-native-runtime/',
'.github/actions/prepare-orcad-prebuilds/',
// These areas also contain worker paths and fixtures opened without an import.
'src/main/persistence/',
'src/main/sqlite/',
'src/main/orcad/',
'src/main/runtime/',
'src/main/claude/',
'src/main/daemon/pty-subprocess/',
'src/main/providers/',
'config/patches/',
'config/tsconfig',
'native/',
'resources/licenses/ripgrep/'
]
export function discoverNodeServerTests(root = ROOT) {
const selectors = nodeServerTestPaths({ artifact: true, crossRuntime: true, template: true })
return globSync(
['src/**/*.test.{ts,tsx}', 'config/scripts/**/*.test.{ts,mjs}', 'tests/e2e/**/*.unit.test.ts'],
{ cwd: root }
)
.map((file) => file.replaceAll('\\', '/'))
.filter((file) => selectors.some((selector) => file.includes(selector)))
.sort()
}
export async function collectNodeServerInputs({ root = ROOT, entryPoints } = {}) {
const [
{ build },
{
externalNativeAddons,
ORCAD_CHILD_ENTRY_POINTS,
ORCAD_ENTRY_POINT,
ORCAD_LAUNCHER_ENTRY_POINT
},
{ createWorkspaceSourceResolver }
] = await Promise.all([
import('esbuild'),
import('./orcad-entry-build.mjs'),
import('./workspace-source-exports.mjs')
])
const workspace = createWorkspaceSourceResolver(root)
const manifests = new Set()
const workspaceSources = {
name: 'workspace-source-exports',
setup(pluginBuild) {
pluginBuild.onResolve({ filter: /^[^./]/ }, ({ path }) => {
const source = workspace.resolve(path)
if (!source) {
return null
}
manifests.add(source.manifest)
return { path: source.path }
})
}
}
const entries = entryPoints ?? [
ORCAD_ENTRY_POINT,
ORCAD_LAUNCHER_ENTRY_POINT,
...Object.values(ORCAD_CHILD_ENTRY_POINTS),
...BUILD_SCRIPTS,
...discoverNodeServerTests(root)
]
const result = await build({
absWorkingDir: root,
entryPoints: entries,
bundle: true,
write: false,
outdir: resolve(root, '.node-server-scope'),
platform: 'node',
format: 'esm',
splitting: true,
packages: 'external',
loader: { '.svg': 'empty', '.png': 'empty', '.webp': 'empty', '.css': 'empty' },
plugins: [workspaceSources, externalNativeAddons],
metafile: true,
logLevel: 'silent'
})
if (result.warnings.length > 0) {
throw new Error(result.warnings.map((warning) => warning.text).join('\n'))
}
return new Set([
...manifests,
...Object.keys(result.metafile.inputs).map((file) =>
file.replaceAll('\\', '/').replace(/\?.*$/, '')
)
])
}
function isSourceUnitTest(file) {
return (
/^src\/(?:[^/]+\/)*[^/]+\.test\.(?:ts|tsx)$/.test(file) &&
!file.includes('/../') &&
!file.includes('/./')
)
}
export async function classifyNodeServerChanges(
changedFiles,
collect = collectNodeServerInputs,
{ deferGraph = false, modifiedFiles } = {}
) {
if (changedFiles.length === 0) {
return { shouldRun: true, reason: 'No complete changed-file evidence' }
}
const modifiedOnly =
Array.isArray(modifiedFiles) &&
new Set(changedFiles).size === changedFiles.length &&
new Set(modifiedFiles).size === modifiedFiles.length &&
modifiedFiles.length === changedFiles.length &&
changedFiles.every((file) => modifiedFiles.includes(file))
const selectors = nodeServerTestPaths({ artifact: true, crossRuntime: true, template: true })
const forced = changedFiles.find(
(file) =>
(ALWAYS_FILES.has(file) && !(modifiedOnly && NODE_SERVER_POLICY_TESTS.has(file))) ||
(ALWAYS_PREFIXES.some((prefix) => file.startsWith(prefix)) && !isSourceUnitTest(file)) ||
/^config\/scripts\/[^/]*orcad[^/]*$/.test(file) ||
selectors.some((selector) => file.includes(selector))
)
if (forced) {
return { shouldRun: true, reason: `Build or CI input changed: ${forced}` }
}
if (deferGraph) {
return { graphRequired: true, reason: 'Installed dependencies are needed to check imports' }
}
try {
const inputs = await collect()
const matchedFiles = changedFiles.filter((file) => inputs.has(file))
const matched = matchedFiles[0]
return {
...(modifiedOnly ? { graphResolved: true, matchedFiles } : {}),
shouldRun: Boolean(matched),
reason: matched
? `Runtime or test dependency changed: ${matched}`
: 'No headless-server inputs changed'
}
} catch (error) {
return {
shouldRun: true,
graphUnavailable: true,
reason: `Dependency graph unavailable: ${String(error)}`
}
}
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
const changedFiles = readFileSync(process.argv[2], 'utf8').split('\0').filter(Boolean)
const modifiedArgument = process.argv.indexOf('--modified-paths')
const modifiedFiles =
modifiedArgument === -1
? undefined
: readFileSync(process.argv[modifiedArgument + 1], 'utf8')
.split('\0')
.filter(Boolean)
const result = await classifyNodeServerChanges(changedFiles, collectNodeServerInputs, {
deferGraph: process.argv.includes('--defer-graph'),
modifiedFiles
})
console.log(result.reason)
const policy = nodeServerQualification(changedFiles, result, {
fullQualification: process.argv.includes('--full-qualification')
})
const diffArgument = process.argv.indexOf('--diff-paths')
const native = result.graphUnavailable
? { nativeCompatibility: true, reason: 'Server dependency graph unavailable' }
: await classifyNativeCompatChanges(changedFiles, collectNodeServerInputs, {
pullRequest: process.argv.includes('--pull-request'),
fullQualification: process.argv.includes('--full-qualification'),
rawDiff:
diffArgument === -1 ? undefined : readFileSync(process.argv[diffArgument + 1], 'utf8'),
deferGraph: process.argv.includes('--defer-graph')
})
console.log(native.reason)
let output = result.graphRequired || native.graphRequired ? 'graph_required=true\n' : ''
if (!result.graphRequired) {
output += `should_run=${result.shouldRun}\nqualification=${policy.qualification}\nrunners=${JSON.stringify(policy.runners)}\n`
}
if (!native.graphRequired) {
output += `native_compatibility=${native.nativeCompatibility}\n`
}
if (process.env.GITHUB_OUTPUT) {
appendFileSync(process.env.GITHUB_OUTPUT, output)
} else {
process.stdout.write(output)
}
}