mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 16:02:32 +00:00
* feat(windows): let a relay host bind the native process table directly The CIM fallback from #16550 answers on relay hosts, but it costs a powershell.exe and ~1.4s per scan where the native reader costs ~57ms. It is a parachute, not the destination. Teach the loader a second source: the desktop app keeps resolving the npm package, and a relay host -- which has none of our node_modules -- binds a bare `windows-process-tree.node` staged beside the bundle. The CIM scan stays as the last resort, so a host with neither is unchanged. Bind the addon directly rather than its package wrapper. lib/index.js adds only a queue over getProcessList, and that queue is the wedge this module already defends against: it latches a module-global requestInProgress with no try/catch. We hold our own single-flight and deadline, so going straight to the addon drops the duplicate. Measured on a Windows 11 SSH host with ~1490 processes, running the relay-externals bundle from the deployed relay directory: no addon staged nativeAvailable=false 1247ms (CIM) addon staged nativeAvailable=true 57ms memory restored Degradation was exercised on that host, not just in fakes: a truncated upload, a text file, and a foreign-arch ELF each fall through to the scan rather than throwing, and restoring a good addon recovers. A file that loads but lacks getProcessList is rejected by shape, because binding to it would reject every read forever where falling through still answers. No artifact is staged yet, so this is inert until the packaging change lands: today every relay takes the same CIM path it does now. * build(relay): ship the Windows process-table addon to relay hosts The CIM scan restored correctness on Windows SSH hosts, but it costs a powershell.exe and ~1.4s per read where the native addon costs ~57ms. It was always the floor, not the destination. The addon cannot be npm-installed on a relay host: it carries a binding.gyp, so npm rebuilds from source and the build wants Spectre-mitigated libraries even where MSVC is already present. The binary inside the published tarball loads, but predates our patch and still caps enumeration at 1024 processes -- on a 1486-process host it returned exactly 1024 rows with the querying process among the missing, which reads as unavailable only under load. No published alternative clears the bar either; the one fork with a working prebuild story still carries the same cap. So build it where a compiler exists and ship the result. The build script refuses unpatched source -- checking the source rather than trusting the install, because the Spectre hunk fails loudly while the 1024 hunk fails silently -- and verifies the PE machine field so a cross-build cannot emit host arch for another target. The artifact is optional: hashed when present so a relay carrying it never shares an immutable directory with one that does not, and never probed, since requiring a file only a Windows build machine can produce would make a correct relay read as MISSING and redeploy forever. Builds on any other OS keep using the scan, unchanged. arm64 cross-compiles from the x64 runner but needs the optional MSVC ARM64 toolset, so it stays best-effort: a runner image without that component should cost arm64 relays the fast path, not fail the release the x64 relay is riding on. ORCA_REQUIRE_RELAY_NATIVE_ADDONS is a per-arch list rather than a flag for exactly that reason. * build(relay): require the arm64 process-table addon too The arm64 cross-compile is no longer unproven. On a Windows x64 machine with the MSVC v143 ARM64 build tools component installed, node-gyp --arch=arm64 produces a genuine ARM64 image: x64 machine=0x8664 152064 bytes arm64 machine=0xaa64 139776 bytes So arm64 stops being best-effort and joins x64 in the required list. It was only best-effort because the component is optional and I had not seen it succeed; a runner image without it now fails the build with MSB8020 naming the missing component, and that step runs before the long packaging step so the failure costs seconds rather than twenty minutes. The env var stays a per-arch list rather than reverting to a flag, so a future arch can land best-effort before being promoted the same way.
290 lines
9.9 KiB
JavaScript
290 lines
9.9 KiB
JavaScript
#!/usr/bin/env node
|
|
/**
|
|
* Bundle the relay daemon and its crash-isolated watcher child per platform.
|
|
*
|
|
* The relay runs on remote hosts via `node relay.js`, so both outputs use
|
|
* self-contained CommonJS bundles with no external dependencies beyond
|
|
* Node.js built-ins. Native addons (node-pty, @parcel/watcher) are
|
|
* marked external and expected to be installed on the remote or
|
|
* gracefully degraded.
|
|
*/
|
|
import { build } from 'esbuild'
|
|
import { createHash } from 'node:crypto'
|
|
import {
|
|
copyFileSync,
|
|
existsSync,
|
|
mkdirSync,
|
|
readdirSync,
|
|
readFileSync,
|
|
rmSync,
|
|
writeFileSync
|
|
} from 'node:fs'
|
|
import { join } from 'node:path'
|
|
import {
|
|
RELAY_BUILD_PLATFORMS,
|
|
RELAY_VERSION_FILENAME,
|
|
RELAY_WINDOWS_PROCESS_TREE_FILENAME,
|
|
relayOptionalArtifactFilenames,
|
|
isWindowsRelayPlatform,
|
|
relayArtifactFilenames
|
|
} from '../../src/shared/relay-artifacts.ts'
|
|
|
|
const __dirname = import.meta.dirname
|
|
// Why: the script lives under config/scripts, so go two levels up to reach the repo root.
|
|
const ROOT = join(__dirname, '..', '..')
|
|
const RELAY_ENTRY = join(ROOT, 'src', 'relay', 'relay.ts')
|
|
const WATCHER_ENTRY = join(ROOT, 'src', 'main', 'ipc', 'parcel-watcher-process-entry.ts')
|
|
const AI_VAULT_SERVICE_ENTRY = join(ROOT, 'src', 'relay', 'ai-vault-service-entry.ts')
|
|
const WSL_TRANSCRIPT_FS_PROCESS_ENTRY = join(
|
|
ROOT,
|
|
'src',
|
|
'main',
|
|
'native-chat',
|
|
'wsl-transcript-fs-process-entry.ts'
|
|
)
|
|
const MANAGED_HOOK_RUNTIME_ENTRY = join(
|
|
ROOT,
|
|
'src',
|
|
'main',
|
|
'agent-hooks',
|
|
'managed-hook-runtime.ts'
|
|
)
|
|
const JSONC_PARSER_ESM_ENTRY = join(ROOT, 'node_modules', 'jsonc-parser', 'lib', 'esm', 'main.js')
|
|
const NODE_PTY_CONSOLE_LIST_PATCH_FILENAME = 'node-pty-1.1.0-console-list-agent-patch.cjs'
|
|
const NODE_PTY_CONSOLE_LIST_PATCH_SOURCE = join(
|
|
ROOT,
|
|
'config',
|
|
'relay-assets',
|
|
NODE_PTY_CONSOLE_LIST_PATCH_FILENAME
|
|
)
|
|
// Written by build-windows-process-tree-relay-addon.mjs, which only runs on a
|
|
// Windows machine.
|
|
const WINDOWS_PROCESS_TREE_BUILD_DIR = join(ROOT, '.build', 'windows-process-tree')
|
|
|
|
// Which Windows arches must have the addon, as a comma-separated list ('all' for
|
|
// every arch). Per-arch rather than a flag because arm64 needs the MSVC ARM64
|
|
// cross toolset, an optional VS component: where it is absent that relay should
|
|
// fall back to the scan, not fail the release the x64 relay is riding on.
|
|
const REQUIRED_ADDON_ARCHES = (process.env.ORCA_REQUIRE_RELAY_NATIVE_ADDONS ?? '')
|
|
.split(',')
|
|
.map((value) => value.trim())
|
|
.filter(Boolean)
|
|
|
|
function stageWindowsProcessTreeAddon(platform, outDir) {
|
|
if (!isWindowsRelayPlatform(platform)) {
|
|
return
|
|
}
|
|
const arch = platform.slice('win32-'.length)
|
|
const source = join(WINDOWS_PROCESS_TREE_BUILD_DIR, arch, RELAY_WINDOWS_PROCESS_TREE_FILENAME)
|
|
if (!existsSync(source)) {
|
|
if (REQUIRED_ADDON_ARCHES.includes(arch) || REQUIRED_ADDON_ARCHES.includes('all')) {
|
|
throw new Error(
|
|
`Relay ${platform} needs ${source}. Run: node config/scripts/build-windows-process-tree-relay-addon.mjs --arch=${arch} (Windows only).`
|
|
)
|
|
}
|
|
console.log(
|
|
`Relay ${platform}: no ${RELAY_WINDOWS_PROCESS_TREE_FILENAME}; relay will use the PowerShell scan.`
|
|
)
|
|
return
|
|
}
|
|
copyFileSync(source, join(outDir, RELAY_WINDOWS_PROCESS_TREE_FILENAME))
|
|
}
|
|
|
|
// Why: lets the packaging contract test build into a temp tree instead of
|
|
// clobbering a developer's out/relay or racing tests that read it.
|
|
const OUT_ROOT = process.env.ORCA_RELAY_OUT_ROOT ?? join(ROOT, 'out', 'relay')
|
|
|
|
const RELAY_VERSION = '0.1.0'
|
|
|
|
for (const platform of RELAY_BUILD_PLATFORMS) {
|
|
const outDir = join(OUT_ROOT, platform)
|
|
// Why: a stale companion left by an earlier build would otherwise satisfy the
|
|
// manifest check and be hashed into .version, shipping mixed-generation bytes.
|
|
rmSync(outDir, { recursive: true, force: true })
|
|
mkdirSync(outDir, { recursive: true })
|
|
|
|
await build({
|
|
entryPoints: [RELAY_ENTRY],
|
|
bundle: true,
|
|
platform: 'node',
|
|
target: 'node18',
|
|
format: 'cjs',
|
|
outfile: join(outDir, 'relay.js'),
|
|
// Native addons cannot be bundled — they must exist on the remote host.
|
|
// The relay gracefully degrades when they are absent.
|
|
external: ['node-pty', '@parcel/watcher', 'electron'],
|
|
sourcemap: false,
|
|
minify: true,
|
|
define: {
|
|
'process.env.NODE_ENV': '"production"'
|
|
}
|
|
})
|
|
|
|
if (isWindowsRelayPlatform(platform)) {
|
|
copyFileSync(
|
|
NODE_PTY_CONSOLE_LIST_PATCH_SOURCE,
|
|
join(outDir, NODE_PTY_CONSOLE_LIST_PATCH_FILENAME)
|
|
)
|
|
}
|
|
stageWindowsProcessTreeAddon(platform, outDir)
|
|
|
|
await build({
|
|
entryPoints: [WATCHER_ENTRY],
|
|
bundle: true,
|
|
platform: 'node',
|
|
target: 'node18',
|
|
format: 'cjs',
|
|
outfile: join(outDir, 'relay-watcher.js'),
|
|
external: ['@parcel/watcher'],
|
|
sourcemap: false,
|
|
minify: true,
|
|
define: {
|
|
'process.env.NODE_ENV': '"production"'
|
|
}
|
|
})
|
|
|
|
await build({
|
|
entryPoints: [AI_VAULT_SERVICE_ENTRY],
|
|
bundle: true,
|
|
platform: 'node',
|
|
target: 'node18',
|
|
format: 'cjs',
|
|
outfile: join(outDir, 'relay-ai-vault-service.js'),
|
|
external: ['electron'],
|
|
sourcemap: false,
|
|
minify: true,
|
|
define: {
|
|
'process.env.NODE_ENV': '"production"'
|
|
}
|
|
})
|
|
|
|
// Why beside the service: the spawn resolves this child next to its own
|
|
// bundle, and a relay host has no desktop out/main to fall back to.
|
|
await build({
|
|
entryPoints: [WSL_TRANSCRIPT_FS_PROCESS_ENTRY],
|
|
bundle: true,
|
|
platform: 'node',
|
|
target: 'node18',
|
|
format: 'cjs',
|
|
outfile: join(outDir, 'wsl-transcript-fs-process-entry.js'),
|
|
external: ['electron'],
|
|
sourcemap: false,
|
|
minify: true,
|
|
define: {
|
|
'process.env.NODE_ENV': '"production"'
|
|
}
|
|
})
|
|
|
|
await build({
|
|
entryPoints: [MANAGED_HOOK_RUNTIME_ENTRY],
|
|
bundle: true,
|
|
platform: 'node',
|
|
target: 'node18',
|
|
format: 'cjs',
|
|
outfile: join(outDir, 'managed-hook-runtime.js'),
|
|
// Why: jsonc-parser's default UMD build keeps relative dynamic requires
|
|
// that break after bundling; its ESM entry is equivalent and self-contained.
|
|
alias: { 'jsonc-parser': JSONC_PARSER_ESM_ENTRY },
|
|
sourcemap: false,
|
|
minify: true,
|
|
define: {
|
|
'process.env.NODE_ENV': '"production"'
|
|
}
|
|
})
|
|
|
|
// Why: include a content hash so the deploy check detects code changes even
|
|
// when RELAY_VERSION hasn't been bumped. Hashing the whole manifest means a
|
|
// companion-only change still selects a fresh immutable relay directory.
|
|
const expected = relayArtifactFilenames(isWindowsRelayPlatform(platform))
|
|
const hash = createHash('sha256')
|
|
for (const filename of expected) {
|
|
const artifactPath = join(outDir, filename)
|
|
if (!existsSync(artifactPath)) {
|
|
throw new Error(
|
|
`Relay ${platform} declares ${filename} in RELAY_ARTIFACTS but never emitted it. ` +
|
|
'Add the build step, or drop it from src/shared/relay-artifacts.ts.'
|
|
)
|
|
}
|
|
hash.update(readFileSync(artifactPath))
|
|
}
|
|
// Why hashed only when present: a relay carrying the native addon answers
|
|
// differently from one that falls back to the scan, so the two must not share
|
|
// an immutable directory -- but a build without it is still valid.
|
|
for (const filename of relayOptionalArtifactFilenames(isWindowsRelayPlatform(platform))) {
|
|
const artifactPath = join(outDir, filename)
|
|
if (existsSync(artifactPath)) {
|
|
hash.update(readFileSync(artifactPath))
|
|
}
|
|
}
|
|
const contentHash = hash.digest('hex').slice(0, 12)
|
|
|
|
// Close the loop: an artifact emitted here but absent from the manifest would
|
|
// ship unhashed and unprobed — exactly how the WSL helper went missing.
|
|
const emitted = readdirSync(outDir).filter((name) => name !== RELAY_VERSION_FILENAME)
|
|
const declared = [
|
|
...expected,
|
|
...relayOptionalArtifactFilenames(isWindowsRelayPlatform(platform))
|
|
]
|
|
const undeclared = emitted.filter((name) => !declared.includes(name))
|
|
if (undeclared.length > 0) {
|
|
throw new Error(
|
|
`Relay ${platform} emitted undeclared artifacts: ${undeclared.join(', ')}. ` +
|
|
'Add them to RELAY_ARTIFACTS in src/shared/relay-artifacts.ts.'
|
|
)
|
|
}
|
|
writeFileSync(join(outDir, RELAY_VERSION_FILENAME), `${RELAY_VERSION}+${contentHash}`)
|
|
|
|
console.log(`Built relay for ${platform} → ${outDir}/relay.js`)
|
|
}
|
|
|
|
// WSL agent-hook relay: a hooks-only guest receiver launched inside WSL
|
|
// distros via wsl.exe. Pure Node built-ins (no node-pty/@parcel/watcher),
|
|
// so a single platform-independent bundle suffices; it ships inside the
|
|
// Windows app via the same out/relay extraResources mapping.
|
|
{
|
|
const wslHookEntry = join(ROOT, 'src', 'relay', 'wsl-agent-hook-relay.ts')
|
|
const wslBrowserNetworkEntry = join(ROOT, 'src', 'relay', 'wsl-browser-network-relay.ts')
|
|
const outDir = join(OUT_ROOT, 'wsl')
|
|
mkdirSync(outDir, { recursive: true })
|
|
await build({
|
|
entryPoints: [wslHookEntry],
|
|
bundle: true,
|
|
platform: 'node',
|
|
target: 'node18',
|
|
format: 'cjs',
|
|
outfile: join(outDir, 'wsl-agent-hook-relay.js'),
|
|
sourcemap: false,
|
|
minify: true,
|
|
define: {
|
|
'process.env.NODE_ENV': '"production"'
|
|
}
|
|
})
|
|
const content = readFileSync(join(outDir, 'wsl-agent-hook-relay.js'))
|
|
const hash = createHash('sha256').update(content).digest('hex').slice(0, 12)
|
|
writeFileSync(join(outDir, '.version'), `${RELAY_VERSION}+${hash}`)
|
|
console.log(`Built WSL hook relay → ${outDir}/wsl-agent-hook-relay.js`)
|
|
|
|
await build({
|
|
entryPoints: [wslBrowserNetworkEntry],
|
|
bundle: true,
|
|
platform: 'node',
|
|
target: 'node18',
|
|
format: 'cjs',
|
|
outfile: join(outDir, 'wsl-browser-network-relay.js'),
|
|
sourcemap: false,
|
|
minify: true,
|
|
define: {
|
|
'process.env.NODE_ENV': '"production"'
|
|
}
|
|
})
|
|
const browserNetworkContent = readFileSync(join(outDir, 'wsl-browser-network-relay.js'))
|
|
const browserNetworkHash = createHash('sha256')
|
|
.update(browserNetworkContent)
|
|
.digest('hex')
|
|
.slice(0, 12)
|
|
writeFileSync(join(outDir, '.browser-network-version'), `${RELAY_VERSION}+${browserNetworkHash}`)
|
|
console.log(`Built WSL browser network relay → ${outDir}/wsl-browser-network-relay.js`)
|
|
}
|
|
|
|
console.log('Relay build complete.')
|