Reduce redundant headless server CI work (#24527)

* ci: avoid unrelated headless server qualification

* ci: skip headless detection for ineligible draft PRs

* ci: preserve cross-host qualification and skip supplied prerequisites

* ci: include Windows server cache validation in change detection
This commit is contained in:
Neil
2026-10-02 01:42:41 -07:00
committed by GitHub
parent 53930a161b
commit 6153fbcfe4
9 changed files with 317 additions and 45 deletions
+45 -10
View File
@@ -18,8 +18,7 @@ on:
- '.github/actions/install-node-dependencies/**'
- '.github/actions/prepare-native-runtime/**'
- '.github/workflows/node-server-tests.yml'
# The pull request qualifies one platform for an unflavoured change; this is where all six
# are re-qualified, so a platform break surfaces minutes after merge instead of next cron.
# Relevant main pushes qualify every platform after the dependency check.
push:
branches: [main]
paths:
@@ -60,15 +59,16 @@ on:
permissions:
contents: read
# Why a run-scoped group for template builds: a release call shares github.ref with main's push
# runs, and cancelling either would drop a release's template or a main qualification.
# Main pushes must finish detection before they can supersede relevant qualification.
concurrency:
group: node-server-${{ inputs.build_template && format('template-{0}', github.run_id) || github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ !inputs.build_template }}
group: node-server-${{ (inputs.build_template || github.event_name == 'push') && format('run-{0}', github.run_id) || github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ !inputs.build_template && github.event_name != 'push' }}
jobs:
changes:
if: github.event_name == 'pull_request'
if: >-
github.event_name == 'push' ||
(github.event_name == 'pull_request' && github.event.pull_request.draft != true)
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
@@ -84,7 +84,20 @@ jobs:
- name: Detect headless-server build and test inputs
id: scope
shell: bash
env:
PUSH_BASE: ${{ github.event.before }}
EVENT_NAME: ${{ github.event_name }}
run: |
if [ "$EVENT_NAME" = push ]; then
# Compare the entire push, including multi-commit pushes and removed files.
if git fetch --no-tags --depth=1 origin "$PUSH_BASE" &&
git diff --name-only --no-renames -z "$PUSH_BASE" HEAD > "$RUNNER_TEMP/node-server-changes"; then
node config/scripts/node-server-change-scope.mjs "$RUNNER_TEMP/node-server-changes" --full-qualification
else
echo 'should_run=true' >> "$GITHUB_OUTPUT"
fi
exit 0
fi
# Compare the tested merge with its base, retaining both sides of renames.
if git diff --name-only --no-renames -z HEAD^1 HEAD > "$RUNNER_TEMP/node-server-changes"; then
node config/scripts/node-server-change-scope.mjs "$RUNNER_TEMP/node-server-changes"
@@ -94,6 +107,9 @@ jobs:
persistence:
needs: changes
concurrency:
group: node-server-persistence-${{ matrix.os }}-${{ github.event_name == 'push' && !inputs.build_template && inputs.ref == '' && github.ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'push' && !inputs.build_template && inputs.ref == '' }}
# Missing/failed detection runs the full matrix; manual runs remain unconditional.
# A draft carries no platform verdict; readiness re-triggers this workflow. Spelled against
# the event name so the push and schedule paths do not rest on a null property comparison.
@@ -126,10 +142,10 @@ jobs:
continue-on-error: true
shell: bash
run: node config/scripts/orcad-windows-prebuild-cache.mjs --fingerprint
- name: Restore the exact Windows server prebuild for this pull request
- name: Restore the exact Windows server prebuild
id: orcad-prebuild-cache-restore
if: >-
github.event_name == 'pull_request' &&
(github.event_name == 'pull_request' || github.event_name == 'push') &&
steps.orcad-prebuild-cache-identity.outcome == 'success' &&
steps.orcad-prebuild-cache-identity.outputs.key != ''
continue-on-error: true
@@ -234,6 +250,9 @@ jobs:
linux_glibc_floor:
needs: [changes, persistence]
concurrency:
group: node-server-linux_glibc_floor-${{ matrix.os }}-${{ github.event_name == 'push' && !inputs.build_template && inputs.ref == '' && github.ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'push' && !inputs.build_template && inputs.ref == '' }}
# A failed smoke already blocks qualification; missing scope still selects every platform.
if: >-
${{ !cancelled() && needs.persistence.result == 'success' &&
@@ -262,7 +281,17 @@ jobs:
PYTHON: /opt/python/cp312-cp312/bin/python3
steps:
- name: Install glibc 2.28 prerequisites
run: dnf install -y git procps-ng unzip which xz
run: |
missing_tool=false
for tool in git ps unzip which xz; do
if ! command -v "$tool" >/dev/null 2>&1; then
missing_tool=true
fi
done
if [ "$missing_tool" = true ]; then
# The image's source-built Git needs no RPM; missing tools come from AlmaLinux.
dnf --disablerepo='epel*' install -y git procps-ng unzip which xz
fi
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref }}
@@ -289,6 +318,9 @@ jobs:
linux_glibc217_compat:
needs: [changes, persistence]
concurrency:
group: node-server-linux_glibc217_compat-${{ github.event_name == 'push' && !inputs.build_template && inputs.ref == '' && github.ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'push' && !inputs.build_template && inputs.ref == '' }}
# A failed smoke already blocks qualification; missing scope still selects every platform.
if: >-
${{ !cancelled() && needs.persistence.result == 'success' &&
@@ -341,6 +373,9 @@ jobs:
linux_musl:
needs: [changes, persistence]
concurrency:
group: node-server-linux_musl-${{ matrix.os }}-${{ github.event_name == 'push' && !inputs.build_template && inputs.ref == '' && github.ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'push' && !inputs.build_template && inputs.ref == '' }}
# A failed smoke already blocks qualification; missing scope still selects every platform.
if: >-
${{ !cancelled() && needs.persistence.result == 'success' &&
+4 -1
View File
@@ -17,6 +17,7 @@ const BUILD_SCRIPTS = [
'config/scripts/pinned-node-downloads.mjs',
'config/scripts/build-orcad.mjs',
'config/scripts/build-orcad-prebuilds.mjs',
'config/scripts/orcad-windows-prebuild-cache.mjs',
'config/scripts/orcad-prebuild-smoke-child.cjs',
'config/scripts/build-windows-process-tree-relay-addon.mjs',
'config/scripts/run-node-server-tests.mjs',
@@ -133,7 +134,9 @@ if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href)
const changedFiles = readFileSync(process.argv[2], 'utf8').split('\0').filter(Boolean)
const result = await classifyNodeServerChanges(changedFiles)
console.log(result.reason)
const policy = nodeServerQualification(changedFiles, result)
const policy = nodeServerQualification(changedFiles, result, {
fullQualification: process.argv.includes('--full-qualification')
})
const output = `should_run=${result.shouldRun}\nqualification=${policy.qualification}\nrunners=${JSON.stringify(policy.runners)}\n`
if (process.env.GITHUB_OUTPUT) {
appendFileSync(process.env.GITHUB_OUTPUT, output)
@@ -113,6 +113,7 @@ describe('the actual Bun build and profile-test dependency graph', () => {
'src/main/worker-thread-entry-path.ts',
'config/scripts/zip-extractor-command.mjs',
'config/scripts/windows-process-tree-gyp-rebuild.mjs',
'config/scripts/orcad-windows-prebuild-cache.mjs',
'config/scripts/profile-state-worker-smoke.mjs',
'config/scripts/vitest-host-ports-setup.ts',
'tests/e2e/daemon-running-work-probe.unit.test.ts'
@@ -131,16 +132,22 @@ describe('the actual Bun build and profile-test dependency graph', () => {
})
})
it('keeps all ten platform jobs and runs them when detection is skipped or fails', () => {
it('keeps every platform job and runs them when detection is skipped or fails', () => {
const workflow = parse(
readFileSync(new URL('../../.github/workflows/node-server-tests.yml', import.meta.url), 'utf8')
)
expect(workflow.on).toHaveProperty('workflow_dispatch')
expect(workflow.jobs.changes.if).toBe("github.event_name == 'pull_request'")
expect(workflow.jobs.changes.if).toBe(
"github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.draft != true)"
)
expect(workflow.jobs.changes.steps[0].with['fetch-depth']).toBe(2)
expect(workflow.jobs.changes.steps[0].with['persist-credentials']).toBe(false)
const detect = workflow.jobs.changes.steps.find((step) => step.id === 'scope')
expect(detect.run).toContain('git diff --name-only --no-renames -z HEAD^1 HEAD')
expect(detect.env.PUSH_BASE).toBe('${{ github.event.before }}')
expect(detect.run).toContain('git fetch --no-tags --depth=1 origin "$PUSH_BASE"')
expect(detect.run).toContain('git diff --name-only --no-renames -z "$PUSH_BASE" HEAD')
expect(detect.run).toContain('node-server-changes" --full-qualification')
expect(workflow.on.pull_request.types).toContain('ready_for_review')
expect(workflow.on.schedule).toHaveLength(1)
// A pull request may qualify one platform, so the merged commit must re-qualify all six.
@@ -0,0 +1,85 @@
import { readFileSync } from 'node:fs'
import { runInNewContext } from 'node:vm'
import { expect, it } from 'vitest'
import { parse } from 'yaml'
const workflow = parse(readFileSync('.github/workflows/node-server-tests.yml', 'utf8'))
function context(event, runId, inputs = {}) {
return {
github: {
event_name: event,
run_id: runId,
ref: 'refs/heads/main',
event: { pull_request: { number: 123 } }
},
inputs: { build_template: false, ref: '', ...inputs },
matrix: { os: 'windows-2022' },
format: (template, value) => template.replace('{0}', value)
}
}
function expression(source, ctx) {
return runInNewContext(source.slice(3, -2).trim(), ctx)
}
function group(policy, ctx) {
return policy.group.replace(/\$\{\{([\s\S]*?)\}\}/g, (_match, source) =>
String(runInNewContext(source, ctx))
)
}
it('skips draft detection and rechecks the same draft once it is ready', () => {
const draft = context('pull_request', 1)
draft.github.event.pull_request.draft = true
expect(runInNewContext(workflow.jobs.changes.if, draft)).toBe(false)
draft.github.event.pull_request.draft = false
expect(runInNewContext(workflow.jobs.changes.if, draft)).toBe(true)
expect(workflow.on.pull_request.types).toContain('ready_for_review')
expect(runInNewContext(workflow.jobs.changes.if, context('push', 2))).toBe(true)
for (const event of ['schedule', 'workflow_dispatch', 'workflow_call']) {
expect(runInNewContext(workflow.jobs.changes.if, context(event, 2))).toBe(false)
}
})
it('lets main pushes finish detection without cancelling another push', () => {
const first = context('push', 1)
const second = context('push', 2)
expect(group(workflow.concurrency, first)).not.toBe(group(workflow.concurrency, second))
expect(expression(workflow.concurrency['cancel-in-progress'], first)).toBe(false)
})
it('still replaces superseded pull requests at workflow level', () => {
const first = context('pull_request', 1)
const second = context('pull_request', 2)
expect(group(workflow.concurrency, first)).toBe(group(workflow.concurrency, second))
expect(expression(workflow.concurrency['cancel-in-progress'], first)).toBe(true)
})
it.each(['persistence', 'linux_glibc_floor', 'linux_glibc217_compat', 'linux_musl'])(
'%s only supersedes eligible main qualification, isolating releases and nightly runs',
(name) => {
const job = workflow.jobs[name]
expect(job.if).toContain("needs.changes.outputs.should_run != 'false'")
const policy = job.concurrency
const first = context('push', 1)
const second = context('push', 2)
expect(group(policy, first)).toBe(group(policy, second))
expect(expression(policy['cancel-in-progress'], first)).toBe(true)
for (const [event, inputs] of [
['schedule', {}],
['workflow_dispatch', {}],
['push', { build_template: true }],
['push', { ref: 'refs/tags/v1' }]
]) {
const isolated = context(event, 2, inputs)
expect(group(policy, isolated)).not.toBe(group(policy, first))
expect(expression(policy['cancel-in-progress'], isolated)).toBe(false)
expect(group(policy, isolated)).not.toBe(group(policy, context(event, 3, inputs)))
}
if (job.strategy?.matrix) {
second.matrix.os = 'windows-11-arm'
expect(group(policy, second)).not.toBe(group(policy, first))
}
}
)
+49 -25
View File
@@ -7,42 +7,66 @@ export const NODE_SERVER_RUNNERS = [
'windows-11-arm'
]
// Only surfaces whose behaviour actually differs per platform. Escalating on `config/`,
// `resources/` and `.github/` wholesale took 36.5% of the last 1100 commits through all six
// platforms where a platform-flavoured predicate takes 19%.
const PLATFORM_PREFIXES = [
// Shared execution and storage changes need every host; explicit platform paths need their family.
const BUILD_PREFIXES = [
'native/',
'config/patches/',
'.github/actions/install-node-dependencies/',
'.github/actions/prepare-native-runtime/',
'.github/actions/prepare-native-runtime/'
]
// A remote target's OS does not identify the client platform that builds its commands.
const CROSS_HOST_PREFIXES = ['src/main/ssh/', 'src/main/providers/', 'src/relay/']
const PLATFORM_PREFIXES = [
'src/main/persistence/',
'src/main/sqlite/',
'src/main/orcad/',
'src/main/providers/',
'src/main/daemon/',
'src/main/ssh/',
'src/main/wsl/',
'src/relay/',
'src/shared/child-process/',
// Every native prebuild slot is compiled and smoked against the pinned runtime.
'src/shared/node-runtime-pin.ts'
'src/shared/child-process/'
]
export function nodeServerQualification(changedFiles, scope) {
const platformSpecific = changedFiles.some(
(file) =>
// A root manifest can move a native dependency on every platform at once.
const PLATFORM_FAMILIES = [
{ pattern: /(?:^|[/.-])(?:windows|win32|wsl)(?:[/.-]|$)/i, prefix: 'windows-' },
{ pattern: /(?:^|[/.-])(?:macos|darwin|posix)(?:[/.-]|$)/i, prefix: 'macos-' },
{ pattern: /(?:^|[/.-])(?:linux|posix)(?:[/.-]|$)/i, prefix: 'ubuntu-' }
]
export function nodeServerQualification(changedFiles, scope, { fullQualification = false } = {}) {
const selected = new Set(['ubuntu-22.04'])
let qualification = false
let full = fullQualification || changedFiles.length === 0 || scope.graphUnavailable === true
for (const file of changedFiles) {
// Build policy and native sources can change every slot, even with a platform in the name.
if (
!file.includes('/') ||
PLATFORM_PREFIXES.some((prefix) => file.startsWith(prefix)) ||
/(?:^|[/.-])(?:windows|win32|wsl|macos|darwin|linux|posix|bun|prebuilds?)(?:[/.-]|$)/i.test(
file
)
)
// A pull request qualifies one platform unless the change is platform-flavoured; the push to
// main re-qualifies all six, so an unescalated miss surfaces minutes after merge, not a day.
const full = changedFiles.length === 0 || scope.graphUnavailable === true || platformSpecific
BUILD_PREFIXES.some((prefix) => file.startsWith(prefix)) ||
CROSS_HOST_PREFIXES.some((prefix) => file.startsWith(prefix)) ||
(/(?:^|[/.-])(?:remote|ssh)(?:[/.-]|$)/i.test(file) &&
PLATFORM_FAMILIES.some(({ pattern }) => pattern.test(file))) ||
file === 'src/shared/node-runtime-pin.ts' ||
file === '.github/workflows/node-server-tests.yml' ||
file.startsWith('config/scripts/node-server-') ||
/(?:^|[/.-])(?:bun|prebuilds?)(?:[/.-]|$)/i.test(file)
) {
full = true
continue
}
const families = PLATFORM_FAMILIES.filter(({ pattern }) => pattern.test(file))
if (families.length > 0) {
for (const { prefix } of families) {
for (const runner of NODE_SERVER_RUNNERS.filter((runner) => runner.startsWith(prefix))) {
selected.add(runner)
}
qualification ||= prefix === 'ubuntu-'
}
} else if (PLATFORM_PREFIXES.some((prefix) => file.startsWith(prefix))) {
full = true
}
}
return {
qualification: full,
runners: full ? NODE_SERVER_RUNNERS : ['ubuntu-22.04']
qualification: full || qualification,
runners: full
? NODE_SERVER_RUNNERS
: NODE_SERVER_RUNNERS.filter((runner) => selected.has(runner))
}
}
@@ -34,11 +34,8 @@ it.each([
'src/main/persistence/profile-state/store.ts',
'src/main/sqlite/database.ts',
'src/main/orcad/entry.ts',
'src/main/runtime/windows-terminal.ts',
'src/shared/linux-glibc.ts',
'src/main/daemon/entry.ts',
'src/relay/index.ts',
'src/main/wsl/runner.ts',
'config/scripts/build-orcad-prebuilds.mjs',
'config/scripts/orcad-prebuild-slot-contents.mjs',
'src/shared/node-runtime-pin.ts'
@@ -58,3 +55,85 @@ it('fails closed to every platform when the evidence is incomplete', () => {
}).qualification
).toBe(true)
})
it.each([
[
'src/main/runtime/windows-terminal.ts',
['ubuntu-22.04', 'windows-2022', 'windows-11-arm'],
false
],
[
'src/main/windows/windows-process-table.ts',
['ubuntu-22.04', 'windows-2022', 'windows-11-arm'],
false
],
['src/main/wsl/runner.ts', ['ubuntu-22.04', 'windows-2022', 'windows-11-arm'], false],
[
'src/main/orcad/orcad-launcher.win32.test.ts',
['ubuntu-22.04', 'windows-2022', 'windows-11-arm'],
false
],
['src/main/daemon/darwin-process.ts', ['ubuntu-22.04', 'macos-14', 'macos-15-intel'], false],
['src/shared/linux-glibc.ts', ['ubuntu-22.04', 'ubuntu-24.04-arm'], true],
[
'src/main/daemon/posix-process.ts',
['ubuntu-22.04', 'ubuntu-24.04-arm', 'macos-14', 'macos-15-intel'],
true
]
])('selects both architectures and a Linux smoke for %s', (file, runners, qualification) => {
expect(nodeServerQualification([file], scope)).toEqual({ runners, qualification })
})
it('combines platform families without adding Linux compatibility work', () => {
expect(
nodeServerQualification(
['src/main/windows/windows-process-table.ts', 'src/main/daemon/darwin-process.ts'],
scope
)
).toEqual({
runners: ['ubuntu-22.04', 'macos-14', 'macos-15-intel', 'windows-2022', 'windows-11-arm'],
qualification: false
})
})
it('keeps all hosts for shared changes alongside a platform-specific change', () => {
expect(
nodeServerQualification(
['src/main/windows/windows-process-table.ts', 'src/main/daemon/entry.ts'],
scope
)
).toEqual({ runners: NODE_SERVER_RUNNERS, qualification: true })
})
it.each([
'src/main/ssh/remote-node-runtime-store-windows.ts',
'src/main/ssh/orcad-remote-node-runtime-windows.ts',
'src/main/ssh/ssh-posix-command-wrapper.test.ts',
'src/main/providers/agent-foreground-process-git-bash.win32.test.ts',
'src/relay/windows-port-scan.ts',
'src/main/runtime/windows-firewall-remote-scope.ts',
'src/shared/remote-windows-path.ts'
])('qualifies every client platform for a remote execution input: %s', (file) => {
expect(nodeServerQualification([file], scope)).toEqual({
runners: NODE_SERVER_RUNNERS,
qualification: true
})
})
it.each([
'.github/workflows/node-server-tests.yml',
'config/scripts/node-server-qualification.mjs'
])('qualifies all hosts when the selection policy changes: %s', (file) => {
expect(nodeServerQualification([file], scope)).toEqual({
runners: NODE_SERVER_RUNNERS,
qualification: true
})
})
it('fully qualifies relevant main pushes even for an unflavoured change', () => {
expect(
nodeServerQualification(['src/main/runtime/rpc/methods/example.ts'], scope, {
fullQualification: true
})
).toEqual({ runners: NODE_SERVER_RUNNERS, qualification: true })
})
@@ -25,7 +25,9 @@ describe('orcad template release wiring (design D2)', () => {
build_template: { type: 'boolean', default: false }
})
// A release call shares github.ref with main's push runs; neither may cancel the other.
expect(nodeServer.concurrency['cancel-in-progress']).toBe('${{ !inputs.build_template }}')
expect(nodeServer.concurrency['cancel-in-progress']).toBe(
"${{ !inputs.build_template && github.event_name != 'push' }}"
)
expect(nodeServer.concurrency.group).toContain('github.run_id')
for (const lane of LANES) {
const steps = nodeServer.jobs[lane].steps
@@ -40,7 +40,7 @@ function context(os, arch, event, ref, template = false) {
describe('Windows server prebuild cache workflow', () => {
it.each([
['pull_request', 'refs/pull/1/merge', false, true, false],
['push', 'refs/heads/main', false, false, true],
['push', 'refs/heads/main', false, true, true],
['schedule', 'refs/heads/main', false, false, true],
['workflow_dispatch', 'refs/heads/main', false, false, true],
['workflow_call', 'refs/heads/main', false, false, false],
+39 -2
View File
@@ -3,6 +3,43 @@
The [September 28 demand rollout](ci-demand-rollout.md) documents staged checks,
unit-selection evidence, headless runtime qualification, review cancellation and daily occupancy reports.
## Headless server follow-up
[PR #24527](https://github.com/stablyai/orca/pull/24527) adds dependency detection to
main pushes. Unrelated pushes skip qualification; relevant pushes still run all
six persistence targets and five Linux compatibility jobs. Explicit Windows or
Mac PR paths select both architectures plus a Linux smoke, while shared
execution/storage changes, SSH/provider/relay inputs, native inputs, manifests, and incomplete evidence
retain the full matrix. Main pushes use the same validated exact Windows slot
cache as PRs; nightly and release builds still compile freshly.
Main detection runs cannot cancel each other. Only eligible qualification jobs
share main concurrency groups, so an unrelated push cannot cancel needed tests.
Release templates, explicit refs, and nightly runs remain isolated.
Draft PRs have no server verdict, so their detector is also skipped. The existing
`ready_for_review` event performs detection and qualification once the PR is ready.
This removes the checkout and dependency installation for a result whose platform
jobs were already ineligible.
The glibc 2.28 prerequisite step checks all five tools before installing anything.
The pinned ARM image already supplies them, including Git 2.55.0 built under
`/usr/local/bin`; installing the Git RPM does not change the Git on PATH. A
missing-tool fallback still installs the original package list and disables EPEL
for that one command. In
the baseline x64 log, EPEL metadata took 4 minutes 50 seconds to download although
every installed package came from AlmaLinux BaseOS or AppStream. The package list,
compiler image, libc floor, native smoke, and persistence tests stay unchanged.
The [DNF command reference](https://dnf.readthedocs.io/en/stable/command_ref.html)
defines `--disablerepo` as a temporary command-level filter, so later commands
retain the image's repository configuration.
A [completed main run](https://github.com/stablyai/orca/actions/runs/36962172614)
used 42 aggregate runner-minutes across 11 test jobs. The
[latest daily demand report](https://github.com/stablyai/orca/actions/runs/36965354205)
estimates 34.9 headless runner-hours, including 23.4 in cancelled runs. These are
baseline observations; post-merge savings have not yet been measured.
## October 1 Windows and dependency cache follow-up
[PR #24355](https://github.com/stablyai/orca/pull/24355) merged at `197ea3a3`.
@@ -17,8 +54,8 @@ tooling is removed from ordinary PR CI.
The existing dependency-native cache and the server's N-API 8 slot serve different
consumers. Cache the small server slot separately, using the exact compiler image,
architecture, dependency/patch/runtime inputs and compilation/validation source.
Only PR qualification restores it. Main qualification still compiles freshly and
saves after persistence/lifecycle tests and the existing x64 Node 18 handoff.
PR and main-push qualification restore it. Nightly qualification still compiles
freshly; main saves after persistence/lifecycle tests and the existing x64 Node 18 handoff.
Templates and explicit-ref calls continue to compile freshly.
| Hosted runner | Fresh build median | Restore median | Difference |