starred/orca
1
0
Fork 0
mirror of https://github.com/stablyai/orca.git synced 2026-10-02 16:02:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
554f7f4ce523907cbbb565f730bf03c0dd066545
orca/.github/workflows/node-server-tests.yml
T
OrcaWinandm4air 554f7f4ce5 feat(packaging): ship the orcad server template in desktop builds (#24155) 
* build(orcad): merge per-runner prebuild slot trees into one matrix

Each node-server lane builds only its own node-pty slot. Release CI needs
their union before `build:orcad-prebuilds --require-slots` and the
template build can run; merge-orcad-prebuilds.mjs verifies every lane's
files against its own manifest, refuses duplicate slots and mismatched
node-pty/N-API/Node-header builds, then writes one merged manifest.

* build(orcad): keep agent-browser out of the desktop deployment template

The template rides inside every desktop build (design D2). Seven ~10 MB
agent-browser binaries would be ~76 MB, more than the rest of the template;
design D2's package contents never listed it, and a slot without one
already reports no headless browser. ORCAD_OMIT_AGENT_BROWSER=1 skips the
copy; standalone build:orcad still includes it.

* feat(packaging): ship the orcad deployment template in desktop builds

Design D2: the server JS and every target's addons ship inside the app,
as out/relay does; the ~120 MB Node runtimes stay excluded and are
downloaded on demand. electron-builder copies out/orcad-template to
Resources/orcad-template on every desktop OS, which is the first path
materializeOrcadArtifact tries (process.resourcesPath).

Platform signing rewrites native bytes the template manifest hashes:
- macOS: the tree is signIgnored (codesign rejects its ELF/PE payloads);
  afterPack signs the darwin targets' Mach-O files with the app identity,
  as notarization requires, then reseals only those manifest entries.
- Windows: SignPath signs after packaging, so release CI reseals from the
  inner-signing list (packaged-orcad-template.cjs --reseal-signed).
Every other file must still match the build's hashes; afterPack verifies.

ORCA_REQUIRE_ORCAD_TEMPLATE=1 makes a missing template fail beforePack and
afterPack; without it a build ships none and SSH relays keep the legacy
path. verify-packaged-orcad-template.test.mjs's "unused, excluded"
contract is reversed on purpose.

* ci(release): build the orcad template from qualified lanes and package it

node-server-tests.yml becomes callable with a ref and build_template.
With build_template, each lane that owns a release slot (macOS, Windows,
the glibc 2.28 and Alpine lanes, and the glibc 2.17 compat lane) uploads
its qualified out/orcad-prebuilds, the Windows lane also uploads both
process-table addons, and desktop_template merges them, gates the full
matrix plus the compat slot with --require-slots, runs
build:orcad-template and uploads the orcad-template artifact.

release-cut calls it at the release tag beside the other gates. The
build and build-mac jobs wait for it, download it into out/orcad-template
(the mac workflow from the parent run), and require it via
ORCA_REQUIRE_ORCAD_TEMPLATE. The Windows signing staging skips the
template's Linux/macOS payloads, and a reseal step records SignPath's
bytes before the installer rebuild. A template-scoped concurrency group
keeps a release call and main's push runs from cancelling each other.

* test(orcad): keep the packaged-lookup imports clear of the compat-slot import edits

* ci(orcad): let a rerun lane replace its template artifacts

upload-artifact v4 refuses a second upload under an existing name in the same
run, so rerunning a flaky node-server lane during a release would fail at the
upload instead of re-qualifying the slot.

* ci(node-server): build the template's Windows addons before the lane switches to Node 18

The addon build script imports TypeScript, which Node 18 cannot load, so every
build_template run (release-cut included) failed on windows-2022.

* fix(build): ship the orcad template's shared node_modules

electron-builder's extraResources filter always drops the root node_modules of
a source directory, so packaged apps lost orcad-template/node_modules and the
afterPack verify failed. Copy it through its own resource entry.

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
2026-10-01 04:01:26 -07:00

394 lines
17 KiB
YAML
Raw Blame History

name: Headless Node server
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
paths:
- 'src/**'
- 'config/**'
- 'native/**'
- 'tests/**'
- 'resources/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'tsconfig.json'
- '.npmrc'
- '.pnpmfile.cjs'
- '.github/actions/install-node-dependencies/**'
- '.github/workflows/node-server-tests.yml'
# The pull request qualifies one platform for an unflavoured change; this is where all six
# are re-qualified, so a platform break surfaces minutes after merge instead of next cron.
push:
branches: [main]
paths:
- 'src/**'
- 'config/**'
- 'native/**'
- 'tests/**'
- 'resources/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'tsconfig.json'
- '.npmrc'
- '.pnpmfile.cjs'
- '.github/actions/install-node-dependencies/**'
- '.github/workflows/node-server-tests.yml'
workflow_dispatch:
inputs:
build_template:
description: Also merge every lane's slot into the desktop orcad template artifact
type: boolean
default: false
# Release packaging calls this to build the orcad template it ships (design D2).
workflow_call:
inputs:
ref:
description: Git ref every lane checks out, e.g. the release tag
type: string
default: ''
build_template:
description: Upload each lane's release slot and merge them into the orcad-template artifact
type: boolean
default: false
schedule:
- cron: '30 11 * * *'
permissions:
contents: read
# Why a run-scoped group for template builds: a release call shares github.ref with main's push
# runs, and cancelling either would drop a release's template or a main qualification.
concurrency:
group: node-server-${{ inputs.build_template && format('template-{0}', github.run_id) || github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ !inputs.build_template }}
jobs:
changes:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
should_run: ${{ steps.scope.outputs.should_run }}
qualification: ${{ steps.scope.outputs.qualification }}
runners: ${{ steps.scope.outputs.runners }}
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 2
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
- name: Detect headless-server build and test inputs
id: scope
shell: bash
run: |
# Compare the tested merge with its base, retaining both sides of renames.
if git diff --name-only --no-renames -z HEAD^1 HEAD > "$RUNNER_TEMP/node-server-changes"; then
node config/scripts/node-server-change-scope.mjs "$RUNNER_TEMP/node-server-changes"
else
echo 'should_run=true' >> "$GITHUB_OUTPUT"
fi
persistence:
needs: changes
# Missing/failed detection runs the full matrix; manual runs remain unconditional.
# A draft carries no platform verdict; readiness re-triggers this workflow. Spelled against
# the event name so the push and schedule paths do not rest on a null property comparison.
if: >-
${{ !cancelled() && needs.changes.outputs.should_run != 'false' &&
(github.event_name != 'pull_request' || github.event.pull_request.draft != true) }}
strategy:
fail-fast: false
matrix:
os: ${{ fromJSON(needs.changes.outputs.runners || '["ubuntu-22.04","ubuntu-24.04-arm","macos-14","macos-15-intel","windows-2022","windows-11-arm"]') }}
runs-on: ${{ matrix.os }}
timeout-minutes: 20
env:
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref }}
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: ${{ runner.os == 'Windows' && 'node' || 'none' }}
# Before setup-node 18: the scripts import the TypeScript runtime pin.
# Linux release slots come from the Ubuntu 20.04 and Alpine lanes below, where their libc
# floors live; this runner's slot only packages orcad for its own tests.
- name: Build and smoke this runner's node-pty prebuild slot under the pinned Node
shell: bash
run: |
pnpm build:orcad-prebuilds
pnpm build:orcad-prebuilds --require-slots "$(node config/scripts/build-orcad-prebuilds.mjs --print-slot)"
pnpm build:orcad-prebuilds --smoke
- run: pnpm build:orcad
# Design D7 upgrade and rollback: the last Bun orcad, built from a main commit that shipped
# it at this PROTOCOL_VERSION, beside this checkout's Node slot. Same lockfile, so its build
# reuses this checkout's node_modules.
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
if: runner.os == 'Linux'
with:
bun-version: 1.4.2
- name: Build the last Bun orcad for the cross-runtime tests
if: runner.os == 'Linux'
shell: bash
env:
BUN_ORCAD_COMMIT: f4092c06d639ee13ad446261dcabc78b27a21fbc
run: |
git fetch --no-tags --depth=1 origin "$BUN_ORCAD_COMMIT"
git worktree add --detach "$RUNNER_TEMP/bun-orcad-source" "$BUN_ORCAD_COMMIT"
ln -s "$GITHUB_WORKSPACE/node_modules" "$RUNNER_TEMP/bun-orcad-source/node_modules"
node "$RUNNER_TEMP/bun-orcad-source/config/scripts/build-orcad-bun.mjs" --out-dir "$RUNNER_TEMP/bun-orcad"
echo "ORCA_BUN_ORCAD_SLOT=$RUNNER_TEMP/bun-orcad" >> "$GITHUB_ENV"
echo "BUN_EXECUTABLE=$(command -v bun)" >> "$GITHUB_ENV"
- run: pnpm test:node-server --artifact ${{ runner.os == 'Linux' && '--cross-runtime' || '' }}
# Only a Windows runner compiles it; arm64 cross-compiles here, as release-cut does for the relay.
# Before the Node 18 check below: the build script imports TypeScript, which Node 18 cannot load.
- name: Build the Windows process-table addons for the desktop template
if: inputs.build_template && matrix.os == 'windows-2022'
shell: bash
run: |
node config/scripts/build-windows-process-tree-relay-addon.mjs --arch=x64
node config/scripts/build-windows-process-tree-relay-addon.mjs --arch=arm64
- name: Keep the Windows process-table addons for the desktop template
if: inputs.build_template && matrix.os == 'windows-2022'
uses: actions/upload-artifact@v7
with:
name: orcad-windows-process-tree
path: .build/windows-process-tree/
include-hidden-files: true
if-no-files-found: error
retention-days: 7
overwrite: true
- uses: actions/setup-node@v6
if: runner.arch == 'X64'
with:
node-version: '18'
- name: Verify Node 18 loads and hands off to the pinned Node
if: runner.arch == 'X64'
shell: bash
run: |
node out/orcad/orcad.js --orcad-smoke-load-check
node out/orcad/orcad.js --orcad-profile-state-preflight 00000000-0000-4000-8000-000000000018 | tee "$RUNNER_TEMP/preflight.json"
node -e "const r=JSON.parse(require('fs').readFileSync(process.argv[1],'utf8'));if(r.runtime!=='node'||!/^24\./.test(r.runtimeVersion))process.exit(1)" "$RUNNER_TEMP/preflight.json"
# Linux release slots come from the floor and Alpine lanes; these runners own the rest.
- name: Keep this runner's qualified slot for the desktop template
if: inputs.build_template && runner.os != 'Linux'
uses: actions/upload-artifact@v7
with:
name: orcad-prebuild-${{ matrix.os }}
path: out/orcad-prebuilds/
if-no-files-found: error
retention-days: 7
# A rerun attempt re-uploads under the same name, which v4 otherwise refuses.
overwrite: true
linux_glibc_floor:
needs: [changes, persistence]
# A failed smoke already blocks qualification; missing scope still selects every platform.
if: >-
${{ !cancelled() && needs.persistence.result == 'success' &&
needs.changes.outputs.should_run != 'false' &&
needs.changes.outputs.qualification != 'false' }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, ubuntu-24.04-arm]
# Design D6: server glibc slots are built and gated on glibc 2.28, where the pinned Node's
# own floor is; the desktop keeps its Ubuntu 20.04 gate. gcc-toolset-14 supplies C++20 and
# links newer libstdc++ symbols statically, so the slot needs only RHEL 8's GLIBCXX_3.4.25.
include:
- os: ubuntu-22.04
image: quay.io/pypa/manylinux_2_28_x86_64@sha256:407f771c51a2c3e83ebe5a7970b4289ead3a6db21d9b9c089168775cad11d328
- os: ubuntu-24.04-arm
image: quay.io/pypa/manylinux_2_28_aarch64@sha256:c22ffd129ac99a8a42d1f2c2f4e88a9089288dd9ee987a7092da1c7dc48f27a9
runs-on: ${{ matrix.os }}
container: ${{ matrix.image }}
timeout-minutes: 25
env:
ORCA_BACKGROUND_LAUNCH: '1'
# The image puts /opt/clang first on PATH; the slot is built with its gcc-toolset.
CC: gcc
CXX: g++
PYTHON: /opt/python/cp312-cp312/bin/python3
steps:
- name: Install glibc 2.28 prerequisites
run: dnf install -y git procps-ng unzip which xz
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref }}
persist-credentials: false
- name: Trust the checked-out workspace
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
- uses: ./.github/actions/install-node-dependencies
- name: Build and smoke this runner's node-pty prebuild slot under the pinned Node
run: |
pnpm build:orcad-prebuilds
pnpm build:orcad-prebuilds --require-slots "$(node config/scripts/build-orcad-prebuilds.mjs --print-slot)"
pnpm build:orcad-prebuilds --smoke
- run: pnpm build:orcad
- run: pnpm test:node-server --artifact
- name: Keep this runner's glibc 2.28 slot for the desktop template
if: inputs.build_template
uses: actions/upload-artifact@v7
with:
name: orcad-prebuild-glibc-${{ matrix.os }}
path: out/orcad-prebuilds/
if-no-files-found: error
retention-days: 7
overwrite: true
linux_glibc217_compat:
needs: [changes, persistence]
# A failed smoke already blocks qualification; missing scope still selects every platform.
if: >-
${{ !cancelled() && needs.persistence.result == 'success' &&
needs.changes.outputs.should_run != 'false' &&
needs.changes.outputs.qualification != 'false' }}
runs-on: ubuntu-22.04
timeout-minutes: 20
env:
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref }}
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
# Design D6 rung B: the opt-in linux-x64-glibc217 slot beside the unofficial glibc-217 Node.
# Why docker run and not container: the runner's own Node for JS actions needs glibc 2.28,
# so the host installs and fetches the pinned Node, and the glibc 2.17 image only builds and
# smokes with that Node. devtoolset-10 supplies C++20; the slot links libstdc++ statically.
- name: Build and smoke the glibc 2.17 compat slot under the glibc-217 Node
run: |
compat_node="$(node config/scripts/build-orcad-prebuilds.mjs --slot=linux-x64-glibc217 --print-runtime)"
case "$compat_node" in
"$GITHUB_WORKSPACE"/*) ;;
*) echo "glibc-217 Node cached outside the workspace: $compat_node" >&2; exit 1 ;;
esac
docker run --rm --init -i \
-e ORCA_BACKGROUND_LAUNCH=1 \
-e COMPAT_NODE="/work/${compat_node#"$GITHUB_WORKSPACE"/}" \
-e CC=gcc -e CXX=g++ \
-e PYTHON=/opt/python/cp312-cp312/bin/python3 \
-v "$GITHUB_WORKSPACE:/work" -w /work \
quay.io/pypa/manylinux2014_x86_64@sha256:6f74cabeac2432570aa4bfdb29f7c1f30313d4d6654d764c44e574b6ffdd4ed5 bash -s <<'GLIBC217_COMPAT_SLOT'
set -eu
export PATH="$(dirname "$COMPAT_NODE"):$PATH"
node config/scripts/build-orcad-prebuilds.mjs --slot=linux-x64-glibc217
node config/scripts/build-orcad-prebuilds.mjs --require-slots linux-x64-glibc217
# The image's devtoolset LD_LIBRARY_PATH must not stand in for a host C++ runtime.
env -u LD_LIBRARY_PATH node config/scripts/build-orcad-prebuilds.mjs --slot=linux-x64-glibc217 --smoke
GLIBC217_COMPAT_SLOT
- name: Keep the glibc 2.17 compat slot for the desktop template
if: inputs.build_template
uses: actions/upload-artifact@v7
with:
name: orcad-prebuild-glibc217
path: out/orcad-prebuilds/
if-no-files-found: error
retention-days: 7
overwrite: true
linux_musl:
needs: [changes, persistence]
# A failed smoke already blocks qualification; missing scope still selects every platform.
if: >-
${{ !cancelled() && needs.persistence.result == 'success' &&
needs.changes.outputs.should_run != 'false' &&
needs.changes.outputs.qualification != 'false' }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, ubuntu-24.04-arm]
runs-on: ${{ matrix.os }}
timeout-minutes: 20
env:
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref }}
persist-credentials: false
- name: Verify native Alpine artifact and persistence
run: |
# Multi-arch index digest of the tag; re-resolve it whenever NODE_RUNTIME_PIN moves.
docker run --rm --init -i \
-e ORCA_BACKGROUND_LAUNCH=1 \
-v "$GITHUB_WORKSPACE:/work" -w /work \
node:24.21.0-alpine@sha256:ebfe2f90462722a7a4de65e91990e97fe0d401c70e0e762c5b53302f905ec1c1 sh -s <<'NODE_SERVER_QUALIFICATION'
set -eu
apk add --no-cache bash git libstdc++ python3 make g++
git config --global --add safe.directory /work
npm install -g "$(node -p "require('./package.json').packageManager.split('+')[0]")"
pnpm install --frozen-lockfile --ignore-scripts
pnpm build:orcad-prebuilds
pnpm build:orcad-prebuilds --require-slots "$(node config/scripts/build-orcad-prebuilds.mjs --print-slot)"
pnpm build:orcad-prebuilds --smoke
pnpm build:orcad
pnpm test:node-server --artifact
NODE_SERVER_QUALIFICATION
- name: Keep this runner's musl slot for the desktop template
if: inputs.build_template
uses: actions/upload-artifact@v7
with:
name: orcad-prebuild-musl-${{ matrix.os }}
path: out/orcad-prebuilds/
if-no-files-found: error
retention-days: 7
overwrite: true
# Design D2: the desktop ships every target's addons, merged from the lanes that qualified them.
desktop_template:
needs: [persistence, linux_glibc_floor, linux_glibc217_compat, linux_musl]
if: >-
${{ !cancelled() && inputs.build_template &&
needs.persistence.result == 'success' && needs.linux_glibc_floor.result == 'success' &&
needs.linux_glibc217_compat.result == 'success' && needs.linux_musl.result == 'success' }}
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref }}
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
- name: Collect every lane's slot
uses: actions/download-artifact@v8
with:
pattern: orcad-prebuild-*
path: ${{ runner.temp }}/orcad-prebuild-lanes
- name: Collect the Windows process-table addons
uses: actions/download-artifact@v8
with:
name: orcad-windows-process-tree
path: .build/windows-process-tree
- name: Merge the lanes and gate the full slot matrix
shell: bash
run: |
node config/scripts/merge-orcad-prebuilds.mjs "$RUNNER_TEMP"/orcad-prebuild-lanes/*
pnpm build:orcad-prebuilds --require-slots
pnpm build:orcad-prebuilds --require-slots linux-x64-glibc217
- run: pnpm build:orcad-template
- name: Report the template size
shell: bash
run: |
{
echo '### orcad template'
echo '```'
du -sh out/orcad-template
du -sh out/orcad-template/targets/*
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@v7
with:
name: orcad-template
path: out/orcad-template/
# The per-target .server-target and .runtime-node markers are dotfiles.
include-hidden-files: true
if-no-files-found: error
retention-days: 7
overwrite: true
Reference in New Issue View Git Blame Copy Permalink