Files
orca/.github/workflows/pr.yml
T
Jinwoo Hong 06780260c0 test(remote-runtime): run an old client and an old server against current code (#12682)
Mixed versions are the normal state of the remote-server feature: users update clients and servers independently. Until now nothing tested that. Every cross-version claim was made by code reading plus unit tests with hand-written old/new shapes — enough to catch design problems, not enough to catch a real skew regression.

This runs the REAL protocol implementations from two builds against each other in one process: the actual host methods and RPC dispatcher on one side, the actual renderer multiplexer on the other, with a transport that reproduces the production asymmetry — each side decodes with its OWN codec and drops frames whose opcode it does not know. A frame survives only if the RECEIVING build understands it, which is what makes this level sufficient without launching two apps. The old side is a genuine checkout extracted from the release tag; the extracted client was confirmed to lack a symbol that exists only on main.

Journey: subscribe, first snapshot, input reaching the process, live output, hide/reveal snapshot, transport drop, resubscribe, input landing again — across old->new, new->old, and a current/current control. Every step ends on an observed-state barrier; no sleeps. The oracle asserts the recorded step list, the exact 16-frame named sequence, negotiated capabilities, the exact input the host wrote to the PTY, rendered content, and zero decoder-rejected frames. A host method the stub lacks is recorded by name and asserted empty, so a harness gap cannot masquerade as a wire break.

Detection is proven per violation shape, and it attributes each to the correct side: an unnegotiated opcode goes red only where a decoder would reject it, a removed published field goes red only where an old client consumes it, and a legal additive field stays green in all three pairings so the harness will not cry wolf on safe changes.

It also documents the three compatibility rules in docs/reference/remote-wire-compatibility.md, linked from AGENTS.md, since they previously existed only as folklore — notably that "decoders reject unknown opcodes" is true for the desktop decoder but NOT for mobile, which silently drops them.

Deliberately scoped: terminal stream only. The session-tab sync channel is not covered, nor agent-session publications, file/Git RPCs, mobile E2EE framing, or the relay transport. Two version points, so a regression introduced and reverted between them is invisible.

CI selection was verified rather than assumed — `vitest list` confirms 0 matches under the shard's exclude and 4 under the dedicated job — because a lane silently running zero tests is precisely how a host-side defect escaped CI earlier in this series. Closes STA-3469.
2026-08-05 01:31:29 -07:00

463 lines
15 KiB
YAML

name: PR Checks
on:
pull_request:
types:
- opened
- synchronize
- reopened
- ready_for_review
concurrency:
group: pr-checks-${{ github.event.pull_request.number }}
cancel-in-progress: true
permissions:
contents: read
jobs:
static_analysis:
name: static analysis
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
- name: Lint
run: pnpm exec oxlint --format github
- name: Enforce focused code-quality plugins
run: pnpm run audit:code-quality:native
- name: Enforce type-aware code-quality baseline
run: pnpm run audit:code-quality:type-aware
- name: Enforce changed-code quality
run: pnpm run check:code-quality:changed -- "${{ github.event.pull_request.base.sha }}"
- name: Enforce React Doctor on changed lines
run: pnpm run check:react-doctor:changed -- "${{ github.event.pull_request.base.sha }}"
- name: Check Zustand selector fan-out budget
run: pnpm run check:zustand-selector-fanout
- name: Check reliability gate manifest
run: pnpm run check:reliability-gates
- name: Enforce max-lines ratchet
run: pnpm run check:max-lines-ratchet
- name: Verify bundled skill guides
run: pnpm run verify:bundled-skill-guides
- name: Verify skill freshness manifest
run: pnpm run verify:skill-bundle-manifest
- name: Verify localization catalog
run: pnpm run verify:localization-catalog
# Why: extraction writes sorted evidence to an isolated temporary path,
# so feature PRs need one normalized AST pass rather than a three-OS matrix.
- name: Verify localization extraction
run: pnpm run verify:localization-extraction
- name: Verify localization coverage
run: pnpm run verify:localization-coverage
# Why: project-owned type declarations must live in .ts so tsc
# actually checks them. TypeScript's skipLibCheck: true (inherited
# from @electron-toolkit/tsconfig) silently widens unresolved names
# in .d.ts to `any`, which is how #1186 shipped a broken IPC signature
# past typecheck. See docs/preload-typecheck-hole.md.
- name: Guard against project-owned .d.ts in preload/shared
run: |
matches=$(find src/preload src/shared -name '*.d.ts' 2>/dev/null || true)
if [ -n "$matches" ]; then
echo "::error::Project-owned .d.ts files are not allowed under src/preload or src/shared."
echo "Move type declarations into a .ts file so skipLibCheck does not hide errors."
echo "See docs/preload-typecheck-hole.md."
echo "Found:"
echo "$matches"
exit 1
fi
- name: Check feature wall asset budget
run: pnpm check:feature-wall-assets
- name: Verify macOS entitlements
run: pnpm verify:macos-entitlements
root_directory_guard:
name: root directory guard
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
persist-credentials: false
- name: Reject new root-level files and folders
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: bash .github/scripts/check-root-directory-entries.sh "$BASE_SHA" "$HEAD_SHA"
typecheck:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
- run: pnpm typecheck
git_compatibility:
name: Git compatibility
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
- name: Verify Git binary compatibility matrix
run: |
pids=()
(
archive="$RUNNER_TEMP/git-2.25.5.tar.gz"
source="$RUNNER_TEMP/git-2.25.5"
curl -fsSL https://www.kernel.org/pub/software/scm/git/git-2.25.5.tar.gz -o "$archive"
echo "41662c52fc16fec4963bfc41075e71f8ead6b5e386797eb6f9a1111ff95a8ddf $archive" \
| sha256sum --check
mkdir -p "$source"
tar -xzf "$archive" -C "$source" --strip-components=1
make -C "$source" -j"$(nproc)" \
NO_GETTEXT=YesPlease NO_TCLTK=YesPlease NO_PYTHON=YesPlease git
ORCA_GIT_COMPAT_BINARY="$source/git" ORCA_GIT_COMPAT_VERSION="2.25.5" \
pnpm exec vitest run --config config/vitest.config.ts \
src/shared/git-binary-compatibility.test.ts
) &
pids+=("$!")
for spec in \
"alpine/git:edge-2.38.1|2.38.1" \
"alpine/git:v2.49.1|2.49.1"; do
(
image="${spec%%|*}"
version="${spec#*|}"
ORCA_GIT_COMPAT_IMAGE="$image" ORCA_GIT_COMPAT_VERSION="$version" \
pnpm exec vitest run --config config/vitest.config.ts \
src/shared/git-binary-compatibility.test.ts
) &
pids+=("$!")
done
status=0
for pid in "${pids[@]}"; do
wait "$pid" || status=1
done
exit "$status"
shell_contracts:
name: shell contracts
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Install zsh
run: sudo apt-get update && sudo apt-get install -y zsh
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: node
- name: Test real shell contracts
run: |
pnpm exec vitest run --config config/vitest.config.ts \
src/main/daemon/shell-ready.test.ts \
src/main/daemon/node-pty-fd-leak.test.ts \
src/main/providers/local-pty-shell-ready.test.ts \
src/main/providers/__tests__/shell-ready-framework-example.test.ts \
src/main/pty/omp-shell-wrapper.node-pty.test.ts \
src/shared/posix-command-path-lookup.test.ts
test:
name: tests node ${{ matrix.node }} ${{ matrix.shard }}/${{ matrix.shard_total }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node: ['24', '26']
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
shard_total: [16]
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: node
node-version: ${{ matrix.node }}
- name: Install Electron package binary for tests
run: node config/scripts/install-electron-package-binary.mjs
- name: Test shard
run: |
pnpm exec vitest run --config config/vitest.config.ts \
--exclude=src/main/daemon/shell-ready.test.ts \
--exclude=src/main/daemon/node-pty-fd-leak.test.ts \
--exclude=src/main/providers/local-pty-shell-ready.test.ts \
--exclude=src/main/providers/__tests__/shell-ready-framework-example.test.ts \
--exclude=src/main/pty/omp-shell-wrapper.node-pty.test.ts \
--exclude=src/shared/posix-command-path-lookup.test.ts \
--exclude=tests/e2e/cross-version-wire/** \
--shard=${{ matrix.shard }}/${{ matrix.shard_total }}
cross-version-wire:
name: cross-version wire compatibility
runs-on: ubuntu-latest
steps:
# Why fetch-depth 0: the harness extracts the newest release tag to skew
# current code against it. The default shallow clone has no tags, which is
# why this cannot ride along in the sharded `test` job.
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: node
# A path filter that matches nothing exits 1 ("No test files found"), so this
# lane cannot report success while running zero tests.
- name: Old/new client and server terminal journey
run: pnpm exec vitest run --config config/vitest.config.ts tests/e2e/cross-version-wire/cross-version-terminal-wire.unit.test.ts
package:
name: package
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Cache electron-builder downloads
uses: actions/cache@v5
with:
path: |
~/.cache/electron
~/.cache/electron-builder
key: electron-builder-linux-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: |
electron-builder-linux-
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- name: Build package inputs
run: |
status=0
pnpm run build:cli || status=1
scripts=(build:relay build:electron-vite:parallel)
pids=()
for script in "${scripts[@]}"; do
pnpm run "$script" &
pids+=("$!")
done
for pid in "${pids[@]}"; do
wait "$pid" || status=1
done
exit "$status"
- name: Project web client from renderer build
run: pnpm run build:web-from-renderer
- name: Build native components
run: pnpm run build:native
- name: Package unpacked app
env:
ORCA_REUSE_PREPARED_NATIVE_RUNTIME: '1'
run: pnpm exec electron-builder --config config/electron-builder.config.cjs --dir
- name: Smoke packaged CLI
run: node config/scripts/smoke-packaged-cli.mjs --app-dir=dist/linux-unpacked
- name: Smoke packaged hang watchdog worker
run: xvfb-run --auto-servernum node config/scripts/smoke-packaged-hang-watchdog-worker.mjs --app-dir=dist/linux-unpacked
package_windows:
name: package (windows)
runs-on: windows-2022
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
run_install: false
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version-file: package.json
cache: pnpm
- name: Cache electron-builder downloads
uses: actions/cache@v5
with:
path: |
~\AppData\Local\electron\Cache
~\AppData\Local\electron-builder\Cache
key: electron-builder-windows-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: |
electron-builder-windows-
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build package inputs
run: pnpm run build:release
- name: Prepare Electron native runtime
run: node config/scripts/ensure-native-runtime.mjs --runtime=electron
- name: Package unpacked app
env:
ORCA_REUSE_PREPARED_NATIVE_RUNTIME: '1'
run: pnpm exec electron-builder --config config/electron-builder.config.cjs --dir
- name: Smoke packaged CLI
run: node config/scripts/smoke-packaged-cli.mjs --app-dir=dist/win-unpacked
# Why: PR E2E is advisory and only validates changed specs; scheduled and
# release runs retain full-suite coverage.
e2e-paths:
name: detect changed e2e specs
runs-on: ubuntu-latest
if: github.event.pull_request.draft != true
# Why: detector only needs to read the checkout; do not inherit repo defaults.
permissions:
contents: read
outputs:
should_run: ${{ steps.filter.outputs.should_run }}
test_files: ${{ steps.filter.outputs.test_files }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
persist-credentials: false
- name: Filter changed E2E specs
id: filter
run: |
set -euo pipefail
BASE="${{ github.event.pull_request.base.sha }}"
HEAD="${{ github.event.pull_request.head.sha }}"
CHANGED="$(git diff --name-only --diff-filter=AMCR --merge-base "$BASE" "$HEAD")"
TEST_FILES="$(printf '%s\n' "$CHANGED" | grep -E '^tests/e2e/.*\.spec\.ts$' || true)"
TEST_FILES_JSON="$(printf '%s\n' "$TEST_FILES" | jq --raw-input --slurp --compact-output 'split("\n") | map(select(length > 0))')"
echo "test_files=$TEST_FILES_JSON" >> "$GITHUB_OUTPUT"
if [ "$TEST_FILES_JSON" != '[]' ]; then
echo "should_run=true" >> "$GITHUB_OUTPUT"
echo "Changed E2E specs: $TEST_FILES_JSON"
else
echo "should_run=false" >> "$GITHUB_OUTPUT"
echo "No changed E2E specs"
fi
e2e:
name: e2e
needs: e2e-paths
if: needs.e2e-paths.outputs.should_run == 'true'
# Why: reusable e2e.yml only checkouts, builds, and uploads artifacts.
permissions:
contents: read
uses: ./.github/workflows/e2e.yml
with:
test_files: ${{ needs.e2e-paths.outputs.test_files }}
verify:
if: always()
needs:
- static_analysis
- root_directory_guard
- typecheck
- git_compatibility
- shell_contracts
- test
- package
- package_windows
runs-on: ubuntu-latest
steps:
# Why: e2e is deliberately absent from needs. The suite is currently red on
# main (every scheduled run), so gating merges on it would block any PR that
# touches tests/e2e/** — including the ones fixing the suite. Until it is
# green the job runs and reports for E2E-path PRs without blocking. To flip
# it on: add `e2e` to needs, add E2E to the env below, and require
# `"$E2E" = success || skipped` after the loop — skipped is the normal
# result for a path-filtered job and must keep passing, so it has to be
# checked outside the loop or it would excuse the jobs above.
- name: Require successful checks
env:
STATIC_ANALYSIS: ${{ needs.static_analysis.result }}
ROOT_DIRECTORY_GUARD: ${{ needs.root_directory_guard.result }}
TYPECHECK: ${{ needs.typecheck.result }}
GIT_COMPATIBILITY: ${{ needs.git_compatibility.result }}
SHELL_CONTRACTS: ${{ needs.shell_contracts.result }}
TEST: ${{ needs.test.result }}
PACKAGE: ${{ needs.package.result }}
PACKAGE_WINDOWS: ${{ needs.package_windows.result }}
run: |
for result in \
"$STATIC_ANALYSIS" \
"$ROOT_DIRECTORY_GUARD" \
"$TYPECHECK" \
"$GIT_COMPATIBILITY" \
"$SHELL_CONTRACTS" \
"$TEST" \
"$PACKAGE" \
"$PACKAGE_WINDOWS"; do
if [ "$result" != "success" ]; then
exit 1
fi
done