starred/orca
1
0
Fork 0
mirror of https://github.com/stablyai/orca.git synced 2026-11-03 18:47:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
37833ce9a02ccd6754dc9b4eb3266d39ddf8a2fd
orca/.github/workflows/e2e.yml
T
Jinwoo Hong 7f0e067256 test(e2e): layout oracle scenarios with window, headless serve and SSH (#26674) 
* test(terminal): check the runtime's workspace layout against structural rules

A pure rules check over the runtime's session partitions: a terminal in two
panes, a pane in two tabs, a tab outside exactly one group, tab rows and tab
bar disagreeing, two tab orders, and ids changing for the same pane, tab or
group across an update. Wired into the seeded layout model test after every
step, plus late-spawn cases (one known on main, kept as an expected failure).

* test(e2e): layout oracle scenarios with window, headless serve and SSH

Drives layout journeys through window gestures, CLI and paired-client runtime
commands, and after every step checks the runtime's layout against the
structural rules, the window's DOM, the paired-client and CLI views, the
expected end layout, restarts (relaunch, cold daemon, in-app daemon restart)
and per-pane terminal markers. Findings main still has are listed with their
cause; anything else fails. Includes headless orcad/Electron serve scenarios
for the serve issues and a Docker SSH scenario.

* test(terminal): check layout rules on every write and race client commands with the window

The layout model now checks the structural rules on every in-memory
workspace-session write, not only at each step's end, naming the writer
of any breach. Three documented two-writer transients (main mints a tab
row before the window writes its tab bar) are allow-listed by rule, op
and writer; they must still heal by the step's end.

New model ops race a stale window save against a paired client's tab
close, a host-created split, and main's PTY-exit retirement. A new file
runs main's updateMobileSessionPaneLayout (old phone pane trees) against
the real Store; the case with no topology revision is an it.fails: main
drops a live pane and its binding there.

* test(e2e): oracle expected outcomes, transient sampling, remount counts, more scenarios, latency

Every step checks the expected panes, which panes went away and that the rest
kept their terminals; a 100 ms sampler runs the rules between steps; xterm
creations are counted per pane and an unexpected recreation fails. Adds host
navigation, window reload (graph republication), quit right after a change
(quit-time save), an old client's stale pane tree, and a latency measurement
(command to drawn update, idle and under an output flood; no pass/fail).

* docs(e2e): describe the oracle's added checks and latency measurement

* test(e2e): #12723 SSH folder scenario, CI lane for the SSH oracle spec, short socket dir

Adds an SSH folder-workspace (no repo) scenario that checks the layout lives in
one partition across relaunch; registers the SSH oracle spec in the Docker SSH
lane's selection lists; headless hosts put long-TMPDIR profiles under ~/.orca-lo
instead of /tmp.

* ci(e2e): run the headless layout oracle in the job that builds orcad; Linux relaunch findings

The headless oracle spec needs out/orcad, which only the orca serve mode-switch
job builds, so it runs there (and fails rather than skips in CI if orcad is
missing). Records what the first Linux CI run found on main: after a relaunch
main can hold tab rows with no tab bar past the settle wait, and Electron serve
on Linux also lists a renamed terminal's title as null after a restart.

* test(e2e): join partition lists in the SSH folder scenario's log lines

* test(e2e): retry the latency spec's reads through a brief terminal-ownership gap

Linux CI saw terminal.list answer terminal_surface_ownership_unavailable while
the window re-published its terminals; reads retry for up to 10 s, commands do not.
2026-10-08 21:18:21 -04:00

750 lines
34 KiB
YAML
Raw Blame History

name: E2E
run-name: E2E ${{ inputs.ref || github.ref }}
# Why: checkout + artifact upload only; callers can only further restrict.
permissions:
contents: read
on:
workflow_call:
inputs:
ref:
description: Ref to check out (defaults to the calling workflow's ref)
required: false
type: string
test_files:
description: JSON array of changed specs; empty runs the full suite
required: false
type: string
run_changed_e2e:
description: PR detector found specs requiring the general changed-spec lane.
required: false
type: boolean
default: true
needs_build:
description: PR detector found an Electron consumer requiring build and native cache.
required: false
type: boolean
default: true
ssh_source_changed:
description: '"true" when the PR touches SSH execution source; gates the Docker-SSH lane'
required: false
type: string
workflow_dispatch:
inputs:
ref:
description: Ref to check out (defaults to the workflow ref)
required: false
type: string
test_files:
description: JSON array of specs to run; empty runs the full suite
required: false
type: string
schedule:
# One complete daily reference run; targeted PR coverage remains unchanged.
- cron: '0 17 * * *'
jobs:
build:
name: build e2e app
if: inputs.test_files == '' || github.event_name != 'pull_request' || inputs.needs_build
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
# Why: the build's plain-Node daemon smoke load resolves node-pty.
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: node
# Why: building here avoids parallel builds inside Playwright globalSetup;
# paired-browser specs also need the standalone web bundle.
- name: Build E2E outputs
env:
VITE_EXPOSE_STORE: 'true'
run: |
status=0
pnpm run build:relay &
relay_pid=$!
pnpm run build:electron-vite:parallel --mode e2e || status=1
wait "$relay_pid" || status=1
exit "$status"
# The CLI emits into out/main too; start only after Electron finishes clearing that tree.
- name: Build shared E2E CLI
id: e2e-cli
background: true
run: |
if node -e 'process.exit(require("./package.json").scripts["prepare:cli-output"] ? 0 : 1)'; then
pnpm run build:cli
else
echo "Older ref: let each consumer build and install its CLI."
fi
- name: Project shared E2E web client
run: pnpm run build:web-from-renderer
- wait: e2e-cli
- name: Upload E2E build output
uses: actions/upload-artifact@v7
with:
name: e2e-build-out
path: out/
# Why: build-relay.mjs writes each relay's marker as `out/relay/<platform>/.version`,
# and upload-artifact drops dotfiles by default — consumers then fail SSH specs with
# "local relay build is missing its version marker".
include-hidden-files: true
retention-days: 1
if-no-files-found: error
# Build Electron-native dependencies once per workflow. Consumer shards restore
# this immutable cache instead of compiling the same ABI concurrently.
prepare-native-cache:
name: prepare Electron native cache
if: inputs.test_files == '' || github.event_name != 'pull_request' || inputs.needs_build
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Install native build tools
env:
ORCA_E2E_APT_PACKAGES: build-essential python3
run: &install_e2e_tools |
read -r -a packages <<< "$ORCA_E2E_APT_PACKAGES"
# Why apt-mirrors.txt: runner images resolve mirror+file: sources through it, so editing the lists alone leaves Azure in use.
for source in /etc/apt/apt-mirrors.txt /etc/apt/sources.list /etc/apt/sources.list.d/*.list /etc/apt/sources.list.d/*.sources; do
if [ -f "$source" ]; then
sudo sed -i 's|https*://azure\.archive\.ubuntu\.com/ubuntu|https://archive.ubuntu.com/ubuntu|g' "$source"
fi
done
sudo tee /etc/apt/apt.conf.d/99-orca-e2e >/dev/null <<'APTCONF'
Acquire::http::Timeout "15";
Acquire::https::Timeout "15";
Acquire::Retries "1";
APTCONF
timeout 120 sudo apt-get update
timeout 300 sudo apt-get install -y "${packages[@]}" || timeout 300 sudo apt-get install -y --fix-missing "${packages[@]}"
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
e2e:
name: e2e ${{ matrix.shard_name }}
needs: [build, prepare-native-cache]
if: inputs.test_files == ''
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
# Fourteen scheduled runs averaged 24.6 minutes per shard; shards 4
# and 9 repeatedly hit the 30-minute cap. A 12-way trial still left
# one 30-minute shard, so 14 gives the suite enough failure headroom.
- shard: '1/14'
shard_name: 1-of-14
- shard: '2/14'
shard_name: 2-of-14
- shard: '3/14'
shard_name: 3-of-14
- shard: '4/14'
shard_name: 4-of-14
- shard: '5/14'
shard_name: 5-of-14
- shard: '6/14'
shard_name: 6-of-14
- shard: '7/14'
shard_name: 7-of-14
- shard: '8/14'
shard_name: 8-of-14
- shard: '9/14'
shard_name: 9-of-14
- shard: '10/14'
shard_name: 10-of-14
- shard: '11/14'
shard_name: 11-of-14
- shard: '12/14'
shard_name: 12-of-14
- shard: '13/14'
shard_name: 13-of-14
- shard: '14/14'
shard_name: 14-of-14
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
# Native cache misses need the compiler, Electron needs Xvfb, and paired
# Quick Open needs ripgrep. Install them in one apt transaction per shard.
- name: Install native build and headless UI tools
# The Azure archive took 16 minutes for one font package; bound setup
# separately so a slow mirror cannot consume the shard's test budget.
env: &e2e_tool_packages
ORCA_E2E_APT_PACKAGES: build-essential fonts-noto-cjk openssh-client python3 ripgrep xvfb zsh openbox x11-utils
run: *install_e2e_tools
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- name: Download E2E build output
uses: actions/download-artifact@v8
with:
name: e2e-build-out
path: out/
# Why: the Electron suite is wall-clock constrained on OSS runners, but
# multiple Electron apps on one Xvfb VM contend on git/Chromium resources.
# Sharding keeps each VM at one Playwright worker while splitting the
# headless suite across separate runners.
# SKIP_BUILD makes Playwright globalSetup reuse the single build job's
# artifact instead of starting five concurrent electron-vite builds.
# ORCA_E2E_FORWARD_APP_LOGS keeps startup failures visible when Electron
# launches but never creates a BrowserWindow.
- name: Balance E2E shard from timing evidence
env:
ORCA_BACKGROUND_LAUNCH: '1'
SKIP_BUILD: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
ORCA_E2E_WEB_CLIENT: '1'
ORCA_RELAY_PATH: ${{ github.workspace }}/out/relay
run: |
mkdir -p ci-shards
pnpm exec playwright test --config tests/playwright.config.ts --project=electron-headless --list --reporter=json > ci-shards/discovery.json
export ORCA_SHARD_SOURCE_SHA="$(git rev-parse HEAD)"
node config/scripts/ci-e2e-shard-plan.mjs ci-shards/discovery.json '${{ matrix.shard }}' ci-shards
pnpm exec playwright test --config tests/playwright.config.ts --project=electron-headless --test-list=ci-shards/selected.txt --list --reporter=json > ci-shards/selected-discovery.json
node config/scripts/ci-e2e-shard-plan.mjs --verify ci-shards/assignment.json ci-shards/selected-discovery.json
- name: Run E2E tests (${{ matrix.shard_name }})
env:
PLAYWRIGHT_JSON_OUTPUT_FILE: ci-shards/results.json
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 ORCA_E2E_WEB_CLIENT=1 ORCA_RELAY_PATH="$GITHUB_WORKSPACE/out/relay" pnpm run test:e2e --test-list=ci-shards/selected.txt --reporter=list,json
- name: Summarize E2E failures
if: always()
continue-on-error: true
run: node config/scripts/ci-e2e-failure-summary.mjs ci-shards/results.json
- name: Upload E2E shard assignment
if: always()
# Diagnostic upload outages must not change the test verdict.
continue-on-error: true
uses: actions/upload-artifact@v7
with:
name: e2e-shard-${{ matrix.shard_name }}-attempt-${{ github.run_attempt }}
path: ci-shards/
retention-days: 14
if-no-files-found: warn
# The frame benchmark needs a mapped window, which the headless shards exclude.
- name: Run worktree first-paint benchmark
if: matrix.shard == '1/14'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 pnpm exec playwright test tests/e2e/worktree-switch-first-paint.spec.ts --config tests/playwright.config.ts --project=electron-headful --workers=1
# Why: Playwright retains traces/screenshots only on failure. Uploading
# them as an artifact makes post-mortem debugging on CI possible without
# re-running locally.
- name: Upload Playwright traces
if: failure() || cancelled()
uses: actions/upload-artifact@v7
with:
name: playwright-traces-${{ matrix.shard_name }}
path: test-results/
retention-days: 7
if-no-files-found: ignore
changed-e2e:
name: changed e2e specs
needs: [build, prepare-native-cache]
if: inputs.test_files != '' && (github.event_name != 'pull_request' || inputs.run_changed_e2e)
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Install native build and headless UI tools
# Why ripgrep: Quick Open's bounded host-side search requires rg instead of an
# unbounded inventory fallback; the paired fixture exercises that real boundary.
# Why openssh-client: the Docker-SSH fixture shells out to ssh/ssh-keygen, and this
# lane now receives those specs from pr.yml's SSH source mapping.
env: *e2e_tool_packages
run: *install_e2e_tools
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- name: Download E2E build output
uses: actions/download-artifact@v8
with:
name: e2e-build-out
path: out/
- name: Run changed E2E specs
env:
TEST_FILES_JSON: ${{ inputs.test_files }}
run: |
if [ -f config/scripts/ci-e2e-job-selection.mjs ]; then
printf '%s\n' "$TEST_FILES_JSON" | node config/scripts/ci-e2e-job-selection.mjs > "$RUNNER_TEMP/general-e2e-specs"
else
# Dispatching an older ref retains its existing dedicated-owner exclusions.
jq -r '.[] | select(
. != "tests/e2e/local-ssh-browser-routing.spec.ts" and
. != "tests/e2e/ssh-client-hosted-browser-drop-reconnect.spec.ts" and
. != "tests/e2e/pty-input-write-queue-ssh.spec.ts" and
. != "tests/e2e/ssh-ai-vault-session-history.spec.ts" and
. != "tests/e2e/ssh-codex-display-artifacts-repro.spec.ts" and
. != "tests/e2e/ssh-cold-activation-restore.spec.ts" and
. != "tests/e2e/ssh-cold-hydration-gap-tab-seeding.spec.ts" and
. != "tests/e2e/ssh-emptied-worktree-reactivation.spec.ts" and
. != "tests/e2e/ssh-docker-five-pane-input-under-flood.spec.ts" and
. != "tests/e2e/ssh-docker-bulk-open-freeze-repro.spec.ts" and
. != "tests/e2e/ssh-docker-half-open-link.spec.ts" and
. != "tests/e2e/ssh-docker-quick-open-large-listing.spec.ts" and
. != "tests/e2e/ssh-docker-reconnect-pane-restore.spec.ts" and
. != "tests/e2e/ssh-docker-relay-stall-credential.spec.ts" and
. != "tests/e2e/ssh-docker-resource-accumulation.spec.ts" and
. != "tests/e2e/ssh-docker-transport-drop-recovery.spec.ts" and
. != "tests/e2e/ssh-external-image-preview.spec.ts" and
. != "tests/e2e/ssh-lost-kill-tab-resurrection.spec.ts" and
. != "tests/e2e/ssh-pi-compatible-agent-title.spec.ts" and
. != "tests/e2e/ssh-port-forward-lifecycle.spec.ts" and
. != "tests/e2e/ssh-reattach-home-partition.spec.ts" and
. != "tests/e2e/ssh-reconnect-tab-destruction.spec.ts" and
. != "tests/e2e/ssh-restart-tab-accumulation.spec.ts" and
. != "tests/e2e/ssh-skill-installation.spec.ts" and
. != "tests/e2e/ssh-stale-resume-execution-host-scope.spec.ts" and
. != "tests/e2e/ssh-startup-local-shadow.spec.ts" and
. != "tests/e2e/ssh-terminal-window-wake-stale-grid-repro.spec.ts" and
. != "tests/e2e/terminal-inline-images-ssh.spec.ts" and
. != "tests/e2e/ssh-docker-watcher-isolation.spec.ts" and
. != "tests/e2e/ssh-terminal-parking.spec.ts" and
. != "tests/e2e/terminal-retention-budget.spec.ts" and
. != "tests/e2e/ssh-startup-exec-readiness.spec.ts" and
. != "tests/e2e/paired-startup-exec-readiness.spec.ts" and
. != "tests/e2e/workspace-layout-oracle-ssh.spec.ts" and
. != "tests/e2e/ssh-browser-network-execution-route.docker.unit.test.ts" and
. != "tests/e2e/ssh-localhost.spec.ts" and
. != "tests/e2e/terminal-ibus-hangul-native.spec.ts" and
. != "tests/e2e/orcad-serve-mode-switch.spec.ts" and
. != "tests/e2e/workspace-layout-oracle-headless.spec.ts" and
. != "tests/e2e/ssh-orcad-auto-convert.spec.ts" and
. != "tests/e2e/windows-missing-appdata-startup.spec.ts" and
. != "tests/e2e/ssh-orcad-idle-exit.spec.ts"
)' <<<"$TEST_FILES_JSON" > "$RUNNER_TEMP/general-e2e-specs"
fi
mapfile -t TEST_FILES < "$RUNNER_TEMP/general-e2e-specs"
if [ "${#TEST_FILES[@]}" -eq 0 ]; then
echo "Changed specs are all owned by dedicated lanes."
exit 0
fi
E2E_ENV=(SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 ORCA_E2E_WEB_CLIENT=1 ORCA_RELAY_PATH="$GITHUB_WORKSPACE/out/relay")
# Second clause: a spec that reads ORCA_E2E_SSH_DOCKER test.skip()s itself without it, so
# naming only one trigger silently skipped every other Docker-SSH spec in this lane.
# The first clause stays because that spec needs Docker without referencing the variable.
if printf '%s\n' "${TEST_FILES[@]}" | grep -qx 'tests/e2e/ephemeral-vm-provisioned-root.spec.ts' \
|| grep -l 'ORCA_E2E_SSH_DOCKER' "${TEST_FILES[@]}" >/dev/null 2>&1; then
E2E_ENV+=(ORCA_E2E_SSH_DOCKER=1)
fi
E2E_PROJECT_ARGS=()
if grep -l '@headful' "${TEST_FILES[@]}" >/dev/null; then
E2E_PROJECT_ARGS+=(--project=electron-headful)
fi
xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env "${E2E_ENV[@]}" \
pnpm run test:e2e "${TEST_FILES[@]}" --workers=1 "${E2E_PROJECT_ARGS[@]}"
- name: Upload Playwright traces
if: failure() || cancelled()
uses: actions/upload-artifact@v7
with:
name: playwright-traces-changed
path: test-results/
retention-days: 7
if-no-files-found: ignore
ssh-docker-watcher-isolation:
name: ssh docker watcher isolation (${{ matrix.shard }}/4)
needs: [build, prepare-native-cache]
# Each excluded changed spec must trigger its dedicated owner.
if: >-
inputs.test_files == '' ||
inputs.ssh_source_changed == 'true' ||
contains(inputs.test_files, 'tests/e2e/local-ssh-browser-routing.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-client-hosted-browser-drop-reconnect.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/pty-input-write-queue-ssh.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-ai-vault-session-history.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-codex-display-artifacts-repro.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-cold-activation-restore.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-cold-hydration-gap-tab-seeding.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-emptied-worktree-reactivation.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-five-pane-input-under-flood.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-bulk-open-freeze-repro.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-half-open-link.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-quick-open-large-listing.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-reconnect-pane-restore.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-relay-stall-credential.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-resource-accumulation.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-transport-drop-recovery.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-external-image-preview.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-lost-kill-tab-resurrection.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-pi-compatible-agent-title.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-port-forward-lifecycle.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-reconnect-tab-destruction.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-restart-tab-accumulation.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-skill-installation.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-stale-resume-execution-host-scope.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-terminal-window-wake-stale-grid-repro.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/terminal-inline-images-ssh.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-docker-watcher-isolation.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-terminal-parking.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/terminal-retention-budget.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/ssh-startup-exec-readiness.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/paired-startup-exec-readiness.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/workspace-layout-oracle-ssh.spec.ts')
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Install native build and headless UI tools
env: *e2e_tool_packages
run: *install_e2e_tools
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- name: Download E2E build output
uses: actions/download-artifact@v8
with:
name: e2e-build-out
path: out/
# Why: this is the release-path proof that the deployed Linux relay keeps
# its PTY and explorer live across a real watcher SIGSEGV.
- name: Run Docker SSH watcher isolation E2E
if: matrix.shard == 1
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 pnpm run test:e2e:ssh-docker-watcher-isolation
# Why: Playwright empties test-results/ when it starts, so each step here used to
# destroy the previous step's traces. Only the last lane's failure was ever
# diagnosable from the artifact; set each lane aside before the next one runs.
- name: Keep watcher-isolation traces
if: always() && matrix.shard == 1
run: |
if [ -d test-results ]; then
mkdir -p e2e-traces
mv test-results "e2e-traces/watcher-isolation"
fi
# Keep every SSH journey after a test failure; stop work on superseded commits.
- name: Run Docker SSH terminal parking + startup readiness E2E
if: '!cancelled() && matrix.shard == 1'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 pnpm run test:e2e:ssh-docker-terminal-parking
- name: Keep terminal-parking traces
if: always() && matrix.shard == 1
run: |
if [ -d test-results ]; then
mkdir -p e2e-traces
mv test-results "e2e-traces/terminal-parking"
fi
# Separate VMs isolate destructive SSH fixtures while keeping one worker per shard.
- name: Run remaining Docker SSH E2E
if: '!cancelled()'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 pnpm run test:e2e:ssh-docker --shard=${{ matrix.shard }}/4
- name: Keep remaining-ssh-docker traces
if: always()
run: |
if [ -d test-results ]; then
mkdir -p e2e-traces
mv test-results "e2e-traces/remaining-ssh-docker"
fi
- name: Upload watcher isolation traces
if: failure() || cancelled()
uses: actions/upload-artifact@v7
with:
name: playwright-traces-ssh-docker-watcher-isolation-${{ matrix.shard }}
path: e2e-traces/
retention-days: 7
if-no-files-found: ignore
ssh-browser-network-route:
name: ssh browser network route
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/ssh-browser-network-execution-route.docker.unit.test.ts')
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: node
- name: Install SSH client
run: sudo apt-get update && sudo apt-get install -y openssh-client
- name: Run Docker SSH browser network route journeys
env:
ORCA_BACKGROUND_LAUNCH: '1'
ORCA_RUN_DOCKER_SSH_BROWSER_E2E: '1'
run: node_modules/.bin/vitest run --config config/vitest.config.ts tests/e2e/ssh-browser-network-execution-route.docker.unit.test.ts
orcad-serve-mode-switch:
name: orca serve Electron/orcad mode switch (D7)
needs: [build, prepare-native-cache]
if: >-
inputs.test_files == '' ||
contains(inputs.test_files, 'tests/e2e/orcad-serve-mode-switch.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/workspace-layout-oracle-headless.spec.ts')
runs-on: ubuntu-latest
timeout-minutes: 45
env:
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
persist-credentials: false
- name: Install headless tools
run: sudo apt-get update && sudo apt-get install -y build-essential ripgrep xvfb openbox x11-utils
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- uses: actions/download-artifact@v8
with:
name: e2e-build-out
path: out/
# The packaged orcad slot the T6-11 launcher runs: its own node-pty under the pinned Node.
# The template is what `orca serve`'s default selection materializes that slot from.
- name: Build this runner's orcad slot and template
run: |
slot="$(node config/scripts/build-orcad-prebuilds.mjs --print-slot)"
pnpm build:orcad-prebuilds
pnpm build:orcad-prebuilds --require-slots "$slot"
pnpm build:orcad
node config/scripts/build-orcad-template.mjs --targets "$slot"
- name: Switch serve hosts on one profile
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/orcad-serve-mode-switch.spec.ts')
env:
SKIP_BUILD: '1'
ORCA_E2E_ORCAD_SERVE: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/orcad-serve-mode-switch.spec.ts --project=electron-headless --workers=1
# The layout oracle's headless scenarios run on both serve hosts, so they need this job's orcad.
- name: Layout oracle on headless serve (orcad and Electron)
if: >-
!cancelled() &&
(inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/workspace-layout-oracle-headless.spec.ts'))
env:
SKIP_BUILD: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/workspace-layout-oracle-headless.spec.ts --project=electron-headless --workers=1
- uses: actions/upload-artifact@v7
if: failure()
with:
name: orcad-serve-mode-switch-traces
path: test-results/
retention-days: 7
if-no-files-found: ignore
# #24979 on a real host: a relay-era Docker host converts to managed orcad on connect, and a managed
# orcad idles out and restarts on the next connect. Needs the
# orcad template for the fixture's target (Debian, linux-x64-glibc), which only this job builds.
orcad-auto-convert-docker:
name: ssh host auto-converts to managed orcad (Docker)
needs: [build, prepare-native-cache]
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/ssh-orcad-auto-convert.spec.ts') || contains(inputs.test_files, 'tests/e2e/ssh-orcad-idle-exit.spec.ts')
runs-on: ubuntu-latest
timeout-minutes: 45
env:
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
persist-credentials: false
- name: Install headless tools
run: sudo apt-get update && sudo apt-get install -y build-essential ripgrep xvfb openbox x11-utils
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- uses: actions/download-artifact@v8
with:
name: e2e-build-out
path: out/
# Built to a path the app does not look at by default, so the relay phase has no template.
- name: Build the linux-x64-glibc orcad template
run: |
pnpm build:orcad-prebuilds
pnpm build:orcad-prebuilds --require-slots linux-x64-glibc
pnpm build:orcad
node config/scripts/build-orcad-template.mjs --targets linux-x64-glibc
mv out/orcad-template "$RUNNER_TEMP/orcad-convert-template"
- name: Convert a relay-era Docker host
env:
SKIP_BUILD: '1'
ORCA_E2E_SSH_DOCKER: '1'
ORCA_E2E_ORCAD_CONVERT_HOST: docker
ORCA_E2E_FORWARD_APP_LOGS: '1'
ORCA_RELAY_PATH: ${{ github.workspace }}/out/relay
run: |
export ORCA_E2E_ORCAD_CONVERT_TEMPLATE="$RUNNER_TEMP/orcad-convert-template"
xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/ssh-orcad-auto-convert.spec.ts tests/e2e/ssh-orcad-idle-exit.spec.ts --project=electron-headless --workers=1
- uses: actions/upload-artifact@v7
if: failure()
with:
name: orcad-auto-convert-docker-traces
path: test-results/
retention-days: 7
if-no-files-found: ignore
# D7 on Windows: both hosts share <userData>\daemon and so one daemon pipe. Built here rather than
# from the Linux e2e build because the slot, template and addons are Windows-native. Also runs the
# missing-AppData startup check, which needs the same Windows e2e build.
orcad-serve-mode-switch-windows:
name: orca serve on Windows (D7 mode switch, missing AppData)
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/orcad-serve-mode-switch.spec.ts') || contains(inputs.test_files, 'tests/e2e/windows-missing-appdata-startup.spec.ts')
runs-on: windows-2022
timeout-minutes: 45
env:
NODE_OPTIONS: --max-old-space-size=4096
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- uses: ./.github/actions/prepare-orcad-prebuilds
with:
resolve-windows-cache: 'true'
restore-windows-cache: 'true'
- name: Build the e2e app, CLI, orcad slot and template
shell: bash
run: |
pnpm run build:relay
pnpm exec electron-vite build --mode e2e
pnpm run build:cli
pnpm build:orcad
node config/scripts/build-windows-process-tree-relay-addon.mjs --arch=x64
node config/scripts/build-orcad-template.mjs --targets win32-x64
# A profile-less Windows session has no roaming AppData; Electron 43 used to crash natively there.
- name: Start serve with no AppData folder
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/windows-missing-appdata-startup.spec.ts')
env:
SKIP_BUILD: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
run: pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/windows-missing-appdata-startup.spec.ts --project=electron-headless --workers=1
- name: Switch serve hosts on one profile
if: ${{ !cancelled() && (inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/orcad-serve-mode-switch.spec.ts')) }}
env:
SKIP_BUILD: '1'
ORCA_E2E_ORCAD_SERVE: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
ORCA_E2E_PRESERVE_PROFILE_LOGS_DIR: ${{ github.workspace }}\test-results\profile-logs
run: pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/orcad-serve-mode-switch.spec.ts --project=electron-headless --workers=1
- uses: actions/upload-artifact@v7
if: failure()
with:
name: orcad-serve-mode-switch-windows-traces
path: test-results/
retention-days: 7
if-no-files-found: ignore
ssh-localhost:
name: localhost SSH terminal and hooks
needs: [build, prepare-native-cache]
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/ssh-localhost.spec.ts')
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Install SSH server and headless tools
run: sudo apt-get update && sudo apt-get install -y build-essential openssh-client openssh-server python3 ripgrep xvfb zsh openbox x11-utils
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- uses: actions/download-artifact@v8
with:
name: e2e-build-out
path: out/
- name: Start isolated localhost SSH server
shell: bash
run: |
# Bare shells install Pi extensions only for an existing agent home.
mkdir -p "$HOME/.pi/agent"
fixture="$RUNNER_TEMP/orca-localhost-sshd"
mkdir -p "$fixture"
ssh-keygen -q -t ed25519 -N '' -f "$fixture/host_key"
ssh-keygen -q -t ed25519 -N '' -f "$fixture/client_key"
cat > "$fixture/sshd_config" <<EOF
Port 22222
ListenAddress 127.0.0.1
HostKey $fixture/host_key
PidFile $fixture/sshd.pid
AuthorizedKeysFile $fixture/client_key.pub
StrictModes no
PasswordAuthentication no
KbdInteractiveAuthentication no
UsePAM yes
AllowUsers $(id -un)
Subsystem sftp internal-sftp
EOF
sudo mkdir -p /run/sshd
sudo /usr/sbin/sshd -f "$fixture/sshd_config" -E "$fixture/sshd.log"
ssh -i "$fixture/client_key" -p 22222 -o BatchMode=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null 127.0.0.1 true || { sudo cat "$fixture/sshd.log"; exit 1; }
{
echo "ORCA_E2E_SSH_PORT=22222"
echo "ORCA_E2E_SSH_USER=$(id -un)"
echo "ORCA_E2E_SSH_IDENTITY_FILE=$fixture/client_key"
} >> "$GITHUB_ENV"
- name: Run localhost SSH terminal and hook journey
env:
SKIP_BUILD: '1'
ORCA_E2E_SSH_LOCALHOST: '1'
ORCA_FEATURE_REMOTE_AGENT_HOOKS: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/ssh-localhost.spec.ts --project=electron-headless --workers=1
- uses: actions/upload-artifact@v7
if: failure() || cancelled()
with:
name: localhost-ssh-traces
path: test-results/
retention-days: 7
if-no-files-found: ignore
Reference in New Issue View Git Blame Copy Permalink