Files
orca/.github/workflows/e2e.yml
T
OrcaWinandOrcaWin 7d6603aa17 [comment audit] Complete repository-wide comment cleanup (#33249)
* [comment audit] Align legacy failure-read counts with cached display

* [comment audit] Condense shared provider lifecycle and browser recovery comments [skip ci]

* [comment audit] Condense shared generation, path and preview comments [skip ci]

* [comment audit] Condense host, file and Git contract rationale [skip ci]

* [comment audit] Tighten Git lifecycle and provider contract comments [skip ci]

* [comment audit] Clarify shared host authority and launch isolation [skip ci]

* [comment audit] simplify local access mobile and native chat contracts [skip ci]

* [comment audit] simplify native chat presentation and state contracts [skip ci]

* [comment audit] simplify native runtime and provider contracts [skip ci]

* [comment audit] simplify migration and orchestration contracts [skip ci]

* [comment audit] simplify plugin trust and process capture rationale [skip ci]

* [comment audit] simplify terminal delivery and remote compatibility rationale [skip ci]

* [comment audit] clarify remote recovery and launch wire rationale [skip ci]

* [comment audit] shorten shared RPC compatibility and runtime evidence comments [skip ci]

* [comment audit] simplify secure storage shell startup and sidebar rationale [skip ci]

* [comment audit] clarify sidebar, skill and SSH invariants [skip ci]

* [comment audit] condense structured chat delivery and timing rationale [skip ci]

* [comment audit] condense terminal replay and telemetry invariants [skip ci]

* [comment audit] clarify launch permission and terminal delivery rationale [skip ci]

* [comment audit] clarify workspace ownership and layout contracts [skip ci]

* [comment audit] wait for fitted and painted SSH terminal links

* [comment audit] clarify layout persistence and session ownership [skip ci]

* [comment audit] clarify host routing and workspace lifecycle contracts [skip ci]

* [comment audit] clarify startup persistence and app lifetime comments [skip ci]

* [comment audit] clarify recovery and activity boundaries [skip ci]

* [comment audit] condense activity and artifact invariants [skip ci]

* [comment audit] condense automation authority contracts [skip ci]

* [comment audit] simplify automation scheduling and history contracts [skip ci]

* [comment audit] clarify browser chrome and annotation lifecycles [skip ci]

* [comment audit] clarify browser routing and retained guest ownership [skip ci]

* [comment audit] clarify browser recovery and document ownership comments [skip ci]

* [comment audit] restore hosted runners required by native IME fixtures

* [comment audit] shorten preview, palette and dashboard invariants [skip ci]

* [comment audit] clarify terminal preview, dashboard and retention comments [skip ci]

* [comment audit] clarify diff-note recovery and editor lifecycle comments [skip ci]

* [comment audit] shorten rich-text and combined-diff ownership comments [skip ci]

* [comment audit] shorten autosave, draft recovery and model ownership comments [skip ci]

* [comment audit] shorten save-queue, notebook and Markdown constraints [skip ci]

* [comment audit] shorten Monaco, PDF and rich Markdown safeguards [skip ci]

* [comment audit] clarify rich Markdown source and selection safeguards [skip ci]

* [comment audit] clarify editor loading and emulator stream lifetimes [skip ci]

* [comment audit] trim emulator and feature tour narration [skip ci]

* [comment audit] clarify floating workspace and GitHub cache ownership [skip ci]

* [comment audit] trim provider controls and Linear filter explanations [skip ci]

* [comment audit] clarify pairing and chat resume lifetimes [skip ci]

* [comment audit] clarify chat delivery and transcript lifetimes [skip ci]

* [comment audit] clarify session options and host-owned status [skip ci]

* [comment audit] clarify draft durability and live-status precedence [skip ci]

* [comment audit] Trim chat send and model ownership explanations [skip ci]

* [comment audit] Shorten transcript ownership and send recovery comments [skip ci]

* [comment audit] Simplify native chat composer and transcript hook comments [skip ci]

* [comment audit] Shorten native chat scroll and structured session explanations [skip ci]

* [comment audit] Trim workspace picker and onboarding commentary [skip ci]

* [comment audit] Simplify pet rendering, review cache and quick open comments [skip ci]

* [comment audit] Trim sidebar and session history explanations [skip ci]

* [comment audit] Condense session resume and review checks comments [skip ci]

* [comment audit] Shorten checks state and file explorer refresh comments [skip ci]

* [comment audit] Trim plugin bridge and source-control action rationale [skip ci]

* [comment audit] Trim source-control listing and review rationale [skip ci]

* [comment audit] Shorten sidebar lifecycle and explorer safeguards [skip ci]

* [comment audit] Condense Settings account and appearance rationale [skip ci]

* [comment audit] Trim pairing and repository Settings explanations [skip ci]

* [comment audit] Condense Settings search SSH and terminal safeguards [skip ci]

* [comment audit] Shorten Settings voice and provider ownership comments [skip ci]

* [comment audit] Condense Settings draft persistence and host-state rationale [skip ci]

* [comment audit] Trim Settings hooks and sidebar setup rationale [skip ci]

* [comment audit] Condense sidebar deletion and board-filter safeguards [skip ci]

* [comment audit] Condense worktree cards, host ownership and Markdown contracts [skip ci]

* [comment audit] Condense sidebar drag, folder and filtering contracts [skip ci]

* [comment audit] Condense sidebar parity, sleep and host ownership contracts [skip ci]

* [comment audit] Prune focus-read baseline after comment cleanup

* [comment audit] Clarify sidebar visibility, board drops and agent evidence [skip ci]

* [comment audit] Tighten sidebar listing, drag and reveal contracts [skip ci]

* [comment audit] Clarify viewport geometry and skill update safeguards [skip ci]

* [comment audit] Tighten skill inventory, usage and host status comments [skip ci]

* [comment audit] Clarify usage ownership and resource kill safeguards [skip ci]

* [comment audit] Tighten tab selection, rename and search contracts [skip ci]

* [comment audit] Tighten tab-strip geometry and drag ownership comments [skip ci]

* [comment audit] Tighten Tasks recovery, pagination and mutation comments [skip ci]

* [comment audit] Tighten provider loading and terminal recovery comments [skip ci]

* [comment audit] Tighten completion, cadence and PTY ownership comments [skip ci]

* [comment audit] Clarify terminal restore and parked-pane boundaries [skip ci]

* [comment audit] Tighten PTY startup and replay explanations [skip ci]

* [comment audit] Clarify PTY identity and delivery ownership [skip ci]

* [comment audit] Shorten remote reconnect and input replay rationale [skip ci]

* [comment audit] Condense terminal parking clipboard and file link comments [skip ci]

* [comment audit] Clarify IME ownership and trim trace history [skip ci]

* [comment audit] Shorten pane recovery and parked watcher explanations [skip ci]

* [comment audit] Condense snapshot replay and visibility recovery rationale [skip ci]

* [comment audit] Shorten terminal parking and keyboard ownership rationale [skip ci]

* [comment audit] Trim renderer lifecycle and menu rationale [skip ci]

* [comment audit] Condense workspace cleanup ownership and settlement notes [skip ci]

* [comment audit] Clarify renderer hook evidence and source ownership [skip ci]

* [comment audit] Tighten IPC ownership and reconnect explanations [skip ci]

* [comment audit] Trim renderer focus, paste and attention comments [skip ci]

* [comment audit] Shorten scroll, locale and agent lifecycle comments [skip ci]

* [comment audit] Trim agent launch, resume and ownership commentary [skip ci]

* [comment audit] Shorten account, browser and editor comments [skip ci]

* [comment audit] Clarify host-mirror waits, previews and input ownership [skip ci]

* [comment audit] Condense launch recovery and editor tokenization comments [skip ci]

* [comment audit] Condense workspace selection and terminal pane rationale [skip ci]

* [comment audit] Condense terminal fitting and rendering recovery comments [skip ci]

* [comment audit] Condense terminal input and host routing explanations [skip ci]

* [comment audit] Condense ownership and sleeping-session recovery rationale [skip ci]

* [comment audit] Condense structured launch and terminal link explanations [skip ci]

* [comment audit] Condense terminal ownership and workspace file routing rationale [skip ci]

* [comment audit] Condense session persistence and workspace activation rationale [skip ci]

* [comment audit] Condense workspace routing and browser ownership rationale [skip ci]

* [comment audit] Condense mirror recovery and layout authority rationale [skip ci]

* [comment audit] Condense runtime transport and file safety rationale [skip ci]

* [comment audit] Condense runtime graph and structured session rationale [skip ci]

* [comment audit] Condense browser creation and remote tab recovery rationale [skip ci]

* [comment audit] Condense snapshot authority and publisher recovery rationale [skip ci]

* [comment audit] Condense frame, startup and provider cache rationale [skip ci]

* [comment audit] Condense host twin and agent lifecycle rationale [skip ci]

* [comment audit] Condense browser authority and editor cleanup rationale [skip ci]

* [comment audit] Condense editor buffer identity and review cache rationale [skip ci]

* [comment audit] Condense provider lifetime and repository ownership rationale [skip ci]

* [comment audit] Condense reconnect, hydration and terminal retirement rationale [skip ci]

* [comment audit] Condense recovery, reconciliation and persistence rationale [skip ci]

* [comment audit] Condense cleanup ownership and listing admission rationale [skip ci]

* [comment audit] Condense teardown ordering and terminal ownership rationale [skip ci]

* [comment audit] Condense recovery ledger and web host-boundary rationale [skip ci]

* [comment audit] Condense web reconnect and credential lifecycle rationale [skip ci]

* [comment audit] Condense cloud rollout and assignment safety rationale [skip ci]

* [comment audit] Condense relay session and database recovery rationale [skip ci]

* [comment audit] Condense relay placement and protocol compatibility rationale [skip ci]

* [comment audit] Condense build and packaging constraints [skip ci]

* [comment audit] Condense CI gates and validation isolation rationale [skip ci]

* [comment audit] Condense catalog identity and browser validation rationale [skip ci]

* [comment audit] Condense mobile browser and frame-budget rationale [skip ci]

* [comment audit] Shorten mobile route and browser test explanations [skip ci]

* [comment audit] Condense editor and session seam test rationale [skip ci]

* [comment audit] Trim session and terminal closure history [skip ci]

* [comment audit] Shorten terminal browser and census explanations [skip ci]

* [comment audit] Preserve root-only E2E diagnostic baseline

* [comment audit] Condense mobile seam and HTML preview test rationale [skip ci]

* [comment audit] Condense preview instrumentation and render harness rationale [skip ci]

* [comment audit] Condense bundle contracts and native-build constraints [skip ci]

* [comment audit] Shorten native packaging and CI routing rationale [skip ci]

* [comment audit] Condense release tooling and benchmark constraints [skip ci]

* [comment audit] Condense packaging and platform gate rationale [skip ci]

* [comment audit] Shorten mobile route and history rationale [skip ci]

* [comment audit] Condense mobile browser and component rationale [skip ci]

* [comment audit] Shorten drawer and workspace-form rationale [skip ci]

* [comment audit] Trim mobile review and editor-document explanations [skip ci]

* [comment audit] Condense editor lifecycle and Markdown explanations [skip ci]

* [comment audit] Trim mobile file, diagnostics and dictation commentary [skip ci]

* [comment audit] Simplify mobile host-list and workspace-view explanations [skip ci]

* [comment audit] Clarify mobile shell lifetime and bridge delivery constraints [skip ci]

* [comment audit] Condense bridge stream, native request and audio rationale [skip ci]

* [comment audit] Reduce bridge schema and page-client commentary [skip ci]

* [comment audit] Tighten mobile page bridge contracts [skip ci]

* [comment audit] Clarify mobile cache and session constraints [skip ci]

* [comment audit] Condense mobile route and lifecycle rationale [skip ci]

* [comment audit] Tighten mobile shell hook timing rationale [skip ci]

* [comment audit] Clarify route handoff and notification compatibility [skip ci]

* [comment audit] Clarify push registration and dictation cleanup constraints [skip ci]

* [comment audit] Condense keyboard geometry and native media ownership rationale [skip ci]

* [comment audit] Tighten mobile chat scroll and composer concurrency rationale [skip ci]

* [comment audit] Condense mobile reply compatibility and upload ownership rationale [skip ci]

* [comment audit] Condense optimistic echo and scroll ownership rationale [skip ci]

* [comment audit] Tighten stream replay and queued session ownership rationale [skip ci]

* [comment audit] Condense mobile composer and reply-schema constraints [skip ci]

* [comment audit] Clarify mobile transcript recovery and session ownership [skip ci]

* [comment audit] Clarify mobile queue and source-control compatibility [skip ci]

* [comment audit] Condense storage and task launch constraints [skip ci]

* [comment audit] Clarify task schemas and terminal document ownership [skip ci]

* [comment audit] Tighten terminal input and viewport rationale [skip ci]

* [comment audit] Shorten terminal and RPC recorder contracts [skip ci]

* [comment audit] Condense recorder engine and connection contracts [skip ci]

* [comment audit] Shorten host credential and relay recovery comments [skip ci]

* [comment audit] Tighten pairing recovery and bundle transport rationale [skip ci]

* [comment audit] Clarify RPC delivery and workspace catalog boundaries [skip ci]

* [comment audit] Condense catalog and relay entry contracts [skip ci]

* [comment audit] Shorten shutdown and SSH relay delivery contracts [skip ci]

* [comment audit] Condense WSL relay and E2E regression rationale [skip ci]

* [comment audit] Shorten cross-version harness and renderer fixture explanations [skip ci]

* [comment audit] Condense paired-host and E2E lifecycle rationale [skip ci]

* [comment audit] Shorten layout oracle and regression explanations [skip ci]

* [comment audit] Condense paired recovery and ownership regression contracts [skip ci]

* [comment audit] Shorten restart, sidebar and SSH lifecycle contracts [skip ci]

* [comment audit] Condense terminal rendering and IME regression rationale [skip ci]

* [comment audit] Shorten terminal recovery and benchmark contracts [skip ci]

* [comment audit] Condense tooling and Windows lifecycle proof rationale [skip ci]

* [comment audit] Condense workflow routing and release rationale [skip ci]

* [comment audit] Condense mobile security and installer contracts [skip ci]

* [comment audit] Shorten native rationale and stylesheet workarounds [skip ci]

* [comment audit] Shorten Windows visibility probe rationale [skip ci]

* [comment audit] Condense embedded release and signing rationale [skip ci]

* [comment audit] Retain current main test behavior in final reconciliation [skip ci]

* [comment audit] Complete final comment review across current source inventory

* [comment audit] Handle consolidated audit reports before final CI

* [comment audit] Preserve credential storage census detail [skip ci]

* [comment audit] Scale broad changed E2E selections with verified timing shards

* [comment audit] Align E2E assertions with browser close and settled session metadata

* [comment audit] Match existing E2Es to current resource and windowing contracts

* [comment audit] Observe the shared server outage notice in reconnect coverage

* [comment audit] Make existing renderer probes measure settled tab and PTY work

* [comment audit] Align palette E2E selector with task search

* [comment audit] Refresh remaining palette search selectors

* [comment audit] Keep selector correction within selected audit specs

* [comment audit] Await emoji fixture settings inside renderer evaluation

* [comment audit] Cancel fixture fit retries between terminal tests

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
2026-10-25 04:05:46 -07:00

742 lines
33 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
name: E2E
run-name: E2E ${{ inputs.ref || github.ref }}
# Why: checkout + artifact upload only; callers can only further restrict.
permissions:
contents: read
on:
workflow_call:
inputs:
ref:
description: Ref to check out (defaults to the calling workflow's ref)
required: false
type: string
test_files:
description: JSON array of changed specs; empty runs the full suite
required: false
type: string
run_changed_e2e:
description: PR detector found specs requiring the general changed-spec lane.
required: false
type: boolean
default: true
needs_build:
description: PR detector found an Electron consumer requiring build and native cache.
required: false
type: boolean
default: true
managed_ssh_shards:
description: JSON shard assignments for the selected managed SSH specs
required: false
type: string
default: '["1/2","2/2"]'
# Why: artifact names are unique per workflow run, and pr.yml calls this workflow twice.
artifact_suffix:
description: Appended to the build and trace artifact names so two calls in one run do not collide
required: false
type: string
default: ''
# Why: the app, orcad and its test template (--dark-test-template) build with it; releases refuse it.
layout_owner:
description: Set to main to build with ORCA_LAYOUT_OWNER=main and record save-divergence reports
required: false
type: string
default: ''
workflow_dispatch:
inputs:
ref:
description: Ref to check out (defaults to the workflow ref)
required: false
type: string
test_files:
description: JSON array of specs to run; empty runs the full suite
required: false
type: string
managed_ssh_shards:
description: Managed SSH shard assignments (use one for small selections)
required: false
type: choice
default: '["1/2","2/2"]'
options: ['["1/2","2/2"]', '["1/1"]']
schedule:
# One complete daily reference run; targeted PR coverage remains unchanged.
- cron: '0 17 * * *'
jobs:
build:
name: build e2e app
if: inputs.test_files == '' || github.event_name != 'pull_request' || inputs.needs_build
runs-on: blacksmith-8vcpu-ubuntu-2404
timeout-minutes: 15
outputs:
changed_e2e_shards: ${{ steps.changed-e2e-shards.outputs.shards }}
env:
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Plan changed E2E allocation
id: changed-e2e-shards
env:
TEST_FILES_JSON: ${{ inputs.test_files }}
run: |
node --input-type=module <<'NODE'
import { appendFileSync, existsSync } from 'node:fs'
const modulePath = './config/scripts/ci-e2e-job-selection.mjs'
const selection = existsSync(modulePath) ? await import(modulePath) : {}
const shards = selection.generalE2eShards?.(process.env.TEST_FILES_JSON) ?? ['1/1']
appendFileSync(process.env.GITHUB_OUTPUT, `shards=${JSON.stringify(shards)}\n`)
NODE
- name: Install native build tools
env:
ORCA_E2E_APT_PACKAGES: build-essential python3
run: &install_e2e_tools |
if [ "$ORCA_E2E_APT_PACKAGES" = 'build-essential python3' ] &&
command -v make >/dev/null && command -v g++ >/dev/null && command -v python3 >/dev/null; then
exit 0
fi
read -r -a packages <<< "$ORCA_E2E_APT_PACKAGES"
if [ -f .github/scripts/prepare-ubuntu-apt.sh ]; then
bash .github/scripts/prepare-ubuntu-apt.sh
else
# Dispatched refs can predate the shared preparation script.
for source in /etc/apt/apt-mirrors.txt /etc/apt/sources.list /etc/apt/sources.list.d/*.list /etc/apt/sources.list.d/*.sources; do
if [ -f "$source" ]; then
sudo sed -i 's|https*://azure\.archive\.ubuntu\.com/ubuntu|https://archive.ubuntu.com/ubuntu|g' "$source"
fi
done
sudo tee /etc/apt/apt.conf.d/99-orca-ci >/dev/null <<'APTCONF'
Acquire::http::Timeout "15";
Acquire::https::Timeout "15";
Acquire::Retries "1";
APTCONF
fi
timeout 120 sudo apt-get update
timeout 300 sudo apt-get install -y "${packages[@]}" || timeout 300 sudo apt-get install -y --fix-missing "${packages[@]}"
# Why: the build's plain-Node daemon smoke load resolves node-pty.
- uses: ./.github/actions/install-node-dependencies
id: deps
with:
native-runtime: node
persist-native-cache: 'false'
cache-electron-package: 'true'
# Save Node's ABI before the Electron preparation replaces the same build paths.
- name: Save compiled Node native modules
if: steps.deps.outputs.native-cache-key != '' && steps.deps.outputs.native-cache-hit != 'true'
uses: actions/cache/save@v5
with:
path: ${{ steps.deps.outputs.native-cache-path }}
key: ${{ steps.deps.outputs.native-cache-key }}
# Why: building here avoids parallel builds inside Playwright globalSetup;
# paired-browser specs also need the standalone web bundle.
- name: Build E2E outputs
env:
VITE_EXPOSE_STORE: 'true'
ORCA_CI_MAIN_SOURCEMAPS: '0'
LAYOUT_OWNER: ${{ inputs.layout_owner }}
run: |
if [ -n "$LAYOUT_OWNER" ]; then
export ORCA_LAYOUT_OWNER="$LAYOUT_OWNER"
fi
pnpm run build:electron-vite:parallel --mode e2e
# The CLI emits into out/main too; start only after Electron finishes clearing that tree.
- name: Build shared E2E CLI
id: e2e-cli
background: true
run: |
if node -e 'process.exit(require("./package.json").scripts["prepare:cli-output"] ? 0 : 1)'; then
pnpm run build:cli
else
echo "Older ref: let each consumer build and install its CLI."
fi
- name: Project shared E2E web client
run: pnpm run build:web-from-renderer
- wait: e2e-cli
# The switch-on specs skip, rather than fail, when the build left no marker.
- name: Check layout-owner build marker
if: inputs.layout_owner == 'main'
run: test -f out/main/ORCA_LAYOUT_OWNER_MAIN
- name: Check shared native runtime action
id: native-runtime-action
run: |
if [ -f .github/actions/prepare-native-runtime/action.yml ]; then
echo 'available=true' >> "$GITHUB_OUTPUT"
else
echo 'available=false' >> "$GITHUB_OUTPUT"
fi
# The orcad builds downstream refuse to run without the page; older refs predate the action.
- uses: ./.github/actions/build-mobile-web-bundle
if: hashFiles('.github/actions/build-mobile-web-bundle/action.yml') != ''
# Foreground: a background JavaScript action reads the event payload while later steps rewrite it.
- name: Upload E2E build output
uses: actions/upload-artifact@v7
with:
name: e2e-build-out${{ inputs.artifact_suffix }}
path: out/
include-hidden-files: true
retention-days: 1
if-no-files-found: error
- name: Prepare shared Electron native runtime
id: electron-native-runtime
if: steps.native-runtime-action.outputs.available == 'true' && steps.deps.outputs.node-version != ''
uses: ./.github/actions/prepare-native-runtime
with:
native-runtime: electron
node-version: ${{ steps.deps.outputs.node-version }}
- name: Prepare Electron natives for older refs
if: steps.native-runtime-action.outputs.available != 'true' || steps.deps.outputs.node-version == ''
uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
e2e:
name: e2e ${{ matrix.shard_name }}
needs: [build]
if: inputs.test_files == ''
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
# Use 14 shards: scheduled average 24.6min and repeated 30min caps remained even in a 12-way trial.
- shard: '1/14'
shard_name: 1-of-14
- shard: '2/14'
shard_name: 2-of-14
- shard: '3/14'
shard_name: 3-of-14
- shard: '4/14'
shard_name: 4-of-14
- shard: '5/14'
shard_name: 5-of-14
- shard: '6/14'
shard_name: 6-of-14
- shard: '7/14'
shard_name: 7-of-14
- shard: '8/14'
shard_name: 8-of-14
- shard: '9/14'
shard_name: 9-of-14
- shard: '10/14'
shard_name: 10-of-14
- shard: '11/14'
shard_name: 11-of-14
- shard: '12/14'
shard_name: 12-of-14
- shard: '13/14'
shard_name: 13-of-14
- shard: '14/14'
shard_name: 14-of-14
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
# Native cache misses need the compiler, Electron needs Xvfb, and paired
# Quick Open needs ripgrep. Install them in one apt transaction per shard.
- name: Install native build and headless UI tools
# The Azure archive took 16 minutes for one font package; bound setup
# separately so a slow mirror cannot consume the shard's test budget.
env: &e2e_tool_packages
ORCA_E2E_APT_PACKAGES: build-essential fonts-noto-cjk openssh-client python3 ripgrep xvfb zsh openbox x11-utils xdotool
run: *install_e2e_tools
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- name: Download E2E build output
uses: actions/download-artifact@v8
with:
name: e2e-build-out${{ inputs.artifact_suffix }}
path: out/
# Shard one worker per VM to avoid Electron/git/Xvfb contention. Reuse one SKIP_BUILD artifact and forward app logs for pre-window failures.
- name: Balance E2E shard from timing evidence
env:
ORCA_BACKGROUND_LAUNCH: '1'
SKIP_BUILD: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
ORCA_E2E_WEB_CLIENT: '1'
run: |
mkdir -p ci-shards
pnpm exec playwright test --config tests/playwright.config.ts --project=electron-headless --list --reporter=json > ci-shards/discovery.json
export ORCA_SHARD_SOURCE_SHA="$(git rev-parse HEAD)"
node config/scripts/ci-e2e-shard-plan.mjs ci-shards/discovery.json '${{ matrix.shard }}' ci-shards
pnpm exec playwright test --config tests/playwright.config.ts --project=electron-headless --test-list=ci-shards/selected.txt --list --reporter=json > ci-shards/selected-discovery.json
node config/scripts/ci-e2e-shard-plan.mjs --verify ci-shards/assignment.json ci-shards/selected-discovery.json
- name: Run E2E tests (${{ matrix.shard_name }})
env:
PLAYWRIGHT_JSON_OUTPUT_FILE: ci-shards/results.json
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 ORCA_E2E_WEB_CLIENT=1 pnpm run test:e2e --test-list=ci-shards/selected.txt --reporter=list,json
- name: Summarize E2E failures
if: always()
continue-on-error: true
run: node config/scripts/ci-e2e-failure-summary.mjs ci-shards/results.json
- name: Upload E2E shard assignment
if: always()
# Diagnostic upload outages must not change the test verdict.
continue-on-error: true
uses: actions/upload-artifact@v7
with:
name: e2e-shard-${{ matrix.shard_name }}-attempt-${{ github.run_attempt }}
path: ci-shards/
retention-days: 14
if-no-files-found: warn
# The frame benchmark needs a mapped window, which the headless shards exclude.
- name: Run worktree first-paint benchmark
if: matrix.shard == '1/14'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 pnpm exec playwright test tests/e2e/worktree-switch-first-paint.spec.ts --config tests/playwright.config.ts --project=electron-headful --workers=1
# Upload failure traces/screenshots so CI failures are diagnosable without reproduction.
- name: Upload Playwright traces
if: failure() || cancelled()
uses: actions/upload-artifact@v7
with:
name: playwright-traces-${{ matrix.shard_name }}
path: test-results/
retention-days: 7
if-no-files-found: ignore
changed-e2e:
name: changed e2e specs (${{ matrix.shard }})
needs: [build]
if: inputs.test_files != '' && (github.event_name != 'pull_request' || inputs.run_changed_e2e)
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
shard: ${{ fromJSON(needs.build.outputs.changed_e2e_shards || '["1/1"]') }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- name: Install native build and headless UI tools
# Install rg for bounded Quick Open searches exercised by the paired fixture.
env: *e2e_tool_packages
run: *install_e2e_tools
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- name: Download E2E build output
uses: actions/download-artifact@v8
with:
name: e2e-build-out${{ inputs.artifact_suffix }}
path: out/
- name: Run changed E2E specs
env:
TEST_FILES_JSON: ${{ inputs.test_files }}
LAYOUT_OWNER: ${{ inputs.layout_owner }}
SHARD: ${{ matrix.shard }}
run: |
if [ -f config/scripts/ci-e2e-job-selection.mjs ]; then
printf '%s\n' "$TEST_FILES_JSON" | node config/scripts/ci-e2e-job-selection.mjs > "$RUNNER_TEMP/general-e2e-specs"
else
# Dispatching an older ref retains its existing dedicated-owner exclusions.
jq -r '.[] | select(
. != "tests/e2e/ssh-browser-network-execution-route.docker.unit.test.ts" and
. != "tests/e2e/terminal-ibus-hangul-native.spec.ts" and
. != "tests/e2e/orcad-serve-mode-switch.spec.ts" and
. != "tests/e2e/workspace-layout-oracle-headless.spec.ts" and
. != "tests/e2e/windows-missing-appdata-startup.spec.ts" and
. != "tests/e2e/file-explorer-copy-file-windows.spec.ts" and
(startswith("tests/e2e/ssh-orcad-") | not)
)' <<<"$TEST_FILES_JSON" > "$RUNNER_TEMP/general-e2e-specs"
fi
mapfile -t TEST_FILES < "$RUNNER_TEMP/general-e2e-specs"
if [ "${#TEST_FILES[@]}" -eq 0 ]; then
echo "Changed specs are all owned by dedicated lanes."
exit 0
fi
E2E_ENV=(SKIP_BUILD=1 ORCA_E2E_FORWARD_APP_LOGS=1 ORCA_E2E_WEB_CLIENT=1)
E2E_PROJECT_ARGS=()
if [ -n "$LAYOUT_OWNER" ]; then
test -f out/main/ORCA_LAYOUT_OWNER_MAIN
mkdir -p layout-divergence
E2E_ENV+=(ORCA_LAYOUT_SAVE_DIVERGENCE_LOG="$PWD/layout-divergence/window-specs.jsonl")
E2E_ENV+=(PLAYWRIGHT_JSON_OUTPUT_FILE="$PWD/layout-divergence/window-specs-results.json")
E2E_PROJECT_ARGS+=('--reporter=list,json')
fi
if grep -l '@headful' "${TEST_FILES[@]}" >/dev/null; then
E2E_PROJECT_ARGS+=(--project=electron-headful)
fi
E2E_SELECTION_ARGS=(--shard="$SHARD")
if [ -f config/scripts/ci-e2e-shard-plan.mjs ]; then
mkdir -p ci-shards-changed
pnpm exec playwright test "${TEST_FILES[@]}" --config tests/playwright.config.ts --project=electron-headless "${E2E_PROJECT_ARGS[@]}" --list --reporter=json > ci-shards-changed/discovery.json
export ORCA_SHARD_SOURCE_SHA="$(git rev-parse HEAD)"
node config/scripts/ci-e2e-shard-plan.mjs ci-shards-changed/discovery.json "$SHARD" ci-shards-changed
pnpm exec playwright test "${TEST_FILES[@]}" --config tests/playwright.config.ts --project=electron-headless "${E2E_PROJECT_ARGS[@]}" --test-list=ci-shards-changed/selected.txt --list --reporter=json > ci-shards-changed/selected-discovery.json
node config/scripts/ci-e2e-shard-plan.mjs --verify ci-shards-changed/assignment.json ci-shards-changed/selected-discovery.json
E2E_SELECTION_ARGS=(--test-list=ci-shards-changed/selected.txt)
fi
xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh env "${E2E_ENV[@]}" \
pnpm run test:e2e "${TEST_FILES[@]}" "${E2E_SELECTION_ARGS[@]}" --workers=1 "${E2E_PROJECT_ARGS[@]}"
- name: Upload changed E2E shard assignment
if: always()
continue-on-error: true
uses: actions/upload-artifact@v7
with:
name: changed-e2e-shard-${{ strategy.job-index }}${{ inputs.artifact_suffix }}-attempt-${{ github.run_attempt }}
path: ci-shards-changed/
retention-days: 14
if-no-files-found: warn
- name: Summarize layout save divergence
if: always() && inputs.layout_owner == 'main'
continue-on-error: true
run: &summarize_layout_divergence |
# Either file names a run: a log with no report is unattributed, a report with no log is zero.
shopt -s nullglob
runs=(layout-divergence/*.jsonl layout-divergence/*-results.json)
for run in $(printf '%s\n' "${runs[@]%.jsonl}" | sed 's/-results\.json$//' | sort -u); do
node config/scripts/layout-save-divergence-summary.mjs "$run.jsonl" "$run-summary.json" "$run-results.json"
if [ -f "$run-results.json" ]; then
node config/scripts/ci-e2e-failure-summary.mjs "$run-results.json"
fi
done
- name: Upload layout save divergence report
if: always() && inputs.layout_owner == 'main'
continue-on-error: true
uses: actions/upload-artifact@v7
with:
name: layout-divergence-changed-${{ strategy.job-index }}${{ inputs.artifact_suffix }}-attempt-${{ github.run_attempt }}
path: layout-divergence/
retention-days: 14
if-no-files-found: warn
- name: Upload Playwright traces
if: failure() || cancelled()
uses: actions/upload-artifact@v7
with:
name: playwright-traces-changed-${{ strategy.job-index }}${{ inputs.artifact_suffix }}
path: test-results/
retention-days: 7
if-no-files-found: ignore
ssh-browser-network-route:
name: ssh browser network route
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/ssh-browser-network-execution-route.docker.unit.test.ts')
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 15
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: node
- name: Install SSH client
run: sudo apt-get update && sudo apt-get install -y openssh-client
- name: Run Docker SSH browser network route journeys
env:
ORCA_BACKGROUND_LAUNCH: '1'
ORCA_RUN_DOCKER_SSH_BROWSER_E2E: '1'
run: node_modules/.bin/vitest run --config config/vitest.config.ts tests/e2e/ssh-browser-network-execution-route.docker.unit.test.ts
orcad-serve-mode-switch:
name: orca serve Electron/orcad mode switch (D7)
needs: [build]
if: >-
inputs.test_files == '' ||
contains(inputs.test_files, 'tests/e2e/orcad-serve-mode-switch.spec.ts') ||
contains(inputs.test_files, 'tests/e2e/workspace-layout-oracle-headless.spec.ts')
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 45
env:
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
persist-credentials: false
- name: Install headless tools
run: sudo apt-get update && sudo apt-get install -y build-essential ripgrep xvfb openbox x11-utils
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- uses: actions/download-artifact@v8
with:
name: e2e-build-out${{ inputs.artifact_suffix }}
path: out/
# The packaged orcad slot the T6-11 launcher runs: its own node-pty under the pinned Node.
# The template is what `orca serve`'s default selection materializes that slot from.
- name: Build this runner's orcad slot and template
env:
LAYOUT_OWNER: ${{ inputs.layout_owner }}
run: |
slot="$(node config/scripts/build-orcad-prebuilds.mjs --print-slot)"
pnpm build:orcad-prebuilds
pnpm build:orcad-prebuilds --require-slots "$slot"
# The orcad host and its materialized template must use the Electron host’s layout owner.
if [ -n "$LAYOUT_OWNER" ]; then
export ORCA_LAYOUT_OWNER="$LAYOUT_OWNER"
pnpm build:orcad
node config/scripts/build-orcad-template.mjs --targets "$slot" --dark-test-template
test -f out/orcad/ORCA_LAYOUT_OWNER_MAIN
test -f out/orcad-template/ORCA_LAYOUT_OWNER_MAIN
else
pnpm build:orcad
node config/scripts/build-orcad-template.mjs --targets "$slot"
fi
- name: Switch serve hosts on one profile
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/orcad-serve-mode-switch.spec.ts')
env:
SKIP_BUILD: '1'
ORCA_E2E_ORCAD_SERVE: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/orcad-serve-mode-switch.spec.ts --project=electron-headless --workers=1
# Only this job builds out/orcad, so the spec's orcad case runs here; the e2e shards run its Electron case.
- name: Phone Markdown on headless orcad serve
if: >-
!cancelled() &&
(inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/headless-serve-mobile-markdown.spec.ts'))
env:
SKIP_BUILD: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
run: xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/headless-serve-mobile-markdown.spec.ts --grep "headless orcad server" --project=electron-headless --workers=1
# The layout oracle's headless scenarios run on both serve hosts, so they need this job's orcad.
- name: Layout oracle on headless serve (orcad and Electron)
if: >-
!cancelled() &&
(inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/workspace-layout-oracle-headless.spec.ts'))
env:
SKIP_BUILD: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
LAYOUT_OWNER: ${{ inputs.layout_owner }}
run: |
REPORT_ARGS=()
if [ -n "$LAYOUT_OWNER" ]; then
test -f out/main/ORCA_LAYOUT_OWNER_MAIN
mkdir -p layout-divergence
export ORCA_LAYOUT_SAVE_DIVERGENCE_LOG="$PWD/layout-divergence/headless-oracle.jsonl"
export PLAYWRIGHT_JSON_OUTPUT_FILE="$PWD/layout-divergence/headless-oracle-results.json"
REPORT_ARGS=('--reporter=list,json')
fi
xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/workspace-layout-oracle-headless.spec.ts --project=electron-headless --workers=1 "${REPORT_ARGS[@]}"
- name: Summarize layout save divergence
if: always() && inputs.layout_owner == 'main'
continue-on-error: true
run: *summarize_layout_divergence
- name: Upload layout save divergence report
if: always() && inputs.layout_owner == 'main'
continue-on-error: true
uses: actions/upload-artifact@v7
with:
name: layout-divergence-headless${{ inputs.artifact_suffix }}-attempt-${{ github.run_attempt }}
path: layout-divergence/
retention-days: 14
if-no-files-found: warn
- uses: actions/upload-artifact@v7
if: failure()
with:
name: orcad-serve-mode-switch-traces${{ inputs.artifact_suffix }}
path: test-results/
retention-days: 7
if-no-files-found: ignore
# Selected managed-orcad SSH specs on a Docker host; daily runs keep the full suite. Needs the
# orcad template for the fixture's target (Debian, linux-x64-glibc), which only this job builds.
orcad-ssh-docker:
name: ssh on managed orcad (Docker ${{ matrix.shard }})
needs: [build]
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/ssh-orcad-')
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
shard: ${{ fromJSON(inputs.managed_ssh_shards || '["1/2","2/2"]') }}
env:
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
persist-credentials: false
# COMPAT(legacy-relay-bridge): the bridge spec builds 1.4.224's relay from its tagged sources.
- name: Fetch the 1.4.224 relay sources
run: git fetch --no-tags --depth=1 origin +refs/tags/v1.4.224:refs/tags/v1.4.224
- name: Install headless tools
run: sudo apt-get update && sudo apt-get install -y build-essential ripgrep xvfb openbox x11-utils
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- uses: actions/download-artifact@v8
with:
name: e2e-build-out${{ inputs.artifact_suffix }}
path: out/
# Built to a path the app does not look at by default, so the relay-era phase has no template.
# The switch-on leg deploys a switch-on server, so both sides of the host run one layout owner.
- name: Build the linux-x64-glibc orcad template
env:
LAYOUT_OWNER: ${{ inputs.layout_owner }}
run: |
pnpm build:orcad-prebuilds
pnpm build:orcad-prebuilds --require-slots linux-x64-glibc
if [ -n "$LAYOUT_OWNER" ]; then
ORCA_LAYOUT_OWNER="$LAYOUT_OWNER" node config/scripts/build-orcad-template.mjs --targets linux-x64-glibc --dark-test-template
test -f out/orcad-template/ORCA_LAYOUT_OWNER_MAIN
else
node config/scripts/build-orcad-template.mjs --targets linux-x64-glibc
fi
mv out/orcad-template "$RUNNER_TEMP/orcad-convert-template"
# The Windows CLI matrix collects here too and skips: it needs a Windows host descriptor.
- name: Run managed SSH specs on a Docker host
env:
TEST_FILES_JSON: ${{ inputs.test_files }}
SKIP_BUILD: '1'
ORCA_E2E_SSH_DOCKER: '1'
ORCA_E2E_ORCAD_CONVERT_HOST: docker
ORCA_E2E_FORWARD_APP_LOGS: '1'
SHARD: ${{ matrix.shard }}
LAYOUT_OWNER: ${{ inputs.layout_owner }}
SHARD_INDEX: ${{ strategy.job-index }}
run: |
node --input-type=module <<'NODE' > "$RUNNER_TEMP/managed-ssh-specs"
import { existsSync } from 'node:fs'
const modulePath = './config/scripts/ci-e2e-job-selection.mjs'
const selection = existsSync(modulePath) ? await import(modulePath) : {}
// Older checkout refs retain their full-suite behavior.
const specs = selection.selectManagedSshE2eSpecs
? selection.selectManagedSshE2eSpecs(process.env.TEST_FILES_JSON)
: ['tests/e2e/ssh-orcad-']
for (const spec of specs) {
console.log(spec.endsWith('.spec.ts') ? spec.replace(/[.*+?^${}()|[\]\\]/g, '\\$&') + '$' : spec)
}
NODE
mapfile -t TEST_FILES < "$RUNNER_TEMP/managed-ssh-specs"
export ORCA_E2E_ORCAD_CONVERT_TEMPLATE="$RUNNER_TEMP/orcad-convert-template"
REPORT_ARGS=()
if [ -n "$LAYOUT_OWNER" ]; then
test -f out/main/ORCA_LAYOUT_OWNER_MAIN
mkdir -p layout-divergence
export ORCA_LAYOUT_SAVE_DIVERGENCE_LOG="$PWD/layout-divergence/managed-ssh-$SHARD_INDEX.jsonl"
export PLAYWRIGHT_JSON_OUTPUT_FILE="$PWD/layout-divergence/managed-ssh-$SHARD_INDEX-results.json"
REPORT_ARGS=('--reporter=list,json')
fi
xvfb-run --auto-servernum bash .github/scripts/e2e-with-window-manager.sh pnpm exec playwright test --config tests/playwright.config.ts "${TEST_FILES[@]}" --project=electron-headless --workers=1 --shard="$SHARD" "${REPORT_ARGS[@]}"
- name: Summarize layout save divergence
if: always() && inputs.layout_owner == 'main'
continue-on-error: true
run: *summarize_layout_divergence
- name: Upload layout save divergence report
if: always() && inputs.layout_owner == 'main'
continue-on-error: true
uses: actions/upload-artifact@v7
with:
name: layout-divergence-managed-ssh-${{ strategy.job-index }}${{ inputs.artifact_suffix }}-attempt-${{ github.run_attempt }}
path: layout-divergence/
retention-days: 14
if-no-files-found: warn
- uses: actions/upload-artifact@v7
if: failure()
with:
name: orcad-ssh-docker-traces-${{ strategy.job-index }}${{ inputs.artifact_suffix }}
path: test-results/
retention-days: 7
if-no-files-found: ignore
# D7 on Windows: both hosts share <userData>\daemon and so one daemon pipe. Built here rather than
# from the Linux e2e build because the slot, template and addons are Windows-native. Also runs the
# missing-AppData startup check and the explorer file-copy check, which need the same Windows e2e build.
orcad-serve-mode-switch-windows:
name: orca serve on Windows (D7 mode switch, missing AppData)
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/orcad-serve-mode-switch.spec.ts') || contains(inputs.test_files, 'tests/e2e/windows-missing-appdata-startup.spec.ts') || contains(inputs.test_files, 'tests/e2e/file-explorer-copy-file-windows.spec.ts')
runs-on: windows-2022
timeout-minutes: 45
env:
NODE_OPTIONS: --max-old-space-size=4096
ORCA_BACKGROUND_LAUNCH: '1'
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
persist-credentials: false
- uses: ./.github/actions/install-node-dependencies
with:
native-runtime: electron
- uses: ./.github/actions/prepare-orcad-prebuilds
with:
resolve-windows-cache: 'true'
restore-windows-cache: 'true'
- uses: ./.github/actions/prepare-windows-cli-launcher
with:
arches: x64
- uses: ./.github/actions/build-mobile-web-bundle
- name: Build the e2e app, CLI, orcad slot and template
shell: bash
env:
ORCA_REUSE_PREPARED_WINDOWS_SECURITY: 'true'
run: |
pnpm run build:wsl-guest
pnpm exec electron-vite build --mode e2e
pnpm run build:cli
pnpm build:orcad
node config/scripts/build-windows-process-tree-addon.mjs --arch=x64 --reuse-prepared-runtime
node config/scripts/build-orcad-template.mjs --targets win32-x64
# A profile-less Windows session has no roaming AppData; Electron 43 used to crash natively there.
- name: Start serve with no AppData folder
if: inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/windows-missing-appdata-startup.spec.ts')
env:
SKIP_BUILD: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
run: pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/windows-missing-appdata-startup.spec.ts --project=electron-headless --workers=1
# Explorer pastes only a real CF_HDROP, which needs the Windows clipboard and shell.
- name: Copy a file from the explorer
if: ${{ !cancelled() && (inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/file-explorer-copy-file-windows.spec.ts')) }}
env:
SKIP_BUILD: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
run: pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/file-explorer-copy-file-windows.spec.ts --project=electron-headless --workers=1
- name: Switch serve hosts on one profile
if: ${{ !cancelled() && (inputs.test_files == '' || contains(inputs.test_files, 'tests/e2e/orcad-serve-mode-switch.spec.ts')) }}
env:
SKIP_BUILD: '1'
ORCA_E2E_ORCAD_SERVE: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
ORCA_E2E_PRESERVE_PROFILE_LOGS_DIR: ${{ github.workspace }}\test-results\profile-logs
run: pnpm exec playwright test --config tests/playwright.config.ts tests/e2e/orcad-serve-mode-switch.spec.ts --project=electron-headless --workers=1
- uses: actions/upload-artifact@v7
if: failure()
with:
name: orcad-serve-mode-switch-windows-traces${{ inputs.artifact_suffix }}
path: test-results/
retention-days: 7
if-no-files-found: ignore