mirror of
https://github.com/stablyai/orca.git
synced 2026-10-30 16:47:34 +00:00
Jinwoo Hong
7f0e067256
* test(terminal): check the runtime's workspace layout against structural rules A pure rules check over the runtime's session partitions: a terminal in two panes, a pane in two tabs, a tab outside exactly one group, tab rows and tab bar disagreeing, two tab orders, and ids changing for the same pane, tab or group across an update. Wired into the seeded layout model test after every step, plus late-spawn cases (one known on main, kept as an expected failure). * test(e2e): layout oracle scenarios with window, headless serve and SSH Drives layout journeys through window gestures, CLI and paired-client runtime commands, and after every step checks the runtime's layout against the structural rules, the window's DOM, the paired-client and CLI views, the expected end layout, restarts (relaunch, cold daemon, in-app daemon restart) and per-pane terminal markers. Findings main still has are listed with their cause; anything else fails. Includes headless orcad/Electron serve scenarios for the serve issues and a Docker SSH scenario. * test(terminal): check layout rules on every write and race client commands with the window The layout model now checks the structural rules on every in-memory workspace-session write, not only at each step's end, naming the writer of any breach. Three documented two-writer transients (main mints a tab row before the window writes its tab bar) are allow-listed by rule, op and writer; they must still heal by the step's end. New model ops race a stale window save against a paired client's tab close, a host-created split, and main's PTY-exit retirement. A new file runs main's updateMobileSessionPaneLayout (old phone pane trees) against the real Store; the case with no topology revision is an it.fails: main drops a live pane and its binding there. * test(e2e): oracle expected outcomes, transient sampling, remount counts, more scenarios, latency Every step checks the expected panes, which panes went away and that the rest kept their terminals; a 100 ms sampler runs the rules between steps; xterm creations are counted per pane and an unexpected recreation fails. Adds host navigation, window reload (graph republication), quit right after a change (quit-time save), an old client's stale pane tree, and a latency measurement (command to drawn update, idle and under an output flood; no pass/fail). * docs(e2e): describe the oracle's added checks and latency measurement * test(e2e): #12723 SSH folder scenario, CI lane for the SSH oracle spec, short socket dir Adds an SSH folder-workspace (no repo) scenario that checks the layout lives in one partition across relaunch; registers the SSH oracle spec in the Docker SSH lane's selection lists; headless hosts put long-TMPDIR profiles under ~/.orca-lo instead of /tmp. * ci(e2e): run the headless layout oracle in the job that builds orcad; Linux relaunch findings The headless oracle spec needs out/orcad, which only the orca serve mode-switch job builds, so it runs there (and fails rather than skips in CI if orcad is missing). Records what the first Linux CI run found on main: after a relaunch main can hold tab rows with no tab bar past the settle wait, and Electron serve on Linux also lists a renamed terminal's title as null after a restart. * test(e2e): join partition lists in the SSH folder scenario's log lines * test(e2e): retry the latency spec's reads through a brief terminal-ownership gap Linux CI saw terminal.list answer terminal_surface_ownership_unavailable while the window re-published its terminals; reads retry for up to 10 s, commands do not.
99 lines
4.3 KiB
JavaScript
99 lines
4.3 KiB
JavaScript
import { spawnSync } from 'node:child_process'
|
|
|
|
const rawExtraArgs = process.argv.slice(2)
|
|
const extraArgs = rawExtraArgs[0] === '--' ? rawExtraArgs.slice(1) : rawExtraArgs
|
|
const pnpm = process.platform === 'win32' ? 'pnpm.cmd' : 'pnpm'
|
|
const env = {
|
|
...process.env,
|
|
ORCA_E2E_SSH_DOCKER: '1',
|
|
ORCA_E2E_LOCAL_SSH_BROWSER: '1',
|
|
ORCA_E2E_SSH_CLIENT_HOSTED_BROWSER: '1',
|
|
ORCA_E2E_WEB_CLIENT: '1'
|
|
}
|
|
|
|
// Why: Node's CVE-2024-27980 hardening rejects .cmd spawns without shell on Windows.
|
|
const spawnOptions = {
|
|
stdio: 'inherit',
|
|
env,
|
|
shell: process.platform === 'win32'
|
|
}
|
|
|
|
const runtime = spawnSync(pnpm, ['run', 'ensure:electron-runtime'], spawnOptions)
|
|
|
|
if (runtime.status !== 0) {
|
|
process.exit(runtime.status ?? 1)
|
|
}
|
|
|
|
// Why one explicit list: these specs self-skip without ORCA_E2E_SSH_DOCKER and no sharded lane
|
|
// sets it, so a spec in no runner runs nowhere. The gate contract proves every flag-reading
|
|
// spec is claimed here, by the watcher-isolation or parking runner, or by a listed exclusion.
|
|
//
|
|
// Deliberately absent, and therefore still covered by no CI trigger:
|
|
// ssh-docker-relay-perf.spec.ts — wall-clock latency thresholds; flaky budgets here would
|
|
// cost the lane its credibility. NOTE: a runner script test:e2e:ssh-docker-perf exists in
|
|
// package.json but NO workflow invokes it, so this spec currently runs in no CI lane at
|
|
// all. Recorded as a real gap, not as coverage living somewhere else.
|
|
// The bulk-open frame probe runs headed: headless Linux compositing schedules idle RAFs
|
|
// roughly 1s apart, so it cannot measure foreground interaction against the same budget.
|
|
//
|
|
// Why both projects: ssh-port-forward-lifecycle is @headful, which the headless project
|
|
// grep-inverts away.
|
|
//
|
|
// Known gaps in SSH e2e coverage, recorded here because nothing else names them:
|
|
// - E2E does not gate merges: `verify.needs` in pr.yml omits `e2e` while the suite is red on
|
|
// main. Nothing in this lane blocks a PR yet. pr.yml's Require-successful-checks comment
|
|
// has the exact wiring to flip it, and the gate contract asserts the current state.
|
|
// - Two specs are gated on env vars no workflow sets, so they run nowhere
|
|
// and are not Docker-gated, which puts them outside this file's contract:
|
|
// nested-runtime-ssh-lifecycle, nested-runtime-ssh-routing (ORCA_E2E_NESTED_RUNTIME_SSH)
|
|
// The nested-runtime runner remains unused by CI.
|
|
const result = spawnSync(
|
|
pnpm,
|
|
[
|
|
'exec',
|
|
'playwright',
|
|
'test',
|
|
'tests/e2e/local-ssh-browser-routing.spec.ts',
|
|
'tests/e2e/ssh-client-hosted-browser-drop-reconnect.spec.ts',
|
|
'tests/e2e/pty-input-write-queue-ssh.spec.ts',
|
|
'tests/e2e/ssh-ai-vault-session-history.spec.ts',
|
|
'tests/e2e/ssh-codex-display-artifacts-repro.spec.ts',
|
|
'tests/e2e/ssh-cold-activation-restore.spec.ts',
|
|
'tests/e2e/ssh-cold-hydration-gap-tab-seeding.spec.ts',
|
|
'tests/e2e/ssh-emptied-worktree-reactivation.spec.ts',
|
|
'tests/e2e/ssh-docker-five-pane-input-under-flood.spec.ts',
|
|
'tests/e2e/ssh-docker-bulk-open-freeze-repro.spec.ts',
|
|
'tests/e2e/ssh-docker-half-open-link.spec.ts',
|
|
'tests/e2e/ssh-docker-quick-open-large-listing.spec.ts',
|
|
'tests/e2e/ssh-docker-reconnect-pane-restore.spec.ts',
|
|
'tests/e2e/ssh-docker-relay-stall-credential.spec.ts',
|
|
'tests/e2e/ssh-docker-resource-accumulation.spec.ts',
|
|
'tests/e2e/ssh-docker-transport-drop-recovery.spec.ts',
|
|
'tests/e2e/ssh-external-image-preview.spec.ts',
|
|
'tests/e2e/ssh-lost-kill-tab-resurrection.spec.ts',
|
|
'tests/e2e/ssh-native-chat-phone-history.spec.ts',
|
|
'tests/e2e/ssh-pi-compatible-agent-title.spec.ts',
|
|
'tests/e2e/ssh-port-forward-lifecycle.spec.ts',
|
|
'tests/e2e/ssh-reattach-home-partition.spec.ts',
|
|
'tests/e2e/ssh-reconnect-tab-destruction.spec.ts',
|
|
'tests/e2e/ssh-restart-tab-accumulation.spec.ts',
|
|
'tests/e2e/ssh-skill-installation.spec.ts',
|
|
'tests/e2e/ssh-stale-resume-execution-host-scope.spec.ts',
|
|
'tests/e2e/ssh-startup-local-shadow.spec.ts',
|
|
'tests/e2e/ssh-terminal-window-wake-stale-grid-repro.spec.ts',
|
|
'tests/e2e/terminal-inline-images-ssh.spec.ts',
|
|
'tests/e2e/workspace-layout-oracle-ssh.spec.ts',
|
|
'--config',
|
|
'tests/playwright.config.ts',
|
|
'--project',
|
|
'electron-headless',
|
|
'--project',
|
|
'electron-headful',
|
|
'--workers=1',
|
|
...extraArgs
|
|
],
|
|
spawnOptions
|
|
)
|
|
|
|
process.exit(result.status ?? 1)
|