test(e2e): on the real WSL lane, background work never starts the stopped distro (E9 guard) (#35590)

* test(e2e): on the real WSL lane, background work never starts the stopped distro (E9 guard)

The Windows real-WSL lane opens a project inside its own imported distro, terminates that distro,
and runs a window of background work elsewhere (focus/blur/reveal, workspace switches, a linked
issue/PR card, Settings panes, Tasks, account switches). Every child the main process starts is
logged; no wsl.exe may run anything in the distro (listings are fine), and `wsl --list --running`,
watched each second, must never show it, which also catches UNC touches. Opening a terminal in the
WSL workspace then starts it. A participation check fails the lane if the guard skipped.

* test(e2e): actually render each Settings pane in the WSL lane's E9 background window

openSettingsTarget alone only set the navigation target, so no pane mounted and none of its reads
(servers, accounts, agents, integrations, CLI status, the repo pane's orca.yaml check) ran. Open
Settings, wait for each pane's section to mount, then close it.

* test(e2e): harden the WSL lane's E9 guard per review

- process-host logs every spawn to ORCA_E2E_SPAWN_LOG when set, so the guard sees worker-thread
  and inheriting-child spawns (e.g. the Codex rate-limit fetch), sync ones included.
- The spawn baseline is taken before `wsl --terminate`, and the running-list watcher starts right
  after it, so work in the stopping window counts.
- A wsl.exe launch counts as starting the distro unless it names another or only lists/terminates.
- The watcher is async with a 5 s timeout and non-overlapping ticks; a failed read is retried and
  fails the guard after three, never reads as "stopped". The recorder throws if it cannot hook,
  and the spawn log must be non-empty. The doc names the watcher as the authoritative assertion.

* test(e2e): bound the terminate grace, keep the spawn log out of shipped runs, tolerate torn lines

- The watcher forgives running reads only within 3 s of `--terminate` and before its first
  not-running read, which the rounds now await; any running read after it fails the guard.
- Main drops ORCA_E2E_SPAWN_LOG outside e2e mode (no ORCA_E2E_USER_DATA_DIR) before anything
  spawns, so no daemon, PTY or agent inherits it; the log holds full argv, tests only.
- readSpawnLog skips a line torn by a concurrent append.

* test(e2e): no blocking wsl.exe in the WSL lane's E9 guard, and a timeline for failures

The first lane run was cancelled at the 30-minute job limit while the guard step had run 25
minutes: past three 5-minute test timeouts, which only a blocked worker survives. Every wsl.exe
call (setup, terminate, cleanup) is now async with a timeout, so a hung one fails the test. The
guard logs a timestamped timeline (each running-list read, terminate, rounds) for diagnosis, and
the job limit goes to 50 minutes for the three added runs.

* test(e2e): let the setup's own WSL terminal launch settle before the terminate; 10 s grace

The lane run failed with "Ubuntu was still listed 3000 ms after --terminate" in all three runs.
The timeline showed why: focusing the WSL workspace in setup is a user act that opens a WSL
terminal, and its launch (codex trust probe, PTY, hook relay) landed about a second after the
terminate and restarted the distro. Setup now waits for that terminal's PTY and for wsl.exe
launches to go quiet for 3 s before terminating. The grace is two running-list reads (10 s).

* test(e2e): interleave wsl.exe launches into the E9 guard's timeline

The lane saw the distro boot about two seconds after the terminate with nothing naming the
launcher; each running-list tick now also prints the wsl.exe launches logged since the last one.

* test(e2e): time and stack every logged spawn and main's WSL UNC touches

The E9 guard printed spawns at its poll time, up to a second late, and with
no caller. Each record now carries when it happened and the caller's stack,
and main's fs touches of \\wsl.localhost paths are logged too, so a boot
with no wsl.exe launch before it can still be attributed.

* test(e2e): keep spawn-log stacks and main's fs hooks strictly to e2e mode

process-host now writes the spawn log, and captures caller stacks, only when
ORCA_E2E_USER_DATA_DIR is set as well, so a stray log variable in a CLI or
daemon environment does nothing. The main-process fs/spawn hook refuses to
install outside an e2e-mode app, and a source scan pins that no shipped code
imports or installs it.

* test(e2e): guard a focused WSL workspace left idle through the stop

Device runs saw a stopped distro restart with its bridge workspace focused
and nothing touched: the stop's own fallout re-ran renderer reads as user
acts. The new case keeps that workspace focused with a folder expanded in
the Explorer and Source Control shown, terminates, and idles. Shared guard
helpers move to tests/e2e/helpers/wsl-boot-guard.ts.

* test(e2e): capture caller stacks deep enough to reach the IPC handler

* test(e2e): give the focused-idle guard's timeout bump its reason
This commit is contained in:
Neil
2026-10-29 06:52:46 -07:00
committed by GitHub
parent 4642bc00d6
commit 2a5f6c79c1
14 changed files with 885 additions and 5 deletions
+22 -1
View File
@@ -26,7 +26,8 @@ concurrency:
jobs:
wsl-terminal:
runs-on: windows-2022
timeout-minutes: 30
# The E9 guard adds three ~2-5 minute runs to the launch, paste and image checks.
timeout-minutes: 50
env:
NODE_OPTIONS: --max-old-space-size=4096
steps:
@@ -75,6 +76,26 @@ jobs:
--output=test-results/terminal-images
--repeat-each=3
--workers=1
- name: Guard that background work never starts the stopped WSL distro
env:
SKIP_BUILD: '1'
ORCA_BACKGROUND_LAUNCH: '1'
ORCA_E2E_FORWARD_APP_LOGS: '1'
# The spec terminates this distro; it is the one this job imported.
ORCA_E2E_WSL_TERMINATE_DISTRO: Ubuntu
PLAYWRIGHT_JSON_OUTPUT_FILE: test-results/wsl-no-background-boot-results.json
run: >-
pnpm exec playwright test
tests/e2e/wsl-bridge-no-background-boot.spec.ts
--config tests/playwright.config.ts
--project=electron-headless
--output=test-results/wsl-no-background-boot
--repeat-each=3
--workers=1
--reporter=list,json
- name: Require all three E9 guard executions
if: always()
run: node config/scripts/verify-wsl-no-background-boot-participation.mjs test-results/wsl-no-background-boot-results.json
- name: Upload terminal image proof
uses: actions/upload-artifact@v7
if: always()
+1 -1
View File
@@ -416,7 +416,7 @@ export const PR_E2E_SOURCE_ROUTES = [
],
matches: (file) =>
isProductSource(file) &&
/^(?:config\/scripts\/(?:verify-wsl-e2e-participation|verify-playwright-participation)\.mjs$|src\/main\/(?:wsl[/-]|pty\/.*wsl|providers\/wsl)|src\/shared\/(?:wsl-|windows-terminal-shell|shell-argument-quoting)|src\/renderer\/src\/.*(?:terminal-paste|pty-paste)|tests\/e2e\/(?:golden-tab-bar-agent-launch\.spec|terminal-windows-shell-paste-ownership\.spec|terminal-inline-images-windows\.spec|helpers\/(?:wsl-golden-stub-agent|golden-stub-agent))|\.github\/(?:actions\/setup-wsl-test-runtime\/|workflows\/windows-wsl-e2e\.yml))/.test(
/^(?:config\/scripts\/(?:verify-wsl-e2e-participation|verify-wsl-no-background-boot-participation|verify-playwright-participation)\.mjs$|src\/main\/(?:wsl[/-]|pty\/.*wsl|providers\/wsl)|src\/shared\/(?:wsl-|windows-terminal-shell|shell-argument-quoting)|src\/renderer\/src\/.*(?:terminal-paste|pty-paste)|tests\/e2e\/(?:golden-tab-bar-agent-launch\.spec|terminal-windows-shell-paste-ownership\.spec|terminal-inline-images-windows\.spec|wsl-bridge-no-background-boot\.spec|helpers\/(?:wsl-golden-stub-agent|golden-stub-agent))|\.github\/(?:actions\/setup-wsl-test-runtime\/|workflows\/windows-wsl-e2e\.yml))/.test(
file
)
},
@@ -0,0 +1,21 @@
import { verifyPlaywrightParticipation } from './verify-playwright-participation.mjs'
import { readFileSync } from 'node:fs'
import { pathToFileURL } from 'node:url'
// A skipped E9 guard would pass silently; the lane must prove it actually ran.
export const WSL_NO_BACKGROUND_BOOT_TITLES = [
'starts nothing in the stopped distro until the user opens a terminal there',
'a focused WSL workspace left idle through the stop never starts it again'
]
export function verifyWslNoBackgroundBootParticipation(report) {
verifyPlaywrightParticipation(report, {
titles: WSL_NO_BACKGROUND_BOOT_TITLES,
label: 'WSL E9 guard'
})
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
verifyWslNoBackgroundBootParticipation(JSON.parse(readFileSync(process.argv[2], 'utf8')))
console.log('Both WSL E9 guards passed three times without skips or retries.')
}
@@ -0,0 +1,39 @@
import { describe, expect, it } from 'vitest'
import {
verifyWslNoBackgroundBootParticipation,
WSL_NO_BACKGROUND_BOOT_TITLES
} from './verify-wsl-no-background-boot-participation.mjs'
function report(runs = 3) {
return {
stats: {
expected: runs * WSL_NO_BACKGROUND_BOOT_TITLES.length,
skipped: 0,
unexpected: 0,
flaky: 0
},
suites: [
{
specs: WSL_NO_BACKGROUND_BOOT_TITLES.map((title) => ({
title,
tests: Array.from({ length: runs }, () => ({
expectedStatus: 'passed',
results: [{ status: 'passed' }]
}))
}))
}
]
}
}
describe('WSL E9 guard participation', () => {
it('accepts both guards executed three times', () => {
expect(() => verifyWslNoBackgroundBootParticipation(report())).not.toThrow()
})
it('rejects a run where the guard skipped', () => {
const value = report()
value.stats.skipped = 3
value.stats.expected = 0
expect(() => verifyWslNoBackgroundBootParticipation(value)).toThrow('participation failed')
})
})
@@ -19,6 +19,8 @@ describe('real WSL terminal lane', () => {
'tests/e2e/golden-tab-bar-agent-launch.spec.ts',
'tests/e2e/terminal-windows-shell-paste-ownership.spec.ts',
'tests/e2e/terminal-inline-images-windows.spec.ts',
'tests/e2e/wsl-bridge-no-background-boot.spec.ts',
'config/scripts/verify-wsl-no-background-boot-participation.mjs',
'.github/actions/setup-wsl-test-runtime/setup.ps1',
'.github/workflows/windows-wsl-e2e.yml'
])('routes %s to both WSL sentinels', (path) => {
@@ -68,5 +70,14 @@ describe('real WSL terminal lane', () => {
expect(receipt.run).toBe(
'node config/scripts/verify-wsl-e2e-participation.mjs test-results/wsl-results.json'
)
// E9: the guard terminates only the distro this job imported, and must prove it ran.
const guard = steps.find(
(step) => step.name === 'Guard that background work never starts the stopped WSL distro'
)
expect(guard.env.ORCA_E2E_WSL_TERMINATE_DISTRO).toBe('Ubuntu')
expect(guard.run).toContain('tests/e2e/wsl-bridge-no-background-boot.spec.ts')
expect(guard.run).toContain('--repeat-each=3')
const guardReceipt = steps.find((step) => step.name === 'Require all three E9 guard executions')
expect(guardReceipt.if).toBe('always()')
})
})
+4
View File
@@ -42,6 +42,10 @@ import { ignoreInheritedWslHostsGateState } from '../shared/wsl-host-internal-ga
import { registerOrcaProtocolClient } from './startup/orca-protocol-registration'
import { clearWindowsStdHandleInheritance } from './startup/windows-std-handle-inheritance'
import { installWindowsIsolatedSpawner } from './startup/windows-isolated-spawner'
import { dropE2eSpawnLogOutsideE2e } from './startup/e2e-spawn-log-env'
// Before anything spawns, so no daemon, PTY or agent inherits it outside e2e.
dropE2eSpawnLogOutsideE2e()
function openMainWindow(options: { revealOnDidFinishLoad?: boolean } = {}): BrowserWindow {
return openMainWindowController(options)
@@ -0,0 +1,44 @@
import { mkdtempSync, existsSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { afterEach, expect, it } from 'vitest'
import { runProcess } from '@orca/process-host'
import { E2E_SPAWN_LOG_ENV } from '@orca/process-host/spawn-observer'
import { dropE2eSpawnLogOutsideE2e } from './e2e-spawn-log-env'
const saved = { log: process.env[E2E_SPAWN_LOG_ENV], userData: process.env.ORCA_E2E_USER_DATA_DIR }
afterEach(() => {
for (const [key, value] of [
[E2E_SPAWN_LOG_ENV, saved.log],
['ORCA_E2E_USER_DATA_DIR', saved.userData]
] as const) {
if (value === undefined) {
delete process.env[key]
} else {
process.env[key] = value
}
}
})
it('drops the spawn log outside e2e mode, so a spawn writes nothing', async () => {
const dir = mkdtempSync(path.join(tmpdir(), 'orca-spawn-log-drop-'))
const log = path.join(dir, 'spawns.jsonl')
try {
delete process.env.ORCA_E2E_USER_DATA_DIR
process.env[E2E_SPAWN_LOG_ENV] = log
dropE2eSpawnLogOutsideE2e()
expect(process.env[E2E_SPAWN_LOG_ENV]).toBeUndefined()
await runProcess({ program: process.execPath, args: ['-e', 'process.exit(0)'] })
expect(existsSync(log)).toBe(false)
} finally {
rmSync(dir, { recursive: true, force: true })
}
})
it('keeps it in e2e mode', () => {
process.env.ORCA_E2E_USER_DATA_DIR = '/tmp/orca-e2e-user-data'
process.env[E2E_SPAWN_LOG_ENV] = '/tmp/spawns.jsonl'
dropE2eSpawnLogOutsideE2e()
expect(process.env[E2E_SPAWN_LOG_ENV]).toBe('/tmp/spawns.jsonl')
})
+12
View File
@@ -0,0 +1,12 @@
import { E2E_SPAWN_LOG_ENV } from '@orca/process-host/spawn-observer'
/**
* The e2e spawn log records full argv, which can include tokens; tests only. Outside e2e mode
* (no ORCA_E2E_USER_DATA_DIR) it is dropped before anything spawns, so no daemon, PTY or agent
* inherits it and a stray variable never writes a user's command lines to disk.
*/
export function dropE2eSpawnLogOutsideE2e(env: NodeJS.ProcessEnv = process.env): void {
if (!env.ORCA_E2E_USER_DATA_DIR) {
delete env[E2E_SPAWN_LOG_ENV]
}
}
@@ -0,0 +1,36 @@
import { readdirSync, readFileSync } from 'node:fs'
import path from 'node:path'
import { describe, expect, it } from 'vitest'
const SRC = path.resolve(__dirname, '../..')
// Markers of the e2e harness that patches main's fs and child_process; shipped code never carries them.
const HARNESS_MARKERS = [
'wsl-boot-diagnostics-log',
'orcaE9SpawnLog',
'recordMainProcessWslDiagnostics'
]
function shippedSources(dir: string): string[] {
return readdirSync(dir, { withFileTypes: true }).flatMap((entry) => {
const full = path.join(dir, entry.name)
if (entry.isDirectory()) {
return entry.name === 'node_modules' ? [] : shippedSources(full)
}
return /\.(ts|tsx|mts|cts|js|mjs)$/.test(entry.name) && !/\.(test|spec)\./.test(entry.name)
? [full]
: []
})
}
describe('E9 WSL boot diagnostics stay in the e2e harness', () => {
it('no shipped source imports or installs the fs and spawn hooks', () => {
const sources = shippedSources(SRC)
// Non-vacuous: the scan really walked the app's sources.
expect(sources.some((file) => file.endsWith(path.join('main', 'index.ts')))).toBe(true)
const offenders = sources.filter((file) => {
const text = readFileSync(file, 'utf8')
return HARNESS_MARKERS.some((marker) => text.includes(marker))
})
expect(offenders).toEqual([])
})
})
@@ -1,11 +1,21 @@
import { afterEach, describe, expect, it } from 'vitest'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { existsSync, mkdtempSync, readFileSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { setSpawnObserver } from './spawn-observer'
import {
E2E_SPAWN_LOG_ENV,
E2E_USER_DATA_DIR_ENV,
hasSpawnObserver,
notifySpawnObserver,
setSpawnObserver
} from './spawn-observer'
import { runProcess, runProcessSync, spawnProcess } from './run-process'
afterEach(() => {
setSpawnObserver(null)
vi.restoreAllMocks()
delete process.env[E2E_SPAWN_LOG_ENV]
delete process.env[E2E_USER_DATA_DIR_ENV]
})
describe('spawn observer', () => {
@@ -65,4 +75,49 @@ describe('spawn observer', () => {
await runProcess({ program: process.execPath, args: ['-e', 'process.exit(0)'] })
expect(calls).toBe(0)
})
it('logs every spawn, sync or async, to the e2e spawn log when one is named', async () => {
const dir = mkdtempSync(path.join(tmpdir(), 'orca-spawn-log-'))
const log = path.join(dir, 'spawns.jsonl')
process.env[E2E_SPAWN_LOG_ENV] = log
process.env[E2E_USER_DATA_DIR_ENV] = dir
try {
await runProcess({ program: process.execPath, args: ['-e', 'process.exit(0)'] })
runProcessSync({ program: process.execPath, args: ['-e', 'process.exit(0)'] })
const lines = readFileSync(log, 'utf8')
.trim()
.split('\n')
.map((line) => JSON.parse(line))
const spawned = {
file: process.execPath,
args: ['-e', 'process.exit(0)'],
at: expect.any(Number),
stack: expect.arrayContaining([expect.stringContaining('spawn-observer.test')])
}
expect(lines).toEqual([spawned, spawned])
} finally {
rmSync(dir, { recursive: true, force: true })
}
})
it('outside e2e mode writes nothing and captures no stack, even with the log variable set', () => {
const dir = mkdtempSync(path.join(tmpdir(), 'orca-spawn-log-'))
const log = path.join(dir, 'spawns.jsonl')
process.env[E2E_SPAWN_LOG_ENV] = log
const errors = vi.spyOn(globalThis, 'Error')
try {
notifySpawnObserver(process.execPath, ['-e', 'process.exit(0)'], 0)
expect(errors).not.toHaveBeenCalled()
expect(existsSync(log)).toBe(false)
expect(hasSpawnObserver()).toBe(false)
// Control: the same call in e2e mode does capture the stack and write the line.
process.env[E2E_USER_DATA_DIR_ENV] = dir
notifySpawnObserver(process.execPath, ['-e', 'process.exit(0)'], 0)
expect(errors).toHaveBeenCalledTimes(1)
expect(existsSync(log)).toBe(true)
} finally {
rmSync(dir, { recursive: true, force: true })
}
})
})
@@ -1,8 +1,24 @@
import { appendFileSync } from 'node:fs'
// Main installs diagnostics here; daemon, relay, and CLI cannot depend on main code.
export type SpawnObserver = (command: string, args: readonly string[], blockMs: number) => void
let observer: SpawnObserver | null = null
/**
* E2E only: every process-host spawn, in any thread or child that inherits the environment, is
* appended here as a JSON line, so a guard can see spawns main's own observer never would.
* Logs full argv, which can include tokens; tests only (main drops it outside e2e mode).
*/
export const E2E_SPAWN_LOG_ENV = 'ORCA_E2E_SPAWN_LOG'
/** Set only in e2e mode; a stray log variable alone never writes argv or captures stacks. */
export const E2E_USER_DATA_DIR_ENV = 'ORCA_E2E_USER_DATA_DIR'
function e2eSpawnLog(): string | undefined {
return (process.env[E2E_USER_DATA_DIR_ENV] && process.env[E2E_SPAWN_LOG_ENV]) || undefined
}
export function setSpawnObserver(next: SpawnObserver | null): void {
observer = next
}
@@ -13,8 +29,27 @@ export function notifySpawnObserver(
blockMs: number
): void {
observer?.(command, args, blockMs)
const log = e2eSpawnLog()
if (log) {
try {
// The caller's stack lets a guard name what launched a spawn it flags.
const limit = Error.stackTraceLimit
Error.stackTraceLimit = 60
const raw = new Error('spawn-log caller').stack ?? ''
Error.stackTraceLimit = limit
const stack = raw
.split('\n')
.slice(2)
.map((line) => line.trim())
.filter((line) => !/\((node:|<anonymous>)|^at (new Promise|process\.)/.test(line))
.slice(0, 30)
appendFileSync(log, `${JSON.stringify({ file: command, args, at: Date.now(), stack })}\n`)
} catch {
// A diagnostics write must never fail the spawn it describes.
}
}
}
export function hasSpawnObserver(): boolean {
return observer !== null
return observer !== null || e2eSpawnLog() !== undefined
}
@@ -0,0 +1,122 @@
/**
* Diagnostics for the E9 WSL guard: a JSON-lines log of every spawn, plus main-process filesystem
* touches of a `\\wsl.localhost` path, each with its time and caller stack, so a boot the
* running-list watcher sees can be matched to the act that caused it. Test harness only: installed
* into an e2e-mode app through Playwright, never imported by shipped code.
*/
import { existsSync, readFileSync } from 'node:fs'
import type { ElectronApplication } from '@stablyai/playwright-test'
/** `file` is the launched program, or `fs` for a filesystem touch whose `args` are [method, path]. */
export type DiagnosticRecord = { file: string; args: string[]; at?: number; stack?: string[] }
export const UNC_TOUCH = 'fs'
/**
* Logs main's direct child_process spawns (spawn, exec and execFile all pass through
* ChildProcess#spawn) and its fs touches of WSL UNC paths, beside the process-host log. node-pty
* terminals spawn elsewhere and need not be logged.
*/
export async function recordMainProcessWslDiagnostics(
app: ElectronApplication,
logFile: string
): Promise<void> {
await app.evaluate((_electron, logFile) => {
// Never patch fs or capture stacks in an app that is not in e2e mode.
if (!process.env.ORCA_E2E_USER_DATA_DIR || !process.env.ORCA_E2E_SPAWN_LOG) {
throw new Error('WSL boot diagnostics need an e2e-mode app with the spawn log named')
}
const fs = process.mainModule?.require('node:fs')
const proto = process.mainModule?.require('node:child_process')?.ChildProcess?.prototype
if (!fs || !proto) {
throw new Error('Cannot hook child_process in the main process')
}
if (proto.spawn.orcaE9SpawnLog) {
return
}
const append = (file: string, args: unknown[]): void => {
// Deep enough to reach the IPC handler past async hops; node internals add nothing.
const limit = Error.stackTraceLimit
Error.stackTraceLimit = 60
const raw = new Error('e9 diagnostics caller').stack ?? ''
Error.stackTraceLimit = limit
const stack = raw
.split('\n')
.slice(3)
.map((line) => line.trim())
.filter((line) => !/\((node:|<anonymous>)|^at (new Promise|process\.)/.test(line))
.slice(0, 30)
fs.appendFileSync(
logFile,
`${JSON.stringify({ file, args: args.map(String), at: Date.now(), stack })}\n`
)
}
const original = proto.spawn
const logged = function (this: unknown, options: { file?: string; args?: string[] }) {
append(String(options?.file ?? ''), options?.args ?? [])
return original.call(this, options)
}
logged.orcaE9SpawnLog = true
proto.spawn = logged
const isWslUnc = (value: unknown): boolean =>
/^([\\/]{2})(wsl\.localhost|wsl\$)[\\/]/i.test(
typeof value === 'string' ? value : value instanceof URL ? value.pathname : ''
)
const wrap = (target: Record<string, unknown>, prefix: string, names: string[]): void => {
for (const name of names) {
const fn = target[name]
if (typeof fn !== 'function') {
continue
}
const wrapped = function (this: unknown, ...args: unknown[]) {
if (isWslUnc(args[0])) {
append('fs', [`${prefix}${name}`, args[0]])
}
return fn.apply(this, args)
}
// Keep attached members such as `realpathSync.native`.
target[name] = Object.assign(wrapped, fn)
}
}
const methods = [
'stat',
'lstat',
'access',
'readFile',
'readdir',
'realpath',
'open',
'opendir'
]
wrap(fs, 'fs.', [...methods, ...methods.map((name) => `${name}Sync`), 'existsSync', 'watch'])
wrap(fs.promises, 'fs.promises.', [...methods, 'writeFile', 'mkdir'])
}, logFile)
}
export function readDiagnosticLog(logFile: string): DiagnosticRecord[] {
if (!existsSync(logFile)) {
return []
}
return readFileSync(logFile, 'utf8')
.split('\n')
.filter(Boolean)
.flatMap((line): DiagnosticRecord[] => {
let parsed: unknown
try {
parsed = JSON.parse(line)
} catch {
// A line torn by a concurrent append; the next read sees it whole.
return []
}
if (!parsed || typeof parsed !== 'object') {
return []
}
const file = 'file' in parsed ? String(parsed.file) : ''
const args = 'args' in parsed && Array.isArray(parsed.args) ? parsed.args.map(String) : []
const at = 'at' in parsed && typeof parsed.at === 'number' ? parsed.at : undefined
const stack =
'stack' in parsed && Array.isArray(parsed.stack) ? parsed.stack.map(String) : undefined
return [{ file, args, ...(at === undefined ? {} : { at }), ...(stack ? { stack } : {}) }]
})
}
+238
View File
@@ -0,0 +1,238 @@
/**
* Shared by the E9 real-WSL guards: wsl.exe reads that can never start a distro, a running-list
* watcher with a timestamped timeline, and the spawn-log check that names what started it.
*/
import { execFile } from 'node:child_process'
import type { Page } from '@stablyai/playwright-test'
import { expect } from './orca-app'
import { buildWslExecArgs } from '../../../src/shared/wsl-login-shell-command'
import { readDiagnosticLog, UNC_TOUCH, type DiagnosticRecord } from './wsl-boot-diagnostics-log'
const WSL_EXE = /(^|[\\/])wsl(\.exe)?$/i
/** Async with a timeout: a hung wsl.exe must fail the test, never block the worker past its timeout. */
export function wsl(args: readonly string[], timeoutMs = 60_000): Promise<string> {
return new Promise((resolve, reject) => {
execFile(
'wsl.exe',
[...args],
// UTF-8 instead of wsl.exe's default UTF-16 for its own listings.
{
encoding: 'utf8',
env: { ...process.env, WSL_UTF8: '1' },
timeout: timeoutMs,
windowsHide: true
},
(error, stdout, stderr) =>
error
? reject(new Error(`wsl ${args.join(' ')} failed: ${String(error)} ${stderr}`))
: resolve(stdout)
)
})
}
/** Timestamped steps in the test output, so a lane failure shows when each read and act happened. */
const startedAt = Date.now()
export function timeline(event: string, at = Date.now()): void {
console.log(`[e9-guard +${Math.round(at - startedAt)}ms] ${event}`)
}
/**
* `--list --running` never starts a distro. Exit 1 is "no running distributions"; any other
* failure (timeout, spawn error, other exit) rejects, so it can never read as "stopped".
*/
export function isDistroRunning(distro: string): Promise<boolean> {
return new Promise((resolve, reject) => {
execFile(
'wsl.exe',
['--list', '--running', '--quiet'],
{
encoding: 'utf8',
env: { ...process.env, WSL_UTF8: '1' },
timeout: 5_000,
windowsHide: true
},
(error, stdout) => {
if (error) {
if (error.code === 1 && !error.killed) {
resolve(false)
} else {
reject(error)
}
return
}
resolve(
stdout
.split(/\r?\n/)
.map((line) => line.replaceAll('\0', '').trim().toLowerCase())
.includes(distro.toLowerCase())
)
}
)
})
}
/** How long a just-terminated instance may still be listed: two reads, each up to 5 s on a loaded runner. */
const TERMINATE_GRACE_MS = 10_000
/**
* Polls the running list once a second, never overlapping, from right after `--terminate`.
* `stopped` resolves on the first not-running read, which must come within the grace; only
* running reads before it are forgiven, so a real boot is never waited out. Any running read
* after it is a boot. Three failed reads in a row fail the guard.
*/
export function watchDistroRunning(
distro: string,
spawnLog: string
): {
stopped: Promise<void>
stop: () => Promise<{ seenRunning: boolean; reads: number }>
} {
let finished = false
let confirmedStopped = false
let seenRunning = false
let reads = 0
let failures = 0
let failure: unknown = null
let timer: ReturnType<typeof setTimeout> | undefined
let current: Promise<void> = Promise.resolve()
let resolveStopped: () => void = () => {}
let rejectStopped: (error: Error) => void = () => {}
const stopped = new Promise<void>((resolve, reject) => {
resolveStopped = resolve
rejectStopped = reject
})
const grace = setTimeout(
() =>
rejectStopped(
new Error(`${distro} was still listed ${TERMINATE_GRACE_MS} ms after --terminate`)
),
TERMINATE_GRACE_MS
)
// Spawns are interleaved into the timeline so a boot can be matched to the launch before it.
let spawnsSeen = readDiagnosticLog(spawnLog).length
const logNewWslSpawns = (): void => {
const spawns = readDiagnosticLog(spawnLog)
for (const spawn of spawns.slice(spawnsSeen)) {
if (WSL_EXE.test(spawn.file) || spawn.file === UNC_TOUCH) {
// `at` is when it happened; the line prints at this poll, up to a second later.
timeline(
`${spawn.file === UNC_TOUCH ? 'unc' : 'spawn'} ${spawn.file} ${spawn.args.join(' ')}`,
spawn.at
)
if (startsDistro(spawn, distro) || spawn.file === UNC_TOUCH) {
for (const frame of spawn.stack ?? []) {
timeline(` ${frame}`, spawn.at)
}
}
}
}
spawnsSeen = spawns.length
}
const tick = (): void => {
logNewWslSpawns()
current = isDistroRunning(distro)
.then((running) => {
reads += 1
failures = 0
timeline(`running list: ${distro} ${running ? 'RUNNING' : 'stopped'}`)
if (confirmedStopped) {
seenRunning ||= running
} else if (!running) {
confirmedStopped = true
clearTimeout(grace)
resolveStopped()
}
})
.catch((error: unknown) => {
failures += 1
timeline(`running-list read failed: ${String(error)}`)
if (failures >= 3) {
failure = error
rejectStopped(new Error(`wsl --list --running kept failing: ${String(error)}`))
}
})
.finally(() => {
if (!finished && failure === null) {
timer = setTimeout(tick, 1_000)
}
})
}
tick()
return {
stopped,
stop: async () => {
finished = true
clearTimeout(timer)
clearTimeout(grace)
await current
logNewWslSpawns()
if (failure !== null) {
throw new Error(`wsl --list --running kept failing: ${String(failure)}`)
}
return { seenRunning, reads }
}
}
}
export function runInDistro(distro: string, script: string): Promise<string> {
return wsl(['-u', 'root', ...buildWslExecArgs(distro, ['sh', '-c', script])])
}
/** wsl.exe verbs that never start a distro; any other launch counts as starting the default one. */
const NON_BOOTING_WSL_VERBS = new Set([
'-l',
'--list',
'--status',
'--version',
'-v',
'-t',
'--terminate',
'--shutdown',
'--help'
])
/** Fails closed: a wsl.exe launch starts `distro` unless it names another one or only lists. */
export function startsDistro(spawn: DiagnosticRecord, distro: string): boolean {
if (!WSL_EXE.test(spawn.file)) {
return false
}
// main's ChildProcess#spawn args carry argv0; process-host's do not.
const raw = WSL_EXE.test(spawn.args[0] ?? '') ? spawn.args.slice(1) : spawn.args
const args = raw.map((arg) => arg.toLowerCase())
const named = args.findIndex((arg) => arg === '-d' || arg === '--distribution')
if (named !== -1) {
return args[named + 1] === distro.toLowerCase()
}
return !NON_BOOTING_WSL_VERBS.has(args[0] ?? '')
}
/** Waits until no wsl.exe launch has been logged for 3 s, so a user act's tail is over. */
export async function waitForWslLaunchesToSettle(spawnLog: string): Promise<void> {
let last = { count: -1, since: 0 }
await expect
.poll(
() => {
const count = readDiagnosticLog(spawnLog).filter((spawn) => WSL_EXE.test(spawn.file)).length
const now = performance.now()
if (count !== last.count) {
last = { count, since: now }
}
return now - last.since >= 3_000
},
{ timeout: 60_000, intervals: [500] }
)
.toBe(true)
}
export async function cycleWindowFocus(page: Page): Promise<void> {
await page.evaluate(() => {
window.dispatchEvent(new Event('blur'))
document.dispatchEvent(new Event('visibilitychange'))
window.dispatchEvent(new Event('focus'))
})
}
export async function activate(page: Page, worktreeId: string | null): Promise<void> {
await page.evaluate((id) => window.__store?.getState().setActiveWorktree(id), worktreeId)
}
@@ -0,0 +1,242 @@
/**
* E9 guard on a real WSL distro: background work never starts a stopped distro. A project inside
* the lane's distro is opened, the distro is terminated, and a window of ordinary background work
* elsewhere must start nothing in it; the user then opening a terminal there starts it. Runs only
* where ORCA_E2E_WSL_TERMINATE_DISTRO names a distro this run owns, since it terminates it.
*
* The running-list watcher is the authoritative assertion: `wsl --list --running`, polled through
* the window, sees every boot, including a plain UNC read or git.exe with a UNC cwd. The spawn log
* is diagnostics that name the caller: process-host spawns from any thread or inheriting child,
* plus main's direct child_process spawns. It cannot see direct spawnSync calls outside
* process-host (wsl-availability's missing-kernel `--exec /bin/true` is a documented exception)
* or spawns in children that drop the environment.
*/
import { rmSync } from 'node:fs'
import os from 'node:os'
import path from 'node:path'
import type { ElectronApplication, Page, TestInfo } from '@stablyai/playwright-test'
import { expect, test } from './helpers/orca-app'
import { ensureTerminalVisible, waitForActiveWorktree, waitForSessionReady } from './helpers/store'
import { waitForActivePanePtyId } from './helpers/terminal'
import { visitSettingsPanes } from './helpers/settings-pane-visits'
import { quotePosixShell } from '../../src/shared/shell-argument-quoting'
import {
readDiagnosticLog,
recordMainProcessWslDiagnostics
} from './helpers/wsl-boot-diagnostics-log'
import {
activate,
cycleWindowFocus,
isDistroRunning,
runInDistro,
startsDistro,
timeline,
waitForWslLaunchesToSettle,
watchDistroRunning,
wsl
} from './helpers/wsl-boot-guard'
const DISTRO = process.env.ORCA_E2E_WSL_TERMINATE_DISTRO?.trim() ?? ''
/** Settings panes whose mount reads host, account, CLI or integration state. */
const SETTINGS_PANES = [
'general',
'integrations',
'accounts',
'agents',
'git',
'tasks',
'session-history',
'automations'
] as const
const SPAWN_LOG = path.join(os.tmpdir(), `orca-e2e-wsl-spawns-${process.pid}-${Date.now()}.jsonl`)
// Every process-host spawn in the app (main, worker threads, inheriting children) lands here.
test.use({ launchEnv: { ORCA_E2E_SPAWN_LOG: SPAWN_LOG } })
type WslWorkspace = { linuxRoot: string; worktreeId: string; repoId: string }
/** A git project inside the lane's distro, opened and focused while it runs, its terminal settled. */
async function openWslWorkspace(
electronApp: ElectronApplication,
page: Page
): Promise<WslWorkspace> {
const linuxRoot = `/tmp/orca-e9-${Date.now()}`
const repoLinuxPath = `${linuxRoot}/repo`
await runInDistro(
DISTRO,
`mkdir -p ${quotePosixShell(`${repoLinuxPath}/src`)} && cd ${quotePosixShell(repoLinuxPath)} && ` +
'git init -q -b main . && printf "# e9\\n" > README.md && printf "e9\\n" > src/main.txt && ' +
'git add . && git -c user.name=e2e -c user.email=e2e@example.com commit -qm init'
)
const uncPath = `\\\\wsl.localhost\\${DISTRO}${repoLinuxPath.replaceAll('/', '\\')}`
// Its sidebar card carries a linked issue and PR, whose refreshes must not start it either.
const opened = await page.evaluate(async (path) => {
const store = window.__store!
const added = await window.api.repos.add({ path, kind: 'git' })
if ('error' in added) {
throw new Error(added.error)
}
await store.getState().fetchWorktrees(added.repo.id)
const worktree = store
.getState()
.worktreesByRepo[added.repo.id]?.find((row) => row.isMainWorktree)
if (!worktree) {
throw new Error('WSL checkout missing')
}
await store.getState().updateWorktreeMeta(worktree.id, { linkedIssue: 1, linkedPR: 2 })
return { worktreeId: worktree.id, repoId: added.repo.id }
}, uncPath)
// Focusing it opens a WSL terminal. Let that launch (its PTY, hook relay and agent probes)
// finish before the terminate, or it boots the distro after it.
await recordMainProcessWslDiagnostics(electronApp, SPAWN_LOG)
await activate(page, opened.worktreeId)
await ensureTerminalVisible(page, 60_000)
await waitForActivePanePtyId(page, 60_000)
await waitForWslLaunchesToSettle(SPAWN_LOG)
timeline('WSL workspace terminal launched and settled')
return { linuxRoot, ...opened }
}
/**
* Terminates the distro, runs `window` once a read has seen it stopped, and fails if anything
* started it meanwhile. Then the user opens a terminal there, which must start it.
*/
async function expectNothingStartsTheStoppedDistro(
page: Page,
testInfo: TestInfo,
workspace: WslWorkspace,
window: () => Promise<void>
): Promise<void> {
// Counted from before the terminate, so work started while it stops counts too.
const spawnsBefore = readDiagnosticLog(SPAWN_LOG).length
timeline('terminate')
await wsl(['--terminate', DISTRO], 30_000)
timeline('terminated')
// WSL1 instances can live only seconds, so watch the running list from the terminate on.
const watcher = watchDistroRunning(DISTRO, SPAWN_LOG)
let watched: { seenRunning: boolean; reads: number } | undefined
try {
// The window starts only once a read has seen it stopped, within the grace.
await watcher.stopped
timeline('confirmed stopped')
await window()
await page.screenshot({ path: testInfo.outputPath('wsl-bridge-background-window.png') })
// ast-grep-ignore: test-no-real-time-wait -- the claim is that nothing boots over a window of real background work
await page.waitForTimeout(15_000)
} finally {
watched = await watcher.stop()
}
// Authoritative: the distro never ran during the window.
expect(watched.reads).toBeGreaterThan(10)
expect(watched.seenRunning).toBe(false)
expect(await isDistroRunning(DISTRO)).toBe(false)
// Diagnostics that name the caller; non-empty so the check cannot pass vacuously.
const background = readDiagnosticLog(SPAWN_LOG).slice(spawnsBefore)
expect(background.length).toBeGreaterThan(0)
expect(background.filter((spawn) => startsDistro(spawn, DISTRO))).toEqual([])
timeline(`window done: ${background.length} spawns`)
// The user opens the WSL workspace and a terminal in it: that act starts the distro.
await activate(page, workspace.worktreeId)
await page.getByRole('button', { name: 'New tab', exact: true }).first().click()
await page
.getByRole('menuitem', { name: /New Terminal/ })
.first()
.click()
await expect(page.locator('.xterm:visible').first()).toBeVisible({ timeout: 60_000 })
await expect.poll(() => isDistroRunning(DISTRO), { timeout: 60_000 }).toBe(true)
await page.screenshot({ path: testInfo.outputPath('wsl-bridge-after-terminal.png') })
}
async function removeWslWorkspace(workspace: WslWorkspace | undefined): Promise<void> {
if (workspace) {
await runInDistro(DISTRO, `rm -rf ${quotePosixShell(workspace.linuxRoot)}`).catch(
(error: unknown) => timeline(`cleanup failed: ${String(error)}`)
)
}
rmSync(SPAWN_LOG, { force: true })
}
test.describe('E9: background work on a stopped real WSL distro', () => {
test.skip(
process.platform !== 'win32' || !DISTRO,
'Needs a Windows run that owns a WSL distro it may terminate (ORCA_E2E_WSL_TERMINATE_DISTRO)'
)
test('starts nothing in the stopped distro until the user opens a terminal there', async ({
electronApp,
orcaPage: page
}, testInfo) => {
// Distro setup, a terminate, a 20 s background window and a boot.
test.setTimeout(300_000)
await waitForSessionReady(page)
const localWorktreeId = await waitForActiveWorktree(page)
let workspace: WslWorkspace | undefined
try {
workspace = await openWslWorkspace(electronApp, page)
const { repoId } = workspace
await activate(page, localWorktreeId)
await expectNothingStartsTheStoppedDistro(page, testInfo, workspace, async () => {
for (let round = 0; round < 2; round += 1) {
timeline(`round ${round}: focus and workspace switches`)
await cycleWindowFocus(page)
await activate(page, null)
await cycleWindowFocus(page)
await activate(page, localWorktreeId)
timeline(`round ${round}: settings`)
await visitSettingsPanes(page, SETTINGS_PANES, repoId)
timeline(`round ${round}: tasks and account switches`)
await page.evaluate(() => window.__store?.getState().setActiveView('tasks'))
await page.evaluate(async () => {
// Account switches re-sync every account home, including WSL ones.
await window.api.codexAccounts.select({ accountId: null }).catch(() => {})
await window.api.claudeAccounts.select({ accountId: null }).catch(() => {})
})
await page.evaluate(() => window.__store?.getState().setActiveView('terminal'))
}
})
} finally {
await removeWslWorkspace(workspace)
}
})
test('a focused WSL workspace left idle through the stop never starts it again', async ({
electronApp,
orcaPage: page
}, testInfo) => {
// Distro setup, an Explorer and Source Control load, a terminate, a 22.5 s idle and a boot.
test.setTimeout(300_000)
await waitForSessionReady(page)
let workspace: WslWorkspace | undefined
try {
workspace = await openWslWorkspace(electronApp, page)
// The user's view: the Explorer with a folder expanded, then Source Control.
await page.evaluate(() => {
const state = window.__store?.getState()
state?.setRightSidebarOpen(true)
state?.setRightSidebarTab('explorer')
})
const row = (name: string) =>
page
.locator('[data-file-explorer-row]')
.filter({ has: page.getByText(name, { exact: true }) })
await row('src').click()
await expect(row('main.txt')).toBeVisible({ timeout: 30_000 })
await page.evaluate(() => window.__store?.getState().setRightSidebarTab('source-control'))
await waitForWslLaunchesToSettle(SPAWN_LOG)
timeline('focused: explorer expanded, source control shown')
// Nothing more from the user: the stop's own fallout (bridge exit, watcher stop, host loss and
// the re-renders they cause) is all that runs, half with Explorer and half with Source Control.
await expectNothingStartsTheStoppedDistro(page, testInfo, workspace, async () => {
await page.evaluate(() => window.__store?.getState().setRightSidebarTab('explorer'))
// ast-grep-ignore: test-no-real-time-wait -- the Explorer must sit visible while the stop lands
await page.waitForTimeout(7_500)
timeline('idle: source control shown')
await page.evaluate(() => window.__store?.getState().setRightSidebarTab('source-control'))
})
} finally {
await removeWslWorkspace(workspace)
}
})
})