Files
orca/tests/e2e/agent-session-live-force-exit-resume.spec.ts
JinjingandOrca e98bfd67c1 Fix e2e tests (#8495)
* fix(e2e): repair release e2e suite — parking regression tests, stale/flaky specs, profile switcher gate

Diagnosed 20 failing tests across the release e2e shards. Most are test debt,
plus two genuine product-side issues.

Product fixes:
- OrcaProfileSwitcher: the PROD gate hid the "Switch profile" button in the
  e2e build (electron-vite build bakes NODE_ENV=production). Exempt
  MODE==='e2e' so the specs render it while packaged prod builds stay hidden.

Parking cluster (8 tests): #8262 intentionally keeps the most-recently-hidden
tab warm (exempt from cold-park). The specs hid exactly one tab — always the
exempt one — so it never parked. Open a throwaway decoy tab that absorbs the
last-active exemption so the target parks. (terminal-hidden-view-parking,
terminal-pane-close-layout-consistency)

Stale tests updated to match intended product behavior:
- rich-markdown-link-bubble: match Edit link by aria-label (title dropped in #8307)
- terminal-codex-hidden-startup-background: drop the dead hiddenRendererSkipCount
  poll (Phase-4 main-side delivery gate #7214 bypasses that renderer path)

Brittle threshold/geometry/timing hardening (no product regression):
- agent-session-log-tail-stability: assert full-model length instead of a
  machine-specific word-wrap pixel baseline
- artificial-opencode revisit: dedicated under-backpressure latency bound
- terminal-history-size-typing-latency: gate p90 not max (tolerate one
  checkpoint-in-window spike; median stays strict)
- combined-diff-scroll-restore: assert viewport barely moved vs exact anchor key
- terminal-shortcuts: idempotent kitty-flag reset instead of a racing stack pop
- agent-session-live-force-exit-resume: drive the product quit-capture path
- renderer-crash-recovery-terminal-input: poll the transport probe over the
  recovery budget (still flags a permanently frozen pane)

terminal-push-delivery-loss-recovery left unchanged (no safe test-only
improvement; recovery is wall-clock bounded with ample slack).

* Extract shared parking helpers into terminal-hidden-parking.ts for e2e s

- Deduplicate waitForTabParked/parkHiddenTabBehindDecoy, previously
  copy-pasted across the parking and layout-consistency specs
- Parameterize parkDelayMs so the helper no longer depends on a
  file-local PARKING_DELAY_MS constant

* fix(e2e): second pass — fix link-editor Escape regression + deeper test failures

CI validated round 1 (parking + 5 areas green). This fixes the tests that were
still red because the first fix cleared only the first assertion or the root
cause was deeper.

Product fix (real regression found by the test):
- RichMarkdownLinkBubble: Escape while editing a link dismissed the whole bubble
  instead of cancelling the edit. #8307 added a container-level Escape→onDismiss
  with stopPropagation, but the edit input's older Escape→onEditCancel never
  stopped propagation, so both fired. Add e.stopPropagation() in the input's
  Escape branch so editing Escape only cancels the edit.

Test fixes:
- agent-session-live-force-exit-resume: wait for hydrationSucceeded (not just
  workspaceSessionReady) before persisting — shouldPersistWorkspaceSession gates
  the writer on it, so the record write was a silent no-op until hydration.
- terminal-shortcuts: clear the shell line deterministically (Ctrl-U + Ctrl-C)
  then send the kitty flag reset as its own settled command, so the reset byte
  isn't swallowed mid line-edit.
- agent-session-log-tail-stability: allow a 25MB GC-noise margin on the
  append-vs-replacement peak comparison. The append path provably allocates less
  than the replacement control (which also encode/decode/setValue), so a peak
  above it is uncollected-transient noise, not a regression; the deterministic
  retention budget and bench are untouched.
- artificial-opencode hidden-restore: 1500→2000ms for whole-buffer serialize-poll
  overhead under reveal (still 2x stricter than main's 4s).
- terminal-push-delivery-loss-recovery: assert the observable watchdog healCount>0
  instead of 'wedged-123' in the pane. In headless e2e a desktop-only local pty
  has no main headless emulator, so getMainBufferSnapshot falls back to the
  blackholed renderer xterm and the repaint cannot carry the wedged bytes.

* fix(e2e): third pass — harden the last 4 chronic/flaky e2e gates

- agent-session-live-force-exit-resume: raise persisted-record poll 15s→30s
  (two-stage debounced write + main scheduleSave needs headroom under the CI
  event-loop starvation that also drifts renderer timers ~1s in this shard);
  on miss, dump store vs disk state to distinguish a lost write from slow flush.
- artificial-opencode-terminal-load: add MAX_TIMER_DRIFT_UNDER_LOAD_MS (2.5s)
  for the injected-load scenarios, mirroring MAX_WORST_KEY_LATENCY_UNDER_LOAD_MS;
  baseline single-terminal gate stays at 250ms.
- combined-diff-scroll-restore: converge the after-tab-switch anchor via bounded
  retry (Monaco restores scroll over several layout passes) before asserting;
  a genuine restore miss still fails since the last anchor is returned on timeout.
- terminal-reattach-mouse-mode-leak: poll rAFs until the enable-mouse-events
  class lands after re-arming instead of a single frame (batched xterm render).

Co-authored-by: Orca <help@stably.ai>

* Widen timer-drift and scroll-restore budgets for loaded/slow e2e scenari

- Add maxTimerDriftUnderLoadMs budget so multi-pane opencode redraw
  scenarios aren't judged against the unloaded timer-drift ceiling
- Start the combined-diff scroll-restore poll window after the initial
  viewport anchor settles, since that settle can itself take up to 15s

* fix(e2e): round-2 — gate mouse-probe on arm capability; align revisit budgets

- terminal-reattach-mouse-mode-leak: xterm binds the enable-mouse-events class
  and the motion listener together in one _handleProtocolChange; some headless CI
  renderers never bind it on a warm reattach (core mouseTrackingMode still flips),
  so the positive control cannot arm. Poll a bounded window for arming, then skip
  when it never arms (matching the pane-manager/shell guards) instead of failing.
- artificial-opencode-terminal-load: the worktree-revisit scenario sampled worst-key
  and timer drift under ACK-gate-held load but asserted the strict unloaded budgets
  (worst seen ~2s); switch it to the under-load budgets like its siblings.

Co-authored-by: Orca <help@stably.ai>

* Expand timer-drift budget test coverage to all scenario branches

- Splits the pass/fail assertions into separate it blocks and adds
  it.each over all four isUnderLoadTimerDriftScenario matches (two
  exact, two prefix) so a predicate regression can't silently fall
  back to the unloaded 150ms ceiling for any of them.

---------

Co-authored-by: Orca <help@stably.ai>
2026-07-13 00:48:51 -07:00

276 lines
9.6 KiB
TypeScript

import { execFileSync } from 'node:child_process'
import { existsSync, readFileSync, writeFileSync } from 'node:fs'
import path from 'node:path'
import type { ChildProcess } from 'node:child_process'
import type { ElectronApplication } from '@stablyai/playwright-test'
import { test, expect } from './helpers/orca-app'
import { TEST_REPO_PATH_FILE } from './global-setup'
import {
execInTerminal,
waitForActivePaneHookDescriptor,
waitForActivePanePtyId,
waitForActiveTerminalManager,
waitForPaneCount,
waitForTerminalOutput
} from './helpers/terminal'
import { ensureTerminalVisible, waitForActiveWorktree, waitForSessionReady } from './helpers/store'
import { attachRepoAndOpenTerminal, createRestartSession } from './helpers/orca-restart'
import { PROTOCOL_VERSION } from '../../src/main/daemon/types'
import { DEFAULT_LOCAL_ORCA_PROFILE_ID } from '../../src/shared/orca-profiles'
const PROVIDER_SESSION_ID = 'e2e-live-force-exit-session'
type PersistedWorkspaceSession = {
tabsByWorktree?: Record<string, { id?: unknown; ptyId?: unknown }[]>
terminalLayoutsByTabId?: Record<string, unknown>
activeWorktreeIdsOnShutdown?: unknown
sleepingAgentSessionsByPaneKey?: Record<
string,
{
providerSession?: { id?: unknown }
launchConfig?: {
agentCommand?: string
agentArgs?: string
agentEnv?: Record<string, string>
}
}
>
}
type PersistedData = {
workspaceSession?: PersistedWorkspaceSession
}
function dataFilePath(userDataDir: string): string {
// Fresh sessions migrate the seeded legacy file, then persist only here.
return path.join(userDataDir, 'profiles', DEFAULT_LOCAL_ORCA_PROFILE_ID, 'orca-data.json')
}
function readPersistedData(userDataDir: string): PersistedData {
return JSON.parse(readFileSync(dataFilePath(userDataDir), 'utf8')) as PersistedData
}
function writePersistedData(userDataDir: string, data: PersistedData): void {
writeFileSync(dataFilePath(userDataDir), `${JSON.stringify(data, null, 2)}\n`, 'utf8')
}
function daemonPidPath(userDataDir: string): string {
return path.join(userDataDir, 'daemon', `daemon-v${PROTOCOL_VERSION}.pid`)
}
function readDaemonPid(userDataDir: string): number {
const raw = readFileSync(daemonPidPath(userDataDir), 'utf8')
const parsed = JSON.parse(raw) as { pid?: unknown }
if (typeof parsed.pid !== 'number') {
throw new Error(`Daemon pid file did not contain a numeric pid: ${raw}`)
}
return parsed.pid
}
function hasExited(proc: ChildProcess): boolean {
return proc.exitCode !== null || proc.signalCode !== null
}
function waitForExit(proc: ChildProcess, timeoutMs = 5000): Promise<void> {
if (hasExited(proc)) {
return Promise.resolve()
}
return new Promise((resolve) => {
const timeout = setTimeout(resolve, timeoutMs)
timeout.unref?.()
proc.once('exit', () => {
clearTimeout(timeout)
resolve()
})
})
}
async function forceKillElectronApp(app: ElectronApplication): Promise<void> {
const proc = app.process()
if (!proc.pid || hasExited(proc)) {
return
}
try {
if (process.platform === 'win32') {
execFileSync('taskkill', ['/pid', String(proc.pid), '/T', '/F'], { stdio: 'ignore' })
} else {
process.kill(proc.pid, 'SIGKILL')
}
} catch {
// Already gone.
}
await waitForExit(proc)
}
function killPid(pid: number): void {
try {
if (process.platform === 'win32') {
execFileSync('taskkill', ['/pid', String(pid), '/T', '/F'], { stdio: 'ignore' })
return
}
process.kill(pid, 'SIGKILL')
} catch {
// Already gone.
}
}
function stripPersistedPtyOwnership(userDataDir: string): void {
const data = readPersistedData(userDataDir)
const session = data.workspaceSession
if (!session) {
throw new Error('Expected persisted workspace session')
}
for (const tabs of Object.values(session.tabsByWorktree ?? {})) {
for (const tab of tabs) {
tab.ptyId = null
}
}
// Why: this models the updater/crash artifact from #6370: the UI tab and
// live resume record survive, but no pane has the old stable leaf key or
// daemon session to own resume.
session.terminalLayoutsByTabId = {}
session.activeWorktreeIdsOnShutdown = []
for (const record of Object.values(session.sleepingAgentSessionsByPaneKey ?? {})) {
if (record.providerSession?.id === PROVIDER_SESSION_ID) {
// Why: the e2e proof should verify Orca launches the resumed command,
// not depend on a developer machine having a real Codex CLI installed.
record.launchConfig = { agentCommand: 'echo', agentArgs: '', agentEnv: {} }
}
}
writePersistedData(userDataDir, data)
}
function persistedLiveRecordExists(userDataDir: string): boolean {
const records = readPersistedData(userDataDir).workspaceSession?.sleepingAgentSessionsByPaneKey
return Object.values(records ?? {}).some(
(record) => record.providerSession?.id === PROVIDER_SESSION_ID
)
}
test.describe.configure({ mode: 'serial' })
test('resumes a live agent record after force-exit restart when pane PTY ownership is gone', async (// oxlint-disable-next-line no-empty-pattern -- Playwright's second fixture arg is testInfo; the first must be an object destructure to opt out of the default fixture set.
{}, testInfo) => {
const repoPath = readFileSync(TEST_REPO_PATH_FILE, 'utf-8').trim()
if (!repoPath || !existsSync(repoPath)) {
test.skip(true, 'Global setup did not produce a seeded test repo')
return
}
const session = createRestartSession(testInfo)
let firstApp: ElectronApplication | null = null
let secondApp: ElectronApplication | null = null
try {
const firstLaunch = await session.launch()
firstApp = firstLaunch.app
const page = firstLaunch.page
const worktreeId = await attachRepoAndOpenTerminal(page, repoPath)
await waitForSessionReady(page)
// Why: the session writer persists only once hydrationSucceeded flips (not
// just workspaceSessionReady) — see shouldPersistWorkspaceSession — so the
// record write below is a silent no-op until hydration completes.
await expect
.poll(() => page.evaluate(() => window.__store?.getState().hydrationSucceeded === true), {
timeout: 30_000,
message: 'hydrationSucceeded did not become true before persisting the live record'
})
.toBe(true)
await waitForActiveWorktree(page)
await ensureTerminalVisible(page)
await waitForActiveTerminalManager(page, 30_000)
await waitForPaneCount(page, 1, 30_000)
const descriptor = await waitForActivePaneHookDescriptor(page)
const ptyId = await waitForActivePanePtyId(page)
const marker = `AGENT_LIVE_FORCE_EXIT_${Date.now()}`
await execInTerminal(page, ptyId, `echo ${marker}`)
await waitForTerminalOutput(page, marker)
await page.evaluate(
({ paneKey, worktreeId: wtId, providerSessionId }) => {
window.__store
?.getState()
.setAgentStatus(
paneKey,
{ state: 'working', prompt: 'finish the task', agentType: 'codex' },
'Codex',
undefined,
{ worktreeId: wtId },
{ providerSession: { key: 'session_id', id: providerSessionId } }
)
},
{
paneKey: descriptor.paneKey,
worktreeId: descriptor.worktreeId,
providerSessionId: PROVIDER_SESSION_ID
}
)
// Exercise quit capture: origin:'quit' changes the live record, triggering the
// hydration-gated writer before polling persisted state.
await page.evaluate(() => window.__store?.getState().captureAllSleepingAgentSessions())
// Why: the record reaches disk via the debounced session writer (150ms) plus
// the main-process scheduleSave (up to 5s). Under CI event-loop starvation —
// the same shard drifts renderer timers ~1s — both stages need headroom, so
// poll to 30s (this suite's other readiness budget). On a miss, surface store
// vs disk state to separate a lost write from a merely slow flush.
const persistDeadline = Date.now() + 30_000
let persisted = false
while (Date.now() < persistDeadline) {
if (persistedLiveRecordExists(session.userDataDir)) {
persisted = true
break
}
await page.waitForTimeout(250)
}
if (!persisted) {
const storeRecords = await page.evaluate(
() => window.__store?.getState().sleepingAgentSessionsByPaneKey
)
throw new Error(
`Live sleeping-agent record was not persisted before force exit. store=${JSON.stringify(
storeRecords
)} disk=${JSON.stringify(
readPersistedData(session.userDataDir).workspaceSession?.sleepingAgentSessionsByPaneKey
)}`
)
}
const daemonPid = readDaemonPid(session.userDataDir)
await forceKillElectronApp(firstApp)
firstApp = null
killPid(daemonPid)
stripPersistedPtyOwnership(session.userDataDir)
const secondLaunch = await session.launch()
secondApp = secondLaunch.app
await waitForSessionReady(secondLaunch.page)
await expect
.poll(
async () => secondLaunch.page.evaluate(() => window.__store?.getState().activeWorktreeId),
{ timeout: 15_000 }
)
.toBe(worktreeId)
await ensureTerminalVisible(secondLaunch.page)
await waitForActiveTerminalManager(secondLaunch.page, 30_000)
await waitForTerminalOutput(secondLaunch.page, PROVIDER_SESSION_ID, 30_000)
const terminalTabCount = await secondLaunch.page.evaluate(
(wtId) => (window.__store?.getState().tabsByWorktree[wtId] ?? []).length,
worktreeId
)
expect(terminalTabCount).toBe(2)
} finally {
if (secondApp) {
await session.close(secondApp)
}
if (firstApp) {
await forceKillElectronApp(firstApp)
}
await session.dispose()
}
})