Files
orca/tests/e2e/cross-version-wire/cross-version-terminal-wire.unit.test.ts
Jinwoo Hong 06780260c0 test(remote-runtime): run an old client and an old server against current code (#12682)
Mixed versions are the normal state of the remote-server feature: users update clients and servers independently. Until now nothing tested that. Every cross-version claim was made by code reading plus unit tests with hand-written old/new shapes — enough to catch design problems, not enough to catch a real skew regression.

This runs the REAL protocol implementations from two builds against each other in one process: the actual host methods and RPC dispatcher on one side, the actual renderer multiplexer on the other, with a transport that reproduces the production asymmetry — each side decodes with its OWN codec and drops frames whose opcode it does not know. A frame survives only if the RECEIVING build understands it, which is what makes this level sufficient without launching two apps. The old side is a genuine checkout extracted from the release tag; the extracted client was confirmed to lack a symbol that exists only on main.

Journey: subscribe, first snapshot, input reaching the process, live output, hide/reveal snapshot, transport drop, resubscribe, input landing again — across old->new, new->old, and a current/current control. Every step ends on an observed-state barrier; no sleeps. The oracle asserts the recorded step list, the exact 16-frame named sequence, negotiated capabilities, the exact input the host wrote to the PTY, rendered content, and zero decoder-rejected frames. A host method the stub lacks is recorded by name and asserted empty, so a harness gap cannot masquerade as a wire break.

Detection is proven per violation shape, and it attributes each to the correct side: an unnegotiated opcode goes red only where a decoder would reject it, a removed published field goes red only where an old client consumes it, and a legal additive field stays green in all three pairings so the harness will not cry wolf on safe changes.

It also documents the three compatibility rules in docs/reference/remote-wire-compatibility.md, linked from AGENTS.md, since they previously existed only as folklore — notably that "decoders reject unknown opcodes" is true for the desktop decoder but NOT for mobile, which silently drops them.

Deliberately scoped: terminal stream only. The session-tab sync channel is not covered, nor agent-session publications, file/Git RPCs, mobile E2EE framing, or the relay transport. Two version points, so a regression introduced and reverted between them is invisible.

CI selection was verified rather than assumed — `vitest list` confirms 0 matches under the shard's exclude and 4 under the dedicated job — because a lane silently running zero tests is precisely how a host-side defect escaped CI earlier in this series. Closes STA-3469.
2026-08-05 01:31:29 -07:00

137 lines
4.5 KiB
TypeScript

import { afterEach, beforeAll, describe, expect, it } from 'vitest'
import { resolveBaselineReleaseRef } from './release-checkout'
import {
JOURNEY_INPUTS,
JOURNEY_STEPS,
runTerminalSkewJourney,
type JourneyRecord
} from './terminal-skew-journey'
import {
loadTerminalWireBuild,
WORKING_TREE,
type TerminalWireBuild
} from './versioned-terminal-wire'
// Why: a cold CI run extracts the baseline checkout before the first journey.
const SUITE_TIMEOUT_MS = 180_000
/**
* The frames one journey must produce, named rather than numbered so a diff reads
* as a protocol change. Any deviation is a change in what a peer publishes or
* accepts, and needs a human decision against docs/reference/remote-wire-compatibility.md.
*/
const EXPECTED_JOURNEY_FRAMES = [
'C>H Subscribe',
'H>C SnapshotStart',
'H>C SnapshotChunk',
'H>C SnapshotEnd',
'C>H Input',
'H>C Output',
'C>H SnapshotRequest',
'H>C SnapshotStart',
'H>C SnapshotChunk',
'H>C SnapshotEnd',
'C>H Subscribe',
'H>C SnapshotStart',
'H>C SnapshotChunk',
'H>C SnapshotEnd',
'C>H Input',
'C>H Unsubscribe'
]
let baselineRef: string
let current: TerminalWireBuild
let baseline: TerminalWireBuild
beforeAll(async () => {
baselineRef = resolveBaselineReleaseRef()
current = await loadTerminalWireBuild(WORKING_TREE)
baseline = await loadTerminalWireBuild(baselineRef)
}, SUITE_TIMEOUT_MS)
afterEach(() => {
// Each journey installs and removes its own window stub; fail loudly if one leaked.
expect(typeof globalThis.window).toBe('undefined')
})
function expectJourneyActuallyRan(record: JourneyRecord): void {
// The anti-vacuous-pass oracle. A harness that connects and then does nothing
// fails here, because "nothing threw" is never enough to call a pairing green.
expect(record.completed).toEqual([...JOURNEY_STEPS])
expect(record.frameSequence).toEqual(EXPECTED_JOURNEY_FRAMES)
expect(record.subscribedEvents).toHaveLength(2)
expect(record.snapshotStarts).toHaveLength(3)
expect(record.missingRuntimeMethods).toEqual([])
}
function expectWireCompatible(record: JourneyRecord): void {
// Rule 2 — no frame may be refused by the receiving build's decoder. An opcode
// the peer does not know is dropped silently, so this is the only signal.
expect(record.rejected).toEqual([])
expect(record.clientErrors).toEqual([])
// The subscribe handshake still negotiates the optional output-pause opcode,
// which is what keeps opcode 16 legal to send on this pairing.
for (const event of record.subscribedEvents) {
expect(event.capabilities).toEqual({ outputPause: 1 })
}
// Input reached the process, before and after the reconnect.
expect(record.inputAtProcess).toEqual([JOURNEY_INPUTS.first, JOURNEY_INPUTS.second])
// Rule 3 — what the host publishes, as the client actually rendered it.
expect(record.snapshotsRendered[0]).toBe(JOURNEY_INPUTS.initialBuffer)
expect(record.dataRendered.join('')).toBe(JOURNEY_INPUTS.output)
expect(record.revealSnapshot?.data).toBe(
`${JOURNEY_INPUTS.initialBuffer}${JOURNEY_INPUTS.output}`
)
expect(record.revealSnapshot).toMatchObject({ cols: 120, rows: 40 })
for (const start of record.snapshotStarts) {
expect(start).toMatchObject({ kind: 'scrollback', cols: 120, rows: 40, source: 'headless' })
}
}
describe('cross-version remote terminal wire', () => {
it(
'skews current code against a real published release',
() => {
expect(baselineRef).toMatch(/^v?\d/)
expect(baseline.revision).toMatch(/^[0-9a-f]{40}$/)
expect(baseline.revision).not.toBe(current.revision)
},
SUITE_TIMEOUT_MS
)
it(
'current client against current server completes the journey',
async () => {
const record = await runTerminalSkewJourney({ hostBuild: current, clientBuild: current })
expectJourneyActuallyRan(record)
expectWireCompatible(record)
},
SUITE_TIMEOUT_MS
)
it(
'old client against new server completes the journey',
async () => {
const record = await runTerminalSkewJourney({ hostBuild: current, clientBuild: baseline })
expect(record.clientRevision).toBe(baseline.revision)
expectJourneyActuallyRan(record)
expectWireCompatible(record)
},
SUITE_TIMEOUT_MS
)
it(
'new client against old server completes the journey',
async () => {
const record = await runTerminalSkewJourney({ hostBuild: baseline, clientBuild: current })
expect(record.hostRevision).toBe(baseline.revision)
expectJourneyActuallyRan(record)
expectWireCompatible(record)
},
SUITE_TIMEOUT_MS
)
})