mirror of
https://github.com/stablyai/orca.git
synced 2026-10-17 08:03:14 +00:00
Suite 2: hidden-reveal seq-reconciliation fuzz + two new snapshot bugs (D, E)
Property-tests the reveal seq-reconciliation byte-stitch (getChunkDataAfterSnapshot / reconcileChunkAgainstRestoredSnapshot in pty-connection.ts), mirrored exactly: N=200 seeded hide/reveal scenarios with a rich agent-TUI hidden prefix snapshot and an append-only racing tail, chunked with seq/rawLength meta, seq-domain restarts, unmetered chunks and droppedOutput markers. Asserts snapshot-at-S + reconciled tail == snapshot-of-everything (seq-neutral) and == always-visible (end-to-end). Runtime ~5s at 200; FUZZ_ITERATIONS override documented. Two NEW real snapshot-limitation garbles found while building it, both distinct from suite 1's serialize bugs and pinned as standalone it.skip repros: - Bug D: the DECSC saved-cursor register is not serialized. A hidden TUI that saves the cursor (ESC 7 / CSI s) and restores it on reveal (ESC 8 / CSI u) lands the restore at home. Repro: 'AB\x1b7\x1b[4;10HCD' + '\x1b8X' -> 'XB' vs 'ABX'. - Bug E: a snapshot taken mid-escape-sequence (a PTY read split an escape) drops the partial sequence (it's parser state, not buffer), so the tail's continuation renders literal. Repro: 'AB\x1b[3' + 'mCD' -> 'ABmCD' vs 'ABCD'. Fired on ~24% of the corpus (tolerated + counted via prefixEndsMidSequence). The append-only-tail design isolates seq reconciliation from these and the Bug C cursor cascade. Full evidence + fix directions in notes. No production changes. Co-authored-by: Orca <help@stably.ai>
This commit is contained in:
Jinwoo-H
Orca
@@ -29,22 +29,23 @@ same bytes. Any such diff is a user-visible garble on reveal.
|
||||
(predicate `bufferHasSerializeHostileWrappedRow`, tolerated + counted) or is
|
||||
listed below.
|
||||
|
||||
## Inventory (seeds 1..2000)
|
||||
## Inventory
|
||||
|
||||
| class | seeds | count | classification |
|
||||
| bug | found by | seeds | classification |
|
||||
| --- | --- | --- | --- |
|
||||
| Bug A — serialize wrap null-cell | 31, 157, 171, 207, 423, 426, 502, 801, 815, 826, 865, 881, 923, 977, 1004, 1119, 1142, 1238, 1241, 1318, 1351, 1374, 1532, 1601, 1657, 1728, 1770 | 27 | (a) real serialize bug, pre-documented + pinned |
|
||||
| Bug B — SGR bold loss (`1;22`) | 435, 770, 1321 | 3 | (a) real serialize bug — garbles style on reveal |
|
||||
| Bug C — cursor off-by-one at right margin | 454, 1696 | 2 | (a) real serialize bug — misplaces cursor on reveal |
|
||||
| A — serialize wrap null-cell | fidelity (suite 1) | 31, 157, 171, 207, 423, 426, 502, 801, 815, 826, 865, 881, 923, 977, 1004, 1119, 1142, 1238, 1241, 1318, 1351, 1374, 1532, 1601, 1657, 1728, 1770 (27 in 1..2000) | (a) real serialize bug, pre-documented + pinned |
|
||||
| B — SGR bold loss (`1;22`) | fidelity (suite 1) | 435, 770, 1321 | (a) real serialize bug — garbles style on reveal |
|
||||
| C — cursor off-by-one at right margin | fidelity (suite 1) | 454, 1696 | (a) real serialize bug — misplaces cursor on reveal |
|
||||
| D — DECSC saved-cursor lost across reveal | reconciliation (suite 2) | seed 3 | (a) real snapshot limitation — garbles a DECRC-in-tail reveal |
|
||||
| E — snapshot boundary mid-escape-sequence | reconciliation (suite 2) | seed 4 (+~24% of corpus) | (a) real snapshot limitation — continuation renders literal |
|
||||
|
||||
The default suite runs seeds 1..300, so it only hits Bug A in range (171, 207,
|
||||
31, 157) — all tolerated — which is why it passes green. The three new
|
||||
divergence *classes* are all reproduced by dedicated minimal `test.skip` repros
|
||||
so they cannot silently regress. `FUZZ_ITERATIONS=2000` (or higher) surfaces
|
||||
Bugs B and C live; the suite skips them via the same wrap-style predicate is NOT
|
||||
applicable — instead they are pinned as standalone skipped repros and excluded
|
||||
from the corpus assertion by keeping the default corpus at 300. See the
|
||||
"Corpus vs deep mode" note at the bottom.
|
||||
The suite-1 default corpus runs seeds 1..300, so it only hits Bug A in range
|
||||
(31, 157, 171, 207) — all tolerated — which is why it passes green. All five
|
||||
bug *classes* are reproduced by dedicated minimal `test.skip` repros so they
|
||||
cannot silently regress. `FUZZ_ITERATIONS=2000` (or higher) surfaces Bugs B and
|
||||
C as live corpus divergences; because they lack a cheap buffer-state predicate
|
||||
(unlike Bug A's wrap predicate) they are pinned as standalone skipped repros and
|
||||
kept out of range by the default 300 corpus. See "Corpus vs deep mode" below.
|
||||
|
||||
Seed 113 (called out in the handoff as a "DECSC/DECRC detour writing colored
|
||||
text mid-line") does not diverge on the current harness. It is a `savedCursor
|
||||
@@ -149,6 +150,100 @@ caret or spinner, and subsequent input can overwrite the wrong cell.
|
||||
|
||||
---
|
||||
|
||||
## Bug D — snapshot does not preserve the DECSC saved-cursor register across a hide/reveal boundary
|
||||
|
||||
**Classification: (a) real bug — a structural snapshot limitation.** New
|
||||
finding, surfaced by the reveal-reconciliation fuzz (suite 2), not the fidelity
|
||||
fuzz.
|
||||
|
||||
**Minimal repro (cols=20):**
|
||||
|
||||
```
|
||||
hidden bytes: "AB\x1b7\x1b[4;10HCD" (write AB, DECSC saves cursor at r0c2,
|
||||
move to r3c9, write CD)
|
||||
tail bytes: "\x1b8X" (DECRC restores the saved cursor, write X)
|
||||
|
||||
live (always visible): rows ["ABX", " CD"] cursor { x: 3, y: 0 }
|
||||
reveal (snapshot+tail): rows ["XB", " CD"] cursor { x: 1, y: 0 }
|
||||
^^ 'X' overwrote 'A' — DECRC landed at home, not r0c2
|
||||
snapshotAnsi: "AB\r\n\r\n\r\n\x1b[9CCD" (no saved-cursor state at all)
|
||||
```
|
||||
|
||||
**Mechanism:** the snapshot is a serialized *screen* (SerializeAddon) plus a few
|
||||
rehydrated modes. The VT100 DECSC/DECRC saved-cursor register (also `CSI s` /
|
||||
`CSI u`) is runtime state that never appears in the serialized buffer, so it
|
||||
cannot survive a snapshot. When a hidden TUI runs `\x1b7` (or `\x1b[s`) before
|
||||
the reveal seq and the racing tail (or any post-reveal output) runs `\x1b8` (or
|
||||
`\x1b[u`), the restore targets the fresh terminal's default saved position
|
||||
(home) instead of where the TUI saved it — the next writes land at the wrong
|
||||
cell and overwrite live content.
|
||||
|
||||
**Why it garbles a real pane:** DECSC/DECRC is common in shell prompts and
|
||||
status-line redraws (save cursor, jump to a corner to paint a clock/token
|
||||
counter, restore). If the save happens while the pane is hidden and the restore
|
||||
fires on reveal, the restored paint clobbers the wrong cells. Found by suite-2
|
||||
seed 3 (a `savedCursorDetour` op whose `\x1b7` fell in the hidden prefix and
|
||||
whose `\x1b8` fell in the racing tail after chunk-splitting).
|
||||
|
||||
**Handling:** suite 2 keeps its racing tail append-only (no DECSC/DECRC, cursor
|
||||
motion, scroll regions, or alt frames) so the seq-reconciliation byte-stitch is
|
||||
tested in isolation from this and the other terminal-state-loss garbles. Bug D
|
||||
is instead pinned as a standalone repro,
|
||||
`hidden-reveal-reconciliation.fuzz.test.ts` →
|
||||
`it.skip('preserves the DECSC saved-cursor register across a hide/reveal …')`.
|
||||
|
||||
**Fix direction (not applied — no production changes in this task):** carry the
|
||||
saved-cursor register out-of-band in the snapshot (like `oscLinks`), or have the
|
||||
emulator re-emit a synthetic DECSC restoring the saved position after replay.
|
||||
|
||||
---
|
||||
|
||||
## Bug E — snapshot boundary mid-escape-sequence drops the partial sequence
|
||||
|
||||
**Classification: (a) real bug — a structural snapshot limitation.** New
|
||||
finding, surfaced by the reveal-reconciliation fuzz (suite 2).
|
||||
|
||||
**Minimal repro (cols=20):**
|
||||
|
||||
```
|
||||
hidden prefix: "AB\x1b[3" (write AB, then ESC [ 3 — no final byte yet)
|
||||
tail bytes: "mCD" ('m' completes ESC[3m = italic, then CD)
|
||||
|
||||
live (always visible): rows ["ABCD"] (ESC[3m parsed atomically, CD italic)
|
||||
reveal (snapshot+tail): rows ["ABmCD"] ← 'm' became a literal character
|
||||
snapshotAnsi: "AB" (the partial ESC[3 is in the parser, gone)
|
||||
```
|
||||
|
||||
**Mechanism:** a PTY read (one delivery record) can split an escape sequence.
|
||||
If the pane is revealed while the emulator's parser sits mid-`ESC[…`, the
|
||||
serialized SCREEN cannot carry the partial sequence (it lives in the parser
|
||||
state machine, not the buffer). The racing tail supplies the sequence's
|
||||
remaining bytes, but with the prefix gone the terminal parses them as literal
|
||||
text. Reproduced end-to-end against the real `HeadlessEmulator.getSnapshot`.
|
||||
|
||||
**Why it garbles a real pane:** any TUI whose output is heavy with escape
|
||||
sequences (all of them) can have a read boundary fall mid-escape; if a reveal
|
||||
lands in that window the continuation renders as stray literal bytes (a rogue
|
||||
`m`, `H`, digits) injected into the visible text.
|
||||
|
||||
**Reachability:** requires the reveal/snapshot to fire in the gap between the two
|
||||
halves of a split escape. `main` writes each PTY read to the emulator and
|
||||
records it as one delivery unit (`session.ts emitSubprocessOutput`), and the
|
||||
snapshot is taken synchronously at a drain — so the window is a single delivered
|
||||
record that ended mid-escape. Narrow but real.
|
||||
|
||||
**Handling:** suite 2 tolerates + counts scenarios whose hidden prefix ends
|
||||
mid-escape-sequence (`prefixEndsMidSequence`), the same way suite 1 tolerates the
|
||||
serialize wrap bug — it fired on ~24% of the corpus, confirming the class is
|
||||
common. Pinned by `hidden-reveal-reconciliation.fuzz.test.ts` →
|
||||
`it.skip('completes an escape sequence split across the hide/reveal boundary')`.
|
||||
|
||||
**Fix direction (not applied):** have `main` hold a trailing incomplete escape
|
||||
out of the drain until it completes (a small parser-aware buffer), or defer the
|
||||
snapshot to a parser-clean boundary.
|
||||
|
||||
---
|
||||
|
||||
## Known-legitimate normalization (NOT bugs)
|
||||
|
||||
- **OSC 8 hyperlink underline** — classification (c). xterm marks OSC-8 link
|
||||
|
||||
@@ -0,0 +1,577 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import {
|
||||
buildAgentTuiStreamOps,
|
||||
mulberry32,
|
||||
splitIntoRandomChunks,
|
||||
type AgentTuiStreamDims
|
||||
} from '../../../../shared/agent-tui-ansi-fuzz-stream'
|
||||
import {
|
||||
POST_REPLAY_LIVE_SNAPSHOT_RESET_PARITY,
|
||||
SNAPSHOT_REPLAY_PREAMBLE_ALT,
|
||||
SNAPSHOT_REPLAY_PREAMBLE_NORMAL,
|
||||
buildParityMainBufferSnapshot,
|
||||
createRendererParityTerminal,
|
||||
cursorPosition,
|
||||
normalBufferRowsTrimmed,
|
||||
visibleRows,
|
||||
visibleRowStyles,
|
||||
writeChunksToTerminal
|
||||
} from '../../../../shared/terminal-restore-parity-fixture'
|
||||
|
||||
// Property fuzz for the hidden-reveal seq-reconciliation path in
|
||||
// pty-connection.ts. When a pane is hidden, main keeps metering PTY output as
|
||||
// seq'd chunks. On reveal the renderer applies a HeadlessEmulator snapshot
|
||||
// captured at seq S (paints everything <= S), then stitches the racing tail
|
||||
// (chunks straddling / after S) on top WITHOUT duplicating or losing a byte.
|
||||
// The byte arithmetic lives in two non-exported closures:
|
||||
// - getChunkDataAfterSnapshot (pty-connection.ts ~L4629): slices a
|
||||
// pending chunk against the snapshot seq.
|
||||
// - reconcileChunkAgainstRestoredSnapshot (pty-connection.ts ~L4660): dedups
|
||||
// backlog, detects a restarted seq domain, and heals dropped-output gaps by
|
||||
// forcing a fresh restore.
|
||||
// This suite mirrors those rules EXACTLY (each branch tagged with its source
|
||||
// line) and asserts the invariant they guarantee: for any reveal boundary,
|
||||
// snapshot(hidden prefix) + reconciled tail reproduces the same screen an
|
||||
// always-visible terminal shows for the full stream. A wrong slice = a garble.
|
||||
//
|
||||
// Runtime knobs:
|
||||
// FUZZ_ITERATIONS=2000 deep/nightly (default 200, <60s with the fidelity suite)
|
||||
// FUZZ_SEED=1234 re-run exactly one seed
|
||||
|
||||
const DEFAULT_ITERATIONS = 200
|
||||
const FIXED_SEED = readPositiveIntEnv('FUZZ_SEED')
|
||||
const ITERATIONS =
|
||||
FIXED_SEED !== null ? 1 : (readPositiveIntEnv('FUZZ_ITERATIONS') ?? DEFAULT_ITERATIONS)
|
||||
|
||||
function readPositiveIntEnv(name: string): number | null {
|
||||
const raw = Number(process.env[name])
|
||||
return Number.isFinite(raw) && raw > 0 ? Math.floor(raw) : null
|
||||
}
|
||||
|
||||
const DIMS: readonly AgentTuiStreamDims[] = [
|
||||
{ cols: 80, rows: 24 },
|
||||
{ cols: 100, rows: 30 }
|
||||
]
|
||||
|
||||
/** A metered chunk exactly as main delivers to onData: text, the seq of the
|
||||
* LAST raw byte, the raw (pre-OSC-strip) length, and delivery-domain markers.
|
||||
* domain increments across a ptyId-exit seq restart (production clears the
|
||||
* restored baseline at that boundary, so the tail writes verbatim). */
|
||||
type MeteredChunk = {
|
||||
data: string
|
||||
seq?: number
|
||||
rawLength?: number
|
||||
domain: number
|
||||
droppedOutput?: boolean
|
||||
}
|
||||
|
||||
/** Mirror of getChunkDataAfterSnapshot (pty-connection.ts ~L4629): how much of
|
||||
* a chunk in the SNAPSHOT'S seq domain survives once the snapshot at
|
||||
* snapshotSeq has painted everything <= snapshotSeq. null = "cannot slice,
|
||||
* force a fresh snapshot" (OSC stripping desynced raw offsets). Byte-identical
|
||||
* to production; each return tagged with its branch. */
|
||||
function chunkDataAfterSnapshot(
|
||||
chunk: MeteredChunk,
|
||||
snapshotSeq: number | undefined
|
||||
): string | null {
|
||||
if (typeof snapshotSeq !== 'number' || typeof chunk.seq !== 'number') {
|
||||
return chunk.data // L4633: unmetered — pass through
|
||||
}
|
||||
const rawLength = chunk.rawLength ?? chunk.data.length
|
||||
const startSeq = chunk.seq - rawLength
|
||||
if (snapshotSeq >= chunk.seq) {
|
||||
return '' // L4638: fully before/at snapshot — already painted
|
||||
}
|
||||
if (snapshotSeq <= startSeq) {
|
||||
return chunk.data // L4641: fully after snapshot — keep whole
|
||||
}
|
||||
const offset = snapshotSeq - startSeq
|
||||
if (rawLength !== chunk.data.length) {
|
||||
return null // L4645: OSC-stripped, offsets unmappable
|
||||
}
|
||||
return chunk.data.slice(offset) // L4648: straddles — drop the painted prefix
|
||||
}
|
||||
|
||||
type RevealResult = {
|
||||
/** Full normal-buffer content (visible + scrollback, trailing blanks
|
||||
* trimmed). Viewport-independent: a byte lost or duplicated by reconciliation
|
||||
* changes this even when the visible viewport happens to line up. */
|
||||
content: string[]
|
||||
rows: string[]
|
||||
styles: string[]
|
||||
cursor: { x: number; y: number }
|
||||
/** True when the buffer scrolled (baseY > 0). The visible viewport's top
|
||||
* anchor after a snapshot restore vs continuous writing can differ by a row
|
||||
* or two purely from trailing-blank trimming — a snapshot-scrollback-depth
|
||||
* concern owned by the fidelity suite, not seq reconciliation — so styles and
|
||||
* cursor (viewport-relative) are only asserted on non-scrolled scenarios. */
|
||||
scrolled: boolean
|
||||
/** True when the terminal ended on the alternate screen. Alt has no
|
||||
* scrollback and a fixed viewport, so its content is the visible rows and the
|
||||
* normal-buffer content comparison does not apply (the alt-screen restore
|
||||
* contract is scrollback-free by design — serializeHeadlessTerminalBuffer). */
|
||||
alternate: boolean
|
||||
forcedFreshRestore: boolean
|
||||
}
|
||||
|
||||
/** Bug E detector: does the hidden prefix (what the snapshot captured) end
|
||||
* mid-escape-sequence? A PTY read (one delivery record) can split an escape;
|
||||
* the emulator then holds partial parser state that the serialized SCREEN
|
||||
* cannot carry, so the racing tail's continuation bytes render as literal text.
|
||||
* Returns true when the prefix ends with a dangling ESC, ESC[…(no final byte),
|
||||
* ESC](no ST/BEL), or a lone ESC-intermediate. */
|
||||
function prefixEndsMidSequence(prefix: string): boolean {
|
||||
const esc = prefix.lastIndexOf('\x1b')
|
||||
if (esc === -1) {
|
||||
return false
|
||||
}
|
||||
const rest = prefix.slice(esc)
|
||||
if (rest === '\x1b') {
|
||||
return true // dangling ESC
|
||||
}
|
||||
const kind = rest[1]
|
||||
if (kind === '[') {
|
||||
// CSI: complete iff a final byte 0x40–0x7e appears after the params.
|
||||
for (let i = 2; i < rest.length; i++) {
|
||||
const code = rest.charCodeAt(i)
|
||||
if (code >= 0x40 && code <= 0x7e) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
if (kind === ']') {
|
||||
// OSC: complete iff terminated by BEL or ST (ESC \).
|
||||
return !(rest.includes('\x07') || rest.includes('\x1b\\'))
|
||||
}
|
||||
// ESC 7 / ESC 8 / ESC-intermediate: two bytes complete it. Dangling iff only
|
||||
// ESC was written (handled above) — a present kind byte means it completed.
|
||||
return false
|
||||
}
|
||||
|
||||
async function readScreen(
|
||||
term: ReturnType<typeof createRendererParityTerminal>
|
||||
): Promise<RevealResult> {
|
||||
const alternate = term.terminal.buffer.active.type === 'alternate'
|
||||
return {
|
||||
content: alternate ? visibleRows(term.terminal) : normalBufferRowsTrimmed(term.terminal),
|
||||
rows: visibleRows(term.terminal),
|
||||
styles: visibleRowStyles(term.terminal),
|
||||
cursor: cursorPosition(term.terminal),
|
||||
scrolled: term.terminal.buffer.active.baseY > 0,
|
||||
alternate,
|
||||
forcedFreshRestore: false
|
||||
}
|
||||
}
|
||||
|
||||
/** The renderer's screen for a hide→reveal cycle. `revealIdx` is the delivery
|
||||
* index at which the pane is revealed: chunks [0, revealIdx) were captured by
|
||||
* the HeadlessEmulator snapshot; chunks [revealIdx, end) are the racing tail.
|
||||
* The snapshot seq is the seq of the last hidden chunk in the snapshot's
|
||||
* domain; the tail is stitched via the production slice/reconcile rules. */
|
||||
async function revealFromSnapshot(
|
||||
dims: AgentTuiStreamDims,
|
||||
chunks: MeteredChunk[],
|
||||
revealIdx: number
|
||||
): Promise<RevealResult> {
|
||||
const source = createRendererParityTerminal(dims)
|
||||
const restored = createRendererParityTerminal(dims)
|
||||
try {
|
||||
// Snapshot source = every hidden chunk painted in order.
|
||||
await writeChunksToTerminal(
|
||||
source.terminal,
|
||||
chunks.slice(0, revealIdx).map((c) => c.data)
|
||||
)
|
||||
// Snapshot seq = seq of the last hidden chunk that carried one (the seq the
|
||||
// emulator would report). undefined when the hidden prefix was unmetered.
|
||||
let snapshotSeq: number | undefined
|
||||
let snapshotDomain = 0
|
||||
for (let i = revealIdx - 1; i >= 0; i--) {
|
||||
if (typeof chunks[i]!.seq === 'number') {
|
||||
snapshotSeq = chunks[i]!.seq
|
||||
snapshotDomain = chunks[i]!.domain
|
||||
break
|
||||
}
|
||||
}
|
||||
const snapshot = buildParityMainBufferSnapshot(source, snapshotSeq ?? 0)
|
||||
const alt = snapshot.alternateScreen
|
||||
await writeChunksToTerminal(restored.terminal, [
|
||||
alt ? SNAPSHOT_REPLAY_PREAMBLE_ALT : SNAPSHOT_REPLAY_PREAMBLE_NORMAL,
|
||||
snapshot.data,
|
||||
POST_REPLAY_LIVE_SNAPSHOT_RESET_PARITY
|
||||
])
|
||||
|
||||
// Stitch the tail. A chunk in the snapshot's domain is sliced against the
|
||||
// snapshot seq (drains the painted prefix); a chunk from a later domain
|
||||
// (post-exit seq restart) writes verbatim — production clears the baseline
|
||||
// at the ptyId-exit boundary, so no seq dedupe applies across domains.
|
||||
let forcedFreshRestore = false
|
||||
for (let i = revealIdx; i < chunks.length; i++) {
|
||||
const chunk = chunks[i]!
|
||||
if (chunk.domain !== snapshotDomain) {
|
||||
await writeChunksToTerminal(restored.terminal, [chunk.data])
|
||||
continue
|
||||
}
|
||||
const sliced = chunkDataAfterSnapshot(chunk, snapshotSeq)
|
||||
if (sliced === null) {
|
||||
// Production re-fetches a fresh snapshot here; a fresh snapshot of the
|
||||
// full stream trivially matches the always-visible screen, so mark the
|
||||
// scenario as a fresh-restore (excluded from the stitch comparison).
|
||||
forcedFreshRestore = true
|
||||
continue
|
||||
}
|
||||
if (sliced) {
|
||||
await writeChunksToTerminal(restored.terminal, [sliced])
|
||||
}
|
||||
}
|
||||
const result = await readScreen(restored)
|
||||
result.forcedFreshRestore = forcedFreshRestore
|
||||
return result
|
||||
} finally {
|
||||
source.terminal.dispose()
|
||||
restored.terminal.dispose()
|
||||
}
|
||||
}
|
||||
|
||||
async function alwaysVisible(
|
||||
dims: AgentTuiStreamDims,
|
||||
chunks: MeteredChunk[]
|
||||
): Promise<RevealResult> {
|
||||
const term = createRendererParityTerminal(dims)
|
||||
try {
|
||||
await writeChunksToTerminal(
|
||||
term.terminal,
|
||||
chunks.map((c) => c.data)
|
||||
)
|
||||
return await readScreen(term)
|
||||
} finally {
|
||||
term.terminal.dispose()
|
||||
}
|
||||
}
|
||||
|
||||
/** Reference for the seq-reconciliation gate: a snapshot of the WHOLE stream
|
||||
* (reveal at the very end, no racing tail). Sharing the snapshot machinery with
|
||||
* revealFromSnapshot cancels out snapshot-fidelity gaps (Bug C cursor restore,
|
||||
* Bug B bold loss — owned by the fidelity suite), so a diff against a mid-point
|
||||
* reveal is attributable purely to the tail-stitch seq arithmetic. */
|
||||
async function fullSnapshotReference(
|
||||
dims: AgentTuiStreamDims,
|
||||
chunks: MeteredChunk[]
|
||||
): Promise<RevealResult> {
|
||||
return revealFromSnapshot(dims, chunks, chunks.length)
|
||||
}
|
||||
|
||||
type Scenario = {
|
||||
seed: number
|
||||
dims: AgentTuiStreamDims
|
||||
chunks: MeteredChunk[]
|
||||
revealIdx: number
|
||||
domains: number
|
||||
hasDropped: boolean
|
||||
}
|
||||
|
||||
const TAIL_WORDS = [
|
||||
'reading',
|
||||
'tokens 12.4k',
|
||||
'你好世界',
|
||||
'터미널',
|
||||
'✅ done',
|
||||
'PASS test.ts',
|
||||
'+142 -37',
|
||||
'diff --git'
|
||||
] as const
|
||||
const TAIL_SGR = ['\x1b[0m', '\x1b[31m', '\x1b[1m', '\x1b[38;5;204m', '\x1b[7m'] as const
|
||||
|
||||
/** Append-only racing tail: SGR runs + text + newlines only. No absolute/
|
||||
* relative cursor motion, scroll regions, alt frames, or DECSC — so the tail
|
||||
* paints wherever the cursor sits and its screen effect is a pure function of
|
||||
* which bytes the seq-reconciliation applies. Terminal-state-loss garbles
|
||||
* (Bugs C/D/E, scroll region) are pinned separately; this suite isolates the
|
||||
* seq slice. */
|
||||
function buildAppendOnlyTail(rng: () => number, lines: number): string {
|
||||
const out: string[] = []
|
||||
for (let i = 0; i < lines; i++) {
|
||||
const words: string[] = []
|
||||
const count = 1 + Math.floor(rng() * 3)
|
||||
for (let w = 0; w < count; w++) {
|
||||
words.push(TAIL_WORDS[Math.floor(rng() * TAIL_WORDS.length)]!)
|
||||
}
|
||||
const sgr = TAIL_SGR[Math.floor(rng() * TAIL_SGR.length)]!
|
||||
out.push(`${sgr}${words.join(' ')}\x1b[0m\r\n`)
|
||||
}
|
||||
return out.join('')
|
||||
}
|
||||
|
||||
/** Builds a seeded metered stream with a random reveal boundary. A running seq
|
||||
* counter tags the LAST byte of each chunk; at most ONE mid-stream restart
|
||||
* bumps the domain and resets the counter low (a revived ptyId with a fresh
|
||||
* main-side counter — production clears the restored baseline at that exit
|
||||
* boundary, so the post-restart tail writes verbatim). Rare unmetered chunks
|
||||
* (no seq) and droppedOutput markers model no-metering runtimes and pending-cap
|
||||
* trims. Op counts are kept modest so most scenarios reach the full
|
||||
* non-scrolled comparison (the tail-stitch invariant); scrolled scenarios fall
|
||||
* back to the snapshot-depth-tolerant path (see the assertion loop). */
|
||||
function buildScenario(seed: number): Scenario {
|
||||
const rng = mulberry32(seed)
|
||||
const dims = DIMS[Math.floor(rng() * DIMS.length)]!
|
||||
// Hidden prefix: full agent-TUI churn (cursor motion, panels, alt frames,
|
||||
// scroll regions, DECSC…) so the SNAPSHOT is taken over rich state.
|
||||
const prefixOps = buildAgentTuiStreamOps(rng, dims, {
|
||||
includeMouseModes: false,
|
||||
includeOscHyperlinks: false,
|
||||
opCount: 3 + Math.floor(rng() * 6)
|
||||
})
|
||||
const prefixStream = prefixOps.join('')
|
||||
const prefixRaw = splitIntoRandomChunks(mulberry32(seed ^ 0x51ed270b), prefixStream, {
|
||||
minLen: 2,
|
||||
maxLen: 64
|
||||
})
|
||||
// Racing tail: append-only content. Isolates the seq-reconciliation byte
|
||||
// stitch from terminal-state-loss garbles (Bugs C/D/E — pinned separately).
|
||||
const tailStream = buildAppendOnlyTail(rng, 2 + Math.floor(rng() * 5))
|
||||
const tailRaw = splitIntoRandomChunks(mulberry32(seed ^ 0x2545f491), tailStream, {
|
||||
minLen: 2,
|
||||
maxLen: 40
|
||||
})
|
||||
|
||||
const chunks: MeteredChunk[] = []
|
||||
let seq = 100 + Math.floor(rng() * 50)
|
||||
let domain = 0
|
||||
let hasDropped = false
|
||||
// At most one restart, placed inside the tail so the snapshot's own domain
|
||||
// has a stable prefix — mirrors a single mid-session ptyId revival.
|
||||
const restartAt =
|
||||
rng() < 0.25 ? prefixRaw.length + Math.floor(rng() * Math.max(1, tailRaw.length)) : -1
|
||||
const allRaw = [...prefixRaw, ...tailRaw]
|
||||
for (let i = 0; i < allRaw.length; i++) {
|
||||
const data = allRaw[i]!
|
||||
if (i === restartAt) {
|
||||
seq = 10 + Math.floor(rng() * 20)
|
||||
domain = 1
|
||||
}
|
||||
const rawLength = data.length
|
||||
seq += rawLength
|
||||
const chunk: MeteredChunk = { data, seq, rawLength, domain }
|
||||
// Rare unmetered / dropped markers, only in the tail (the prefix must meter
|
||||
// so the snapshot carries a seq).
|
||||
if (i >= prefixRaw.length && rng() < 0.05) {
|
||||
delete chunk.seq
|
||||
delete chunk.rawLength
|
||||
}
|
||||
if (i >= prefixRaw.length && rng() < 0.04) {
|
||||
chunk.droppedOutput = true
|
||||
hasDropped = true
|
||||
}
|
||||
chunks.push(chunk)
|
||||
}
|
||||
|
||||
// Reveal at or after the prefix ends, so the racing tail is purely the
|
||||
// append-only region — the snapshot captures all cursor-motion/scroll/alt
|
||||
// state and the tail cannot depend on unserialized terminal state. Landing
|
||||
// the reveal at various points INSIDE the append-only tail exercises the
|
||||
// straddle and "snapshot seq inside the tail" seq-slice cases.
|
||||
const tailSpan = Math.max(1, chunks.length - prefixRaw.length)
|
||||
const revealIdx = Math.min(chunks.length, prefixRaw.length + Math.floor(rng() * tailSpan))
|
||||
return { seed, dims, chunks, revealIdx, domains: domain + 1, hasDropped }
|
||||
}
|
||||
|
||||
function firstRowDiff(a: string[], b: string[]): number | null {
|
||||
const n = Math.max(a.length, b.length)
|
||||
for (let i = 0; i < n; i++) {
|
||||
if ((a[i] ?? '') !== (b[i] ?? '')) {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
function formatFailure(s: Scenario, stage: string, expected: unknown, actual: unknown): string {
|
||||
return [
|
||||
`hidden-reveal reconciliation divergence — stage: ${stage}`,
|
||||
`seed: ${s.seed} (re-run: FUZZ_SEED=${s.seed})`,
|
||||
`dims: ${s.dims.cols}x${s.dims.rows} revealIdx: ${s.revealIdx}/${s.chunks.length} domains: ${s.domains} dropped: ${s.hasDropped}`,
|
||||
`chunks: ${JSON.stringify(
|
||||
s.chunks.map((c) => ({ seq: c.seq, len: c.data.length, dom: c.domain }))
|
||||
)}`,
|
||||
`expected (always-visible): ${JSON.stringify(expected)}`,
|
||||
`actual (reveal-from-snapshot): ${JSON.stringify(actual)}`
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
describe('hidden reveal seq-reconciliation fuzz', () => {
|
||||
it('is a byte-exact identity when the snapshot seq splits a straddling chunk', async () => {
|
||||
// Pin the core invariant on a hand-built case so the property test cannot
|
||||
// pass vacuously. rawLength === data.length so the straddle slices cleanly.
|
||||
// Chunk 0 fully hidden; chunk 1 straddles the reveal; chunk 2 is the tail.
|
||||
const dims = { cols: 40, rows: 6 }
|
||||
const c0 = 'red line one\r\n'
|
||||
const c1 = 'green straddle\r\n'
|
||||
const c2 = 'plain tail rest'
|
||||
const chunks: MeteredChunk[] = [
|
||||
{ data: c0, seq: c0.length, rawLength: c0.length, domain: 0 },
|
||||
{ data: c1, seq: c0.length + c1.length, rawLength: c1.length, domain: 0 },
|
||||
{ data: c2, seq: c0.length + c1.length + c2.length, rawLength: c2.length, domain: 0 }
|
||||
]
|
||||
// Reveal after chunk 1 (snapshot seq = end of c1); tail = c2. Also exercise
|
||||
// the straddle by revealing at chunk 1 with a snapshot seq mid-c1 handled by
|
||||
// chunkDataAfterSnapshot when c1 is in the tail — covered by revealIdx=1.
|
||||
const revealAfterC1 = await revealFromSnapshot(dims, chunks, 2)
|
||||
const control = await alwaysVisible(dims, chunks)
|
||||
expect(revealAfterC1.rows).toEqual(control.rows)
|
||||
expect(revealAfterC1.styles).toEqual(control.styles)
|
||||
expect(revealAfterC1.cursor).toEqual(control.cursor)
|
||||
|
||||
// Reveal at chunk 1: c0 hidden (snapshot seq = end of c0), c1+c2 are the
|
||||
// tail. c1 is fully after the snapshot seq so it writes whole — identity.
|
||||
const revealAtC1 = await revealFromSnapshot(dims, chunks, 1)
|
||||
expect(revealAtC1.rows).toEqual(control.rows)
|
||||
expect(revealAtC1.styles).toEqual(control.styles)
|
||||
expect(revealAtC1.cursor).toEqual(control.cursor)
|
||||
})
|
||||
|
||||
// ── Bug D: DECSC saved-cursor register lost across the hide/reveal boundary ──
|
||||
// The snapshot is a serialized screen; the VT100 saved-cursor register
|
||||
// (DECSC ESC 7 / CSI s) is runtime state it cannot carry. When a hidden TUI
|
||||
// saves the cursor before the reveal seq and the racing tail restores it
|
||||
// (DECRC ESC 8 / CSI u), the restore lands at home instead — the next writes
|
||||
// clobber the wrong cells. Found by fuzz seed 3 (a savedCursorDetour op split
|
||||
// across the reveal boundary). See notes/garble-fuzz-divergences.md (Bug D).
|
||||
// Fix direction: carry the saved-cursor out-of-band in the snapshot (like
|
||||
// oscLinks) or re-emit a synthetic DECSC after replay. Unskip once landed.
|
||||
it.skip('preserves the DECSC saved-cursor register across a hide/reveal boundary', async () => {
|
||||
const dims = { cols: 20, rows: 4 }
|
||||
// Hidden: write 'AB', DECSC saves cursor at r0c2, move to r3c9, write 'CD'.
|
||||
const hidden: MeteredChunk = {
|
||||
data: 'AB\x1b7\x1b[4;10HCD',
|
||||
seq: 'AB\x1b7\x1b[4;10HCD'.length,
|
||||
rawLength: 'AB\x1b7\x1b[4;10HCD'.length,
|
||||
domain: 0
|
||||
}
|
||||
// Tail: DECRC restores the saved cursor, write 'X' → live shows 'ABX'.
|
||||
const tail: MeteredChunk = {
|
||||
data: '\x1b8X',
|
||||
seq: hidden.seq! + 3,
|
||||
rawLength: 3,
|
||||
domain: 0
|
||||
}
|
||||
const reveal = await revealFromSnapshot(dims, [hidden, tail], 1)
|
||||
const live = await alwaysVisible(dims, [hidden, tail])
|
||||
// Fails today: reveal shows 'XB' (DECRC landed at home) not live's 'ABX'.
|
||||
expect(reveal.rows).toEqual(live.rows)
|
||||
expect(reveal.cursor).toEqual(live.cursor)
|
||||
})
|
||||
|
||||
// ── Bug E: snapshot boundary mid-escape-sequence drops the partial sequence ──
|
||||
// A PTY read (one delivery record) can split an escape. If the pane is
|
||||
// revealed with the emulator parser mid-ESC[…, the serialized SCREEN cannot
|
||||
// carry the partial sequence, so the racing tail's continuation bytes render
|
||||
// as literal text. Found by fuzz seed 4 (a hidden prefix chunk that ended on
|
||||
// ESC[3, tail supplied 'm'). See notes/garble-fuzz-divergences.md (Bug E).
|
||||
// Fix direction: have main hold a trailing incomplete escape out of the drain
|
||||
// until it completes, or snapshot only at a parser-clean boundary. Unskip once
|
||||
// landed.
|
||||
it.skip('completes an escape sequence split across the hide/reveal boundary', async () => {
|
||||
const dims = { cols: 20, rows: 3 }
|
||||
// Hidden prefix ends mid-escape: 'AB' then ESC[3 (no final byte).
|
||||
const hidden: MeteredChunk = { data: 'AB\x1b[3', seq: 5, rawLength: 5, domain: 0 }
|
||||
// Tail completes it: 'm' → ESC[3m (italic), then 'CD' italic. Live: 'ABCD'.
|
||||
const tail: MeteredChunk = { data: 'mCD', seq: 8, rawLength: 3, domain: 0 }
|
||||
const reveal = await revealFromSnapshot(dims, [hidden, tail], 1)
|
||||
const live = await alwaysVisible(dims, [hidden, tail])
|
||||
// Fails today: reveal shows 'ABmCD' (the 'm' became literal).
|
||||
expect(reveal.rows).toEqual(live.rows)
|
||||
})
|
||||
|
||||
it(`stitches the racing tail losslessly across ${ITERATIONS} seeded hide/reveal scenarios`, async () => {
|
||||
let statsCompared = 0
|
||||
let statsSkippedScrolled = 0
|
||||
let statsForcedFresh = 0
|
||||
let statsMidSequenceBugE = 0
|
||||
for (let i = 0; i < ITERATIONS; i++) {
|
||||
const seed = FIXED_SEED ?? 1 + i
|
||||
const scenario = buildScenario(seed)
|
||||
// Bug E: the snapshot boundary falls mid-escape-sequence (a PTY read
|
||||
// split an escape). The partial sequence lives in the emulator parser,
|
||||
// not the serialized screen, so the tail's continuation renders literal.
|
||||
// Tolerated + counted; pinned by the skipped repro below.
|
||||
const hiddenPrefix = scenario.chunks
|
||||
.slice(0, scenario.revealIdx)
|
||||
.map((c) => c.data)
|
||||
.join('')
|
||||
if (prefixEndsMidSequence(hiddenPrefix)) {
|
||||
statsMidSequenceBugE += 1
|
||||
continue
|
||||
}
|
||||
// Primary control: a snapshot of the WHOLE stream. Sharing the snapshot
|
||||
// machinery isolates the seq-reconciliation tail-stitch from
|
||||
// snapshot-fidelity gaps (fidelity suite's Bugs B/C). If snapshot-at-S +
|
||||
// tail differs from snapshot-of-everything, a byte was lost/duped/mis-
|
||||
// sliced. Also compare against always-visible where the two agree, to
|
||||
// catch a tail that diverges from live output.
|
||||
const [reveal, reference, live] = await Promise.all([
|
||||
revealFromSnapshot(scenario.dims, scenario.chunks, scenario.revealIdx),
|
||||
fullSnapshotReference(scenario.dims, scenario.chunks),
|
||||
alwaysVisible(scenario.dims, scenario.chunks)
|
||||
])
|
||||
if (reveal.forcedFreshRestore || reference.forcedFreshRestore) {
|
||||
statsForcedFresh += 1
|
||||
continue
|
||||
}
|
||||
if (reveal.alternate !== reference.alternate) {
|
||||
expect.fail(
|
||||
formatFailure(scenario, 'alt-screen-state', reference.alternate, reveal.alternate)
|
||||
)
|
||||
}
|
||||
// Scrolled scenarios: top-row survival is a snapshot-depth question
|
||||
// (fidelity suite), not seq reconciliation.
|
||||
if (reveal.scrolled || reference.scrolled) {
|
||||
statsSkippedScrolled += 1
|
||||
continue
|
||||
}
|
||||
statsCompared += 1
|
||||
// Seq-reconciliation gate (snapshot-fidelity-neutral).
|
||||
if (firstRowDiff(reveal.content, reference.content) !== null) {
|
||||
expect.fail(
|
||||
formatFailure(scenario, 'tail-stitch-content', reference.content, reveal.content)
|
||||
)
|
||||
}
|
||||
if (firstRowDiff(reveal.rows, reference.rows) !== null) {
|
||||
expect.fail(formatFailure(scenario, 'tail-stitch-visible', reference.rows, reveal.rows))
|
||||
}
|
||||
if (firstRowDiff(reveal.styles, reference.styles) !== null) {
|
||||
expect.fail(formatFailure(scenario, 'tail-stitch-styles', reference.styles, reveal.styles))
|
||||
}
|
||||
if (JSON.stringify(reveal.cursor) !== JSON.stringify(reference.cursor)) {
|
||||
expect.fail(formatFailure(scenario, 'tail-stitch-cursor', reference.cursor, reveal.cursor))
|
||||
}
|
||||
// Stronger gate: when a full-stream SNAPSHOT already matches the
|
||||
// always-visible LIVE screen (no fidelity gap in play), the mid-point
|
||||
// reveal must match live too — a true end-to-end garble check.
|
||||
if (
|
||||
!live.scrolled &&
|
||||
live.alternate === reference.alternate &&
|
||||
firstRowDiff(reference.rows, live.rows) === null &&
|
||||
firstRowDiff(reference.styles, live.styles) === null &&
|
||||
JSON.stringify(reference.cursor) === JSON.stringify(live.cursor)
|
||||
) {
|
||||
if (firstRowDiff(reveal.rows, live.rows) !== null) {
|
||||
expect.fail(formatFailure(scenario, 'live-visible', live.rows, reveal.rows))
|
||||
}
|
||||
if (firstRowDiff(reveal.styles, live.styles) !== null) {
|
||||
expect.fail(formatFailure(scenario, 'live-styles', live.styles, reveal.styles))
|
||||
}
|
||||
if (JSON.stringify(reveal.cursor) !== JSON.stringify(live.cursor)) {
|
||||
expect.fail(formatFailure(scenario, 'live-cursor', live.cursor, reveal.cursor))
|
||||
}
|
||||
}
|
||||
}
|
||||
// Guard against a degenerate corpus that skips its way to green: a healthy
|
||||
// fraction of scenarios must reach the full non-scrolled comparison.
|
||||
expect(statsCompared).toBeGreaterThan(ITERATIONS * 0.2)
|
||||
// The mid-escape (Bug E) tolerance must not swallow the suite.
|
||||
expect(statsMidSequenceBugE).toBeLessThan(Math.max(3, ITERATIONS * 0.5))
|
||||
expect(
|
||||
statsCompared + statsSkippedScrolled + statsForcedFresh + statsMidSequenceBugE
|
||||
).toBeLessThanOrEqual(ITERATIONS)
|
||||
}, 120_000)
|
||||
})
|
||||
Reference in New Issue
Block a user