Files
orca/src/shared/terminal-query-reply.test.ts
T
09073086a8 feat(terminal): inline images via @xterm/addon-image (perf-first) (#19512)
* feat(terminal): inline images via @xterm/addon-image, perf-first

Add opt-in inline terminal images (SIXEL, iTerm2 IIP, Kitty graphics)
through @xterm/addon-image, designed to keep idle terminals unaffected.

Performance:
- The addon (base64-inlined wasm decoders + protocol handlers) loads off
  the boot critical path via a deferred loader that mirrors the WebGL
  addon: primed after first paint only when the setting is on, read back
  synchronously at attach, with a 3-attempt cap so a transient failure
  never disables images for the session and a missing chunk never
  refetches per pane. renderer-boot-graph guards against eager import.
- enableSizeReports:false so the addon never sets windowOptions and
  double-answers Orca's own CSI 14t/16t responder.
- Perf-tuned decode/storage limits (storageLimit, sixel/iip/kitty size
  caps) in one place.

Correctness:
- Orca's DA1 handler wins over the addon's (last-registered-first), and
  the default DA1 response never advertised Sixel (;4), so DA1-detecting
  tools (chafa, img2sixel, viu, timg) never emitted it. The winning
  handler now appends ;4 while the setting is on, resolved per query so a
  live toggle changes the next DA1; idempotent against the ConPTY
  response that already lists it.
- ORCA_IMAGE_PROTOCOL=kitty is exported to spawned shells (local, daemon,
  relay/SSH) and forwarded across the WSL boundary, so image-capable
  agents can pick an encoder. Unknown image sequences are swallowed by
  xterm when the addon is detached, so this never garbles output.
- Settings toggle (default on) gates rendering and DA1 advertisement.

Cross-checked against community PRs #7775, #11706, and #19201 at the end;
credited below.

Co-authored-by: s546126 <s546126@users.noreply.github.com>
Co-authored-by: XRX193 <XRX193@users.noreply.github.com>
Co-authored-by: lmsh7 <lmsh7@users.noreply.github.com>

* fix(terminal): bound inline image memory and classify Kitty replies

* fix(terminal): bound image decode and release image resources on cleanup

* fix(terminal): address image addon review feedback

* test(terminal): stub setPaneInlineImagesEnabled in appearance manager fakes

* fix(terminal): evict unplaced kitty payloads before displayed images

Byte-budget eviction dropped the oldest transmitted blob regardless of
placement, so a new upload could erase a visible image while abandoned
blobs still held budget. Unplaced payloads now go first and displayed
ones only when that is not enough. The incoming image is always stored,
so an oversized one overshoots the cap by one payload instead of being
dropped after the protocol already acked OK.

* fix(terminal): gate DA1 Sixel on real addon attachment; claim SSH image spec in CI

- DA1 advertised Sixel from the setting alone, so a pane whose lazy addon
  chunk was still loading (or had failed all three attempts) told
  feature-detecting tools to emit DCS that nothing could render. Track the
  attached decoder per terminal and require it before setting the ;4 bit.
- tests/e2e/terminal-inline-images-ssh.spec.ts was Docker-gated but claimed
  by no lane runner, so pr-e2e-gate-contract failed and the spec would have
  self-skipped green forever.
- Reject non-positive PNG IHDR dimensions before decode: they are parsed with
  signed shifts, so a dimension >= 0x80000000 came back negative and slipped
  past the pixel-limit comparison.
- One resolveTerminalInlineImagesEnabled() for the default-on setting; the
  four call sites mixed '?? true' with '!== false', which disagree on null.
- One readInlineImageResources() walk of the addon internals instead of two
  copies that could drift against the patched dependency.
- Isolate the deferred-attach drain per pane; make the zoom-invariance and
  backing-storage e2e assertions fail when the feature is dead.

* refactor(terminal): one lazy xterm addon loader for webgl and image

terminal-image-addon-loader was a structural clone of the webgl one — same
memo, attempt cap, and .then(ok,err)-clears-memo recovery. Both now wrap
createLazyXtermAddonLoader; each keeps its literal import() specifier so the
bundler still splits the chunk (verified against a fresh build: addon-image
stays out of the boot graph).

* refactor(terminal): name openTerminal's addon flags; pin image addon limits

Two adjacent optional booleans could be swapped without a type error once
inline images added the second one.

* docs(terminal): state the real per-pane image ceiling; drop test ordering dependency

storageLimit:32 reads like the pane's budget but keys three pools — decoded
pixels, retained encoded Kitty blobs, and pending WASM decoders — so the worst
case is ~98 MB per pane with no cross-pane governor. Say so at the constant.

pane-inline-images.test.ts's deferred case needed to run first; it now takes a
fresh module instead, and the rest prime in beforeAll. Verified by running the
file with that test moved last.

* fix(terminal): satisfy rebased static analysis gate

* fix(terminal): complete casting gate cleanup

* fix(terminal): recover failed image addon loads

* fix(terminal): bound image decoder allocations

---------

Co-authored-by: m4air <m4air@m4airs-MacBook-Air.local>
Co-authored-by: s546126 <s546126@users.noreply.github.com>
Co-authored-by: XRX193 <XRX193@users.noreply.github.com>
Co-authored-by: lmsh7 <lmsh7@users.noreply.github.com>
Co-authored-by: Neil <4138956+nwparker@users.noreply.github.com>
Co-authored-by: Neil <neil@stably.ai>
2026-09-18 16:32:49 -07:00

272 lines
12 KiB
TypeScript

import { Terminal } from '@xterm/headless'
import { describe, expect, it, vi } from 'vitest'
import {
extractOnlyCookedEchoSafeQueryReplies,
extractOnlyTerminalQueryReplies,
isTerminalQueryReply,
needsCookedEchoSafeQueryReply
} from './terminal-query-reply'
import { PtyStartupIngress } from './pty-startup-ingress'
describe('isTerminalQueryReply', () => {
it('routes complete Kitty graphics acknowledgements as replies', () => {
const ok = '\x1b_Gi=31;OK\x1b\\'
const error = '\x1b_Gi=31,p=2;ENOENT:image not found\x1b\\'
expect(isTerminalQueryReply(ok)).toBe(true)
expect(isTerminalQueryReply(error)).toBe(true)
expect(extractOnlyTerminalQueryReplies(ok + error)).toEqual([ok, error])
expect(needsCookedEchoSafeQueryReply(ok)).toBe(false)
for (const input of [
'\x1b_',
'\x1b_Gi=31;OK',
'\x1b_Gi=31;OK\x07',
'\x1b_Gi=31;hello\x1b\\',
'\x1b_Ga=q,i=31;AAAA\x1b\\',
`${ok}typed`
]) {
expect(isTerminalQueryReply(input)).toBe(false)
expect(extractOnlyTerminalQueryReplies(input)).toBeNull()
}
})
it('matches synthetic query replies that must be sent immediately', () => {
// CPR cursor position report (answer to CSI 6n) — the #7329 culprit.
expect(isTerminalQueryReply('\x1b[3;1R')).toBe(true)
expect(isTerminalQueryReply('\x1b[22;1R')).toBe(true)
// DSR device status.
expect(isTerminalQueryReply('\x1b[0n')).toBe(true)
// Contour color-scheme report (answer to CSI ?996n / mode-2031 push).
expect(isTerminalQueryReply('\x1b[?997;1n')).toBe(true)
expect(isTerminalQueryReply('\x1b[?997;2n')).toBe(true)
// DA1/DA2/DA3 device attributes.
expect(isTerminalQueryReply('\x1b[?1;2c')).toBe(true)
expect(isTerminalQueryReply('\x1b[?61;4c')).toBe(true)
expect(isTerminalQueryReply('\x1b[>0;276;0c')).toBe(true)
// Window/cell pixel-size reports.
expect(isTerminalQueryReply('\x1b[6;16;8t')).toBe(true)
expect(isTerminalQueryReply('\x1b[4;384;640t')).toBe(true)
// DECRPM mode report — private (with ?) and ANSI (without ?).
expect(isTerminalQueryReply('\x1b[?2026;2$y')).toBe(true)
expect(isTerminalQueryReply('\x1b[4;1$y')).toBe(true)
// OSC 10/11 color responses (the #7329 culprit) — BEL and ST terminated.
expect(isTerminalQueryReply('\x1b]11;rgb:2828/2c2c/3434\x1b\\')).toBe(true)
expect(isTerminalQueryReply('\x1b]10;rgb:c0c0/c0c0/c0c0\x07')).toBe(true)
// DECXCPR extended cursor position report (answer to CSI ? 6n).
expect(isTerminalQueryReply('\x1b[?12;5R')).toBe(true)
// Text-area size in characters (answer to CSI 18t).
expect(isTerminalQueryReply('\x1b[8;24;80t')).toBe(true)
// Kitty keyboard flags report (answer to CSI ? u) — crossterm probes this
// at startup, so a debounced reply corrupts the same way CPR did.
expect(isTerminalQueryReply('\x1b[?0u')).toBe(true)
expect(isTerminalQueryReply('\x1b[?31u')).toBe(true)
// DCS DECRQSS reports (vim queries cursor style via DCS $ q) + XTVERSION.
expect(isTerminalQueryReply('\x1bP1$r2 q\x1b\\')).toBe(true)
expect(isTerminalQueryReply('\x1bP1$r0m\x1b\\')).toBe(true)
expect(isTerminalQueryReply('\x1bP0$r\x1b\\')).toBe(true)
expect(isTerminalQueryReply('\x1bP>|xterm.js(5.6.0)\x1b\\')).toBe(true)
})
it('classifies the fully framed XTVERSION reply emitted by real xterm', async () => {
const terminal = new Terminal()
const replies: string[] = []
const disposable = terminal.onData((data) => replies.push(data))
try {
await new Promise<void>((resolve) => terminal.write('\x1b[>q', resolve))
expect(replies).toHaveLength(1)
const reply = replies[0]
expect(reply.startsWith('\x1bP>|xterm.js(')).toBe(true)
expect(reply.endsWith(')\x1b\\')).toBe(true)
expect(isTerminalQueryReply(reply)).toBe(true)
} finally {
disposable.dispose()
terminal.dispose()
}
})
it('documents the accepted modified-F3/CPR collision', () => {
// xterm.js encodes Shift+F3 as CSI 1;2R — byte-identical to a CPR report.
// Classified as a reply on purpose: order is still preserved (the immediate
// path flushes pending input first); see the comment in terminal-query-reply.ts.
expect(isTerminalQueryReply('\x1b[1;2R')).toBe(true)
})
it('routes only cooked-echo-risk replies through the ECHO-safe write path', () => {
// Color-scheme private DSR + OSC color — cooked prompt paint risk (#13137).
expect(needsCookedEchoSafeQueryReply('\x1b[?997;1n')).toBe(true)
expect(needsCookedEchoSafeQueryReply('\x1b[?997;2n')).toBe(true)
expect(needsCookedEchoSafeQueryReply('\x1b]11;rgb:00/00/00\x07')).toBe(true)
// Latency-critical CPR / public DSR / DA stay immediate (#7329).
expect(needsCookedEchoSafeQueryReply('\x1b[3;1R')).toBe(false)
expect(needsCookedEchoSafeQueryReply('\x1b[0n')).toBe(false)
expect(needsCookedEchoSafeQueryReply('\x1b[?1;2c')).toBe(false)
// Ordinary input must never take the echo-safe reply path.
expect(needsCookedEchoSafeQueryReply('y')).toBe(false)
expect(needsCookedEchoSafeQueryReply('\x1b[A')).toBe(false)
// Dual-answerer coalesced payload is not a single reply — use extract.
expect(needsCookedEchoSafeQueryReply('\x1b[?997;1n\x1b[?997;1n')).toBe(false)
expect(extractOnlyCookedEchoSafeQueryReplies('\x1b[?997;1n\x1b[?997;1n')).toEqual([
'\x1b[?997;1n',
'\x1b[?997;1n'
])
expect(extractOnlyCookedEchoSafeQueryReplies('\x1b[?997;1ny')).toBe(null)
expect(extractOnlyTerminalQueryReplies('\x1b[?1;2c\x1b[1;1R')).toEqual([
'\x1b[?1;2c',
'\x1b[1;1R'
])
expect(extractOnlyTerminalQueryReplies('\x1b]11;rgb:2828/2c2c/3434\x1b\\\x1b[?1;2c')).toEqual([
'\x1b]11;rgb:2828/2c2c/3434\x1b\\',
'\x1b[?1;2c'
])
expect(extractOnlyTerminalQueryReplies('\x1b[?1;2chello')).toBe(null)
})
it('does NOT match ordinary typed input or navigation sequences', () => {
// Plain text.
expect(isTerminalQueryReply('yes')).toBe(false)
expect(isTerminalQueryReply('y')).toBe(false)
expect(isTerminalQueryReply('\r')).toBe(false)
expect(isTerminalQueryReply('\x03')).toBe(false) // Ctrl-C
// Arrow keys / navigation — must stay batched (coalesced auto-repeat).
expect(isTerminalQueryReply('\x1b[A')).toBe(false)
expect(isTerminalQueryReply('\x1b[B')).toBe(false)
expect(isTerminalQueryReply('\x1b[C')).toBe(false)
expect(isTerminalQueryReply('\x1b[D')).toBe(false)
expect(isTerminalQueryReply('\x1b[H')).toBe(false) // Home
expect(isTerminalQueryReply('\x1b[F')).toBe(false) // End
// Function keys (end in ~).
expect(isTerminalQueryReply('\x1b[15~')).toBe(false)
expect(isTerminalQueryReply('\x1b[3~')).toBe(false) // Delete
// Bare Escape key.
expect(isTerminalQueryReply('\x1b')).toBe(false)
// Alt+key (including Alt+Shift+P, whose bytes prefix the DCS grammar).
expect(isTerminalQueryReply('\x1bb')).toBe(false)
expect(isTerminalQueryReply('\x1bP')).toBe(false)
// Kitty-protocol KEYSTROKES (CSI code;mods u, no "?") must stay batched.
expect(isTerminalQueryReply('\x1b[97;5u')).toBe(false)
expect(isTerminalQueryReply('\x1b[13u')).toBe(false)
// Modified F1/F2/F4 (CSI 1;<mod> P/Q/S) are keystrokes, not replies.
expect(isTerminalQueryReply('\x1b[1;2P')).toBe(false)
expect(isTerminalQueryReply('\x1b[1;2Q')).toBe(false)
expect(isTerminalQueryReply('\x1b[1;2S')).toBe(false)
// Bracketed paste markers are input framing, not replies.
expect(isTerminalQueryReply('\x1b[200~')).toBe(false)
expect(isTerminalQueryReply('\x1b[201~')).toBe(false)
// Incomplete / non-terminated OSC and DCS must not match.
expect(isTerminalQueryReply('\x1b]11;rgb:2828/2c2c/3434')).toBe(false)
expect(isTerminalQueryReply('\x1bP1$r2 q')).toBe(false)
})
})
// Regression for the `gh auth login` report: termenv writes `OSC 11 ;? ST` then
// `CSI 6n` and reads exactly one response, treating a CPR-first answer as "no OSC
// support" without draining further. Orca defers the color reply behind an ECHO
// probe, so a CPR taken straight to the PTY overtakes it and leaves `ESC ]` in the
// tty for the next program — bubbletea then dies with
// "unexpected escape sequence from terminal: ['\x1b' ']']".
describe('query reply ordering (termenv OSC-then-CPR)', () => {
const OSC_11_REPLY = '\x1b]11;rgb:1e1e/1e1e/1e1e\x1b\\'
const CPR_REPLY = '\x1b[1;1R'
function hostWrites(ingress: PtyStartupIngress, pty: string[]) {
return (data: string): void => {
if (!ingress.answerLiveQueryReply(data)) {
pty.push(data)
}
}
}
it('delivers the color reply before the CPR the querying program stops at', async () => {
vi.useFakeTimers()
const pty: string[] = []
const ingress = new PtyStartupIngress({
ownerBackend: 'posix-pty',
write: (data) => pty.push(data),
onEmission: () => {}
})
const write = hostWrites(ingress, pty)
write(OSC_11_REPLY)
write(CPR_REPLY)
await vi.advanceTimersByTimeAsync(200)
expect(pty).toEqual([OSC_11_REPLY, CPR_REPLY])
// Nothing survives the CPR, so the next program's stdin opens clean.
expect(pty.slice(1).join('')).not.toContain('\x1b]')
ingress.drainAndClose()
vi.useRealTimers()
})
it('preserves reverse query order when CPR arrives before the color query', async () => {
vi.useFakeTimers()
const pty: string[] = []
const ingress = new PtyStartupIngress({
ownerBackend: 'posix-pty',
write: (data) => pty.push(data),
onEmission: () => {}
})
const write = hostWrites(ingress, pty)
write(CPR_REPLY)
write(OSC_11_REPLY)
await vi.advanceTimersByTimeAsync(200)
expect(pty).toEqual([CPR_REPLY, OSC_11_REPLY])
ingress.drainAndClose()
vi.useRealTimers()
})
it('keeps a CPR immediate when no color reply is deferred', () => {
vi.useFakeTimers()
const pty: string[] = []
const ingress = new PtyStartupIngress({
ownerBackend: 'posix-pty',
write: (data) => pty.push(data),
onEmission: () => {}
})
hostWrites(ingress, pty)(CPR_REPLY)
expect(pty).toEqual([CPR_REPLY])
ingress.drainAndClose()
vi.useRealTimers()
})
it('never takes ordinary typed input', () => {
const pty: string[] = []
const ingress = new PtyStartupIngress({
ownerBackend: 'posix-pty',
write: (data) => pty.push(data),
onEmission: () => {}
})
for (const keystroke of ['y', 'gh auth login\r', '\x1b[A', '\x1b', '\x03']) {
expect(ingress.answerLiveQueryReply(keystroke)).toBe(false)
}
ingress.drainAndClose()
})
})
// Known limitation, tracked for follow-up: the guarantee is FIFO among recognised
// query replies, not over every byte written to the pty. Both branches of
// takeLiveQueryReply match whole strings, so a coalesced payload is not a reply, and
// ordinary input never rides the queue at all. Pinned so the boundary is explicit —
// if a change makes these take the ordered path, that is an improvement, not a break.
describe('writes that still bypass the ordered queue', () => {
const BYPASSING = [
{ what: 'reply coalesced with a keystroke', data: '\x1b[6;1Ry' },
{ what: 'keystroke coalesced with a reply', data: 'y\x1b[6;1R' },
{ what: 'ordinary typed input', data: 'ls\r' }
]
it.each(BYPASSING)('does not take $what', ({ data }) => {
const ingress = new PtyStartupIngress({
ownerBackend: 'posix-pty',
write: () => {},
onEmission: () => {}
})
// Deferral is open, so an ordered write WOULD be queued here.
expect(ingress.answerLiveQueryReply('\x1b]11;rgb:00/00/00\x07')).toBe(true)
expect(ingress.answerLiveQueryReply(data)).toBe(false)
ingress.drainAndClose()
})
})