mirror of
https://github.com/stablyai/orca.git
synced 2026-10-04 08:02:09 +00:00
* feat(terminal): inline images via @xterm/addon-image, perf-first Add opt-in inline terminal images (SIXEL, iTerm2 IIP, Kitty graphics) through @xterm/addon-image, designed to keep idle terminals unaffected. Performance: - The addon (base64-inlined wasm decoders + protocol handlers) loads off the boot critical path via a deferred loader that mirrors the WebGL addon: primed after first paint only when the setting is on, read back synchronously at attach, with a 3-attempt cap so a transient failure never disables images for the session and a missing chunk never refetches per pane. renderer-boot-graph guards against eager import. - enableSizeReports:false so the addon never sets windowOptions and double-answers Orca's own CSI 14t/16t responder. - Perf-tuned decode/storage limits (storageLimit, sixel/iip/kitty size caps) in one place. Correctness: - Orca's DA1 handler wins over the addon's (last-registered-first), and the default DA1 response never advertised Sixel (;4), so DA1-detecting tools (chafa, img2sixel, viu, timg) never emitted it. The winning handler now appends ;4 while the setting is on, resolved per query so a live toggle changes the next DA1; idempotent against the ConPTY response that already lists it. - ORCA_IMAGE_PROTOCOL=kitty is exported to spawned shells (local, daemon, relay/SSH) and forwarded across the WSL boundary, so image-capable agents can pick an encoder. Unknown image sequences are swallowed by xterm when the addon is detached, so this never garbles output. - Settings toggle (default on) gates rendering and DA1 advertisement. Cross-checked against community PRs #7775, #11706, and #19201 at the end; credited below. Co-authored-by: s546126 <s546126@users.noreply.github.com> Co-authored-by: XRX193 <XRX193@users.noreply.github.com> Co-authored-by: lmsh7 <lmsh7@users.noreply.github.com> * fix(terminal): bound inline image memory and classify Kitty replies * fix(terminal): bound image decode and release image resources on cleanup * fix(terminal): address image addon review feedback * test(terminal): stub setPaneInlineImagesEnabled in appearance manager fakes * fix(terminal): evict unplaced kitty payloads before displayed images Byte-budget eviction dropped the oldest transmitted blob regardless of placement, so a new upload could erase a visible image while abandoned blobs still held budget. Unplaced payloads now go first and displayed ones only when that is not enough. The incoming image is always stored, so an oversized one overshoots the cap by one payload instead of being dropped after the protocol already acked OK. * fix(terminal): gate DA1 Sixel on real addon attachment; claim SSH image spec in CI - DA1 advertised Sixel from the setting alone, so a pane whose lazy addon chunk was still loading (or had failed all three attempts) told feature-detecting tools to emit DCS that nothing could render. Track the attached decoder per terminal and require it before setting the ;4 bit. - tests/e2e/terminal-inline-images-ssh.spec.ts was Docker-gated but claimed by no lane runner, so pr-e2e-gate-contract failed and the spec would have self-skipped green forever. - Reject non-positive PNG IHDR dimensions before decode: they are parsed with signed shifts, so a dimension >= 0x80000000 came back negative and slipped past the pixel-limit comparison. - One resolveTerminalInlineImagesEnabled() for the default-on setting; the four call sites mixed '?? true' with '!== false', which disagree on null. - One readInlineImageResources() walk of the addon internals instead of two copies that could drift against the patched dependency. - Isolate the deferred-attach drain per pane; make the zoom-invariance and backing-storage e2e assertions fail when the feature is dead. * refactor(terminal): one lazy xterm addon loader for webgl and image terminal-image-addon-loader was a structural clone of the webgl one — same memo, attempt cap, and .then(ok,err)-clears-memo recovery. Both now wrap createLazyXtermAddonLoader; each keeps its literal import() specifier so the bundler still splits the chunk (verified against a fresh build: addon-image stays out of the boot graph). * refactor(terminal): name openTerminal's addon flags; pin image addon limits Two adjacent optional booleans could be swapped without a type error once inline images added the second one. * docs(terminal): state the real per-pane image ceiling; drop test ordering dependency storageLimit:32 reads like the pane's budget but keys three pools — decoded pixels, retained encoded Kitty blobs, and pending WASM decoders — so the worst case is ~98 MB per pane with no cross-pane governor. Say so at the constant. pane-inline-images.test.ts's deferred case needed to run first; it now takes a fresh module instead, and the rest prime in beforeAll. Verified by running the file with that test moved last. * fix(terminal): satisfy rebased static analysis gate * fix(terminal): complete casting gate cleanup * fix(terminal): recover failed image addon loads * fix(terminal): bound image decoder allocations --------- Co-authored-by: m4air <m4air@m4airs-MacBook-Air.local> Co-authored-by: s546126 <s546126@users.noreply.github.com> Co-authored-by: XRX193 <XRX193@users.noreply.github.com> Co-authored-by: lmsh7 <lmsh7@users.noreply.github.com> Co-authored-by: Neil <4138956+nwparker@users.noreply.github.com> Co-authored-by: Neil <neil@stably.ai>
170 lines
7.5 KiB
TypeScript
170 lines
7.5 KiB
TypeScript
// Why this module exists: xterm's public onData stream mixes real keystrokes
|
|
// with the parser's synthetic replies to terminal queries a program embedded in
|
|
// its output (CPR/DSR cursor + device-status reports, DA device attributes,
|
|
// DECRPM mode reports, window/cell pixel-size reports, OSC 10/11 color reports,
|
|
// kitty keyboard flag reports, DCS-framed DECRQSS/XTVERSION reports).
|
|
// A querying program (e.g. starship/orb) reads these replies synchronously in
|
|
// raw mode with a short timeout, so on the remote path they must NOT sit behind
|
|
// the input debounce — a late reply lands on the shell prompt in cooked mode,
|
|
// which echoes it literally and splices it into the next typed line (#7329).
|
|
// This classifier lets the transport send replies immediately while keeping
|
|
// ordinary typed input (including bursty arrow-key auto-repeat) coalesced.
|
|
|
|
const ESC = String.fromCharCode(0x1b)
|
|
|
|
// Built via new RegExp from \u-escaped strings so no literal control
|
|
// characters appear in the source. // Final bytes of xterm's own query-reply grammars:
|
|
// R — CPR / DECXCPR cursor position report (answer to CSI 6n / CSI ? 6n)
|
|
// n — DSR device status report (answer to CSI 5n → CSI 0n)
|
|
// c — DA1/DA2/DA3 device attributes (answer to CSI c / CSI > c / CSI = c)
|
|
// t — window/cell pixel-size + text-area-size reports (CSI 14t/16t/18t)
|
|
// y — DECRPM mode report (answer to CSI ? Ps $ p), body ends "$y"
|
|
// u — kitty keyboard flags report (answer to CSI ? u), carries "?"
|
|
/* oxlint-disable no-control-regex -- grammars match terminal ESC/BEL sequences by definition */
|
|
// Known accepted collision: xterm.js encodes MODIFIED F3 (Shift/Ctrl/Alt+F3) as
|
|
// `CSI 1 ; <mod> R`, which is indistinguishable from a CPR report (a classic
|
|
// VT ambiguity). Such a keystroke is sent immediately and shares the bounded reply
|
|
// budget, so a pathological reply flood can shed it. It is also held behind a deferred
|
|
// reply like any CPR, so within that window (bounded by the host's echo budget) a
|
|
// keystroke typed after it can reach the pty first. Accepted: it needs the chord and a
|
|
// following keypress inside a live colour-query deferral, and we keep the reply grammar
|
|
// complete rather than special-casing an unresolvable ambiguity.
|
|
const CPR_OR_DSR_PREFIX_RE = new RegExp('^\\u001b\\[\\??[0-9;]*[Rn]')
|
|
const DEVICE_ATTRIBUTES_PREFIX_RE = new RegExp('^\\u001b\\[[?>=]?[0-9;]*c')
|
|
// 4/6 = pixel-size reports, 8 = text-area size in characters (answer to CSI 18t).
|
|
const WINDOW_SIZE_REPORT_PREFIX_RE = new RegExp('^\\u001b\\[[468];[0-9]+;[0-9]+t')
|
|
// `?` optional: private-mode reports carry it (DECRPM), ANSI-mode reports don't.
|
|
const DECRPM_PREFIX_RE = new RegExp('^\\u001b\\[\\??[0-9;]*\\$y')
|
|
// Kitty keyboard protocol flags report: CSI ? flags u. The `?` distinguishes it
|
|
// from kitty-protocol *keystrokes* (CSI code;mods u), which must stay batched.
|
|
const KITTY_FLAGS_PREFIX_RE = new RegExp('^\\u001b\\[\\?[0-9]+u')
|
|
// Kitty graphics acknowledgements must bypass user-input activity and debounce.
|
|
const KITTY_GRAPHICS_PREFIX_RE = new RegExp(
|
|
'^\\u001b_Gi=[0-9]+(?:,p=[0-9]+)?;(?:OK|E[A-Z]+:[^\\u0000-\\u001f]*)\\u001b\\\\'
|
|
)
|
|
// OSC color/title responses: ESC ] Ps ; body ST (ST = BEL or ESC backslash).
|
|
const OSC_RESPONSE_PREFIX_RE = new RegExp(
|
|
'^\\u001b\\][0-9]+;[^\\u0007\\u001b]*(?:\\u0007|\\u001b\\\\)'
|
|
)
|
|
// DCS-framed reports xterm emits: DECRQSS "ESC P 1 $ r Pt ST" / "ESC P 0 $ r ST"
|
|
// (vim queries cursor style this way) and XTVERSION "ESC P > | text ST".
|
|
const DCS_RESPONSE_PREFIX_RE = new RegExp(
|
|
'^\\u001bP(?:[01]\\$r[^\\u001b]*|>\\|[^\\u001b]*)\\u001b\\\\'
|
|
)
|
|
// Private-mode DSR (CSI ? … n) — e.g. color-scheme `?997;1n` — often lands cooked.
|
|
// Prefix form peels consecutive replies out of one coalesced payload.
|
|
const COOKED_ECHO_RISK_PRIVATE_DSR_PREFIX_RE = new RegExp('^\\u001b\\[\\?[0-9;]*n')
|
|
const COOKED_ECHO_RISK_OSC_PREFIX_RE = new RegExp(
|
|
'^\\u001b\\][0-9]+;[^\\u0007\\u001b]*(?:\\u0007|\\u001b\\\\)'
|
|
)
|
|
const QUERY_REPLY_PREFIX_RES = [
|
|
CPR_OR_DSR_PREFIX_RE,
|
|
DEVICE_ATTRIBUTES_PREFIX_RE,
|
|
WINDOW_SIZE_REPORT_PREFIX_RE,
|
|
DECRPM_PREFIX_RE,
|
|
KITTY_FLAGS_PREFIX_RE,
|
|
KITTY_GRAPHICS_PREFIX_RE,
|
|
OSC_RESPONSE_PREFIX_RE,
|
|
DCS_RESPONSE_PREFIX_RE
|
|
] as const
|
|
/* oxlint-enable no-control-regex */
|
|
|
|
function terminalQueryReplyEnd(data: string, start: number): number {
|
|
if (start >= data.length || data[start] !== ESC) {
|
|
return -1
|
|
}
|
|
const slice = data.slice(start)
|
|
for (const re of QUERY_REPLY_PREFIX_RES) {
|
|
const match = re.exec(slice)
|
|
if (match?.[0]) {
|
|
return start + match[0].length
|
|
}
|
|
}
|
|
return -1
|
|
}
|
|
|
|
/**
|
|
* True when `data` (from xterm.onData) is a synthetic reply the emulator
|
|
* generated in response to a query — not something the user typed. These are
|
|
* latency-critical and must bypass input coalescing on the remote transport.
|
|
*
|
|
* Conservative by design: matches only complete, well-formed reply grammars so
|
|
* ordinary keystrokes and navigation sequences (arrows CSI A/B/C/D, Home/End,
|
|
* function keys ending in ~, kitty CSI-u keystrokes) are never misclassified
|
|
* as replies — with the single documented modified-F3/CPR collision above.
|
|
*/
|
|
export function isTerminalQueryReply(data: string): boolean {
|
|
return data.length >= 3 && terminalQueryReplyEnd(data, 0) === data.length
|
|
}
|
|
|
|
/** End index (exclusive) of one cooked-echo-risk reply at `start`, else -1. */
|
|
function cookedEchoSafeReplyEnd(data: string, start: number): number {
|
|
if (start >= data.length || data[start] !== ESC) {
|
|
return -1
|
|
}
|
|
const slice = data.slice(start)
|
|
const dsr = COOKED_ECHO_RISK_PRIVATE_DSR_PREFIX_RE.exec(slice)
|
|
if (dsr?.[0]) {
|
|
return start + dsr[0].length
|
|
}
|
|
const osc = COOKED_ECHO_RISK_OSC_PREFIX_RE.exec(slice)
|
|
if (osc?.[0]) {
|
|
return start + osc[0].length
|
|
}
|
|
return -1
|
|
}
|
|
|
|
/**
|
|
* If `data` is entirely one or more consecutive cooked-echo-risk replies, return each
|
|
* reply. Repeats still arrive coalesced from a rapid theme flip or an older client that
|
|
* answers 2031 subscribes (this host stopped — #9993), so peel them individually.
|
|
* Mixed payloads (reply + keystroke) return null so hosts fall through to raw write.
|
|
*/
|
|
export function extractOnlyCookedEchoSafeQueryReplies(data: string): string[] | null {
|
|
if (data.length < 4 || data[0] !== ESC) {
|
|
return null
|
|
}
|
|
const replies: string[] = []
|
|
let offset = 0
|
|
while (offset < data.length) {
|
|
const end = cookedEchoSafeReplyEnd(data, offset)
|
|
if (end === -1) {
|
|
return null
|
|
}
|
|
replies.push(data.slice(offset, end))
|
|
offset = end
|
|
}
|
|
return replies.length > 0 ? replies : null
|
|
}
|
|
|
|
/** If `data` is entirely one or more consecutive query replies, return each reply. */
|
|
export function extractOnlyTerminalQueryReplies(data: string): string[] | null {
|
|
if (data.length < 3 || data[0] !== ESC) {
|
|
return null
|
|
}
|
|
const replies: string[] = []
|
|
let offset = 0
|
|
while (offset < data.length) {
|
|
const end = terminalQueryReplyEnd(data, offset)
|
|
if (end === -1) {
|
|
return null
|
|
}
|
|
replies.push(data.slice(offset, end))
|
|
offset = end
|
|
}
|
|
return replies.length > 0 ? replies : null
|
|
}
|
|
|
|
/**
|
|
* Query replies that must use the ECHO-safe write path on POSIX PTYs so cooked
|
|
* prompts do not paint reply bytes (e.g. `997;1n` on `npx` confirm, #13137).
|
|
* Latency-critical CPR/DSR without `?` stay on the immediate write path.
|
|
*
|
|
* Whole-string only: a single complete reply. For repeated / write-queue
|
|
* coalesced payloads use {@link extractOnlyCookedEchoSafeQueryReplies}.
|
|
*/
|
|
export function needsCookedEchoSafeQueryReply(data: string): boolean {
|
|
const replies = extractOnlyCookedEchoSafeQueryReplies(data)
|
|
return replies !== null && replies.length === 1
|
|
}
|