mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 00:02:31 +00:00
* feat(terminal): inline images via @xterm/addon-image, perf-first Add opt-in inline terminal images (SIXEL, iTerm2 IIP, Kitty graphics) through @xterm/addon-image, designed to keep idle terminals unaffected. Performance: - The addon (base64-inlined wasm decoders + protocol handlers) loads off the boot critical path via a deferred loader that mirrors the WebGL addon: primed after first paint only when the setting is on, read back synchronously at attach, with a 3-attempt cap so a transient failure never disables images for the session and a missing chunk never refetches per pane. renderer-boot-graph guards against eager import. - enableSizeReports:false so the addon never sets windowOptions and double-answers Orca's own CSI 14t/16t responder. - Perf-tuned decode/storage limits (storageLimit, sixel/iip/kitty size caps) in one place. Correctness: - Orca's DA1 handler wins over the addon's (last-registered-first), and the default DA1 response never advertised Sixel (;4), so DA1-detecting tools (chafa, img2sixel, viu, timg) never emitted it. The winning handler now appends ;4 while the setting is on, resolved per query so a live toggle changes the next DA1; idempotent against the ConPTY response that already lists it. - ORCA_IMAGE_PROTOCOL=kitty is exported to spawned shells (local, daemon, relay/SSH) and forwarded across the WSL boundary, so image-capable agents can pick an encoder. Unknown image sequences are swallowed by xterm when the addon is detached, so this never garbles output. - Settings toggle (default on) gates rendering and DA1 advertisement. Cross-checked against community PRs #7775, #11706, and #19201 at the end; credited below. Co-authored-by: s546126 <s546126@users.noreply.github.com> Co-authored-by: XRX193 <XRX193@users.noreply.github.com> Co-authored-by: lmsh7 <lmsh7@users.noreply.github.com> * fix(terminal): bound inline image memory and classify Kitty replies * fix(terminal): bound image decode and release image resources on cleanup * fix(terminal): address image addon review feedback * test(terminal): stub setPaneInlineImagesEnabled in appearance manager fakes * fix(terminal): evict unplaced kitty payloads before displayed images Byte-budget eviction dropped the oldest transmitted blob regardless of placement, so a new upload could erase a visible image while abandoned blobs still held budget. Unplaced payloads now go first and displayed ones only when that is not enough. The incoming image is always stored, so an oversized one overshoots the cap by one payload instead of being dropped after the protocol already acked OK. * fix(terminal): gate DA1 Sixel on real addon attachment; claim SSH image spec in CI - DA1 advertised Sixel from the setting alone, so a pane whose lazy addon chunk was still loading (or had failed all three attempts) told feature-detecting tools to emit DCS that nothing could render. Track the attached decoder per terminal and require it before setting the ;4 bit. - tests/e2e/terminal-inline-images-ssh.spec.ts was Docker-gated but claimed by no lane runner, so pr-e2e-gate-contract failed and the spec would have self-skipped green forever. - Reject non-positive PNG IHDR dimensions before decode: they are parsed with signed shifts, so a dimension >= 0x80000000 came back negative and slipped past the pixel-limit comparison. - One resolveTerminalInlineImagesEnabled() for the default-on setting; the four call sites mixed '?? true' with '!== false', which disagree on null. - One readInlineImageResources() walk of the addon internals instead of two copies that could drift against the patched dependency. - Isolate the deferred-attach drain per pane; make the zoom-invariance and backing-storage e2e assertions fail when the feature is dead. * refactor(terminal): one lazy xterm addon loader for webgl and image terminal-image-addon-loader was a structural clone of the webgl one — same memo, attempt cap, and .then(ok,err)-clears-memo recovery. Both now wrap createLazyXtermAddonLoader; each keeps its literal import() specifier so the bundler still splits the chunk (verified against a fresh build: addon-image stays out of the boot graph). * refactor(terminal): name openTerminal's addon flags; pin image addon limits Two adjacent optional booleans could be swapped without a type error once inline images added the second one. * docs(terminal): state the real per-pane image ceiling; drop test ordering dependency storageLimit:32 reads like the pane's budget but keys three pools — decoded pixels, retained encoded Kitty blobs, and pending WASM decoders — so the worst case is ~98 MB per pane with no cross-pane governor. Say so at the constant. pane-inline-images.test.ts's deferred case needed to run first; it now takes a fresh module instead, and the rest prime in beforeAll. Verified by running the file with that test moved last. * fix(terminal): satisfy rebased static analysis gate * fix(terminal): complete casting gate cleanup * fix(terminal): recover failed image addon loads * fix(terminal): bound image decoder allocations --------- Co-authored-by: m4air <m4air@m4airs-MacBook-Air.local> Co-authored-by: s546126 <s546126@users.noreply.github.com> Co-authored-by: XRX193 <XRX193@users.noreply.github.com> Co-authored-by: lmsh7 <lmsh7@users.noreply.github.com> Co-authored-by: Neil <4138956+nwparker@users.noreply.github.com> Co-authored-by: Neil <neil@stably.ai>
130 lines
5.0 KiB
JavaScript
130 lines
5.0 KiB
JavaScript
import { createRequire } from 'node:module'
|
|
import { describe, expect, it } from 'vitest'
|
|
|
|
const require = createRequire(import.meta.url)
|
|
const { Terminal } = require('@xterm/xterm')
|
|
const { ImageAddon } = require('@xterm/addon-image')
|
|
|
|
function createTerminal(options = {}) {
|
|
const terminal = new Terminal({ allowProposedApi: true })
|
|
const addon = new ImageAddon({
|
|
enableSizeReports: false,
|
|
storageLimit: 32,
|
|
kittySizeLimit: 8 * 1024 * 1024,
|
|
...options
|
|
})
|
|
terminal.loadAddon(addon)
|
|
const handler = addon._handlers.get('kitty')
|
|
if (options.kittyStorageLimit !== undefined) {
|
|
handler._kittyStorage._storage.setLimit(options.kittyStorageLimit)
|
|
}
|
|
return { terminal, addon, handler }
|
|
}
|
|
|
|
function writeKitty(terminal, command, payload) {
|
|
return new Promise((resolve) => terminal.write(`\x1b_G${command};${payload}\x1b\\`, resolve))
|
|
}
|
|
|
|
describe('xterm image memory contract', () => {
|
|
it('does not emit a reply when evicting an id-less upload', async () => {
|
|
const { terminal, handler } = createTerminal()
|
|
const replies = []
|
|
terminal.onData((data) => replies.push(data))
|
|
try {
|
|
await writeKitty(terminal, 'a=t,f=32,s=1,v=1,m=1', 'AAAA')
|
|
await writeKitty(terminal, 'a=t,f=32,s=1,v=1,i=1,m=1,q=2', 'AAAA')
|
|
await writeKitty(terminal, 'a=t,f=32,s=1,v=1,i=2,m=1,q=2', 'AAAA')
|
|
expect(handler._pendingTransmissions.has(0)).toBe(false)
|
|
expect(handler._pendingTransmissions.size).toBe(2)
|
|
expect(replies).toEqual([])
|
|
} finally {
|
|
terminal.dispose()
|
|
}
|
|
})
|
|
|
|
it('bounds abandoned uploads by retained decoder capacity and accepts a continuation', async () => {
|
|
const { terminal, handler } = createTerminal()
|
|
try {
|
|
for (let id = 1; id <= 40; id++) {
|
|
await writeKitty(terminal, `a=t,f=32,s=1,v=1,i=${id},m=1,q=2`, 'AAAA')
|
|
const pending = [...handler._pendingTransmissions.values()]
|
|
const retainedBytes = pending.reduce(
|
|
(total, upload) => total + upload.decoder._mem.buffer.byteLength,
|
|
0
|
|
)
|
|
expect(retainedBytes).toBeLessThanOrEqual(32_000_000)
|
|
expect(pending.length).toBeLessThanOrEqual(2)
|
|
}
|
|
await writeKitty(terminal, 'm=0,q=2', 'AA==')
|
|
expect(handler._kittyStorage.getImage(40).data.size).toBe(4)
|
|
terminal.dispose()
|
|
expect(handler._pendingTransmissions.size).toBe(0)
|
|
} finally {
|
|
terminal.dispose()
|
|
}
|
|
})
|
|
|
|
it('rejects a decoder that cannot fit the storage budget before allocation', async () => {
|
|
const { terminal, handler } = createTerminal({ storageLimit: 8 })
|
|
try {
|
|
await writeKitty(terminal, 'a=t,f=32,s=1,v=1,i=9,m=1,q=2', 'AAAA')
|
|
expect(handler._pendingTransmissions.size).toBe(0)
|
|
expect(handler._aborted).toBe(true)
|
|
} finally {
|
|
terminal.dispose()
|
|
}
|
|
})
|
|
|
|
it('evicts transmitted images by byte size before placement', async () => {
|
|
const { terminal, handler } = createTerminal({ storageLimit: 12, kittyStorageLimit: 0.5 })
|
|
const payload = Buffer.alloc(200_000, 1).toString('base64')
|
|
try {
|
|
for (let id = 1; id <= 4; id++) {
|
|
await writeKitty(terminal, `a=t,f=32,s=250,v=200,i=${id},q=2`, payload)
|
|
const retainedBytes = [...handler._kittyStorage.images.values()].reduce(
|
|
(total, image) => total + image.data.size,
|
|
0
|
|
)
|
|
expect(retainedBytes).toBeLessThanOrEqual(500_000)
|
|
}
|
|
expect(handler._kittyStorage.getImage(1)).toBeUndefined()
|
|
expect(handler._kittyStorage.getImage(3).data.size).toBe(200_000)
|
|
expect(handler._kittyStorage.getImage(4).data.size).toBe(200_000)
|
|
await writeKitty(terminal, 'a=d,d=A,q=2', '')
|
|
expect(handler._kittyStorage.images.size).toBe(0)
|
|
} finally {
|
|
terminal.dispose()
|
|
}
|
|
})
|
|
|
|
it('evicts unplaced payloads before displayed ones', async () => {
|
|
const { terminal, handler } = createTerminal({ storageLimit: 12, kittyStorageLimit: 0.5 })
|
|
const storage = handler._kittyStorage
|
|
const payload = Buffer.alloc(200_000, 1).toString('base64')
|
|
try {
|
|
await writeKitty(terminal, 'a=t,f=32,s=250,v=200,i=1,q=2', payload)
|
|
await writeKitty(terminal, 'a=t,f=32,s=250,v=200,i=2,q=2', payload)
|
|
// Placement bookkeeping only; a real addImage needs a canvas this env lacks.
|
|
storage._kittyIdToStorageId.set(1, 1001)
|
|
storage._storageIdToKittyId.set(1001, 1)
|
|
await writeKitty(terminal, 'a=t,f=32,s=250,v=200,i=3,q=2', payload)
|
|
expect(storage.getImage(1)).toBeDefined()
|
|
expect(storage.getImage(2)).toBeUndefined()
|
|
expect(storage.getImage(3).data.size).toBe(200_000)
|
|
} finally {
|
|
terminal.dispose()
|
|
}
|
|
})
|
|
|
|
it('stores an image larger than the byte budget rather than acking a dropped one', async () => {
|
|
const { terminal, handler } = createTerminal({ storageLimit: 12, kittyStorageLimit: 0.5 })
|
|
const payload = Buffer.alloc(600_000, 1).toString('base64')
|
|
try {
|
|
await writeKitty(terminal, 'a=t,f=32,s=500,v=300,i=1,q=2', payload)
|
|
expect(handler._kittyStorage.getImage(1).data.size).toBe(600_000)
|
|
} finally {
|
|
terminal.dispose()
|
|
}
|
|
})
|
|
})
|