Files
orca/config/patches/xterm-src/@xterm__addon-serialize@0.15.0-beta.300.src.patch
T
Jinwoo Hong 2c7609bf6a fix(terminal): serialize only the visible width after a column shrink (#22586)
* fix(terminal): serialize only the visible width after a column shrink

xterm does not reflow the alternate buffer (or a normal buffer under
pre-21376 ConPTY), so after a shrink each line keeps its old length.
SerializeAddon walked every non-final row to line.length, so any snapshot
taken after a shrink carried stale right-hand cells that wrapped into extra
rows on replay; restores repainted that garbage and a differential TUI such
as OpenCode never cleared it.

Clamp the row walk and the wrap-boundary lookups to the terminal's columns
in Orca's addon-serialize source patch, and regenerate the bundles, maps and
lockfile hash per docs/reference/xterm-patch-regeneration.md.

* test(terminal): read shrink-snapshot fixtures through public APIs

Drops the private-terminal casts the casting gate flags; the normal-buffer case
now drives a plain pre-21376 ConPTY terminal and its SerializeAddon directly.

* fix(terminal): blank a wide glyph clipped by a column shrink when serializing

After a non-reflowing shrink a width-2 glyph can have its lead half in the last
column and its trailing half past the grid. Serializing the lead half makes the
replay wrap it to the next row and shift every row below, so serialize that
cell as a blank and keep the row exactly the grid's width. A glyph ending
exactly at the edge is unchanged.

* test(mobile): move the session closure pin past the main agent status modules

#22452 added src/shared/main-agent-status.ts and src/shared/agent-turn-outcome.ts,
which agent-status-types.ts imports, so the session route's closure grew by two
local modules (4218 -> 4220). That change was src/shared-only, so its own CI never
ran this suite; main has been at 4220 since, and any PR that fires the mobile web
app job fails on the stale pin. Measured on 4064653740 and on origin/main 3ea15dd0a2.

* test(terminal): differential serialize round-trip fuzz and transcript replay

Seeded VT streams (text, CJK/emoji/combining, SGR, cursor/edit ops, scroll
regions, DECAWM/IRM, alt-screen variants, DECSC, and shrink-heavy resizes)
drive a source terminal in three modes: reflowing normal buffer, alternate
buffer, and a non-reflowing pre-21376 ConPTY normal buffer. At each checkpoint
every SerializeAddon build under test serializes it, and each output is
replayed into a fresh terminal of the same size and compared cell by cell,
plus cursor, active buffer and modes.

CI runs 25 seeds per mode against a pinned list of pre-existing divergences,
and replays the committed PTY transcripts (the existing agent fixtures plus new
vim, less, pico, and OpenCode captures) under four resize schedules. Point
ORCA_OLD_SERIALIZE_ADDON at a previous patched build to also check byte
identity when no line is wider than the grid, and that no checkpoint regresses.

* test(terminal): build the differential serialize baseline from any git ref

config/scripts/build-serialize-addon-at-ref.mjs reverse-applies the patch that
produced the installed @xterm/addon-serialize dist, applies the ref's patch,
and verifies each step against the patches' blob hashes, so the fuzz can use
origin/main (or any fix commit) as its baseline without a second install.
ORCA_NEW_SERIALIZE_ADDON swaps in a built dist for the build under test, and a
seed-pinned test replays the nine I3 regressions found against origin/main.

* fix(terminal): serialize a clipped wide glyph as a width-1 blank

The stand-in for a wide glyph clipped by a column shrink came from getNullCell(),
whose width is 0. _nextCell skipped it as a wide trailer, and the row-end wrap
check counted the width-0 _backgroundCell as content, so a soft wrap after the
clipped column was taken as natural and replayed one column early
(conpty seed 1149: `abcdefghi中WRAPPED` at 12 -> 10 cols replayed as
`abcdefghiR`/`APPED`). Blank the cell in place instead: width 1, no codepoint,
its own attributes, so it counts as one empty column and forces the wrap.

Differential sweep vs origin/main, 7000 cases per mode: I1 0 byte diffs, seed
1149 fixed; the remaining I3 regressions are the trailing background-row seeds.

* test(terminal): neutral paths in the serialize fixtures and usage comment

The OpenCode transcript carried this machine's lane paths in its footer; replace
them with same-length neutral paths so the recorded cursor layout is unchanged.

* fix(terminal): keep trailing background-only rows when serializing

Without scrollback, _serializeString trims rows after the last content cursor.
A row made only of background-colored blanks emits its erase in _rowEnd but
never moved that cursor, so two or more such rows at the bottom were dropped
(4x3 `r1\r\n\e[48;5;157m\e[J\e[0m\e[3;1H` replayed with the last row blank).
Track the erase separately and extend the kept rows to it, except when the
cursor is wrap-pending: relative moves back from those rows cannot re-create
that state, and doing so regressed normal 508, alt 1425/6647, conpty 4699.

Harness: I1 now exempts checkpoints with a background row after the last text
row, the one place this fix changes bytes on purpose (scope helpers move to
serialize-grid-variant-scope.ts); conpty seed 5 leaves the pinned pre-existing
list. Sweep vs origin/main, 7000 cases per mode: I1 0, I3 regressions 0;
fixed/both-fail normal 967/3914, alt 2628/8316, conpty 6657/2937 (was
323/4558, 2296/8648, 5466/4128).

* test(terminal): narrow the I1 carve-out to where the serializer keeps background rows

Trailing background-only rows change bytes only when the serialized range has no
scrollback (the trimming path) and the cursor is not wrap-pending; checkpoints with
scrollback or a wrap-pending cursor are held to byte identity again. The 7000-per-mode
sweep against origin/main stays at I1 0 and I3 0.

* test(native-chat): pin that screen scrapers ignore kept background rows

The serializer now keeps trailing background-only rows, so a painted TUI's screen
read as text ends in \r\n\x1b[NX rows. Serialize the same frame with and without
them and check the Claude option scrape, the empty-prompt check and the fork
transcript read the same thing.

* test(terminal): read xterm core internals through a checked parser, not Reflect.get

The fuzz oracle reached xterm's private _core with Reflect.get, which the
low-evidence gate rejects. Narrow _core, writeSync and the DECSTBM bounds with
in/typeof checks into one named XtermCoreInternals shape instead.

* test(terminal): keep captured serialize transcripts byte-exact on Windows checkouts
2026-09-25 01:18:14 -04:00

332 lines
15 KiB
Diff

diff --git a/src/SerializeAddon.ts b/src/SerializeAddon.ts
index e1728feb219c362dfa2ecb602ff99f830d520757..265b9792403601c286bc341f9d046dcd3062e1be 100644
--- a/src/SerializeAddon.ts
+++ b/src/SerializeAddon.ts
@@ -5,13 +5,66 @@
* (EXPERIMENTAL) This Addon is still under development
*/
-import type { IBuffer, IBufferCell, IBufferRange, ITerminalAddon, Terminal } from '@xterm/xterm';
+import type { IBuffer, IBufferCell, IBufferLine, IBufferRange, ITerminalAddon, Terminal } from '@xterm/xterm';
import type { IHTMLSerializeOptions, SerializeAddon as ISerializeApi, ISerializeOptions, ISerializeRange } from '@xterm/addon-serialize';
import { IColor } from 'common/Types';
import { IAttributeData } from 'common/buffer/Types';
import { DEFAULT_ANSI_COLORS } from 'browser/Types';
import { UnderlineStyle } from 'common/buffer/Constants';
+type OscLinkData = { id?: string; uri: string };
+type OscLinkedCell = { extended?: { urlId?: number } };
+type TerminalWithOscLinks = Terminal & {
+ _core?: {
+ _inputHandler?: { _curAttrData?: IAttributeData & OscLinkedCell };
+ _oscLinkService?: { getLinkData: (linkId: number) => OscLinkData | undefined };
+ };
+};
+
+function getOscLinkId(cell: IBufferCell | IAttributeData): number {
+ return (cell as typeof cell & OscLinkedCell).extended?.urlId ?? 0;
+}
+
+function getOscLinkOpenSequence(terminal: Terminal, linkId: number): string {
+ if (!linkId) {
+ return '';
+ }
+ const data = (terminal as TerminalWithOscLinks)._core?._oscLinkService?.getLinkData(linkId);
+ if (!data) {
+ return '';
+ }
+ const params = data.id === undefined ? '' : `id=${data.id}`;
+ return `\u001b]8;${params};${data.uri}\u001b\\`;
+}
+
+function getActiveOscLinkSequence(terminal: Terminal): string {
+ const attrs = (terminal as TerminalWithOscLinks)._core?._inputHandler?._curAttrData;
+ return attrs ? getOscLinkOpenSequence(terminal, getOscLinkId(attrs)) : '';
+}
+
+// PATCH(orca): a non-reflowing shrink (always the alternate buffer) leaves each
+// line at its old length; cells past the grid are not on screen and would wrap
+// into extra rows on replay, so serialize only the terminal's width.
+function visibleCellCount(line: IBufferLine, cols: number): number {
+ return Math.min(line.length, cols);
+}
+
+// PATCH(orca): a wide glyph whose trailing half fell past the grid cannot be
+// replayed in the last column (it would wrap and shift every row below), so it
+// is serialized as a blank that keeps the row exactly the grid's width.
+function isClippedWideCell(line: IBufferLine, col: number, cols: number, cell: IBufferCell): boolean {
+ return col === cols - 1 && line.length > cols && cell.getWidth() > 1;
+}
+
+// Width 1 with no codepoint (CellData.content layout). A width-0 stand-in would read as a
+// wide glyph's trailer: skipped by _nextCell and treated as content by the wrap check.
+const BLANK_CELL_CONTENT = 1 << 22;
+
+function blankClippedWideCell(cell: IBufferCell): IBufferCell {
+ (cell as IBufferCell & { content: number }).content = BLANK_CELL_CONTENT;
+ return cell;
+}
+
function constrain(value: number, low: number, high: number): number {
return Math.max(low, Math.min(value, high));
}
@@ -27,7 +80,8 @@ function escapeHTMLChar(c: string): string {
// TODO: Refine this template class later
abstract class BaseSerializeHandler {
constructor(
- protected readonly _buffer: IBuffer
+ protected readonly _buffer: IBuffer,
+ private readonly _cols: number
) {
}
@@ -48,13 +102,16 @@ abstract class BaseSerializeHandler {
const line = this._buffer.getLine(row);
if (line) {
const startLineColumn = row === range.start.y ? startColumn : 0;
- const endLineColumn = row === range.end.y ? endColumn: line.length;
+ const endLineColumn = row === range.end.y ? endColumn: visibleCellCount(line, this._cols);
for (let col = startLineColumn; col < endLineColumn; col++) {
- const c = line.getCell(col, oldCell === cell1 ? cell2 : cell1);
+ let c = line.getCell(col, oldCell === cell1 ? cell2 : cell1);
if (!c) {
console.warn(`Can't get cell at row=${row}, col=${col}`);
continue;
}
+ if (isClippedWideCell(line, col, this._cols, c)) {
+ c = blankClippedWideCell(c);
+ }
this._nextCell(c, oldCell, row, col);
oldCell = c;
}
@@ -148,18 +205,22 @@ class StringSerializeHandler extends BaseSerializeHandler {
// this is a null cell for reference for checking whether background is empty or not
private _backgroundCell: IBufferCell = this._buffer.getNullCell();
+ private _defaultCell: IBufferCell = this._buffer.getNullCell();
private _firstRow: number = 0;
private _lastCursorRow: number = 0;
private _lastCursorCol: number = 0;
private _lastContentCursorRow: number = 0;
private _lastContentCursorCol: number = 0;
+ private _activeOscLinkId: number = 0;
+ private _lastBackgroundCursorRow: number = -1;
+ private _lastBackgroundCursorCol: number = 0;
constructor(
buffer: IBuffer,
private readonly _terminal: Terminal
) {
- super(buffer);
+ super(buffer, _terminal.cols);
}
protected _beforeSerialize(rows: number, start: number, end: number): void {
@@ -178,6 +239,9 @@ class StringSerializeHandler extends BaseSerializeHandler {
if (this._nullCellCount > 0 && !equalBg(this._cursorStyle, this._backgroundCell)) {
// use clear right to set background.
this._currentRow += `\u001b[${this._nullCellCount}X`;
+ // PATCH(orca): the erase paints this row, so _serializeString must not trim it as trailing blank.
+ this._lastBackgroundCursorRow = this._lastCursorRow;
+ this._lastBackgroundCursorCol = this._lastCursorCol;
}
let rowSeparator = '';
@@ -202,8 +266,12 @@ class StringSerializeHandler extends BaseSerializeHandler {
this._lastCursorCol = 0;
} else {
rowSeparator = '';
- const thisRowLastChar = currentLine.getCell(currentLine.length - 1, this._thisRowLastChar)!;
- const thisRowLastSecondChar = currentLine.getCell(currentLine.length - 2, this._thisRowLastSecondChar)!;
+ const currentLineCells = visibleCellCount(currentLine, this._terminal.cols);
+ let thisRowLastChar = currentLine.getCell(currentLineCells - 1, this._thisRowLastChar)!;
+ if (isClippedWideCell(currentLine, currentLineCells - 1, this._terminal.cols, thisRowLastChar)) {
+ thisRowLastChar = blankClippedWideCell(thisRowLastChar);
+ }
+ const thisRowLastSecondChar = currentLine.getCell(currentLineCells - 2, this._thisRowLastSecondChar)!;
const nextRowFirstChar = nextLine.getCell(0, this._nextRowFirstChar)!;
const isNextRowFirstCharDoubleWidth = nextRowFirstChar.getWidth() > 1;
@@ -214,7 +282,7 @@ class StringSerializeHandler extends BaseSerializeHandler {
if (
// you must output character to cause overflow, control sequence can't do this
nextRowFirstChar.getChars() &&
- isNextRowFirstCharDoubleWidth ? this._nullCellCount <= 1 : this._nullCellCount <= 0
+ (isNextRowFirstCharDoubleWidth ? this._nullCellCount <= 1 : this._nullCellCount <= 0)
) {
if (
// the last character can't be null,
@@ -251,9 +319,14 @@ class StringSerializeHandler extends BaseSerializeHandler {
if (this._nullCellCount > 0) {
// do these because we filled the last several null slot, which we shouldn't
rowSeparator += '\u001b[A';
- rowSeparator += `\u001b[${currentLine.length - this._nullCellCount}C`;
+ const contentCellCount = currentLineCells - this._nullCellCount;
+ if (contentCellCount > 0) {
+ rowSeparator += `\u001b[${contentCellCount}C`;
+ }
rowSeparator += `\u001b[${this._nullCellCount}X`;
- rowSeparator += `\u001b[${currentLine.length - this._nullCellCount}D`;
+ if (contentCellCount > 0) {
+ rowSeparator += `\u001b[${contentCellCount}D`;
+ }
rowSeparator += '\u001b[B';
}
@@ -310,7 +383,20 @@ class StringSerializeHandler extends BaseSerializeHandler {
}
if (flagsChanged) {
if (cell.isInverse() !== oldCell.isInverse()) { sgrSeq.push(cell.isInverse() ? 7 : 27); }
- if (cell.isBold() !== oldCell.isBold()) { sgrSeq.push(cell.isBold() ? 1 : 22); }
+ // PATCH(orca): bold (1) and dim (2) share the single reset param 22, so
+ // they must be diffed as one intensity group with the clearing 22 emitted
+ // BEFORE any re-set. Upstream's independent per-flag diff could emit
+ // "1;22" (bold set, then wiped by dim's clear — \x1b[2mA\x1b[22m\x1b[1mB
+ // loses B's bold on round-trip) or a bare "22" that drops a still-set
+ // bold/dim, garbling Orca's hidden-terminal snapshot restores.
+ const boldChanged = cell.isBold() !== oldCell.isBold();
+ const dimChanged = cell.isDim() !== oldCell.isDim();
+ if (boldChanged || dimChanged) {
+ const clearsIntensity = (boldChanged && !cell.isBold()) || (dimChanged && !cell.isDim());
+ if (clearsIntensity) { sgrSeq.push(22); }
+ if (cell.isBold() && (boldChanged || clearsIntensity)) { sgrSeq.push(1); }
+ if (cell.isDim() && (dimChanged || clearsIntensity)) { sgrSeq.push(2); }
+ }
if (!equalUnderline(cell, oldCell)) {
const style = cell.getUnderlineStyle();
if (style === UnderlineStyle.NONE) {
@@ -337,7 +423,7 @@ class StringSerializeHandler extends BaseSerializeHandler {
if (cell.isBlink() !== oldCell.isBlink()) { sgrSeq.push(cell.isBlink() ? 5 : 25); }
if (cell.isInvisible() !== oldCell.isInvisible()) { sgrSeq.push(cell.isInvisible() ? 8 : 28); }
if (cell.isItalic() !== oldCell.isItalic()) { sgrSeq.push(cell.isItalic() ? 3 : 23); }
- if (cell.isDim() !== oldCell.isDim()) { sgrSeq.push(cell.isDim() ? 2 : 22); }
+ // PATCH(orca): dim handled in the intensity group above.
if (cell.isStrikethrough() !== oldCell.isStrikethrough()) { sgrSeq.push(cell.isStrikethrough() ? 9 : 29); }
}
}
@@ -346,6 +432,20 @@ class StringSerializeHandler extends BaseSerializeHandler {
return sgrSeq;
}
+ private _setOscLink(linkId: number): string {
+ if (linkId === this._activeOscLinkId) {
+ return '';
+ }
+ let sequence = this._activeOscLinkId ? '\u001b]8;;\u001b\\' : '';
+ this._activeOscLinkId = 0;
+ const openSequence = getOscLinkOpenSequence(this._terminal, linkId);
+ if (openSequence) {
+ this._activeOscLinkId = linkId;
+ sequence += openSequence;
+ }
+ return sequence;
+ }
+
protected _nextCell(cell: IBufferCell, oldCell: IBufferCell, row: number, col: number): void {
// a width 0 cell don't need to be count because it is just a placeholder after a CJK character;
const isPlaceHolderCell = cell.getWidth() === 0;
@@ -356,12 +456,21 @@ class StringSerializeHandler extends BaseSerializeHandler {
// this cell don't have content
const isEmptyCell = cell.getChars() === '';
+ const nextLine = isEmptyCell && cell.isInverse() ? this._buffer.getLine(row + 1) : undefined;
+ const nextRowFirstCell = nextLine?.getCell(0, this._nextRowFirstChar);
+ // A pending wide glyph recreates its own final-column padding during replay.
+ const isWideWrapPadding = col === this._terminal.cols - 1 &&
+ nextLine?.isWrapped &&
+ (nextRowFirstCell?.getWidth() ?? 0) > 1 &&
+ !!nextRowFirstCell && attributesEquals(cell, nextRowFirstCell);
+ // Cursor movement cannot reproduce an inverse cell's visible background.
+ const materializeEmptyCell = isEmptyCell && !!cell.isInverse() && !isWideWrapPadding;
const sgrSeq = this._diffStyle(cell, this._cursorStyle);
- // the empty cell style is only assumed to be changed when background changed, because
- // foreground is always 0.
- const styleChanged = isEmptyCell ? !equalBg(this._cursorStyle, cell) : sgrSeq.length > 0;
+ const styleChanged = isEmptyCell
+ ? materializeEmptyCell ? sgrSeq.length > 0 : !equalBg(this._cursorStyle, cell)
+ : sgrSeq.length > 0;
/**
* handles style change
@@ -395,7 +504,7 @@ class StringSerializeHandler extends BaseSerializeHandler {
/**
* handles actual content
*/
- if (isEmptyCell) {
+ if (isEmptyCell && !materializeEmptyCell) {
this._nullCellCount += cell.getWidth();
} else {
if (this._nullCellCount > 0) {
@@ -411,7 +520,22 @@ class StringSerializeHandler extends BaseSerializeHandler {
this._nullCellCount = 0;
}
- this._currentRow += cell.getChars();
+ // PATCH(orca): styling alone leaves restored OSC 8 links looking live but unclickable.
+ this._currentRow += this._setOscLink(getOscLinkId(cell));
+
+ if (materializeEmptyCell) {
+ const hasDecoration = !!cell.isUnderline() || !!cell.isStrikethrough() || !!cell.isOverline();
+ if (hasDecoration) {
+ this._currentRow += '\u001b[24;29;55m';
+ }
+ this._currentRow += ' '.repeat(cell.getWidth());
+ if (hasDecoration) {
+ const restoreSgrSeq = this._diffStyle(cell, this._defaultCell);
+ this._currentRow += `\u001b[0m\u001b[${restoreSgrSeq.join(';')}m`;
+ }
+ } else {
+ this._currentRow += cell.getChars();
+ }
// update cursor
this._lastContentCursorRow = this._lastCursorRow = row;
@@ -425,6 +549,15 @@ class StringSerializeHandler extends BaseSerializeHandler {
// the fixup is only required for data without scrollback
// because it will always be placed at last line otherwise
if (this._buffer.length - this._firstRow <= this._terminal.rows) {
+ // PATCH(orca): keep trailing background-only rows, unless the cursor is wrap-pending:
+ // relative moves back from those rows cannot re-create that state.
+ if (
+ this._lastBackgroundCursorRow > this._lastContentCursorRow &&
+ this._buffer.cursorX < this._terminal.cols
+ ) {
+ this._lastContentCursorRow = this._lastBackgroundCursorRow;
+ this._lastContentCursorCol = this._lastBackgroundCursorCol;
+ }
rowEnd = this._lastContentCursorRow + 1 - this._firstRow;
this._lastCursorCol = this._lastContentCursorCol;
this._lastCursorRow = this._lastContentCursorRow;
@@ -439,6 +572,9 @@ class StringSerializeHandler extends BaseSerializeHandler {
}
}
+ // Each buffer is self-contained so links cannot leak into a following buffer.
+ content += this._setOscLink(0);
+
// restore the cursor
if (!excludeFinalCursorPosition) {
const realCursorRow = this._buffer.baseY + this._buffer.cursorY;
@@ -616,6 +752,9 @@ export class SerializeAddon implements ITerminalAddon, ISerializeApi {
content += this._serializeScrollRegion(this._terminal);
}
+ // Restore the source terminal's live OSC pen only after all buffers are complete.
+ content += getActiveOscLinkSequence(this._terminal);
+
return content;
}
@@ -642,7 +781,7 @@ export class HTMLSerializeHandler extends BaseSerializeHandler {
private readonly _terminal: Terminal,
private readonly _options: Partial<IHTMLSerializeOptions>
) {
- super(buffer);
+ super(buffer, _terminal.cols);
// For xterm headless: fallback to ansi colors
if ((_terminal as any)._core._themeService) {