Files
orca/tests/e2e/terminal-ime-midline-occlusion-probe.ts
Neilandrayim 453237cc57 fix(terminal): render the row tail the IME preedit overlay covers (#15014)
* fix(terminal): render the covered row tail inside the IME preedit overlay

Closes #12545.

Composing mid-line hid the character at the cursor for the whole composition.
The preedit overlay is an opaque box anchored to the cursor cell, and nothing
reaches the pty while composing, so those cells still held their characters —
the box simply covered them.

`CompositionHelper` now draws the rest of the row after the preedit inside the
view, so the composition reads as inserted text pushing the tail right. Four
details come with it:

- The view is start-anchored while it carries a tail, so the preedit stays put
  and the pushed tail clips at the right edge; alone, `rtl` still keeps a long
  preedit's end in view.
- It is themed from `options.theme` instead of the stock `#000`/`#FFF`, with any
  alpha dropped — the view masks the cells it draws over, so a see-through
  background would re-expose the very characters the tail stands in for.
- The helper textarea syncs to the preedit's own bounds, so IME candidate
  dialogs anchor to the composing text rather than past the rendered tail.
- A TUI can repaint the row under an open composition, so
  `updateCompositionElements` — which already runs on every render — re-reads
  the remainder and re-renders on change. A string compare adds no layout read.

The tail is read with an explicit end column: the cacheable form of
`translateToString` arms the line string cache's self-renewing idle-clear timer,
and the composition path must own no timers.

Geometry is not the cause. Two mature reference terminal implementations compose
marked text into the grid rather than into a floating box, and both still blank
the cells under it — one of them literally substitutes the marked characters
into the row's character array before rasterizing. Moving off the overlay would
not have fixed this report; rendering the covered tail is what does.

The e2e arm asserts the invariant an opaque overlay owes the grid: it must
render every committed cell its bounding rect covers. That is measured from the
real rect against the real cell grid, so it fails on the unfixed build with
`covers "하" / renders "가"`.

Known limitation: the rendered tail is plain-styled while composing (theme
foreground on theme background, no per-cell colors); colors return on commit.
This is inherent to the overlay, and drawing the preedit into the cell renderer
instead would be a far larger change.

Co-authored-by: rayim <rayim@fxy.global>

* test(e2e): assert the occlusion invariant, not the runner's cell width

CI covered four columns where this machine covers two — 34.4px over an 8.43px
grid against 12.3px over an 8px grid — so pinning the covered text verbatim
pinned the font metrics rather than the behaviour. Assert instead that every
committed cell the overlay covers appears in what it draws, which is the actual
invariant and holds at any cell width.

Still fails against main: covers "하" / renders "가".

* fix(terminal): keep the rendered tail's spacing on the grid

The composition view is white-space: nowrap, which collapses runs of spaces
exactly like normal — it only suppresses wrapping. So a committed tail carrying
padding drew its trailing glyph cells left of where the grid has them: measured
in Chromium with xterm's own rule, twenty spaces plus a border rendered two
cells wide instead of twenty-one.

The visible case is Orca's most common IME context — composing inside an agent
TUI input box, where the row is a prompt, padding, then a real border glyph the
trim cannot drop. A stray border appeared a cell after the preedit while the
real one stayed put.

xterm sets white-space: pre on its grid rows for this reason; the view was only
nowrap-safe while it held preedit text alone.

The existing fixtures are all space-free, and the e2e invariant is that the
overlay renders everything it covers — collapsing makes it cover less, so both
stayed green. Pinned with a padded-row fixture.

---------

Co-authored-by: rayim <rayim@fxy.global>
2026-08-17 00:18:11 -07:00

124 lines
5.0 KiB
TypeScript
Raw Permalink Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import type { Page } from '@stablyai/playwright-test'
/**
* Measures what an active preedit overlay hides, against what it renders.
*
* The overlay has an opaque background and is absolutely positioned over the grid, so every cell
* its bounding rect covers is unreadable for the duration of the composition. That gives one
* invariant worth asserting on screen: **the overlay must render everything it covers**. Composing
* mid-line broke it — the box covered the character at the cursor and rendered only the preedit.
*
* This has to be measured from the real rect against the real cell grid. The class, the
* `textContent` and `display` are all identical either way, and a DOM emulator reports every rect
* as zero, so a unit-level arm cannot see the difference.
*/
export type MidlinePreeditOcclusionSample = {
/** Row text from the cursor rightwards — the committed characters a mid-line preedit sits over. */
rowTailFromCursor: string
/** Characters in the grid columns the overlay's bounding rect covers. */
hiddenByOverlay: string
/** Overlay text as rendered, LRM marks stripped. */
overlayText: string
overlayActive: boolean
cursorColumn: number
/** Columns covered by the overlay, as `[first, last]`; null when it covers none. */
coveredColumns: [number, number] | null
cellWidth: number
/** Kept in the sample so a geometry mismatch prints the rects that produced it. */
overlayRect: { left: number; right: number; width: number }
screenRect: { left: number; width: number }
}
function readMidlinePreeditOcclusion(): MidlinePreeditOcclusionSample {
const state = window.__store?.getState()
const worktreeId = state?.activeWorktreeId
const tabId =
state?.activeTabType === 'terminal'
? state.activeTabId
: worktreeId
? (state?.activeTabIdByWorktree?.[worktreeId] ?? null)
: null
const manager = tabId ? window.__paneManagers?.get(tabId) : null
const pane = manager?.getActivePane?.() ?? manager?.getPanes?.()[0] ?? null
if (!pane) {
throw new Error('No active terminal pane to sample')
}
const terminal = pane.terminal
const screen = pane.container.querySelector<HTMLElement>('.xterm-screen')
const view = pane.container.querySelector<HTMLElement>('.composition-view')
if (!screen || !view) {
throw new Error('Active terminal has no screen element or composition view')
}
const buffer = terminal.buffer.active
const line = buffer.getLine(buffer.baseY + buffer.cursorY)
const cursorColumn = buffer.cursorX
const rowTailFromCursor = (line?.translateToString(true, cursorColumn) ?? '').trimEnd()
const screenRect = screen.getBoundingClientRect()
const overlayRect = view.getBoundingClientRect()
const cellWidth = terminal.cols > 0 ? screenRect.width / terminal.cols : 0
// A column counts as hidden when the overlay covers most of its cell, which keeps the sample
// stable against the sub-pixel width a proportional text node lands on.
let first: number | null = null
let last = -1
let hiddenByOverlay = ''
if (cellWidth > 0 && overlayRect.width > 0) {
for (let column = 0; column < terminal.cols; column++) {
const cellLeft = screenRect.left + column * cellWidth
const overlap =
Math.min(cellLeft + cellWidth, overlayRect.right) - Math.max(cellLeft, overlayRect.left)
if (overlap <= cellWidth / 2) {
continue
}
first ??= column
last = column
hiddenByOverlay += line?.getCell(column)?.getChars() ?? ''
}
}
return {
rowTailFromCursor,
hiddenByOverlay: hiddenByOverlay.trimEnd(),
// The overlay wraps its text in LRM marks; strip them so assertions read as the user sees it.
overlayText: (view.textContent ?? '').replaceAll('‎', ''),
overlayActive: view.classList.contains('active'),
cursorColumn,
coveredColumns: first === null ? null : [first, last],
cellWidth,
overlayRect: {
left: overlayRect.left,
right: overlayRect.right,
width: overlayRect.width
},
screenRect: { left: screenRect.left, width: screenRect.width }
}
}
export async function sampleMidlinePreeditOcclusion(
page: Page
): Promise<MidlinePreeditOcclusionSample> {
return page.evaluate(readMidlinePreeditOcclusion)
}
/** Writes straight to the emulator: the defect is in what the overlay draws, not in the pty. */
export async function writeToActiveTerminal(page: Page, data: string): Promise<void> {
await page.evaluate(async (payload: string) => {
const state = window.__store?.getState()
const worktreeId = state?.activeWorktreeId
const tabId =
state?.activeTabType === 'terminal'
? state.activeTabId
: worktreeId
? (state?.activeTabIdByWorktree?.[worktreeId] ?? null)
: null
const manager = tabId ? window.__paneManagers?.get(tabId) : null
const pane = manager?.getActivePane?.() ?? manager?.getPanes?.()[0] ?? null
if (!pane) {
throw new Error('No active terminal pane to write to')
}
await new Promise<void>((resolve) => pane.terminal.write(payload, resolve))
}, data)
}