diff --git a/notes/terminal-performance-initiative.md b/notes/terminal-performance-initiative.md new file mode 100644 index 00000000000..66c6441f55d --- /dev/null +++ b/notes/terminal-performance-initiative.md @@ -0,0 +1,186 @@ +# Terminal Performance Initiative + +Working plan for the `orca-performance` branch. Goal: make Orca's terminal as +performant as the architecture allows, with every claim backed by a number. +Started 2026-07-02. + +## Why (user-reported, from the team meeting) + +1. Typing in the terminal is sometimes laggy — occasionally seconds of delay. +2. Users say the terminal is slower than iTerm (unclear if typing or scrolling). +3. Scrolling in Claude Code / OpenCode is slow. +4. Idle memory is high (1–2 GB). +5. Battery usage is high. + +Goals: legit performance complaints ≤ 1/week; sampled P90 typing/scrolling +latency down significantly; lower memory with 0–1 agents. + +## Ground truth (verified against source, 2026-07-02) + +Research corpus: xterm.js 6 / VS Code / Ghostty internals study (verified +file:line claims) — see the archived digest and the "xterm.js vs Ghostty" +deep-dive. The Orca-specific findings below were re-verified against this +repo's code: + +- **Electron main sits on every terminal byte's path** (daemon → main → + renderer). VS Code ships the same xterm.js but bypasses main entirely: its + ptyHost is a UtilityProcess with a direct MessagePort to each renderer. +- **The PTY producer is never paused.** `acknowledgeDataEvent` is a no-op in + both `LocalPtyProvider` and `DaemonPtyAdapter`. Only main→renderer delivery + is watermarked (512 KB, `src/main/ipc/pty.ts:1374`); main's own buffer can + grow toward a 512 MB cap under flood. VS Code pauses the actual pty at 100k + unacked chars (kernel backpressure blocks the shell). +- Renderer terminals share one thread with the entire React app; xterm.js + parses in 12 ms slices at a documented 5–35 MB/s ceiling. +- Renderer scrollback default is 5,000 rows (`src/shared/terminal-scrollback-policy.ts`), + 5× VS Code's default; 12 B/cell plus per-line JS objects; O(all lines) + reflow on column resize. +- Latency physics: Ghostty ~4 ms median keypress latency, VS Code ~31 ms + (same-library reference), native class 5–10 ms. Realistic target: beat + VS Code, close on iTerm2, eliminate the stall/jank class entirely (P99 + dominates perception). + +## Current state + +Branch `orca-performance` (long-lived testing line, from main @ `8e8a08ac7`): + +1. `tools/benchmarks/terminal-pipeline-bench.mjs` — cross-terminal rig + (see Benchmark protocol below). +2. Merge of PR #7153 = #7150 (freeze/memory: backlog caps, wedge guards, + probe-certified replay release) + #7139 (cooperative drain: paced backlog + draining keeps typing responsive under floods). Post-merge on this base: + `pnpm typecheck` clean, 626 targeted tests green (scheduler, guards, + pty/pty-connection/pty-transport suites). #7153 itself is a disposable + testing PR; #7139 and #7150 land separately on main. + +## Workstreams + +### 1. Baseline benchmarks (now; human-in-terminal required) + +Run the rig in each terminal on the same machine — Orca pane, iTerm2, Ghostty, +Terminal.app, VS Code (T3Code if available): + +``` +node tools/benchmarks/terminal-pipeline-bench.mjs --label - +node tools/benchmarks/terminal-pipeline-bench.mjs report +``` + +These numbers answer "are we actually slower than iTerm, and where," and are +the before/after for everything below. + +### 2. Validate #7153 on orca-performance (this week, extended testing) + +Watch for: typing responsiveness under agent floods, bounded memory, +skip-notice + snapshot repaint on overflow, no permanent input loss. When +validated, land #7139 and #7150 as separate PRs on main. + +### 3. Revive term-speed-2 (the headline structural work) + +History: nwparker's ~38-branch chain (+20k lines) implementing the terminal +model/view contract — hidden view parking, hidden delivery gate, side-effect +authority in main, model query authority, skip-grammar deletion — all +kill-switched, documented in +`origin/nwparker/term-speed-2-architecture-docs:docs/reference/terminal-model-view-contract.md`. +It shipped only in v1.4.78-rc.1, a deliberate personal-testing build; it was +never rejected and never reached main. Directly targets complaints 3–5 +(hidden panes stop receiving bytes and unmount their xterm + WebGL atlases). + +Merge scout (2026-07-02, chain tip into orca-performance): 144 files, 34 +conflicted, 115 hunks. Hotspots: `pty-connection.ts` (31), `pty.ts` (16), +`daemon-pty-adapter.ts` (6), `orca-runtime.ts` (5). +`pane-terminal-output-scheduler.ts` does NOT conflict — #7139/#7150 and the +chain touch different layers; runtime interaction (drain pacing × hidden +gate) still needs deliberate testing. + +Execution: dedicated focused session; resolve on `revive/term-speed-2` off +orca-performance; keep both sides' kill switches; validate with typecheck + +the contract tests listed in the model-view-contract doc + #7153's suites; +merge back to orca-performance for extended testing. Estimated ~1 day of +careful resolution + validation. + +### 4. Remaining stall-bug fixes (parallel, independently shippable) + +The "seconds of delay" class = discrete thread-blocking events, not +steady-state latency: + +- PR #7105 (open): skip synchronous cold-restore replay for live daemon + sessions in doSpawn. +- `SerializeAddon.serialize()` audit: ~1.2 s renderer block at 50k scrollback + rows (#5096 follow-up, never done). Call sites include the mobile snapshot + path (`pty-connection.ts:2861`) and sleep/hibernate serialization. +- #2836 frozen-terminal leads: replay-guard latch, codex-stale gate, uncapped + buffers (repro harness exists). +- Checkpoint-RPC main-thread scrub (measured ~2–10 ms bursts per hot 5 s + tick; small, part of the same program). + +### 5. Producer-side PTY flow control + +Ack-driven pause/resume of the actual PTY through the daemon protocol +(node-pty supports it), watermarks per the xterm.js flow-control guide +(≤500 KB). Converts flood-induced buffered lag into shell blocking — the +correct physics. Sequence after #7139 lands (interacts with its drain pacing). + +### 6. Extend the measurement rig + +- True keypress→pixel latency: Typometer manual protocol (the DSR probe stops + at the parser reply, before paint). +- Idle memory + battery: per-process RSS breakdown + `powermetrics` sampling + at 0/1/5 agents (goal-3 metric). +- FPS under flood; event-loop-delay probes (`monitorEventLoopDelay`) in + main/daemon/renderer behind a debug flag for pipeline attribution. + +### 7. utilityProcess terminal router (structural endgame; gated on data) + +An Electron UtilityProcess owns the daemon socket and hands each renderer a +MessagePort — VS Code's topology while keeping Orca's detached daemon (warm +reattach). Takes main off the terminal data path entirely; daemon-side +history persistence falls out naturally. Prototype only after baselines show +how much tail latency lives in the main hop. + +### 8. Production P90 telemetry + +Sampled keypress→echo latency + long-task/stall counts from real users; +defines the success criterion and becomes the permanent regression gate. +Design after the local rig stabilizes so the metrics match. + +## Benchmark protocol + +`tools/benchmarks/terminal-pipeline-bench.mjs` measures, from inside any +terminal: + +- **DSR idle latency** — ESC[6n round trips (p50/p90/p99); replies come only + after the parser reaches the query, so it proxies the input pipeline + without keystroke injection. +- **Fenced throughput** — 4 deterministic fixtures (`ascii-log`, `cjk-emoji`, + `agent-tui` — Claude-Code-shaped transcript + DEC-2026 status repaints — + and labeled-pathological `styles-stress`), each run ended by a DSR fence so + xterm.js-class ingest queues can't flatter the result. +- **DSR under load** — latency sampled during a paced 1 MB/s agent-TUI + stream: "typing while the agent works," quantified. + +Rules: same machine, AC power, comparable window size, no tmux/screen, hands +off the keyboard during runs. Never compare numbers across machines. + +## Sequencing + +``` +now: [1] baselines [2] #7153 testing (parallel) +next: [3] term-speed-2 revival (dedicated session) +parallel: [4] stall fixes, [6] rig extensions +after 2/3: [5] flow control +gated: [7] utility router [8] telemetry +``` + +BMW-group crash work remains the team's priority gate above all of this +(#7150's wedge guards overlap it); this plan runs measurement and revival +prep in parallel without displacing it. + +## Success criteria (baseline-relative; finalize after task 1) + +- DSR-under-load p90 in Orca within striking distance of iTerm2 on the same + box; zero DSR timeouts (today's freeze class). +- Fenced agent-tui throughput ≥ VS Code on the same box. +- Idle RSS with 0–1 agents materially down (target set after the memory + harness lands; hidden-pane parking is the main lever). +- Zero >100 ms event-loop stalls in main/renderer during a 10 MB flood. +- Production P90 typing latency down and monitored continuously.