Merge origin/main (v1.4.127-rc.0) into orca-performance

46 commits from main. Conflict resolutions (branch structure wins, main semantics grafted): - pending-backlog cap (#7630): main's capPendingPtyData/droppedBacklog 2MB tail-trim EXCISED — this branch's droppedOutput sentinel mechanism (O(1) drop + query salvage + settings-scaled cap + restore marker) already bounds the unsent backlog and strictly supersedes it; carrying both would race two cap policies over one buffer. Kept main's makePtyDataPayload helper and exit-time pending flush (adapted to the sentinel). Main's duplicate never-ACK cap test removed (covered by the existing sentinel cap tests); its reload-gate tests kept with the runtime mock extended. - replay guard (#7661): main's blind REPLAY_GUARD_RELEASE_FALLBACK_MS release dropped — this branch's probe-certified stall release covers the same cold-restore reattach lockout without the auto-reply leak a timed release can cause in agent TUIs. Main's three regression tests KEPT, adapted to probe timing (explicit stallCheckMs). - hidden-pressure scenario: kept this branch's STRICT 1500ms restore budget (the background keep-tail global budget earns it; main relaxed to 4s for CI drain overhead) + grafted main's timer-drift gate const. - ssh-docker-relay-perf: kept this branch's ACK-stalled background PTY test AND grafted main's new file-streams+git-churn test (4 tests). - preload/renderer: droppedBacklog threading excised with the mechanism; main's unrelated changes (ssh importConfig reAdopt, etc.) kept. Gates: typecheck clean; ipc+daemon+terminal-pane+pane-manager suites 4,396 passed / 14 skipped. Co-authored-by: Orca <help@stably.ai>
2026-10-17 16:02:55 +00:00 · 2026-07-07 15:57:43 -04:00
parent d0b067e7f0 599f9f2138
commit e9cc89c476
227 changed files with 11285 additions and 1282 deletions
@@ -1,5 +1,5 @@
-<svg xmlns="http://www.w3.org/2000/svg" width="113" height="20" role="img" aria-label="downloads: 3.7m">
-  <title>downloads: 3.7m</title>
+<svg xmlns="http://www.w3.org/2000/svg" width="113" height="20" role="img" aria-label="downloads: 3.9m">
+  <title>downloads: 3.9m</title>
  <linearGradient id="s" x2="0" y2="100%">
    <stop offset="0" stop-color="#bbb" stop-opacity=".1"/>
    <stop offset="1" stop-opacity=".1"/>
@@ -15,7 +15,7 @@
  <g fill="#fff" text-anchor="middle" font-family="Verdana,Geneva,DejaVu Sans,sans-serif" text-rendering="geometricPrecision" font-size="11">
    <text x="37" y="15" fill="#010101" fill-opacity=".3">downloads</text>
    <text x="37" y="14">downloads</text>
-    <text x="93.5" y="15" fill="#010101" fill-opacity=".3">3.7m</text>
-    <text x="93.5" y="14">3.7m</text>
+    <text x="93.5" y="15" fill="#010101" fill-opacity=".3">3.9m</text>
+    <text x="93.5" y="14">3.9m</text>
  </g>
 </svg>
@@ -238,10 +238,6 @@ Conecte ao app desktop para monitorar e conduzir seus agentes pelo celular.

 - **Discord:** Entre na comunidade no **[Discord](https://discord.gg/fzjDKHxv8Q)**.
 - **Twitter / X:** Siga **[@orca_build](https://x.com/orca_build)** para atualizações e anúncios.
- **WeChat:** Escaneie o QR code para entrar na comunidade.
-
-  <img src="../assets/wechat-qr.png" alt="QR code do WeChat para a comunidade Orca" width="160" />
-
 - **Feedback e ideias:** Lançamos rápido. Sentiu falta de algo? [Peça um novo recurso](https://github.com/stablyai/orca/issues).
 - **Privacidade:** Veja a [documentação de privacidade e telemetria](https://www.onorca.dev/docs/telemetry) para saber quais dados anônimos de uso o Orca coleta e como desativar.
 - **Mostre apoio:** Dê uma [estrela](https://github.com/stablyai/orca) neste repositório para acompanhar nossos lançamentos diários.
@@ -235,6 +235,11 @@ yay -S stably-orca-bin

 - **Discord:** 加入 **[Discord](https://discord.gg/fzjDKHxv8Q)** 社区。
 - **Twitter / X:** 关注 **[@orca_build](https://x.com/orca_build)** 获取更新和公告。
+- **微信:** 扫描二维码加入社区。如果第一个群已满，请使用备用微信群。
+
+  <img src="../assets/wechat-qr.png" alt="Orca 社区微信群二维码" width="160" />
+  <img src="../assets/wechat-qr-backup.jpg" alt="Orca 社区备用微信群二维码" width="160" />
+
 - **反馈与想法:** 我们发布很快。缺少什么功能？[提交功能请求](https://github.com/stablyai/orca/issues)。
 - **隐私:** 查看[隐私与遥测文档](https://www.onorca.dev/docs/telemetry)，了解 Orca 收集哪些匿名使用数据以及如何退出。
 - **支持我们:** 给这个仓库点 [Star](https://github.com/stablyai/orca)，关注我们的日常发布。
@@ -0,0 +1,115 @@
+# SSH Typing Latency Under Relay Load
+
+Date: 2026-07-06
+
+## Symptom
+
+Typing in an SSH-host terminal can become extremely slow (hundreds of ms to
+seconds per echoed keystroke) while other SSH work is active — file previews,
+source-control refresh, search. A quiet SSH shell stays snappy, which made the
+reports look unreproducible.
+
+This is distinct from the local warm-switch lag documented in
+`terminal-switch-typing-lag-investigation.md` (daemon `listSessions()`
+snapshot cost — already fixed; `TerminalHost.listSessions()` is now
+metadata-only via `getAppliedSize()`).
+
+## Root Cause
+
+The relay and the Electron client share ONE ordered SSH channel for all
+JSON-RPC traffic: PTY input/output, file streams, git responses, search, port
+scans. Two mechanisms turned bulk traffic into typing latency:
+
+1. **Relay-side head-of-line blocking (primary).** The `fs.readFileStream`
+   pump (`src/relay/fs-handler-file-read.ts`) wrote every 256KB chunk (~340KB
+   framed) into the relay's stdout as fast as local disk reads completed,
+   ignoring the `write() === false` backpressure signal. A 10MB preview
+   enqueued ~13.6MB into the pipe at once; a `pty.data` echo emitted
+   mid-stream queued behind ALL of it. At 2MB/s WAN that is multiple seconds
+   of echo delay per open file. The reproduced measurement: 4,195,592 bytes
+   (the entire remaining 3MB test file, framed) queued ahead of one echo
+   frame.
+
+2. **Client-side O(n²) frame buffering (secondary).** `FrameDecoder.feed()`
+   in both `src/main/ssh/relay-protocol.ts` and `src/relay/protocol.ts` did
+   `Buffer.concat([buffered, chunk])` per data event, re-copying the whole
+   backlog for every ~32KB TCP chunk — ~2MB of memcpy per 340KB frame,
+   ~80MB per 10MB file — on the Electron main thread, delaying `pty:write`
+   IPC dispatch and echo delivery to the renderer.
+
+## Fix
+
+- **Bulk lane with sink backpressure** (`RelayDispatcher.notifyBulk`):
+  `fs.streamChunk` frames are serialized per client and each send waits for
+  the sink's `drain` when `write()` returns `false`. Interactive frames
+  (`pty.data`) still use plain `notify()` and are admitted immediately —
+  because the bulk lane keeps the outbound buffer at ~1 frame, an echo jumps
+  ahead of every not-yet-admitted chunk. Sinks (`relay.ts` stdout + Unix
+  socket clients) surface `write()`'s boolean and a one-shot
+  `waitWriteDrain`; every death path (EPIPE, stdin end, detach, dispose)
+  flushes parked waiters so a pump can never hang on a dead pipe.
+
+- **Credit-based flow control** (`fs.streamAck`): the client requests
+  streams with `flowControl: 'ack'` and acks each processed chunk; the relay
+  caps unacked chunks at `STREAM_ACK_WINDOW_CHUNKS = 4` (~1MB raw). This
+  bounds in-flight bulk bytes even past the relay's own pipe (sshd/TCP
+  buffers) and paces the relay's base64/JSON encode loop so incoming
+  keystroke frames get event-loop turns. A parked pump wakes on ack, cancel,
+  release, client detach, and a 1s staleness recheck.
+
+- **Cross-version compatibility**: old client + new relay → no
+  `flowControl` param → legacy unpaced pump (still drain-bounded, which is
+  transparent). New client + old relay → `fs.streamAck` is an unknown
+  notification and is silently ignored. `STREAM_CHUNK_SIZE` (256KB) is
+  intentionally unchanged — both sides bake it into chunk offset math, so
+  changing it would corrupt cross-version streams.
+
+- **FrameDecoder rewrite** (both mirrored protocol files): chunk-list
+  buffering; each frame is assembled exactly once (one copy) instead of
+  re-concatenating the backlog per feed.
+
+## What Stays Fast / Unchanged
+
+- Relay PTY output batching (interactive echo fast path in
+  `src/relay/pty-handler.ts`) is untouched.
+- No `listSessions()` / provider inventory was added anywhere near typing,
+  focus, switch, resume, resize, or render paths.
+- Remote file streaming correctness: per-chunk length checks, chunk-count and
+  byte-count invariants, cancel paths, and the 12MB round-trip integration
+  test all pass unchanged.
+
+## Regression Coverage
+
+- `src/relay/fs-stream-pty-echo-backpressure.integration.test.ts` — real
+  mux ↔ dispatcher ↔ FsHandler over a congestible in-memory pipe; asserts
+  deterministic BYTE bounds (not wall-clock): echo queues behind < 2 framed
+  chunks when congested (pre-fix: whole file), ack window caps in-flight
+  chunks, legacy no-ack clients still get complete streams.
+- `src/relay/fs-handler-stream.test.ts` — pump parks at the ack window,
+  resumes per ack, and releases its file handle when cancelled while parked.
+- `src/relay/dispatcher.test.ts` — notifyBulk semantics: drain gating,
+  per-client targeting, dispose releases parked senders, interactive
+  notify() not gated behind a stalled bulk lane.
+- `src/main/ssh/relay-protocol.test.ts` — decoder byte-at-a-time boundary
+  straddling, oversized-frame resync across odd chunk sizes, and a
+  no-`Buffer.concat`-during-feed guard that locks out the O(n²) shape.
+- `tests/e2e/ssh-docker-relay-perf.spec.ts` — new "busy relay" scenario:
+  types while two 8MB remote file-read loops and a git.status loop run;
+  budgets median < 500ms, worst < 2000ms. (On loopback Docker the pre-fix
+  HOL is only tens of ms — the in-process byte-bound test is the
+  authoritative red/green; the e2e guards the end-to-end wiring.)
+
+## Residual Gaps
+
+- **Large single-frame responses**: `git.*` responses (stdout capped at
+  `MAX_GIT_BUFFER` = 10MB → ~13MB framed) and other request/response results
+  are one atomic frame on the wire; a huge diff can still delay an echo by
+  its own transfer time. Fixing this requires response chunking (a protocol
+  change); bounded by the 16MB `MAX_MESSAGE_SIZE`.
+- **Outbound direction**: a large `fs.writeFile` request frame (client →
+  relay) can queue ahead of keystroke `pty.data` frames on `channel.stdin`.
+  Same shape, opposite direction, rarer trigger (saving a large remote file
+  while typing).
+- `pty.ackData` flow control for PTY output remains unenforced
+  (`src/relay/pty-handler.ts` — "not yet enforced"); PTY output floods are
+  already paced by the relay's 8ms batch/16KB slice scheduler.