Files
orca/src/relay
Jinjing ce5b639e03 fix(P1-B): recover SSH targets and remote file watchers after a network drop (#12032)
* fix(P1-B): recover system-SSH targets after a network drop

Two defects stopped a remote workspace auto-recovering after a blip.

runReconnectAttempt classified failures with isTransientError, which only
matches ETIMEDOUT/ECONNREFUSED/ECONNRESET by errno code or literal
substring. The system-SSH transport — the only transport FIDO2 and
ProxyUseFdpass targets can use — reports network failures as OpenSSH
prose ("System SSH connection timed out"), so the ladder published a
permanent 'error' on the first timeout and the target never came back
without a manual reconnect. isTransientReconnectError adds a
network-shaped prose table on top of isTransientError and is used only on
the reconnect path: connect() keeps the narrow classifier so an
unreachable host still fails fast instead of burning five 30s attempts
and five security-key touch prompts. Auth and passphrase failures stay
permanent on both paths.

runReconnectAttempt also had no generation fence, so a superseded attempt
published its cancellation as a permanent error over the winner's live
connection — reachable when a system-transport proc.onExit schedules a
reconnect while an attempt is still in flight. Cancellation now carries a
stable error name, and both connect() and runReconnectAttempt claim their
connectGeneration and stay silent when a newer attempt owns the state.

* fix(P1-B): retry a dropped watcher overflow marker on real capacity

emitWatcherOverflowToClient published the {kind:'overflow'} resync marker
with controlOverflow:'reject'. A full control queue rejects at admission
with no settlement callback, so the marker was silently discarded and the
remote File Explorer stayed stale until some later watcher event happened
to produce another one — for a quiet tree, possibly never.

The emitter now retains a rejected marker per (client, root) and
republishes it when the sink actually frees up. The existing
onLegacyPtyCapacity signal cannot drive that: it is gated on producer
retention, so it stays silent exactly under the dual-queue pressure that
caused the rejection. RelayDispatcher.onClientCapacity is an ungated
per-client capacity signal that fires on every writer settlement and
drain. It lives on the dispatcher rather than the writer so a retained
marker survives setWrite() replacing the primary sink, and setWrite
notifies capacity once afterwards so the marker does not wait on traffic
that may never arrive.

Retention is bounded to one marker per (client, root), released on
settlement and purged on client detach.

* fix(P1-B): address all review findings on SSH network recovery

Fix four issues from code review:

1. **Bug — admitted overflow markers lost on setWrite**: Retain markers when
   settlement fails `ok: false`, not just on admission rejection. Prevents
   desynced filesystem trees after SSH sink replacement.

2. **SSH error classification expanded**: Add missing OpenSSH patterns
   (`ssh_exchange_identification`, `connection closed by remote`) and new
   `isDefiniteSystemSshHostFailure()` classifier.

3. **ControlMaster retry optimization**: Skip second probe when first failure is
   already definite host-level (network timeout, refused, unreachable). Saves
   ~30s per reconnect ladder step.

4. **Overflow flush under dual-queue pressure**: Gate pending marker retries on
   control-lane headroom instead of re-attempting on every capacity notification.
   Reduces thrash proportional to producer traffic.

Add regression tests for marker republish on sink replacement and validate auth
error detection against live OpenSSH credential rejection messages.

* rm random doc

* fix(P1-B): skip credential-failure retries and recover watcher markers o

- Auth and passphrase errors fail immediately without retry attempts
- Bare "System SSH probe failed (exit 255)" is transient only for reconnect
- Watcher markers survive client invalidation when switching SSH connections
- Add network error patterns: "lost connection", "remote end closed"
2026-08-01 22:21:37 -07:00
..