Charge reconnect attempts when connection restores mid-backoff, preventing
infinite loop on transports that flap between states. Refactor control overflow
handling to use entry property instead of WeakSet marker for clarity.
Replay is never retried, so it uses the control lane where overflow
is fatal — the writer closes the client and reconnect reloads history
rather than stranding a short buffer.
Validate that shed subagent rosters match their wire digest and turn identity before
restoration, preventing stale roster resurrection. Compact interactive prompts for waiting
states instead of dropping them. Demote control-queue overflow to non-fatal rejection so
clients can retry on capacity recovery, keeping the link alive during transient backpressure.
* fix(runtime): drop stale local agent rows from worktree.ps after tab close
attachAgentRowsToSummaries attached every hydrated hook row by worktreeId
with no check that the pane/tab still exists, so agents from closed tabs
(last-status.json hydrates for days) kept showing on mobile as current
activity. Local rows now require the tab in a session/runtime graph or a
connected PTY; remote rows are exempt since their tabs may only exist on
the remote host.
Fixes#6072
* fix(runtime): resolve legacy numeric pane keys through the stale-row filter
Non-UUID leaves produce tabId:paneRuntimeId keys with no tabId field;
without parsing them the stale filter was bypassed entirely for such rows.
* fix(runtime): filter stale WSL agent rows
* fix(runtime): ignore persisted tabs for agent liveness
* fix(runtime): restore session-tab liveness and thread OSC transport through the stale-row filter
Review loop pass 1 (3 independent same-model reviewers, findings converged):
- Revert a829e8f9cf's `!this.tabs.has(tabId)` to `mirroredWorktreeId ===
undefined`. The renderer graph is structurally empty under headless serve
(index.ts publishes {tabs: [], leaves: []}), is cleared by
markGraphUnavailable, and omits unvisited/cold-parked workspaces, so
graph-only existence dropped live agent rows in all those states and broke
worktree.ps/session.tabs.list parity. Every close path prunes the persisted
tab, so session tabs remain valid liveness evidence; the stale-persisted-tab
premise did not survive tracing.
- Restore the rename and legacy-pane-key tests to their session-only fixtures
(the graph syncs added with the flipped predicate masked the contract
change) and pin the restored contract in a named test.
- Thread the pane's connectionId through RuntimeAgentRowSnapshot so
OSC-retained rows keep the SSH exemption; previously a fresher OSC ping
hardcoded null and stripped it.
- Pin each rescue conjunct individually (paneKey-only, tabId-only, ptyId after
binding clear), the WSL keep direction, the unresolvable-paneKey guard, and
row presence in the freshness cases.
* fix(runtime): carry the OSC-observed ptyId when a hook row wins the freshness race
Hook payloads have no ptyId field, so overwriting the rowSources entry
discarded the OSC-observed one and the connected-PTY ptyId rescue went dead
for hook-fresh panes during a binding-clear window (pass-2 review P3). Also
corrects the incarnation-change comment on the OSC rescue test.
* batch remote watcher events and defer File Explorer refreshes
Remote filesystem watcher events now batch with the shared 150ms trailing and 500ms
max-wait window, coalescing per-path like local events. File Explorer tree and
directory refreshes are scheduled with debounce and transport-aware concurrency caps
(16 local, 8 runtime, 4 SSH). Stale directory cache tracking prevents trusting
collapsed listings skipped by full refresh; they are re-read on re-expansion. Relay
implements a 15-minute idle-only grace cap for zero-PTY relays via PTY pool
lifecycle tracking, independent of explicitly configured grace time.
* fix(watch/relay): bound remote watcher fan-out and read the live relay grace
Three P1 fixes from the SSH/remote freeze audit:
- Remote watchers now debounce on the same 150/500 window as local ones
(finding D), and every teardown path drops the trailing flush timer
instead of letting it fire into a dead watch. The deferred send is
wrapped so a frame disposed mid-window can't escape as a fatal
main-process exception.
- File Explorer refreshes are scheduled and concurrency-capped rather
than fanned out unbounded over expanded dirs (finding C). Local
transports use a zero window, since main already coalesced the burst.
- relay.startGrace reads ptyHandler.configuredGraceTimeMs instead of the
launch-time argv closure, so a grace raised after launch is honored.
The branch selection moves to relay-grace-branch.ts because relay.ts
has no exports and calls main() at import, making it untestable.
Consequence: a host-sleep relay holding zero PTYs now exits after the
idle cap. Pinned by test and documented in
docs/reference/relay-grace-time-reconfiguration.md.
Also drops the duplicated 150/500/5000 constants in the runtime-RPC
batcher in favor of the shared window module.
* docs(relay): correct grace-reconfiguration line numbers after the relay.ts edit
Co-authored-by: Orca <help@stably.ai>
* refactor(file-explorer): use useMemo for paths; remove relay reference
Replace manual ref-based caching with proper React hooks for content-stable path memoization. Remove outdated relay grace-time reference documentation from code review cycle.
* rm design doc
* fix(remote-watcher): prevent stranded timer after close
An in-flight provider receive can land after the batch is torn down.
Without a guard, pushing events to a closed batch would re-arm a timer
that would never be cleared, stranding the task indefinitely. Track the
closed state and skip pushes after close().
Relay.ts comment clarifies why pool watches remain registered during
grace-period shutdown deferral — the socket server stays listening so
a reconnecting client can cancel the grace and resume.
---------
Co-authored-by: Orca <help@stably.ai>
Use async profile probes and serialize the current-time due decision plus mutations under one async owner so sync flushes and detached writers cannot double-shift or miss the recovery interval. Add syscall, timer-liveness, parity, and held-I/O interleaving coverage.
* fix(relay): chunk fs.changed notifications to fit the client frame capacity
emitRelayWatcherEvents published an entire watcher batch as one
fs.changed notification on the ordinary lane. A batch runs routinely
tens of times over the per-frame producer capacity (12,288 bytes against
a Node <=21 socket, 49,152 against Node >=22), and an over-capacity
frame there is not queued or trimmed -- notify() fails admission and
closes the client, tearing down every in-flight request on that
connection. Regressed in 5f7807497e (#11005).
The emitter now sizes chunks against the tightest capacity across
attached clients, measured in encoded bytes through the same envelope
admission measures, and preserves event order. A single event too large
for an empty envelope cannot be chunked, so the ordinary-lane close
remains its backstop: a silent drop on this lane has no resync contract.
* chore(relay): shorten watcher chunking comments
- Oversized relay responses now fail their request instead of closing the connection,
preventing one frame from killing every pane on the host
- Restore subagent state for correct hibernation; don't resurrect stale prose
across turns
- Account for relay re-establishment and PTY reattach time in SSH flap delay caps
- Only log drops of final unsendable envelopes, not temporary rejections during
measurement probes
- Fix watcher overflow marker release race when notification admission rejects
without settlement; use precise byte counting for event batching
Round 3 + 4 review fixes.
Watcher chunking is now gated on the *client's* retention reserve rather than
the dispatcher-wide one, so one stalled peer no longer forces a healthy client
into a full file-tree resync. The relay-lost redeploy ladder no longer burns its
6-attempt budget while the SSH transport itself is down: it holds at the 15s step
with a non-terminal status and rearms, so a laptop that slept past the ladder
comes back instead of landing on a terminal "give up" banner.
The shedFields wire marker had no consumer, so an agent-hook envelope whose
subagent roster was dropped to fit the frame read as "roster cleared" on the Orca
side: live child rows blanked and a done pane became hibernation-eligible while
its teammates were still running. ingestRemote now restores shed fields from the
cached payload (interactivePrompt deliberately excluded — a stale answerable
question card is worse than none).
Also: stream terminal-frame slots are counted per client, since the control queue
they protect is per client; the chunking fast path no longer logs a drop for a
batch it goes on to deliver in full; -32010 is now RelayErrorCode.ResponseOverCapacity.
Test debt from the review: pending-pane eviction, per-client stream isolation, and
the reconnect budget are now asserted rather than assumed; four fragile exact-byte
pins dropped in favour of the tier comparisons that carry the requirement.
Swap the worktree-name emoji picker from emojibase-data's `github` shortcode preset to `emojibase`, which carries both `flag_kr` and `south_korea` style flag names, and drop the hand-maintained `kr` entry that patched around the gap. Filter skin-tone aliases so they neither crowd the suggestion list nor clobber base-emoji branch names.
Search now matches anywhere in the shortcode, ranked exact > prefix > word-start > substring, so `:korea` surfaces both Koreas.
Emoji-derived branch names now prefer spelled-out aliases: flags use country names (japan, germany, south-korea) and cryptic stubs are skipped (thumbsdown over no, victory over v).
Keep only the durable docs already allowlisted for tracking
(STYLEGUIDE, assets, localized readme, and reference compatibility
guides). Drop feature design notes, plans, and repro artifacts that
were force-added past the existing docs ignore rules.
Remove one-off incident docs and committed test-results noise, move
dev/repro/bench tools under tests/tools, and relocate i18next config
into config/ so the GitHub root scrolls to the description faster.
Round-2 review fixes.
The round-0 change from close-on-rejection to silent drop removed the only
redelivery path for agent.hook envelopes: they are fire-and-forget and the
per-pane cache only replays on handler install, so a saturated link stranded
a pane on a stale Working spinner until reconnect. Closing used to guarantee
delivery by forcing that replay. Envelopes now publish per client and pend
for bounded latest-wins redelivery when the producer queue rejects them.
Shed fields are now named on the wire. The subagent roster is not cosmetic --
the renderer replaces rather than merges it, and hibernation gates on its
length -- so an unmarked shed could sleep a live pane.
fs.streamEnd rode the control lane because it must not be dropped, but that
lane kills rather than drops. The stream's concurrency slot is now held until
the terminal frame settles rather than until the fd closes, capping queued
terminal frames well under the control budget; overflow costs one refused
read instead of the connection.
The watcher chunk walk now stops while producer retention sits past its
reserve and degrades to a resync, so a 5000-event flood cannot fill the queue
that interactive PTY traffic shares and stall every remote terminal.
The reconnect ladder caps its flap-path delay so delay plus handshake timeout
cannot cross the relay grace floor and let the remote daemon kill live PTYs.
Also: the suppression key no longer embeds a NUL byte, which had made the
file binary to git and grep; producerEnvelopeBudget no longer reports
infinite capacity for a departed client; the drop logger no longer encodes a
frame it will not log; and an over-capacity response substitution no longer
settles as if the result had been delivered.
* perf(mobile): avoid unchanged worktree catalog payloads
* fix(mobile): isolate catalog snapshots by limit
* review: reassert host truth on unchanged polls; content-address snapshots
Client — the `changed` gate meant an unchanged poll skipped setWorktrees /
setLastKnownWorktrees / setCachedWorktrees, so optimistic local edits
(togglePin, handleDeleteWorktree's failure re-add) and the #8498 cache guard
were no longer repaired while the host catalog was stable. The gate bought
nothing: setCachedWorktrees is an in-memory Map write and areWorktreeListsEqual
already ran every poll, so the steady state still short-circuits on array
identity. All wire savings are unaffected. admit() now just returns the
confirmed rows and HostScreen applies them exactly as it did pre-PR.
Also on the client:
- a stale response from a superseded client/host no longer clears the token the
current client/host just established
- discriminate on `worktrees` rather than on `'unchanged' in response`, so a
future catalog field named `unchanged` can't reclassify a full response
- useRef over useMemo for the snapshot client; React may discard memoized values
- hoist WORKTREE_PS_FULL_LIMIT so the truncates-at-200 rationale travels with it
Host — replace the per-limit snapshot cache with a content-addressed id (ETag
semantics). Ownership lives in the id, so concurrent clients, differing limits,
and runtime restarts are correct by construction; this drops the LRU, the
eviction policy, the per-runtime WeakMap, and the retention of up to 8 full
catalogs. The remaining cache is a pure memo: because ids derive from content,
dropping or thrashing it costs CPU and nothing else. Keeping the memo also
keeps the measured steady-state cost — hashing every poll instead measured
2.24ms vs 0.75ms for the compare on a 310KB catalog.
Verified: mobile 2784 passed / 3 skipped, src/main/runtime/rpc 1064 passed,
node + mobile typechecks, oxlint, oxfmt, max-lines ratchet.
* fix(runtime): isolate catalog snapshot memo
---------
Co-authored-by: Brennan Benson <79079362+brennanb2025@users.noreply.github.com>
Round-1 review fixes on the P0 freeze work.
The control-lane overflow marker could reinstate the exact failure this P0
removes: dispatcher-client-writer closes the client when control-lane
admission fails, and admitControl is the only lane that returns an error, so
one marker per failing batch accumulated to the 256-frame/1MB bound and
dropped the link. Markers are now deduped to one outstanding per
(client, root), cleared on settle.
Chunking also defeated the renderer's per-payload directory dedupe -- events
are now stable-grouped by parent directory so one directory lands in one
chunk -- and the halving walk overshot the byte minimum ~1.7x while the fast
path paid three JSON encodes; both are fixed by publishing first and sizing
from a measured bytes-per-event estimate.
Agent-hook shedding now surrenders the blocking interactive prompt LAST
rather than first, so a degraded envelope cannot strand a pane at
state=waiting with no answerable question card.
The dropped-notification log now distinguishes over-capacity from producer
queue backpressure and no longer lets the first dropped method silence every
other producer for the life of the connection.
Three independent connection killers found in the SSH/remote freeze audit.
FINDING A - the reconnect ladder never escalated for post-handshake drops.
scheduleReconnect() used the single published state.reconnectAttempt for both
the delay index and the give-up test, and runReconnectAttempt() zeroed it
before connecting (ssh.ts gates the relay redeploy on 0-at-connected). Every
post-handshake drop therefore re-entered at 1000ms forever, ~3600 relay
redeploys/hour, and 'reconnection-failed' was unreachable for a flapping host.
New SshReconnectLadder splits the delay index (advanced by every retry) from
the failure streak (advanced only by a failed handshake), so flaps back off
while give-up semantics stay byte-identical to shipped.
FINDING B - notify() closed the client whenever a frame exceeded the producer
frame capacity, conflating a permanently un-sendable frame with transient
backpressure. A 5000-event fs.changed is 425KB against a 49KB cap, so the
watcher flood killed the link and re-killed on every reattach+replay. notify()
now drops and logs once per generation; fs.changed is chunked to each sink's
capacity with a control-lane overflow marker as the resync fallback; agent-hook
envelopes shed lastAssistantMessage/interactivePrompt/subagents to fit.
FINDING B2 - sendResponse routed >1MB responses to a lane whose admission
ignores the frame cap and closed the client on rejection, so a large
fs.listFiles dropped the SSH host. It now substitutes a JSON-RPC error so the
request fails instead of the connection.
Also moves fs.streamEnd/fs.streamError to the control lane so a terminal frame
cannot be dropped by the producer-lane check.
Co-authored-by: Orca <help@stably.ai>
* fix(preflight): route landing banner through the runtime-aware preflight slice
Landing called window.api.preflight.check directly, which always probes the
local client. The preflight slice is the only caller that consults
getActiveRuntimeTarget and forwards to preflight.check on the active runtime
environment, so while connected to a remote runtime the landing banner
reported the client machine's git/gh state instead of the server's.
Delegate to refreshPreflightStatus and derive the issue list from
state.preflightStatus. This also drops Landing's duplicate probe: the slice
dedupes concurrent and forced checks, so the mount/focus/poll paths now share
one in-flight request with the rest of the app.
* fix(preflight): refresh landing status across runtime sessions
* test(preflight): cover paired runtime session races
* test: make landing preflight oracle behavioral
* fix(preflight): scope runtime session invalidation
* test(preflight): cover headed runtime switching
* test(preflight): isolate runtime status toast
---------
Co-authored-by: Marty <marty@localhost>
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
* fix(mobile): stop serving a pre-write host-list snapshot to loads issued after the write
removeHost/persistHost await hostListMutation, but the in-flight loadHosts()
de-dupe handed back a pass that started BEFORE the write committed, so a load
issued after removal repainted the removed host card (#8791). Every durable
write now drops the shared pass via host-list-load-sharing.ts so the next
caller reads fresh; concurrent loads with no write between them still share
one Keychain pass.
Also extracts the host action sheet into host-list-action-sheet-actions.ts to
pin closeBeforePress on Edit host + Remove (the freeze half of #8791, already
fixed by #8536).
* fix(mobile): invalidate host loads after token writes
* fix(mobile): protect host token cache from stale reads
* fix(remote): unthrottle host renderer while serving a paired client
A paired desktop host left in the background could not open or close
agent sessions for its remote/relay client: the action stalled and
eventually failed with the host-side "Timed out waiting for terminal
surface after creation" (10s) error, while an already-live terminal's
keystrokes stayed fast.
Root cause: creating/closing a session routes through the host
renderer's setTimeout-coalesced graph sync to publish the terminal
surface, but the host window runs with Electron background throttling
(the hidden-window default, reaffirmed on macOS). When the window is
backgrounded/occluded, those renderer timers are throttled to a crawl
and the surface publication misses the 10s deadline. Live keystrokes are
unaffected because PTY I/O flows through the main process, never the
renderer.
Keep the authoritative renderer unthrottled while at least one remote
client is connected and restore the throttled power-saving default once
the last one disconnects. Connect/disconnect are driven from the shared
MobileSocketWiring onReady/onClose, so both direct-WS and cloud-relay
clients are covered; headless serve has no window and is a safe no-op.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(remote): tidy renderer-throttle comment and test per review
Address automated review nits on #11581:
- Trim the module-level rationale comment to the non-obvious contract,
matching the repo's concise-comment guideline.
- Drop the dead `detachedThrottle` variable from the reapply test; the
detached-target scenario is already covered by the lazy-resolution
test, so the case now asserts only what it exercises.
No behavior change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(remote): scope paired terminal publication throttling
Keep headed paired terminal creation and close renderer-owned so host inventory, input routing, ACK recovery, and cleanup retain the established lifecycle. Hold a reference-counted background-throttle lease only while the renderer publishes a paired operation, and epoch-fence async resolution so renderer reloads reject before any request or PTY spawn. Preserve headless main ownership and prevent paired clients from falling back to a local terminal.
* test(e2e): verify minimized host terminal repaint
* fix(remote): preserve paired terminal inventory through graph gaps
---------
Co-authored-by: fanyunqian.1 <fanyunqian.1@bytedance.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
* fix(mobile): keep source-control layout steady while Create PR eligibility loads
The Create PR entry unmounted until the first hostedReview.getCreationEligibility
answer arrived, so on a cold open the changed-files list painted first and then
shifted down 54pt (createPrBlock marginTop 12 + createPrButton minHeight 42)
when the button appeared — while the user was already tapping (#8411).
- buildMobileCreatePrAction: cold loading now reserves the row with a disabled
placeholder instead of unmounting it.
- useMobileHostedReviewEligibility: a fetch-imminent idle frame renders as an
in-flight load, so the reservation is present on the first painted frame.
- New per-worktree+branch memory of the last resolved eligibility seeds cold
loads, so branches whose answer is hidden (existing review, unsupported
provider) do not get a placeholder that collapses on every reopen.
Fixes#8411
* fix(mobile): harden source-control layout reservation
* fix(mobile): keep review status row footprint fixed
* fix(mobile): derive eligibility state from keyed snapshots
Jinjing
Brennan Benson
Neil
OrcaWinandNeil
Jinwoo HongandOrcaWin
fsdwen
Rod BoevandOrcaWin
github-actions[bot]