* fix(native-chat): kill hung WSL operations via child process
Stalled UNC file operations hold libuv permits even after the gate
timeout expires, blocking Chat tab recovery. Two stalled operations
fill both permits and freeze all WSL access until restart.
Fork file I/O for UNC paths into a separate child process. On deadline
expiry, kill the process to force the hung syscall to exit. This frees
the permit for the affected tab's next read. Temporarily quarantine the
stalled route to avoid retry storms.
* chore: drop internal review artifact from the repo root
* fix(native-chat): harden the WSL transcript fs sidecar
Review follow-ups on the sidecar isolation change:
- Only the deadline may abort running gate work. The sole waiter's
same-duration timeout fired first, killed healthy children on caller
abandonment, and settled the task before the deadline could quarantine
a stalled route - leaving the back-off dead for every dedupe:false op.
- Resolve the fork entry from out/main/chunks too: the resolver compiles
into a shared chunk, and the scanner service child has no
process.resourcesPath, so packaged WSL vault scans threw entry-not-found
(masked as an empty tree).
- Allowlist the fork env instead of spreading process.env; ambient
NODE_OPTIONS would halt or --require code into every child.
- Wrap transport faults (spawn failure, child death) in
WslTranscriptFsError('unavailable') so discovery reports them as scan
issues instead of misreading them as missing paths or empty trees.
- Gate the vitest in-process fallback on the vitest worker global so a
leaked VITEST=true cannot revert production to in-process UNC syscalls.
- Reap idle sidecar processes after 60s instead of holding them for the
app session.
- Split 'open' into its own protocol union member so the reusable-call
Exclude actually strips it from the pooled-process API.
- Guard kill('SIGKILL') against the teardown race where an exiting child
emits an unlistened 'error', and dispatch reads by handle kind before
path spelling.
* fix(native-chat): probe stalled WSL routes instead of a fixed quarantine
Remaining review follow-ups:
- Escalating route quarantine: first strike lifts after 5s so a distro
that was cold-booting when its op hit the deadline recovers on the
next poll (~35s total instead of ~90s); repeat stalls double the
back-off toward the prior 2x-timeout cap, and any settle the deadline
did not force clears the strikes. Queued same-route tasks fail fast
at quarantine instead of stranding one waiter deadline per file in
sequential scans.
- Single request implementation: the vitest in-process fallback now runs
the child's own dispatcher (WslTranscriptFsProcessOperations + decode),
so unit suites exercise exactly what the forked process executes and
the per-call-site fallback closures are gone. Dirent fixtures gained
the full kind-flag set the serializer reads.
- Dropped the production-dead per-route close queue; UNC FileHandles
(test fallback only) mirror the process-handle close contract.
- Error class, messages, and factories move to wsl-transcript-fs-error
(re-exported from the gate) to keep the gate under the lines budget.
* fix(native-chat): harden WSL transcript fs with route quarantine strike
Extract quarantine logic into a dedicated module with strike decay: stalls older
than 5 minutes restart from base back-off, and concurrent-lane timeouts count as
one incident. Allow joining live in-flight tasks on quarantined routes (they cost
no new I/O). Preserve quarantine across transport faults (child death). Handle
file shrinking during tail reads by detecting short reads and returning empty.
Defer file closes that arrive mid-read instead of refusing, preventing slot
leaks. Separate process slot and boundary-finding concerns into focused modules.
* fix(native-chat): enforce route quarantine windows and isolate lanes per
A late result arriving after the deadline was incorrectly lifting the route
quarantine, allowing subsequent work to start before the back-off period
expired. Now late results are correctly recognized as stale and never cut
the quarantine short.
Process work is now isolated per (route, priority) lane so a scan stall
cannot block exact reads on the same distro. Each lane gets its own client
and process pool; late results and handle faults stay scoped to their lane.
Tests now fake performance.now() alongside timers (the quarantine clock
depends on it) and wait for the full back-off window to expire rather than
advancing by 0. Gate state is reset between test cases since late releases
never lift the quarantine.
---------
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
The three usage scanners and their stores, plus the renderer usage-overview
model, each carried a file-level `eslint-disable max-lines` and had grown to
338-769 counted lines against a 300-line budget. AGENTS.md calls for splitting
rather than suppressing, and config/max-lines-baseline.txt is a shrink-only
ratchet, so this removes all seven suppressions and prunes their entries
(341 -> 334).
Each file is cut along the seams it already had -- and that several of the
suppression comments named out loud: filesystem discovery / record parsing /
attribution / aggregation for the scanners, and pricing policy / scope filters /
rollups / session rows / automation attribution for the stores.
Pure move, no behavior change. Code is relocated verbatim; the only edits are
import plumbing and, where a private class method became a free function, the
mechanical `this.state` -> `state` parameter threading. Every converted call
site passes `this.state` at call time and the automation path takes a live
`getState: () => this.state` getter, so no state is snapshotted. No barrel
exports: each new module owns real logic and importers point at the owner.
Verified: oxlint clean, ratchet passes, typecheck clean, full unit suite green
(remaining failures are pre-existing load flakes in untouched files, each green
when re-run serially), no import cycles among the 64 affected modules, and a
statement-level diff of every split confirms the moves are verbatim.
Parsers now properly tear down gated transcript streams in finally blocks,
even when throwing mid-parse, so stalled gate deadlines do not leak file
handles into later scans. Handle close queues are isolated per WSL route
so a stuck distro cannot strand closes on healthy ones. Route key logic is
extracted to a shared module used by both the gate's admission and the
close queue's serialization.
* fix(ai-vault): gate post-resolution WSL transcript I/O (STA-4049)
PR #14090 admitted only path *resolution* through the WSL transcript
filesystem gate. Every byte read afterwards from the resulting
\\wsl.localhost\... UNC path ran raw, so a distro that answers the first
access() and then stalls hung Native Chat at "loading" and AI Vault at
"scanning" with no timeout and no error.
Route that I/O through a new wsl-transcript-fs-access accessor, which is
a verbatim node:fs passthrough off UNC and an admitted, deadlined task on
it. open/positional-read opt out of coalescing (dedupe: false): joiners
would share one FileHandle or one caller's buffer.
Refusals now surface as the existing retryable message rather than
notFound, per-root scan failures are contained to an AiVaultScanIssue,
and the memoized Codex/Kimi indexes evict on refusal so a stall cannot
pin "no titles"/"no cwd" until the index changes.
* fix(ai-vault): stop caching WSL gate refusals as results (STA-4049)
Code review 1 P1 fixes on top of the transcript gate:
- transcript-read-cache: never store a gate refusal. The refusal leaves the
file's mtime untouched, so the cached error would have been served to every
later call until the transcript itself changed.
- kimi/grok/opencode parsers: rethrow WslTranscriptFsError instead of folding it
into "no session"/"no transcript", so the session parse cache cannot store a
null or partial answer under an unchanged mtime. Ordinary missing/half-written
files stay contained.
- opencode-usage scanner: gate the data-directory readdir and the absolute
OPENCODE_DB stat. The AI Vault's primary OpenCode source reaches them
transitively, which is why the direct-import guard never saw them.
- gated stat/lstat: accept an AbortSignal, matching gated open/read, so a
cancelled watch install or title probe detaches immediately instead of holding
a waiter to its deadline.
- gated open: close a FileHandle whose syscall lands after the last waiter gave
up, and close handles off UNC verbatim (awaited, failures surfaced).
Co-authored-by: Orca <help@stably.ai>
* fix(native-chat): decode gated chunks incrementally and cancel drain I/O (STA-4049)
Addresses the CR2 blockers.
UTF-8 chunk-boundary corruption: the UNC branch yielded raw 1 MiB Buffer
slices that `decodeTranscriptStream` decoded independently, so any multibyte
codepoint straddling a boundary became U+FFFD on both sides — corrupting the
JSONL line and shifting `consumedBytes` (which seeds fallback message ids).
`gatedChunks` now holds a StringDecoder when `encoding` is set, and
`decodeTranscriptStream` holds one for the Buffer path, matching what
`createReadStream`'s decoder already did off UNC.
Watcher teardown: `installTranscriptWatcher` owns an AbortController that
`unsubscribe()` aborts, threaded through every gated call on the drain path.
Waiters now detach at teardown instead of holding to the 30s deadline, and
the gate's aborted-signal pre-check stops an in-flight drain from admitting
new tasks after close.
Rovo `session_context.json`: `readJsonObjectIfExists` rethrows
WslTranscriptFsError so `parseSessionCandidate` records a scan issue, instead
of caching an un-enriched session under an unchanged mtime that never re-reads.
Primary OpenCode source: `listOpenCodeDatabases` takes an optional refusal
reporter so a refused `OPENCODE_DB`/`XDG_DATA_HOME` surfaces an
AiVaultScanIssue, matching `listOpenCodeDatabasesInDirectory`.
`boundaryFingerprint` moved to its own module to keep the watcher engine
under the max-lines cap.
* refactor(native-chat): consolidate transcript I/O and remove fallback te
- Move boundaryFingerprint from its own module to transcript-file-version.ts
- Extract runPathOperation helper to eliminate duplicate UNC path routing
- Remove tests for fallback behaviors when transcripts are unavailable or incomplete
- Clean up implementation comments and verbose test documentation
* consolidate scan issues and gate session scanner I/O (STA-4049)
Both local and remote session scans hit stalled WSL distros identically:
one failed probe per discovered path. Unifying issue recording and gate
refusal handling prevents duplication and ensures consistent behavior.
- Gate all file operations (stat, readdir, read, open) through WSL
stall detection instead of scattered or missing gates
- Serve cached transcripts when stat stalls; distinguish gate refusals
from missing files
- Serialize UNC close operations to prevent thread pool exhaustion
- Incremental chunk decoding in streams handles codepoint boundaries
correctly
---------
Co-authored-by: Orca <help@stably.ai>
* refactor(usage): share the session/daily fold between Codex and OpenCode
The Codex and OpenCode scanners each carried their own byte-identical copy of
the ~325-line aggregation pipeline (createEmptySession, the three breakdown
folds, finalizeSessions, mergeSessions, mergeDailyAggregates). Two copies means
a token-accounting fix — a bucket that double-counts, a merge that drops a
breakdown row — lands in one provider and silently not the other. The copies had
already started to drift in comments only; the next drift would have been in
arithmetic.
The providers differ in exactly one dimension: the extra metric folded alongside
the token counters (Codex `hasInferredPricing`, OpenCode `estimatedCostUsd`).
That is now injected as an empty/fromEvent/fold triple, so the shared code stays
generic without collapsing the two record schemas into a nullable union. The
clone strategy stays per-provider (`cloneSessionForMerge` vs `structuredClone`)
rather than being unified on the assumption that the difference is accidental.
`usage-provider-contract.ts` is the seam a plugin-contributed usage source will
implement. It is deliberately generic over each provider's record types: Claude
bills per turn while Codex/OpenCode bill per event, and `cachedInput` is a subset
of `input` for the latter but a peer bucket for Claude, so a single normalized
record would push nullable handling onto every consumer.
No behavior change. Emitted objects are byte-identical, including key insertion
order — verified by diffing JSON.stringify of the scan output before and after
across mixed models, mixed locations, an inferred-pricing flip, and null vs
non-null cost. Persisted field names and schemaVersion are untouched, so caches
do not invalidate.
* refactor(usage): make the provider contract load-bearing and dedupe worktree refs
Follow-up to the aggregation extraction, addressing three review points.
`UsageProvider`/`UsageScanResult` were declaration-only, which is the same
speculative-interface problem #12077 just deleted 8,900 lines of. They are now
implemented by both real providers via `satisfies`, so the seam is typechecked
against actual scan functions rather than asserted. The blocker was that codex
returns `processedFiles` and opencode returns `processedDatabases`; rather than
rename persisted-adjacent fields, the source key is a type parameter, so each
provider keeps its own on-disk name and the contract still binds. Verified the
constraint bites: swapping the key to 'processedSources' fails typecheck.
`schemaVersion` is part of provider identity in the contract, so each provider's
SCHEMA_VERSION constant (with its cache-invalidation rationale) moves into the
provider module and the store imports it. Values are unchanged (codex 5,
opencode 2) and the stores compare them exactly as before, so no cache
invalidates. This also keeps store -> provider -> scanner acyclic.
`UsageWorktreeRef` collided with the existing export in usage-worktree-metadata
(3 fields, no repoId). Two different exported types under one name in src/main
is worse than the duplication being removed, so the scan-input type is now
`UsageScanWorktreeRef`; usage-worktree-metadata is untouched.
`createWorktreeRefs` was triplicated. Codex, OpenCode, and Claude copies are
byte-identical apart from the return type name (verified by diff), and all three
ref types have the same four fields, so one shared copy replaces all three. This
is the only change to claude-usage/.
No behavior change: same functions, same arguments, same call order. The store
tests' `./scanner` mock still intercepts scanning because the provider captures
the mocked binding; their now-inert `createWorktreeRefs` mock key is dropped so
it does not read as still mocking something.
* fix(worktrees): prevent deletion from blocking Orca
* test(worktrees): loosen async history-delete event-loop bound for CI
The main-thread safety check failed on a loaded runner when a single
timer gap hit ~48ms under the prior 30ms threshold. Keep the bound well
below a recursive sync-rm stall without treating CI jitter as a block.
* test(worktrees): measure history-delete critical path, not timer gaps
setInterval gaps during async rm of thousands of files still flake under
CI scheduling. deleteWorktreeHistoryDir is sync and must only rename, so
assert that critical-path wall time stays well below a recursive walk.
* fix(worktrees): prevent deletion from blocking Orca
Add timeout-based draining of watcher closes so SSH round-trip delays don't
indefinitely block the worktree removal path. Also: order durable temp-file
sweeps ahead of writes to reclaim orphans before accumulation, skip own-process
temps to avoid deleting live writes, swallow persistence errors so disk failures
don't cascade to query callers, and measure history-deletion progress by loop
turns rather than timer gaps to detect blocking on CI runners.
* fix(worktrees): prevent deletion from blocking Orca
Worktree deletion can now proceed even if filesystem watchers or history cleanup operations hang, preventing Orca from freezing. Changes:
- Fence install slots with tokens instead of counters so removals can abandon wedged installs without corrupting later removals
- Timeout-bound watcher unsubscribe operations with a shared drain budget
- Move JSON serialization of large usage caches from queue-time to write-time to avoid blocking main thread
- Async tombstone + schedule history tree deletion instead of blocking recursive rmSync during GC, preventing main-thread stalls ~10s after startup
* Extract usage cache writer into reusable durable snapshot class
Consolidates serialized durable-write and generation-veto logic from
three usage stores into UsageCacheSnapshotWriter. Eliminates duplication,
centralizes multi-MB JSON serialization on the main thread via write-queue
serialization, and vetoes superseded snapshots to avoid wasted rewrites.
* fix(worktrees): prevent deletion from blocking Orca
Worktree deletion used to recursively delete large session trees (hundreds
of MB) on the critical path, stalling the event loop. Instead, rename trees
into a `.pending-delete` tombstone queue and reclaim them asynchronously
off the removal's critical path.
Extracted host tree removal into a reusable helper (`removeHostTree`) that
centralizes Windows retry logic. Added usage-cache flush on quit to prevent
data loss when scans complete right before shutdown. Improved watcher
removal deadline management with reserved tail slices for the final
unsubscribe, and added retry logic for tombstone removals that fail once
under transient Windows locking.
* fix(history): retry failed session tree removals
Tombstoned session trees whose removal fails transiently (e.g., EBUSY
under Windows AV) are now re-queued in-process with bounded exponential
backoff instead of sitting until the next HistoryManager construction.
Prevents a single stuck tree from blocking the entire Orca process.
* perf(runtime): remove timer clamps from cooperative yields
Renderer paste and input loops can schedule more than a thousand zero-delay timer yields for a maximum-size payload. Chromium clamps nested timers to 4ms, adding seconds of idle wall time.
Use MessageChannel tasks in renderer runtimes and setImmediate in Node while retaining a timer fallback for tests and unsupported environments.
* fix(runtime): preserve pacing and release yield callbacks
Adversarial review found that concurrent producers could retain resolved callbacks until global quiescence. Route renderer yields by token and delete each resolver before resuming its producer.
Keep timer pacing in terminal paste and accepted-write loops where SSH and local PTYs do not provide drain acknowledgement. Use the shared scheduler for the OpenCode scanner.
Backups that own no sessions have nothing to reclaim, so reparsing
them on every live-db update was wasted work; only reparse siblings
that still hold claimable session ownership.
* fix: dedupe fork-copied usage history across Claude, Codex, and OpenCode scanners
Coding-agent CLIs copy transcript/rollout history into new files on
resume/fork, and the usage scanners deduped per-file only (or not at
all), so copied history was re-counted once per descendant file
(issue #8006: 38.6B tokens / $72,981 reported vs ~2.3B real).
- claude-usage: cross-file turn ownership keyed on message.id:requestId;
per-file ownedDedupeKeys persisted; deterministic sorted-path claim
order; schema v3 -> v4 so inflated caches rebuild.
- codex-usage: cross-file token_count event ownership keyed on the raw
record identity (sessionId + timestamp + token tuples); fixes both the
copied-prefix re-count and the total-only branch that re-counted the
entire cumulative session per descendant rollout; legacy
.orca-session-copies skip-bytes bridge unchanged; schema v3 -> v4.
- opencode-usage: each sessionId is counted from exactly one database;
the canonical opencode.db claims ahead of stale sibling copies
(opencode-backup.db etc.) so backups no longer double totals, while
backup-only sessions are still counted; schema v1 -> v2.
Regression tests cover fork-copied files counted once (including the
Codex total-only variant), duplicated OpenCode databases, and dedupe
stability across cached incremental rescans.
Co-authored-by: Orca <help@stably.ai>
* Fix cross-tool usage double-counting for fork/resume-copied history
- Widen Claude dedupe keys with message-id and uuid fallbacks so forks
missing requestId still dedupe correctly, and drop Codex's sessionId
from event keys since fork/resume rewrites session_meta.id while
copying identical token_count records.
- Track hasDeferredClaims per cached file/database across Claude, Codex,
and OpenCode scanners so that when an owning file is deleted, only
files that deferred a claim need reparsing to reclaim those turns
instead of rescanning the entire corpus.
- Let OpenCode's live opencode.db reclaim sessions from a stale backup
claim once it reappears, avoiding a frozen stale snapshot.
- Bump schema versions to invalidate caches built with the old,
narrower ownership keys (#8006, #8013 follow-up).
---------
Co-authored-by: Orca <help@stably.ai>
110 files carried an eslint/oxlint-disable max-lines directive but are
already under the default max-lines budget (300 .ts / 400 .tsx / 600 .mjs
/ 800 test), so the suppression is dead. Removing it restores real
max-lines coverage on these files with zero behavior change.
Each removed directive had max-lines as its only rule; verified via a
full oxlint run (0 max-lines violations, 0 new errors). Diff is pure
deletions (200 lines, 0 additions) — no code touched.
Co-authored-by: Orca <help@stably.ai>
* chore(lint): upgrade oxlint to 1.71 and enable 7 new rules
Upgrade oxlint 1.67.0 -> 1.71.0 (1.72 was blocked by the repo's 3-day
minimum-release-age supply-chain guard; nothing here needs it). The
bump is a no-op on the existing config.
Enable 3 error rules (backlog autofixed to zero in this commit) and
4 warn rules (surface signal without gating CI):
error (autofixed, behavior-preserving):
- unicorn/prefer-node-protocol (~1531 sites: bare builtin -> node:)
- typescript/no-import-type-side-effects (~36: all-inline-type -> import type)
- unicorn/no-array-reverse (19: copy-then-reverse -> toReversed)
warn (real signal, current fires are test-only/correct):
- unicorn/no-array-fill-with-reference-type (aliasing footgun guard)
- typescript/no-unsafe-function-type (bans bare Function type)
- unicorn/prefer-array-flat-map (map().flat() -> flatMap())
- unicorn/prefer-regexp-test (.match() in bool ctx -> .test())
mobile/.oxlintrc.json extends root, so it inherits all 7; the autofix
ran from root and covered mobile/ too.
Verification (all green): oxlint 0 errors (root+mobile+aux configs),
oxfmt clean, typecheck (node+cli+web), vitest 22795 passed / 0 failed,
builds (electron-vite + web + cli) succeed. node: rewrites confirmed to
skip embedded SSH/CLI string payloads (AST-only); all toReversed sites
verified to operate on fresh copies or write-once locals.
* chore(lint): bump mobile oxlint to 1.71 so inherited rules parse
mobile/ is a standalone pnpm project pinning its own oxlint@1.67, which
lacks unicorn/no-array-fill-with-reference-type (needs >=1.70). Since
mobile/.oxlintrc.json extends the root config, mobile CI's 'cd mobile &&
oxlint' failed to parse the new rule. Bump mobile to match root (1.71).
Verified in mobile/: oxlint 0 errors, oxfmt --check clean, tsc --noEmit
pass, vitest 978 passed / 0 failed.
Co-authored-by: Orca <help@stably.ai>
---------
Co-authored-by: Orca <help@stably.ai>
* fix: write stats file in chunks to avoid Electron UTF-8 abort
orca-stats.json gains an event on every agent start/stop. After about a
month of use mine had grown to ~3.6k events / ~608 KB, and the app started
hard-crashing a few seconds after every launch (SIGTRAP, no catchable JS
stack):
Assertion failed: (length + 1) <= (capacity())
node::MaybeStackBuffer<char>::SetLengthAndZeroTerminate <- node::Utf8Value
The crash is in StatsCollector.writeToDiskSync(), which saves the whole
file in one writeFileSync(JSON.stringify(data)). Electron 42.3.2's bundled
Node aborts when encoding a string that large to UTF-8 in a single write;
stock Node 24 handles the same file fine and the data is well-formed, so
it's an Electron/Node encoding limit, not bad data. The save runs on a
debounce after agent_start, which restored agents fire on launch -- so it
crashed right after opening.
Write the JSON in 64 KB slices through one fd instead (never splitting a
surrogate pair), and lower MAX_EVENTS 10k -> 1k so the file can't grow back
this large. Lifetime aggregates are unaffected.
Verified by reproducing the abort standalone with the real 608 KB file
under ELECTRON_RUN_AS_NODE, confirming the chunked writer round-trips it
byte-for-byte with no crash, and running a patched build that loads the
file without crashing. The underlying encode abort is an Electron/Node bug
to report upstream.
* fix: harden stats JSON writes
* fix: chunk app state UTF-8 writes
* fix: stabilize status and terminal polling
---------
Co-authored-by: thiagomsoares <5190162+thiagomsoares@users.noreply.github.com>
Neil
OrcaWinandOrcaWin
Jinjing
Jinwoo HongandOrca
