7 Commits

Include renames

Author	SHA1	Message	Date
OrcaWin	54a19c2ba3	fix(pdf): render CJK text with pdf.js resources ... Merged after PR-specific checks passed. CI failures are unrelated baseline findings in ClientHostedBrowserPagePane.markup.test.tsx and pane-title-update-global-scan-budget.test.tsx.	2026-09-19 01:28:09 -07:00
Jinwoo Hong	631b51f508	perf(usage): run the Claude/Codex/OpenCode usage scans on a worker thread (#21114) ... * perf(codex-usage): resume rollout scans at the last parsed byte Codex rollout files are append-only and grow all day, but any append changed both mtime and size, so `canReuse` discarded the cached entry and the scanner re-read the whole file from byte 0 on the Electron main process. On one real corpus that was 6.59 GB re-read per cycle across 26.63 GB / 21,110 files. Each parsed file now persists a resume point: the offset just past the last newline-terminated line, the parse context at that offset (session id, cwd, model, running totals), a sha256 of the 4 KiB before it, and the file's dev:ino. A grown file resumes there and merges the appended rollup into the cached one; anything unproven falls back to a full reparse — truncation, an in-place rewrite, rotation, a counted tail with no trailing newline, a legacy copied-session suffix offset, or a file that must reclaim deferred fork claims. Resume never depends on mtime equality, so a coarse-mtime filesystem cannot hide an append. Fixture: a 75,737-byte rollout with a 758-byte append re-read 76,495 bytes before and 8,950 after (the append plus two bounded 4 KiB boundary windows). Also bounds the automation-attribution force predicate for both Codex and Claude: it keyed on `lastScanError`, so a persistently failing scan forced a fresh full rescan on every single lookup. It now keys on the most recent scan attempt, which is one forced scan per run regardless of outcome. * perf(usage): run the Claude/Codex/OpenCode usage scans on a worker thread The three first-party usage scans walk whole rollout and transcript corpora and read OpenCode's SQLite synchronously, all on the Electron main process. They rarely produce a long stall — the JSONL reader streams, so it yields to the loop between chunks — but they pin the main-process event loop at ~95% utilization for the scan's whole duration, which is what every IPC message, timer and window event then queues behind. Move that work to one lazily-spawned, unref'd worker thread shared by all three providers, following the OpenCode SQLite scanner precedent (#8864). Measured on a synthetic 4,000-rollout corpus (25.8 MB cache): a cold scan drops from 2,147 ms of main-thread time to 31 ms, and a steady-state incremental scan from 165 ms to 64 ms. The worker is stateless and the cache crosses the boundary both ways. That costs ~64 ms of structured clone at this corpus size, against 2,147 ms saved on the cold path, and it keeps the persisted cache the single source of truth — a worker-owned copy would need an invalidation protocol and a second resident copy of the same multi-MB array. Failure is closed, never a silent empty result: a worker that cannot spawn, times out, or crash-loops rejects, and the store records the scan error and keeps the previous projection. Two clients already carried the same FIFO/timeout/crash-cap machinery, so extract it once as WorkerThreadRequestQueue (with the packaged entry-path resolver as worker-thread-entry-path) and move all three onto it, rather than adding a third copy. Their existing tests pass unchanged. The oracle is event-loop utilization on the calling thread, not a stopwatch: usage-scan-worker-event-loop.test.ts runs the same scan both ways and asserts the worker leg leaves the caller idle while the main-thread leg does not, so CI load moves both legs together (#18788). * test(usage): compare the two scan arms instead of two fixed thresholds The event-loop oracle claimed to be self-calibrating — its header said "the ratio is self-calibrating, so CI load moves both legs together (#18788) instead of tipping a fixed millisecond threshold." It computed no ratio. Two separate `it()` blocks each asserted an absolute threshold against its own arm, run separately, so load moved them independently. The comment described a test nobody wrote, and the flake it promised was impossible is the one that landed: `activeRatio > 0.8` on the calling-thread arm measured 0.764 on an ubuntu runner. Fixing the comment is not enough, because the fraction is the wrong quantity. CPU contention drags the calling-thread arm's active/wall fraction *down* toward the worker's, since the loop parks waiting on a contended libuv pool. A 4-vCPU Linux container measured that arm at 0.175-0.756 across twenty runs, idle and loaded — never once above 0.8. Active *milliseconds* move the other way: contention stretches the caller's JS time far more than it stretches the worker arm's fixed post-and-deserialize cost, so the gap widens under load. Merge the two arms into one case over one corpus and assert the worker arm costs the caller under a fifth of the inline arm's active milliseconds. Same twenty Linux runs: 10.9x-83.6x, passing throughout. Keep the presence preconditions on both arms — an arm that silently scanned nothing satisfies the comparison trivially — and extend them to the calling-thread arm, which previously checked only file and session counts. * fix(ports): name the dropped command when the probe queue is full The shared-queue extraction turned `Port scan command queue is full; dropped ${command}.` into a constant string, because `describeFull` was given no way to see the request. Pile-up is per-probe, so the name is the only thing in that log that identifies which of lsof/ps/netstat was shed. Pass the rejected request to `describeFull` and restore the name. The request is built before the cap check so it exists to be named; the id it burns is a correlation token, so a gap costs nothing. The existing overflow test asserted only the error class, which is why the regression escaped a 29-test suite. It now dispatches the overflow under a different command than the accepted ones and asserts the message text, so a message that names the wrong request fails too. Also add a direct WorkerThreadRequestQueue test. Three subsystems share the queue and each client test only sees the parts its own protocol exercises, with `queueCap` reachable from port-scan alone. Covers one-at-a-time FIFO dispatch, the deadline starting at dispatch rather than enqueue, the consecutive-death cap, and both points where that count clears. And record the child-process hazard at the usage worker entry. `terminate()` reaps nothing the thread spawned, and OpenCode discovery reaches a fork today: `wslGated*` forks the WSL transcript sidecar for a `\\wsl$\...` path, which a Windows `OPENCODE_DB` or `XDG_DATA_HOME` can be. One scan through that entry with a UNC `OPENCODE_DB` forked a sidecar that outlived `terminate()`. * test(ai-vault): assert the OpenCode worker messages exactly, not by fragment Checked every message string in the two clients the shared-queue extraction rewrote against origin/main. Only the port-scan queue-full one regressed (fixed in the previous commit); the OpenCode SQLite client's four messages render identically, the remaining source diffs being renames — `error.message` to `lastError`, `call.timeoutMs` and `CALL_DEADLINE_MS` to `timeoutMs`. `session-scanner-worker-client.ts` was not touched by the extraction. But its suite could not have caught it either. `/timed out/`, `/exited with code/` and a bare `rejects.toThrow()` all still match a message that has lost its interpolated value, which is the same blind spot that let the port-scan regression through. Assert the rendered text instead: the timeout names its deadline, the exit names its code, and the crash-loop drain still carries the text of the fault that killed the run. * fix(usage): correct the worker entry's child-process note The previous note said `worker.terminate()` leaves a forked sidecar orphaned. It does not, and the reproduction that appeared to show it used a stub sidecar missing the `process.on('disconnect', () => process.exit(0))` the real entry has. With a faithful one: the sidecar lives exactly as long as the thread and is gone within 2s of `terminate()`, because tearing the thread down closes the IPC channel it owned. Two worker lifecycles forked two sidecars and leaked neither, and the pre-worker main-thread path reaps its sidecar the same way, on host exit. What is true and worth recording: a fork is reachable from this bundle at all, which is easy to miss; it survives only as long as the channel does; and the sidecar is now re-forked per worker lifecycle instead of pooled for the app's life. State those, and warn that a future child which does not exit on channel close would not get the same free cleanup. * fix(usage): kill a wedged scan worker on no progress, not on wall clock `USAGE_SCAN_TIMEOUT_MS` was a 10-minute deadline on the whole scan. A cold scan of a real history is legitimately minutes — 637 s measured on a 30 GB corpus with 300 worktrees before the per-cwd memo, ~51 s after — so a larger corpus or a slower disk crosses it. Crossing it killed the worker, recorded a scan error and left the cache unadvanced, so the next refresh started cold and died at the same point, forever. The deadline is now a no-progress window. The worker posts a file counter as it walks the corpus (`UsageScanWorkerProgress`, rate-limited to one message a second), and `WorkerThreadRequestQueue` re-arms the active call's timer on each one via the new optional `isProgress`. Clients that do not pass it keep the plain wall-clock deadline. `MAX_CONSECUTIVE_DEATHS` and idle teardown are unchanged. * refactor(usage): report scan progress as a file count, not one call per file Claude's scanner walks batches, so a per-file callback made it loop just to bump a counter.	2026-09-16 23:03:37 -04:00
Neil	26721bd632	fix(codex): stop blocking the main thread on trust grants (#16441) (#16594) ... * fix(codex): stop blocking the main thread on trust grants (#16441) Codex hook trust was granted by blocking the Electron main thread on `spawnSync` of a bundled ELECTRON_RUN_AS_NODE entry for the whole app-server deadline: 15s native, 35s WSL, ~45s on the real-home path (rebase inspect + repair + grant). Cold start and every Codex pane launch showed "Not Responding"; the reported event-loop gap was 15,049 ms. The subprocess only ever existed to donate an event loop to a deliberately blocked parent — `runCodexHookTrustGrantSession` was already the real async implementation. Make the callers async and the fork is unnecessary, so the bridge, the forked entry and its envelope are deleted along with their build/knip/tsconfig registrations. The CLI `agent hooks prepare-codex` handler is already async, so it awaits the in-process session and saves a process spawn per managed-home shell. `resolveCodexTrustGrantHost` is async too; the WSL identity probe moves from `execFileSync` to `runProcess`, dropping that file from the child-process import allowlist. Status reads keep a synchronous native-only stamp path. Two invariants that held only because the lane blocked: - Overlapping capability probes were impossible by construction. `GitCapabilityCache`'s dedupe engine is extracted to a shared `CapabilityProbeCache` and `CodexAppServerCapabilityCache` now inherits it, so concurrent launches against a cold host share one app-server session instead of one each. - Two grants on one `config.toml` could not interleave capture and restore. A reentrant per-file lane now serializes the whole install sequence (managed, WSL runtime, real-home ensure, legacy sweep) and the grant and rebase inside it. Cold-start work moves off the critical path: retained-home reconciliation (N sequential sessions) is fire-and-forget behind the daemon provider, and the startup real-home ensure chains into managed hook reconciliation instead of blocking app init. Every preserved semantic is unchanged: never throws, the ORCA_DISABLE_CODEX_TRUST_RPC kill switch, ledger hits, backfill-pending and cooldown fallbacks, config rollback on every failure path, pre-grant self-computed trust removal, the verify-failure taxonomy, diagnostics and telemetry. * fix(codex): widen the trust-config lane to every config.toml writer Review follow-ups on #16441's async trust grant: - `markCodexProjectTrusted` now runs inside the runtime+system config.toml lanes, so a project-trust write can no longer land inside a hook grant's capture->restore window and be silently reverted. Its callers await it. - `install`/`refreshRuntimeUserHooks`/`remove` hold the system config.toml lane as well as the runtime one — they promote approvals into ~/.codex/config.toml and mirror it back. Lock order is runtime-before-system everywhere. - The real-home ensure chain resumes after a rejection instead of returning the same rejected promise to every later pane launch, and resolving the real home is now inside the module's never-throws boundary. - `buildSpawnEnv` awaits inside a cancelable pending-spawn registration, so shutdown during the (now long) env build stops the PTY from launching. `prepareLocalPtySpawn` generalizes into `awaitCancelableLocalPtySpawn`. - CapabilityProbeCache drops the test-only `nowMs` passthrough; its probe backstop comment now describes what it actually guards. - Preflight is a plain async function; the trust dispatch in orca-runtime collapses into one `markWorkspaceTrustedForAgent`. * test(codex): exercise the trust-config lane under real concurrency The async grant makes two pane launches overlap for the first time. These drive the real modules end to end on real files: a rollback swallowing a sibling's grant, a markCodexProjectTrusted write landing inside a capture -> restore window, shared capability-probe dedupe on a cold host, the host-scoped transient cooldown, and reentrancy from inside an installer. Each was verified to fail against a deliberately broken implementation (lane removed, dedupe disabled, cooldown made global, reentrancy pass- through disabled). * test(codex): stop hook-service suites spawning the developer's real codex The forked grant bundle never existed under vitest, so the RPC lane was unreachable in tests on main. Running it in-process makes these suites spawn a real `codex app-server` when one is installed: 38 spawns and two failures in hook-service-runtime-trust-repair on a machine with codex, green in CI where there is none. Stand in for the missing binary so both environments exercise the same fallback lane. * docs(codex): scope the trust-RPC kill switch comment to what it actually gates The comment read as though the flag forces the fallback lane everywhere. It gates the managed grant only: the real-home rebase still runs its own inspect/repair app-server sessions when Orca's insertion shifts a user's hook positions, and never reads the flag. Verified by exercise, not by reading — with the flag set, both inspect-user-hook-trust and repair-user-hook-trust still ran. Pre-existing: main has no check there either, it just blocked the main thread while doing it. Widening the flag to cover the rebase is a follow-up; this only stops the comment promising something the constant does not do.	2026-08-26 16:44:55 -07:00
OrcaWinandOrcaWin	4ec6bbf588	Kill hung WSL transcript filesystem operations via child process with route quarantine (#15381) ... * fix(native-chat): kill hung WSL operations via child process Stalled UNC file operations hold libuv permits even after the gate timeout expires, blocking Chat tab recovery. Two stalled operations fill both permits and freeze all WSL access until restart. Fork file I/O for UNC paths into a separate child process. On deadline expiry, kill the process to force the hung syscall to exit. This frees the permit for the affected tab's next read. Temporarily quarantine the stalled route to avoid retry storms. * chore: drop internal review artifact from the repo root * fix(native-chat): harden the WSL transcript fs sidecar Review follow-ups on the sidecar isolation change: - Only the deadline may abort running gate work. The sole waiter's same-duration timeout fired first, killed healthy children on caller abandonment, and settled the task before the deadline could quarantine a stalled route - leaving the back-off dead for every dedupe:false op. - Resolve the fork entry from out/main/chunks too: the resolver compiles into a shared chunk, and the scanner service child has no process.resourcesPath, so packaged WSL vault scans threw entry-not-found (masked as an empty tree). - Allowlist the fork env instead of spreading process.env; ambient NODE_OPTIONS would halt or --require code into every child. - Wrap transport faults (spawn failure, child death) in WslTranscriptFsError('unavailable') so discovery reports them as scan issues instead of misreading them as missing paths or empty trees. - Gate the vitest in-process fallback on the vitest worker global so a leaked VITEST=true cannot revert production to in-process UNC syscalls. - Reap idle sidecar processes after 60s instead of holding them for the app session. - Split 'open' into its own protocol union member so the reusable-call Exclude actually strips it from the pooled-process API. - Guard kill('SIGKILL') against the teardown race where an exiting child emits an unlistened 'error', and dispatch reads by handle kind before path spelling. * fix(native-chat): probe stalled WSL routes instead of a fixed quarantine Remaining review follow-ups: - Escalating route quarantine: first strike lifts after 5s so a distro that was cold-booting when its op hit the deadline recovers on the next poll (~35s total instead of ~90s); repeat stalls double the back-off toward the prior 2x-timeout cap, and any settle the deadline did not force clears the strikes. Queued same-route tasks fail fast at quarantine instead of stranding one waiter deadline per file in sequential scans. - Single request implementation: the vitest in-process fallback now runs the child's own dispatcher (WslTranscriptFsProcessOperations + decode), so unit suites exercise exactly what the forked process executes and the per-call-site fallback closures are gone. Dirent fixtures gained the full kind-flag set the serializer reads. - Dropped the production-dead per-route close queue; UNC FileHandles (test fallback only) mirror the process-handle close contract. - Error class, messages, and factories move to wsl-transcript-fs-error (re-exported from the gate) to keep the gate under the lines budget. * fix(native-chat): harden WSL transcript fs with route quarantine strike Extract quarantine logic into a dedicated module with strike decay: stalls older than 5 minutes restart from base back-off, and concurrent-lane timeouts count as one incident. Allow joining live in-flight tasks on quarantined routes (they cost no new I/O). Preserve quarantine across transport faults (child death). Handle file shrinking during tail reads by detecting short reads and returning empty. Defer file closes that arrive mid-read instead of refusing, preventing slot leaks. Separate process slot and boundary-finding concerns into focused modules. * fix(native-chat): enforce route quarantine windows and isolate lanes per A late result arriving after the deadline was incorrectly lifting the route quarantine, allowing subsequent work to start before the back-off period expired. Now late results are correctly recognized as stale and never cut the quarantine short. Process work is now isolated per (route, priority) lane so a scan stall cannot block exact reads on the same distro. Each lane gets its own client and process pool; late results and handle faults stay scoped to their lane. Tests now fake performance.now() alongside timers (the quarantine clock depends on it) and wait for the full back-off window to expire rather than advancing by 0. Gate state is reset between test cases since late releases never lift the quarantine. --------- Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>	2026-08-18 16:57:35 -07:00
Jinjing	4c69e45552	Strengthen plain-node-entry-guard with entry name validation (#12761) ... * Strengthen plain-node-entry-guard with entry name validation - Add buildStart hook to validate guarded entry names exist in rollup inputs, preventing stale names from silently stopping guards - Extend electron require detection to subpaths (electron/main, etc) - Improve smoke test signal/exit handling and use constants - Add comprehensive tests for entry validation and new behaviors * Add SIGKILL escalation to plain-node-entry-guard timeout Switch from spawnSync to async spawn to properly handle daemons that trap SIGTERM. spawnSync's timeout only sends the signal and waits, so a daemon that ignores SIGTERM causes the build to hang. The new runDaemonEntry function escalates to SIGKILL after a grace period to enforce the deadline. Configurable timeouts and grace periods via SmokeTimings type; closeBundle hook becomes async to support the change.	2026-08-11 12:20:20 -07:00
Brennan Benson	5df2ddbc9c	perf(ai-vault): isolate tab title resolution (#13377) ... * perf(ai-vault): isolate tab title resolution * fix(ai-vault): preserve background scan caches * fix(ai-vault): resolve nested worker from chunks	2026-08-09 16:45:52 -07:00
Jinjing	fde816e4ee	move folders (#12758)	2026-08-05 12:09:24 -07:00