Commit Graph
7 Commits
Author SHA1 Message Date
Orca Worker f8e351ebae feat(native-chat): enable Windows structured sessions 2026-08-31 14:57:30 -07:00
Brennan Benson fd9125ea8c feat(native-chat): Codex structured native chat restructure (#16729)
* feat(native-chat): port structured Codex sessions from restructure-recovery

Rebuilds the desktop structured native-chat implementation from
brennanb2025/native-chat-restructure-recovery (tip 4e31c08db3) on top of
current main as a single commit, scoped to the local Codex path.

Ported:
- Structured agent-session core: durable record store + single-writer lease,
  canonical journal, agent-session wire host/attach/eviction/subscribers,
  `agentSession.*` RPC surface (registered via ALL_RPC_METHODS; host-side
  mobile allowlist included for wire compat), pty write gate, transcript
  additions, and the Codex app-server adapter/launch resolution.
- Renderer: NativeChatStructuredSession view/composer stack, structured
  launch path with the single-flight guard, local structured session tabs
  sync, activation gate + structured inventory (read-only
  `agentSession.handoffStatus` probe), agent-session tabs in the tab strip,
  AI-vault structured session activation, and the settings pane with the
  parent Experimental Chat UI toggle plus the nested "Use updated structured
  native chat" toggle. New sessions require both flags, agent codex, no
  prompt, and a local non-WSL, non-Windows-host execution host
  (structured-native-chat-availability).
- Fixes 72c013cea6 (verified Codex launch recovery), 8ddbaf5e3d (defer
  native terminal view switching affordances), and 4e31c08db3 (release the
  launch gate after a visibility retry) with their regression tests,
  including the third-launch-after-retry guard case.
- Cross-version agent-session wire test + CI lane, packaging entries
  (proper-lockfile, agent-tooling asar excludes), and the wire-compat doc
  section.

Deliberately not ported: mobile/ changes, the Claude structured runtime
(only the claude-transcript-branch-proof and claude-structured-owner-identity
leaf modules remain, backing the kept TUI-recovery arms), the terminal↔chat
adoption/handoff flow (`agentSession.adoptTerminal`/`requestHandoff`, the
handoff request engine, TUI adoption machinery, orca-runtime adoption
methods), renderer switching affordances and their dead leftovers, the
hook/subagent-status refactor cluster, and unrelated branch changes. The
crash-during-acquisition recovery path (restart handoff adjudication,
restore/reverse re-acquire, lease schema handoff keys) is kept because every
plain direct launch depends on it; a trimmed handoff coordinator exposes
only status/restore/close.

Branch edits that targeted files main has since split (ipc/pty.ts,
worktrees.ts, rpc/methods/terminal.ts, useIpcEvents, pty-connection,
store/slices/terminals.ts, runtime-types, web preload) were re-applied to
the split modules, preserving main's newer logic (Windows CIM fallback,
browser tab close rework, cold-restore resume flow, dispatcher threading).

Known seam: the mobile clipboard image-provenance CONSUMER gate ships
(agentSession.send refuses unproven mobile image refs with
agent_session_image_untrusted) but the producer hunk in
rpc/methods/clipboard.ts stays with the unported mobile cluster, so mobile
image sends into structured chat fail closed until that side ports.

* fix(native-chat): trust only authenticated local image uploads

* fix(build): preserve Windows process-tree patch application

* test(windows): include process creation time in addon fixture

* fix(build): run windows-process-tree node-gyp from the physical package dir

gyp expands the node-addon-api dependency by probing node, whose cwd
resolves to the package's physical directory in the store, so the emitted
target is a store-relative ../../../../node-addon-api@... hop. gyp then
resolves that hop against the rebuild cwd; from the node_modules
symlink/junction it escapes the store and configure fails with
"node_addon_api.gyp not found" (run 32999886072).

Rebuild from realpath(package dir) so both bases agree, matching how the
package manager itself runs native install scripts. The regression test
replays gyp's expansion+resolution against the planned cwd and fails
without the fix.

* fix(native-chat): keep chat tabs visible through terminal closes and empty-worktree launches

Two proven blockers in the native Codex tab contract:

closeTerminalTab pre-empted the canonical unified close. With one terminal
left it deactivated the worktree on a terminal/editor/browser-only check,
blanking a workspace that still held a renderable agent-session tab; with
two or more it pre-picked a successor from terminal entities only,
re-stamping the group active before closeUnifiedTab's MRU/neighbor repair
could land on the chat tab. Successor choice now defers to the unified
contract whenever the terminal has a unified row, and deactivation is
gated on the unified renderable count (matching leaveWorktreeIfEmpty),
with the legacy pre-pick kept only for terminals without a unified row.

A structured session created on an empty worktree was published into the
host's headless group while preserveLocalLayout froze the local layout,
leaving the tab in store but permanently off screen. A preserveLocalLayout
owner now always takes client-owned placement — repairing a rendered
leaf whose group record is missing, or materializing a rendered group on a
truly empty worktree — and applies the client-derived layout repair while
still rejecting host-authored layout.

Regression tests drive the real store through closeTerminalTab (git
worktree and folder workspace) and the real snapshot applier for the
empty-worktree adoption states; all fail without the fixes.

* fix(native-chat): close stale turns and retry rejected sends

* fix(native-chat): retire hosted rows on structured tab activation

* fix(native-chat): preserve rpc defaults across main merge

* chore: format remote wire compatibility guide

* test(native-chat): cover retry after unconfirmed send

* fix(native-chat): reload outbox on session switch

* docs(settings): disclose structured chat platform limits

* fix(native-chat): await Codex launch-home preparation

* fix(codex): align child-process allowlist with async trust bridge

* test(identity): update inventory for tab surface refactor

* fix(windows): preserve process-tree CRLF patch sources

* fix(native-chat): anchor an unmatched chat echo where it was sent (#16117)

* fix(native-chat): anchor an unmatched chat echo where it was sent

The reported symptom was old user messages replaying below every new turn, so the
conversation read as scrambled. The cause was not that the echo failed to match a
transcript row. Claude consumes a mid-turn send through a `queued_command`
attachment and writes no `type:"user"` record for it, so some echoes can never
match, and no amount of matching will change that. The cause was WHERE an
unmatched echo rendered: buildMobileNativeChatTransientData appended every pending
item after the entire transcript, so it re-read below each turn that landed
afterwards.

Render each echo directly after the transcript row it was sent against, using the
baseline the send already captures. An unmatched echo is then at worst a duplicate
in the right position rather than a scrambled one, and it stays visible. Echoes
sharing an anchor keep send order; a send with no baseline, or one whose anchor
folding dropped, still falls back to the tail.

Deliberately NOT fixed by deleting the echo. Inferring from send ordering that an
echo can never match, then removing it, loses the user's own text for a message
the agent did receive, and it cannot fire in the common case anyway - measured
drain groups are 1,017 of size 1 against 55 larger. It also escalates an existing
gap: the count pass has no baseline-tail guard, unlike the glue pass, while
`messages` is a 40-row window that head-trims, resets on reconnect and grows at
the front on loadEarlier, so a false landing there would license deleting a
DIFFERENT outstanding message.

That count-pass gap is real and left for a separate change; anchoring makes its
worst case a duplicate in place rather than a scrambled conversation.

* fix(native-chat): preserve folded echo anchors

* fix(native-chat): preserve forward-folded echo anchors

* fix(native-chat): keep leading folded echoes in place

* fix(workspace-cleanup): show git status for every row (#16690)

* fix(native-chat): refuse structured chat on every Windows execution path

canUseStructuredNativeChat only refused win32 when a project runtime
resolved, so folder-workspace keys (and other keys with no project
runtime) failed open into structured chat on Windows. Fail closed on
win32 unconditionally after the host check, matching the settings copy:
local macOS/Linux only; Windows/WSL/SSH stay on terminal chat.

* fix(native-chat): restore runtime refusals behind the win32 gate

506d375de3 replaced the project-runtime checks with a bare platform test,
so a WSL or repair-required runtime resolution would no longer refuse
structured chat off-win32. Keep the unconditional win32 refusal and
re-run the runtime resolution after it, so the gate does not depend on
the resolver's own platform guard. Tests inject WSL and repair-required
resolutions on darwin/linux and fail against the regressed gate.

* fix structured session journal durability

* fix structured tab active pointer after restart

* fix(native-chat): await optional lease renewal callbacks

* refactor(skills): extract install error messages

* fix(agent-session): harden recovery ownership

* fix(native-chat): retain panes across tab activation

* fix(native-chat): address round-one review findings

* test(native-chat): align integration coverage after main merge

* fix(native-chat): harden round-two reliability

* fix(native-chat): harden round-three reliability

* fix(native-chat): close round-four recovery gaps

* fix(native-chat): separate bounded journal key forms

* fix(native-chat): reset outbox error in render on session switch

The switch effect adjusted error state after the sessionId prop changed,
tripping react-doctor's no-adjust-state-on-prop-change on the changed-code
gate and flashing the old session's banner for a frame. Reset it with the
render-time previous-value guard instead.

* fix(native-chat): invalidate stale outbox settlements

* test(native-chat): restore settled-error session-switch regression

a6e2379bd1 replaced this test with the in-flight settlement race test,
leaving the render-time error reset unpinned: deleting the reset block
still passed the whole native-chat suite. Keep both scenarios pinned;
they are distinct (settled error clears on switch vs stale settlement
invalidated in the commit-to-passive window).

* test(wire): make release checkouts race safe

* test(wire): pin cross-process checkout single-flight and importer specifier contract

* test(wire): harden release checkout lifecycle

* fix(build): drop CR-byte residue from windows-process-tree patch

The two trailing CR bytes on the patch's deletion lines are a proven
no-op: pnpm hashes patches CRLF-normalized (both forms hash to the
lockfile's 946ffb2b) and materializes this package without applying the
patch in either form, so the load-bearing build edits come solely from
applyWindowsProcessTreeBuildFixes() (#16947), which handles both source
EOL forms. Restore byte-identity with main and repin the contract test
to the post-#16947 reality: LF-only patch bytes plus lockfile hash sync.

* fix(native-chat): skip empty startup recovery
2026-08-28 16:45:58 -07:00
Neil 2b1b094aa8 fix(cli): pair every resolved CLI with its runtime, and ratchet it (#16383)
Follow-up to #16365, which paired 8 spawn sites by hand. Hand-pairing is how
the class got introduced, so close it structurally instead.

cliPath is now required on CodexAppServerInvocation, `null` only for the
guest-side wsl.exe launcher where a host path pairs nothing. Optional let a
native builder omit it and silently fall back to pairing against a cmd.exe
wrapper with no type error. Every production site already passed it; only
test fixtures needed updating, which is the type doing its job.

Four more sites now pair. codex-state-db-backfill-recovery spawns the same
`codex app-server` subcommand #16365 fixed elsewhere. cli/handlers/account
was the worst case: addAgentNodePaths prepends the *newest* version-manager
bin, which is not necessarily where the CLI being launched lives, so it
actively created the mismatch — pairing now runs last so the CLI's own node
wins. commit-message-text-generation and skills/skill-update-run spawn
resolved binaries with inherited env.

cli/handlers/skills had grown its own buildNpxPath: a weaker local copy that
prepended unconditionally, ignored the Windows `Path` key, and special-cased
a '.' dirname. Deleted in favor of the shared helper, which checks the
sibling node actually exists — the behavior change one test had pinned.

The ratchet is the point: any file that resolves a CLI and spawns must
reference withCliRuntimeOnPath, with a shrink-only allowlist. It caught
skill-update-run, which I had missed. Its first draft required a call paren
and so let dependency-injected resolvers (`resolveCommand: resolveCodexCommand`)
through — verified by removing a pairing and watching it stay green, then
widened until it failed. A second assertion fails on a stale allowlist entry
so an exemption cannot outlive its reason.

external-editor-launch stays allowlisted: it launches a GUI editor, not a
Node CLI whose ABI matters.
2026-08-24 23:12:37 -07:00
Neil a7505fd911 fix(cli): spawn a version-manager CLI with its own node runtime (#16365)
* fix(cli): spawn a version-manager CLI with its own node runtime

resolveCliCommand falls back to scanning every version-manager install when
PATH misses, so it can hand back ~/.nvm/versions/node/v20.x/bin/codex while
PATH still leads with v22. Nothing paired the binary with the runtime it was
installed against, so its `#!/usr/bin/env node` shebang loaded a v20-built
native module under a v22 ABI and the agent died on first require (#10932).

Reproduced with a real addon rather than asserted: a CLI requiring a
cpu-features build for NODE_MODULE_VERSION 115, spawned with v24 leading
PATH, fails with ERR_DLOPEN_FAILED and exit 1. With the CLI's own bin
directory prepended it runs clean.

withCliRuntimeOnPath prepends the resolved command's directory when that
directory ships a sibling node, and is a no-op otherwise — so a Homebrew or
/usr/local CLI is untouched, and the WSL paths pass a bare `codex`/`claude`
that is not absolute and so never matches.

Host CLI resolution in the Claude login path is now lazy, keeping the WSL
branch from resolving a host binary it never spawns.

* fix(cli): split PATH on the delimiter we join with, pair app-server too

Readiness review findings, all four addressed.

withCliRuntimeOnPath chose its join delimiter from the platform option but
split with the host's. Passing platform:'win32' from a posix host turned
`C:\Windows;C:\Windows\System32` into `C;\Windows;C;\Windows\System32` —
every drive letter torn off at its colon. Latent, since no shipped caller
passes platform, but the sole win32 test was written against the corrupted
value and asserted one split segment, so it green-lit the shredding.

That test's other assertion was vacuous: it seeded only `Path`, so the
`PATH` key it asserted absent could never exist. Deleting the whole
case-dedupe block left the suite green. It now seeds both keys and asserts
the full joined string; removing the block fails it.

Nothing covered the wiring, and the argument choice is the easy thing to get
silently wrong. Note it only diverges on win32 — on posix
getSpawnArgsForWindows returns the CLI itself, so pairing the spawn command
is indistinguishable there. The new test drives the win32 branch with a .cmd
fixture; pairing spawnCmd or dropping the wrapper both fail it now.

codex-trust-grant-host and codex-session-index-heal spawn the same
`codex app-server` subcommand through runCodexAppServerSession and were left
unpaired. Pair centrally there via a new optional cliPath, since
invocation.command may be a cmd.exe wrapper.

Pairing tests live in their own file: adding them inline pushed
codex-fetcher.test.ts past the 800-line ratchet.

* fix(cli): read the Windows path key the child will actually use

Round-2 review finding. The read was narrower than the delete: the key was
picked from exactly two spellings (`Path`, else `PATH`), while the twin
dedupe removed every key whose lowercase form is `path`. A block spelling it
`path` or `pATh` therefore had its value deleted without ever being read,
handing the child a PATH containing only the CLI's own directory — a strictly
worse outcome than not pairing at all.

Win32 resolves env names case-insensitively and object order preserves block
order, so the entry the child reads is the first case-insensitive match. The
repo already encodes that rule in resolvePathEnvKey
(src/main/pty/windows-path-segment-merge.ts); src/shared cannot import from
src/main, so mirror it locally.

Verified by execution across six env shapes: lowercase, mixed-case, Path-only,
PATH-only, both twins, and a PATHEXT control that must not be touched. All
preserve the original PATH; before the fix the first two lost it entirely.
Reverting the selector fails the new test and nothing else.
2026-08-24 22:30:04 -07:00
NeilandOrca aab112933e Revert "fix(memory): bound OOM-prone accumulators (#10179)" (#10255)
Co-authored-by: Orca <help@stably.ai>
2026-07-23 18:35:31 -07:00
Neil 8f40ddf328 fix(memory): bound OOM-prone accumulators (#10179) 2026-07-23 06:22:56 -07:00
Brennan BensonandOrcaWin e58de71f5e feat(codex): real-home routing + self-contained multi-account homes (#9501)
* feat(codex): backfill managed-home sessions into the real Codex home once per host

Orca-launched Codex sessions currently land only in the Orca-managed
runtime home, so the user's own `codex resume` picker and app history
never see them (#4444, #8612). Backfill the managed sessions tree into
the real ~/.codex/sessions/YYYY/MM/DD layout once per host:

- hardlink first (one physical rollout log), copy as the cross-volume
  fallback; existing target files are always skipped, nothing in either
  home is deleted or moved
- idempotent; per-file failures leave the completion marker unset so the
  next startup retries cheaply
- JSONL audit log of every link/copy/failure under
  <userData>/codex-session-backfill/
- honors the custom Codex session source home override, mirroring the
  existing system->managed bridge

WSL managed homes are distro-local and need an in-distro variant; that
is a follow-up.

* feat(codex): flag-gated system-default real-home routing scaffolding

Staged internal flag (default OFF, no settings UI): route the SYSTEM-DEFAULT
Codex account at the user's real ~/.codex instead of Orca's managed runtime
home. Flag OFF is byte-identical to today; managed (multi-account) selections
are unchanged in either state.

Routing (flag ON + host system default = no managed account):
- CodexRuntimeHomeService.prepareForCodexLaunch / prepareForRateLimitFetch
  return null so the PTY/env layer injects no managed CODEX_HOME and the
  rate-limit fetcher + auth-presence gate fall back to ~/.codex (the background
  poller stops spawning Codex against the managed home — the #5370 auth war).
- buildPtyHostEnv strips only a nested-Orca-inherited Orca-owned override
  (CODEX_HOME matching the private ORCA_CODEX_HOME marker), preserving a
  user-set CODEX_HOME. Shell-ready re-exports already no-op without the marker.
- The headless commit-message Codex path strips the same inherited override.

Hook install for the real-home lane (append-last into ~/.codex/hooks.json,
trust via the app-server client) lands with the trust plumbing; the managed
hook install is skipped for this lane meanwhile.

Credit @jellychoco (#8606) for the native-home routing direction.

Depends on the codex trust-rpc-grant plumbing for the real-home hook installer.

* fix(codex): strip the daemon-inherited Orca CODEX_HOME override for real-home routing

The daemon spawns PTYs from its own inherited environment and honors only
spawnOptions.envToDelete, so mutating the sparse env object was not enough to
strip an Orca-owned CODEX_HOME the daemon already carries. Add the strip to
envToDelete for both daemon host-spawn paths, preserving a user-set CODEX_HOME.

Verified live via CDP against a sandboxed dev instance (flag ON): an
Orca-spawned pane reports empty CODEX_HOME/ORCA_CODEX_HOME, so Codex resolves
its own ~/.codex. Adds daemon-path unit coverage (strip Orca-owned, preserve
user-owned, no-op when flag OFF).

* fix(codex): harden one-time session backfill

* test(codex): cover staged cross-volume install

* feat(codex): app-server trust-grant client, capability cache, and grant ledger

Short-lived codex app-server JSON-RPC client (hooks/list + config/batchWrite,
the same pair the Codex TUI 'Trust all' flow calls), run in a bundled
ELECTRON_RUN_AS_NODE entry so synchronous launch prep can block on it with a
hard deadline and guaranteed child reap. Capability cache modeled on
GitCapabilityCache, scoped per execution host (native vs each WSL distro),
with a narrow unknown-method/missing-subcommand unsupported predicate. The
grant ledger records verified grants so steady-state launches skip the RPC.

* fix(codex): grant managed hook trust via codex app-server RPCs in install/refresh

Host and WSL installs now grant trust for Orca's managed status hooks through
codex's own hooks/list -> config/batchWrite -> re-list verify, scoped to
exactly the managed entries; the previous computeTrustedHash lane is the
unchanged fallback for incapable/erroring CLIs. getStatus and the removal
paths recognize ledger-recorded codex hashes so drift between codex's real
algorithm and the replica no longer misreports or strands trust. SSH remote
install is untouched by design.

* test(codex): cover app-server trust grant client, cache, ledger, and lanes

* test(codex): cover commit-message real-home override strip/preserve

Adds the two cases for the headless commit-message Codex env under real-home
routing: a nested-Orca-inherited Orca-owned CODEX_HOME is stripped, and a
user-owned CODEX_HOME is preserved.

* test(codex): WSL grant-lane coverage — in-distro invocation and fallback parity

* feat(codex): real-home hook installer trusted via the codex app-server grant client

With the real-home flag ON and the system-default selection, install Orca's
status hook into the user's real ~/.codex before any pane spawns:

- entry APPENDED LAST per managed event: codex hook trust keys are positional
  (source:event:group:handler), so appending keeps every user entry's position
  and trust record intact; user entries and unknown top-level hooks.json fields
  are preserved verbatim
- trust is granted exclusively through the codex app-server client
  (hooks/list + config/batchWrite, verified by re-list); Orca never writes
  [hooks.state] into the user's real config.toml itself
- if the grant lane is unavailable (old binary, unsupported RPC, verify
  failure), the appended entry is rolled back byte-exactly and the host keeps
  the managed-home lane end to end (PTY env, rate limits, commit messages)
  via a lane gate on the runtime-home service
- one-time pristine backup of the user's hooks.json under Orca's userData;
  a rolling .bak sits next to the file (existing atomic writer)
- hook opt-out sweeps Orca entries from the real home and drops Orca-owned
  trust records; flag-off downgrade re-arms the existing legacy system-home
  sweep, which removes the entry and its trust keys cleanly
- the legacy system-home sweep is suppressed only while the real-home lane
  owns ~/.codex/hooks.json, so managed installs cannot delete the entry

* fix(codex): resolve the trust-grant entry without requiring electron

The grant bridge is reachable from plain-Node CLI entries, where the
plain-node entry guard rejects any chunk containing require("electron").
Resolve the bundled session entry from __dirname (root chunk and chunks/
layouts) with an app.asar -> app.asar.unpacked rewrite for packaged runs,
instead of electron's app path APIs.

* fix(codex): keep session backfill off main thread

Use asynchronous, sequential filesystem operations for the one-time rollout backfill, and avoid repeated target-directory probes. Treat inaccessible managed session roots as retryable failures instead of writing a false completion marker.

* fix(codex): harden app-server trust grant fallback

* fix(codex): install cross-volume session backfill copies atomically

On a real Codex home whose filesystem supports no hardlinks (exFAT/FAT,
some network mounts), the staged cross-volume copy was installed with a
non-atomic copyFile(..., COPYFILE_EXCL) straight into the final
rollout-*.jsonl name. An install interrupted mid-copy (app quit, crash,
ENOSPC during the deferred run) could strand a truncated rollout that the
next run then skips as already-present, defeating the staging design's own
guarantee that a failed copy never leaves a partial session behind.

Install the fully-staged copy with an atomic rename instead, guarded by an
existence re-check so it keeps the never-overwrite contract (and the rename
source is the same immutable managed rollout, so any clobber would be
byte-identical). Cover the no-hardlink-support target and an interrupted
install that must leave no partial in the user's sessions tree.

* fix(codex): resolve grant entry from __dirname so plain-node CLI entries stay electron-free

The build guard rejects any electron require reachable from plain-node
entries; the bridge now maps app.asar to app.asar.unpacked by string
replacement instead of consulting electron app paths. CLI typecheck project
lists the new trust-grant module graph.

* fix(codex): harden trust grant reconciliation

* fix(codex): restore trust config permissions on rollback

* fix(codex): harden real-home routing cleanup and retries

* fix(codex): preserve unicode trust RPC responses

* fix(codex): preserve remote env and complete real-home cleanup

* fix(codex): preserve real-home lane invariants

* test(terminal): isolate replacement idle reset assertion

* fix(codex): preserve real-home dotfile links

* fix(codex): preserve verified trust grants across launch prep

* fix(codex): preserve dangling config symlinks on rollback

* fix(codex): don't revoke a just-granted WSL home on a false 'missing' probe

The async wsl.exe canonical-path settlement could report the runtime home
'missing' immediately after a verified RPC grant (a false negative — codex
had just written and re-listed trust there), which drove the reconciliation
'remove' branch to delete all six granted [hooks.state] tables, leaving a bare
[hooks.state] the launching pane read as 'hooks need review'. A 'missing'
settlement now revokes only when no successful install ran this generation; a
genuinely moved home still resolves to a different path and reinstalls.

* test(codex): model codex config/batchWrite faithfully on Windows

The grant-lane stub simulated codex by calling Orca's upsertHookTrustEntries,
which writes both separator variants for a Windows key (a fallback-lane compat
shim real codex never does) — fabricating duplicate tables and whitespace the
RPC path never produces, so the byte-stable and no-duplicate assertions failed
on win32. Replace it with a single-variant, blank-line-separated writer that
matches the real 0.144.x binary's output.

* feat(codex): collapse duplicate session listings across Codex roots

Backfilled/bridged rollouts are hardlinked into both the real ~/.codex and
Orca's managed runtime home, so AI Vault listed each session once per root
(#7521). Dedup candidates by rollout file name pre-parse and parsed sessions
by session id post-parse, keeping the canonical root: host real home first
(unprefixed resume), then the managed runtime home, then other homes. Applies
to local, WSL, and SSH-remote scans.

* feat(codex): background sqlite index heal for backfilled sessions

Codex's own state-DB metadata backfill is one-shot, so rollouts hardlinked in
by Orca's session backfill never become visible to Codex's DB-driven surfaces.
Extract the app-server stdio JSONL transport into codex-app-server-session
(shared with the trust-grant client) and add a bounded, resumable background
pass that drives Codex's lazy indexing via thread/read per backfilled session:
recent-first, batched onto one short-lived server per batch with small
concurrency, ledger + marker so steady-state startups are a no-op, stop-aware
on quit, and capability-aware on CLIs without the app-server surface.

* fix(codex): preserve session identity during dedup heal

* fix(codex): preserve user trust during real-home cleanup

* fix(codex): harden real-home heal boundaries

* fix(codex): fail closed on unsafe backfill install

* fix: harden real-home hook cleanup

* fix(ai-vault): preserve execution boundaries and reap children

* fix(codex): narrow app-server unsupported detection

* fix(codex): bound user hook trust rebase retries per host

The rebase lane ran a codex app-server session on every launch prep while a
host was stuck (CLI without app-server support, or keys hooks/list cannot
match). Gate the transaction on the shared capability cache and add the same
5-minute transient cooldown the grant lane uses, so sweep and legacy-cleanup
retries cost plain fs reads instead of a codex session per pane spawn.

* fix(codex): enforce real-home resume and heal boundaries

* fix(codex): establish real-home lane before cleanup

* fix(codex): stop index heal before delayed spawn

* fix(codex): protect symlinked rolling backups

* fix(ai-vault): preserve resume env deletion through drag

* fix(codex): strip inherited Codex homes on mobile real-home resume

The mobile resume surface types a bare real-home codex resume into a
freshly created pane, but never asked for CODEX_HOME/ORCA_CODEX_HOME
deletion at pane spawn, so an agentDefaultEnv-pinned or daemon-inherited
Codex home rerouted the resume away from the user's real ~/.codex while
the same session resumed correctly on desktop. Share the deletion helper
from the AI Vault resume builders and forward it through the mobile
launch and session.tabs.createTerminal call.

* fix(codex): gate session migration on real-home lane

* fix(codex): stop session backfill after opt-out

* fix(codex): keep session heal failures retryable

* fix(codex): keep session migration state recoverable

* fix(codex): retry republished missing session heals

* fix(codex): preserve hook symlink trust path

* fix(codex): disambiguate POSIX trust paths

* fix(codex): align hook trust source paths

* fix(codex): harden trust grant lifecycle

* fix(codex): restore envToDelete on client invocation type after base reconcile

* test(codex): type child.stdout as PassThrough for oversized-output write

* Assemble RC: reconcile app-server transport API across PRs

Unify on the object RPC surface from the index-heal transport (#8921) while
preserving the default-home env strip (#8828) and the narrowed missing-app-server
capability signal (#8847): adapt the user-hook-trust-rebase consumer + tests,
port envToDelete stripping into the shared session, and route stderr
classification through the canonical capability-signal module.

* RC: enable system-default real-home routing by default (flag ON)

Flip codexSystemDefaultRealHomeEnabled to default ON for this RC's staged
rollout (a user can still opt out by setting it false, which stays byte-identical
to managed-home behavior). This is the only intended behavior difference between
the RC branch and the individual PRs. Updates the two tests that assumed the
prior OFF default.

* fix(codex): snapshot hooks.json bytes+parse in one read to close real-home clobber race

The install/sweep/legacy-cleanup paths parsed hooks.json, then did a separate
later read to capture the previous bytes for the pre-write generation guard.
A concurrent save (second Orca instance or the user editing the file) could
land between the parse and that second read and be silently overwritten.
readHooksJsonWithRaw returns the raw bytes and parse from a single read so the
guard compares against exactly what it parsed. Adds a regression test that
mutates hooks.json mid-RPC and asserts the sweep aborts without clobbering.

* fix(codex): sanitize managed account config trust

* fix(codex): guard OAuth add for custom providers

* fix(codex): persist outgoing managed tokens before real-home lane takeover (PR-C)

prepareForCodexLaunch returns null early for the real-home / system-default
lane before syncForCurrentSelection runs. If a managed account is still
recorded as synced when the selection has dropped to the system default
(nulled without a sync pass, or auto-deselect on missing managed auth), a
Codex-refreshed token stranded in the shared runtime home is never persisted
to its canonical per-account home -> token loss.

Read the outgoing managed account's refreshed token back before the real home
takes over. The real-home lane implies host === null, so running the
managed->system-default transition restores only Orca's runtime mirror from
~/.codex and never writes the real ~/.codex. It is a no-op once the selection
has already been reconciled, so the normal select path does not double-write.

* fix(codex): preserve refreshes across all default transitions

* feat(codex): show system-default/real-home account identity in switcher (PR-B)

The account switcher modeled the system-default Codex account as
activeAccountId:null with no identity fields, so the null row rendered
blank ("System default" / generic subtitle) even though its effective
login is whatever ~/.codex/auth.json currently is.

Add a CodexSystemDefaultIdentity descriptor {hasAuth, authKind, email,
providerAccountId, workspaceLabel} to CodexRateLimitAccountsState,
resolved live and READ-ONLY from ~/.codex by the accounts service and
returned from listAccounts()/getSnapshot(). The settings switcher now
renders the null (system-default) row as that real identity: the OAuth
email when signed in, "Custom provider — no usage tracked." for
env-key/custom-provider logins (auth.json with OPENAI_API_KEY, or an
OPENAI_API_KEY env with no auth.json), and the generic fallback when
signed out. Identity is host-scoped (per-distro WSL keeps the generic
label). Orca never writes ~/.codex; managed-account switches only touch
Orca-owned homes, so the system-default identity stays a stable,
displayed source of truth. Usage already routes to the real home via
getSystemCodexHomePath, so the switcher now attributes it to a real face.

Tests (sandboxed temp homes only): OAuth email/provider resolution,
api-key auth.json and env-key (no auth.json) as custom-provider,
signed-out, and select/deselect of a managed account never mutating
~/.codex/auth.json.

* fix(codex): parse multiline provider pins in OAuth guard

* fix(codex): harden managed trust sanitization

* fix(codex): harden system-default identity rendering

* feat(codex): give each managed account a self-contained CODEX_HOME; retire shared mirror (PR-E)

With the real-home flag ON, a host managed account now launches directly
against its own codex-accounts/<id>/home instead of the shared runtime
mirror + auth.json hot-swap:

- codex-home-paths: syncSystemCodexResourcesIntoManagedHome links system
  resources into any managed home (ownership-marker discipline; never
  symlinks into / mutates ~/.codex).
- runtime-home-service: prepareForCodexLaunch / prepareForRateLimitFetch /
  syncForCurrentSelection route the per-account home directly and skip the
  shared-home hot-swap + token read-back; each home keeps its own auth in
  place (fixes GAP-5 concurrent auth race). Session discovery scans every
  per-account home.
- hook-service / hook-trust-promotion: install/getStatus/refresh accept a
  runtimeHomePath so hooks + RPC-granted trust land in the per-account home.
- service: config mirror into a self-contained home uses the trust-
  preserving merge so granted hook/project trust survives account switches.
- codex-session-root-dedup: rank codex-accounts/<id>/home as canonical
  managed alongside the shared runtime home.

Flag-OFF and the system-default real-home (null) lane are unchanged; the
nested-Orca CODEX_HOME===ORCA_CODEX_HOME daemon strip (#5370) is preserved.
Sandboxed tests only; ~/.codex is never mutated.

* fix(codex): validate per-account home ownership

* fix(codex): keep managed rollouts discoverable across real-home opt-out

WI-4 lossless migration/rollback validation for pre-E shared-mirror managed
accounts. Session discovery gated the per-account home scan on the real-home
flag, so opting back out (flag OFF) hid every rollout an account accumulated
while the flag was ON — the data stayed on disk but vanished from the AI Vault
until the flag flipped back on.

Scan a managed host home whenever it holds a sessions/ tree, independent of the
flag; a never-enabled install keeps its homes credential-only so opt-out stays
byte-identical to today. Forward migration was already lossless (the shared
mirror is always scanned) and the opt-out credential read-back already refuses
to overwrite a fresher per-account token; add tests locking all three
invariants. Sandboxed tests only; ~/.codex is never touched.

* fix(codex): migrate stranded shared auth on E takeover

* test(e2e): isolate Electron from developer Codex home

* test(codex): add real-account validation harness

* fix(codex): finish C and E matcher composition

* fix(codex): bound validation harness shutdown

* test(codex): isolate hook lifecycle user data

* test(codex): cover realistic account-home migration

* fix(codex): keep standalone home tripwire active

* test(codex): fingerprint system auth in validation reports

* fix(codex): bind managed homes to account ownership

* fix(codex): normalize Windows trust source identity

* fix(codex): make Windows trust upgrade transactional

* test(codex): use TypeScript pipeline for validation scripts

* test(codex): run validation modules through native node

* test(codex): allow slow Windows tripwire startup

* fix(codex): survive lingering Windows codex login processes in add-account

On Windows, codex login can keep running (with descendants) after it has
written auth.json, holding OS handles on the per-account managed home
(log/codex-login.log). That made doAddAccount's post-login cleanup fail
with ENOTEMPTY (rmSync) and left an orphaned codex-accounts/<id>/home.

- runCodexLogin now watches for auth.json on Windows and force-kills the
  login process tree (taskkill /t) if it lingers past a short grace
  period; the forced exit is treated as a successful login. The 120s
  timeout path also kills the whole tree instead of only the direct
  child. macOS/Linux behavior is unchanged.
- safeRemoveManagedHome now removes homes with rmSync maxRetries /
  retryDelay (mirroring the local-worktree-filesystem Windows policy)
  and no longer lets a cleanup failure mask the original add error.
- run-codex-real-account-validation.mjs accepts --temp-parent /
  ORCA_CODEX_VALIDATION_TEMP_PARENT so the disposable root can live
  outside %USERPROFILE% on Windows, and fails with an actionable message
  before creating anything when the temp parent is inside the primary
  home. The real-home guard is unchanged.

* fix(codex): preserve managed-account MCP .credentials.json on per-account-home migration (#8440)

Codex file-mode MCP OAuth tokens live in $CODEX_HOME/.credentials.json,
keyed by MCP server URL with no account identity of their own. The legacy
shared-mirror -> per-account-home migration only carried auth.json, so an
existing managed account with authed MCP servers had its tokens stranded on
upgrade and silently needed re-auth.

Carry the shared mirror's .credentials.json into the same identity-proven
per-account home alongside auth.json: only into the single uniquely-matched
active account (no cross-account leak), only when the destination has none yet
(never clobber a newer file the account authed in its own home), atomic 0600,
absent-source no-op. New MCP auth already lands in the per-account home since
that home is CODEX_HOME.

* fix(codex): preserve Windows reauthentication login flow

* test(codex): build real-account validation harness cross-platform on Windows

The harness built its app with execFileSync('npx', ['electron-vite', ...]),
but npx resolves to a .cmd shim on Windows that execFileSync cannot launch
(ENOENT), so the harness could not build its own app there and required
--skip-build with a prebuilt out/main/index.js.

Extract resolveElectronViteBuildCommand(repoRoot): it runs the repository-local
electron-vite JS entry (node_modules/electron-vite/bin/electron-vite.js) with
the current Node binary (process.execPath), which resolves identically on
macOS, Linux, and Windows with no shell. It throws a clear error if the local
entry is missing (install deps or pass --skip-build). --skip-build behavior is
unchanged.

Add regression coverage asserting the build command uses process.execPath and
the repo-local JS entry (not npx), and that a missing entry fails clearly.

* fix(codex): version the MCP creds migration independently of the auth marker

The auth carry and the MCP .credentials.json carry (#8440) shared one
existence-only v1 marker, so any build that stamped the auth-only marker
first would strand the MCP store forever. The MCP carry now concludes via
its own per-account-mcp-creds-migration-v1.json marker and runs even when
the auth marker is already present; ordering is code-enforced instead of
landing-discipline-enforced.

Also isolate per-account read failures: one stale or deleted account home
no longer aborts the whole migration. The broken account stays in the
unique-identity ambiguity gate via its stored fields but is never read or
written, so the active account still migrates.

* fix(codex): fail corrupt managed auth.json without echoing credential bytes

A raw JSON.parse SyntaxError from loadOAuthCredentials could carry auth
file fragments into logs and the add/reauth error surface. Throw a
sanitized error instead; filesystem errors still propagate unchanged.

* fix(mobile): give the pairing runtime a disposable home for the E2E boot guard

The main-process guard now refuses to start with ORCA_E2E_USER_DATA_DIR
set but the real user home, and this was the one caller not updated —
the temporary pairing runtime crashed before emitting its pairing URL.

* test(codex): canonicalize harness containment guards and retry cleanup

Resolve symlinks before the disposable-root containment checks so a
symlinked temp parent cannot smuggle the throwaway home inside the
primary home, and give the final cleanup rm Windows retry/force so a
briefly lingering codex handle cannot strand the credential-bearing
root.

* test(codex): add lane-aware containment mode to the real-account harness

The Windows gate-D run proved strict zero-event whole-profile containment
is structurally unreachable with the real-home flag ON: system-default
spawn sites deliberately delete CODEX_HOME so native codex resolves the
real ~/.codex, and on Windows the binary ignores the USERPROFILE sandbox.
Its own volatile runtime churn (root sqlite/WAL/SHM, tmp/, log/) is the
shipped Phase-1 design, not a candidate defect.

--lane-aware-containment records those designed events without aborting
while every other real-home write — auth.json, config.toml,
.credentials.json, hooks.json, sessions/, anything unknown — remains a
hard violation and still aborts the run. Default behavior is unchanged
(strict); the absolute zero-event claim stays carried by macOS runs,
where HOME does sandbox native codex.

* test(codex): allow the real-account harness to pin the real-home flag off

--system-default-real-home off seeds and env-pins the flag OFF so every
codex spawn gets an explicit managed CODEX_HOME and native codex never
resolves the OS profile. This is the only Windows configuration where the
strict zero-event whole-profile tripwire is reachable, and it matches the
stable-rollout default; flag-ON runs keep lane-aware classification.

* test(codex): correct the flag-off harness comment to kill-switch rationale

The rollout ships all codex-home changes at once (no phased rollout), so
flag OFF is the emergency kill-switch lane, not the stable default.

* test(e2e): canonicalize the isolated E2E home path

The disposable HOME lives under os.tmpdir(), whose spelling is an alias
on CI (macOS /var symlink, Windows 8.3 RUNNER~1). Git canonicalizes
worktree paths, so worktrees created under the aliased home never
matched the app's listing — golden core flows and the packaged
crash-survival harness failed with 'worktree created but not found in
listing'. Resolve the home to its canonical spelling at creation in
both the e2e helper and the packaged-app driver.

* fix(codex): address CodeRabbit review on the landing PR

- carry envToDelete through the mobile agent-resume startup plan so a
  real-home Codex resume cannot inherit an ambient CODEX_HOME
- strip Orca-owned Codex overrides in the commit-message WSL fallback,
  matching the host fallback
- strip ELECTRON_RUN_AS_NODE in the computer-e2e driver like every other
  home-isolation caller
- drop the unused hooksEnabled parameter from isRealHomeCodexHookLaneUsable

* feat(codex): ship real-home routing unconditionally, remove the rollout flag

The codexSystemDefaultRealHomeEnabled setting is gone from types and
constants and the helper no longer consults settings — the system-default
real-home lane and per-account homes ship for everyone in one release.
This also un-strands profiles that rc-era builds stamped with false (the
setting had no UI, so every stored false was a seeded artifact that would
have silently kept those users on the legacy mirror forever).

The ORCA_CODEX_SYSTEM_DEFAULT_REAL_HOME env override survives strictly as
a test-rig control: the containment harness pins the legacy lane for
strict zero-event Windows runs, e2e home isolation pins lanes inside
disposable homes, and the legacy-lane test suites now route their
per-test lane selection through it.

---------

Co-authored-by: OrcaWin <alpha-eng@stably.ai>
2026-07-20 14:34:53 -07:00