From 84f1c520a8027f5ecf49ba8eaaf7959cee453ebc Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Sun, 15 Nov 2026 22:31:24 -0800 Subject: [PATCH] Keep one copy of Codex history shared by every account (#42614) * Share Codex account history through folder links into ~/.codex Each Codex account home keeps its login; its history folders become folder links (junctions on Windows) into the user's ~/.codex, and Codex's databases follow through a sqlite_home line in the account's own config. Migration is automatic before the next launch in each home; the store's copy wins a name conflict and the account's copy is kept beside it. The per-file history bridge, its link pass and the WSL bridge are removed. Claude's merge-then-link moves to a shared async module both providers use. * Make account history moves safe under concurrency and keep the DB with the folders Files move by link-then-unlink so a move never replaces an existing name, host passes run one at a time per store, index markers are written before a move, account-only folders are shared, lock folders are moved rather than deleted, and sqlite_home is written only once a home's sessions are linked. * Let the history pass own sqlite_home, link locks first, and index per store The pass that links a home's folders now writes and removes its sqlite_home line; the config mirror only carries it forward. thread-writer-locks is linked before any history, index markers are kept once per store (hard-linked duplicates included), WSL links the launched home before Codex starts, passes queue per home in request order, and re-sharing after off then on works with earlier conflicts still kept. * Never overwrite on move, gate indexing on moves, and read the shared DB for backfill Files move by link or exclusive copy; repeat conflicts get their own pending folder; secrets and sandbox setup stay per home; the sqlite_home write retries against a concurrent mirror; indexing waits until no move is running; backfill checks read the DB where sqlite_home puts it. * Keep live writers' files, require both history links, and keep unmoved history on removal Files move only by a new name for their inode; a home is shared only when sessions and archived_sessions both link; reverted-thread rollouts get index markers; settled homes skip repeat passes in either mode; account removal keeps a real history folder. * Give resume test runtime-home mocks the account history step * Keep unmoved WSL history on removal, prepare the launched home, and retry incomplete work WSL account removal keeps real sessions/archived_sessions; a launch prepares the account it returns even if the selection changed meanwhile; an incomplete unshare and a failed WSL index pass stay eligible for a later launch; the guest helper rechecks the distro before each step. * Keep pinned resumes reachable, finish interrupted unshares, and keep index work from racing moves A pinned resume only shares; a launch re-prepares when the selection or setting changed; Orca's sqlite_home line is read only at the config root; a writer-lock link elsewhere keeps the home private; an unshare finishes its database move on retry; indexing keeps markers a newer move recorded, retries an incomplete run on a later launch, and merges WSL marker lists. * Move account history into a store one pass at a time and retry incomplete moves Two homes merging into one store at once raced on its folders and stranded the second home's history in its merge folder. Passes now run one at a time per store (host) and per distro (WSL), a folder that appears mid-move is merged into, and a share with failed moves is not settled. * Keep one rollout form in ~/.codex, record what sharing left behind, and guard Windows homes from older builds - A rollout present as .jsonl and .jsonl.zst lands in the store in one form only; the other form moves to the home's .orca-set-aside-rollouts, never deleted. - Each account home gets a versioned cleanup manifest: set-aside databases, left-behind rollouts with a background content check against the store, store twin pairs, and the shared streak. - Native Windows homes get a renamed ownership marker before their first junction, so older builds refuse to use or delete them. - Account removal keeps the set-aside rollouts and the manifest. * Set a rollout form aside only beside its twin in the store, inventory leftovers off the launch path, and keep account databases on removal - The .zst moves first and a form is kept out only when the store already holds its twin, so a failed move never leaves the store with neither form. - The rollout inventory and store twin walk run in the detached check, not the awaited pass; manifest writes are serialized per home and a check applies only to unchanged files; checkpoints are time-spaced. - A WSL check that runs out of budget resumes on a later launch instead of looping. - The ownership check reads either marker in one step and the marker rename retries on Windows. - Account removal keeps the account's Codex databases for the later cleanup release. * Read the cleanup manifest through a type guard * Read the cleanup manifest through a type guard over unknown * Settle only a check's own request, skip queued WSL passes after quit, and validate manifest entries - A pass finishing during a check keeps its new check request on host and WSL. - A WSL pass queued before quit no longer starts guest work. - Manifest rollouts must carry a path, size and mtime to be read. - The store twin scan reuses the twin module's folder walk. * Walk the store outside the manifest write and keep databases on removal only for shared homes - The inventory writes only its own fields onto a freshly read manifest, so it never undoes a streak reset from another process and a launch never waits behind the walk. - Account removal keeps the home's Codex databases only when the cleanup manifest shows the home was shared. * Lock the cleanup manifest across processes and validate every field later passes read - Manifest writes take a file lock, so a WSL check helper can't publish over another helper's streak reset. - The guard also checks storePath, the content check's store facts, database entries and sharedPasses; anything else is rebuilt from disk. --- config/knip.json | 1 + config/main-blocking-sqlite-baseline.txt | 2 +- config/main-sync-fs-baseline.txt | 89 +++-- .../wsl-guest/build-wsl-guest.mjs | 7 +- .../profiles/claude-profile-report.ts | 6 +- .../claude-profile-history-windows.test.ts | 7 +- .../setup/claude-profile-history.test.ts | 13 +- .../profiles/setup/claude-profile-history.ts | 125 +----- .../setup/claude-profile-prompt-history.ts | 4 +- .../profiles/wsl/claude-profile-wsl-router.ts | 39 +- .../codex-managed-home-lifecycle.test.ts | 78 ++++ .../codex-managed-home-lifecycle.ts | 112 +++++- .../host-codex-managed-home-ownership.test.ts | 25 +- .../host-codex-managed-home-ownership.ts | 30 +- .../runtime-home-service-home-routing.ts | 1 - .../routing/runtime-home-service-launch.ts | 41 +- .../runtime-home-service-managed-home.ts | 138 ++++--- ...home-service-per-account-migration.test.ts | 13 +- .../runtime-home-service-state.ts | 6 +- .../runtime-home-wsl-managed-accounts.test.ts | 17 +- .../runtime-home-wsl-session-bridge.test.ts | 14 +- .../settings/codex-config-mirror.test.ts | 37 ++ .../settings/codex-config-mirror.ts | 23 +- .../codex-config-runtime-owned-values.ts | 20 + .../config-toml-runtime-owned-sections.ts | 7 +- .../toml/config-toml-shared-sqlite-home.ts | 85 +++++ .../codex-account-session-bridge.test.ts | 142 ------- .../bridges/codex-account-session-bridge.ts | 141 ------- .../bridges/wsl-codex-session-bridge.test.ts | 129 ------- .../bridges/wsl-codex-session-bridge.ts | 357 ------------------ .../index/codex-account-session-index-heal.ts | 58 +-- .../index/codex-session-index-heal-state.ts | 15 +- .../history/index/codex-state-db.test.ts | 29 +- .../codex/history/index/codex-state-db.ts | 52 ++- ...codex-session-resume-launch-preparation.ts | 1 + .../rollouts/codex-rollout-compressed-twin.ts | 88 +++++ .../rollouts/codex-rollout-thread-id.ts | 14 + .../rollouts/codex-session-link-pass.ts | 299 --------------- .../codex-account-history-cleanup-manifest.ts | 351 +++++++++++++++++ .../codex-account-history-content-check.ts | 137 +++++++ .../codex-account-history-index.ts | 152 ++++++++ .../codex-account-history-links.test.ts | 230 +++++++++++ .../codex-account-history-links.ts | 344 +++++++++++++++++ .../codex-account-history-passes.ts | 148 ++++++++ .../codex-account-history-wsl-entry.ts | 58 +++ .../codex-account-sqlite-home-config.ts | 41 ++ .../shared-store/wsl-codex-account-history.ts | 342 +++++++++++++++++ src/main/filesystem/merged-directory-link.ts | 287 ++++++++++++++ .../codex-account-restart-resume.test.ts | 1 + .../main-process-quit-pending-writer.test.ts | 4 +- .../main-process-quit-update-veto.test.ts | 2 +- src/main/startup/quit/main-process-quit.ts | 4 +- .../wsl/execution/wsl-guest-node-helper.ts | 79 ++++ .../remote-bundle-artifacts.ts | 2 + 54 files changed, 2900 insertions(+), 1547 deletions(-) create mode 100644 src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.test.ts create mode 100644 src/main/codex/configuration/settings/codex-config-runtime-owned-values.ts create mode 100644 src/main/codex/configuration/toml/config-toml-shared-sqlite-home.ts delete mode 100644 src/main/codex/history/bridges/codex-account-session-bridge.test.ts delete mode 100644 src/main/codex/history/bridges/codex-account-session-bridge.ts delete mode 100644 src/main/codex/history/bridges/wsl-codex-session-bridge.test.ts delete mode 100644 src/main/codex/history/bridges/wsl-codex-session-bridge.ts create mode 100644 src/main/codex/history/rollouts/codex-rollout-compressed-twin.ts create mode 100644 src/main/codex/history/rollouts/codex-rollout-thread-id.ts delete mode 100644 src/main/codex/history/rollouts/codex-session-link-pass.ts create mode 100644 src/main/codex/history/shared-store/codex-account-history-cleanup-manifest.ts create mode 100644 src/main/codex/history/shared-store/codex-account-history-content-check.ts create mode 100644 src/main/codex/history/shared-store/codex-account-history-index.ts create mode 100644 src/main/codex/history/shared-store/codex-account-history-links.test.ts create mode 100644 src/main/codex/history/shared-store/codex-account-history-links.ts create mode 100644 src/main/codex/history/shared-store/codex-account-history-passes.ts create mode 100644 src/main/codex/history/shared-store/codex-account-history-wsl-entry.ts create mode 100644 src/main/codex/history/shared-store/codex-account-sqlite-home-config.ts create mode 100644 src/main/codex/history/shared-store/wsl-codex-account-history.ts create mode 100644 src/main/filesystem/merged-directory-link.ts create mode 100644 src/main/wsl/execution/wsl-guest-node-helper.ts diff --git a/config/knip.json b/config/knip.json index 6998964c0f5..5b61c365145 100644 --- a/config/knip.json +++ b/config/knip.json @@ -53,6 +53,7 @@ "src/main/orcad/orcad-local-serve-selection-entry.ts", "src/main/orcad/launcher.ts", "src/main/claude-accounts/profiles/wsl/claude-profile-wsl-entry.ts", + "src/main/codex/history/shared-store/codex-account-history-wsl-entry.ts", "src/main/ai-vault/scanners/opencode/reader-transport/session-scanner-opencode-sqlite-process-entry.ts", "src/wsl-guest/wsl-browser-network-relay.ts", "src/renderer/src/space.tsx", diff --git a/config/main-blocking-sqlite-baseline.txt b/config/main-blocking-sqlite-baseline.txt index 20ee5db56fa..43a4c5eda85 100644 --- a/config/main-blocking-sqlite-baseline.txt +++ b/config/main-blocking-sqlite-baseline.txt @@ -28,7 +28,7 @@ 1 src/main/browser/cookies/chromium/prepare-import.ts 5 src/main/browser/cookies/sources/browser-cookie-firefox-import.ts 25 src/main/browser/cookies/staging/browser-cookie-staged-import.ts -8 src/main/codex/history/index/codex-state-db.ts +5 src/main/codex/history/index/codex-state-db.ts 3 src/main/foreign-sqlite-readers/readers/cursor-profile.ts 44 src/main/github/pr-watch/pr-check-event-tables.ts 14 src/main/github/pr-watch/pr-pending-actions.ts diff --git a/config/main-sync-fs-baseline.txt b/config/main-sync-fs-baseline.txt index 02c1106398e..d2f73c0729a 100644 --- a/config/main-sync-fs-baseline.txt +++ b/config/main-sync-fs-baseline.txt @@ -76,26 +76,25 @@ 1 src/main/claude-accounts/profiles/routing/claude-observed-config-dirs.ts 1 src/main/claude-accounts/profiles/routing/claude-profile-launch-preparation.ts 5 src/main/claude-accounts/profiles/routing/claude-profile-router.ts -14 src/main/claude-accounts/profiles/setup/claude-profile-history.ts +5 src/main/claude-accounts/profiles/setup/claude-profile-history.ts 26 src/main/claude-accounts/profiles/setup/claude-profile-prompt-history.ts 5 src/main/claude-accounts/profiles/setup/claude-profile-provisioning.ts 1 src/main/claude-accounts/profiles/setup/claude-profile-settings.ts 1 src/main/claude-accounts/profiles/setup/claude-profile-setup-worker.ts 1 src/main/claude-accounts/profiles/setup/claude-profile-setup.ts 12 src/main/claude-accounts/profiles/setup/claude-profile-sharing.ts -1 src/main/claude-accounts/profiles/wsl/claude-profile-wsl-router.ts 2 src/main/claude-accounts/runtime-auth/keychain.ts 1 src/main/claude-accounts/runtime-auth/runtime-paths.ts 7 src/main/claude-accounts/sign-in/claude-account-folder.ts 3 src/main/claude/agent-view/agent-view-install.ts 1 src/main/claude/configuration/claude-diff-panel-auto-open.ts 14 src/main/claude/configuration/claude-folder-trust-file.ts -2 src/main/claude/hooks/claude-profile-hook-target.ts 1 src/main/claude/configuration/claude-rejected-api-key.ts -1 src/main/claude/transcripts/claude-rewind-transcript.ts 1 src/main/claude/hook-service.ts +2 src/main/claude/hooks/claude-profile-hook-target.ts 3 src/main/claude/hooks/statusline-install.ts 6 src/main/claude/hooks/windows-hook-files.ts +1 src/main/claude/transcripts/claude-rewind-transcript.ts 1 src/main/cli/appimage/appimage-copy-pre-extraction.ts 3 src/main/cli/appimage/appimage-extracted-root.ts 1 src/main/cli/appimage/appimage-extraction-pruning.ts @@ -108,27 +107,27 @@ 2 src/main/cli/launchers/linux-bare-orca-dispatcher.ts 7 src/main/cli/launchers/linux-terminal-orca-cli-shim.ts 2 src/main/cli/wsl/wsl-managed-cli.ts -2 src/main/codex-accounts/credentials/codex-account-identity.ts 3 src/main/codex-accounts/account-management/codex-account-registration.ts -4 src/main/codex-accounts/managed-homes/codex-config-mirror.ts +2 src/main/codex-accounts/credentials/codex-account-identity.ts 1 src/main/codex-accounts/credentials/codex-credential-absence-grace.ts +1 src/main/codex-accounts/credentials/managed-codex-auth-readiness.ts +36 src/main/codex-accounts/legacy-wsl-auth-drain/legacy-wsl-runtime-auth-drain-script-harness.ts +6 src/main/codex-accounts/legacy-wsl-auth-drain/legacy-wsl-runtime-auth-drain-script-interference-shims.ts 1 src/main/codex-accounts/login/codex-login-auth-snapshot.ts +4 src/main/codex-accounts/managed-homes/codex-config-mirror.ts 5 src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.ts 6 src/main/codex-accounts/managed-homes/codex-managed-home-path.ts 1 src/main/codex-accounts/managed-homes/guest-codex-account-homes.ts 6 src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.ts -2 src/main/codex-accounts/runtime-home/legacy-host/legacy-shared-auth-migration.ts -36 src/main/codex-accounts/legacy-wsl-auth-drain/legacy-wsl-runtime-auth-drain-script-harness.ts -6 src/main/codex-accounts/legacy-wsl-auth-drain/legacy-wsl-runtime-auth-drain-script-interference-shims.ts -1 src/main/codex-accounts/credentials/managed-codex-auth-readiness.ts 7 src/main/codex-accounts/runtime-home/credentials/runtime-home-service-auth-core.ts 5 src/main/codex-accounts/runtime-home/credentials/runtime-home-service-auth-provenance.ts 23 src/main/codex-accounts/runtime-home/credentials/runtime-home-service-auth-sync.ts +2 src/main/codex-accounts/runtime-home/credentials/runtime-home-service-wsl-core.ts +2 src/main/codex-accounts/runtime-home/legacy-host/legacy-shared-auth-migration.ts 19 src/main/codex-accounts/runtime-home/legacy-host/runtime-home-service-legacy-migration.ts 1 src/main/codex-accounts/runtime-home/routing/runtime-home-service-managed-home.ts 14 src/main/codex-accounts/runtime-home/runtime-home-service-paths.ts 1 src/main/codex-accounts/runtime-home/runtime-home-service-sync.ts -2 src/main/codex-accounts/runtime-home/credentials/runtime-home-service-wsl-core.ts 1 src/main/codex-usage/codex-session-file-discovery.ts 1 src/main/codex/configuration/codex-path-observation.ts 21 src/main/codex/configuration/homes/codex-home-paths.ts @@ -243,16 +242,16 @@ 17 src/main/emulator/serve-sim-runtime-materializer.ts 12 src/main/external-editors/external-editor-launch.ts 14 src/main/filesystem/atomic-file-publication.ts -3 src/main/filesystem/file-content-observation.ts # Existing reads/existence checks relocated from observed callers; no new synchronous I/O. 9 src/main/filesystem/durable-file-write.ts +3 src/main/filesystem/file-content-observation.ts # Existing reads/existence checks relocated from observed callers; no new synchronous I/O. 2 src/main/filesystem/rolling-file-backup.ts 2 src/main/foreign-sqlite-readers/foreign-sqlite-reader-entry-path.ts 1 src/main/foreign-sqlite-readers/foreign-sqlite-reader-spawn.ts 1 src/main/foreign-sqlite-readers/readers/cursor-profile.ts 1 src/main/foreign-sqlite-readers/readers/opencode-binder-sessions.ts -1 src/main/git/source-control/ignores/huge-folder-ignore.ts 5 src/main/git/repositories/discovery/repo-detection.ts 13 src/main/git/repositories/discovery/repo-git-marker-scan.ts +1 src/main/git/source-control/ignores/huge-folder-ignore.ts 3 src/main/git/worktrees/catalog/worktree-git-path.ts 1 src/main/github/client/check/check-job-log-cache-retention-child.mjs 3 src/main/grok/grok-hook-backup-cleanup.ts @@ -315,8 +314,8 @@ 4 src/main/muse/hook-service.ts 2 src/main/native-chat/agent-session-attachments/agent-session-attachment-claims.ts 8 src/main/native-chat/agent-session-journal/database/journal-domain-database.ts -3 src/main/native-chat/agent-session-journal/database/journal-legacy-source-receipt.ts 1 src/main/native-chat/agent-session-journal/database/journal-host-database.ts +3 src/main/native-chat/agent-session-journal/database/journal-legacy-source-receipt.ts 1 src/main/native-chat/transcript-discovery/host-readable-transcript-path.ts 2 src/main/native-chat/transcript-filesystem/process/wsl-transcript-fs-process-spawn.ts 1 src/main/node-runtime/appimage-bundled-node-runtime.ts @@ -342,40 +341,40 @@ 4 src/main/orca-profiles/project-transfer/profile-project-state-file.ts 2 src/main/orca-profiles/storage/new-profile-state.ts 5 src/main/orca-profiles/storage/profile-legacy-state-import.ts -2 src/main/orcad/launcher.ts -7 src/main/orcad/native-runtime/node-pty-prebuilt-slot.ts -2 src/main/orcad/native-runtime/node-pty-precondition.ts -2 src/main/orcad/browser-providers/orcad-agent-browser-binary.ts -1 src/main/orcad/host-runtime/orcad-app-paths.ts +4 src/main/orcad/account-launch/orcad-host-account-launch-env-store.ts +2 src/main/orcad/account-launch/orcad-wsl-host-claude-profile-refresh.ts 1 src/main/orcad/artifact-slots/orcad-artifact-identity.ts 2 src/main/orcad/artifact-slots/orcad-artifact-materializer.ts -12 src/main/orcad/runtime-launch/orcad-bundled-runtime.ts -2 src/main/orcad/serve-startup/orcad-cli-launcher.ts -1 src/main/orcad/serve-startup/orcad-command-arguments.ts -1 src/main/orcad/host-lifecycle/managed-stop/orcad-completed-stop-receipt.ts 2 src/main/orcad/artifact-slots/orcad-daemon-slot-pins.ts -2 src/main/orcad/instance-lock/orcad-data-root-locality.ts -3 src/main/orcad/instance-lock/orcad-data-root-privacy.ts -4 src/main/orcad/account-launch/orcad-host-account-launch-env-store.ts -1 src/main/orcad/host-runtime/orcad-host-adapters.ts -1 src/main/orcad/profile-state/orcad-host-identity-reset.ts +2 src/main/orcad/browser-providers/orcad-agent-browser-binary.ts +4 src/main/orcad/credentials/orcad-plaintext-credential-migration.ts +11 src/main/orcad/credentials/orcad-secret-key-record.ts 2 src/main/orcad/host-lifecycle/idle-exit/orcad-idle-stop-record.ts -3 src/main/orcad/instance-lock/orcad-instance-lock-hold.ts -16 src/main/orcad/instance-lock/orcad-instance-lock-reclaim.ts -4 src/main/orcad/instance-lock/orcad-instance-lock.ts -7 src/main/orcad/serve-startup/orcad-local-serve-selection.ts -1 src/main/orcad/instance-lock/orcad-lock-clone-machine.ts +1 src/main/orcad/host-lifecycle/managed-stop/orcad-completed-stop-receipt.ts 1 src/main/orcad/host-lifecycle/managed-stop/orcad-managed-stop-cancellation.ts 3 src/main/orcad/host-lifecycle/managed-stop/orcad-managed-stop-decision.ts 2 src/main/orcad/host-lifecycle/managed-stop/orcad-managed-stop-request.ts -4 src/main/orcad/credentials/orcad-plaintext-credential-migration.ts -5 src/main/orcad/instance-lock/orcad-process-start-time.ts -11 src/main/orcad/credentials/orcad-secret-key-record.ts -3 src/main/orcad/instance-lock/orcad-stale-lock-reclaim.ts 1 src/main/orcad/host-lifecycle/managed-stop/orcad-stop-request-listener.ts +1 src/main/orcad/host-runtime/orcad-app-paths.ts +1 src/main/orcad/host-runtime/orcad-host-adapters.ts +2 src/main/orcad/instance-lock/orcad-data-root-locality.ts +3 src/main/orcad/instance-lock/orcad-data-root-privacy.ts +3 src/main/orcad/instance-lock/orcad-instance-lock-hold.ts +16 src/main/orcad/instance-lock/orcad-instance-lock-reclaim.ts +4 src/main/orcad/instance-lock/orcad-instance-lock.ts +1 src/main/orcad/instance-lock/orcad-lock-clone-machine.ts +5 src/main/orcad/instance-lock/orcad-process-start-time.ts +3 src/main/orcad/instance-lock/orcad-stale-lock-reclaim.ts 1 src/main/orcad/instance-lock/orcad-wsl-holder-release.ts -2 src/main/orcad/account-launch/orcad-wsl-host-claude-profile-refresh.ts +2 src/main/orcad/launcher.ts +7 src/main/orcad/native-runtime/node-pty-prebuilt-slot.ts +2 src/main/orcad/native-runtime/node-pty-precondition.ts +1 src/main/orcad/profile-state/orcad-host-identity-reset.ts 1 src/main/orcad/profile-state/serve-profile-ssh-targets.ts +12 src/main/orcad/runtime-launch/orcad-bundled-runtime.ts +2 src/main/orcad/serve-startup/orcad-cli-launcher.ts +1 src/main/orcad/serve-startup/orcad-command-arguments.ts +7 src/main/orcad/serve-startup/orcad-local-serve-selection.ts 6 src/main/persistence/atomic-snapshot-writer.ts 5 src/main/persistence/loading-store/user-data-path.ts 4 src/main/persistence/migrating-orcad-catalog/blob-transfer/orcad-migration-blob-installers.ts @@ -425,11 +424,11 @@ 8 src/main/pi/managed-extensions/legacy-omp-overlay-migration.ts 5 src/main/pi/managed-extensions/titlebar-extension-service.ts 3 src/main/plugins/appearance/plugin-appearance-service.ts +2 src/main/plugins/development/plugin-dev-watcher.ts 2 src/main/plugins/installation/plugin-install-staging.ts 1 src/main/plugins/installation/plugin-install.ts 1 src/main/plugins/panels/guest-isolation/plugin-frame-guest-sessions.ts 1 src/main/plugins/panels/plugin-source-transpile-client.ts -2 src/main/plugins/development/plugin-dev-watcher.ts 3 src/main/plugins/storage/plugin-secrets-store.ts 3 src/main/plugins/storage/plugin-storage-store.ts 1 src/main/plugins/testing/plugin-editor-static-check.ts @@ -476,8 +475,8 @@ 1 src/main/runtime/client/navigation/desktop-chat-creation-history.ts 9 src/main/runtime/host-instance-identity.ts 4 src/main/runtime/hosts/platform-probe/host-descriptor.ts -5 src/main/runtime/mobile-cloud-relay/region-selection/relay-region-preference.ts 2 src/main/runtime/mobile-cloud-relay/pairing/relay-revoke-outbox.ts +5 src/main/runtime/mobile-cloud-relay/region-selection/relay-region-preference.ts 1 src/main/runtime/orchestration/db/runtime-orchestration-database-access.ts 2 src/main/runtime/push/push-unregister-outbox.ts 1 src/main/runtime/rpc/methods/mobile-delivery/bundled-mobile-web-bundle.ts @@ -517,22 +516,22 @@ 3 src/main/ssh/system-transport/system-ssh-binary.ts 3 src/main/startup/cli-launch/cli-launch-redirect.ts 2 src/main/startup/configure-process.ts +1 src/main/startup/diagnostics/startup-diagnostics.ts 4 src/main/startup/gpu-recovery/gpu-fallback-marker.ts -2 src/main/startup/network-compatibility/http1-compatibility-marker.ts -5 src/main/startup/network-compatibility/http1-compatibility-profile-state.ts +6 src/main/startup/linux-display/x-display-artifacts.ts 1 src/main/startup/linux-runtime/linux-dev-shm-policy.ts 4 src/main/startup/linux-runtime/linux-realtime-scheduling-guard.ts 1 src/main/startup/linux-runtime/linux-rttime-limit.ts +2 src/main/startup/network-compatibility/http1-compatibility-marker.ts +5 src/main/startup/network-compatibility/http1-compatibility-profile-state.ts 2 src/main/startup/profile-startup/profile-state-recovery-preflight.ts -1 src/main/startup/diagnostics/startup-diagnostics.ts -1 src/main/startup/windows-startup/windows-app-data-path.ts 5 src/main/startup/windows-install-permissions/windows-install-dir-acl-poison-marker.ts 1 src/main/startup/windows-install-permissions/windows-install-dir-acl-probe.ts 1 src/main/startup/windows-install-permissions/windows-install-dir-package-acl-grant.ts 4 src/main/startup/windows-install-permissions/windows-install-dir-package-acl-repair.ts +1 src/main/startup/windows-startup/windows-app-data-path.ts 3 src/main/startup/windows-startup/windows-renderer-runtime-files.ts 3 src/main/startup/windows-startup/windows-user-data-acl.ts -6 src/main/startup/linux-display/x-display-artifacts.ts 2 src/main/stats/stats-file-loader.ts 2 src/main/text-generation/prompts/source-control-prompt-files.ts 1 src/main/updater/lifecycle/windows-update-install-directory.ts diff --git a/config/scripts/remote-host-build/wsl-guest/build-wsl-guest.mjs b/config/scripts/remote-host-build/wsl-guest/build-wsl-guest.mjs index 56008774dfb..68dd7219c1b 100644 --- a/config/scripts/remote-host-build/wsl-guest/build-wsl-guest.mjs +++ b/config/scripts/remote-host-build/wsl-guest/build-wsl-guest.mjs @@ -8,7 +8,8 @@ import { join } from 'node:path' import { OPENCODE_SQLITE_READER_FILENAME, OPENCODE_WSL_TEXT_TASK_FILENAME, - WSL_CLAUDE_PROFILE_HELPER_FILENAME + WSL_CLAUDE_PROFILE_HELPER_FILENAME, + WSL_CODEX_ACCOUNT_HISTORY_HELPER_FILENAME } from '../../../../src/shared/remote-runtime/orcad-distribution/remote-bundle-artifacts.ts' const __dirname = import.meta.dirname @@ -58,6 +59,10 @@ await bundle( 'src/main/claude-accounts/profiles/wsl/claude-profile-wsl-entry.ts', WSL_CLAUDE_PROFILE_HELPER_FILENAME ) +await bundle( + 'src/main/codex/history/shared-store/codex-account-history-wsl-entry.ts', + WSL_CODEX_ACCOUNT_HISTORY_HELPER_FILENAME +) await bundle( 'src/main/ai-vault/scanners/opencode/reader-transport/session-scanner-opencode-sqlite-process-entry.ts', OPENCODE_SQLITE_READER_FILENAME, diff --git a/src/main/claude-accounts/profiles/claude-profile-report.ts b/src/main/claude-accounts/profiles/claude-profile-report.ts index 0722d6314da..8745fb5250a 100644 --- a/src/main/claude-accounts/profiles/claude-profile-report.ts +++ b/src/main/claude-accounts/profiles/claude-profile-report.ts @@ -1,3 +1,4 @@ +import { DirectoryLinkError } from '../../filesystem/merged-directory-link' /** A setup step or shared resource name, such as `hooks` or `settings.json`. */ export type ClaudeProfileSurface = string @@ -51,7 +52,10 @@ export function warnClaudeProfile( ): void { report.warnings.push({ surface, - code: error instanceof ClaudeProfileSurfaceError ? error.code : 'failed', + code: + error instanceof ClaudeProfileSurfaceError || error instanceof DirectoryLinkError + ? error.code + : 'failed', detail: error instanceof Error ? error.message : String(error) }) } diff --git a/src/main/claude-accounts/profiles/setup/claude-profile-history-windows.test.ts b/src/main/claude-accounts/profiles/setup/claude-profile-history-windows.test.ts index d81168e8c1f..5a1fa2789e9 100644 --- a/src/main/claude-accounts/profiles/setup/claude-profile-history-windows.test.ts +++ b/src/main/claude-accounts/profiles/setup/claude-profile-history-windows.test.ts @@ -13,6 +13,11 @@ vi.mock('node:fs', async (original) => { symlinkSync: vi.fn(actual.symlinkSync) } }) +vi.mock('node:fs/promises', async (original) => { + const actual = await original() + return { ...actual, symlink: vi.fn(actual.symlink) } +}) +import * as fsPromises from 'node:fs/promises' vi.mock('../../../filesystem/atomic-file-publication', async (original) => { const actual = await original() return { ...actual, writeFileAtomically: vi.fn(actual.writeFileAtomically) } @@ -61,7 +66,7 @@ describe('Claude profile history sharing on Windows', () => { expect(report.warnings).toEqual([]) expect(report.surfaces.projects).toBe('linked') expect(report.surfaces['history.jsonl']).toBe('linked') - expect(fs.symlinkSync).toHaveBeenCalledWith( + expect(fsPromises.symlink).toHaveBeenCalledWith( join(f.defaultHome, 'projects'), join(f.profileHome, 'projects'), 'junction' diff --git a/src/main/claude-accounts/profiles/setup/claude-profile-history.test.ts b/src/main/claude-accounts/profiles/setup/claude-profile-history.test.ts index 25fc1db4bca..e786c516473 100644 --- a/src/main/claude-accounts/profiles/setup/claude-profile-history.test.ts +++ b/src/main/claude-accounts/profiles/setup/claude-profile-history.test.ts @@ -11,6 +11,11 @@ vi.mock('node:fs', async (original) => { statSync: vi.fn(actual.statSync) } }) +vi.mock('node:fs/promises', async (original) => { + const actual = await original() + return { ...actual, link: vi.fn(actual.link) } +}) +import * as fsPromises from 'node:fs/promises' import { shareClaudeProfileHistory } from './claude-profile-history' const roots: string[] = [] function fixture() { @@ -96,19 +101,19 @@ describe('Claude profile history sharing', () => { for (const name of ['a.json', 'b.json', 'c.json']) { fs.writeFileSync(join(f.profileHome, 'todos', name), name) } - const actual = vi.mocked(fs.renameSync).getMockImplementation()! - vi.mocked(fs.renameSync).mockImplementation((from, to) => { + const actual = vi.mocked(fsPromises.link).getMockImplementation()! + vi.mocked(fsPromises.link).mockImplementation(async (from, to) => { if (String(from).endsWith('b.json')) { throw Object.assign(new Error('busy'), { code: 'EBUSY' }) } - actual(from, to) + await actual(from, to) }) const report = await f.share() expect(fs.readdirSync(join(f.defaultHome, 'todos')).sort()).toEqual(['a.json', 'c.json']) expect(report.warnings).toContainEqual( expect.objectContaining({ surface: 'todos', code: 'failed' }) ) - vi.mocked(fs.renameSync).mockReset() + vi.mocked(fsPromises.link).mockImplementation(actual) await f.share() expect(fs.readdirSync(join(f.defaultHome, 'todos')).sort()).toEqual([ 'a.json', diff --git a/src/main/claude-accounts/profiles/setup/claude-profile-history.ts b/src/main/claude-accounts/profiles/setup/claude-profile-history.ts index b558e0bcc02..48f4549be4b 100644 --- a/src/main/claude-accounts/profiles/setup/claude-profile-history.ts +++ b/src/main/claude-accounts/profiles/setup/claude-profile-history.ts @@ -1,30 +1,16 @@ -import { - mkdirSync, - readdirSync, - realpathSync, - rmdirSync, - statSync, - symlinkSync, - unlinkSync -} from 'node:fs' +import { mkdirSync, realpathSync, rmdirSync, unlinkSync } from 'node:fs' import { join } from 'node:path' import '../../../filesystem/atomic-file-publication' -import { renameFileWithWindowsRetry } from '../../../../shared/file-write/windows-retry-file-operations' +import { mergeDirectoryIntoLink } from '../../../filesystem/merged-directory-link' import { resolveClaudeDefaultHome } from '../claude-profile-paths' import { - ClaudeProfileSurfaceError, createClaudeProfileReport, runClaudeProfileSurface, warnClaudeProfile, type ClaudeProfileReport, type ClaudeProfileSurfaceOutcome } from '../claude-profile-report' -import { - CLAUDE_PROFILE_MERGE_SUFFIX, - crossFilesystem, - lstatIfPresent, - mergeClaudeProfilePromptHistory -} from './claude-profile-prompt-history' +import { lstatIfPresent, mergeClaudeProfilePromptHistory } from './claude-profile-prompt-history' // Exclude `sessions`: its live-process registry belongs to each folder's daemon. export const CLAUDE_PROFILE_HISTORY_DIRS = [ @@ -39,48 +25,6 @@ export const CLAUDE_PROFILE_HISTORY_DIRS = [ 'transcripts' ] as const -type MoveResult = { retained: number; failed: unknown[] } - -// One locked entry must not prevent merging the rest. -function moveHistoryTree(source: string, destination: string, result: MoveResult): void { - for (const item of readdirSync(source, { withFileTypes: true })) { - const from = join(source, item.name) - const to = join(destination, item.name) - try { - const existing = lstatIfPresent(to) - if (!existing) { - renameFileWithWindowsRetry(from, to) - } else if (item.isDirectory() && existing.isDirectory()) { - moveHistoryTree(from, to, result) - } else { - result.retained += 1 - } - } catch (error) { - result.failed.push(error) - } - } - if (readdirSync(source).length === 0) { - rmdirSync(source) - } -} - -function drainDirectory( - pending: string, - destination: string, - report: ClaudeProfileReport, - name: string -): void { - const result: MoveResult = { retained: 0, failed: [] } - moveHistoryTree(pending, destination, result) - if (result.failed.length > 0) { - warnClaudeProfile(report, name, result.failed[0]) - } - if (result.retained > 0) { - const detail = `${result.retained} conflicting entries kept in ${pending}` - warnClaudeProfile(report, name, new ClaudeProfileSurfaceError('retained-conflict', detail)) - } -} - /** Link the default home's `name` folder in place of the profile's, moving its entries there. */ export function mergeClaudeProfileDirectory( profile: string, @@ -88,62 +32,13 @@ export function mergeClaudeProfileDirectory( name: string, platform: NodeJS.Platform, report: ClaudeProfileReport -): ClaudeProfileSurfaceOutcome { - const source = join(profile, name) - const destination = join(home, name) - const pending = `${source}${CLAUDE_PROFILE_MERGE_SUFFIX}` - mkdirSync(destination, { recursive: true }) - const sameDevice = (file: string): boolean => statSync(file).dev === statSync(destination).dev - if (lstatIfPresent(pending)?.isDirectory()) { - if (!sameDevice(pending)) { - if (!lstatIfPresent(source)) { - renameFileWithWindowsRetry(pending, source) - } - throw crossFilesystem() - } - try { - drainDirectory(pending, destination, report, name) - } catch (error) { - // Preserve unreadable leftovers without preventing link publication. - warnClaudeProfile(report, name, error) - } - } - const current = lstatIfPresent(source) - if (current?.isSymbolicLink()) { - return realpathSync(source) === realpathSync(destination) ? 'unchanged' : 'user-owned' - } - if (current && !current.isDirectory()) { - return 'user-owned' - } - if (current) { - if (!sameDevice(source)) { - throw crossFilesystem() - } - if (lstatIfPresent(pending)) { - throw new ClaudeProfileSurfaceError( - 'retained-conflict', - `Earlier conflicts kept in ${pending}` - ) - } - renameFileWithWindowsRetry(source, pending) - } - try { - // Junctions avoid Windows symlink privilege requirements. - symlinkSync(destination, source, platform === 'win32' ? 'junction' : 'dir') - } catch (error) { - if (current) { - if (!lstatIfPresent(source)) { - renameFileWithWindowsRetry(pending, source) - } else if (lstatIfPresent(source)?.isDirectory()) { - moveHistoryTree(pending, source, { retained: 0, failed: [] }) - } - } - throw new ClaudeProfileSurfaceError('link-failed', String(error)) - } - if (current) { - drainDirectory(pending, destination, report, name) - } - return 'linked' +): Promise { + return mergeDirectoryIntoLink({ + source: join(profile, name), + destination: join(home, name), + platform, + onWarning: (error) => warnClaudeProfile(report, name, error) + }) } // Undo legacy session sharing; Claude recreates its per-folder process registry. diff --git a/src/main/claude-accounts/profiles/setup/claude-profile-prompt-history.ts b/src/main/claude-accounts/profiles/setup/claude-profile-prompt-history.ts index 1a58f07b2e0..2b2f43b6220 100644 --- a/src/main/claude-accounts/profiles/setup/claude-profile-prompt-history.ts +++ b/src/main/claude-accounts/profiles/setup/claude-profile-prompt-history.ts @@ -19,6 +19,7 @@ import { join } from 'node:path' import { isDefinitiveAbsence } from '../../../../shared/errors/definitive-filesystem-absence' import { writeFileAtomically } from '../../../filesystem/atomic-file-publication' import { renameFileWithWindowsRetry } from '../../../../shared/file-write/windows-retry-file-operations' +import { MERGE_PENDING_SUFFIX } from '../../../filesystem/merged-directory-link' import { ClaudeProfileSurfaceError, warnClaudeProfile, @@ -26,9 +27,8 @@ import { type ClaudeProfileSurfaceOutcome } from '../claude-profile-report' -export const CLAUDE_PROFILE_MERGE_SUFFIX = '.orca-profile-merge' const HISTORY = 'history.jsonl' -const PENDING = `${HISTORY}${CLAUDE_PROFILE_MERGE_SUFFIX}` +const PENDING = `${HISTORY}${MERGE_PENDING_SUFFIX}` // Track Windows hardlink identity to detect replacement of the shared file. const LINK_RECORD = `${HISTORY}.orca-profile-link` diff --git a/src/main/claude-accounts/profiles/wsl/claude-profile-wsl-router.ts b/src/main/claude-accounts/profiles/wsl/claude-profile-wsl-router.ts index 129b8c2da8c..3885955a2f6 100644 --- a/src/main/claude-accounts/profiles/wsl/claude-profile-wsl-router.ts +++ b/src/main/claude-accounts/profiles/wsl/claude-profile-wsl-router.ts @@ -1,7 +1,5 @@ -import { existsSync } from 'node:fs' import { lstat, readFile } from 'node:fs/promises' -import { join, posix } from 'node:path' -import { getAppEnvironment } from '../../../../shared/app-lifecycle/app-environment' +import { posix } from 'node:path' import { CLAUDE_INJECTED_CONFIG_DIR_ENV, CLAUDE_PROFILE_POINTER_ENV, @@ -10,9 +8,8 @@ import { import { WSL_CLAUDE_PROFILE_HELPER_FILENAME } from '../../../../shared/remote-runtime/orcad-distribution/remote-bundle-artifacts' import { parseWslUncPath, toWindowsWslPath } from '../../../../shared/file-access/wsl-paths' import { getWslHomeAsync, listRunningWslDistrosAsync } from '../../../wsl/discovery/distro-catalog' -import { ensureWslPinnedRuntime } from '../../../wsl/execution/wsl-pinned-runtime' -import { relayBundleCandidates } from '../../../wsl/execution/relay-bundle-paths' import { runWslProcess, type WslSpec } from '../../../wsl/execution/wsl-runner' +import { runWslGuestNodeHelper } from '../../../wsl/execution/wsl-guest-node-helper' import { describeWslProcessFailure } from '../../../wsl/execution/wsl-process-failure' import { blockedWslDistroForPath } from '../../../wsl/filesystem/wsl-unc-access-gate' import { claudeProfileMarkerPath, type ClaudeProfileDescriptor } from '../claude-profile-paths' @@ -284,32 +281,14 @@ async function runGuest(distro: string, spec: WslSpec, timeoutMs = 15_000): Prom /** Run guest setup with the pinned Node runtime. */ const runWslSetup: WslSetup = async (distro, home, accountId) => { - const helper = relayBundleCandidates('wsl') - .map((dir) => join(dir, WSL_CLAUDE_PROFILE_HELPER_FILENAME)) - .find(existsSync) - if (!helper) { - throw new Error('The bundled WSL Claude account helper is missing. Reinstall Orca.') - } - const run = (spec: WslSpec, timeoutMs?: number) => runGuest(distro, spec, timeoutMs) - const node = await ensureWslPinnedRuntime( - run, - join(getAppEnvironment().getPath('userData'), 'orcad-artifacts'), - AbortSignal.timeout(180_000) - ) - const guestHelper = await run({ - program: 'wslpath', - args: ['-a', '-u', helper], - loginPath: 'none' - }) // Successful setup is silent; warnings produce a report and refusal exits nonzero. - const report = await run( - { - program: '/usr/bin/env', - args: ['-u', 'NODE_OPTIONS', node, guestHelper, home, distro, accountId], - loginPath: 'none' - }, - 120_000 - ) + const report = await runWslGuestNodeHelper({ + distro, + helperFilename: WSL_CLAUDE_PROFILE_HELPER_FILENAME, + helperLabel: 'Claude account', + helperArgs: [home, distro, accountId], + timeoutMs: 120_000 + }) if (report) { console.warn('[claude-profile] WSL account setup was incomplete:', report) } diff --git a/src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.test.ts b/src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.test.ts new file mode 100644 index 00000000000..db75d8304d9 --- /dev/null +++ b/src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.test.ts @@ -0,0 +1,78 @@ +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + realpathSync, + rmSync, + symlinkSync, + writeFileSync +} from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { CodexManagedHomeLifecycle } from './codex-managed-home-lifecycle' +import type { CodexManagedHomePath } from './codex-managed-home-path' + +const roots: string[] = [] +afterEach(() => { + for (const root of roots.splice(0)) { + rmSync(root, { recursive: true, force: true }) + } +}) + +function fixture() { + const root = realpathSync(mkdtempSync(join(tmpdir(), 'codex-home-removal-'))) + roots.push(root) + const accountsRoot = join(root, 'codex-accounts') + const home = join(accountsRoot, 'a', 'home') + const store = join(root, '.codex') + mkdirSync(home, { recursive: true }) + mkdirSync(join(store, 'sessions'), { recursive: true }) + // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: safeRemove reads only these two members. + const paths = { + assert: async (candidate: string) => candidate, + getRoot: () => accountsRoot + } as unknown as CodexManagedHomePath + return { home, store, accountsRoot, lifecycle: new CodexManagedHomeLifecycle(paths) } +} + +describe('CodexManagedHomeLifecycle.safeRemove', () => { + it("removes the login but keeps the account's history links and conflicting copies", async () => { + const f = fixture() + writeFileSync(join(f.store, 'sessions', 'rollout.jsonl'), 'shared\n') + symlinkSync(join(f.store, 'sessions'), join(f.home, 'sessions'), 'dir') + mkdirSync(join(f.home, 'sessions.orca-profile-merge')) + writeFileSync(join(f.home, 'sessions.orca-profile-merge', 'rollout.jsonl'), 'diverged\n') + writeFileSync(join(f.home, 'auth.json'), '{}') + mkdirSync(join(f.home, 'log')) + + await f.lifecycle.safeRemove(f.home, 'a') + + expect(readdirSync(f.home).sort()).toEqual(['sessions', 'sessions.orca-profile-merge']) + expect(readFileSync(join(f.home, 'sessions', 'rollout.jsonl'), 'utf8')).toBe('shared\n') + expect(readFileSync(join(f.store, 'sessions', 'rollout.jsonl'), 'utf8')).toBe('shared\n') + }) + + it('keeps history that never moved into the store', async () => { + const f = fixture() + mkdirSync(join(f.home, 'sessions')) + writeFileSync(join(f.home, 'sessions', 'rollout.jsonl'), 'only copy\n') + writeFileSync(join(f.home, 'auth.json'), '{}') + + await f.lifecycle.safeRemove(f.home, 'a') + + expect(readdirSync(f.home)).toEqual(['sessions']) + expect(readFileSync(join(f.home, 'sessions', 'rollout.jsonl'), 'utf8')).toBe('only copy\n') + }) + + it('removes a home that never shared history entirely', async () => { + const f = fixture() + writeFileSync(join(f.home, 'auth.json'), '{}') + + await f.lifecycle.safeRemove(f.home, 'a') + + expect(existsSync(join(f.accountsRoot, 'a'))).toBe(false) + }) +}) diff --git a/src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.ts b/src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.ts index 3b3b8975225..ffb522ee390 100644 --- a/src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.ts +++ b/src/main/codex-accounts/managed-homes/codex-managed-home-lifecycle.ts @@ -1,5 +1,6 @@ -import { mkdirSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs' -import { dirname, join, resolve, sep } from 'node:path' +import { mkdirSync, readFileSync, realpathSync, rmSync, writeFileSync, type Dirent } from 'node:fs' +import { readdir, rm } from 'node:fs/promises' +import { join, resolve, sep } from 'node:path' import { isDefinitiveAbsence } from '../../../shared/errors/definitive-filesystem-absence' import { WINDOWS_RM_MAX_RETRIES, @@ -13,11 +14,85 @@ import type { ManagedCodexHomeLocation } from '../account-management/codex-account-service-types' import { writeFileAtomically } from '../../filesystem/atomic-file-publication' +import { MERGE_PENDING_SUFFIX, removeDirectoryLink } from '../../filesystem/merged-directory-link' +import { + CODEX_DATABASE_FILE, + HISTORY_CLEANUP_MANIFEST, + PRE_SHARE_DATABASES_DIR, + SET_ASIDE_ROLLOUTS_DIR +} from '../../codex/history/shared-store/codex-account-history-cleanup-manifest' import { ManagedCodexHomeTemporarilyUnavailableError } from './host-codex-managed-home-ownership' import type { CodexManagedHomePath } from './codex-managed-home-path' const WSL_MANAGED_HOME_TIMEOUT_MS = 5_000 +// Codex recorded the account's conversations under its own path in the shared index, a home +// that was busy at every launch may still hold its only copy, and a merge keeps conflicting +// copies (and set-aside rollout forms, listed in the cleanup manifest) beside the links; all of +// it outlives the login. A home that was shared (it has the manifest) keeps its own databases +// too: deleting them waits for a later release that has seen the shared store work. +const KEPT_HISTORY = [ + 'sessions', + 'archived_sessions', + SET_ASIDE_ROLLOUTS_DIR, + HISTORY_CLEANUP_MANIFEST, + PRE_SHARE_DATABASES_DIR +] +const KEEP_ON_REMOVAL_SCRIPT = [ + 'set -u', + 'home=$1', + 'kept() {', + ' case "${1##*/}" in', + ' *.orca-profile-merge) return 0 ;;', + ` ${KEPT_HISTORY.join(' | ')}) return 0 ;;`, + ` *.sqlite | *.sqlite-wal | *.sqlite-shm) [ -f "$home/${HISTORY_CLEANUP_MANIFEST}" ] ;;`, + ' *) return 1 ;;', + ' esac', + '}', + 'any_kept=0', + 'for entry in "$home"/* "$home"/.[!.]* "$home"/..?*; do', + ' if kept "$entry"; then any_kept=1; fi', + 'done', + 'if [ "$any_kept" = 0 ]; then', + ' rm -rf -- "$home"', + ' rmdir -- "${home%/*}" 2>/dev/null || true', + ' exit 0', + 'fi', + // GNU rm removes a link itself, never what it points to. + 'for entry in "$home"/* "$home"/.[!.]* "$home"/..?*; do', + ' if [ -e "$entry" ] || [ -L "$entry" ]; then kept "$entry" || rm -rf -- "$entry"; fi', + 'done' +].join('\n') + +/** + * Deletes the login and private state but keeps shared history; true when anything was kept. + * Links are removed one by one first, so no recursive delete ever meets a Windows junction. + */ +async function removeManagedHomeKeepingSharedHistory(home: string): Promise { + const entries = await readdir(home, { withFileTypes: true }) + const sharedOnce = entries.some((entry) => entry.name === HISTORY_CLEANUP_MANIFEST) + const kept = (entry: Dirent): boolean => + KEPT_HISTORY.includes(entry.name) || + entry.name.endsWith(MERGE_PENDING_SUFFIX) || + (sharedOnce && entry.isFile() && CODEX_DATABASE_FILE.test(entry.name)) + for (const entry of entries.filter((candidate) => !kept(candidate))) { + const path = join(home, entry.name) + if (entry.isSymbolicLink()) { + // Never recurse through a link (a Windows junction included). + await removeDirectoryLink(path) + continue + } + await rm(path, { + recursive: true, + force: true, + maxRetries: WINDOWS_RM_MAX_RETRIES, + retryDelay: WINDOWS_RM_RETRY_DELAY_MS + }) + } + // Read again: history a writer created meanwhile must not fall to the whole-home removal. + return (await readdir(home, { withFileTypes: true })).some(kept) +} + function removeManagedHomeTreeSync(targetPath: string): void { // Retry transient Windows locks held by login descendants to avoid orphaning the home. rmSync(targetPath, { @@ -99,7 +174,31 @@ export class CodexManagedHomeLifecycle { return } + const wslHome = parseWslUncPath(managedHomePath) + if (wslHome) { + // Never walk a guest tree from Windows: a Linux link must not be followed over 9P. + try { + const result = await runWslProcess({ + distro: wslHome.distro, + loginPath: 'none', + script: KEEP_ON_REMOVAL_SCRIPT, + args: [wslHome.linuxPath], + shell: 'bash', + timeoutMs: WSL_MANAGED_HOME_TIMEOUT_MS + }) + if (result.code !== 0 || result.timedOut) { + throw new Error(`WSL cleanup exited with ${result.timedOut ? 'a timeout' : result.code}`) + } + } catch (error) { + console.warn('[codex-accounts] Failed to remove managed home:', error) + } + return + } + try { + if (await removeManagedHomeKeepingSharedHistory(managedHomePath)) { + return + } removeManagedHomeTreeSync(managedHomePath) } catch (error) { // Cleanup failures must not mask the original error. @@ -107,15 +206,6 @@ export class CodexManagedHomeLifecycle { return } - if (parseWslUncPath(managedHomePath)) { - try { - removeManagedHomeTreeSync(dirname(managedHomePath)) - } catch { - // Best-effort cleanup - } - return - } - // Remove the account directory left after deleting its home leaf. try { const parentDir = resolve(managedHomePath, '..') diff --git a/src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.test.ts b/src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.test.ts index b04dca5b386..8a11452d1e4 100644 --- a/src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.test.ts +++ b/src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.test.ts @@ -1,5 +1,13 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest' -import { mkdtempSync, mkdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' +import { + mkdtempSync, + mkdirSync, + readFileSync, + realpathSync, + rmSync, + symlinkSync, + writeFileSync +} from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { assertOwnedHostCodexManagedHomePath } from './host-codex-managed-home-ownership' @@ -46,6 +54,21 @@ describe('assertOwnedHostCodexManagedHomePath', () => { expect(readFileSync(sentinelPath, 'utf-8')).toBe('system-auth\n') }) + it('owns a home whose marker was renamed when it got junctions', () => { + const accountHome = join(userDataDir, 'codex-accounts', 'account-1', 'home') + mkdirSync(accountHome, { recursive: true }) + writeFileSync(join(accountHome, '.orca-managed-home-linked'), 'account-1\n', 'utf-8') + + expect( + assertOwnedHostCodexManagedHomePath({ + candidatePath: accountHome, + managedAccountsRoot: join(userDataDir, 'codex-accounts'), + systemCodexHomePath: systemHomePath, + expectedAccountId: 'account-1' + }) + ).toBe(realpathSync(accountHome)) + }) + it('rejects a persisted home or marker belonging to another account', () => { const accountHome = join(userDataDir, 'codex-accounts', 'account-2', 'home') mkdirSync(accountHome, { recursive: true }) diff --git a/src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.ts b/src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.ts index 813a736dee5..01b5c924bf9 100644 --- a/src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.ts +++ b/src/main/codex-accounts/managed-homes/host-codex-managed-home-ownership.ts @@ -11,6 +11,30 @@ type HostCodexManagedHomeOwnershipOptions = { export const MISSING_MANAGED_HOME_MESSAGE = 'Managed Codex home directory does not exist on disk.' +export const MANAGED_HOME_MARKER = '.orca-managed-home' +/** + * Replaces the marker before a Windows home gets junctions: older builds refuse to use or delete + * a home without the first one, and their recursive delete would follow a junction into ~/.codex. + */ +export const LINKED_MANAGED_HOME_MARKER = '.orca-managed-home-linked' + +// The linked marker is read second: a share pass renames the first into it, never back, so a +// rename landing between the two reads still finds one of them. +function readMarker(home: string): { isFile: boolean; contents: string } { + for (const name of [MANAGED_HOME_MARKER, LINKED_MANAGED_HOME_MARKER]) { + const path = join(home, name) + try { + const isFile = lstatSync(path).isFile() + return { isFile, contents: isFile ? readFileSync(path, 'utf-8') : '' } + } catch (error) { + if (name === LINKED_MANAGED_HOME_MARKER || !isDefinitiveAbsence(error)) { + throw error + } + } + } + throw new Error('unreachable') +} + /** Only `untrusted` may clear selection; `indeterminate` must refuse use without erasing state. */ export type HostCodexManagedHomeVerdict = | { kind: 'owned'; homePath: string } @@ -127,12 +151,12 @@ function evaluate({ } } - const markerPath = join(canonicalCandidate, '.orca-managed-home') let markerIsRegularFile: boolean let markerContents: string try { - markerIsRegularFile = lstatSync(markerPath).isFile() - markerContents = markerIsRegularFile ? readFileSync(markerPath, 'utf-8') : '' + const marker = readMarker(canonicalCandidate) + markerIsRegularFile = marker.isFile + markerContents = marker.contents } catch (error) { // A missing ownership marker disproves trust; an unreadable one leaves it unknown. if (isDefinitiveAbsence(error)) { diff --git a/src/main/codex-accounts/runtime-home/routing/runtime-home-service-home-routing.ts b/src/main/codex-accounts/runtime-home/routing/runtime-home-service-home-routing.ts index 797a1b83492..49eb777fb3c 100644 --- a/src/main/codex-accounts/runtime-home/routing/runtime-home-service-home-routing.ts +++ b/src/main/codex-accounts/runtime-home/routing/runtime-home-service-home-routing.ts @@ -218,7 +218,6 @@ export abstract class CodexRuntimeHomeRouting extends CodexRuntimeHomeManagedHom homePath: string | null ): void { this.syncWslConfigAndGlobalInstructionsForLaunch(target, homePath) - this.startWslSessionBridgeForLaunch(target, homePath) } protected syncWslConfigAndGlobalInstructionsForLaunch( diff --git a/src/main/codex-accounts/runtime-home/routing/runtime-home-service-launch.ts b/src/main/codex-accounts/runtime-home/routing/runtime-home-service-launch.ts index 0be2b424cb1..e0d816014ea 100644 --- a/src/main/codex-accounts/runtime-home/routing/runtime-home-service-launch.ts +++ b/src/main/codex-accounts/runtime-home/routing/runtime-home-service-launch.ts @@ -1,9 +1,7 @@ import { resolveHostCodexSessionSourceHome } from '../../../codex/history/rollouts/codex-session-source-home' import { startSystemCodexSessionBridgeInBackground } from '../../../codex/history/bridges/codex-session-bridge' -import { startCodexAccountSessionBridgeInBackground } from '../../../codex/history/bridges/codex-account-session-bridge' import { isCodexHistorySharedAcrossAccounts } from './codex-history-sharing' import { - getSystemCodexHomePath, resolveOrcaManagedCodexHomePath, syncSystemCodexResourcesIntoManagedHome } from '../../../codex/configuration/homes/codex-home-paths' @@ -46,6 +44,7 @@ export abstract class CodexRuntimeHomeLaunch extends CodexRuntimeHomeRouting { const homePath = this.getWslCodexHomePathForSelection(wslTarget) this.startLegacyWslAuthDrain(wslTarget) this.finishWslLaunchPreparation(wslTarget, homePath) + void this.prepareWslAccountHistory(wslTarget, homePath) return homePath } const selfContainedAccount = this.getSelfContainedManagedHostAccount() @@ -63,7 +62,6 @@ export abstract class CodexRuntimeHomeLaunch extends CodexRuntimeHomeRouting { // Null leaves CODEX_HOME uninjected; refresh the retired mirror only for retained panes. this.copyMirrorLoginIntoEmptySystemHome() this.reconcileLegacySharedHomeForRetainedPanes() - this.startSystemDefaultSessionBridgeForLaunch() return null } this.invalidateBackfillAfterManagedSystemDefaultLaunch(launchEnv) @@ -87,27 +85,6 @@ export abstract class CodexRuntimeHomeLaunch extends CodexRuntimeHomeRouting { return this.getRuntimeHomePath() } - // Real-home launches see managed-account history too, or switching back to system default hides it (#25000). - private startSystemDefaultSessionBridgeForLaunch(): void { - const settings = this.store.getSettings() - const sourceCodexHomePaths = this.getManagedHostAccountHomesForSessionDiscovery() - const targetCodexHomePath = - resolveHostCodexSessionSourceHome(settings) ?? getSystemCodexHomePath() - if ( - sourceCodexHomePaths.length === 0 || - !isCodexHistorySharedAcrossAccounts(settings) || - this.isHostHistorySourceInWsl() - ) { - return - } - // The user's own ~/.codex gets hardlinks or nothing: a symlink there dangles once an account goes. - void startCodexAccountSessionBridgeInBackground({ - targetCodexHomePath, - sourceCodexHomePaths, - hardlinkOnly: true - }) - } - /** Resolve the launch home without side effects for catalog reads; null means real ~/.codex. */ resolveHostCodexHomePathForLaunchReadOnly(launchEnv?: NodeJS.ProcessEnv): string | null { const selfContainedAccount = this.getSelfContainedManagedHostAccount() @@ -137,6 +114,20 @@ export abstract class CodexRuntimeHomeLaunch extends CodexRuntimeHomeRouting { options?: CodexHomeLaunchOptions ): Promise { if (target?.runtime !== 'wsl') { + // Codex must start on linked history, so the launched home's pass finishes first; the + // selection or the setting may change while it runs, so prepare until both held still. + let prepared: string | null = null + for (let attempt = 0; attempt < 3; attempt += 1) { + const account = this.getSelfContainedManagedHostAccount() + const accountHome = account ? this.getTrustedSelfContainedManagedHomePath(account) : null + const shared = isCodexHistorySharedAcrossAccounts(this.store.getSettings()) + if (!accountHome || `${accountHome}\0${shared}` === prepared) { + break + } + await this.prepareAccountHistory(accountHome) + prepared = `${accountHome}\0${shared}` + } + this.prepareAllAccountHistoryInBackground() return this.prepareForCodexLaunch(target, launchEnv, options) } const wslTarget = this.resolveWslDefaultTarget(target) @@ -153,6 +144,8 @@ export abstract class CodexRuntimeHomeLaunch extends CodexRuntimeHomeRouting { const homePath = this.getWslCodexHomePathForSelection(wslTarget) // Drain potentially fresher retired-home credentials before the first direct-home spawn. await this.startLegacyWslAuthDrain(wslTarget, { throwOnFailure: true }) + // Codex must start on linked history (and the guest's sqlite_home line) in this home. + await this.prepareWslAccountHistory(wslTarget, homePath) this.finishWslLaunchPreparation(wslTarget, homePath) return homePath } diff --git a/src/main/codex-accounts/runtime-home/routing/runtime-home-service-managed-home.ts b/src/main/codex-accounts/runtime-home/routing/runtime-home-service-managed-home.ts index 438781c299d..d140a657bec 100644 --- a/src/main/codex-accounts/runtime-home/routing/runtime-home-service-managed-home.ts +++ b/src/main/codex-accounts/runtime-home/routing/runtime-home-service-managed-home.ts @@ -3,16 +3,12 @@ import { join } from 'node:path' import { isDefinitiveAbsence } from '../../../../shared/errors/definitive-filesystem-absence' import { syncSystemCodexResourcesIntoManagedHome, - getSystemCodexHomePath, - resolveOrcaManagedCodexHomePath + getSystemCodexHomePath } from '../../../codex/configuration/homes/codex-home-paths' import { syncSystemConfigIntoManagedCodexHome } from '../../../codex/configuration/settings/codex-config-mirror' -import { startCodexAccountSessionBridgeInBackground } from '../../../codex/history/bridges/codex-account-session-bridge' +import { CodexAccountHistoryPasses } from '../../../codex/history/shared-store/codex-account-history-passes' import { isCodexHistorySharedAcrossAccounts } from './codex-history-sharing' -import { - resolveHostCodexSessionSourceHome, - resolveWslCodexSessionSourceHome -} from '../../../codex/history/rollouts/codex-session-source-home' +import { resolveHostCodexSessionSourceHome } from '../../../codex/history/rollouts/codex-session-source-home' import { parseWslUncPath } from '../../../../shared/file-access/wsl-paths' import { normalizeRuntimePathForComparison } from '../../../../shared/file-access/cross-platform-path' import { normalizeCodexRuntimeSelection } from '../../account-management/runtime-selection' @@ -27,14 +23,14 @@ import { markCodexSessionBackfillMarkerNeedsFullScan } from '../../../codex/history/backfill/codex-session-backfill-marker' import { getDefaultWslDistro } from '../../../wsl/discovery/distro-catalog' -import { startWslCodexSessionBridgeInBackground } from '../../../codex/history/bridges/wsl-codex-session-bridge' +import { WslCodexAccountHistoryLaunches } from '../../../codex/history/shared-store/wsl-codex-account-history' import type { CodexAccountSelectionTarget } from '../../../../shared/agent-accounts/codex/codex-selection-lane' import type { CodexManagedAccount } from '../../../../shared/agent-accounts/managed-account-types' import { CodexRuntimeHomeSync } from '../runtime-home-service-sync' export abstract class CodexRuntimeHomeManagedHome extends CodexRuntimeHomeSync { - // One WSL sibling-bridge loop per target home; keyed by distro and Linux path. - private readonly wslSessionBridgesByTarget = new Map>() + private readonly accountHistoryPasses = new CodexAccountHistoryPasses() + private readonly wslAccountHistory = new WslCodexAccountHistoryLaunches() // Host accounts refresh auth in separate homes so concurrent accounts cannot race one auth.json. protected getSelfContainedManagedHostAccount(): CodexManagedAccount | null { @@ -140,19 +136,39 @@ export abstract class CodexRuntimeHomeManagedHome extends CodexRuntimeHomeSync { } else { syncConfig() } - this.startSelfContainedSessionBridgeForLaunch(perAccountHome) return perAccountHome } - // Bridge history in the background: /resume only sees launch-home rollouts, and scans can be large. - protected startSelfContainedSessionBridgeForLaunch(perAccountHome: string): void { + /** Links this home's history into ~/.codex (fresh private folders when sharing is off). */ + prepareAccountHistory(accountHome: string): Promise { + return this.accountHistoryPasses.run( + accountHome, + isCodexHistorySharedAcrossAccounts(this.store.getSettings()) + ) + } + + /** Brings every trusted host account's history into the store without waiting for its launch. */ + protected prepareAllAccountHistoryInBackground(): void { + for (const home of this.getManagedHostAccountHomesForSessionDiscovery()) { + void this.prepareAccountHistory(home) + } + } + + /** + * For launches pinned to a home other than the selected account's, such as a resume. Only + * shares: provenance chose this home for a transcript that unsharing would move out of reach. + */ + async prepareAccountHistoryForPinnedHome(home: string): Promise { if (!isCodexHistorySharedAcrossAccounts(this.store.getSettings())) { return } - void startCodexAccountSessionBridgeInBackground({ - targetCodexHomePath: perAccountHome, - sourceCodexHomePaths: this.getSelfContainedSessionBridgeSourceHomes() - }) + const key = normalizeRuntimePathForComparison(home) + const accountHome = this.getManagedHostAccountHomesForSessionDiscovery().find( + (candidate) => normalizeRuntimePathForComparison(candidate) === key + ) + if (accountHome) { + await this.prepareAccountHistory(accountHome) + } } // Reading a distro's history over 9P would boot it (E9); the WSL lane owns those homes. @@ -163,20 +179,6 @@ export abstract class CodexRuntimeHomeManagedHome extends CodexRuntimeHomeSync { return parseWslUncPath(sourceHome) !== null } - protected getSelfContainedSessionBridgeSourceHomes(): string[] { - return [ - // Honor custom history sources without changing credential routing. - ...(this.isHostHistorySourceInWsl() - ? [] - : [ - resolveHostCodexSessionSourceHome(this.store.getSettings()) ?? getSystemCodexHomePath() - ]), - // Resolve the mirror path without creating it for per-account launches. - resolveOrcaManagedCodexHomePath(), - ...this.getManagedHostAccountHomesForSessionDiscovery() - ] - } - // Per-account homes own auth refreshes in place; selection needs no shared-mirror read-back. protected syncSelfContainedManagedSelection(account: CodexManagedAccount): void { const resolved = this.resolveSelfContainedManagedHome(account) @@ -190,8 +192,8 @@ export abstract class CodexRuntimeHomeManagedHome extends CodexRuntimeHomeSync { this.lastHostAccountUsedSelfContainedHome = true this.sharedAuthRefreshBlockedByManagedTransition = true this.markSharedRuntimeAuthManaged(account.id) - // Start history bridging at selection so /resume is ready before the next launch. - this.startSelfContainedSessionBridgeForLaunch(perAccountHome) + // Link history at selection so the next launch finds it done. + void this.prepareAccountHistory(perAccountHome) return } this.clearSelfContainedManagedSelection(account) @@ -277,55 +279,41 @@ export abstract class CodexRuntimeHomeManagedHome extends CodexRuntimeHomeSync { return this.prepareHostSystemDefaultSessionMigrationPass() } - protected startWslSessionBridgeForLaunch( + // Inside the distro its ~/.codex is the store; host and guest stores stay separate. + /** The launched WSL account home's pass is awaited; the distro's other homes run behind it. */ + protected prepareWslAccountHistory( target: CodexAccountSelectionTarget, runtimeHomePath: string | null - ): void { - if ( - process.platform !== 'win32' || - !runtimeHomePath || - !isCodexHistorySharedAcrossAccounts(this.store.getSettings()) - ) { - return + ): Promise { + if (process.platform !== 'win32' || !runtimeHomePath) { + return Promise.resolve() } - const runtimeHomeWsl = parseWslUncPath(runtimeHomePath) - const distro = target.wslDistro?.trim() || runtimeHomeWsl?.distro || getDefaultWslDistro() - if (!distro) { - return + const distro = + target.wslDistro?.trim() || parseWslUncPath(runtimeHomePath)?.distro || getDefaultWslDistro() + const storeHome = distro + ? this.getWslSystemCodexHomePath({ runtime: 'wsl', wslDistro: distro }) + : null + if (!distro || !storeHome) { + return Promise.resolve() } - // Honor custom WSL history sources without changing credential routing. - const systemCodexHomePath = - resolveWslCodexSessionSourceHome(this.store.getSettings(), distro) ?? - this.getWslSystemCodexHomePath({ runtime: 'wsl', wslDistro: distro }) const linuxPathOf = (home: string): string => parseWslUncPath(home)?.linuxPath ?? home - const targetLinuxPath = linuxPathOf(runtimeHomePath) + const launched = linuxPathOf(runtimeHomePath) + const run = (accountHome: string): Promise => + this.wslAccountHistory.run({ + distro, + accountHome, + storeHome: linuxPathOf(storeHome), + shared: isCodexHistorySharedAcrossAccounts(this.store.getSettings()) + }) const accountHomes = this.store .getSettings() - .codexManagedAccounts.map((account) => this.getWslLaunchCodexHomePath(account, distro)) - // Siblings only flow into a selected account's home; system default's own ~/.codex gets none (#24511). - const targetIsAccountHome = accountHomes.some( - (home) => !!home && linuxPathOf(home) === targetLinuxPath - ) - const sourceHomes = [ - systemCodexHomePath, - // Sibling account homes are sources too, or a switch hides the previous account's sessions (#18486). - ...(targetIsAccountHome ? accountHomes : []) - ].filter((home): home is string => !!home && linuxPathOf(home) !== targetLinuxPath) - const targetKey = `${distro.toLowerCase()}\0${targetLinuxPath}` - if (sourceHomes.length === 0 || this.wslSessionBridgesByTarget.has(targetKey)) { - // A concurrent launch joins the loop already bridging this home. - return + .codexManagedAccounts.flatMap( + (account) => this.getWslLaunchCodexHomePath(account, distro) ?? [] + ) + .map(linuxPathOf) + for (const home of accountHomes.filter((candidate) => candidate !== launched)) { + void run(home) } - // Create history links inside the distro so Codex can see them in /resume; one wsl.exe at a time. - const loop = (async () => { - for (const sourceHome of new Set(sourceHomes)) { - await startWslCodexSessionBridgeInBackground({ - distro, - systemCodexHomePath: sourceHome, - managedCodexHomePath: runtimeHomePath - }) - } - })().finally(() => this.wslSessionBridgesByTarget.delete(targetKey)) - this.wslSessionBridgesByTarget.set(targetKey, loop) + return accountHomes.includes(launched) ? run(launched) : Promise.resolve() } } diff --git a/src/main/codex-accounts/runtime-home/routing/runtime-home-service-per-account-migration.test.ts b/src/main/codex-accounts/runtime-home/routing/runtime-home-service-per-account-migration.test.ts index 334b79a30b9..f73e8028d0c 100644 --- a/src/main/codex-accounts/runtime-home/routing/runtime-home-service-per-account-migration.test.ts +++ b/src/main/codex-accounts/runtime-home/routing/runtime-home-service-per-account-migration.test.ts @@ -23,20 +23,15 @@ vi.mock('node:os', async () => { const actual = await vi.importActual('node:os') return { ...actual, homedir: () => testState.home } }) -// Prevent the selection-triggered history bridge from spawning a real app-server. -vi.mock('../../../codex/history/index/codex-account-session-index-heal', () => { - const upToDate = async () => ({ +// Prevent the selection-triggered history indexing from spawning a real app-server. +vi.mock('../../../codex/history/index/codex-account-session-index-heal', () => ({ + healPendingCodexAccountThreads: async () => ({ outcome: 'up-to-date', healedThreads: 0, missingThreads: 0, failedThreads: 0 }) - return { - createCodexAccountStateDb: async () => false, - healCodexAccountSessionIndex: upToDate, - healPendingCodexAccountThreads: upToDate - } -}) +})) beforeEach(() => { vi.resetModules() diff --git a/src/main/codex-accounts/runtime-home/runtime-home-service-state.ts b/src/main/codex-accounts/runtime-home/runtime-home-service-state.ts index 418cb0dfcda..0cee2d50b99 100644 --- a/src/main/codex-accounts/runtime-home/runtime-home-service-state.ts +++ b/src/main/codex-accounts/runtime-home/runtime-home-service-state.ts @@ -63,8 +63,6 @@ export abstract class CodexRuntimeHomeState { account: CodexManagedAccount, unavailableManagedHomePath?: string ): string | null - protected abstract startSelfContainedSessionBridgeForLaunch(perAccountHome: string): void - protected abstract getSelfContainedSessionBridgeSourceHomes(): string[] protected abstract syncSelfContainedManagedSelection(account: CodexManagedAccount): void protected abstract resolveSelfContainedManagedHome( account: CodexManagedAccount @@ -79,10 +77,10 @@ export abstract class CodexRuntimeHomeState { protected abstract invalidateBackfillAfterManagedSystemDefaultLaunch( launchEnv?: NodeJS.ProcessEnv ): boolean | null - protected abstract startWslSessionBridgeForLaunch( + protected abstract prepareWslAccountHistory( target: CodexAccountSelectionTarget, runtimeHomePath: string | null - ): void + ): Promise abstract getHostCodexHomePathsForSessionDiscovery(): string[] abstract getSelectedHostAccountCodexHomePath(): string | null diff --git a/src/main/codex-accounts/runtime-home/wsl/runtime-home-wsl-managed-accounts.test.ts b/src/main/codex-accounts/runtime-home/wsl/runtime-home-wsl-managed-accounts.test.ts index 9177a2b3d55..ee98460f1e2 100644 --- a/src/main/codex-accounts/runtime-home/wsl/runtime-home-wsl-managed-accounts.test.ts +++ b/src/main/codex-accounts/runtime-home/wsl/runtime-home-wsl-managed-accounts.test.ts @@ -2,7 +2,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs' import { join } from 'node:path' import type * as LegacyWslRuntimeAuthDrain from '../../legacy-wsl-auth-drain/legacy-wsl-runtime-auth-drain' -import type * as WslCodexSessionBridge from '../../../codex/history/bridges/wsl-codex-session-bridge' +import type * as WslCodexAccountHistory from '../../../codex/history/shared-store/wsl-codex-account-history' import { createSettings } from '../runtime-home-settings-test-fixtures' import { createCodexAuthJson, @@ -23,10 +23,17 @@ vi.mock('../../legacy-wsl-auth-drain/legacy-wsl-runtime-auth-drain', async (impo ...(await importOriginal()), startLegacyWslRuntimeAuthDrain: async () => {} })) -vi.mock('../../../codex/history/bridges/wsl-codex-session-bridge', async (importOriginal) => ({ - ...(await importOriginal()), - startWslCodexSessionBridgeInBackground: async () => {} -})) +vi.mock( + '../../../codex/history/shared-store/wsl-codex-account-history', + async (importOriginal) => ({ + ...(await importOriginal()), + WslCodexAccountHistoryLaunches: class { + run(): Promise { + return Promise.resolve() + } + } + }) +) vi.mock('electron', () => ({ app: { getPath: () => testState.userDataDir diff --git a/src/main/codex-accounts/runtime-home/wsl/runtime-home-wsl-session-bridge.test.ts b/src/main/codex-accounts/runtime-home/wsl/runtime-home-wsl-session-bridge.test.ts index 7e1016eb55a..6d3bb000357 100644 --- a/src/main/codex-accounts/runtime-home/wsl/runtime-home-wsl-session-bridge.test.ts +++ b/src/main/codex-accounts/runtime-home/wsl/runtime-home-wsl-session-bridge.test.ts @@ -6,7 +6,7 @@ import type * as CodexConfigMirror from '../../../codex/configuration/settings/c import type * as CodexHomePaths from '../../../codex/configuration/homes/codex-home-paths' import type * as LegacyWslRuntimeAuthDrain from '../../legacy-wsl-auth-drain/legacy-wsl-runtime-auth-drain' import type * as WslCodexAuthBatchReader from './wsl-codex-auth-batch-reader' -import type * as WslCodexSessionBridge from '../../../codex/history/bridges/wsl-codex-session-bridge' +import type * as WslCodexAccountHistory from '../../../codex/history/shared-store/wsl-codex-account-history' import { createSettings } from '../runtime-home-settings-test-fixtures' import { createCodexAuthJson, @@ -143,10 +143,14 @@ describe('CodexRuntimeHomeService', () => { ) // Launch-time sibling bridging is covered elsewhere; this pins only the drain's own target. vi.doMock( - '../../../codex/history/bridges/wsl-codex-session-bridge', + '../../../codex/history/shared-store/wsl-codex-account-history', async (importOriginal) => ({ - ...(await importOriginal()), - startWslCodexSessionBridgeInBackground: vi.fn(() => Promise.resolve()) + ...(await importOriginal()), + WslCodexAccountHistoryLaunches: class { + run(): Promise { + return Promise.resolve() + } + } }) ) const retiredHome = join(guestHome, '.local', 'share', 'orca', 'codex-runtime-home', 'home') @@ -219,7 +223,7 @@ describe('CodexRuntimeHomeService', () => { await Promise.all(drainTasks) expect(existsSync(linkedSessionPath)).toBe(false) } finally { - vi.doUnmock('../../../codex/history/bridges/wsl-codex-session-bridge') + vi.doUnmock('../../../codex/history/shared-store/wsl-codex-account-history') vi.doUnmock('../../../codex/configuration/settings/codex-config-mirror') vi.doUnmock('../../../codex/configuration/homes/codex-home-paths') vi.doUnmock('../../legacy-wsl-auth-drain/legacy-wsl-runtime-auth-drain') diff --git a/src/main/codex/configuration/settings/codex-config-mirror.test.ts b/src/main/codex/configuration/settings/codex-config-mirror.test.ts index 6e45a3dc5bc..65274ff2c4d 100644 --- a/src/main/codex/configuration/settings/codex-config-mirror.test.ts +++ b/src/main/codex/configuration/settings/codex-config-mirror.test.ts @@ -77,6 +77,43 @@ afterEach(() => { }) describe('syncSystemConfigIntoManagedCodexHome', () => { + it("keeps a shared account home's sqlite_home and never copies it into ~/.codex/config.toml", () => { + const accountHome = join(userDataDir, 'codex-accounts', 'a', 'home') + mkdirSync(accountHome, { recursive: true }) + const orcaLine = 'sqlite_home = "/store/.codex" # orca: Codex history shared across accounts' + writeFileSync(join(accountHome, 'config.toml'), `${orcaLine}\nmodel = "old"\n`, 'utf-8') + const systemConfig = ['model = "gpt"', '', '[features]', 'hooks = true', ''].join('\n') + writeFileSync(getSystemConfigPath(), systemConfig, 'utf-8') + const homes = { runtimeHomePath: accountHome, systemHomePath: getSystemCodexHomePath() } + + syncSystemConfigIntoManagedCodexHome(homes) + syncSystemConfigIntoManagedCodexHome(homes) + + const accountConfig = readFileSync(join(accountHome, 'config.toml'), 'utf-8') + expect(accountConfig.split('\n')[0]).toBe(orcaLine) + expect(accountConfig).toContain('model = "gpt"') + expect(accountConfig.match(/sqlite_home/g)).toHaveLength(1) + expect(readFileSync(getSystemConfigPath(), 'utf-8')).toBe(systemConfig) + }) + + it("drops Orca's sqlite_home once the user's own config sets one", () => { + const accountHome = join(userDataDir, 'codex-accounts', 'a', 'home') + mkdirSync(accountHome, { recursive: true }) + writeFileSync( + join(accountHome, 'config.toml'), + 'sqlite_home = "/store/.codex" # orca: Codex history shared across accounts\n', + 'utf-8' + ) + writeFileSync(getSystemConfigPath(), 'sqlite_home = "/mine"\n', 'utf-8') + + syncSystemConfigIntoManagedCodexHome({ + runtimeHomePath: accountHome, + systemHomePath: getSystemCodexHomePath() + }) + + expect(readFileSync(join(accountHome, 'config.toml'), 'utf-8')).toBe('sqlite_home = "/mine"\n') + }) + it('preserves an existing runtime config when the system config is blank', () => { // A zero-byte cloud-synced config may be half-written or unhydrated. mkdirSync(join(userDataDir, 'codex-runtime-home', 'home'), { recursive: true }) diff --git a/src/main/codex/configuration/settings/codex-config-mirror.ts b/src/main/codex/configuration/settings/codex-config-mirror.ts index 8c3ab91a440..68d661516a4 100644 --- a/src/main/codex/configuration/settings/codex-config-mirror.ts +++ b/src/main/codex/configuration/settings/codex-config-mirror.ts @@ -22,7 +22,7 @@ import { import { observeCodexSettingsBaseline, readCodexSettingsBaseline } from './config-settings-baseline' import { getCodexConfigSyncStatus, reportCodexConfigSyncOutcome } from './config-sync-stall' import { preserveRuntimeConflictValues } from './codex-config-settings-preservation' -import { applyCodexDaemonSocketGuard } from '../shared-server/codex-daemon-socket-path-guard' +import { applyRuntimeOwnedConfig } from './codex-config-runtime-owned-values' import { stripRuntimeOwnedTomlSections } from '../toml/config-toml-runtime-owned-sections' import { mergeSystemCodexConfigIntoRuntime } from './codex-config-mirror-merge' @@ -111,7 +111,7 @@ export function ensureCodexDaemonSocketGuard(runtimeHomePath: string): void { } function writeCodexDaemonSocketGuard(runtimeHomePath: string, runtimeConfig: string | null): void { - const guarded = applyCodexDaemonSocketGuard(runtimeConfig ?? '', runtimeHomePath) + const guarded = applyRuntimeOwnedConfig(runtimeConfig ?? '', runtimeHomePath, runtimeConfig) if (guarded !== (runtimeConfig ?? '')) { writeFileAtomicallyIfUnchanged( join(runtimeHomePath, 'config.toml'), @@ -166,9 +166,10 @@ export function syncSystemConfigIntoLegacySharedCodexHome( homes.runtimeHomePath ) } - const nextRuntimeConfig = applyCodexDaemonSocketGuard( + const nextRuntimeConfig = applyRuntimeOwnedConfig( mirroredRuntimeConfig, - homes.runtimeHomePath + homes.runtimeHomePath, + runtimeConfigBeforeMirror ) if ((runtimeConfigBeforeMirror ?? '') === nextRuntimeConfig) { return @@ -206,9 +207,10 @@ type CodexConfigMirrorResult = } function syncSystemConfigIntoManagedCodexHomeUnsafe( - { runtimeHomePath, systemHomePath, systemConfigDir }: CodexSettingsPromotionHomes, + homes: CodexSettingsPromotionHomes, promotionPlan: CodexSettingsPromotionPlan ): CodexConfigMirrorResult { + const { runtimeHomePath, systemHomePath, systemConfigDir } = homes const systemConfigPath = join(systemHomePath, 'config.toml') const runtimeConfigPath = join(runtimeHomePath, 'config.toml') // Distinguish unavailable paths from absent ones to avoid replacing an unreadable runtime as fresh. @@ -241,9 +243,10 @@ function syncSystemConfigIntoManagedCodexHomeUnsafe( const sourceConfigDir = resolveCodexConfigMirrorSourceDirectory(systemHomePath, systemConfigDir) if (!runtimeConfigExists) { - const freshRuntimeConfig = applyCodexDaemonSocketGuard( + const freshRuntimeConfig = applyRuntimeOwnedConfig( prepareSystemConfigForFreshRuntimeMirror(rawSystemConfig, sourceConfigDir, runtimeHomePath), - runtimeHomePath + runtimeHomePath, + null ) const ownership = readMcpServerTomlOwnership(freshRuntimeConfig) writeFileAtomically(runtimeConfigPath, freshRuntimeConfig, RUNTIME_CONFIG_WRITE) @@ -274,7 +277,11 @@ function syncSystemConfigIntoManagedCodexHomeUnsafe( ), promotionPlan.runtimeValuesToPreserve ) - const nextRuntimeConfig = applyCodexDaemonSocketGuard(preserved.content, runtimeHomePath) + const nextRuntimeConfig = applyRuntimeOwnedConfig( + preserved.content, + runtimeHomePath, + runtimeConfig + ) if (nextRuntimeConfig !== runtimeConfig) { writeFileAtomically(runtimeConfigPath, nextRuntimeConfig, RUNTIME_CONFIG_WRITE) } diff --git a/src/main/codex/configuration/settings/codex-config-runtime-owned-values.ts b/src/main/codex/configuration/settings/codex-config-runtime-owned-values.ts new file mode 100644 index 00000000000..4343a62c2c2 --- /dev/null +++ b/src/main/codex/configuration/settings/codex-config-runtime-owned-values.ts @@ -0,0 +1,20 @@ +import { applyCodexDaemonSocketGuard } from '../shared-server/codex-daemon-socket-path-guard' +import { + applyCodexSharedSqliteHome, + readCodexSharedSqliteHome +} from '../toml/config-toml-shared-sqlite-home' + +/** + * Values the managed home owns: never mirrored from, or promoted to, the user's own config. + * The shared sqlite_home line belongs to the history link pass; a mirror only carries it forward. + */ +export function applyRuntimeOwnedConfig( + config: string, + runtimeHomePath: string, + previousConfig: string | null +): string { + return applyCodexSharedSqliteHome( + applyCodexDaemonSocketGuard(config, runtimeHomePath), + readCodexSharedSqliteHome(previousConfig ?? '') + ) +} diff --git a/src/main/codex/configuration/toml/config-toml-runtime-owned-sections.ts b/src/main/codex/configuration/toml/config-toml-runtime-owned-sections.ts index 9549b4187d4..2f80c044a39 100644 --- a/src/main/codex/configuration/toml/config-toml-runtime-owned-sections.ts +++ b/src/main/codex/configuration/toml/config-toml-runtime-owned-sections.ts @@ -1,4 +1,5 @@ import { stripCodexDaemonOverride } from '../shared-server/codex-daemon-socket-path-guard' +import { stripCodexSharedSqliteHome } from './config-toml-shared-sqlite-home' import { createTomlLineScanState, getTomlTableHeader, @@ -240,6 +241,8 @@ export function extractOrdinaryCodexSettings(config: string): string { .filter((section) => isRuntimeProjectTomlSection(section)) .map((section) => getTomlSectionHeaderKey(section)) ) - // The daemon override belongs to the managed home and must not be promoted. - return stripCodexDaemonOverride(stripRuntimeOwnedTomlSections(config, projectHeaders)).trimEnd() + // The daemon override and shared sqlite_home belong to the managed home and must not be promoted. + return stripCodexSharedSqliteHome( + stripCodexDaemonOverride(stripRuntimeOwnedTomlSections(config, projectHeaders)) + ).trimEnd() } diff --git a/src/main/codex/configuration/toml/config-toml-shared-sqlite-home.ts b/src/main/codex/configuration/toml/config-toml-shared-sqlite-home.ts new file mode 100644 index 00000000000..60e90b7656d --- /dev/null +++ b/src/main/codex/configuration/toml/config-toml-shared-sqlite-home.ts @@ -0,0 +1,85 @@ +import { + createTomlLineScanState, + getTomlTableHeader, + isTomlStructuralLine, + parseTomlSingleLineStringValue, + updateTomlLineScanState +} from './config-toml-line-scan' + +// An account home's databases live with the shared history store; the line is Orca's, per home. +export const CODEX_SHARED_SQLITE_HOME_MARKER = '# orca: Codex history shared across accounts' + +/** Remove Orca's line so it never reaches the user's own ~/.codex config. */ +export function stripCodexSharedSqliteHome(config: string): string { + if (!config.includes(CODEX_SHARED_SQLITE_HOME_MARKER)) { + return config + } + let scan = createTomlLineScanState() + const kept: string[] = [] + for (const line of config.split('\n')) { + const structural = isTomlStructuralLine(scan) + scan = updateTomlLineScanState(scan, line) + if (structural && line.trimEnd().endsWith(CODEX_SHARED_SQLITE_HOME_MARKER)) { + continue + } + kept.push(line) + } + return kept.join('\n') +} + +/** The store path in Orca's own line, or null when the config has none. */ +export function readCodexSharedSqliteHome(config: string): string | null { + const line = findTopLevelSqliteHomeLine(config) + if (line) { + const match = /^sqlite_home = (".*") # orca: /.exec(line.trimEnd()) + if (match && line.trimEnd().endsWith(CODEX_SHARED_SQLITE_HOME_MARKER)) { + try { + const value: unknown = JSON.parse(match[1]) + return typeof value === 'string' ? value : null + } catch { + return null + } + } + } + return null +} + +/** The top-level sqlite_home line (the user's or Orca's), or null when the config has none. */ +function findTopLevelSqliteHomeLine(config: string): string | null { + let scan = createTomlLineScanState() + for (const line of config.split('\n')) { + const structural = isTomlStructuralLine(scan) + scan = updateTomlLineScanState(scan, line) + if (!structural) { + continue + } + if (getTomlTableHeader(line)) { + return null + } + if (/^\s*(?:sqlite_home|"sqlite_home"|'sqlite_home')\s*=/.test(line)) { + return line + } + } + return null +} + +function hasTopLevelSqliteHome(config: string): boolean { + return findTopLevelSqliteHomeLine(config) !== null +} + +/** Where a home's config puts Codex's databases, from a single-line string value. */ +export function readCodexConfiguredSqliteHome(config: string): string | null { + const line = findTopLevelSqliteHomeLine(config) + return (line && parseTomlSingleLineStringValue(line, line.indexOf('=') + 1)?.value) || null +} + +/** Points Codex's databases at `sqliteHome` unless the user's own config sets one; null removes Orca's line. */ +export function applyCodexSharedSqliteHome(config: string, sqliteHome: string | null): string { + const stripped = stripCodexSharedSqliteHome(config) + if (sqliteHome === null || hasTopLevelSqliteHome(stripped)) { + return stripped + } + const newline = stripped.includes('\r\n') ? '\r\n' : '\n' + // Top-level keys must precede every table. + return `sqlite_home = ${JSON.stringify(sqliteHome)} ${CODEX_SHARED_SQLITE_HOME_MARKER}${newline}${stripped}` +} diff --git a/src/main/codex/history/bridges/codex-account-session-bridge.test.ts b/src/main/codex/history/bridges/codex-account-session-bridge.test.ts deleted file mode 100644 index 8cf741a2598..00000000000 --- a/src/main/codex/history/bridges/codex-account-session-bridge.test.ts +++ /dev/null @@ -1,142 +0,0 @@ -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' -import { tmpdir } from 'node:os' -import { join } from 'node:path' -import { - _internals, - bridgeCodexSessionsIntoAccountHome, - startCodexAccountSessionBridgeInBackground, - stopCodexAccountSessionBridges -} from './codex-account-session-bridge' -import { writeCodexStateDbBackfillStatus } from '../index/codex-state-db-test-fixture' - -let workspaceRoot: string -const healIndexStub = vi.fn(async () => ({ - outcome: 'up-to-date' as const, - healedThreads: 0, - missingThreads: 0, - failedThreads: 0 -})) - -function writeRollout(homePath: string, relativePath: string, contents: string): string { - const filePath = join(homePath, 'sessions', relativePath) - mkdirSync(join(filePath, '..'), { recursive: true }) - writeFileSync(filePath, contents) - return filePath -} - -function rolloutPath(homePath: string, relativePath: string): string { - return join(homePath, 'sessions', relativePath) -} -const THREAD_B = '019a0000-0000-7000-8000-00000000000b' -const ROLLOUT_A = join( - '2026', - '07', - '20', - 'rollout-2026-07-20T10-00-00-019a0000-0000-7000-8000-00000000000a.jsonl' -) -const ROLLOUT_B = join( - '2026', - '07', - '21', - 'rollout-2026-07-21T10-00-00-019a0000-0000-7000-8000-00000000000b.jsonl' -) - -beforeEach(() => { - workspaceRoot = mkdtempSync(join(tmpdir(), 'codex-account-session-bridge-')) - _internals.resetBackgroundBridgeTasks() - healIndexStub.mockClear() -}) - -afterEach(() => { - rmSync(workspaceRoot, { recursive: true, force: true }) -}) - -describe('bridgeCodexSessionsIntoAccountHome', () => { - it('does not hand a rollout it failed to link to the index heal', async () => { - const systemHome = join(workspaceRoot, 'system') - const targetHome = join(workspaceRoot, 'account') - writeRollout(systemHome, ROLLOUT_A, 'session\n') - writeRollout(systemHome, ROLLOUT_B, 'session\n') - // A file where the target's day directory belongs makes that one link fail. - mkdirSync(join(targetHome, 'sessions', '2026', '07'), { recursive: true }) - writeFileSync(join(targetHome, 'sessions', '2026', '07', '20'), 'not a directory') - const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) - - const summary = await bridgeCodexSessionsIntoAccountHome({ - targetCodexHomePath: targetHome, - sourceCodexHomePaths: [systemHome] - }) - - warn.mockRestore() - expect(summary.linkedFiles).toBe(1) - expect(summary.bridgedThreads).toEqual(new Map([[THREAD_B, '2026-07-21T10-00-00']])) - }) -}) - -type BridgeDependencies = NonNullable< - Parameters[1] -> - -// Why: the real steps spawn `codex app-server`; their behavior is covered separately. -function startBridge( - targetCodexHomePath: string, - sourceCodexHomePaths: string[], - dependencies: BridgeDependencies = {} -): Promise { - return startCodexAccountSessionBridgeInBackground( - { targetCodexHomePath, sourceCodexHomePaths }, - { createStateDb: async () => false, healIndex: healIndexStub, ...dependencies } - ) -} - -describe('startCodexAccountSessionBridgeInBackground', () => { - // Older CLIs and sqlite_home can leave the home without a DB across launches. - - it('skips linking while Codex is still indexing the home', async () => { - const systemHome = join(workspaceRoot, 'system') - const targetHome = join(workspaceRoot, 'account') - writeRollout(systemHome, ROLLOUT_A, 'session\n') - writeCodexStateDbBackfillStatus(targetHome, 'running') - - await startBridge(targetHome, [systemHome]) - - expect(existsSync(rolloutPath(targetHome, ROLLOUT_A))).toBe(false) - expect(healIndexStub).not.toHaveBeenCalled() - }) - - it('reports why it skips a home whose state DB cannot be read', async () => { - const systemHome = join(workspaceRoot, 'system') - const targetHome = join(workspaceRoot, 'account') - writeRollout(systemHome, ROLLOUT_A, 'session\n') - mkdirSync(targetHome, { recursive: true }) - writeFileSync(join(targetHome, 'state_5.sqlite'), 'not a sqlite database') - const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) - - await startBridge(targetHome, [systemHome]) - - expect(existsSync(rolloutPath(targetHome, ROLLOUT_A))).toBe(false) - expect(warn).toHaveBeenCalledWith( - expect.stringContaining('Codex state DB is unreadable'), - expect.any(String) - ) - warn.mockRestore() - }) - - it('links nothing when the app quits while Codex creates the state DB', async () => { - const systemHome = join(workspaceRoot, 'system') - const targetHome = join(workspaceRoot, 'account') - writeRollout(systemHome, ROLLOUT_A, 'session\n') - - await startBridge(targetHome, [systemHome], { - createStateDb: async (home) => { - writeCodexStateDbBackfillStatus(home, 'complete') - stopCodexAccountSessionBridges() - return true - } - }) - - expect(existsSync(rolloutPath(targetHome, ROLLOUT_A))).toBe(false) - expect(healIndexStub).not.toHaveBeenCalled() - }) -}) diff --git a/src/main/codex/history/bridges/codex-account-session-bridge.ts b/src/main/codex/history/bridges/codex-account-session-bridge.ts deleted file mode 100644 index bdb90d8da52..00000000000 --- a/src/main/codex/history/bridges/codex-account-session-bridge.ts +++ /dev/null @@ -1,141 +0,0 @@ -import { join } from 'node:path' -import { normalizeRuntimePathForComparison } from '../../../../shared/file-access/cross-platform-path' -import { - createCodexAccountStateDb, - healCodexAccountSessionIndex -} from '../index/codex-account-session-index-heal' -import { - linkMissingCodexSessionRollouts, - type CodexSessionLinkPass -} from '../rollouts/codex-session-link-pass' -import { - countCodexSessionFilesUpTo, - findNewestCodexStateDbPath, - readCodexStateDbBackfillStatus -} from '../index/codex-state-db' - -// Hardlink history across homes so account switching preserves /resume entries and one physical log. -export type CodexAccountSessionBridgeSummary = CodexSessionLinkPass - -const backgroundBridgeTasksByTargetHome = new Map>() -let stopping = false - -type BackgroundBridgeDependencies = { - createStateDb: typeof createCodexAccountStateDb - healIndex: typeof healCodexAccountSessionIndex -} - -const defaultBackgroundBridgeDependencies: BackgroundBridgeDependencies = { - createStateDb: createCodexAccountStateDb, - healIndex: healCodexAccountSessionIndex -} - -// Create the DB before linking history to avoid first-launch backfill timeouts; heal indexes later. -async function isReadyForBridgedHistory( - targetCodexHomePath: string, - sourceCodexHomePaths: readonly string[], - dependencies: BackgroundBridgeDependencies -): Promise { - if ( - !findNewestCodexStateDbPath(targetCodexHomePath) && - !hasSessionRollouts(targetCodexHomePath) - ) { - if ( - !sourceCodexHomePaths.some(hasSessionRollouts) || - !(await dependencies.createStateDb(targetCodexHomePath)) - ) { - return false - } - } - const status = readCodexStateDbBackfillStatus(targetCodexHomePath) - if (status.kind === 'unreadable') { - // Why: contention clears by the next launch; corruption would skip every launch, so surface it. - console.warn( - '[codex-account-session-bridge] Skipping history bridge; Codex state DB is unreadable:', - status.error - ) - return false - } - // Existing history without a local DB implies sqlite_home or an older CLI; linking is safe. - // `not-tracked` is a pre-backfill schema that Codex migrates to `pending`. - return status.kind === 'complete' || status.kind === 'missing' -} - -function hasSessionRollouts(codexHomePath: string): boolean { - return countCodexSessionFilesUpTo(join(codexHomePath, 'sessions'), 1) > 0 -} - -export function startCodexAccountSessionBridgeInBackground( - args: { - targetCodexHomePath: string - sourceCodexHomePaths: readonly string[] - /** Skip rather than symlink when a hardlink fails (cross-volume): for the user's own ~/.codex. */ - hardlinkOnly?: boolean - }, - dependenciesOverride: Partial = {} -): Promise { - if (stopping) { - return Promise.resolve() - } - const key = normalizeRuntimePathForComparison(args.targetCodexHomePath) - const inFlight = backgroundBridgeTasksByTargetHome.get(key) - if (inFlight) { - return inFlight - } - const dependencies = { ...defaultBackgroundBridgeDependencies, ...dependenciesOverride } - const task = isReadyForBridgedHistory( - args.targetCodexHomePath, - args.sourceCodexHomePaths, - dependencies - ) - .then(async (ready) => { - // Why: skip rather than feed a running backfill; the next launch retries. - if (!ready || stopping) { - return - } - const summary = await bridgeCodexSessionsIntoAccountHome(args) - await dependencies.healIndex(args.targetCodexHomePath, summary.bridgedThreads, { - shouldStop: () => stopping - }) - }) - .catch((error: unknown) => { - console.warn('[codex-account-session-bridge] Background session bridge failed:', error) - }) - .then(() => undefined) - backgroundBridgeTasksByTargetHome.set(key, task) - void task.finally(() => { - if (backgroundBridgeTasksByTargetHome.get(key) === task) { - backgroundBridgeTasksByTargetHome.delete(key) - } - }) - return task -} - -/** Stops background bridges at quit; links and index reads made so far are kept. */ -export function stopCodexAccountSessionBridges(): void { - stopping = true -} - -export function isCodexAccountSessionBridgeStopping(): boolean { - return stopping -} - -export async function bridgeCodexSessionsIntoAccountHome(args: { - targetCodexHomePath: string - sourceCodexHomePaths: readonly string[] - hardlinkOnly?: boolean -}): Promise { - return linkMissingCodexSessionRollouts({ - sourceCodexHomePaths: args.sourceCodexHomePaths, - targetCodexHomePath: args.targetCodexHomePath, - hardlinkOnly: args.hardlinkOnly, - shouldStop: () => stopping - }) -} - -export const _internals = { - resetBackgroundBridgeTasks: (): void => { - backgroundBridgeTasksByTargetHome.clear() - stopping = false - } -} diff --git a/src/main/codex/history/bridges/wsl-codex-session-bridge.test.ts b/src/main/codex/history/bridges/wsl-codex-session-bridge.test.ts deleted file mode 100644 index d307c1adacc..00000000000 --- a/src/main/codex/history/bridges/wsl-codex-session-bridge.test.ts +++ /dev/null @@ -1,129 +0,0 @@ -import { beforeEach, describe, expect, it, vi } from 'vitest' -import { execFileSync } from 'node:child_process' -import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' -import { tmpdir } from 'node:os' -import { join } from 'node:path' - -const { runWslProcessMock } = vi.hoisted(() => ({ - runWslProcessMock: vi.fn() -})) - -// The E9 running check has its own cases; these cover the chain once the distro runs. -vi.mock('../../../wsl/filesystem/wsl-unc-access-gate', () => ({ - assertWslUncPathRunning: async () => {} -})) -vi.mock('../../../wsl/execution/wsl-runner', () => ({ - runWslProcess: runWslProcessMock -})) -import { _internals } from './wsl-codex-session-bridge' -import { buildWslCodexSessionBridgeShellCommand } from './wsl-codex-session-bridge-script' - -beforeEach(() => { - runWslProcessMock.mockReset() - _internals.reset() -}) - -describe('buildWslCodexSessionBridgeShellCommand', () => { - it.skipIf(process.platform === 'win32')( - 'publishes a verified guest-side copy across filesystems', - () => { - const root = mkdtempSync(join(tmpdir(), 'orca-wsl-session-bridge-cross-fs-')) - const sourceSessionsRoot = join(root, 'legacy', 'sessions') - const managedSessionsRoot = join(root, 'managed', 'sessions') - const binDir = join(root, 'bin') - const relativePath = join('2026', '08', '26', 'retired.jsonl') - const sourcePath = join(sourceSessionsRoot, relativePath) - const targetPath = join(managedSessionsRoot, relativePath) - mkdirSync(join(sourcePath, '..'), { recursive: true }) - mkdirSync(binDir) - writeFileSync(sourcePath, '{"session":"retired"}\n', 'utf-8') - const lnShimPath = join(binDir, 'ln') - writeFileSync( - lnShimPath, - `#!/bin/sh -if [ "$2" = "$BRIDGE_SOURCE" ] && [ "$3" = "$BRIDGE_TARGET" ]; then - exit 1 -fi -exec /bin/ln "$@" -` - ) - chmodSync(lnShimPath, 0o755) - const shellCommand = buildWslCodexSessionBridgeShellCommand({ - systemSessionsRoot: sourceSessionsRoot, - managedSessionsRoot - }) - - try { - execFileSync('/bin/bash', ['-c', shellCommand], { - env: { - ...process.env, - BRIDGE_SOURCE: sourcePath, - BRIDGE_TARGET: targetPath, - PATH: `${binDir}:${process.env.PATH ?? ''}` - } - }) - expect(readFileSync(targetPath, 'utf-8')).toBe('{"session":"retired"}\n') - } finally { - rmSync(root, { recursive: true, force: true }) - } - } - ) - - it.skipIf(process.platform === 'win32')( - 'leaves an existing copied session to its current writer', - () => { - const root = mkdtempSync(join(tmpdir(), 'orca-wsl-session-bridge-existing-')) - const sourceSessionsRoot = join(root, 'legacy', 'sessions') - const managedSessionsRoot = join(root, 'managed', 'sessions') - const relativePath = join('2026', '08', '26', 'retired.jsonl') - const sourcePath = join(sourceSessionsRoot, relativePath) - const targetPath = join(managedSessionsRoot, relativePath) - mkdirSync(join(sourcePath, '..'), { recursive: true }) - mkdirSync(join(targetPath, '..'), { recursive: true }) - writeFileSync(sourcePath, '{"session":"retired"}\n{"event":"legacy"}\n', 'utf-8') - writeFileSync(targetPath, '{"session":"retired"}\n', 'utf-8') - const shellCommand = buildWslCodexSessionBridgeShellCommand({ - systemSessionsRoot: sourceSessionsRoot, - managedSessionsRoot - }) - - try { - execFileSync('/bin/bash', ['-c', shellCommand]) - expect(readFileSync(targetPath, 'utf-8')).toBe('{"session":"retired"}\n') - } finally { - rmSync(root, { recursive: true, force: true }) - } - } - ) - - it.skipIf(process.platform === 'win32')( - 'fails in the guest shell when a missing session cannot be linked, then retries cleanly', - () => { - const root = mkdtempSync(join(tmpdir(), 'orca-wsl-session-bridge-')) - const sourceSessionsRoot = join(root, 'legacy', 'sessions') - const managedSessionsRoot = join(root, 'managed', 'sessions') - const relativePath = join('2026', '08', '26', 'retired.jsonl') - const sourcePath = join(sourceSessionsRoot, relativePath) - const blockingPath = join(managedSessionsRoot, '2026') - mkdirSync(join(sourcePath, '..'), { recursive: true }) - mkdirSync(managedSessionsRoot, { recursive: true }) - writeFileSync(sourcePath, '{"session":"retired"}\n', 'utf-8') - writeFileSync(blockingPath, 'not-a-directory\n', 'utf-8') - const shellCommand = buildWslCodexSessionBridgeShellCommand({ - systemSessionsRoot: sourceSessionsRoot, - managedSessionsRoot - }) - - try { - expect(() => execFileSync('/bin/bash', ['-c', shellCommand])).toThrow() - rmSync(blockingPath) - execFileSync('/bin/bash', ['-c', shellCommand]) - expect(readFileSync(join(managedSessionsRoot, relativePath), 'utf-8')).toBe( - '{"session":"retired"}\n' - ) - } finally { - rmSync(root, { recursive: true, force: true }) - } - } - ) -}) diff --git a/src/main/codex/history/bridges/wsl-codex-session-bridge.ts b/src/main/codex/history/bridges/wsl-codex-session-bridge.ts deleted file mode 100644 index 1f746843b56..00000000000 --- a/src/main/codex/history/bridges/wsl-codex-session-bridge.ts +++ /dev/null @@ -1,357 +0,0 @@ -import { posix as pathPosix } from 'node:path' -import { normalizeRuntimePathForComparison } from '../../../../shared/file-access/cross-platform-path' -import { parseWslUncPath } from '../../../../shared/file-access/wsl-paths' -import { runWslProcess } from '../../../wsl/execution/wsl-runner' -import { assertWslUncPathRunning } from '../../../wsl/filesystem/wsl-unc-access-gate' -import type { onRunningWslDistrosStopped } from '../../../wsl/discovery/wsl-running-distro-cache' -import { onWslDistroGateNote } from '../../../wsl/discovery/wsl-background-distro-gate' -import { isCodexAccountSessionBridgeStopping } from './codex-account-session-bridge' -import { runCodexAppServerSession } from '../../app-server/codex-app-server-session' -import { - createCodexAccountStateDb, - healPendingCodexAccountThreads -} from '../index/codex-account-session-index-heal' -import { - buildWslHealInvocation, - type CodexSettledThreadRead -} from '../index/codex-session-index-heal' -import { parseCodexRolloutThreadId } from '../index/codex-session-index-heal-state' -import { - buildWslCodexSessionBridgeShellCommand, - WSL_SESSION_BRIDGE_TIMEOUT_MS -} from './wsl-codex-session-bridge-script' - -export type WslCodexSessionBridgeTarget = { - distro: string - systemCodexHomePath: string - managedCodexHomePath: string -} - -export type WslCodexSessionBridgeLinuxPaths = { - systemSessionsRoot: string - managedSessionsRoot: string - managedHomePath: string - indexPendingRoot: string -} - -export type WslCodexSessionBridgeSummary = { - scannedFiles: number - linkedFiles: number - /** Marker names in `indexPendingRoot`: rollout file names Codex has yet to index. */ - pendingRollouts: string[] - /** Some rollout could not be linked; the rest of the summary is still accurate. */ - bridgeFailed: boolean -} - -// Why: Windows cannot take SQLite's locks over \\wsl.localhost, so the host -// cannot diff bridged threads against the guest's Codex index the way the -// native bridge does. A marker per linked rollout records the work instead, -// and dies when `thread/read` settles that thread. -const WSL_INDEX_PENDING_DIR_NAME = '.orca-index-pending' - -// Why: app-server answers `initialize` only once Codex's startup backfill is -// complete; on a home an older Orca filled before Codex ever opened it, that -// is the minutes-long backfill itself. Codex's own 30s gate bounds the case -// where another process holds the backfill. -const WSL_STATE_DB_OPEN_TIMEOUT_MS = 15 * 60_000 - -// Why: marker names are ~75 bytes, so this reports ~900k pending rollouts -// before the summary line could be truncated away. -const WSL_SESSION_BRIDGE_MAX_OUTPUT_BYTES = 64 * 1024 * 1024 - -type WslBridgeDependencies = { - openStateDb: typeof createCodexAccountStateDb - healPending: typeof healPendingCodexAccountThreads - /** Throws unless the distro runs; each step outlives the launch that started the chain (E9). */ - requireRunning: (distro: string) => Promise - subscribeStops: typeof onRunningWslDistrosStopped -} - -const defaultWslBridgeDependencies: WslBridgeDependencies = { - openStateDb: createCodexAccountStateDb, - healPending: healPendingCodexAccountThreads, - requireRunning: (distro) => assertWslUncPathRunning('', distro), - // Gate notes carry running-list stops and explicit ones (a relay or watcher exit) no list saw yet. - subscribeStops: (listener) => - onWslDistroGateNote((note) => { - if (note.kind === 'stopped') { - listener([note.distro], []) - } - }) -} - -const emptySummary: WslCodexSessionBridgeSummary = { - scannedFiles: 0, - linkedFiles: 0, - pendingRollouts: [], - bridgeFailed: false -} -const backgroundWslSessionBridgeTasks = new Map>() -// A complete backfill stays complete, so each home needs the startup probe once per run. -const homesWithCompleteIndex = new Set() - -export function startWslCodexSessionBridgeInBackground( - target: WslCodexSessionBridgeTarget, - dependenciesOverride: Partial = {} -): Promise { - if (isCodexAccountSessionBridgeStopping()) { - return Promise.resolve() - } - const taskKey = getWslSessionBridgeTaskKey(target) - const existingTask = backgroundWslSessionBridgeTasks.get(taskKey) - if (existingTask) { - return existingTask - } - - const task = bridgeAndIndexWslCodexSessions(target, { - ...defaultWslBridgeDependencies, - ...dependenciesOverride - }) - .catch((error: unknown) => { - console.warn('[codex-session-bridge] Background WSL session bridge failed:', error) - }) - .then(() => undefined) - backgroundWslSessionBridgeTasks.set(taskKey, task) - void task.finally(() => { - if (backgroundWslSessionBridgeTasks.get(taskKey) === task) { - backgroundWslSessionBridgeTasks.delete(taskKey) - } - }) - return task -} - -/** - * The WSL twin of the native account bridge (#22971): history lands only in a - * home whose Codex index is complete, then Codex indexes it newest first. - */ -async function bridgeAndIndexWslCodexSessions( - target: WslCodexSessionBridgeTarget, - dependencies: WslBridgeDependencies -): Promise { - const paths = resolveWslCodexSessionBridgeLinuxPaths(target) - if (!paths) { - return - } - const buildInvocation = (_codexHomePath: string, timeoutMs: number) => - buildWslHealInvocation(target.distro, paths.managedHomePath, timeoutMs) - const homeKey = normalizeRuntimePathForComparison(target.managedCodexHomePath) - // Every wsl.exe step below re-checks: the user may stop the distro mid-chain, and a step would boot it. - const requireRunning = (): Promise => dependencies.requireRunning(target.distro) - if (!homesWithCompleteIndex.has(homeKey)) { - await requireRunning() - // Why: linking before Codex's first open makes that open index everything, - // and every other Codex on the home gives up after 30s (#20669). - const complete = await dependencies.openStateDb(target.managedCodexHomePath, { - buildInvocation, - timeoutMs: WSL_STATE_DB_OPEN_TIMEOUT_MS - }) - if (!complete) { - return - } - homesWithCompleteIndex.add(homeKey) - } - if (isCodexAccountSessionBridgeStopping()) { - return - } - await requireRunning() - const summary = await syncWslCodexSessionsIntoManagedHome(target) - const pendingThreads = new Map() - const markerByThreadId = new Map() - const markersWithoutThread: string[] = [] - for (const rolloutName of summary.pendingRollouts) { - const rollout = parseCodexRolloutThreadId(rolloutName) - if (rollout) { - pendingThreads.set(rollout.threadId, rollout.rolloutStamp) - markerByThreadId.set(rollout.threadId, rolloutName) - } else { - markersWithoutThread.push(rolloutName) - } - } - if (markersWithoutThread.length > 0) { - // Why: no thread id means thread/read can never settle it, so it would be reported forever. - await requireRunning() - await clearIndexPendingMarkers(target.distro, paths, markersWithoutThread).catch( - (error: unknown) => { - console.warn('[codex-session-bridge] Could not clear WSL index markers:', error) - } - ) - } - if (pendingThreads.size > 0 && !isCodexAccountSessionBridgeStopping()) { - await requireRunning() - // Batches spawn synchronously from here on; a confirmed stop ends the heal before the next one. - const distroKey = target.distro.toLowerCase() - let distroStopped = false - const unsubscribe = dependencies.subscribeStops((stopped) => { - distroStopped ||= stopped.some((distro) => distro.toLowerCase() === distroKey) - }) - try { - await dependencies.healPending(target.managedCodexHomePath, pendingThreads, { - buildInvocation, - // Each batch spawns a guest codex; the inter-batch delay leaves time for the user to stop it. - runSession: async (invocation, body) => { - await requireRunning().catch((error: unknown) => { - distroStopped = true - throw error - }) - return runCodexAppServerSession(invocation, body) - }, - shouldStop: () => distroStopped || isCodexAccountSessionBridgeStopping(), - afterBatch: async (settled) => { - // Also the check before the next batch: a refused read ends the heal there. - await requireRunning().catch((error: unknown) => { - distroStopped = true - throw error - }) - await clearSettledIndexPendingMarkers(target.distro, paths, settled, markerByThreadId) - } - }) - } finally { - unsubscribe() - } - } - if (summary.bridgeFailed) { - throw new Error(`WSL codex session bridge could not link every session for ${target.distro}`) - } -} - -/** Drops the markers of threads Codex indexed or no longer has; a failed read stays pending. */ -async function clearSettledIndexPendingMarkers( - distro: string, - paths: WslCodexSessionBridgeLinuxPaths, - settled: readonly CodexSettledThreadRead[], - markerByThreadId: ReadonlyMap -): Promise { - await clearIndexPendingMarkers( - distro, - paths, - settled - .filter(({ outcome }) => outcome !== 'failed') - .flatMap(({ threadId }) => markerByThreadId.get(threadId) ?? []) - ) -} - -async function clearIndexPendingMarkers( - distro: string, - paths: WslCodexSessionBridgeLinuxPaths, - markers: readonly string[] -): Promise { - if (markers.length === 0) { - return - } - const result = await runWslProcess({ - distro, - loginPath: 'none', - script: 'cd -- "$1" || exit 1\nshift\nrm -f -- "$@"', - args: [paths.indexPendingRoot, ...markers], - timeoutMs: WSL_SESSION_BRIDGE_TIMEOUT_MS - }) - if (result.code !== 0 || result.timedOut) { - throw new Error(`Could not clear WSL Codex index markers (code ${result.code})`) - } -} - -export async function syncWslCodexSessionsIntoManagedHome( - target: WslCodexSessionBridgeTarget -): Promise { - const paths = resolveWslCodexSessionBridgeLinuxPaths(target) - if (!paths) { - return emptySummary - } - - const result = await runWslProcess({ - distro: target.distro, - loginPath: 'none', - script: buildWslCodexSessionBridgeShellCommand(paths), - // The script uses Bash [[ ... ]] and read -d, which dash rejects. - shell: 'bash', - timeoutMs: WSL_SESSION_BRIDGE_TIMEOUT_MS, - maxOutputBytes: WSL_SESSION_BRIDGE_MAX_OUTPUT_BYTES - }) - // Why: the summary is the script's last act, and a link failure still prints it so one bad - // file cannot strand every marker; any other failure exits before it. - const summary = result.timedOut ? null : parseWslSessionBridgeSummary(result.stdout) - if (!summary) { - throw Object.assign( - new Error(`WSL codex session bridge failed for ${target.distro} (code ${result.code})`), - { code: result.code, stderr: result.stderr, timedOut: result.timedOut } - ) - } - return { ...summary, bridgeFailed: result.code !== 0 } -} - -export function resolveWslCodexSessionBridgeLinuxPaths( - target: WslCodexSessionBridgeTarget -): WslCodexSessionBridgeLinuxPaths | null { - const systemHomePath = getLinuxPathForWslDistro(target.systemCodexHomePath, target.distro) - const managedHomePath = getLinuxPathForWslDistro(target.managedCodexHomePath, target.distro) - if (!systemHomePath || !managedHomePath) { - return null - } - - return { - systemSessionsRoot: joinLinuxPath(systemHomePath, 'sessions'), - managedSessionsRoot: joinLinuxPath(managedHomePath, 'sessions'), - managedHomePath, - indexPendingRoot: joinLinuxPath(managedHomePath, WSL_INDEX_PENDING_DIR_NAME) - } -} - -function getWslSessionBridgeTaskKey(target: WslCodexSessionBridgeTarget): string { - return [target.distro, target.systemCodexHomePath, target.managedCodexHomePath].join('\0') -} - -function getLinuxPathForWslDistro(path: string, distro: string): string | null { - const wslPath = parseWslUncPath(path) - if (wslPath) { - return wslDistroNamesMatch(wslPath.distro, distro) ? wslPath.linuxPath : null - } - return path.startsWith('/') ? path : null -} - -function wslDistroNamesMatch(left: string, right: string): boolean { - return left.toLowerCase() === right.toLowerCase() -} - -function joinLinuxPath(basePath: string, ...segments: string[]): string { - return pathPosix.join(basePath, ...segments) -} - -function parseWslSessionBridgeSummary(stdout: string): WslCodexSessionBridgeSummary | null { - // Why: login/profile scripts may write stdout before the bridge output. - const lines = stdout - .split(/\r?\n/) - .map((line) => line.trim()) - .filter((line) => line.length > 0) - let parsed: unknown - try { - parsed = JSON.parse(lines.at(-1) ?? '') - } catch { - return null - } - if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) { - return null - } - const summary = parsed as Record - if (typeof summary.scannedFiles !== 'number' || typeof summary.linkedFiles !== 'number') { - return null - } - // Why: count every listed marker, not just Codex-shaped names, so one odd `rollout-*.jsonl` - // cannot fail the count on every launch and strand the home's whole index heal. - const markerLines = lines.slice(0, -1).filter((line) => line.startsWith('rollout-')) - // Why: a clipped or noisy list would index the wrong set; its markers wait for the next launch. - const pendingComplete = - typeof summary.pendingFiles === 'number' && summary.pendingFiles === markerLines.length - return { - scannedFiles: summary.scannedFiles, - linkedFiles: summary.linkedFiles, - pendingRollouts: pendingComplete ? markerLines : [], - bridgeFailed: false - } -} - -export const _internals = { - reset: (): void => { - backgroundWslSessionBridgeTasks.clear() - homesWithCompleteIndex.clear() - } -} diff --git a/src/main/codex/history/index/codex-account-session-index-heal.ts b/src/main/codex/history/index/codex-account-session-index-heal.ts index 19d9fd4a1c5..d178b450f73 100644 --- a/src/main/codex/history/index/codex-account-session-index-heal.ts +++ b/src/main/codex/history/index/codex-account-session-index-heal.ts @@ -1,75 +1,21 @@ import { normalizeRuntimePathForComparison } from '../../../../shared/file-access/cross-platform-path' import { - isCodexAppServerUnsupportedError, - runCodexAppServerSession -} from '../../app-server/codex-app-server-session' -import { - buildNativeHealInvocation, readCodexThreadsForIndexHeal, type CodexSessionIndexHealOptions, type CodexThreadReadPassOutcome } from './codex-session-index-heal' -import { readIndexedCodexThreadIds } from './codex-state-db' -// thread/read lazily indexes bridged history so /resume's indexed-only "All" view can show it. +// thread/read indexes history moved into a store so /resume's indexed-only "All" view can show it. export type CodexAccountSessionIndexHealSummary = { - outcome: CodexThreadReadPassOutcome | 'up-to-date' | 'no-index' + outcome: CodexThreadReadPassOutcome | 'up-to-date' healedThreads: number missingThreads: number failedThreads: number } -export type CodexAccountSessionIndexHealOptions = CodexSessionIndexHealOptions & { - readIndexedThreadIds?: (codexHomePath: string) => Set | null -} - -// app-server finishes state DB startup before answering initialize. -const STATE_DB_CREATE_TIMEOUT_MS = 15_000 - // Cache failed reads until restart to avoid respawning app-server for the same refused/missing thread. const failedThreadIdsByHome = new Map>() -/** Starts Codex so the state DB exists and startup backfill is complete (app-server answers initialize only after both); false if Codex could not start. */ -export async function createCodexAccountStateDb( - codexHomePath: string, - options: Pick & { - timeoutMs?: number - } = {} -): Promise { - const buildInvocation = options.buildInvocation ?? buildNativeHealInvocation - const runSession = options.runSession ?? runCodexAppServerSession - const timeoutMs = options.timeoutMs ?? STATE_DB_CREATE_TIMEOUT_MS - try { - await runSession(buildInvocation(codexHomePath, timeoutMs), async () => {}) - return true - } catch (error) { - // Why: a Codex without app-server predates the state DB, so linking cannot stall it. - if (isCodexAppServerUnsupportedError(error)) { - return true - } - console.warn('[codex-account-session-index-heal] Failed to create Codex state DB:', error) - return false - } -} - -/** Diff against the state DB each pass so interrupted indexing resumes on the next launch. */ -export async function healCodexAccountSessionIndex( - codexHomePath: string, - bridgedThreads: ReadonlyMap, - options: CodexAccountSessionIndexHealOptions = {} -): Promise { - if (bridgedThreads.size === 0) { - return upToDateSummary() - } - // No local DB means an older CLI or sqlite_home owns indexing elsewhere. - const indexed = (options.readIndexedThreadIds ?? readIndexedCodexThreadIds)(codexHomePath) - if (!indexed) { - return { ...upToDateSummary(), outcome: 'no-index' } - } - const unindexed = new Map([...bridgedThreads].filter(([threadId]) => !indexed.has(threadId))) - return healPendingCodexAccountThreads(codexHomePath, unindexed, options) -} - /** * Indexes `pendingThreads` (thread id -> rollout timestamp), newest rollout * first, skipping threads that already failed in this home since Orca started. diff --git a/src/main/codex/history/index/codex-session-index-heal-state.ts b/src/main/codex/history/index/codex-session-index-heal-state.ts index fd0635c1e9a..008deeb78af 100644 --- a/src/main/codex/history/index/codex-session-index-heal-state.ts +++ b/src/main/codex/history/index/codex-session-index-heal-state.ts @@ -6,6 +6,7 @@ import { } from '../../../../shared/file-access/cross-platform-path' import { writeFileAtomically } from '../../../filesystem/atomic-file-publication' import { streamCodexSessionLedgerRecords } from '../rollouts/codex-session-ledger-stream' +import { parseCodexRolloutThreadId } from '../rollouts/codex-rollout-thread-id' // Bump after semantics changes to retry outcomes recorded under older versions. export const CODEX_SESSION_INDEX_HEAL_VERSION = 3 @@ -14,9 +15,6 @@ export const CODEX_SESSION_INDEX_HEAL_VERSION = 3 const HEAL_UNSUPPORTED_RETRY_INTERVAL_MS = 24 * 60 * 60 * 1000 const HEAL_FAILED_THREAD_RETRY_INTERVAL_MS = 24 * 60 * 60 * 1000 -const CODEX_ROLLOUT_THREAD_ID_PATTERN = - /^rollout-(.+)-([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})\.jsonl(?:\.zst)?$/i - export type CodexSessionIndexHealPaths = { auditLogPath: string systemSessionsRoot: string @@ -82,17 +80,6 @@ export async function collectPendingHealThreads( ) } -export function parseCodexRolloutThreadId( - filePath: string -): { threadId: string; rolloutStamp: string } | null { - const match = CODEX_ROLLOUT_THREAD_ID_PATTERN.exec(lastPathSegment(filePath)) - return match ? { threadId: match[2].toLowerCase(), rolloutStamp: match[1] } : null -} - -function lastPathSegment(filePath: string): string { - return filePath.split(/[\\/]/).at(-1) ?? '' -} - async function readProcessedHealThreads(paths: CodexSessionIndexHealPaths): Promise<{ healedAuditRecords: Set legacyHealedThreadIds: Set diff --git a/src/main/codex/history/index/codex-state-db.test.ts b/src/main/codex/history/index/codex-state-db.test.ts index d80729ab7dd..2bf3824b4d2 100644 --- a/src/main/codex/history/index/codex-state-db.test.ts +++ b/src/main/codex/history/index/codex-state-db.test.ts @@ -1,14 +1,13 @@ import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' -import { afterEach, describe, expect, it, vi } from 'vitest' +import { afterEach, describe, expect, it } from 'vitest' import SyncDatabase from '../../../sqlite/sync-database' import { findNewestCodexStateDbPath, isCodexStateDbBackfillPending, readCodexStateDbBackfillPendingState, - readCodexStateDbBackfillStatus, - readIndexedCodexThreadIds + readCodexStateDbBackfillStatus } from './codex-state-db' const temporaryHomes: string[] = [] @@ -80,27 +79,3 @@ describe('readCodexStateDbBackfillPendingState', () => { expect(isCodexStateDbBackfillPending(home)).toBe(false) }) }) - -describe('readIndexedCodexThreadIds', () => { - it('returns lower-cased thread ids from the newest state DB', async () => { - const home = await createHome() - const path = createBackfillDb(home, 5, 'complete') - const db = new SyncDatabase(path) - db.exec( - "CREATE TABLE threads (id TEXT PRIMARY KEY); INSERT INTO threads (id) VALUES ('ABC'), ('def')" - ) - db.close() - - expect(readIndexedCodexThreadIds(home)).toEqual(new Set(['abc', 'def'])) - }) - - it('returns null when there is no state DB or no threads table', async () => { - const home = await createHome() - expect(readIndexedCodexThreadIds(home)).toBeNull() - - createBackfillDb(home, 5, 'complete') - const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) - expect(readIndexedCodexThreadIds(home)).toBeNull() - warn.mockRestore() - }) -}) diff --git a/src/main/codex/history/index/codex-state-db.ts b/src/main/codex/history/index/codex-state-db.ts index ec42b74ad29..213c3035ec1 100644 --- a/src/main/codex/history/index/codex-state-db.ts +++ b/src/main/codex/history/index/codex-state-db.ts @@ -1,6 +1,9 @@ import { readdirSync } from 'node:fs' import { join } from 'node:path' import SyncDatabase from '../../../sqlite/sync-database' +import { parseWslUncPath, toWindowsWslPath } from '../../../../shared/file-access/wsl-paths' +import { readAgentStateFileSync } from '../../configuration/agent-state-file-reader' +import { readCodexConfiguredSqliteHome } from '../../configuration/toml/config-toml-shared-sqlite-home' const STATE_DB_FILE_PATTERN = /^state_(\d+)\.sqlite$/ @@ -32,8 +35,27 @@ export function findNewestCodexStateDbPath(codexHomePath: string): string | null return newest ? join(codexHomePath, newest.name) : null } +// Codex's own order: the home's config, then CODEX_SQLITE_HOME, then the home itself. A shared +// account home keeps its databases in the history store. +function resolveCodexSqliteHome(codexHomePath: string): string { + let config = '' + try { + config = readAgentStateFileSync(join(codexHomePath, 'config.toml')) + } catch { + // No readable config: Codex uses its defaults too. + } + const configured = readCodexConfiguredSqliteHome(config) ?? process.env.CODEX_SQLITE_HOME?.trim() + if (!configured) { + return codexHomePath + } + const wslHome = parseWslUncPath(codexHomePath) + return wslHome && configured.startsWith('/') + ? toWindowsWslPath(configured, wslHome.distro) + : configured +} + export function readCodexStateDbBackfillStatus(codexHomePath: string): CodexStateDbBackfillStatus { - const stateDbPath = findNewestCodexStateDbPath(codexHomePath) + const stateDbPath = findNewestCodexStateDbPath(resolveCodexSqliteHome(codexHomePath)) if (!stateDbPath) { return { kind: 'missing' } } @@ -70,34 +92,6 @@ export function readCodexStateDbBackfillStatus(codexHomePath: string): CodexStat } } -/** Null means the database is absent or unreadable. */ -export function readIndexedCodexThreadIds(codexHomePath: string): Set | null { - const stateDbPath = findNewestCodexStateDbPath(codexHomePath) - if (!stateDbPath) { - return null - } - let db: SyncDatabase | null = null - try { - db = new SyncDatabase(stateDbPath, { readonly: true, fileMustExist: true }) - const ids = new Set() - for (const row of db.prepare('SELECT id FROM threads').all()) { - if (typeof row.id === 'string') { - ids.add(row.id.toLowerCase()) - } - } - return ids - } catch (error) { - console.warn('[codex-state-db] Failed to read indexed Codex threads:', error) - return null - } finally { - try { - db?.close() - } catch { - // A close failure cannot change the read-only result already collected. - } - } -} - export function countCodexSessionFilesUpTo(sessionsRoot: string, limit: number): number { let count = 0 const pendingDirectories = [sessionsRoot] diff --git a/src/main/codex/history/resume/codex-session-resume-launch-preparation.ts b/src/main/codex/history/resume/codex-session-resume-launch-preparation.ts index 60a0ca26236..98ba14ba8b0 100644 --- a/src/main/codex/history/resume/codex-session-resume-launch-preparation.ts +++ b/src/main/codex/history/resume/codex-session-resume-launch-preparation.ts @@ -48,6 +48,7 @@ export function createCodexSessionResumeLaunchPreparation(deps: { console.warn('[codex-hook-service] failed to prepare automatic resume home:', error) } if (!isSystemHome) { + await deps.getRuntimeHome()?.prepareAccountHistoryForPinnedHome(home) // Hook repair may skip or fail before its config mirror applies the guard. ensureCodexDaemonSocketGuard(home) } diff --git a/src/main/codex/history/rollouts/codex-rollout-compressed-twin.ts b/src/main/codex/history/rollouts/codex-rollout-compressed-twin.ts new file mode 100644 index 00000000000..9c8584dbecf --- /dev/null +++ b/src/main/codex/history/rollouts/codex-rollout-compressed-twin.ts @@ -0,0 +1,88 @@ +import type { BigIntStats, Dirent } from 'node:fs' +import { readdir } from 'node:fs/promises' +import { join } from 'node:path' +import { isDefinitiveAbsence } from '../../../../shared/errors/definitive-filesystem-absence' +import { lstatIfPresent } from '../../../filesystem/merged-directory-link' +import { parseCodexRolloutThreadId } from './codex-rollout-thread-id' + +export const COMPRESSED_ROLLOUT_SUFFIX = '.zst' + +/** The rollout's other form: `X.jsonl` for `X.jsonl.zst` and the reverse. */ +export function rolloutTwinPath(path: string): string { + return path.endsWith(COMPRESSED_ROLLOUT_SUFFIX) + ? path.slice(0, -COMPRESSED_ROLLOUT_SUFFIX.length) + : `${path}${COMPRESSED_ROLLOUT_SUFFIX}` +} + +// Codex compresses a cold rollout keeping its mtime and appends to the `.jsonl` when both exist, +// so a `.jsonl` no newer than a non-empty `.zst` holds nothing the `.zst` lacks. +function plainWins(plain: BigIntStats, compressed: BigIntStats): boolean { + return compressed.size === 0n || plain.mtimeNs > compressed.mtimeNs +} + +function isFile(stats: BigIntStats | undefined): stats is BigIntStats { + return stats?.isFile() ?? false +} + +/** Sorts `.zst` names first, so a folder's compressed form reaches the store before its twin. */ +export function compressedRolloutsFirst(left: string, right: string): number { + return ( + Number(right.endsWith(COMPRESSED_ROLLOUT_SUFFIX)) - + Number(left.endsWith(COMPRESSED_ROLLOUT_SUFFIX)) + ) +} + +/** + * Whether the rollout at `from` stays out of the store at `to` because the store already holds + * its other form, so the store keeps one form: a `.jsonl` joins only when newer than the store's + * `.zst` (its appended lines are missing there). Deciding against the store, never a sibling still + * waiting to move, means a form is set aside only once its twin is safely in the store. + */ +export async function isRolloutTwinKeptFromStore(from: string, to: string): Promise { + if (!parseCodexRolloutThreadId(from)) { + return false + } + const [self, storeTwin] = await Promise.all([ + lstatIfPresent(from), + lstatIfPresent(rolloutTwinPath(to)) + ]) + if (!isFile(self) || !isFile(storeTwin)) { + return false + } + return from.endsWith(COMPRESSED_ROLLOUT_SUFFIX) || !plainWins(self, storeTwin) +} + +/** Plain-form paths of the rollouts under `folder` present in both forms, at most `limit`. */ +export async function listRolloutTwinPairs(folder: string, limit = Infinity): Promise { + let entries: Dirent[] + try { + entries = await readdir(folder, { withFileTypes: true }) + } catch (error) { + if (isDefinitiveAbsence(error)) { + return [] + } + throw error + } + const names = new Set(entries.map((entry) => entry.name)) + const pairs: string[] = [] + for (const entry of entries) { + if (pairs.length >= limit) { + break + } + if (entry.isDirectory()) { + pairs.push(...(await listRolloutTwinPairs(join(folder, entry.name), limit - pairs.length))) + } else if ( + entry.name.endsWith(COMPRESSED_ROLLOUT_SUFFIX) && + names.has(rolloutTwinPath(entry.name)) && + parseCodexRolloutThreadId(entry.name) + ) { + pairs.push(join(folder, rolloutTwinPath(entry.name))) + } + } + return pairs +} + +/** Whether a folder holds both forms of a rollout, so it must merge file by file. */ +export async function holdsRolloutTwins(folder: string): Promise { + return (await listRolloutTwinPairs(folder, 1)).length > 0 +} diff --git a/src/main/codex/history/rollouts/codex-rollout-thread-id.ts b/src/main/codex/history/rollouts/codex-rollout-thread-id.ts new file mode 100644 index 00000000000..0d1d932027f --- /dev/null +++ b/src/main/codex/history/rollouts/codex-rollout-thread-id.ts @@ -0,0 +1,14 @@ +// A reverted thread's rollout adds `_` after the thread id. +const CODEX_ROLLOUT_THREAD_ID_PATTERN = + /^rollout-(.+)-([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})(?:_[0-9a-f-]{36})?\.jsonl(?:\.zst)?$/i + +export function parseCodexRolloutThreadId( + filePath: string +): { threadId: string; rolloutStamp: string } | null { + const match = CODEX_ROLLOUT_THREAD_ID_PATTERN.exec(lastPathSegment(filePath)) + return match ? { threadId: match[2].toLowerCase(), rolloutStamp: match[1] } : null +} + +function lastPathSegment(filePath: string): string { + return filePath.split(/[\\/]/).at(-1) ?? '' +} diff --git a/src/main/codex/history/rollouts/codex-session-link-pass.ts b/src/main/codex/history/rollouts/codex-session-link-pass.ts deleted file mode 100644 index 56f6d22726e..00000000000 --- a/src/main/codex/history/rollouts/codex-session-link-pass.ts +++ /dev/null @@ -1,299 +0,0 @@ -import type { Dirent } from 'node:fs' -import { lstat, mkdir, readdir, rmdir, stat } from 'node:fs/promises' -import { dirname, join } from 'node:path' -import { setImmediate as yieldToEventLoop } from 'node:timers/promises' -import { normalizeRuntimePathForComparison } from '../../../../shared/file-access/cross-platform-path' -import { isDefinitiveAbsence } from '../../../../shared/errors/definitive-filesystem-absence' -import { parseCodexRolloutThreadId } from '../index/codex-session-index-heal-state' -import { isCodexSessionRolloutFileName } from './codex-session-file-listing' -import { linkCodexSessionFileAsync } from './codex-session-link' - -export type CodexSessionLinkPass = { - scannedFiles: number - linkedFiles: number - /** Rollouts left unlinked because only a hardlink was allowed and none could be made. */ - skippedFiles: number - /** Thread ID to rollout timestamp for history present in the target home. */ - bridgedThreads: Map -} - -// Up to this many rollouts, per-path checks cost less than listing a wide target directory. -const SPARSE_TARGET_LOOKUP_LIMIT = 4 -const ENTRIES_PER_YIELD = 256 - -/** - * Links rollouts missing from the target by diffing directory listings off the main thread, so - * history the target already has costs no per-rollout synchronous call. Sources are walked - * together so each target directory is examined once; earlier sources are tried first. - */ -export async function linkMissingCodexSessionRollouts(args: { - sourceCodexHomePaths: readonly string[] - targetCodexHomePath: string - /** Skip rather than symlink when a hardlink fails (cross-volume): for the user's own ~/.codex. */ - hardlinkOnly?: boolean - shouldStop?: () => boolean -}): Promise { - const pass: CodexSessionLinkPass = { - scannedFiles: 0, - linkedFiles: 0, - skippedFiles: 0, - bridgedThreads: new Map() - } - const sourceRoots = dedupeSourceHomes(args.sourceCodexHomePaths, args.targetCodexHomePath).map( - (home) => join(home, 'sessions') - ) - if (sourceRoots.length > 0) { - await new RolloutLinkWalk( - sourceRoots, - join(args.targetCodexHomePath, 'sessions'), - args.hardlinkOnly === true, - args.shouldStop ?? (() => false), - pass - ).visit([]) - } - return pass -} - -type TargetEntry = 'present' | 'dangling' - -class RolloutLinkWalk { - private entriesSinceYield = 0 - - constructor( - private readonly sourceRoots: readonly string[], - private readonly targetRoot: string, - private readonly hardlinkOnly: boolean, - private readonly shouldStop: () => boolean, - private readonly pass: CodexSessionLinkPass - ) {} - - async visit(relativeParts: readonly string[]): Promise { - if (this.shouldStop()) { - return - } - // Rollout name to every source directory holding it, in caller priority order. - const rollouts = new Map() - const subdirectories = new Set() - for (const sourceRoot of this.sourceRoots) { - const sourceDir = join(sourceRoot, ...relativeParts) - for (const entry of (await readSourceDirectory(sourceDir)) ?? []) { - if (entry.isDirectory()) { - subdirectories.add(entry.name) - } else if (entry.isFile() && isCodexSessionRolloutFileName(entry.name)) { - const holders = rollouts.get(entry.name) - if (holders) { - holders.push(sourceDir) - } else { - rollouts.set(entry.name, [sourceDir]) - } - } - await this.yieldPeriodically() - } - } - if (rollouts.size > 0) { - await this.linkDirectory(join(this.targetRoot, ...relativeParts), rollouts) - } - for (const name of subdirectories) { - await this.visit([...relativeParts, name]) - } - } - - private async linkDirectory( - targetDir: string, - rollouts: ReadonlyMap - ): Promise { - const targetEntries = await this.readTargetEntries(targetDir, rollouts) - if (!targetEntries) { - this.pass.scannedFiles += rollouts.size - return - } - let createdDirs: { firstCreated: string | undefined } | null = null - let linkedHere = 0 - let examined = 0 - for (const [name, sourceDirs] of rollouts) { - if (this.shouldStop()) { - break - } - await this.yieldPeriodically() - examined += 1 - this.pass.scannedFiles += 1 - const targetEntry = targetEntries.get(name) - // A dangling link blocks the name, and resume cannot read it; leave it out of healing. - if (targetEntry === 'dangling') { - continue - } - let outcome: 'linked' | 'existing' | 'skipped' | 'failed' = 'existing' - // Rollout names carry the session UUID, so an existing name is this conversation. - if (!targetEntry) { - if (!createdDirs) { - try { - createdDirs = { firstCreated: await mkdir(targetDir, { recursive: true }) } - } catch (error) { - console.warn('[codex-session-link] Failed to create session directory:', error) - // The rest of this directory cannot be linked either. - this.pass.scannedFiles += rollouts.size - examined - return - } - } - outcome = await this.linkFromFirstUsableSource(name, sourceDirs, targetDir) - } - if (outcome === 'skipped') { - this.pass.skippedFiles += 1 - continue - } - if (outcome === 'failed') { - continue - } - if (outcome === 'linked') { - linkedHere += 1 - this.pass.linkedFiles += 1 - } - const rollout = parseCodexRolloutThreadId(name) - if (rollout) { - this.pass.bridgedThreads.set(rollout.threadId, rollout.rolloutStamp) - } - } - if (linkedHere === 0 && createdDirs?.firstCreated) { - // A skipped link leaves no folders behind. - await removeEmptyCreatedDirs(targetDir, createdDirs.firstCreated) - } - } - - // Each source holding the name gets its own attempt, as a per-source pass would give it. - private async linkFromFirstUsableSource( - name: string, - sourceDirs: readonly string[], - targetDir: string - ): Promise<'linked' | 'existing' | 'skipped' | 'failed'> { - let outcome: 'linked' | 'existing' | 'skipped' | 'failed' = 'failed' - for (const sourceDir of sourceDirs) { - outcome = await linkCodexSessionFileAsync(join(sourceDir, name), join(targetDir, name), { - hardlinkOnly: this.hardlinkOnly - }) - if (outcome === 'existing' && !(await isTargetPresent(join(targetDir, name)))) { - // A dangling link another writer made after the listing is not readable history. - return 'failed' - } - if (outcome === 'linked' || outcome === 'existing') { - return outcome - } - } - return outcome - } - - /** Target entries named like a source rollout; null when the target cannot be read. */ - private async readTargetEntries( - targetDir: string, - rollouts: ReadonlyMap - ): Promise | null> { - const entries = new Map() - try { - if (rollouts.size <= SPARSE_TARGET_LOOKUP_LIMIT) { - for (const name of rollouts.keys()) { - const entry = await readTargetEntry(join(targetDir, name)) - if (entry) { - entries.set(name, entry) - } - } - return entries - } - for (const dirent of await readdir(targetDir, { withFileTypes: true })) { - if (rollouts.has(dirent.name)) { - entries.set( - dirent.name, - dirent.isSymbolicLink() - ? await resolveSymlinkEntry(join(targetDir, dirent.name)) - : 'present' - ) - } - await this.yieldPeriodically() - } - return entries - } catch (error) { - if (isDefinitiveAbsence(error)) { - return entries - } - console.warn('[codex-session-link] Failed to read target sessions:', error) - return null - } - } - - // Wide directories must not hold the main thread while names are matched. - private async yieldPeriodically(): Promise { - this.entriesSinceYield += 1 - if (this.entriesSinceYield >= ENTRIES_PER_YIELD) { - this.entriesSinceYield = 0 - await yieldToEventLoop() - } - } -} - -async function readSourceDirectory(sourceDir: string): Promise { - try { - return await readdir(sourceDir, { withFileTypes: true }) - } catch (error) { - // A source without this directory is normal; other sources may still fill it. - if (!isDefinitiveAbsence(error)) { - console.warn('[codex-session-link] Failed to list Codex sessions:', error) - } - return null - } -} - -async function readTargetEntry(targetPath: string): Promise { - try { - const entry = await lstat(targetPath) - return entry.isSymbolicLink() ? await resolveSymlinkEntry(targetPath) : 'present' - } catch (error) { - if (isDefinitiveAbsence(error)) { - return null - } - throw error - } -} - -async function isTargetPresent(targetPath: string): Promise { - try { - return (await readTargetEntry(targetPath)) === 'present' - } catch { - return false - } -} - -async function resolveSymlinkEntry(targetPath: string): Promise { - try { - await stat(targetPath) - return 'present' - } catch { - return 'dangling' - } -} - -async function removeEmptyCreatedDirs(deepest: string, firstCreated: string): Promise { - for (let dir = deepest; ; dir = dirname(dir)) { - try { - // Non-recursive: a directory something else wrote into in the meantime stays. - await rmdir(dir) - } catch { - return - } - if (dir === firstCreated) { - return - } - } -} - -function dedupeSourceHomes( - sourceCodexHomePaths: readonly string[], - targetCodexHomePath: string -): string[] { - const seen = new Set([normalizeRuntimePathForComparison(targetCodexHomePath)]) - return sourceCodexHomePaths.filter((sourceHomePath) => { - const key = normalizeRuntimePathForComparison(sourceHomePath) - if (seen.has(key)) { - return false - } - seen.add(key) - return true - }) -} diff --git a/src/main/codex/history/shared-store/codex-account-history-cleanup-manifest.ts b/src/main/codex/history/shared-store/codex-account-history-cleanup-manifest.ts new file mode 100644 index 00000000000..301faa732a7 --- /dev/null +++ b/src/main/codex/history/shared-store/codex-account-history-cleanup-manifest.ts @@ -0,0 +1,351 @@ +import { randomUUID } from 'node:crypto' +import type { BigIntStats } from 'node:fs' +import { readFile, readdir, unlink, writeFile } from 'node:fs/promises' +import { join, relative } from 'node:path' +import { isDefinitiveAbsence } from '../../../../shared/errors/definitive-filesystem-absence' +import { renameFileWithWindowsRetryAsync } from '../../../../shared/file-write/windows-retry-file-operations' +import { withFileTransactionLock } from '../../../filesystem/file-transaction-lock' +import { MERGE_PENDING_SUFFIX, lstatIfPresent } from '../../../filesystem/merged-directory-link' +import { listRolloutTwinPairs, rolloutTwinPath } from '../rollouts/codex-rollout-compressed-twin' +import { parseCodexRolloutThreadId } from '../rollouts/codex-rollout-thread-id' + +/** + * In each account home: the old data sharing left behind, for a later release to delete once the + * store has proven itself. This release deletes none of it. + */ +export const HISTORY_CLEANUP_MANIFEST = '.orca-history-cleanup.json' +/** One form of a rollout whose other form the store holds, kept out so it holds one. */ +export const SET_ASIDE_ROLLOUTS_DIR = '.orca-set-aside-rollouts' +/** The account's own databases from before sharing stopped; kept, never read again. */ +export const PRE_SHARE_DATABASES_DIR = '.orca-pre-share-db' +export const CODEX_DATABASE_FILE = /\.sqlite(?:-wal|-shm)?$/ + +const MANIFEST_VERSION = 1 +const HISTORY_FOLDERS = ['sessions', 'archived_sessions'] + +/** `path` is relative to its home with `/` separators. */ +type FileFacts = { path: string; bytes: number; mtimeMs: number } + +export type ContentCheck = { + at: string + /** Byte-equal once either form is decompressed. */ + sameContent: boolean + sha256: string + /** The store file as it was checked; a change since voids the check. */ + store: FileFacts +} + +export type LeftBehindRollout = FileFacts & { + kind: 'set-aside-twin' | 'merge-conflict' + /** The store file holding this content: the other form of a twin, else the same name. */ + storePath: string | null + check: ContentCheck | null +} + +type CodexHistoryCleanupManifest = { + version: typeof MANIFEST_VERSION + storeHome: string + /** The first and latest fully shared pass (about one per Orca run) since sharing was off. */ + firstSharedAt: string | null + lastSharedAt: string | null + sharedPasses: number + /** Databases Codex no longer opens here: sqlite_home points at the store. */ + databases: FileFacts[] + rollouts: LeftBehindRollout[] + /** Store rollouts holding both forms, left there (relative to the store, `.jsonl` form). */ + storeTwinPairs: string[] + updatedAt: string +} + +type CodexHistoryShareOutcome = 'shared' | 'incomplete' | 'unshared' + +function toManifestPath(root: string, path: string): string { + return relative(root, path).split(/[\\/]/).join('/') +} + +export function fromManifestPath(root: string, path: string): string { + return join(root, ...path.split('/')) +} + +export function factsOf(root: string, path: string, stats: BigIntStats): FileFacts { + return { + path: toManifestPath(root, path), + bytes: Number(stats.size), + mtimeMs: Number(stats.mtimeNs / 1_000_000n) + } +} + +async function readEntries(path: string) { + try { + return await readdir(path, { withFileTypes: true }) + } catch (error) { + if (isDefinitiveAbsence(error)) { + return [] + } + throw error + } +} + +async function* regularFiles(folder: string): AsyncGenerator<{ path: string; stats: BigIntStats }> { + for (const entry of await readEntries(folder)) { + const path = join(folder, entry.name) + if (entry.isDirectory()) { + yield* regularFiles(path) + } else if (entry.isFile()) { + const stats = await lstatIfPresent(path) + if (stats?.isFile()) { + yield { path, stats } + } + } + } +} + +export async function storeFileFacts(storeHome: string, path: string): Promise { + const stats = await lstatIfPresent(fromManifestPath(storeHome, path)) + return stats?.isFile() ? factsOf(storeHome, fromManifestPath(storeHome, path), stats) : null +} + +/** A newer version's manifest is left alone; anything unreadable or malformed is rebuilt. */ +export async function readManifest( + accountHome: string +): Promise { + let parsed: unknown + try { + parsed = JSON.parse(await readFile(join(accountHome, HISTORY_CLEANUP_MANIFEST), 'utf8')) + } catch (error) { + if (error instanceof SyntaxError || isDefinitiveAbsence(error)) { + return null + } + throw error + } + if (!parsed || typeof parsed !== 'object' || !('version' in parsed)) { + return null + } + if (typeof parsed.version === 'number' && parsed.version > MANIFEST_VERSION) { + return 'newer' + } + return isCurrentManifest(parsed) ? parsed : null +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null +} + +function isFileFacts(value: unknown): boolean { + return ( + isRecord(value) && + typeof value.path === 'string' && + typeof value.bytes === 'number' && + typeof value.mtimeMs === 'number' + ) +} + +// Every field a later pass or the cleanup release reads; anything else is rebuilt from disk. +function isLeftBehindRollout(value: unknown): boolean { + return ( + isFileFacts(value) && + isRecord(value) && + (value.storePath === null || typeof value.storePath === 'string') && + (value.check === null || (isRecord(value.check) && isFileFacts(value.check.store))) + ) +} + +function isCurrentManifest(value: unknown): value is CodexHistoryCleanupManifest { + return ( + isRecord(value) && + value.version === MANIFEST_VERSION && + typeof value.sharedPasses === 'number' && + Array.isArray(value.rollouts) && + value.rollouts.every(isLeftBehindRollout) && + Array.isArray(value.databases) && + value.databases.every(isFileFacts) && + Array.isArray(value.storeTwinPairs) + ) +} + +async function writeManifest( + accountHome: string, + manifest: CodexHistoryCleanupManifest +): Promise { + const target = join(accountHome, HISTORY_CLEANUP_MANIFEST) + const staged = `${target}.${randomUUID()}.tmp` + await writeFile(staged, `${JSON.stringify(manifest, null, 2)}\n`) + try { + await renameFileWithWindowsRetryAsync(staged, target) + } catch (error) { + await unlink(staged).catch(() => {}) + throw error + } +} + +// One read-modify-write at a time per home (a pass and its background check). +const manifestUpdates = new Map>() + +export function updateManifest( + accountHome: string, + update: (current: CodexHistoryCleanupManifest | null) => Promise +): Promise { + // The file lock covers other processes too: each WSL pass and check is its own guest helper. + const run = (manifestUpdates.get(accountHome) ?? Promise.resolve()).then(() => + withFileTransactionLock(join(accountHome, HISTORY_CLEANUP_MANIFEST), async () => { + const current = await readManifest(accountHome) + if (current !== 'newer') { + await writeManifest(accountHome, await update(current)) + } + }) + ) + const settled = run.catch(() => {}) + manifestUpdates.set(accountHome, settled) + void settled.then(() => { + if (manifestUpdates.get(accountHome) === settled) { + manifestUpdates.delete(accountHome) + } + }) + return run +} + +export function sameFacts(left: FileFacts, right: FileFacts | null): boolean { + return right !== null && left.bytes === right.bytes && left.mtimeMs === right.mtimeMs +} + +async function leftBehindRollouts( + accountHome: string, + storeHome: string, + previous: Map +): Promise { + // A pending folder is `[.].orca-profile-merge`; its files conflict by name. + const sources: { root: string; folder: string; kind: LeftBehindRollout['kind'] }[] = [] + for (const folder of await readEntries(join(accountHome, SET_ASIDE_ROLLOUTS_DIR))) { + if (folder.isDirectory()) { + sources.push({ + root: join(accountHome, SET_ASIDE_ROLLOUTS_DIR, folder.name), + folder: folder.name, + kind: 'set-aside-twin' + }) + } + } + for (const entry of await readEntries(accountHome)) { + if (entry.isDirectory() && entry.name.endsWith(MERGE_PENDING_SUFFIX)) { + const folder = entry.name.slice(0, -MERGE_PENDING_SUFFIX.length).replace(/\.\d+$/, '') + sources.push({ root: join(accountHome, entry.name), folder, kind: 'merge-conflict' }) + } + } + const rollouts: LeftBehindRollout[] = [] + for (const source of sources) { + for await (const file of regularFiles(source.root)) { + const inFolder = `${source.folder}/${toManifestPath(source.root, file.path)}` + // A set-aside form matches the store's other form; a conflict usually its same name. + const [first, second] = + source.kind === 'set-aside-twin' + ? [rolloutTwinPath(inFolder), inFolder] + : [inFolder, rolloutTwinPath(inFolder)] + const store = + (await storeFileFacts(storeHome, first)) ?? + (parseCodexRolloutThreadId(inFolder) ? await storeFileFacts(storeHome, second) : null) + const facts = factsOf(accountHome, file.path, file.stats) + const prior = previous.get(facts.path) + const keepsCheck = + prior?.check && + sameFacts(prior, facts) && + prior.storePath === (store?.path ?? null) && + sameFacts(prior.check.store, store) + rollouts.push({ + ...facts, + kind: source.kind, + storePath: store?.path ?? null, + check: keepsCheck ? prior.check : null + }) + } + } + return rollouts +} + +async function databasesLeftBehind( + accountHome: string, + includeHomeRoot: boolean +): Promise { + const databases: FileFacts[] = [] + if (includeHomeRoot) { + for (const entry of await readEntries(accountHome)) { + const stats = CODEX_DATABASE_FILE.test(entry.name) + ? await lstatIfPresent(join(accountHome, entry.name)) + : undefined + if (stats?.isFile()) { + databases.push(factsOf(accountHome, join(accountHome, entry.name), stats)) + } + } + } + for await (const file of regularFiles(join(accountHome, PRE_SHARE_DATABASES_DIR))) { + databases.push(factsOf(accountHome, file.path, file.stats)) + } + return databases +} + +/** Plain-form paths of store rollouts present in both forms; names only, no file reads. */ +async function storeTwinPairs(storeHome: string): Promise { + const pairs: string[] = [] + for (const folder of HISTORY_FOLDERS) { + pairs.push(...(await listRolloutTwinPairs(join(storeHome, folder)))) + } + return pairs.map((path) => toManifestPath(storeHome, path)).sort() +} + +/** + * After a pass: the shared streak (a fully shared pass extends it, turning sharing off resets it) + * and the databases left behind. Only cheap reads, since a launch awaits the pass; the rollout + * inventory is rebuilt by `inventoryCodexHistoryLeftBehind` off that path. + */ +export async function recordCodexHistoryLeftBehind(args: { + accountHome: string + storeHome: string + outcome: CodexHistoryShareOutcome + /** Whether sqlite_home now points the home's Codex at the store's databases. */ + databasesMovedToStore: boolean +}): Promise { + await updateManifest(args.accountHome, async (current) => { + const now = new Date().toISOString() + const sharedNow = args.outcome === 'shared' + const streak = args.outcome !== 'unshared' && current?.storeHome === args.storeHome + return { + version: MANIFEST_VERSION, + storeHome: args.storeHome, + firstSharedAt: (streak ? current?.firstSharedAt : null) ?? (sharedNow ? now : null), + lastSharedAt: sharedNow ? now : streak ? (current?.lastSharedAt ?? null) : null, + sharedPasses: (streak ? (current?.sharedPasses ?? 0) : 0) + (sharedNow ? 1 : 0), + databases: await databasesLeftBehind( + args.accountHome, + args.outcome !== 'unshared' && args.databasesMovedToStore + ), + rollouts: current?.rollouts ?? [], + storeTwinPairs: current?.storeTwinPairs ?? [], + updatedAt: now + } + }) +} + +/** Rebuilds the left-behind rollouts and store twin pairs from disk, keeping still-valid checks. */ +export async function inventoryCodexHistoryLeftBehind(args: { + accountHome: string + storeHome: string +}): Promise { + const before = await readManifest(args.accountHome) + if (!before || before === 'newer') { + return + } + // Walked outside the write, which then touches only these fields: a pass (or, in WSL, another + // process) may rewrite the streak meanwhile, and a launch never waits behind the walk. + const previous = new Map(before.rollouts.map((rollout) => [rollout.path, rollout])) + const rollouts = await leftBehindRollouts(args.accountHome, args.storeHome, previous) + const pairs = await storeTwinPairs(args.storeHome) + if (pairs.length > 0) { + console.warn( + `[codex-account-history] ${pairs.length} rollouts are in ${args.storeHome} as both .jsonl and .jsonl.zst; both kept` + ) + } + await updateManifest(args.accountHome, async (current) => ({ + ...(current ?? before), + rollouts, + storeTwinPairs: pairs, + updatedAt: new Date().toISOString() + })) +} diff --git a/src/main/codex/history/shared-store/codex-account-history-content-check.ts b/src/main/codex/history/shared-store/codex-account-history-content-check.ts new file mode 100644 index 00000000000..79cb5d1f2a7 --- /dev/null +++ b/src/main/codex/history/shared-store/codex-account-history-content-check.ts @@ -0,0 +1,137 @@ +import { createHash } from 'node:crypto' +import { createReadStream } from 'node:fs' +import { pipeline } from 'node:stream/promises' +import { createZstdDecompress } from 'node:zlib' +import { lstatIfPresent } from '../../../filesystem/merged-directory-link' +import { COMPRESSED_ROLLOUT_SUFFIX } from '../rollouts/codex-rollout-compressed-twin' +import { + factsOf, + inventoryCodexHistoryLeftBehind, + fromManifestPath, + readManifest, + sameFacts, + storeFileFacts, + updateManifest, + type ContentCheck, + type LeftBehindRollout +} from './codex-account-history-cleanup-manifest' + +async function contentSha256(path: string): Promise { + const hash = createHash('sha256') + const raw = createReadStream(path, { highWaterMark: 1024 * 1024 }) + const digest = async (content: AsyncIterable): Promise => { + for await (const chunk of content) { + hash.update(chunk) + } + } + await (path.endsWith(COMPRESSED_ROLLOUT_SUFFIX) + ? pipeline(raw, createZstdDecompress(), digest) + : pipeline(raw, digest)) + return hash.digest('hex') +} + +async function checkContent( + accountHome: string, + storeHome: string, + rollout: LeftBehindRollout +): Promise { + if (!rollout.storePath) { + return null + } + const store = await storeFileFacts(storeHome, rollout.storePath) + if (!store) { + return null + } + let sha256: string + let storeSha256: string + try { + ;[sha256, storeSha256] = await Promise.all([ + contentSha256(fromManifestPath(accountHome, rollout.path)), + contentSha256(fromManifestPath(storeHome, rollout.storePath)) + ]) + } catch (error) { + // A corrupt `.zst` never matches; anything else (a moved file) is checked on a later pass. + const code = (error as NodeJS.ErrnoException | null)?.code ?? '' + if (!code.startsWith('ZSTD_')) { + return null + } + return { at: new Date().toISOString(), sameContent: false, sha256: '', store } + } + const home = await lstatIfPresent(fromManifestPath(accountHome, rollout.path)) + const homeFacts = home?.isFile() + ? factsOf(accountHome, fromManifestPath(accountHome, rollout.path), home) + : null + if ( + !sameFacts(rollout, homeFacts) || + !sameFacts(store, await storeFileFacts(storeHome, store.path)) + ) { + return null + } + return { at: new Date().toISOString(), sameContent: sha256 === storeSha256, sha256, store } +} + +// Checkpoints rewrite the whole manifest, so they are spaced by time, not by count. +const SAVE_INTERVAL_MS = 30_000 + +/** + * Rebuilds the home's left-behind inventory, then compares each unchecked rollout with its store + * copy, saving progress as it goes. False when `shouldStop` cut it short. + */ +export async function checkCodexHistoryLeftBehind(args: { + accountHome: string + storeHome: string + shouldStop: () => boolean +}): Promise { + if (!(await readManifest(args.accountHome))) { + return true + } + await inventoryCodexHistoryLeftBehind(args) + const manifest = await readManifest(args.accountHome) + if (!manifest || manifest === 'newer') { + return true + } + const checks = new Map() + const save = async (): Promise => { + const pending = new Map(checks) + checks.clear() + await updateManifest(args.accountHome, async (current) => { + const base = current ?? manifest + return { + ...base, + rollouts: base.rollouts.map((rollout) => { + const checked = pending.get(rollout.path) + // A pass meanwhile may have seen either file change; such a check is stale. + return checked && + !rollout.check && + sameFacts(rollout, checked.rollout) && + rollout.storePath === checked.check.store.path + ? { ...rollout, check: checked.check } + : rollout + }) + } + }) + } + let finished = true + let savedAt = Date.now() + for (const rollout of manifest.rollouts) { + if (rollout.check) { + continue + } + if (args.shouldStop()) { + finished = false + break + } + const check = await checkContent(args.accountHome, args.storeHome, rollout) + if (check) { + checks.set(rollout.path, { rollout, check }) + } + if (checks.size > 0 && Date.now() - savedAt >= SAVE_INTERVAL_MS) { + await save() + savedAt = Date.now() + } + } + if (checks.size > 0) { + await save() + } + return finished +} diff --git a/src/main/codex/history/shared-store/codex-account-history-index.ts b/src/main/codex/history/shared-store/codex-account-history-index.ts new file mode 100644 index 00000000000..748c2d73c8c --- /dev/null +++ b/src/main/codex/history/shared-store/codex-account-history-index.ts @@ -0,0 +1,152 @@ +import { unlink } from 'node:fs/promises' +import { join } from 'node:path' +import { normalizeRuntimePathForComparison } from '../../../../shared/file-access/cross-platform-path' +import { isDefinitiveAbsence } from '../../../../shared/errors/definitive-filesystem-absence' +import { healPendingCodexAccountThreads } from '../index/codex-account-session-index-heal' +import type { CodexSessionIndexHealOptions } from '../index/codex-session-index-heal' +import { parseCodexRolloutThreadId } from '../rollouts/codex-rollout-thread-id' +import { listCodexIndexPendingRollouts } from './codex-account-history-links' + +let stopping = false +const hostHealsByPendingDir = new Map>() +const rerunRequested = new Set() +// Stores whose last heal stopped or failed with work left; a later launch retries them. +const incompleteHeals = new Set() + +export function hasIncompleteHostCodexIndexHeal(pendingDir: string): boolean { + return incompleteHeals.has(normalizeRuntimePathForComparison(pendingDir)) +} + +/** Stops background index work at quit; settled rows and markers are kept. */ +export function stopCodexAccountHistoryWork(): void { + stopping = true +} + +export function isCodexAccountHistoryWorkStopping(): boolean { + return stopping +} + +/** + * Codex's own `thread/read` against the store indexes each moved rollout, newest first, so + * the store's /resume list shows it; a marker dies once its read settles (a failure retries + * on a later launch). + */ +export async function healCodexIndexPendingRollouts(args: { + storeHome: string + rolloutNames: readonly string[] + clear: (rolloutNames: readonly string[]) => Promise + /** True once a history pass started after this heal did: its markers may name unmoved files. */ + passStartedSince?: () => boolean + options?: CodexSessionIndexHealOptions +}): Promise { + const pending = new Map() + // A thread may own several rollouts (a .zst twin, a revert); one read settles them all. + const markersByThreadId = new Map() + const withoutThread: string[] = [] + for (const name of args.rolloutNames) { + const rollout = parseCodexRolloutThreadId(name) + if (rollout) { + if ((pending.get(rollout.threadId) ?? '') < rollout.rolloutStamp) { + pending.set(rollout.threadId, rollout.rolloutStamp) + } + const markers = markersByThreadId.get(rollout.threadId) ?? [] + markers.push(name) + markersByThreadId.set(rollout.threadId, markers) + } else { + withoutThread.push(name) + } + } + if (withoutThread.length > 0) { + // No thread id means no read can ever settle it. + await args.clear(withoutThread) + } + if (pending.size === 0) { + return true + } + if (stopping) { + return false + } + const options = args.options ?? {} + const summary = await healPendingCodexAccountThreads(args.storeHome, pending, { + ...options, + shouldStop: () => stopping || options.shouldStop?.() === true, + afterBatch: async (settled) => { + await options.afterBatch?.(settled) + // A file a newer pass has yet to move reads as missing; its marker must stay. + const keepMissing = args.passStartedSince?.() === true + await args.clear( + settled + .filter(({ outcome }) => outcome === 'healed' || (outcome === 'missing' && !keepMissing)) + .flatMap(({ threadId }) => markersByThreadId.get(threadId) ?? []) + ) + } + }) + return summary.outcome === 'completed' || summary.outcome === 'up-to-date' +} + +/** + * One background pass per store, and only while no history pass runs (`isPaused`): a pass may + * have recorded markers for files not moved yet, which a read would wrongly settle as missing. + * A request while one runs reruns it afterwards, from whatever markers are left. + */ +export function requestHostCodexIndexHeal( + pendingDir: string, + storeHome: string, + passes: { isPaused: () => boolean; started: () => number } +): void { + const { isPaused } = passes + const key = normalizeRuntimePathForComparison(pendingDir) + if (stopping || isPaused()) { + return + } + if (hostHealsByPendingDir.has(key)) { + rerunRequested.add(key) + return + } + const startedAtRequest = passes.started() + const task = listCodexIndexPendingRollouts(pendingDir) + .then((rolloutNames) => + healCodexIndexPendingRollouts({ + storeHome, + rolloutNames, + clear: async (names) => { + for (const name of names) { + await unlink(join(pendingDir, name)).catch((error: unknown) => { + if (!isDefinitiveAbsence(error)) { + throw error + } + }) + } + }, + passStartedSince: () => passes.started() !== startedAtRequest, + options: { shouldStop: isPaused } + }) + ) + .then((finished) => { + if (finished) { + incompleteHeals.delete(key) + } else { + incompleteHeals.add(key) + } + }) + .catch((error: unknown) => { + incompleteHeals.add(key) + console.warn('[codex-account-history] Indexing shared history failed:', error) + }) + .finally(() => { + hostHealsByPendingDir.delete(key) + if (rerunRequested.delete(key)) { + requestHostCodexIndexHeal(pendingDir, storeHome, passes) + } + }) + hostHealsByPendingDir.set(key, task) +} + +export const _internals = { + reset: (): void => { + stopping = false + hostHealsByPendingDir.clear() + rerunRequested.clear() + incompleteHeals.clear() + } +} diff --git a/src/main/codex/history/shared-store/codex-account-history-links.test.ts b/src/main/codex/history/shared-store/codex-account-history-links.test.ts new file mode 100644 index 00000000000..7008ddfa284 --- /dev/null +++ b/src/main/codex/history/shared-store/codex-account-history-links.test.ts @@ -0,0 +1,230 @@ +import { + existsSync, + linkSync, + lstatSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + realpathSync, + rmSync, + utimesSync, + writeFileSync +} from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { zstdCompressSync } from 'node:zlib' +import { afterEach, describe, expect, it } from 'vitest' +import { checkCodexHistoryLeftBehind } from './codex-account-history-content-check' +import { shareCodexAccountHistory, unshareCodexAccountHistory } from './codex-account-history-links' + +const ROLLOUT = 'rollout-2026-11-14T10-00-00-0199a1b2-c3d4-7e5f-8a9b-0c1d2e3f4a5b.jsonl' +const OTHER = 'rollout-2026-11-14T11-00-00-0199a1b2-c3d4-7e5f-8a9b-0c1d2e3f4a5c.jsonl' +const DAY = join('2026', '11', '14') + +const roots: string[] = [] +function fixture(platform: NodeJS.Platform = 'linux') { + const root = realpathSync(mkdtempSync(join(tmpdir(), 'codex-account-history-'))) + roots.push(root) + const store = join(root, 'store') + const account = join(root, 'account') + mkdirSync(join(store, 'sessions', DAY), { recursive: true }) + mkdirSync(join(account, 'sessions', DAY), { recursive: true }) + const pending = join(root, 'index-pending') + const warnings: unknown[] = [] + const share = () => + shareCodexAccountHistory({ + accountHome: account, + storeHome: store, + sqliteHome: store, + indexPendingDir: pending, + platform, + onWarning: (error) => warnings.push(error) + }) + const unshare = () => + unshareCodexAccountHistory({ + accountHome: account, + storeHome: store, + onWarning: (error) => warnings.push(error) + }) + return { store, account, pending, warnings, share, unshare } +} +afterEach(() => { + for (const root of roots.splice(0)) { + rmSync(root, { recursive: true, force: true }) + } +}) + +describe('shareCodexAccountHistory', () => { + it('links history into the store, keeps the store copy of a conflict and the login private', async () => { + const f = fixture() + writeFileSync(join(f.store, 'sessions', DAY, ROLLOUT), 'store version\n') + writeFileSync(join(f.account, 'sessions', DAY, ROLLOUT), 'account version\n') + writeFileSync(join(f.account, 'sessions', DAY, OTHER), 'account only\n') + writeFileSync(join(f.account, 'auth.json'), '{"account":"a"}') + mkdirSync(join(f.account, 'memories')) + writeFileSync(join(f.account, 'memories', 'note.md'), 'remember\n') + mkdirSync(join(f.store, 'app-server-control')) + mkdirSync(join(f.account, 'log')) + + expect(await f.share()).toBe('shared') + + expect(realpathSync(join(f.account, 'sessions'))).toBe(join(f.store, 'sessions')) + expect(readFileSync(join(f.account, 'memories', 'note.md'), 'utf8')).toBe('remember\n') + expect(realpathSync(join(f.account, 'memories'))).toBe(join(f.store, 'memories')) + expect(readFileSync(join(f.store, 'sessions', DAY, ROLLOUT), 'utf8')).toBe('store version\n') + expect(readFileSync(join(f.store, 'sessions', DAY, OTHER), 'utf8')).toBe('account only\n') + expect(readFileSync(join(f.account, 'sessions.orca-profile-merge', DAY, ROLLOUT), 'utf8')).toBe( + 'account version\n' + ) + expect(readdirSync(f.pending)).toEqual([OTHER]) + expect(readFileSync(join(f.account, 'config.toml'), 'utf8')).toBe( + `sqlite_home = ${JSON.stringify(f.store)} # orca: Codex history shared across accounts\n` + ) + expect(lstatSync(join(f.account, 'auth.json')).isFile()).toBe(true) + expect(lstatSync(join(f.account, 'log')).isSymbolicLink()).toBe(false) + expect(existsSync(join(f.account, 'app-server-control'))).toBe(false) + }) + + it('drops a hard-linked duplicate name but still owes the store its index row', async () => { + const f = fixture() + writeFileSync(join(f.store, 'sessions', DAY, ROLLOUT), 'same file\n') + linkSync(join(f.store, 'sessions', DAY, ROLLOUT), join(f.account, 'sessions', DAY, ROLLOUT)) + + await f.share() + + expect(readFileSync(join(f.store, 'sessions', DAY, ROLLOUT), 'utf8')).toBe('same file\n') + expect(existsSync(join(f.account, 'sessions.orca-profile-merge'))).toBe(false) + expect(readdirSync(f.pending)).toEqual([ROLLOUT]) + expect(f.warnings).toEqual([]) + }) + + it('shares again after sharing was turned off and on, with earlier conflicts still kept', async () => { + const f = fixture() + writeFileSync(join(f.store, 'sessions', DAY, ROLLOUT), 'store version\n') + writeFileSync(join(f.account, 'sessions', DAY, ROLLOUT), 'account version\n') + await f.share() + await f.unshare() + mkdirSync(join(f.account, 'sessions', DAY), { recursive: true }) + writeFileSync(join(f.account, 'sessions', DAY, OTHER), 'private while off\n') + + expect(await f.share()).toBe('shared') + + expect(realpathSync(join(f.account, 'sessions'))).toBe(join(f.store, 'sessions')) + expect(readFileSync(join(f.store, 'sessions', DAY, OTHER), 'utf8')).toBe('private while off\n') + expect(readFileSync(join(f.account, 'sessions.orca-profile-merge', DAY, ROLLOUT), 'utf8')).toBe( + 'account version\n' + ) + }) + + it('keeps one form of a rollout in the store, sets the other aside and records it', async () => { + const f = fixture() + const at = (path: string, seconds: number) => utimesSync(path, seconds, seconds) + const compressed = zstdCompressSync(Buffer.from('plain\n')) + const storeDay = join(f.store, 'sessions', DAY) + const accountDay = join(f.account, 'sessions', DAY) + writeFileSync(join(storeDay, `${ROLLOUT}.zst`), compressed) + at(join(storeDay, `${ROLLOUT}.zst`), 1000) + writeFileSync(join(accountDay, ROLLOUT), 'plain\n') + at(join(accountDay, ROLLOUT), 1000) + writeFileSync(join(storeDay, `${OTHER}.zst`), compressed) + at(join(storeDay, `${OTHER}.zst`), 1000) + writeFileSync(join(accountDay, OTHER), 'plain\nappended after compression\n') + at(join(accountDay, OTHER), 2000) + // A day the store lacks, holding both forms, merges file by file. + const newDay = join(f.account, 'sessions', '2026', '11', '15') + mkdirSync(newDay, { recursive: true }) + writeFileSync(join(newDay, ROLLOUT), 'plain\n') + writeFileSync(join(newDay, `${ROLLOUT}.zst`), compressed) + at(join(newDay, ROLLOUT), 1000) + at(join(newDay, `${ROLLOUT}.zst`), 1000) + writeFileSync(join(f.account, 'state_5.sqlite'), 'db') + + expect(await f.share()).toBe('shared') + await checkCodexHistoryLeftBehind({ + accountHome: f.account, + storeHome: f.store, + shouldStop: () => false + }) + + expect(readdirSync(storeDay).sort()).toEqual([`${ROLLOUT}.zst`, OTHER, `${OTHER}.zst`]) + expect(readdirSync(join(f.store, 'sessions', '2026', '11', '15'))).toEqual([`${ROLLOUT}.zst`]) + const manifest = JSON.parse(readFileSync(join(f.account, '.orca-history-cleanup.json'), 'utf8')) + expect(manifest).toMatchObject({ + version: 1, + sharedPasses: 1, + databases: [{ path: 'state_5.sqlite', bytes: 2 }], + storeTwinPairs: [`sessions/2026/11/14/${OTHER}`] + }) + const checked = { + kind: 'set-aside-twin', + check: { + sameContent: true, + sha256: 'dacf36547c7774a0a170806363b5d412991fbc0d6260b2c00b1d3a80a816c23f' + } + } + expect([...manifest.rollouts].sort((a, b) => a.path.localeCompare(b.path))).toMatchObject([ + { + ...checked, + path: `.orca-set-aside-rollouts/sessions/2026/11/14/${ROLLOUT}`, + storePath: `sessions/2026/11/14/${ROLLOUT}.zst` + }, + { + ...checked, + path: `.orca-set-aside-rollouts/sessions/2026/11/15/${ROLLOUT}`, + storePath: `sessions/2026/11/15/${ROLLOUT}.zst` + } + ]) + }) + + it('renames the ownership marker before a Windows home gets junctions', async () => { + const f = fixture('win32') + writeFileSync(join(f.account, '.orca-managed-home'), 'account-1\n') + + expect(await f.share()).toBe('shared') + + expect(existsSync(join(f.account, '.orca-managed-home'))).toBe(false) + expect(readFileSync(join(f.account, '.orca-managed-home-linked'), 'utf8')).toBe('account-1\n') + }) + + it('links nothing into a Windows home without an ownership marker', async () => { + const f = fixture('win32') + + expect(await f.share()).toBe('private') + + expect(lstatSync(join(f.account, 'sessions')).isSymbolicLink()).toBe(false) + }) + + it('waits while a Codex in the home holds a thread lock', async () => { + const f = fixture() + mkdirSync(join(f.account, 'thread-writer-locks')) + writeFileSync(join(f.account, 'thread-writer-locks', '0199a1b2.lock'), '') + writeFileSync(join(f.account, 'sessions', DAY, OTHER), 'account only\n') + + expect(await f.share()).toBe('busy') + + expect(lstatSync(join(f.account, 'sessions')).isSymbolicLink()).toBe(false) + expect(existsSync(join(f.store, 'sessions', DAY, OTHER))).toBe(false) + }) +}) + +describe('unshareCodexAccountHistory', () => { + it('gives the home fresh private folders and sets its old databases aside', async () => { + const f = fixture() + writeFileSync(join(f.account, 'sessions', DAY, OTHER), 'shared earlier\n') + await f.share() + writeFileSync(join(f.account, 'state_5.sqlite'), 'pre-share index') + + await f.unshare() + + expect(lstatSync(join(f.account, 'sessions')).isSymbolicLink()).toBe(false) + expect(readdirSync(join(f.account, 'sessions'))).toEqual([]) + expect(readFileSync(join(f.account, 'config.toml'), 'utf8')).toBe('') + expect(readFileSync(join(f.store, 'sessions', DAY, OTHER), 'utf8')).toBe('shared earlier\n') + expect(existsSync(join(f.account, 'state_5.sqlite'))).toBe(false) + const [held] = readdirSync(join(f.account, '.orca-pre-share-db')) + expect( + readFileSync(join(f.account, '.orca-pre-share-db', held, 'state_5.sqlite'), 'utf8') + ).toBe('pre-share index') + }) +}) diff --git a/src/main/codex/history/shared-store/codex-account-history-links.ts b/src/main/codex/history/shared-store/codex-account-history-links.ts new file mode 100644 index 00000000000..142c9c0ba71 --- /dev/null +++ b/src/main/codex/history/shared-store/codex-account-history-links.ts @@ -0,0 +1,344 @@ +import { mkdir, readdir, realpath, rename, unlink, writeFile } from 'node:fs/promises' +import { basename, join, relative } from 'node:path' +import { isDefinitiveAbsence } from '../../../../shared/errors/definitive-filesystem-absence' +import { renameFileWithWindowsRetryAsync } from '../../../../shared/file-write/windows-retry-file-operations' +import { + DirectoryLinkError, + MERGE_PENDING_SUFFIX, + isLinkedDirectory, + lstatIfPresent, + mergeDirectoryIntoLink, + removeDirectoryLink +} from '../../../filesystem/merged-directory-link' +import { + LINKED_MANAGED_HOME_MARKER, + MANAGED_HOME_MARKER +} from '../../../codex-accounts/managed-homes/host-codex-managed-home-ownership' +import { writeSharedSqliteHome } from './codex-account-sqlite-home-config' +import { + CODEX_DATABASE_FILE, + PRE_SHARE_DATABASES_DIR, + SET_ASIDE_ROLLOUTS_DIR, + recordCodexHistoryLeftBehind +} from './codex-account-history-cleanup-manifest' +import { + compressedRolloutsFirst, + holdsRolloutTwins, + isRolloutTwinKeptFromStore +} from '../rollouts/codex-rollout-compressed-twin' +import { parseCodexRolloutThreadId } from '../rollouts/codex-rollout-thread-id' + +// Created in the store when missing, so a first account still links its history. +const STORE_DIRECTORIES = ['sessions', 'archived_sessions', 'rollout-migrations'] + +// Local logs and scratch; per-home process endpoints (a linked daemon socket would attach an +// account to the ~/.codex login's daemon); state keyed to the home's own path (encrypted secrets +// whose keyring passphrase is named from CODEX_HOME, the Windows sandbox setup); resources Orca +// links or copies itself, including the home's own bundled marketplaces inside `.tmp`. +const PRIVATE_DIRECTORIES = new Set([ + 'log', + 'tmp', + '.tmp', + 'app-server-control', + 'app-server-daemon', + 'ipc', + 'secrets', + '.sandbox', + '.sandbox-bin', + '.sandbox-secrets', + 'skills', + 'hooks', + 'plugins', + 'plugin-state', + 'profile-v2', + 'themes', + 'prompts', + 'rules' +]) + +const HISTORY_DIRECTORIES = new Set(['sessions', 'archived_sessions']) +// Linked before any history, so a home never writes shared rollouts under private locks. +const WRITER_LOCKS = 'thread-writer-locks' +// Present from the first unlinked folder until the databases moved aside, so a retry finishes. +const UNSHARE_PENDING = '.orca-unshare-pending' + +/** + * `shared` once the home's history resolves to the store; `incomplete` when it does but some entry + * failed to move (a later pass retries it); `private` when a step refused. + */ +export type CodexAccountHistoryShareResult = 'shared' | 'incomplete' | 'private' | 'busy' + +export type CodexAccountHistoryShareArgs = { + accountHome: string + storeHome: string + /** Where Codex keeps the store's databases, written into the home's config; null writes none. */ + sqliteHome: string | null + /** Rollouts the store's index still has to learn, one empty file each, shared by every account. */ + indexPendingDir: string + platform: NodeJS.Platform + onWarning: (error: unknown) => void +} + +function isSharedDirectoryName(name: string): boolean { + return ( + !PRIVATE_DIRECTORIES.has(name) && + !name.startsWith('.orca') && + !name.endsWith(MERGE_PENDING_SUFFIX) + ) +} + +async function readDirectoryEntries(path: string) { + try { + return await readdir(path, { withFileTypes: true }) + } catch (error) { + if (isDefinitiveAbsence(error)) { + return [] + } + throw error + } +} + +/** Every folder of the store or the home is shared; files (login, config, prompt recall) stay. */ +async function sharedDirectoryNames(storeHome: string, accountHome: string): Promise { + const names = new Set(STORE_DIRECTORIES) + for (const home of [storeHome, accountHome]) { + for (const entry of await readDirectoryEntries(home)) { + if (entry.isDirectory()) { + names.add(entry.name) + } + } + } + names.delete(WRITER_LOCKS) + return [...names].filter(isSharedDirectoryName) +} + +// Codex deletes a thread lock on release, so one left may guard a live writer; Codex itself +// clears stale ones on its next start, after which a later launch shares the home. +async function hasThreadWriter(accountHome: string): Promise { + const locks = join(accountHome, WRITER_LOCKS) + if ((await lstatIfPresent(locks))?.isSymbolicLink()) { + return false + } + return (await readDirectoryEntries(locks)).some( + (entry) => entry.name.endsWith('.lock') && entry.name !== '.coordination.lock' + ) +} + +async function isSameDirectory(left: string, right: string): Promise { + const resolve = (path: string): Promise => + realpath(path).catch((error: unknown) => { + if (isDefinitiveAbsence(error)) { + return null + } + throw error + }) + const [a, b] = await Promise.all([resolve(left), resolve(right)]) + return a !== null && a === b +} + +// Recorded before the move, so a crash mid-move cannot strand a rollout without its marker. +async function recordIndexPending(pendingDir: string, path: string): Promise { + const entries = await readDirectoryEntries(path) + if (entries.length === 0 && parseCodexRolloutThreadId(path)) { + await mkdir(pendingDir, { recursive: true }) + await writeFile(join(pendingDir, basename(path)), '') + return + } + for (const entry of entries) { + await recordIndexPending(pendingDir, join(path, entry.name)) + } +} + +/** False when the home has no ownership marker; a link must never land in an unowned home. */ +async function markHomeLinked(accountHome: string): Promise { + try { + await renameFileWithWindowsRetryAsync( + join(accountHome, MANAGED_HOME_MARKER), + join(accountHome, LINKED_MANAGED_HOME_MARKER) + ) + return true + } catch (error) { + if (!isDefinitiveAbsence(error)) { + throw error + } + } + return (await lstatIfPresent(join(accountHome, LINKED_MANAGED_HOME_MARKER)))?.isFile() ?? false +} + +/** Both history folders: Codex archives by moving a rollout between them. */ +export async function isCodexAccountHistoryLinked( + accountHome: string, + storeHome: string +): Promise { + const linked = await Promise.all( + [...HISTORY_DIRECTORIES].map((name) => + isLinkedDirectory(join(accountHome, name), join(storeHome, name)) + ) + ) + return linked.every(Boolean) +} + +/** + * Replaces the account home's folders with links into the store (the user's ~/.codex), moving + * the home's own entries there first; the store's copy wins a name conflict and the home's copy + * stays in `.orca-profile-merge`, and a rollout form the store must not hold beside its + * twin waits in `.orca-set-aside-rollouts`. Nothing is deleted. Codex's databases follow once the + * history is linked. + */ +export async function shareCodexAccountHistory( + args: CodexAccountHistoryShareArgs +): Promise { + if (await isSameDirectory(args.accountHome, args.storeHome)) { + // Linking the store into itself would park its history in a pending folder. + return 'private' + } + // An older build's recursive removal would follow a junction and delete the store's history. + if (args.platform === 'win32' && !(await markHomeLinked(args.accountHome))) { + return 'private' + } + if (await hasThreadWriter(args.accountHome)) { + return (await isCodexAccountHistoryLinked(args.accountHome, args.storeHome)) ? 'shared' : 'busy' + } + await Promise.all( + [WRITER_LOCKS, ...STORE_DIRECTORIES].map((name) => + mkdir(join(args.storeHome, name), { recursive: true }) + ) + ) + let failed = false + const onWarning = (error: unknown): void => { + // A kept conflict is final; anything else left an entry behind for a later pass. + failed ||= !(error instanceof DirectoryLinkError && error.code === 'retained-conflict') + args.onWarning(error) + } + const link = (name: string) => + mergeDirectoryIntoLink({ + source: join(args.accountHome, name), + destination: join(args.storeHome, name), + platform: args.platform, + onWarning, + beforeShare: HISTORY_DIRECTORIES.has(name) + ? (path) => recordIndexPending(args.indexPendingDir, path) + : undefined, + ...(HISTORY_DIRECTORIES.has(name) && { + setAsidePath: async (from: string, to: string) => + (await isRolloutTwinKeptFromStore(from, to)) + ? join( + args.accountHome, + SET_ASIDE_ROLLOUTS_DIR, + name, + relative(join(args.storeHome, name), to) + ) + : null, + movesWhole: async (folder: string) => !(await holdsRolloutTwins(folder)), + // The `.zst` reaches the store first, so its `.jsonl` is set aside only beside it. + entryOrder: compressedRolloutsFirst + }), + dropEmptyConflicts: name.endsWith('-locks') + }) + try { + // A held lock file moves by hard link, so its inode keeps guarding the thread. + await link(WRITER_LOCKS) + } catch (error) { + onWarning(error) + return 'private' + } + // A user's own link elsewhere would split the lock namespace from the history's. + if ( + !(await isLinkedDirectory( + join(args.accountHome, WRITER_LOCKS), + join(args.storeHome, WRITER_LOCKS) + )) + ) { + return 'private' + } + for (const name of await sharedDirectoryNames(args.storeHome, args.accountHome)) { + try { + await link(name) + } catch (error) { + onWarning(error) + } + } + if (!(await isCodexAccountHistoryLinked(args.accountHome, args.storeHome))) { + return 'private' + } + await writeSharedSqliteHome(args.accountHome, args.sqliteHome) + const result = failed ? 'incomplete' : 'shared' + await recordCodexHistoryLeftBehind({ + accountHome: args.accountHome, + storeHome: args.storeHome, + outcome: result, + databasesMovedToStore: args.sqliteHome !== null + }).catch(args.onWarning) + return result +} + +/** + * Sharing turned off: later conversations stay private. Links become fresh empty folders, and the + * home's pre-share databases (they list files that now live in the store) move aside so Codex + * starts a private index. History already shared stays in the store. False when a link stayed. + */ +export async function unshareCodexAccountHistory(args: { + accountHome: string + storeHome: string + onWarning: (error: unknown) => void +}): Promise { + let unlinked = false + let complete = true + for (const entry of await readDirectoryEntries(args.accountHome)) { + const link = join(args.accountHome, entry.name) + if ( + !isSharedDirectoryName(entry.name) || + // Lock folders stay shared: swapping them could split a live writer's namespace. + entry.name.endsWith('-locks') || + !(await isLinkedDirectory(link, join(args.storeHome, entry.name))) + ) { + continue + } + try { + if (!unlinked) { + await writeFile(join(args.accountHome, UNSHARE_PENDING), '') + } + await removeDirectoryLink(link) + await mkdir(link) + unlinked = true + } catch (error) { + complete = false + args.onWarning(error) + } + } + await writeSharedSqliteHome(args.accountHome, null) + const pending = join(args.accountHome, UNSHARE_PENDING) + if (!unlinked && !(await lstatIfPresent(pending))) { + return complete + } + const holding = join(args.accountHome, PRE_SHARE_DATABASES_DIR, String(Date.now())) + for (const entry of await readDirectoryEntries(args.accountHome)) { + if (!entry.isFile() || !CODEX_DATABASE_FILE.test(entry.name)) { + continue + } + try { + await mkdir(holding, { recursive: true }) + await rename(join(args.accountHome, entry.name), join(holding, entry.name)) + } catch (error) { + complete = false + args.onWarning(error) + } + } + if (complete) { + await unlink(pending) + } + await recordCodexHistoryLeftBehind({ + accountHome: args.accountHome, + storeHome: args.storeHome, + outcome: 'unshared', + databasesMovedToStore: false + }).catch(args.onWarning) + return complete +} + +/** Rollout file names whose rows the store's index still owes. */ +export async function listCodexIndexPendingRollouts(pendingDir: string): Promise { + return (await readDirectoryEntries(pendingDir)) + .filter((entry) => entry.isFile()) + .map((entry) => entry.name) +} diff --git a/src/main/codex/history/shared-store/codex-account-history-passes.ts b/src/main/codex/history/shared-store/codex-account-history-passes.ts new file mode 100644 index 00000000000..be627a728dc --- /dev/null +++ b/src/main/codex/history/shared-store/codex-account-history-passes.ts @@ -0,0 +1,148 @@ +import { join } from 'node:path' +import { normalizeRuntimePathForComparison } from '../../../../shared/file-access/cross-platform-path' +import { + getOrcaUserDataPath, + getSystemCodexHomePath +} from '../../configuration/homes/codex-home-paths' +import { checkCodexHistoryLeftBehind } from './codex-account-history-content-check' +import { + hasIncompleteHostCodexIndexHeal, + isCodexAccountHistoryWorkStopping, + requestHostCodexIndexHeal +} from './codex-account-history-index' +import { shareCodexAccountHistory, unshareCodexAccountHistory } from './codex-account-history-links' + +type QueuedPass = { shared: boolean; pass: Promise } + +const INDEX_PENDING_DIR = (): string => + join(getOrcaUserDataPath(), 'codex-account-history', 'index-pending') + +/** + * Host account history passes, one at a time in request order (homes merging into one store at + * once would race on its folders): a request joins only the latest one queued for that home with + * the same mode. A home settled in a mode this run skips later passes in it. + */ +export class CodexAccountHistoryPasses { + private readonly latest = new Map() + private tail: Promise = Promise.resolve() + private readonly settledThisRun = new Map() + private activePasses = 0 + // Homes whose left-behind rollouts still await a content check, checked once no pass runs. + private readonly owedChecks = new Map() + private checking = false + private startedPasses = 0 + private readonly passProgress = { + isPaused: (): boolean => this.activePasses > 0, + started: (): number => this.startedPasses + } + + run(accountHome: string, shared: boolean): Promise { + const key = normalizeRuntimePathForComparison(accountHome) + const previous = this.latest.get(key) + if (previous?.shared === shared) { + return previous.pass + } + const pass = this.tail.then(async () => { + if (this.settledThisRun.get(key) === shared) { + // Moves are done; only index work a stopped or failed heal left behind is retried. + if (hasIncompleteHostCodexIndexHeal(INDEX_PENDING_DIR())) { + requestHostCodexIndexHeal( + INDEX_PENDING_DIR(), + getSystemCodexHomePath(), + this.passProgress + ) + } + return + } + this.activePasses += 1 + this.startedPasses += 1 + try { + await this.runPass(key, accountHome, shared) + } finally { + this.activePasses -= 1 + } + // Index once no pass is moving files. + requestHostCodexIndexHeal(INDEX_PENDING_DIR(), getSystemCodexHomePath(), this.passProgress) + this.checkOwed() + }) + this.tail = pass.catch(() => {}) + this.latest.set(key, { shared, pass }) + void pass.finally(() => { + if (this.latest.get(key)?.pass === pass) { + this.latest.delete(key) + } + }) + return pass + } + + private async runPass(key: string, accountHome: string, shared: boolean): Promise { + this.settledThisRun.delete(key) + const storeHome = getSystemCodexHomePath() + const onWarning = (error: unknown): void => { + console.warn('[codex-account-history] Account history was not fully shared:', error) + } + try { + if (!shared) { + if (await unshareCodexAccountHistory({ accountHome, storeHome, onWarning })) { + this.settledThisRun.set(key, false) + } + return + } + const indexPendingDir = INDEX_PENDING_DIR() + const result = await shareCodexAccountHistory({ + accountHome, + storeHome, + // A user's own CODEX_SQLITE_HOME already points every home's databases at the store. + sqliteHome: process.env.CODEX_SQLITE_HOME?.trim() ? null : storeHome, + indexPendingDir, + platform: process.platform, + onWarning + }) + if (result === 'shared') { + this.settledThisRun.set(key, true) + } + if (result === 'shared' || result === 'incomplete') { + this.owedChecks.set(key, { accountHome }) + } + } catch (error) { + onWarning(error) + } + } + + // Hashing a large history must not delay a launch: a starting pass stops it after one file. + private checkOwed(): void { + if (this.checking || this.activePasses > 0 || isCodexAccountHistoryWorkStopping()) { + return + } + const next = this.owedChecks.entries().next() + if (next.done) { + return + } + const [key, request] = next.value + // A pass that finished meanwhile owes a fresh check; only this run's request is settled. + const settle = (): void => { + if (this.owedChecks.get(key) === request) { + this.owedChecks.delete(key) + } + } + this.checking = true + void checkCodexHistoryLeftBehind({ + accountHome: request.accountHome, + storeHome: getSystemCodexHomePath(), + shouldStop: () => this.activePasses > 0 || isCodexAccountHistoryWorkStopping() + }) + .then((finished) => { + if (finished) { + settle() + } + }) + .catch((error: unknown) => { + settle() + console.warn('[codex-account-history] Checking left-behind history failed:', error) + }) + .finally(() => { + this.checking = false + this.checkOwed() + }) + } +} diff --git a/src/main/codex/history/shared-store/codex-account-history-wsl-entry.ts b/src/main/codex/history/shared-store/codex-account-history-wsl-entry.ts new file mode 100644 index 00000000000..cf0c1f6d9b2 --- /dev/null +++ b/src/main/codex/history/shared-store/codex-account-history-wsl-entry.ts @@ -0,0 +1,58 @@ +import { homedir } from 'node:os' +import { join } from 'node:path' +import { + listCodexIndexPendingRollouts, + shareCodexAccountHistory, + unshareCodexAccountHistory +} from './codex-account-history-links' +import { checkCodexHistoryLeftBehind } from './codex-account-history-content-check' + +// Under the host's helper timeout; an unfinished check resumes on a later launch. +const CHECK_BUDGET_MS = 8 * 60_000 + +// Runs inside a WSL distro on Orca's pinned Node: ` `. +// Prints each rollout still owed an index row, then `{"result":…,"pendingDir":…}` last. +async function main(): Promise { + const [mode, accountHome = '', storeHome = ''] = process.argv.slice(2) + if (!accountHome.startsWith('/') || !storeHome.startsWith('/')) { + throw new Error('Codex account history needs absolute Linux homes') + } + const onWarning = (error: unknown): void => { + console.error(error instanceof Error ? error.message : String(error)) + } + // Beside the guest's account homes, so it is per store and never inside the user's ~/.codex. + const pendingDir = join(homedir(), '.local', 'share', 'orca', 'codex-index-pending') + let result: string + if (mode === 'check') { + const deadline = Date.now() + CHECK_BUDGET_MS + const finished = await checkCodexHistoryLeftBehind({ + accountHome, + storeHome, + shouldStop: () => Date.now() > deadline + }) + process.stdout.write( + `${JSON.stringify({ result: finished ? 'checked' : 'stopped', pendingDir })}\n` + ) + return + } + if (mode === 'unshare') { + const complete = await unshareCodexAccountHistory({ accountHome, storeHome, onWarning }) + result = complete ? 'unshared' : 'incomplete' + } else { + result = await shareCodexAccountHistory({ + accountHome, + storeHome, + sqliteHome: storeHome, + indexPendingDir: pendingDir, + platform: 'linux', + onWarning + }) + } + const pending = await listCodexIndexPendingRollouts(pendingDir) + process.stdout.write(pending.map((name) => `${name}\n`).join('')) + process.stdout.write(`${JSON.stringify({ result, pendingDir })}\n`) +} +void main().catch((error: unknown) => { + console.error(error instanceof Error ? error.message : String(error)) + process.exitCode = 1 +}) diff --git a/src/main/codex/history/shared-store/codex-account-sqlite-home-config.ts b/src/main/codex/history/shared-store/codex-account-sqlite-home-config.ts new file mode 100644 index 00000000000..6205b8a4373 --- /dev/null +++ b/src/main/codex/history/shared-store/codex-account-sqlite-home-config.ts @@ -0,0 +1,41 @@ +import { randomUUID } from 'node:crypto' +import { readFile, rename, unlink, writeFile } from 'node:fs/promises' +import { join } from 'node:path' +import { isDefinitiveAbsence } from '../../../../shared/errors/definitive-filesystem-absence' +import { applyCodexSharedSqliteHome } from '../../configuration/toml/config-toml-shared-sqlite-home' + +async function readConfig(configPath: string): Promise { + try { + return await readFile(configPath, 'utf8') + } catch (error) { + if (isDefinitiveAbsence(error)) { + return '' + } + throw error + } +} + +/** Orca's sqlite_home line in the home's own config; never the user's ~/.codex config. */ +export async function writeSharedSqliteHome( + accountHome: string, + sqliteHome: string | null +): Promise { + const configPath = join(accountHome, 'config.toml') + // A mirror writing meanwhile wins the race; the line is then applied to its content. + for (let attempt = 0; attempt < 5; attempt += 1) { + const current = await readConfig(configPath) + const next = applyCodexSharedSqliteHome(current, sqliteHome) + if (next === current) { + return + } + const staged = `${configPath}.${randomUUID()}.tmp` + // Mirrored MCP environment tokens require owner-only writes. + await writeFile(staged, next, { mode: 0o600 }) + if ((await readConfig(configPath)) === current) { + await rename(staged, configPath) + return + } + await unlink(staged) + } + throw new Error(`Codex config kept changing; sqlite_home not updated in ${configPath}`) +} diff --git a/src/main/codex/history/shared-store/wsl-codex-account-history.ts b/src/main/codex/history/shared-store/wsl-codex-account-history.ts new file mode 100644 index 00000000000..61feb577dee --- /dev/null +++ b/src/main/codex/history/shared-store/wsl-codex-account-history.ts @@ -0,0 +1,342 @@ +import { WSL_CODEX_ACCOUNT_HISTORY_HELPER_FILENAME } from '../../../../shared/remote-runtime/orcad-distribution/remote-bundle-artifacts' +import { runWslGuestNodeHelper } from '../../../wsl/execution/wsl-guest-node-helper' +import { runWslProcess } from '../../../wsl/execution/wsl-runner' +import { assertWslUncPathRunning } from '../../../wsl/filesystem/wsl-unc-access-gate' +import type { onRunningWslDistrosStopped } from '../../../wsl/discovery/wsl-running-distro-cache' +import { onWslDistroGateNote } from '../../../wsl/discovery/wsl-background-distro-gate' +import { runCodexAppServerSession } from '../../app-server/codex-app-server-session' +import { buildWslHealInvocation } from '../index/codex-session-index-heal' +import { + healCodexIndexPendingRollouts, + isCodexAccountHistoryWorkStopping +} from './codex-account-history-index' + +export type WslCodexAccountHistoryTarget = { + distro: string + /** Linux paths inside the distro. */ + accountHome: string + storeHome: string + shared: boolean +} + +type WslAccountHistoryDependencies = { + runHelper: typeof runWslGuestNodeHelper + /** Throws unless the distro runs; each step outlives the launch that started it (E9). */ + requireRunning: (distro: string) => Promise + subscribeStops: typeof onRunningWslDistrosStopped +} + +const defaultDependencies: WslAccountHistoryDependencies = { + runHelper: runWslGuestNodeHelper, + requireRunning: (distro) => assertWslUncPathRunning('', distro), + // Gate notes carry running-list stops and explicit ones (a relay or watcher exit) no list saw yet. + subscribeStops: (listener) => + onWslDistroGateNote((note) => { + if (note.kind === 'stopped') { + listener([note.distro], []) + } + }) +} + +// The helper may move a whole history on first run; pending names are ~75 bytes each. +const HELPER_TIMEOUT_MS = 10 * 60_000 +const HELPER_MAX_OUTPUT_BYTES = 64 * 1024 * 1024 +const CLEAR_TIMEOUT_MS = 30_000 + +type HelperOutput = { result: string; pendingDir: string; pending: string[] } + +/** Links (or unlinks) the home inside its distro; the guest helper also owns its sqlite_home. */ +async function linkInGuest( + target: WslCodexAccountHistoryTarget, + dependencies: WslAccountHistoryDependencies +): Promise { + // Every wsl.exe step re-checks: the user may stop the distro mid-pass, and a step would boot it. + const stdout = await dependencies.runHelper({ + distro: target.distro, + helperFilename: WSL_CODEX_ACCOUNT_HISTORY_HELPER_FILENAME, + helperLabel: 'Codex account history', + helperArgs: [target.shared ? 'share' : 'unshare', target.accountHome, target.storeHome], + timeoutMs: HELPER_TIMEOUT_MS, + maxOutputBytes: HELPER_MAX_OUTPUT_BYTES, + requireRunning: () => dependencies.requireRunning(target.distro) + }) + return parseHelperOutput(stdout) +} + +/** Compares the home's left-behind rollouts with the guest store within the helper's budget. */ +async function checkInGuest( + target: WslCodexAccountHistoryTarget, + dependencies: WslAccountHistoryDependencies +): Promise { + const stdout = await dependencies.runHelper({ + distro: target.distro, + helperFilename: WSL_CODEX_ACCOUNT_HISTORY_HELPER_FILENAME, + helperLabel: 'Codex account history', + helperArgs: ['check', target.accountHome, target.storeHome], + timeoutMs: HELPER_TIMEOUT_MS, + requireRunning: () => dependencies.requireRunning(target.distro) + }) + if (!parseHelperOutput(stdout)) { + throw new Error('The WSL Codex history check printed no summary') + } +} + +/** Marker lines precede the summary; a missing summary means the list may be clipped. */ +function parseHelperOutput(stdout: string): HelperOutput | null { + const lines = stdout + .split(/\r?\n/) + .map((line) => line.trim()) + .filter((line) => line.length > 0) + let summary: unknown + try { + summary = JSON.parse(lines.at(-1) ?? '') + } catch { + return null + } + if ( + !summary || + typeof summary !== 'object' || + !('pendingDir' in summary) || + typeof summary.pendingDir !== 'string' + ) { + return null + } + // Login scripts may print before the helper; only rollout names are markers. + return { + result: 'result' in summary && typeof summary.result === 'string' ? summary.result : '', + pendingDir: summary.pendingDir, + pending: lines.slice(0, -1).filter((line) => line.startsWith('rollout-')) + } +} + +/** Codex's own `thread/read` inside the distro indexes moved rollouts against the guest store. */ +async function healInGuest( + target: WslCodexAccountHistoryTarget, + output: HelperOutput, + dependencies: WslAccountHistoryDependencies, + passes: { isPaused: () => boolean; startedSince: () => boolean } +): Promise { + const distroKey = target.distro.toLowerCase() + let distroStopped = false + const unsubscribe = dependencies.subscribeStops((stopped) => { + distroStopped ||= stopped.some((distro) => distro.toLowerCase() === distroKey) + }) + const stillRunning = async (): Promise => { + await dependencies.requireRunning(target.distro).catch((error: unknown) => { + distroStopped = true + throw error + }) + } + try { + return await healCodexIndexPendingRollouts({ + storeHome: target.storeHome, + rolloutNames: output.pending, + clear: async (names) => { + await stillRunning() + await clearPendingMarkers(target.distro, output.pendingDir, names) + }, + passStartedSince: passes.startedSince, + options: { + // Windows can't take SQLite's locks over \\wsl.localhost; the guest's own codex indexes. + buildInvocation: (_home, timeoutMs) => + buildWslHealInvocation(target.distro, target.storeHome, timeoutMs), + runSession: async (invocation, body) => { + await stillRunning() + return runCodexAppServerSession(invocation, body) + }, + shouldStop: () => distroStopped || passes.isPaused() + } + }) + } finally { + unsubscribe() + } +} + +async function clearPendingMarkers( + distro: string, + pendingDir: string, + names: readonly string[] +): Promise { + if (names.length === 0) { + return + } + const result = await runWslProcess({ + distro, + loginPath: 'none', + script: 'cd -- "$1" || exit 1\nshift\nrm -f -- "$@"', + args: [pendingDir, ...names], + timeoutMs: CLEAR_TIMEOUT_MS + }) + if (result.code !== 0 || result.timedOut) { + throw new Error(`Could not clear WSL Codex index markers (code ${result.code})`) + } +} + +type QueuedPass = { shared: boolean; pass: Promise } + +/** + * WSL account history passes, one at a time per distro in request order (homes merging into its + * store at once would race on its folders; a request joins only the latest queued one for that + * home with the same mode); indexing runs detached, one at a time per distro. + */ +export class WslCodexAccountHistoryLaunches { + private readonly latest = new Map() + private readonly tails = new Map>() + private readonly heals = new Map>() + // The newest guest marker list per distro, indexed once no pass runs there. + private readonly owedHeals = new Map< + string, + { target: WslCodexAccountHistoryTarget; output: HelperOutput } + >() + // A heal that failed waits for the next launch in that distro, not an immediate retry. + private readonly failedHeals = new Map< + string, + { target: WslCodexAccountHistoryTarget; output: HelperOutput } + >() + private readonly activePasses = new Map() + private readonly startedPasses = new Map() + // A home settled in a mode this run skips later guest passes in it. + private readonly settledThisRun = new Map() + // Shared homes whose left-behind rollouts await a content check, one guest check per distro. + private readonly owedChecks = new Map() + private readonly checks = new Set() + + constructor(private readonly dependencies: WslAccountHistoryDependencies = defaultDependencies) {} + + run(target: WslCodexAccountHistoryTarget): Promise { + if (isCodexAccountHistoryWorkStopping()) { + return Promise.resolve() + } + const distroKey = target.distro.toLowerCase() + const key = `${distroKey}\0${target.accountHome}` + const previous = this.latest.get(key) + if (previous?.shared === target.shared) { + return previous.pass + } + const pass = (this.tails.get(distroKey) ?? Promise.resolve()) + .then(async () => { + // Quit doesn't await history work, so a pass queued before it must not start guest work. + if (isCodexAccountHistoryWorkStopping()) { + return + } + if (this.settledThisRun.get(key) === target.shared) { + this.retryFailedHeal(distroKey) + return + } + this.settledThisRun.delete(key) + this.activePasses.set(distroKey, (this.activePasses.get(distroKey) ?? 0) + 1) + this.startedPasses.set(distroKey, (this.startedPasses.get(distroKey) ?? 0) + 1) + try { + const output = await linkInGuest(target, this.dependencies) + if (output && output.result === (target.shared ? 'shared' : 'unshared')) { + this.settledThisRun.set(key, target.shared) + } + if (output && output.pending.length > 0) { + this.oweHeal(distroKey, { target, output }) + } + if (target.shared && (output?.result === 'shared' || output?.result === 'incomplete')) { + this.owedChecks.set(key, { ...target }) + } + } finally { + this.activePasses.set(distroKey, (this.activePasses.get(distroKey) ?? 1) - 1) + this.startOwedHeal(distroKey) + this.startOwedCheck(distroKey) + } + }) + .catch((error: unknown) => { + console.warn('[codex-account-history] WSL account history pass failed:', error) + }) + this.tails.set(distroKey, pass) + this.latest.set(key, { shared: target.shared, pass }) + void pass.finally(() => { + if (this.latest.get(key)?.pass === pass) { + this.latest.delete(key) + } + }) + return pass + } + + // Passes in one distro share its marker folder and may report in any order: keep every name. + private oweHeal( + distroKey: string, + owed: { target: WslCodexAccountHistoryTarget; output: HelperOutput } + ): void { + const earlier = this.owedHeals.get(distroKey)?.output.pending ?? [] + const pending = [...new Set([...earlier, ...owed.output.pending])] + this.owedHeals.set(distroKey, { ...owed, output: { ...owed.output, pending } }) + } + + private retryFailedHeal(distroKey: string): void { + const failed = this.failedHeals.get(distroKey) + if (failed) { + this.failedHeals.delete(distroKey) + this.oweHeal(distroKey, failed) + this.startOwedHeal(distroKey) + } + } + + // A pass may have recorded markers for files not moved yet, which a read would settle as missing. + private startOwedHeal(distroKey: string): void { + const isPaused = (): boolean => (this.activePasses.get(distroKey) ?? 0) > 0 + const owed = this.owedHeals.get(distroKey) + if (!owed || isPaused() || this.heals.has(distroKey) || isCodexAccountHistoryWorkStopping()) { + return + } + this.owedHeals.delete(distroKey) + const started = this.startedPasses.get(distroKey) ?? 0 + const heal = healInGuest(owed.target, owed.output, this.dependencies, { + isPaused, + startedSince: () => (this.startedPasses.get(distroKey) ?? 0) !== started + }) + .then((finished) => { + if (!finished) { + this.failedHeals.set(distroKey, owed) + } + }) + .catch((error: unknown) => { + console.warn('[codex-account-history] Indexing WSL shared history failed:', error) + this.failedHeals.set(distroKey, owed) + }) + .finally(() => { + this.heals.delete(distroKey) + this.startOwedHeal(distroKey) + }) + this.heals.set(distroKey, heal) + } + + // Detached: hashing a large history must not hold up a launch's pass. + private startOwedCheck(distroKey: string): void { + if ( + this.checks.has(distroKey) || + (this.activePasses.get(distroKey) ?? 0) > 0 || + isCodexAccountHistoryWorkStopping() + ) { + return + } + const owed = [...this.owedChecks].find( + ([, target]) => target.distro.toLowerCase() === distroKey + ) + if (!owed) { + return + } + const [key, target] = owed + this.checks.add(distroKey) + // An unfinished check resumes on a later launch, never in a loop now. + // A pass that finished meanwhile owes a fresh check; only this run's request is settled. + const settle = (): void => { + if (this.owedChecks.get(key) === target) { + this.owedChecks.delete(key) + } + } + void checkInGuest(target, this.dependencies) + .then(settle) + .catch((error: unknown) => { + settle() + console.warn('[codex-account-history] Checking WSL left-behind history failed:', error) + }) + .finally(() => { + this.checks.delete(distroKey) + this.startOwedCheck(distroKey) + }) + } +} diff --git a/src/main/filesystem/merged-directory-link.ts b/src/main/filesystem/merged-directory-link.ts new file mode 100644 index 00000000000..8862163ae75 --- /dev/null +++ b/src/main/filesystem/merged-directory-link.ts @@ -0,0 +1,287 @@ +import { + link, + lstat, + mkdir, + readdir, + readlink, + realpath, + rmdir, + stat, + symlink, + unlink +} from 'node:fs/promises' +import type { BigIntStats } from 'node:fs' +import { dirname, join } from 'node:path' +import { isDefinitiveAbsence } from '../../shared/errors/definitive-filesystem-absence' +import { renameFileWithWindowsRetryAsync } from '../../shared/file-write/windows-retry-file-operations' + +/** Where a folder waits while its entries move into the shared one; conflicts stay here. */ +export const MERGE_PENDING_SUFFIX = '.orca-profile-merge' + +export type MergedDirectoryLinkOutcome = 'linked' | 'unchanged' | 'user-owned' + +export type DirectoryLinkErrorCode = 'cross-filesystem' | 'retained-conflict' | 'link-failed' + +export class DirectoryLinkError extends Error { + readonly code: DirectoryLinkErrorCode + + constructor(code: DirectoryLinkErrorCode, message: string) { + super(message) + this.code = code + } +} + +export type MergedDirectoryLinkArgs = { + /** The private folder that becomes a link. */ + source: string + /** The shared folder it links to; created when missing. */ + destination: string + platform: NodeJS.Platform + /** Problems that leave the link published, such as retained conflicts. */ + onWarning: (error: unknown) => void + /** + * Runs before an entry joins `destination`, with its current path: one that moves (a folder + * moves whole) or a hard-linked duplicate whose name is dropped. + */ + beforeShare?: (path: string) => void | Promise + /** Empty files that collide with a shared name are dropped (lock files hold no data). */ + dropEmptyConflicts?: boolean + /** Where a file joining `destination` at `to` waits instead; null shares it. */ + setAsidePath?: (from: string, to: string) => Promise + /** False merges a folder file by file, so `setAsidePath` sees each one. */ + movesWhole?: (folder: string) => Promise + /** The order a folder's entries move in. */ + entryOrder?: (left: string, right: string) => number +} + +type MoveOptions = Pick< + MergedDirectoryLinkArgs, + 'beforeShare' | 'dropEmptyConflicts' | 'setAsidePath' | 'movesWhole' | 'entryOrder' +> + +export async function lstatIfPresent(file: string): Promise { + try { + return await lstat(file, { bigint: true }) + } catch (error) { + if (!isDefinitiveAbsence(error)) { + throw error + } + return undefined + } +} + +type MoveResult = { retained: number; failed: unknown[] } + +// Never replaces a file already set aside, so a name kept there keeps waiting in the pending folder. +async function setAside(from: string, aside: string, item: BigIntStats): Promise { + await mkdir(dirname(aside), { recursive: true }) + try { + return await moveEntry(from, aside, item) + } catch (error) { + if ((error as NodeJS.ErrnoException | null)?.code === 'EEXIST') { + return false + } + throw error + } +} + +const HARD_LINK_UNSUPPORTED = new Set(['EPERM', 'ENOTSUP', 'EOPNOTSUPP', 'EMLINK', 'EXDEV']) + +// A plain rename replaces an existing file (and on Windows a racing one), and a copy would detach +// a live writer's handle, so a file moves only by a new name for its inode; false keeps it in place. +async function moveEntry(from: string, to: string, item: BigIntStats): Promise { + if (item.isDirectory()) { + // A directory rename never replaces a non-empty one. + await renameFileWithWindowsRetryAsync(from, to) + return true + } + try { + await (item.isSymbolicLink() ? symlink(await readlink(from), to) : link(from, to)) + } catch (error) { + const code = (error as NodeJS.ErrnoException | null)?.code ?? '' + if (HARD_LINK_UNSUPPORTED.has(code) && item.isFile()) { + return false + } + throw error + } + await unlink(from) + return true +} + +// One locked entry must not prevent merging the rest. +async function moveTree( + source: string, + destination: string, + result: MoveResult, + options: MoveOptions +): Promise { + for (const name of (await readdir(source)).sort(options.entryOrder)) { + const from = join(source, name) + const to = join(destination, name) + try { + const [item, existing] = await Promise.all([ + lstat(from, { bigint: true }), + lstatIfPresent(to) + ]) + const aside = !existing && item.isFile() ? await options.setAsidePath?.(from, to) : null + if (aside) { + if (!(await setAside(from, aside, item))) { + result.retained += 1 + } + } else if ( + !existing && + item.isDirectory() && + options.movesWhole && + !(await options.movesWhole(from)) + ) { + await mkdir(to, { recursive: true }) + await moveTree(from, to, result, options) + } else if (!existing) { + await options.beforeShare?.(from) + try { + if (!(await moveEntry(from, to, item))) { + result.retained += 1 + } + } catch (error) { + // Another merge into the same destination created this folder first: merge into it. + if (!item.isDirectory() || !(await lstatIfPresent(to))?.isDirectory()) { + throw error + } + await moveTree(from, to, result, options) + } + } else if (item.isDirectory() && existing.isDirectory()) { + await moveTree(from, to, result, options) + } else if (item.isFile() && item.dev === existing.dev && item.ino === existing.ino) { + // A hard link to the shared file: dropping this name loses no bytes. + await options.beforeShare?.(from) + await unlink(from) + } else if (options.dropEmptyConflicts && item.isFile() && item.size === 0n) { + await unlink(from) + } else { + result.retained += 1 + } + } catch (error) { + result.failed.push(error) + } + } + if ((await readdir(source)).length === 0) { + await rmdir(source) + } +} + +async function drain( + pending: string, + destination: string, + args: MergedDirectoryLinkArgs +): Promise { + const result: MoveResult = { retained: 0, failed: [] } + await moveTree(pending, destination, result, args) + if (result.failed.length > 0) { + args.onWarning(result.failed[0]) + } + if (result.retained > 0) { + const detail = `${result.retained} conflicting entries kept in ${pending}` + args.onWarning(new DirectoryLinkError('retained-conflict', detail)) + } +} + +function crossFilesystem(): DirectoryLinkError { + return new DirectoryLinkError('cross-filesystem', 'History stays private across filesystems') +} + +/** Link `destination` in place of `source`, moving its entries there; existing names stay shared. */ +export async function mergeDirectoryIntoLink( + args: MergedDirectoryLinkArgs +): Promise { + const { source, destination } = args + const pending = `${source}${MERGE_PENDING_SUFFIX}` + await mkdir(destination, { recursive: true }) + const destinationDevice = (await stat(destination, { bigint: true })).dev + const sameDevice = async (file: string): Promise => + (await stat(file, { bigint: true })).dev === destinationDevice + if ((await lstatIfPresent(pending))?.isDirectory()) { + if (!(await sameDevice(pending))) { + if (!(await lstatIfPresent(source))) { + await renameFileWithWindowsRetryAsync(pending, source) + } + throw crossFilesystem() + } + try { + await drain(pending, destination, args) + } catch (error) { + // Preserve unreadable leftovers without preventing link publication. + args.onWarning(error) + } + } + const current = await lstatIfPresent(source) + if (current?.isSymbolicLink()) { + return (await realpath(source)) === (await realpath(destination)) ? 'unchanged' : 'user-owned' + } + if (current && !current.isDirectory()) { + return 'user-owned' + } + if (current) { + if (!(await sameDevice(source))) { + throw crossFilesystem() + } + if (await lstatIfPresent(pending)) { + // Earlier conflicts wait there (sharing was turned off and on): merge straight in, then set + // the new conflicts beside the old ones so the folder can still become a link. + await drain(source, destination, args) + if (await lstatIfPresent(source)) { + await moveTree(source, pending, { retained: 0, failed: [] }, {}) + } + if (await lstatIfPresent(source)) { + // A name conflicting in both places: set this version aside in its own folder. + await renameFileWithWindowsRetryAsync( + source, + `${source}.${Date.now()}${MERGE_PENDING_SUFFIX}` + ) + } + } else { + await renameFileWithWindowsRetryAsync(source, pending) + } + } + try { + // Junctions avoid Windows symlink privilege requirements. + await symlink(destination, source, args.platform === 'win32' ? 'junction' : 'dir') + } catch (error) { + if (current) { + // Something recreated the folder mid-swap (such as a starting CLI); put its entries back. + if (!(await lstatIfPresent(source))) { + await renameFileWithWindowsRetryAsync(pending, source) + } else if ((await lstatIfPresent(source))?.isDirectory()) { + await moveTree(pending, source, { retained: 0, failed: [] }, {}) + } + } + throw new DirectoryLinkError('link-failed', String(error)) + } + if (current) { + await drain(pending, destination, args) + } + return 'linked' +} + +/** Whether `source` is a link (or junction) that resolves to `destination`. */ +export async function isLinkedDirectory(source: string, destination: string): Promise { + if (!(await lstatIfPresent(source))?.isSymbolicLink()) { + return false + } + try { + return (await realpath(source)) === (await realpath(destination)) + } catch (error) { + if (isDefinitiveAbsence(error)) { + return false + } + throw error + } +} + +/** Remove a link without following it; a Windows junction is removed as a directory. */ +export async function removeDirectoryLink(path: string): Promise { + try { + await unlink(path) + } catch { + await rmdir(path) + } +} diff --git a/src/main/startup/agent-accounts/codex-account-restart-resume.test.ts b/src/main/startup/agent-accounts/codex-account-restart-resume.test.ts index c665233f8d7..6123ac41fca 100644 --- a/src/main/startup/agent-accounts/codex-account-restart-resume.test.ts +++ b/src/main/startup/agent-accounts/codex-account-restart-resume.test.ts @@ -9,6 +9,7 @@ vi.mock('../main-process-state', () => ({ mainProcessState: { codexRuntimeHome: { isHostSystemDefaultRealHomeSelected: () => false, + prepareAccountHistoryForPinnedHome: async () => {}, getHostCodexHomePathsForSessionDiscovery: () => [homes.original, homes.selected], resolveSelectedHostAccountCodexHomePathForResume: () => homes.selected || null }, diff --git a/src/main/startup/quit/main-process-quit-pending-writer.test.ts b/src/main/startup/quit/main-process-quit-pending-writer.test.ts index c2127141428..e65be0ca83e 100644 --- a/src/main/startup/quit/main-process-quit-pending-writer.test.ts +++ b/src/main/startup/quit/main-process-quit-pending-writer.test.ts @@ -67,8 +67,8 @@ const dependencies: Record> = { '../../codex/history/index/codex-state-db-backfill-recovery': { stopCodexStateDbBackfillRecoveries: settled() }, - '../../codex/history/bridges/codex-account-session-bridge': { - stopCodexAccountSessionBridges: idle() + '../../codex/history/shared-store/codex-account-history-index': { + stopCodexAccountHistoryWork: idle() }, '../../git/refs/maintenance/local-repo-ref-maintenance': { awaitPackedRefsLockRelease: settled() diff --git a/src/main/startup/quit/main-process-quit-update-veto.test.ts b/src/main/startup/quit/main-process-quit-update-veto.test.ts index 053177162a0..e14c803a1ee 100644 --- a/src/main/startup/quit/main-process-quit-update-veto.test.ts +++ b/src/main/startup/quit/main-process-quit-update-veto.test.ts @@ -39,7 +39,7 @@ const dependencyExports: [string, string[]][] = [ '../../codex/history/index/codex-state-db-backfill-recovery', ['stopCodexStateDbBackfillRecoveries'] ], - ['../../codex/history/bridges/codex-account-session-bridge', ['stopCodexAccountSessionBridges']], + ['../../codex/history/shared-store/codex-account-history-index', ['stopCodexAccountHistoryWork']], ['../../git/refs/maintenance/local-repo-ref-maintenance', ['awaitPackedRefsLockRelease']], ['../../workspaces/removal/worktree-background-removal', ['stopBackgroundWorktreeRemovals']], ['./quit-teardown-deadline', ['settleTeardownWithinDeadline', 'settleWithinMs']], diff --git a/src/main/startup/quit/main-process-quit.ts b/src/main/startup/quit/main-process-quit.ts index 4b52b74cba7..63812a5ab6f 100644 --- a/src/main/startup/quit/main-process-quit.ts +++ b/src/main/startup/quit/main-process-quit.ts @@ -17,7 +17,7 @@ import { shutdownPairedRuntimeBrowserClientHosts } from '../../browser/client-ho import { disposeDesktopBrowserBridge } from '../../ipc/browser/desktop-browser-bridge-wiring' import { browserManager } from '../../browser/guest-lifecycle/manager/browser-manager' import { stopCodexStateDbBackfillRecoveries } from '../../codex/history/index/codex-state-db-backfill-recovery' -import { stopCodexAccountSessionBridges } from '../../codex/history/bridges/codex-account-session-bridge' +import { stopCodexAccountHistoryWork } from '../../codex/history/shared-store/codex-account-history-index' import { awaitPackedRefsLockRelease } from '../../git/refs/maintenance/local-repo-ref-maintenance' import { worktreeRemovalLifetime } from '../../workspaces/removal/worktree-removal-lifetime' import { settleTeardownWithinDeadline, settleWithinMs } from './quit-teardown-deadline' @@ -138,7 +138,7 @@ function installWillQuitHandler(): void { state.pluginUpdateReviewNotices = null const pluginHostShutdown = state.pluginService?.dispose() ?? Promise.resolve() const codexBackfillRecoveryShutdown = stopCodexStateDbBackfillRecoveries() - stopCodexAccountSessionBridges() + stopCodexAccountHistoryWork() // Stamp resume reasons before stopping sessions; an update restart was not a user quit. setStructuredAgentSessionTeardownTrigger(updateQuitInProgress ? 'update' : 'quit') const structuredAgentSessionShutdown = stopStructuredAgentSessionRuntime() diff --git a/src/main/wsl/execution/wsl-guest-node-helper.ts b/src/main/wsl/execution/wsl-guest-node-helper.ts new file mode 100644 index 00000000000..021adea1661 --- /dev/null +++ b/src/main/wsl/execution/wsl-guest-node-helper.ts @@ -0,0 +1,79 @@ +import { access } from 'node:fs/promises' +import { join } from 'node:path' +import { getAppEnvironment } from '../../../shared/app-lifecycle/app-environment' +import { ensureWslPinnedRuntime } from './wsl-pinned-runtime' +import { relayBundleCandidates } from './relay-bundle-paths' +import { describeWslProcessFailure } from './wsl-process-failure' +import { runWslProcess, type WslSpec } from './wsl-runner' + +async function runGuest( + distro: string, + spec: WslSpec, + timeoutMs = 15_000, + maxOutputBytes = 256 * 1024 +): Promise { + const result = await runWslProcess({ ...spec, distro, timeoutMs, maxOutputBytes }) + if (result.code !== 0 || result.timedOut) { + throw new Error(`WSL ${distro}: ${describeWslProcessFailure(result)}`) + } + return result.stdout.trim() +} + +async function findBundledHelper(filename: string): Promise { + for (const directory of relayBundleCandidates('wsl')) { + const candidate = join(directory, filename) + if ( + await access(candidate).then( + () => true, + () => false + ) + ) { + return candidate + } + } + return null +} + +/** Runs a bundled guest helper on Orca's pinned Node inside the distro; returns its stdout. */ +export async function runWslGuestNodeHelper(args: { + distro: string + helperFilename: string + /** Names the helper in the error when the bundle lacks it. */ + helperLabel: string + helperArgs: readonly string[] + timeoutMs: number + maxOutputBytes?: number + /** Throws once the distro stopped; `wsl -d` would boot it again. */ + requireRunning?: () => Promise +}): Promise { + const helper = await findBundledHelper(args.helperFilename) + if (!helper) { + throw new Error(`The bundled WSL ${args.helperLabel} helper is missing. Reinstall Orca.`) + } + const requireRunning = args.requireRunning ?? (async () => {}) + const run = async (spec: WslSpec, timeoutMs?: number) => { + await requireRunning() + return runGuest(args.distro, spec, timeoutMs) + } + const node = await ensureWslPinnedRuntime( + run, + join(getAppEnvironment().getPath('userData'), 'orcad-artifacts'), + AbortSignal.timeout(180_000) + ) + const guestHelper = await run({ + program: 'wslpath', + args: ['-a', '-u', helper], + loginPath: 'none' + }) + await requireRunning() + return runGuest( + args.distro, + { + program: '/usr/bin/env', + args: ['-u', 'NODE_OPTIONS', node, guestHelper, ...args.helperArgs], + loginPath: 'none' + }, + args.timeoutMs, + args.maxOutputBytes + ) +} diff --git a/src/shared/remote-runtime/orcad-distribution/remote-bundle-artifacts.ts b/src/shared/remote-runtime/orcad-distribution/remote-bundle-artifacts.ts index b12b097b2c2..835532d5d80 100644 --- a/src/shared/remote-runtime/orcad-distribution/remote-bundle-artifacts.ts +++ b/src/shared/remote-runtime/orcad-distribution/remote-bundle-artifacts.ts @@ -16,5 +16,7 @@ export type RemoteBundlePlatform = (typeof REMOTE_BUNDLE_PLATFORMS)[number] export const OPENCODE_SQLITE_READER_FILENAME = 'opencode-sqlite-reader.cjs' /** Built into the WSL guest bundle dir (out/relay/wsl). */ export const WSL_CLAUDE_PROFILE_HELPER_FILENAME = 'claude-profile-wsl.cjs' +/** Built into the WSL guest bundle dir; links a Codex account's history into the distro's ~/.codex. */ +export const WSL_CODEX_ACCOUNT_HISTORY_HELPER_FILENAME = 'codex-account-history-wsl.cjs' /** Built into the WSL guest bundle dir; runs the owned OpenCode text task inside the distro. */ export const OPENCODE_WSL_TEXT_TASK_FILENAME = 'opencode-text-task-wsl.cjs'