From e2fa611916a6d550fb7417d443c7483bf5d4496c Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:22:39 -0700 Subject: [PATCH] feat(native-chat): chat scratch folders expire after 30 days unused (#28394) * fix(files): never write through a dangling symlink A path that doesn't exist yet was authorized by walking up to the nearest existing folder. A dangling symlink fails realpath with ENOENT, so the walk stepped over it and approved the link's own path; the write then followed the link and created its target outside the allowed folders. The walk now continues from the link's target, and returns that real destination. * refactor(native-chat): rename the chat visuals folder to the chat scratch folder The per-chat folder agents write into is the chat's scratch space, not only a visuals folder. Its root becomes /chat-scratch/ (the hash is unchanged) and the agent variable becomes ORCA_CHAT_SCRATCH_DIR. The old ORCA_CHAT_VISUALS_DIR is never set and is still stripped from what every agent child inherits, Pi included. The visuals skill names the new variable and says the folder is temporary. * test(native-chat): keep the delivery test to the rename The launch parity snapshot already pins that every agent drops the pre-rename variable. * feat(native-chat): chat scratch folders expire after 30 days unused A chat's scratch folder was removed only when its record was gone or its local workspace was proven deleted, so the folder of a long-idle chat lived forever. The sweep now expires a chat's folder once its newest journal row and the newest entry anywhere in the folder (complete, no-follow scan) are older than 30 days, unless the host holds a running agent, a delivery or an open dispatch for the chat. The final recheck and a rename to .expired run in the chat's session lane and then a per-folder lock that Orca scratch saves (desktop IPC and host RPC) also take; the delete runs outside both. A save that settles after the age scan marks that expiry stale. Leftover .expired folders and the pre-rename native-chat-visuals root are removed. The workspace-removed rule and its catalog verdict are deleted. * fix(files): refuse any dangling symlink in the missing-path walk Following a dangling link by hand resolved relative targets against the lexical parent, not the link's real folder: through a folder link the walk could loop forever or authorize the wrong file. Refuse instead: realpath can't say where a dangling link leads, but a write would follow it. * test(files): drop the loop case that duplicated the dangling-link refusal * refactor(native-chat): trim the scratch expiry and test the real live-work veto - Remove the other-profile catalog options (running profile id, never-written profiles) that only the deleted workspace verdict used; history cleanup's reader is unchanged. - The sweep keeps its original remove-and-warn block; an unused chat's folder is parked by one function. Drop the unused `now` option and the candidate identity check. - The live-agent case runs against the host's real runWhileExecutionIdle. * style(native-chat): keep the session host within its line limit after the main sync * fix(native-chat): every scratch save and every overlapping sweep share the folder lock Binary (base64) runtime saves now take the scratch folder lock like text saves, and each folder keeps a set of expiry candidates so one sweep finishing no longer hides a save from another. --- config/scripts/ci-shard-timings.json | 1 - .../scripts/vitest-sqlite-runtime-files.mjs | 1 + .../filesystem/filesystem-write-handlers.ts | 6 +- .../journal-row-table.ts | 11 + ...red-agent-session-conversation-lifetime.ts | 26 +- .../structured-agent-session-host.ts | 5 +- .../native-chat-scratch-folder-lock.ts | 72 +++++ .../native-chat/native-chat-scratch-folder.ts | 6 + .../native-chat-visuals-sweep.test.ts | 157 +++++++--- .../native-chat/native-chat-visuals-sweep.ts | 145 +++++++--- .../other-profile-workspace-catalog.test.ts | 25 -- .../other-profile-workspace-catalog.ts | 47 +-- src/main/runtime/files/file-mutations.ts | 20 +- ...ive-chat-visuals-workspace-verdict.test.ts | 271 ------------------ .../native-chat-visuals-workspace-verdict.ts | 204 ------------- .../runtime/orca-runtime-get-worktree-ps.ts | 10 +- .../structured-agent-session-runtime.ts | 15 +- 17 files changed, 364 insertions(+), 658 deletions(-) create mode 100644 src/main/native-chat/native-chat-scratch-folder-lock.ts delete mode 100644 src/main/runtime/native-chat-visuals-workspace-verdict.test.ts delete mode 100644 src/main/runtime/native-chat-visuals-workspace-verdict.ts diff --git a/config/scripts/ci-shard-timings.json b/config/scripts/ci-shard-timings.json index ead2162455b..19d51b803bb 100644 --- a/config/scripts/ci-shard-timings.json +++ b/config/scripts/ci-shard-timings.json @@ -4456,7 +4456,6 @@ "src/main/runtime/mobile-subscribe-idle-pane-hydrate.test.ts": 5902, "src/main/runtime/mobile-subscribe-integration.test.ts": 22684, "src/main/runtime/multi-client-navigation-isolation.integration.test.ts": 5282, - "src/main/runtime/native-chat-visuals-workspace-verdict.test.ts": 326, "src/main/runtime/omp-readiness-transcripts.test.ts": 7797, "src/main/runtime/opencode-agent-row-enter-transcript.test.ts": 562, "src/main/runtime/opencode-agent-row-structure-transcript.test.ts": 347, diff --git a/config/scripts/vitest-sqlite-runtime-files.mjs b/config/scripts/vitest-sqlite-runtime-files.mjs index c21975ecdc0..540856c7122 100644 --- a/config/scripts/vitest-sqlite-runtime-files.mjs +++ b/config/scripts/vitest-sqlite-runtime-files.mjs @@ -152,6 +152,7 @@ export const SQLITE_RUNTIME_INCLUDE = [ 'src/main/native-chat/agent-session-wire/structured-agent-session-refusal-retry.test.ts', 'src/main/native-chat/agent-session-wire/structured-agent-session-repeat-press.test.ts', 'src/main/native-chat/agent-session-wire/structured-agent-session-repeated-stop.test.ts', + 'src/main/native-chat/native-chat-visuals-sweep.test.ts', 'src/main/native-chat/agent-session-wire/structured-agent-session-resend-answer.test.ts', 'src/main/native-chat/agent-session-wire/structured-agent-session-replaced-server-turn.test.ts', 'src/main/native-chat/agent-session-wire/structured-agent-session-resend-ledger-refusal.test.ts', diff --git a/src/main/ipc/filesystem/filesystem-write-handlers.ts b/src/main/ipc/filesystem/filesystem-write-handlers.ts index ee9646ac3e6..6bf20868f28 100644 --- a/src/main/ipc/filesystem/filesystem-write-handlers.ts +++ b/src/main/ipc/filesystem/filesystem-write-handlers.ts @@ -14,6 +14,8 @@ import { registerFilesystemMutationHandlers } from '../filesystem-mutations' import { registerAgentSessionAttachmentUploadHandlers } from '../agent-session-attachment-upload-ipc' import type { FilesystemHandlerContext } from './filesystem-handler-context' import { assertLocalWriteTargetIsRegularFile } from './local-regular-file-read' +import { runNativeChatScratchSave } from '../../native-chat/native-chat-scratch-folder-lock' +import { getProfileUserDataPath } from '../../orca-profiles/profile-storage-paths' export function registerFilesystemWriteHandlers(context: FilesystemHandlerContext): void { const { store } = context @@ -50,7 +52,9 @@ export function registerFilesystemWriteHandlers(context: FilesystemHandlerContex } } await assertLocalWriteTargetIsRegularFile(filePath) - await writeFile(filePath, args.content, 'utf-8') + await runNativeChatScratchSave(getProfileUserDataPath(), filePath, () => + writeFile(filePath, args.content, 'utf-8') + ) } ) diff --git a/src/main/native-chat/agent-session-journal/journal-row-table.ts b/src/main/native-chat/agent-session-journal/journal-row-table.ts index 7ff185857cf..8a108c5f80f 100644 --- a/src/main/native-chat/agent-session-journal/journal-row-table.ts +++ b/src/main/native-chat/agent-session-journal/journal-row-table.ts @@ -21,12 +21,23 @@ const SELECT_ROWS_AFTER = `SELECT seq, ts, row_json FROM journal_rows WHERE session_id = ? AND epoch = ? AND seq > ? ORDER BY seq ASC` const SELECT_ROWS_AFTER_LIMITED = `${SELECT_ROWS_AFTER} LIMIT ?` const DELETE_EPOCH = 'DELETE FROM journal_rows WHERE session_id = ? AND epoch = ?' +const SELECT_LAST_ACTIVITY = 'SELECT MAX(ts) AS ts FROM journal_rows WHERE session_id = ?' export function readJournalSessionEpoch(db: Database.Database, sessionId: string): string | null { const epoch = db.prepare(SELECT_EPOCH).get(sessionId)?.epoch return typeof epoch === 'string' ? epoch : null } +/** When the chat's newest row was saved, across every epoch; null for a chat with no rows. One + * aggregate over the chat's key range, so no row body reaches this process. */ +export function readJournalSessionLastActivityAt( + db: Database.Database, + sessionId: string +): number | null { + const ts = db.prepare(SELECT_LAST_ACTIVITY).get(sessionId)?.ts + return typeof ts === 'number' ? ts : null +} + /** Points the chat at `epoch`. Only an epoch change writes this row. */ export function publishJournalSessionEpoch( db: Database.Database, diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-lifetime.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-lifetime.ts index 431ca9e761d..2574f6c3a8a 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-lifetime.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-lifetime.ts @@ -77,6 +77,13 @@ export function createStructuredAgentSessionConversationLifetime(host: { return closed } + const hasOpenDispatch = (sessionId: string): boolean => { + const record = deps().store.getRecord(sessionId) + return record !== null && deps().hasOpenDispatch?.(record) === true + } + const providerHoldsDispatch = (sessionId: string): boolean => + deps().adapter.holdsDispatch?.(sessionId) === true + const idleSweep = new StructuredAgentSessionIdleSweep({ sessions, serialize, @@ -84,11 +91,8 @@ export function createStructuredAgentSessionConversationLifetime(host: { isDisposed: () => disposed, deliveryActive: host.deliveryActive, childWork: host.readChildWork, - hasOpenDispatch: (sessionId) => { - const record = deps().store.getRecord(sessionId) - return record !== null && deps().hasOpenDispatch?.(record) === true - }, - providerHoldsDispatch: (sessionId) => deps().adapter.holdsDispatch?.(sessionId) === true, + hasOpenDispatch, + providerHoldsDispatch, // The host puts an idle agent to rest: a turn it cuts short is news, not the user's Stop. stopAgent: (sessionId) => stopAgent(sessionId, { cause: 'evict', resting: true }), // A host stop: the child's end rejects what it was handed and writes the one error row, and @@ -130,6 +134,18 @@ export function createStructuredAgentSessionConversationLifetime(host: { }) ) }, + /** Runs `task` in the chat's lane, so no start interleaves with it, unless quit has begun or + * the host holds a child, a delivery or a dispatch for the chat; null then. */ + runWhileExecutionIdle: (sessionId: string, task: () => Promise): Promise => + serialize(sessionId, async () => + disposed || + sessions.get(sessionId)?.child || + host.deliveryActive(sessionId) || + providerHoldsDispatch(sessionId) || + hasOpenDispatch(sessionId) + ? null + : await task() + ), /** Quit has begun: nothing opens a conversation or sweeps one after this. */ dispose: (): void => { disposed = true diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts index b5716d6cdfc..3e364c4a8ab 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts @@ -177,8 +177,7 @@ export class StructuredAgentSessionHost { private now = (): number => this.deps.now?.() ?? Date.now() hasSession = (sessionId: string): boolean => this.sessions.has(sessionId) - hasLoadedProviders = (): boolean => - [...this.sessions.values()].some(({ child }) => child !== null) + hasLoadedProviders = () => [...this.sessions.values()].some(({ child }) => child !== null) sessionAgent = (sessionId: string) => this.deps.store.getRecord(sessionId)?.provider ?? null handleAdapterEvent = (event: Parameters[0]) => @@ -232,6 +231,8 @@ export class StructuredAgentSessionHost { showSessionTabs = this.tabs.showSessionTabs setSessionTabVisibility = this.tabs.setSessionTabVisibility notifySessionTabHidden = this.tabs.notifySessionTabHidden + runWhileExecutionIdle = (sessionId: string, task: () => Promise): Promise => + this.lifetime.runWhileExecutionIdle(sessionId, task) /** This runtime holds a chat record, readable or not. */ holdsSessions = (): boolean => this.deps.store.holdsRecords() onSessionsHeld = (listener: () => void): (() => void) => this.deps.store.onFirstRecord(listener) diff --git a/src/main/native-chat/native-chat-scratch-folder-lock.ts b/src/main/native-chat/native-chat-scratch-folder-lock.ts new file mode 100644 index 00000000000..a270f804151 --- /dev/null +++ b/src/main/native-chat/native-chat-scratch-folder-lock.ts @@ -0,0 +1,72 @@ +// Orders an Orca save into a chat's scratch folder against the sweep expiring that folder. Expiry +// renames the folder aside under this lock, so a save either lands first (its candidate mark keeps +// the folder) or runs after and fails on the missing folder, leaving the draft unsaved. A save +// never takes the session lane, so it never waits for an agent start. + +import { relative } from 'node:path' +import { runKeyedSerializedOperation } from '../cli/keyed-promise-queue' +import { + isNativeChatScratchFolderName, + nativeChatScratchRootFor +} from './native-chat-scratch-folder' + +type ExpiryCandidate = { current: boolean } + +const folderQueues = new Map>() +// The age scan runs outside the lock, so a save that settles after it starts makes it stale. +// A set per folder: each runtime install runs its own sweep, so two can scan one folder at once. +const expiryCandidates = new Map>() + +/** The scratch folder `filePath` lies inside, or null for any other path. */ +function scratchFolderNameOf(stateDirectory: string, filePath: string): string | null { + const segments = relative(nativeChatScratchRootFor(stateDirectory), filePath).split(/[\\/]/) + return segments.length > 1 && isNativeChatScratchFolderName(segments[0]) ? segments[0] : null +} + +/** Runs an Orca save; one into a chat scratch folder holds that folder's lock. */ +export function runNativeChatScratchSave( + stateDirectory: string, + filePath: string, + save: () => Promise +): Promise { + const folderName = scratchFolderNameOf(stateDirectory, filePath) + if (!folderName) { + return save() + } + return runKeyedSerializedOperation(folderQueues, folderName, async () => { + try { + return await save() + } finally { + for (const candidate of expiryCandidates.get(folderName) ?? []) { + candidate.current = false + } + } + }) +} + +export function runUnderNativeChatScratchFolderLock( + folderName: string, + task: () => Promise +): Promise { + return runKeyedSerializedOperation(folderQueues, folderName, task) +} + +/** Registered before the sweep's age scan; the scan stays usable only while `isCurrent`. */ +export function trackNativeChatScratchExpiryCandidate(folderName: string): { + isCurrent: () => boolean + dispose: () => void +} { + const candidate: ExpiryCandidate = { current: true } + const candidates = expiryCandidates.get(folderName) ?? new Set() + candidates.add(candidate) + expiryCandidates.set(folderName, candidates) + return { + isCurrent: () => candidate.current, + dispose: () => { + candidates.delete(candidate) + if (candidates.size === 0) { + expiryCandidates.delete(folderName) + } + } + } +} diff --git a/src/main/native-chat/native-chat-scratch-folder.ts b/src/main/native-chat/native-chat-scratch-folder.ts index e52acaf1624..bd82f646b36 100644 --- a/src/main/native-chat/native-chat-scratch-folder.ts +++ b/src/main/native-chat/native-chat-scratch-folder.ts @@ -5,6 +5,7 @@ import { createHash } from 'node:crypto' import { join } from 'node:path' const NATIVE_CHAT_SCRATCH_DIR_NAME = 'chat-scratch' +const FOLDER_NAME = /^[0-9a-f]{32}$/ /** Where every chat's scratch folder on this host lives. */ export function nativeChatScratchRootFor(stateDirectory: string): string { @@ -16,6 +17,11 @@ export function nativeChatScratchFolderName(sessionId: string): string { return createHash('sha256').update(sessionId, 'utf8').digest('hex').slice(0, 32) } +/** A name this host minted for a chat's folder; nothing else under the root is a chat's. */ +export function isNativeChatScratchFolderName(name: string): boolean { + return FOLDER_NAME.test(name) +} + /** * `/chat-scratch/`. Keyed by the session id * alone, the chat record's primary key, so the folder needs no workspace lookup to find or remove; diff --git a/src/main/native-chat/native-chat-visuals-sweep.test.ts b/src/main/native-chat/native-chat-visuals-sweep.test.ts index d5daee4eaa3..6592f3d6735 100644 --- a/src/main/native-chat/native-chat-visuals-sweep.test.ts +++ b/src/main/native-chat/native-chat-visuals-sweep.test.ts @@ -1,16 +1,35 @@ -import { existsSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' +import { + existsSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + statSync, + symlinkSync, + utimesSync, + writeFileSync +} from 'node:fs' +import { writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' -import { join } from 'node:path' +import { basename, join } from 'node:path' import { afterEach, describe, expect, it, vi } from 'vitest' -import type { AgentSessionExecutionLocation } from '../../shared/agent-session-record' -import { LOCAL_EXECUTION_HOST_ID } from '../../shared/execution-host' +import { runKeyedSerializedOperation } from '../cli/keyed-promise-queue' import type { StructuredAgentSessionLogger } from './agent-session-wire/structured-agent-session-logger' import { nativeChatScratchFolderFor, nativeChatScratchRootFor } from './native-chat-scratch-folder' +import { + runNativeChatScratchSave, + trackNativeChatScratchExpiryCandidate +} from './native-chat-scratch-folder-lock' +import { + attach, + hostTestState +} from './agent-session-wire/structured-agent-session-host-test-harness' +import { HOST_TEST_SESSION } from './agent-session-wire/structured-agent-session-host-test-data' import { scheduleNativeChatVisualsSweep, sweepNativeChatVisualsFolders, - type NativeChatVisualsSweepDeps, - type NativeChatVisualsWorkspaceVerdict + type NativeChatVisualsSweepDeps } from './native-chat-visuals-sweep' const scratch: string[] = [] @@ -34,25 +53,36 @@ function visualsFor(state: string, sessionId: string): string { return folder } -const location = (workspaceId: string): AgentSessionExecutionLocation => ({ - executionHostId: LOCAL_EXECUTION_HOST_ID, - wslDistro: null, - workspaceId, - workspaceKind: 'git-worktree' -}) +const DAY_MS = 24 * 60 * 60_000 + +/** Sets the mtime of `path` and everything under it to `daysAgo`, children before parents. */ +function age(path: string, daysAgo = 40): void { + if (statSync(path).isDirectory()) { + for (const name of readdirSync(path)) { + age(join(path, name), daysAgo) + } + } + const at = new Date(Date.now() - daysAgo * DAY_MS) + utimesSync(path, at, at) +} function deps( state: string, held: readonly string[] | null, - verdicts: Record = {} + facts: { journal?: Record } = {} ): NativeChatVisualsSweepDeps & { logger: { warn: ReturnType> } + lanes: Map> } { + const lanes = new Map>() return { stateDirectory: state, listHeldSessionIds: () => held, - locationOf: (sessionId) => (held?.includes(sessionId) ? location(`ws-${sessionId}`) : null), - workspaceVerdicts: () => async (where) => verdicts[where.workspaceId] ?? 'present', + lastJournalActivityAt: (sessionId) => facts.journal?.[sessionId] ?? null, + // Stands in for the host's lane: one task at a time per chat. + runWhileExecutionIdle: (sessionId: string, task: () => Promise) => + runKeyedSerializedOperation(lanes, sessionId, task), + lanes, logger: { warn: vi.fn(), error: vi.fn() } } } @@ -72,32 +102,86 @@ describe('the visuals folder sweep', () => { expect(existsSync(orphan)).toBe(false) }) - it('removes a chat folder only when its local workspace is proven removed', async () => { + it('expires a chat unused for 30 days and keeps one with recent use', async () => { const state = stateDirectory() - const removed = visualsFor(state, 'chat-a') - const unverifiable = visualsFor(state, 'chat-b') - const present = visualsFor(state, 'chat-c') - await sweepNativeChatVisualsFolders( - deps(state, ['chat-a', 'chat-b', 'chat-c'], { - 'ws-chat-a': 'removed', - 'ws-chat-b': 'unverifiable' + const unused = visualsFor(state, 'chat-unused') + const talkedTo = visualsFor(state, 'chat-journal') + const edited = visualsFor(state, 'chat-nested') + mkdirSync(join(edited, 'drafts')) + writeFileSync(join(edited, 'drafts', 'v2.html'), '

v2

') + const legacy = join(state, 'native-chat-visuals', 'a'.repeat(32)) + mkdirSync(legacy, { recursive: true }) + age(nativeChatScratchRootFor(state)) + const fresh = new Date() + utimesSync(join(edited, 'drafts', 'v2.html'), fresh, fresh) + + const result = await sweepNativeChatVisualsFolders( + deps(state, ['chat-unused', 'chat-journal', 'chat-nested'], { + journal: { 'chat-unused': Date.now() - 31 * DAY_MS, 'chat-journal': Date.now() - DAY_MS } }) ) - expect(existsSync(removed)).toBe(false) - expect(existsSync(unverifiable)).toBe(true) - expect(existsSync(present)).toBe(true) + + expect(result).toEqual({ removed: 1, failed: 0 }) + expect(existsSync(unused)).toBe(false) + expect(existsSync(`${unused}.expired`)).toBe(false) + expect(existsSync(talkedTo)).toBe(true) + expect(existsSync(edited)).toBe(true) + expect(existsSync(join(state, 'native-chat-visuals'))).toBe(false) }) - it('keeps a held chat whose workspace verdict throws', async () => { + it('keeps a chat past 30 days while the host runs its agent', async () => { + await attach() + const state = stateDirectory() + const folder = visualsFor(state, HOST_TEST_SESSION) + age(folder) + const result = await sweepNativeChatVisualsFolders({ + ...deps(state, [HOST_TEST_SESSION]), + runWhileExecutionIdle: hostTestState().host.runWhileExecutionIdle + }) + expect(result).toEqual({ removed: 0, failed: 0 }) + expect(existsSync(folder)).toBe(true) + }) + + it('gives a start queued behind expiry a fresh folder', async () => { const state = stateDirectory() const folder = visualsFor(state, 'chat-a') - await sweepNativeChatVisualsFolders({ - ...deps(state, ['chat-a']), - workspaceVerdicts: () => async () => { - throw new Error('catalog unavailable') + age(folder) + const sweep = deps(state, ['chat-a']) + const result = await sweepNativeChatVisualsFolders({ + ...sweep, + runWhileExecutionIdle: (sessionId, task) => { + const expiry = sweep.runWhileExecutionIdle(sessionId, task) + // The start's own lane task, as the host's agent start runs it. + void runKeyedSerializedOperation(sweep.lanes, sessionId, async () => { + mkdirSync(folder, { recursive: true }) + writeFileSync(join(folder, 'new.html'), '

new

') + }) + return expiry } }) - expect(existsSync(folder)).toBe(true) + expect(result).toEqual({ removed: 1, failed: 0 }) + expect(readdirSync(folder)).toEqual(['new.html']) + expect(existsSync(`${folder}.expired`)).toBe(false) + }) + + it('keeps a folder saved into after its age scan', async () => { + const state = stateDirectory() + const folder = visualsFor(state, 'chat-a') + age(folder) + const sweep = deps(state, ['chat-a']) + await sweepNativeChatVisualsFolders({ + ...sweep, + runWhileExecutionIdle: async (sessionId, task) => { + // An overlapping sweep of the same folder ends first; this sweep's candidate must survive. + trackNativeChatScratchExpiryCandidate(basename(folder)).dispose() + // Settles between the scan and the locked recheck, which does not scan again. + await runNativeChatScratchSave(state, join(folder, 'chart.html'), () => + writeFile(join(folder, 'chart.html'), '

saved

') + ) + return sweep.runWhileExecutionIdle(sessionId, task) + } + }) + expect(readFileSync(join(folder, 'chart.html'), 'utf8')).toBe('

saved

') }) it('removes nothing when the held chats cannot be read', async () => { @@ -177,13 +261,4 @@ describe('the visuals folder sweep', () => { await sweepNativeChatVisualsFolders(deps(state, [])) expect(existsSync(join(elsewhere, 'a'.repeat(32)))).toBe(true) }) - - it('takes one verdict snapshot per run', async () => { - const state = stateDirectory() - visualsFor(state, 'chat-a') - visualsFor(state, 'chat-b') - const workspaceVerdicts = vi.fn(() => async () => 'present' as const) - await sweepNativeChatVisualsFolders({ ...deps(state, ['chat-a', 'chat-b']), workspaceVerdicts }) - expect(workspaceVerdicts).toHaveBeenCalledOnce() - }) }) diff --git a/src/main/native-chat/native-chat-visuals-sweep.ts b/src/main/native-chat/native-chat-visuals-sweep.ts index 8306a2aeb9b..01cfd046153 100644 --- a/src/main/native-chat/native-chat-visuals-sweep.ts +++ b/src/main/native-chat/native-chat-visuals-sweep.ts @@ -1,40 +1,49 @@ // How a chat's scratch folder dies. The host never deletes a chat record, so no deletion event -// exists to hang cleanup on; instead each folder's life is derived from the records every run: +// exists to hang cleanup on; instead each folder's life is re-derived every run: // 1. a folder no chat record (readable or not) maps to is removed, so any future record // deletion cleans up for free; -// 2. a folder whose chat ran in a local workspace this host can prove was removed is removed. -// Anything unproven is kept. A failure is logged and never blocks anything else. +// 2. a folder whose chat went unused for the retention window is removed: its newest journal row +// and everything in the folder are older than that, and the host holds no execution for it. +// Rule 2 decides again under the chat's lane and then its folder lock, so a start or a save that +// lands after the first look keeps the folder. A failure is logged and never blocks anything else. import type { Dirent } from 'node:fs' -import { lstat, readdir, rm } from 'node:fs/promises' +import { lstat, readdir, rename, rm } from 'node:fs/promises' import { join } from 'node:path' -import type { AgentSessionExecutionLocation } from '../../shared/agent-session-record' import { isDefinitiveAbsence } from '../../shared/definitive-filesystem-absence' import { yieldToEventLoop } from '../../shared/event-loop-yield' import type { StructuredAgentSessionLogger } from './agent-session-wire/structured-agent-session-logger' -import { nativeChatScratchFolderName, nativeChatScratchRootFor } from './native-chat-scratch-folder' +import { + isNativeChatScratchFolderName, + nativeChatScratchFolderName, + nativeChatScratchRootFor +} from './native-chat-scratch-folder' +import { + runUnderNativeChatScratchFolderLock, + trackNativeChatScratchExpiryCandidate +} from './native-chat-scratch-folder-lock' -/** `removed` only on positive proof from the host that owns the workspace. */ -export type NativeChatVisualsWorkspaceVerdict = 'present' | 'removed' | 'unverifiable' +/** How long a chat's scratch files outlive its last use. */ +export const NATIVE_CHAT_SCRATCH_RETENTION_MS = 30 * 24 * 60 * 60_000 export type NativeChatVisualsSweepDeps = { stateDirectory: string /** Every chat this host holds a row for, readable or not; null when that can't be trusted. */ listHeldSessionIds: () => readonly string[] | null - locationOf: (sessionId: string) => AgentSessionExecutionLocation | null - /** Called at most once per run, so one run judges every chat against one catalog snapshot. - * Absent: only rule 1 runs. */ - workspaceVerdicts?: () => ( - location: AgentSessionExecutionLocation - ) => Promise + /** When the chat's newest journal row was saved; null when it has none. */ + lastJournalActivityAt: (sessionId: string) => number | null + /** Runs `task` in the chat's lane unless the host holds execution for it; null when it does. */ + runWhileExecutionIdle: (sessionId: string, task: () => Promise) => Promise logger: StructuredAgentSessionLogger remove?: (path: string) => Promise } export type NativeChatVisualsSweepResult = { removed: number; failed: number } -// Only names this host minted: anything else in the folder is never touched. -const FOLDER_NAME = /^[0-9a-f]{32}$/ +// Renamed under the locks and deleted after them; a cut-short delete leaves it for a later run. +const EXPIRED_SUFFIX = '.expired' +// The pre-rename root: native chat had no real users, and an update restarts every old child. +const LEGACY_ROOT_NAME = 'native-chat-visuals' // Bounds one run's disk work; whatever is left goes in a later run. const MAX_REMOVALS_PER_RUN = 200 const YIELD_EVERY = 32 @@ -60,6 +69,13 @@ export async function sweepNativeChatVisualsFolders( ): Promise { const result = { removed: 0, failed: 0 } const root = nativeChatScratchRootFor(deps.stateDirectory) + await rm(join(deps.stateDirectory, LEGACY_ROOT_NAME), { recursive: true, force: true }).catch( + (error: unknown) => + deps.logger.warn('the pre-rename chat visuals root could not be removed', { + scope: 'nativeChatVisuals.sweep', + error + }) + ) // Listed before the records are read: a folder is created only after its chat's record, so // every folder seen here already has its record in the snapshot below. const entries = await listFolders(root) @@ -71,9 +87,7 @@ export async function sweepNativeChatVisualsFolders( held.map((sessionId) => [nativeChatScratchFolderName(sessionId), sessionId]) ) const remove = deps.remove ?? ((path: string) => rm(path, { recursive: true, force: true })) - let verdict: ReturnType> | undefined - const verdictFor = (location: AgentSessionExecutionLocation) => - (verdict ??= deps.workspaceVerdicts?.())?.(location) + const cutoff = Date.now() - NATIVE_CHAT_SCRATCH_RETENTION_MS let visited = 0 for (const entry of entries) { if (isStopped() || result.removed + result.failed >= MAX_REMOVALS_PER_RUN) { @@ -82,18 +96,22 @@ export async function sweepNativeChatVisualsFolders( if (++visited % YIELD_EVERY === 0) { await yieldToEventLoop() } + const expired = entry.name.endsWith(EXPIRED_SUFFIX) + const folderName = expired ? entry.name.slice(0, -EXPIRED_SUFFIX.length) : entry.name // A Dirent describes the entry itself, so a symlink is never followed or removed here. - if (!entry.isDirectory() || !FOLDER_NAME.test(entry.name)) { - continue - } - const sessionId = sessionByFolder.get(entry.name) - const reason = - sessionId === undefined ? 'no-record' : await removedWorkspace(deps, sessionId, verdictFor) - if (!reason) { + if (!entry.isDirectory() || !isNativeChatScratchFolderName(folderName)) { continue } + const sessionId = expired ? undefined : sessionByFolder.get(entry.name) + const reason = expired ? 'expired' : sessionId === undefined ? 'no-record' : 'unused' try { - await remove(join(root, entry.name)) + if ( + sessionId !== undefined && + !(await parkUnusedFolder(deps, root, folderName, sessionId, cutoff)) + ) { + continue + } + await remove(join(root, sessionId === undefined ? entry.name : folderName + EXPIRED_SUFFIX)) result.removed += 1 } catch (error) { result.failed += 1 @@ -108,26 +126,69 @@ export async function sweepNativeChatVisualsFolders( return result } -async function removedWorkspace( - deps: Pick, +/** Renames a chat's folder to `.expired` under the chat's lane and then its folder lock; + * false when the chat was used since `cutoff`, holds live work, or a save landed after the scan. */ +async function parkUnusedFolder( + deps: NativeChatVisualsSweepDeps, + root: string, + name: string, sessionId: string, - verdictFor: ( - location: AgentSessionExecutionLocation - ) => Promise | undefined -): Promise<'workspace-removed' | null> { - const location = deps.locationOf(sessionId) - if (!location) { - return null - } + cutoff: number +): Promise { + const folder = join(root, name) + const usedSince = (): boolean => (deps.lastJournalActivityAt(sessionId) ?? 0) >= cutoff + // Before the scan, so a save already running when it starts still marks it stale. + const candidate = trackNativeChatScratchExpiryCandidate(name) try { - return (await verdictFor(location)) === 'removed' ? 'workspace-removed' : null - } catch { - return null + if (usedSince() || (await changedSince(folder, cutoff))) { + return false + } + const parked = await deps.runWhileExecutionIdle(sessionId, () => + runUnderNativeChatScratchFolderLock(name, async () => { + if (!candidate.isCurrent() || usedSince()) { + return false + } + await rename(folder, folder + EXPIRED_SUFFIX) + return true + }) + ) + return parked === true + } catch (error) { + if (isDefinitiveAbsence(error)) { + return false + } + throw error + } finally { + candidate.dispose() } } -// Why minutes, not at boot: the first sweep must not compete with session restore and the -// workspace catalog loading; after that, a few runs a day is enough for files nothing reads. +/** Whether the folder or anything under it changed at or after `cutoff`. Every entry is read with + * lstat, so a link is judged as itself and never followed; the scan stops at the first change. */ +async function changedSince(folder: string, cutoff: number): Promise { + const pending = [folder] + let visited = 0 + for (let dir = pending.pop(); dir !== undefined; dir = pending.pop()) { + if ((await lstat(dir)).mtimeMs >= cutoff) { + return true + } + for (const entry of await readdir(dir, { withFileTypes: true })) { + if (++visited % YIELD_EVERY === 0) { + await yieldToEventLoop() + } + const path = join(dir, entry.name) + if (entry.isDirectory()) { + pending.push(path) + } else if ((await lstat(path)).mtimeMs >= cutoff) { + return true + } + } + } + return false +} + +// Why minutes, not at boot: the first sweep must not compete with session restore; after that, a +// few runs a day is enough for files nothing reads. const FIRST_SWEEP_DELAY_MS = 2 * 60_000 const SWEEP_INTERVAL_MS = 6 * 60 * 60_000 diff --git a/src/main/orca-profiles/other-profile-workspace-catalog.test.ts b/src/main/orca-profiles/other-profile-workspace-catalog.test.ts index 43fd24d919b..ae85981096a 100644 --- a/src/main/orca-profiles/other-profile-workspace-catalog.test.ts +++ b/src/main/orca-profiles/other-profile-workspace-catalog.test.ts @@ -287,29 +287,4 @@ describe('readOtherProfileWorkspaceCatalog', () => { ]) expect(readOtherProfileWorkspaceCatalog(root).unreadableProfiles).toBe(1) }) - - it('skips the running profile even when the index already names another as active', () => { - const root = userDataWithProfiles('next', [ - { id: 'running', state: { repos: [{ id: 'repo-running' }] } }, - { id: 'next', state: { repos: [{ id: 'repo-next' }] } } - ]) - const catalog = readOtherProfileWorkspaceCatalog(root, { runningProfileId: 'running' }) - expect(catalog.repoIds).toEqual(new Set(['repo-next'])) - }) - - it('counts a never-written profile as empty only when asked', () => { - const root = userDataWithProfiles('active', [{ id: 'active', state: {} }, { id: 'fresh' }]) - expect(readOtherProfileWorkspaceCatalog(root).unreadableProfiles).toBe(1) - const lenient = readOtherProfileWorkspaceCatalog(root, { neverWrittenIsEmpty: true }) - expect(lenient.unreadableProfiles).toBe(0) - expect(lenient.repoIds.size).toBe(0) - }) - - it('never counts a profile with only backups of its data file as empty', () => { - const root = userDataWithProfiles('active', [{ id: 'active', state: {} }, { id: 'lost' }]) - writeFileSync(join(root, 'profiles', 'lost', 'orca-data.json.bak.0'), '{}') - expect( - readOtherProfileWorkspaceCatalog(root, { neverWrittenIsEmpty: true }).unreadableProfiles - ).toBe(1) - }) }) diff --git a/src/main/orca-profiles/other-profile-workspace-catalog.ts b/src/main/orca-profiles/other-profile-workspace-catalog.ts index 1ba2e91a114..a52f7316e79 100644 --- a/src/main/orca-profiles/other-profile-workspace-catalog.ts +++ b/src/main/orca-profiles/other-profile-workspace-catalog.ts @@ -1,5 +1,4 @@ import { lstatSync, readFileSync } from 'node:fs' -import { isDefinitiveAbsence } from '../../shared/definitive-filesystem-absence' import { folderWorkspaceKey } from '../../shared/workspace-scope' import { getOrcaProfileDataFile, @@ -9,10 +8,6 @@ import { import { getOrcaProfileIndexPath, readProfileIndex } from './profile-index-store' import { readProfileStateDomains } from '../persistence/profile-state/profile-state-domain-reader' import { assertNoRetainedProfileStateExports } from '../persistence/profile-state/profile-state-recovery-required' -import { - PROFILE_STATE_LEGACY_BACKUP_COUNT, - profileStateLegacyBackupPath -} from '../persistence/profile-state/legacy-json/profile-state-legacy-backup-path' /** * Worktree ids owned by Orca profiles OTHER than the running one. @@ -45,20 +40,10 @@ type ProfileWorkspaceIds = { repoIds: Set } -export type OtherProfileCatalogOptions = { - /** The profile this process runs, whose catalog the caller reads live. Defaults to the index's - * active profile, which a profile switch rewrites before the running one exits. */ - runningProfileId?: string - /** A profile with no state database, sidecar or data file at all has never been written and - * holds nothing; by default it counts as unreadable like a missing file anywhere else. */ - neverWrittenIsEmpty?: boolean -} - /** The workspaces and projects every profile other than the running one holds. Any profile whose * state can't be read is counted, never skipped silently: its ids are unknown. */ export function readOtherProfileWorkspaceCatalog( - userDataPath = getProfileUserDataPath(), - options: OtherProfileCatalogOptions = {} + userDataPath = getProfileUserDataPath() ): ProfileWorkspaceIds & { unreadableProfiles: number } { const ids = new Set() const repoIds = new Set() @@ -67,12 +52,11 @@ export function readOtherProfileWorkspaceCatalog( return { ids, repoIds, unreadableProfiles: 0 } } let unreadableProfiles = 0 - const running = options.runningProfileId ?? index.activeProfileId for (const profile of index.profiles) { - if (profile.id === running) { + if (profile.id === index.activeProfileId) { continue } - const collected = readProfileWorktreeIds(profile.id, userDataPath, options) + const collected = readProfileWorktreeIds(profile.id, userDataPath) if (!collected) { unreadableProfiles += 1 continue @@ -85,8 +69,7 @@ export function readOtherProfileWorkspaceCatalog( function readProfileWorktreeIds( profileId: string, - userDataPath: string, - options: OtherProfileCatalogOptions = {} + userDataPath: string ): ProfileWorkspaceIds | null { const databaseFile = getOrcaProfileStateDatabaseFile(profileId, userDataPath) // A present database is authoritative. In particular, do not fall back to a @@ -105,31 +88,9 @@ function readProfileWorktreeIds( } catch { return null } - if (options.neverWrittenIsEmpty && neverWritten(dataFile)) { - return { ids: new Set(), repoIds: new Set() } - } return readProfileWorktreeIdsFromJson(dataFile) } -/** No data file and none of Orca's own backups of it: a lost primary file is not a fresh profile. */ -function neverWritten(dataFile: string): boolean { - for (let index = 0; index < PROFILE_STATE_LEGACY_BACKUP_COUNT; index += 1) { - if (!definitivelyAbsent(profileStateLegacyBackupPath(dataFile, index))) { - return false - } - } - return definitivelyAbsent(dataFile) -} - -function definitivelyAbsent(path: string): boolean { - try { - lstatSync(path) - return false - } catch (error) { - return isDefinitiveAbsence(error) - } -} - function profileStateDatabasePresence(path: string): 'absent' | 'present' | 'unreadable' { let mainDatabasePresent = false try { diff --git a/src/main/runtime/files/file-mutations.ts b/src/main/runtime/files/file-mutations.ts index d7a98d651e8..7f8f1858d58 100644 --- a/src/main/runtime/files/file-mutations.ts +++ b/src/main/runtime/files/file-mutations.ts @@ -6,6 +6,8 @@ import { resolveFileExplorerPaths } from './file-target' import type { RuntimeFileCommandHost } from './file-command-host' +import { runNativeChatScratchSave } from '../../native-chat/native-chat-scratch-folder-lock' +import { getProfileUserDataPath } from '../../orca-profiles/profile-storage-paths' import { resolveRuntimeAuthorizedPath } from './chat-scratch-file-root' import { constants, copyFile, mkdir, rm } from 'node:fs/promises' import { dirname } from 'node:path' @@ -33,7 +35,9 @@ export class RuntimeFileMutations { expectedSshTargetId, expectedSshConnectionGeneration ) - await requireRuntimeFileProvider(target, this.host).writeFile(target.path, content) + await runNativeChatScratchSave(getProfileUserDataPath(), target.path, () => + requireRuntimeFileProvider(target, this.host).writeFile(target.path, content) + ) return { ok: true } } @@ -52,7 +56,9 @@ export class RuntimeFileMutations { expectedSshTargetId, expectedSshConnectionGeneration ) - await requireRuntimeFileProvider(target, this.host).writeFileBase64(target.path, contentBase64) + await runNativeChatScratchSave(getProfileUserDataPath(), target.path, () => + requireRuntimeFileProvider(target, this.host).writeFileBase64(target.path, contentBase64) + ) return { ok: true } } @@ -72,10 +78,12 @@ export class RuntimeFileMutations { expectedSshTargetId, expectedSshConnectionGeneration ) - await requireRuntimeFileProvider(target, this.host).writeFileBase64Chunk( - target.path, - contentBase64, - append + await runNativeChatScratchSave(getProfileUserDataPath(), target.path, () => + requireRuntimeFileProvider(target, this.host).writeFileBase64Chunk( + target.path, + contentBase64, + append + ) ) return { ok: true } } diff --git a/src/main/runtime/native-chat-visuals-workspace-verdict.test.ts b/src/main/runtime/native-chat-visuals-workspace-verdict.test.ts deleted file mode 100644 index 07ad7372d75..00000000000 --- a/src/main/runtime/native-chat-visuals-workspace-verdict.test.ts +++ /dev/null @@ -1,271 +0,0 @@ -import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' -import { tmpdir } from 'node:os' -import { join } from 'node:path' -import { afterEach, describe, expect, it, vi } from 'vitest' -import type { AgentSessionExecutionLocation } from '../../shared/agent-session-record' -import { FLOATING_TERMINAL_WORKTREE_ID } from '../../shared/constants' -import { LOCAL_EXECUTION_HOST_ID } from '../../shared/execution-host' -import { - createNativeChatVisualsWorkspaceVerdicts, - readNativeChatVisualsWorkspaceCatalogs, - type NativeChatVisualsFilesystem, - type NativeChatVisualsWorkspaceCatalogs -} from './native-chat-visuals-workspace-verdict' - -const REPO_PATH = '/work/repo-1' -const WORKTREE_PATH = '/work/repo-1-feature' -const WORKTREE_ID = `repo-1::${WORKTREE_PATH}` - -function catalogs( - options: { - repos?: string[] - meta?: string[] - folders?: string[] - others?: Partial - } = {} -): NativeChatVisualsWorkspaceCatalogs { - return { - active: { - getRepo: (id: string) => - (options.repos ?? ['repo-1']).includes(id) ? { path: REPO_PATH } : undefined, - getAllWorktreeMeta: () => Object.fromEntries((options.meta ?? []).map((id) => [id, {}])), - getFolderWorkspaces: () => (options.folders ?? []).map((id) => ({ id })) - }, - others: { ids: new Set(), repoIds: new Set(), unreadableProfiles: 0, ...options.others } - } -} - -const at = ( - workspaceId: string, - overrides: Partial = {} -): AgentSessionExecutionLocation => ({ - executionHostId: LOCAL_EXECUTION_HOST_ID, - wslDistro: null, - workspaceId, - workspaceKind: 'git-worktree', - ...overrides -}) - -type Presence = 'present' | 'absent' | 'unknown' - -function verdictWith( - known: NativeChatVisualsWorkspaceCatalogs | null, - paths: Record = {}, - gitRecords: boolean | null = false -) { - const fs: NativeChatVisualsFilesystem = { - presence: vi.fn(async (path: string): Promise => paths[path] ?? 'present'), - gitRecordsWorktree: vi.fn(async () => gitRecords) - } - return { verdict: createNativeChatVisualsWorkspaceVerdicts(() => known, fs)(), fs } -} - -describe('whether a chat workspace is provably removed', () => { - it('is removed when its project was removed from Orca in every profile', async () => { - const { verdict, fs } = verdictWith(catalogs({ repos: [] })) - await expect(verdict(at(WORKTREE_ID))).resolves.toBe('removed') - expect(fs.presence).not.toHaveBeenCalled() - }) - - it("keeps a chat whose project or folder only another profile holds (chats are shared, catalogs aren't)", async () => { - const { verdict } = verdictWith( - catalogs({ - repos: [], - others: { repoIds: new Set(['repo-1']), ids: new Set(['folder:f-2']) } - }) - ) - await expect(verdict(at(WORKTREE_ID))).resolves.toBe('unverifiable') - await expect(verdict(at('folder:f-2', { workspaceKind: 'folder' }))).resolves.toBe('present') - }) - - it('decides nothing while any profile cannot be read', async () => { - const { verdict } = verdictWith(catalogs({ repos: [], others: { unreadableProfiles: 1 } })) - await expect(verdict(at(WORKTREE_ID))).resolves.toBe('unverifiable') - await expect(verdict(at('folder:gone', { workspaceKind: 'folder' }))).resolves.toBe( - 'unverifiable' - ) - }) - - it('is present while any profile still tracks the worktree', async () => { - for (const known of [ - catalogs({ meta: [WORKTREE_ID] }), - catalogs({ others: { ids: new Set([WORKTREE_ID]) } }) - ]) { - const { verdict, fs } = verdictWith(known, { [WORKTREE_PATH]: 'absent' }) - await expect(verdict(at(WORKTREE_ID))).resolves.toBe('present') - expect(fs.presence).not.toHaveBeenCalled() - } - }) - - it('is removed when the worktree folder is gone and git no longer records it', async () => { - const { verdict, fs } = verdictWith(catalogs(), { [WORKTREE_PATH]: 'absent' }, false) - await expect(verdict(at(WORKTREE_ID))).resolves.toBe('removed') - expect(fs.gitRecordsWorktree).toHaveBeenCalledWith(REPO_PATH, WORKTREE_PATH) - }) - - it('keeps a worktree on an unmounted drive: git still records it, or the project is unreachable', async () => { - await expect( - verdictWith(catalogs(), { [WORKTREE_PATH]: 'absent' }, true).verdict(at(WORKTREE_ID)) - ).resolves.toBe('unverifiable') - await expect( - verdictWith(catalogs(), { [WORKTREE_PATH]: 'absent' }, null).verdict(at(WORKTREE_ID)) - ).resolves.toBe('unverifiable') - await expect( - verdictWith(catalogs(), { [WORKTREE_PATH]: 'absent', [REPO_PATH]: 'absent' }).verdict( - at(WORKTREE_ID) - ) - ).resolves.toBe('unverifiable') - }) - - it("never removes the project's own checkout while the project is known", async () => { - const { verdict } = verdictWith(catalogs(), { [REPO_PATH]: 'absent' }) - await expect(verdict(at(`repo-1::${REPO_PATH}`))).resolves.toBe('unverifiable') - }) - - it('keeps a worktree whose folder is there or unreadable', async () => { - await expect(verdictWith(catalogs()).verdict(at(WORKTREE_ID))).resolves.toBe('present') - await expect( - verdictWith(catalogs(), { [WORKTREE_PATH]: 'unknown' }).verdict(at(WORKTREE_ID)) - ).resolves.toBe('unverifiable') - }) - - it('reads a folder workspace from every catalog', async () => { - const folder = (id: string) => at(`folder:${id}`, { workspaceKind: 'folder' }) - const { verdict } = verdictWith(catalogs({ folders: ['f-1'] })) - await expect(verdict(folder('f-1'))).resolves.toBe('present') - await expect(verdict(folder('f-2'))).resolves.toBe('removed') - }) - - it('never decides for another host, a WSL distro, the floating workspace, or without catalogs', async () => { - const { verdict, fs } = verdictWith(catalogs({ repos: [] })) - await expect(verdict(at(WORKTREE_ID, { executionHostId: 'ssh:box' }))).resolves.toBe( - 'unverifiable' - ) - await expect(verdict(at(WORKTREE_ID, { wslDistro: 'Ubuntu' }))).resolves.toBe('unverifiable') - await expect( - verdict(at(FLOATING_TERMINAL_WORKTREE_ID, { workspaceKind: 'folder' })) - ).resolves.toBe('unverifiable') - await expect(verdictWith(null).verdict(at(WORKTREE_ID))).resolves.toBe('unverifiable') - expect(fs.presence).not.toHaveBeenCalled() - }) - - it('reads the catalogs once per run, however many chats it judges', async () => { - const read = vi.fn(() => catalogs({ repos: [] })) - const verdict = createNativeChatVisualsWorkspaceVerdicts(read)() - await verdict(at(WORKTREE_ID)) - await verdict(at('repo-1::/work/other')) - expect(read).toHaveBeenCalledOnce() - }) -}) - -describe("git's own record of a linked worktree", () => { - const scratch: string[] = [] - afterEach(() => { - for (const dir of scratch.splice(0)) { - rmSync(dir, { recursive: true, force: true }) - } - }) - - function repoWithWorktreeRecord(recordedWorktree: string | null): string { - const repo = realpathSync(mkdtempSync(join(tmpdir(), 'orca-visuals-git-'))) - scratch.push(repo) - mkdirSync(join(repo, '.git', 'worktrees', 'feature'), { recursive: true }) - if (recordedWorktree) { - writeFileSync( - join(repo, '.git', 'worktrees', 'feature', 'gitdir'), - `${recordedWorktree}/.git\n` - ) - } - return repo - } - - it('removes only a worktree git no longer records', async () => { - const gone = join(tmpdir(), 'orca-visuals-missing-worktree-a') - const verdicts = (repo: string) => - createNativeChatVisualsWorkspaceVerdicts(() => ({ - ...catalogs(), - active: { - ...catalogs().active, - getRepo: () => ({ path: repo }) - } - }))() - const recorded = repoWithWorktreeRecord(gone) - await expect(verdicts(recorded)(at(`repo-1::${gone}`))).resolves.toBe('unverifiable') - const pruned = repoWithWorktreeRecord(null) - await expect(verdicts(pruned)(at(`repo-1::${gone}`))).resolves.toBe('removed') - }) - - it('reads a record git wrote relative to its own folder', async () => { - const repo = repoWithWorktreeRecord(null) - const worktree = join(repo, '..', 'orca-visuals-relative-worktree-b') - writeFileSync( - join(repo, '.git', 'worktrees', 'feature', 'gitdir'), - '../../../../orca-visuals-relative-worktree-b/.git\n' - ) - const verdict = createNativeChatVisualsWorkspaceVerdicts(() => ({ - ...catalogs(), - active: { ...catalogs().active, getRepo: () => ({ path: repo }) } - }))() - await expect(verdict(at(`repo-1::${worktree}`))).resolves.toBe('unverifiable') - }) - - it('resolves a relative record against the real path when the project was added through a link', async () => { - const real = realpathSync(mkdtempSync(join(tmpdir(), 'orca-visuals-real-'))) - scratch.push(real) - const nested = join(real, 'a', 'b', 'c') - mkdirSync(join(nested, 'repo', '.git', 'worktrees', 'wt'), { recursive: true }) - const linkedParent = join(real, 'deep') - symlinkSync(nested, linkedParent) - const worktree = join(real, 'wts', 'wt') - // What git writes: relative from the record folder's real path. - writeFileSync( - join(nested, 'repo', '.git', 'worktrees', 'wt', 'gitdir'), - '../../../../../../../wts/wt/.git\n' - ) - const verdict = createNativeChatVisualsWorkspaceVerdicts(() => ({ - ...catalogs(), - active: { ...catalogs().active, getRepo: () => ({ path: join(linkedParent, 'repo') }) } - }))() - await expect(verdict(at(`repo-1::${worktree}`))).resolves.toBe('unverifiable') - }) -}) - -describe('reading the catalogs for a sweep run', () => { - it('skips the running profile named by its own storage folder', () => { - const root = mkdtempSync(join(tmpdir(), 'orca-visuals-profiles-')) - try { - mkdirSync(join(root, 'profiles', 'running'), { recursive: true }) - mkdirSync(join(root, 'profiles', 'next'), { recursive: true }) - const profile = (id: string) => ({ - id, - name: id, - kind: 'local', - createdAt: 0, - updatedAt: 0, - lastOpenedAt: 0, - avatar: { kind: 'initials', initials: id.slice(0, 2), color: 'neutral' } - }) - // A switch already named the next profile active; this process still runs the old one. - writeFileSync( - join(root, 'orca-profile-index.json'), - JSON.stringify({ activeProfileId: 'next', profiles: [profile('running'), profile('next')] }) - ) - for (const id of ['running', 'next']) { - writeFileSync( - join(root, 'profiles', id, 'orca-data.json'), - JSON.stringify({ repos: [{ id: `repo-${id}` }] }) - ) - } - const read = readNativeChatVisualsWorkspaceCatalogs( - { - ...catalogs().active, - getProfileStorageDirectory: () => join(root, 'profiles', 'running') - }, - root - ) - expect([...read.others.repoIds]).toEqual(['repo-next']) - } finally { - rmSync(root, { recursive: true, force: true }) - } - }) -}) diff --git a/src/main/runtime/native-chat-visuals-workspace-verdict.ts b/src/main/runtime/native-chat-visuals-workspace-verdict.ts deleted file mode 100644 index 3dce96bb0f7..00000000000 --- a/src/main/runtime/native-chat-visuals-workspace-verdict.ts +++ /dev/null @@ -1,204 +0,0 @@ -// Whether the local workspace a chat ran in is provably gone, for the scratch folder sweep. -// Chat records and visuals live at the user-data root, shared by every Orca profile, while each -// profile keeps its own workspace catalog: so only absence from EVERY profile's catalog counts. A -// worktree still in a known project is gone only when git itself no longer records it. Never runs -// git or lists a repo's worktrees: that walk can touch protected folders. - -import { lstat, readdir, readFile, realpath, stat } from 'node:fs/promises' -import { basename, dirname, isAbsolute, join, resolve } from 'node:path' -import type { AgentSessionExecutionLocation } from '../../shared/agent-session-record' -import { normalizeRuntimePathForComparison } from '../../shared/cross-platform-path' -import { isDefinitiveAbsence } from '../../shared/definitive-filesystem-absence' -import { LOCAL_EXECUTION_HOST_ID } from '../../shared/execution-host' -import { isFloatingWorkspaceId } from '../../shared/floating-workspace-worktree' -import { splitWorktreeIdForFilesystem } from '../../shared/worktree/id' -import { folderWorkspaceKey, parseWorkspaceKey } from '../../shared/workspace-scope' -import type { Store } from '../persistence' -import { readOtherProfileWorkspaceCatalog } from '../orca-profiles/other-profile-workspace-catalog' -import { - getOrcaProfilesDirectory, - getProfileUserDataPath -} from '../orca-profiles/profile-storage-paths' -import type { NativeChatVisualsWorkspaceVerdict } from '../native-chat/native-chat-visuals-sweep' - -/** This profile's live catalog and every other profile's persisted one. The running profile is - * named by its own storage folder, not the profile index, which a switch rewrites first. */ -export function readNativeChatVisualsWorkspaceCatalogs( - store: NativeChatVisualsWorkspaceCatalogs['active'] & Pick, - userDataPath = getProfileUserDataPath() -): NativeChatVisualsWorkspaceCatalogs { - const storage = store.getProfileStorageDirectory() - const runningProfileId = samePath(dirname(storage), getOrcaProfilesDirectory(userDataPath)) - ? basename(storage) - : undefined - return { - active: store, - others: readOtherProfileWorkspaceCatalog(userDataPath, { - ...(runningProfileId ? { runningProfileId } : {}), - // Chat records are never deleted, so this rule is what cleans up; a profile created but never - // opened must not switch it off. - neverWrittenIsEmpty: true - }) - } -} - -export type NativeChatVisualsWorkspaceCatalogs = { - /** The running profile's live catalog: the Store, narrowed to what a verdict reads. */ - active: { - getRepo: (id: string) => { path: string } | undefined - getAllWorktreeMeta: () => Readonly> - getFolderWorkspaces: () => readonly { id: string }[] - } - /** Every other profile's worktree ids, folder workspace keys and project ids. */ - others: { ids: ReadonlySet; repoIds: ReadonlySet; unreadableProfiles: number } -} - -export type NativeChatVisualsFilesystem = { - /** `present`, `absent` (definitively), or `unknown` for any other answer. */ - presence: (path: string) => Promise<'present' | 'absent' | 'unknown'> - /** Whether git in the repo at `repoPath` records a linked worktree at `worktreePath`; null when - * its records can't be read. */ - gitRecordsWorktree: (repoPath: string, worktreePath: string) => Promise -} - -async function presence(path: string): Promise<'present' | 'absent' | 'unknown'> { - try { - await stat(path) - return 'present' - } catch (error) { - return isDefinitiveAbsence(error) ? 'absent' : 'unknown' - } -} - -const samePath = (left: string, right: string): boolean => - normalizeRuntimePathForComparison(left) === normalizeRuntimePathForComparison(right) - -async function gitRecordsWorktree(repoPath: string, worktreePath: string): Promise { - const gitDir = join(repoPath, '.git') - try { - // A `.git` file (the repo is itself a linked worktree or a submodule) is not read here. - if (!(await lstat(gitDir)).isDirectory()) { - return null - } - let names: string[] - try { - names = await readdir(join(gitDir, 'worktrees')) - } catch (error) { - if (isDefinitiveAbsence(error)) { - return false - } - throw error - } - for (const name of names) { - let recorded: string - try { - recorded = await readFile(join(gitDir, 'worktrees', name, 'gitdir'), 'utf8') - } catch (error) { - if (isDefinitiveAbsence(error)) { - continue - } - throw error - } - // The record names the worktree's own `.git` file; git 2.48+ may write it relative to the - // record folder's real path, which differs from the stored one when that runs through a link. - const gitFile = recorded.trim() - const absolute = isAbsolute(gitFile) - ? gitFile - : resolve(await realpath(join(gitDir, 'worktrees', name)), gitFile) - if (samePath(dirname(absolute), worktreePath)) { - return true - } - } - return false - } catch { - return null - } -} - -const NODE_FILESYSTEM: NativeChatVisualsFilesystem = { presence, gitRecordsWorktree } - -async function worktreeVerdict( - catalogs: NativeChatVisualsWorkspaceCatalogs, - workspaceId: string, - fs: NativeChatVisualsFilesystem -): Promise { - const worktree = splitWorktreeIdForFilesystem(workspaceId) - if (!worktree?.repoId || !worktree.worktreePath) { - return 'unverifiable' - } - const repo = catalogs.active.getRepo(worktree.repoId) - // The project was removed from Orca in every profile. - if (!repo && !catalogs.others.repoIds.has(worktree.repoId)) { - return 'removed' - } - if ( - Object.hasOwn(catalogs.active.getAllWorktreeMeta(), workspaceId) || - catalogs.others.ids.has(workspaceId) - ) { - return 'present' - } - // The project's own checkout, or one only another profile can say anything about. - if (!repo || samePath(repo.path, worktree.worktreePath)) { - return 'unverifiable' - } - const own = await fs.presence(worktree.worktreePath) - if (own !== 'absent') { - return own === 'present' ? 'present' : 'unverifiable' - } - // A worktree on an unmounted drive is still recorded by git, so absence alone proves nothing. - if ((await fs.presence(repo.path)) !== 'present') { - return 'unverifiable' - } - return (await fs.gitRecordsWorktree(repo.path, worktree.worktreePath)) === false - ? 'removed' - : 'unverifiable' -} - -/** - * A verdict function over one snapshot of every profile's catalog; call once per sweep run. - * Null catalogs, or any profile whose state can't be read, decide nothing. - */ -export function createNativeChatVisualsWorkspaceVerdicts( - readCatalogs: () => NativeChatVisualsWorkspaceCatalogs | null, - fs: NativeChatVisualsFilesystem = NODE_FILESYSTEM -): () => (location: AgentSessionExecutionLocation) => Promise { - return () => { - let snapshot: NativeChatVisualsWorkspaceCatalogs | null | undefined - const catalogs = (): NativeChatVisualsWorkspaceCatalogs | null => { - if (snapshot === undefined) { - try { - snapshot = readCatalogs() - } catch { - snapshot = null - } - } - return snapshot && snapshot.others.unreadableProfiles === 0 ? snapshot : null - } - return async (location) => { - // Another host or a WSL distro owns its own answer; loss of contact is never removal. - if (location.executionHostId !== LOCAL_EXECUTION_HOST_ID || location.wslDistro !== null) { - return 'unverifiable' - } - // The floating workspace names a setting, not a place the user removes. - if (isFloatingWorkspaceId(location.workspaceId)) { - return 'unverifiable' - } - const known = catalogs() - if (!known) { - return 'unverifiable' - } - if (location.workspaceKind !== 'folder') { - return worktreeVerdict(known, location.workspaceId, fs) - } - if (parseWorkspaceKey(location.workspaceId)?.type !== 'folder') { - return 'unverifiable' - } - const listed = - known.others.ids.has(location.workspaceId) || - known.active - .getFolderWorkspaces() - .some((workspace) => folderWorkspaceKey(workspace.id) === location.workspaceId) - return listed ? 'present' : 'removed' - } - } -} diff --git a/src/main/runtime/orca-runtime-get-worktree-ps.ts b/src/main/runtime/orca-runtime-get-worktree-ps.ts index 845ed09c28c..795c29fb92b 100644 --- a/src/main/runtime/orca-runtime-get-worktree-ps.ts +++ b/src/main/runtime/orca-runtime-get-worktree-ps.ts @@ -56,10 +56,6 @@ import { withoutHiddenHosts } from './runtime-visible-catalog' import { claudeCliFlagSupport } from '../claude/claude-cli-flag-support' -import { - createNativeChatVisualsWorkspaceVerdicts, - readNativeChatVisualsWorkspaceCatalogs -} from './native-chat-visuals-workspace-verdict' export class OrcaRuntimeWithGetWorktreePs extends OrcaRuntimeWithStartTuiIdleVisibleReadProbe { async getWorktreePs( @@ -241,11 +237,7 @@ export class OrcaRuntimeWithGetWorktreePs extends OrcaRuntimeWithStartTuiIdleVis // Wired only here, so a test runtime never runs a real `claude --version`. claudeCliFlags: claudeCliFlagSupport, nativeChatVisuals: { - isEnabled: () => this.requireStore().getSettings().nativeChatInlineVisuals !== false, - // Chats and their visuals are shared by every profile; each profile keeps its own catalog. - workspaceVerdicts: createNativeChatVisualsWorkspaceVerdicts(() => - this.store ? readNativeChatVisualsWorkspaceCatalogs(this.store) : null - ) + isEnabled: () => this.requireStore().getSettings().nativeChatInlineVisuals !== false }, resolveShellEnvironmentPolicy: () => nativeChatShellEnvironmentPolicy(this.requireStore().getSettings()), diff --git a/src/main/runtime/structured-agent-session-runtime.ts b/src/main/runtime/structured-agent-session-runtime.ts index 3ec22cc3ac0..a9e2d69dda1 100644 --- a/src/main/runtime/structured-agent-session-runtime.ts +++ b/src/main/runtime/structured-agent-session-runtime.ts @@ -67,10 +67,8 @@ import { type RuntimeAgentAccountHomeResolver } from './structured-agent-model-catalog-wiring' import type { ClaudeCliFlagSupport } from '../claude/claude-cli-flag-support' -import { - scheduleNativeChatVisualsSweep, - type NativeChatVisualsSweepDeps -} from '../native-chat/native-chat-visuals-sweep' +import { scheduleNativeChatVisualsSweep } from '../native-chat/native-chat-visuals-sweep' +import { readJournalSessionLastActivityAt } from '../native-chat/agent-session-journal/journal-row-table' /** Whether this profile holds a structured chat: a record or tab in the journal database. */ export function hasPersistedStructuredAgentSessionStore( @@ -104,11 +102,11 @@ export type StructuredAgentSessionRuntimeDeps = { /** Which version-gated flags a Claude CLI takes; absent never passes one. */ claudeCliFlags?: ClaudeCliFlagSupport /** Gives each chat a scratch folder and the visuals skill that teaches it, and sweeps folders - * whose chat is gone. Wired by the real hosts only, so a test runtime never loads the bundled skill. */ + * whose chat is gone or long unused. Wired by the real hosts only, so a test runtime never loads + * the bundled skill. */ nativeChatVisuals?: { /** Read this host's preference when a chat starts a provider process. */ isEnabled: () => boolean - workspaceVerdicts: NonNullable } /** Provider transports are overridden only to drive the runtime against scripted children. */ openCodexConnection?: CodexStructuredSessionAdapterDeps['openConnection'] @@ -368,8 +366,9 @@ async function installOnJournal( ? scheduleNativeChatVisualsSweep({ stateDirectory: deps.stateDirectory, listHeldSessionIds: () => (store.readOnly ? null : store.listHeldSessionIds()), - locationOf: (sessionId) => store.getRecord(sessionId)?.location ?? null, - workspaceVerdicts: deps.nativeChatVisuals.workspaceVerdicts, + lastJournalActivityAt: (sessionId) => + readJournalSessionLastActivityAt(journalDatabase.db, sessionId), + runWhileExecutionIdle: host.runWhileExecutionIdle, logger: deps.logger }) : undefined