From 254a07bb0543c16ea4e03b14b147566f28dce78e Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Thu, 10 Sep 2026 19:30:47 -0700 Subject: [PATCH 1/8] fix(release): port three release-gate fixes to main so cuts stop re-inheriting them (#19945) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(release): prune optional natives before the linux arch floor check The linux-arm64 release build failed packaging, and retried three times: [verify-linux-glibc-floor] 1 bundled native binary is built for the wrong architecture (target arm64): resources/node_modules/@parcel/watcher-linux-x64-glibc/watcher.node is x64, expected arm64 (from its own path) `afterPack` ran `verifyLinuxGlibcFloor` at its top, before `prunePackagedRuntimeNodeModules`. A cross-build intentionally installs every optional native variant, so at that point the arm64 slice still carries the x64 `@parcel/watcher` package that the prune exists to drop. The check was reading a file that was never going to ship. Move the floor check below the prune so it inspects the binaries actually packed. Same ordering as 35f1b0ebb91 on the v1.4.197 release branch, which shipped green and was never merged back to main. (cherry picked from commit 6400598212aa7cb51fcff39b5db2d77123b6983b) (cherry picked from commit a883e17116354abe8401f7e0cc856cdf19e753b7) * fix(release): restore the minified telemetry constant fallback The macOS, Windows, and both Linux release builds all failed on "Verify telemetry constants present in app.asar", blocking publish-release: ::error::BUILD_IDENTITY constant missing or unexpected value in dist/mac-arm64/Orca.app/Contents/Resources/app.asar The verifier printed no context sample, because the string `BUILD_IDENTITY` does not occur anywhere in the shipped bundle at all. #17527 added `minify: 'oxc'` to the main bundle, so oxc renames the module-local `const BUILD_IDENTITY` to a short identifier. `BUILD_IDENTITY_RE` keys off the literal name and therefore cannot match a production bundle. `MINIFIED_TELEMETRY_RE` matches the adjacent injected identity/key pair instead, which survives renaming. #11019 created these patterns without that fallback, and its own comment predicted this exact failure if minification were ever enabled on the main bundle. The fallback was written on the v1.4.197 release branch in 35f1b0ebb91 and never merged back to main, so main has never been able to verify a minified bundle. Verified against a real local bundle built with the release env vars (ORCA_BUILD_IDENTITY=stable, ORCA_POSTHOG_WRITE_KEY=phc_...): the bundle does not contain "BUILD_IDENTITY"; both the narrowed and the widened BUILD_IDENTITY_RE fail; MINIFIED_TELEMETRY_RE matches and recovers identity=stable plus the write key. (cherry picked from commit 39c3e44cd1c1cf8b2dc7a58bb01e78a83b625c45) (cherry picked from commit d0bbe4475de3d943dede9ee5dae614ca238f1ef2) * fix(skills): stop asserting readdir order in the skill root walk test The Windows skill-sharing release gate — which blocks publish-release — failed on this test, wedging the 1.4.198 cut. The implementation is correct; the assertion was not. `findSkillFiles` pushes results in `readdir` order and dedupes visited directories by realpath, so of the 32 junctions pointing at one target exactly one survives. The assertion hardcoded both the array order and which link won. `readdir` order is filesystem-dependent: APFS and ext4 return SKILL.md first, while NTFS enumerates its filename index alphabetically on the uppercased name, where "LINK00" sorts before "SKILL.MD" (L=0x4C < S=0x53). Windows therefore returned the same two paths in the opposite order. Assert the contract instead: the real file is present, exactly one link-routed path survives dedup, and nothing else does. (cherry picked from commit 9a832e9bda4aaa98c04efa75e3265a2baf77ad75) (cherry picked from commit 505b85cd0d8eddd91c38a86fb3392294cd954520) * test(release): pin ported release gate fixes * style(skills): apply pinned test formatting --------- Co-authored-by: Neil Co-authored-by: Merge Sim --- config/electron-builder.config.cjs | 23 ++++--- ...lectron-builder-runtime-resources.test.mjs | 68 ++++++++++++++++++- .../telemetry-bundle-constant-patterns.mjs | 8 ++- ...elemetry-bundle-constant-patterns.test.mjs | 11 ++- config/scripts/verify-telemetry-constants.mjs | 18 +++-- src/main/skills/skill-root-file-walk.test.ts | 13 ++-- 6 files changed, 119 insertions(+), 22 deletions(-) diff --git a/config/electron-builder.config.cjs b/config/electron-builder.config.cjs index 7e0009b3a24..6b36938a3d1 100644 --- a/config/electron-builder.config.cjs +++ b/config/electron-builder.config.cjs @@ -279,16 +279,6 @@ module.exports = { } }, afterPack: async (context) => { - // Why: a Linux runner-image glibc bump silently shipped a node-pty pty.node - // requiring GLIBC_2.34, crashing the app on startup on Ubuntu 20.04 (#9902). - // Fail packaging if any bundled native binary exceeds the supported floor. - if (context.electronPlatformName === 'linux') { - // Why the arch is passed: symbol-version checks pass happily on a wrong-architecture binary, - // so a cross-built slice could ship the host's pty.node and only fail at runtime. - verifyLinuxGlibcFloor(context.appOutDir, { - targetArch: { 1: 'x64', 3: 'arm64' }[context.arch] - }) - } const resourcesDir = context.electronPlatformName === 'darwin' ? join( @@ -326,6 +316,19 @@ module.exports = { } stampPackagedCliVersion(resourcesDir, context.packager.appInfo.version) prunePackagedRuntimeNodeModules(resourcesDir, context.electronPlatformName, context.arch) + // Why: a Linux runner-image glibc bump silently shipped a node-pty pty.node + // requiring GLIBC_2.34, crashing the app on startup on Ubuntu 20.04 (#9902). + // Fail packaging if any bundled native binary exceeds the supported floor. + // Why after the prune: cross-builds intentionally install every optional + // native variant, so an arm64 slice still carries the x64 @parcel/watcher + // until prunePackagedRuntimeNodeModules drops it. + if (context.electronPlatformName === 'linux') { + // Why the arch is passed: symbol-version checks pass happily on a wrong-architecture binary, + // so a cross-built slice could ship the host's pty.node and only fail at runtime. + verifyLinuxGlibcFloor(context.appOutDir, { + targetArch: { 1: 'x64', 3: 'arm64' }[context.arch] + }) + } verifyPackagedMainRuntimeDeps(resourcesDir) // Why: boot the packaged daemon-entry under plain Node, but only for the // slice matching the packaging host's arch — daemon-entry.js is JS, yet it diff --git a/config/scripts/electron-builder-runtime-resources.test.mjs b/config/scripts/electron-builder-runtime-resources.test.mjs index 5a93ec12c25..45145572b80 100644 --- a/config/scripts/electron-builder-runtime-resources.test.mjs +++ b/config/scripts/electron-builder-runtime-resources.test.mjs @@ -2,7 +2,7 @@ import { readFileSync, readdirSync } from 'node:fs' import { cp, mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises' import { createRequire } from 'node:module' import { tmpdir } from 'node:os' -import { dirname, join, relative, resolve } from 'node:path' +import { delimiter, dirname, join, relative, resolve } from 'node:path' import { describe, expect, it } from 'vitest' const require = createRequire(import.meta.url) @@ -387,6 +387,72 @@ describe('packaged runtime resources', () => { } }) + it.skipIf(process.platform === 'win32')( + 'prunes non-target native packages before the Linux glibc gate', + async () => { + const root = await mkdtemp(join(tmpdir(), 'orca-after-pack-prune-order-')) + const previousPath = process.env.PATH + try { + const appOutDir = join(root, 'linux-unpacked') + const resourcesDir = join(appOutDir, 'resources') + await cp( + join(process.cwd(), 'resources', 'plugins', 'launch'), + join(resourcesDir, 'plugins', 'launch'), + { recursive: true } + ) + + const unpackedMainDir = join(resourcesDir, 'app.asar.unpacked', 'out', 'main') + await mkdir(unpackedMainDir, { recursive: true }) + await writeFile(join(unpackedMainDir, 'daemon-entry.js'), '', 'utf8') + await writeFile( + join(resourcesDir, 'app.asar.unpacked', 'out', 'package.json'), + `${JSON.stringify({ name: 'orca-compiled-output', type: 'commonjs', private: true })}\n`, + 'utf8' + ) + + const unpackedCliDir = join(resourcesDir, 'app.asar.unpacked', 'out', 'cli') + await mkdir(join(unpackedCliDir, 'handlers'), { recursive: true }) + await writeFile(join(unpackedCliDir, 'handlers', 'skills.js'), '', 'utf8') + await writeFile(join(unpackedCliDir, 'index.js'), '', 'utf8') + + const target = + process.arch === 'x64' + ? { electronArch: 3, machine: 0xb7, nonTarget: 'x64' } + : { electronArch: 1, machine: 0x3e, nonTarget: 'arm64' } + const wrongArchPackage = join( + resourcesDir, + 'node_modules', + '@parcel', + `watcher-linux-${target.nonTarget}-glibc` + ) + await mkdir(wrongArchPackage, { recursive: true }) + const wrongArchElf = Buffer.alloc(20) + wrongArchElf.set([0x7f, 0x45, 0x4c, 0x46]) + wrongArchElf[5] = 1 + wrongArchElf.writeUInt16LE(target.machine, 18) + await writeFile(join(wrongArchPackage, 'watcher.node'), wrongArchElf) + + const stubBinDir = join(root, 'bin') + await mkdir(stubBinDir) + await writeFile(join(stubBinDir, 'objdump'), '#!/bin/sh\nexit 0\n', { mode: 0o755 }) + process.env.PATH = `${stubBinDir}${delimiter}${previousPath ?? ''}` + + await expect( + electronBuilderConfig.afterPack({ + appOutDir, + electronPlatformName: 'linux', + arch: target.electronArch, + packager: { appInfo: { version: '9.9.9' } } + }) + ).resolves.toBeUndefined() + await expect(stat(wrongArchPackage)).rejects.toMatchObject({ code: 'ENOENT' }) + } finally { + process.env.PATH = previousPath + await rm(root, { recursive: true, force: true }) + } + } + ) + it.skipIf(process.platform === 'win32')( 'marks packaged Unix CLI launchers executable', async () => { diff --git a/config/scripts/telemetry-bundle-constant-patterns.mjs b/config/scripts/telemetry-bundle-constant-patterns.mjs index 04944b7b156..87c2ae43cf7 100644 --- a/config/scripts/telemetry-bundle-constant-patterns.mjs +++ b/config/scripts/telemetry-bundle-constant-patterns.mjs @@ -1,2 +1,6 @@ -export const BUILD_IDENTITY_RE = /\b(?:const|let|var)\s+BUILD_IDENTITY\s*=\s*"(rc|stable)"/ -export const WRITE_KEY_RE = /\b(?:const|let|var)\s+WRITE_KEY\s*=\s*"(phc_[A-Za-z0-9_-]+)"/ +// The unminified bundle keeps these names. Production minification may rename +// them, but the injected identity and key remain adjacent in the declaration. +export const BUILD_IDENTITY_RE = /\b(?:const|let|var)\s+BUILD_IDENTITY\s*=\s*["`](rc|stable)["`]/ +export const WRITE_KEY_RE = /\b(?:const|let|var)\s+WRITE_KEY\s*=\s*["`](phc_[A-Za-z0-9_-]+)["`]/ +export const MINIFIED_TELEMETRY_RE = + /\b(?:const|let|var)\s+[$\w]+\s*=\s*["'`](rc|stable)["'`][\s\S]{0,200}?[,$]\s*[$\w]+\s*=\s*["'`](phc_[A-Za-z0-9_-]+)["'`]/ diff --git a/config/scripts/telemetry-bundle-constant-patterns.test.mjs b/config/scripts/telemetry-bundle-constant-patterns.test.mjs index b8df5ec3d0f..ca87e3ee971 100644 --- a/config/scripts/telemetry-bundle-constant-patterns.test.mjs +++ b/config/scripts/telemetry-bundle-constant-patterns.test.mjs @@ -1,5 +1,9 @@ import { describe, expect, it } from 'vitest' -import { BUILD_IDENTITY_RE, WRITE_KEY_RE } from './telemetry-bundle-constant-patterns.mjs' +import { + BUILD_IDENTITY_RE, + MINIFIED_TELEMETRY_RE, + WRITE_KEY_RE +} from './telemetry-bundle-constant-patterns.mjs' describe('telemetry bundle constant patterns', () => { it.each(['const', 'let', 'var'])('accepts %s declarations', (declaration) => { @@ -13,4 +17,9 @@ describe('telemetry bundle constant patterns', () => { expect('const WRITE_KEY = null').not.toMatch(WRITE_KEY_RE) expect('const WRITE_KEY = "example-key"').not.toMatch(WRITE_KEY_RE) }) + + it('accepts minified adjacent declarations', () => { + const bundle = 'var dde=`stable`,fde=`phc_example-key_123`,pde=(dde===`stable`)' + expect(bundle).toMatch(MINIFIED_TELEMETRY_RE) + }) }) diff --git a/config/scripts/verify-telemetry-constants.mjs b/config/scripts/verify-telemetry-constants.mjs index 3bdcd8b3ec6..836a3e4546f 100644 --- a/config/scripts/verify-telemetry-constants.mjs +++ b/config/scripts/verify-telemetry-constants.mjs @@ -38,7 +38,11 @@ import { join, resolve } from 'node:path' // `node_modules`). If electron-builder ever drops it, promote this to a // direct devDependency in package.json. import { extractFile, listPackage } from '@electron/asar' -import { BUILD_IDENTITY_RE, WRITE_KEY_RE } from './telemetry-bundle-constant-patterns.mjs' +import { + BUILD_IDENTITY_RE, + MINIFIED_TELEMETRY_RE, + WRITE_KEY_RE +} from './telemetry-bundle-constant-patterns.mjs' // Why resolve from import.meta.url instead of cwd: a release runner (or a // developer debugging locally) may invoke this script from a non-root cwd. @@ -156,8 +160,14 @@ function verifyAsar(asarPath) { const buildIdentityMatch = BUILD_IDENTITY_RE.exec(indexJs) const writeKeyMatch = WRITE_KEY_RE.exec(indexJs) + const minifiedTelemetryMatch = MINIFIED_TELEMETRY_RE.exec(indexJs) - if (!buildIdentityMatch) { + // Rolldown renames module-local constants in production output. In that + // form, verify the adjacent injected identity/key declaration instead. + const verifiedIdentity = buildIdentityMatch?.[1] ?? minifiedTelemetryMatch?.[1] + const verifiedWriteKey = writeKeyMatch?.[1] ?? minifiedTelemetryMatch?.[2] + + if (!verifiedIdentity) { console.error(`::error::BUILD_IDENTITY constant missing or unexpected value in ${asarPath}`) const sample = indexJs.match(/.{0,80}BUILD_IDENTITY.{0,80}/g)?.slice(0, 5) ?? [] for (const line of sample) { @@ -165,7 +175,7 @@ function verifyAsar(asarPath) { } return null } - if (!writeKeyMatch) { + if (!verifiedWriteKey) { console.error(`::error::PostHog WRITE_KEY missing from ${asarPath}`) const sample = indexJs.match(/.{0,80}WRITE_KEY.{0,80}/g)?.slice(0, 5) ?? [] for (const line of sample) { @@ -174,7 +184,7 @@ function verifyAsar(asarPath) { return null } - return { asarPath, buildIdentity: buildIdentityMatch[1], writeKey: writeKeyMatch[1] } + return { asarPath, buildIdentity: verifiedIdentity, writeKey: verifiedWriteKey } } // Why verify every match (not just the first): macOS dual-arch produces one diff --git a/src/main/skills/skill-root-file-walk.test.ts b/src/main/skills/skill-root-file-walk.test.ts index ad84eced6b6..81e9b167fcc 100644 --- a/src/main/skills/skill-root-file-walk.test.ts +++ b/src/main/skills/skill-root-file-walk.test.ts @@ -66,10 +66,15 @@ describe('findSkillFiles', () => { expect(await findSkillFiles(root, 4)).toEqual([join(edge, 'SKILL.md')]) expect(statPaths).toEqual([]) - expect(await findSkillFiles(root, 5)).toEqual([ - join(edge, 'SKILL.md'), - join(edge, 'link00', 'SKILL.md') - ]) + // Why not a fixed array: `readdir` order is filesystem-dependent, and both + // the result order and which link survives dedup follow it. NTFS enumerates + // its name index alphabetically, so `link00` precedes `SKILL.md` on Windows + // and follows it on APFS/ext4. All 32 links share one realpath, so the + // visited set collapses them to a single entry beside the real file. + const withinDepth = await findSkillFiles(root, 5) + expect(withinDepth).toContain(join(edge, 'SKILL.md')) + expect(withinDepth.filter((path) => /[\\/]link\d{2}[\\/]SKILL\.md$/.test(path))).toHaveLength(1) + expect(withinDepth).toHaveLength(2) expect(statPaths).toHaveLength(32) }) From ae729128b630eb45d391dfed19cf127cde6fc67e Mon Sep 17 00:00:00 2001 From: Jinjing <6427696+AmethystLiang@users.noreply.github.com> Date: Thu, 10 Sep 2026 19:35:22 -0700 Subject: [PATCH 2/8] refactor(renderer): share path head elision (#19938) --- .../new-workspace/ProjectComboboxRow.test.tsx | 16 ++++- .../new-workspace/ProjectComboboxRow.tsx | 14 ++-- .../project-combobox-matching.test.ts | 15 +---- .../project-combobox-matching.ts | 13 ---- .../src/lib/path-head-elision.test.ts | 67 +++++++++++++++++++ src/renderer/src/lib/path-head-elision.ts | 52 ++++++++++++++ 6 files changed, 145 insertions(+), 32 deletions(-) create mode 100644 src/renderer/src/lib/path-head-elision.test.ts create mode 100644 src/renderer/src/lib/path-head-elision.ts diff --git a/src/renderer/src/components/new-workspace/ProjectComboboxRow.test.tsx b/src/renderer/src/components/new-workspace/ProjectComboboxRow.test.tsx index 69f85ee8d00..29d78fa55a0 100644 --- a/src/renderer/src/components/new-workspace/ProjectComboboxRow.test.tsx +++ b/src/renderer/src/components/new-workspace/ProjectComboboxRow.test.tsx @@ -3,7 +3,7 @@ import { act } from 'react' import { createRoot, type Root } from 'react-dom/client' import { afterEach, beforeEach, describe, expect, it } from 'vitest' -import { MatchedText } from './ProjectComboboxRow' +import { MatchedText, ProjectOptionDetail } from './ProjectComboboxRow' import { rankProjectOptions } from './project-combobox-matching' import type { NewWorkspaceProjectOption } from '@/lib/new-workspace-project-options' @@ -69,3 +69,17 @@ describe('MatchedText', () => { expect(container.textContent).toBe('🚀 orca') }) }) + +describe('ProjectOptionDetail', () => { + it('keeps Windows path matches highlighted in the preserved tail', () => { + const detail = 'C:\\Users\\ada\\projects\\orca\\src\\renderer\\app.ts' + const start = detail.indexOf('src') + + act(() => { + root.render() + }) + + expect(container.textContent).toBe(detail) + expect(markedText()).toBe('src') + }) +}) diff --git a/src/renderer/src/components/new-workspace/ProjectComboboxRow.tsx b/src/renderer/src/components/new-workspace/ProjectComboboxRow.tsx index 471fd9a42cc..d9568dd5526 100644 --- a/src/renderer/src/components/new-workspace/ProjectComboboxRow.tsx +++ b/src/renderer/src/components/new-workspace/ProjectComboboxRow.tsx @@ -3,7 +3,7 @@ import { FolderOpen } from 'lucide-react' import { RepoBadgeMark } from '@/components/repo/RepoBadgeLabel' import { cn } from '@/lib/utils' import type { NewWorkspaceProjectOption } from '@/lib/new-workspace-project-options' -import { splitDetailForElision } from './project-combobox-matching' +import { splitPathHeadForElision } from '@/lib/path-head-elision' /** Identity mark shared by the field and every row, so a project reads the same in both. */ export function ProjectOptionMark({ @@ -70,7 +70,8 @@ export function ProjectOptionDetail({ hits?: readonly number[] className?: string }): React.JSX.Element { - const split = splitDetailForElision(detail) + const ranges = hits?.map((index) => ({ start: index, end: index + 1 })) + const split = splitPathHeadForElision(detail, ranges) if (!split) { return ( @@ -80,9 +81,14 @@ export function ProjectOptionDetail({ } return ( - {/* Head collapses first; the tail only truncates once the head is gone. */} {split.head} - /{split.tail} + + {split.tailRanges.length > 0 ? ( + range.start)} /> + ) : ( + split.tail + )} + ) } diff --git a/src/renderer/src/components/new-workspace/project-combobox-matching.test.ts b/src/renderer/src/components/new-workspace/project-combobox-matching.test.ts index 6e16dae386d..28c422e6c2b 100644 --- a/src/renderer/src/components/new-workspace/project-combobox-matching.test.ts +++ b/src/renderer/src/components/new-workspace/project-combobox-matching.test.ts @@ -3,8 +3,7 @@ import type { NewWorkspaceProjectOption } from '@/lib/new-workspace-project-opti import { getAmbiguousProjectOptionIds, rankProjectOptions, - sectionProjectOptions, - splitDetailForElision + sectionProjectOptions } from './project-combobox-matching' function project(id: string, displayName: string, detail: string): NewWorkspaceProjectOption { @@ -76,15 +75,3 @@ describe('getAmbiguousProjectOptionIds', () => { expect(ids).toEqual(new Set(['a', 'b'])) }) }) - -describe('splitDetailForElision', () => { - it('keeps the last two segments so sibling paths stay distinguishable', () => { - const split = splitDetailForElision('~/Developer/work/acme/monorepo/services/checkout-api') - expect(split?.tail).toBe('services/checkout-api') - }) - - it('leaves short or shallow details alone', () => { - expect(splitDetailForElision('stablyai/orca')).toBeNull() - expect(splitDetailForElision('3 hosts configured')).toBeNull() - }) -}) diff --git a/src/renderer/src/components/new-workspace/project-combobox-matching.ts b/src/renderer/src/components/new-workspace/project-combobox-matching.ts index 2e3459ac745..6ee21dd2194 100644 --- a/src/renderer/src/components/new-workspace/project-combobox-matching.ts +++ b/src/renderer/src/components/new-workspace/project-combobox-matching.ts @@ -143,16 +143,3 @@ export function getAmbiguousProjectOptionIds( options.filter((o) => (counts.get(o.displayName) ?? 0) > 1).map((option) => option.id) ) } - -/** - * A deep path's identity lives in its tail (`…/services/checkout-api`), which is - * exactly what a plain truncate throws away — two sibling paths then render - * identically. Split so the head can elide while the tail keeps its width. - */ -export function splitDetailForElision(detail: string): { head: string; tail: string } | null { - const segments = detail.split('/') - if (segments.length <= 3 || detail.length <= 28) { - return null - } - return { head: segments.slice(0, -2).join('/'), tail: segments.slice(-2).join('/') } -} diff --git a/src/renderer/src/lib/path-head-elision.test.ts b/src/renderer/src/lib/path-head-elision.test.ts new file mode 100644 index 00000000000..3fd110dd27e --- /dev/null +++ b/src/renderer/src/lib/path-head-elision.test.ts @@ -0,0 +1,67 @@ +import { describe, expect, it } from 'vitest' +import { splitPathHeadForElision } from './path-head-elision' + +describe('splitPathHeadForElision', () => { + it('keeps the last two segments as the tail', () => { + expect(splitPathHeadForElision('/Users/me/projects/orca/proposals/create-button.html')).toEqual( + { + head: '/Users/me/projects/orca', + tail: '/proposals/create-button.html', + tailRanges: [] + } + ) + }) + + it('supports Windows paths without changing their separators', () => { + const path = 'C:\\Users\\me\\projects\\orca\\src\\renderer\\app.ts' + const start = path.indexOf('src') + + expect(splitPathHeadForElision(path, [{ start, end: start + 3 }])).toEqual({ + head: 'C:\\Users\\me\\projects\\orca', + tail: '\\src\\renderer\\app.ts', + tailRanges: [{ start: 1, end: 4 }] + }) + }) + + it('keeps backslashes inside POSIX segment names', () => { + expect(splitPathHeadForElision('/tmp/project/src/name\\with\\slashes.ts')).toEqual({ + head: '/tmp/project', + tail: '/src/name\\with\\slashes.ts', + tailRanges: [] + }) + }) + + it('leaves short or shallow paths whole', () => { + expect(splitPathHeadForElision('src/app.ts')).toBeNull() + expect(splitPathHeadForElision('a/b/c')).toBeNull() + expect(splitPathHeadForElision('/tmp/orca-create-button/create-button.html')).toEqual({ + head: '/tmp', + tail: '/orca-create-button/create-button.html', + tailRanges: [] + }) + }) + + it('extends the tail back to the first matched segment and re-bases ranges', () => { + const path = '/Users/me/projects/orca/new-create-button-design/proposals/create-button.html' + const start = path.indexOf('create-butt') + const split = splitPathHeadForElision(path, [{ start, end: start + 'create-butt'.length }]) + expect(split).toEqual({ + head: '/Users/me/projects/orca', + tail: '/new-create-button-design/proposals/create-button.html', + tailRanges: [{ start: 5, end: 16 }] + }) + }) + + it('pulls a segment the match starts in fully into the tail', () => { + const path = '/Users/me/projects/orca/deep/nested/file.ts' + const split = splitPathHeadForElision(path, [{ start: 20, end: 30 }]) + expect(split?.head).toBe('/Users/me/projects') + expect(split?.tail).toBe('/orca/deep/nested/file.ts') + expect(split?.tailRanges).toEqual([{ start: 2, end: 12 }]) + }) + + it('returns null when the match sits in the first segment', () => { + const path = '/Users-long-prefix/me/projects/orca/deep/file.ts' + expect(splitPathHeadForElision(path, [{ start: 1, end: 6 }])).toBeNull() + }) +}) diff --git a/src/renderer/src/lib/path-head-elision.ts b/src/renderer/src/lib/path-head-elision.ts new file mode 100644 index 00000000000..e21bac240d6 --- /dev/null +++ b/src/renderer/src/lib/path-head-elision.ts @@ -0,0 +1,52 @@ +import type { MatchRange } from './palette-match/normalized-text' +import { isWindowsAbsolutePathLike } from '../../../shared/cross-platform-path' + +export type PathHeadElisionSplit = { + head: string + tail: string + tailRanges: readonly MatchRange[] +} + +const MIN_ELISION_LENGTH = 28 +const MIN_SEGMENTS = 4 +const TAIL_SEGMENTS = 2 + +export function splitPathHeadForElision( + path: string, + ranges: readonly MatchRange[] = [] +): PathHeadElisionSplit | null { + if (path.length <= MIN_ELISION_LENGTH) { + return null + } + const windowsPath = isWindowsAbsolutePathLike(path) + const separators: number[] = [] + for (let index = 0; index < path.length; index += 1) { + if (path[index] === '/' || (windowsPath && path[index] === '\\')) { + separators.push(index) + } + } + if (separators.length < MIN_SEGMENTS - 1) { + return null + } + let tailStart = separators[separators.length - TAIL_SEGMENTS]! + const firstMatchStart = ranges.reduce( + (earliest, range) => (range.start < range.end ? Math.min(earliest, range.start) : earliest), + Number.POSITIVE_INFINITY + ) + if (firstMatchStart < tailStart) { + const segmentSeparator = separators.findLast((separator) => separator < firstMatchStart) + tailStart = segmentSeparator ?? 0 + } + const head = path.slice(0, tailStart) + if (!(windowsPath ? /[^/\\]/ : /[^/]/).test(head)) { + return null + } + return { + head, + tail: path.slice(tailStart), + tailRanges: ranges.map((range) => ({ + start: range.start - tailStart, + end: range.end - tailStart + })) + } +} From 54216a7868a9aab8993ad7bc61909e9940bf2bec Mon Sep 17 00:00:00 2001 From: Jinjing <6427696+AmethystLiang@users.noreply.github.com> Date: Thu, 10 Sep 2026 19:35:22 -0700 Subject: [PATCH 3/8] feat(cmd-j): compact palette location layout (#19939) * feat(cmd-j): compact palette location layout * chore(i18n): sync palette runtime fallbacks * fix(cmd-j): preserve agent snippet context * fix(cmd-j): elide all file-backed tab paths --- ...ee-jump-palette-browser-simulator-rows.tsx | 42 ++-- ...jump-palette-interleaved-sections.test.tsx | 9 +- .../worktree-jump-palette-primitives.test.tsx | 115 ++++++++++- .../worktree-jump-palette-primitives.tsx | 195 ++++++++++-------- ...orktree-jump-palette-workspace-tab-row.tsx | 24 +-- src/renderer/src/i18n/locales/en.json | 2 - src/renderer/src/i18n/locales/fr.json | 2 - 7 files changed, 260 insertions(+), 129 deletions(-) diff --git a/src/renderer/src/components/worktree-jump-palette-browser-simulator-rows.tsx b/src/renderer/src/components/worktree-jump-palette-browser-simulator-rows.tsx index 95220a25f2e..4a3acb10f11 100644 --- a/src/renderer/src/components/worktree-jump-palette-browser-simulator-rows.tsx +++ b/src/renderer/src/components/worktree-jump-palette-browser-simulator-rows.tsx @@ -1,15 +1,14 @@ import type React from 'react' import { Smartphone } from 'lucide-react' import { CommandItem } from '@/components/ui/command' -import { RepoBadgeMark } from '@/components/repo/RepoBadgeLabel' import { getPaletteHostBadge } from '@/components/cmd-j/palette-host-badge' import { cn } from '@/lib/utils' import { translate } from '@/i18n/i18n' import type { BrowserPaletteItem, SimulatorPaletteItem } from './worktree-jump-palette-model' import type { WorktreeJumpPaletteController } from './use-worktree-jump-palette-controller' import { - HighlightedText, PaletteHostBadgeChip, + PaletteLocationChip, PaletteOpenTabPrimaryLine, PaletteRowShortcutBadge } from './worktree-jump-palette-primitives' @@ -67,8 +66,6 @@ export function WorktreeJumpPaletteSimulatorRow({ secondaryText={result.secondaryText} secondaryRanges={result.secondaryRanges} secondaryMatches={result.secondaryMatches} - worktreeName={result.worktreeName} - worktreeRanges={result.worktreeRanges} sessionAge={simulatorSessionAge} leadingBadges={ <> @@ -94,16 +91,15 @@ export function WorktreeJumpPaletteSimulatorRow({ ) : null} -
+
- {simulatorRepoName && ( - - - - - - - )} + @@ -187,16 +182,15 @@ export function WorktreeJumpPaletteBrowserRow({ } />
-
+
- {browserRepoName && ( - - - - - - - )} + { const title = row?.querySelector('[data-slot="palette-open-tab-title"]') const worktree = row?.querySelector('[data-slot="palette-open-tab-worktree"]') expect(title?.textContent).toBe(longTitle) - expect(title?.classList.contains('flex-auto')).toBe(true) + expect(title?.classList.contains('min-w-0')).toBe(true) + expect(title?.classList.contains('shrink-0')).toBe(false) expect(worktree?.textContent).toBe('user-support') + const locationChip = worktree?.closest('[data-slot="palette-location-chip"]') + expect(locationChip).not.toBeNull() + expect(locationChip?.parentElement?.classList.contains('max-w-[40%]')).toBe(true) + expect(locationChip?.parentElement?.classList.contains('min-w-0')).toBe(true) expect(worktree?.compareDocumentPosition(title ?? document.createElement('span'))).toBe( Node.DOCUMENT_POSITION_PRECEDING ) }) - it('tags the worktree rail label as a branch when the visible name is the branch', async () => { + it('shows the branch in the location chip when the workspace display name is empty', async () => { await renderPalette({ worktreesByRepo: { 'repo-1': [makeWorktree('wt-tabs', '', { displayName: '' })] diff --git a/src/renderer/src/components/worktree-jump-palette-primitives.test.tsx b/src/renderer/src/components/worktree-jump-palette-primitives.test.tsx index a54a48329be..8834c240086 100644 --- a/src/renderer/src/components/worktree-jump-palette-primitives.test.tsx +++ b/src/renderer/src/components/worktree-jump-palette-primitives.test.tsx @@ -3,7 +3,7 @@ import { cleanup, render, type RenderResult, screen } from '@testing-library/react' import { afterEach, expect, it } from 'vitest' import { TooltipProvider } from '@/components/ui/tooltip' -import { PaletteOpenTabPrimaryLine } from './worktree-jump-palette-primitives' +import { PaletteLocationChip, PaletteOpenTabPrimaryLine } from './worktree-jump-palette-primitives' afterEach(() => cleanup()) @@ -18,8 +18,6 @@ function renderPrimaryLine( secondaryText="src/app.ts" secondaryRanges={[]} secondaryMatches={secondaryMatches} - worktreeName="Workspace" - worktreeRanges={[]} /> ) @@ -45,3 +43,114 @@ it('renders no badge when every secondary match is already shown', () => { expect(screen.queryByText(/^\+\d+$/)).toBeNull() }) + +it('elides a deep path from the head so the matched tail stays visible', () => { + const path = '/Users/me/projects/orca/new-create-button-design/proposals/create-button.html' + const start = path.indexOf('create-butt') + const { container } = render( + + + + ) + + const secondary = container.querySelector('[data-slot="palette-open-tab-secondary"]') + expect(secondary?.textContent).toBe(path) + const [head, tail] = Array.from(secondary?.children ?? []) + expect(head?.textContent).toBe('/Users/me/projects/orca') + expect(tail?.textContent).toBe('/new-create-button-design/proposals/create-button.html') + expect(tail?.querySelector('.font-semibold')?.textContent).toBe('create-butt') +}) + +it('keeps slash-separated agent snippets intact', () => { + const snippet = + 'Ran pnpm test src/renderer/src/components/worktree-jump-palette-primitives.test.tsx' + const start = snippet.indexOf('worktree-jump') + const { container } = render( + + + + ) + + const secondary = container.querySelector('[data-slot="palette-open-tab-secondary"]') + expect(secondary?.textContent).toBe(snippet) + expect(secondary?.children).toHaveLength(1) + expect(secondary?.querySelector('.font-semibold')?.textContent).toBe('worktree-jump') +}) + +it('folds the worktree into the repo chip and drops it when it repeats the repo name', () => { + const { container, rerender } = render( + + + + ) + expect(container.querySelector('[data-slot="palette-location-chip"]')?.textContent).toBe( + 'orca·new-create-button-design' + ) + const chip = container.querySelector('[data-slot="palette-location-chip"]') + const repo = container.querySelector('[data-slot="palette-location-repo"]') + const worktree = container.querySelector('[data-slot="palette-open-tab-worktree"]') + expect(chip?.className).toContain('overflow-hidden') + expect(chip?.className).toContain('min-w-0') + expect(repo?.className).toContain('max-w-[55%]') + expect(repo?.className).toContain('min-w-[3ch]') + expect(worktree?.className).toContain('min-w-[3ch]') + expect(worktree?.getAttribute('data-state')).toBeNull() + expect(worktree?.getAttribute('tabindex')).toBeNull() + + rerender( + + + + ) + expect(container.querySelector('[data-slot="palette-location-chip"]')?.textContent).toBe('orca') + expect( + container.querySelector('[data-slot="palette-location-repo"] .font-semibold')?.textContent + ).toBe('orca') +}) + +it('keeps a short title at its natural width so the session age stays beside it', () => { + const { container } = render( + + + + ) + + const title = container.querySelector('[data-slot="palette-open-tab-title"]') + expect(title?.className).toContain('min-w-0') + expect(title?.className).not.toContain('shrink-0') +}) + +it('caps the title only when it shares the line with secondary text', () => { + const { container } = renderPrimaryLine([]) + + const title = container.querySelector('[data-slot="palette-open-tab-title"]') + expect(title?.className).toContain('max-w-[62%]') + expect(title?.className).toContain('shrink-0') +}) diff --git a/src/renderer/src/components/worktree-jump-palette-primitives.tsx b/src/renderer/src/components/worktree-jump-palette-primitives.tsx index 4162a56cb6e..aadad46748d 100644 --- a/src/renderer/src/components/worktree-jump-palette-primitives.tsx +++ b/src/renderer/src/components/worktree-jump-palette-primitives.tsx @@ -1,11 +1,12 @@ -import React, { useLayoutEffect, useRef, useState } from 'react' +import React from 'react' import { ShortcutKeyCombo } from '@/components/ShortcutKeyCombo' import { translate } from '@/i18n/i18n' import type { PaletteHostBadge } from '@/components/cmd-j/palette-host-badge' import type { MatchRange, PaletteSearchResult } from '@/lib/worktree-palette-search' import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip' -import type { Worktree } from '../../../shared/worktree/types' -import { resolveWorktreeBranchLabel } from '@/lib/worktree-default-display-name' +import { splitPathHeadForElision } from '@/lib/path-head-elision' +import { RepoBadgeMark } from '@/components/repo/RepoBadgeLabel' +import { cn } from '@/lib/utils' const NO_SECONDARY_MATCHES: readonly { text: string; ranges: readonly MatchRange[] }[] = [] @@ -67,14 +68,51 @@ export function HighlightedText({ return <>{parts} } +function PaletteOpenTabSecondaryText({ + text, + ranges, + elidePathHead +}: { + text: string + ranges: readonly MatchRange[] + elidePathHead: boolean +}): React.JSX.Element { + const split = elidePathHead ? splitPathHeadForElision(text, ranges) : null + const content = split ? ( + + {split.head} + + + + + ) : ( + + + + ) + return ( + + {content} + + {text} + + + ) +} + export function PaletteOpenTabPrimaryLine({ title, titleRanges, secondaryText, secondaryRanges, secondaryMatches = NO_SECONDARY_MATCHES, - worktreeName, - worktreeRanges, + elideSecondaryPathHead = false, sessionAge, leadingBadges }: { @@ -83,13 +121,11 @@ export function PaletteOpenTabPrimaryLine({ secondaryText: string secondaryRanges: readonly MatchRange[] secondaryMatches?: readonly { text: string; ranges: readonly MatchRange[] }[] - worktreeName: string - worktreeRanges: readonly MatchRange[] + elideSecondaryPathHead?: boolean sessionAge?: string leadingBadges?: React.ReactNode }): React.JSX.Element { const showSecondary = secondaryText.trim().length > 0 - const showWorktree = worktreeName.trim().length > 0 const additionalSecondaryMatches = secondaryMatches.filter( (match) => match.text && match.text !== secondaryText ) @@ -98,7 +134,10 @@ export function PaletteOpenTabPrimaryLine({
@@ -116,12 +155,11 @@ export function PaletteOpenTabPrimaryLine({ ) : null} {leadingBadges} {showSecondary ? ( - <> - · - - - - + ) : null} {additionalSecondaryMatches.length ? ( <> @@ -154,85 +192,76 @@ export function PaletteOpenTabPrimaryLine({ ) : null} - {showWorktree ? ( - <> - · - - - - - ) : null}
) } -function resolveOpenTabWorktreeRailTooltip({ - isBranch, - truncated, - name +export function PaletteLocationChip({ + repoName, + repoRanges, + repoColor, + worktreeName, + worktreeRanges, + className }: { - isBranch: boolean - truncated: boolean - name: string -}): string { - if (truncated) { - return name - } - return isBranch - ? translate('auto.components.WorktreeJumpPalette.paletteOpenTabBranch', 'Branch name') - : translate('auto.components.WorktreeJumpPalette.paletteOpenTabWorkspace', 'Workspace name') -} - -export function PaletteOpenTabWorktreeRailLabel({ - name, - matchRanges, - worktree, - className, - slot = 'palette-open-tab-worktree' -}: { - name: string - matchRanges: readonly MatchRange[] - worktree?: Pick | null + repoName: string + repoRanges: readonly MatchRange[] + repoColor?: string + worktreeName: string + worktreeRanges: readonly MatchRange[] className?: string - slot?: string }): React.JSX.Element | null { - const [truncated, setTruncated] = useState(false) - const labelRef = useRef(null) - useLayoutEffect(() => { - const node = labelRef.current - if (!node) { - setTruncated(false) - return - } - const updateTruncated = (): void => { - const next = node.scrollWidth > node.clientWidth - setTruncated((current) => (current === next ? current : next)) - } - updateTruncated() - if (typeof ResizeObserver === 'undefined') { - return - } - const observer = new ResizeObserver(updateTruncated) - observer.observe(node) - return () => observer.disconnect() - }, [name]) - if (name.trim().length === 0) { + const showRepo = repoName.trim().length > 0 + const repeatedName = showRepo && worktreeName === repoName + const showWorktree = worktreeName.trim().length > 0 && !repeatedName + if (!showRepo && !showWorktree) { return null } - const isBranch = worktree != null && name === resolveWorktreeBranchLabel(worktree) - const tooltip = resolveOpenTabWorktreeRailTooltip({ isBranch, truncated, name }) + const repoMatchRanges = repeatedName + ? [...repoRanges, ...worktreeRanges].sort((left, right) => left.start - right.start) + : repoRanges + const label = [showRepo ? repoName : '', showWorktree ? worktreeName : ''] + .filter(Boolean) + .join(' · ') + const chip = ( + + {showRepo ? ( + <> + + + + + + ) : null} + {showRepo && showWorktree ? ( + + · + + ) : null} + {showWorktree ? ( + + + + ) : null} + + ) return ( - - - - - - - {tooltip} + {chip} + + {label} ) diff --git a/src/renderer/src/components/worktree-jump-palette-workspace-tab-row.tsx b/src/renderer/src/components/worktree-jump-palette-workspace-tab-row.tsx index 787cbff449c..0763300eca7 100644 --- a/src/renderer/src/components/worktree-jump-palette-workspace-tab-row.tsx +++ b/src/renderer/src/components/worktree-jump-palette-workspace-tab-row.tsx @@ -3,20 +3,20 @@ import { FileText, SquareTerminal } from 'lucide-react' import { AgentIcon } from '@/lib/agent-catalog' import { CommandItem } from '@/components/ui/command' import { PaletteRecentTabStatusDot } from '@/components/cmd-j/palette-live-status' -import { RepoBadgeMark } from '@/components/repo/RepoBadgeLabel' import { getPaletteHostBadge } from '@/components/cmd-j/palette-host-badge' import { cn } from '@/lib/utils' import { translate } from '@/i18n/i18n' import type { WorkspaceTabPaletteItem } from './worktree-jump-palette-model' import type { WorktreeJumpPaletteController } from './use-worktree-jump-palette-controller' import { - HighlightedText, PaletteHostBadgeChip, + PaletteLocationChip, PaletteOpenTabPrimaryLine, PaletteRowShortcutBadge } from './worktree-jump-palette-primitives' import { formatPaletteSessionAge } from '@/components/cmd-j/palette-session-age' import { resolvePaletteRepoForWorktree } from '@/lib/palette-repo-resolution' +import { isEditorTabContentType } from '@/store/slices/editor/tabs/editor-tab-content-type' export function WorktreeJumpPaletteWorkspaceTabRow({ entry, @@ -76,8 +76,7 @@ export function WorktreeJumpPaletteWorkspaceTabRow({ secondaryText={result.secondaryText} secondaryRanges={result.secondaryRanges} secondaryMatches={result.secondaryMatches} - worktreeName={result.worktreeName} - worktreeRanges={result.worktreeRanges} + elideSecondaryPathHead={isEditorTabContentType(result.contentType)} sessionAge={sessionAge} leadingBadges={ <> @@ -103,16 +102,15 @@ export function WorktreeJumpPaletteWorkspaceTabRow({ ) : null}
-
+
- {workspaceTabRepoName && ( - - - - - - - )} + Date: Thu, 10 Sep 2026 22:37:52 -0400 Subject: [PATCH 4/8] refactor(mobile): name the RPC acceptance policies call sites hand-rolled (#19960) --- .../codex-reset-credit-capability.ts | 6 +- .../src/host-screen/use-host-repo-metadata.ts | 2 +- ...le-structured-agent-session-launch.test.ts | 28 +++ .../mobile-structured-agent-session-launch.ts | 15 +- .../terminal/terminal-viewport-refit-state.ts | 16 +- .../transport/mobile-relay-direct-upgrade.ts | 27 ++- .../mobile-relay-pairing-recovery.ts | 14 +- .../src/transport/mobile-relay-rpc-streams.ts | 3 +- .../pre-profile-pairing-coordinator.ts | 25 ++- .../transport/rpc-acceptance-policies.test.ts | 168 ++++++++++++++++++ .../src/transport/rpc-acceptance-policies.ts | 33 ++++ .../transport/rpc-client-stream-registry.ts | 3 +- 12 files changed, 278 insertions(+), 62 deletions(-) create mode 100644 mobile/src/transport/rpc-acceptance-policies.test.ts create mode 100644 mobile/src/transport/rpc-acceptance-policies.ts diff --git a/mobile/src/components/codex-reset-credit-capability.ts b/mobile/src/components/codex-reset-credit-capability.ts index 1a32ef37873..8e88f74f2f9 100644 --- a/mobile/src/components/codex-reset-credit-capability.ts +++ b/mobile/src/components/codex-reset-credit-capability.ts @@ -2,6 +2,7 @@ import { useEffect, useState } from 'react' import { CODEX_RESET_CREDIT_RUNTIME_CAPABILITY } from '../../../src/shared/protocol-version' import type { RpcClient } from '../transport/rpc-client' import { startRuntimeCapabilityProbe } from '../transport/runtime-capability-probe' +import { rpcObjectResultOrNull } from '../transport/rpc-acceptance-policies' // Why: source the capability string from the shared contract so a host bump can never // silently drift from the mobile probe. @@ -12,10 +13,7 @@ export async function readCodexResetCreditCapability( ): Promise { try { const response = await client.sendRequest('status.get') - if (!response.ok || !response.result || typeof response.result !== 'object') { - return false - } - const capabilities = (response.result as { capabilities?: unknown }).capabilities + const capabilities = rpcObjectResultOrNull(response)?.capabilities return ( Array.isArray(capabilities) && capabilities.includes(MOBILE_CODEX_RESET_CREDIT_CAPABILITY) ) diff --git a/mobile/src/host-screen/use-host-repo-metadata.ts b/mobile/src/host-screen/use-host-repo-metadata.ts index 198efbaf3ea..6840184c3a0 100644 --- a/mobile/src/host-screen/use-host-repo-metadata.ts +++ b/mobile/src/host-screen/use-host-repo-metadata.ts @@ -18,7 +18,7 @@ type SshTargetSummaryRow = { id: string; label: string } async function requestResult(client: RpcClient, method: string): Promise { try { const response = await client.sendRequest(method) - return response.ok ? (response as RpcSuccess).result : null + return response.ok ? response.result : null } catch { // Best-effort: hosts that predate a method still list repos; labels degrade to host ids. return null diff --git a/mobile/src/session/mobile-structured-agent-session-launch.test.ts b/mobile/src/session/mobile-structured-agent-session-launch.test.ts index 8a020d2eea9..2deb3042ca2 100644 --- a/mobile/src/session/mobile-structured-agent-session-launch.test.ts +++ b/mobile/src/session/mobile-structured-agent-session-launch.test.ts @@ -228,6 +228,34 @@ describe('mobile structured agent-session launch', () => { }) }) + describe.each(['top-level', 'nested'])('%s refusal messages', (location) => { + function refusalClient(message: unknown) { + const refusal = { code: 'method_not_found', ...(message === undefined ? {} : { message }) } + return clientReturning( + { ok: true, result: { supported: true } }, + location === 'top-level' + ? { ok: false, error: refusal } + : { ok: true, result: { ok: false, refusal } } + ) + } + + it.each( + [undefined, null, 42, false, { text: 'unavailable' }, ['unavailable']].map((message) => ({ + message + })) + )('keeps a malformed message $message unknown', async ({ message }) => { + await expect( + createMobileStructuredAgentSession(refusalClient(message), 'workspace-1', 'codex') + ).resolves.toMatchObject({ kind: 'unknown' }) + }) + + it('preserves the fallback for an empty string message', async () => { + await expect( + createMobileStructuredAgentSession(refusalClient(''), 'workspace-1', 'codex') + ).resolves.toEqual({ kind: 'failed', message: 'Could not open Codex chat.' }) + }) + }) + it.each(['structured_agent_session_unsupported', 'method_not_found'])( 'treats a top-level %s as a definitive refusal', async (code) => { diff --git a/mobile/src/session/mobile-structured-agent-session-launch.ts b/mobile/src/session/mobile-structured-agent-session-launch.ts index 9e26eaab91e..bd15e595736 100644 --- a/mobile/src/session/mobile-structured-agent-session-launch.ts +++ b/mobile/src/session/mobile-structured-agent-session-launch.ts @@ -137,18 +137,22 @@ export async function createMobileStructuredAgentSession( } } + // Why: this path distrusts the declared RpcResponse type — a malformed reply must read as + // unconfirmed, not as a refusal we can classify. if (!response || typeof response !== 'object' || typeof response.ok !== 'boolean') { return unknownCreateResult(agent, new Error(unconfirmedMessage(agent))) } if (!response.ok) { + const error = response.error as { code?: unknown; message?: unknown } | null | undefined if ( - !response.error || - typeof response.error !== 'object' || - typeof response.error.code !== 'string' + !error || + typeof error !== 'object' || + typeof error.code !== 'string' || + typeof error.message !== 'string' ) { return unknownCreateResult(agent, new Error(unconfirmedMessage(agent))) } - return classifyCreateRefusal(agent, response.error.code, response.error.message) + return classifyCreateRefusal(agent, error.code, error.message) } const result = response.result as AgentSessionMutationResult if (!result || typeof result !== 'object' || typeof result.ok !== 'boolean') { @@ -158,7 +162,8 @@ export async function createMobileStructuredAgentSession( if ( !result.refusal || typeof result.refusal !== 'object' || - typeof result.refusal.code !== 'string' + typeof result.refusal.code !== 'string' || + typeof result.refusal.message !== 'string' ) { return unknownCreateResult(agent, new Error(unconfirmedMessage(agent))) } diff --git a/mobile/src/terminal/terminal-viewport-refit-state.ts b/mobile/src/terminal/terminal-viewport-refit-state.ts index dfa4daadf4c..766345d8c73 100644 --- a/mobile/src/terminal/terminal-viewport-refit-state.ts +++ b/mobile/src/terminal/terminal-viewport-refit-state.ts @@ -1,4 +1,8 @@ import type { RpcResponse } from '../transport/types' +import { + isMethodNotFoundRefusal, + rpcObjectResultOrNull +} from '../transport/rpc-acceptance-policies' export type TerminalUpdateViewportCapability = 'unknown' | 'supported' | 'unsupported' @@ -14,17 +18,11 @@ export type TerminalViewportRefitTargetState = { } export function isTerminalUpdateViewportUpdated(response: RpcResponse): boolean { - if (!response.ok || typeof response.result !== 'object' || response.result == null) { - return false - } - return (response.result as { updated?: unknown }).updated === true + return rpcObjectResultOrNull(response)?.updated === true } export function isTerminalUpdateViewportApplied(response: RpcResponse): boolean { - if (!response.ok || typeof response.result !== 'object' || response.result == null) { - return false - } - return (response.result as { applied?: unknown }).applied === true + return rpcObjectResultOrNull(response)?.applied === true } export function resolveTerminalUpdateViewportCapability( @@ -33,7 +31,7 @@ export function resolveTerminalUpdateViewportCapability( if (response.ok) { return 'supported' } - return response.error.code === 'method_not_found' ? 'unsupported' : 'unknown' + return isMethodNotFoundRefusal(response) ? 'unsupported' : 'unknown' } // Why: defer height refits while typing, then coalesce every skipped layout diff --git a/mobile/src/transport/mobile-relay-direct-upgrade.ts b/mobile/src/transport/mobile-relay-direct-upgrade.ts index b804d6fb610..ed019262bca 100644 --- a/mobile/src/transport/mobile-relay-direct-upgrade.ts +++ b/mobile/src/transport/mobile-relay-direct-upgrade.ts @@ -21,7 +21,11 @@ import { type MobileRelayDirectUpgradeJournal } from './mobile-relay-direct-upgrade-journal' import type { RpcClient } from './rpc-client' -import type { HostProfile, RpcResponse } from './types' +import type { HostProfile } from './types' +import { + isMethodNotFoundRefusal, + requireRpcResultOrThrowCodedError +} from './rpc-acceptance-policies' export type MobileRelayDirectUpgradeResult = { host: HostProfile @@ -79,11 +83,13 @@ export async function upgradeDirectMobileRelay(args: { reqId: journal.reqId, newResumeTokenHash: journal.pendingResumeTokenHash }) - if (isMethodNotFound(provisionResponse)) { + if (isMethodNotFoundRefusal(provisionResponse)) { await dependencies.clearJournal(args.host.id) return null } - const installed = DeviceCredentialInstalledSchema.parse(requireSuccess(provisionResponse)) + const installed = DeviceCredentialInstalledSchema.parse( + requireRpcResultOrThrowCodedError(provisionResponse) + ) assertDirectInstall(journal, installed) const reconciled = await getEndpoints(args.client, journal.reqId) if (reconciled === 'method-not-found') { @@ -136,10 +142,10 @@ async function getEndpoints( installReqId: string ): Promise { const response = await client.sendRequest('pairing.getEndpoints', { installReqId }) - if (isMethodNotFound(response)) { + if (isMethodNotFoundRefusal(response)) { return 'method-not-found' } - return PairingGetEndpointsResultSchema.parse(requireSuccess(response)) + return PairingGetEndpointsResultSchema.parse(requireRpcResultOrThrowCodedError(response)) } function assertDirectInstall( @@ -162,14 +168,3 @@ function assertCommitted( throw new Error('relay credential install was not authoritatively reconciled') } } - -function requireSuccess(response: RpcResponse): unknown { - if (!response.ok) { - throw new Error(`${response.error.code}: ${response.error.message}`) - } - return response.result -} - -function isMethodNotFound(response: RpcResponse): boolean { - return !response.ok && response.error.code === 'method_not_found' -} diff --git a/mobile/src/transport/mobile-relay-pairing-recovery.ts b/mobile/src/transport/mobile-relay-pairing-recovery.ts index 26dc08620de..b37a18a06c2 100644 --- a/mobile/src/transport/mobile-relay-pairing-recovery.ts +++ b/mobile/src/transport/mobile-relay-pairing-recovery.ts @@ -25,7 +25,8 @@ import { type PairingCandidateClient } from './mobile-relay-physical-client' import { createRecoveringPairingRelayCandidate } from './pairing-relay-candidate' -import type { HostProfile, RpcResponse } from './types' +import type { HostProfile } from './types' +import { requireRpcResultOrThrowCodedError } from './rpc-acceptance-policies' export type MobileRelayPairingRecoveryResult = 'none' | 'recovered' | 'deferred' | 'abandoned' @@ -128,7 +129,7 @@ async function runRecovery( if (credential.kind === 'invite' && endpoints.installStatus?.state === 'not-found') { journal = await transitionToInviteAuthorization(journal, dependencies) const installed = DeviceCredentialInstalledSchema.parse( - requireSuccess( + requireRpcResultOrThrowCodedError( await client.sendRequest('pairing.provisionRelay', { reqId: journal.metadata.installReqId, newResumeTokenHash: journal.metadata.pendingResumeTokenHash @@ -220,7 +221,7 @@ async function getRecoveryStatus( kind: 'resume' | 'invite' ) { return PairingGetEndpointsResultSchema.parse( - requireSuccess( + requireRpcResultOrThrowCodedError( await client.sendRequest('pairing.getEndpoints', { installReqId: journal.metadata.installReqId, ...(kind === 'resume' ? { resumeConfirmReqId: journal.metadata.resumeConfirmReqId } : {}) @@ -294,13 +295,6 @@ function pairingRelay(journal: MobileRelayPairingJournal): PairingRelay { return { ...journal.metadata.relay, inviteToken: journal.secrets.inviteToken } } -function requireSuccess(response: RpcResponse): unknown { - if (!response.ok) { - throw new Error(`${response.error.code}: ${response.error.message}`) - } - return response.result -} - function assertCommitted( endpoints: ReturnType, installed: DeviceCredentialInstalled diff --git a/mobile/src/transport/mobile-relay-rpc-streams.ts b/mobile/src/transport/mobile-relay-rpc-streams.ts index abb2c12564b..3af5b2b3f5e 100644 --- a/mobile/src/transport/mobile-relay-rpc-streams.ts +++ b/mobile/src/transport/mobile-relay-rpc-streams.ts @@ -9,6 +9,7 @@ import { updateTerminalSubscriptionViewport } from './rpc-client-terminal-subscription' import { buildReadyStreamUnsubscribe } from './rpc-client-server-subscription' +import { isStreamingOpenerReply } from './rpc-acceptance-policies' import type { RpcClient } from './rpc-client' import type { RpcResponse, RpcSuccess } from './types' @@ -119,7 +120,7 @@ export class MobileRelayRpcStreams { } } } - if (response.ok && response.streaming !== true) { + if (response.ok && !isStreamingOpenerReply(response)) { this.cancelledSubscriptions.delete(response.id) } return true diff --git a/mobile/src/transport/pre-profile-pairing-coordinator.ts b/mobile/src/transport/pre-profile-pairing-coordinator.ts index a9b04320471..eb1f524334e 100644 --- a/mobile/src/transport/pre-profile-pairing-coordinator.ts +++ b/mobile/src/transport/pre-profile-pairing-coordinator.ts @@ -7,7 +7,11 @@ import { } from '../../../src/shared/mobile-relay-credential-contract' import { connect, type ConnectOptions } from './rpc-client' import { resolvePairingHostIdentity, saveHost } from './host-store' -import type { HostProfile, PairingOffer, RpcResponse } from './types' +import type { HostProfile, PairingOffer } from './types' +import { + isMethodNotFoundRefusal, + requireRpcResultOrThrowCodedError +} from './rpc-acceptance-policies' import { createMobileRelayPairingJournal, type MobileRelayPairingJournal @@ -219,7 +223,7 @@ async function runPairing( reqId: journal.metadata.installReqId, newResumeTokenHash: journal.metadata.pendingResumeTokenHash }) - if (isMethodNotFound(provision)) { + if (isMethodNotFoundRefusal(provision)) { if (winner.path !== 'direct') { throw new Error('relay pairing RPC unavailable after relay path authentication') } @@ -227,9 +231,11 @@ async function runPairing( await dependencies.clearJournal(journal.metadata.journalId) return { hostId } } - const installed = DeviceCredentialInstalledSchema.parse(requireSuccess(provision)) + const installed = DeviceCredentialInstalledSchema.parse( + requireRpcResultOrThrowCodedError(provision) + ) const endpoints = PairingGetEndpointsResultSchema.parse( - requireSuccess( + requireRpcResultOrThrowCodedError( await winner.client.sendRequest('pairing.getEndpoints', { installReqId: journal.metadata.installReqId }) @@ -283,17 +289,6 @@ function relayWebSocketUrl(relay: MobileRelayEndpoint): string { return url.toString() } -function requireSuccess(response: RpcResponse): unknown { - if (!response.ok) { - throw new Error(`${response.error.code}: ${response.error.message}`) - } - return response.result -} - -function isMethodNotFound(response: RpcResponse): boolean { - return !response.ok && response.error.code === 'method_not_found' -} - function assertCommittedInstall( status: | { state: 'not-found' } diff --git a/mobile/src/transport/rpc-acceptance-policies.test.ts b/mobile/src/transport/rpc-acceptance-policies.test.ts new file mode 100644 index 00000000000..2f2abc90e16 --- /dev/null +++ b/mobile/src/transport/rpc-acceptance-policies.test.ts @@ -0,0 +1,168 @@ +import { describe, expect, it } from 'vitest' +import type { RpcResponse } from './types' +import { + isMethodNotFoundRefusal, + isStreamingOpenerReply, + requireRpcResultOrThrowCodedError, + rpcObjectResultOrNull +} from './rpc-acceptance-policies' + +const meta = { runtimeId: 'runtime-1' } + +function success(result: unknown, streaming?: true): RpcResponse { + return { id: 'rpc-1', ok: true, result, _meta: meta, ...(streaming ? { streaming } : {}) } +} + +function refusal(code: string, message = 'Nope'): RpcResponse { + return { id: 'rpc-1', ok: false, error: { code, message }, _meta: meta } +} + +/** Every result partition a policy has to survive. */ +const resultPartitions: [string, unknown][] = [ + ['object result', { value: 1 }], + ['undefined result', undefined], + ['null result', null], + ['empty object result', {}], + ['numeric result', 7], + ['zero result', 0], + ['string result', 'done'], + ['empty string result', ''], + ['boolean result', false], + ['array result', [1, 2]], + ['empty array result', []] +] + +describe('requireRpcResultOrThrowCodedError', () => { + it.each(resultPartitions)('returns the %s untouched', (_label, result) => { + expect(requireRpcResultOrThrowCodedError(success(result))).toEqual(result) + }) + + it('returns an absent result field as undefined', () => { + const response = { id: 'rpc-1', ok: true, _meta: meta } as unknown as RpcResponse + expect(requireRpcResultOrThrowCodedError(response)).toBeUndefined() + }) + + it('throws a code-prefixed message on refusal', () => { + expect(() => requireRpcResultOrThrowCodedError(refusal('method_not_found', 'no such'))).toThrow( + 'method_not_found: no such' + ) + }) + + it('throws even when the refusal carries an empty message', () => { + expect(() => requireRpcResultOrThrowCodedError(refusal('runtime_error', ''))).toThrow( + 'runtime_error: ' + ) + }) +}) + +describe('rpcObjectResultOrNull', () => { + it('accepts a plain object result', () => { + expect(rpcObjectResultOrNull(success({ value: 1 }))).toEqual({ value: 1 }) + }) + + it('accepts an empty object result', () => { + expect(rpcObjectResultOrNull(success({}))).toEqual({}) + }) + + it('accepts an array result, because arrays are objects', () => { + expect(rpcObjectResultOrNull(success([1, 2]))).toEqual([1, 2]) + }) + + it.each([ + ['null', null], + ['undefined', undefined], + ['numeric', 7], + ['zero', 0], + ['string', 'done'], + ['empty string', ''], + ['boolean', false], + ['true', true] + ])('refuses a %s result', (_label, result) => { + expect(rpcObjectResultOrNull(success(result))).toBeNull() + }) + + it('refuses a refusal regardless of its code', () => { + expect(rpcObjectResultOrNull(refusal('method_not_found'))).toBeNull() + expect(rpcObjectResultOrNull(refusal('runtime_error'))).toBeNull() + }) +}) + +// A refusal that illegally carries success-shaped fields: without the `ok` check each of these +// would read the stray field and answer as if the call had succeeded. +describe('a refusal carrying stray success fields', () => { + const strayRefusal = { + id: 'rpc-1', + ok: false, + error: { code: 'method_not_found', message: 'Nope' }, + result: { value: 1 }, + streaming: true, + _meta: meta + } as unknown as RpcResponse + + it('yields null rather than the stray result', () => { + expect(rpcObjectResultOrNull(strayRefusal)).toBeNull() + }) + + it('is still recognised as method-not-found', () => { + expect(isMethodNotFoundRefusal(strayRefusal)).toBe(true) + }) + + // The mirror case: a success carrying a stray error must not read as a refusal. + it('does not read a success carrying a stray error as a refusal', () => { + const straySuccess = { + id: 'rpc-1', + ok: true, + result: { value: 1 }, + error: { code: 'method_not_found', message: 'Nope' }, + _meta: meta + } as unknown as RpcResponse + expect(isMethodNotFoundRefusal(straySuccess)).toBe(false) + }) +}) + +describe('isMethodNotFoundRefusal', () => { + it('matches only the method_not_found code', () => { + expect(isMethodNotFoundRefusal(refusal('method_not_found'))).toBe(true) + expect(isMethodNotFoundRefusal(refusal('runtime_error'))).toBe(false) + expect(isMethodNotFoundRefusal(refusal('METHOD_NOT_FOUND'))).toBe(false) + }) + + it('never matches a success, including one with a null result', () => { + expect(isMethodNotFoundRefusal(success(null))).toBe(false) + expect(isMethodNotFoundRefusal(success({ code: 'method_not_found' }))).toBe(false) + }) +}) + +describe('isStreamingOpenerReply', () => { + it('accepts a success flagged streaming', () => { + expect(isStreamingOpenerReply(success({ subscriptionId: 's1' }, true))).toBe(true) + }) + + it('refuses a success with no streaming flag', () => { + expect(isStreamingOpenerReply(success({ subscriptionId: 's1' }))).toBe(false) + }) + + // A truthy non-boolean off the wire must not open a stream: the registry would route it to + // handleStreamingResponse and wait for frames that never come. + it.each([['yes'], [1], [{}]])('refuses a truthy non-boolean streaming flag %j', (flag) => { + const response = { + id: 'rpc-1', + ok: true, + result: { subscriptionId: 's1' }, + streaming: flag, + _meta: meta + } as unknown as RpcResponse + expect(isStreamingOpenerReply(response)).toBe(false) + }) + + it('refuses a refusal even when it carries a streaming flag', () => { + const response = { + id: 'rpc-1', + ok: false, + error: { code: 'runtime_error', message: 'Nope' }, + streaming: true, + _meta: meta + } as unknown as RpcResponse + expect(isStreamingOpenerReply(response)).toBe(false) + }) +}) diff --git a/mobile/src/transport/rpc-acceptance-policies.ts b/mobile/src/transport/rpc-acceptance-policies.ts new file mode 100644 index 00000000000..742d94aafa7 --- /dev/null +++ b/mobile/src/transport/rpc-acceptance-policies.ts @@ -0,0 +1,33 @@ +import type { RpcResponse, RpcSuccess } from './types' + +// Named acceptance policies for RPC replies. Call sites used to hand-roll these +// predicates and did not agree with each other; each policy here preserves one +// call site's existing acceptance exactly. Do not merge two policies without +// proving every caller of both tolerates the wider or narrower set. + +/** Throws `code: message` on refusal. Diagnostic text; not user-facing. */ +export function requireRpcResultOrThrowCodedError(response: RpcResponse): unknown { + if (!response.ok) { + throw new Error(`${response.error.code}: ${response.error.message}`) + } + return response.result +} + +/** Accepts only a success whose result is a non-null object. Arrays qualify. */ +export function rpcObjectResultOrNull(response: RpcResponse): Record | null { + if (!response.ok || typeof response.result !== 'object' || response.result === null) { + return null + } + return response.result as Record +} + +export function isMethodNotFoundRefusal(response: RpcResponse): boolean { + return !response.ok && response.error.code === 'method_not_found' +} + +/** A success that opened a stream rather than delivering a terminal result. */ +export function isStreamingOpenerReply( + response: RpcResponse +): response is RpcSuccess & { streaming: true } { + return response.ok && response.streaming === true +} diff --git a/mobile/src/transport/rpc-client-stream-registry.ts b/mobile/src/transport/rpc-client-stream-registry.ts index ac20acdecc0..68e50a6e6dd 100644 --- a/mobile/src/transport/rpc-client-stream-registry.ts +++ b/mobile/src/transport/rpc-client-stream-registry.ts @@ -8,6 +8,7 @@ import { updateTerminalSubscriptionViewport } from './rpc-client-terminal-subscription' import { buildReadyStreamUnsubscribe } from './rpc-client-server-subscription' +import { isStreamingOpenerReply } from './rpc-acceptance-policies' import { isStreamingSubscriptionReadyResult, isTerminalSubscribedResult @@ -112,7 +113,7 @@ export class RpcClientStreamRegistry { } handleResponse(response: RpcResponse): boolean { - if (response.ok && response.streaming === true) { + if (isStreamingOpenerReply(response)) { this.handleStreamingResponse(response) return true } From 47b6c756f0b5fb0c8110ca60250eb34c43a21741 Mon Sep 17 00:00:00 2001 From: Jinwoo Hong <73622457+Jinwoo-H@users.noreply.github.com> Date: Thu, 10 Sep 2026 23:26:14 -0400 Subject: [PATCH 5/8] fix: prompt unexpectedly signed-out Cloud users once per version (#19966) * fix: prompt unexpectedly signed-out Cloud users once per version * fix: align sign-in card English catalog with runtime defaults * fix: stack notification cards by their rendered height * fix: wait for fresh auth before showing signout card * fix: require verified auth before signout recovery * fix: retry transient auth readiness failures --- .../profile-cloud-auth-status.test.ts | 106 +++++++ .../runtime/rpc/methods/client-ui-schemas.ts | 1 + .../runtime/rpc/methods/client-ui.test.ts | 2 + .../src/app-shell/AppRootSurfaces.tsx | 27 +- .../src/components/NotificationCardStack.tsx | 9 + src/renderer/src/components/StarNagCard.tsx | 17 +- .../src/components/UnexpectedSignoutCard.tsx | 269 ++++++++++++++++++ .../components/UpdateCard.error-card.test.tsx | 7 +- src/renderer/src/components/UpdateCard.tsx | 5 +- .../unexpected-signout-card.test.tsx | 152 ++++++++++ .../unexpected-signout-visibility.test.ts | 150 ++++++++++ .../unexpected-signout-visibility.ts | 22 ++ src/renderer/src/i18n/locales/en.json | 15 + .../store/slices/ui-notice-dismissals.test.ts | 78 +++++ .../ui/ui-slice-contract-preferences.ts | 4 + .../slices/ui/ui-slice-hydration-actions.ts | 2 + .../ui/ui-slice-hydration-sanitizers.ts | 13 + .../slices/ui/ui-slice-update-actions.ts | 12 + src/shared/constants.ts | 1 + src/shared/persisted-ui-state-types.ts | 2 + 20 files changed, 866 insertions(+), 28 deletions(-) create mode 100644 src/main/orca-profiles/profile-cloud-auth-status.test.ts create mode 100644 src/renderer/src/components/NotificationCardStack.tsx create mode 100644 src/renderer/src/components/UnexpectedSignoutCard.tsx create mode 100644 src/renderer/src/components/unexpected-signout/unexpected-signout-card.test.tsx create mode 100644 src/renderer/src/components/unexpected-signout/unexpected-signout-visibility.test.ts create mode 100644 src/renderer/src/components/unexpected-signout/unexpected-signout-visibility.ts diff --git a/src/main/orca-profiles/profile-cloud-auth-status.test.ts b/src/main/orca-profiles/profile-cloud-auth-status.test.ts new file mode 100644 index 00000000000..7a28783e9a9 --- /dev/null +++ b/src/main/orca-profiles/profile-cloud-auth-status.test.ts @@ -0,0 +1,106 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest' +import type { ActiveOrcaProfileState } from './profile-index-store' +import type { OrcaCloudSessionReadResult } from './profile-cloud-session-store' +import { getOrcaProfileAuthStatusFromProfile } from './profile-cloud-auth-status' + +const { readSession, configuration } = vi.hoisted(() => ({ + readSession: vi.fn<() => OrcaCloudSessionReadResult>(), + configuration: { configured: true } +})) + +vi.mock('./profile-cloud-session-store', () => ({ readOrcaCloudSession: readSession })) +vi.mock('./profile-cloud-auth-config', () => ({ + getOrcaCloudAuthConfig: () => configuration, + isOrcaCloudDevAuthEnabled: () => false +})) + +function activeProfile(linked: boolean): ActiveOrcaProfileState { + const profile: ActiveOrcaProfileState['profile'] = { + id: 'profile-1', + name: 'Personal', + avatar: { kind: 'initials', initials: 'P', color: 'neutral' }, + kind: linked ? 'cloud-linked' : 'local', + createdAt: 0, + updatedAt: 0, + lastOpenedAt: 0, + ...(linked + ? { + cloud: { + cloudProfileId: 'cloud-1', + userId: 'user-1', + email: 'a@example.com', + linkedAt: 0 + } + } + : {}) + } + return { + profile, + index: { schemaVersion: 1, activeProfileId: profile.id, profiles: [profile] }, + dataFile: '', + profileDirectory: '' + } +} + +const absentSessions: OrcaCloudSessionReadResult[] = [ + { status: 'missing', persistence: 'none' }, + { status: 'decrypt-failed', persistence: 'none', error: 'Cannot decrypt' }, + { status: 'unreadable', persistence: 'none', error: 'Permission denied' } +] + +describe('unexpected sign-out auth evidence', () => { + beforeEach(() => { + readSession.mockReset() + configuration.configured = true + }) + + it.each(absentSessions)('requires a preserved cloud link for $status credentials', (session) => { + readSession.mockReturnValue(session) + const linked = activeProfile(true) + expect(getOrcaProfileAuthStatusFromProfile(linked, '')).toMatchObject({ + state: 'reconnect-required', + cloud: linked.profile.cloud, + persistence: 'none', + credentialError: 'error' in session ? session.error : undefined + }) + readSession.mockClear() + const signedOut = getOrcaProfileAuthStatusFromProfile(activeProfile(false), '') + expect(signedOut.state).toBe('local') + expect(signedOut.cloud).toBeUndefined() + expect(readSession).not.toHaveBeenCalled() + }) + + it.each(absentSessions)( + 'keeps unconfigured linked profiles out of reconnect for $status', + (session) => { + configuration.configured = false + readSession.mockReturnValue(session) + expect(getOrcaProfileAuthStatusFromProfile(activeProfile(true), '').state).toBe( + 'unconfigured' + ) + expect(getOrcaProfileAuthStatusFromProfile(activeProfile(false), '').state).toBe( + 'unconfigured' + ) + } + ) + + it('treats a live memory-only session as connected, then reconnects after its loss', () => { + readSession.mockReturnValue({ + status: 'found', + persistence: 'memory-only', + session: { + accessToken: 'access', + refreshToken: 'refresh', + expiresAt: Date.now() + 60_000, + capabilities: { flags: {}, refreshedAt: 0 } + } + }) + const linked = activeProfile(true) + expect(getOrcaProfileAuthStatusFromProfile(linked, '')).toMatchObject({ + state: 'connected', + persistence: 'memory-only' + }) + readSession.mockReturnValue({ status: 'missing', persistence: 'none' }) + expect(getOrcaProfileAuthStatusFromProfile(linked, '').state).toBe('reconnect-required') + }) +}) diff --git a/src/main/runtime/rpc/methods/client-ui-schemas.ts b/src/main/runtime/rpc/methods/client-ui-schemas.ts index 943d0081fdf..32e62e105d7 100644 --- a/src/main/runtime/rpc/methods/client-ui-schemas.ts +++ b/src/main/runtime/rpc/methods/client-ui-schemas.ts @@ -171,6 +171,7 @@ const UiUpdateFields = z usagePercentageDisplay: z.enum(['used', 'remaining']).optional(), statusBarUsageMode: z.enum(['verbose', 'compact']).optional(), dismissedUpdateVersion: NullableString.optional(), + dismissedUnexpectedSignoutVersion: NullableString.optional(), lastUpdateCheckAt: z.number().finite().nullable().optional(), pendingUpdateNudgeId: NullableString.optional(), dismissedUpdateNudgeId: NullableString.optional(), diff --git a/src/main/runtime/rpc/methods/client-ui.test.ts b/src/main/runtime/rpc/methods/client-ui.test.ts index 39048611155..3a26b1c615d 100644 --- a/src/main/runtime/rpc/methods/client-ui.test.ts +++ b/src/main/runtime/rpc/methods/client-ui.test.ts @@ -607,6 +607,8 @@ describe('client UI RPC methods', () => { ], ['taskResumeState.jiraPreset', { taskResumeState: { jiraPreset: 'assigned' } }], ['taskResumeState.jiraQuery', { taskResumeState: { jiraQuery: 'ENG' } }], + ['dismissedUnexpectedSignoutVersion', { dismissedUnexpectedSignoutVersion: '1.2.3' }], + ['dismissedUnexpectedSignoutVersion null', { dismissedUnexpectedSignoutVersion: null }], ['activeView', { activeView: 'tasks' }], ['showDotfilesByWorktree', { showDotfilesByWorktree: { 'repo::/worktree': true } }], ['setupGuideSidebarDismissed', { setupGuideSidebarDismissed: true }], diff --git a/src/renderer/src/app-shell/AppRootSurfaces.tsx b/src/renderer/src/app-shell/AppRootSurfaces.tsx index 202c99df4d2..31de9b1ce8d 100644 --- a/src/renderer/src/app-shell/AppRootSurfaces.tsx +++ b/src/renderer/src/app-shell/AppRootSurfaces.tsx @@ -1,3 +1,4 @@ +import { NotificationCardStack } from '../components/NotificationCardStack' import { Suspense } from 'react' import { lazyWithRetry as lazy } from '@/lib/lazy-with-retry' import { translate } from '@/i18n/i18n' @@ -58,6 +59,11 @@ const SshPassphraseDialog = lazy(() => const UpdateCard = lazy(() => import('../components/UpdateCard').then((module) => ({ default: module.UpdateCard })) ) +const UnexpectedSignoutCard = lazy(() => + import('../components/UnexpectedSignoutCard').then((module) => ({ + default: module.UnexpectedSignoutCard + })) +) const RemoteServerUpdateDialog = lazy( () => import('../components/settings/RemoteServerUpdateDialog') ) @@ -273,16 +279,23 @@ export function AppRootSurfaces(props: { ) : null} - {shouldMountUpdateCard ? ( + + {shouldMountUpdateCard ? ( + + + + + + ) : null} - - + + - ) : null} - - - + + + + diff --git a/src/renderer/src/components/NotificationCardStack.tsx b/src/renderer/src/components/NotificationCardStack.tsx new file mode 100644 index 00000000000..9d3d5cddf3c --- /dev/null +++ b/src/renderer/src/components/NotificationCardStack.tsx @@ -0,0 +1,9 @@ +import type { ReactNode } from 'react' + +export function NotificationCardStack({ children }: { children: ReactNode }): React.JSX.Element { + return ( +
+ {children} +
+ ) +} diff --git a/src/renderer/src/components/StarNagCard.tsx b/src/renderer/src/components/StarNagCard.tsx index f0e184143fc..0bf3312463e 100644 --- a/src/renderer/src/components/StarNagCard.tsx +++ b/src/renderer/src/components/StarNagCard.tsx @@ -2,7 +2,6 @@ import { useCallback, useEffect, useState } from 'react' import { ExternalLink, Star, X } from 'lucide-react' import { Card } from './ui/card' import { Button } from './ui/button' -import { useAppStore } from '../store' import { useMountedRef } from '@/hooks/useMountedRef' import { translate } from '@/i18n/i18n' @@ -25,12 +24,6 @@ export function StarNagCard(): React.JSX.Element | null { const [busy, setBusy] = useState(false) const [mode, setMode] = useState('gh') const mountedRef = useMountedRef() - // Why: UpdateCard lives at the same bottom-right slot. When it is visible - // (any non-idle / non-not-available state), stack the star-nag card above - // it instead of overlapping — we must not cover a pending update prompt - // because that's a higher-priority action. - const updateStatus = useAppStore((s) => s.updateStatus) - const updateCardVisible = updateStatus.state !== 'idle' && updateStatus.state !== 'not-available' useEffect(() => { const unsubscribeShow = window.api.starNag.onShow((payload) => { @@ -147,15 +140,7 @@ export function StarNagCard(): React.JSX.Element | null { } return ( -
+
diff --git a/src/renderer/src/components/UnexpectedSignoutCard.tsx b/src/renderer/src/components/UnexpectedSignoutCard.tsx new file mode 100644 index 00000000000..ee0e8949ec3 --- /dev/null +++ b/src/renderer/src/components/UnexpectedSignoutCard.tsx @@ -0,0 +1,269 @@ +import { useEffect, useRef, useState } from 'react' +import { BookOpen, ChevronDown, CircleUserRound, Files, Smartphone, X } from 'lucide-react' +import { useAppStore } from '../store' +import { translate } from '@/i18n/i18n' +import { cn } from '@/lib/utils' +import { Button } from './ui/button' +import { Card } from './ui/card' +import { Collapsible, CollapsibleContent, CollapsibleTrigger } from './ui/collapsible' +import { shouldShowUnexpectedSignoutCard } from './unexpected-signout/unexpected-signout-visibility' + +function readPreviewFlag(): boolean { + if (!import.meta.env.DEV) { + return false + } + try { + if (new URLSearchParams(window.location.search).get('showSignoutCard') === '1') { + return true + } + return window.localStorage.getItem('orca-debug-show-signout-card') === '1' + } catch { + return false + } +} + +function FeatureRow({ + icon: Icon, + title, + description +}: { + icon: typeof Files + title: string + description: string +}): React.JSX.Element { + return ( +
+ +
+

{title}

+

{description}

+
+
+ ) +} + +export function UnexpectedSignoutCard(): React.JSX.Element | null { + const authStatus = useAppStore((s) => s.orcaProfileAuthStatus) + const persistedUIReady = useAppStore((s) => s.persistedUIReady) + const persistedDismissedVersion = useAppStore((s) => s.dismissedUnexpectedSignoutVersion) + const dismissedVersions = useAppStore((s) => s.unexpectedSignoutDismissedVersions) + const dismissForVersion = useAppStore((s) => s.dismissUnexpectedSignoutCard) + const connecting = useAppStore((s) => s.orcaProfileConnecting) + const connect = useAppStore((s) => s.connectCurrentOrcaProfile) + const [appVersion, setAppVersion] = useState(null) + const [authRefreshReady, setAuthRefreshReady] = useState(false) + const [expanded, setExpanded] = useState(false) + const [preview] = useState(readPreviewFlag) + const [previewDismissed, setPreviewDismissed] = useState(false) + const reconnectingProfile = useRef(null) + + useEffect(() => { + let cancelled = false + let attempts = 0 + const refresh = (): void => { + attempts += 1 + void useAppStore + .getState() + .fetchOrcaProfileAuthStatus() + .then((status) => { + if (cancelled) { + return + } + if (status != null) { + setAuthRefreshReady(true) + } else if (attempts < 3) { + window.setTimeout(refresh, 500) + } + }) + } + refresh() + return () => { + cancelled = true + } + }, []) + + useEffect(() => { + let cancelled = false + void window.api.updater + .getVersion() + .then((version) => { + if (!cancelled) { + setAppVersion(version) + } + }) + .catch(() => { + if (!cancelled) { + setAppVersion(null) + } + }) + return () => { + cancelled = true + } + }, []) + + const dismissedVersion = + appVersion && dismissedVersions.includes(appVersion) ? appVersion : persistedDismissedVersion + const eligible = shouldShowUnexpectedSignoutCard({ + authStatus, + persistedUIReady, + appVersion, + dismissedVersion + }) + + const visible = preview ? persistedUIReady && !previewDismissed : authRefreshReady && eligible + + // Observe recovery independently of visibility and asynchronous version/hydration reads. + useEffect(() => { + if (preview || !authRefreshReady) { + return + } + if (authStatus?.state === 'reconnect-required' && authStatus.configured && authStatus.cloud) { + reconnectingProfile.current = authStatus.activeProfileId + } else if (authStatus?.state === 'connected') { + if ( + reconnectingProfile.current === authStatus.activeProfileId && + persistedUIReady && + appVersion + ) { + reconnectingProfile.current = null + if (dismissedVersion !== appVersion) { + dismissForVersion(appVersion) + } + } + } else { + reconnectingProfile.current = null + } + }, [ + preview, + authRefreshReady, + authStatus, + persistedUIReady, + appVersion, + dismissedVersion, + dismissForVersion + ]) + + if (!visible) { + return null + } + + const email = authStatus?.cloud?.email?.trim() || null + const canConnect = authStatus?.configured === true + + const handleDismiss = (): void => { + if (preview) { + setPreviewDismissed(true) + } else if (appVersion) { + dismissForVersion(appVersion) + } + } + + return ( +
+ +
+
+
+ +

+ {translate( + 'auto.components.UnexpectedSignoutCard.9f2c1a4b7d', + "You've been signed out" + )} +

+
+ +
+ +

+ {email + ? translate( + 'auto.components.UnexpectedSignoutCard.7b4d9e1f2a', + 'Sign in again as {{value0}} to restore Artifact sharing, Orca Relay, and skill sharing.', + { value0: email } + ) + : translate( + 'auto.components.UnexpectedSignoutCard.5a1c8d3e6f', + 'Sign in again to restore Artifact sharing, Orca Relay, and skill sharing.' + )} +

+ + + + + + + + + + + + +
+ +
+
+ +
+ ) +} diff --git a/src/renderer/src/components/UpdateCard.error-card.test.tsx b/src/renderer/src/components/UpdateCard.error-card.test.tsx index 1e6da6879f7..186a285b26d 100644 --- a/src/renderer/src/components/UpdateCard.error-card.test.tsx +++ b/src/renderer/src/components/UpdateCard.error-card.test.tsx @@ -5,6 +5,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import type { LinuxPackageInstallRecovery, UpdateStatus } from '../../../shared/update-status-types' import { useAppStore } from '../store' import { UpdateCard } from './UpdateCard' +import { NotificationCardStack } from './NotificationCardStack' const openUrl = vi.fn() const download = vi.fn() @@ -31,7 +32,11 @@ function renderWithInitialStatus(updateStatus: UpdateStatus): RenderResult { updateCardCollapsed: false, updateReassuranceSeen: true }) - return render() + return render( + + + + ) } function renderAfterAvailableStatus(): RenderResult { diff --git a/src/renderer/src/components/UpdateCard.tsx b/src/renderer/src/components/UpdateCard.tsx index 4ccf95ff252..e2023ae21e1 100644 --- a/src/renderer/src/components/UpdateCard.tsx +++ b/src/renderer/src/components/UpdateCard.tsx @@ -239,10 +239,7 @@ export function UpdateCard(): React.JSX.Element | null { !reassuranceSeen && ((status.state === 'available' && !status.externallyManaged) || status.state === 'downloading') return ( -
+
{showReassurance && (
diff --git a/src/renderer/src/components/unexpected-signout/unexpected-signout-card.test.tsx b/src/renderer/src/components/unexpected-signout/unexpected-signout-card.test.tsx new file mode 100644 index 00000000000..f220d803a01 --- /dev/null +++ b/src/renderer/src/components/unexpected-signout/unexpected-signout-card.test.tsx @@ -0,0 +1,152 @@ +// @vitest-environment happy-dom +import { act, cleanup, fireEvent, render, screen, waitFor } from '@testing-library/react' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { useAppStore } from '../../store' +import { getDefaultUIState } from '../../../../shared/constants' +import { UnexpectedSignoutCard } from '../UnexpectedSignoutCard' +import type { OrcaProfileAuthStatus } from '../../../../shared/orca-profiles' + +const status: OrcaProfileAuthStatus = { + activeProfileId: 'profile-1', + configured: true, + state: 'reconnect-required', + persistence: 'none', + cloud: { + cloudProfileId: 'cloud-1', + userId: 'user-1', + email: 'user@example.com', + displayName: 'User', + linkedAt: 0 + } +} +const persist = vi.fn().mockResolvedValue(undefined) + +beforeEach(() => { + vi.stubEnv('DEV', false) + persist.mockClear() + window.localStorage.clear() + window.history.replaceState({}, '', '/') + Object.defineProperty(window, 'api', { + configurable: true, + value: { + ui: { set: persist }, + updater: { getVersion: vi.fn().mockResolvedValue('1.4.197') } + } + }) + useAppStore.setState(useAppStore.getInitialState(), true) + useAppStore.setState({ + orcaProfileAuthStatus: status, + persistedUIReady: true, + fetchOrcaProfileAuthStatus: vi.fn().mockResolvedValue(status) + }) +}) +afterEach(() => { + cleanup() + vi.unstubAllEnvs() +}) + +async function showCard(): Promise { + render() + await screen.findByRole('complementary') +} + +describe('unexpected signout lifecycle', () => { + it('stamps successful sign-in and never re-arms in the same version', async () => { + await showCard() + act(() => useAppStore.setState({ orcaProfileAuthStatus: { ...status, state: 'connected' } })) + await waitFor(() => + expect(persist).toHaveBeenCalledWith({ dismissedUnexpectedSignoutVersion: '1.4.197' }) + ) + act(() => useAppStore.setState({ orcaProfileAuthStatus: status })) + expect(screen.queryByRole('complementary')).toBeNull() + cleanup() + render() + await act(async () => {}) + expect(screen.queryByRole('complementary')).toBeNull() + expect(persist).toHaveBeenCalledTimes(1) + }) + + it('does not treat a cached connected status as a successful re-sign-in', async () => { + useAppStore.setState({ orcaProfileAuthStatus: { ...status, state: 'connected' } }) + render() + await act(async () => {}) + act(() => useAppStore.setState({ orcaProfileAuthStatus: status })) + expect(screen.queryByRole('complementary')).not.toBeNull() + expect(persist).not.toHaveBeenCalled() + }) + + it('waits for hydration before stamping an already recovered session', async () => { + useAppStore.setState({ persistedUIReady: false }) + render() + await act(async () => {}) + act(() => useAppStore.setState({ orcaProfileAuthStatus: { ...status, state: 'connected' } })) + expect(persist).not.toHaveBeenCalled() + act(() => useAppStore.setState({ persistedUIReady: true })) + await waitFor(() => expect(persist).toHaveBeenCalledTimes(1)) + act(() => useAppStore.setState({ orcaProfileAuthStatus: { ...status, state: 'connected' } })) + expect(persist).toHaveBeenCalledTimes(1) + }) + + it('keeps a failed sign-in available without stamping dismissal', async () => { + useAppStore.setState({ connectCurrentOrcaProfile: vi.fn().mockResolvedValue(undefined) }) + await showCard() + fireEvent.click(screen.getByRole('button', { name: 'Sign in to Orca' })) + await act(async () => {}) + expect(screen.queryByRole('complementary')).not.toBeNull() + expect(persist).not.toHaveBeenCalled() + }) + + it('retains a reopened dismissal through stale UI sync and a renderer remount', async () => { + useAppStore.getState().hydratePersistedUI( + { + ...getDefaultUIState(), + dismissedUnexpectedSignoutVersion: '1.4.197' + }, + 'startup' + ) + render() + await act(async () => {}) + act(() => useAppStore.getState().hydratePersistedUI(getDefaultUIState())) + expect(screen.queryByRole('complementary')).toBeNull() + cleanup() + render() + await act(async () => {}) + expect(screen.queryByRole('complementary')).toBeNull() + expect(persist).not.toHaveBeenCalledWith( + expect.objectContaining({ dismissedUnexpectedSignoutVersion: expect.anything() }) + ) + }) + + it('persists X dismissal and stays hidden after remount', async () => { + await showCard() + fireEvent.click(screen.getByRole('button', { name: 'Dismiss' })) + expect(persist).toHaveBeenCalledWith({ dismissedUnexpectedSignoutVersion: '1.4.197' }) + cleanup() + render() + await act(async () => {}) + expect(screen.queryByRole('complementary')).toBeNull() + }) + + it.each(['storage', 'query'])('ignores the %s preview flag in production', async (source) => { + if (source === 'storage') { + window.localStorage.setItem('orca-debug-show-signout-card', '1') + } else { + window.history.replaceState({}, '', '/?showSignoutCard=1') + } + useAppStore.setState({ orcaProfileAuthStatus: { ...status, state: 'local', cloud: undefined } }) + render() + await act(async () => {}) + expect(screen.queryByRole('complementary')).toBeNull() + expect(persist).not.toHaveBeenCalled() + }) + + it('dismisses a development preview without writing real dismissal state', async () => { + vi.stubEnv('DEV', true) + window.localStorage.setItem('orca-debug-show-signout-card', '1') + useAppStore.setState({ orcaProfileAuthStatus: { ...status, state: 'local', cloud: undefined } }) + await showCard() + fireEvent.click(screen.getByRole('button', { name: 'Dismiss' })) + expect(screen.queryByRole('complementary')).toBeNull() + expect(persist).not.toHaveBeenCalled() + }) +}) diff --git a/src/renderer/src/components/unexpected-signout/unexpected-signout-visibility.test.ts b/src/renderer/src/components/unexpected-signout/unexpected-signout-visibility.test.ts new file mode 100644 index 00000000000..c1b8459204f --- /dev/null +++ b/src/renderer/src/components/unexpected-signout/unexpected-signout-visibility.test.ts @@ -0,0 +1,150 @@ +import { describe, expect, it } from 'vitest' +import type { OrcaProfileAuthStatus } from '../../../../shared/orca-profiles' +import { shouldShowUnexpectedSignoutCard } from './unexpected-signout-visibility' + +function reconnectRequired(): OrcaProfileAuthStatus { + return { + activeProfileId: 'profile-1', + configured: true, + state: 'reconnect-required', + persistence: 'none', + cloud: { + cloudProfileId: 'cloud-1', + userId: 'user-1', + email: 'user@example.com', + displayName: 'User', + linkedAt: 0 + } + } +} + +describe('shouldShowUnexpectedSignoutCard', () => { + it.each([ + { name: 'unknown auth', auth: null, visible: false }, + { + name: 'missing cloud link', + auth: { ...reconnectRequired(), cloud: undefined }, + visible: false + }, + { + name: 'unconfigured linked profile', + auth: { ...reconnectRequired(), configured: false, state: 'unconfigured' }, + visible: false + }, + { + name: 'unconfigured reconnect status', + auth: { ...reconnectRequired(), configured: false }, + visible: false + }, + { + name: 'local with stale cloud metadata', + auth: { ...reconnectRequired(), state: 'local' }, + visible: false + }, + { + name: 'live memory-only session', + auth: { ...reconnectRequired(), state: 'connected', persistence: 'memory-only' }, + visible: false + }, + { + name: 'decrypt failure with retained link', + auth: { ...reconnectRequired(), credentialError: 'Cannot decrypt' }, + visible: true + }, + { + name: 'unreadable session with retained link', + auth: { ...reconnectRequired(), credentialError: 'Permission denied' }, + visible: true + } + ] satisfies { name: string; auth: OrcaProfileAuthStatus | null; visible: boolean }[])( + '$name', + ({ auth, visible }) => { + expect( + shouldShowUnexpectedSignoutCard({ + authStatus: auth, + persistedUIReady: true, + appVersion: '1.4.197', + dismissedVersion: null + }) + ).toBe(visible) + } + ) + + it('shows when linked but the session is gone', () => { + expect( + shouldShowUnexpectedSignoutCard({ + authStatus: reconnectRequired(), + persistedUIReady: true, + appVersion: '1.4.197', + dismissedVersion: null + }) + ).toBe(true) + }) + + it('hides after an explicit sign-out (link removed)', () => { + expect( + shouldShowUnexpectedSignoutCard({ + authStatus: { + activeProfileId: 'profile-1', + configured: true, + state: 'local', + persistence: 'none' + }, + persistedUIReady: true, + appVersion: '1.4.197', + dismissedVersion: null + }) + ).toBe(false) + }) + + it('hides when connected', () => { + const status = reconnectRequired() + status.state = 'connected' + expect( + shouldShowUnexpectedSignoutCard({ + authStatus: status, + persistedUIReady: true, + appVersion: '1.4.197', + dismissedVersion: null + }) + ).toBe(false) + }) + + it('shows only once per app version', () => { + expect( + shouldShowUnexpectedSignoutCard({ + authStatus: reconnectRequired(), + persistedUIReady: true, + appVersion: '1.4.197', + dismissedVersion: '1.4.197' + }) + ).toBe(false) + expect( + shouldShowUnexpectedSignoutCard({ + authStatus: reconnectRequired(), + persistedUIReady: true, + appVersion: '1.4.198', + dismissedVersion: '1.4.197' + }) + ).toBe(true) + }) + + it('waits for hydration and version', () => { + expect( + shouldShowUnexpectedSignoutCard({ + authStatus: reconnectRequired(), + persistedUIReady: false, + appVersion: '1.4.197', + dismissedVersion: null + }) + ).toBe(false) + expect( + shouldShowUnexpectedSignoutCard({ + authStatus: reconnectRequired(), + persistedUIReady: true, + appVersion: null, + dismissedVersion: null + }) + ).toBe(false) + }) +}) diff --git a/src/renderer/src/components/unexpected-signout/unexpected-signout-visibility.ts b/src/renderer/src/components/unexpected-signout/unexpected-signout-visibility.ts new file mode 100644 index 00000000000..a9a47b2d98b --- /dev/null +++ b/src/renderer/src/components/unexpected-signout/unexpected-signout-visibility.ts @@ -0,0 +1,22 @@ +import type { OrcaProfileAuthStatus } from '../../../../shared/orca-profiles' + +export type UnexpectedSignoutGate = { + authStatus: OrcaProfileAuthStatus | null + persistedUIReady: boolean + appVersion: string | null + dismissedVersion: string | null +} + +export function shouldShowUnexpectedSignoutCard(gate: UnexpectedSignoutGate): boolean { + if (!gate.persistedUIReady || gate.appVersion === null) { + return false + } + if (gate.dismissedVersion === gate.appVersion) { + return false + } + return ( + gate.authStatus?.configured === true && + gate.authStatus.state === 'reconnect-required' && + gate.authStatus.cloud != null + ) +} diff --git a/src/renderer/src/i18n/locales/en.json b/src/renderer/src/i18n/locales/en.json index fe4e115ae77..607359d1b71 100644 --- a/src/renderer/src/i18n/locales/en.json +++ b/src/renderer/src/i18n/locales/en.json @@ -16883,6 +16883,21 @@ "HostedReviewUnlinkMenuItem": { "label": "Unlink {{value0}} from workspace", "description": "Orca will hide {{value0}} {{value1}} details for this workspace. The {{value0}} and branch on {{value2}} won’t be changed." + }, + "UnexpectedSignoutCard": { + "9f2c1a4b7d": "You've been signed out", + "3e8f5c2a91": "Dismiss", + "7b4d9e1f2a": "Sign in again as {{value0}} to restore Artifact sharing, Orca Relay, and skill sharing.", + "5a1c8d3e6f": "Sign in again to restore Artifact sharing, Orca Relay, and skill sharing.", + "1f6b2c9d4e": "What you get back", + "8d2e4f7a1b": "Artifact sharing", + "2c9a5b6e8d": "Publish HTML and Markdown files and manage every shared link from Orca.", + "6e3f1a9c5b": "Orca Relay", + "4b7d2e8f1a": "Connect Orca Mobile to this desktop across cellular or any Wi-Fi.", + "9a4c6b2d7e": "Skill sharing", + "3d8e5f1b9c": "Share skills behind an unlisted link and install them on any machine you use.", + "7e1a9c4d2f": "Signing in…", + "c5b3e8a17d": "Sign in to Orca" } }, "i18n": { diff --git a/src/renderer/src/store/slices/ui-notice-dismissals.test.ts b/src/renderer/src/store/slices/ui-notice-dismissals.test.ts index f17c77c78e3..180f96e427a 100644 --- a/src/renderer/src/store/slices/ui-notice-dismissals.test.ts +++ b/src/renderer/src/store/slices/ui-notice-dismissals.test.ts @@ -280,3 +280,81 @@ describe('createUISlice clearOsc52ClipboardDefaultOnNotice', () => { expect(setUI).toHaveBeenCalledWith({ osc52ClipboardDefaultOnNoticePending: false }) }) }) + +describe('unexpected sign-out dismissal persistence', () => { + it('persists a dismissal once even when effects repeat', () => { + const setUI = vi.fn(() => Promise.resolve()) + vi.stubGlobal('window', { api: { ui: { set: setUI } } }) + const store = createUIStore() + + store.getState().dismissUnexpectedSignoutCard('1.2.3') + store.getState().dismissUnexpectedSignoutCard('1.2.3') + + expect(store.getState().dismissedUnexpectedSignoutVersion).toBe('1.2.3') + expect(setUI).toHaveBeenCalledExactlyOnceWith({ dismissedUnexpectedSignoutVersion: '1.2.3' }) + }) + + it.each([undefined, null, '1.2.2'])( + 'does not re-arm a local dismissal from stale hydration (%s)', + (dismissedUnexpectedSignoutVersion) => { + vi.stubGlobal('window', { api: { ui: { set: vi.fn(() => Promise.resolve()) } } }) + const store = createUIStore() + store.getState().dismissUnexpectedSignoutCard('1.2.3') + + store.getState().hydratePersistedUI(makePersistedUI({ dismissedUnexpectedSignoutVersion })) + + expect(store.getState().unexpectedSignoutDismissedVersions).toContain('1.2.3') + } + ) + + it('defaults legacy profiles and restores dismissal on reopen', () => { + const store = createUIStore() + expect(store.getState().dismissedUnexpectedSignoutVersion).toBeNull() + store + .getState() + .hydratePersistedUI( + makePersistedUI({ dismissedUnexpectedSignoutVersion: undefined }), + 'startup' + ) + expect(store.getState().dismissedUnexpectedSignoutVersion).toBeNull() + const reopened = createUIStore() + reopened + .getState() + .hydratePersistedUI( + makePersistedUI({ dismissedUnexpectedSignoutVersion: '1.2.3' }), + 'startup' + ) + expect(reopened.getState().dismissedUnexpectedSignoutVersion).toBe('1.2.3') + }) + + it('accepts another window dismissal after hydrating an older version', () => { + const store = createUIStore() + store + .getState() + .hydratePersistedUI( + makePersistedUI({ dismissedUnexpectedSignoutVersion: '1.2.2' }), + 'startup' + ) + store + .getState() + .hydratePersistedUI(makePersistedUI({ dismissedUnexpectedSignoutVersion: '1.2.3' })) + expect(store.getState().dismissedUnexpectedSignoutVersion).toBe('1.2.3') + }) +}) + +describe('unexpected sign-out hydrated dismissal history', () => { + it.each([undefined, null, '1.2.2', '1.2.4'])( + 'retains an observed dismissal after a different version sync (%s)', + (dismissedUnexpectedSignoutVersion) => { + const store = createUIStore() + store + .getState() + .hydratePersistedUI( + makePersistedUI({ dismissedUnexpectedSignoutVersion: '1.2.3' }), + 'startup' + ) + store.getState().hydratePersistedUI(makePersistedUI({ dismissedUnexpectedSignoutVersion })) + expect(store.getState().unexpectedSignoutDismissedVersions).toContain('1.2.3') + } + ) +}) diff --git a/src/renderer/src/store/slices/ui/ui-slice-contract-preferences.ts b/src/renderer/src/store/slices/ui/ui-slice-contract-preferences.ts index dff5f50c7e3..d913f998d2c 100644 --- a/src/renderer/src/store/slices/ui/ui-slice-contract-preferences.ts +++ b/src/renderer/src/store/slices/ui/ui-slice-contract-preferences.ts @@ -175,6 +175,10 @@ export type UISlicePersistence = { dismissedUpdateVersion: string | null dismissUpdate: (versionOverride?: string) => void clearDismissedUpdateVersion: () => void + /** App version that dismissed the unexpected-sign-out card; null = never dismissed. */ + dismissedUnexpectedSignoutVersion: string | null + unexpectedSignoutDismissedVersions: string[] + dismissUnexpectedSignoutCard: (version: string) => void /** Dev-only channel override; null follows the running build's own channel. */ releaseChannelOverride: ReleaseChannel | null setReleaseChannelOverride: (channel: ReleaseChannel | null) => void diff --git a/src/renderer/src/store/slices/ui/ui-slice-hydration-actions.ts b/src/renderer/src/store/slices/ui/ui-slice-hydration-actions.ts index a5daf8a500d..6ca8d702bd3 100644 --- a/src/renderer/src/store/slices/ui/ui-slice-hydration-actions.ts +++ b/src/renderer/src/store/slices/ui/ui-slice-hydration-actions.ts @@ -52,6 +52,7 @@ import { } from '../persisted-ui-write-baseline' import { hydrateTrustedOrcaHooks, + hydrateUnexpectedSignoutDismissal, normalizeHydratedVisibleWorkspaceHostIds, preserveStringArrayIdentity, sanitizeHydratedActiveView, @@ -226,6 +227,7 @@ export function createUiHydrationActions(set: UISliceSet, _get: UISliceGet): Par return DEFAULT_PET_ID })(), dismissedUpdateVersion: ui.dismissedUpdateVersion ?? null, + ...hydrateUnexpectedSignoutDismissal(s, ui.dismissedUnexpectedSignoutVersion), // Why: a persisted value from a build that knew a different channel set // would otherwise survive as-is; activeChannel only falls back on null, // so an unknown string reaches listBuilds and the segmented control. diff --git a/src/renderer/src/store/slices/ui/ui-slice-hydration-sanitizers.ts b/src/renderer/src/store/slices/ui/ui-slice-hydration-sanitizers.ts index bfed25a75cc..9737a575f06 100644 --- a/src/renderer/src/store/slices/ui/ui-slice-hydration-sanitizers.ts +++ b/src/renderer/src/store/slices/ui/ui-slice-hydration-sanitizers.ts @@ -238,3 +238,16 @@ export function migrateStatusBarItems(items: readonly string[] | undefined): Sta } return out as StatusBarItem[] } + +export function hydrateUnexpectedSignoutDismissal( + state: Pick, + version: string | null | undefined +): Pick { + const observed = state.unexpectedSignoutDismissedVersions + return { + dismissedUnexpectedSignoutVersion: version ?? null, + // A later sync must never undo any dismissal observed in this session. + unexpectedSignoutDismissedVersions: + typeof version === 'string' && !observed.includes(version) ? [...observed, version] : observed + } +} diff --git a/src/renderer/src/store/slices/ui/ui-slice-update-actions.ts b/src/renderer/src/store/slices/ui/ui-slice-update-actions.ts index e4beadf4c6c..942ff9610a5 100644 --- a/src/renderer/src/store/slices/ui/ui-slice-update-actions.ts +++ b/src/renderer/src/store/slices/ui/ui-slice-update-actions.ts @@ -43,6 +43,18 @@ export function createUiUpdateActions(set: UISliceSet, get: UISliceGet): Partial updateChangelog: null, updateUserInitiatedCycle: false, dismissedUpdateVersion: null, + dismissedUnexpectedSignoutVersion: null, + unexpectedSignoutDismissedVersions: [], + dismissUnexpectedSignoutCard: (version) => { + if (get().unexpectedSignoutDismissedVersions.includes(version)) { + return + } + set({ + dismissedUnexpectedSignoutVersion: version, + unexpectedSignoutDismissedVersions: [...get().unexpectedSignoutDismissedVersions, version] + }) + void window.api.ui.set({ dismissedUnexpectedSignoutVersion: version }).catch(console.error) + }, clearDismissedUpdateVersion: () => { set({ dismissedUpdateVersion: null }) }, diff --git a/src/shared/constants.ts b/src/shared/constants.ts index 7a06e11dba3..6840d92adc9 100644 --- a/src/shared/constants.ts +++ b/src/shared/constants.ts @@ -296,6 +296,7 @@ export function getDefaultUIState(): PersistedUIState { usagePercentageDisplay: DEFAULT_USAGE_PERCENTAGE_DISPLAY, statusBarUsageMode: DEFAULT_STATUS_BAR_USAGE_MODE, dismissedUpdateVersion: null, + dismissedUnexpectedSignoutVersion: null, lastUpdateCheckAt: null, trustedOrcaHooks: {}, setupScriptPromptDismissedRepoIds: [], diff --git a/src/shared/persisted-ui-state-types.ts b/src/shared/persisted-ui-state-types.ts index 943813d7255..2a3eb411a0e 100644 --- a/src/shared/persisted-ui-state-types.ts +++ b/src/shared/persisted-ui-state-types.ts @@ -125,6 +125,8 @@ export type PersistedUIState = { /** Client-side footer presentation; verbose preserves the pre-roster all-window default. */ statusBarUsageMode?: StatusBarUsageMode dismissedUpdateVersion: string | null + /** App version that last dismissed the unexpected-sign-out card; null = never. Re-arms on each new version while still signed out. */ + dismissedUnexpectedSignoutVersion?: string | null lastUpdateCheckAt: number | null /** Dev-only update channel override; absent means the build's own channel. */ releaseChannelOverride?: ReleaseChannel | null From 2c984bcdaa7950e628a18180aad9537243e1a92f Mon Sep 17 00:00:00 2001 From: Jinwoo Hong <73622457+Jinwoo-H@users.noreply.github.com> Date: Thu, 10 Sep 2026 23:28:39 -0400 Subject: [PATCH 6/8] feat(ai-vault-search): session search index as a transcript reader consumer (#19687) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(ai-vault-search): add the session search index schema and row modules The FTS5 index that PR 2 folds the transcript reader's message stream into: two FTS tables behind visibility views, the identifier shadow column, the resumable fork digest, redaction at the row-insert choke point, and the bounded compaction, warm-up and retention lanes. Schema version starts at 1: this branch drops the query log and the fts5vocab table, which the query engine reintroduces with its own bump. * feat(ai-vault-search): stage index writes and publish them atomically The writer stages a read's rows against an unpublished session row and a batch id, then flips the whole read visible in one transaction: both FTS tables are written together per row, and publish nulls the batch pointer before dropping the batch so a recycled rowid can never name a later in-flight write. Buffering replaces the branch's streamed producer. The transcript reader pushes messages synchronously, so a staged write cannot make it wait; rows are held to 128 of them or 256 KB and then flushed in one transaction, which bounds both the retained bytes and one stall. The store owns the database and the set of files the index is behind on. It carries no query, coverage or scheduling: draining that set is the indexer's. * feat(ai-vault-search): register the index as a transcript reader consumer The index keeps its own per-file cursor in the `files` table and never consults the parse cache; the branch's AsyncLocalStorage capture scope is gone. Three refusals, each leaving the cursor where it was and recording the file for a later whole re-read: an append whose predecessor offset is not this index's own cursor (or whose dev/ino changed) is declined in `beginRead`, a staging failure ends the read's rows without throwing back at the reader, and an `incomplete` outcome never publishes. A null session drops the file's rows and still advances the cursor, because the file was read through. Nothing in production registers it: the indexer decides when the index is live. * test(ai-vault-search): measure the index's disk and latency cost The write benchmark indexes a synthetic corpus through the real reader and reports rows/s, per-table bytes per transcript MB, write amplification and rebuild time; the retention benchmark compares the batched purge against a whole-file delete. The corpus generator ships with them, so the numbers are reproducible on any host and no real transcript is ever read. * fix(ai-vault-search): decline a source the message channel cannot reach An OpenCode SQLite candidate decodes inside a worker, so the reader reports every read of it as incomplete. Staging a batch first meant one staging session and one tombstone per scan of every such session, forever. Declining in `beginRead` also keeps it out of the re-read set, because no re-read helps: the index cannot cover that source until its capture path lands. * fix(ai-vault-search): redact a message before it is cut into rows A credential is a shape, and a shape split across two chunks matches neither half: an 8000-char boundary landing inside a PEM block or a JWT indexed the key material in full, identifier shadow terms included. Redaction moves ahead of chunking, and the row type is branded so only `searchMessageRows` can mint a row an insert site accepts. `upsertFile` stops erasing dev/ino when a candidate carries none. A host that cannot stat identity (SSH, WSL, a degraded Windows stat) was overwriting a proven identity with NULL, which made the next rename-replace of that path undetectable. * test(ai-vault-search): pin each index guard on its own Three guards were each shadowed by another, so mutating any one of them left every test green. Driving the store directly separates the writer's own predecessor-offset check from the consumer's cursor check, and a stubbed store proves `beginRead` refuses before the writer is ever asked. `publishable` gets the case it never had: a second read of the same path publishes first, and the stale stage is tombstoned rather than resurrected. It covers the overlap the parse file lane normally prevents, so the invalidation slot being per-path is not load-bearing on its own. Two ratchets: every provider fingerprint needs a fixture that reaches past `SECRET_ANCHOR`, and every FTS read in the index modules must subtract staged rows through a visibility view. * fix(ai-vault-search): rebuild an index that cannot be trusted, and index retention Opening self-heals. A file too torn to open, or one whose recovery fails, is unlinked with its sidecars and reopened once; a second failure throws. Before this a torn index broke open permanently, even though the index is a cache over the transcripts and throwing it away costs only a re-scan. A `meta` table with no readable version row is now `stale`, not `fresh`. Seeding the current version over it would have kept whatever rows the old schema left. `fresh` means no `meta` table at all, and no longer triggers a rebuild: the first open of a new profile was doing create-remove-create. Recovery moves inside open, because retiring a batch that outlived its writer is part of opening the index rather than of using it. Its append case gets the test it never had: 200 rows staged onto a live session and abandoned leave the published generation intact and nothing else. Also an index on files(mtime_ms), so the retention pass seeks the expiring end of the list instead of scanning and sorting it, a pragma read-back test, and a sessions.file_path comment that says what is actually true of OpenCode. * fix(ai-vault-search): key a session's cwd the way the sidebar already does `cwd_key` had its own normalizer, which qualified a WSL cwd with its distro: `/home/me/repo` was stored as `//wsl/ubuntu/home/me/repo` while the sidebar's `folderGroupKey` stored `/home/me/repo`. Any later join between an indexed hit and a sidebar group would have returned nothing for exactly the WSL users the qualification was meant to help. The key is now the shared normalizer verbatim, pinned against `folderGroupKey` for POSIX, Windows drive, /mnt/c, both WSL UNC aliases, a Linux path, and the root. The collision it guarded against is real: two distros both spell `/home/me/repo`. It is also a collision every SSH host has, neither key qualifies for SSH, and the honest fix is a column naming the execution host rather than a path key that only some hosts spell differently. `/` now keys as `/` instead of the empty string. An empty key cannot be told apart from a session with no cwd, and the scope filter builds its child prefix as `key + '/'`, which would have been `//`. * test(ai-vault-search): widen the FTS visibility ratchet past whole literals Concatenating or interpolating a table name hid it from the per-literal scan, so `'SELECT rowid FROM ' + table` passed. The unit is now the file, the scan covers src/main and src/relay rather than one directory, and both bypass shapes are checker unit tests. Exemptions carry a reason and the census fails on one that has stopped being needed, which is how the schema module lost its. * fix(ai-vault-search): keep a failed rebuild's real cause, and read per statement Closing the stale handle before the unlink left `db` dangling: if the unlink or the reopen threw, the failure path closed it a second time and node:sqlite's ERR_INVALID_STATE replaced the real cause. Nothing classifies that as worth retrying, so an index a virus scanner or a second Orca was holding would never rebuild. The handle is nulled while none is open. The visibility ratchet moves from the file to the statement. A file is far too coarse for what it exists to guard: the query module will name both views somewhere, and that whitelisted every raw read in it. Statements come from literals and their concatenation chains, split on `;`, and a table name assembled at runtime counts as a read in any file that names an FTS table. The roots now cover cli, shared and preload as well. Half a recorded identity is now stated to be no identity. `remote-session-file-stat` spreads dev and ino independently and the COALESCE preserves whichever half a host could prove, so one number cannot tell a rename-replace from a same-file re-read; comparing it would decline healthy resumes on a coincidence. `discard` clearing the staging slot gets a test: without it the map grew one entry per path for the store's life. * fix(ai-vault-search): give a behind consumer a way to get the read it needs `markStale` promised a whole re-read that nothing could deliver. The reader picks append or replace from the session list's resume point, so with an empty index and a warm parse cache every read arrives as `append`, the consumer declines every one, and nothing is ever indexed. That is the state on first enablement inside a running app. `requestWholeTranscriptRead` in the reader drops that path's resume point, which is the one lever that changes the next read's mode, and it lives with the cache that owns it rather than with the consumer that wants it. `takeStale` documents that its paths need it before a scan is re-dispatched. Pausing no longer empties the re-read set. `acceptsCandidate` answers whether a write may start now and was being used for both jobs, so reconfiguring retention while paused dropped every entry and a declined read during a pause was never recorded at all. Retention alone prunes; a paused decline is recorded, bounded, with the oldest dropped and the drops counted, because a set that silently forgets is worse than one that says it is incomplete. A read that decodes no session now tombstones its staged rows before the store schedules cleanup. The cleanup lane reads the tombstone table the moment it is scheduled, so the old order left that batch on disk until some later write, and for the last read before a shutdown that is never. * fix(ai-vault-search): create the directory the index lives in Nothing made `/ai-vault-search/`, and SQLite's failure for a missing parent is `unable to open database file`, which is correctly not classified as corruption — so the retry never fired and the feature stranded on any profile that had never held an index. * refactor(ai-vault-search): store transcript content as written Decision: the index does not redact. A secret in a transcript is already plaintext under the user's home directory and is treated as compromised, so the index is a second copy of content the user already holds, not a new exposure. The reference agent-session-search products do not redact either. What a snippet may carry once it leaves this machine is a transport policy and belongs where the wire is, not in the write path. Removes the redaction module and its census, the branded row type that existed to prove redaction had run before an insert, and the two chunk-boundary tests. `insertSearchMessage` takes a plain chunk again and identifier shadow terms come off the raw text. `src/main/observability/redactor.ts` goes back to what main has, so this PR no longer touches it at all. Chunking and the fork digest are unchanged; the digest always folded raw message text, ahead of chunking, so its tests hold as written. Cost: redaction was 14 ms per 10.5 MB of transcript, about 3% of a rebuild and below the benchmark's run-to-run spread, so the write numbers are unchanged at roughly 22-25k rows/s and 22-27 MB/s. * fix(ai-vault-search): hide a tombstoned session's messages, not only its row `visible_sessions` already subtracted session-keyed tombstones; the message half filtered on the batch pointer alone. A published row outlives its session row until the cleanup lane reaches it, so between a `replace` publish and that drain both generations answered, and a removed file kept answering after its session was gone. Both views now subtract the same set, over a partial index so neither read scans the tombstone table. * fix(ai-vault-search): drain the rows a read that never published staged Only `writePublished` and `removeFile` scheduled the cleanup lane. A read that staged rows and then declined to publish them tombstoned its batch and told nobody, so 256 rows of a 300-message incomplete read sat in `messages` and both FTS tables until some unrelated write happened to schedule a pass. Open-time recovery had the same hole: it wrote the tombstones and drained none. The abandoned branch now schedules the drain the published branch already got, and opening schedules one pass for what recovery just tombstoned. Recovery no longer adds a batch tombstone when the session-keyed one already covers those rows, which it did once more on every reopen. * fix(ai-vault-search): continue the cursor of a file that decoded no session A read that decodes no session — an excluded Codex worker transcript — advances the cursor and leaves `files.session_row_id` null. Every later append was then declined, so a file that only ever grows would be re-read whole on every pass for the rest of its life. An append now only has to continue this index's own cursor; it creates the session row when there is none to resume. That also collapses the `append` flag, which meant both "resume this session" and "do not own it", into the resumed row id itself. * refactor(ai-vault-search): drop the index path module nothing calls No caller in this PR or the two that follow it: the composition root that would capture the userData path is PR 3b's. * refactor(ai-vault-search): read the schema version as stale or not Only the stale answer was ever acted on; the fresh/current split named two ways of being fine. * fix(ai-vault-search): read the resumed session id off an optional row * refactor(ai-vault-search): leave page warmup and the freshness check to PR 4 `warm()` and `session-search-page-warmup.ts` have their first caller in the query engine, which is what knows which pages are worth warming; the module itself went with the previous commit. `isSessionSearchFileCurrent` has its first caller in the reconciler. `session-search-file-cursor.ts` stays, because `fileIdentity` and both its types are load-bearing here. * refactor(ai-vault-search): stop pruning the re-read set on a retention change The prune walked the whole set to drop what the next `beginRead` would refuse anyway: `acceptsCandidate` applies the window when the re-read is dispatched, and `markStale` applies it again before recording anything. The bound stays. * fix(ai-vault-search): keep a batch tombstone from outliving its batch row Draining a session-keyed tombstone deletes the session's batch rows, but left any batch-keyed tombstone naming them in place. `search_write_batches` empties on every publish, so ids restart low and the next read takes the freed rowid; the stale tombstone then deleted that live batch's staged rows and the session published missing messages, with no re-read to fill the gap. The session drain now clears those tombstones in the same transaction. * refactor(ai-vault-search): commit a file's rows and its cursor in one transaction The staged-publish model is replaced by one SQLite transaction per file in WAL mode. A read buffers its decoded rows and writes them, its session and its cursor together; a reader on another handle sees the last committed state, which is the "never a torn session" guarantee the staging machinery was built to provide. A crash rolls the whole file back and it is re-read. Gone with it: `search_write_batches`, `search_pending_deletes`, `messages.batch_id`, `sessions.index_ready`, both `visible_*` views, the open-time recovery pass, the cleanup lane and `settled()`, and the ratchet test that made every query site read through a view. Rounds 2 through 6 were all seams between those pieces. A file whose rows exceed `SESSION_SEARCH_COMMIT_CHARS` is cut into chunks. The reader only hands out a byte offset when a read finishes, so a chunk records one no append can continue from: its rows answer searches as a coherent prefix of the session, and the next whole read replaces them. Retention keeps no record of unfinished work. It cuts a session loose from its file in one small transaction, which is what stops it answering, then reclaims its rows in bounded batches and hands the freed pages back as it goes. Rows whose session row is gone are the record of what an interrupted purge left. Deleted for want of a caller in PR 2, 3 or 4: the WAL budget (a buffered write has no staging window to grow one across), the compaction module (folded into the drain), the `sessions_content_hash` index (fork folding runs in JS over rows PR 4 already holds), `lastWriteAt`, `failures`, `SessionSearchStoreOptions`, `openStageCount` and `chunkMessageText`. * test(ai-vault-search): price a per-file commit and the ceiling that bounds it The write benchmark now times every transaction, because with one per file that is the whole stall a file costs. A second phase indexes a single synthetic 100 MB transcript, which is what the commit ceiling is chosen against. * fix(ai-vault-search): stop a fenced write reopening a transaction per message A chunk write that finds the file record moved under it — a `removeFile`, or an overlapping read of the same path — can never land anything afterwards. It kept buffering, so every remaining message re-entered `BEGIN IMMEDIATE` only to roll back, once per message for the rest of a file that may be a hundred megabytes. It now stops at the first refusal and drops what it holds. * fix(ai-vault-search): never hand a live session the id of a purged one `sessions.id` was a plain rowid alias, so SQLite reissued it as max+1. That id names rows in `messages` for far longer than the row itself lives: retention cuts a session loose in one transaction and reclaims its messages over many, and a session created inside that window was handed a freed id and adopted whatever of the purged conversation the drain had not reached. The drain then skipped those rows for good, because their session row exists again, and a search answered for a purged transcript under a live session's name. Reachable with no crash at all: an append of a growing transcript the parser decoded no session from creates its session row mid-purge. AUTOINCREMENT is the class fix; the schema version goes to 3 so a file written by an earlier commit of this branch rebuilds rather than keeping a plain-rowid table under a `CREATE TABLE IF NOT EXISTS`. `messages.id` was checked and is not exposed to the same window: a row and its two FTS entries always go in one transaction. Two smaller ones in the same pass. `removeFile` did not fence a read of a path this index had never written: `current()` compared a cursor, and on an unknown path the absent row and the absent expectation are both undefined, so the write recreated the source after its owner had proven it gone. The writer now counts removals per path and a write compares that count, which is a positive fence rather than an inferred one. And `drainOrphanedMessages`, `CONTENT_HASH_MESSAGE_LIMIT` and `CONTENT_HASH_MIN_MESSAGES` are no longer exported: nothing outside their own modules reads them. * fix(ai-vault-search): report a half-written file as held, not as unknown `indexedFile` returned null for a file a chunked read left partway through, the same answer it gives for a path the index has never seen. A caller asking "do you hold this file" therefore read a half-written one as new, asked for whatever read the parse cache offered, and the reader picked append — which the consumer then declined. Only the stale set healed it, a cycle later. It now reports the record with a null `byteOffset`, and `requiresWholeRead(indexed)` names that state. Null rather than an added flag because the mtime and size on that record are the file's real ones: a freshness check comparing only those would call a half-written file current, and a boolean is easy to not read, while every site that does arithmetic on the offset has to say what null means at compile time. The test also found the writer accepting an append that passed the partial sentinel back as its predecessor offset. Nothing in the reader produces a negative offset, but the sentinel is not a byte offset and no caller should be able to continue it; `beginWrite` now refuses it outright. * fix(ai-vault-search): cut a chunk at whitespace, never mid-token The 8,000-char chunker backed up only to a newline, and only when one sat in the second half of the window. A wrapped paragraph, a CJK transcript or a minified log has no newline there, so the cut landed inside whatever word straddled the target: the user's term was filed as two halves and matched neither. It now backs up to the last Unicode whitespace in the second half, and falls through to the target when there is none, because 4,000 characters without a space is not a word. A phrase that straddles a chunk boundary is still not matched. Chunks are separate FTS rows and FTS5 cannot span them; that is stated at the chunker. * feat(ai-vault-search): cap an indexed tool row at 3,072 characters Tool output is 80-97 % of a transcript's bytes and a single message may be a quarter of a megabyte, so without a cap the index, the buffer a read holds and the transaction it commits are all sized by how much a tool printed rather than by how much is worth searching. A `tool` message now becomes one row of at most 3,072 characters, kept from the head, where the command and the first lines of its output are. User and assistant text is never capped. Measured on the write benchmark, 40 sessions with tool output at 95 % of message text (the real band), per transcript MB: index 1,334,126 -> 353,973 bytes, write amplification 1.27x -> 0.34x, rows 18,804 -> 9,600, rebuild 1,676 -> 379 ms, largest transaction 49.7 -> 12.1 ms. On the default corpus, whose tool results are 1.4 KB and so under the cap, nothing changes at all: 2,167,887 bytes per transcript MB on both arms. The corpus generator takes `toolResultWords` and the benchmark reads ORCA_SEARCH_BENCH_TOOL_WORDS so both arms are the same harness. * fix(ai-vault-search): check the commit ceiling per row, not per message The buffered-chars check ran after a whole message had been folded into rows, so a single message could carry a transaction as far past the ceiling as it was large. A conversation turn is one message and can be megabytes; the ceiling exists to bound how long one commit holds the process and how large a WAL it produces, and neither bound survived a message that overshot it. * style(ai-vault-search): undo a stray reformat of untouched assertions Three assertions in the file-write test were expanded by a formatter run that was not the repo's, and the expansion survived because a multi-line object literal is preserved once it exists. Restored to what they were. * feat(ai-vault-search): write a session's identity with its first commit A chunked read created its session row with an empty session id, an empty title, an empty resume command and null cwd and timestamps, and only filled them in on the final commit. Those chunks answer searches the moment they land, so until the read ended every hit they produced named a session nothing could identify — and a crash between chunks left it that way for good, since the re-read that heals it is the same read starting over. The reader already knows the id, cwd and timestamps from a transcript's opening lines; it just had nowhere to put them. `TranscriptReadStart` now carries an optional `identity()` the consumer calls during the read, backed by an optional `identity()` on `ResumableSessionParseState`: one line in the shared accumulator fold (which covers cursor, copilot, droid, gemini, antigravity and the graph parsers) and one each in the Claude and Codex folds, which keep their own state. A chunk commit writes what it returns; the final commit overwrites it from the decoded session, so the mid-read title stays provisional. Two `cwd` guards in the Codex fold collapse into the `?? ` form the `branch` line beside one of them already used. `extractString` never returns an empty string, so it is the same assignment in one line instead of four, and it is what keeps the file under the 300-line cap with the identity accessor added. * refactor(ai-vault-search): drop conversation_fts for a column filter The second FTS table existed for query latency alone: a column-filtered `messages_fts MATCH '{user_text assistant_text}: q'` returns the identical rowid set, which PR 2's round 5 review proved and PR 4's benchmark re-checked per query. PR 4 then measured the filter at 1.16-1.36x the p95 of the second table on a 105 MB corpus across two points in the tool-output band, under the 2x bar the decision was set at, and closes the stack's open decision 3. It cost a quarter of the index on a corpus whose tool output is half the message text. With the tool-row cap it is a smaller saving than the decision was framed around, but it is still a table, a write per conversational row and a delete per reclaimed one. Schema version 4: `CREATE TABLE IF NOT EXISTS` is a no-op over an existing index, so only the bump makes a file that still carries the table go. * feat(ai-vault-search): expose the index handle a composed reader queries PR 4's engine reads the index this store owns. One getter lets it compose over the store's handle instead of opening a second connection to the same file, and it carries the two rules this PR measured: never hold a read transaction across an await, and no `.iterate()` outliving its statement. Either pins a read snapshot, and a checkpoint cannot pass one, so the WAL grows without bound for as long as it is held (10 MB to 266 MB on the write benchmark). * fix(ai-vault-search): cut a chunk at punctuation, not only whitespace The 8,000-character chunker backed up to the last whitespace in the second half of the window and fell through to the target when it found none. A minified tool result has none to find: valid minified JSON runs past 8,000 characters without a space, so the cut landed inside whatever word straddled the target, and a search for `pericardium` matched neither `perica` nor `rdium`. That is the shape most likely to be chunked in the first place, because tool output is 80-97 % of a transcript's bytes. The backoff now takes any character that cannot be inside a token — anything outside a letter, digit or underscore — and still falls through to the target when the window holds none, because 4,000 characters without one is not a word. Surrogates are excluded from the class so a cut never lands between the halves of an astral character. A few of the tokenizer's own `tokenchars` (`. - / +`) are cut on even though unicode61 keeps them inside a token. That costs the joined form of a path, and only in a window with no whitespace anywhere, which is a far smaller loss than the torn word this exists to prevent. * fix(ai-vault-search): never chunk a read that cannot name its session `updateProvisionalSession` is fed by the resumable readers alone. Claude and Codex carry an `identity()` off their fold; `readWholeTranscript` supplies none, because a format rewritten in place has no resumable state to ask. So a Grok, Cursor, Gemini or OpenCode file large enough to pass the commit ceiling published its chunks under a session with an empty id, an empty title and a null cwd — rows that answer searches at once and that an interrupted read leaves behind for good, since the re-read that heals them is the same read starting over. `add` now commits a chunk only while the read can name what it is writing. A read with no identity, or one whose parser has decoded no id yet, keeps buffering and commits whole at `finish`. The whole-file formats are the ones with nothing to give and they are small — the largest on this machine is 5 MB — so buffering one to the end costs nothing, and chunking stays reserved for the readers that can say which session a prefix belongs to. `updateProvisionalSession` takes a non-null identity now, so the invariant is the type rather than a guard that silently wrote nothing. * perf(ai-vault-search): cut a replaced generation loose instead of deleting it The first transaction of a replace deleted every row of the old session before inserting its own bounded chunk, so the transaction was sized by the history being replaced rather than by the rows being written. On the synthetic 100 MB transcript the longest replace transaction was 1,255-1,283 ms against 668-672 ms for the same file read fresh, and the gap grows with the session. A replace now mints a new session row, points the `files` row at it and deletes the one old `sessions` row, all in the same transaction. Every retrieval joins `sessions`, so the old generation stops answering the moment that commits — the same thing retention's first transaction does — and its messages are reclaimed afterwards by `drainOrphanedMessages`, the bounded batch loop that already exists for exactly this set. `sessions.id` is AUTOINCREMENT, so the freed id is never handed to another session while those rows still name it. The longest replace transaction is now 746-848 ms, the same band as a fresh read. The trade is stated plainly: the pass does more total work, 3.7 s against 3.1 s, because the reclaim is 360 bounded transactions with an incremental_vacuum step each instead of one large delete. It yields between them, so none of it holds the process, and returning the pages per batch also takes the index from 173.6 MB to 162.0 MB. The store schedules the drain off the committing stack: an async function runs synchronously to its first `await`, so calling it inline would put the first batch back inside the transaction's own call. One drain at a time, with a repeat flag for a replace that commits while one is running. * fix(ai-vault-search): preserve search tokens and index Codex tools * fix(ai-vault-search): match SQLite marks and Codex file-change records * fix(ai-vault-search): preserve stat pairs and validate benchmark results --- .../session-search-retention-benchmark.ts | 148 +++++ .../scripts/session-search-write-benchmark.ts | 266 ++++++++ .../session-search-content-hash.test.ts | 48 ++ .../session-search-content-hash.ts | 45 ++ .../session-search-cwd-key.test.ts | 37 ++ .../session-search-file-cursor.ts | 41 ++ .../session-search-file-records.ts | 134 ++++ .../session-search-file-write.test.ts | 614 ++++++++++++++++++ .../session-search-identifier-split.test.ts | 29 + .../session-search-identifier-split.ts | 54 ++ .../session-search-index-consumer.test.ts | 373 +++++++++++ .../session-search-index-consumer.ts | 118 ++++ .../session-search-index-test-fixture.ts | 124 ++++ .../session-search-index-writer.test.ts | 228 +++++++ .../session-search-index-writer.ts | 359 ++++++++++ .../session-search-live-transcript.test.ts | 208 ++++++ .../session-search-message-rows.test.ts | 249 +++++++ .../session-search-message-rows.ts | 135 ++++ .../session-search-retention-delete.test.ts | 188 ++++++ .../session-search-retention-delete.ts | 102 +++ .../session-search-row-identity.test.ts | 116 ++++ .../session-search-schema.test.ts | 350 ++++++++++ .../ai-vault-search/session-search-schema.ts | 198 ++++++ .../ai-vault-search/session-search-store.ts | 253 ++++++++ .../session-search-synthetic-corpus.test.ts | 44 ++ .../session-search-synthetic-corpus.ts | 154 +++++ .../session-search-transcript-fixtures.ts | 118 ++++ .../ai-vault/session-scanner-accumulator.ts | 29 +- .../session-scanner-codex-message-records.ts | 9 + .../ai-vault/session-scanner-codex-parser.ts | 19 +- .../session-scanner-codex-record-fast-path.ts | 14 +- ...session-scanner-codex-tool-records.test.ts | 125 ++++ .../session-scanner-codex-tool-records.ts | 95 +++ ...ession-scanner-omp-subagent-transcripts.ts | 1 + .../session-scanner-primary-parsers.ts | 2 + src/main/ai-vault/session-scanner-types.ts | 8 +- .../ai-vault/session-transcript-consumers.ts | 22 + .../ai-vault/session-transcript-reader.ts | 28 +- 38 files changed, 5070 insertions(+), 15 deletions(-) create mode 100644 config/scripts/session-search-retention-benchmark.ts create mode 100644 config/scripts/session-search-write-benchmark.ts create mode 100644 src/main/ai-vault-search/session-search-content-hash.test.ts create mode 100644 src/main/ai-vault-search/session-search-content-hash.ts create mode 100644 src/main/ai-vault-search/session-search-cwd-key.test.ts create mode 100644 src/main/ai-vault-search/session-search-file-cursor.ts create mode 100644 src/main/ai-vault-search/session-search-file-records.ts create mode 100644 src/main/ai-vault-search/session-search-file-write.test.ts create mode 100644 src/main/ai-vault-search/session-search-identifier-split.test.ts create mode 100644 src/main/ai-vault-search/session-search-identifier-split.ts create mode 100644 src/main/ai-vault-search/session-search-index-consumer.test.ts create mode 100644 src/main/ai-vault-search/session-search-index-consumer.ts create mode 100644 src/main/ai-vault-search/session-search-index-test-fixture.ts create mode 100644 src/main/ai-vault-search/session-search-index-writer.test.ts create mode 100644 src/main/ai-vault-search/session-search-index-writer.ts create mode 100644 src/main/ai-vault-search/session-search-live-transcript.test.ts create mode 100644 src/main/ai-vault-search/session-search-message-rows.test.ts create mode 100644 src/main/ai-vault-search/session-search-message-rows.ts create mode 100644 src/main/ai-vault-search/session-search-retention-delete.test.ts create mode 100644 src/main/ai-vault-search/session-search-retention-delete.ts create mode 100644 src/main/ai-vault-search/session-search-row-identity.test.ts create mode 100644 src/main/ai-vault-search/session-search-schema.test.ts create mode 100644 src/main/ai-vault-search/session-search-schema.ts create mode 100644 src/main/ai-vault-search/session-search-store.ts create mode 100644 src/main/ai-vault-search/session-search-synthetic-corpus.test.ts create mode 100644 src/main/ai-vault-search/session-search-synthetic-corpus.ts create mode 100644 src/main/ai-vault-search/session-search-transcript-fixtures.ts create mode 100644 src/main/ai-vault/session-scanner-codex-tool-records.test.ts create mode 100644 src/main/ai-vault/session-scanner-codex-tool-records.ts diff --git a/config/scripts/session-search-retention-benchmark.ts b/config/scripts/session-search-retention-benchmark.ts new file mode 100644 index 00000000000..095eb5f29b0 --- /dev/null +++ b/config/scripts/session-search-retention-benchmark.ts @@ -0,0 +1,148 @@ +import assert from 'node:assert/strict' +import { mkdtemp, rm, stat } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { setImmediate as yieldToEventLoop } from 'node:timers/promises' +import { + syntheticCandidate, + syntheticSession, + userMessages +} from '../../src/main/ai-vault-search/session-search-index-test-fixture' +import { SessionSearchStore } from '../../src/main/ai-vault-search/session-search-store' +import SyncDatabase from '../../src/main/sqlite/sync-database' + +// Bundle with esbuild --bundle --platform=node, then run on the host under test. +// Every mode seeds through SessionSearchStore so the three arms are comparable; +// only `whole-file` leaves the shipped path, because it is the baseline the +// batched purge exists to replace. + +const ROWS = 60_000 + +/** The purge yields with `setImmediate` between chunks, so a peer chain samples each gap. */ +async function sampleLoopStalls(running: () => boolean, intervals: number[]): Promise { + let previous = performance.now() + while (running()) { + await yieldToEventLoop() + const now = performance.now() + intervals.push(now - previous) + previous = now + } +} + +/** What a search would still return: rows whose session row is still there. */ +function visibleRows(db: SyncDatabase): number { + return ( + db + .prepare(`SELECT count(*) AS n FROM messages m JOIN sessions s ON s.id = m.session_row_id`) + .get() as { n: number } + ).n +} + +const root = await mkdtemp(join(tmpdir(), 'orca-search-retention-bench-')) +try { + for (const mode of ['whole-file', 'batched', 'batched-pinned-reader']) { + const path = join(root, `${mode}.sqlite`) + const errors: unknown[] = [] + const store = new SessionSearchStore(path, (error) => errors.push(error)) + let reader: SyncDatabase | null = null + try { + const write = store.beginWrite(syntheticCandidate(), 'replace', 0)! + for (const message of userMessages( + 'synthetic benchmark needle repeated context for a representative coding conversation with commands and paths src/example.ts', + ROWS + )) { + write.add(message) + } + assert.equal( + write.commit({ + session: syntheticSession(), + byteOffset: 4096, + incomplete: false + }), + true + ) + assert.deepEqual(errors, []) + // Truncating first is what makes walBytes below the purge's own growth. + const checkpoint = new SyncDatabase(path) + checkpoint.pragma('wal_checkpoint(TRUNCATE)') + checkpoint.close() + if (mode === 'batched-pinned-reader') { + reader = new SyncDatabase(path, { readonly: true }) + reader.exec('BEGIN') + reader.prepare('SELECT count(*) FROM messages').get() + } + const probe = new SyncDatabase(path, { readonly: true }) + const intervals: number[] = [] + const started = performance.now() + if (mode === 'whole-file') { + const raw = new SyncDatabase(path) + try { + raw.exec('BEGIN IMMEDIATE') + const ids = raw.prepare('SELECT id FROM messages').all() as { + id: number + }[] + for (const { id } of ids) { + raw.prepare('DELETE FROM messages_fts WHERE rowid=?').run(id) + } + raw.exec('DELETE FROM messages; DELETE FROM sessions; DELETE FROM files; COMMIT') + } finally { + raw.close() + } + intervals.push(performance.now() - started) + } else { + let purging = true + const purge = store.purgeOlderThan(Date.now() + 60_000) + // Hiding is immediate: cutting the session loose from its file is the + // first transaction, so a read one turn in already sees nothing, long + // before the rows are gone. + const hiddenEarly = yieldToEventLoop().then(() => visibleRows(probe)) + const sampler = sampleLoopStalls(() => purging, intervals) + await purge + purging = false + await sampler + assert.equal(await hiddenEarly, 0) + assert.deepEqual(errors, []) + } + const wallMs = performance.now() - started + probe.close() + reader?.exec('COMMIT') + reader?.close() + reader = null + const after = new SyncDatabase(path, { readonly: true }) + try { + for (const table of ['messages_fts']) { + assert.equal( + ( + after.prepare(`SELECT count(*) AS n FROM ${table}`).get() as { + n: number + } + ).n, + 0 + ) + } + } finally { + after.close() + } + const walBytes = (await stat(`${path}-wal`)).size + intervals.sort((a, b) => a - b) + console.log( + JSON.stringify({ + mode, + platform: process.platform, + node: process.version, + rows: ROWS, + wallMs: Math.round(wallMs), + samples: intervals.length, + maxStepMs: Math.round(intervals.at(-1) ?? 0), + p95StepMs: Math.round(intervals[Math.floor(intervals.length * 0.95)] ?? 0), + walBytes + }) + ) + } finally { + reader?.close() + store.close() + } + } +} finally { + await rm(root, { recursive: true, force: true }) +} diff --git a/config/scripts/session-search-write-benchmark.ts b/config/scripts/session-search-write-benchmark.ts new file mode 100644 index 00000000000..db09275cd98 --- /dev/null +++ b/config/scripts/session-search-write-benchmark.ts @@ -0,0 +1,266 @@ +import assert from 'node:assert/strict' +import { rm, stat } from 'node:fs/promises' +import { join } from 'node:path' +import { setImmediate as yieldToEventLoop } from 'node:timers/promises' +import { + createSessionParseStats, + parseAgentSessionFileCached, + resetSessionParseCacheForTests +} from '../../src/main/ai-vault/session-scanner-parse-cache' +import { resetTranscriptConsumersForTests } from '../../src/main/ai-vault/session-transcript-consumers' +import { requestWholeTranscriptRead } from '../../src/main/ai-vault/session-transcript-reader' +import { registerSessionSearchIndexConsumer } from '../../src/main/ai-vault-search/session-search-index-consumer' +import { SessionSearchStore } from '../../src/main/ai-vault-search/session-search-store' +import { writeSyntheticTranscriptCorpus } from '../../src/main/ai-vault-search/session-search-synthetic-corpus' +import { sessionCandidate } from '../../src/main/ai-vault-search/session-search-transcript-fixtures' +import SyncDatabase from '../../src/main/sqlite/sync-database' + +// Measures the real transcript reader and search store over a synthetic corpus. +// Never point this at a real transcript tree. + +/** + * How long the longest single transaction held the process. + * + * With one transaction per file that is the whole stall a file costs, so it is + * the number the commit ceiling exists to bound. Measured by wrapping `exec`, + * because the writer's transactions are the only ones this benchmark runs. + */ +function recordTransactionDurations(durations: number[]): () => void { + const exec = SyncDatabase.prototype.exec + let started = 0 + SyncDatabase.prototype.exec = function (this: SyncDatabase, sql: string): void { + if (sql === 'BEGIN IMMEDIATE') { + started = performance.now() + } + exec.call(this, sql) + if (sql === 'COMMIT' && started > 0) { + durations.push(performance.now() - started) + started = 0 + } + } + return () => { + SyncDatabase.prototype.exec = exec + } +} + +/** The writer commits synchronously, so a peer chain samples the gap each read leaves. */ +async function sampleLoopStalls(running: () => boolean, stalls: number[]): Promise { + let previous = performance.now() + while (running()) { + await yieldToEventLoop() + const now = performance.now() + stalls.push(now - previous) + previous = now + } +} + +function tableBytes(db: SyncDatabase): Record { + const rows = db.prepare('SELECT name, sum(pgsize) AS bytes FROM dbstat GROUP BY name').all() as { + name: string + bytes: number + }[] + const group = (prefix: string): number => + rows + .filter((row) => row.name === prefix || row.name.startsWith(`${prefix}_`)) + .reduce((sum, row) => sum + row.bytes, 0) + return { + messagesFts: group('messages_fts'), + messages: group('messages') - group('messages_fts'), + sessions: group('sessions'), + total: rows.reduce((sum, row) => sum + row.bytes, 0) + } +} + +function assertIndexedMessages(db: SyncDatabase, expected: number): number { + const { n } = db + .prepare('SELECT count(*) AS n FROM messages m JOIN sessions s ON s.id = m.session_row_id') + .get() as { n: number } + assert.equal(n, expected, 'indexed message count') + return n +} + +async function checkpointedFileBytes(db: SyncDatabase, path: string): Promise { + // Flush committed WAL pages before reporting the final database footprint. + const [checkpoint] = db.pragma('wal_checkpoint(TRUNCATE)') as { busy: number }[] + assert.equal(checkpoint?.busy, 0, 'storage measurement requires a completed checkpoint') + return (await stat(path)).size +} + +// The default corpus puts tool output at about half the message text; set this +// far higher to price the tool-row cap against the real 80-97 % band. +const toolResultWords = Number(process.env.ORCA_SEARCH_BENCH_TOOL_WORDS ?? 200) +const corpus = await writeSyntheticTranscriptCorpus({ toolResultWords }) +const indexPath = join(corpus.root, 'index.sqlite') +try { + const errors: unknown[] = [] + const store = new SessionSearchStore(indexPath, (error) => errors.push(error)) + const unregister = registerSessionSearchIndexConsumer(store) + const stalls: number[] = [] + const transactions: number[] = [] + const restoreExec = recordTransactionDurations(transactions) + let indexing = true + try { + const stats = createSessionParseStats() + const started = performance.now() + const sampler = sampleLoopStalls(() => indexing, stalls) + for (const path of corpus.files) { + await parseAgentSessionFileCached( + await sessionCandidate('claude', path), + process.platform, + stats + ) + } + indexing = false + await sampler + restoreExec() + const rebuildMs = performance.now() - started + assert.deepEqual(errors, []) + + const reader = new SyncDatabase(indexPath, { readonly: true }) + try { + const rows = assertIndexedMessages(reader, corpus.messageCount) + const sessions = ( + reader.prepare('SELECT count(*) AS n FROM sessions').get() as { + n: number + } + ).n + assert.equal(sessions, corpus.files.length) + const bytes = tableBytes(reader) + const perMb = (value: number): number => + Math.round((value / (corpus.transcriptBytes / (1024 * 1024))) * 10) / 10 + const fileBytes = await checkpointedFileBytes(store.connection, indexPath) + stalls.sort((a, b) => a - b) + transactions.sort((a, b) => a - b) + console.log( + JSON.stringify( + { + platform: process.platform, + node: process.version, + transcriptMb: Math.round((corpus.transcriptBytes / (1024 * 1024)) * 100) / 100, + toolResultWords, + sessions, + rows, + rebuildMs: Math.round(rebuildMs), + rowsPerSecond: Math.round(rows / (rebuildMs / 1000)), + transcriptMbPerSecond: + Math.round((corpus.transcriptBytes / (1024 * 1024) / (rebuildMs / 1000)) * 100) / 100, + bytesPerTranscriptMb: { + messagesFts: perMb(bytes.messagesFts), + messages: perMb(bytes.messages), + sessions: perMb(bytes.sessions), + total: perMb(bytes.total) + }, + writeAmplification: Math.round((bytes.total / corpus.transcriptBytes) * 100) / 100, + fileWriteAmplification: Math.round((fileBytes / corpus.transcriptBytes) * 100) / 100, + transactions: transactions.length, + maxTransactionMs: Math.round((transactions.at(-1) ?? 0) * 100) / 100, + maxLoopStallMs: Math.round(stalls.at(-1) ?? 0), + p95LoopStallMs: Math.round(stalls[Math.floor(stalls.length * 0.95)] ?? 0), + loopStallSamples: stalls.length, + parseStats: stats + }, + null, + 2 + ) + ) + } finally { + reader.close() + } + } finally { + indexing = false + restoreExec() + unregister() + resetTranscriptConsumersForTests() + resetSessionParseCacheForTests() + store.close() + } +} finally { + await rm(corpus.root, { recursive: true, force: true }) +} + +// Phase two: one transcript far larger than any real one, to price the ceiling +// that decides whether a file commits once or in chunks. +const largeTurns = Number(process.env.ORCA_SEARCH_BENCH_LARGE_TURNS ?? 23_000) +const large = await writeSyntheticTranscriptCorpus({ + sessions: 1, + turnsPerSession: largeTurns, + seed: 2 +}) +const largeIndexPath = join(large.root, 'index.sqlite') +try { + const errors: unknown[] = [] + const store = new SessionSearchStore(largeIndexPath, (error) => errors.push(error)) + const unregister = registerSessionSearchIndexConsumer(store) + const transactions: number[] = [] + const restoreExec = recordTransactionDurations(transactions) + try { + const stats = createSessionParseStats() + const started = performance.now() + await parseAgentSessionFileCached( + await sessionCandidate('claude', large.files[0]!), + process.platform, + stats + ) + const indexMs = performance.now() - started + restoreExec() + assert.deepEqual(errors, []) + assertIndexedMessages(store.connection, large.messageCount) + transactions.sort((a, b) => a - b) + + // The same file again, over a generation the index already holds. That is + // the pass a growing transcript really costs, and the one whose transaction + // used to be sized by the old session rather than by the chunk being + // written. The drain that reclaims the cut-loose generation runs after the + // commit, so its bounded batches are in `replaceTransactions` too. + const replaceTransactions: number[] = [] + const restoreReplaceExec = recordTransactionDurations(replaceTransactions) + requestWholeTranscriptRead(large.files[0]!) + const replaceStarted = performance.now() + await parseAgentSessionFileCached( + await sessionCandidate('claude', large.files[0]!), + process.platform, + stats + ) + const replaceMs = performance.now() - replaceStarted + // Finishes whatever the scheduled drain has not reached, so the reclaim is + // priced rather than left half done under the next measurement. + const reclaimStarted = performance.now() + await store.purgeOlderThan(null) + const reclaimMs = performance.now() - reclaimStarted + restoreReplaceExec() + assert.deepEqual(errors, []) + assertIndexedMessages(store.connection, large.messageCount) + replaceTransactions.sort((a, b) => a - b) + + console.log( + JSON.stringify( + { + phase: 'single-large-file', + transcriptMb: Math.round((large.transcriptBytes / (1024 * 1024)) * 100) / 100, + indexMs: Math.round(indexMs), + transactions: transactions.length, + maxTransactionMs: Math.round(transactions.at(-1) ?? 0), + replaceMs: Math.round(replaceMs), + replaceTransactions: replaceTransactions.length, + maxReplaceTransactionMs: Math.round(replaceTransactions.at(-1) ?? 0), + reclaimMs: Math.round(reclaimMs), + indexMb: + Math.round( + ((await checkpointedFileBytes(store.connection, largeIndexPath)) / (1024 * 1024)) * + 100 + ) / 100 + }, + null, + 2 + ) + ) + } finally { + restoreExec() + unregister() + resetTranscriptConsumersForTests() + resetSessionParseCacheForTests() + store.close() + } +} finally { + await rm(large.root, { recursive: true, force: true }) +} diff --git a/src/main/ai-vault-search/session-search-content-hash.test.ts b/src/main/ai-vault-search/session-search-content-hash.test.ts new file mode 100644 index 00000000000..1e7232fc855 --- /dev/null +++ b/src/main/ai-vault-search/session-search-content-hash.test.ts @@ -0,0 +1,48 @@ +import { expect, it } from 'vitest' +import { + EMPTY_CONTENT_HASH, + foldContentHash, + isCollapsibleContentHash +} from './session-search-content-hash' +import { userMessages } from './session-search-index-test-fixture' + +it('reaches the same digest whether the prefix arrives whole or in two appends', () => { + const messages = userMessages('turn', 5) + const whole = foldContentHash(EMPTY_CONTENT_HASH, messages) + const resumed = foldContentHash( + foldContentHash(EMPTY_CONTENT_HASH, messages.slice(0, 2)), + messages.slice(2) + ) + + expect(resumed).toEqual(whole) + expect(whole.count).toBe(5) +}) + +it('freezes once the prefix limit is reached so later appends cannot move it', () => { + // Found rather than imported: the limit is the module's business, and a test + // that reads it off the export cannot notice the fold ignoring it. + const capped = foldContentHash(EMPTY_CONTENT_HASH, userMessages('turn', 64)) + expect(capped.count).toBeLessThan(64) + expect(foldContentHash(capped, userMessages('later', 20))).toEqual(capped) +}) + +it('separates two conversations that share an opening prompt', () => { + const shared = userMessages('same opening', 1) + const first = foldContentHash(EMPTY_CONTENT_HASH, [ + ...shared, + { role: 'user', text: 'left', timestamp: null } + ]) + const second = foldContentHash(EMPTY_CONTENT_HASH, [ + ...shared, + { role: 'user', text: 'right', timestamp: null } + ]) + expect(first.hash).not.toBe(second.hash) +}) + +it('refuses to collapse on a prefix too short to mean anything', () => { + const one = foldContentHash(EMPTY_CONTENT_HASH, userMessages('only turn', 1)) + expect(isCollapsibleContentHash(one.hash, one.count)).toBe(false) + const two = foldContentHash(EMPTY_CONTENT_HASH, userMessages('two turns', 2)) + expect(isCollapsibleContentHash(two.hash, two.count)).toBe(true) + expect(isCollapsibleContentHash(null, 9)).toBe(false) +}) diff --git a/src/main/ai-vault-search/session-search-content-hash.ts b/src/main/ai-vault-search/session-search-content-hash.ts new file mode 100644 index 00000000000..a9d2ab229da --- /dev/null +++ b/src/main/ai-vault-search/session-search-content-hash.ts @@ -0,0 +1,45 @@ +import { createHash } from 'node:crypto' +import type { TranscriptMessage } from '../ai-vault/session-transcript-consumers' + +// Why: Claude `--resume` and Codex fork copy the parent transcript into a new +// file under a new session id, so one conversation lands N times in results. +// The shared opening prefix is what identifies the copy; the tail diverges. +const CONTENT_HASH_MESSAGE_LIMIT = 8 +// One shared opening prompt is not evidence of a fork; two turns is. +const CONTENT_HASH_MIN_MESSAGES = 2 + +export type SessionContentHash = { hash: string | null; count: number } + +export const EMPTY_CONTENT_HASH: SessionContentHash = { hash: null, count: 0 } + +/** + * Chained digest over the first `CONTENT_HASH_MESSAGE_LIMIT` messages. Chaining + * (rather than hashing one joined string) makes it resumable, so an `append` + * can finish a prefix a short `replace` started; once the limit is reached the + * value is frozen and later appends leave it untouched. + */ +export function foldContentHash( + previous: SessionContentHash, + messages: readonly TranscriptMessage[] +): SessionContentHash { + let { hash, count } = previous + for (const message of messages) { + if (count >= CONTENT_HASH_MESSAGE_LIMIT) { + break + } + hash = createHash('sha256') + .update(hash ?? '') + .update('\0') + .update(message.role) + .update('\0') + .update(message.text) + .digest('hex') + count += 1 + } + return { hash, count } +} + +/** Sessions collapse only on a hash that covers enough turns to mean anything. */ +export function isCollapsibleContentHash(hash: string | null, count: number): hash is string { + return hash !== null && count >= CONTENT_HASH_MIN_MESSAGES +} diff --git a/src/main/ai-vault-search/session-search-cwd-key.test.ts b/src/main/ai-vault-search/session-search-cwd-key.test.ts new file mode 100644 index 00000000000..24d915eedc9 --- /dev/null +++ b/src/main/ai-vault-search/session-search-cwd-key.test.ts @@ -0,0 +1,37 @@ +import { expect, it } from 'vitest' +import { folderGroupKey } from '../../shared/ai-vault-session-filters' +import { cwdKey } from './session-search-file-records' + +// The sidebar groups sessions by `folderGroupKey`, which is the shared +// normalizer under a `folder:` prefix. A hit's `cwd_key` has to be the same +// string, or joining an indexed hit to a sidebar group returns nothing. +const CASES: [name: string, cwd: string][] = [ + ['a POSIX path', '/repo/app'], + ['a trailing slash', '/repo/app/'], + ['a Windows drive', 'C:\\Users\\me\\repo'], + ['a WSL interop mount', '/mnt/c/Users/me/repo'], + ['a WSL UNC path', '\\\\wsl.localhost\\Ubuntu\\home\\me\\repo'], + ['the wsl$ alias for the same path', '//wsl$/Ubuntu/home/me/repo'], + ['a Linux path from inside WSL', '/home/me/repo'], + ['the filesystem root', '/'] +] + +it.each(CASES)('keys %s exactly as the sidebar does', (_name, cwd) => { + expect(`folder:${cwdKey(cwd)}`).toBe(folderGroupKey(cwd)) +}) + +it('keeps the root as a path rather than collapsing it to nothing', () => { + // An empty key is indistinguishable from "no cwd", and the scope filter builds + // its child prefix as `key + '/'`, which would be `//` for an empty key. + expect(cwdKey('/')).toBe('/') +}) + +it('has no key for a session whose cwd the transcript never recorded', () => { + expect(cwdKey(null)).toBeNull() +}) + +it('folds the two WSL UNC aliases onto one key', () => { + expect(cwdKey('\\\\wsl.localhost\\Ubuntu\\home\\me\\repo')).toBe( + cwdKey('//wsl$/ubuntu/home/me/repo') + ) +}) diff --git a/src/main/ai-vault-search/session-search-file-cursor.ts b/src/main/ai-vault-search/session-search-file-cursor.ts new file mode 100644 index 00000000000..2ba978b00ae --- /dev/null +++ b/src/main/ai-vault-search/session-search-file-cursor.ts @@ -0,0 +1,41 @@ +import type { FileWithMtime } from '../ai-vault/session-scanner-types' + +// Why the index keeps its own cursor: the parse cache's cursor answers "what +// does the session list already show", which is a different question from "what +// bytes of this file are already rows". They diverge the moment either side +// declines a read, so neither may consult the other. + +/** Filesystem identity, when discovery could prove it. */ +export type SessionSearchFileIdentity = { dev: number; ino: number } | null + +/** + * What the index holds for one transcript. + * + * A null `byteOffset` is a file the index holds rows for and cannot continue: + * a chunked read committed a prefix, and the reader only hands out an offset + * when a read finishes. Null rather than a flag because every caller that does + * arithmetic on the offset then has to say what it means here, at compile time, + * instead of ignoring a boolean it did not know to read. + */ +export type SessionSearchIndexedFile = { + byteOffset: number | null + mtimeMs: number + sizeBytes: number | null +} + +/** + * Whether this file has to be read from the start, whatever its stat says. + * + * The mtime and size are the real ones, so a freshness check that compares only + * those would call a half-written file current and never re-read it. Every such + * check must start here. + */ +export function requiresWholeRead(indexed: SessionSearchIndexedFile | null): boolean { + return indexed !== null && indexed.byteOffset === null +} + +export function fileIdentity(file: FileWithMtime): SessionSearchFileIdentity { + return typeof file.dev === 'number' && typeof file.ino === 'number' + ? { dev: file.dev, ino: file.ino } + : null +} diff --git a/src/main/ai-vault-search/session-search-file-records.ts b/src/main/ai-vault-search/session-search-file-records.ts new file mode 100644 index 00000000000..2ee79cbdc13 --- /dev/null +++ b/src/main/ai-vault-search/session-search-file-records.ts @@ -0,0 +1,134 @@ +import { fileIdentity } from './session-search-file-cursor' +import type { AiVaultSession } from '../../shared/ai-vault-types' +import type { TranscriptSessionIdentity } from '../ai-vault/session-transcript-consumers' +import type { SessionFileCandidate } from '../ai-vault/session-scanner-types' +import type SyncDatabase from '../sqlite/sync-database' +import { EMPTY_CONTENT_HASH, type SessionContentHash } from './session-search-content-hash' +import { normalizeRuntimePathForComparison } from '../../shared/cross-platform-path' + +/** + * The stored comparison key for a session's working directory. + * + * Why the shared normalizer verbatim: the sidebar already groups sessions by + * `folderGroupKey`, which is this function under a prefix. A second spelling + * here means any later join between an indexed hit and a sidebar group returns + * nothing. An earlier version qualified a WSL cwd with its distro so two + * distros could not collide at `/home/me/repo`; that is a real collision, but it + * is one every SSH host has too, neither key qualifies for SSH, and the fix for + * it is a column that names the execution host, not a path key that only some + * hosts spell differently. + */ +export function cwdKey(cwd: string | null): string | null { + return cwd ? normalizeRuntimePathForComparison(cwd) : null +} + +export class SessionSearchFileRecords { + constructor(private readonly db: SyncDatabase) {} + /** + * The row a read hangs its messages off, before the parser has said what the + * session is. The same transaction fills it in: from the decoded session when + * the read finished, and from `updateProvisionalSession` when this is a chunk + * of one that has not. + */ + createSessionRow(candidate: SessionFileCandidate): number { + return Number( + this.db + .prepare( + `INSERT INTO sessions(agent,session_id,file_path,title,resume_command) + VALUES (?,'',?,'','')` + ) + .run(candidate.agent, candidate.file.path).lastInsertRowid + ) + } + + /** + * Writes what the parser knows so far onto a session a chunk is committing. + * + * Rows a chunk commits answer searches the moment they land, so the session + * they hang off has to be nameable before the read producing it ends — and it + * may never end, because a crash between chunks leaves exactly this row. That + * is why the identity is required rather than optional: a read that has none + * does not chunk at all. The final commit overwrites all of it from the + * decoded session; until then the title in particular is provisional. + */ + updateProvisionalSession(rowId: number, identity: TranscriptSessionIdentity): void { + this.db + .prepare( + `UPDATE sessions SET session_id = ?, title = ?, cwd = ?, cwd_key = ?, + created_at = ?, updated_at = ? WHERE id = ?` + ) + .run( + identity.sessionId, + identity.title ?? '', + identity.cwd, + cwdKey(identity.cwd), + identity.createdAt, + identity.updatedAt, + rowId + ) + } + + contentHash(rowId: number): SessionContentHash { + const row = this.db + .prepare('SELECT content_hash, content_hash_count FROM sessions WHERE id = ?') + .get(rowId) as { content_hash: string | null; content_hash_count: number } | undefined + return row ? { hash: row.content_hash, count: row.content_hash_count } : EMPTY_CONTENT_HASH + } + + updateSession(session: AiVaultSession, rowId: number, contentHash: SessionContentHash): void { + const values = [ + session.agent, + session.sessionId, + session.filePath, + session.codexHome, + session.title, + session.cwd, + cwdKey(session.cwd), + session.branch, + session.createdAt, + session.updatedAt, + session.messageCount, + session.resumeCommand, + contentHash.hash, + contentHash.count + ] + this.db + .prepare( + `UPDATE sessions SET agent = ?, session_id = ?, file_path = ?, codex_home = ?, title = ?, + cwd = ?, cwd_key = ?, branch = ?, created_at = ?, updated_at = ?, message_count = ?, resume_command = ?, + content_hash = ?, content_hash_count = ? WHERE id = ?` + ) + .run(...values, rowId) + } + + upsertFile( + candidate: SessionFileCandidate, + byteOffset: number, + sessionRowId: number | null + ): void { + const { file } = candidate + const identity = fileIdentity(file) + this.db + .prepare( + `INSERT INTO files(path, dev, ino, byte_offset, mtime_ms, size_bytes, session_row_id) + VALUES (?, ?, ?, ?, ?, ?, ?) + ON CONFLICT(path) DO UPDATE SET + -- Partial observations must never create a pair that no stat proved. + dev = CASE WHEN excluded.dev IS NOT NULL AND excluded.ino IS NOT NULL + THEN excluded.dev ELSE files.dev END, + ino = CASE WHEN excluded.dev IS NOT NULL AND excluded.ino IS NOT NULL + THEN excluded.ino ELSE files.ino END, + byte_offset = excluded.byte_offset, mtime_ms = excluded.mtime_ms, + size_bytes = excluded.size_bytes, session_row_id = excluded.session_row_id` + ) + .run( + file.path, + identity?.dev ?? null, + identity?.ino ?? null, + byteOffset, + file.mtimeMs, + file.sizeBytes ?? null, + sessionRowId + ) + } +} diff --git a/src/main/ai-vault-search/session-search-file-write.test.ts b/src/main/ai-vault-search/session-search-file-write.test.ts new file mode 100644 index 00000000000..942565e18d6 --- /dev/null +++ b/src/main/ai-vault-search/session-search-file-write.test.ts @@ -0,0 +1,614 @@ +import { afterEach, beforeEach, expect, it, vi } from 'vitest' +import { resetTranscriptConsumersForTests } from '../ai-vault/session-transcript-consumers' +import SyncDatabase from '../sqlite/sync-database' +import { registerSessionSearchIndexConsumer } from './session-search-index-consumer' +import { cwdKey } from './session-search-file-records' +import { requiresWholeRead } from './session-search-file-cursor' +import { SessionSearchIndexWriter } from './session-search-index-writer' +import { deleteExpiredSearchFiles } from './session-search-retention-delete' +import { + openSessionSearchIndexFile, + replayTranscriptRead, + syntheticCandidate, + syntheticSession, + SYNTHETIC_TRANSCRIPT, + userMessages, + type SessionSearchIndexFile +} from './session-search-index-test-fixture' +import { SessionSearchStore } from './session-search-store' + +// Every assertion here reads through `index.db`, a second connection to the same +// file. That is the whole consistency model: one transaction per file in WAL +// mode, so another handle sees the last committed state and never a session part +// way through being rewritten. + +let index: SessionSearchIndexFile +let store: SessionSearchStore +let errors: unknown[] + +beforeEach(async () => { + index = await openSessionSearchIndexFile('ss-file-write') + errors = [] + store = new SessionSearchStore(index.path, (error) => errors.push(error)) + registerSessionSearchIndexConsumer(store) +}) + +afterEach(async () => { + vi.restoreAllMocks() + resetTranscriptConsumersForTests() + store.close() + await index.close() +}) + +function matches(db: SyncDatabase, table: string, term: string): number { + return ( + db + .prepare( + `SELECT count(*) AS n FROM ${table} JOIN messages m ON m.id = ${table}.rowid + JOIN sessions s ON s.id = m.session_row_id WHERE ${table} MATCH ?` + ) + .get(term) as { n: number } + ).n +} + +/** Fails the nth statement matching `pick`, wherever the writer prepares it. */ +function failOnStatement(pick: (sql: string) => boolean, nth: number): void { + const prepare = SyncDatabase.prototype.prepare + let seen = 0 + vi.spyOn(SyncDatabase.prototype, 'prepare').mockImplementation(function ( + this: SyncDatabase, + sql: string + ) { + if (pick(sql) && ++seen === nth) { + throw new Error('index write crashed mid transaction') + } + return prepare.call(this, sql) + }) +} + +function counts(db: SyncDatabase): Record { + const one = (sql: string): number => (db.prepare(sql).get() as { n: number }).n + return { + sessions: one('SELECT count(*) AS n FROM sessions'), + messages: one('SELECT count(*) AS n FROM messages'), + files: one('SELECT count(*) AS n FROM files'), + full: one('SELECT count(*) AS n FROM messages_fts') + } +} + +it('writes a whole read in one transaction', () => { + replayTranscriptRead({ messages: userMessages('needle text', 300) }) + + const after = counts(index.db) + expect(after.sessions).toBe(1) + expect(after.messages).toBe(300) + expect(after.full).toBe(300) + expect(errors).toEqual([]) +}) + +it('files every row in one FTS table, under the column its role owns', () => { + replayTranscriptRead({ + messages: [ + { role: 'user', text: 'alpha question', timestamp: null }, + { role: 'assistant', text: 'beta answer', timestamp: null }, + { role: 'tool', text: 'gamma tool output', timestamp: null } + ] + }) + + // One table carries all three; the conversation scope is a column filter over + // it, which is what the second table used to be. + expect(counts(index.db).full).toBe(3) + expect(matches(index.db, 'messages_fts', 'gamma')).toBe(1) + expect(matches(index.db, 'messages_fts', '{user_text assistant_text}: gamma')).toBe(0) + expect(matches(index.db, 'messages_fts', '{user_text assistant_text}: beta')).toBe(1) +}) + +it('leaves the index exactly as it found it when a read never finishes', () => { + const write = store.beginWrite(syntheticCandidate(), 'replace', 0)! + for (const message of userMessages('neverfinished', 200)) { + write.add(message) + } + // The process dies here: the rows only ever existed in this buffer. + expect(counts(index.db)).toMatchObject({ + sessions: 0, + messages: 0, + files: 0 + }) +}) + +it('rolls a whole file back when a write throws part way through its transaction', () => { + replayTranscriptRead({ + messages: userMessages('firstgeneration', 3), + outcome: { byteOffset: 40 } + }) + const before = counts(index.db) + + failOnStatement((sql) => sql.startsWith('INSERT INTO messages('), 50) + replayTranscriptRead({ + messages: userMessages('crashedgeneration', 100), + outcome: { byteOffset: 900 } + }) + vi.restoreAllMocks() + + // Not one of the 49 rows that were already inserted survived, the previous + // generation is untouched, and the cursor still describes what is really here. + expect(counts(index.db)).toEqual(before) + expect(matches(index.db, 'messages_fts', 'crashedgeneration')).toBe(0) + expect(matches(index.db, 'messages_fts', 'firstgeneration')).toBe(3) + expect(store.indexedFile(SYNTHETIC_TRANSCRIPT, null)?.byteOffset).toBe(40) + expect(errors).toHaveLength(1) + // The file is owed a re-read, which is the only reason anything was lost. + expect(store.takeStale().map((candidate) => candidate.file.path)).toEqual([SYNTHETIC_TRANSCRIPT]) + + // And the connection is usable again: a transaction left open by the failure + // would take down every write after it, not just the one that threw. + replayTranscriptRead({ + messages: userMessages('afterthecrash', 2), + outcome: { byteOffset: 900 } + }) + expect(matches(index.db, 'messages_fts', 'afterthecrash')).toBe(2) + expect(store.indexedFile(SYNTHETIC_TRANSCRIPT, null)?.byteOffset).toBe(900) +}) + +it('takes the rows back when recording the cursor is what fails', () => { + replayTranscriptRead({ + messages: userMessages('firstgeneration', 3), + outcome: { byteOffset: 40 } + }) + + // The cursor is written last, so this is the crash point that would leave rows + // no cursor describes: a later append would continue from an offset those rows + // already cover, and index the same span twice. + failOnStatement((sql) => sql.startsWith('INSERT INTO files('), 1) + replayTranscriptRead({ + messages: userMessages('crashedgeneration', 5), + outcome: { byteOffset: 900 } + }) + vi.restoreAllMocks() + + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 3 }) + expect(matches(index.db, 'messages_fts', 'firstgeneration')).toBe(3) + expect(matches(index.db, 'messages_fts', 'crashedgeneration')).toBe(0) + expect(store.indexedFile(SYNTHETIC_TRANSCRIPT, null)?.byteOffset).toBe(40) +}) + +it('shows a reader on another handle one generation or the other, never a mixture', async () => { + replayTranscriptRead({ + messages: userMessages('firstgeneration', 3), + outcome: { byteOffset: 40 } + }) + expect(counts(index.db).messages).toBe(3) + + const write = store.beginWrite(syntheticCandidate(), 'replace', 0)! + for (const message of userMessages('secondgeneration', 7)) { + write.add(message) + // Every point at which the other handle could issue a query mid-read. + expect(counts(index.db).messages).toBe(3) + expect(matches(index.db, 'messages_fts', 'secondgeneration')).toBe(0) + } + expect( + write.commit({ + session: syntheticSession(), + byteOffset: 900, + incomplete: false + }) + ).toBe(true) + + expect(matches(index.db, 'messages_fts', 'firstgeneration')).toBe(0) + expect(matches(index.db, 'messages_fts', 'secondgeneration')).toBe(7) + // The old three are cut loose, not deleted, so they are still on disk and + // already unreachable; the drain the store scheduled hands them back. + expect(counts(index.db).messages).toBe(10) + await vi.waitFor(() => { + expect(counts(index.db).messages).toBe(7) + }) +}) + +// Four of these fill the 400-char ceiling the two tests below construct. +const CHUNKED_MESSAGE = `chunkedneedle ${'filler '.repeat(12)}nd` + +const PROVISIONAL_IDENTITY = { + sessionId: 'aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee', + cwd: '/repo/app', + title: 'provisional title', + createdAt: '2026-05-01T10:00:00.000Z', + updatedAt: '2026-05-01T10:05:00.000Z' +} + +// Only a read that can name its session chunks at all, so every test below that +// wants a chunk has to supply one. +const named = (): typeof PROVISIONAL_IDENTITY => PROVISIONAL_IDENTITY + +it('leaves the session consistent after every chunk of a file too large for one transaction', () => { + expect(CHUNKED_MESSAGE.length).toBe(100) + const writer = new SessionSearchIndexWriter(index.db, 400) + const write = writer.beginWrite(syntheticCandidate(), 'replace', 0, named)! + for (const [position, message] of userMessages(CHUNKED_MESSAGE, 10).entries()) { + write.add(message) + const rows = counts(index.db).messages + // Four messages per chunk, and nothing else reaches the file between them. + expect(rows).toBe(Math.floor((position + 1) / 4) * 4) + // Whatever landed is a coherent prefix of this session and answers searches. + expect(matches(index.db, 'messages_fts', 'chunkedneedle')).toBe(rows) + if (rows > 0) { + // The cursor a chunk leaves refuses every append rather than inventing an + // offset the reader never gave it. + expect(requiresWholeRead(writer.indexedFile(SYNTHETIC_TRANSCRIPT, null))).toBe(true) + expect(writer.beginWrite(syntheticCandidate(), 'append', 0)).toBeNull() + } + } + expect(counts(index.db).messages).toBe(8) + + expect( + write.commit({ + session: syntheticSession(), + byteOffset: 4096, + incomplete: false + }) + ).toBe(true) + expect(counts(index.db)).toMatchObject({ + sessions: 1, + messages: 10, + full: 10 + }) + expect(writer.indexedFile(SYNTHETIC_TRANSCRIPT, null)?.byteOffset).toBe(4096) +}) + +it('holds the ceiling against a single message larger than it', () => { + const writer = new SessionSearchIndexWriter(index.db, 8000) + const write = writer.beginWrite(syntheticCandidate(), 'replace', 0, named)! + const exec = SyncDatabase.prototype.exec + let opened = 0 + vi.spyOn(SyncDatabase.prototype, 'exec').mockImplementation(function ( + this: SyncDatabase, + sql: string + ) { + if (sql === 'BEGIN IMMEDIATE') { + opened += 1 + } + exec.call(this, sql) + }) + + // One conversation turn, three times the ceiling. Checked once per message, + // this commits all 24,000 characters in a single transaction — the ceiling + // bounds nothing that a message can exceed on its own. + write.add({ role: 'assistant', text: 'a'.repeat(24_000), timestamp: null }) + vi.restoreAllMocks() + + expect(opened).toBe(3) + expect(counts(index.db).messages).toBe(3) + expect( + write.commit({ + session: syntheticSession(), + byteOffset: 4096, + incomplete: false + }) + ).toBe(true) + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 3 }) +}) + +it('names a session on its first chunk, not only when the read ends', () => { + const writer = new SessionSearchIndexWriter(index.db, 400) + const write = writer.beginWrite(syntheticCandidate(), 'replace', 0, named)! + for (const message of userMessages(CHUNKED_MESSAGE, 10)) { + write.add(message) + } + + // The chunks that landed already answer searches, so the session they hang + // off has to be nameable on another handle before the read ends. This is also + // the whole record a crash between chunks leaves behind. + expect(counts(index.db).messages).toBe(8) + expect( + index.db.prepare('SELECT session_id, cwd, cwd_key, title, created_at FROM sessions').get() + ).toEqual({ + session_id: PROVISIONAL_IDENTITY.sessionId, + cwd: '/repo/app', + cwd_key: cwdKey('/repo/app'), + title: 'provisional title', + created_at: '2026-05-01T10:00:00.000Z' + }) + + // And the decoded session still wins at the end: the mid-read title is + // provisional, never a value the final commit has to defer to. + expect( + write.commit({ + session: syntheticSession({ title: 'the settled title' }), + byteOffset: 4096, + incomplete: false + }) + ).toBe(true) + expect(index.db.prepare('SELECT title FROM sessions').get()).toEqual({ + title: 'the settled title' + }) +}) + +it('commits a whole-file read over the ceiling in one transaction, never a chunk', () => { + // The whole-file readers (Grok, Cursor, Gemini, OpenCode) pass no identity: + // their formats are rewritten in place and have no resumable state to ask. + const writer = new SessionSearchIndexWriter(index.db, 400) + const write = writer.beginWrite(syntheticCandidate(), 'replace', 0)! + const exec = SyncDatabase.prototype.exec + let opened = 0 + vi.spyOn(SyncDatabase.prototype, 'exec').mockImplementation(function ( + this: SyncDatabase, + sql: string + ) { + if (sql === 'BEGIN IMMEDIATE') { + opened += 1 + } + exec.call(this, sql) + }) + + for (const message of userMessages(CHUNKED_MESSAGE, 10)) { + write.add(message) + // Chunking here would publish rows under a session with an empty id, an + // empty title and a null cwd, and an interrupted read would leave that + // prefix answering searches for good. + expect(counts(index.db)).toMatchObject({ sessions: 0, messages: 0, files: 0 }) + } + expect(write.commit({ session: syntheticSession(), byteOffset: 4096, incomplete: false })).toBe( + true + ) + vi.restoreAllMocks() + + expect(opened).toBe(1) + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 10, full: 10 }) + // And a real cursor, not the partial sentinel a chunk would have left. + expect(writer.indexedFile(SYNTHETIC_TRANSCRIPT, null)?.byteOffset).toBe(4096) +}) + +it('starts chunking only once the parser has an id to name the session with', () => { + const writer = new SessionSearchIndexWriter(index.db, 400) + let decoded: typeof PROVISIONAL_IDENTITY | null = null + const write = writer.beginWrite(syntheticCandidate(), 'replace', 0, () => decoded)! + for (const message of userMessages(CHUNKED_MESSAGE, 4)) { + write.add(message) + } + // Past the ceiling, but the parser has decoded nothing: the buffer keeps + // growing rather than naming a session it cannot name. + expect(counts(index.db).messages).toBe(0) + + decoded = PROVISIONAL_IDENTITY + write.add(userMessages(CHUNKED_MESSAGE, 1)[0]!) + + // Everything held goes with the first chunk that can say what it is. + expect(counts(index.db).messages).toBe(5) + expect(index.db.prepare('SELECT session_id, cwd FROM sessions').get()).toEqual({ + session_id: PROVISIONAL_IDENTITY.sessionId, + cwd: '/repo/app' + }) +}) + +it('reports a chunk-partial file as held, and as one that must be read whole', () => { + const writer = new SessionSearchIndexWriter(index.db, 400) + const write = writer.beginWrite(syntheticCandidate(), 'replace', 0, named)! + for (const message of userMessages(CHUNKED_MESSAGE, 10)) { + write.add(message) + } + + // Held, with no cursor to continue. Reporting nothing here reads as "never + // indexed", so a caller asks for whatever read the parse cache offers, the + // reader picks append, and only a decline heals it a cycle later. + const held = writer.indexedFile(SYNTHETIC_TRANSCRIPT, null) + expect(held).not.toBeNull() + expect(held?.byteOffset).toBeNull() + expect(requiresWholeRead(held)).toBe(true) + expect(held?.mtimeMs).toBe(syntheticCandidate().file.mtimeMs) + + // A file this index has never seen is still the other answer, so the two + // states a caller has to tell apart are distinguishable. + expect(writer.indexedFile('/never-seen.jsonl', null)).toBeNull() + expect(requiresWholeRead(null)).toBe(false) + + // And no offset continues it, including the one the chunk recorded. + for (const offset of [0, -1, 400, 1000]) { + expect(writer.beginWrite(syntheticCandidate(), 'append', offset)).toBeNull() + } +}) + +it('re-reads a chunked file whole when its writer died between chunks', async () => { + const writer = new SessionSearchIndexWriter(index.db, 400) + const abandoned = writer.beginWrite(syntheticCandidate(), 'replace', 0, named)! + for (const message of userMessages(CHUNKED_MESSAGE, 10)) { + abandoned.add(message) + } + expect(counts(index.db).messages).toBe(8) + + // Nothing can continue that prefix, so the only way forward is a whole re-read, + // and that replaces every row the dead writer left. + expect(requiresWholeRead(writer.indexedFile(SYNTHETIC_TRANSCRIPT, null))).toBe(true) + const replacement = writer.beginWrite(syntheticCandidate(), 'replace', 0)! + replacement.add(userMessages('wholereread', 1)[0]!) + expect( + replacement.commit({ + session: syntheticSession(), + byteOffset: 4096, + incomplete: false + }) + ).toBe(true) + // The eight stranded rows stop answering the moment the replace commits, and + // the drain hands them back after it rather than inside it. + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 9 }) + expect(matches(index.db, 'messages_fts', 'chunkedneedle')).toBe(0) + await deleteExpiredSearchFiles(index.db, null, () => false) + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 1 }) +}) + +it('stops a chunked read whose file was removed between its chunks', () => { + const writer = new SessionSearchIndexWriter(index.db, 400) + const write = writer.beginWrite(syntheticCandidate(), 'replace', 0, named)! + const messages = userMessages(CHUNKED_MESSAGE, 10) + for (const message of messages.slice(0, 4)) { + write.add(message) + } + expect(counts(index.db).messages).toBe(4) + + writer.removeFile(SYNTHETIC_TRANSCRIPT) + const exec = SyncDatabase.prototype.exec + let opened = 0 + vi.spyOn(SyncDatabase.prototype, 'exec').mockImplementation(function ( + this: SyncDatabase, + sql: string + ) { + if (sql === 'BEGIN IMMEDIATE') { + opened += 1 + } + exec.call(this, sql) + }) + for (const message of messages.slice(4)) { + write.add(message) + } + expect(write.commit({ session: syntheticSession(), byteOffset: 4096, incomplete: false })).toBe( + false + ) + vi.restoreAllMocks() + + // Not one row of the removed source came back. The read stopped at the first + // refusal rather than reopening a transaction it already knows will roll back, + // once for every message left in a file that may be a hundred megabytes. + expect(opened).toBe(1) + expect(counts(index.db)).toMatchObject({ sessions: 0, messages: 0, files: 0, full: 0 }) +}) + +it('fences a first-ever read whose file was removed before it committed', () => { + const candidate = syntheticCandidate({ path: '/never-indexed.jsonl' }) + const write = store.beginWrite(candidate, 'replace', 0)! + for (const message of userMessages('removedbeforefirstcommit', 3)) { + write.add(message) + } + // The path was never indexed, so there is no cursor for the removal to move. + // PR 3's retirement sweep removes exactly these: paths the index deferred over + // budget and never wrote, while the registered consumer is fed concurrently. + store.removeFile('/never-indexed.jsonl') + + expect(write.commit({ session: syntheticSession(), byteOffset: 300, incomplete: false })).toBe( + false + ) + expect(counts(index.db)).toMatchObject({ sessions: 0, messages: 0, files: 0, full: 0 }) +}) + +it('replaces the previous generation without ever showing both', async () => { + replayTranscriptRead({ messages: userMessages('firstgeneration', 10) }) + replayTranscriptRead({ messages: userMessages('secondgeneration', 10) }) + + expect(matches(index.db, 'messages_fts', 'firstgeneration')).toBe(0) + expect(matches(index.db, 'messages_fts', 'secondgeneration')).toBe(10) + await vi.waitFor(() => { + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 10, full: 10 }) + }) +}) + +it('replaces a generation by cutting the old one loose, not by deleting it inline', async () => { + const writer = new SessionSearchIndexWriter(index.db) + const first = writer.beginWrite(syntheticCandidate(), 'replace', 0)! + for (const message of userMessages('firstgeneration', 200)) { + first.add(message) + } + expect(first.commit({ session: syntheticSession(), byteOffset: 100, incomplete: false })).toBe( + true + ) + const before = index.db.prepare('SELECT id FROM sessions').get() as { id: number } + expect(counts(index.db).messages).toBe(200) + + const second = writer.beginWrite(syntheticCandidate(), 'replace', 0)! + for (const message of userMessages('secondgeneration', 3)) { + second.add(message) + } + expect(second.commit({ session: syntheticSession(), byteOffset: 200, incomplete: false })).toBe( + true + ) + + // The transaction inserted three rows and deleted one, rather than deleting + // two hundred: all 203 are still on disk, and the old 200 already answer + // nothing, because every retrieval joins `sessions`. + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 203, full: 203 }) + expect(matches(index.db, 'messages_fts', 'firstgeneration')).toBe(0) + expect(matches(index.db, 'messages_fts', 'secondgeneration')).toBe(3) + + // A new session row, with `files` repointed at it in that same transaction. + // AUTOINCREMENT never hands the freed id back while orphans still name it. + const after = index.db.prepare('SELECT id FROM sessions').get() as { id: number } + expect(after.id).toBeGreaterThan(before.id) + expect(index.db.prepare('SELECT session_row_id FROM files').get()).toEqual({ + session_row_id: after.id + }) + + await deleteExpiredSearchFiles(index.db, null, () => false) + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 3, full: 3 }) +}) + +it('drains what a replace cut loose without being asked', async () => { + replayTranscriptRead({ messages: userMessages('firstgeneration', 200) }) + replayTranscriptRead({ messages: userMessages('secondgeneration', 3) }) + + // The store schedules the reclaim the way it schedules retention's. Hiding a + // generation and never reclaiming it would grow the file by every re-read. + expect(matches(index.db, 'messages_fts', 'firstgeneration')).toBe(0) + await vi.waitFor(() => { + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 3, full: 3 }) + }) + expect(errors).toEqual([]) +}) + +it('continues a session across an append rather than replaying it', () => { + replayTranscriptRead({ + messages: userMessages('openingturn', 3), + outcome: { byteOffset: 40 } + }) + replayTranscriptRead({ + messages: userMessages('laterturn', 2), + mode: 'append', + previousByteOffset: 40, + outcome: { byteOffset: 90 } + }) + + expect(counts(index.db)).toMatchObject({ sessions: 1, messages: 5 }) + expect(matches(index.db, 'messages_fts', 'openingturn')).toBe(3) + expect(matches(index.db, 'messages_fts', 'laterturn')).toBe(2) + expect(store.indexedFile(SYNTHETIC_TRANSCRIPT, null)?.byteOffset).toBe(90) +}) + +it('stops answering for a removed file the moment it is removed', () => { + replayTranscriptRead({ messages: userMessages('removedneedle', 3) }) + store.removeFile(SYNTHETIC_TRANSCRIPT) + + expect(counts(index.db)).toMatchObject({ + sessions: 0, + messages: 0, + files: 0, + full: 0 + }) + expect(matches(index.db, 'messages_fts', 'removedneedle')).toBe(0) +}) + +it('writes nothing for an incomplete read and owes the file a whole re-read', () => { + replayTranscriptRead({ + messages: userMessages('incompleteread', 300), + outcome: { incomplete: true } + }) + + expect(counts(index.db)).toMatchObject({ + sessions: 0, + messages: 0, + files: 0, + full: 0 + }) + expect(store.pendingFileCount).toBe(1) + expect(errors).toEqual([]) +}) + +it('exposes the handle a composed reader queries through', () => { + replayTranscriptRead({ messages: userMessages('composedreader', 3) }) + + // PR 4's engine reads through this rather than opening a second connection, + // so it sees a write the moment the transaction commits. + expect(store.connection.prepare('SELECT count(*) AS n FROM messages').get()).toEqual({ n: 3 }) +}) + +it('closes twice without turning the second call into an error', () => { + store.close() + // node:sqlite throws ERR_INVALID_STATE on a second close of one handle, and a + // store is closed both by whoever owns it and by a teardown that cannot know. + expect(() => store.close()).not.toThrow() + store = new SessionSearchStore(index.path, (error) => errors.push(error)) +}) diff --git a/src/main/ai-vault-search/session-search-identifier-split.test.ts b/src/main/ai-vault-search/session-search-identifier-split.test.ts new file mode 100644 index 00000000000..24f8a67d5c3 --- /dev/null +++ b/src/main/ai-vault-search/session-search-identifier-split.test.ts @@ -0,0 +1,29 @@ +import { expect, it } from 'vitest' +import { identifierShadowTerms, identifierShadowText } from './session-search-identifier-split' + +it('splits a camel-case symbol into its pieces and keeps the whole', () => { + expect(identifierShadowTerms('call resolveTerminalPath here')).toEqual([ + 'resolveterminalpath', + 'resolve', + 'terminal', + 'path' + ]) +}) + +it('splits a path into its segments and extension', () => { + // The whole path already tokenizes on its own; only the pieces need shadowing. + expect(identifierShadowText('src/main/foo-bar.ts')).toBe('src main foo bar ts') +}) + +it('leaves ordinary prose alone', () => { + expect(identifierShadowTerms('the quick brown fox')).toEqual([]) +}) + +it('shadows a screaming-case constant', () => { + expect(identifierShadowTerms('MAX_RETRIES')).toEqual(['max', 'retries']) +}) + +it('stops at the term limit rather than growing with the message', () => { + const text = Array.from({ length: 50 }, (_unused, index) => `alpha_beta${index}`).join(' ') + expect(identifierShadowTerms(text, 10)).toHaveLength(10) +}) diff --git a/src/main/ai-vault-search/session-search-identifier-split.ts b/src/main/ai-vault-search/session-search-identifier-split.ts new file mode 100644 index 00000000000..e2df1822cfa --- /dev/null +++ b/src/main/ai-vault-search/session-search-identifier-split.ts @@ -0,0 +1,54 @@ +// Identifier shadow terms: `resolveTerminalPath` → `resolve terminal path`, +// `src/main/foo-bar.ts` → `src main foo bar ts`. Stored in a separate FTS5 +// column so a partial identifier still matches; the largest single accuracy +// win measured in the retrieval shoot-out (MRR 0.50 → 0.55). + +const RAW_TOKEN = /[A-Za-z0-9_./-]+/g +const CAMEL_PIECE = /[A-Z]+(?![a-z])|[A-Z][a-z0-9]*|[a-z0-9]+/g +const SEPARATOR = /[_./-]+/ +// Worth shadowing: has a separator, a camel boundary, or is SCREAMING_CASE. +const INTERESTING = /[_./-]|[a-z0-9][A-Z]|^[A-Z]{2,}[0-9_]*$/ +const MIN_TOKEN = 3 +const MAX_TOKEN = 120 +const MIN_PIECE = 2 + +function hasMixedCase(piece: string): boolean { + return /[a-z]/.test(piece) && /[A-Z]/.test(piece) +} + +export function identifierShadowTerms(text: string, limit = 4000): string[] { + const out: string[] = [] + const seen = new Set() + for (const match of text.matchAll(RAW_TOKEN)) { + const token = match[0] + if (token.length < MIN_TOKEN || token.length > MAX_TOKEN || !INTERESTING.test(token)) { + continue + } + const parts: string[] = [] + for (const piece of token.split(SEPARATOR)) { + if (!piece) { + continue + } + parts.push(piece) + if (hasMixedCase(piece)) { + parts.push(...(piece.match(CAMEL_PIECE) ?? [])) + } + } + for (const part of parts) { + const lowered = part.toLowerCase() + if (lowered.length < MIN_PIECE || seen.has(lowered)) { + continue + } + seen.add(lowered) + out.push(lowered) + if (out.length >= limit) { + return out + } + } + } + return out +} + +export function identifierShadowText(text: string, limit?: number): string { + return identifierShadowTerms(text, limit).join(' ') +} diff --git a/src/main/ai-vault-search/session-search-index-consumer.test.ts b/src/main/ai-vault-search/session-search-index-consumer.test.ts new file mode 100644 index 00000000000..ca02333cf0b --- /dev/null +++ b/src/main/ai-vault-search/session-search-index-consumer.test.ts @@ -0,0 +1,373 @@ +import { afterEach, beforeEach, expect, it } from 'vitest' +import { resetTranscriptConsumersForTests } from '../ai-vault/session-transcript-consumers' +import { registerSessionSearchIndexConsumer } from './session-search-index-consumer' +import { + openSessionSearchIndexFile, + replayTranscriptRead, + syntheticCandidate, + syntheticSession, + SYNTHETIC_TRANSCRIPT, + userMessages, + type SessionSearchIndexFile +} from './session-search-index-test-fixture' +import { SessionSearchStore, STALE_PATH_LIMIT } from './session-search-store' + +let index: SessionSearchIndexFile +let store: SessionSearchStore +let errors: unknown[] + +beforeEach(async () => { + index = await openSessionSearchIndexFile('ss-index-consumer') + errors = [] + store = new SessionSearchStore(index.path, (error) => errors.push(error)) + registerSessionSearchIndexConsumer(store) +}) + +afterEach(async () => { + resetTranscriptConsumersForTests() + store.close() + await index.close() +}) + +function indexedMessages(): number { + return ( + index.db.prepare('SELECT count(*) AS n FROM messages').get() as { + n: number + } + ).n +} + +function cursor(): number | null | undefined { + return store.indexedFile(SYNTHETIC_TRANSCRIPT, null)?.byteOffset +} + +it('appends onto its own cursor and carries the content hash forward', async () => { + replayTranscriptRead({ + messages: userMessages('first half', 3), + outcome: { byteOffset: 100 } + }) + const first = index.db + .prepare('SELECT content_hash AS hash, content_hash_count AS count FROM sessions') + .get() as { hash: string; count: number } + + replayTranscriptRead({ + mode: 'append', + previousByteOffset: 100, + messages: userMessages('second half', 2), + outcome: { byteOffset: 220 } + }) + + expect(indexedMessages()).toBe(5) + expect(cursor()).toBe(220) + const second = index.db + .prepare('SELECT content_hash AS hash, content_hash_count AS count FROM sessions') + .get() as { hash: string; count: number } + expect(second.count).toBe(first.count + 2) + expect(second.hash).not.toBe(first.hash) + expect(store.takeStale()).toEqual([]) +}) + +it('appends onto a file it read through and decoded no session from', async () => { + // An excluded Codex worker transcript: read through, nothing to index, and + // still growing. Its cursor is sound, so a re-read of the whole file every + // pass buys nothing. + replayTranscriptRead({ + messages: userMessages('excluded span', 3), + outcome: { session: null, byteOffset: 100 } + }) + expect(cursor()).toBe(100) + expect(store.takeStale()).toEqual([]) + + replayTranscriptRead({ + mode: 'append', + previousByteOffset: 100, + messages: userMessages('decoded at last', 2), + outcome: { byteOffset: 220 } + }) + + expect(indexedMessages()).toBe(2) + expect(cursor()).toBe(220) + expect(store.takeStale()).toEqual([]) +}) + +it('declines an append that starts past its own cursor and records the file', async () => { + replayTranscriptRead({ + messages: userMessages('indexed span', 3), + outcome: { byteOffset: 100 } + }) + + // The session list read further than this index did, so the appended span + // continues from bytes the index never saw. + replayTranscriptRead({ + mode: 'append', + previousByteOffset: 900, + messages: userMessages('unseen span', 4), + outcome: { byteOffset: 1200 } + }) + + expect(indexedMessages()).toBe(3) + expect(cursor()).toBe(100) + expect(store.takeStale().map((candidate) => candidate.file.path)).toEqual([SYNTHETIC_TRANSCRIPT]) +}) + +it('declines a file whose identity changed under the same path', async () => { + const original = syntheticCandidate({ dev: 1, ino: 10 }) + replayTranscriptRead({ + candidate: original, + messages: userMessages('original file', 2), + outcome: { byteOffset: 100 } + }) + + replayTranscriptRead({ + candidate: syntheticCandidate({ dev: 1, ino: 77 }), + mode: 'append', + previousByteOffset: 100, + messages: userMessages('replacement file', 2), + outcome: { byteOffset: 200 } + }) + + expect(indexedMessages()).toBe(2) + expect(store.takeStale()).toHaveLength(1) +}) + +it('never advances the cursor for an incomplete read', async () => { + replayTranscriptRead({ + messages: userMessages('complete span', 3), + outcome: { byteOffset: 100 } + }) + + replayTranscriptRead({ + mode: 'append', + previousByteOffset: 100, + messages: userMessages('partial span', 5), + outcome: { byteOffset: 400, incomplete: true } + }) + + expect(indexedMessages()).toBe(3) + expect(cursor()).toBe(100) + expect( + ( + index.db.prepare('SELECT count(*) AS n FROM messages').get() as { + n: number + } + ).n + ).toBe(3) + expect(store.takeStale()).toHaveLength(1) +}) + +it('indexes nothing at all from a read that was incomplete from the start', async () => { + replayTranscriptRead({ + messages: userMessages('unreachable', 4), + outcome: { byteOffset: 0, incomplete: true } + }) + + expect(index.db.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 0 + }) + expect(index.db.prepare('SELECT count(*) AS n FROM messages').get()).toEqual({ + n: 0 + }) + expect(cursor()).toBeUndefined() +}) + +it('drops a file whose parser returned no session', async () => { + replayTranscriptRead({ + messages: userMessages('was indexed', 3), + outcome: { byteOffset: 100 } + }) + + replayTranscriptRead({ + messages: userMessages('now rejected', 2), + outcome: { session: null, byteOffset: 300 } + }) + + expect(index.db.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 0 + }) + expect(index.db.prepare('SELECT count(*) AS n FROM messages').get()).toEqual({ + n: 0 + }) + // The file is still read through, so a later scan does not re-read it. + expect(cursor()).toBe(300) +}) + +it('writes nothing for a source whose parser cannot reach the channel', async () => { + // An OpenCode SQLite candidate decodes in a worker, so every read of it is + // incomplete, and no re-read would help. + const candidate = { + ...syntheticCandidate({ path: '/opencode/opencode.db#session-1' }), + agent: 'opencode' as const + } + replayTranscriptRead({ + candidate, + messages: [], + outcome: { byteOffset: 0, incomplete: true } + }) + + expect(index.db.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 0 + }) + expect(store.takeStale()).toEqual([]) +}) + +it('ignores a candidate older than the retention cutoff', async () => { + store.setRetentionCutoffMs(Date.now()) + replayTranscriptRead({ messages: userMessages('too old', 3) }) + + expect(index.db.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 0 + }) + expect(store.takeStale()).toEqual([]) +}) + +it('stops writing while the store refuses writes, but remembers what it skipped', async () => { + store.setAcceptingWrites(false) + replayTranscriptRead({ messages: userMessages('paused', 3) }) + + expect(index.db.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 0 + }) + expect(errors).toEqual([]) + // A pause is exactly the window in which every read is declined. Forgetting + // them would leave the whole paused span unindexed with nothing to replay it. + expect(store.takeStale().map((candidate) => candidate.file.path)).toEqual([SYNTHETIC_TRANSCRIPT]) +}) + +it('keeps the paused re-read set when the retention window is reconfigured', async () => { + store.setAcceptingWrites(false) + replayTranscriptRead({ messages: userMessages('paused', 2) }) + expect(store.pendingFileCount).toBe(1) + + // The set records what still has to be read, not what is worth keeping. A + // window that now excludes this file is enforced where the re-read is + // dispatched, so nothing is written and the file leaves the set there. + store.setRetentionCutoffMs(Date.now()) + expect(store.pendingFileCount).toBe(1) + + store.setAcceptingWrites(true) + expect(store.takeStale()).toHaveLength(1) + replayTranscriptRead({ messages: userMessages('outside the window now', 2) }) + expect(index.db.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 0 + }) + expect(store.pendingFileCount).toBe(0) +}) + +it('drops the oldest record rather than growing without a bound, and says so', () => { + store.setAcceptingWrites(false) + for (let index = 0; index < STALE_PATH_LIMIT + 5; index++) { + store.markStale(syntheticCandidate({ path: `/transcript-${index}.jsonl` })) + } + + expect(store.pendingFileCount).toBe(STALE_PATH_LIMIT) + expect(store.droppedPendingFileCount).toBe(5) + const kept = store.takeStale().map((candidate) => candidate.file.path) + expect(kept).not.toContain('/transcript-0.jsonl') + expect(kept).toContain(`/transcript-${STALE_PATH_LIMIT + 4}.jsonl`) +}) + +it('keeps the session list running when the index write fails', async () => { + replayTranscriptRead({ + messages: userMessages('healthy', 2), + outcome: { byteOffset: 100 } + }) + index.db.exec('DROP TABLE messages_fts') + + expect(() => + replayTranscriptRead({ + mode: 'append', + previousByteOffset: 100, + messages: userMessages('broken', 400), + outcome: { byteOffset: 500 } + }) + ).not.toThrow() + expect(errors.length).toBeGreaterThan(0) + expect(store.takeStale()).toHaveLength(1) +}) + +it('unregisters cleanly, leaving later reads unindexed', async () => { + resetTranscriptConsumersForTests() + replayTranscriptRead({ messages: userMessages('after unregister', 3) }) + + expect(index.db.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 0 + }) +}) + +it('drops a removed source and keeps its cursor gone', async () => { + replayTranscriptRead({ + messages: userMessages('present', 3), + outcome: { byteOffset: 100 } + }) + store.removeFile(SYNTHETIC_TRANSCRIPT) + + expect(cursor()).toBeUndefined() + expect(index.db.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 0 + }) + expect(index.db.prepare('SELECT count(*) AS n FROM messages').get()).toEqual({ + n: 0 + }) +}) + +it('writes the session metadata the read decoded', async () => { + replayTranscriptRead({ + messages: userMessages('metadata', 1), + outcome: { + session: syntheticSession({ + sessionId: 'abc-123', + title: 'a titled session', + cwd: '/repo/app', + branch: 'main', + messageCount: 1, + resumeCommand: 'claude --resume abc-123' + }), + byteOffset: 42 + } + }) + + expect( + index.db + .prepare('SELECT session_id, title, cwd, cwd_key, branch, resume_command FROM sessions') + .get() + ).toEqual({ + session_id: 'abc-123', + title: 'a titled session', + cwd: '/repo/app', + cwd_key: '/repo/app', + branch: 'main', + resume_command: 'claude --resume abc-123' + }) +}) + +it('keeps a proven file identity when a later read cannot stat it', async () => { + const withIdentity = syntheticCandidate({ dev: 1, ino: 10 }) + replayTranscriptRead({ + candidate: withIdentity, + messages: userMessages('first', 2), + outcome: { byteOffset: 100 } + }) + + // A host that cannot prove identity re-reads the same file. + replayTranscriptRead({ + candidate: syntheticCandidate(), + mode: 'append', + previousByteOffset: 100, + messages: userMessages('second', 2), + outcome: { byteOffset: 200 } + }) + expect(indexedMessages()).toBe(4) + + // The stored identity survived, so a rename-replace is still detectable. + replayTranscriptRead({ + candidate: syntheticCandidate({ dev: 1, ino: 99 }), + mode: 'append', + previousByteOffset: 200, + messages: userMessages('replacement', 2), + outcome: { byteOffset: 300 } + }) + + expect(indexedMessages()).toBe(4) + expect(cursor()).toBe(200) + expect(store.takeStale()).toHaveLength(1) +}) diff --git a/src/main/ai-vault-search/session-search-index-consumer.ts b/src/main/ai-vault-search/session-search-index-consumer.ts new file mode 100644 index 00000000000..e4fa6da4a8e --- /dev/null +++ b/src/main/ai-vault-search/session-search-index-consumer.ts @@ -0,0 +1,118 @@ +import { parserPublishesMessages } from '../ai-vault/session-scanner-agent-parser' +import { + registerTranscriptConsumer, + type TranscriptConsumer, + type TranscriptMessage, + type TranscriptReadConsumer, + type TranscriptReadOutcome, + type TranscriptReadStart +} from '../ai-vault/session-transcript-consumers' +import { fileIdentity } from './session-search-file-cursor' +import type { SessionSearchFileWrite } from './session-search-index-writer' +import type { SessionSearchStore } from './session-search-store' + +/** + * The search index as a consumer of the transcript reader. + * + * It keeps its own cursor in the `files` table and never consults the parse + * cache: the two answer different questions and diverge the moment either + * declines a read. Three refusals, each of which leaves the cursor where it + * was and records the file for a later whole re-read: + * + * - `beginRead` returns null when this index's cursor is behind the offset an + * `append` continues from, or when the file's identity changed. + * - a buffering failure stops the read's rows without failing the session list. + * - an `incomplete` outcome never commits; those rows are not the whole span. + */ +export class SessionSearchIndexConsumer implements TranscriptConsumer { + constructor(private readonly store: SessionSearchStore) {} + + beginRead(start: TranscriptReadStart): TranscriptReadConsumer | null { + const { candidate } = start + if (!this.store.acceptsCandidate(candidate)) { + // A pause is a reason not to write now, not a reason to forget the read. + // `markStale` applies the retention rule itself, so a candidate that is + // out of scope rather than merely paused is still dropped here. + this.store.markStale(candidate) + return null + } + // A parser that decodes where the channel cannot reach it reports every read + // as incomplete. Declining here is not the same as being behind: no re-read + // would help, so the file is not recorded either. + if (!parserPublishesMessages(candidate)) { + return null + } + if (start.mode === 'append') { + const cursor = this.store.indexedFile(candidate.file.path, fileIdentity(candidate.file)) + if (!cursor || cursor.byteOffset !== start.previousByteOffset) { + // This index never saw the span before `previousByteOffset`; appending + // here would leave a hole no later read can fill. A null cursor is the + // file a chunked read left half written, which no offset continues. + this.store.markStale(candidate) + return null + } + } + const write = this.store.beginWrite( + candidate, + start.mode, + start.previousByteOffset, + start.identity + ) + if (!write) { + this.store.markStale(candidate) + return null + } + return new SessionSearchReadConsumer(this.store, start, write) + } +} + +class SessionSearchReadConsumer implements TranscriptReadConsumer { + private failed = false + + constructor( + private readonly store: SessionSearchStore, + private readonly start: TranscriptReadStart, + private readonly write: SessionSearchFileWrite + ) {} + + message(message: TranscriptMessage): void { + if (this.failed) { + return + } + try { + this.write.add(message) + } catch (error) { + // Never throws back into the reader: the channel would drop this consumer + // for the rest of the read and `finish` would never run. Failing here + // keeps the whole read on one path — the buffer is dropped and the file is + // re-read. + this.failed = true + this.store.reportWriteFailure(error) + } + } + + finish(outcome: TranscriptReadOutcome): void { + const { candidate } = this.start + let committed = false + try { + // An incomplete read's rows are not the whole span, so the cursor must not + // move past them; the file is re-read whole instead. + committed = !this.failed && !outcome.incomplete && this.write.commit(outcome) + } catch (error) { + this.store.reportWriteFailure(error) + } + if (committed) { + this.store.writeCommitted(candidate) + return + } + this.store.markStale(candidate) + } +} + +/** + * Registers the index with the reader and returns the unregister function. + * Nothing in production calls this yet: PR 3 owns when the index is live. + */ +export function registerSessionSearchIndexConsumer(store: SessionSearchStore): () => void { + return registerTranscriptConsumer(new SessionSearchIndexConsumer(store)) +} diff --git a/src/main/ai-vault-search/session-search-index-test-fixture.ts b/src/main/ai-vault-search/session-search-index-test-fixture.ts new file mode 100644 index 00000000000..baa3e2976fe --- /dev/null +++ b/src/main/ai-vault-search/session-search-index-test-fixture.ts @@ -0,0 +1,124 @@ +import { mkdtemp } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { removeTree } from '../../shared/windows-transient-lock-removal' +import type { AiVaultSession } from '../../shared/ai-vault-types' +import type { SessionFileCandidate } from '../ai-vault/session-scanner-types' +import { TranscriptMessageChannel } from '../ai-vault/session-transcript-channel' +import type { + TranscriptMessage, + TranscriptReadOutcome, + TranscriptReadStart +} from '../ai-vault/session-transcript-consumers' +import type SyncDatabase from '../sqlite/sync-database' +import { openSessionSearchDatabase } from './session-search-schema' + +export const SYNTHETIC_TRANSCRIPT = 'synthetic-transcript' + +export function syntheticCandidate( + overrides: Partial = {} +): SessionFileCandidate { + const at = new Date(1740000000000) + return { + agent: 'claude', + codexHome: null, + file: { + path: SYNTHETIC_TRANSCRIPT, + mtimeMs: at.getTime(), + modifiedAt: at.toISOString(), + sizeBytes: 4096, + ...overrides + } + } +} + +export function syntheticSession(overrides: Partial = {}): AiVaultSession { + const at = new Date(1740000000000).toISOString() + return { + id: 'fixture', + executionHostId: 'local', + agent: 'claude', + sessionId: 'fixture', + title: 'fixture session', + cwd: '/fixture', + branch: null, + model: null, + filePath: SYNTHETIC_TRANSCRIPT, + codexHome: null, + createdAt: at, + updatedAt: at, + modifiedAt: at, + messageCount: 0, + totalTokens: 0, + previewMessages: [], + queuedMessageCount: 0, + subagentTranscriptCount: 0, + resumeCommand: '', + subagent: null, + ...overrides + } +} + +export function userMessages(text: string, count: number): TranscriptMessage[] { + return Array.from({ length: count }, (_unused, index) => ({ + role: 'user' as const, + text, + timestamp: new Date(1740000000000 + index * 1000).toISOString() + })) +} + +/** + * Drives one read through the real fan-out channel, so a test exercises the + * registration path the transcript reader uses rather than the consumer alone. + */ +export function replayTranscriptRead(args: { + candidate?: SessionFileCandidate + mode?: TranscriptReadStart['mode'] + previousByteOffset?: number + messages: TranscriptMessage[] + outcome?: Partial +}): void { + const candidate = args.candidate ?? syntheticCandidate() + const mode = args.mode ?? 'replace' + const channel = new TranscriptMessageChannel() + channel.beginRead({ + candidate, + mode, + previousByteOffset: args.previousByteOffset ?? 0 + }) + for (const message of args.messages) { + channel.push(message) + } + channel.finishRead({ + session: syntheticSession(), + byteOffset: 4096, + incomplete: false, + ...args.outcome + }) +} + +export type SessionSearchIndexFile = { + path: string + /** The store keeps its own connection private, so row assertions need this one. */ + db: SyncDatabase + close: () => Promise +} + +/** An on-disk index: `:memory:` is per-connection, so a second reader needs a real file. */ +export async function openSessionSearchIndexFile(name: string): Promise { + const root = await mkdtemp(join(tmpdir(), `${name}-`)) + const path = join(root, 'index.sqlite') + const db = openSessionSearchDatabase(path) + let open = true + return { + path, + db, + close: async () => { + if (open) { + open = false + db.close() + } + await removeTree(root) + } + } +} diff --git a/src/main/ai-vault-search/session-search-index-writer.test.ts b/src/main/ai-vault-search/session-search-index-writer.test.ts new file mode 100644 index 00000000000..46d147dcce0 --- /dev/null +++ b/src/main/ai-vault-search/session-search-index-writer.test.ts @@ -0,0 +1,228 @@ +import { afterEach, beforeEach, expect, it } from 'vitest' +import { SessionSearchIndexConsumer } from './session-search-index-consumer' +import { + openSessionSearchIndexFile, + syntheticCandidate, + syntheticSession, + SYNTHETIC_TRANSCRIPT, + userMessages, + type SessionSearchIndexFile +} from './session-search-index-test-fixture' +import { SessionSearchStore } from './session-search-store' + +// The store is driven directly here. Every guard below is also shadowed by the +// consumer's own check, so a test that goes through the consumer proves nothing +// about which of the two is holding. + +let index: SessionSearchIndexFile +let store: SessionSearchStore +let errors: unknown[] + +beforeEach(async () => { + index = await openSessionSearchIndexFile('ss-index-writer') + errors = [] + store = new SessionSearchStore(index.path, (error) => errors.push(error)) +}) + +afterEach(async () => { + store.close() + await index.close() +}) + +function count(table: string): number { + return ( + index.db.prepare(`SELECT count(*) AS n FROM ${table}`).get() as { + n: number + } + ).n +} + +function indexRead(previousByteOffset: number, byteOffset: number, text: string): boolean { + const write = store.beginWrite( + syntheticCandidate(), + previousByteOffset === 0 ? 'replace' : 'append', + previousByteOffset + ) + if (!write) { + return false + } + for (const message of userMessages(text, 2)) { + write.add(message) + } + return write.commit({ + session: syntheticSession(), + byteOffset, + incomplete: false + }) +} + +it('refuses an append whose predecessor offset is not the committed cursor', () => { + expect(indexRead(0, 100, 'first')).toBe(true) + + expect(store.beginWrite(syntheticCandidate(), 'append', 900)).toBeNull() + expect(store.beginWrite(syntheticCandidate(), 'append', 99)).toBeNull() + // The one offset that does continue the committed span is accepted. + expect(store.beginWrite(syntheticCandidate(), 'append', 100)).not.toBeNull() +}) + +it('refuses to commit a write whose cursor moved underneath it', () => { + const stale = store.beginWrite(syntheticCandidate(), 'replace', 0)! + for (const message of userMessages('stalegeneration', 40)) { + stale.add(message) + } + // A second read of the same path finishes first. Without the parse file lane + // this is the overlap that would otherwise resurrect the stale rows. + expect(indexRead(0, 200, 'winninggeneration')).toBe(true) + + expect( + stale.commit({ + session: syntheticSession(), + byteOffset: 100, + incomplete: false + }) + ).toBe(false) + expect(store.indexedFile(SYNTHETIC_TRANSCRIPT, null)?.byteOffset).toBe(200) + expect(count('sessions')).toBe(1) + expect(count('messages')).toBe(2) + expect(errors).toEqual([]) +}) + +it('refuses to commit a write whose file was removed mid-read', () => { + expect(indexRead(0, 100, 'firstgeneration')).toBe(true) + const write = store.beginWrite(syntheticCandidate(), 'append', 100)! + for (const message of userMessages('afterremoval', 10)) { + write.add(message) + } + store.removeFile(SYNTHETIC_TRANSCRIPT) + + // Committing here would put a source back that its owner proved was deleted. + expect( + write.commit({ + session: syntheticSession(), + byteOffset: 300, + incomplete: false + }) + ).toBe(false) + expect(store.indexedFile(SYNTHETIC_TRANSCRIPT, null)).toBeNull() + expect(count('sessions')).toBe(0) + expect(count('messages')).toBe(0) + expect(count('files')).toBe(0) +}) + +it('declines a behind cursor in beginRead before it ever reaches the store', () => { + const attempted: number[] = [] + const stub = { + acceptsCandidate: () => true, + indexedFile: () => ({ byteOffset: 100, mtimeMs: 1, sizeBytes: 1 }), + beginWrite: (_candidate: unknown, _mode: unknown, previousByteOffset: number) => { + attempted.push(previousByteOffset) + return { add: () => undefined, commit: () => true } + }, + markStale: () => undefined + } as unknown as SessionSearchStore + const consumer = new SessionSearchIndexConsumer(stub) + + expect( + consumer.beginRead({ + candidate: syntheticCandidate(), + mode: 'append', + previousByteOffset: 900 + }) + ).toBeNull() + // The store was never asked, so the writer's own guard cannot be what refused. + expect(attempted).toEqual([]) + expect( + consumer.beginRead({ + candidate: syntheticCandidate(), + mode: 'append', + previousByteOffset: 100 + }) + ).not.toBeNull() + expect(attempted).toEqual([100]) +}) + +it("hands the read's identity accessor to the store", () => { + const captured: unknown[] = [] + const stub = { + acceptsCandidate: () => true, + indexedFile: () => null, + beginWrite: ( + _candidate: unknown, + _mode: unknown, + _previousByteOffset: unknown, + identity: unknown + ) => { + captured.push(identity) + return { add: () => undefined, commit: () => true } + }, + markStale: () => undefined + } as unknown as SessionSearchStore + const identity = (): null => null + + new SessionSearchIndexConsumer(stub).beginRead({ + candidate: syntheticCandidate(), + mode: 'replace', + previousByteOffset: 0, + identity + }) + + // Dropped here, a chunked read writes rows under a session with no id and no + // cwd for as long as the read lasts, and for ever if it crashes first. + expect(captured).toEqual([identity]) +}) + +it('treats half a recorded identity as no identity at all', () => { + // New partial observations are not stored as identities. + const partial = { + ...syntheticCandidate({ dev: 7 }), + agent: 'claude' as const + } + const write = store.beginWrite(partial, 'replace', 0)! + for (const message of userMessages('halfidentity', 2)) { + write.add(message) + } + write.commit({ + session: syntheticSession(), + byteOffset: 100, + incomplete: false + }) + expect(index.db.prepare('SELECT dev, ino FROM files').get()).toEqual({ + dev: null, + ino: null + }) + // Older indexes may still carry a half-pair. + index.db.exec('UPDATE files SET dev = 7') + + // One matching number is not proof of sameness, and one mismatching number is + // not proof of replacement. Neither compares, so neither declines. + expect(store.indexedFile(SYNTHETIC_TRANSCRIPT, { dev: 7, ino: 99 })?.byteOffset).toBe(100) + expect(store.indexedFile(SYNTHETIC_TRANSCRIPT, { dev: 8, ino: 99 })?.byteOffset).toBe(100) + expect(store.beginWrite(syntheticCandidate({ dev: 8, ino: 99 }), 'append', 100)).not.toBeNull() +}) + +it.each([ + [null, { dev: null, ino: null }], + [ + { dev: 7, ino: 11 }, + { dev: 7, ino: 11 } + ] +])('never combines partial stats with the previous identity %j', (initial, expected) => { + const observations = [initial ?? {}, { dev: 9 }, { ino: 13 }, { dev: 17, ino: 19 }] + for (const [position, identity] of observations.entries()) { + const write = store.beginWrite( + syntheticCandidate(identity), + position ? 'append' : 'replace', + position * 100 + )! + expect( + write.commit({ + session: syntheticSession(), + byteOffset: (position + 1) * 100, + incomplete: false + }) + ).toBe(true) + expect(index.db.prepare('SELECT dev, ino FROM files').get()).toEqual( + position === 3 ? { dev: 17, ino: 19 } : expected + ) + } +}) diff --git a/src/main/ai-vault-search/session-search-index-writer.ts b/src/main/ai-vault-search/session-search-index-writer.ts new file mode 100644 index 00000000000..a5c981b5bb2 --- /dev/null +++ b/src/main/ai-vault-search/session-search-index-writer.ts @@ -0,0 +1,359 @@ +import type SyncDatabase from '../sqlite/sync-database' +import type { SessionFileCandidate } from '../ai-vault/session-scanner-types' +import type { + TranscriptMessage, + TranscriptReadOutcome, + TranscriptSessionIdentity +} from '../ai-vault/session-transcript-consumers' +import { EMPTY_CONTENT_HASH, foldContentHash } from './session-search-content-hash' +import type { + SessionSearchFileIdentity, + SessionSearchIndexedFile +} from './session-search-file-cursor' +import { SessionSearchFileRecords } from './session-search-file-records' +import { + deleteSearchMessages, + insertSearchMessage, + searchMessageRows +} from './session-search-message-rows' + +/** + * How much decoded text one transaction may carry. + * + * A file's rows are buffered in memory and written in one transaction, so the + * whole read is either in the index or not. The ceiling is what keeps that + * promise affordable: at the measured 26 MB of transcript per second it caps a + * single commit near a second and the WAL it produces near 64 MB, and it is far + * above the largest real transcript (the 40-session benchmark corpus is 10.5 MB + * in total), so an ordinary file never reaches it. Above the ceiling the read is + * cut into chunks that each leave the index consistent — but only a read that + * can name its session chunks at all. See `add`. + */ +export const SESSION_SEARCH_COMMIT_CHARS = 32 * 1024 * 1024 + +/** + * The cursor of a file whose rows are a prefix, written by a chunk of a read + * that has not reached the end of the file. + * + * The reader hands out byte offsets only when a read finishes, so a chunk has + * no honest offset to record. This one is unusable on purpose: `indexedFile` + * reports no cursor for it, so an append is declined and the file is re-read + * whole. The rows are still a coherent prefix of that session and answer + * searches until the re-read replaces them. + */ +const PARTIAL_FILE_CURSOR = -1 + +type FileRow = { + dev: number | null + ino: number | null + byte_offset: number + mtime_ms: number + size_bytes: number | null + session_row_id: number | null +} + +type FileCursor = Pick + +export type SessionSearchFileWrite = { + /** + * Buffers one message, committing a chunk when the buffer reaches the ceiling + * — and only while this read can name the session it is writing. + * + * A chunk's rows answer searches the moment they land, so a read with no + * `identity` would publish them under a session with an empty id, an empty + * title and a null cwd, and an interrupted read would leave that prefix + * behind for good. The readers that supply no identity are the whole-file + * ones (Grok, Cursor, Gemini, OpenCode), whose formats are rewritten in place + * and have no resumable state to ask; they are also small — the largest on + * the author's machine is 5 MB — so buffering one to the end and committing + * it whole costs nothing. Chunking stays reserved for the readers that can + * say which session this is before the read ends. + */ + add(message: TranscriptMessage): void + /** + * Writes this file's rows, its session and its cursor in one transaction. + * False when the file's record changed under this read — it was removed, or + * another writer moved the cursor these rows continue from. A read that never + * calls this leaves the index exactly as it found it, unless it chunked. + */ + commit(outcome: TranscriptReadOutcome): boolean +} + +export class SessionSearchIndexWriter { + private readonly records: SessionSearchFileRecords + // Removals per path, so a write can prove its source was not dropped under it + // rather than infer it from the cursor. In memory is enough: one process owns + // the index, and a removal only has to fence writes this process opened. + private readonly removals = new Map() + + constructor( + private readonly db: SyncDatabase, + private readonly commitChars: number = SESSION_SEARCH_COMMIT_CHARS, + /** + * Called after a transaction that left a session's messages with no session + * row, so the owner can start the bounded drain that reclaims them. + * Synchronous work here would put the cost back where it was taken from. + */ + private readonly onOrphanedRows: () => void = () => undefined + ) { + this.records = new SessionSearchFileRecords(db) + } + + /** + * What the index holds for this file, or null when it holds nothing usable: + * an unknown path, or one whose recorded identity no longer matches. + * + * A file a chunked read left half written is reported, with a null cursor. + * Reporting nothing for it would read as "never indexed", so the caller would + * ask for whatever read the parse cache offers, the reader would pick append, + * and the decline would be the only thing that ever forced the whole read. + */ + indexedFile(path: string, identity: SessionSearchFileIdentity): SessionSearchIndexedFile | null { + const row = this.db + .prepare( + 'SELECT dev, ino, byte_offset, mtime_ms, size_bytes, session_row_id FROM files WHERE path = ?' + ) + .get(path) as FileRow | undefined + if (!row) { + return null + } + // Older indexes can carry half-pairs; only a complete identity can prove replacement. + if (identity && row.dev !== null && row.ino !== null) { + if (row.dev !== identity.dev || row.ino !== identity.ino) { + return null + } + } + return { + byteOffset: row.byte_offset === PARTIAL_FILE_CURSOR ? null : row.byte_offset, + mtimeMs: row.mtime_ms, + sizeBytes: row.size_bytes + } + } + + /** + * Opens a buffered write for one read, or returns null when the read cannot + * extend what the index holds: an `append` whose predecessor byte offset is + * not this index's own cursor covers a span the index never saw. + */ + beginWrite( + candidate: SessionFileCandidate, + mode: 'replace' | 'append', + previousByteOffset: number, + identity?: () => TranscriptSessionIdentity | null + ): SessionSearchFileWrite | null { + const path = candidate.file.path + const cursor = this.cursor(path) + if (mode === 'append') { + // The partial sentinel is not a byte offset, so nothing continues it — + // including a caller that reads it back off the row and passes it in. + if (cursor === undefined || cursor.byte_offset === PARTIAL_FILE_CURSOR) { + return null + } + if (cursor.byte_offset !== previousByteOffset) { + return null + } + } + // A file the index read through and decoded no session from still has a + // cursor worth continuing: it has no session row to hang new rows off, so + // this read makes one. Declining instead would force a whole re-read of + // that file on every pass for as long as it grows. + return this.buffered(candidate, cursor, mode === 'append', identity) + } + + /** + * Drops a source: its session, its rows and its file record, in one + * transaction. Unbounded on purpose — the caller has proven this one file is + * gone and expects it out of results when the call returns, and a read of it + * that is still in flight is fenced by the cursor its commit re-reads. + */ + removeFile(path: string): void { + this.removals.set(path, (this.removals.get(path) ?? 0) + 1) + const cursor = this.cursor(path) + this.db.exec('BEGIN IMMEDIATE') + try { + this.dropSession(cursor?.session_row_id ?? null) + this.db.prepare('DELETE FROM files WHERE path = ?').run(path) + this.db.exec('COMMIT') + } catch (error) { + this.db.exec('ROLLBACK') + throw error + } + } + + private cursor(path: string): FileCursor | undefined { + return this.db + .prepare('SELECT session_row_id,byte_offset FROM files WHERE path = ?') + .get(path) as FileCursor | undefined + } + + private buffered( + candidate: SessionFileCandidate, + opened: FileCursor | undefined, + append: boolean, + identity?: () => TranscriptSessionIdentity | null + ): SessionSearchFileWrite { + const db = this.db + const path = candidate.file.path + const buffer: TranscriptMessage[] = [] + let bufferedChars = 0 + // What this write believes the file record holds. Re-read inside every + // transaction: a `removeFile` or another writer between two chunks means + // these rows no longer continue anything, and committing on top of that + // would resurrect a deleted source or duplicate a span. + let expected = opened + const removalsAtStart = this.removals.get(path) ?? 0 + // The session row is reused across re-reads of one file, so a `replace` + // swaps a session's rows rather than minting a second generation of it. + let session = opened?.session_row_id ?? null + let hash = append && session !== null ? this.records.contentHash(session) : EMPTY_CONTENT_HASH + // A replace owns the session's whole row set, so the old generation goes in + // the same transaction as the first of the new one. Chunk two onwards must + // not repeat it. + // + // It goes by being cut loose, not by being deleted. Deleting every old row + // inline sizes the transaction by the session being replaced rather than by + // the chunk being written: 1,286 ms against 720 ms fresh on the 100 MB + // corpus, and it grows with the history. Instead the first transaction + // mints a new session row, points `files` at it and deletes the one old + // `sessions` row. Every retrieval joins `sessions`, so the old generation + // stops answering the moment that commits, and its messages are reclaimed + // afterwards by the same bounded drain retention uses — which is where the + // old rows would have ended up had the process died here anyway. + // `sessions.id` is AUTOINCREMENT, so the freed id is never handed to + // another session while those rows still name it (round 8). + let replaced = append + // Set by the transaction that cut a generation loose; read once it commits. + let orphaned = false + // Set when the file record moved under this read. Nothing this write holds + // can land after that, so it stops buffering rather than reopening a + // transaction it already knows will roll back, once per remaining message. + let fenced = false + + // Why a counter and not the cursor alone: on a path this index never wrote, + // `expected` and the absent row are both undefined, so the cursor compare + // reads a removal as no change and the write recreates the source. + const current = (): boolean => { + if ((this.removals.get(path) ?? 0) !== removalsAtStart) { + return false + } + const row = this.cursor(path) + return ( + row?.session_row_id === expected?.session_row_id && + row?.byte_offset === expected?.byte_offset + ) + } + + /** + * `outcome` is null for a chunk of a read that has not reached the file's + * end, and `named` is what that chunk writes onto its session row. + */ + const write = ( + outcome: TranscriptReadOutcome | null, + named: TranscriptSessionIdentity | null + ): boolean => { + const decoded = outcome?.session ?? null + db.exec('BEGIN IMMEDIATE') + try { + if (!current()) { + db.exec('ROLLBACK') + return false + } + if (outcome && !decoded) { + // Read through, but nothing to search: the cursor advances so the file + // is not re-read whole on every pass, and whatever generation was here + // — including this read's own committed chunks — goes with it. + this.dropSession(session) + session = null + this.records.upsertFile(candidate, outcome.byteOffset, null) + } else { + if (replaced) { + session ??= this.records.createSessionRow(candidate) + } else { + const previous = session + session = this.records.createSessionRow(candidate) + if (previous !== null) { + db.prepare('DELETE FROM sessions WHERE id = ?').run(previous) + orphaned = true + } + replaced = true + } + for (const row of buffer) { + insertSearchMessage(db, session, row) + } + if (decoded) { + this.records.updateSession(decoded, session, hash) + } else if (named) { + // A chunk's rows answer searches as soon as they land, so the + // session they hang off is written with whatever the parser has + // decoded rather than left empty until a read that may never end. + // `add` refuses to chunk without this, so it is never absent here. + this.records.updateProvisionalSession(session, named) + } + this.records.upsertFile( + candidate, + outcome ? outcome.byteOffset : PARTIAL_FILE_CURSOR, + session + ) + } + db.exec('COMMIT') + } catch (error) { + db.exec('ROLLBACK') + throw error + } + // After the transaction that cut them loose is durable, never before: a + // rollback leaves the old session row standing and nothing to reclaim. + if (orphaned) { + orphaned = false + this.onOrphanedRows() + } + expected = { + session_row_id: session, + byte_offset: outcome ? outcome.byteOffset : PARTIAL_FILE_CURSOR + } + buffer.length = 0 + bufferedChars = 0 + return true + } + + return { + add: (message) => { + if (fenced) { + return + } + hash = foldContentHash(hash, [message]) + // The ceiling is checked per row, not per message: one message is a whole + // conversation turn and may be megabytes, so checking it after the whole + // message had been buffered let a single one carry a transaction as far + // past the ceiling as it was large. + for (const row of searchMessageRows([message])) { + buffer.push(row) + bufferedChars += row.text.length + if (bufferedChars < this.commitChars) { + continue + } + // Publishing a chunk under a session nothing can identify is worse + // than holding the buffer: the rows answer searches at once, and an + // interrupted read leaves that prefix for good. A read with nothing + // to name it keeps buffering and commits whole at `finish`. + const named = identity?.() ?? null + if (named && !write(null, named)) { + fenced = true + buffer.length = 0 + bufferedChars = 0 + return + } + } + }, + commit: (outcome) => !fenced && write(outcome, null) + } + } + + /** Caller's transaction: drops a session and every row that hangs off it. */ + private dropSession(sessionRowId: number | null): void { + if (sessionRowId === null) { + return + } + deleteSearchMessages(this.db, sessionRowId) + this.db.prepare('DELETE FROM sessions WHERE id = ?').run(sessionRowId) + } +} diff --git a/src/main/ai-vault-search/session-search-live-transcript.test.ts b/src/main/ai-vault-search/session-search-live-transcript.test.ts new file mode 100644 index 00000000000..7f37ca662cb --- /dev/null +++ b/src/main/ai-vault-search/session-search-live-transcript.test.ts @@ -0,0 +1,208 @@ +import { mkdtemp, rm, writeFile, appendFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, expect, it } from 'vitest' +import { resetSessionParseCacheForTests } from '../ai-vault/session-scanner-parse-cache' +import { + registerTranscriptConsumer, + resetTranscriptConsumersForTests, + type TranscriptSessionIdentity +} from '../ai-vault/session-transcript-consumers' +import { requestWholeTranscriptRead } from '../ai-vault/session-transcript-reader' +import SyncDatabase from '../sqlite/sync-database' +import { registerSessionSearchIndexConsumer } from './session-search-index-consumer' +import { SessionSearchStore } from './session-search-store' +import { + assistantRecord, + CLAUDE_SESSION_ID as SESSION_ID, + CODEX_ROLLOUT_FILE, + CODEX_SESSION_ID, + codexRolloutLines, + parseTranscript, + userRecord +} from './session-search-transcript-fixtures' + +let tempRoots: string[] = [] +let store: SessionSearchStore +// The store keeps its connection private, so row assertions need a second one. +let reader: SyncDatabase +let errors: unknown[] + +beforeEach(async () => { + resetSessionParseCacheForTests() + resetTranscriptConsumersForTests() + errors = [] + const path = join(await makeTempDir(), 'index.sqlite') + store = new SessionSearchStore(path, (error) => errors.push(error)) + registerSessionSearchIndexConsumer(store) + reader = new SyncDatabase(path, { readonly: true }) +}) + +afterEach(async () => { + resetTranscriptConsumersForTests() + reader.close() + store.close() + await Promise.all(tempRoots.map((root) => rm(root, { recursive: true, force: true }))) + tempRoots = [] +}) + +async function makeTempDir(): Promise { + const root = await mkdtemp(join(tmpdir(), 'orca-session-search-live-')) + tempRoots.push(root) + return root +} + +/** Sessions a query would return for one FTS term, read on a second handle. */ +function sessionsMatching(term: string): string[] { + return ( + reader + .prepare( + `SELECT DISTINCT s.session_id AS id FROM messages_fts + JOIN messages m ON m.id = messages_fts.rowid + JOIN sessions s ON s.id = m.session_row_id + WHERE messages_fts MATCH ? ORDER BY s.session_id` + ) + .all(term) as { id: string }[] + ).map((row) => row.id) +} + +it('indexes a Claude transcript through the reader and resumes on append', async () => { + const root = await makeTempDir() + const path = join(root, `${SESSION_ID}.jsonl`) + await writeFile( + path, + `${[ + userRecord(0, 'find the flaky terminal reattach'), + assistantRecord(1, 'look at resolveTerminalPath first') + ].join('\n')}\n` + ) + await parseTranscript(path) + expect(errors).toEqual([]) + expect(sessionsMatching('reattach')).toEqual([SESSION_ID]) + // The identifier column shadows a camel-case symbol into its pieces. + expect(sessionsMatching('terminal')).toEqual([SESSION_ID]) + + await appendFile(path, `${assistantRecord(2, 'the zygomorphic follow-up landed')}\n`) + const resumed = await parseTranscript(path) + // The reader resumed, so the index saw an `append`, not a whole re-read. + expect(resumed.stats).toMatchObject({ incremental: 1, fullParses: 0 }) + expect(errors).toEqual([]) + expect(sessionsMatching('zygomorphic')).toEqual([SESSION_ID]) + // An append extends one session rather than creating a second. + expect(reader.prepare('SELECT count(*) AS n FROM sessions').get()).toEqual({ + n: 1 + }) +}) + +it('keeps a tool result searchable but out of the conversation half', async () => { + const root = await makeTempDir() + const codexHome = await makeTempDir() + const path = join(root, CODEX_ROLLOUT_FILE) + await writeFile( + path, + `${codexRolloutLines( + ['rg', 'pericardium'], + `outputonly ${'padding '.repeat(600)}tailonly`, + 'promptonly search for the module' + ).join('\n')}\n` + ) + await parseTranscript(path, 'codex', codexHome) + expect(errors).toEqual([]) + + expect(sessionsMatching('pericardium')).toHaveLength(1) + // The prompt is conversation; the command output is not, and the column + // filter is what tells them apart. + expect(sessionsMatching('outputonly')).toHaveLength(1) + expect(sessionsMatching('tailonly')).toHaveLength(0) + expect(sessionsMatching('rg')).toHaveLength(1) + expect(sessionsMatching('{user_text assistant_text}: promptonly')).toHaveLength(1) + expect(sessionsMatching('{user_text assistant_text}: outputonly')).toHaveLength(0) + expect(sessionsMatching('{user_text assistant_text}: rg')).toHaveLength(0) +}) + +/** What `start.identity()` returns at each message of one read. */ +function recordIdentityPerMessage(): (TranscriptSessionIdentity | null)[] { + const seen: (TranscriptSessionIdentity | null)[] = [] + registerTranscriptConsumer({ + beginRead: (start) => ({ + message: () => { + seen.push(start.identity?.() ?? null) + }, + finish: () => undefined + }) + }) + return seen +} + +it('names the session mid-read, before the reader has finished the file', async () => { + const root = await makeTempDir() + const path = join(root, `${SESSION_ID}.jsonl`) + await writeFile( + path, + `${[ + userRecord(0, 'find the flaky terminal reattach'), + assistantRecord(1, 'look at resolveTerminalPath first') + ].join('\n')}\n` + ) + const seen = recordIdentityPerMessage() + await parseTranscript(path) + + // A chunked read commits partway through a file this size or larger, so what + // it can name the session with is exactly this. + expect(seen.length).toBeGreaterThan(0) + expect(seen[0]).toMatchObject({ + sessionId: SESSION_ID, + cwd: '/repo/app', + createdAt: expect.any(String) + }) +}) + +it('names a Codex session mid-read from its own opening record', async () => { + const root = await makeTempDir() + const codexHome = await makeTempDir() + const path = join(root, CODEX_ROLLOUT_FILE) + await writeFile( + path, + `${codexRolloutLines(['rg', 'pericardium'], 'src/main/pericardium.ts:12: match', 'search for the pericardium module').join('\n')}\n` + ) + const seen = recordIdentityPerMessage() + await parseTranscript(path, 'codex', codexHome) + + // Codex builds its own resumable state rather than the shared accumulator + // fold, so it is the other half of the surface a chunked commit depends on. + expect(seen[0]).toMatchObject({ + sessionId: CODEX_SESSION_ID, + cwd: '/repo/app' + }) +}) + +it('indexes a file the session list already read past, once a whole read is asked for', async () => { + const root = await makeTempDir() + const path = join(root, `${SESSION_ID}.jsonl`) + await writeFile(path, `${userRecord(0, 'the opening prompt')}\n`) + + // The state on first enablement inside a running app: the session list has + // read this file, so the parse cache is warm, while the index is empty. + resetTranscriptConsumersForTests() + await parseTranscript(path) + registerSessionSearchIndexConsumer(store) + + await appendFile(path, `${assistantRecord(1, 'a zygomorphic reply')}\n`) + const appended = await parseTranscript(path) + expect(appended.stats).toMatchObject({ incremental: 1, fullParses: 0 }) + // The append continued from a byte offset the index never saw, so it declined. + expect(sessionsMatching('zygomorphic')).toEqual([]) + + const behind = store.takeStale() + expect(behind.map((candidate) => candidate.file.path)).toEqual([path]) + for (const candidate of behind) { + requestWholeTranscriptRead(candidate.file.path) + } + + const reread = await parseTranscript(path) + expect(reread.stats).toMatchObject({ incremental: 0, fullParses: 1 }) + expect(errors).toEqual([]) + expect(sessionsMatching('zygomorphic')).toEqual([SESSION_ID]) + expect(sessionsMatching('opening')).toEqual([SESSION_ID]) + expect(store.takeStale()).toEqual([]) +}) diff --git a/src/main/ai-vault-search/session-search-message-rows.test.ts b/src/main/ai-vault-search/session-search-message-rows.test.ts new file mode 100644 index 00000000000..d2cf2fbca61 --- /dev/null +++ b/src/main/ai-vault-search/session-search-message-rows.test.ts @@ -0,0 +1,249 @@ +import { expect, it } from 'vitest' +import type { TranscriptMessage } from '../ai-vault/session-transcript-consumers' +import { insertSearchMessage, searchMessageRows } from './session-search-message-rows' +import { + openSessionSearchIndexFile, + type SessionSearchIndexFile +} from './session-search-index-test-fixture' + +/** Every column of the FTS table, so an assertion cannot miss the shadow terms. */ +async function indexedColumns( + index: SessionSearchIndexFile, + message: TranscriptMessage +): Promise { + for (const row of searchMessageRows([message])) { + insertSearchMessage(index.db, 1, row) + } + const full = index.db + .prepare('SELECT user_text, assistant_text, tool_text, identifiers FROM messages_fts') + .all() as Record[] + return full.flatMap((row) => Object.values(row)) +} + +it('splits an oversized message on a line boundary and keeps every character', () => { + const line = `${'padding '.repeat(11)}word\n` + const text = line.repeat(400) + const chunks = [...searchMessageRows([{ role: 'user', text, timestamp: null }])].map( + (row) => row.text + ) + + expect(chunks.length).toBeGreaterThan(1) + expect(chunks.join('')).toBe(text) + for (const chunk of chunks) { + expect(chunk.length).toBeLessThanOrEqual(8000) + expect(chunk.endsWith('\n')).toBe(true) + } +}) + +it('cuts at whitespace rather than through the word on the boundary', async () => { + const index = await openSessionSearchIndexFile('ss-rows-whitespace') + try { + // The 8,000th character lands inside `pericardium`. Cutting at the target + // would file `per` under one row and `icardium` under another, and the word + // the user types would match neither. + const text = `${' '.repeat(7997)}pericardium` + const chunks = [...searchMessageRows([{ role: 'user', text, timestamp: null }])] + expect(chunks.map((row) => row.text).join('')).toBe(text) + for (const row of chunks) { + insertSearchMessage(index.db, 1, row) + } + + expect( + index.db + .prepare('SELECT count(*) AS n FROM messages_fts WHERE messages_fts MATCH ?') + .get('pericardium') + ).toEqual({ n: 1 }) + } finally { + await index.close() + } +}) + +it.each(['/repo/pericardium.ts', 'PROJ-12345', 'C++', 'cafe\u0301ine'])( + 'preserves the exact FTS token %s at a chunk boundary', + async (token) => { + const index = await openSessionSearchIndexFile('ss-rows-tokenchars') + try { + const text = ' '.repeat(7998) + token + const chunks = [...searchMessageRows([{ role: 'user', text, timestamp: null }])] + expect(chunks.map((row) => row.text).join('')).toBe(text) + for (const row of chunks) { + insertSearchMessage(index.db, 1, row) + } + expect( + index.db + .prepare('SELECT count(*) AS n FROM messages_fts WHERE messages_fts MATCH ?') + .get(`"${token}"`) + ).toEqual({ n: 1 }) + } finally { + await index.close() + } + } +) + +it.each(['\u0305', '\u030d', '\u0332'])( + 'cuts at a combining mark unicode61 treats as a separator: %s', + async (mark) => { + const index = await openSessionSearchIndexFile('ss-rows-unicode-separator') + try { + const text = `${'x'.repeat(7997)}${mark}pericardium` + for (const row of searchMessageRows([{ role: 'user', text, timestamp: null }])) { + insertSearchMessage(index.db, 1, row) + } + expect( + index.db + .prepare("SELECT count(*) AS n FROM messages_fts WHERE messages_fts MATCH 'pericardium'") + .get() + ).toEqual({ n: 1 }) + } finally { + await index.close() + } + } +) + +it('backs up to any whitespace, not only a newline', () => { + // An ideographic space separates words in a CJK transcript exactly as a + // space does here, and a newline-only backoff tears the token after it. + const text = `${'\u4e00'.repeat(7000)}\u3000${'\u4e8c'.repeat(2000)}` + const chunks = [...searchMessageRows([{ role: 'user', text, timestamp: null }])].map( + (row) => row.text + ) + + expect(chunks[0]).toBe(`${'\u4e00'.repeat(7000)}\u3000`) + expect(chunks.join('')).toBe(text) +}) + +it('cuts at punctuation when the window holds no whitespace at all', async () => { + const index = await openSessionSearchIndexFile('ss-rows-minified') + try { + // Valid minified JSON, the shape a tool result carries: 8,000 characters + // without a single space. The 8,000th lands inside `pericardium`, and a + // whitespace-only backoff has nothing in the window to back up to, so it + // files `perica` under one row and `rdium` under the next. + const text = `{"pad":"${'x'.repeat(7976)}","note":"pericardium"}` + expect(JSON.parse(text)).toEqual({ pad: 'x'.repeat(7976), note: 'pericardium' }) + expect(text.slice(7994, 8005)).toBe('pericardium') + expect(/\s/.test(text)).toBe(false) + + const chunks = [...searchMessageRows([{ role: 'user', text, timestamp: null }])] + expect(chunks.map((row) => row.text).join('')).toBe(text) + for (const row of chunks) { + insertSearchMessage(index.db, 1, row) + } + + expect( + index.db + .prepare('SELECT count(*) AS n FROM messages_fts WHERE messages_fts MATCH ?') + .get('pericardium') + ).toEqual({ n: 1 }) + } finally { + await index.close() + } +}) + +it('keeps a 9,000-character identifier whole rather than cutting at its underscores', () => { + // `_` sits inside a token for this tokenizer, so it is not a boundary. A + // snake_case name that long holds none at all, and the target itself is the + // honest cut — backing up to every `_` would file the name in pieces. + const text = 'ab_'.repeat(3000) + expect(text.length).toBe(9000) + const chunks = [...searchMessageRows([{ role: 'user', text, timestamp: null }])].map( + (row) => row.text + ) + + expect(chunks.map((chunk) => chunk.length)).toEqual([8000, 1000]) + expect(chunks.join('')).toBe(text) +}) + +it('still chunks a message that holds no whitespace at all', () => { + // A 20,000-character token is not a word, so the target itself is the cut and + // the message is still bounded. + const chunks = [ + ...searchMessageRows([{ role: 'user', text: 'a'.repeat(20_000), timestamp: null }]) + ] + expect(chunks.map((row) => row.text.length)).toEqual([8000, 8000, 4000]) +}) + +it('leaves a message that fits as a single row', () => { + const rows = [...searchMessageRows([{ role: 'user', text: 'short enough', timestamp: null }])] + expect(rows.map((row) => row.text)).toEqual(['short enough']) +}) + +it('caps a tool row at its head and never caps the conversation', async () => { + const index = await openSessionSearchIndexFile('ss-rows-tool-cap') + try { + // The reader hands over untruncated text (its own bound is 256 KB per + // message and a consumer may be handed more); the cap is this module's. + const output = `pericardium ${'padding '.repeat(140_000)}` + expect(output.length).toBeGreaterThan(1024 * 1024) + + const toolRows = [...searchMessageRows([{ role: 'tool', text: output, timestamp: null }])] + expect(toolRows).toHaveLength(1) + expect(toolRows[0]!.text.length).toBe(3072) + // The head is what identifies what ran, so it is what survives. + expect(toolRows[0]!.text.startsWith('pericardium ')).toBe(true) + + // The same text as an assistant turn is conversation, and keeps every byte. + const assistantRows = [ + ...searchMessageRows([{ role: 'assistant', text: output, timestamp: null }]) + ] + expect(assistantRows.map((row) => row.text).join('')).toBe(output) + expect(assistantRows.length).toBeGreaterThan(100) + + for (const row of toolRows) { + insertSearchMessage(index.db, 1, row) + } + expect( + index.db + .prepare('SELECT count(*) AS n FROM messages_fts WHERE messages_fts MATCH ?') + .get('pericardium') + ).toEqual({ n: 1 }) + } finally { + await index.close() + } +}) + +it('files a tool row under the tool column alone', async () => { + const index = await openSessionSearchIndexFile('ss-message-rows-tool') + try { + for (const row of searchMessageRows([ + { role: 'tool', text: 'rg pericardium', timestamp: null } + ])) { + insertSearchMessage(index.db, 1, row) + } + expect(index.db.prepare('SELECT count(*) AS n FROM messages_fts').get()).toEqual({ n: 1 }) + // What makes a conversation-scoped search exclude it: the column filter, not + // a second table. + expect( + index.db + .prepare('SELECT count(*) AS n FROM messages_fts WHERE messages_fts MATCH ?') + .get('{user_text assistant_text}: pericardium') + ).toEqual({ n: 0 }) + expect( + index.db + .prepare('SELECT count(*) AS n FROM messages_fts WHERE messages_fts MATCH ?') + .get('{tool_text}: pericardium') + ).toEqual({ n: 1 }) + } finally { + await index.close() + } +}) + +it('stores a chunk exactly as the transcript wrote it', async () => { + const index = await openSessionSearchIndexFile('ss-rows-verbatim') + try { + const text = 'deploy with AKIAIOSFODNN7EXAMPLE and the resolveTerminalPath fix' + const stored = await indexedColumns(index, { + role: 'assistant', + text, + timestamp: null + }) + + // The index is a second copy of content the user already holds in plaintext, + // so it neither rewrites nor drops any of it. + expect(stored).toContain(text) + // Identifier shadow terms come off that same raw chunk. + expect(stored.some((column) => column.includes('resolve terminal path'))).toBe(true) + } finally { + await index.close() + } +}) diff --git a/src/main/ai-vault-search/session-search-message-rows.ts b/src/main/ai-vault-search/session-search-message-rows.ts new file mode 100644 index 00000000000..21254c183aa --- /dev/null +++ b/src/main/ai-vault-search/session-search-message-rows.ts @@ -0,0 +1,135 @@ +import type SyncDatabase from '../sqlite/sync-database' +import type { TranscriptMessage } from '../ai-vault/session-transcript-consumers' +import { sliceAtCodeUnitLimit } from '../ai-vault/session-scanner-text-normalization' +import { identifierShadowText } from './session-search-identifier-split' + +const CHUNK_TARGET_CHARS = 8000 + +/** + * How much of one tool output is indexed. Its head: a command, its arguments and + * the first lines of what it printed are what a user searches for, while the + * tail is the padding that makes these messages large in the first place. + * + * Tool output is 80-97 % of a transcript's bytes, and a single one can be a + * quarter of a megabyte (the reader's own per-message bound). Without this the + * index, the in-memory buffer a read holds and the transaction it commits are + * all sized by how much a tool printed rather than by how much is worth + * searching. 3 KB was the accuracy/size sweet spot in the original design + * measurement. User and assistant text is never capped: it is the conversation, + * and it is small. + */ +const TOOL_ROW_CHARS = 3072 + +// Keep unicode61's tokenchars intact, including before an available space. +// SQLite ext/fts5/fts5_unicode2.c: sqlite3Fts5UnicodeIsdiacritic, with remove_diacritics=1. +const FOLDED_DIACRITIC = + /[\u0300-\u0304\u0306-\u030c\u030f\u0311\u031b\u0323-\u0328\u032d-\u032e\u0330-\u0331]/ +const TOKEN_BOUNDARY = /[^\p{L}\p{N}\p{Co}_.\-/+\uD800-\uDFFF]/u + +/** + * Index just past the last token boundary in `[floor, end)`, or -1 when the + * window holds none. Not only a newline: a wrapped paragraph, a CJK transcript + * separated by ideographic spaces and a minified log all chunk on a boundary a + * tokenizer would have picked anyway. + */ +function lastTokenBoundaryEnd(text: string, floor: number, end: number): number { + for (let at = end - 1; at >= floor; at--) { + if (TOKEN_BOUNDARY.test(text[at]!) && !FOLDED_DIACRITIC.test(text[at]!)) { + return at + 1 + } + } + return -1 +} + +/** + * Splits an oversized message into rows of at most `CHUNK_TARGET_CHARS`, cutting + * on a token boundary so no token is torn in half and every word stays + * searchable. A phrase that straddles two chunks is not matched: chunks are + * separate FTS rows and FTS5 cannot span them. + */ +function* textChunks(text: string): Generator { + if (text.length <= CHUNK_TARGET_CHARS) { + yield text + return + } + let start = 0 + while (start < text.length) { + let end = Math.min(text.length, start + CHUNK_TARGET_CHARS) + if (end < text.length) { + // Only the second half of the window: backing up further would trade a + // torn token for chunks half the size. No boundary at all in 4,000 + // characters is not a word, so the target itself is the honest cut. + const split = lastTokenBoundaryEnd(text, start + CHUNK_TARGET_CHARS / 2, end) + if (split > start) { + end = split + } + } + yield text.slice(start, end) + start = end + } +} + +/** + * The row policy for one message: a `tool` message becomes one capped row, and + * anything else becomes N chunks, because FTS5 ranks a short row far better + * than a huge one. + */ +export function* searchMessageRows( + messages: Iterable +): Generator { + for (const message of messages) { + if (message.role === 'tool') { + yield { + ...message, + text: sliceAtCodeUnitLimit(message.text, TOOL_ROW_CHARS) + } + continue + } + for (const text of textChunks(message.text)) { + yield { ...message, text } + } + } +} + +/** + * Writes one row into `messages` and `messages_fts` in the caller's + * transaction, so a message is never present in one and absent from the other. + * A conversation-scoped query filters the columns rather than reading a second + * table (see the schema). + */ +export function insertSearchMessage( + db: SyncDatabase, + sessionId: number, + message: TranscriptMessage +): void { + const text = message.text + const id = db + .prepare('INSERT INTO messages(session_row_id, role, ts) VALUES (?, ?, ?)') + .run(sessionId, message.role, message.timestamp).lastInsertRowid + const user = message.role === 'user' ? text : '' + const assistant = message.role === 'assistant' ? text : '' + const tool = message.role === 'tool' ? text : '' + db.prepare( + 'INSERT INTO messages_fts(rowid,user_text,assistant_text,tool_text,identifiers) VALUES (?,?,?,?,?)' + ).run(id, user, assistant, tool, identifierShadowText(text)) +} + +/** + * Deletes up to `limit` of a session's rows from `messages` and `messages_fts`, + * in the caller's transaction, and reports how many went. Bounded + * because a retention sweep must not hold one transaction over a whole + * session; a replace passes no limit, since its rows and their replacements + * have to land together. + */ +export function deleteSearchMessages(db: SyncDatabase, sessionId: number, limit = -1): number { + const ids = db + .prepare('SELECT id FROM messages WHERE session_row_id = ? LIMIT ?') + .all(sessionId, limit) as { id: number }[] + const full = db.prepare('DELETE FROM messages_fts WHERE rowid = ?') + const message = db.prepare('DELETE FROM messages WHERE id = ?') + for (const { id } of ids) { + full.run(id) + message.run(id) + } + return ids.length +} diff --git a/src/main/ai-vault-search/session-search-retention-delete.test.ts b/src/main/ai-vault-search/session-search-retention-delete.test.ts new file mode 100644 index 00000000000..c21c8d7fe2b --- /dev/null +++ b/src/main/ai-vault-search/session-search-retention-delete.test.ts @@ -0,0 +1,188 @@ +import { expect, it } from 'vitest' +import type SyncDatabase from '../sqlite/sync-database' +import { + deleteExpiredSearchFiles, + RETENTION_DELETE_ROWS_PER_STEP +} from './session-search-retention-delete' +import { openSessionSearchIndexFile } from './session-search-index-test-fixture' +import { SessionSearchStore } from './session-search-store' + +function seed(db: SyncDatabase, id: number, rows: number, mtime: number): void { + db.prepare( + `INSERT INTO sessions(id,agent,session_id,file_path,title,cwd,cwd_key,resume_command) + VALUES (?, 'claude', ?, ?, 'synthetic retention', '/fixture', '/fixture', '')` + ).run(id, String(id), String(id)) + db.prepare('INSERT INTO files(path,byte_offset,mtime_ms,session_row_id) VALUES (?,1,?,?)').run( + String(id), + mtime, + id + ) + db.exec('BEGIN') + for (let i = 0; i < rows; i++) { + const row = db + .prepare("INSERT INTO messages(session_row_id,role) VALUES (?,'user')") + .run(id).lastInsertRowid + db.prepare('INSERT INTO messages_fts(rowid,user_text) VALUES (?,?)').run(row, 'retentionneedle') + } + db.exec('COMMIT') +} + +/** + * Sessions a search would still return. Every retrieval joins a message to its + * session, which is what makes cutting the session loose enough to hide the + * whole thing while its rows are still being reclaimed. + */ +function visibleSessionIds(db: SyncDatabase): string[] { + return ( + db + .prepare( + `SELECT DISTINCT s.session_id AS id FROM messages_fts + JOIN messages m ON m.id = messages_fts.rowid + JOIN sessions s ON s.id = m.session_row_id + WHERE messages_fts MATCH 'retentionneedle' ORDER BY s.session_id` + ) + .all() as { id: string }[] + ).map((row) => row.id) +} + +function count(db: SyncDatabase, table: string): number { + return (db.prepare(`SELECT count(*) AS n FROM ${table}`).get() as { n: number }).n +} + +it('seeks the expiring end of the file list instead of scanning it', async () => { + const index = await openSessionSearchIndexFile('ss-retention-plan') + try { + seed(index.db, 1, 1, 1) + const plan = ( + index.db + .prepare('EXPLAIN QUERY PLAN SELECT path FROM files WHERE mtime_ms < ? ORDER BY mtime_ms') + .all(100) as { detail: string }[] + ) + .map((row) => row.detail) + .join(' ') + // Without files_mtime this is "SCAN files" plus a "USE TEMP B-TREE FOR ORDER BY". + expect(plan).toContain('files_mtime') + expect(plan).not.toContain('TEMP B-TREE') + } finally { + await index.close() + } +}) + +it('hides an expiring session at once, then reclaims its rows in bounded steps', async () => { + const index = await openSessionSearchIndexFile('ss-retention-yield') + seed(index.db, 1, 1025, 1) + seed(index.db, 2, 1, 200) + let previous = 1025 + const steps: number[] = [] + try { + await deleteExpiredSearchFiles( + index.db, + 100, + () => false, + async () => { + const left = count(index.db, 'messages WHERE session_row_id=1') + steps.push(previous - left) + previous = left + // Cut loose in the very first transaction, so no query ever sees it with + // some of its messages already gone. + expect(visibleSessionIds(index.db)).toEqual(['2']) + } + ) + // The file transaction, then one bounded batch per step until the rows are gone. + expect(steps).toEqual([0, RETENTION_DELETE_ROWS_PER_STEP, 256, 256, 256, 1]) + expect(count(index.db, 'messages_fts')).toBe(1) + expect(count(index.db, 'sessions')).toBe(1) + } finally { + await index.close() + } +}) + +it('finishes an interrupted deletion after reopening', async () => { + const index = await openSessionSearchIndexFile('ss-retention-resume') + let store = new SessionSearchStore(index.path) + let closed = false + let steps = 0 + try { + seed(index.db, 1, 513, 1) + await deleteExpiredSearchFiles( + index.db, + 100, + () => closed, + async () => { + if (++steps === 2) { + store.close() + closed = true + } + } + ) + // Some rows went, the rest did not, and nothing recorded that anywhere. + const stranded = count(index.db, 'messages') + expect(stranded).toBeGreaterThan(0) + expect(stranded).toBeLessThan(513) + expect(visibleSessionIds(index.db)).toEqual([]) + + store = new SessionSearchStore(index.path) + closed = false + // Rows nothing points at are the whole record of unfinished work, so the + // rest goes even with retention now unlimited. + await store.purgeOlderThan(null) + expect(count(index.db, 'messages')).toBe(0) + expect(count(index.db, 'messages_fts')).toBe(0) + } finally { + if (!closed) { + store.close() + } + await index.close() + } +}) + +it('cancels retention between batches and resumes without exposing a partial session', async () => { + const index = await openSessionSearchIndexFile('ss-retention-cancel') + const store = new SessionSearchStore(index.path) + try { + seed(index.db, 1, 1025, 1) + const controller = new AbortController() + const purge = store.purgeOlderThan(100, controller.signal) + setImmediate(() => controller.abort()) + await purge + const remaining = count(index.db, 'messages') + expect(remaining).toBeGreaterThan(0) + expect(remaining).toBeLessThan(1025) + expect(visibleSessionIds(index.db)).toEqual([]) + await store.purgeOlderThan(null) + expect(count(index.db, 'messages')).toBe(0) + } finally { + store.close() + await index.close() + } +}) + +it('keeps a file a read refreshed after the expiry list was taken', async () => { + const index = await openSessionSearchIndexFile('ss-retention-refreshed') + try { + seed(index.db, 1, 2, 1) + seed(index.db, 2, 2, 2) + let refreshed = false + // The scan of `files` happens once, up front. A read of the second transcript + // lands while the first is being deleted, which makes it new enough to keep. + await deleteExpiredSearchFiles( + index.db, + 100, + () => false, + async () => { + if (!refreshed) { + refreshed = true + index.db.prepare('UPDATE files SET mtime_ms = 500 WHERE path = ?').run('2') + } + } + ) + + // Only the per-file transaction re-reading the mtime it is about to act on + // keeps that session; the list it came from says both should go. + expect(count(index.db, 'files')).toBe(1) + expect(visibleSessionIds(index.db)).toEqual(['2']) + expect(count(index.db, 'messages')).toBe(2) + } finally { + await index.close() + } +}) diff --git a/src/main/ai-vault-search/session-search-retention-delete.ts b/src/main/ai-vault-search/session-search-retention-delete.ts new file mode 100644 index 00000000000..e8d407f8be9 --- /dev/null +++ b/src/main/ai-vault-search/session-search-retention-delete.ts @@ -0,0 +1,102 @@ +import { setImmediate as yieldToEventLoop } from 'node:timers/promises' +import type SyncDatabase from '../sqlite/sync-database' +import { deleteSearchMessages } from './session-search-message-rows' + +export const RETENTION_DELETE_ROWS_PER_STEP = 256 +// Why in step with the deletes rather than one sweep at the end: `auto_vacuum = +// INCREMENTAL` holds every freed page until something asks for it back, and +// asking for a whole purge's worth at once is one long stall (40 ms per 22 MB +// freed, measured) instead of many short ones. +const RECLAIM_PAGES_PER_STEP = 2000 + +/** + * Drops every file older than the cutoff, then hands its rows back in bounded + * steps. + * + * The two halves are separate on purpose. Cutting a session loose from its file + * is one small transaction, and it is what makes the session stop answering + * searches — every read joins `sessions`, so a row whose session is gone is + * already unreachable. Reclaiming those rows is the expensive half, and it can + * be paused, interrupted or resumed at any point without a reader ever seeing a + * session that is half deleted. A crash in the middle leaves rows nothing + * points at, and `drainOrphanedMessages` finds them on the next pass. + */ +export async function deleteExpiredSearchFiles( + db: SyncDatabase, + cutoffMs: number | null, + closed: () => boolean, + yieldStep: () => Promise = yieldToEventLoop +): Promise { + if (cutoffMs !== null) { + const expired = db + .prepare('SELECT path FROM files WHERE mtime_ms < ? ORDER BY mtime_ms') + .all(cutoffMs) as { path: string }[] + for (const { path } of expired) { + if (closed()) { + return + } + db.exec('BEGIN IMMEDIATE') + try { + // Re-read under the lock: a read of this file may have landed since the + // list was taken, which makes it new enough to keep. + const file = db + .prepare('SELECT session_row_id FROM files WHERE path = ? AND mtime_ms < ?') + .get(path, cutoffMs) as { session_row_id: number | null } | undefined + if (file) { + db.prepare('DELETE FROM sessions WHERE id = ?').run(file.session_row_id) + db.prepare('DELETE FROM files WHERE path = ?').run(path) + } + db.exec('COMMIT') + } catch (error) { + db.exec('ROLLBACK') + throw error + } + await yieldStep() + } + } + await drainOrphanedMessages(db, closed, yieldStep) +} + +/** + * Deletes rows whose session no longer exists, a bounded batch per transaction. + * + * That set is exactly what retention, a replace that cut its old generation + * loose, a removed source and an interrupted earlier drain leave behind, so the + * index needs no record of unfinished work beyond the rows themselves. + * + * Exported for the store, which runs it after a replace commits for the same + * reason retention runs it after its own small transaction: cutting a session + * loose is what hides it, and reclaiming its rows is the half that must not + * hold one transaction. + */ +export async function drainOrphanedMessages( + db: SyncDatabase, + closed: () => boolean, + yieldStep: () => Promise = yieldToEventLoop +): Promise { + // Ordered by session so one call to this walks a session's rows to the end + // before paying for the scan that finds the next one. + const nextOrphan = db.prepare( + `SELECT session_row_id FROM messages + WHERE session_row_id NOT IN (SELECT id FROM sessions) LIMIT 1` + ) + let orphan = (nextOrphan.get() as { session_row_id: number } | undefined)?.session_row_id + while (orphan !== undefined && !closed()) { + db.exec('BEGIN IMMEDIATE') + let deleted = 0 + try { + deleted = deleteSearchMessages(db, orphan, RETENTION_DELETE_ROWS_PER_STEP) + db.exec('COMMIT') + } catch (error) { + db.exec('ROLLBACK') + throw error + } + db.pragma(`incremental_vacuum(${RECLAIM_PAGES_PER_STEP})`) + if (deleted < RETENTION_DELETE_ROWS_PER_STEP) { + orphan = (nextOrphan.get() as { session_row_id: number } | undefined)?.session_row_id + } + await yieldStep() + } + // A `removeFile` frees its pages outside this loop and may leave none to drain. + db.pragma(`incremental_vacuum(${RECLAIM_PAGES_PER_STEP})`) +} diff --git a/src/main/ai-vault-search/session-search-row-identity.test.ts b/src/main/ai-vault-search/session-search-row-identity.test.ts new file mode 100644 index 00000000000..4adc655b584 --- /dev/null +++ b/src/main/ai-vault-search/session-search-row-identity.test.ts @@ -0,0 +1,116 @@ +import { afterEach, beforeEach, expect, it } from 'vitest' +import type SyncDatabase from '../sqlite/sync-database' +import { deleteExpiredSearchFiles } from './session-search-retention-delete' +import { + openSessionSearchIndexFile, + syntheticCandidate, + syntheticSession, + userMessages, + type SessionSearchIndexFile +} from './session-search-index-test-fixture' +import { SessionSearchStore } from './session-search-store' + +// A session row id outlives the row: it names the rows in `messages` until a +// retention drain has walked all of them, which takes many transactions. These +// tests are about what may be handed that id in the meantime. + +let index: SessionSearchIndexFile +let store: SessionSearchStore +let errors: unknown[] + +beforeEach(async () => { + index = await openSessionSearchIndexFile('ss-row-identity') + errors = [] + store = new SessionSearchStore(index.path, (error) => errors.push(error)) +}) + +afterEach(async () => { + store.close() + await index.close() +}) + +const OLD_MTIME = 1_000 +const LIVE_MTIME = 1_000_000 +const LIVE_PATH = '/live.jsonl' + +function count(db: SyncDatabase, table: string): number { + return (db.prepare(`SELECT count(*) AS n FROM ${table}`).get() as { n: number }).n +} + +/** Rows a search would return for a term: the join every retrieval makes. */ +function matches(db: SyncDatabase, term: string): number { + return ( + db + .prepare( + `SELECT count(*) AS n FROM messages_fts JOIN messages m ON m.id = messages_fts.rowid + JOIN sessions s ON s.id = m.session_row_id WHERE messages_fts MATCH ?` + ) + .get(term) as { n: number } + ).n +} + +function indexFile(path: string, mtimeMs: number, text: string, rows: number): void { + const write = store.beginWrite(syntheticCandidate({ path, mtimeMs }), 'replace', 0)! + for (const message of userMessages(text, rows)) { + write.add(message) + } + expect(write.commit({ session: syntheticSession(), byteOffset: 50, incomplete: false })).toBe( + true + ) +} + +it('never hands a live session the rows of a purged one', async () => { + // Two expiring transcripts, each large enough that reclaiming their rows takes + // several transactions, and one live transcript the parser decoded no session + // from — so it holds a cursor and no session row of its own. + indexFile('/old-a.jsonl', OLD_MTIME, 'purgedneedle', 400) + indexFile('/old-b.jsonl', OLD_MTIME, 'purgedneedle', 400) + const live = syntheticCandidate({ path: LIVE_PATH, mtimeMs: LIVE_MTIME }) + const opening = store.beginWrite(live, 'replace', 0)! + opening.add(userMessages('excluded', 1)[0]!) + expect(opening.commit({ session: null, byteOffset: 50, incomplete: false })).toBe(true) + + let appended = false + await deleteExpiredSearchFiles( + index.db, + LIVE_MTIME, + () => false, + async () => { + // The window: both expiring sessions are cut loose, most of their rows are + // still on disk, and the live transcript grows. The append is legitimate — + // it continues this index's own cursor — and it needs a session row. + if (appended || count(index.db, 'sessions') > 0) { + return + } + appended = true + const write = store.beginWrite(live, 'append', 50)! + for (const message of userMessages('liveneedle', 2)) { + write.add(message) + } + expect( + write.commit({ session: syntheticSession(), byteOffset: 120, incomplete: false }) + ).toBe(true) + } + ) + + expect(appended).toBe(true) + // Reusing a freed id would adopt whatever of that session's rows the drain had + // not reached, and put them behind a live session no purge will visit again. + expect(matches(index.db, 'purgedneedle')).toBe(0) + expect(matches(index.db, 'liveneedle')).toBe(2) + expect(count(index.db, 'messages')).toBe(2) + expect(errors).toEqual([]) +}) + +it('never reissues a session row id a delete freed', () => { + for (const path of ['/a.jsonl', '/b.jsonl', '/c.jsonl']) { + indexFile(path, OLD_MTIME, 'seeded', 1) + } + const before = (index.db.prepare('SELECT max(id) AS id FROM sessions').get() as { id: number }).id + index.db.exec('DELETE FROM sessions') + + indexFile('/d.jsonl', OLD_MTIME, 'seeded', 1) + expect((index.db.prepare('SELECT id FROM sessions').get() as { id: number }).id).toBeGreaterThan( + before + ) +}) diff --git a/src/main/ai-vault-search/session-search-schema.test.ts b/src/main/ai-vault-search/session-search-schema.test.ts new file mode 100644 index 00000000000..b23f36cde55 --- /dev/null +++ b/src/main/ai-vault-search/session-search-schema.test.ts @@ -0,0 +1,350 @@ +import type * as NodeFs from 'node:fs' +import { mkdtemp, readFile, stat, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + removeTree, + WINDOWS_RM_MAX_RETRIES, + WINDOWS_RM_RETRY_DELAY_MS +} from '../../shared/windows-transient-lock-removal' +import SyncDatabase from '../sqlite/sync-database' +import { + SESSION_SEARCH_SCHEMA_VERSION, + openSessionSearchDatabase, + removeSessionSearchDatabase +} from './session-search-schema' + +const recordedRmSync = vi.hoisted(() => vi.fn()) +vi.mock('node:fs', async () => { + const actual = await vi.importActual('node:fs') + return { + ...actual, + rmSync: (...args: Parameters) => { + recordedRmSync(...args) + return actual.rmSync(...args) + } + } +}) + +let roots: string[] = [] + +afterEach(async () => { + await Promise.all(roots.map((root) => removeTree(root))) + roots = [] +}) + +async function tempDatabasePath(): Promise { + const root = await mkdtemp(join(tmpdir(), 'orca-session-search-schema-')) + roots.push(root) + return join(root, 'index.sqlite') +} + +function schemaVersion(db: SyncDatabase): string | undefined { + return ( + db.prepare("SELECT value FROM meta WHERE key = 'schema_version'").get() as + | { value: string } + | undefined + )?.value +} + +describe('openSessionSearchDatabase', () => { + it('keeps a current-version index and its rows', async () => { + const path = await tempDatabasePath() + const first = openSessionSearchDatabase(path) + first.prepare("INSERT INTO files(path,byte_offset,mtime_ms) VALUES ('a',1,1)").run() + first.close() + + const second = openSessionSearchDatabase(path) + expect(schemaVersion(second)).toBe(String(SESSION_SEARCH_SCHEMA_VERSION)) + expect(second.prepare('SELECT COUNT(*) AS c FROM files').get()).toEqual({ + c: 1 + }) + second.close() + }) + + it('carries one FTS table and throws away an index that carries two', async () => { + const path = await tempDatabasePath() + const fresh = openSessionSearchDatabase(path) + const tables = (): string[] => + ( + fresh + .prepare("SELECT name FROM sqlite_master WHERE type = 'table' AND name LIKE '%_fts'") + .all() as { name: string }[] + ).map((row) => row.name) + expect(tables()).toEqual(['messages_fts']) + + // What an index written before this bump looks like: the second table, and + // rows in it. `CREATE TABLE IF NOT EXISTS` would leave both in place, so + // only the version bump makes that file go. + fresh.exec('CREATE VIRTUAL TABLE conversation_fts USING fts5(user_text, assistant_text)') + fresh.prepare("INSERT INTO files(path,byte_offset,mtime_ms) VALUES ('a',1,1)").run() + fresh.prepare("UPDATE meta SET value = '3' WHERE key = 'schema_version'").run() + fresh.close() + + const rebuilt = openSessionSearchDatabase(path) + expect(schemaVersion(rebuilt)).toBe(String(SESSION_SEARCH_SCHEMA_VERSION)) + expect( + rebuilt + .prepare("SELECT count(*) AS n FROM sqlite_master WHERE name = 'conversation_fts'") + .get() + ).toEqual({ n: 0 }) + expect(rebuilt.prepare('SELECT COUNT(*) AS c FROM files').get()).toEqual({ c: 0 }) + rebuilt.close() + }) + + it('replaces the file on a version mismatch instead of dropping tables in place', async () => { + const path = await tempDatabasePath() + const stale = openSessionSearchDatabase(path) + stale.prepare("INSERT INTO files(path,byte_offset,mtime_ms) VALUES ('a',1,1)").run() + stale + .prepare("UPDATE meta SET value = ? WHERE key = 'schema_version'") + .run(String(SESSION_SEARCH_SCHEMA_VERSION + 1)) + stale.close() + // Why: a stale sidecar must go with the main file, or SQLite replays it into the new one. + await writeFile(`${path}-wal`, 'stale wal bytes') + const before = await stat(path) + + const fresh = openSessionSearchDatabase(path) + expect(schemaVersion(fresh)).toBe(String(SESSION_SEARCH_SCHEMA_VERSION)) + expect(fresh.prepare('SELECT COUNT(*) AS c FROM files').get()).toEqual({ + c: 0 + }) + fresh.close() + // Why not inode: ext4 hands a freed inode straight back to the next create. + // The planted sidecar is gone (a fresh WAL is checkpointed away on close). + await expect(stat(`${path}-wal`)).rejects.toMatchObject({ code: 'ENOENT' }) + expect((await stat(path)).mtimeMs).toBeGreaterThanOrEqual(before.mtimeMs) + }) + + it('removes the database with every sidecar', async () => { + const path = await tempDatabasePath() + openSessionSearchDatabase(path).close() + await writeFile(`${path}-shm`, '') + removeSessionSearchDatabase(path) + for (const suffix of ['', '-wal', '-shm']) { + await expect(stat(`${path}${suffix}`)).rejects.toMatchObject({ + code: 'ENOENT' + }) + } + }) +}) + +it('rebuilds a file too corrupt to open instead of refusing forever', async () => { + const path = await tempDatabasePath() + const healthy = openSessionSearchDatabase(path) + healthy.prepare("INSERT INTO files(path,byte_offset,mtime_ms) VALUES ('a',1,1)").run() + healthy.close() + // A torn page, not a truncation: SQLite opens the header and fails on the read. + const bytes = await readFile(path) + bytes.fill(0x7f, 4096, Math.min(bytes.length, 12_288)) + await writeFile(path, bytes) + + const rebuilt = openSessionSearchDatabase(path) + try { + expect(schemaVersion(rebuilt)).toBe(String(SESSION_SEARCH_SCHEMA_VERSION)) + expect(rebuilt.prepare('SELECT COUNT(*) AS c FROM files').get()).toEqual({ + c: 0 + }) + } finally { + rebuilt.close() + } +}) + +it('rebuilds a file that is not a database at all', async () => { + const path = await tempDatabasePath() + await writeFile(path, 'not a SQLite database') + + const rebuilt = openSessionSearchDatabase(path) + try { + expect(schemaVersion(rebuilt)).toBe(String(SESSION_SEARCH_SCHEMA_VERSION)) + } finally { + rebuilt.close() + } +}) + +it('gives up rather than looping when a fresh file still cannot be opened', async () => { + const path = await tempDatabasePath() + await writeFile(path, 'not a SQLite database') + // Every open of this path fails, so the one permitted retry is exhausted. + const open = vi.spyOn(SyncDatabase.prototype, 'pragma').mockImplementation(() => { + throw Object.assign(new Error('database disk image is malformed'), { + code: 'SQLITE_CORRUPT' + }) + }) + try { + expect(() => openSessionSearchDatabase(path)).toThrow(/malformed/) + } finally { + open.mockRestore() + } +}) + +it('surfaces the unlink failure itself when a stale index cannot be removed', async () => { + const path = await tempDatabasePath() + const stale = openSessionSearchDatabase(path) + stale + .prepare("UPDATE meta SET value = ? WHERE key = 'schema_version'") + .run(String(SESSION_SEARCH_SCHEMA_VERSION + 1)) + stale.close() + recordedRmSync.mockReset() + recordedRmSync.mockImplementation(() => { + throw Object.assign(new Error('EPERM: operation not permitted, unlink'), { + code: 'EPERM' + }) + }) + try { + // The stale handle is closed before the unlink, so the failure path must not + // close it again: ERR_INVALID_STATE would bury the cause and would not be + // classified as worth a rebuild. + expect(() => openSessionSearchDatabase(path)).toThrow(/EPERM/) + expect(() => openSessionSearchDatabase(path)).not.toThrow(/not open/) + } finally { + recordedRmSync.mockReset() + } +}) + +it('creates the directory the index lives in', async () => { + const root = await mkdtemp(join(tmpdir(), 'orca-session-search-mkdir-')) + roots.push(root) + // The real layout: `/ai-vault-search/index.sqlite`, where nothing + // has made that folder yet. SQLite would fail with `unable to open database + // file`, which is correctly not treated as corruption, so it never retries. + const db = openSessionSearchDatabase(join(root, 'ai-vault-search', 'index.sqlite')) + try { + expect(schemaVersion(db)).toBe(String(SESSION_SEARCH_SCHEMA_VERSION)) + } finally { + db.close() + } +}) + +it('rebuilds a newer index rather than reading a schema it does not know', async () => { + const path = await tempDatabasePath() + const newer = openSessionSearchDatabase(path) + newer.prepare("INSERT INTO files(path,byte_offset,mtime_ms) VALUES ('a',1,1)").run() + newer + .prepare("UPDATE meta SET value = ? WHERE key = 'schema_version'") + .run(String(SESSION_SEARCH_SCHEMA_VERSION + 1)) + newer.close() + + const rebuilt = openSessionSearchDatabase(path) + try { + expect(schemaVersion(rebuilt)).toBe(String(SESSION_SEARCH_SCHEMA_VERSION)) + expect(rebuilt.prepare('SELECT COUNT(*) AS c FROM files').get()).toEqual({ + c: 0 + }) + } finally { + rebuilt.close() + } +}) + +it('rebuilds when meta exists but its version row is gone', async () => { + const path = await tempDatabasePath() + const damaged = openSessionSearchDatabase(path) + damaged.prepare("INSERT INTO files(path,byte_offset,mtime_ms) VALUES ('a',1,1)").run() + // A meta table with no version is a damaged index, never a fresh one: seeding + // the current version over it would keep whatever the old schema left behind. + damaged.prepare("DELETE FROM meta WHERE key = 'schema_version'").run() + damaged.close() + + const rebuilt = openSessionSearchDatabase(path) + try { + expect(schemaVersion(rebuilt)).toBe(String(SESSION_SEARCH_SCHEMA_VERSION)) + expect(rebuilt.prepare('SELECT COUNT(*) AS c FROM files').get()).toEqual({ + c: 0 + }) + } finally { + rebuilt.close() + } +}) + +it('opens with the pragmas the write path depends on', async () => { + const db = openSessionSearchDatabase(await tempDatabasePath()) + try { + // auto_vacuum=2 is INCREMENTAL, and only takes on an empty file: without it + // a purge cannot hand pages back in bounded steps. + expect(Number(db.pragma('auto_vacuum', { simple: true }))).toBe(2) + expect(String(db.pragma('journal_mode', { simple: true })).toLowerCase()).toBe('wal') + expect(Number(db.pragma('synchronous', { simple: true }))).toBe(1) + // A WAL with no size limit never hands its space back after a large write. + expect(Number(db.pragma('journal_size_limit', { simple: true }))).toBe(8388608) + // Zero here turns every contended write into an immediate SQLITE_BUSY. + expect(Number(db.pragma('busy_timeout', { simple: true }))).toBe(5000) + } finally { + db.close() + } +}) + +it("walks a session's rows through an index rather than scanning the table", async () => { + const db = openSessionSearchDatabase(await tempDatabasePath()) + try { + // The replace delete and the orphan drain both take this path, once per file. + const plan = ( + db + .prepare('EXPLAIN QUERY PLAN SELECT id FROM messages WHERE session_row_id = ? LIMIT ?') + .all(1, 1) as { detail: string }[] + ) + .map((row) => row.detail) + .join(' ') + expect(plan).toContain('messages_session') + } finally { + db.close() + } +}) + +it('keeps only the session indexes a retrieval query can seek', async () => { + const db = openSessionSearchDatabase(await tempDatabasePath()) + try { + const names = ( + db + .prepare("SELECT name FROM sqlite_master WHERE type='index' AND tbl_name='sessions'") + .all() as { name: string }[] + ) + .map((row) => row.name) + .sort() + // One per shape PR 4's retrieval seeks: the agent filter, the newest-first + // order and date window, and the folder-prefix range scan. Fork folding reads + // `content_hash` off rows it already holds, so that column is not indexed. + expect(names).toEqual(['sessions_agent', 'sessions_cwd_key', 'sessions_updated_at']) + } finally { + db.close() + } +}) + +it("retries a Windows lock that outlives rmSync's own retries", async () => { + const path = await tempDatabasePath() + openSessionSearchDatabase(path).close() + vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') + recordedRmSync.mockReset() + const locked = Object.assign(new Error('EPERM: operation not permitted'), { + code: 'EPERM' + }) + recordedRmSync.mockImplementationOnce(() => { + throw locked + }) + try { + expect(() => removeSessionSearchDatabase(path)).not.toThrow() + expect(recordedRmSync.mock.calls.length).toBe(5) + await expect(stat(path)).rejects.toMatchObject({ code: 'ENOENT' }) + } finally { + recordedRmSync.mockReset() + vi.restoreAllMocks() + } +}) + +it('gives Windows the shared retry options for a late handle release', async () => { + const path = await tempDatabasePath() + vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') + recordedRmSync.mockClear() + try { + removeSessionSearchDatabase(path) + expect(recordedRmSync).toHaveBeenCalled() + for (const [, options] of recordedRmSync.mock.calls) { + expect(options).toMatchObject({ + maxRetries: WINDOWS_RM_MAX_RETRIES, + retryDelay: WINDOWS_RM_RETRY_DELAY_MS + }) + } + } finally { + vi.restoreAllMocks() + } +}) diff --git a/src/main/ai-vault-search/session-search-schema.ts b/src/main/ai-vault-search/session-search-schema.ts new file mode 100644 index 00000000000..2da1e64bc11 --- /dev/null +++ b/src/main/ai-vault-search/session-search-schema.ts @@ -0,0 +1,198 @@ +import { mkdirSync } from 'node:fs' +import { dirname } from 'node:path' +import SyncDatabase from '../sqlite/sync-database' +import { removeTreeSync } from '../../shared/windows-transient-lock-removal' + +// The index stores transcript content as written, with no redaction. A secret in +// a transcript is already plaintext under the user's home directory and is +// treated as compromised; this is a second copy of content the user already +// holds. What a snippet may carry once it leaves this machine is a transport +// policy, decided where the wire is. + +// Bump to drop and rebuild: the index is a cache over the transcripts, never a source. +export const SESSION_SEARCH_SCHEMA_VERSION = 5 + +// unicode61 keeps `_ . - /` inside tokens so paths and identifiers match exactly; +// the `identifiers` column carries the split form (see session-search-identifier-split). +// Why: `+` keeps `C++` a token of its own instead of the letter `c`; `#` is +// left out so `#123` still answers a search for `123`. +const TOKENIZER = `tokenize="unicode61 tokenchars '_.-/+'"` + +const SCHEMA_SQL = ` +CREATE TABLE IF NOT EXISTS meta(key TEXT PRIMARY KEY, value TEXT NOT NULL); +CREATE TABLE IF NOT EXISTS sessions( + -- AUTOINCREMENT, because this id names rows in the messages table for longer + -- than the row itself lives: retention cuts a session loose in one + -- transaction and reclaims its messages over many. A plain rowid is reissued + -- as max+1, so a session created inside that window would be handed a freed + -- id and adopt whatever of the purged conversation the drain had not reached, + -- behind a live session no later purge visits. + id INTEGER PRIMARY KEY AUTOINCREMENT, + agent TEXT NOT NULL, + session_id TEXT NOT NULL, + -- The transcript this session was decoded from. Not unique: OpenCode's SQLite + -- sessions all report the store's own path here, while files.path holds the + -- synthetic db#sessionId candidate that really is one per session. + file_path TEXT NOT NULL, + codex_home TEXT, + title TEXT NOT NULL, + cwd TEXT, + cwd_key TEXT, + branch TEXT, + created_at TEXT, + updated_at TEXT, + message_count INTEGER NOT NULL DEFAULT 0, + resume_command TEXT NOT NULL, + -- Chained digest of the first N messages; forks of one conversation share it. + content_hash TEXT, + content_hash_count INTEGER NOT NULL DEFAULT 0 +); +CREATE INDEX IF NOT EXISTS sessions_agent ON sessions(agent); +CREATE INDEX IF NOT EXISTS sessions_updated_at ON sessions(updated_at); +CREATE INDEX IF NOT EXISTS sessions_cwd_key ON sessions(cwd_key); +CREATE TABLE IF NOT EXISTS files( + path TEXT PRIMARY KEY, + dev INTEGER, + ino INTEGER, + byte_offset INTEGER NOT NULL, + mtime_ms REAL NOT NULL, + size_bytes INTEGER, + session_row_id INTEGER +); +-- Retention walks the expiring end of this column; without it that is a full scan and a sort. +CREATE INDEX IF NOT EXISTS files_mtime ON files(mtime_ms); +CREATE TABLE IF NOT EXISTS messages( + id INTEGER PRIMARY KEY, + session_row_id INTEGER NOT NULL, + role TEXT NOT NULL, + ts TEXT +); +-- Both the replace delete and the orphan drain walk a session's rows through this. +CREATE INDEX IF NOT EXISTS messages_session ON messages(session_row_id); +-- One FTS table, not two. A conversation-scoped search is a column filter on +-- this one — 'MATCH {user_text assistant_text}: q' with bm25 weights that zero +-- the other two — and PR 4 measured that at 1.16-1.36x the p95 of a dedicated +-- second table on a 105 MB corpus, under the 2x bar the decision was set at. +CREATE VIRTUAL TABLE IF NOT EXISTS messages_fts USING fts5( + user_text, assistant_text, tool_text, identifiers, ${TOKENIZER}, detail=full +); +` + +/** + * Opens the index, rebuilding it whenever what is on disk cannot be trusted: + * a different schema version, a version SQLite cannot report, or a file torn + * badly enough that opening or recovery fails. The index is a cache over the + * transcripts, so throwing away a bad one costs a re-scan and nothing else; + * refusing to open would strand the feature until a human deleted the file. + */ +export function openSessionSearchDatabase(path: string): SyncDatabase { + // SQLite will not create the directory, and its failure is `unable to open + // database file`, which is correctly not corruption — so without this the + // feature strands on a profile that has never held an index. + if (path !== ':memory:') { + mkdirSync(dirname(path), { recursive: true }) + } + try { + return openExisting(path) + } catch (error) { + if (!isUnusableDatabaseError(error)) { + throw error + } + // One retry only: a second failure on a file we just created is not corruption. + removeSessionSearchDatabase(path) + return openExisting(path) + } +} + +function openExisting(path: string): SyncDatabase { + // Nulled while no handle is open, because closing an already-closed handle + // throws ERR_INVALID_STATE, which would replace whatever really failed — + // an unlink refused by a virus scanner or a second Orca holding the file — + // with an error nothing classifies as worth rebuilding for. + let db: SyncDatabase | null = openWithPragmas(path) + try { + if (isStaleSchema(db)) { + // Why: DROP TABLE on a multi-GB FTS index takes minutes and runs inside the + // scanner service's init, past its ready timeout; unlinking is instant. + db.close() + db = null + removeSessionSearchDatabase(path) + db = openWithPragmas(path) + } + db.exec(SCHEMA_SQL) + db.prepare('INSERT OR REPLACE INTO meta(key, value) VALUES (?, ?)').run( + 'schema_version', + String(SESSION_SEARCH_SCHEMA_VERSION) + ) + return db + } catch (error) { + db?.close() + throw error + } +} + +// SQLite reports a torn file at the first statement that has to read a page, so +// this has to match on the message as well as the code. +const UNUSABLE_DATABASE = + /SQLITE_CORRUPT|SQLITE_NOTADB|file is not a database|database disk image is malformed/i + +function isUnusableDatabaseError(error: unknown): boolean { + if (!(error instanceof Error)) { + return false + } + const code = (error as { code?: unknown }).code + return ( + (typeof code === 'string' && UNUSABLE_DATABASE.test(code)) || + UNUSABLE_DATABASE.test(error.message) + ) +} + +function openWithPragmas(path: string): SyncDatabase { + const db = new SyncDatabase(path) + try { + // Why: only takes effect on an empty file; it is what lets a purge hand pages + // back in bounded steps instead of a full VACUUM. Set before any table exists. + db.pragma('auto_vacuum = INCREMENTAL') + // The whole consistency model: a file's rows and its cursor land in one + // transaction, and a reader on another handle sees the last committed state + // of the index rather than a session half way through being rewritten. + db.pragma('journal_mode = WAL') + db.pragma('synchronous = NORMAL') + db.pragma('journal_size_limit = 8388608') + db.pragma('busy_timeout = 5000') + return db + } catch (error) { + db?.close() + throw error + } +} + +export function removeSessionSearchDatabase(path: string): void { + if (path === ':memory:') { + return + } + for (const suffix of ['', '-wal', '-shm', '-journal']) { + removeTreeSync(`${path}${suffix}`) + } +} + +/** + * Whether what is on disk has to be thrown away. No `meta` table at all is a + * file with nothing in it to throw away, and removing it would make the first + * open of every new profile a create-remove-create. A meta table whose version + * row is missing or unparseable is a damaged index rather than a new one: + * seeding the current version over it would keep whatever rows the old schema + * left. + */ +function isStaleSchema(db: SyncDatabase): boolean { + const table = db + .prepare("SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'meta'") + .get() + if (!table) { + return false + } + const row = db.prepare("SELECT value FROM meta WHERE key = 'schema_version'").get() as + | { value: string } + | undefined + return (row ? Number(row.value) : Number.NaN) !== SESSION_SEARCH_SCHEMA_VERSION +} diff --git a/src/main/ai-vault-search/session-search-store.ts b/src/main/ai-vault-search/session-search-store.ts new file mode 100644 index 00000000000..da9f4d6f731 --- /dev/null +++ b/src/main/ai-vault-search/session-search-store.ts @@ -0,0 +1,253 @@ +import type SyncDatabase from '../sqlite/sync-database' +import type { SessionFileCandidate } from '../ai-vault/session-scanner-types' +import type { TranscriptSessionIdentity } from '../ai-vault/session-transcript-consumers' +import type { + SessionSearchFileIdentity, + SessionSearchIndexedFile +} from './session-search-file-cursor' +import { + SESSION_SEARCH_COMMIT_CHARS, + SessionSearchIndexWriter, + type SessionSearchFileWrite +} from './session-search-index-writer' +import { deleteExpiredSearchFiles, drainOrphanedMessages } from './session-search-retention-delete' +import { openSessionSearchDatabase } from './session-search-schema' + +// A paused store keeps recording what it declined, so the set needs a ceiling. +// Above it the oldest record goes and the drop is counted, because a re-read set +// that silently forgets is worse than one that says it is incomplete. +export const STALE_PATH_LIMIT = 20_000 + +/** + * Owns the index database. PR 2 scope: the write half only — the transcript + * consumer writes through it and nothing reads from it yet. Lifecycle (who + * indexes, when, and how the re-read set is drained) belongs to the service. + */ +export class SessionSearchStore { + private readonly db: SyncDatabase + private readonly writer: SessionSearchIndexWriter + private closed = false + private acceptingWrites = true + private retentionCutoffMs: number | null = null + // Files this index knows it is behind on. Filled by a declined or abandoned + // read; PR 3's indexer drains it. Nothing here schedules the re-read. + private readonly stale = new Map() + private droppedStalePaths = 0 + // One drain at a time. A replace that commits while one is running asks for + // another pass rather than starting a second walk of the same rows. + private draining = false + private drainRequested = false + + constructor( + path: string, + private readonly onError: (error: unknown) => void = (error) => + console.warn( + '[ai-vault-search] index write failed:', + error instanceof Error ? error.name : 'IndexError' + ) + ) { + this.db = openSessionSearchDatabase(path) + this.writer = new SessionSearchIndexWriter(this.db, SESSION_SEARCH_COMMIT_CHARS, () => + this.scheduleOrphanDrain() + ) + } + + /** + * Reclaims the rows a replace cut loose, once its transaction has committed. + * + * The same split retention makes, for the same reason: deleting the old + * session row is what stops it answering, because every retrieval joins + * `sessions`, and handing its messages back is the expensive half that must + * not hold one transaction. Nothing records the work: rows whose session row + * is gone are the whole record, so a crash before or during a drain is found + * by the next one. + */ + private scheduleOrphanDrain(): void { + this.drainRequested = true + if (this.draining || this.closed) { + return + } + this.draining = true + // Off the committing stack. An async function runs synchronously up to its + // first `await`, so calling the drain here would put its first batch back + // inside the call that committed the replace — the cost this took out. + void Promise.resolve().then(() => this.runOrphanDrain()) + } + + private async runOrphanDrain(): Promise { + try { + while (this.drainRequested && !this.closed) { + this.drainRequested = false + await drainOrphanedMessages(this.db, () => this.closed) + } + } catch (error) { + if (!this.closed) { + this.onError(error) + } + } finally { + this.draining = false + } + } + + /** + * The index handle, for a reader composed over this store (PR 4's engine). + * + * Two rules come with it, both measured in this PR. **Never hold a read + * transaction across an `await`**: a checkpoint cannot pass an open read + * snapshot, so a paginated read that opened `BEGIN` and yielded between pages + * takes the WAL from 10 MB to 266 MB and it does not come back. And **no + * `.iterate()` that outlives its statement**, which is the same pin by + * another name. Every retrieval a single synchronous statement is the whole + * contract. + */ + get connection(): SyncDatabase { + return this.db + } + + setAcceptingWrites(accept: boolean): void { + this.acceptingWrites = accept + } + + /** The oldest transcript mtime worth indexing; PR 3 derives it from the retention setting. */ + setRetentionCutoffMs(cutoffMs: number | null): void { + this.retentionCutoffMs = cutoffMs + } + + /** Whether this candidate is new enough to be worth holding rows for at all. */ + private withinRetention(candidate: SessionFileCandidate): boolean { + return this.retentionCutoffMs === null || candidate.file.mtimeMs >= this.retentionCutoffMs + } + + /** Whether a write for this candidate may start right now. */ + acceptsCandidate(candidate: SessionFileCandidate): boolean { + return !this.closed && this.acceptingWrites && this.withinRetention(candidate) + } + + indexedFile(path: string, identity: SessionSearchFileIdentity): SessionSearchIndexedFile | null { + try { + return this.writer.indexedFile(path, identity) + } catch (error) { + this.onError(error) + return null + } + } + + /** Null when this read cannot extend the index, or when the store refuses writes. */ + beginWrite( + candidate: SessionFileCandidate, + mode: 'replace' | 'append', + previousByteOffset: number, + identity?: () => TranscriptSessionIdentity | null + ): SessionSearchFileWrite | null { + if (!this.acceptsCandidate(candidate)) { + return null + } + try { + return this.writer.beginWrite(candidate, mode, previousByteOffset, identity) + } catch (error) { + this.reportWriteFailure(error) + return null + } + } + + writeCommitted(candidate: SessionFileCandidate): void { + // Why: a list scan queues every file the backfill has not reached yet; once + // one lands, a later pass must not re-read the whole queue. + this.stale.delete(candidate.file.path) + } + + reportWriteFailure(error: unknown): void { + this.onError(error) + } + + /** + * Records a file whose content the index is behind on, for a later whole + * re-read. Recorded while paused too: a pause is exactly the window in which + * reads are declined, so refusing to remember them would lose every file the + * pause covered. + */ + markStale(candidate: SessionFileCandidate): void { + if (this.closed || !this.withinRetention(candidate)) { + return + } + // Re-inserting moves the path to the end, so the oldest record is the one + // dropped when a long pause overruns the bound. + this.stale.delete(candidate.file.path) + this.stale.set(candidate.file.path, candidate) + while (this.stale.size > STALE_PATH_LIMIT) { + const oldest = this.stale.keys().next() + if (oldest.done) { + break + } + this.stale.delete(oldest.value) + this.droppedStalePaths += 1 + } + } + + /** + * Files the index knew it was behind on and could not keep a record of. A + * non-zero count means the re-read set is incomplete, so coverage cannot be + * reported as whole until a full pass runs. + */ + get droppedPendingFileCount(): number { + return this.droppedStalePaths + } + + /** + * Hands the re-read set to its scheduler and clears it. + * + * These paths are behind, not merely dirty: the index declined their last read + * because it covered a span the index never saw. Re-dispatching a scan is not + * enough on its own, because the reader picks `append` from the session list's + * resume point and the consumer will decline again. The caller must pass each + * path to `requestWholeTranscriptRead` first. + */ + takeStale(): SessionFileCandidate[] { + const candidates = [...this.stale.values()] + this.stale.clear() + return candidates + } + + get pendingFileCount(): number { + return this.stale.size + } + + /** + * Drops a source's rows. Only a proven deletion may call this: an unreadable + * source is `unverifiable`, not `missing`, and keeps its rows + * (docs/reference/ssh-execution-boundary.md). + */ + removeFile(path: string): void { + this.stale.delete(path) + try { + this.writer.removeFile(path) + } catch (error) { + this.onError(error) + } + } + + /** Cuts expired sessions loose at once, then reclaims their rows in resumable batches. */ + async purgeOlderThan(cutoffMs: number | null, signal?: AbortSignal): Promise { + try { + await deleteExpiredSearchFiles( + this.db, + cutoffMs, + () => this.closed || signal?.aborted === true + ) + } catch (error) { + if (!this.closed) { + this.onError(error) + } + } + } + + close(): void { + // node:sqlite throws ERR_INVALID_STATE on a second close, and a store is + // closed both by its owner and by a test's teardown. + if (this.closed) { + return + } + this.closed = true + this.db.close() + } +} diff --git a/src/main/ai-vault-search/session-search-synthetic-corpus.test.ts b/src/main/ai-vault-search/session-search-synthetic-corpus.test.ts new file mode 100644 index 00000000000..9b6a48beb4e --- /dev/null +++ b/src/main/ai-vault-search/session-search-synthetic-corpus.test.ts @@ -0,0 +1,44 @@ +import { rm } from 'node:fs/promises' +import { join } from 'node:path' +import { expect, it } from 'vitest' +import { resetSessionParseCacheForTests } from '../ai-vault/session-scanner-parse-cache' +import { resetTranscriptConsumersForTests } from '../ai-vault/session-transcript-consumers' +import { registerSessionSearchIndexConsumer } from './session-search-index-consumer' +import { SessionSearchStore } from './session-search-store' +import { writeSyntheticTranscriptCorpus } from './session-search-synthetic-corpus' +import { parseTranscript } from './session-search-transcript-fixtures' + +it.each([Infinity, -Infinity, Number.NaN, -1, 1.5])( + 'rejects invalid corpus loop bounds: %s', + async (value) => { + for (const field of ['sessions', 'turnsPerSession', 'toolResultWords']) { + await expect(writeSyntheticTranscriptCorpus({ [field]: value })).rejects.toThrow(RangeError) + } + } +) + +it.each([0, 200, 2000])( + 'counts the indexed messages with %s tool words', + async (toolResultWords) => { + const corpus = await writeSyntheticTranscriptCorpus({ + sessions: 1, + turnsPerSession: 1, + toolResultWords + }) + const store = new SessionSearchStore(join(corpus.root, 'index.sqlite')) + const unregister = registerSessionSearchIndexConsumer(store) + try { + await parseTranscript(corpus.files[0]!) + expect(corpus.messageCount).toBe(toolResultWords === 0 ? 3 : 4) + expect(store.connection.prepare('SELECT count(*) AS n FROM messages').get()).toEqual({ + n: corpus.messageCount + }) + } finally { + unregister() + resetTranscriptConsumersForTests() + resetSessionParseCacheForTests() + store.close() + await rm(corpus.root, { recursive: true, force: true }) + } + } +) diff --git a/src/main/ai-vault-search/session-search-synthetic-corpus.ts b/src/main/ai-vault-search/session-search-synthetic-corpus.ts new file mode 100644 index 00000000000..14e8a27fe5f --- /dev/null +++ b/src/main/ai-vault-search/session-search-synthetic-corpus.ts @@ -0,0 +1,154 @@ +import { mkdtemp, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' + +// Why synthetic and in-repo: the cost model has to be reproducible on any host +// and must never read a real transcript. The shapes here mirror what a Claude +// JSONL transcript actually holds — prose turns, a pasted diff, tool calls and +// their output — because the index's disk cost tracks the mix, not the size. + +const WORDS = [ + 'terminal', + 'reattach', + 'worktree', + 'resolveTerminalPath', + 'src/main/ai-vault/session-transcript-reader.ts', + 'the', + 'index', + 'cursor', + 'byteOffset', + 'publish', + 'staged', + 'transaction', + 'MAX_RETRIES', + 'relay', + 'daemon', + 'pty', + 'snapshot', + 'because' +] + +/** Deterministic: the same seed gives the same corpus on every host and run. */ +function mulberry32(seed: number): () => number { + let state = seed >>> 0 + return () => { + state = (state + 0x6d2b79f5) >>> 0 + let t = Math.imul(state ^ (state >>> 15), 1 | state) + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t + return ((t ^ (t >>> 14)) >>> 0) / 4294967296 + } +} + +function words(random: () => number, count: number): string { + const out: string[] = [] + for (let index = 0; index < count; index++) { + out.push(WORDS[Math.floor(random() * WORDS.length)]) + } + return out.join(' ') +} + +export type SyntheticCorpus = { + root: string + files: string[] + /** Total bytes of transcript written, the denominator of write amplification. */ + transcriptBytes: number + messageCount: number +} + +export type SyntheticCorpusOptions = { + sessions?: number + turnsPerSession?: number + seed?: number + /** + * Words per tool result. The default keeps tool output at about half the + * message text; the real distribution is 80-97 %, which is what prices the + * tool-row cap, so the benchmark runs a second arm well above the default. + */ + toolResultWords?: number +} + +/** Writes a corpus of Claude JSONL transcripts and reports what it cost on disk. */ +export async function writeSyntheticTranscriptCorpus( + options: SyntheticCorpusOptions = {} +): Promise { + const sessions = options.sessions ?? 40 + const turns = options.turnsPerSession ?? 60 + const toolWords = options.toolResultWords ?? 200 + for (const [name, value] of Object.entries({ + sessions, + turnsPerSession: turns, + toolResultWords: toolWords + })) { + if (!Number.isSafeInteger(value) || value < 0) { + throw new RangeError(`${name} must be a finite non-negative safe integer`) + } + } + const random = mulberry32(options.seed ?? 1) + const root = await mkdtemp(join(tmpdir(), 'orca-search-corpus-')) + const files: string[] = [] + let transcriptBytes = 0 + let messageCount = 0 + + for (let session = 0; session < sessions; session++) { + const sessionId = `00000000-0000-4000-8000-${String(session).padStart(12, '0')}` + const lines: string[] = [] + for (let turn = 0; turn < turns; turn++) { + const at = new Date(1740000000000 + turn * 60_000).toISOString() + lines.push( + JSON.stringify({ + type: 'user', + sessionId, + timestamp: at, + cwd: `/repo/app-${session % 7}`, + gitBranch: 'main', + message: { role: 'user', content: words(random, 40) } + }) + ) + lines.push( + JSON.stringify({ + type: 'assistant', + sessionId, + timestamp: at, + message: { + role: 'assistant', + model: 'claude-fable-5', + content: [ + { type: 'text', text: words(random, 120) }, + { + type: 'tool_use', + name: 'Bash', + input: { command: `rg ${words(random, 3)}` } + } + ] + } + }) + ) + lines.push( + JSON.stringify({ + type: 'user', + sessionId, + timestamp: at, + message: { + role: 'user', + content: [ + { + type: 'tool_result', + tool_use_id: 'toolu_1', + content: words(random, toolWords) + } + ] + } + }) + ) + // Empty tool results emit no searchable message. + messageCount += toolWords === 0 ? 3 : 4 + } + const path = join(root, `${sessionId}.jsonl`) + const body = `${lines.join('\n')}\n` + await writeFile(path, body) + transcriptBytes += Buffer.byteLength(body) + files.push(path) + } + + return { root, files, transcriptBytes, messageCount } +} diff --git a/src/main/ai-vault-search/session-search-transcript-fixtures.ts b/src/main/ai-vault-search/session-search-transcript-fixtures.ts new file mode 100644 index 00000000000..bc7eda9a8ff --- /dev/null +++ b/src/main/ai-vault-search/session-search-transcript-fixtures.ts @@ -0,0 +1,118 @@ +import { stat } from 'node:fs/promises' +import { + createSessionParseStats, + parseAgentSessionFileCached, + type SessionParseStats +} from '../ai-vault/session-scanner-parse-cache' +import type { SessionFileCandidate } from '../ai-vault/session-scanner-types' + +// Transcript builders shared by the session-search store tests; each file owns +// its temp directories, this module only shapes records and drives the parser. + +export const CLAUDE_SESSION_ID = 'aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee' +export const CODEX_SESSION_ID = '019f0000-1111-7222-8333-444444444444' +export const CODEX_ROLLOUT_FILE = `rollout-2026-05-01T10-00-00-${CODEX_SESSION_ID}.jsonl` + +const RECORD_EPOCH_MS = 1740000000000 + +export function recordTimestamp(index: number): string { + return new Date(RECORD_EPOCH_MS + index * 60_000).toISOString() +} + +export function userRecord( + index: number, + content: unknown, + sessionId = CLAUDE_SESSION_ID, + cwd = '/repo/app' +): string { + return JSON.stringify({ + type: 'user', + sessionId, + timestamp: recordTimestamp(index), + cwd, + gitBranch: 'main', + message: { role: 'user', content } + }) +} + +export function assistantRecord( + index: number, + content: unknown, + sessionId = CLAUDE_SESSION_ID +): string { + return JSON.stringify({ + type: 'assistant', + sessionId, + timestamp: recordTimestamp(index), + message: { role: 'assistant', model: 'claude-fable-5', content } + }) +} + +export async function sessionCandidate( + agent: SessionFileCandidate['agent'], + path: string, + codexHome: string | null = null +): Promise { + const fileStat = await stat(path) + return { + agent, + codexHome, + file: { + path, + mtimeMs: fileStat.mtimeMs, + modifiedAt: fileStat.mtime.toISOString(), + sizeBytes: fileStat.size, + dev: fileStat.dev, + ino: fileStat.ino + } + } +} + +export async function parseTranscript( + path: string, + agent: SessionFileCandidate['agent'] = 'claude', + codexHome: string | null = null +): Promise<{ stats: SessionParseStats }> { + const stats = createSessionParseStats() + await parseAgentSessionFileCached( + await sessionCandidate(agent, path, codexHome), + process.platform, + stats + ) + return { stats } +} + +function codexLine(record: Record): string { + return JSON.stringify(record) +} + +/** Minimal Codex rollout: meta, one user message, one completed shell command. */ +export function codexRolloutLines(command: string[], output: string, prompt: string): string[] { + return [ + codexLine({ + timestamp: recordTimestamp(0), + type: 'session_meta', + payload: { id: CODEX_SESSION_ID, cwd: '/repo/app', git: { branch: 'main' } } + }), + codexLine({ + timestamp: recordTimestamp(1), + type: 'response_item', + payload: { type: 'message', role: 'user', content: prompt } + }), + codexLine({ + timestamp: recordTimestamp(2), + type: 'response_item', + payload: { + type: 'function_call', + call_id: 'call-1', + name: 'shell', + arguments: JSON.stringify({ command }) + } + }), + codexLine({ + timestamp: recordTimestamp(3), + type: 'response_item', + payload: { type: 'function_call_output', call_id: 'call-1', output } + }) + ] +} diff --git a/src/main/ai-vault/session-scanner-accumulator.ts b/src/main/ai-vault/session-scanner-accumulator.ts index 88e09627eb7..18f273d6be3 100644 --- a/src/main/ai-vault/session-scanner-accumulator.ts +++ b/src/main/ai-vault/session-scanner-accumulator.ts @@ -23,7 +23,11 @@ import { normalizePreviewText, timestampMs } from './session-scanner-values' -import { NO_TRANSCRIPT_MESSAGES, type TranscriptMessageSink } from './session-transcript-consumers' +import { + NO_TRANSCRIPT_MESSAGES, + type TranscriptMessageSink, + type TranscriptSessionIdentity +} from './session-transcript-consumers' import { boundedText, transcriptMessageRole, @@ -64,6 +68,28 @@ export function createAccumulator(args: { } } +/** + * The session identity a fold holds right now. Null until it has an id, which + * every supported format writes in the opening lines of the transcript. + */ +export function accumulatorSessionIdentity( + accumulator: SessionAccumulator +): TranscriptSessionIdentity | null { + const sessionId = accumulator.sessionId.trim() + if (!sessionId) { + return null + } + return { + sessionId, + cwd: accumulator.cwd, + // The generated fallback is `finalizeSession`'s, not this one's: a title + // that is still absent mid-read is better said to be absent. + title: accumulator.title ?? accumulator.fallbackTitle, + createdAt: accumulator.createdAt, + updatedAt: accumulator.updatedAt + } +} + export function cloneSessionAccumulator(accumulator: SessionAccumulator): SessionAccumulator { return { ...accumulator, previewMessages: [...accumulator.previewMessages] } } @@ -77,6 +103,7 @@ export function accumulatorFoldResumeState( ): ResumableSessionParseState { return { consumeLine: (line) => consumeRecordLine(accumulator, line), + identity: () => accumulatorSessionIdentity(accumulator), clone: () => accumulatorFoldResumeState(cloneSessionAccumulator(accumulator), consumeRecordLine), touchFile: (file) => { diff --git a/src/main/ai-vault/session-scanner-codex-message-records.ts b/src/main/ai-vault/session-scanner-codex-message-records.ts index 5aa739275ff..a8a5c3c9d13 100644 --- a/src/main/ai-vault/session-scanner-codex-message-records.ts +++ b/src/main/ai-vault/session-scanner-codex-message-records.ts @@ -1,3 +1,7 @@ +import { + publishCodexResponseTool, + publishCodexCompletedTool +} from './session-scanner-codex-tool-records' import { normalizePromptField } from '../../shared/agent-status-field-normalization' import { addPreviewContent } from './session-scanner-accumulator' import type { SessionAccumulator } from './session-scanner-types' @@ -8,6 +12,10 @@ export function consumeCodexResponseMessage( payload: Record, timestamp: unknown ): boolean { + publishCodexResponseTool(accumulator, payload, timestamp) + if (payload.type !== 'message') { + return false + } accumulator.messageCount++ const role = payload.role === 'assistant' ? 'assistant' : payload.role === 'user' ? 'user' : 'unknown' @@ -24,6 +32,7 @@ export function consumeCodexCompletedMessage( payload: Record, timestamp: unknown ): boolean { + publishCodexCompletedTool(accumulator, payload, timestamp) const item = asRecord(payload.item) if (!item) { return false diff --git a/src/main/ai-vault/session-scanner-codex-parser.ts b/src/main/ai-vault/session-scanner-codex-parser.ts index 02a385400de..b3571d4a337 100644 --- a/src/main/ai-vault/session-scanner-codex-parser.ts +++ b/src/main/ai-vault/session-scanner-codex-parser.ts @@ -4,6 +4,7 @@ import type { AiVaultSession } from '../../shared/ai-vault-types' import { readCodexSessionIndexTitle } from './session-scanner-codex-title-index' import type { ExecutionHostId } from '../../shared/execution-host' import { + accumulatorSessionIdentity, cloneSessionAccumulator, createAccumulator, finalizeSession, @@ -153,19 +154,13 @@ function consumeCodexRecordLine(state: CodexSessionParseState, line: string): vo accumulator.title = metadataTitle state.titleSource = 'meta' } - const cwd = extractString(payload.cwd) - if (cwd) { - accumulator.cwd = cwd - } + accumulator.cwd = extractString(payload.cwd) ?? accumulator.cwd accumulator.branch = extractGitBranch(payload.git) ?? accumulator.branch return } if (record.type === 'turn_context' && payload) { - const cwd = extractString(payload.cwd) - if (cwd) { - accumulator.cwd = cwd - } + accumulator.cwd = extractString(payload.cwd) ?? accumulator.cwd const model = extractModel(payload) if (model) { accumulator.model = model @@ -177,7 +172,7 @@ function consumeCodexRecordLine(state: CodexSessionParseState, line: string): vo return } - if (record.type === 'response_item' && payload.type === 'message') { + if (record.type === 'response_item') { if (state.historyMode === 'paginated') { return } @@ -285,7 +280,10 @@ function codexResumeStateFromParseState( return { consumeLine: (line) => consumeCodexRecordLine(state, line), consumeLineBytes: (line) => { - const timelineOnlyRecord = readCodexTimelineOnlyRecord(line) + const timelineOnlyRecord = readCodexTimelineOnlyRecord( + line, + state.accumulator.messages.active && state.historyMode !== 'paginated' + ) if (timelineOnlyRecord) { updateTimeline(state.accumulator, timelineOnlyRecord.timestamp) } else { @@ -293,6 +291,7 @@ function codexResumeStateFromParseState( } }, shouldStop: () => state.rejectedWorkerSession, + identity: () => accumulatorSessionIdentity(state.accumulator), clone: () => codexResumeStateFromParseState(cloneCodexParseState(state), codexHome, titleReader), touchFile: (file) => { diff --git a/src/main/ai-vault/session-scanner-codex-record-fast-path.ts b/src/main/ai-vault/session-scanner-codex-record-fast-path.ts index 1c4322f89f6..852ec174e95 100644 --- a/src/main/ai-vault/session-scanner-codex-record-fast-path.ts +++ b/src/main/ai-vault/session-scanner-codex-record-fast-path.ts @@ -1,3 +1,5 @@ +import { CODEX_TOOL_RESPONSE_TYPES } from './session-scanner-codex-tool-records' + // Records below this size are decoded and parsed exactly: JSON.parse on a // kilobyte costs less than the risk of a prefix heuristic, and the scan cost // this path exists to remove is entirely in megabyte-scale records. @@ -22,7 +24,10 @@ const PARSED_EVENT_TYPES = new Set([ ]) /** Returns the timestamp only when the record cannot affect other visible session fields. */ -export function readCodexTimelineOnlyRecord(line: Buffer): { timestamp: string } | null { +export function readCodexTimelineOnlyRecord( + line: Buffer, + includeTools = false +): { timestamp: string } | null { if (line.length <= CODEX_RECORD_PREFIX_LIMIT) { return null } @@ -41,6 +46,13 @@ export function readCodexTimelineOnlyRecord(line: Buffer): { timestamp: string } if (!payloadType) { return null } + if ( + includeTools && + recordType === 'response_item' && + CODEX_TOOL_RESPONSE_TYPES.has(payloadType) + ) { + return null + } const parsedPayloadTypes = recordType === 'response_item' ? PARSED_RESPONSE_ITEM_TYPES : PARSED_EVENT_TYPES return parsedPayloadTypes.has(payloadType) ? null : { timestamp } diff --git a/src/main/ai-vault/session-scanner-codex-tool-records.test.ts b/src/main/ai-vault/session-scanner-codex-tool-records.test.ts new file mode 100644 index 00000000000..a5aa909bfa1 --- /dev/null +++ b/src/main/ai-vault/session-scanner-codex-tool-records.test.ts @@ -0,0 +1,125 @@ +import { expect, it } from 'vitest' +import { createCodexSessionResumeState } from './session-scanner-codex-parser' +import type { TranscriptMessage } from './session-transcript-consumers' +import { readCodexTimelineOnlyRecord } from './session-scanner-codex-record-fast-path' + +const timestamp = '2026-05-01T10:00:00.000Z' +const file = { + path: '/fixture/rollout.jsonl', + mtimeMs: Date.parse(timestamp), + modifiedAt: timestamp +} +const record = (type: string, payload: Record): Buffer => + Buffer.from(JSON.stringify({ timestamp, type, payload })) + +it.each(['function_call_output', 'custom_tool_call_output'])( + 'reads large %s records only when a consumer needs them', + (type) => { + const line = record('response_item', { type, output: 'outputonly '.repeat(300) }) + expect(readCodexTimelineOnlyRecord(line)).toEqual({ timestamp }) + expect(readCodexTimelineOnlyRecord(line, true)).toBeNull() + const messages: TranscriptMessage[] = [] + const state = createCodexSessionResumeState(file, null, { + active: true, + push: (message) => messages.push(message) + }) + state.consumeLineBytes!(line) + expect(messages).toEqual([{ role: 'tool', text: 'outputonly '.repeat(300), timestamp }]) + } +) + +it.each([false, true])( + 'uses one tool representation across append when paginated=%s', + async (paginated) => { + const messages: TranscriptMessage[] = [] + let state = createCodexSessionResumeState(file, null, { + active: true, + push: (message) => messages.push(message) + }) + const consume = (type: string, payload: Record) => + state.consumeLineBytes!(record(type, payload)) + consume('session_meta', { id: 'session-1', history_mode: paginated ? 'paginated' : 'full' }) + consume('response_item', { type: 'message', role: 'user', content: 'promptonly' }) + consume('event_msg', { + type: 'item_completed', + item: { type: 'UserMessage', content: [{ type: 'text', text: 'promptonly' }] } + }) + consume('response_item', { + type: 'function_call', + name: 'shell', + arguments: '{"command":"commandonly"}' + }) + // The next scan resumes between the call and its output. + state = state.clone() + consume('response_item', { type: 'function_call_output', output: 'outputonly' }) + consume('event_msg', { + type: 'item_completed', + item: { type: 'CommandExecution', command: ['commandonly'], aggregated_output: 'outputonly' } + }) + expect(messages.filter((message) => message.text.includes('commandonly'))).toHaveLength(1) + expect(messages.filter((message) => message.text === 'outputonly')).toEqual([ + { role: 'tool', text: 'outputonly', timestamp } + ]) + expect(messages.filter((message) => message.role === 'user')).toHaveLength(1) + expect(await state.finalize(process.platform)).toMatchObject({ messageCount: 1 }) + } +) + +it.each([ + { type: 'add', content: '+ addedneedle' }, + { type: 'delete', content: '+ addedneedle' }, + { type: 'update', unified_diff: '+ addedneedle', move_path: null } +])('publishes paginated $type file changes', (change) => { + const messages: TranscriptMessage[] = [] + const state = createCodexSessionResumeState(file, null, { + active: true, + push: (message) => messages.push(message) + }) + state.consumeLineBytes!(record('session_meta', { id: 'session-1', history_mode: 'paginated' })) + state.consumeLineBytes!( + record('event_msg', { + type: 'item_completed', + item: { type: 'FileChange', changes: { 'src/changed.ts': change } } + }) + ) + expect(messages.map((message) => [message.role, message.text])).toEqual([ + ['tool', 'apply_patch: src/changed.ts'], + ['tool', '+ addedneedle'] + ]) +}) + +it('normalizes custom calls and structured results through the existing content reader', () => { + const messages: TranscriptMessage[] = [] + const state = createCodexSessionResumeState(file, null, { + active: true, + push: (message) => messages.push(message) + }) + state.consumeLineBytes!( + record('response_item', { type: 'custom_tool_call', name: 'apply_patch', input: 'patchneedle' }) + ) + state.consumeLineBytes!( + record('response_item', { + type: 'custom_tool_call_output', + output: { content: [{ type: 'text', text: 'resultneedle' }] } + }) + ) + expect(messages.map((message) => message.text)).toEqual([ + 'apply_patch: patchneedle', + 'resultneedle' + ]) +}) + +it('keeps local shell argv searchable', () => { + const messages: TranscriptMessage[] = [] + const state = createCodexSessionResumeState(file, null, { + active: true, + push: (message) => messages.push(message) + }) + state.consumeLineBytes!( + record('response_item', { + type: 'local_shell_call', + action: { type: 'exec', command: ['rg', 'argvneedle'] } + }) + ) + expect(messages.map((message) => message.text)).toEqual(['tool: rg argvneedle']) +}) diff --git a/src/main/ai-vault/session-scanner-codex-tool-records.ts b/src/main/ai-vault/session-scanner-codex-tool-records.ts new file mode 100644 index 00000000000..976d5eff36d --- /dev/null +++ b/src/main/ai-vault/session-scanner-codex-tool-records.ts @@ -0,0 +1,95 @@ +import { timestampIso } from './session-scanner-accumulator' +import { asRecord } from './session-scanner-record-value' +import type { SessionAccumulator } from './session-scanner-types' +import { transcriptMessagesFromContent } from './session-transcript-message-content' + +export const CODEX_TOOL_RESPONSE_TYPES = new Set([ + 'function_call', + 'local_shell_call', + 'custom_tool_call', + 'function_call_output', + 'custom_tool_call_output' +]) + +function publishToolContent( + accumulator: SessionAccumulator, + content: unknown, + timestamp: unknown +): void { + for (const message of transcriptMessagesFromContent('tool', content, timestampIso(timestamp))) { + accumulator.messages.push(message) + } +} + +export function publishCodexResponseTool( + accumulator: SessionAccumulator, + payload: Record, + timestamp: unknown +): void { + if (!accumulator.messages.active || !CODEX_TOOL_RESPONSE_TYPES.has(String(payload.type))) { + return + } + if (payload.type === 'function_call_output' || payload.type === 'custom_tool_call_output') { + const output = asRecord(payload.output) + publishToolContent( + accumulator, + [{ type: 'tool_result', content: output?.content ?? output?.output ?? payload.output }], + timestamp + ) + return + } + const input = payload.arguments ?? payload.input ?? payload.action + const action = asRecord(input) + const normalizedInput = + action && Array.isArray(action.command) + ? { ...action, command: action.command.filter((part) => typeof part === 'string').join(' ') } + : input + publishToolContent( + accumulator, + [ + { + type: 'tool_use', + name: payload.name ?? 'tool', + input: normalizedInput + } + ], + timestamp + ) +} + +export function publishCodexCompletedTool( + accumulator: SessionAccumulator, + payload: Record, + timestamp: unknown +): void { + if (!accumulator.messages.active) { + return + } + const item = asRecord(payload.item) + if (item?.type === 'CommandExecution' || item?.type === 'command_execution') { + const command = Array.isArray(item.command) + ? item.command.filter((part) => typeof part === 'string').join(' ') + : item.command + publishToolContent( + accumulator, + [ + { type: 'tool_use', name: 'shell', input: command }, + { type: 'tool_result', content: item.aggregated_output ?? item.aggregatedOutput } + ], + timestamp + ) + } else if (item?.type === 'FileChange' || item?.type === 'file_change') { + const changes = asRecord(item.changes) ?? {} + for (const [path, value] of Object.entries(changes)) { + const change = asRecord(value) + publishToolContent( + accumulator, + [ + { type: 'tool_use', name: 'apply_patch', input: { path } }, + { type: 'tool_result', content: change?.unified_diff ?? change?.content } + ], + timestamp + ) + } + } +} diff --git a/src/main/ai-vault/session-scanner-omp-subagent-transcripts.ts b/src/main/ai-vault/session-scanner-omp-subagent-transcripts.ts index 57cadebeee0..07f3646b537 100644 --- a/src/main/ai-vault/session-scanner-omp-subagent-transcripts.ts +++ b/src/main/ai-vault/session-scanner-omp-subagent-transcripts.ts @@ -101,6 +101,7 @@ export function withOmpSubagentTranscriptCount( ): ResumableSessionParseState { return { consumeLine: (line) => state.consumeLine(line), + identity: () => state.identity?.() ?? null, clone: () => withOmpSubagentTranscriptCount(state.clone(), transcriptFilePath), touchFile: (file) => state.touchFile(file), finalize: async (platform, options) => { diff --git a/src/main/ai-vault/session-scanner-primary-parsers.ts b/src/main/ai-vault/session-scanner-primary-parsers.ts index 9783f8a0520..62149920b5a 100644 --- a/src/main/ai-vault/session-scanner-primary-parsers.ts +++ b/src/main/ai-vault/session-scanner-primary-parsers.ts @@ -12,6 +12,7 @@ import type { } from './session-scanner-types' import type { TranscriptMessageSink } from './session-transcript-consumers' import { + accumulatorSessionIdentity, addPreviewContent, createAccumulator, finalizeSession, @@ -205,6 +206,7 @@ function claudeResumeStateFromParseState( ): ResumableSessionParseState { return { consumeLine: (line) => consumeClaudeSessionLine(state, line), + identity: () => accumulatorSessionIdentity(state.accumulator), clone: () => claudeResumeStateFromParseState(cloneClaudeSessionParseState(state)), touchFile: (file) => { state.accumulator.modifiedAt = file.modifiedAt diff --git a/src/main/ai-vault/session-scanner-types.ts b/src/main/ai-vault/session-scanner-types.ts index b1d480aa944..f4b60270544 100644 --- a/src/main/ai-vault/session-scanner-types.ts +++ b/src/main/ai-vault/session-scanner-types.ts @@ -5,7 +5,10 @@ import type { AiVaultSessionPreviewMessage } from '../../shared/ai-vault-types' import type { ExecutionHostId } from '../../shared/execution-host' -import type { TranscriptMessageSink } from './session-transcript-consumers' +import type { + TranscriptMessageSink, + TranscriptSessionIdentity +} from './session-transcript-consumers' import type { SessionSidecarObservation } from './session-sidecar-stat' export type AiVaultScanOptions = { @@ -103,6 +106,9 @@ export type ResumableSessionParseState = { consumeLineBytes?(line: Buffer): void // Lets a parser terminate an excluded transcript without draining the file. shouldStop?(): boolean + // What the fold knows about the session right now, for a consumer that has to + // commit before the read ends (see TranscriptSessionIdentity). + identity?(): TranscriptSessionIdentity | null clone(): ResumableSessionParseState // Refresh per-scan file metadata (mtime display string) without re-parsing. touchFile(file: FileWithMtime): void diff --git a/src/main/ai-vault/session-transcript-consumers.ts b/src/main/ai-vault/session-transcript-consumers.ts index 6707b298586..7aff8dcf87b 100644 --- a/src/main/ai-vault/session-transcript-consumers.ts +++ b/src/main/ai-vault/session-transcript-consumers.ts @@ -27,12 +27,34 @@ export const NO_TRANSCRIPT_MESSAGES: TranscriptMessageSink = { push: () => undefined } +/** + * What a parser has decoded about the session so far, mid-read. + * + * Provisional by construction: it is read before the file ends, so a title can + * still change and a timestamp can still move. Every field the transcript + * formats put in their opening lines, which is what a consumer that has to + * commit before the read finishes needs to name what it is holding. + */ +export type TranscriptSessionIdentity = { + sessionId: string + cwd: string | null + title: string | null + createdAt: string | null + updatedAt: string | null +} + export type TranscriptReadStart = { candidate: SessionFileCandidate /** `replace`: the whole file is being re-read; `append`: a resumed read. */ mode: 'replace' | 'append' /** Byte offset the messages of this read continue from. */ previousByteOffset: number + /** + * The session identity decoded so far, or null before the parser has an id. + * Called during the read, never here: nothing is decoded yet when a read + * begins. Absent when the read has no resumable parse state to ask. + */ + identity?: () => TranscriptSessionIdentity | null } export type TranscriptReadOutcome = { diff --git a/src/main/ai-vault/session-transcript-reader.ts b/src/main/ai-vault/session-transcript-reader.ts index 228b0c832a3..3fc0ddcc146 100644 --- a/src/main/ai-vault/session-transcript-reader.ts +++ b/src/main/ai-vault/session-transcript-reader.ts @@ -3,7 +3,10 @@ import type { AiVaultSession } from '../../shared/ai-vault-types' import { parseAgentSessionFile, parserPublishesMessages } from './session-scanner-agent-parser' import { consumeCompleteJsonlLines } from './session-scanner-jsonl-reader' import type { ResumableSessionParseState, SessionFileCandidate } from './session-scanner-types' -import type { SessionParseResumePoint } from './session-parse-cache-store' +import { + invalidateSessionParseCacheEntry, + type SessionParseResumePoint +} from './session-parse-cache-store' import { TranscriptMessageChannel } from './session-transcript-channel' const NEWLINE_BYTE = 0x0a @@ -29,6 +32,24 @@ export type ResumableTranscriptRead = { resume: SessionParseResumePoint } +/** + * Ask for the next read of `path` to be a whole-file `replace`. + * + * Why this lives here: a consumer never chooses its own mode. The reader picks + * `append` or `replace` from the resume point the session list left behind, so a + * consumer that declined an append has no way to get the span it missed — with + * an empty index and a warm parse cache, every read arrives as `append`, every + * one is declined, and nothing is ever indexed. Dropping the resume point is the + * one lever that changes the next read's mode, and only the reader's own cache + * owns it. + * + * The cost is a re-parse for the session list too. That is the honest price of a + * second consumer being behind, and it is paid once per file rather than per scan. + */ +export function requestWholeTranscriptRead(path: string): void { + invalidateSessionParseCacheEntry(path) +} + /** * Read an append-only transcript, resuming from `resume` when the file only * grew and the recorded offset still sits on a line boundary. Anything else @@ -70,7 +91,10 @@ export async function readResumableTranscript(args: { channel.beginRead({ candidate: args.candidate, mode: canResume ? 'append' : 'replace', - previousByteOffset: startOffset + previousByteOffset: startOffset, + // Read by a consumer during the read, not here: the fold has decoded + // nothing yet at this point of a whole-file read. + identity: () => state.identity?.() ?? null }) try { const readResult = await consumeCompleteJsonlLines({ From ecd7b19ad42ab78d190b37ca22cd57ab783103fb Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Thu, 10 Sep 2026 20:50:40 -0700 Subject: [PATCH 7/8] fix(native-chat): pass agent-implemented slash commands through to the agent (#19929) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(native-chat): pass agent-implemented slash commands through to the agent Claim what the host implements; pass through what the agent implements. Claude's harness expands a slash command out of the message text, so the host claimed catalog commands it had no way to run and answered "/init is not available in chat sessions" for commands Claude does run. Codex's app-server has no slash parser at all, so its catalog stays claimed — except /goal, which the model carries out through its own goal tools. * fix(native-chat): offer the agent-run commands in the structured picker Codex reports no command catalog, so its structured `/` menu is the host fallback -- which listed only the host's own commands and hid `/goal`, the one command the model itself acts on. The picker now appends the profile's text-driven commands, described from the curated catalog, so a command that passes through is discoverable and not merely typable. The menu invariant holds either way: a pick is answered by the host or run by the agent, never refused with "not available in chat sessions". * fix mobile structured command reconciliation * fix(mobile): keep native chat controller within lint budget * fix mobile controller lint budget --------- Co-authored-by: Merge Sim --- .../src/session/MobileNativeChatComposer.tsx | 2 +- .../use-mobile-native-chat-controller.ts | 7 +- ...structured-native-chat-send-bridge.test.ts | 88 +++++++++++++ ...bile-structured-native-chat-send-bridge.ts | 15 +-- .../native-chat/NativeChatComposer.test.tsx | 14 +- .../use-native-chat-composer-catalog.test.tsx | 20 +++ .../use-native-chat-composer-catalog.ts | 2 +- ...ive-chat-structured-composer-send.test.tsx | 82 ++++++++++++ src/shared/native-chat-agent-profiles.test.ts | 31 ++++- src/shared/native-chat-agent-profiles.ts | 48 ++++++- .../structured-agent-session-composer.test.ts | 123 +++++++++++++++++- .../structured-agent-session-composer.ts | 32 ++++- 12 files changed, 426 insertions(+), 38 deletions(-) create mode 100644 mobile/src/session/use-mobile-structured-native-chat-send-bridge.test.ts create mode 100644 src/renderer/src/components/native-chat/use-native-chat-structured-composer-send.test.tsx diff --git a/mobile/src/session/MobileNativeChatComposer.tsx b/mobile/src/session/MobileNativeChatComposer.tsx index c16b69ced89..20c43aa6c8b 100644 --- a/mobile/src/session/MobileNativeChatComposer.tsx +++ b/mobile/src/session/MobileNativeChatComposer.tsx @@ -130,7 +130,7 @@ export function MobileNativeChatComposer({ if (trigger.kind === 'slash') { const commands = structuredCommands !== undefined - ? structuredSlashCommands(structuredCommands) + ? structuredSlashCommands(structuredCommands, agent) : agent ? getVerifiedNativeChatCommands(agent) : [] diff --git a/mobile/src/session/use-mobile-native-chat-controller.ts b/mobile/src/session/use-mobile-native-chat-controller.ts index d3d68b85032..5b941367451 100644 --- a/mobile/src/session/use-mobile-native-chat-controller.ts +++ b/mobile/src/session/use-mobile-native-chat-controller.ts @@ -140,7 +140,7 @@ export function useMobileNativeChatController(args: { ) const { permission: legacyNativeChatPermission, - question: legacyNativeChatQuestion, + question: legacyQuestion, detectedAsk: nativeChatDetectedAsk, ask: nativeChatAskPrompt } = useMobileNativeChatPrompts({ @@ -242,6 +242,7 @@ export function useMobileNativeChatController(args: { }) const structuredNativeChatSend = useMobileStructuredNativeChatSendBridge({ + agent: activeChatResolution?.agent === 'claude' ? 'claude' : 'codex', sendStructured: structuredNativeChat.sendWithOutcome, captureSendOrigin, clearDraftForSend, @@ -309,9 +310,7 @@ export function useMobileNativeChatController(args: { nativeChatPermission: activeChatStructured ? structuredNativeChat.permission : legacyNativeChatPermission, - nativeChatQuestion: activeChatStructured - ? structuredNativeChat.question - : legacyNativeChatQuestion, + nativeChatQuestion: activeChatStructured ? structuredNativeChat.question : legacyQuestion, nativeChatAsk: !activeChatStructured && showNativeChatAsk ? nativeChatAskPrompt : null, nativeChatAskKey, dismissNativeChatAsk, diff --git a/mobile/src/session/use-mobile-structured-native-chat-send-bridge.test.ts b/mobile/src/session/use-mobile-structured-native-chat-send-bridge.test.ts new file mode 100644 index 00000000000..3cf432381c2 --- /dev/null +++ b/mobile/src/session/use-mobile-structured-native-chat-send-bridge.test.ts @@ -0,0 +1,88 @@ +import { createElement } from 'react' +import { act, create, type ReactTestRenderer } from 'react-test-renderer' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import type { AgentSessionHandleProvider } from '../../../src/shared/agent-session-provider-handle' +import type { MobileNativeChatSendOutcome } from './mobile-native-chat-send' +import type { MobileNativeChatSendOrigin } from './use-mobile-native-chat-drafts' +import { useMobileStructuredNativeChatSendBridge } from './use-mobile-structured-native-chat-send-bridge' + +const ORIGIN: MobileNativeChatSendOrigin = { + draftKey: 'draft', + draftEditGeneration: 0, + pendingKey: 'pending', + normalizedText: 'command', + baselineOccurrences: 0, + baselineTailMessageId: null, + baselineResolved: true +} + +describe('useMobileStructuredNativeChatSendBridge', () => { + let renderer: ReactTestRenderer | null = null + let sendWithOutcome: (text: string) => Promise + const acceptSend = vi.fn() + const captureSendOrigin = vi.fn(() => ORIGIN) + const clearDraftForSend = vi.fn() + const holdUnconfirmedSend = vi.fn() + const onSendError = vi.fn() + const restoreRejectedDraft = vi.fn() + const sendStructured = vi.fn() + + function Harness({ agent }: { agent: AgentSessionHandleProvider }): null { + sendWithOutcome = useMobileStructuredNativeChatSendBridge({ + agent, + acceptSend, + captureSendOrigin, + clearDraftForSend, + holdUnconfirmedSend, + onSendError, + restoreRejectedDraft, + sendStructured + }).sendWithOutcome + return null + } + + function mount(agent: AgentSessionHandleProvider): void { + act(() => { + renderer = create(createElement(Harness, { agent })) + }) + } + + beforeEach(() => { + vi.clearAllMocks() + }) + + afterEach(() => { + act(() => renderer?.unmount()) + renderer = null + }) + + it('optimistically echoes accepted commands owned by the active agent', async () => { + sendStructured.mockResolvedValue('accepted') + mount('claude') + + await expect(sendWithOutcome('/init')).resolves.toBe('accepted') + + expect(acceptSend).toHaveBeenCalledWith(ORIGIN, '/init', undefined) + expect(restoreRejectedDraft).not.toHaveBeenCalled() + }) + + it('holds unknown delivery for commands owned by the active agent', async () => { + sendStructured.mockResolvedValue('unknown') + mount('claude') + + await expect(sendWithOutcome('/review')).resolves.toBe('unknown') + + expect(holdUnconfirmedSend).toHaveBeenCalledWith(ORIGIN, '/review', expect.any(Function)) + expect(restoreRejectedDraft).not.toHaveBeenCalled() + }) + + it('keeps host-command reconciliation for Codex', async () => { + sendStructured.mockResolvedValue('unknown') + mount('codex') + + await expect(sendWithOutcome('/review')).resolves.toBe('unknown') + + expect(restoreRejectedDraft).toHaveBeenCalledWith(ORIGIN, '/review') + expect(holdUnconfirmedSend).not.toHaveBeenCalled() + }) +}) diff --git a/mobile/src/session/use-mobile-structured-native-chat-send-bridge.ts b/mobile/src/session/use-mobile-structured-native-chat-send-bridge.ts index 70261e9df9b..d1cfe3d42f4 100644 --- a/mobile/src/session/use-mobile-structured-native-chat-send-bridge.ts +++ b/mobile/src/session/use-mobile-structured-native-chat-send-bridge.ts @@ -1,4 +1,5 @@ import { useCallback } from 'react' +import type { AgentSessionHandleProvider } from '../../../src/shared/agent-session-provider-handle' import { isStructuredAgentSessionComposerCommand } from '../../../src/shared/structured-agent-session-composer' import type { MobileNativeChatSendOutcome } from './mobile-native-chat-send' import type { MobileNativeChatSendOrigin } from './use-mobile-native-chat-drafts' @@ -10,6 +11,7 @@ type StructuredNativeChatAttachment = { } export function useMobileStructuredNativeChatSendBridge(args: { + agent: AgentSessionHandleProvider sendStructured: ( text: string, images?: string[], @@ -37,6 +39,7 @@ export function useMobileStructuredNativeChatSendBridge(args: { } { const { acceptSend, + agent, captureSendOrigin, clearDraftForSend, holdUnconfirmedSend, @@ -56,6 +59,7 @@ export function useMobileStructuredNativeChatSendBridge(args: { onSendError('Message not sent (disconnected)') return 'rejected' } + const isHostCommand = isStructuredAgentSessionComposerCommand(text, agent) clearDraftForSend(origin, text) const outcome = attachments !== undefined @@ -66,19 +70,13 @@ export function useMobileStructuredNativeChatSendBridge(args: { ? await sendStructured(text, images) : await sendStructured(text) if (outcome === 'accepted') { - if ( - !isStructuredAgentSessionComposerCommand(text, 'codex') && - !isStructuredAgentSessionComposerCommand(text, 'claude') - ) { + if (!isHostCommand) { acceptSend(origin, text.trimEnd(), images) } return 'accepted' } if (outcome === 'unknown') { - if ( - isStructuredAgentSessionComposerCommand(text, 'codex') || - isStructuredAgentSessionComposerCommand(text, 'claude') - ) { + if (isHostCommand) { restoreRejectedDraft(origin, text) return 'unknown' } @@ -92,6 +90,7 @@ export function useMobileStructuredNativeChatSendBridge(args: { }, [ acceptSend, + agent, captureSendOrigin, clearDraftForSend, holdUnconfirmedSend, diff --git a/src/renderer/src/components/native-chat/NativeChatComposer.test.tsx b/src/renderer/src/components/native-chat/NativeChatComposer.test.tsx index 45d95140f9e..055230c1a4e 100644 --- a/src/renderer/src/components/native-chat/NativeChatComposer.test.tsx +++ b/src/renderer/src/components/native-chat/NativeChatComposer.test.tsx @@ -306,12 +306,14 @@ describe('NativeChatComposer', () => { expect(mocks.setDraft).toHaveBeenCalledWith('') }) - // The structured menu offers only what the dispatcher can carry out. Listing the - // agent's TUI catalog here answered every pick with "not available in chat sessions". + // The structured menu offers only what a pick can carry out: the host's own + // commands, plus the ones the agent itself runs from message text (Codex `/goal`). + // Listing the agent's whole TUI catalog here answered every pick with + // "not available in chat sessions". it.each([ - ['claude', 'compact'], - ['codex', 'vim'] - ] as const)('offers %s only actionable structured slash commands', (agent, withheld) => { + ['claude', 'compact', ['model', 'effort']], + ['codex', 'vim', ['model', 'effort', 'goal']] + ] as const)('offers %s only actionable structured slash commands', (agent, withheld, offered) => { mocks.draft = '/' render( { const names = (mocks.fieldProps?.autocomplete?.items ?? []) .filter((item) => item.kind === 'command') .map((item) => item.name) - expect(names).toEqual(['model', 'effort']) + expect(names).toEqual([...offered]) expect(names).not.toContain(withheld) }) diff --git a/src/renderer/src/components/native-chat/use-native-chat-composer-catalog.test.tsx b/src/renderer/src/components/native-chat/use-native-chat-composer-catalog.test.tsx index 9e7b3abc5b2..427a56c739f 100644 --- a/src/renderer/src/components/native-chat/use-native-chat-composer-catalog.test.tsx +++ b/src/renderer/src/components/native-chat/use-native-chat-composer-catalog.test.tsx @@ -7,6 +7,7 @@ import { EMPTY_HISTORY } from './native-chat-composer-state' import { useNativeChatComposerCatalog } from './use-native-chat-composer-catalog' import type { NativeChatStructuredComposerTransport } from './native-chat-composer-types' import { getVerifiedNativeChatCommands } from '../../../../shared/native-chat-agent-profiles' +import { sessionSlashCommandSuggestions } from '../../../../shared/native-chat-slash-commands' import { structuredSlashCommands } from '../../../../shared/structured-agent-session-composer' function transport(sessionCommands?: NativeChatStructuredComposerTransport['sessionCommands']) { @@ -47,6 +48,25 @@ describe('composer catalog authority', () => { 'clear' ]) }) + // Codex reports no catalog, so the hook's fallback is its entire `/` menu; the + // agent has to reach structuredSlashCommands or `/goal` is invisible there. + it('offers Codex the commands its model runs from message text', () => { + const { result } = renderHook(() => useNativeChatComposerCatalog('codex', transport())) + expect(result.current.agentCommands).toEqual(structuredSlashCommands([], 'codex')) + expect(result.current.agentCommands.map(({ name }) => name)).toContain('goal') + }) + it('leaves the Claude route on its own commands, reported or not', () => { + const reported = [{ name: 'init', kind: 'command' as const }] + const withReport = renderHook(() => useNativeChatComposerCatalog('claude', transport(reported))) + expect(withReport.result.current.agentCommands).toEqual( + sessionSlashCommandSuggestions('claude', reported) + ) + const withoutReport = renderHook(() => useNativeChatComposerCatalog('claude', transport())) + expect(withoutReport.result.current.agentCommands.map(({ name }) => name)).toEqual([ + 'model', + 'effort' + ]) + }) it('respects empty catalogs and command-only catalogs without reviving disk skills', () => { const { result, rerender } = renderHook( ({ reported }) => useNativeChatComposerCatalog('claude', transport(reported)), diff --git a/src/renderer/src/components/native-chat/use-native-chat-composer-catalog.ts b/src/renderer/src/components/native-chat/use-native-chat-composer-catalog.ts index 273e9fbfa60..d2e0b93ac40 100644 --- a/src/renderer/src/components/native-chat/use-native-chat-composer-catalog.ts +++ b/src/renderer/src/components/native-chat/use-native-chat-composer-catalog.ts @@ -34,7 +34,7 @@ export function useNativeChatComposerCatalog( ? getVerifiedNativeChatCommands(agent) : reported !== undefined ? sessionSlashCommandSuggestions(agent, reported) - : structuredSlashCommands(conversationCommands), + : structuredSlashCommands(conversationCommands, agent), [agent, conversationCommands, reported, structured] ) const sessionSkillNames = useMemo( diff --git a/src/renderer/src/components/native-chat/use-native-chat-structured-composer-send.test.tsx b/src/renderer/src/components/native-chat/use-native-chat-structured-composer-send.test.tsx new file mode 100644 index 00000000000..82cd0f0cb03 --- /dev/null +++ b/src/renderer/src/components/native-chat/use-native-chat-structured-composer-send.test.tsx @@ -0,0 +1,82 @@ +// @vitest-environment happy-dom +import { renderHook } from '@testing-library/react' +import { describe, expect, it, vi } from 'vitest' +import { dispatchStructuredAgentSessionComposerCommand } from '../../../../shared/structured-agent-session-composer' +import type { AgentType } from '../../../../shared/agent-status-types' +import type { NativeChatStructuredComposerTransport } from './native-chat-composer-types' +import type { NativeChatComposerImageAttachment } from './NativeChatComposerField' +import { useNativeChatStructuredComposerSend } from './use-native-chat-structured-composer-send' + +vi.mock('@/lib/native-chat-telemetry', () => ({ emitNativeChatMessageSent: vi.fn() })) +vi.mock('@/lib/worker-terminal-takeover-report', () => ({ + reportStructuredSessionUserInput: vi.fn() +})) + +const ATTACHMENT = { id: 'a1', path: '/tmp/shot.png' } as NativeChatComposerImageAttachment + +function harness(agent: AgentType) { + const structuredTransport = { + send: vi.fn(() => true), + dispatchCommand: (text: string) => + dispatchStructuredAgentSessionComposerCommand(text, { + agent, + snapshot: [], + invokeAction: async () => true, + setOption: async () => true, + conversationCommands: ['clear', 'compact'], + runConversationCommand: async () => ({ accepted: true, error: null }) + }), + optionSnapshot: [], + onError: vi.fn(), + runtime: 'local', + sessionId: 'session-test', + runtimeEnvironmentId: null + } as unknown as NativeChatStructuredComposerTransport + const { result } = renderHook(() => + useNativeChatStructuredComposerSend({ + agent, + draft: '', + imageAttachments: [ATTACHMENT], + structuredTransport, + clearImageAttachments: vi.fn(), + clearSkillOrigin: vi.fn(), + setHistory: vi.fn(), + setDraft: vi.fn(), + setCaret: vi.fn() + }) + ) + return { send: result.current, structuredTransport } +} + +// The guard exists because a host command sends no message, so its attachments +// would be dropped without a word. A pass-through command IS the message, so the +// attachments ride along with it. +describe('attachment guard follows what the host claims', () => { + it.each([ + ['claude', '/clear'], + ['claude', '/model'], + ['codex', '/permissions'] + ] as const)('refuses attachments on the host-claimed %s command %s', (agent, text) => { + const { send, structuredTransport } = harness(agent) + send(text) + expect(structuredTransport.onError).toHaveBeenCalledWith( + 'Remove attachments before using a chat-session command.' + ) + expect(structuredTransport.send).not.toHaveBeenCalled() + }) + + it.each([ + ['claude', '/init'], + ['claude', '/review'], + ['codex', '/goal ship the fix'] + ] as const)('sends %s attachments along with the passed-through %s', async (agent, text) => { + const { send, structuredTransport } = harness(agent) + send(text) + await vi.waitFor(() => + expect(structuredTransport.send).toHaveBeenCalledWith(text, [ATTACHMENT]) + ) + expect(structuredTransport.onError).not.toHaveBeenCalledWith( + 'Remove attachments before using a chat-session command.' + ) + }) +}) diff --git a/src/shared/native-chat-agent-profiles.test.ts b/src/shared/native-chat-agent-profiles.test.ts index 4d64e283c01..546a4bf9db0 100644 --- a/src/shared/native-chat-agent-profiles.test.ts +++ b/src/shared/native-chat-agent-profiles.test.ts @@ -1,5 +1,9 @@ import { describe, expect, it } from 'vitest' -import { getNativeChatAgentProfile } from './native-chat-agent-profiles' +import { + getHostClaimedNativeChatCommands, + getNativeChatAgentProfile, + getVerifiedNativeChatCommands +} from './native-chat-agent-profiles' describe('native chat agent picker profiles', () => { // The composer types the same `/` for every agent; skillPrefix is only the @@ -27,3 +31,28 @@ describe('native chat agent picker profiles', () => { expect(getNativeChatAgentProfile('custom-agent')).toBeNull() }) }) + +describe('host-claimed native chat commands', () => { + function names(agent: string): string[] { + return getHostClaimedNativeChatCommands(agent).map((command) => command.name) + } + + // Claude's harness expands a slash command out of the message body, so claiming + // its catalog only answered "/init is not available" for commands that do run. + it('claims nothing from the Claude-family catalog', () => { + expect(names('claude')).toEqual([]) + expect(names('openclaude')).toEqual([]) + }) + + it('keeps the Codex catalog claimed except the model-driven /goal', () => { + expect(names('codex')).toContain('permissions') + expect(names('codex')).toContain('vim') + expect(names('codex')).not.toContain('goal') + expect(getVerifiedNativeChatCommands('codex').map((command) => command.name)).toContain('goal') + }) + + it('claims the whole catalog for agents with no pass-through policy', () => { + expect(names('custom-agent')).toEqual(['clear', 'help']) + expect(names('grok')).toEqual([]) + }) +}) diff --git a/src/shared/native-chat-agent-profiles.ts b/src/shared/native-chat-agent-profiles.ts index 6f86313d17d..82fe88e9ac3 100644 --- a/src/shared/native-chat-agent-profiles.ts +++ b/src/shared/native-chat-agent-profiles.ts @@ -5,20 +5,31 @@ export type NativeChatAgentProfile = { skillPrefix: '$' | '/' /** OpenClaude reads Claude-owned roots, so this can differ from the agent. */ skillSourceOwner: AgentType + /** The agent's own harness expands a slash command out of the message text, so + * the chat host claims only the commands it implements itself. */ + expandsSlashCommandsFromText?: true + /** Catalog commands the model acts on when they arrive as prose, even though + * the runtime has no slash parser of its own. */ + textDrivenCommands?: readonly string[] } const NATIVE_CHAT_AGENT_PROFILES: Partial> = { codex: { skillPrefix: '$', - skillSourceOwner: 'codex' + skillSourceOwner: 'codex', + // The app-server has no slash parser, but the model owns goal tools and + // calls create_goal itself when `/goal ` reaches it as prose. + textDrivenCommands: ['goal'] }, claude: { skillPrefix: '/', - skillSourceOwner: 'claude' + skillSourceOwner: 'claude', + expandsSlashCommandsFromText: true }, openclaude: { skillPrefix: '/', - skillSourceOwner: 'claude' + skillSourceOwner: 'claude', + expandsSlashCommandsFromText: true }, grok: { skillPrefix: '/', @@ -38,3 +49,34 @@ export function getNativeChatAgentProfile( export function getVerifiedNativeChatCommands(agent: AgentType): readonly SlashCommandSuggestion[] { return agent === 'grok' ? [] : getAgentSlashCommands(agent) } + +/** The mirror of the claimed set: catalog commands this agent acts on when they + * arrive as message text. The picker offers these too, so a command the agent + * implements is discoverable and not merely typable. */ +export function getTextDrivenNativeChatCommands( + agent: AgentType | null | undefined +): readonly SlashCommandSuggestion[] { + if (!agent) { + return [] + } + const names = new Set(getNativeChatAgentProfile(agent)?.textDrivenCommands ?? []) + return names.size === 0 + ? [] + : getVerifiedNativeChatCommands(agent).filter((command) => names.has(command.name)) +} + +/** Catalog commands the chat host answers itself. Whatever is left over reaches + * the agent as ordinary text, which is only correct where the agent implements + * the command — so an agent unclaims a command only via the profile above. + * Claiming stays the default: it is what stops a hand-typed `/clear` from being + * sent to the model as literal prompt text. */ +export function getHostClaimedNativeChatCommands( + agent: AgentType +): readonly SlashCommandSuggestion[] { + const profile = getNativeChatAgentProfile(agent) + if (profile?.expandsSlashCommandsFromText) { + return [] + } + const passedThrough = new Set(profile?.textDrivenCommands ?? []) + return getVerifiedNativeChatCommands(agent).filter((command) => !passedThrough.has(command.name)) +} diff --git a/src/shared/structured-agent-session-composer.test.ts b/src/shared/structured-agent-session-composer.test.ts index ec3b301a8e2..b51de0396be 100644 --- a/src/shared/structured-agent-session-composer.test.ts +++ b/src/shared/structured-agent-session-composer.test.ts @@ -6,15 +6,59 @@ import { } from './structured-agent-session-composer' describe('structuredSlashCommands', () => { - // The composer menu and the dispatcher read this one list. When they disagreed, + const hostController = { + snapshot: [], + invokeAction: async () => true, + setOption: async () => true, + conversationCommands: ['clear', 'compact'] as const, + runConversationCommand: async () => ({ accepted: true, error: null }) + } + const REFUSAL = /is not available in chat sessions/ + + // The composer menu and the dispatcher read the same policy. When they disagreed, // a Claude session was offered Codex-only tokens that missed the command guard - // and reached the model as literal prompt text instead of erroring. - it.each(['codex', 'claude'] as const)('offers %s only commands it also accepts', (agent) => { - const offered = structuredSlashCommands() - expect(offered.length).toBeGreaterThan(0) - for (const command of offered) { - expect(isStructuredAgentSessionComposerCommand(`/${command.name}`, agent)).toBe(true) + // and reached the model as literal prompt text instead of erroring. A row is + // honored either way now: the host answers it, or it passes through to the agent. + it.each(['codex', 'claude'] as const)( + 'offers %s only commands the host answers or the agent runs', + async (agent) => { + const offered = structuredSlashCommands(['clear', 'compact'], agent) + expect(offered.length).toBeGreaterThan(0) + for (const command of offered) { + const outcome = await dispatchStructuredAgentSessionComposerCommand(`/${command.name}`, { + ...hostController, + agent + }) + expect(outcome.error ?? '').not.toMatch(REFUSAL) + } } + ) + + // Codex reports no catalog of its own, so this fallback is its whole `/` menu — + // without the row, a command that now works is impossible to discover. + it('offers Codex the /goal the model acts on, described from the catalog', () => { + const offered = structuredSlashCommands(['clear', 'compact'], 'codex') + expect(offered.map((command) => command.name)).toEqual([ + 'model', + 'effort', + 'clear', + 'compact', + 'goal' + ]) + expect(offered.find((command) => command.name === 'goal')?.description).toBe( + 'Set or view the goal' + ) + // Picking it must reach the model, not the host's refusal. + expect(isStructuredAgentSessionComposerCommand('/goal', 'codex')).toBe(false) + }) + + it('adds nothing for an agent whose own harness expands its commands', () => { + expect(structuredSlashCommands(['clear', 'compact'], 'claude').map((c) => c.name)).toEqual([ + 'model', + 'effort', + 'clear', + 'compact' + ]) }) it('offers only the commands a chat session can carry out', () => { @@ -98,3 +142,68 @@ describe('dispatchStructuredAgentSessionComposerCommand', () => { expect(runConversationCommand).not.toHaveBeenCalled() }) }) + +describe('agent-implemented commands pass through to the agent', () => { + const controller = { + snapshot: [], + invokeAction: async () => true, + setOption: async () => true + } + const PASSED_THROUGH = { handled: false, accepted: false, error: null } + + // Claude's harness runs a slash command it finds in the message text, so + // claiming these answered "not available" for commands that do work. + it.each(['init', 'review', 'help'] as const)( + 'sends /%s on to the Claude harness instead of refusing it', + async (name) => { + expect(isStructuredAgentSessionComposerCommand(`/${name}`, 'claude')).toBe(false) + expect( + await dispatchStructuredAgentSessionComposerCommand(`/${name}`, { + ...controller, + agent: 'claude' + }) + ).toEqual(PASSED_THROUGH) + } + ) + + it.each(['clear', 'compact', 'model', 'effort'] as const)( + 'still claims the host-owned /%s on Claude', + async (name) => { + expect(isStructuredAgentSessionComposerCommand(`/${name}`, 'claude')).toBe(true) + expect( + ( + await dispatchStructuredAgentSessionComposerCommand(`/${name}`, { + ...controller, + agent: 'claude' + }) + ).handled + ).toBe(true) + } + ) + + // Codex's app-server has no slash parser, but the model owns goal tools and + // creates a real goal from `/goal ` arriving as prose. + it('passes /goal through on Codex, arguments and all', async () => { + expect(isStructuredAgentSessionComposerCommand('/goal', 'codex')).toBe(false) + expect( + await dispatchStructuredAgentSessionComposerCommand('/goal ship the fix', { + ...controller, + agent: 'codex' + }) + ).toEqual(PASSED_THROUGH) + }) + + it('keeps refusing a Codex command the model cannot carry out', async () => { + expect(isStructuredAgentSessionComposerCommand('/permissions', 'codex')).toBe(true) + expect( + await dispatchStructuredAgentSessionComposerCommand('/permissions', { + ...controller, + agent: 'codex' + }) + ).toMatchObject({ + handled: true, + error: + '/permissions is not available in chat sessions. Use the slash menu to see available commands.' + }) + }) +}) diff --git a/src/shared/structured-agent-session-composer.ts b/src/shared/structured-agent-session-composer.ts index 70d10c34cfa..093d588effb 100644 --- a/src/shared/structured-agent-session-composer.ts +++ b/src/shared/structured-agent-session-composer.ts @@ -1,4 +1,7 @@ -import { getVerifiedNativeChatCommands } from './native-chat-agent-profiles' +import { + getHostClaimedNativeChatCommands, + getTextDrivenNativeChatCommands +} from './native-chat-agent-profiles' import type { AgentType } from './agent-status-types' import type { SessionOptionDescriptor, SessionOptionValue } from './native-chat-session-options' import type { SlashCommandSuggestion } from './native-chat-slash-commands' @@ -50,30 +53,45 @@ function commandParts(text: string): { name: string; argument: string } | null { return match ? { name: match[1]!.toLowerCase(), argument: match[2]?.trim() ?? '' } : null } -/** The commands the composer menu offers. Strictly what the dispatcher honors, - * so a menu pick is never answered with "not available". */ +/** The commands the composer menu offers when the host reports no catalog of its + * own: the host's own commands, plus the ones this agent acts on from message + * text. Both are honored — the first here, the second by the agent — so a menu + * pick is never answered with "not available". */ export function structuredSlashCommands( - commands: readonly AgentSessionConversationCommand[] = [] + commands: readonly AgentSessionConversationCommand[] = [], + agent?: AgentType | null ): readonly SlashCommandSuggestion[] { - return [ + const hostOwned = [ ...STRUCTURED_AGENT_SESSION_SLASH_COMMANDS, ...CONVERSATION_COMMANDS.filter((entry) => commands.includes(entry.name as AgentSessionConversationCommand) ) ] + // Why: a host with no catalog to report would otherwise hide the commands the + // agent itself implements, e.g. Codex's `/goal`. + return [ + ...hostOwned, + ...getTextDrivenNativeChatCommands(agent).filter( + (entry) => !hostOwned.some((offered) => offered.name === entry.name) + ) + ] } /** Wider than the offered menu on purpose: a TUI-only command still has to be * claimed here and answered, or a hand-typed `/clear` reaches the model as - * literal prompt text. */ + * literal prompt text. Commands the agent itself implements are deliberately + * absent — the profile unclaims those so they pass through as text. */ function structuredRecognizedCommands(agent: AgentType): readonly SlashCommandSuggestion[] { return [ ...STRUCTURED_AGENT_SESSION_SLASH_COMMANDS, ...CONVERSATION_COMMANDS, - ...getVerifiedNativeChatCommands(agent) + ...getHostClaimedNativeChatCommands(agent) ] } +/** Whether the chat host, rather than the agent, owns this command. Callers also + * use it to refuse attachments: a host command sends no message, so attachments + * would be silently dropped, whereas a pass-through command is a real send. */ export function isStructuredAgentSessionComposerCommand( text: string, agent: AgentType = 'codex' From cdf41df37c6c16acae89b06795f0ee5267182632 Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Thu, 10 Sep 2026 21:09:04 -0700 Subject: [PATCH 8/8] perf(terminal): stop rebuilding per-workspace collections on a worktree switch (#19975) Two allocations scale with the workspace count and are rebuilt on inputs that cannot change their result. `workspaceSurfaces` took `renderedActiveWorktreeId` as a memo dep, but `projectWorkspaceSurfaces` reads that id only behind a truthy `activeWorkspaceResolvedHostId` (the folder-collision tie-break), which is null unless the active workspace is itself a folder workspace. Every git-worktree switch therefore re-projected every surface and re-derived the id array to reach an identical answer. Gate the id on the host so the memo holds. The parked-watcher sync built a fresh empty `Set` for every workspace surface, even though only a mounted workspace can park a tab and the sync only reads the set. Share one empty instance for the rest. Both keep every effect firing on exactly the inputs it fired on before. --- ...minal-parked-watcher-sync-entries.test.tsx | 110 ++++++++++++++++++ .../terminal-workspace-surface-ids.test.tsx | 79 +++++++++++++ .../use-terminal-watcher-effects.ts | 14 ++- .../use-terminal-workspace-foundation.ts | 8 +- .../workspace-surface-projection.test.ts | 18 +++ .../workspace-surface-projection.ts | 1 + 6 files changed, 224 insertions(+), 6 deletions(-) create mode 100644 src/renderer/src/components/terminal-parked-watcher-sync-entries.test.tsx diff --git a/src/renderer/src/components/terminal-parked-watcher-sync-entries.test.tsx b/src/renderer/src/components/terminal-parked-watcher-sync-entries.test.tsx new file mode 100644 index 00000000000..e14faedbc18 --- /dev/null +++ b/src/renderer/src/components/terminal-parked-watcher-sync-entries.test.tsx @@ -0,0 +1,110 @@ +// @vitest-environment happy-dom +import { act } from 'react' +import { createRoot, type Root } from 'react-dom/client' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { useTerminalWatcherEffects } from './use-terminal-watcher-effects' +import type { ParkedTerminalTabWatcherSyncEntry } from './terminal-pane/terminal-parked-tab-watchers' +import type { TerminalColdActivationController } from './terminal-cold-activation' + +const mocks = vi.hoisted(() => ({ + sync: vi.fn(), + prune: vi.fn() +})) +vi.mock('@/store', () => ({ + useAppStore: Object.assign(() => 'unverifiable', { + getState: () => ({ activeWorktreeId: null }) + }) +})) +vi.mock('@/lib/workspace-terminal-host-authority', () => ({ + createWorkspaceTerminalHostAuthoritySelector: () => () => 'unverifiable' +})) +vi.mock('./terminal-pane/terminal-parked-tab-watchers', () => ({ + canWatcherCoverParkedTerminalTab: () => true, + disposeAllParkedTerminalWatchers: vi.fn(), + pruneParkedTerminalWatchers: mocks.prune, + syncParkedTerminalTabWatchersForWorkspaces: mocks.sync, + terminalWatcherLiveWorkspaceIds: (ids: Iterable) => new Set(ids) +})) +;(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true + +const SURFACE_COUNT = 423 +const PARKED_WORKTREE_ID = 'repo-1::/worktree-0' +const surfaceIds = Array.from({ length: SURFACE_COUNT }, (_, index) => `repo-1::/worktree-${index}`) + +let root: Root | undefined +afterEach(async () => { + await act(async () => root?.unmount()) + vi.clearAllMocks() +}) + +function renderWatcherEffects(): Promise { + function Watcher(): null { + useTerminalWatcherEffects({ + activationDeferredMountTabIdsByWorktreeRef: { current: new Map() }, + activeTabId: null, + activeTabIdByWorktree: {}, + activeView: 'terminal', + activeWorktreeId: null, + activityTerminalPortals: [], + anyMountedWorktreeHasLayout: false, + backgroundMountRevision: 0, + effectiveParkedTerminalWorktreeIds: new Set([PARKED_WORKTREE_ID]), + evictionExemptTerminalTabIds: new Set(['tab-exempt']), + getEffectiveLayoutForWorktree: () => null, + groupsByWorktree: {}, + hydrationSucceeded: false, + measurableBackgroundWorktreeIdsRef: { current: new Set() }, + mountedWorktreeIdsRef: { current: new Set([PARKED_WORKTREE_ID]) }, + pendingStartupByTabId: {}, + // Another workspace is on screen, so the mounted one is hidden and parks. + renderedActiveWorktreeId: 'repo-1::/worktree-9', + tabsByWorktree: { + [PARKED_WORKTREE_ID]: [{ id: 'tab-parked' }, { id: 'tab-exempt' }] + }, + terminalParkingEnabled: true, + terminalStartupRestorationReady: false, + terminalTitleSnapshotAuthorityEnabled: true, + workspaceSessionReady: false, + workspaceSurfaceIds: surfaceIds + } as unknown as TerminalColdActivationController) + return null + } + root = createRoot(document.createElement('div')) + return act(async () => root?.render()) +} + +function lastSyncEntries(): Map { + return mocks.sync.mock.calls.at(-1)?.[0] as Map +} + +describe('parked terminal watcher sync entries', () => { + it('publishes an entry for every surface so closed-tab disposal still sees it', async () => { + await renderWatcherEffects() + + const entries = lastSyncEntries() + expect(entries.size).toBe(SURFACE_COUNT) + expect([...entries.keys()]).toEqual(surfaceIds) + expect(mocks.prune).toHaveBeenCalledWith(new Set(surfaceIds)) + }) + + it('parks the hidden mounted workspace tabs and exempts the eviction-exempt tab', async () => { + await renderWatcherEffects() + + const parkedEntry = lastSyncEntries().get(PARKED_WORKTREE_ID) + expect([...(parkedEntry?.parkedTabIds ?? [])]).toEqual(['tab-parked']) + }) + + it('does not allocate a parked-tab-id set per unmounted surface', async () => { + await renderWatcherEffects() + + const entries = lastSyncEntries() + const unmountedSets = new Set( + [...entries] + .filter(([workspaceId]) => workspaceId !== PARKED_WORKTREE_ID) + .map(([, entry]) => entry.parkedTabIds) + ) + // Pre-fix this was one empty Set per surface (422 of them) on every fire. + expect(unmountedSets.size).toBe(1) + expect([...unmountedSets][0]?.size).toBe(0) + }) +}) diff --git a/src/renderer/src/components/terminal-workspace-surface-ids.test.tsx b/src/renderer/src/components/terminal-workspace-surface-ids.test.tsx index 770b1cfc47b..59984d7fe06 100644 --- a/src/renderer/src/components/terminal-workspace-surface-ids.test.tsx +++ b/src/renderer/src/components/terminal-workspace-surface-ids.test.tsx @@ -8,6 +8,7 @@ import { makeRepo, makeWorktree } from './worktree-jump-palette-test-fixtures' import { useTerminalWorkspaceFoundation } from './use-terminal-workspace-foundation' import { applyTerminalColdActivation } from './terminal-cold-activation' import { collectTerminalParkingPassCandidates } from './terminal-parking-pass-candidates' +import type { FolderWorkspace } from '../../../shared/folder-workspace-types' import type { WorkspaceSurface } from './workspace-surface-projection' import type { TerminalParkingFoundation } from './use-terminal-parking-foundation' @@ -147,6 +148,84 @@ describe('workspace surface ids', () => { ]) expect(hiddenSince.has('repo::/stale')).toBe(false) }) + it('keeps the surface projection stable across a git-worktree switch', () => { + const repo = makeRepo() + const worktrees = Array.from({ length: 423 }, (_, index) => + makeWorktree(`repo-1::/worktree-${index}`, `Workspace ${index}`) + ) + useAppStore.setState({ + worktreesByRepo: { [repo.id]: worktrees }, + activeWorktreeId: worktrees[0].id + }) + + const { result } = renderHook(() => useTerminalWorkspaceFoundation()) + const surfaces = result.current.workspaceSurfaces + const ids = result.current.workspaceSurfaceIds + const idSet = result.current.workspaceSurfaceIdSet + expect(surfaces).toHaveLength(423) + + // Pre-fix every switch re-ran the projection (423 surface objects) and re-derived + // the id array, because the active id was a dep even with no folder host to tie-break. + for (let index = 1; index < 10; index += 1) { + act(() => { + useAppStore.setState({ activeWorktreeId: worktrees[index].id }) + }) + expect(result.current.renderedActiveWorktreeId).toBe(worktrees[index].id) + expect(result.current.workspaceSurfaces).toBe(surfaces) + expect(result.current.workspaceSurfaceIds).toBe(ids) + expect(result.current.workspaceSurfaceIdSet).toBe(idSet) + } + }) + + it('still re-projects when the active folder workspace owns the collision tie-break', () => { + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) + const localFolder: FolderWorkspace = { + id: 'folder-shared', + projectGroupId: 'group-shared', + name: 'orca', + folderPath: '/work/orca-local', + connectionId: null, + executionHostId: 'local', + linkedTask: null, + comment: '', + isArchived: false, + isUnread: false, + isPinned: false, + sortOrder: 0, + lastActivityAt: 1, + createdAt: 1, + updatedAt: 1 + } + const runtimeFolder: FolderWorkspace = { + ...localFolder, + folderPath: '/remote/orca', + executionHostId: 'runtime:env-1' + } + useAppStore.setState({ + folderWorkspaces: [localFolder, runtimeFolder], + activeWorktreeId: 'folder:folder-shared', + activeWorkspaceExecutionHostId: 'runtime:env-1' + }) + + const { result } = renderHook(() => useTerminalWorkspaceFoundation()) + expect(result.current.workspaceSurfaces).toEqual([ + { id: 'folder:folder-shared', path: '/remote/orca' } + ]) + + // Leaving the folder workspace drops the tie-break, so the projection must re-run + // and fall back to first-wins rather than mount the previous host's path. + act(() => { + useAppStore.setState({ + activeWorktreeId: 'repo-1::/worktree-0', + activeWorkspaceExecutionHostId: null + }) + }) + expect(result.current.workspaceSurfaces).toEqual([ + { id: 'folder:folder-shared', path: '/work/orca-local' } + ]) + warn.mockRestore() + }) + it('stops idle worktree writes from re-firing surface-keyed terminal effects', () => { const repo = makeRepo() const worktrees = Array.from({ length: 20 }, (_, index) => diff --git a/src/renderer/src/components/use-terminal-watcher-effects.ts b/src/renderer/src/components/use-terminal-watcher-effects.ts index 3c6a89fb323..fc44352c942 100644 --- a/src/renderer/src/components/use-terminal-watcher-effects.ts +++ b/src/renderer/src/components/use-terminal-watcher-effects.ts @@ -17,6 +17,10 @@ import { getStructuredAgentLaunchStatus } from '@/lib/structured-agent-session-l import { AGENT_SESSION_PROVIDER_HANDLE_PROVIDERS } from '../../../shared/agent-session-provider-handle' import type { TerminalColdActivationController } from './terminal-cold-activation' +// Why shared: only a mounted workspace can park a tab, and the watcher sync only reads +// this set, so every other surface would otherwise allocate its own empty one per fire. +const NO_PARKED_TAB_IDS: ReadonlySet = new Set() + export function useTerminalWatcherEffects(controller: TerminalColdActivationController): void { const { activationDeferredMountTabIdsByWorktreeRef, @@ -58,9 +62,11 @@ export function useTerminalWatcherEffects(controller: TerminalColdActivationCont continue } const tabs = tabsByWorktree[workspaceId] ?? [] - const parkedTabIds = new Set() + let parkedTabIds: ReadonlySet = NO_PARKED_TAB_IDS let deferredTabIds: ReadonlySet | null = null if (!anyMountedWorktreeHasLayout && mountedWorktreeIdsRef.current.has(workspaceId)) { + const mountedParkedTabIds = new Set() + parkedTabIds = mountedParkedTabIds const isVisible = activeView === 'terminal' && workspaceId === renderedActiveWorktreeId const shouldMeasureHiddenWorktree = !isVisible && measurableBackgroundWorktreeIdsRef.current.has(workspaceId) @@ -75,7 +81,7 @@ export function useTerminalWatcherEffects(controller: TerminalColdActivationCont tabId: tab.id }) if (!activityTerminalPortal && !evictionExemptTerminalTabIds.has(tab.id)) { - parkedTabIds.add(tab.id) + mountedParkedTabIds.add(tab.id) } } } @@ -83,14 +89,14 @@ export function useTerminalWatcherEffects(controller: TerminalColdActivationCont for (const tab of tabs) { if ( deferredTabIds?.has(tab.id) && - !parkedTabIds.has(tab.id) && + !mountedParkedTabIds.has(tab.id) && canWatcherCoverParkedTerminalTab(workspaceId, tab) && !findActivityTerminalPortal(activityTerminalPortals, { worktreeId: workspaceId, tabId: tab.id }) ) { - parkedTabIds.add(tab.id) + mountedParkedTabIds.add(tab.id) } } } diff --git a/src/renderer/src/components/use-terminal-workspace-foundation.ts b/src/renderer/src/components/use-terminal-workspace-foundation.ts index 61726fa9643..d68c5104da2 100644 --- a/src/renderer/src/components/use-terminal-workspace-foundation.ts +++ b/src/renderer/src/components/use-terminal-workspace-foundation.ts @@ -37,15 +37,19 @@ export function useTerminalWorkspaceFoundation() { parseWorkspaceKey(renderedActiveWorktreeId ?? '')?.type === 'folder' ? activeWorktreeDeferralHostId : null + // Why gate the id on the host: the projection reads `activeWorkspaceId` only behind a + // truthy resolved host, so without one every active id yields the same surfaces — and a + // worktree switch must not re-project (and re-identify) every surface to rediscover that. + const activeFolderSurfaceId = activeFolderSurfaceHostId ? renderedActiveWorktreeId : null const workspaceSurfaces = useMemo( () => projectWorkspaceSurfaces({ worktreesById, folderWorkspaces, - activeWorkspaceId: renderedActiveWorktreeId, + activeWorkspaceId: activeFolderSurfaceId, activeWorkspaceResolvedHostId: activeFolderSurfaceHostId }), - [worktreesById, folderWorkspaces, renderedActiveWorktreeId, activeFolderSurfaceHostId] + [worktreesById, folderWorkspaces, activeFolderSurfaceId, activeFolderSurfaceHostId] ) // Why split the ids out: every mount/park/activation pass reads only `.id`, but // the surface array is re-identified on any worktree write. Reusing the previous diff --git a/src/renderer/src/components/workspace-surface-projection.test.ts b/src/renderer/src/components/workspace-surface-projection.test.ts index d5226bc5662..7f7c1709ca9 100644 --- a/src/renderer/src/components/workspace-surface-projection.test.ts +++ b/src/renderer/src/components/workspace-surface-projection.test.ts @@ -118,6 +118,24 @@ describe('projectWorkspaceSurfaces', () => { expect(surfaces).toEqual([{ id: 'folder:folder-shared', path: '/remote/orca' }]) }) + it('ignores the active workspace id entirely when no resolved host disambiguates', () => { + // The terminal foundation memo relies on this: with no resolved folder host it passes + // `null` instead of the active id, so a worktree switch cannot change the projection. + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) + const folderWorkspaces = [localFolder, runtimeFolder] + const worktrees = [localWorktree] + + const withoutActiveId = project({ worktrees, folderWorkspaces, activeWorkspaceId: null }) + for (const activeWorkspaceId of [ + 'folder:folder-shared', + 'folder:other-workspace', + SHARED_WORKTREE_ID + ]) { + expect(project({ worktrees, folderWorkspaces, activeWorkspaceId })).toEqual(withoutActiveId) + } + warn.mockRestore() + }) + it('keeps the first row when no resolved host disambiguates the folder collision', () => { const surfaces = project({ folderWorkspaces: [runtimeFolder, localFolder] diff --git a/src/renderer/src/components/workspace-surface-projection.ts b/src/renderer/src/components/workspace-surface-projection.ts index 4a3d28ea17f..9b0a5731be2 100644 --- a/src/renderer/src/components/workspace-surface-projection.ts +++ b/src/renderer/src/components/workspace-surface-projection.ts @@ -49,6 +49,7 @@ export function projectWorkspaceSurfaces({ }: { worktreesById: ReadonlyMap> folderWorkspaces: readonly FolderWorkspaceSurfaceRow[] + /** Read only when `activeWorkspaceResolvedHostId` is set; inert otherwise. */ activeWorkspaceId: string | null /** Resolved (not user-selected) host of the active workspace; the folder tie-break. */ activeWorkspaceResolvedHostId: ExecutionHostId | null