refactor(ai-vault): split the session scanner into a transcript reader and consumers (#19666)

* refactor(ai-vault): split the session scanner into a transcript reader and consumers

The scanner's only output was the Session History summary; a second reader
of the same transcripts (a search index) had nowhere to plug in without
hooking the parse itself. Extract a reader that owns each file read, keeps
the resumable cursor and publishes every decoded message to registered
consumers. The session list stays a fold inside the parser and is the only
consumer here. Parsers take an optional message sink instead of a scope.

Also: read Cursor chats/<md5>/<uuid>/meta.json for cwd, title and
timestamps (Cursor transcripts carry only role and message); share the
lazily spawned worker-thread host between the OpenCode SQLite reader and
the port-scan probe; probe OpenCode's schema before querying; keep the
newest-N discovery set with a bounded insert instead of sort+slice.

Session list output is byte-identical to main across all 18 providers
cold and append-resumed; the one Cursor session gains cwd/timestamps
from meta.json.

* fix(ai-vault): serialize per-path parses and report unpublished reads

Overlapping parses of one transcript share the cached resume point's
message channel, so the second beginRead dropped the first read's
consumers and the first finishRead handed them the wrong outcome. Two
callers really do overlap: a forced refresh restarts a scan while the
aborted scan's parse is still in flight, and the title reader parses
outside any scan. Restore the per-path lane around the whole
lookup-read-store sequence.

OpenCode's SQLite sessions are decoded on a worker thread the channel
cannot reach, so their reads published no messages while reporting a
complete span. Finish those reads as incomplete instead, so a consumer
never records a cursor for a stream it did not receive.

* fix(ai-vault): degrade a refused cursor chats read instead of dropping sessions

A refused WSL read of Cursor's chats tree rethrew, and the per-file catch
in discovery then recorded an issue and skipped the transcript. Before
the meta.json join Cursor had no content dependency, so a stalled distro
could not hide a Cursor session at all. Degrade to no metadata for the
scan and report the chats root once. The parse cache stays honest without
the throw: discovery stats no meta.json on a refused scan, so the entry's
recorded size omits it and the next healthy scan re-reads the transcript.

The per-scan index scope covered discovery only, so every Cursor finalize
re-read the chats root to validate the module cache. Move the scope to
scanAiVaultSessions, which spans discovery and parse.

Also drop the unused signal parameters the sink threading added to the
Devin and Hermes content parsers, by giving each file parser a private
record parser instead.

* fix(ai-vault): do not cache a cursor parse whose meta.json read was refused

Discovery stats meta.json into the candidate's cache key, so when only
the meta.json read is refused the un-enriched session was stored under a
key that looks unchanged and reuseCachedSession never re-ran the enrich
hook. The session stayed without cwd until Cursor rewrote the file. The
enrich hook now reports 'refused', the resumable state exposes
isCacheable, and the parse cache drops the entry instead of storing it,
so the next healthy scan re-parses. The index-read branch is unaffected:
it never stats meta.json, so its key is honest already.

* fix(ai-vault): separate the transcript's size from its cache key

sizeBytes folds a content dependency's size in, so it is a cache key
rather than a file length. The reader compared a transcript byte offset
against it and reported it as a whole-file read offset, which for Cline
handed consumers an offset past the end of the file it read. Carry the
dependency's own size on FileWithMtime and subtract it in the reader.

A refused sibling stat rethrew, so discovery recorded an issue and
skipped the transcript, the same drop removed for the readdir and read
paths. Degrade to no dependency, note the tree once, and mark the key
untrustworthy.

An untrustworthy key no longer costs the resume cursor: the entry is
stored under an mtime no stat can produce, so unchanged is false while
the resume point survives and the next scan resumes instead of re-reading
the whole transcript.

* test(ai-vault): pin the untrustworthy-key mechanism, not just its effect

Both refusal tests asserted that a later healthy scan re-enriches, which
a plain store would also satisfy once the resume cursor was preserved.
Assert the cache entry directly: its mtime is the unmatchable sentinel
and its resume point survives. The sentinel is exported so the tests name
the contract instead of repeating -1.

* refactor(ai-vault): track a session's sidecar file apart from its transcript

Folding Cursor's meta.json stat into the transcript's mtime/size made one
key mean two things, and every round of review found another consequence:
a byte offset could not be compared against it, a refused sibling read
took the transcript down with it, and an un-enriched parse cached under
it looked current forever. Main already had the answer for a file the
transcript key cannot see: Codex titles are refreshed at reuse time over
the cached session, not folded into the key.

Discovery now records the sibling as its own observation, unknown when it
could not be read. A cache hit needs both the transcript key and the
sidecar to match. When only the sidecar moved, Cursor re-merges it over
the stored un-enriched fold result and never re-reads the transcript;
Cline, which reads its sibling as part of the parse, re-parses.

Merging over the fold result rather than the accumulator makes enrichment
pure, so a meta.json rewritten with a new cwd replaces the old one
instead of losing to it. That was unreachable while the merge used ??= on
a session it had already enriched.

Cline and the remote scanner move to the same field, so the fold is gone
from both discovery paths.

* fix(ai-vault): tell an absent sidecar from an unreadable one

Three places collapsed the two. sidecarUnchanged returned true for any
observed 'none' without reading the entry, so a sidecar that was deleted,
or one that was unreadable last scan, both read as cache hits. Native
discovery mapped every non-WSL stat failure to 'none', so an EACCES on
meta.json left a session enriched from a file nobody can see, with no
scan issue. Remote discovery could not tell a missing sibling from a
failed stat, because statRemoteSessionFile returns null for both.

'none' is now a claim: absent-now is a hit only when it was absent before
or the agent never had a sidecar, and only ENOENT/ENOTDIR reads as
absent. statRemoteSessionFile grows an opt-in rethrow so its caller can
distinguish the two failures it already reports.

Also rewrites three comments in the cursor chat-meta reader that still
described the deleted fold.
This commit is contained in:
Jinwoo Hong
2026-09-09 03:36:41 -04:00
committed by GitHub
parent acd501486d
commit 65631e449a
52 changed files with 3265 additions and 744 deletions
+12 -2
View File
@@ -13,7 +13,13 @@ export async function statRemoteSessionFile(
agent: AiVaultAgent,
executionHostId: ExecutionHostId,
issues: AiVaultScanIssue[],
options?: { missingIsExpected?: boolean; signal?: AbortSignal }
options?: {
missingIsExpected?: boolean
signal?: AbortSignal
// Lets a caller tell a missing path from a failed stat, which both report
// as null; the issue is recorded either way before this rethrows.
rethrowFailures?: boolean
}
): Promise<FileWithMtime | null> {
try {
throwIfAiVaultScanCancelled(options?.signal)
@@ -31,7 +37,8 @@ export async function statRemoteSessionFile(
}
} catch (error) {
throwIfAiVaultScanCancelled(options?.signal)
if (!options?.missingIsExpected || !isMissingRemoteSessionPathError(error)) {
const missing = isMissingRemoteSessionPathError(error)
if (!options?.missingIsExpected || !missing) {
recordSessionScanIssue(issues, {
executionHostId,
agent,
@@ -39,6 +46,9 @@ export async function statRemoteSessionFile(
message: errorMessage(error)
})
}
if (options?.rethrowFailures && !missing) {
throw error
}
return null
}
}
@@ -1,5 +1,6 @@
import type { AiVaultSession } from '../../shared/ai-vault-types'
import type { RemoteScannerContext, RemoteSessionCandidate } from './remote-session-scanner-types'
import { sidecarUnchanged, type SessionSidecarObservation } from './session-sidecar-stat'
// Matches the local scanner's cap. The relay sidecar is forked with
// --max-old-space-size=384, and a retained session row is a title, a preview
@@ -11,6 +12,7 @@ type RemoteSessionParseCacheEntry = {
mtimeMs: number
sizeBytes: number | null
hostKey: string
sidecar?: SessionSidecarObservation
session: AiVaultSession | null
}
@@ -55,13 +57,15 @@ function storeEntry(path: string, entry: RemoteSessionParseCacheEntry): void {
* (#13753). The local scanner has had `parseAgentSessionFileCached` for exactly
* this reason; this is its remote counterpart.
*
* `(mtimeMs, sizeBytes)` is a sound validity key here because discovery already
* folds a source's `contentDependencyPath` stat into both fields
* (remote-session-scanner-discovery.ts), so a metadata-only transcript whose
* companion file changed still looks changed. Sources whose parse reads a file
* discovery does not stat — Codex looks its title up in `session_index.jsonl` —
* are not covered by that key and pass `refreshReusedSession` to re-derive the
* uncovered part without touching the transcript.
* `(mtimeMs, sizeBytes)` covers the transcript, and the sidecar observation
* discovery records beside it (remote-session-scanner-discovery.ts) covers a
* source's companion file, so a metadata-only transcript whose companion
* changed still looks changed. Remote Cline is the only such source; remote
* Cursor streams transcript content with no sibling to read. Sources whose
* parse reads a file discovery does not stat — Codex looks its title up in
* `session_index.jsonl` — are not covered by either and pass
* `refreshReusedSession` to re-derive the uncovered part without touching the
* transcript.
*
* Only a completed parse is stored. A read that threw stays uncached so a
* transient filesystem failure cannot pin a wrong answer for the corpus's life.
@@ -80,7 +84,10 @@ export async function parseRemoteSessionFileCached(args: {
entry !== undefined &&
entry.hostKey === args.hostKey &&
entry.mtimeMs === file.mtimeMs &&
(entry.sizeBytes === null || file.sizeBytes === undefined || entry.sizeBytes === file.sizeBytes)
(entry.sizeBytes === null ||
file.sizeBytes === undefined ||
entry.sizeBytes === file.sizeBytes) &&
sidecarUnchanged(entry.sidecar, file.sidecar)
if (unchanged) {
if (args.stats) {
args.stats.reused++
@@ -101,6 +108,7 @@ export async function parseRemoteSessionFileCached(args: {
mtimeMs: file.mtimeMs,
sizeBytes: file.sizeBytes ?? null,
hostKey: args.hostKey,
sidecar: file.sidecar,
session
})
return session
@@ -4,6 +4,7 @@ import type { ExecutionHostId } from '../../shared/execution-host'
import { joinRemotePath } from '../ssh/ssh-remote-platform'
import { isMissingRemoteSessionPathError, statRemoteSessionFile } from './remote-session-file-stat'
import type { FileWithMtime } from './session-scanner-types'
import type { SessionSidecarObservation } from './session-sidecar-stat'
import { errorMessage } from './session-scanner-values'
import { mapRemoteScanBatches } from './remote-session-scan-batching'
import { throwIfAiVaultScanCancelled } from './ai-vault-scan-cancellation'
@@ -61,23 +62,38 @@ async function statRemoteCandidateFile(
if (!file || !source.contentDependencyPath) {
return file
}
const dependency = await statRemoteSessionFile(
context.provider,
source.contentDependencyPath(path),
source.agent,
context.executionHostId,
issues,
{ missingIsExpected: true, signal: context.signal }
)
if (!dependency) {
return file
}
const mtimeMs = Math.max(file.mtimeMs, dependency.mtimeMs)
return {
...file,
mtimeMs,
modifiedAt: new Date(mtimeMs).toISOString(),
sizeBytes: (file.sizeBytes ?? 0) + (dependency.sizeBytes ?? 0)
const sidecarPath = source.contentDependencyPath(path)
// Recorded beside the transcript's own stat, never folded into it: one key
// cannot mean both "the transcript grew" and "the sibling changed".
return { ...file, sidecar: await observeRemoteSidecar(source, context, sidecarPath, issues) }
}
/**
* A stat that failed for any reason other than a missing path is `'unknown'`,
* not `'none'`: serving the cached session over an unreadable sibling would
* publish metadata nobody can currently see. `statRemoteSessionFile` already
* recorded the issue for the failure.
*/
async function observeRemoteSidecar(
source: RemoteSessionSource,
context: RemoteScannerContext,
sidecarPath: string,
issues: AiVaultScanIssue[]
): Promise<SessionSidecarObservation> {
try {
const sidecar = await statRemoteSessionFile(
context.provider,
sidecarPath,
source.agent,
context.executionHostId,
issues,
{ missingIsExpected: true, signal: context.signal, rethrowFailures: true }
)
return sidecar
? { path: sidecarPath, mtimeMs: sidecar.mtimeMs, sizeBytes: sidecar.sizeBytes ?? 0 }
: 'none'
} catch {
return 'unknown'
}
}
@@ -0,0 +1,84 @@
import { describe, expect, it } from 'vitest'
import { getRemoteHostPlatform } from '../ssh/ssh-remote-platform'
import { scanRemoteAiVaultSessions } from './remote-session-scanner'
import { MemoryRemoteProvider } from './remote-session-scanner-test-fixtures'
import { resetRemoteSessionParseCacheForTests } from './remote-session-parse-cache'
describe('remote sidecar observations', () => {
const sessionId = '1786466194549_sidecar'
const sessionDir = `/home/ada/.cline/data/sessions/${sessionId}`
const messagesPath = `${sessionDir}/${sessionId}.messages.json`
function addCline(provider: MemoryRemoteProvider, firstPrompt: string): void {
provider.addFile(
`${sessionDir}/${sessionId}.json`,
JSON.stringify({
version: 1,
session_id: sessionId,
started_at: '2026-08-11T16:36:34.551Z',
cwd: '/home/ada/repo'
}),
10
)
provider.addFile(
messagesPath,
JSON.stringify({
version: 1,
updated_at: '2026-08-11T16:38:00.000Z',
sessionId,
messages: [{ role: 'user', content: [{ type: 'text', text: firstPrompt }] }]
}),
11
)
}
const scan = (provider: MemoryRemoteProvider): ReturnType<typeof scanRemoteAiVaultSessions> =>
scanRemoteAiVaultSessions({
provider,
executionHostId: 'ssh:dev-box',
remoteHome: '/home/ada',
hostPlatform: getRemoteHostPlatform('linux-x64')
})
it('re-parses when a messages-file stat fails rather than serving the cached row', async () => {
resetRemoteSessionParseCacheForTests()
const provider = new MemoryRemoteProvider()
addCline(provider, 'first prompt')
expect((await scan(provider)).sessions[0]).toMatchObject({ title: 'first prompt' })
// The sidecar changed underneath, and its stat now fails for a reason that
// is not "missing": nothing about it may be assumed, so the row is re-read.
addCline(provider, 'second prompt')
provider.failStat(
messagesPath,
Object.assign(new Error('permission denied'), { code: 'EACCES' })
)
const refused = await scan(provider)
expect(refused.sessions[0]).toMatchObject({ title: 'second prompt' })
expect(refused.issues.map((issue) => issue.path)).toContain(messagesPath)
})
it('treats a genuinely missing messages file as no sidecar, not as unknown', async () => {
resetRemoteSessionParseCacheForTests()
const provider = new MemoryRemoteProvider()
provider.addFile(
`${sessionDir}/${sessionId}.json`,
JSON.stringify({
version: 1,
session_id: sessionId,
started_at: '2026-08-11T16:36:34.551Z',
cwd: '/home/ada/repo'
}),
10
)
const first = await scan(provider)
const second = await scan(provider)
expect(first.issues).toEqual([])
expect(second.issues).toEqual([])
expect(second.sessions[0]?.sessionId).toBe(sessionId)
})
})
@@ -0,0 +1,31 @@
import { expect, it } from 'vitest'
import { SessionNewestFiles } from './session-newest-files'
import type { FileWithMtime } from './session-scanner-types'
function file(i: number): FileWithMtime {
const mtimeMs = (i * 7919) % 997
return { path: String(i), mtimeMs, modifiedAt: new Date(mtimeMs).toISOString() }
}
it('retains at most 12 of 100,000 candidates with stable newest-first ties', () => {
const all = Array.from({ length: 100_000 }, (_, i) => file(i))
const retained = new SessionNewestFiles(12)
let peak = 0
for (const candidate of all) {
retained.add(candidate)
peak = Math.max(peak, retained.size)
}
expect(peak).toBe(12)
expect(retained.newest()).toEqual(all.sort((a, b) => b.mtimeMs - a.mtimeMs).slice(0, 12))
})
it('supports full backfill and empty requests', () => {
const all = new SessionNewestFiles(Infinity)
const none = new SessionNewestFiles(0)
for (let i = 0; i < 100; i++) {
all.add(file(i))
none.add(file(i))
}
expect(all.newest()).toHaveLength(100)
expect(none.newest()).toEqual([])
})
+50
View File
@@ -0,0 +1,50 @@
import type { FileWithMtime } from './session-scanner-types'
/** Retain only the requested newest files, preserving traversal order on ties. */
export class SessionNewestFiles {
private readonly files: FileWithMtime[] = []
private readonly limit: number
constructor(limit: number) {
this.limit = Math.max(0, Math.trunc(limit) || 0)
}
add(file: FileWithMtime): void {
// The backfill enumerates with no limit; skip the insert search entirely.
if (!Number.isFinite(this.limit)) {
this.files.push(file)
return
}
if (this.limit <= 0) {
return
}
const last = this.files.at(-1)
if (this.files.length >= this.limit && last && file.mtimeMs <= last.mtimeMs) {
return
}
let low = 0
let high = this.files.length
while (low < high) {
const middle = (low + high) >>> 1
if (this.files[middle].mtimeMs >= file.mtimeMs) {
low = middle + 1
} else {
high = middle
}
}
this.files.splice(low, 0, file)
if (this.files.length > this.limit) {
this.files.pop()
}
}
get size(): number {
return this.files.length
}
/** The unbounded path appends in traversal order, so the sort is not redundant. */
newest(): FileWithMtime[] {
return [...this.files].sort((a, b) => b.mtimeMs - a.mtimeMs)
}
}
@@ -29,9 +29,12 @@ import {
parseAgentSessionFileCached,
resetSessionParseCacheForTests,
seedSessionParseCache,
snapshotSessionParseCacheForPersistence,
type PersistedSessionParseCacheEntry,
type SessionParseStats
} from './session-scanner-parse-cache'
import { getSessionParseCacheEntry } from './session-parse-cache-store'
import type { SessionSidecarObservation } from './session-sidecar-stat'
import { isolatedScanRoots } from './session-scanner-test-fixtures'
import { parseClaudeSessionFile } from './session-scanner-primary-parsers'
import type { FileWithMtime, SessionFileCandidate } from './session-scanner-types'
@@ -526,3 +529,38 @@ describe('session parse cache persistence', () => {
debugSpy.mockRestore()
})
})
describe('sidecar observations survive the round trip', () => {
const OBSERVATIONS: [string, SessionSidecarObservation | undefined][] = [
['an object', { path: '/chats/a/meta.json', mtimeMs: 42, sizeBytes: 7 }],
['none', 'none'],
['unknown', 'unknown'],
['absent', undefined]
]
it.each(OBSERVATIONS)('restores %s exactly', async (_label, sidecar) => {
const root = await makeTempDir()
const cacheFile = join(root, 'session-parse-cache.json')
const path = await writeTranscript(root)
initSessionParseCachePersistence({ filePath: cacheFile, appVersion: APP_VERSION })
await ensureSessionParseCacheLoaded()
const stats = createSessionParseStats()
await parseAgentSessionFileCached(await claudeCandidate(path), process.platform, stats)
const seeded = snapshotSessionParseCacheForPersistence().map(
([entryPath, entry]): [string, PersistedSessionParseCacheEntry] => [
entryPath,
sidecar === undefined ? entry : { ...entry, sidecar }
]
)
resetSessionParseCacheForTests()
seedSessionParseCache(seeded)
scheduleSessionParseCachePersist(stats)
await flushSessionParseCachePersistForTests()
simulateRestart(cacheFile)
await ensureSessionParseCacheLoaded()
expect(getSessionParseCacheEntry(path)?.sidecar).toEqual(sidecar)
})
})
@@ -11,6 +11,7 @@ import {
type PersistedSessionParseCacheEntry,
type SessionParseStats
} from './session-scanner-parse-cache'
import type { SessionSidecarObservation } from './session-sidecar-stat'
// Bump when the persisted entry layout or cached session semantics change; a
// mismatched file is discarded whole.
@@ -181,17 +182,37 @@ function parsePersistedEntry(item: unknown): [string, PersistedSessionParseCache
if (entry.session !== null && typeof entry.session !== 'object') {
return null
}
const sidecar = parsePersistedSidecar(entry.sidecar)
return [
path,
{
mtimeMs: entry.mtimeMs,
sizeBytes: entry.sizeBytes,
platform: entry.platform as NodeJS.Platform,
session: entry.session as PersistedSessionParseCacheEntry['session']
session: entry.session as PersistedSessionParseCacheEntry['session'],
...(sidecar === undefined ? {} : { sidecar })
}
]
}
// Why: added after SCHEMA_VERSION 2 shipped, so a file an older build wrote has
// no such field. Absent (or unreadable) means unknown, which costs one re-parse
// of the rows that have a sibling and nothing at all for the rest.
function parsePersistedSidecar(value: unknown): SessionSidecarObservation | undefined {
if (value === 'none' || value === 'unknown') {
return value
}
if (typeof value !== 'object' || value === null) {
return undefined
}
const record = value as Record<string, unknown>
return typeof record.path === 'string' &&
typeof record.mtimeMs === 'number' &&
typeof record.sizeBytes === 'number'
? { path: record.path, mtimeMs: record.mtimeMs, sizeBytes: record.sizeBytes }
: undefined
}
async function persistSnapshot(current: SessionParseCachePersistenceOptions): Promise<void> {
const directory = dirname(current.filePath)
const tempPath = join(directory, `session-parse-cache-${process.pid}-${Date.now()}.tmp`)
@@ -0,0 +1,111 @@
import type { AiVaultSession } from '../../shared/ai-vault-types'
import type { ResumableSessionParseState } from './session-scanner-types'
import type { SessionSidecarObservation } from './session-sidecar-stat'
import type { TranscriptMessageChannel } from './session-transcript-channel'
// Sized past the default recency cap (1000) plus the in-scope cap (2000) so a
// full steady-state result set stays resident between forced rescans.
const MAX_CACHE_ENTRIES = 4096
export type SessionParseResumePoint = {
state: ResumableSessionParseState
// Byte offset just past the last complete ('\n'-terminated) line consumed;
// a trailing unterminated line is deliberately left before this point.
byteOffset: number
// Bound to the cached state, which keeps the reference its parsers were built
// with; a resumed read re-points this channel instead of replacing it.
channel: TranscriptMessageChannel
}
export type SessionParseCacheEntry = {
mtimeMs: number
sizeBytes: number | null
platform: NodeJS.Platform
session: AiVaultSession | null
// What the sibling file looked like when `session` was built. Tracked apart
// from the transcript's key so each can go stale on its own.
sidecar?: SessionSidecarObservation
// The session the transcript alone produced, before any sibling was merged
// onto it. In-memory only: without it a sibling change costs one re-parse.
foldSession?: AiVaultSession | null
resume: SessionParseResumePoint | null
}
const cache = new Map<string, SessionParseCacheEntry>()
export function resetSessionParseCacheForTests(): void {
cache.clear()
}
// Drops one entry after its file is deleted. Cleanliness, not correctness:
// discovery walks disk first, so a trashed file is never rediscovered anyway.
export function invalidateSessionParseCacheEntry(path: string): void {
cache.delete(path)
}
// Persisted subset of a cache entry: the non-serializable `resume` parser
// state is dropped, and `foldSession` with it, so a restart pays one re-parse
// for a session whose sibling moved rather than storing every row twice
// (see session-parse-cache-persistence.ts).
export type PersistedSessionParseCacheEntry = Omit<SessionParseCacheEntry, 'resume' | 'foldSession'>
export function snapshotSessionParseCacheForPersistence(): [
string,
PersistedSessionParseCacheEntry
][] {
return [...cache].map(([path, entry]): [string, PersistedSessionParseCacheEntry] => [
path,
{
mtimeMs: entry.mtimeMs,
sizeBytes: entry.sizeBytes,
platform: entry.platform,
session: entry.session,
...(entry.sidecar === undefined ? {} : { sidecar: entry.sidecar })
}
])
}
// Seeded entries carry `resume: null`: after a restart an unchanged file is a
// cache hit; a file that changed while the app was closed pays one full
// (not incremental) re-parse.
export function seedSessionParseCache(
entries: Iterable<[string, PersistedSessionParseCacheEntry]>
): void {
const list = [...entries]
// Snapshot order is oldest→newest (LRU); an over-cap list keeps the newest
// tail rather than seeding the oldest entries and dropping the tail.
for (const [path, entry] of list.slice(Math.max(0, list.length - MAX_CACHE_ENTRIES))) {
if (cache.size >= MAX_CACHE_ENTRIES) {
return
}
// In-process entries are always fresher than persisted ones; never clobber.
if (cache.has(path)) {
continue
}
cache.set(path, {
mtimeMs: entry.mtimeMs,
sizeBytes: entry.sizeBytes,
platform: entry.platform,
session: entry.session,
// Absent in files an older build wrote; `sidecarUnchanged` reads that as
// unknown, so such a row re-enriches on its first scan.
sidecar: entry.sidecar,
resume: null
})
}
}
export function getSessionParseCacheEntry(path: string): SessionParseCacheEntry | undefined {
return cache.get(path)
}
export function storeSessionParseCacheEntry(path: string, entry: SessionParseCacheEntry): void {
cache.delete(path)
cache.set(path, entry)
if (cache.size > MAX_CACHE_ENTRIES) {
const oldest = cache.keys().next()
if (!oldest.done) {
cache.delete(oldest.value)
}
}
}
@@ -0,0 +1,28 @@
const pending = new Map<string, Promise<unknown>>()
/**
* Serializes parses of one transcript path.
*
* Two callers really do overlap on the same file: a forced refresh aborts the
* running scan while its in-flight parse keeps going as the replacement scan
* starts it again, and `session-title-file-reader.ts` parses on its own, with
* no scan involved. Overlapping reads share the cached resume point's message
* channel, so the second `beginRead` would drop the first read's consumers and
* the first `finishRead` would hand them the wrong outcome; the later store
* could also move the cursor backwards.
*/
export async function inSessionParseFileLane<T>(path: string, parse: () => Promise<T>): Promise<T> {
const previous = pending.get(path)
const run = (async () => {
await previous?.catch(() => undefined)
return parse()
})()
pending.set(path, run)
try {
return await run
} finally {
if (pending.get(path) === run) {
pending.delete(path)
}
}
}
@@ -23,6 +23,12 @@ import {
normalizePreviewText,
timestampMs
} from './session-scanner-values'
import { NO_TRANSCRIPT_MESSAGES, type TranscriptMessageSink } from './session-transcript-consumers'
import {
boundedText,
transcriptMessageRole,
transcriptMessagesFromContent
} from './session-transcript-message-content'
const SESSION_PREVIEW_MESSAGE_LIMIT = 5
@@ -30,9 +36,12 @@ export function createAccumulator(args: {
agent: AiVaultAgent
file: FileWithMtime
sessionId: string
// Where every decoded message goes; absent for one-shot parses with no reader.
messages?: TranscriptMessageSink
}): SessionAccumulator {
return {
agent: args.agent,
messages: args.messages ?? NO_TRANSCRIPT_MESSAGES,
sessionId: args.sessionId,
title: null,
fallbackTitle: null,
@@ -75,6 +84,9 @@ export function accumulatorFoldResumeState(
},
// Finalize a snapshot: the live accumulator (and its preview array) keeps
// accumulating appended lines after this session object is handed out.
// A sibling file's metadata is merged onto this result by the parse cache,
// never into the fold, so re-merging it later starts from what the
// transcript alone said (see session-scanner-sidecar-enrichment.ts).
finalize: (platform, options) =>
finalizeSession(cloneSessionAccumulator(accumulator), platform, options)
}
@@ -96,7 +108,7 @@ export function finalizeSession(
const title =
accumulator.title ||
accumulator.fallbackTitle ||
`${aiVaultAgentLabel(accumulator.agent)} ${sessionId.slice(0, 8)}`
generatedSessionTitle(accumulator.agent, sessionId)
const executionHostId = options.executionHostId ?? LOCAL_EXECUTION_HOST_ID
@@ -137,6 +149,15 @@ export function finalizeSession(
}
}
/**
* The title a session gets when neither the transcript nor the agent named it.
* Exported so a later merge can tell "the fold found no title" from a real one
* without re-deriving the string (session-scanner-sidecar-enrichment.ts).
*/
export function generatedSessionTitle(agent: AiVaultAgent, sessionId: string): string {
return `${aiVaultAgentLabel(agent)} ${sessionId.slice(0, 8)}`
}
export function updateTimeline(accumulator: SessionAccumulator, timestamp: unknown): void {
const parsed = timestampMs(timestamp)
if (!Number.isFinite(parsed)) {
@@ -161,8 +182,13 @@ export function addPreviewMessage(
// Why: Claude meta/injected turns still preview, but must not seed the
// copyable first-prompt row.
seedFirstUserPrompt?: boolean
// Set false by callers that already published this record's messages.
publishMessage?: boolean
}
): void {
if (args.publishMessage !== false && accumulator.messages.active) {
publishTranscriptMessage(accumulator, args.role, args.text, args.timestamp)
}
// Seeded before the preview-empty return so the copy body never depends on
// preview-only normalization rules.
seedFullFirstUserPrompt(
@@ -199,15 +225,41 @@ export function addPreviewContent(
() => extractFullFirstUserPromptText(content),
options?.seedFirstUserPrompt
)
// Published from the content value, not the preview string: a consumer needs
// the whole turn, including the tool blocks the 220-char preview drops.
if (accumulator.messages.active) {
for (const message of transcriptMessagesFromContent(role, content, timestampIso(timestamp))) {
accumulator.messages.push(message)
}
}
addPreviewMessage(accumulator, {
role,
text: extractPreviewContentText(content),
timestamp,
// Content path already seeded above when capture is enabled.
seedFirstUserPrompt: false
seedFirstUserPrompt: false,
publishMessage: false
})
}
/** One already-flattened turn; the content path publishes per block instead. */
function publishTranscriptMessage(
accumulator: SessionAccumulator,
role: AiVaultSessionPreviewMessage['role'],
text: string | null,
timestamp: unknown
): void {
const messageRole = transcriptMessageRole(role)
const messageText = text === null ? null : boundedText(text)
if (messageRole && messageText) {
accumulator.messages.push({
role: messageRole,
text: messageText,
timestamp: timestampIso(timestamp)
})
}
}
/**
* Seed the copyable first prompt from the first real user turn. `fullText` is a
* thunk so list scans (capture mode `none`) never pay the extraction cost.
@@ -6,7 +6,10 @@ import { parseClineSessionFile } from './session-scanner-cline-parser'
import { parseGrokSessionFile } from './session-scanner-grok-parser'
import { parseMessageGraphSessionFile, parseRovoSessionFile } from './session-scanner-graph-parsers'
import { parseKimiSessionFile } from './session-scanner-kimi-parser'
import { splitOpenCodeSqliteCandidate } from './session-scanner-opencode-sqlite-paths'
import {
looksLikeOpenCodeSqliteCandidate,
splitOpenCodeSqliteCandidate
} from './session-scanner-opencode-sqlite-paths'
import { parseOpenCodeSqliteSessionViaWorker } from './session-scanner-opencode-sqlite-worker-spawn'
import { parseClaudeSessionFile } from './session-scanner-primary-parsers'
import { parseGeminiSessionFile } from './session-scanner-gemini-parsers'
@@ -16,6 +19,16 @@ import { parseCursorSessionFile } from './session-scanner-cursor-parser'
import { parseHermesSessionFile } from './session-scanner-hermes-parser'
import { parseOpenCodeSessionFile } from './session-scanner-opencode-parser'
import type { SessionFileCandidate } from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
/**
* False when a parser decodes its messages somewhere the channel cannot reach.
* OpenCode's SQLite sessions are read on a worker thread, so their messages
* never come back over the sink and the read must not be reported as complete.
*/
export function parserPublishesMessages(candidate: SessionFileCandidate): boolean {
return candidate.agent !== 'opencode' || !looksLikeOpenCodeSqliteCandidate(candidate.file.path)
}
/**
* Parse a single agent session file into an `AiVaultSession`. Routes to the
@@ -24,25 +37,33 @@ import type { SessionFileCandidate } from './session-scanner-types'
* `parseOpenCodeSqliteSession` instead of the legacy JSON parser.
* @param candidate - The session file candidate to parse.
* @param platform - The platform to use for resume command generation.
* @param messages - Where the parser publishes every decoded message.
* @returns The parsed `AiVaultSession`, or `null` if parsing fails.
*/
export async function parseAgentSessionFile(
candidate: SessionFileCandidate,
platform: NodeJS.Platform
platform: NodeJS.Platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
switch (candidate.agent) {
case 'claude':
return parseClaudeSessionFile(candidate.file, platform)
return parseClaudeSessionFile(candidate.file, platform, messages)
case 'codex':
return parseCodexSessionFile(candidate.file, platform, candidate.codexHome)
return parseCodexSessionFile(
candidate.file,
platform,
candidate.codexHome,
undefined,
messages
)
case 'gemini':
return parseGeminiSessionFile(candidate.file, platform)
return parseGeminiSessionFile(candidate.file, platform, messages)
case 'antigravity':
return parseAntigravitySessionFile(candidate.file, platform)
return parseAntigravitySessionFile(candidate.file, platform, messages)
case 'copilot':
return parseCopilotSessionFile(candidate.file, platform)
return parseCopilotSessionFile(candidate.file, platform, messages)
case 'cursor':
return parseCursorSessionFile(candidate.file, platform)
return parseCursorSessionFile(candidate.file, platform, messages)
case 'opencode': {
// Why: OpenCode 1.17.x sessions are read from SQLite via a synthetic
// <dbPath>#<sessionId> candidate path. Legacy file-based sessions use
@@ -55,29 +76,29 @@ export async function parseAgentSessionFile(
platform
})
}
return parseOpenCodeSessionFile(candidate.file, platform)
return parseOpenCodeSessionFile(candidate.file, platform, messages)
}
case 'grok':
return parseGrokSessionFile(candidate.file, platform)
return parseGrokSessionFile(candidate.file, platform, messages)
case 'hermes':
return parseHermesSessionFile(candidate.file, platform)
return parseHermesSessionFile(candidate.file, platform, messages)
case 'rovo':
return parseRovoSessionFile(candidate.file, platform)
return parseRovoSessionFile(candidate.file, platform, messages)
case 'openclaw':
return parseMessageGraphSessionFile('openclaw', candidate.file, platform)
return parseMessageGraphSessionFile('openclaw', candidate.file, platform, messages)
case 'pi':
return parseMessageGraphSessionFile('pi', candidate.file, platform)
return parseMessageGraphSessionFile('pi', candidate.file, platform, messages)
case 'omp':
return parseMessageGraphSessionFile('omp', candidate.file, platform)
return parseMessageGraphSessionFile('omp', candidate.file, platform, messages)
case 'prime-agent':
return parseMessageGraphSessionFile('prime-agent', candidate.file, platform)
return parseMessageGraphSessionFile('prime-agent', candidate.file, platform, messages)
case 'droid':
return parseDroidSessionFile(candidate.file, platform)
return parseDroidSessionFile(candidate.file, platform, messages)
case 'cline':
return parseClineSessionFile(candidate.file, platform)
return parseClineSessionFile(candidate.file, platform, messages)
case 'devin':
return parseDevinSessionFile(candidate.file, platform)
return parseDevinSessionFile(candidate.file, platform, messages)
case 'kimi':
return parseKimiSessionFile(candidate.file, platform)
return parseKimiSessionFile(candidate.file, platform, messages)
}
}
@@ -8,6 +8,7 @@ import {
clineMessagesPathForMetadata,
isClineSessionMetadataPath
} from './session-scanner-cline-parser'
import { cursorChatMetaPath } from './session-scanner-cursor-chat-meta'
import { resolveKimiSessionsDir } from './session-scanner-kimi-paths'
import { OMP_SESSION_ARTIFACT_DIR_PATTERN } from './session-scanner-omp-subagent-transcripts'
import { claudeProjectsRootDirs, OMP_SESSIONS_DIR, sessionRootDirs } from './session-scanner-roots'
@@ -61,8 +62,9 @@ export type AiVaultAgentSource = {
rootDirs: (options: AiVaultScanOptions, wslHomeDirs: readonly string[]) => string[]
extensions: readonly string[]
filePredicate?: (filePath: string) => boolean
// A sibling whose stat participates in candidate freshness and recency.
contentDependencyPath?: (filePath: string) => string
// A sibling whose stat participates in candidate freshness and recency; async
// for agents that have to look the sibling up rather than derive its path.
contentDependencyPath?: (filePath: string) => string | undefined | Promise<string | undefined>
// Return false to skip a directory; depth 0 is a child of the root.
directoryPredicate?: (name: string, depth: number) => boolean
// Roots that are alternates for one install rather than distinct locations,
@@ -126,7 +128,8 @@ export const AI_VAULT_AGENT_SOURCES: AiVaultAgentSourceTable = {
'projects'
]),
extensions: ['.jsonl'],
filePredicate: (filePath) => pathSegments(filePath).includes('agent-transcripts')
filePredicate: (filePath) => pathSegments(filePath).includes('agent-transcripts'),
contentDependencyPath: cursorChatMetaPath
},
grok: {
rootDirs: (options, wslHomeDirs) =>
@@ -15,6 +15,7 @@ import type {
ResumableSessionParseState,
SessionAccumulator
} from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import { extractString, normalizeTitleText, parseJsonObject } from './session-scanner-values'
type ParserSessionOptions = {
@@ -24,12 +25,13 @@ type ParserSessionOptions = {
export async function parseAntigravitySessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const input = openTranscriptReadStream(file.path, { encoding: 'utf-8' }, 'scan')
const lines = createInterface({ input, crlfDelay: Infinity })
try {
return await parseAntigravitySessionLines({ file, lines, platform })
return await parseAntigravitySessionLines({ file, lines, platform, messages })
} finally {
// readline.close() leaves the underlying stream open; destroy it so a
// mid-parse throw cannot leak the gated transcript handle.
@@ -54,13 +56,14 @@ export async function parseAntigravitySessionContent(
}
export function createAntigravitySessionResumeState(
file: FileWithMtime
file: FileWithMtime,
messages?: TranscriptMessageSink
): ResumableSessionParseState {
const sessionId = antigravityConversationIdFromTranscriptPath(file.path) ?? ''
// Why: the transcript has no cwd/model fields. Workspace enrichment is a
// separate, conservative history join; protobuf/SQLite blobs are unstable.
return accumulatorFoldResumeState(
createAccumulator({ agent: 'antigravity', file, sessionId }),
createAccumulator({ agent: 'antigravity', file, sessionId, messages }),
consumeAntigravityRecordLine
)
}
@@ -70,8 +73,9 @@ async function parseAntigravitySessionLines(args: {
lines: AsyncIterable<string> | Iterable<string>
platform: NodeJS.Platform
options?: ParserSessionOptions
messages?: TranscriptMessageSink
}): Promise<AiVaultSession | null> {
const state = createAntigravitySessionResumeState(args.file)
const state = createAntigravitySessionResumeState(args.file, args.messages)
for await (const line of args.lines) {
state.consumeLine(line)
}
@@ -0,0 +1,46 @@
import { readCodexRolloutSessionMetaId } from '../codex/codex-rollout-session-meta'
import { codexRolloutHardlinkIdentity, dedupeCodexRolloutAliases } from './codex-session-root-dedup'
import { antigravityHistoryPathForBrainDir } from './session-scanner-antigravity-paths'
import { codexHomeForSessionsDir } from './session-scanner-codex-paths'
import { DEFAULT_CODEX_HOME_DIR } from './session-scanner-source-discovery'
import type {
AiVaultScanOptions,
SessionFileCandidate,
SessionFileDiscovery
} from './session-scanner-types'
/** Newest-first parse candidates for a discovery set, with Codex hardlink aliases collapsed. */
export async function sessionCandidatesFromDiscoveries(
discoveries: SessionFileDiscovery[],
options: AiVaultScanOptions
): Promise<SessionFileCandidate[]> {
return dedupeCodexRolloutAliases(
discoveries
.flatMap((discovery) =>
discovery.files.map((file): SessionFileCandidate => ({
agent: discovery.agent,
file,
codexHome:
discovery.agent === 'codex'
? codexHomeForSessionsDir(
discovery.rootDir,
options.defaultCodexHomeDir ?? DEFAULT_CODEX_HOME_DIR
)
: null,
antigravityHistoryPath:
discovery.agent === 'antigravity'
? antigravityHistoryPathForBrainDir(discovery.rootDir)
: undefined
}))
)
.sort((left, right) => right.file.mtimeMs - left.file.mtimeMs),
{
isCodex: (candidate) => candidate.agent === 'codex',
getFilePath: (candidate) => candidate.file.path,
getCodexHome: (candidate) => candidate.codexHome,
getHardlinkIdentity: (candidate) => codexRolloutHardlinkIdentity(candidate.file)
},
(filePath) => readCodexRolloutSessionMetaId(filePath, options.signal, 'scan'),
options.signal
)
}
@@ -9,6 +9,7 @@ import {
updateTimeline
} from './session-scanner-accumulator'
import type { FileWithMtime } from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
arrayValue,
asRecord,
@@ -35,7 +36,8 @@ export function clineMessagesPathForMetadata(filePath: string): string {
export async function parseClineSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messageSink?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const metadataContent = await wslGatedReadFile(file.path, 'utf-8', 'scan')
let messagesContent: string | null = null
@@ -52,7 +54,7 @@ export async function parseClineSessionFile(
throw error
}
}
return parseClineSessionContent(file, metadataContent, messagesContent, platform)
return parseClineSessionContent(file, metadataContent, messagesContent, platform, {}, messageSink)
}
function isMissingSessionPathError(error: unknown): boolean {
@@ -68,7 +70,8 @@ export function parseClineSessionContent(
metadataContent: string,
messagesContent: string | null,
platform: NodeJS.Platform = process.platform,
options: ParserSessionOptions = {}
options: ParserSessionOptions = {},
messageSink?: TranscriptMessageSink
): AiVaultSession | null {
const metadata = parseJsonRecord(metadataContent)
if (!metadata) {
@@ -76,7 +79,12 @@ export function parseClineSessionContent(
}
const pathSegments = file.path.replace(/\\/g, '/').split('/').filter(Boolean)
const sessionId = extractString(metadata.session_id) ?? pathSegments.at(-2) ?? ''
const accumulator = createAccumulator({ agent: 'cline', file, sessionId })
const accumulator = createAccumulator({
agent: 'cline',
file,
sessionId,
messages: messageSink
})
accumulator.cwd = extractString(metadata.cwd) ?? extractString(metadata.workspace_root)
accumulator.model = extractString(metadata.model)
updateTimeline(accumulator, metadata.started_at)
@@ -22,6 +22,7 @@ import type {
ResumableSessionParseState,
SessionAccumulator
} from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
addCodexUsage,
asRecord,
@@ -29,18 +30,22 @@ import {
extractModel,
extractString,
normalizeCodexUsage,
normalizeTitleText,
parseJsonObject,
subtractCodexUsage
} from './session-scanner-values'
import { remoteSessionContentLines } from './remote-session-content-lines'
import { readCodexTimelineOnlyRecord } from './session-scanner-codex-record-fast-path'
import {
extractCodexSessionMetadataTitle,
isCodexWorkerSession
} from './session-scanner-codex-session-meta'
export async function parseCodexSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform,
codexHome: string | null = null,
executionHostId?: ExecutionHostId
executionHostId?: ExecutionHostId,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const lines = createInterface({
input: openTranscriptReadStream(file.path, { encoding: 'utf-8' }, 'scan'),
@@ -53,6 +58,7 @@ export async function parseCodexSessionFile(
platform,
codexHome,
executionHostId,
messages,
titleReader: (sessionId) => readCodexSessionIndexTitle(file.path, codexHome, sessionId)
})
}
@@ -89,12 +95,16 @@ type CodexSessionParseState = {
titleSource: 'meta' | 'user' | null
}
function createCodexParseState(file: FileWithMtime): CodexSessionParseState {
function createCodexParseState(
file: FileWithMtime,
messages?: TranscriptMessageSink
): CodexSessionParseState {
return {
accumulator: createAccumulator({
agent: 'codex',
file,
sessionId: sessionIdFromFileName(file.path)
sessionId: sessionIdFromFileName(file.path),
messages
}),
previousTotals: null,
rejectedWorkerSession: false,
@@ -257,10 +267,13 @@ async function finalizeCodexParseState(
export function createCodexSessionResumeState(
file: FileWithMtime,
codexHome: string | null
codexHome: string | null,
messages?: TranscriptMessageSink
): ResumableSessionParseState {
return codexResumeStateFromParseState(createCodexParseState(file), codexHome, (sessionId) =>
readCodexSessionIndexTitle(file.path, codexHome, sessionId)
return codexResumeStateFromParseState(
createCodexParseState(file, messages),
codexHome,
(sessionId) => readCodexSessionIndexTitle(file.path, codexHome, sessionId)
)
}
@@ -298,8 +311,9 @@ async function parseCodexSessionLines(args: {
executionHostId?: ExecutionHostId
executionHostPlatform?: NodeJS.Platform | null
titleReader?: (sessionId: string) => Promise<string | null>
messages?: TranscriptMessageSink
}): Promise<AiVaultSession | null> {
const state = createCodexParseState(args.file)
const state = createCodexParseState(args.file, args.messages)
for await (const line of args.lines) {
consumeCodexRecordLine(state, line)
if (state.rejectedWorkerSession) {
@@ -314,21 +328,3 @@ async function parseCodexSessionLines(args: {
executionHostPlatform: args.executionHostPlatform
})
}
function isCodexWorkerSession(payload: Record<string, unknown>): boolean {
const threadSource = extractString(payload.thread_source) ?? extractString(payload.threadSource)
if (threadSource) {
return threadSource.toLowerCase() !== 'user'
}
const source = asRecord(payload.source)
return Boolean(asRecord(source?.subagent))
}
function extractCodexSessionMetadataTitle(payload: Record<string, unknown>): string | null {
return (
normalizeTitleText(extractString(payload.title) ?? '') ??
normalizeTitleText(extractString(payload.thread_name) ?? '') ??
normalizeTitleText(extractString(payload.threadName) ?? '')
)
}
@@ -0,0 +1,22 @@
import { asRecord, extractString, normalizeTitleText } from './session-scanner-values'
// Field readers for Codex's `session_meta` record, whose key spelling has drifted
// across Codex releases (snake_case rollouts, camelCase app-server rollouts).
export function isCodexWorkerSession(payload: Record<string, unknown>): boolean {
const threadSource = extractString(payload.thread_source) ?? extractString(payload.threadSource)
if (threadSource) {
return threadSource.toLowerCase() !== 'user'
}
const source = asRecord(payload.source)
return Boolean(asRecord(source?.subagent))
}
export function extractCodexSessionMetadataTitle(payload: Record<string, unknown>): string | null {
return (
normalizeTitleText(extractString(payload.title) ?? '') ??
normalizeTitleText(extractString(payload.thread_name) ?? '') ??
normalizeTitleText(extractString(payload.threadName) ?? '')
)
}
@@ -8,6 +8,7 @@ import type {
ResumableSessionParseState,
SessionAccumulator
} from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
accumulatorFoldResumeState,
addPreviewMessage,
@@ -32,13 +33,14 @@ type ParserSessionOptions = {
export async function parseCopilotSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const lines = createInterface({
input: openTranscriptReadStream(file.path, { encoding: 'utf-8' }, 'scan'),
crlfDelay: Infinity
})
return parseCopilotSessionLines({ file, lines, platform })
return parseCopilotSessionLines({ file, lines, platform, messages })
}
export async function parseCopilotSessionContent(
@@ -107,9 +109,17 @@ function consumeCopilotRecordLine(accumulator: SessionAccumulator, line: string)
}
}
export function createCopilotSessionResumeState(file: FileWithMtime): ResumableSessionParseState {
export function createCopilotSessionResumeState(
file: FileWithMtime,
messages?: TranscriptMessageSink
): ResumableSessionParseState {
return accumulatorFoldResumeState(
createAccumulator({ agent: 'copilot', file, sessionId: sessionIdFromFileName(file.path) }),
createAccumulator({
agent: 'copilot',
file,
sessionId: sessionIdFromFileName(file.path),
messages
}),
consumeCopilotRecordLine
)
}
@@ -119,8 +129,9 @@ async function parseCopilotSessionLines(args: {
lines: AsyncIterable<string> | Iterable<string>
platform: NodeJS.Platform
options?: ParserSessionOptions
messages?: TranscriptMessageSink
}): Promise<AiVaultSession | null> {
const state = createCopilotSessionResumeState(args.file)
const state = createCopilotSessionResumeState(args.file, args.messages)
for await (const line of args.lines) {
state.consumeLine(line)
}
@@ -0,0 +1,552 @@
import { appendFile, mkdir, mkdtemp, rm, stat, utimes, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { WslTranscriptFsError } from '../native-chat/wsl-transcript-fs-error'
// Why: a refused WSL read is the one build failure that must not be cached.
let failNextChatsReaddir = false
let failNextChatsRootReaddir = false
let failMetaJsonReads = false
let failMetaJsonStats: false | true | 'eacces' = false
let chatsRootReads = 0
vi.mock('../native-chat/wsl-transcript-fs-access', async (importOriginal) => {
const actual = await importOriginal<typeof WslTranscriptFsAccess>()
return {
...actual,
wslGatedReaddir: (
...args: Parameters<typeof actual.wslGatedReaddir>
): ReturnType<typeof actual.wslGatedReaddir> => {
if (args[0].endsWith('chats')) {
chatsRootReads += 1
if (failNextChatsRootReaddir) {
failNextChatsRootReaddir = false
return Promise.reject(new WslTranscriptFsError('timeout', 'wsl fs timed out'))
}
}
if (failNextChatsReaddir && args[0].includes('workspace-hash')) {
failNextChatsReaddir = false
return Promise.reject(new WslTranscriptFsError('timeout', 'wsl fs timed out'))
}
return actual.wslGatedReaddir(...args)
},
wslGatedReadFile: (
...args: Parameters<typeof actual.wslGatedReadFile>
): ReturnType<typeof actual.wslGatedReadFile> => {
if (failMetaJsonReads && String(args[0]).endsWith('meta.json')) {
return Promise.reject(new WslTranscriptFsError('timeout', 'wsl fs timed out'))
}
return actual.wslGatedReadFile(...args)
},
wslGatedStat: (
...args: Parameters<typeof actual.wslGatedStat>
): ReturnType<typeof actual.wslGatedStat> => {
if (failMetaJsonStats && String(args[0]).endsWith('meta.json')) {
return Promise.reject(
failMetaJsonStats === 'eacces'
? Object.assign(new Error('permission denied'), { code: 'EACCES' })
: new WslTranscriptFsError('timeout', 'wsl fs timed out')
)
}
return actual.wslGatedStat(...args)
}
}
})
import type * as WslTranscriptFsAccess from '../native-chat/wsl-transcript-fs-access'
import {
cursorChatMetaPath,
readCursorChatMeta,
resetCursorChatMetaIndexCacheForTests,
withCursorChatMetaScan
} from './session-scanner-cursor-chat-meta'
import { parseCursorSessionContent } from './session-scanner-cursor-parser'
import type { AiVaultScanIssue, AiVaultSession } from '../../shared/ai-vault-types'
import { AI_VAULT_AGENT_SOURCES } from './session-scanner-agent-sources'
import { discoverFiles } from './session-scanner-discovery'
import { scanAiVaultSessions } from './session-scanner'
import {
createSessionParseStats,
parseAgentSessionFileCached,
resetSessionParseCacheForTests,
seedSessionParseCache,
snapshotSessionParseCacheForPersistence,
type SessionParseStats
} from './session-scanner-parse-cache'
import {
getSessionParseCacheEntry,
type PersistedSessionParseCacheEntry
} from './session-parse-cache-store'
import { isolatedScanRoots } from './session-scanner-test-fixtures'
import type { FileWithMtime } from './session-scanner-types'
import type { SessionSidecarStat } from './session-sidecar-stat'
// Cursor's real meta.json keys (~/.cursor/chats/<md5 of cwd>/<uuid>/meta.json, 2026-09).
type CursorMetaFixture = {
schemaVersion: number
createdAtMs: number
updatedAtMs: number
cwd: string
hasConversation: boolean
title?: string
}
const CREATED_AT_MS = 1_787_039_612_017
const UPDATED_AT_MS = 1_787_039_640_532
let tempRoots: string[] = []
afterEach(async () => {
resetCursorChatMetaIndexCacheForTests()
resetSessionParseCacheForTests()
failNextChatsRootReaddir = false
failMetaJsonReads = false
failMetaJsonStats = false
await Promise.all(tempRoots.map((root) => rm(root, { recursive: true, force: true })))
tempRoots = []
})
async function createCursorHome(): Promise<string> {
const root = await mkdtemp(join(tmpdir(), 'orca-cursor-chat-meta-'))
tempRoots.push(root)
const cursorHome = join(root, '.cursor')
await mkdir(cursorHome, { recursive: true })
return cursorHome
}
async function writeTranscript(
cursorHome: string,
projectSlug: string,
chatId: string,
lines: string[]
): Promise<string> {
const chatDir = join(cursorHome, 'projects', projectSlug, 'agent-transcripts', chatId)
await mkdir(chatDir, { recursive: true })
const transcriptPath = join(chatDir, `${chatId}.jsonl`)
await writeFile(transcriptPath, lines.map((line) => `${line}\n`).join(''))
return transcriptPath
}
async function writeChatMeta(
cursorHome: string,
workspaceHash: string,
chatId: string,
meta: Partial<CursorMetaFixture> = {}
): Promise<string> {
const chatDir = join(cursorHome, 'chats', workspaceHash, chatId)
await mkdir(chatDir, { recursive: true })
const metaPath = join(chatDir, 'meta.json')
await writeFile(
metaPath,
JSON.stringify({
schemaVersion: 1,
createdAtMs: CREATED_AT_MS,
updatedAtMs: UPDATED_AT_MS,
cwd: '/private/tmp/workspace',
hasConversation: true,
...meta
} satisfies CursorMetaFixture)
)
return metaPath
}
function fileWithMtime(path: string): FileWithMtime {
return { path, mtimeMs: 1, modifiedAt: new Date(1).toISOString() }
}
describe('cursor chat meta', () => {
it('resolves the meta.json under the workspace hash that holds the chat id', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'aa37220647fb7ce5eb044aa4bda60807', 'other-chat')
const metaPath = await writeChatMeta(cursorHome, '96fa26ac0f433670ebec73ecef20b47b', 'chat-1', {
title: 'Shell Command Hostname'
})
const transcriptPath = await writeTranscript(cursorHome, 'private-tmp-workspace', 'chat-1', [])
expect(await cursorChatMetaPath(transcriptPath)).toBe(metaPath)
expect(await readCursorChatMeta(transcriptPath)).toEqual({
title: 'Shell Command Hostname',
cwd: '/private/tmp/workspace',
createdAt: new Date(CREATED_AT_MS).toISOString(),
updatedAt: new Date(UPDATED_AT_MS).toISOString()
})
})
it('re-indexes after a chat appears under an already indexed workspace', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-first')
const firstTranscript = await writeTranscript(cursorHome, 'slug', 'chat-first', [])
expect(await cursorChatMetaPath(firstTranscript)).toBeDefined()
const laterMetaPath = await writeChatMeta(cursorHome, 'workspace-hash', 'chat-later')
const laterTranscript = await writeTranscript(cursorHome, 'slug', 'chat-later', [])
expect(await cursorChatMetaPath(laterTranscript)).toBe(laterMetaPath)
})
it('does not cache a metadata index whose build was refused by the WSL gate', async () => {
const cursorHome = await createCursorHome()
const metaPath = await writeChatMeta(cursorHome, 'workspace-hash', 'chat-refused')
const transcriptPath = await writeTranscript(cursorHome, 'slug', 'chat-refused', [])
failNextChatsReaddir = true
// A refusal degrades to "no metadata" rather than taking the session down.
await expect(cursorChatMetaPath(transcriptPath)).resolves.toBeUndefined()
// The next scan rebuilds instead of replaying the rejected promise.
await expect(cursorChatMetaPath(transcriptPath)).resolves.toBe(metaPath)
})
it('validates the index once per scan, not once per transcript', async () => {
const cursorHome = await createCursorHome()
const transcripts: string[] = []
for (const chatId of ['chat-a', 'chat-b', 'chat-c']) {
await writeChatMeta(cursorHome, 'workspace-hash', chatId)
transcripts.push(await writeTranscript(cursorHome, 'slug', chatId, []))
}
chatsRootReads = 0
const inScan = await withCursorChatMetaScan(() =>
Promise.all(transcripts.map((path) => cursorChatMetaPath(path)))
)
expect(inScan.every(Boolean)).toBe(true)
expect(chatsRootReads).toBe(1)
// Outside a scan every lookup re-validates, which is what the parse path needs.
chatsRootReads = 0
await Promise.all(transcripts.map((path) => cursorChatMetaPath(path)))
expect(chatsRootReads).toBe(3)
})
it('yields nothing and does not throw when there is no chats tree', async () => {
const cursorHome = await createCursorHome()
const transcriptPath = await writeTranscript(cursorHome, 'slug', 'chat-orphan', [])
await expect(cursorChatMetaPath(transcriptPath)).resolves.toBeUndefined()
await expect(readCursorChatMeta(transcriptPath)).resolves.toBeNull()
await expect(readCursorChatMeta('/nowhere/near/cursor/chat.jsonl')).resolves.toBeNull()
})
it('yields nothing and does not throw when meta.json is malformed', async () => {
const cursorHome = await createCursorHome()
const chatDir = join(cursorHome, 'chats', 'workspace-hash', 'chat-bad')
await mkdir(chatDir, { recursive: true })
await writeFile(join(chatDir, 'meta.json'), '{ not json')
const transcriptPath = await writeTranscript(cursorHome, 'slug', 'chat-bad', [])
await expect(readCursorChatMeta(transcriptPath)).resolves.toBeNull()
})
})
async function cursorCandidate(cursorHome: string): Promise<FileWithMtime> {
const issues: AiVaultScanIssue[] = []
const discovery = await discoverFiles({
rootDir: join(cursorHome, 'projects'),
limit: 10,
agent: 'cursor',
issues,
extensions: [...AI_VAULT_AGENT_SOURCES.cursor.extensions],
filePredicate: AI_VAULT_AGENT_SOURCES.cursor.filePredicate,
contentDependencyPath: AI_VAULT_AGENT_SOURCES.cursor.contentDependencyPath
})
return discovery.files[0]
}
/** The production path: the parse cache owns the sidecar merge, not the parser. */
function parseCursorCached(
file: FileWithMtime,
stats: SessionParseStats = createSessionParseStats()
): Promise<{ session: AiVaultSession | null; stats: SessionParseStats }> {
return withCursorChatMetaScan(async () => {
const session = await parseAgentSessionFileCached(
{ agent: 'cursor', file, codexHome: null },
'darwin',
stats
)
return { session, stats }
})
}
async function writeCursorScanFixture(chatIds: string[]): Promise<{
cursorHome: string
scanOptions: ReturnType<typeof isolatedScanRoots> & { cursorProjectsDir: string }
}> {
const cursorHome = await createCursorHome()
for (const chatId of chatIds) {
await writeChatMeta(cursorHome, 'workspace-hash', chatId, { cwd: `/tmp/ws-${chatId}` })
await writeTranscript(cursorHome, 'slug', chatId, [
JSON.stringify({ role: 'user', message: { content: [{ type: 'text', text: chatId }] } })
])
}
const root = join(cursorHome, '..')
return {
cursorHome,
scanOptions: {
...isolatedScanRoots(root),
cursorProjectsDir: join(cursorHome, 'projects')
}
}
}
describe('cursor discovery sidecar observation', () => {
it('records meta.json beside the transcript stat instead of folding it in', async () => {
const cursorHome = await createCursorHome()
const metaPath = await writeChatMeta(cursorHome, 'workspace-hash', 'chat-7')
const transcriptPath = await writeTranscript(cursorHome, 'slug', 'chat-7', [])
const before = await cursorCandidate(cursorHome)
const future = new Date(Date.now() + 10_000)
await utimes(metaPath, future, future)
const after = await cursorCandidate(cursorHome)
// The transcript's own key is untouched by a sibling rewrite.
const transcriptStat = await stat(transcriptPath)
expect(after.mtimeMs).toBe(transcriptStat.mtimeMs)
expect(after.sizeBytes).toBe(transcriptStat.size)
expect(after.mtimeMs).toBe(before.mtimeMs)
// The sibling is observed separately, and it did move.
expect(before.sidecar).toMatchObject({ path: metaPath })
expect(after.sidecar).toMatchObject({ path: metaPath })
expect((after.sidecar as SessionSidecarStat).mtimeMs).toBeGreaterThan(
(before.sidecar as SessionSidecarStat).mtimeMs
)
})
})
describe('cursor sidecar enrichment', () => {
it('fills cwd, timestamps and title from meta.json', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-2', { title: 'Named From Meta' })
await writeTranscript(cursorHome, 'slug', 'chat-2', [
JSON.stringify({ role: 'assistant', message: { content: 'hello' } })
])
resetSessionParseCacheForTests()
const { session } = await parseCursorCached(await cursorCandidate(cursorHome))
expect(session?.cwd).toBe('/private/tmp/workspace')
expect(session?.title).toBe('Named From Meta')
expect(session?.createdAt).toBe(new Date(CREATED_AT_MS).toISOString())
expect(session?.updatedAt).toBe(new Date(UPDATED_AT_MS).toISOString())
})
it('keeps a transcript title and timestamps over meta.json', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-3', { title: 'Meta Title' })
await writeTranscript(cursorHome, 'slug', 'chat-3', [
JSON.stringify({
role: 'user',
timestamp: '2026-01-01T00:00:00.000Z',
message: { content: 'transcript first prompt' }
})
])
resetSessionParseCacheForTests()
const { session } = await parseCursorCached(await cursorCandidate(cursorHome))
expect(session?.title).toBe('transcript first prompt')
expect(session?.createdAt).toBe('2026-01-01T00:00:00.000Z')
// cwd is never in the transcript, so it still comes from meta.json.
expect(session?.cwd).toBe('/private/tmp/workspace')
})
it('builds the resume command from the meta.json cwd', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-4', { cwd: '/repo/from-meta' })
await writeTranscript(cursorHome, 'slug', 'chat-4', [
JSON.stringify({ role: 'user', message: { content: 'hi' } })
])
resetSessionParseCacheForTests()
const { session } = await parseCursorCached(await cursorCandidate(cursorHome))
expect(session?.resumeCommand).toContain('/repo/from-meta')
})
it('leaves remote content parses to the transcript alone', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-5', { title: 'Meta Title' })
const transcriptPath = await writeTranscript(cursorHome, 'slug', 'chat-5', [])
const session = await parseCursorSessionContent(
fileWithMtime(transcriptPath),
`${JSON.stringify({ role: 'assistant', message: { content: 'remote' } })}\n`,
'linux'
)
expect(session?.cwd).toBeNull()
expect(session?.title).not.toBe('Meta Title')
})
it('re-enriches without a parse when only the sidecar is rewritten', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-8', { cwd: '/repo/first' })
await writeTranscript(cursorHome, 'slug', 'chat-8', [
JSON.stringify({ role: 'user', message: { content: 'hi' } })
])
resetSessionParseCacheForTests()
await parseCursorCached(await cursorCandidate(cursorHome))
const future = new Date(Date.now() + 10_000)
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-8', { cwd: '/repo/second' })
await utimes(join(cursorHome, 'chats', 'workspace-hash', 'chat-8', 'meta.json'), future, future)
const { session, stats } = await parseCursorCached(await cursorCandidate(cursorHome))
// The transcript is not re-read: the merge runs over the stored fold result.
expect(stats.reused).toBe(1)
expect(stats.fullParses).toBe(0)
expect(stats.incremental).toBe(0)
// A rewritten cwd REPLACES the merged one; `??=` on the cached session could
// never do this, because the cached cwd is already non-null.
expect(session?.cwd).toBe('/repo/second')
expect(session?.resumeCommand).toContain('/repo/second')
})
it('treats a persisted entry with no sidecar as unknown and enriches once', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-9', { cwd: '/repo/persisted' })
await writeTranscript(cursorHome, 'slug', 'chat-9', [
JSON.stringify({ role: 'user', message: { content: 'hi' } })
])
resetSessionParseCacheForTests()
await parseCursorCached(await cursorCandidate(cursorHome))
// What a build older than the sidecar field wrote: no such key.
const persisted = snapshotSessionParseCacheForPersistence().map(
([path, entry]): [string, PersistedSessionParseCacheEntry] => {
const { sidecar: _sidecar, ...rest } = entry
return [path, rest]
}
)
resetSessionParseCacheForTests()
seedSessionParseCache(persisted)
const { session, stats } = await parseCursorCached(await cursorCandidate(cursorHome))
expect(stats.reused).toBe(0)
expect(session?.cwd).toBe('/repo/persisted')
})
})
describe('cursor chat meta scan failures', () => {
it('lists cursor sessions without metadata when the chats tree is refused, then heals', async () => {
const { cursorHome, scanOptions } = await writeCursorScanFixture(['chat-a', 'chat-b'])
resetSessionParseCacheForTests()
failNextChatsRootReaddir = true
const refused = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
const refusedCursor = refused.sessions.filter((session) => session.agent === 'cursor')
expect(refusedCursor).toHaveLength(2)
expect(refusedCursor.map((session) => session.cwd)).toEqual([null, null])
// One issue for the chats root, not one per transcript.
expect(refused.issues).toHaveLength(1)
expect(refused.issues[0].path).toBe(join(cursorHome, 'chats'))
expect(refused.issues[0].agent).toBe('cursor')
const healed = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
expect(healed.issues).toEqual([])
expect(
healed.sessions
.filter((session) => session.agent === 'cursor')
.map((session) => session.cwd)
.sort()
).toEqual(['/tmp/ws-chat-a', '/tmp/ws-chat-b'])
})
it('re-enriches after a refused meta.json read without losing the resume cursor', async () => {
const { scanOptions } = await writeCursorScanFixture(['chat-a'])
resetSessionParseCacheForTests()
failMetaJsonReads = true
const refused = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
const listed = refused.sessions.find((session) => session.agent === 'cursor')
expect(listed?.cwd).toBeNull()
expect(refused.issues).toHaveLength(1)
// The sibling alone is unknown; the transcript's work and its resume point
// are kept, so the next healthy scan merges without re-reading bytes.
const entry = getSessionParseCacheEntry(listed?.filePath ?? '')
expect(entry?.sidecar).toBe('unknown')
expect(entry?.resume).not.toBeNull()
failMetaJsonReads = false
const healed = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
expect(healed.issues).toEqual([])
expect(healed.sessions.find((session) => session.agent === 'cursor')?.cwd).toBe(
'/tmp/ws-chat-a'
)
})
it('treats a local EACCES on the sidecar stat as unknown, not as absent', async () => {
const { scanOptions } = await writeCursorScanFixture(['chat-a'])
resetSessionParseCacheForTests()
// On mac/Linux/Windows the gated stat is a bare fs stat, so a permissions
// failure is not a WslTranscriptFsError and must not read as "no sidecar".
failMetaJsonStats = 'eacces'
const refused = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
const listed = refused.sessions.find((session) => session.agent === 'cursor')
expect(listed?.sessionId).toBeTruthy()
expect(refused.issues).toHaveLength(1)
expect(refused.issues[0].agent).toBe('cursor')
expect(getSessionParseCacheEntry(listed?.filePath ?? '')?.sidecar).toBe('unknown')
failMetaJsonStats = false
const healed = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
expect(healed.issues).toEqual([])
expect(healed.sessions.find((session) => session.agent === 'cursor')?.cwd).toBe(
'/tmp/ws-chat-a'
)
})
it('lists a cursor session whose meta.json stat is refused instead of dropping it', async () => {
const { scanOptions } = await writeCursorScanFixture(['chat-a'])
resetSessionParseCacheForTests()
failMetaJsonStats = true
const refused = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
expect(refused.sessions.filter((session) => session.agent === 'cursor')).toHaveLength(1)
expect(refused.issues).toHaveLength(1)
expect(refused.issues[0].agent).toBe('cursor')
failMetaJsonStats = false
const healed = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
expect(healed.issues).toEqual([])
expect(healed.sessions.find((session) => session.agent === 'cursor')?.cwd).toBe(
'/tmp/ws-chat-a'
)
})
it('reads the chats root once per scan across discovery and parse', async () => {
const { scanOptions } = await writeCursorScanFixture(['chat-a', 'chat-b', 'chat-c'])
resetSessionParseCacheForTests()
chatsRootReads = 0
const result = await scanAiVaultSessions({ ...scanOptions, platform: 'darwin', limit: 20 })
expect(result.sessions.filter((session) => session.agent === 'cursor')).toHaveLength(3)
expect(chatsRootReads).toBe(1)
})
it('resumes an appended transcript after a refused sidecar scan', async () => {
const cursorHome = await createCursorHome()
await writeChatMeta(cursorHome, 'workspace-hash', 'chat-r', { cwd: '/repo/resume' })
const transcriptPath = await writeTranscript(cursorHome, 'slug', 'chat-r', [
JSON.stringify({ role: 'user', message: { content: 'one' } })
])
resetSessionParseCacheForTests()
await parseCursorCached(await cursorCandidate(cursorHome))
await appendFile(
transcriptPath,
`${JSON.stringify({ role: 'user', message: { content: 'two' } })}\n`
)
failMetaJsonReads = true
const { stats: refusedStats } = await parseCursorCached(await cursorCandidate(cursorHome))
expect(refusedStats.incremental).toBe(1)
failMetaJsonReads = false
const { session, stats } = await parseCursorCached(await cursorCandidate(cursorHome))
expect(stats.reused).toBe(1)
expect(stats.fullParses).toBe(0)
expect(session?.cwd).toBe('/repo/resume')
})
})
@@ -0,0 +1,294 @@
import { AsyncLocalStorage } from 'node:async_hooks'
import { basename, dirname, join } from 'node:path'
import { wslGatedReaddir, wslGatedStat } from '../native-chat/wsl-transcript-fs-access'
import { WslTranscriptFsError } from '../native-chat/wsl-transcript-fs-gate'
import { timestampIso } from './session-scanner-accumulator'
import { extractString, normalizeTitleText, readJsonObjectIfExists } from './session-scanner-values'
// Cursor keeps a chat's transcript and its metadata in two unrelated trees:
// <cursor>/projects/<slug>/agent-transcripts/<uuid>/<uuid>.jsonl holds the
// messages, while <cursor>/chats/<md5 of cwd>/<uuid>/meta.json holds the cwd,
// title and timestamps. The md5 hashes the very cwd we are looking for, so the
// only way across is an index of the chat directories.
const CURSOR_CHATS_DIR = 'chats'
const CURSOR_CHAT_META_FILE = 'meta.json'
const CURSOR_TRANSCRIPTS_DIR = 'agent-transcripts'
const CURSOR_PROJECTS_DIR = 'projects'
// Why: custom and WSL Cursor homes can vary over a long-lived main process.
const CURSOR_CHAT_META_INDEX_CACHE_MAX = 8
export type CursorChatMeta = {
title: string | null
cwd: string | null
createdAt: string | null
updatedAt: string | null
}
type CursorChatMetaIndexEntry = {
signature: string
metaPathByChatId: Map<string, string>
}
const cursorChatMetaIndexCache = new Map<string, Promise<CursorChatMetaIndexEntry>>()
type CursorChatMetaScan = {
index: Map<string, Promise<Map<string, string>>>
// Chats roots this scan could not read, reported once by the scan owner.
refusals: Map<string, string>
// Transcripts whose own meta.json read was refused, so the metadata merged
// onto them is not what the file on disk says.
refusedTranscripts: Set<string>
}
// Why: validating the module cache costs a readdir of the chats root plus a stat
// per workspace, and it cannot be skipped because the signature is built from
// those stats. Discovery asks once per transcript and finalize asks again, so
// the scope has to span both phases for one scan to see the tree once.
const scanScopedIndex = new AsyncLocalStorage<CursorChatMetaScan>()
export function resetCursorChatMetaIndexCacheForTests(): void {
cursorChatMetaIndexCache.clear()
}
/** Runs one whole scan, discovery and parse; every Cursor transcript in it shares one index read. */
export function withCursorChatMetaScan<T>(fn: () => Promise<T>): Promise<T> {
return scanScopedIndex.run(
{ index: new Map(), refusals: new Map(), refusedTranscripts: new Set() },
fn
)
}
/**
* True when this transcript's own meta.json read was refused, so the caller
* records the sidecar as unknown rather than as the observation discovery made.
* The transcript's own work and its resume point are kept either way.
*/
export function wasCursorChatMetaRefused(transcriptPath: string): boolean {
return scanScopedIndex.getStore()?.refusedTranscripts.has(transcriptPath) ?? false
}
/** Chats roots the current scan was refused, for the caller to report as scan issues. */
export function cursorChatMetaRefusals(): { chatsRoot: string; message: string }[] {
const scan = scanScopedIndex.getStore()
return scan ? [...scan.refusals].map(([chatsRoot, message]) => ({ chatsRoot, message })) : []
}
/** Path a discovery stat can watch so a rewritten meta.json invalidates the parse cache. */
export async function cursorChatMetaPath(transcriptPath: string): Promise<string | undefined> {
const chatsRoot = cursorChatsRootFromTranscriptPath(transcriptPath)
const chatId = cursorChatIdFromTranscriptPath(transcriptPath)
if (!chatsRoot || !chatId) {
return undefined
}
const index = await readCursorChatMetaIndexOncePerScan(chatsRoot)
return index.get(chatId)
}
function readCursorChatMetaIndexOncePerScan(chatsRoot: string): Promise<Map<string, string>> {
const scan = scanScopedIndex.getStore()
if (!scan) {
return readCursorChatMetaIndexOrNone(chatsRoot)
}
let pending = scan.index.get(chatsRoot)
if (!pending) {
pending = readCursorChatMetaIndexOrNone(chatsRoot)
scan.index.set(chatsRoot, pending)
}
return pending
}
/**
* A refused WSL read is not "no chats", but it must not take the transcript
* down with it: before this join a stalled distro could not hide a Cursor
* session at all. Degrade to no metadata for the scan and report the root once.
* The session still lists from its transcript alone, and the sidecar is
* recorded as unknown, so the next healthy scan merges the real metadata in
* without re-reading a byte of the transcript.
*/
async function readCursorChatMetaIndexOrNone(chatsRoot: string): Promise<Map<string, string>> {
try {
return await readCursorChatMetaIndex(chatsRoot)
} catch (error) {
if (!(error instanceof WslTranscriptFsError)) {
throw error
}
recordCursorChatMetaRefusal(chatsRoot, error.message)
return new Map()
}
}
function recordCursorChatMetaRefusal(chatsRoot: string, message: string): void {
const scan = scanScopedIndex.getStore()
if (scan && !scan.refusals.has(chatsRoot)) {
scan.refusals.set(chatsRoot, message)
}
}
export async function readCursorChatMeta(transcriptPath: string): Promise<CursorChatMeta | null> {
const metaPath = await cursorChatMetaPath(transcriptPath)
if (!metaPath) {
return null
}
let record: Record<string, unknown> | null
try {
record = await readJsonObjectIfExists(metaPath)
} catch (error) {
if (!(error instanceof WslTranscriptFsError)) {
throw error
}
// The session still lists, but unlike the index read this transcript's key
// already includes meta.json's stat, so the caller must not cache the
// un-enriched result. One issue per chats root, as for a refused index.
recordCursorChatMetaRefusal(
cursorChatsRootFromTranscriptPath(transcriptPath) ?? metaPath,
error.message
)
scanScopedIndex.getStore()?.refusedTranscripts.add(transcriptPath)
return null
}
if (!record) {
return null
}
return {
title: normalizeTitleText(extractString(record.title) ?? ''),
cwd: extractString(record.cwd),
createdAt: timestampIso(record.createdAtMs),
updatedAt: timestampIso(record.updatedAtMs)
}
}
function cursorChatIdFromTranscriptPath(transcriptPath: string): string | null {
const chatDir = dirname(transcriptPath)
return basename(dirname(chatDir)) === CURSOR_TRANSCRIPTS_DIR ? basename(chatDir) : null
}
function cursorChatsRootFromTranscriptPath(transcriptPath: string): string | null {
let currentDir = dirname(transcriptPath)
while (currentDir && dirname(currentDir) !== currentDir) {
// The chats tree is a sibling of the projects tree, custom Cursor homes included.
if (basename(currentDir) === CURSOR_PROJECTS_DIR) {
return join(dirname(currentDir), CURSOR_CHATS_DIR)
}
currentDir = dirname(currentDir)
}
return null
}
async function readCursorChatMetaIndex(chatsRoot: string): Promise<Map<string, string>> {
let workspaceDirs: string[]
try {
workspaceDirs = (await wslGatedReaddir(chatsRoot, 'scan'))
.filter((entry) => entry.isDirectory())
.map((entry) => entry.name)
.sort()
} catch (error) {
// Why: a refused WSL read is not "no chats"; letting it through keeps the
// session out of the parse cache instead of caching it without metadata.
if (error instanceof WslTranscriptFsError) {
throw error
}
return new Map()
}
const signature = await readCursorChatsSignature(chatsRoot, workspaceDirs)
const cached = await readCachedCursorChatMetaIndex(chatsRoot, signature)
if (cached) {
return cached
}
const pending = buildCursorChatMetaIndex(chatsRoot, workspaceDirs).then((metaPathByChatId) => ({
signature,
metaPathByChatId
}))
storeCursorChatMetaIndexEntry(chatsRoot, pending)
// Why: a rejected build (a refused WSL read) must not be served from the
// cache forever; the next scan rebuilds while this one still sees the error.
pending.catch(() => {
if (cursorChatMetaIndexCache.get(chatsRoot) === pending) {
cursorChatMetaIndexCache.delete(chatsRoot)
}
})
return (await pending).metaPathByChatId
}
// Why: a new chat only bumps its own workspace directory, so the chats root's
// own mtime would keep serving an index that is missing the newest sessions.
async function readCursorChatsSignature(
chatsRoot: string,
workspaceDirs: string[]
): Promise<string> {
const parts = await Promise.all(
workspaceDirs.map(async (name) => {
try {
const dirStat = await wslGatedStat(join(chatsRoot, name), 'scan')
return `${name}:${dirStat.mtimeMs}`
} catch {
return `${name}:?`
}
})
)
return parts.join('|')
}
async function buildCursorChatMetaIndex(
chatsRoot: string,
workspaceDirs: string[]
): Promise<Map<string, string>> {
const metaPathByChatId = new Map<string, string>()
for (const workspaceDir of workspaceDirs) {
let chatDirs
try {
chatDirs = await wslGatedReaddir(join(chatsRoot, workspaceDir), 'scan')
} catch (error) {
if (error instanceof WslTranscriptFsError) {
throw error
}
continue
}
for (const chatDir of chatDirs) {
// Why: the same chat id never appears under two workspace hashes, so the
// first hit wins and a duplicate would only cost a wasted read.
if (chatDir.isDirectory() && !metaPathByChatId.has(chatDir.name)) {
metaPathByChatId.set(
chatDir.name,
join(chatsRoot, workspaceDir, chatDir.name, CURSOR_CHAT_META_FILE)
)
}
}
}
return metaPathByChatId
}
async function readCachedCursorChatMetaIndex(
chatsRoot: string,
signature: string
): Promise<Map<string, string> | undefined> {
const cached = cursorChatMetaIndexCache.get(chatsRoot)
if (!cached) {
return undefined
}
const entry = await cached
if (entry.signature !== signature) {
return undefined
}
// Why: a concurrent scan can replace this Promise while it resolves; only the
// still-current entry may refresh recency without bypassing the cap.
if (cursorChatMetaIndexCache.get(chatsRoot) === cached) {
cursorChatMetaIndexCache.delete(chatsRoot)
cursorChatMetaIndexCache.set(chatsRoot, cached)
}
return entry.metaPathByChatId
}
function storeCursorChatMetaIndexEntry(
chatsRoot: string,
pending: Promise<CursorChatMetaIndexEntry>
): void {
cursorChatMetaIndexCache.delete(chatsRoot)
cursorChatMetaIndexCache.set(chatsRoot, pending)
if (cursorChatMetaIndexCache.size > CURSOR_CHAT_META_INDEX_CACHE_MAX) {
const oldest = cursorChatMetaIndexCache.keys().next()
if (!oldest.done) {
cursorChatMetaIndexCache.delete(oldest.value)
}
}
}
@@ -8,6 +8,7 @@ import type {
ResumableSessionParseState,
SessionAccumulator
} from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
accumulatorFoldResumeState,
addPreviewContent,
@@ -30,13 +31,14 @@ type ParserSessionOptions = {
export async function parseCursorSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const lines = createInterface({
input: openTranscriptReadStream(file.path, { encoding: 'utf-8' }, 'scan'),
crlfDelay: Infinity
})
return parseCursorSessionLines({ file, lines, platform })
return parseCursorSessionLines({ file, lines, platform, messages })
}
export async function parseCursorSessionContent(
@@ -75,9 +77,17 @@ function consumeCursorRecordLine(accumulator: SessionAccumulator, line: string):
}
}
export function createCursorSessionResumeState(file: FileWithMtime): ResumableSessionParseState {
export function createCursorSessionResumeState(
file: FileWithMtime,
messages?: TranscriptMessageSink
): ResumableSessionParseState {
return accumulatorFoldResumeState(
createAccumulator({ agent: 'cursor', file, sessionId: sessionIdFromFileName(file.path) }),
createAccumulator({
agent: 'cursor',
file,
sessionId: sessionIdFromFileName(file.path),
messages
}),
consumeCursorRecordLine
)
}
@@ -87,8 +97,9 @@ async function parseCursorSessionLines(args: {
lines: AsyncIterable<string> | Iterable<string>
platform: NodeJS.Platform
options?: ParserSessionOptions
messages?: TranscriptMessageSink
}): Promise<AiVaultSession | null> {
const state = createCursorSessionResumeState(args.file)
const state = createCursorSessionResumeState(args.file, args.messages)
for await (const line of args.lines) {
state.consumeLine(line)
}
@@ -2,6 +2,7 @@ import { wslGatedReadFile } from '../native-chat/wsl-transcript-fs-access'
import type { AiVaultSession } from '../../shared/ai-vault-types'
import type { ExecutionHostId } from '../../shared/execution-host'
import type { FileWithMtime } from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
addPreviewContent,
createAccumulator,
@@ -25,20 +26,34 @@ type ParserSessionOptions = {
export async function parseDevinSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
return parseDevinSessionContent(
return parseDevinSessionRecord(
file,
await wslGatedReadFile(file.path, 'utf-8', 'scan'),
platform
platform,
{},
messages
)
}
/** Remote transcript content, streamed from a host that has no reader attached. */
export function parseDevinSessionContent(
file: FileWithMtime,
content: string,
platform: NodeJS.Platform = process.platform,
options: ParserSessionOptions = {}
): AiVaultSession | null {
return parseDevinSessionRecord(file, content, platform, options)
}
function parseDevinSessionRecord(
file: FileWithMtime,
content: string,
platform: NodeJS.Platform,
options: ParserSessionOptions,
messages?: TranscriptMessageSink
): AiVaultSession | null {
const record = asRecord(JSON.parse(content) as unknown)
if (!record) {
@@ -48,7 +63,7 @@ export function parseDevinSessionContent(
extractString(record.session_id) ??
extractString(record.sessionId) ??
sessionIdFromFileName(file.path)
const accumulator = createAccumulator({ agent: 'devin', file, sessionId })
const accumulator = createAccumulator({ agent: 'devin', file, sessionId, messages })
const agentRecord = asRecord(record.agent)
accumulator.model =
extractString(agentRecord?.model_name) ??
+104 -59
View File
@@ -1,10 +1,12 @@
import type { Dirent } from 'node:fs'
import { extname, join } from 'node:path'
import { SessionNewestFiles } from './session-newest-files'
import type { SessionSidecarObservation } from './session-sidecar-stat'
import type { AiVaultAgent, AiVaultScanIssue } from '../../shared/ai-vault-types'
import { wslGatedReaddir, wslGatedStat } from '../native-chat/wsl-transcript-fs-access'
import { WslTranscriptFsError } from '../native-chat/wsl-transcript-fs-gate'
import { recordSessionScanIssue } from './session-scan-issues'
import type { FileWithMtime, SessionFileDiscovery } from './session-scanner-types'
import type { SessionFileDiscovery } from './session-scanner-types'
import { errorMessage } from './session-scanner-values'
export async function discoverFiles(args: {
@@ -14,16 +16,55 @@ export async function discoverFiles(args: {
issues: AiVaultScanIssue[]
extensions: string[]
filePredicate?: (path: string) => boolean
contentDependencyPath?: (path: string) => string
contentDependencyPath?: (path: string) => string | undefined | Promise<string | undefined>
directoryPredicate?: (name: string, depth: number) => boolean
}): Promise<SessionFileDiscovery> {
let paths: string[]
const files = new SessionNewestFiles(args.limit)
let refusedSidecar = false
try {
paths = await walkSessionFiles(args.rootDir, args.agent, args.issues, {
extensions: new Set(args.extensions),
filePredicate: args.filePredicate,
directoryPredicate: args.directoryPredicate
})
await forEachSessionFile(
args.rootDir,
args.agent,
args.issues,
{
extensions: new Set(args.extensions),
filePredicate: args.filePredicate,
directoryPredicate: args.directoryPredicate
},
async (path) => {
try {
const fileStat = await wslGatedStat(path, 'scan')
const sidecarPath = await args.contentDependencyPath?.(path)
const sidecar = await observeSessionSidecar(sidecarPath)
if (sidecar === 'unknown' && !refusedSidecar) {
// One issue per root: a refused sibling is a property of the tree,
// not of each transcript that happens to point at it.
refusedSidecar = true
recordSessionScanIssue(args.issues, {
agent: args.agent,
path: sidecarPath ?? args.rootDir,
message: 'Session metadata could not be read this scan.'
})
}
files.add({
path,
mtimeMs: fileStat.mtimeMs,
modifiedAt: new Date(fileStat.mtimeMs).toISOString(),
sizeBytes: fileStat.size,
sidecar,
dev: fileStat.dev,
ino: fileStat.ino,
nlink: fileStat.nlink
})
} catch (err) {
recordSessionScanIssue(args.issues, {
agent: args.agent,
path,
message: errorMessage(err)
})
}
}
)
} catch (err) {
// Why: discoverAiVaultSessionSources fans out with Promise.all, so one
// stalled distro would otherwise reject the whole vault scan — including
@@ -38,68 +79,74 @@ export async function discoverFiles(args: {
})
return { agent: args.agent, rootDir: args.rootDir, files: [] }
}
const files: FileWithMtime[] = []
for (const path of paths) {
try {
const fileStat = await wslGatedStat(path, 'scan')
const dependencyStat = await optionalContentDependencyStat(args.contentDependencyPath?.(path))
const mtimeMs = Math.max(fileStat.mtimeMs, dependencyStat?.mtimeMs ?? 0)
files.push({
path,
mtimeMs,
modifiedAt: new Date(mtimeMs).toISOString(),
sizeBytes: fileStat.size + (dependencyStat?.size ?? 0),
dev: fileStat.dev,
ino: fileStat.ino,
nlink: fileStat.nlink
})
} catch (err) {
recordSessionScanIssue(args.issues, {
agent: args.agent,
path,
message: errorMessage(err)
})
}
}
return {
agent: args.agent,
rootDir: args.rootDir,
files: files.sort((left, right) => right.mtimeMs - left.mtimeMs).slice(0, args.limit)
}
return { agent: args.agent, rootDir: args.rootDir, files: files.newest() }
}
async function optionalContentDependencyStat(
/**
* A sibling that cannot be statted is not "no sibling": it must not take the
* transcript down with it, and it must not read as absent either, or the parse
* cache would treat a session enriched from a file nobody can see as current
* forever. Only a genuinely missing path is `'none'`; every other failure —
* a stalled WSL distro, EACCES, EIO — is `'unknown'`.
*/
async function observeSessionSidecar(
filePath: string | undefined
): Promise<{ mtimeMs: number; size: number } | null> {
): Promise<SessionSidecarObservation> {
if (!filePath) {
return null
return 'none'
}
try {
const fileStat = await wslGatedStat(filePath, 'scan')
return { mtimeMs: fileStat.mtimeMs, size: fileStat.size }
return { path: filePath, mtimeMs: fileStat.mtimeMs, sizeBytes: fileStat.size }
} catch (error) {
if (error instanceof WslTranscriptFsError) {
throw error
}
return null
return isMissingSidecarError(error) ? 'none' : 'unknown'
}
}
function isMissingSidecarError(error: unknown): boolean {
if (error instanceof WslTranscriptFsError) {
return false
}
const code =
error && typeof error === 'object' && 'code' in error && typeof error.code === 'string'
? error.code
: null
return code === 'ENOENT' || code === 'ENOTDIR'
}
export type SessionFileWalkOptions = {
extensions: Set<string>
filePredicate?: (path: string) => boolean
// Return false to skip descending into a directory; depth 0 is a child of
// rootDir, so pruned subtrees are never stat'd or parsed.
directoryPredicate?: (name: string, depth: number) => boolean
readDirectory?: (dirPath: string) => Promise<Dirent[]>
signal?: AbortSignal
}
/** Collecting form for callers that want every match; bounded scans stream. */
export async function walkSessionFiles(
dirPath: string,
agent: AiVaultAgent,
issues: AiVaultScanIssue[],
options: {
extensions: Set<string>
filePredicate?: (path: string) => boolean
// Return false to skip descending into a directory; depth 0 is a child of
// rootDir, so pruned subtrees are never stat'd or parsed.
directoryPredicate?: (name: string, depth: number) => boolean
readDirectory?: (dirPath: string) => Promise<Dirent[]>
signal?: AbortSignal
},
depth = 0
options: SessionFileWalkOptions
): Promise<string[]> {
const files: string[] = []
await forEachSessionFile(dirPath, agent, issues, options, async (path) => {
files.push(path)
})
return files
}
/** Streams matches to `onFile` so a bounded consumer never retains the whole tree. */
export async function forEachSessionFile(
dirPath: string,
agent: AiVaultAgent,
issues: AiVaultScanIssue[],
options: SessionFileWalkOptions,
onFile: (path: string) => Promise<void>,
depth = 0
): Promise<void> {
options.signal?.throwIfAborted()
let entries
try {
@@ -113,10 +160,9 @@ export async function walkSessionFiles(
if (error instanceof WslTranscriptFsError) {
throw error
}
return []
return
}
const files: string[] = []
for (const entry of entries) {
options.signal?.throwIfAborted()
const fullPath = join(dirPath, entry.name)
@@ -124,7 +170,7 @@ export async function walkSessionFiles(
// Skip whole subtrees an agent never wants (e.g. subagent transcripts),
// avoiding the readdir cost of descending into them.
if (options.directoryPredicate?.(entry.name, depth) ?? true) {
files.push(...(await walkSessionFiles(fullPath, agent, issues, options, depth + 1)))
await forEachSessionFile(fullPath, agent, issues, options, onFile, depth + 1)
}
continue
}
@@ -133,8 +179,7 @@ export async function walkSessionFiles(
options.extensions.has(extname(entry.name).toLowerCase()) &&
(options.filePredicate?.(fullPath) ?? true)
) {
files.push(fullPath)
await onFile(fullPath)
}
}
return files
}
@@ -8,6 +8,7 @@ import type {
ResumableSessionParseState,
SessionAccumulator
} from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
accumulatorFoldResumeState,
addPreviewMessage,
@@ -32,12 +33,13 @@ type ParserSessionOptions = {
export async function parseDroidSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const input = openTranscriptReadStream(file.path, { encoding: 'utf-8' }, 'scan')
const lines = createInterface({ input, crlfDelay: Infinity })
try {
return await parseDroidSessionLines({ file, lines, platform })
return await parseDroidSessionLines({ file, lines, platform, messages })
} finally {
// readline.close() leaves the underlying stream open; destroy it so a
// mid-parse throw cannot leak the gated transcript handle.
@@ -94,9 +96,17 @@ function consumeDroidRecordLine(accumulator: SessionAccumulator, line: string):
}
}
export function createDroidSessionResumeState(file: FileWithMtime): ResumableSessionParseState {
export function createDroidSessionResumeState(
file: FileWithMtime,
messages?: TranscriptMessageSink
): ResumableSessionParseState {
return accumulatorFoldResumeState(
createAccumulator({ agent: 'droid', file, sessionId: sessionIdFromFileName(file.path) }),
createAccumulator({
agent: 'droid',
file,
sessionId: sessionIdFromFileName(file.path),
messages
}),
consumeDroidRecordLine
)
}
@@ -106,8 +116,9 @@ async function parseDroidSessionLines(args: {
lines: AsyncIterable<string> | Iterable<string>
platform: NodeJS.Platform
options?: ParserSessionOptions
messages?: TranscriptMessageSink
}): Promise<AiVaultSession | null> {
const state = createDroidSessionResumeState(args.file)
const state = createDroidSessionResumeState(args.file, args.messages)
for await (const line of args.lines) {
state.consumeLine(line)
}
@@ -8,6 +8,7 @@ import type {
ResumableSessionParseState,
SessionAccumulator
} from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
accumulatorFoldResumeState,
addPreviewContent,
@@ -27,16 +28,19 @@ import {
export async function parseGeminiSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
if (file.path.endsWith('.jsonl')) {
return parseGeminiJsonlSessionFile(file, platform)
return parseGeminiJsonlSessionFile(file, platform, messages)
}
return parseGeminiJsonSessionContent(
file,
await wslGatedReadFile(file.path, 'utf-8', 'scan'),
platform
platform,
{},
messages
)
}
@@ -62,7 +66,8 @@ function parseGeminiJsonSessionContent(
file: FileWithMtime,
content: string,
platform: NodeJS.Platform,
options: ResumableParseFinalizeOptions = {}
options: ResumableParseFinalizeOptions = {},
messages?: TranscriptMessageSink
): AiVaultSession | null {
const record = asRecord(JSON.parse(content) as unknown)
if (!record) {
@@ -71,7 +76,8 @@ function parseGeminiJsonSessionContent(
const accumulator = createAccumulator({
agent: 'gemini',
file,
sessionId: extractString(record.sessionId) ?? sessionIdFromFileName(file.path)
sessionId: extractString(record.sessionId) ?? sessionIdFromFileName(file.path),
messages
})
updateTimeline(accumulator, extractString(record.startTime))
updateTimeline(accumulator, extractString(record.lastUpdated))
@@ -83,13 +89,14 @@ function parseGeminiJsonSessionContent(
export async function parseGeminiJsonlSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform
platform: NodeJS.Platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const lines = createInterface({
input: openTranscriptReadStream(file.path, { encoding: 'utf-8' }, 'scan'),
crlfDelay: Infinity
})
return parseGeminiJsonlSessionLines({ file, lines, platform })
return parseGeminiJsonlSessionLines({ file, lines, platform, messages })
}
function consumeGeminiJsonlRecordLine(accumulator: SessionAccumulator, line: string): void {
@@ -114,10 +121,16 @@ function consumeGeminiJsonlRecordLine(accumulator: SessionAccumulator, line: str
// Resumable only for the JSONL log format; Gemini's legacy single-JSON
// session documents are rewritten in place and must be re-read whole.
export function createGeminiJsonlSessionResumeState(
file: FileWithMtime
file: FileWithMtime,
messages?: TranscriptMessageSink
): ResumableSessionParseState {
return accumulatorFoldResumeState(
createAccumulator({ agent: 'gemini', file, sessionId: sessionIdFromFileName(file.path) }),
createAccumulator({
agent: 'gemini',
file,
sessionId: sessionIdFromFileName(file.path),
messages
}),
consumeGeminiJsonlRecordLine
)
}
@@ -127,8 +140,9 @@ async function parseGeminiJsonlSessionLines(args: {
lines: AsyncIterable<string> | Iterable<string>
platform: NodeJS.Platform
options?: ResumableParseFinalizeOptions
messages?: TranscriptMessageSink
}): Promise<AiVaultSession | null> {
const state = createGeminiJsonlSessionResumeState(args.file)
const state = createGeminiJsonlSessionResumeState(args.file, args.messages)
for await (const line of args.lines) {
state.consumeLine(line)
}
@@ -10,6 +10,7 @@ import type {
ResumableSessionParseState,
SessionAccumulator
} from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
accumulatorFoldResumeState,
addPreviewContent,
@@ -38,7 +39,8 @@ type ParserSessionOptions = {
export async function parseRovoSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const metadata = asRecord(
JSON.parse(await wslGatedReadFile(file.path, 'utf-8', 'scan')) as unknown
@@ -49,7 +51,8 @@ export async function parseRovoSessionFile(
const accumulator = createAccumulator({
agent: 'rovo',
file,
sessionId: basename(dirname(file.path))
sessionId: basename(dirname(file.path)),
messages
})
accumulator.title = firstString(metadata, ['title', 'name', 'summary'])
accumulator.cwd = firstString(metadata, [
@@ -174,12 +177,13 @@ export type MessageGraphAgent = 'openclaw' | 'pi' | 'omp' | 'prime-agent'
export async function parseMessageGraphSessionFile(
agent: MessageGraphAgent,
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const input = openTranscriptReadStream(file.path, { encoding: 'utf-8' }, 'scan')
const lines = createInterface({ input, crlfDelay: Infinity })
try {
return await parseMessageGraphSessionLines({ agent, file, lines, platform })
return await parseMessageGraphSessionLines({ agent, file, lines, platform, messages })
} finally {
// readline.close() leaves the underlying stream open; destroy it so a
// mid-parse throw cannot leak the gated transcript handle.
@@ -245,10 +249,11 @@ function consumeMessageGraphRecordLine(accumulator: SessionAccumulator, line: st
export function createMessageGraphSessionResumeState(
agent: MessageGraphAgent,
file: FileWithMtime
file: FileWithMtime,
messages?: TranscriptMessageSink
): ResumableSessionParseState {
const state = accumulatorFoldResumeState(
createAccumulator({ agent, file, sessionId: sessionIdFromFileName(file.path) }),
createAccumulator({ agent, file, sessionId: sessionIdFromFileName(file.path), messages }),
consumeMessageGraphRecordLine
)
// Why: only OMP materializes task-subagent transcripts beside its sessions
@@ -263,8 +268,9 @@ async function parseMessageGraphSessionLines(args: {
lines: AsyncIterable<string> | Iterable<string>
platform: NodeJS.Platform
options?: ParserSessionOptions
messages?: TranscriptMessageSink
}): Promise<AiVaultSession | null> {
const state = createMessageGraphSessionResumeState(args.agent, args.file)
const state = createMessageGraphSessionResumeState(args.agent, args.file, args.messages)
for await (const line of args.lines) {
state.consumeLine(line)
}
@@ -4,6 +4,7 @@ import { dirname, join } from 'node:path'
import { createInterface } from 'node:readline'
import type { AiVaultSession } from '../../shared/ai-vault-types'
import type { FileWithMtime, SessionAccumulator } from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
addPreviewMessage,
createAccumulator,
@@ -34,7 +35,8 @@ const GROK_USER_QUERY_PREVIEW_SCAN_LIMIT = 4096
export async function parseGrokSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const record = asRecord(JSON.parse(await wslGatedReadFile(file.path, 'utf-8', 'scan')) as unknown)
if (!record) {
@@ -42,7 +44,7 @@ export async function parseGrokSessionFile(
}
const info = asRecord(record.info)
const sessionId = extractString(info?.id) ?? sessionIdFromFileName(dirname(file.path))
const accumulator = createAccumulator({ agent: 'grok', file, sessionId })
const accumulator = createAccumulator({ agent: 'grok', file, sessionId, messages })
accumulator.cwd = extractString(info?.cwd)
accumulator.title =
normalizeTitleText(extractString(record.generated_title) ?? '') ??
@@ -2,6 +2,7 @@ import { wslGatedReadFile } from '../native-chat/wsl-transcript-fs-access'
import type { AiVaultSession } from '../../shared/ai-vault-types'
import type { ExecutionHostId } from '../../shared/execution-host'
import type { FileWithMtime } from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
addPreviewContent,
createAccumulator,
@@ -24,20 +25,34 @@ type ParserSessionOptions = {
export async function parseHermesSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
return parseHermesSessionContent(
return parseHermesSessionRecord(
file,
await wslGatedReadFile(file.path, 'utf-8', 'scan'),
platform
platform,
{},
messages
)
}
/** Remote transcript content, streamed from a host that has no reader attached. */
export async function parseHermesSessionContent(
file: FileWithMtime,
content: string,
platform: NodeJS.Platform = process.platform,
options: ParserSessionOptions = {}
): Promise<AiVaultSession | null> {
return parseHermesSessionRecord(file, content, platform, options)
}
async function parseHermesSessionRecord(
file: FileWithMtime,
content: string,
platform: NodeJS.Platform,
options: ParserSessionOptions,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const record = asRecord(JSON.parse(content) as unknown)
if (!record) {
@@ -46,7 +61,8 @@ export async function parseHermesSessionContent(
const accumulator = createAccumulator({
agent: 'hermes',
file,
sessionId: extractString(record.session_id) ?? sessionIdFromFileName(file.path)
sessionId: extractString(record.session_id) ?? sessionIdFromFileName(file.path),
messages
})
accumulator.model = extractString(record.model)
accumulator.cwd = extractString(record.cwd)
@@ -16,6 +16,7 @@ import {
readKimiWorkDirBySessionId
} from './session-scanner-kimi-paths'
import type { FileWithMtime, SessionAccumulator } from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
asRecord,
extractContentText,
@@ -32,7 +33,8 @@ import {
// session_index.jsonl; model/messages/tokens come from the wire transcript.
export async function parseKimiSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
let stateRecord: Record<string, unknown> | null
try {
@@ -53,7 +55,7 @@ export async function parseKimiSessionFile(
}
const sessionId = kimiSessionIdFromStatePath(file.path)
const accumulator = createAccumulator({ agent: 'kimi', file, sessionId })
const accumulator = createAccumulator({ agent: 'kimi', file, sessionId, messages })
// Why: Kimi sessions are work-dir-scoped — the resume command must `cd` into
// the original directory or the CLI rejects it. That path lives only in the
@@ -3,6 +3,7 @@ import { WslTranscriptFsError } from '../native-chat/wsl-transcript-fs-gate'
import { join } from 'node:path'
import type { AiVaultSession } from '../../shared/ai-vault-types'
import type { FileWithMtime, SessionAccumulator } from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
addPreviewMessage,
createAccumulator,
@@ -25,14 +26,15 @@ import {
export async function parseOpenCodeSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const record = asRecord(JSON.parse(await wslGatedReadFile(file.path, 'utf-8', 'scan')) as unknown)
if (!record) {
return null
}
const sessionId = extractString(record.id) ?? sessionIdFromFileName(file.path)
const accumulator = createAccumulator({ agent: 'opencode', file, sessionId })
const accumulator = createAccumulator({ agent: 'opencode', file, sessionId, messages })
accumulator.title = normalizeTitleText(extractString(record.title) ?? '')
accumulator.cwd = extractString(record.directory)
updateTimeline(accumulator, timeObjectValue(record.time, 'created'))
@@ -0,0 +1,29 @@
import type SyncDatabase from '../sqlite/sync-database'
import { columnExists, tableExists } from '../opencode-usage/schema-helpers'
// Why: OpenCode's schema has moved more than once, so every read probes for the
// columns it names. These are the two shapes the session parser depends on;
// keeping them here stops each reader from inventing its own partial gate.
/** Enough of `message` to count a session's turns. */
export function canCountOpenCodeMessages(db: SyncDatabase): boolean {
return (
tableExists(db, 'message') &&
columnExists(db, 'message', 'session_id') &&
columnExists(db, 'message', 'data')
)
}
/** Enough of `message`×`part` to read a session's parts in turn order. */
export function canReadOpenCodeMessageParts(db: SyncDatabase): boolean {
return (
canCountOpenCodeMessages(db) &&
columnExists(db, 'message', 'id') &&
// Every parts read orders by it; unprobed, a schema without it throws mid-read.
columnExists(db, 'message', 'time_created') &&
tableExists(db, 'part') &&
columnExists(db, 'part', 'message_id') &&
columnExists(db, 'part', 'time_created') &&
columnExists(db, 'part', 'data')
)
}
@@ -1,4 +1,4 @@
import type { Worker } from 'node:worker_threads'
import { LazyWorkerThreadHost, type WorkerThreadFactory } from '../lazy-worker-thread-host'
import type { AiVaultScanIssue, AiVaultSession } from '../../shared/ai-vault-types'
import type {
OpenCodeSqliteListRequest,
@@ -11,9 +11,10 @@ import type { SessionFileCandidate } from './session-scanner-types'
import { errorMessage } from './session-scanner-values'
// Why (#8864): a lazily-spawned, unref'd worker runs OpenCode SQLite reads off
// the main-process event loop. Lifecycle (idle teardown, FIFO one-at-a-time
// dispatch, per-call timeouts, respawn-on-fault) mirrors src/main/speech/
// stt-service.ts. The default spawn + shared singleton live in
// the main-process event loop. This module owns the request half (FIFO
// one-at-a-time dispatch, per-call timeouts, respawn-on-fault); the thread's
// own lifetime belongs to LazyWorkerThreadHost, shared with the port-scan probe
// client. The default spawn + shared singleton live in
// session-scanner-opencode-sqlite-worker-spawn.ts.
export const LIST_TIMEOUT_MS = 30_000
@@ -25,8 +26,6 @@ export const IDLE_TEARDOWN_MS = 30_000
// fresh scan burst starts from idle (so the cap is per-scan, not process-wide).
export const MAX_CONSECUTIVE_DEATHS = 3
export type WorkerFactory = () => Worker
// Omit<union, 'id'> collapses to the shared keys, so omit each member and let
// the client stamp the correlation id.
type OpenCodeSqliteRequestBody =
@@ -53,20 +52,27 @@ class OpenCodeSqliteWorkerUnavailableError extends Error {}
* no worker can be spawned rather than moving SQLite work onto the main thread.
*/
export class OpenCodeSqliteWorkerClient {
private worker: Worker | null = null
private active: PendingCall | null = null
private queue: PendingCall[] = []
private idleTimer: NodeJS.Timeout | null = null
private consecutiveDeaths = 0
private nextId = 1
private loggedWorkerUnavailable = false
private cleanupWorkerListeners: (() => void) | null = null
private readonly workerFactory: WorkerFactory
private readonly log: (message: string) => void
private readonly host: LazyWorkerThreadHost<OpenCodeSqliteWorkerResponse>
constructor(options: { workerFactory: WorkerFactory; log?: (message: string) => void }) {
this.workerFactory = options.workerFactory
this.log = options.log ?? ((message) => console.warn(message))
constructor(options: { workerFactory: WorkerThreadFactory; log?: (message: string) => void }) {
const log = options.log ?? ((message: string) => console.warn(message))
this.host = new LazyWorkerThreadHost<OpenCodeSqliteWorkerResponse>({
factory: options.workerFactory,
idleTeardownMs: IDLE_TEARDOWN_MS,
onMessage: (response) => this.onMessage(response),
onError: (error) => this.onWorkerFault(error),
onExit: (code) => this.onWorkerExit(code),
isIdle: () => !this.active && this.queue.length === 0,
// Why (#8864): never fall back to synchronous SQLite reads here; a missing
// bundle or resource-exhausted spawn must omit OpenCode history rather than
// reintroduce the main-process hang this worker boundary prevents.
onUnavailable: (err) =>
log(`OpenCode SQLite worker unavailable; skipping its history. ${errorMessage(err)}`)
})
}
/**
@@ -167,7 +173,7 @@ export class OpenCodeSqliteWorkerClient {
if (this.active || this.queue.length === 0) {
return
}
const worker = this.ensureWorker()
const worker = this.host.ensure()
if (!worker) {
this.failQueuedAsUnavailable()
return
@@ -177,7 +183,7 @@ export class OpenCodeSqliteWorkerClient {
return
}
this.active = call
this.clearIdleTimer()
this.host.clearIdleTimer()
// Timeout clock starts at dispatch (not enqueue): a batch may enqueue up to
// 8 parses at once, and a queue-inclusive timeout would fire falsely.
call.timer = setTimeout(() => this.onTimeout(call), call.timeoutMs)
@@ -185,39 +191,6 @@ export class OpenCodeSqliteWorkerClient {
worker.postMessage(call.request)
}
private ensureWorker(): Worker | null {
if (this.worker) {
return this.worker
}
try {
const worker = this.workerFactory()
const onMessage = (response: OpenCodeSqliteWorkerResponse): void => this.onMessage(response)
const onError = (error: Error): void => this.onWorkerFault(error)
const onExit = (code: number): void => this.onWorkerExit(code)
worker.on('message', onMessage)
worker.on('error', onError)
worker.on('exit', onExit)
this.cleanupWorkerListeners = () => {
worker.off('message', onMessage)
worker.off('error', onError)
worker.off('exit', onExit)
}
// Never keep the app alive for a scan worker.
worker.unref?.()
this.worker = worker
return worker
} catch (err) {
// Why (#8864): never fall back to synchronous SQLite reads here; a missing
// bundle or resource-exhausted spawn must omit OpenCode history rather than
// reintroduce the main-process hang this worker boundary prevents.
if (!this.loggedWorkerUnavailable) {
this.loggedWorkerUnavailable = true
this.log(`OpenCode SQLite worker unavailable; skipping its history. ${errorMessage(err)}`)
}
return null
}
}
private onMessage(response: OpenCodeSqliteWorkerResponse): void {
const call = this.active
if (!call || call.request.id !== response.id) {
@@ -243,7 +216,7 @@ export class OpenCodeSqliteWorkerClient {
// A clean self-exit is not a death, but the stale handle must be dropped
// or the next dispatch would post into the dead worker and stall to timeout.
if (code === 0 && !this.active && this.queue.length === 0) {
this.destroyWorker()
this.host.destroy()
return
}
this.onWorkerFault(new Error(`OpenCode SQLite worker exited with code ${code}`))
@@ -251,7 +224,7 @@ export class OpenCodeSqliteWorkerClient {
private onWorkerFault(error: Error): void {
const failed = this.active
this.destroyWorker()
this.host.destroy()
this.consecutiveDeaths++
if (failed) {
this.settle(failed, () => failed.reject(error))
@@ -302,46 +275,7 @@ export class OpenCodeSqliteWorkerClient {
if (this.queue.length > 0) {
this.pump()
} else {
this.scheduleIdleTeardown()
this.host.scheduleIdleTeardown()
}
}
private scheduleIdleTeardown(): void {
this.clearIdleTimer()
if (!this.worker) {
return
}
this.idleTimer = setTimeout(() => this.teardownIfIdle(), IDLE_TEARDOWN_MS)
this.idleTimer.unref?.()
}
private teardownIfIdle(): void {
this.idleTimer = null
// Only tear down with nothing active AND nothing queued: a request arriving
// as the timer fires must never be lost to a self-exiting worker.
if (this.active || this.queue.length > 0) {
return
}
this.destroyWorker()
}
private clearIdleTimer(): void {
if (this.idleTimer) {
clearTimeout(this.idleTimer)
this.idleTimer = null
}
}
private destroyWorker(): void {
this.clearIdleTimer()
const worker = this.worker
this.worker = null
if (!worker) {
return
}
this.cleanupWorkerListeners?.()
this.cleanupWorkerListeners = null
worker.removeAllListeners()
void worker.terminate().catch(() => undefined)
}
}
@@ -10,6 +10,10 @@ import {
shouldCaptureFullFirstUserPrompt
} from './session-scanner-first-user-prompt'
import { readOpenCodeDatabase } from './session-scanner-opencode-sqlite-open'
import {
canCountOpenCodeMessages,
canReadOpenCodeMessageParts
} from './session-scanner-opencode-sqlite-schema'
import { normalizeTitleText } from './session-scanner-values'
import type SyncDatabase from '../sqlite/sync-database'
import { columnExists, tableExists } from '../opencode-usage/schema-helpers'
@@ -70,14 +74,6 @@ function sessionNumberColumnSelect(db: SyncDatabase, columnName: string): string
return columnExists(db, 'session', columnName) ? `s.${columnName}` : '0'
}
function canCountOpenCodeMessages(db: SyncDatabase): boolean {
return (
tableExists(db, 'message') &&
columnExists(db, 'message', 'session_id') &&
columnExists(db, 'message', 'data')
)
}
function buildSessionQuery(db: SyncDatabase): string {
const messageCountSubquery = canCountOpenCodeMessages(db)
? `(SELECT COUNT(*) FROM message m
@@ -154,14 +150,7 @@ function extractPartText(partData: string): string | null {
}
function readFirstUserPromptFromOpenCodeDb(db: SyncDatabase, sessionId: string): string | null {
if (
!canCountOpenCodeMessages(db) ||
!tableExists(db, 'part') ||
!columnExists(db, 'message', 'id') ||
!columnExists(db, 'part', 'message_id') ||
!columnExists(db, 'part', 'time_created') ||
!columnExists(db, 'part', 'data')
) {
if (!canReadOpenCodeMessageParts(db)) {
return null
}
@@ -206,14 +195,7 @@ function readFirstUserPromptFromOpenCodeDb(db: SyncDatabase, sessionId: string):
}
function buildPreviewQuery(db: SyncDatabase): string | null {
if (
!canCountOpenCodeMessages(db) ||
!tableExists(db, 'part') ||
!columnExists(db, 'message', 'id') ||
!columnExists(db, 'part', 'message_id') ||
!columnExists(db, 'part', 'time_created') ||
!columnExists(db, 'part', 'data')
) {
if (!canReadOpenCodeMessageParts(db)) {
return null
}
return `SELECT json_extract(m.data, '$.role') AS role,
@@ -205,6 +205,61 @@ describe('codex-specific resume behavior', () => {
})
describe('non-resumable formats keep reuse-only caching', () => {
it('re-parses cline when only its messages sidecar changed', async () => {
const root = await makeTempDir()
const sessionDir = join(root, 'cline-1')
await mkdir(sessionDir, { recursive: true })
const metadataPath = join(sessionDir, 'cline-1.json')
const messagesPath = join(sessionDir, 'cline-1.messages.json')
await writeFile(
metadataPath,
JSON.stringify({
session_id: 'cline-1',
cwd: '/tmp/cline',
started_at: '2026-05-01T10:00:00Z'
})
)
const writeMessages = (text: string): Promise<void> =>
writeFile(
messagesPath,
JSON.stringify({
updated_at: '2026-05-01T10:00:01Z',
messages: [{ role: 'user', content: [{ type: 'text', text }] }]
})
)
await writeMessages('first ask')
// Cline reads the sidecar as part of its parse, so a change to it has to
// re-parse; there is no metadata-only merge to re-run.
const candidate = async (): Promise<SessionFileCandidate> => {
const base = await candidateFor('cline', metadataPath)
const sidecarStat = await stat(messagesPath)
return {
...base,
file: {
...base.file,
sidecar: {
path: messagesPath,
mtimeMs: sidecarStat.mtimeMs,
sizeBytes: sidecarStat.size
}
}
}
}
const stats = createSessionParseStats()
const seeded = await parseAgentSessionFileCached(await candidate(), process.platform, stats)
expect(seeded?.title).toBe('first ask')
await parseAgentSessionFileCached(await candidate(), process.platform, stats)
expect(stats).toMatchObject({ fullParses: 1, reused: 1 })
await writeMessages('second ask, rather longer than the first')
const rewritten = await parseAgentSessionFileCached(await candidate(), process.platform, stats)
expect(rewritten?.title).toBe('second ask, rather longer than the first')
expect(stats).toMatchObject({ fullParses: 2, reused: 1 })
})
it('re-parses a changed grok summary fully and reuses it when unchanged', async () => {
const root = await makeTempDir()
const sessionDir = join(root, 'session-1')
+124 -228
View File
@@ -1,7 +1,6 @@
import { readTranscriptSlice } from '../native-chat/wsl-transcript-fs-access'
import type { AiVaultSession } from '../../shared/ai-vault-types'
import { inSessionParseFileLane } from './session-parse-file-lane'
import { createAntigravitySessionResumeState } from './session-scanner-antigravity-parser'
import { parseAgentSessionFile } from './session-scanner-agent-parser'
import { createCodexSessionResumeState } from './session-scanner-codex-parser'
import { createDroidSessionResumeState } from './session-scanner-droid-parser'
import { createMessageGraphSessionResumeState } from './session-scanner-graph-parsers'
@@ -13,27 +12,30 @@ import { countSubagentTranscripts } from './session-scanner-subagent-transcripts
import { countOmpSubagentTranscripts } from './session-scanner-omp-subagent-transcripts'
import type { ResumableSessionParseState, SessionFileCandidate } from './session-scanner-types'
import { refreshCachedCodexTitle } from './session-scanner-codex-cached-title'
import { consumeCompleteJsonlLines } from './session-scanner-jsonl-reader'
import {
getSessionParseCacheEntry,
storeSessionParseCacheEntry,
type SessionParseCacheEntry
} from './session-parse-cache-store'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import { sidecarUnchanged } from './session-sidecar-stat'
import {
enrichSessionFromSidecar,
sidecarEnrichesWithoutReparse
} from './session-scanner-sidecar-enrichment'
import {
readResumableTranscript,
readWholeTranscript,
type TranscriptReadStats
} from './session-transcript-reader'
// Sized past the default recency cap (1000) plus the in-scope cap (2000) so a
// full steady-state result set stays resident between forced rescans.
const MAX_CACHE_ENTRIES = 4096
const NEWLINE_BYTE = 0x0a
type ResumePoint = {
state: ResumableSessionParseState
// Byte offset just past the last complete ('\n'-terminated) line consumed;
// a trailing unterminated line is deliberately left before this point.
byteOffset: number
}
type SessionParseCacheEntry = {
mtimeMs: number
sizeBytes: number | null
platform: NodeJS.Platform
session: AiVaultSession | null
resume: ResumePoint | null
}
export {
invalidateSessionParseCacheEntry,
resetSessionParseCacheForTests,
seedSessionParseCache,
snapshotSessionParseCacheForPersistence,
type PersistedSessionParseCacheEntry
} from './session-parse-cache-store'
// Incremental append-parsing applies only to transcripts that are append-only
// JSONL line-folds. Whole-JSON documents (grok/rovo/devin/hermes/gemini-json)
@@ -44,31 +46,32 @@ type SessionParseCacheEntry = {
// cached state instead, never pay for a throwaway accumulator.
function resumableStateFactoryFor(
candidate: SessionFileCandidate
): (() => ResumableSessionParseState) | null {
): ((messages: TranscriptMessageSink) => ResumableSessionParseState) | null {
switch (candidate.agent) {
case 'claude':
return () => createClaudeSessionResumeState(candidate.file)
return (messages) => createClaudeSessionResumeState(candidate.file, messages)
case 'codex':
return () => createCodexSessionResumeState(candidate.file, candidate.codexHome)
return (messages) =>
createCodexSessionResumeState(candidate.file, candidate.codexHome, messages)
case 'cursor':
return () => createCursorSessionResumeState(candidate.file)
return (messages) => createCursorSessionResumeState(candidate.file, messages)
case 'copilot':
return () => createCopilotSessionResumeState(candidate.file)
return (messages) => createCopilotSessionResumeState(candidate.file, messages)
case 'droid':
return () => createDroidSessionResumeState(candidate.file)
return (messages) => createDroidSessionResumeState(candidate.file, messages)
case 'openclaw':
case 'pi':
case 'omp':
case 'prime-agent': {
const agent = candidate.agent
return () => createMessageGraphSessionResumeState(agent, candidate.file)
return (messages) => createMessageGraphSessionResumeState(agent, candidate.file, messages)
}
case 'gemini':
return candidate.file.path.endsWith('.jsonl')
? () => createGeminiJsonlSessionResumeState(candidate.file)
? (messages) => createGeminiJsonlSessionResumeState(candidate.file, messages)
: null
case 'antigravity':
return () => createAntigravitySessionResumeState(candidate.file)
return (messages) => createAntigravitySessionResumeState(candidate.file, messages)
case 'devin':
case 'grok':
case 'hermes':
@@ -80,245 +83,138 @@ function resumableStateFactoryFor(
}
}
export type SessionParseStats = {
export type SessionParseStats = TranscriptReadStats & {
reused: number
incremental: number
fullParses: number
// Transcripts the parser already excluded (Codex workers), re-listed after a
// write and dismissed without reading. Counted apart from `incremental` so a
// scan span still shows how much work the early stop actually removed.
earlyStopped: number
bytesRead: number
}
export function createSessionParseStats(): SessionParseStats {
return { reused: 0, incremental: 0, fullParses: 0, earlyStopped: 0, bytesRead: 0 }
}
const cache = new Map<string, SessionParseCacheEntry>()
export function resetSessionParseCacheForTests(): void {
cache.clear()
}
// Drops one entry after its file is deleted. Cleanliness, not correctness:
// discovery walks disk first, so a trashed file is never rediscovered anyway.
export function invalidateSessionParseCacheEntry(path: string): void {
cache.delete(path)
}
// Persisted subset of a cache entry: the non-serializable `resume` parser
// state is dropped (see session-parse-cache-persistence.ts).
export type PersistedSessionParseCacheEntry = Omit<SessionParseCacheEntry, 'resume'>
export function snapshotSessionParseCacheForPersistence(): [
string,
PersistedSessionParseCacheEntry
][] {
return [...cache].map(([path, entry]): [string, PersistedSessionParseCacheEntry] => [
path,
{
mtimeMs: entry.mtimeMs,
sizeBytes: entry.sizeBytes,
platform: entry.platform,
session: entry.session
}
])
}
// Seeded entries carry `resume: null`: after a restart an unchanged file is a
// cache hit; a file that changed while the app was closed pays one full
// (not incremental) re-parse.
export function seedSessionParseCache(
entries: Iterable<[string, PersistedSessionParseCacheEntry]>
): void {
const list = [...entries]
// Snapshot order is oldest→newest (LRU); an over-cap list keeps the newest
// tail rather than seeding the oldest entries and dropping the tail.
for (const [path, entry] of list.slice(Math.max(0, list.length - MAX_CACHE_ENTRIES))) {
if (cache.size >= MAX_CACHE_ENTRIES) {
return
}
// In-process entries are always fresher than persisted ones; never clobber.
if (cache.has(path)) {
continue
}
cache.set(path, {
mtimeMs: entry.mtimeMs,
sizeBytes: entry.sizeBytes,
platform: entry.platform,
session: entry.session,
resume: null
})
}
}
function storeEntry(path: string, entry: SessionParseCacheEntry): void {
cache.delete(path)
cache.set(path, entry)
if (cache.size > MAX_CACHE_ENTRIES) {
const oldest = cache.keys().next()
if (!oldest.done) {
cache.delete(oldest.value)
}
}
}
/**
* Parse a session file, reusing prior work where the file is provably
* unchanged (mtime+size) and, for append-only JSONL transcripts (Claude,
* Codex, Cursor, Copilot, Droid, OpenClaw/Pi/OMP, Gemini-JSONL), resuming the
* parse from the last consumed byte when the file only grew. This is what
* keeps the renderer's ~5s forced rescans from re-reading gigabytes of
* transcripts (STA-1278/STA-1417: main process pegging one core during
* multi-agent workloads).
* The session list's cursor over the transcript reader: it remembers what each
* file looked like when it was last listed, reuses that work where the file is
* provably unchanged (mtime+size), and otherwise asks the reader to resume from
* the last consumed byte or re-read the file whole. This is what keeps the
* renderer's ~5s forced rescans from re-reading gigabytes of transcripts
* (STA-1278/STA-1417: main process pegging one core during multi-agent
* workloads). Other consumers of the reader keep their own equivalent cursor
* and never consult this one.
*/
export async function parseAgentSessionFileCached(
candidate: SessionFileCandidate,
platform: NodeJS.Platform,
stats?: SessionParseStats
): Promise<AiVaultSession | null> {
const { file } = candidate
const entry = cache.get(file.path)
// The whole lookup-read-store sequence runs in the lane: a concurrent parse of
// the same path shares this entry's resume point and its message channel.
return inSessionParseFileLane(candidate.file.path, () =>
parseCachedInLane(candidate, platform, stats)
)
}
const unchanged =
async function parseCachedInLane(
candidate: SessionFileCandidate,
platform: NodeJS.Platform,
stats?: SessionParseStats
): Promise<AiVaultSession | null> {
const { file } = candidate
const entry = getSessionParseCacheEntry(file.path)
const transcriptUnchanged =
entry !== undefined &&
entry.platform === platform &&
entry.mtimeMs === file.mtimeMs &&
(entry.sizeBytes === null || file.sizeBytes === undefined || entry.sizeBytes === file.sizeBytes)
if (unchanged) {
if (stats) {
stats.reused++
if (transcriptUnchanged) {
if (sidecarUnchanged(entry.sidecar, file.sidecar)) {
return reuseCachedSession(candidate, entry, stats)
}
// A zero-turn transcript usually never changes again, but its sibling
// subagent dir (Claude `<session>/subagents/`, OMP's same-named artifact
// dir) can gain files after the parent's last write (a still-running
// subagent finishing). The mtime+size key can't see that, so refresh the
// cheap directory count on reuse.
if (entry.session && entry.session.messageCount === 0) {
const subagentTranscriptCount =
candidate.agent === 'claude'
? await countSubagentTranscripts(file.path)
: candidate.agent === 'omp'
? await countOmpSubagentTranscripts(file.path)
: null
if (
subagentTranscriptCount !== null &&
subagentTranscriptCount !== entry.session.subagentTranscriptCount
) {
entry.session = { ...entry.session, subagentTranscriptCount }
// Only the sibling moved. For an agent whose sibling just adds metadata,
// re-merge it onto the stored fold result; the transcript is not re-read.
if (sidecarEnrichesWithoutReparse(candidate) && entry.foldSession !== undefined) {
const enriched = await enrichSessionFromSidecar(candidate, entry.foldSession, platform)
entry.session = enriched.session
entry.sidecar = enriched.refused ? 'unknown' : file.sidecar
storeSessionParseCacheEntry(file.path, entry)
if (stats) {
stats.reused++
}
return entry.session
}
// Codex titles come from session_index.jsonl, which mtime+size can't see.
// Remote counterpart: remote-session-scanner.ts's reusedCodexTitleRefresh.
if (entry.session && candidate.agent === 'codex') {
entry.session = await refreshCachedCodexTitle(candidate, entry.session)
}
storeEntry(file.path, entry)
return entry.session
}
const stateFactory = resumableStateFactoryFor(candidate)
if (stateFactory) {
const parsed = await parseResumableCandidate({
const read = await readResumableTranscript({
candidate,
platform,
entry,
stats,
stateFactory
resume: entry?.platform === platform ? entry.resume : null,
stateFactory,
stats
})
storeEntry(file.path, parsed)
return parsed.session
const enriched = await enrichSessionFromSidecar(candidate, read.session, platform)
storeSessionParseCacheEntry(file.path, {
mtimeMs: file.mtimeMs,
sizeBytes: file.sizeBytes ?? null,
platform,
session: enriched.session,
// A refused sibling leaves the transcript's own work cached and resumable;
// only the sibling is recorded as unknown, so the next healthy scan
// re-merges it without re-reading the transcript.
sidecar: enriched.refused ? 'unknown' : file.sidecar,
foldSession: read.session,
resume: read.resume
})
return enriched.session
}
if (stats) {
stats.fullParses++
stats.bytesRead += file.sizeBytes ?? 0
}
const session = await parseAgentSessionFile(candidate, platform)
storeEntry(file.path, {
const session = await readWholeTranscript({ candidate, platform, stats })
storeSessionParseCacheEntry(file.path, {
mtimeMs: file.mtimeMs,
sizeBytes: file.sizeBytes ?? null,
platform,
session,
// A whole-file parse reads the sibling itself, so a change to it re-parses.
sidecar: file.sidecar,
foldSession: session,
resume: null
})
return session
}
async function parseResumableCandidate(args: {
candidate: SessionFileCandidate
platform: NodeJS.Platform
entry: SessionParseCacheEntry | undefined
async function reuseCachedSession(
candidate: SessionFileCandidate,
entry: SessionParseCacheEntry,
stats?: SessionParseStats
stateFactory: () => ResumableSessionParseState
}): Promise<SessionParseCacheEntry> {
const { file } = args.candidate
const resume = args.entry?.platform === args.platform ? args.entry.resume : null
const canResume =
resume !== null &&
resume !== undefined &&
typeof file.sizeBytes === 'number' &&
file.sizeBytes >= resume.byteOffset &&
(resume.byteOffset === 0 || (await endsWithNewlineAt(file.path, resume.byteOffset)))
// Clone before consuming: a failed read must not corrupt the cached state,
// or the next resume would double-count the lines applied before the error.
const state = canResume ? resume.state.clone() : args.stateFactory()
const startOffset = canResume ? resume.byteOffset : 0
// Mirrors the reader's entry guard so a dismissed transcript is not reported
// as an incremental parse that read nothing.
const stoppedBeforeRead = state.shouldStop?.() === true
if (args.stats) {
if (stoppedBeforeRead) {
args.stats.earlyStopped++
} else if (canResume) {
args.stats.incremental++
} else {
args.stats.fullParses++
): Promise<AiVaultSession | null> {
if (stats) {
stats.reused++
}
// A zero-turn transcript usually never changes again, but its sibling
// subagent dir (Claude `<session>/subagents/`, OMP's same-named artifact
// dir) can gain files after the parent's last write (a still-running
// subagent finishing). The mtime+size key can't see that, so refresh the
// cheap directory count on reuse.
if (entry.session && entry.session.messageCount === 0) {
const subagentTranscriptCount =
candidate.agent === 'claude'
? await countSubagentTranscripts(candidate.file.path)
: candidate.agent === 'omp'
? await countOmpSubagentTranscripts(candidate.file.path)
: null
if (
subagentTranscriptCount !== null &&
subagentTranscriptCount !== entry.session.subagentTranscriptCount
) {
entry.session = { ...entry.session, subagentTranscriptCount }
}
}
const readResult = await consumeCompleteJsonlLines({
path: file.path,
start: startOffset,
onLine: (line) => state.consumeLine(line),
// Bound: the optional hooks are declared as methods, so a parser written
// with method syntax must not lose `this` on the way into the reader.
onLineBytes: state.consumeLineBytes?.bind(state),
shouldStop: state.shouldStop?.bind(state)
})
if (args.stats) {
args.stats.bytesRead += readResult.bytesRead
}
// The stat this scan displays is current even when nothing new was consumed.
state.touchFile(file)
// Keep parity with the one-shot parser: a final unterminated line is shown,
// but stays out of the resumable state so the (possibly still-growing) line
// is re-read once complete instead of being half-counted.
let displayState = state
if (readResult.trailingPartialLine !== null) {
displayState = state.clone()
displayState.consumeLine(readResult.trailingPartialLine)
}
return {
mtimeMs: file.mtimeMs,
sizeBytes: file.sizeBytes ?? null,
platform: args.platform,
session: await displayState.finalize(args.platform),
resume: { state, byteOffset: readResult.consumedThrough }
// Codex titles come from session_index.jsonl, which mtime+size can't see.
// Remote counterpart: remote-session-scanner.ts's reusedCodexTitleRefresh.
if (entry.session && candidate.agent === 'codex') {
entry.session = await refreshCachedCodexTitle(candidate, entry.session)
}
}
// A resume point is only valid if it still sits just past a line break;
// anything else means the file was rewritten, not appended. Heuristic: a
// grown rewrite keeping '\n' at exactly this byte would slip through, but
// agent transcripts are append-only so that trade is accepted (worst case is
// a stale vault row until the file is next truncated or the app restarts).
async function endsWithNewlineAt(path: string, offset: number): Promise<boolean> {
const slice = await readTranscriptSlice(path, offset - 1, 1, 'scan')
return slice.length === 1 && slice[0] === NEWLINE_BYTE
storeSessionParseCacheEntry(candidate.file.path, entry)
return entry.session
}
@@ -10,6 +10,7 @@ import type {
ResumableSessionParseState,
SessionAccumulator
} from './session-scanner-types'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import {
addPreviewContent,
createAccumulator,
@@ -42,12 +43,16 @@ export type ClaudeSessionParseState = {
firstUserTitle: string | null
}
export function createClaudeSessionParseState(file: FileWithMtime): ClaudeSessionParseState {
export function createClaudeSessionParseState(
file: FileWithMtime,
messages?: TranscriptMessageSink
): ClaudeSessionParseState {
return {
accumulator: createAccumulator({
agent: 'claude',
file,
sessionId: sessionIdFromFileName(file.path)
sessionId: sessionIdFromFileName(file.path),
messages
}),
metaTitle: null,
generatedTitle: null,
@@ -188,8 +193,11 @@ export async function finalizeClaudeSessionParseState(
return finalizeSession(snapshot.accumulator, platform, options)
}
export function createClaudeSessionResumeState(file: FileWithMtime): ResumableSessionParseState {
return claudeResumeStateFromParseState(createClaudeSessionParseState(file))
export function createClaudeSessionResumeState(
file: FileWithMtime,
messages?: TranscriptMessageSink
): ResumableSessionParseState {
return claudeResumeStateFromParseState(createClaudeSessionParseState(file, messages))
}
function claudeResumeStateFromParseState(
@@ -207,13 +215,14 @@ function claudeResumeStateFromParseState(
export async function parseClaudeSessionFile(
file: FileWithMtime,
platform: NodeJS.Platform = process.platform
platform: NodeJS.Platform = process.platform,
messages?: TranscriptMessageSink
): Promise<AiVaultSession | null> {
const lines = createInterface({
input: openTranscriptReadStream(file.path, { encoding: 'utf-8' }, 'scan'),
crlfDelay: Infinity
})
return parseClaudeSessionLines({ file, lines, platform })
return parseClaudeSessionLines({ file, lines, platform, messages })
}
export async function parseClaudeSessionContent(
@@ -236,8 +245,9 @@ async function parseClaudeSessionLines(args: {
lines: AsyncIterable<string> | Iterable<string>
platform: NodeJS.Platform
options?: ParserSessionOptions
messages?: TranscriptMessageSink
}): Promise<AiVaultSession | null> {
const state = createClaudeSessionParseState(args.file)
const state = createClaudeSessionParseState(args.file, args.messages)
for await (const line of args.lines) {
consumeClaudeSessionLine(state, line)
}
@@ -0,0 +1,79 @@
import type { AiVaultSession } from '../../shared/ai-vault-types'
import { buildAiVaultResumeCommand } from '../../shared/ai-vault-resume-command'
import { generatedSessionTitle } from './session-scanner-accumulator'
import { readCursorChatMeta, wasCursorChatMetaRefused } from './session-scanner-cursor-chat-meta'
import type { SessionFileCandidate } from './session-scanner-types'
/**
* Merges an agent's sibling file onto the session its transcript alone
* produced. Kept out of the fold and applied here so it is a pure function of
* (fold result, sibling): re-running it starts from what the transcript said,
* never from a previous merge, so a rewritten sibling replaces the fields it
* supplied last time instead of losing to them.
*
* The reuse-time counterpart of session-scanner-codex-cached-title.ts, for
* agents whose sibling the transcript key cannot see.
*/
export type SidecarEnrichment = {
session: AiVaultSession | null
/** The sibling could not be read; the caller records the observation as unknown. */
refused: boolean
}
/** True when the sibling only adds metadata, so a change to it needs no re-parse. */
export function sidecarEnrichesWithoutReparse(candidate: SessionFileCandidate): boolean {
return candidate.agent === 'cursor'
}
export async function enrichSessionFromSidecar(
candidate: SessionFileCandidate,
foldSession: AiVaultSession | null,
platform: NodeJS.Platform
): Promise<SidecarEnrichment> {
if (candidate.agent !== 'cursor' || !foldSession) {
return { session: foldSession, refused: false }
}
const meta = await readCursorChatMeta(candidate.file.path)
if (!meta) {
return { session: foldSession, refused: wasCursorChatMetaRefused(candidate.file.path) }
}
return { session: mergeCursorChatMeta(foldSession, meta, platform), refused: false }
}
/** Fills only what the transcript never recorded; its own records always win. */
export function mergeCursorChatMeta(
session: AiVaultSession,
meta: {
title: string | null
cwd: string | null
createdAt: string | null
updatedAt: string | null
},
platform: NodeJS.Platform
): AiVaultSession {
// A generated title means the fold found none, so the sibling's may stand in.
const named = session.title !== generatedSessionTitle(session.agent, session.sessionId)
const cwd = session.cwd ?? meta.cwd
const merged: AiVaultSession = {
...session,
title: named ? session.title : (meta.title ?? session.title),
cwd,
createdAt: session.createdAt ?? meta.createdAt,
updatedAt: session.updatedAt ?? meta.updatedAt
}
if (cwd === session.cwd) {
return merged
}
// The resume command embeds the cwd, so it has to be rebuilt with it.
return {
...merged,
resumeCommand: buildAiVaultResumeCommand({
agent: merged.agent,
sessionId: merged.sessionId,
resumeFilePath: merged.filePath,
cwd,
platform
})
}
}
@@ -17,6 +17,8 @@ export async function discoverAiVaultSessionSources(args: {
const { options, limitPerAgent, issues } = args
const wslHomeDirs = normalizedWslHomeDirs(options.wslHomeDirs)
// The Cursor chat-meta scan scope is owned by scanAiVaultSessions: it has to
// span parse as well, and finalize runs after this returns.
return Promise.all([
// Why: OpenCode 1.17.x migrated sessions from per-session JSON files to a
// SQLite DB. discoverOpenCodeSessions runs both the file scanner (legacy)
+10 -1
View File
@@ -5,6 +5,8 @@ import type {
AiVaultSessionPreviewMessage
} from '../../shared/ai-vault-types'
import type { ExecutionHostId } from '../../shared/execution-host'
import type { TranscriptMessageSink } from './session-transcript-consumers'
import type { SessionSidecarObservation } from './session-sidecar-stat'
export type AiVaultScanOptions = {
claudeProjectsDir?: string
@@ -54,8 +56,12 @@ export type FileWithMtime = {
modifiedAt: string
// Present when discovery statted the file; lets the parse cache detect
// unchanged/truncated files without a second stat. Synthetic candidates
// such as OpenCode SQLite rows omit it.
// such as OpenCode SQLite rows omit it. The transcript's own length: a byte
// offset into it may be compared against this directly.
sizeBytes?: number
// What discovery saw of the agent's sibling file, tracked apart from the
// transcript's own stat (see session-sidecar-stat.ts).
sidecar?: SessionSidecarObservation
// Present when discovery can prove filesystem identity. Codex dual-root
// scans use a multi-link inode to collapse only actual hardlink aliases.
dev?: number
@@ -108,6 +114,9 @@ export type ResumableSessionParseState = {
export type SessionAccumulator = {
agent: AiVaultAgent
// Every decoded message this fold sees also goes here, for the reader's
// consumers. Shared by clones on purpose: one read, one message stream.
messages: TranscriptMessageSink
sessionId: string
title: string | null
fallbackTitle: string | null
+82 -106
View File
@@ -6,19 +6,13 @@ import type {
import { LOCAL_EXECUTION_HOST_ID, type ExecutionHostId } from '../../shared/execution-host'
import { withSpan } from '../observability/tracer'
import { sessionSortTime } from './session-scanner-accumulator'
import {
codexRolloutHardlinkIdentity,
dedupeCodexRolloutAliases,
dedupeCodexSessionsBySessionId
} from './codex-session-root-dedup'
import { readCodexRolloutSessionMetaId } from '../codex/codex-rollout-session-meta'
import { dedupeCodexSessionsBySessionId } from './codex-session-root-dedup'
import {
createAntigravityWorkspaceResolver,
readLocalAntigravityHistory,
type AntigravityWorkspaceResolver
} from './session-scanner-antigravity-history'
import { antigravityHistoryPathForBrainDir } from './session-scanner-antigravity-paths'
import { codexHomeForSessionsDir } from './session-scanner-codex-paths'
import { sessionCandidatesFromDiscoveries } from './session-scanner-candidates'
import {
ensureSessionParseCacheLoaded,
scheduleSessionParseCachePersist
@@ -30,10 +24,8 @@ import {
} from './session-scanner-parse-cache'
import { recordSessionScanIssue } from './session-scan-issues'
import { discoverInScopeClaudeFiles } from './session-scanner-scope-discovery'
import {
DEFAULT_CODEX_HOME_DIR,
discoverAiVaultSessionSources
} from './session-scanner-source-discovery'
import { discoverAiVaultSessionSources } from './session-scanner-source-discovery'
import { cursorChatMetaRefusals, withCursorChatMetaScan } from './session-scanner-cursor-chat-meta'
import type {
AiVaultScanOptions,
SessionFileCandidate,
@@ -62,103 +54,87 @@ export async function scanAiVaultSessions(
// The span makes scan cost visible in the local trace file: STA-1278-style
// "one core pegged" reports need to show whether transcript scanning is the
// subsystem burning CPU, and how much of each scan the cache absorbed.
return withSpan('aiVault.scan', async (span) => {
const limit = options.unlimited
? Number.POSITIVE_INFINITY
: clampPositiveInteger(options.limit, DEFAULT_AI_VAULT_SCAN_LIMIT)
const limitPerAgent = options.unlimited
? Number.POSITIVE_INFINITY
: clampPositiveInteger(options.limitPerAgent, limit * SESSION_PARSE_CANDIDATE_MULTIPLIER)
const platform = options.platform ?? process.platform
const executionHostId = options.executionHostId ?? LOCAL_EXECUTION_HOST_ID
const issues: AiVaultScanIssue[] = []
const parseStats = createSessionParseStats()
const antigravityWorkspaceResolver = createAntigravityWorkspaceResolver(
readLocalAntigravityHistory
)
// Why: persisted entries must be seeded before any candidate is parsed, or
// the cold scan gains nothing from the cache file (#9210).
throwIfAiVaultScanCancelled(options.signal)
await ensureSessionParseCacheLoaded()
const discoveries = await discoverAiVaultSessionSources({ options, limitPerAgent, issues })
throwIfAiVaultScanCancelled(options.signal)
// The Cursor chat-meta scope spans discovery AND parse: its sibling meta.json
// is looked up in both phases, and one scan must read the chats tree once.
return withSpan('aiVault.scan', (span) =>
withCursorChatMetaScan(async () => {
const limit = options.unlimited
? Number.POSITIVE_INFINITY
: clampPositiveInteger(options.limit, DEFAULT_AI_VAULT_SCAN_LIMIT)
const limitPerAgent = options.unlimited
? Number.POSITIVE_INFINITY
: clampPositiveInteger(options.limitPerAgent, limit * SESSION_PARSE_CANDIDATE_MULTIPLIER)
const platform = options.platform ?? process.platform
const executionHostId = options.executionHostId ?? LOCAL_EXECUTION_HOST_ID
const issues: AiVaultScanIssue[] = []
const parseStats = createSessionParseStats()
const antigravityWorkspaceResolver = createAntigravityWorkspaceResolver(
readLocalAntigravityHistory
)
// Why: persisted entries must be seeded before any candidate is parsed, or
// the cold scan gains nothing from the cache file (#9210).
throwIfAiVaultScanCancelled(options.signal)
await ensureSessionParseCacheLoaded()
const discoveries = await discoverAiVaultSessionSources({ options, limitPerAgent, issues })
throwIfAiVaultScanCancelled(options.signal)
const candidates = await dedupeCodexRolloutAliases(
discoveries
.flatMap((discovery) =>
discovery.files.map((file): SessionFileCandidate => ({
agent: discovery.agent,
file,
codexHome:
discovery.agent === 'codex'
? codexHomeForSessionsDir(
discovery.rootDir,
options.defaultCodexHomeDir ?? DEFAULT_CODEX_HOME_DIR
)
: null,
antigravityHistoryPath:
discovery.agent === 'antigravity'
? antigravityHistoryPathForBrainDir(discovery.rootDir)
: undefined
}))
)
.sort((left, right) => right.file.mtimeMs - left.file.mtimeMs),
{
isCodex: (candidate) => candidate.agent === 'codex',
getFilePath: (candidate) => candidate.file.path,
getCodexHome: (candidate) => candidate.codexHome,
getHardlinkIdentity: (candidate) => codexRolloutHardlinkIdentity(candidate.file)
},
(filePath) => readCodexRolloutSessionMetaId(filePath, options.signal, 'scan'),
options.signal
)
const candidates = await sessionCandidatesFromDiscoveries(discoveries, options)
const parsedSessions = await parseSessionCandidates({
candidates: candidates.slice(0, limit * SESSION_PARSE_CANDIDATE_MULTIPLIER),
limit,
platform,
executionHostId,
issues,
parseStats,
signal: options.signal,
antigravityWorkspaceResolver
const parsedSessions = await parseSessionCandidates({
candidates: candidates.slice(0, limit * SESSION_PARSE_CANDIDATE_MULTIPLIER),
limit,
platform,
executionHostId,
issues,
parseStats,
signal: options.signal,
antigravityWorkspaceResolver
})
const cappedSessions = dedupeCodexSessionsBySessionId(parsedSessions)
.sort((left, right) => sessionSortTime(right) - sessionSortTime(left))
.slice(0, limit)
const scopeSessions = await scanInScopeSessions({
discoveries,
scopePaths: options.scopePaths ?? [],
limit,
alreadyParsedFilePaths: new Set(cappedSessions.map((session) => session.filePath)),
platform,
executionHostId,
issues,
parseStats,
signal: options.signal
})
// Scope discovery can return without parsing anything, so an abort landing
// here would otherwise persist and return a cancelled scan as complete.
throwIfAiVaultScanCancelled(options.signal)
for (const refusal of cursorChatMetaRefusals()) {
// One issue per refused chats root, not one per Cursor transcript.
recordSessionScanIssue(issues, {
agent: 'cursor',
path: refusal.chatsRoot,
message: refusal.message
})
}
span.setAttribute('candidates', candidates.length)
span.setAttribute('reused', parseStats.reused)
span.setAttribute('incremental', parseStats.incremental)
span.setAttribute('fullParses', parseStats.fullParses)
span.setAttribute('earlyStopped', parseStats.earlyStopped)
span.setAttribute('bytesRead', parseStats.bytesRead)
span.setAttribute('issues', issues.length)
scheduleSessionParseCachePersist(parseStats)
return {
sessions: mergeSessions(cappedSessions, scopeSessions),
issues: issues.map((issue) => ({ executionHostId, ...issue })),
scannedAt: new Date().toISOString()
}
})
const cappedSessions = dedupeCodexSessionsBySessionId(parsedSessions)
.sort((left, right) => sessionSortTime(right) - sessionSortTime(left))
.slice(0, limit)
const scopeSessions = await scanInScopeSessions({
discoveries,
scopePaths: options.scopePaths ?? [],
limit,
alreadyParsedFilePaths: new Set(cappedSessions.map((session) => session.filePath)),
platform,
executionHostId,
issues,
parseStats,
signal: options.signal
})
// Scope discovery can return without parsing anything, so an abort landing
// here would otherwise persist and return a cancelled scan as complete.
throwIfAiVaultScanCancelled(options.signal)
span.setAttribute('candidates', candidates.length)
span.setAttribute('reused', parseStats.reused)
span.setAttribute('incremental', parseStats.incremental)
span.setAttribute('fullParses', parseStats.fullParses)
span.setAttribute('earlyStopped', parseStats.earlyStopped)
span.setAttribute('bytesRead', parseStats.bytesRead)
span.setAttribute('issues', issues.length)
scheduleSessionParseCachePersist(parseStats)
return {
sessions: mergeSessions(cappedSessions, scopeSessions),
issues: issues.map((issue) => ({ executionHostId, ...issue })),
scannedAt: new Date().toISOString()
}
})
)
}
// In-scope sessions are guaranteed regardless of the recency cap, so the global
@@ -0,0 +1,74 @@
import { describe, expect, it } from 'vitest'
import { sidecarUnchanged, type SessionSidecarObservation } from './session-sidecar-stat'
const META = { path: '/chats/a/meta.json', mtimeMs: 100, sizeBytes: 20 } as const
const OTHER = { path: '/chats/a/meta.json', mtimeMs: 101, sizeBytes: 20 } as const
type Named = [label: string, value: SessionSidecarObservation | undefined]
const ENTRIES: Named[] = [
['undefined', undefined],
["'none'", 'none'],
["'unknown'", 'unknown'],
['object', { ...META }]
]
const OBSERVED: Named[] = [
['undefined', undefined],
["'none'", 'none'],
["'unknown'", 'unknown'],
['same object', { ...META }],
['different object', { ...OTHER }]
]
// entry (row) x observed (column). A `true` cell is a cache hit.
const TRUTH_TABLE: Record<string, Record<string, boolean>> = {
undefined: {
undefined: true,
"'none'": true,
"'unknown'": false,
'same object': false,
'different object': false
},
"'none'": {
undefined: true,
"'none'": true,
"'unknown'": false,
'same object': false,
'different object': false
},
"'unknown'": {
undefined: false,
"'none'": false,
"'unknown'": false,
'same object': false,
'different object': false
},
object: {
undefined: false,
"'none'": false,
"'unknown'": false,
'same object': true,
'different object': false
}
}
describe.each(ENTRIES)('cached %s', (entryLabel, entry) => {
it.each(OBSERVED)(`vs observed %s`, (observedLabel, observed) => {
expect(sidecarUnchanged(entry, observed)).toBe(TRUTH_TABLE[entryLabel][observedLabel])
})
})
it('treats a vanished sidecar as a change, not as "never had one"', () => {
expect(sidecarUnchanged({ ...META }, 'none')).toBe(false)
})
it('never concludes anything from an unreadable sidecar, in either position', () => {
expect(sidecarUnchanged('unknown', 'none')).toBe(false)
expect(sidecarUnchanged('unknown', { ...META })).toBe(false)
expect(sidecarUnchanged({ ...META }, 'unknown')).toBe(false)
})
it('keeps an agent with no sidecar at all a cache hit', () => {
expect(sidecarUnchanged(undefined, undefined)).toBe(true)
expect(sidecarUnchanged(undefined, 'none')).toBe(true)
})
+49
View File
@@ -0,0 +1,49 @@
// Why: some agents keep part of a session beside its transcript — Cursor's
// chat meta.json, Cline's messages file. Folding that file's stat into the
// transcript's own mtime/size makes one key mean two things, so a byte offset
// into the transcript can no longer be compared against it and a refused read
// of the sibling takes the transcript down with it. The sidecar is observed
// separately and compared separately.
export type SessionSidecarStat = {
path: string
mtimeMs: number
sizeBytes: number
}
export type SessionSidecarObservation =
| SessionSidecarStat
/** This agent declares no sidecar, or it does not exist. */
| 'none'
/** It could not be read this scan; nothing may be concluded from its absence. */
| 'unknown'
/**
* Whether a cached observation still describes what discovery just saw.
*
* Asymmetric on purpose: `file` is observed now, so a missing value means the
* agent has no sidecar, while `entry` may predate the field (an entry seeded
* from a cache file an older build wrote), so a missing value means unknown.
*
* `'none'` is a claim, not an absence of one: a sidecar that was there and is
* gone changed, and one that was unreadable last time is still unknown now.
*/
export function sidecarUnchanged(
entry: SessionSidecarObservation | undefined,
file: SessionSidecarObservation | undefined
): boolean {
const observed = file ?? 'none'
if (observed === 'unknown' || entry === 'unknown') {
return false
}
if (observed === 'none') {
// Absent now: a hit only if it was absent before, or the agent never had one.
return entry === undefined || entry === 'none'
}
return (
typeof entry === 'object' &&
entry.path === observed.path &&
entry.mtimeMs === observed.mtimeMs &&
entry.sizeBytes === observed.sizeBytes
)
}
@@ -31,6 +31,9 @@ async function readOneTitle(
if (!stats.isFile() || signal?.aborted) {
return null
}
// Why: this key is a raw lstat with no content dependency, so it only
// matches the scanner's for providers that declare none — today claude and
// codex, which is all this request type carries.
const session = await parseAgentSessionFileCached(
{
agent: request.agent,
@@ -0,0 +1,92 @@
import {
hasTranscriptConsumers,
transcriptConsumers,
type TranscriptMessage,
type TranscriptMessageSink,
type TranscriptReadConsumer,
type TranscriptReadOutcome,
type TranscriptReadStart
} from './session-transcript-consumers'
/**
* The sink a parser pushes into, and the fan-out to every registered consumer.
*
* One channel belongs to one file for as long as its resumable parse state
* lives, because the cached state (and every clone of it) holds this reference.
* A read re-points the channel at that read's consumers instead of replacing it.
*/
export class TranscriptMessageChannel implements TranscriptMessageSink {
private readers: TranscriptReadConsumer[] = []
private muted = false
/** True while a read is open with at least one consumer attached. */
get active(): boolean {
return this.readers.length > 0
}
beginRead(start: TranscriptReadStart): void {
this.muted = false
this.readers = []
// Keeps a scan with no consumers allocation-free on its hottest path.
if (!hasTranscriptConsumers()) {
return
}
for (const consumer of transcriptConsumers()) {
try {
const reader = consumer.beginRead(start)
if (reader) {
this.readers.push(reader)
}
} catch {
// A consumer that cannot open this read simply does not see it.
}
}
}
push(message: TranscriptMessage): void {
if (this.muted || this.readers.length === 0) {
return
}
// A throwing consumer is dropped for the rest of the read rather than
// failing the parse; it then gets no `finish`, so it never records a cursor
// for a stream it did not see in full.
let index = 0
while (index < this.readers.length) {
try {
this.readers[index].message(message)
index++
} catch {
this.readers.splice(index, 1)
}
}
}
/**
* Suppresses emission for a display-only re-read: the trailing unterminated
* line is shown in the list but is re-read once complete, so emitting it here
* would hand every consumer the same line twice. `fn` must be synchronous.
*/
mute<T>(fn: () => T): T {
const previous = this.muted
this.muted = true
try {
return fn()
} finally {
this.muted = previous
}
}
finishRead(outcome: TranscriptReadOutcome): void {
const readers = this.readers
this.readers = []
this.muted = false
for (const reader of readers) {
try {
reader.finish(outcome)
} catch {
// A consumer failure must never fail the session list.
}
}
}
}
@@ -0,0 +1,330 @@
import { appendFile, mkdir, mkdtemp, rm, stat, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, expect, it, vi } from 'vitest'
import { scanAiVaultSessions } from './session-scanner'
import {
parseAgentSessionFileCached,
resetSessionParseCacheForTests
} from './session-scanner-parse-cache'
import { isolatedScanRoots, jsonLines } from './session-scanner-test-fixtures'
import type { FileWithMtime, SessionFileCandidate } from './session-scanner-types'
import { readWholeTranscript } from './session-transcript-reader'
const OPENCODE_SQLITE_SESSION = {
id: 'local:opencode:sqlite-session:db',
agent: 'opencode' as const,
sessionId: 'sqlite-session'
}
// Stands in for the worker thread: the point is that its messages never come
// back over the channel, not what the SQLite read returns.
vi.mock('./session-scanner-opencode-sqlite-worker-spawn', async (importOriginal) => ({
...(await importOriginal<typeof OpenCodeSqliteWorkerSpawn>()),
parseOpenCodeSqliteSessionViaWorker: () => Promise.resolve(OPENCODE_SQLITE_SESSION)
}))
import type * as OpenCodeSqliteWorkerSpawn from './session-scanner-opencode-sqlite-worker-spawn'
import {
registerTranscriptConsumer,
resetTranscriptConsumersForTests,
type TranscriptMessage,
type TranscriptReadOutcome,
type TranscriptReadStart
} from './session-transcript-consumers'
type RecordedRead = {
start: TranscriptReadStart
messages: TranscriptMessage[]
outcome: TranscriptReadOutcome | null
}
function recordingConsumer(): { reads: RecordedRead[]; unregister: () => void } {
const reads: RecordedRead[] = []
const unregister = registerTranscriptConsumer({
beginRead: (start) => {
const read: RecordedRead = { start, messages: [], outcome: null }
reads.push(read)
return {
message: (message) => read.messages.push(message),
finish: (outcome) => {
read.outcome = outcome
}
}
}
})
return { reads, unregister }
}
function textsFor(reads: RecordedRead[], agent: string): string[] {
return reads
.filter((read) => read.start.candidate.agent === agent)
.flatMap((read) => read.messages.map((message) => `${message.role}:${message.text}`))
}
let tempRoots: string[] = []
afterEach(async () => {
resetTranscriptConsumersForTests()
resetSessionParseCacheForTests()
await Promise.all(tempRoots.map((root) => rm(root, { recursive: true, force: true })))
tempRoots = []
})
function claudeTurns(from: number, to: number): unknown[] {
const records: unknown[] = []
for (let index = from; index <= to; index++) {
records.push({
type: 'user',
sessionId: 'claude-session',
timestamp: `2026-05-01T10:0${index}:00.000Z`,
cwd: '/tmp/claude',
message: { role: 'user', content: `ask ${index}` }
})
records.push({
type: 'assistant',
sessionId: 'claude-session',
timestamp: `2026-05-01T10:0${index}:01.000Z`,
message: {
role: 'assistant',
content: [
{ type: 'text', text: `reply ${index}` },
{ type: 'tool_use', name: 'Bash', input: { command: `ls ${index}` } }
]
}
})
}
return records
}
async function writeClaudeFixture(): Promise<{
root: string
roots: ReturnType<typeof isolatedScanRoots>
transcript: string
}> {
const root = await mkdtemp(join(tmpdir(), 'orca-transcript-consumers-'))
tempRoots.push(root)
const roots = isolatedScanRoots(root)
const transcript = join(roots.claudeProjectsDir, 'project', 'claude-session.jsonl')
await mkdir(join(roots.claudeProjectsDir, 'project'), { recursive: true })
await writeFile(transcript, `${jsonLines(claudeTurns(1, 4))}\n`)
return { root, roots, transcript }
}
it('delivers one message stream to every registered consumer', async () => {
const { roots } = await writeClaudeFixture()
const first = recordingConsumer()
const second = recordingConsumer()
const result = await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
expect(result.issues).toEqual([])
const stream = textsFor(first.reads, 'claude')
expect(stream).toEqual(textsFor(second.reads, 'claude'))
expect(stream).toEqual([
'user:ask 1',
'assistant:reply 1',
'tool:Bash: ls 1',
'user:ask 2',
'assistant:reply 2',
'tool:Bash: ls 2',
'user:ask 3',
'assistant:reply 3',
'tool:Bash: ls 3',
'user:ask 4',
'assistant:reply 4',
'tool:Bash: ls 4'
])
// The list's own fold keeps only the newest five preview turns, so the stream
// is demonstrably the reader's, not a projection of the session row.
const session = result.sessions.find((entry) => entry.agent === 'claude')
expect(session?.previewMessages).toHaveLength(5)
expect(session?.messageCount).toBe(8)
})
it('leaves the session list identical whether or not a consumer is registered', async () => {
const withoutConsumer = await writeClaudeFixture()
const bare = await scanAiVaultSessions({
...withoutConsumer.roots,
platform: 'darwin',
limit: 20
})
resetSessionParseCacheForTests()
recordingConsumer()
const observed = await scanAiVaultSessions({
...withoutConsumer.roots,
platform: 'darwin',
limit: 20
})
expect(observed.sessions).toEqual(bare.sessions)
})
it('replays only the appended lines on a resumed read', async () => {
const { roots, transcript } = await writeClaudeFixture()
const consumer = recordingConsumer()
await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
const firstRead = consumer.reads.at(-1)
expect(firstRead?.start.mode).toBe('replace')
expect(firstRead?.start.previousByteOffset).toBe(0)
expect(firstRead?.outcome?.incomplete).toBe(false)
await appendFile(transcript, `${jsonLines(claudeTurns(5, 5))}\n`)
consumer.reads.length = 0
await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
const resumed = consumer.reads.find((read) => read.start.candidate.agent === 'claude')
expect(resumed?.start.mode).toBe('append')
expect(resumed?.start.previousByteOffset).toBe(firstRead?.outcome?.byteOffset)
expect(textsFor(consumer.reads, 'claude')).toEqual([
'user:ask 5',
'assistant:reply 5',
'tool:Bash: ls 5'
])
})
it('publishes a trailing unterminated line once, when it is complete', async () => {
const { roots, transcript } = await writeClaudeFixture()
const consumer = recordingConsumer()
await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
// A half-written record: the list shows it, the stream must not carry it yet.
const [partial] = claudeTurns(5, 5)
await appendFile(transcript, JSON.stringify(partial))
consumer.reads.length = 0
await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
expect(textsFor(consumer.reads, 'claude')).toEqual([])
await appendFile(transcript, '\n')
consumer.reads.length = 0
await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
expect(textsFor(consumer.reads, 'claude')).toEqual(['user:ask 5'])
})
it('keeps the session list working when a consumer throws', async () => {
const { roots } = await writeClaudeFixture()
registerTranscriptConsumer({
beginRead: () => ({
message: () => {
throw new Error('consumer exploded')
},
finish: () => undefined
})
})
const healthy = recordingConsumer()
const result = await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
expect(result.issues).toEqual([])
expect(result.sessions.find((entry) => entry.agent === 'claude')?.messageCount).toBe(8)
expect(textsFor(healthy.reads, 'claude')).toHaveLength(12)
})
it('skips a read a consumer declines without disturbing the others', async () => {
const { roots } = await writeClaudeFixture()
registerTranscriptConsumer({ beginRead: () => null })
const healthy = recordingConsumer()
await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
expect(textsFor(healthy.reads, 'claude')).toHaveLength(12)
})
async function claudeCandidate(transcript: string): Promise<SessionFileCandidate> {
const stats = await stat(transcript)
const file: FileWithMtime = {
path: transcript,
mtimeMs: stats.mtimeMs,
modifiedAt: stats.mtime.toISOString(),
sizeBytes: stats.size
}
return { agent: 'claude', file, codexHome: null }
}
it('serializes overlapping parses of one path so no consumer read is orphaned', async () => {
const { transcript } = await writeClaudeFixture()
// Seed a resume point: the channel it stores is what concurrent reads share.
await parseAgentSessionFileCached(await claudeCandidate(transcript), 'darwin')
await appendFile(transcript, `${jsonLines(claudeTurns(5, 5))}\n`)
const consumer = recordingConsumer()
const appended = await claudeCandidate(transcript)
const [first, second] = await Promise.all([
parseAgentSessionFileCached(appended, 'darwin'),
parseAgentSessionFileCached(appended, 'darwin')
])
// Every read that opened must also close, or its consumer keeps a half-read
// stream forever and never learns the outcome.
expect(consumer.reads.filter((read) => read.outcome === null)).toEqual([])
expect(consumer.reads).toHaveLength(1)
expect(consumer.reads[0].start.mode).toBe('append')
expect(textsFor(consumer.reads, 'claude')).toEqual([
'user:ask 5',
'assistant:reply 5',
'tool:Bash: ls 5'
])
// The later caller reuses the stored entry rather than moving the cursor back.
expect(first?.messageCount).toBe(10)
expect(second?.messageCount).toBe(10)
})
it('reports a read whose parser cannot publish its messages as not complete', async () => {
const root = await mkdtemp(join(tmpdir(), 'orca-transcript-opencode-'))
tempRoots.push(root)
const dbPath = join(root, 'opencode.db')
await writeFile(dbPath, '')
const consumer = recordingConsumer()
const session = await readWholeTranscript({
candidate: {
agent: 'opencode',
codexHome: null,
file: {
path: `${dbPath}#sqlite-session`,
mtimeMs: 1,
modifiedAt: new Date(1).toISOString(),
sizeBytes: 10
}
},
platform: 'darwin'
})
expect(session).toEqual(OPENCODE_SQLITE_SESSION)
expect(consumer.reads).toHaveLength(1)
expect(consumer.reads[0].messages).toEqual([])
expect(consumer.reads[0].outcome?.incomplete).toBe(true)
})
it('reports the transcript size, not the cache key, as a whole-file read offset', async () => {
const root = await mkdtemp(join(tmpdir(), 'orca-transcript-cline-'))
tempRoots.push(root)
const roots = isolatedScanRoots(root)
// Cline is whole-file and declares a sibling content dependency, so its cache
// key covers two files while the read covers one.
const sessionDir = join(roots.clineSessionsDir, 'cline-session')
await mkdir(sessionDir, { recursive: true })
const metadataPath = join(sessionDir, 'cline-session.json')
await writeFile(
metadataPath,
JSON.stringify({
session_id: 'cline-session',
started_at: '2026-05-01T10:00:00.000Z',
cwd: '/tmp/cline'
})
)
await writeFile(
join(sessionDir, 'cline-session.messages.json'),
JSON.stringify({
updated_at: '2026-05-01T10:00:01.000Z',
messages: [{ role: 'user', content: [{ type: 'text', text: 'x'.repeat(400) }] }]
})
)
const consumer = recordingConsumer()
await scanAiVaultSessions({ ...roots, platform: 'darwin', limit: 20 })
const read = consumer.reads.find((entry) => entry.start.candidate.agent === 'cline')
expect(read?.outcome?.byteOffset).toBe((await stat(metadataPath)).size)
})
@@ -0,0 +1,85 @@
import type { AiVaultSession } from '../../shared/ai-vault-types'
import type { SessionFileCandidate } from './session-scanner-types'
// Why: the transcript reader owns discovery, per-file cursors and decoding; a
// consumer only folds the message stream. Registering a second consumer (a
// search index, a digest) must not require touching the reader or the parse
// cache, so the reader publishes reads rather than knowing who reads them.
export type TranscriptMessageRole = 'user' | 'assistant' | 'tool'
export type TranscriptMessage = {
role: TranscriptMessageRole
/** Untruncated decoded text; caps and redaction are consumer policy. */
text: string
timestamp: string | null
}
/** Where a parser hands its decoded messages; the reader supplies the instance. */
export type TranscriptMessageSink = {
/** False when nobody is listening: parsers skip the extraction entirely. */
readonly active: boolean
push(message: TranscriptMessage): void
}
export const NO_TRANSCRIPT_MESSAGES: TranscriptMessageSink = {
active: false,
push: () => undefined
}
export type TranscriptReadStart = {
candidate: SessionFileCandidate
/** `replace`: the whole file is being re-read; `append`: a resumed read. */
mode: 'replace' | 'append'
/** Byte offset the messages of this read continue from. */
previousByteOffset: number
}
export type TranscriptReadOutcome = {
/** Null when the parser rejected the file (an excluded Codex worker transcript). */
session: AiVaultSession | null
/** Byte offset just past the last complete line this read consumed. */
byteOffset: number
/**
* The messages of this read are not the whole span: the read failed part way,
* or the parser decodes where the channel cannot reach it. A consumer must
* not record a cursor for an incomplete read.
*/
incomplete: boolean
}
/** One consumer's view of one file read. */
export type TranscriptReadConsumer = {
message(message: TranscriptMessage): void
finish(outcome: TranscriptReadOutcome): void
}
export type TranscriptConsumer = {
/**
* Open this read, or return null to ignore it. A consumer whose own cursor is
* behind `previousByteOffset` declines here and re-reads on its own schedule;
* it must never ask another consumer where it is.
*/
beginRead(start: TranscriptReadStart): TranscriptReadConsumer | null
}
const consumers = new Set<TranscriptConsumer>()
export function registerTranscriptConsumer(consumer: TranscriptConsumer): () => void {
consumers.add(consumer)
return () => {
consumers.delete(consumer)
}
}
export function transcriptConsumers(): readonly TranscriptConsumer[] {
return [...consumers]
}
export function hasTranscriptConsumers(): boolean {
return consumers.size > 0
}
export function resetTranscriptConsumersForTests(): void {
consumers.clear()
}
@@ -0,0 +1,61 @@
import { expect, it } from 'vitest'
import { transcriptMessagesFromContent } from './session-transcript-message-content'
const AT = '2026-05-01T10:00:00.000Z'
it('keeps a plain string turn under the record role', () => {
expect(transcriptMessagesFromContent('user', 'just words', AT)).toEqual([
{ role: 'user', text: 'just words', timestamp: AT }
])
})
it('drops turns whose role a consumer cannot use', () => {
expect(transcriptMessagesFromContent('system', 'boot', AT)).toEqual([])
expect(transcriptMessagesFromContent('unknown', 'noise', AT)).toEqual([])
})
it('joins text blocks and appends tool blocks as their own messages', () => {
expect(
transcriptMessagesFromContent(
'assistant',
[
{ type: 'text', text: 'first' },
{ type: 'tool_use', name: 'Bash', input: { command: 'ls -la', description: 'ignored' } },
{ type: 'thinking', text: 'second' },
{ type: 'image', source: {} }
],
AT
)
).toEqual([
{ role: 'assistant', text: 'first\nsecond', timestamp: AT },
{ role: 'tool', text: 'Bash: ls -la', timestamp: AT }
])
})
it('reads a tool result carried on a user record as a tool message', () => {
expect(
transcriptMessagesFromContent(
'user',
[{ type: 'tool_result', content: [{ type: 'text', text: 'exit 0' }] }],
AT
)
).toEqual([{ role: 'tool', text: 'exit 0', timestamp: AT }])
})
it('names a tool call even with no recognisable argument', () => {
expect(
transcriptMessagesFromContent('assistant', [{ type: 'tool_use', name: 'Read', input: {} }], AT)
).toEqual([{ role: 'tool', text: 'Read', timestamp: AT }])
})
it('emits nothing for blank or absent content', () => {
expect(transcriptMessagesFromContent('user', ' ', AT)).toEqual([])
expect(transcriptMessagesFromContent('user', null, AT)).toEqual([])
expect(transcriptMessagesFromContent('assistant', [{ type: 'tool_use' }], AT)).toEqual([])
})
it('does not apply the list preview cap', () => {
const long = 'x'.repeat(5000)
const [message] = transcriptMessagesFromContent('user', [{ type: 'text', text: long }], AT)
expect(message.text).toHaveLength(5000)
})
@@ -0,0 +1,138 @@
import { asRecord } from './session-scanner-record-value'
import { sliceAtCodeUnitLimit } from './session-scanner-text-normalization'
import type { AiVaultSessionPreviewMessage } from '../../shared/ai-vault-types'
import type { TranscriptMessage, TranscriptMessageRole } from './session-transcript-consumers'
// Safety bound only: a consumer applies its own caps. Matches the first-prompt
// copy path's ceiling so one pathological paste cannot dominate a scan.
const TRANSCRIPT_MESSAGE_TEXT_LIMIT = 256 * 1024
const TOOL_ARGUMENT_SCAN_LIMIT = 2000
const TEXT_BLOCK_TYPES = new Set(['text', 'input_text', 'output_text', 'thinking', 'reasoning'])
// The argument that identifies what a tool call actually did.
const TOOL_INPUT_KEYS = ['command', 'cmd', 'file_path', 'path', 'pattern', 'query', 'description']
type PreviewRole = AiVaultSessionPreviewMessage['role']
/** Only conversational roles reach consumers; system/unknown turns are noise. */
export function transcriptMessageRole(role: PreviewRole): TranscriptMessageRole | null {
return role === 'user' || role === 'assistant' || role === 'tool' ? role : null
}
export function toolCallText(name: unknown, input: unknown): string | null {
const toolName = typeof name === 'string' && name.trim() ? name.trim() : null
const inputRecord = asRecord(input)
let argument: string | null = null
if (inputRecord) {
for (const key of TOOL_INPUT_KEYS) {
const value = inputRecord[key]
if (typeof value === 'string' && value.trim()) {
argument = value
break
}
}
} else if (typeof input === 'string' && input.trim()) {
argument = input
}
if (!toolName && !argument) {
return null
}
const bounded = argument ? sliceAtCodeUnitLimit(argument, TOOL_ARGUMENT_SCAN_LIMIT) : null
return toolName && bounded ? `${toolName}: ${bounded}` : (toolName ?? bounded)
}
/** Flattens a tool_result body (a string, or an array of text blocks). */
function toolResultText(content: unknown): string | null {
if (typeof content === 'string') {
return content.trim() ? content : null
}
if (!Array.isArray(content)) {
return null
}
const parts: string[] = []
let length = 0
for (const item of content) {
const text = typeof item === 'string' ? item : asRecord(item)?.text
if (typeof text === 'string' && text) {
parts.push(text)
length += text.length
if (length >= TRANSCRIPT_MESSAGE_TEXT_LIMIT) {
break
}
}
}
const joined = parts.join('\n')
return joined.trim() ? joined : null
}
/**
* Splits one provider content value into the messages it decodes to. Text
* blocks keep the record's role; tool_use and tool_result blocks become `tool`
* messages whichever record carried them (Claude stores tool results on user
* records), so a consumer never has to know a provider's record shapes.
*/
export function transcriptMessagesFromContent(
role: PreviewRole,
content: unknown,
timestamp: string | null
): TranscriptMessage[] {
const messages: TranscriptMessage[] = []
const textRole = transcriptMessageRole(role)
if (typeof content === 'string') {
const text = boundedText(content)
return text && textRole ? [{ role: textRole, text, timestamp }] : []
}
const blocks = Array.isArray(content) ? content : content != null ? [content] : []
const textParts: string[] = []
for (const block of blocks) {
if (typeof block === 'string') {
textParts.push(block)
continue
}
const item = asRecord(block)
if (!item) {
continue
}
const type = typeof item.type === 'string' ? item.type : null
if (type === 'tool_use') {
pushMessage(messages, 'tool', toolCallText(item.name, item.input), timestamp)
continue
}
if (type === 'tool_result') {
pushMessage(messages, 'tool', toolResultText(item.content), timestamp)
continue
}
if (type !== null && !TEXT_BLOCK_TYPES.has(type)) {
continue
}
const text = typeof item.text === 'string' ? item.text : item.content
if (typeof text === 'string' && text) {
textParts.push(text)
}
}
if (textRole && textParts.length > 0) {
// The record's own words lead; its tool blocks follow in transcript order.
const text = boundedText(textParts.join('\n'))
if (text) {
messages.unshift({ role: textRole, text, timestamp })
}
}
return messages
}
function pushMessage(
messages: TranscriptMessage[],
role: TranscriptMessageRole,
text: string | null,
timestamp: string | null
): void {
const bounded = text === null ? null : boundedText(text)
if (bounded) {
messages.push({ role, text: bounded, timestamp })
}
}
export function boundedText(value: string): string | null {
const bounded = sliceAtCodeUnitLimit(value, TRANSCRIPT_MESSAGE_TEXT_LIMIT)
return bounded.trim() ? bounded : null
}
@@ -0,0 +1,150 @@
import { readTranscriptSlice } from '../native-chat/wsl-transcript-fs-access'
import type { AiVaultSession } from '../../shared/ai-vault-types'
import { parseAgentSessionFile, parserPublishesMessages } from './session-scanner-agent-parser'
import { consumeCompleteJsonlLines } from './session-scanner-jsonl-reader'
import type { ResumableSessionParseState, SessionFileCandidate } from './session-scanner-types'
import type { SessionParseResumePoint } from './session-parse-cache-store'
import { TranscriptMessageChannel } from './session-transcript-channel'
const NEWLINE_BYTE = 0x0a
// Why: this layer owns reading a transcript and nothing else. It decides where
// a read starts, drives the parser, publishes the decoded messages to every
// registered consumer, and reports where the read ended. Which of those results
// are cached, listed or indexed belongs to the callers.
export type TranscriptReadStats = {
incremental: number
fullParses: number
// Transcripts the parser already excluded (Codex workers), re-listed after a
// write and dismissed without reading. Counted apart from `incremental` so a
// scan span still shows how much work the early stop actually removed.
earlyStopped: number
bytesRead: number
}
export type ResumableTranscriptRead = {
session: AiVaultSession | null
/** The fold to resume from next time, and the channel bound to it. */
resume: SessionParseResumePoint
}
/**
* Read an append-only transcript, resuming from `resume` when the file only
* grew and the recorded offset still sits on a line boundary. Anything else
* (a rewrite, a truncation, a platform change) re-reads the whole file.
*/
export async function readResumableTranscript(args: {
candidate: SessionFileCandidate
platform: NodeJS.Platform
resume: SessionParseResumePoint | null
stateFactory: (messages: TranscriptMessageChannel) => ResumableSessionParseState
stats?: TranscriptReadStats
}): Promise<ResumableTranscriptRead> {
const { file } = args.candidate
const resume = args.resume
const canResume =
resume !== null &&
typeof file.sizeBytes === 'number' &&
file.sizeBytes >= resume.byteOffset &&
(resume.byteOffset === 0 || (await endsWithNewlineAt(file.path, resume.byteOffset)))
// Clone before consuming: a failed read must not corrupt the cached state,
// or the next resume would double-count the lines applied before the error.
const channel = canResume ? resume.channel : new TranscriptMessageChannel()
const state = canResume ? resume.state.clone() : args.stateFactory(channel)
const startOffset = canResume ? resume.byteOffset : 0
// Mirrors the reader's entry guard so a dismissed transcript is not reported
// as an incremental parse that read nothing.
const stoppedBeforeRead = state.shouldStop?.() === true
if (args.stats) {
if (stoppedBeforeRead) {
args.stats.earlyStopped++
} else if (canResume) {
args.stats.incremental++
} else {
args.stats.fullParses++
}
}
channel.beginRead({
candidate: args.candidate,
mode: canResume ? 'append' : 'replace',
previousByteOffset: startOffset
})
try {
const readResult = await consumeCompleteJsonlLines({
path: file.path,
start: startOffset,
onLine: (line) => state.consumeLine(line),
// Bound: the optional hooks are declared as methods, so a parser written
// with method syntax must not lose `this` on the way into the reader.
onLineBytes: state.consumeLineBytes?.bind(state),
shouldStop: state.shouldStop?.bind(state)
})
if (args.stats) {
args.stats.bytesRead += readResult.bytesRead
}
// The stat this scan displays is current even when nothing new was consumed.
state.touchFile(file)
// Keep parity with the one-shot parser: a final unterminated line is shown,
// but stays out of the resumable state so the (possibly still-growing) line
// is re-read once complete instead of being half-counted.
let displayState = state
if (readResult.trailingPartialLine !== null) {
const partialLine = readResult.trailingPartialLine
displayState = state.clone()
channel.mute(() => displayState.consumeLine(partialLine))
}
const session = await displayState.finalize(args.platform)
channel.finishRead({ session, byteOffset: readResult.consumedThrough, incomplete: false })
return {
session,
resume: { state, byteOffset: readResult.consumedThrough, channel }
}
} catch (error) {
channel.finishRead({ session: null, byteOffset: startOffset, incomplete: true })
throw error
}
}
/**
* Read a transcript whose format is rewritten in place rather than appended
* (whole-JSON documents, Kimi's state doc, OpenCode). There is no cursor to
* keep, so every read is a whole-file `replace`.
*/
export async function readWholeTranscript(args: {
candidate: SessionFileCandidate
platform: NodeJS.Platform
stats?: TranscriptReadStats
}): Promise<AiVaultSession | null> {
const { file } = args.candidate
if (args.stats) {
args.stats.fullParses++
args.stats.bytesRead += file.sizeBytes ?? 0
}
const publishes = parserPublishesMessages(args.candidate)
const channel = new TranscriptMessageChannel()
channel.beginRead({ candidate: args.candidate, mode: 'replace', previousByteOffset: 0 })
try {
const session = await parseAgentSessionFile(args.candidate, args.platform, channel)
channel.finishRead({ session, byteOffset: file.sizeBytes ?? 0, incomplete: !publishes })
return session
} catch (error) {
channel.finishRead({ session: null, byteOffset: 0, incomplete: true })
throw error
}
}
// A resume point is only valid if it still sits just past a line break;
// anything else means the file was rewritten, not appended. Heuristic: a
// grown rewrite keeping '\n' at exactly this byte would slip through, but
// agent transcripts are append-only so that trade is accepted (worst case is
// a stale vault row until the file is next truncated or the app restarts).
async function endsWithNewlineAt(path: string, offset: number): Promise<boolean> {
const slice = await readTranscriptSlice(path, offset - 1, 1, 'scan')
return slice.length === 1 && slice[0] === NEWLINE_BYTE
}
+102
View File
@@ -0,0 +1,102 @@
import type { Worker } from 'node:worker_threads'
export type WorkerThreadFactory = () => Worker
/**
* Owns the lifetime of one lazily-spawned worker thread: spawn on demand,
* listener wiring, teardown, and idle expiry. It holds no request state, so
* every decision about which call a message belongs to stays with its client,
* and a failed spawn is reported rather than thrown so the client can fail its
* queued calls closed instead of moving the work back onto the main thread.
*/
export class LazyWorkerThreadHost<TResponse> {
private worker: Worker | null = null
private idleTimer: NodeJS.Timeout | null = null
private cleanupListeners: (() => void) | null = null
private reportedUnavailable = false
constructor(
private readonly options: {
factory: WorkerThreadFactory
idleTeardownMs: number
onMessage: (response: TResponse) => void
onError: (error: Error) => void
onExit: (code: number) => void
/** Nothing active and nothing queued, checked again when the idle timer fires. */
isIdle: () => boolean
/** First spawn failure only: a repeating one must not repeat the log. */
onUnavailable: (error: unknown) => void
}
) {}
get current(): Worker | null {
return this.worker
}
/** The live worker, spawning one if needed; null when no worker can be had. */
ensure(): Worker | null {
if (this.worker) {
return this.worker
}
try {
const worker = this.options.factory()
const onMessage = (response: TResponse): void => this.options.onMessage(response)
const onError = (error: Error): void => this.options.onError(error)
const onExit = (code: number): void => this.options.onExit(code)
worker.on('message', onMessage)
worker.on('error', onError)
worker.on('exit', onExit)
this.cleanupListeners = () => {
worker.off('message', onMessage)
worker.off('error', onError)
worker.off('exit', onExit)
}
// Never keep the app alive for background work.
worker.unref?.()
this.worker = worker
return worker
} catch (err) {
if (!this.reportedUnavailable) {
this.reportedUnavailable = true
this.options.onUnavailable(err)
}
return null
}
}
destroy(): void {
this.clearIdleTimer()
const worker = this.worker
this.worker = null
if (!worker) {
return
}
this.cleanupListeners?.()
this.cleanupListeners = null
worker.removeAllListeners()
void worker.terminate().catch(() => undefined)
}
scheduleIdleTeardown(): void {
this.clearIdleTimer()
if (!this.worker) {
return
}
this.idleTimer = setTimeout(() => {
this.idleTimer = null
// Re-checked here: a request arriving as the timer fires must never be
// lost to a self-exiting worker.
if (this.options.isIdle()) {
this.destroy()
}
}, this.options.idleTeardownMs)
this.idleTimer.unref?.()
}
clearIdleTimer(): void {
if (this.idleTimer) {
clearTimeout(this.idleTimer)
this.idleTimer = null
}
}
}
+28 -93
View File
@@ -2,6 +2,7 @@ import { existsSync } from 'node:fs'
import { getAppEnvironment, hasAppEnvironment } from '../../shared/app-environment'
import { join } from 'node:path'
import { Worker } from 'node:worker_threads'
import { LazyWorkerThreadHost, type WorkerThreadFactory } from '../lazy-worker-thread-host'
import {
PORT_SCAN_COMMAND_TIMEOUT_MS,
PortScanCommandTimeoutError,
@@ -11,11 +12,10 @@ import {
// Why (#11161): a lazily-spawned, unref'd worker runs the port scan's probe
// spawns off the Electron main-process event loop, because libuv performs
// process creation inline on the calling thread. Lifecycle (FIFO one-at-a-time
// dispatch, per-call deadlines, respawn-on-fault, idle teardown, fail-closed)
// mirrors src/main/ai-vault/session-scanner-opencode-sqlite-worker-client.ts;
// the duplicated ~150 lines are cheaper than a premature shared abstraction, so
// a third adopter should extract one.
// process creation inline on the calling thread. This module owns the request
// half (FIFO one-at-a-time dispatch, per-call deadlines, respawn-on-fault); the
// thread's own lifetime belongs to LazyWorkerThreadHost, shared with
// src/main/ai-vault/session-scanner-opencode-sqlite-worker-client.ts.
//
// This module used to contain the literal text require('electron'), which fails the
// plain-Node entry guard even inside a try/catch. It reads the AppEnvironment port
@@ -36,7 +36,7 @@ export const MAX_CONSECUTIVE_DEATHS = 3
export const MAX_QUEUED_CALLS = 8
export type PortScanCommandResult = { stdout: string; spawnMs: number }
export type PortScanWorkerFactory = () => Worker
export type PortScanWorkerFactory = WorkerThreadFactory
// Distinguishes "no worker at all" from a timeout or crash so the scanner can
// log it once and callers never mistake it for a command timeout.
@@ -62,20 +62,27 @@ type PendingCall = {
* be spawned rather than moving process creation back onto the main thread.
*/
export class PortScanCommandClient {
private worker: Worker | null = null
private active: PendingCall | null = null
private queue: PendingCall[] = []
private idleTimer: NodeJS.Timeout | null = null
private consecutiveDeaths = 0
private nextId = 1
private loggedWorkerUnavailable = false
private cleanupWorkerListeners: (() => void) | null = null
private readonly workerFactory: PortScanWorkerFactory
private readonly log: (message: string) => void
private readonly host: LazyWorkerThreadHost<PortScanCommandResponse>
constructor(options: { workerFactory: PortScanWorkerFactory; log?: (message: string) => void }) {
this.workerFactory = options.workerFactory
this.log = options.log ?? ((message) => console.warn(message))
const log = options.log ?? ((message: string) => console.warn(message))
this.host = new LazyWorkerThreadHost<PortScanCommandResponse>({
factory: options.workerFactory,
idleTeardownMs: IDLE_TEARDOWN_MS,
onMessage: (response) => this.onMessage(response),
onError: (error) => this.onWorkerFault(error),
onExit: (code) => this.onWorkerExit(code),
isIdle: () => !this.active && this.queue.length === 0,
// Why (#11161): never fall back to in-process execFile here; a missing
// bundle must report port scanning as unavailable rather than reintroduce
// the main-thread freeze this worker boundary exists to prevent.
onUnavailable: (err) =>
log(`[workspace-ports] probe worker unavailable. ${errorMessage(err)}`)
})
}
/**
@@ -109,7 +116,7 @@ export class PortScanCommandClient {
if (this.active || this.queue.length === 0) {
return
}
const worker = this.ensureWorker()
const worker = this.host.ensure()
if (!worker) {
this.failQueuedAsUnavailable()
return
@@ -119,7 +126,7 @@ export class PortScanCommandClient {
return
}
this.active = call
this.clearIdleTimer()
this.host.clearIdleTimer()
// Why (#11161): one at a time. uv_spawn blocks the worker's own loop, so a
// second concurrent request would have its deadline armed while the first
// spawn is still stalling the thread, producing a false timeout.
@@ -128,39 +135,6 @@ export class PortScanCommandClient {
worker.postMessage(call.request)
}
private ensureWorker(): Worker | null {
if (this.worker) {
return this.worker
}
try {
const worker = this.workerFactory()
const onMessage = (response: PortScanCommandResponse): void => this.onMessage(response)
const onError = (error: Error): void => this.onWorkerFault(error)
const onExit = (code: number): void => this.onWorkerExit(code)
worker.on('message', onMessage)
worker.on('error', onError)
worker.on('exit', onExit)
this.cleanupWorkerListeners = () => {
worker.off('message', onMessage)
worker.off('error', onError)
worker.off('exit', onExit)
}
// Never keep the app alive for a port scan.
worker.unref?.()
this.worker = worker
return worker
} catch (err) {
// Why (#11161): never fall back to in-process execFile here; a missing
// bundle must report port scanning as unavailable rather than reintroduce
// the main-thread freeze this worker boundary exists to prevent.
if (!this.loggedWorkerUnavailable) {
this.loggedWorkerUnavailable = true
this.log(`[workspace-ports] probe worker unavailable. ${errorMessage(err)}`)
}
return null
}
}
private onMessage(response: PortScanCommandResponse): void {
const call = this.active
if (!call || call.request.id !== response.id) {
@@ -191,7 +165,7 @@ export class PortScanCommandClient {
// A clean self-exit is not a death, but the stale handle must be dropped or
// the next dispatch would post into a dead worker and stall to its deadline.
if (code === 0 && !this.active && this.queue.length === 0) {
this.destroyWorker()
this.host.destroy()
return
}
this.onWorkerFault(new Error(`Port scan probe worker exited with code ${code}`))
@@ -199,7 +173,7 @@ export class PortScanCommandClient {
private onWorkerFault(error: Error): void {
const failed = this.active
this.destroyWorker()
this.host.destroy()
this.consecutiveDeaths++
if (failed) {
this.settle(failed, () => failed.reject(error))
@@ -248,50 +222,11 @@ export class PortScanCommandClient {
if (this.queue.length > 0) {
this.pump()
} else {
this.scheduleIdleTeardown()
// Terminating can orphan a probe child mid-spawn; the worker reaps what it
// can on exit, and every probe here is short-lived.
this.host.scheduleIdleTeardown()
}
}
private scheduleIdleTeardown(): void {
this.clearIdleTimer()
if (!this.worker) {
return
}
this.idleTimer = setTimeout(() => this.teardownIfIdle(), IDLE_TEARDOWN_MS)
this.idleTimer.unref?.()
}
private teardownIfIdle(): void {
this.idleTimer = null
// Only tear down with nothing active AND nothing queued: a request arriving
// as the timer fires must never be lost to a self-exiting worker.
if (this.active || this.queue.length > 0) {
return
}
this.destroyWorker()
}
private clearIdleTimer(): void {
if (this.idleTimer) {
clearTimeout(this.idleTimer)
this.idleTimer = null
}
}
private destroyWorker(): void {
this.clearIdleTimer()
const worker = this.worker
this.worker = null
if (!worker) {
return
}
this.cleanupWorkerListeners?.()
this.cleanupWorkerListeners = null
worker.removeAllListeners()
// Terminating can orphan a probe child mid-spawn; the worker reaps what it
// can on exit, and every probe here is short-lived.
void worker.terminate().catch(() => undefined)
}
}
function errorMessage(error: unknown): string {