orca/docs/reference/agent-session-search-query-tuning.md

Agent session search: query tuning

What a search costs, and what the knobs in src/main/ai-vault-search/session-search-engine.ts buy. Every number here comes from config/scripts/session-search-query-benchmark.ts over the synthetic corpus in session-search-synthetic-corpus.ts, except the conversation_fts shoot-out, which writes its own corpus because the answer turns on how much of a transcript is tool output. Nothing in this file was measured against a real transcript, and neither benchmark must ever be pointed at one.

Running it

The benchmark is a top-level-await module that imports the main-process tree by extensionless path, so it needs a bundler-backed runner rather than bare node:

cat > src/main/ai-vault-search/zz-bench.test.ts <<'EOF'
import { it } from 'vitest'
it('runs', { timeout: 1_800_000 }, async () => {
  await import('../../../config/scripts/session-search-query-benchmark')
})
EOF
BENCH_OUT=/tmp/ss-query-bench.json pnpm test src/main/ai-vault-search/zz-bench.test.ts
rm src/main/ai-vault-search/zz-bench.test.ts

The conversation_fts shoot-out below runs the same way, importing config/scripts/session-search-conversation-fts-benchmark instead, with CORPUS_MB and TOOL_SHARE to size and shape its corpus. config/scripts is not inside any typecheck project, so while that throwaway test exists tsc reports TS6307 for each script it pulls in; delete it and the run is clean again.

BENCH_OUT exists because vitest intercepts console.log; the report is written to that path as well as printed.

Scope: what the second FTS table buys a reader

Corpus: 40 synthetic Claude transcripts, 10.5 MB, 9,600 messages, indexed through the real store. Eight queries, one per rung of the route ladder plus the two shapes that skip it; 5 warm-up runs and 25 samples each. Apple silicon, warm page cache, machine otherwise idle. Milliseconds, and p95 over 25 samples moves several milliseconds run to run if anything else is competing for the disk.

Scope	p50	p95
all	7.22	8.94
conversation	5.33	7.86

Per query, all then conversation (p50 / p95):

Query	all	conversation
"terminal reattach" (phrase)	5.24 / 8.42	2.97 / 3.24
resolveTerminalPath (identifier)	7.55 / 8.94	6.47 / 6.72
src/main/…/session-transcript-reader.ts (path)	8.69 / 10.12	7.78 / 8.04
why is the daemon snapshot stale (prose)	7.84 / 8.57	5.90 / 7.01
reattahc worktre (typo repair)	7.30 / 7.39	5.53 / 5.89
index (common term)	5.45 / 5.66	3.81 / 4.02
repo:app-3 (operator only)	0.12 / 0.16	0.10 / 0.10
worktree scoped to one cwd	1.47 / 1.63	1.25 / 1.49

Reading it:

• conversation is about 1.4x faster at p50 and 1.1x at p95, and it is a column filter over the same table rather than a table of its own. Narrowing to the two prose columns is what buys the gap: fewer postings to score. It is also the scope where a match is something a person wrote rather than something a tool printed.
• A scopePaths query is the cheapest real search on the page. It is the one narrowing SQL can express exactly, so it seeks sessions_cwd_key and hands ranking a small candidate set.
• The operator-only figure is a floor, not a typical cost. repo: and path: are applied in JS over retrieved rows (see session-search-row-filter for why they cannot be pushed into SQL), so their cost tracks how many sessions the walk has to read before it fills a candidate set. This corpus has 40 sessions, which is one page of that walk; an index where few sessions match the operator will read up to the ceiling in session-search-retrieval instead.

What the conversation scope costs at real corpus size

conversation was a second FTS table holding a copy of the two prose columns. It is a column filter now — {user_text assistant_text}: (…) with bm25 weights that zero the other two — and PR 2 deleted the table on the strength of the shoot-out this section used to hold: the filter came in at 1.16-1.36x the p95 of the dedicated table, under the 2x bar, while the table cost a tenth of the index to maintain. What follows is what the shipped schema actually does, measured again on the same corpus after the table went and tool rows were capped.

Corpus: Claude transcripts from config/scripts/session-search-tool-heavy-corpus.ts, 105 MB, indexed through the real store, at two points in the 80-97% band a real transcript tree sits in. Half the tokens in tool output are words the conversation also uses, so a conversation term really does have postings the filter must discard. Twenty queries per rung, both scopes interleaved query by query, warm cache; config/scripts/session-search-scope-benchmark.ts, run twice.

Tool share	Rung	all p50 / p95	conversation p50 / p95
86%	phrase	16.69 / 17.48	13.08 / 13.52
86%	or	31.91 / 35.74	22.25 / 23.87
86%	and	70.04 / 74.00	53.47 / 59.39
93%	phrase	9.14 / 13.36	7.23 / 8.51
93%	or	16.46 / 18.70	12.34 / 14.88
93%	and	39.65 / 43.44	31.05 / 32.92

Three things to read out of it.

The filter is a win, not a cost. Every rung is faster narrow than wide, by 1.2x to 1.4x at p50. The shoot-out compared the filter against a table built for exactly this query; against the wide table it replaces, it does what the second table did, which is read fewer postings.

The and rung is where the corpus size shows. Those queries are eight terms, chosen so no ordered run that long occurs and the phrase rung has to miss; a real two-term AND sits nearer the phrase row. It is also the noisiest: the second run's p95 reached 140 ms on one bucket, which is what twenty samples of a 70 ms query buys. Read the p50 column.

The index is far smaller than the shoot-out's was. 57 MB at 93% tool output and 103 MB at 86%, against roughly 150 MB for messages_fts alone before PR 2 capped an indexed tool row at 3,072 characters. Most of a tool-heavy transcript is now not in the index at all, which moves every number above and is the larger effect of the two.

What is not measured here is relevance, and the column filter does carry one ranking difference the deleted table did not. FTS5's bm25 normalises by the whole row's length and has no per-column length, so two rows with identical prose score differently when one also holds tool output. The rowid set is unchanged, which is what the deletion was decided on; the order within it can move. session-search-engine.test.ts pins the direction.

sessionCandidateLimit

The reviewer's F13: this is a tunable default, not a constant. It bounds how many sessions the SQL hands ranking, so it bounds both retrieval cost and how deep a caller can page before the answer simply stops.

The limit only costs anything once more sessions match than the limit allows, so this is measured over a second corpus: 2,500 one-turn transcripts, 10.9 MB, every one of them matching the query. Limits are interleaved sample by sample, because run back to back the first configuration pays for every page the OS cache had not seen and the ordering alone moves p95 further than the limit does.

Limit	p50	p95	Pages of 20 a caller can reach
200	6.85	7.21	10
600	7.93	8.36	30
1200	9.55	10.53	60
2400	12.32	13.45	120

600 is the default: it costs about 16% over 200 at p50 and buys three times the reachable depth, and the curve only turns steep past 1200. A host with a much larger index can raise it; the result's truncated.candidates says when the limit was the thing that cut the answer, so a caller never has to guess.

What is not measured here is relevance. These numbers say what a limit costs, not what it retrieves. The MRR figures quoted in the BM25 weights (session-search-retrieval.ts) and in the identifier shadow column (session-search-identifier-split.ts) come from the original retrieval shoot-out on real transcripts and are not reproducible from this repository. Any change to the limit justified on relevance grounds needs an eval set, not this benchmark.

What typo repair costs

The repair is the one rung whose cost tracks the size of the vocabulary rather than the size of a result. It only runs for a term the scope has no posting for, so an ordinary query never pays it; a query of nonsense pays it once per term.

Measured over a synthetic vocabulary of 1.6 M distinct terms, every term in two rows so none is filtered out:

Query	p50
one known term (no repair)	11 ms
one unknown term	10 ms
39 unknown 12-character terms (480 ch)	387 ms
12 unknown 40-character terms	99 ms

Two things follow. The cost is linear in unknown terms and in vocabulary size, and search is synchronous, so a 512-character query of nonsense holds the thread for a third of a second on an index that large. And the scoped-count fix made this cheaper rather than dearer — it was 737 ms before — because ordering the vocabulary scan by term drops the sort that ordering by doc required, and the counts it added are at most eight bounded probes per prefix. A cap on unknown terms per query is recorded as a follow-up in the split plan.

Page warmup, dropped

PR 2 deferred warm() — a sliced read of messages that pulls its pages into the OS cache before the first query — to whoever knew which pages a read touches. It is not re-added here, for two reasons. The measurement that justified it (first query 1.3 s to 0.45 s) was on a 4 GB index, and neither corpus in this file is within an order of magnitude of that, so PR 4 cannot show a win: removing the call moved the 10.5 MB corpus's p50 by less than the run-to-run spread. And it is a cancellable background pass, which needs an owner with a lifecycle; a query library that holds no timers has nothing to hang the stopped() on, and a fire-and-forget async read from a synchronous search is a rejection nothing can supervise. It belongs with the indexer in PR 3b, which already owns starting and stopping work.

Not settled here

Which process may open, unlink and rebuild the index is PR 3b's decision. A second handle that finds an older schema version replaces the file while a live store keeps answering from the unlinked inode, and this PR is what first makes that reachable, because it is the first thing that reads. What PR 4 does is refuse to make it worse. The engine restores its derived vocabulary and generation triggers before a search. A missing messages_fts fails clearly; the connection owner must rebuild the source index. There is no degraded-search capability state or query logging. Logging can be added by a caller when an evaluation consumer exists.

Each search checks the generation before retrieval and after its final content read. A concurrent commit rejects the page with stale-generation, including a first page without a cursor. The caller can retry from page one. No long-lived read transaction is needed, and a mixed page is never returned as a valid snapshot.

Repository/path operators are applied before a phrase or AND route is accepted. Candidate truncation is reported by the rung that answered, not by every rung tried. Each rung of the ladder matches a superset of the one before it, so a rung that reached its cap with no eligible sessions is always followed by one that reaches it too: a full candidate set stays explicit either way.

The phrase and AND rungs run for prose as well as for literal-looking input, over the query's tokens as typed rather than the stop-word-stripped OR body. A sentence pasted out of a transcript is ordinary words in order; over OR its common words fill the candidate limit with recent sessions and the old session holding the sentence never reaches ranking. The cost is two FTS queries that usually miss, which on the corpus above sits inside this harness's run-to-run noise. A one-token query still takes the rung only when it looked literal.