Files
orca/src/main/ipc/worktree-head-identity-reader.test.ts
T
Neil 1603810dde perf(worktree): make head-identity refresh incremental (#17843)
* perf(worktree): make head-identity refresh incremental

Head-identity refresh re-read `gitdir` + `HEAD` + a loose ref for every
linked worktree on every watcher burst. On a 973-worktree checkout that is
~2,800 metadata reads (~1.0s of main-process fs I/O) per event, and the
debounced pipeline fires on every commit in any worktree — so fleet-wide
agent activity degenerated into a continuous scan loop.

Watcher events already name the admin dir that changed. Classify each event
into a head-identity scope, memoize per-entry identities, and re-read only
the scoped entries. Refs resolved during a pass are replayed onto cached
entries that share the same branch, so `git worktree add --force` siblings
stay current without extra reads.

Invalidation stays conservative: an absent scope (watcher failure, event
overflow, cold start) means a full re-read, `packed-refs` writes invalidate
every entry, misses are never memoized, and one refresh per minute is
promoted back to a full re-read to bound the window where a ref moves with
no event under any admin dir.

Measured on the reported 989-entry checkout (macOS/APFS): one-worktree
commit 2,816 -> 2 file reads, 61ms -> 0.5ms p50 with an identical page
cache; a 20-worktree debounce burst costs 57 reads / 9.7ms; an external
`git worktree add`/`remove` costs one readdir / 1.0ms.

Refs #17828

* fix(worktree): harden incremental head-identity invalidation

Two holes found in self-review:

- An admin entry name removed and immediately reused inside one debounce
  window coalesced into a listing-only scope, so the reused entry kept
  serving the removed worktree's cached head. Name the entry alongside the
  listing on every `worktrees/<name>` create/delete.
- A non-ENOENT `readdir` failure on `worktrees/` collapsed the memo to the
  primary row, which then re-emitted every identity on recovery. Mirror
  worktree-git-common-polling: only a genuinely absent dir means empty; any
  other error keeps the previous listing.

* fix(worktree): let empty-scope bursts still take the head re-baseline

Adversarial review found the 60s full-rebaseline promotion was unreachable
whenever the triggering burst had an empty head-identity scope: the skip
guarded on the raw caller scope and returned before `resolveScope` ran, so
`lastFullReadAtMs` was never re-evaluated. A repo whose only churn is
`git worktree lock`/`unlock` or a sparse toggle — Orca's own prepared-checkout
flow locks and unlocks on every create — could starve the promotion forever
and hold a stale head indefinitely. Resolve the scope first and skip on the
effective scope.

Also stop deferring an add/remove that arrived while the `worktrees/` listing
was transiently unreadable: forget the memoized listing so the next refresh
re-enumerates whatever its scope, instead of waiting for another listing event.

Both fixes carry a test verified to fail without them.

* fix(worktree): return head-read completeness instead of sniffing the memo

Adversarial review round two. Six fixes, each with a test verified to fail
without it.

- `readGitCommonHeadIdentities` now returns `{ identities, listingComplete }`.
  The refresh layer was inferring "enumeration failed" from `cache.entryNames
  === null`, a reader-owned field whose null also means "cold start" — fragile
  in production and impossible to express in a mock.
- A read discarded by teardown, or one that could not enumerate `worktrees/`,
  no longer arms the 60s freshness clock.
- A queued refresh whose re-run met a destroyed window (macOS recreates the
  window while the watch lives on) was cleared and dropped. It now stays armed
  and is folded into the next request.
- An incomplete listing carries forward the baseline rows it could not observe,
  so recovery does not report every linked worktree as changed.
- The baseline advances after notifying, so a send into destroyed chrome leaves
  the move to be retried instead of diffing it away.
- A scope naming an entry the memoized listing does not know now forces a
  re-enumeration instead of resolving to zero work — this removes an unstated
  dependency on `diffGitCommon` emitting a dir-level create for new entries.
- Overflow states FULL at its construction site rather than relying on a
  downstream `?? FULL` for an absent field.

Also documents the load-bearing invariant behind the empty-scope skip (an empty
scope only reaches the refresh from a structural burst, which forces
`emit: false` and is always paired with a catalog notification for every repo
on the watch), and strengthens two tests that could not distinguish the
behaviour they claimed.

* fix(worktree): bound head-identity staleness with a one-shot catch-up

The previous re-baseline was opportunistic: it rode the next refresh, so a ref
that moves with no watched write (`git update-ref refs/heads/x` from a sibling
worktree) stayed stale until an event happened to arrive after the interval.
Pre-PR the very next event anywhere in the repo corrected it, so this was a
real narrowing of correctness, not just a pre-existing gap.

Arm a one-shot, unref'd timer when a SCOPED pass completes, firing one full
re-baseline an interval after the last full read, then disarming. A full pass
disarms instead of arming, so it never becomes a background poll, and the timer
only exists after an event — an idle repo still schedules nothing and reads
nothing. Cost is O(1) timer per active repo and at most one full read per
interval: the same operation the old code ran per event, 60x rarer.

This also converts "stale until some later event" into "stale at most one
interval, period", which is what bounds the blast radius of any invalidation
bug in the scoping itself.

Cleared on watch disposal. Three tests, each verified to fail without its fix:
the catch-up runs with no further events; a quiet repo issues no background
reads and the timer disarms after firing; disposal stops it.

* fix(worktree): treat an unreadable head as unknown, not absent

Reported independently by two PR reviewers. `readTrimmedFile` collapsed every
errno to `null`, so an EIO/EACCES/ENFILE on a `gitdir`, `HEAD`, loose ref, or
`packed-refs` read was indistinguishable from the file being absent — and the
caller deletes the cached identity on `null`. Same conflation AGENTS.md forbids
for the SSH verdict vocabulary: loss of contact is not evidence of absence.

Reads now report three outcomes, and an unknown:

- keeps the entry's last verified identity instead of evicting it,
- is never replayed onto siblings sharing the branch as "this ref is gone",
- marks the entry unverified so the very next pass re-reads it whatever its
  scope, and
- reports the pass incomplete, so it cannot arm the freshness clock.

The reviewers' stated consequence — that an evicted entry stays evicted until
the next full pass — did not hold, because `!cache.entries.has(name)` already
forced a re-read. The real cost was that one EMFILE evicted every entry it
touched and the next pass re-read all of them, which is exactly the full scan
this PR exists to remove, plus a spurious re-publish of every row.

Renames `listingComplete` to `complete`: it now covers entry reads too.
2026-09-01 02:53:05 -07:00

158 lines
5.8 KiB
TypeScript

import { afterEach, describe, expect, it } from 'vitest'
import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { readGitCommonHeadIdentities } from './worktree-head-identity-reader'
const readIdentities = async (
...args: Parameters<typeof readGitCommonHeadIdentities>
): Promise<Awaited<ReturnType<typeof readGitCommonHeadIdentities>>['identities']> =>
(await readGitCommonHeadIdentities(...args)).identities
const OID_A = 'a'.repeat(40)
const OID_B = 'b'.repeat(40)
const OID_C = 'c'.repeat(40)
describe('readGitCommonHeadIdentities', () => {
const roots: string[] = []
afterEach(async () => {
await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true })))
})
async function makeCommonDir(): Promise<string> {
const root = await mkdtemp(join(tmpdir(), 'orca-head-reader-'))
roots.push(root)
const commonDir = join(root, 'checkout', '.git')
await mkdir(commonDir, { recursive: true })
return commonDir
}
async function writeLooseRef(commonDir: string, ref: string, oid: string): Promise<void> {
const refPath = join(commonDir, ...ref.split('/'))
await mkdir(dirname(refPath), { recursive: true })
await writeFile(refPath, `${oid}\n`)
}
async function addLinkedWorktree(
commonDir: string,
name: string,
worktreePath: string,
headContent: string
): Promise<void> {
const entry = join(commonDir, 'worktrees', name)
await mkdir(entry, { recursive: true })
await writeFile(join(entry, 'HEAD'), `${headContent}\n`)
await writeFile(join(entry, 'gitdir'), `${join(worktreePath, '.git')}\n`)
}
it('resolves the primary checkout and linked worktrees from loose refs', async () => {
const commonDir = await makeCommonDir()
await writeFile(join(commonDir, 'HEAD'), 'ref: refs/heads/main\n')
await writeLooseRef(commonDir, 'refs/heads/main', OID_A)
await writeLooseRef(commonDir, 'refs/heads/feature', OID_B)
const linkedPath = join(dirname(commonDir), '..', 'linked-wt')
await addLinkedWorktree(commonDir, 'linked-wt', linkedPath, 'ref: refs/heads/feature')
const identities = await readIdentities(commonDir)
expect(identities).toContainEqual({
worktreePath: dirname(commonDir),
head: OID_A,
branch: 'refs/heads/main'
})
expect(identities).toContainEqual({
worktreePath: linkedPath,
head: OID_B,
branch: 'refs/heads/feature'
})
})
it('falls back to packed-refs when the loose ref is absent', async () => {
const commonDir = await makeCommonDir()
await writeFile(join(commonDir, 'HEAD'), 'ref: refs/heads/main\n')
await writeFile(
join(commonDir, 'packed-refs'),
`# pack-refs with: peeled fully-peeled sorted\n${OID_C} refs/heads/main\n^${OID_A}\n`
)
const identities = await readIdentities(commonDir)
expect(identities).toEqual([
{ worktreePath: dirname(commonDir), head: OID_C, branch: 'refs/heads/main' }
])
})
it('reports detached HEAD as a raw oid with a null branch', async () => {
const commonDir = await makeCommonDir()
await writeFile(join(commonDir, 'HEAD'), `${OID_B}\n`)
const identities = await readIdentities(commonDir)
expect(identities).toEqual([{ worktreePath: dirname(commonDir), head: OID_B, branch: null }])
})
it('skips unborn branches instead of emitting partial identities', async () => {
const commonDir = await makeCommonDir()
await writeFile(join(commonDir, 'HEAD'), 'ref: refs/heads/unborn\n')
expect(await readIdentities(commonDir)).toEqual([])
})
it('resolves relative gitdir entries against the metadata dir', async () => {
const commonDir = await makeCommonDir()
await writeLooseRef(commonDir, 'refs/heads/feature', OID_B)
const entry = join(commonDir, 'worktrees', 'rel-wt')
await mkdir(entry, { recursive: true })
await writeFile(join(entry, 'HEAD'), 'ref: refs/heads/feature\n')
await writeFile(join(entry, 'gitdir'), `${join('..', '..', '..', '..', 'rel-wt', '.git')}\n`)
const identities = await readIdentities(commonDir)
expect(identities).toEqual([
{
worktreePath: join(dirname(commonDir), '..', 'rel-wt'),
head: OID_B,
branch: 'refs/heads/feature'
}
])
})
it('rejects traversal-shaped symrefs instead of reading outside the common dir', async () => {
const commonDir = await makeCommonDir()
// A crafted repo can put arbitrary content in HEAD; backslash segments
// would traverse on Windows where join treats them as separators too.
await writeFile(join(dirname(commonDir), '..', 'outside.txt'), 'secret\n')
for (const ref of [
'refs/heads/../../../outside.txt',
'refs\\..\\..\\..\\outside.txt',
'refs/heads/evil:name',
'refs//heads'
]) {
await writeFile(join(commonDir, 'HEAD'), `ref: ${ref}\n`)
expect(await readIdentities(commonDir)).toEqual([])
}
})
it('never emits resolved content that is not a hex object id', async () => {
const commonDir = await makeCommonDir()
await writeFile(join(commonDir, 'HEAD'), 'ref: refs/heads/main\n')
await writeLooseRef(commonDir, 'refs/heads/main', 'not-an-object-id')
expect(await readIdentities(commonDir)).toEqual([])
await writeFile(join(commonDir, 'HEAD'), 'this is not a detached oid\n')
expect(await readIdentities(commonDir)).toEqual([])
})
it('omits the primary row for non-standard common dir layouts', async () => {
const root = await mkdtemp(join(tmpdir(), 'orca-head-reader-'))
roots.push(root)
const commonDir = join(root, 'bare-repo')
await mkdir(commonDir, { recursive: true })
await writeFile(join(commonDir, 'HEAD'), 'ref: refs/heads/main\n')
await writeLooseRef(commonDir, 'refs/heads/main', OID_A)
expect(await readIdentities(commonDir)).toEqual([])
})
})