mirror of
https://github.com/stablyai/orca.git
synced 2026-10-29 16:17:51 +00:00
* perf(worktree): make head-identity refresh incremental Head-identity refresh re-read `gitdir` + `HEAD` + a loose ref for every linked worktree on every watcher burst. On a 973-worktree checkout that is ~2,800 metadata reads (~1.0s of main-process fs I/O) per event, and the debounced pipeline fires on every commit in any worktree — so fleet-wide agent activity degenerated into a continuous scan loop. Watcher events already name the admin dir that changed. Classify each event into a head-identity scope, memoize per-entry identities, and re-read only the scoped entries. Refs resolved during a pass are replayed onto cached entries that share the same branch, so `git worktree add --force` siblings stay current without extra reads. Invalidation stays conservative: an absent scope (watcher failure, event overflow, cold start) means a full re-read, `packed-refs` writes invalidate every entry, misses are never memoized, and one refresh per minute is promoted back to a full re-read to bound the window where a ref moves with no event under any admin dir. Measured on the reported 989-entry checkout (macOS/APFS): one-worktree commit 2,816 -> 2 file reads, 61ms -> 0.5ms p50 with an identical page cache; a 20-worktree debounce burst costs 57 reads / 9.7ms; an external `git worktree add`/`remove` costs one readdir / 1.0ms. Refs #17828 * fix(worktree): harden incremental head-identity invalidation Two holes found in self-review: - An admin entry name removed and immediately reused inside one debounce window coalesced into a listing-only scope, so the reused entry kept serving the removed worktree's cached head. Name the entry alongside the listing on every `worktrees/<name>` create/delete. - A non-ENOENT `readdir` failure on `worktrees/` collapsed the memo to the primary row, which then re-emitted every identity on recovery. Mirror worktree-git-common-polling: only a genuinely absent dir means empty; any other error keeps the previous listing. * fix(worktree): let empty-scope bursts still take the head re-baseline Adversarial review found the 60s full-rebaseline promotion was unreachable whenever the triggering burst had an empty head-identity scope: the skip guarded on the raw caller scope and returned before `resolveScope` ran, so `lastFullReadAtMs` was never re-evaluated. A repo whose only churn is `git worktree lock`/`unlock` or a sparse toggle — Orca's own prepared-checkout flow locks and unlocks on every create — could starve the promotion forever and hold a stale head indefinitely. Resolve the scope first and skip on the effective scope. Also stop deferring an add/remove that arrived while the `worktrees/` listing was transiently unreadable: forget the memoized listing so the next refresh re-enumerates whatever its scope, instead of waiting for another listing event. Both fixes carry a test verified to fail without them. * fix(worktree): return head-read completeness instead of sniffing the memo Adversarial review round two. Six fixes, each with a test verified to fail without it. - `readGitCommonHeadIdentities` now returns `{ identities, listingComplete }`. The refresh layer was inferring "enumeration failed" from `cache.entryNames === null`, a reader-owned field whose null also means "cold start" — fragile in production and impossible to express in a mock. - A read discarded by teardown, or one that could not enumerate `worktrees/`, no longer arms the 60s freshness clock. - A queued refresh whose re-run met a destroyed window (macOS recreates the window while the watch lives on) was cleared and dropped. It now stays armed and is folded into the next request. - An incomplete listing carries forward the baseline rows it could not observe, so recovery does not report every linked worktree as changed. - The baseline advances after notifying, so a send into destroyed chrome leaves the move to be retried instead of diffing it away. - A scope naming an entry the memoized listing does not know now forces a re-enumeration instead of resolving to zero work — this removes an unstated dependency on `diffGitCommon` emitting a dir-level create for new entries. - Overflow states FULL at its construction site rather than relying on a downstream `?? FULL` for an absent field. Also documents the load-bearing invariant behind the empty-scope skip (an empty scope only reaches the refresh from a structural burst, which forces `emit: false` and is always paired with a catalog notification for every repo on the watch), and strengthens two tests that could not distinguish the behaviour they claimed. * fix(worktree): bound head-identity staleness with a one-shot catch-up The previous re-baseline was opportunistic: it rode the next refresh, so a ref that moves with no watched write (`git update-ref refs/heads/x` from a sibling worktree) stayed stale until an event happened to arrive after the interval. Pre-PR the very next event anywhere in the repo corrected it, so this was a real narrowing of correctness, not just a pre-existing gap. Arm a one-shot, unref'd timer when a SCOPED pass completes, firing one full re-baseline an interval after the last full read, then disarming. A full pass disarms instead of arming, so it never becomes a background poll, and the timer only exists after an event — an idle repo still schedules nothing and reads nothing. Cost is O(1) timer per active repo and at most one full read per interval: the same operation the old code ran per event, 60x rarer. This also converts "stale until some later event" into "stale at most one interval, period", which is what bounds the blast radius of any invalidation bug in the scoping itself. Cleared on watch disposal. Three tests, each verified to fail without its fix: the catch-up runs with no further events; a quiet repo issues no background reads and the timer disarms after firing; disposal stops it. * fix(worktree): treat an unreadable head as unknown, not absent Reported independently by two PR reviewers. `readTrimmedFile` collapsed every errno to `null`, so an EIO/EACCES/ENFILE on a `gitdir`, `HEAD`, loose ref, or `packed-refs` read was indistinguishable from the file being absent — and the caller deletes the cached identity on `null`. Same conflation AGENTS.md forbids for the SSH verdict vocabulary: loss of contact is not evidence of absence. Reads now report three outcomes, and an unknown: - keeps the entry's last verified identity instead of evicting it, - is never replayed onto siblings sharing the branch as "this ref is gone", - marks the entry unverified so the very next pass re-reads it whatever its scope, and - reports the pass incomplete, so it cannot arm the freshness clock. The reviewers' stated consequence — that an evicted entry stays evicted until the next full pass — did not hold, because `!cache.entries.has(name)` already forced a re-read. The real cost was that one EMFILE evicted every entry it touched and the next pass re-read all of them, which is exactly the full scan this PR exists to remove, plus a spurious re-publish of every row. Renames `listingComplete` to `complete`: it now covers entry reads too.
218 lines
8.9 KiB
TypeScript
218 lines
8.9 KiB
TypeScript
import type { BrowserWindow } from 'electron'
|
|
import { notifyWorktreeHeadIdentitiesChanged } from './worktree-remote'
|
|
import {
|
|
createWorktreeHeadIdentityCache,
|
|
readGitCommonHeadIdentities,
|
|
type WorktreeHeadIdentityCache
|
|
} from './worktree-head-identity-reader'
|
|
import {
|
|
EMPTY_HEAD_IDENTITY_SCOPE,
|
|
FULL_HEAD_IDENTITY_SCOPE,
|
|
isEmptyHeadIdentityScope,
|
|
mergeHeadIdentityScopes,
|
|
type WorktreeHeadIdentityScope
|
|
} from './worktree-head-identity-scope'
|
|
|
|
type HeadIdentityWatchHost = {
|
|
path: string
|
|
repos: ReadonlyMap<string, unknown>
|
|
mainWindow: BrowserWindow
|
|
disposed: boolean
|
|
}
|
|
|
|
export type WorktreeHeadIdentityRefreshState = {
|
|
/** worktreePath → `${head} ${branch}` from the last metadata-file read. */
|
|
baseline: Map<string, string> | null
|
|
cache: WorktreeHeadIdentityCache
|
|
lastFullReadAtMs: number
|
|
inFlight: boolean
|
|
queuedScope: WorktreeHeadIdentityScope | null
|
|
queuedEmit: boolean
|
|
/** One-shot catch-up armed only by a scoped pass; see `scheduleRebaseline`. */
|
|
rebaselineTimer: ReturnType<typeof setTimeout> | null
|
|
}
|
|
|
|
// Why: a ref can move with no event under any admin dir — `git update-ref
|
|
// refs/heads/x` from a sibling worktree appends no HEAD reflog for the worktree
|
|
// that has `x` checked out (verified on git 2.44). Scoped refreshes cannot see
|
|
// that, so a full re-read is forced this long after the last one, whether or
|
|
// not another event arrives. This is also what bounds the blast radius of any
|
|
// invalidation bug in the scoping itself.
|
|
export const HEAD_IDENTITY_FULL_REBASELINE_INTERVAL_MS = 60_000
|
|
|
|
export function createWorktreeHeadIdentityRefreshState(): WorktreeHeadIdentityRefreshState {
|
|
return {
|
|
baseline: null,
|
|
cache: createWorktreeHeadIdentityCache(),
|
|
lastFullReadAtMs: 0,
|
|
inFlight: false,
|
|
queuedScope: null,
|
|
queuedEmit: false,
|
|
rebaselineTimer: null
|
|
}
|
|
}
|
|
|
|
export function disposeWorktreeHeadIdentityRefreshState(
|
|
state: WorktreeHeadIdentityRefreshState
|
|
): void {
|
|
clearTimeout(state.rebaselineTimer ?? undefined)
|
|
state.rebaselineTimer = null
|
|
}
|
|
|
|
/** Arms the one-shot catch-up that turns "stale until some later event happens
|
|
* to arrive" into "stale at most one interval". Only a scoped pass arms it, so
|
|
* the timer exists only after an event: an idle repo schedules nothing, and a
|
|
* full pass disarms because nothing is outstanding after one. */
|
|
function scheduleRebaseline(
|
|
host: HeadIdentityWatchHost,
|
|
state: WorktreeHeadIdentityRefreshState
|
|
): void {
|
|
disposeWorktreeHeadIdentityRefreshState(state)
|
|
if (host.disposed || host.mainWindow.isDestroyed()) {
|
|
return
|
|
}
|
|
const dueInMs = Math.max(
|
|
0,
|
|
state.lastFullReadAtMs + HEAD_IDENTITY_FULL_REBASELINE_INTERVAL_MS - Date.now()
|
|
)
|
|
const timer = setTimeout(() => {
|
|
state.rebaselineTimer = null
|
|
// `emit: true`: unlike a structural burst, nothing else runs alongside this
|
|
// to correct the drift, so a silent re-baseline would bury it forever.
|
|
void refreshWorktreeHeadIdentities(host, state, true, FULL_HEAD_IDENTITY_SCOPE)
|
|
}, dueInMs)
|
|
// Never hold the process open for a freshness backstop.
|
|
timer.unref?.()
|
|
state.rebaselineTimer = timer
|
|
}
|
|
|
|
function headIdentitySignature(identity: { head: string; branch: string | null }): string {
|
|
return `${identity.head} ${identity.branch ?? ''}`
|
|
}
|
|
|
|
function resolveScope(
|
|
state: WorktreeHeadIdentityRefreshState,
|
|
scope: WorktreeHeadIdentityScope
|
|
): WorktreeHeadIdentityScope {
|
|
if (scope.all || state.baseline === null) {
|
|
return FULL_HEAD_IDENTITY_SCOPE
|
|
}
|
|
return Date.now() - state.lastFullReadAtMs >= HEAD_IDENTITY_FULL_REBASELINE_INTERVAL_MS
|
|
? FULL_HEAD_IDENTITY_SCOPE
|
|
: scope
|
|
}
|
|
|
|
/** Diffs metadata-file head reads against the previous baseline and notifies
|
|
* only actual head moves, so status-only churn (index rewrites from external
|
|
* `git status`) stays silent and never re-enters structural fanout. Passing
|
|
* `emit: false` re-baselines without notifying — structural ticks already
|
|
* run the authoritative worktree listing.
|
|
*
|
|
* `scope` narrows the read to the worktrees a watcher burst could have moved;
|
|
* omitting it (watcher errors, event overflow, cold start) re-reads everything. */
|
|
export async function refreshWorktreeHeadIdentities(
|
|
host: HeadIdentityWatchHost,
|
|
state: WorktreeHeadIdentityRefreshState,
|
|
emit: boolean,
|
|
scope: WorktreeHeadIdentityScope = FULL_HEAD_IDENTITY_SCOPE
|
|
): Promise<void> {
|
|
if (host.disposed || host.mainWindow.isDestroyed()) {
|
|
return
|
|
}
|
|
if (state.inFlight) {
|
|
state.queuedScope = mergeHeadIdentityScopes(
|
|
state.queuedScope ?? EMPTY_HEAD_IDENTITY_SCOPE,
|
|
scope
|
|
)
|
|
state.queuedEmit ||= emit
|
|
return
|
|
}
|
|
// Why: the queued re-run below can be handed to a window that was destroyed
|
|
// mid-read (macOS recreates it while the watch lives on), and that call
|
|
// returns at the guard above. Fold anything still queued into this request so
|
|
// a stranded scope is never dropped on the floor.
|
|
const requestedScope = state.queuedScope
|
|
? mergeHeadIdentityScopes(state.queuedScope, scope)
|
|
: scope
|
|
const requestedEmit = emit || state.queuedEmit
|
|
state.queuedScope = null
|
|
state.queuedEmit = false
|
|
// Resolve BEFORE the skip: an empty scope is still an opportunity to take the
|
|
// periodic re-baseline, and a repo whose only churn is `git worktree
|
|
// lock`/`unlock` or a sparse toggle must not be able to starve it forever.
|
|
const effectiveScope = resolveScope(state, requestedScope)
|
|
// Nothing the burst touched can move a head (a `locked` or `config.worktree`
|
|
// write) and no re-baseline is due: read nothing.
|
|
//
|
|
// Load-bearing invariant behind the promotion above: an empty scope only ever
|
|
// reaches here from `structuralChange(repoIds, EMPTY)`, which populates
|
|
// `pendingStructure` for EVERY repo on this watch (`allRepoIds`), which forces
|
|
// `emit: false`. So a promoted re-baseline triggered by an empty-scope burst
|
|
// corrects the baseline silently and publishes nothing — safe precisely
|
|
// because the same flush already sent that watch's repos a catalog
|
|
// notification, and the renderer's authoritative listing carries the head.
|
|
if (state.baseline !== null && isEmptyHeadIdentityScope(effectiveScope)) {
|
|
return
|
|
}
|
|
state.inFlight = true
|
|
try {
|
|
const { identities, complete } = await readGitCommonHeadIdentities(
|
|
host.path,
|
|
state.cache,
|
|
effectiveScope
|
|
)
|
|
if (host.disposed || host.mainWindow.isDestroyed()) {
|
|
return
|
|
}
|
|
// After the teardown check, so a read whose result is discarded cannot pass
|
|
// for a checkpoint. A read that could not enumerate `worktrees/`, or that hit
|
|
// an unreadable entry, has not observed the whole repo — not one either.
|
|
if (effectiveScope.all && complete) {
|
|
state.lastFullReadAtMs = Date.now()
|
|
}
|
|
const baseline = state.baseline
|
|
// Rows this pass could not observe are carried forward: dropping them would
|
|
// make the next successful listing report every linked worktree as changed.
|
|
const nextBaseline = complete ? new Map<string, string>() : new Map(baseline ?? [])
|
|
for (const identity of identities) {
|
|
nextBaseline.set(identity.worktreePath, headIdentitySignature(identity))
|
|
}
|
|
// Why `emit: false` is safe to publish nothing: the only classification that
|
|
// reaches here with a head-moving scope and no emit is a structural burst,
|
|
// and structural bursts notify the worktree catalog for every repo on this
|
|
// watch — the renderer's authoritative listing carries the head. So a silent
|
|
// pass is always paired with a stronger refresh, never a dropped update.
|
|
const changed =
|
|
baseline && requestedEmit
|
|
? identities.filter(
|
|
(identity) => baseline.get(identity.worktreePath) !== headIdentitySignature(identity)
|
|
)
|
|
: []
|
|
for (const repoId of changed.length > 0 ? host.repos.keys() : []) {
|
|
notifyWorktreeHeadIdentitiesChanged(host.mainWindow, repoId, changed)
|
|
}
|
|
// Last, so a send that throws part-way leaves the old baseline and the next
|
|
// refresh re-diffs instead of silently dropping the move.
|
|
state.baseline = nextBaseline
|
|
} catch (error) {
|
|
console.warn(`[worktree-base-watcher] head identity read failed for ${host.path}:`, error)
|
|
// A failed read leaves the cache in an unknown state; force the next
|
|
// refresh to re-read every entry rather than trust a partial memo.
|
|
state.cache = createWorktreeHeadIdentityCache()
|
|
state.lastFullReadAtMs = 0
|
|
} finally {
|
|
state.inFlight = false
|
|
if (state.queuedScope && !host.disposed) {
|
|
// Leave the queue armed: if this call cannot proceed (destroyed window),
|
|
// the next refresh folds it back in at the entry above.
|
|
void refreshWorktreeHeadIdentities(host, state, state.queuedEmit, state.queuedScope)
|
|
} else if (effectiveScope.all) {
|
|
// A full pass just ran (or failed while running full — re-arming there
|
|
// would spin on a persistent fs error). Nothing is outstanding.
|
|
disposeWorktreeHeadIdentityRefreshState(state)
|
|
} else {
|
|
scheduleRebaseline(host, state)
|
|
}
|
|
}
|
|
}
|