Files
orca/src/main/runtime/fetch-remote-cache.test.ts
T
Neil d7123591ce perf(git): pack the loose refs Orca's own fetches leave behind (#17857)
* perf(git): pack the loose refs Orca's own fetches leave behind

Orca strips git's auto-maintenance off every fetch it issues
(GIT_FETCH_SKIP_AUTO_MAINTENANCE_CONFIG_ARGS) and never compensated, so
nothing in an Orca-driven checkout ever packs refs. One real machine
reached 36,574 loose refs, where `git show-ref -- main` costs 5.2s and
every worktree create pays for it.

Add an idle-time, per-repo `git pack-refs --all --prune`, armed by the
fetches that create the debt. It runs only after ten minutes of quiet on
that repo, only above 1000 loose refs (probed with a walk bounded by that
threshold, not by the backlog), one at a time across the whole app, at
the background admission tier, and never while an agent is working, a
create is prepared or in flight, a worktree removal is deleting refs, the
app is quitting, or the machine is on battery. A user who set
`maintenance.auto=false` or `gc.auto=0` has opted out.

Measured on a 36,001-loose-ref fixture (macOS/APFS, git 2.44):
`show-ref` 5.5-12.2s -> 30-49ms, `for-each-ref` 4.0-10.8s -> 43-48ms.

Also fixes a pre-existing bug the split exposed: `--path-format=absolute`
is ignored before git 2.31, and taking rev-parse's stdout raw collapsed
every repo on such a host onto one fetch-serialization key.

Refs #17828

* perf(git): make idle ref maintenance preemptible and cheaper to probe

The idle veto was one-directional: it stopped a pack from starting during
a create, removal, or agent work, but nothing stopped those from starting
during a pack. A user-clicked Fetch, a branch delete, or a worktree
removal that needed `packed-refs.lock` mid-rewrite could fail with
`unable to create packed-refs.lock` -- a git error with no visible cause.

Make the pack cancellable end to end. An AbortSignal now reaches the
`pack-refs` child and both pre-pack probes, and `pause()` aborts what is
running, waits for it to actually stop, and holds a suspension count so
nothing new starts until the caller releases. Every entry point that
deletes a ref takes that pause: gitFetch, gitPull, gitFastForward,
removeWorktree, forceDeleteLocalBranch, prepareWorktreeCreateCheckout,
addWorktree. Five more triggers close the rest of the window: battery
drop, window focus, quit, the attempt deadline, and any other git command
queueing for an admission slot.

Judge a pack by re-probing the backlog rather than by the child's exit
code. Measured in the field: another Orca session moved a branch
mid-pack, git reported `cannot lock ref`, skipped that ref and packed the
rest -- 36,688 loose refs down to 3. On a machine running several
sessions that is the normal case, and retrying it would be wrong.

Probe with one batched `readdir` per directory instead of streaming
`opendir`, which issues a thread-pool round trip every 32 entries: 177ms
-> 23ms on a real 36,600-ref repository, with half the event-loop lag.
The walk stays strictly sequential so it can never occupy more than one
of libuv's four filesystem threads.

`PackRefsLockOwnership` makes a lock left by SIGKILL attributable, and
only reclaims one when a marker exists, the lock is older than any
pack-refs could run for, and the recorded process is gone.

Refs #17828

* fix(git): wait out the packed-refs lock instead of killing the pack

Measured on Git 2.55/APFS with 37k loose refs: a full `pack-refs --all
--prune` takes 23-32s but holds `packed-refs.lock` for only 0.03-1.37s of
it. The other ~95% is the prune phase, during which a concurrent `fetch
--prune`, `branch -D` or `update-ref` succeeds every time -- per-ref locks
last microseconds and git retries for `core.filesRefLockTimeout`.

So the abort-on-everything design was strictly harmful. SIGTERM into the
prune loop strands an empty `refs/**/*.lock` about one time in five
(9/30, 5/40, 6/30 kills): `tempfile.c` opens the lock O_EXCL before
`activate_tempfile()` links it into the list the signal handler walks,
and a pack does ~36k lock cycles. Afterwards `update-ref -d` on that ref
fails with `cannot lock ref ... File exists`, permanently. On Windows
`taskkill /f` never runs git's handlers at all, so an abort inside the
rewrite strands `packed-refs.lock` every time.

Never signal the child. `packRefs` no longer takes an abort signal; it
polls `packed-refs.lock` and reports the window through a
`PackedRefsLockReporter`. `pause()` resolves when the lock is released --
bounded, and free during the prune -- while the suspension counter still
blocks new attempts. Battery and window-focus become do-not-start rather
than stop-what-is-running, and quit waits for the lock and lets the child
finish orphaned.

For strands that already exist, `PackRefsLockOwnership` now also reclaims
`refs/**/*.lock` under the same three conditions plus a 0-byte check, and
a lock carrying our own not-yet-reclaimable marker records `locked` with
a 30min retry instead of the 6h failure cooldown -- so a Windows strand
self-heals in half an hour rather than six.

Reverts the git admission-scheduler event bus, which existed only to
drive the abort this removes.

Refs #17828

* test(git): make the ref-maintenance waits survive a loaded runner

CI shard 4/8 failed on `restarts every armed countdown when the user does
ref work themselves`, which passes locally. The `until()` helper spun a
fixed 200 event-loop turns and then returned silently, so on a contended
runner the filesystem probe had not finished and the assertion that
followed failed with an unrelated message.

Bound the wait by wall clock instead and throw a named error, which
immediately exposed a second latent bug: the single-flight test's second
wait could never succeed, because the deferred repo's retry is on a faked
`setTimeout` that spinning the real loop never advances. It had been
passing only because the old helper gave up quietly. Add a timer-aware
variant for those, and have the countdown test await a signal the fake
pack resolves rather than polling at all.

Verified stable across five sequential runs and once under load average
32 with six concurrent suites.

Refs #17828
2026-09-01 19:06:44 -07:00

427 lines
15 KiB
TypeScript

import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
// Why: these tests cover the §3.3 Lifecycle rules on
// `OrcaRuntimeService.fetchRemoteWithCache` — in particular that a rejected
// fetch evicts its Map entry AND does not advance the freshness timestamp,
// and that two concurrent callers serialize on a single underlying fetch.
// They live in a dedicated file so we can mock `gitExecFileAsync` cleanly
// without disturbing the large orca-runtime.test.ts mock surface.
const gitExecFileAsyncMock = vi.hoisted(() => vi.fn())
vi.mock('../git/runner', async (importOriginal) => {
const actual = (await importOriginal()) as Record<string, unknown>
return {
...actual,
gitExecFileAsync: gitExecFileAsyncMock
}
})
// Why: orca-runtime.ts imports heavy modules (hooks, ipc/*, etc.) at top
// level. We only exercise the fetch cache, so we let those imports load
// normally — none of them trigger IO until a runtime method is called.
import { OrcaRuntimeService } from './orca-runtime'
function isFetchArgs(argv: unknown): argv is string[] {
if (!Array.isArray(argv)) {
return false
}
let commandIndex = 0
while (argv[commandIndex] === '-c' && typeof argv[commandIndex + 1] === 'string') {
commandIndex += 2
}
return argv[commandIndex] === 'fetch'
}
function fetchCallCount(): number {
return gitExecFileAsyncMock.mock.calls.filter(([argv]) => isFetchArgs(argv)).length
}
function exactBaseRefreshOptions(cwd: string): {
cwd: string
timeout: number
useConfiguredSshCommandForNetwork: boolean
} {
return { cwd, timeout: 60_000, useConfiguredSshCommandForNetwork: true }
}
function exactBaseRefreshArgs(branch = 'main'): string[] {
return [
'-c',
'maintenance.auto=false',
'-c',
'maintenance.commit-graph.auto=0',
'-c',
'gc.auto=0',
'fetch',
'--no-tags',
'origin',
`+refs/heads/${branch}:refs/remotes/origin/${branch}`
]
}
// Why (STA-1292): the broad create-path fetch must carry a timeout so a Windows
// credential-manager GUI hang can't wedge worktree creation forever.
function fullRemoteFetchOptions(cwd: string): { cwd: string; timeout: number } {
return { cwd, timeout: 60_000 }
}
function mockFetchResults(results: unknown[]): void {
let fetchIndex = 0
gitExecFileAsyncMock.mockImplementation((argv: string[]) => {
if (argv[0] === 'rev-parse') {
return Promise.reject(new Error('not a repo in cache-key test'))
}
const result = results[fetchIndex++]
return result instanceof Promise ? result : Promise.resolve(result)
})
}
describe('OrcaRuntimeService.fetchRemoteWithCache', () => {
beforeEach(() => {
gitExecFileAsyncMock.mockReset()
})
afterEach(() => {
vi.useRealTimers()
})
it('evicts the in-flight Map entry on rejection so the next caller re-fetches', async () => {
// First call rejects, second call resolves. Without §3.3 Lifecycle
// `.finally()` eviction, the second caller would await the rejected
// promise forever (or throw the same error) — the regression pattern
// described in §3.3.
mockFetchResults([Promise.reject(new Error('network down')), { stdout: '', stderr: '' }])
const runtime = new OrcaRuntimeService(null)
await runtime.fetchRemoteWithCache('/repo/a', 'origin')
await runtime.fetchRemoteWithCache('/repo/a', 'origin')
expect(fetchCallCount()).toBe(2)
})
it('does not advance the freshness timestamp when the fetch rejects', async () => {
// A rejected fetch that wrote the timestamp would make the 30s freshness
// cache "lie" — the next caller would skip the fetch on a repo whose
// last real sync is unknown. §3.3 mandates success-only writes.
mockFetchResults([Promise.reject(new Error('boom')), { stdout: '', stderr: '' }])
const runtime = new OrcaRuntimeService(null)
await runtime.fetchRemoteWithCache('/repo/b', 'origin')
// Immediately call again — if the freshness window were armed we would
// short-circuit and skip the fetch. It must still dispatch a real fetch.
await runtime.fetchRemoteWithCache('/repo/b', 'origin')
expect(fetchCallCount()).toBe(2)
})
it('serializes two concurrent callers onto a single git fetch', async () => {
// Two callers hitting the same repo+remote at the same time must share
// one underlying fetch. Without the in-flight Map they would each
// dispatch an independent `git fetch`, tripling the network load in the
// worst case (renderer create + dispatch probe + CLI create).
let resolveFetch!: () => void
const pending = new Promise<{ stdout: string; stderr: string }>((resolve) => {
resolveFetch = () => resolve({ stdout: '', stderr: '' })
})
mockFetchResults([pending])
const runtime = new OrcaRuntimeService(null)
const first = runtime.fetchRemoteWithCache('/repo/c', 'origin')
const second = runtime.fetchRemoteWithCache('/repo/c', 'origin')
// Allow both callers to register before we resolve. Each canonicalizes the
// repo key first, so the dispatch lands several microtasks in.
for (let tick = 0; tick < 8; tick += 1) {
await Promise.resolve()
}
expect(fetchCallCount()).toBe(1)
resolveFetch()
await Promise.all([first, second])
expect(fetchCallCount()).toBe(1)
})
it('skips the fetch inside the 30s freshness window after a successful fetch', async () => {
mockFetchResults([{ stdout: '', stderr: '' }])
const runtime = new OrcaRuntimeService(null)
await runtime.fetchRemoteWithCache('/repo/d', 'origin')
await runtime.fetchRemoteWithCache('/repo/d', 'origin')
// Second call must short-circuit on the freshness window (no new exec).
expect(fetchCallCount()).toBe(1)
})
it('bounds process-lifetime fetch cache maps for churned repo paths', async () => {
mockFetchResults(Array.from({ length: 520 }, () => ({ stdout: '', stderr: '' })))
const runtime = new OrcaRuntimeService(null)
const caches = runtime as unknown as {
canonicalFetchKeyCache: Map<string, string>
fetchLastCompletedAt: Map<string, number>
}
for (let i = 0; i < 520; i += 1) {
await runtime.fetchRemoteWithCache(`/repo/cache-${i}`, 'origin')
}
expect(caches.canonicalFetchKeyCache.size).toBeLessThanOrEqual(512)
expect(caches.fetchLastCompletedAt.size).toBeLessThanOrEqual(512)
expect(caches.canonicalFetchKeyCache.has('/repo/cache-0::origin')).toBe(false)
expect(caches.fetchLastCompletedAt.has('/repo/cache-0::origin')).toBe(false)
})
it('resolves remote-tracking bases with longest configured remote matching', async () => {
gitExecFileAsyncMock.mockResolvedValue({ stdout: 'foo\nfoo/bar\norigin\n', stderr: '' })
const runtime = new OrcaRuntimeService(null)
await expect(runtime.resolveRemoteTrackingBase('/repo/e', 'foo/bar/main')).resolves.toEqual({
remote: 'foo/bar',
branch: 'main',
ref: 'refs/remotes/foo/bar/main',
base: 'foo/bar/main'
})
})
it('resolves full remote-tracking refs with longest configured remote matching', async () => {
gitExecFileAsyncMock.mockResolvedValue({ stdout: 'foo\nfoo/bar\norigin\n', stderr: '' })
const runtime = new OrcaRuntimeService(null)
await expect(
runtime.resolveRemoteTrackingBase('/repo/e', 'refs/remotes/foo/bar/main')
).resolves.toEqual({
remote: 'foo/bar',
branch: 'main',
ref: 'refs/remotes/foo/bar/main',
base: 'foo/bar/main'
})
})
it('refreshes a remote-tracking base with an exact no-tags refspec', async () => {
mockFetchResults([{ stdout: '', stderr: '' }])
const runtime = new OrcaRuntimeService(null)
await runtime.getOrStartRemoteTrackingBaseRefresh('/repo/f', {
remote: 'origin',
branch: 'main',
ref: 'refs/remotes/origin/main',
base: 'origin/main'
})
expect(gitExecFileAsyncMock).toHaveBeenCalledWith(
exactBaseRefreshArgs(),
exactBaseRefreshOptions('/repo/f')
)
})
it('keeps automatic maintenance enabled for ordinary full remote fetches', async () => {
mockFetchResults([{ stdout: '', stderr: '' }])
const runtime = new OrcaRuntimeService(null)
await runtime.getOrStartRemoteFetch('/repo/full-maintenance', 'origin')
expect(gitExecFileAsyncMock).toHaveBeenCalledWith(
['fetch', 'origin'],
fullRemoteFetchOptions('/repo/full-maintenance')
)
})
it('shares an in-flight remote-tracking base refresh and reuses exact-base freshness', async () => {
let resolveFetch!: () => void
const pending = new Promise<{ stdout: string; stderr: string }>((resolve) => {
resolveFetch = () => resolve({ stdout: '', stderr: '' })
})
mockFetchResults([pending, { stdout: '', stderr: '' }])
const runtime = new OrcaRuntimeService(null)
const base = {
remote: 'origin',
branch: 'main',
ref: 'refs/remotes/origin/main',
base: 'origin/main'
}
const first = runtime.getOrStartRemoteTrackingBaseRefresh('/repo/g', base)
const second = runtime.getOrStartRemoteTrackingBaseRefresh('/repo/g', base)
await new Promise((resolve) => setTimeout(resolve, 0))
expect(fetchCallCount()).toBe(1)
resolveFetch()
await Promise.all([first, second])
await runtime.getOrStartRemoteTrackingBaseRefresh('/repo/g', base)
expect(fetchCallCount()).toBe(1)
})
it('does not advance exact-base freshness when a remote-tracking refresh fails', async () => {
mockFetchResults([Promise.reject(new Error('network down')), { stdout: '', stderr: '' }])
const runtime = new OrcaRuntimeService(null)
const base = {
remote: 'origin',
branch: 'main',
ref: 'refs/remotes/origin/main',
base: 'origin/main'
}
await expect(
runtime.getOrStartRemoteTrackingBaseRefresh('/repo/g-fail', base)
).resolves.toEqual({
ok: false,
errorKind: 'git_error'
})
await expect(
runtime.getOrStartRemoteTrackingBaseRefresh('/repo/g-fail', base)
).resolves.toEqual({ ok: true })
expect(fetchCallCount()).toBe(2)
})
it('does not treat a recent full remote fetch as exact-base freshness', async () => {
mockFetchResults([
{ stdout: '', stderr: '' },
{ stdout: '', stderr: '' }
])
const runtime = new OrcaRuntimeService(null)
const base = {
remote: 'origin',
branch: 'main',
ref: 'refs/remotes/origin/main',
base: 'origin/main'
}
await runtime.getOrStartRemoteFetch('/repo/g-full', 'origin')
await expect(
runtime.getOrStartRemoteTrackingBaseRefresh('/repo/g-full', base)
).resolves.toEqual({ ok: true })
expect(fetchCallCount()).toBe(2)
})
it('queues a full remote fetch behind an in-flight remote-tracking base refresh', async () => {
let resolveBaseFetch!: () => void
let resolveFullFetch!: () => void
const pendingBaseFetch = new Promise<{ stdout: string; stderr: string }>((resolve) => {
resolveBaseFetch = () => resolve({ stdout: '', stderr: '' })
})
const pendingFullFetch = new Promise<{ stdout: string; stderr: string }>((resolve) => {
resolveFullFetch = () => resolve({ stdout: '', stderr: '' })
})
mockFetchResults([pendingBaseFetch, pendingFullFetch])
const runtime = new OrcaRuntimeService(null)
const base = {
remote: 'origin',
branch: 'main',
ref: 'refs/remotes/origin/main',
base: 'origin/main'
}
const baseRefresh = runtime.getOrStartRemoteTrackingBaseRefresh('/repo/h', base)
await new Promise((resolve) => setTimeout(resolve, 0))
expect(fetchCallCount()).toBe(1)
const fullFetch = runtime.getOrStartRemoteFetch('/repo/h', 'origin')
await new Promise((resolve) => setTimeout(resolve, 0))
expect(fetchCallCount()).toBe(1)
resolveBaseFetch()
await vi.waitFor(() => expect(fetchCallCount()).toBe(2))
resolveFullFetch()
await expect(Promise.all([baseRefresh, fullFetch])).resolves.toEqual([
{ ok: true },
{ ok: true }
])
const fetchCalls = gitExecFileAsyncMock.mock.calls.filter(([argv]) => isFetchArgs(argv))
expect(fetchCalls).toEqual([
[exactBaseRefreshArgs(), exactBaseRefreshOptions('/repo/h')],
[['fetch', 'origin'], fullRemoteFetchOptions('/repo/h')]
])
})
it('runs a queued exact base refresh after an in-flight full remote fetch succeeds', async () => {
let resolveFullFetch!: () => void
let resolveBaseFetch!: () => void
const pendingFullFetch = new Promise<{ stdout: string; stderr: string }>((resolve) => {
resolveFullFetch = () => resolve({ stdout: '', stderr: '' })
})
const pendingBaseFetch = new Promise<{ stdout: string; stderr: string }>((resolve) => {
resolveBaseFetch = () => resolve({ stdout: '', stderr: '' })
})
mockFetchResults([pendingFullFetch, pendingBaseFetch])
const runtime = new OrcaRuntimeService(null)
const base = {
remote: 'origin',
branch: 'main',
ref: 'refs/remotes/origin/main',
base: 'origin/main'
}
const fullFetch = runtime.getOrStartRemoteFetch('/repo/i', 'origin')
await new Promise((resolve) => setTimeout(resolve, 0))
expect(fetchCallCount()).toBe(1)
const baseRefresh = runtime.getOrStartRemoteTrackingBaseRefresh('/repo/i', base)
await new Promise((resolve) => setTimeout(resolve, 0))
expect(fetchCallCount()).toBe(1)
resolveFullFetch()
await vi.waitFor(() => expect(fetchCallCount()).toBe(2))
resolveBaseFetch()
await expect(Promise.all([fullFetch, baseRefresh])).resolves.toEqual([
{ ok: true },
{ ok: true }
])
const fetchCalls = gitExecFileAsyncMock.mock.calls.filter(([argv]) => isFetchArgs(argv))
expect(fetchCalls).toEqual([
[['fetch', 'origin'], fullRemoteFetchOptions('/repo/i')],
[exactBaseRefreshArgs(), exactBaseRefreshOptions('/repo/i')]
])
})
it('runs a queued exact base refresh when an in-flight full remote fetch fails', async () => {
let rejectFullFetch!: () => void
let resolveBaseFetch!: () => void
const pendingFullFetch = new Promise<{ stdout: string; stderr: string }>((_resolve, reject) => {
rejectFullFetch = () => reject(new Error('network unavailable'))
})
const pendingBaseFetch = new Promise<{ stdout: string; stderr: string }>((resolve) => {
resolveBaseFetch = () => resolve({ stdout: '', stderr: '' })
})
mockFetchResults([pendingFullFetch, pendingBaseFetch])
const runtime = new OrcaRuntimeService(null)
const base = {
remote: 'origin',
branch: 'main',
ref: 'refs/remotes/origin/main',
base: 'origin/main'
}
const fullFetch = runtime.getOrStartRemoteFetch('/repo/i-fail', 'origin')
await new Promise((resolve) => setTimeout(resolve, 0))
expect(fetchCallCount()).toBe(1)
const baseRefresh = runtime.getOrStartRemoteTrackingBaseRefresh('/repo/i-fail', base)
await new Promise((resolve) => setTimeout(resolve, 0))
expect(fetchCallCount()).toBe(1)
rejectFullFetch()
await vi.waitFor(() => expect(fetchCallCount()).toBe(2))
resolveBaseFetch()
await expect(Promise.all([fullFetch, baseRefresh])).resolves.toEqual([
{ ok: false, errorKind: 'git_error' },
{ ok: true }
])
const fetchCalls = gitExecFileAsyncMock.mock.calls.filter(([argv]) => isFetchArgs(argv))
expect(fetchCalls).toEqual([
[['fetch', 'origin'], fullRemoteFetchOptions('/repo/i-fail')],
[exactBaseRefreshArgs(), exactBaseRefreshOptions('/repo/i-fail')]
])
})
})