mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 08:02:28 +00:00
perf(git): pack the loose refs Orca's own fetches leave behind (#17857)
* perf(git): pack the loose refs Orca's own fetches leave behind Orca strips git's auto-maintenance off every fetch it issues (GIT_FETCH_SKIP_AUTO_MAINTENANCE_CONFIG_ARGS) and never compensated, so nothing in an Orca-driven checkout ever packs refs. One real machine reached 36,574 loose refs, where `git show-ref -- main` costs 5.2s and every worktree create pays for it. Add an idle-time, per-repo `git pack-refs --all --prune`, armed by the fetches that create the debt. It runs only after ten minutes of quiet on that repo, only above 1000 loose refs (probed with a walk bounded by that threshold, not by the backlog), one at a time across the whole app, at the background admission tier, and never while an agent is working, a create is prepared or in flight, a worktree removal is deleting refs, the app is quitting, or the machine is on battery. A user who set `maintenance.auto=false` or `gc.auto=0` has opted out. Measured on a 36,001-loose-ref fixture (macOS/APFS, git 2.44): `show-ref` 5.5-12.2s -> 30-49ms, `for-each-ref` 4.0-10.8s -> 43-48ms. Also fixes a pre-existing bug the split exposed: `--path-format=absolute` is ignored before git 2.31, and taking rev-parse's stdout raw collapsed every repo on such a host onto one fetch-serialization key. Refs #17828 * perf(git): make idle ref maintenance preemptible and cheaper to probe The idle veto was one-directional: it stopped a pack from starting during a create, removal, or agent work, but nothing stopped those from starting during a pack. A user-clicked Fetch, a branch delete, or a worktree removal that needed `packed-refs.lock` mid-rewrite could fail with `unable to create packed-refs.lock` -- a git error with no visible cause. Make the pack cancellable end to end. An AbortSignal now reaches the `pack-refs` child and both pre-pack probes, and `pause()` aborts what is running, waits for it to actually stop, and holds a suspension count so nothing new starts until the caller releases. Every entry point that deletes a ref takes that pause: gitFetch, gitPull, gitFastForward, removeWorktree, forceDeleteLocalBranch, prepareWorktreeCreateCheckout, addWorktree. Five more triggers close the rest of the window: battery drop, window focus, quit, the attempt deadline, and any other git command queueing for an admission slot. Judge a pack by re-probing the backlog rather than by the child's exit code. Measured in the field: another Orca session moved a branch mid-pack, git reported `cannot lock ref`, skipped that ref and packed the rest -- 36,688 loose refs down to 3. On a machine running several sessions that is the normal case, and retrying it would be wrong. Probe with one batched `readdir` per directory instead of streaming `opendir`, which issues a thread-pool round trip every 32 entries: 177ms -> 23ms on a real 36,600-ref repository, with half the event-loop lag. The walk stays strictly sequential so it can never occupy more than one of libuv's four filesystem threads. `PackRefsLockOwnership` makes a lock left by SIGKILL attributable, and only reclaims one when a marker exists, the lock is older than any pack-refs could run for, and the recorded process is gone. Refs #17828 * fix(git): wait out the packed-refs lock instead of killing the pack Measured on Git 2.55/APFS with 37k loose refs: a full `pack-refs --all --prune` takes 23-32s but holds `packed-refs.lock` for only 0.03-1.37s of it. The other ~95% is the prune phase, during which a concurrent `fetch --prune`, `branch -D` or `update-ref` succeeds every time -- per-ref locks last microseconds and git retries for `core.filesRefLockTimeout`. So the abort-on-everything design was strictly harmful. SIGTERM into the prune loop strands an empty `refs/**/*.lock` about one time in five (9/30, 5/40, 6/30 kills): `tempfile.c` opens the lock O_EXCL before `activate_tempfile()` links it into the list the signal handler walks, and a pack does ~36k lock cycles. Afterwards `update-ref -d` on that ref fails with `cannot lock ref ... File exists`, permanently. On Windows `taskkill /f` never runs git's handlers at all, so an abort inside the rewrite strands `packed-refs.lock` every time. Never signal the child. `packRefs` no longer takes an abort signal; it polls `packed-refs.lock` and reports the window through a `PackedRefsLockReporter`. `pause()` resolves when the lock is released -- bounded, and free during the prune -- while the suspension counter still blocks new attempts. Battery and window-focus become do-not-start rather than stop-what-is-running, and quit waits for the lock and lets the child finish orphaned. For strands that already exist, `PackRefsLockOwnership` now also reclaims `refs/**/*.lock` under the same three conditions plus a 0-byte check, and a lock carrying our own not-yet-reclaimable marker records `locked` with a 30min retry instead of the 6h failure cooldown -- so a Windows strand self-heals in half an hour rather than six. Reverts the git admission-scheduler event bus, which existed only to drive the abort this removes. Refs #17828 * test(git): make the ref-maintenance waits survive a loaded runner CI shard 4/8 failed on `restarts every armed countdown when the user does ref work themselves`, which passes locally. The `until()` helper spun a fixed 200 event-loop turns and then returned silently, so on a contended runner the filesystem probe had not finished and the assertion that followed failed with an unrelated message. Bound the wait by wall clock instead and throw a named error, which immediately exposed a second latent bug: the single-flight test's second wait could never succeed, because the deferred repo's retry is on a faked `setTimeout` that spinning the real loop never advances. It had been passing only because the old helper gave up quietly. Add a timer-aware variant for those, and have the countdown test await a signal the fake pack resolves rather than polling at all. Verified stable across five sequential runs and once under load average 32 with six concurrent suites. Refs #17828
This commit is contained in:
@@ -117,6 +117,11 @@ export class AgentAwakeService {
|
||||
}
|
||||
}
|
||||
|
||||
/** Agents this runtime has seen working recently, independent of the awake setting. */
|
||||
getWorkingAgentCount(): number {
|
||||
return this.getEligibleRunningStatusCount()
|
||||
}
|
||||
|
||||
subscribe(listener: (status: ComputerAwakeStatus) => void): () => void {
|
||||
this.statusListeners.add(listener)
|
||||
return () => this.statusListeners.delete(listener)
|
||||
|
||||
@@ -35,7 +35,9 @@ export class CliCommandInstallation extends CliCommandInspection {
|
||||
|
||||
const inspected = await this.inspectStableSymlink(commandPath, launcherPath)
|
||||
if (inspected.status.state === 'conflict') {
|
||||
throw new Error(`Refusing to replace non-Orca command at ${commandPath}. Remove it and register again if it is no longer needed.`)
|
||||
throw new Error(
|
||||
`Refusing to replace non-Orca command at ${commandPath}. Remove it and register again if it is no longer needed.`
|
||||
)
|
||||
}
|
||||
if (inspected.status.state === 'installed') {
|
||||
return
|
||||
@@ -54,7 +56,9 @@ export class CliCommandInstallation extends CliCommandInspection {
|
||||
|
||||
if (!(await capturedExpectedEntry(quarantine, inspected))) {
|
||||
await this.restoreQuarantinedCommand(quarantine, commandPath)
|
||||
throw new Error(`Refusing to replace non-Orca command at ${commandPath}. Remove it and register again if it is no longer needed.`)
|
||||
throw new Error(
|
||||
`Refusing to replace non-Orca command at ${commandPath}. Remove it and register again if it is no longer needed.`
|
||||
)
|
||||
}
|
||||
|
||||
try {
|
||||
|
||||
@@ -116,7 +116,9 @@ export class CliInstaller extends CliPathRegistration {
|
||||
throw new Error(initialStatus.detail ?? 'CLI registration is unavailable on this build.')
|
||||
}
|
||||
if (initialStatus.state === 'conflict') {
|
||||
throw new Error(`Refusing to replace non-Orca command at ${initialStatus.commandPath}. Remove it and register again if it is no longer needed.`)
|
||||
throw new Error(
|
||||
`Refusing to replace non-Orca command at ${initialStatus.commandPath}. Remove it and register again if it is no longer needed.`
|
||||
)
|
||||
}
|
||||
const extractedRoot = await this.ensureLinuxAppImagePayload()
|
||||
const status = extractedRoot
|
||||
@@ -126,7 +128,9 @@ export class CliInstaller extends CliPathRegistration {
|
||||
throw new Error(status.detail ?? 'CLI registration is unavailable on this build.')
|
||||
}
|
||||
if (status.state === 'conflict') {
|
||||
throw new Error(`Refusing to replace non-Orca command at ${status.commandPath}. Remove it and register again if it is no longer needed.`)
|
||||
throw new Error(
|
||||
`Refusing to replace non-Orca command at ${status.commandPath}. Remove it and register again if it is no longer needed.`
|
||||
)
|
||||
}
|
||||
|
||||
// eslint-disable-next-line unicorn/prefer-ternary -- Why: the install path performs async side effects and is easier to audit as an explicit branch than as an awaited ternary.
|
||||
|
||||
@@ -207,7 +207,9 @@ export class WslCliInstaller {
|
||||
throw new Error(status.detail ?? 'WSL CLI registration is unavailable.')
|
||||
}
|
||||
if (status.state === 'conflict') {
|
||||
throw new Error(`Refusing to replace non-Orca command at ${status.commandPath}. Remove it and register again if it is no longer needed.`)
|
||||
throw new Error(
|
||||
`Refusing to replace non-Orca command at ${status.commandPath}. Remove it and register again if it is no longer needed.`
|
||||
)
|
||||
}
|
||||
|
||||
await this.run(
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const gitExecFileAsyncMock = vi.hoisted(() => vi.fn())
|
||||
|
||||
vi.mock('./runner', async (importOriginal) => ({
|
||||
...((await importOriginal()) as Record<string, unknown>),
|
||||
gitExecFileAsync: gitExecFileAsyncMock
|
||||
}))
|
||||
|
||||
import {
|
||||
_resetCanonicalRepoKeyCacheForTests,
|
||||
getCanonicalRepoKey,
|
||||
readGitCommonDir
|
||||
} from './canonical-repo-key'
|
||||
|
||||
beforeEach(() => {
|
||||
_resetCanonicalRepoKeyCacheForTests()
|
||||
gitExecFileAsyncMock.mockReset()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
describe('readGitCommonDir', () => {
|
||||
it('reads the absolute answer modern Git gives', () => {
|
||||
expect(readGitCommonDir('/repo/.git\n', '/repo/worktrees/a')).toBe('/repo/.git')
|
||||
})
|
||||
|
||||
it('drops the flag Git older than 2.31 echoes back, and resolves the relative answer', () => {
|
||||
// Without this every repository on such a host would answer `.git` and collide.
|
||||
expect(readGitCommonDir('--path-format=absolute\n.git\n', '/repo')).toBe('/repo/.git')
|
||||
})
|
||||
|
||||
it('resolves a WSL answer in Git execution space, not against the UNC path', () => {
|
||||
expect(readGitCommonDir('.git\n', '//wsl$/Ubuntu/home/dev/repo')).toBe('/home/dev/repo/.git')
|
||||
})
|
||||
|
||||
it('tolerates CRLF and blank lines', () => {
|
||||
expect(readGitCommonDir('\r\n/repo/.git\r\n', '/repo')).toBe('/repo/.git')
|
||||
})
|
||||
|
||||
it('returns undefined when Git printed nothing usable', () => {
|
||||
expect(readGitCommonDir('\n', '/repo')).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
describe('getCanonicalRepoKey', () => {
|
||||
it('gives every worktree of one repository the same key', async () => {
|
||||
gitExecFileAsyncMock.mockResolvedValue({ stdout: '/repo/.git\n', stderr: '' })
|
||||
|
||||
await expect(getCanonicalRepoKey('/repo')).resolves.toBe('local::/repo/.git')
|
||||
await expect(getCanonicalRepoKey('/repo/worktrees/a')).resolves.toBe('local::/repo/.git')
|
||||
})
|
||||
|
||||
it('scopes the key to the execution host', async () => {
|
||||
gitExecFileAsyncMock.mockResolvedValue({ stdout: '/home/dev/repo/.git\n', stderr: '' })
|
||||
|
||||
await expect(
|
||||
getCanonicalRepoKey('//wsl$/Ubuntu/home/dev/repo', { wslDistro: 'Ubuntu' })
|
||||
).resolves.toBe('wsl:Ubuntu::/home/dev/repo/.git')
|
||||
})
|
||||
|
||||
it('caches so repeated arming costs no subprocess', async () => {
|
||||
gitExecFileAsyncMock.mockResolvedValue({ stdout: '/repo/.git\n', stderr: '' })
|
||||
|
||||
await getCanonicalRepoKey('/repo')
|
||||
await getCanonicalRepoKey('/repo')
|
||||
|
||||
expect(gitExecFileAsyncMock).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('falls back to the caller path when Git cannot answer', async () => {
|
||||
gitExecFileAsyncMock.mockRejectedValue(new Error('not a git repository'))
|
||||
|
||||
await expect(getCanonicalRepoKey('/not-a-repo')).resolves.toBe('local::/not-a-repo')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,72 @@
|
||||
import { toWslExecutionSpace } from '../../shared/wsl-paths'
|
||||
import { gitExecFileAsync } from './runner'
|
||||
import { resolveRevParsePath } from './worktree-path-comparison'
|
||||
|
||||
/**
|
||||
* One repository on one execution host, named by its Git common dir.
|
||||
*
|
||||
* Shared by the fetch controller (which serializes fetches on it) and idle ref
|
||||
* maintenance (which scopes all of its state to it), so both agree on what "the
|
||||
* same repo" means across every worktree that points at it.
|
||||
*/
|
||||
|
||||
export type CanonicalRepoKeyOptions = { wslDistro?: string }
|
||||
|
||||
const CACHE_MAX = 512
|
||||
const cache = new Map<string, string>()
|
||||
|
||||
/**
|
||||
* Git < 2.31 ignores `--path-format=absolute`: it echoes the unrecognized flag,
|
||||
* exits 0, and prints a relative `.git`. Taking the raw stdout there would give
|
||||
* every repository on the host the same key.
|
||||
*/
|
||||
export function readGitCommonDir(stdout: string, repoPath: string): string | undefined {
|
||||
const commonDir = stdout
|
||||
.split('\n')
|
||||
.map((line) => (line.endsWith('\r') ? line.slice(0, -1) : line))
|
||||
.findLast((line) => line.length > 0 && !line.startsWith('-'))
|
||||
return commonDir ? resolveRevParsePath(toWslExecutionSpace(repoPath), commonDir) : undefined
|
||||
}
|
||||
|
||||
function remember(cacheKey: string, value: string): string {
|
||||
cache.delete(cacheKey)
|
||||
cache.set(cacheKey, value)
|
||||
while (cache.size > CACHE_MAX) {
|
||||
const oldest = cache.keys().next()
|
||||
if (oldest.done) {
|
||||
break
|
||||
}
|
||||
cache.delete(oldest.value)
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/** `${runtimeKey}::${gitCommonDir}`, falling back to the caller's path. */
|
||||
export async function getCanonicalRepoKey(
|
||||
repoPath: string,
|
||||
options: CanonicalRepoKeyOptions = {}
|
||||
): Promise<string> {
|
||||
const runtimeKey = options.wslDistro ? `wsl:${options.wslDistro}` : 'local'
|
||||
const cacheKey = `${runtimeKey}::${repoPath}`
|
||||
const cached = cache.get(cacheKey)
|
||||
if (cached !== undefined) {
|
||||
return remember(cacheKey, cached)
|
||||
}
|
||||
try {
|
||||
const { stdout } = await gitExecFileAsync(
|
||||
['rev-parse', '--path-format=absolute', '--git-common-dir'],
|
||||
{ cwd: repoPath, ...options }
|
||||
)
|
||||
const commonDir = readGitCommonDir(stdout, repoPath)
|
||||
if (commonDir) {
|
||||
return remember(cacheKey, `${runtimeKey}::${commonDir}`)
|
||||
}
|
||||
} catch {
|
||||
// The caller path remains a safe serialization key when canonicalization fails.
|
||||
}
|
||||
return remember(cacheKey, cacheKey)
|
||||
}
|
||||
|
||||
export function _resetCanonicalRepoKeyCacheForTests(): void {
|
||||
cache.clear()
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const gitExecFileAsyncMock = vi.hoisted(() => vi.fn())
|
||||
const readRepoCommonDirFromGitMock = vi.hoisted(() => vi.fn())
|
||||
|
||||
vi.mock('./runner', async (importOriginal) => ({
|
||||
...((await importOriginal()) as Record<string, unknown>),
|
||||
gitExecFileAsync: gitExecFileAsyncMock
|
||||
}))
|
||||
|
||||
vi.mock('./worktree-list-reader', async (importOriginal) => ({
|
||||
...((await importOriginal()) as Record<string, unknown>),
|
||||
readRepoCommonDirFromGit: readRepoCommonDirFromGitMock
|
||||
}))
|
||||
|
||||
import { _resetCanonicalRepoKeyCacheForTests } from './canonical-repo-key'
|
||||
import {
|
||||
_resetLocalRepoRefMaintenanceForTests,
|
||||
armLocalRepoRefMaintenance,
|
||||
createLocalRepoRefMaintenanceTarget,
|
||||
getLocalRepoRefMaintenance,
|
||||
setRepoMaintenanceActivityProbe,
|
||||
withRepoRefMaintenancePaused
|
||||
} from './local-repo-ref-maintenance'
|
||||
|
||||
const NO_ABORT = new AbortController().signal
|
||||
|
||||
function target(wslDistro?: string): ReturnType<typeof createLocalRepoRefMaintenanceTarget> {
|
||||
return createLocalRepoRefMaintenanceTarget({
|
||||
key: 'local::/repo/.git',
|
||||
repoPath: wslDistro ? '//wsl$/Ubuntu/home/dev/repo' : '/repo',
|
||||
...(wslDistro ? { wslDistro } : {})
|
||||
})
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
gitExecFileAsyncMock.mockReset()
|
||||
readRepoCommonDirFromGitMock.mockReset()
|
||||
delete process.env.ORCA_DISABLE_REPO_REF_MAINTENANCE
|
||||
_resetCanonicalRepoKeyCacheForTests()
|
||||
_resetLocalRepoRefMaintenanceForTests()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
delete process.env.ORCA_DISABLE_REPO_REF_MAINTENANCE
|
||||
_resetLocalRepoRefMaintenanceForTests()
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
describe('local repo ref maintenance target', () => {
|
||||
it('never hands the pack child an abort signal', async () => {
|
||||
// Killing a `pack-refs` strands a `refs/**` lock about one time in five, and
|
||||
// on Windows a force-kill inside the rewrite strands `packed-refs.lock`
|
||||
// every time. The child must always be allowed to finish.
|
||||
readRepoCommonDirFromGitMock.mockResolvedValue('/repo/.git')
|
||||
gitExecFileAsyncMock.mockResolvedValue({ stdout: '', stderr: '' })
|
||||
|
||||
await target().packRefs({ setHeld: () => {} })
|
||||
|
||||
const packCall = gitExecFileAsyncMock.mock.calls.find(
|
||||
([argv]) => (argv as string[])[0] === 'pack-refs'
|
||||
)
|
||||
expect(packCall?.[1]).not.toHaveProperty('signal')
|
||||
})
|
||||
|
||||
it('runs pack-refs at the background tier with a long deadline', async () => {
|
||||
gitExecFileAsyncMock.mockResolvedValue({ stdout: '', stderr: '' })
|
||||
|
||||
await target().packRefs({ setHeld: () => {} })
|
||||
|
||||
expect(gitExecFileAsyncMock).toHaveBeenCalledWith(
|
||||
['pack-refs', '--all', '--prune'],
|
||||
expect.objectContaining({ cwd: '/repo', admissionTier: 'background', timeout: 15 * 60_000 })
|
||||
)
|
||||
})
|
||||
|
||||
it('reads either Git auto-maintenance opt-out, and unset keys as consent', async () => {
|
||||
for (const stdout of [
|
||||
'maintenance.auto false\n',
|
||||
'gc.auto 0\n',
|
||||
'gc.auto 6700\nmaintenance.auto false\n'
|
||||
]) {
|
||||
gitExecFileAsyncMock.mockResolvedValue({ stdout, stderr: '' })
|
||||
await expect(target().isOptedOut?.(NO_ABORT)).resolves.toBe(true)
|
||||
}
|
||||
|
||||
gitExecFileAsyncMock.mockResolvedValue({
|
||||
stdout: 'maintenance.auto true\ngc.auto 6700\n',
|
||||
stderr: ''
|
||||
})
|
||||
await expect(target().isOptedOut?.(NO_ABORT)).resolves.toBe(false)
|
||||
|
||||
// `git config --get-regexp` exits non-zero when nothing matches.
|
||||
gitExecFileAsyncMock.mockRejectedValue(new Error('exit 1'))
|
||||
await expect(target().isOptedOut?.(NO_ABORT)).resolves.toBe(false)
|
||||
})
|
||||
|
||||
it('walks the POSIX refs directory for a native repo', async () => {
|
||||
readRepoCommonDirFromGitMock.mockResolvedValue('/repo/.git')
|
||||
|
||||
await expect(target().resolveRefsDirectory(NO_ABORT)).resolves.toBe('/repo/.git/refs')
|
||||
})
|
||||
|
||||
it('translates a WSL repo answer back to the UNC path the main process can open', async () => {
|
||||
// Git answers in its own execution space, which for WSL is a Linux path.
|
||||
readRepoCommonDirFromGitMock.mockResolvedValue('/home/dev/repo/.git')
|
||||
|
||||
await expect(target('Ubuntu').resolveRefsDirectory(NO_ABORT)).resolves.toBe(
|
||||
'\\\\wsl.localhost\\Ubuntu\\home\\dev\\repo\\.git\\refs'
|
||||
)
|
||||
})
|
||||
|
||||
it('reports an unresolvable repository rather than guessing a path', async () => {
|
||||
readRepoCommonDirFromGitMock.mockResolvedValue(undefined)
|
||||
|
||||
await expect(target().resolveRefsDirectory(NO_ABORT)).resolves.toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
describe('local repo ref maintenance scheduling', () => {
|
||||
it('schedules nothing when the kill switch is set', () => {
|
||||
process.env.ORCA_DISABLE_REPO_REF_MAINTENANCE = '1'
|
||||
const arm = vi.spyOn(getLocalRepoRefMaintenance(), 'arm')
|
||||
|
||||
armLocalRepoRefMaintenance({ key: 'local::/repo/.git', repoPath: '/repo' })
|
||||
|
||||
expect(arm).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('arms through the shared single-flight instance otherwise', () => {
|
||||
const arm = vi.spyOn(getLocalRepoRefMaintenance(), 'arm')
|
||||
|
||||
armLocalRepoRefMaintenance({ key: 'local::/repo/.git', repoPath: '/repo' })
|
||||
|
||||
expect(arm).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('is free when nothing has ever been armed', async () => {
|
||||
// The common case by far: no timers, no instance, no reason to pay anything.
|
||||
await expect(withRepoRefMaintenancePaused('git-fetch', async () => 'done')).resolves.toBe(
|
||||
'done'
|
||||
)
|
||||
})
|
||||
|
||||
it('holds the window shut for the duration of ref-touching work', async () => {
|
||||
readRepoCommonDirFromGitMock.mockResolvedValue('/repo/.git')
|
||||
_resetLocalRepoRefMaintenanceForTests({ quietPeriodMs: 1, looseRefThreshold: 0 })
|
||||
setRepoMaintenanceActivityProbe(() => false)
|
||||
const maintenance = getLocalRepoRefMaintenance()
|
||||
const packRefs = vi.fn(async () => {})
|
||||
maintenance.arm({
|
||||
key: 'local::/repo/.git',
|
||||
resolveRefsDirectory: async () => '/repo/.git/refs',
|
||||
packRefs
|
||||
})
|
||||
|
||||
await withRepoRefMaintenancePaused('branch-delete', async () => {
|
||||
await new Promise((resolve) => setTimeout(resolve, 25))
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
await vi.waitFor(() => expect(packRefs).toHaveBeenCalledTimes(1))
|
||||
})
|
||||
|
||||
it('routes the app activity probe into the shared instance', async () => {
|
||||
readRepoCommonDirFromGitMock.mockResolvedValue('/repo/.git')
|
||||
let busy = true
|
||||
setRepoMaintenanceActivityProbe(() => busy)
|
||||
const maintenance = getLocalRepoRefMaintenance()
|
||||
const packRefs = vi.fn(async () => {})
|
||||
|
||||
maintenance.arm({
|
||||
key: 'local::/repo/.git',
|
||||
resolveRefsDirectory: async () => '/repo/.git/refs',
|
||||
packRefs
|
||||
})
|
||||
await maintenance.whenAttemptSettled()
|
||||
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
busy = false
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,272 @@
|
||||
import { posix, win32 } from 'node:path'
|
||||
import { isWindowsAbsolutePathLike } from '../../shared/cross-platform-path'
|
||||
import { RepoRefMaintenance } from '../../shared/repo-ref-maintenance'
|
||||
import {
|
||||
PACK_REFS_ARGS,
|
||||
PACK_REFS_TIMEOUT_MS,
|
||||
RefMaintenanceRepoLocked,
|
||||
type PackedRefsLockReporter,
|
||||
type RepoRefMaintenanceOptions,
|
||||
type RepoRefMaintenanceTarget
|
||||
} from '../../shared/repo-ref-maintenance-policy'
|
||||
import { isWslUncPath, toWindowsWslPath } from '../../shared/wsl-paths'
|
||||
import { withSpan } from '../observability/tracer'
|
||||
import { PackRefsLockOwnership } from './pack-refs-lock-ownership'
|
||||
import { gitExecFileAsync } from './runner'
|
||||
import { readRepoCommonDirFromGit } from './worktree-list-reader'
|
||||
|
||||
/**
|
||||
* Main-process wiring for idle loose-ref packing on the local execution host
|
||||
* (native and WSL).
|
||||
*
|
||||
* SSH-hosted repos are deliberately out of scope: the execution host owns
|
||||
* anything that touches execution, so maintaining them means running host-side
|
||||
* on the relay, which today has neither admission control nor spans. Keying all
|
||||
* state by execution host is what keeps this path from reaching across.
|
||||
*/
|
||||
|
||||
export type RepoMaintenanceActivityProbe = () => boolean
|
||||
|
||||
const REPO_BUSY_PROBE_MAX = 64
|
||||
|
||||
let activityProbe: RepoMaintenanceActivityProbe | null = null
|
||||
let shared: RepoRefMaintenance | null = null
|
||||
// Why keyed here rather than captured in the target: a repo can be armed from
|
||||
// the fetch controller or from a user-initiated fetch, and every arming must see
|
||||
// the same "this repo has work in flight" answer, not whichever closure was last.
|
||||
const repoBusyProbes = new Map<string, () => boolean>()
|
||||
|
||||
/** Register the owner of "this repo has a fetch in flight" for `key`. */
|
||||
export function setRepoRefMaintenanceBusyProbe(key: string, probe: () => boolean): void {
|
||||
repoBusyProbes.delete(key)
|
||||
repoBusyProbes.set(key, probe)
|
||||
while (repoBusyProbes.size > REPO_BUSY_PROBE_MAX) {
|
||||
const oldest = repoBusyProbes.keys().next()
|
||||
if (oldest.done) {
|
||||
break
|
||||
}
|
||||
repoBusyProbes.delete(oldest.value)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the app-wide "do not start maintenance now" signal. Owned by the
|
||||
* main entry point because the inputs (live agents, battery, quit) are not
|
||||
* visible from the git layer.
|
||||
*/
|
||||
export function setRepoMaintenanceActivityProbe(probe: RepoMaintenanceActivityProbe | null): void {
|
||||
activityProbe = probe
|
||||
}
|
||||
|
||||
/** Support escape hatch: kills the sweep without touching the user's git config. */
|
||||
function isDisabled(): boolean {
|
||||
return process.env.ORCA_DISABLE_REPO_REF_MAINTENANCE === '1'
|
||||
}
|
||||
|
||||
function localMaintenanceOptions(): RepoRefMaintenanceOptions {
|
||||
return {
|
||||
// Fail closed: without the app-level gate installed we cannot see agents,
|
||||
// creates, or battery, and running blind is worse than not running.
|
||||
isBusy: () => activityProbe?.() ?? true,
|
||||
observe: (attempt) =>
|
||||
withSpan('repo.ref_maintenance', (span) => attempt(span), {
|
||||
attributes: { kind: 'git', 'repo.maintenance_host': 'local' }
|
||||
}),
|
||||
onError: (error) => {
|
||||
console.warn('[repo-ref-maintenance] attempt failed:', error)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export function getLocalRepoRefMaintenance(): RepoRefMaintenance {
|
||||
shared ??= new RepoRefMaintenance(localMaintenanceOptions())
|
||||
return shared
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancels every armed timer and waits out any `packed-refs` rewrite in progress.
|
||||
*
|
||||
* Deliberately does not kill the child. A pack orphaned by the app quitting
|
||||
* finishes on its own; a pack signalled mid-prune strands a ref lock about one
|
||||
* time in five, and on Windows a force-kill inside the rewrite strands
|
||||
* `packed-refs.lock` every time -- which blocks every later ref deletion.
|
||||
*/
|
||||
export function disposeLocalRepoRefMaintenance(): Promise<void> {
|
||||
const settling = shared?.awaitPackedRefsLockRelease() ?? Promise.resolve()
|
||||
shared?.dispose()
|
||||
shared = null
|
||||
repoBusyProbes.clear()
|
||||
return settling
|
||||
}
|
||||
|
||||
/**
|
||||
* Hold every repository open while `run` touches refs.
|
||||
*
|
||||
* A ref deletion needs `packed-refs.lock`, which a running pack holds only while
|
||||
* it rewrites the file -- 0.03-1.37s of a 23-32s run. Waiting that out turns the
|
||||
* collision into a short pause. Cancelling the pack instead would strand a
|
||||
* `refs/**` lock about one time in five, which Git never clears, so the ref
|
||||
* stays undeletable indefinitely.
|
||||
*/
|
||||
export async function withRepoRefMaintenancePaused<T>(
|
||||
reason: string,
|
||||
run: () => Promise<T>
|
||||
): Promise<T> {
|
||||
// Taken unconditionally rather than only when something is already armed: a
|
||||
// fetch inside `run` can arm the sweep, and one counter bump against an idle
|
||||
// instance costs a microtask. This can rebuild the instance after the
|
||||
// quit-time dispose; harmless, because a fresh one has no armed timers and its
|
||||
// activity probe is gone, so it fails closed.
|
||||
const release = await getLocalRepoRefMaintenance().pause(reason)
|
||||
try {
|
||||
return await run()
|
||||
} finally {
|
||||
release()
|
||||
}
|
||||
}
|
||||
|
||||
/** Wait out a `packed-refs` rewrite without holding the window open. For shutdown. */
|
||||
export function awaitPackedRefsLockRelease(): Promise<void> {
|
||||
return shared ? shared.awaitPackedRefsLockRelease() : Promise.resolve()
|
||||
}
|
||||
|
||||
/**
|
||||
* Count user-initiated ref work as activity and restart every armed countdown.
|
||||
*
|
||||
* Deliberately not keyed to a repo: resolving one would cost a `rev-parse` on a
|
||||
* path the user is waiting on, and a manual fetch or pull says the user is at
|
||||
* the keyboard, which is a reason to defer every repository.
|
||||
*/
|
||||
export function postponeRepoRefMaintenance(): void {
|
||||
shared?.postponeAll()
|
||||
}
|
||||
|
||||
/** `overrides` preseeds the shared instance so a test can shorten the quiet period. */
|
||||
export function _resetLocalRepoRefMaintenanceForTests(
|
||||
overrides?: Partial<RepoRefMaintenanceOptions>
|
||||
): void {
|
||||
shared?.dispose()
|
||||
shared = overrides ? new RepoRefMaintenance({ ...localMaintenanceOptions(), ...overrides }) : null
|
||||
activityProbe = null
|
||||
repoBusyProbes.clear()
|
||||
}
|
||||
|
||||
/**
|
||||
* Git reports the common dir in its own execution space, so a WSL repo answers
|
||||
* with a Linux path the Windows main process cannot open. Translate it back to
|
||||
* the UNC spelling for the dirent walk; the walk reads directories, not files,
|
||||
* so the handful of round trips stays cheap even over the share.
|
||||
*/
|
||||
function refsDirectoryForMainProcess(commonDir: string, wslDistro: string | undefined): string {
|
||||
if (wslDistro && !isWslUncPath(commonDir) && !isWindowsAbsolutePathLike(commonDir)) {
|
||||
return win32.join(toWindowsWslPath(commonDir, wslDistro), 'refs')
|
||||
}
|
||||
// Decided by path syntax, not by platform: `win32.isAbsolute` accepts POSIX paths too.
|
||||
return (isWindowsAbsolutePathLike(commonDir) ? win32 : posix).join(commonDir, 'refs')
|
||||
}
|
||||
|
||||
/**
|
||||
* `maintenance.auto=false` and `gc.auto=0` are the two knobs a user reaches for
|
||||
* to tell Git to stop maintaining a repository on its own. Orca sets both on its
|
||||
* own fetches, but only as per-invocation `-c` flags, so this probe sees the
|
||||
* user's persisted config and never Orca's own suppression.
|
||||
*/
|
||||
export function isGitAutoMaintenanceDisabled(configOutput: string): boolean {
|
||||
return configOutput
|
||||
.split('\n')
|
||||
.map((line) => line.trim())
|
||||
.some((line) => line === 'maintenance.auto false' || line === 'gc.auto 0')
|
||||
}
|
||||
|
||||
/**
|
||||
* The common dir in the spelling the main process can open.
|
||||
*
|
||||
* Derived from the converted refs path, not the raw one: a WSL answer arrives as
|
||||
* a Linux path but converts to a UNC path with no `/` in it, so choosing the
|
||||
* path flavour before conversion collapses the whole thing to `.`.
|
||||
*/
|
||||
function gitCommonDirForMainProcess(commonDir: string, wslDistro: string | undefined): string {
|
||||
const refs = refsDirectoryForMainProcess(commonDir, wslDistro)
|
||||
return (isWindowsAbsolutePathLike(refs) ? win32 : posix).dirname(refs)
|
||||
}
|
||||
|
||||
export type LocalRepoRefMaintenanceTargetArgs = {
|
||||
/** `${runtimeKey}::${gitCommonDir}` -- already scoped to the execution host. */
|
||||
readonly key: string
|
||||
readonly repoPath: string
|
||||
readonly wslDistro?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Record a write to this repo and restart its quiet-period countdown. The only
|
||||
* entry point callers need: the kill switch is honoured before anything is
|
||||
* scheduled, so a disabled build arms no timers at all.
|
||||
*/
|
||||
export function armLocalRepoRefMaintenance(args: LocalRepoRefMaintenanceTargetArgs): void {
|
||||
if (isDisabled()) {
|
||||
return
|
||||
}
|
||||
getLocalRepoRefMaintenance().arm(createLocalRepoRefMaintenanceTarget(args))
|
||||
}
|
||||
|
||||
export function createLocalRepoRefMaintenanceTarget(
|
||||
args: LocalRepoRefMaintenanceTargetArgs
|
||||
): RepoRefMaintenanceTarget {
|
||||
const gitOptions = args.wslDistro ? { wslDistro: args.wslDistro } : {}
|
||||
// The engine always probes before it packs, so the pack reuses this answer
|
||||
// rather than spending a second rev-parse on the same repository.
|
||||
let commonDir: string | undefined
|
||||
const resolveCommonDir = async (signal?: AbortSignal): Promise<string | undefined> => {
|
||||
commonDir ??= await readRepoCommonDirFromGit(args.repoPath, {
|
||||
...gitOptions,
|
||||
...(signal ? { signal } : {})
|
||||
})
|
||||
return commonDir
|
||||
}
|
||||
return {
|
||||
key: args.key,
|
||||
isBusy: () => repoBusyProbes.get(args.key)?.() ?? false,
|
||||
async resolveRefsDirectory(signal: AbortSignal) {
|
||||
const resolved = await resolveCommonDir(signal)
|
||||
return resolved ? refsDirectoryForMainProcess(resolved, args.wslDistro) : undefined
|
||||
},
|
||||
async isOptedOut(signal: AbortSignal) {
|
||||
try {
|
||||
const { stdout } = await gitExecFileAsync(
|
||||
['config', '--get-regexp', '^(maintenance\\.auto|gc\\.auto)$'],
|
||||
{ cwd: args.repoPath, ...gitOptions, admissionTier: 'background', signal }
|
||||
)
|
||||
return isGitAutoMaintenanceDisabled(stdout)
|
||||
} catch {
|
||||
// Neither key set is the common case and exits non-zero; that is consent.
|
||||
return false
|
||||
}
|
||||
},
|
||||
async packRefs(lock: PackedRefsLockReporter) {
|
||||
const resolved = await resolveCommonDir()
|
||||
const owner = resolved
|
||||
? new PackRefsLockOwnership(gitCommonDirForMainProcess(resolved, args.wslDistro))
|
||||
: null
|
||||
const claim = owner ? await owner.claim() : { ok: true as const }
|
||||
if (!claim.ok) {
|
||||
throw new RefMaintenanceRepoLocked(claim.reason)
|
||||
}
|
||||
// Report the rewrite window rather than accepting a signal. A pack that is
|
||||
// killed mid-prune strands a `refs/**` lock about one time in five, and
|
||||
// Git never clears those; waiting out the window costs at most ~1.4s.
|
||||
const watch = owner?.watchLock((held) => lock.setHeld(held))
|
||||
try {
|
||||
await gitExecFileAsync([...PACK_REFS_ARGS], {
|
||||
cwd: args.repoPath,
|
||||
...gitOptions,
|
||||
admissionTier: 'background',
|
||||
timeout: PACK_REFS_TIMEOUT_MS
|
||||
})
|
||||
} finally {
|
||||
watch?.stop()
|
||||
lock.setHeld(false)
|
||||
await owner?.release()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { PackRefsLockOwnership } from './pack-refs-lock-ownership'
|
||||
|
||||
const roots: string[] = []
|
||||
|
||||
async function gitCommonDir(): Promise<string> {
|
||||
const root = await mkdtemp(join(tmpdir(), 'orca-pack-refs-lock-'))
|
||||
roots.push(root)
|
||||
return root
|
||||
}
|
||||
|
||||
function paths(commonDir: string): { lock: string; marker: string } {
|
||||
return {
|
||||
lock: join(commonDir, 'packed-refs.lock'),
|
||||
marker: join(commonDir, 'packed-refs.orca-owner')
|
||||
}
|
||||
}
|
||||
|
||||
async function exists(path: string): Promise<boolean> {
|
||||
try {
|
||||
await stat(path)
|
||||
return true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/** A pid that cannot be running: the kernel rejects it outright. */
|
||||
const DEAD_PID = 0x7fffffff
|
||||
const ABANDONED_LOCK_AGE_MS = 15 * 60_000
|
||||
const PID_REUSE_HORIZON_MS = 24 * 60 * 60_000
|
||||
|
||||
/** `claim` takes `now`, so age cases need no sleeping and no mtime forgery. */
|
||||
function laterBy(ms: number): number {
|
||||
return Date.now() + ms
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true })))
|
||||
})
|
||||
|
||||
describe('packed-refs lock ownership', () => {
|
||||
it('claims a repository with no lock and records the owner', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
const { marker } = paths(commonDir)
|
||||
|
||||
await expect(new PackRefsLockOwnership(commonDir).claim()).resolves.toEqual({ ok: true })
|
||||
|
||||
await expect(readFile(marker, 'utf-8')).resolves.toContain(String(process.pid))
|
||||
})
|
||||
|
||||
it('drops the owner marker on release', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
const ownership = new PackRefsLockOwnership(commonDir)
|
||||
await ownership.claim()
|
||||
|
||||
await ownership.release()
|
||||
|
||||
await expect(exists(paths(commonDir).marker)).resolves.toBe(false)
|
||||
})
|
||||
|
||||
it('refuses a lock it cannot prove is its own', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
// A lock with no marker belongs to the user's own git, or to another tool.
|
||||
await writeFile(paths(commonDir).lock, 'someone else')
|
||||
|
||||
await expect(new PackRefsLockOwnership(commonDir).claim()).resolves.toMatchObject({ ok: false })
|
||||
await expect(exists(paths(commonDir).lock)).resolves.toBe(true)
|
||||
})
|
||||
|
||||
it('refuses a lock whose recorded owner is still running', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
const { lock, marker } = paths(commonDir)
|
||||
await writeFile(lock, 'in progress')
|
||||
await writeFile(marker, JSON.stringify({ pid: process.pid }))
|
||||
|
||||
await expect(
|
||||
new PackRefsLockOwnership(commonDir).claim(laterBy(ABANDONED_LOCK_AGE_MS + 1))
|
||||
).resolves.toMatchObject({ ok: false })
|
||||
await expect(exists(lock)).resolves.toBe(true)
|
||||
})
|
||||
|
||||
it('reclaims the lock its own dead process left behind', async () => {
|
||||
// SIGKILL and power loss bypass git's cleanup, and git never clears this itself.
|
||||
const commonDir = await gitCommonDir()
|
||||
const { lock, marker } = paths(commonDir)
|
||||
await writeFile(lock, 'abandoned mid-rewrite')
|
||||
await writeFile(marker, JSON.stringify({ pid: DEAD_PID }))
|
||||
|
||||
const claimed = await new PackRefsLockOwnership(commonDir).claim(
|
||||
laterBy(ABANDONED_LOCK_AGE_MS + 1)
|
||||
)
|
||||
|
||||
expect(claimed).toEqual({ ok: true })
|
||||
await expect(exists(lock)).resolves.toBe(false)
|
||||
await expect(readFile(marker, 'utf-8')).resolves.toContain(String(process.pid))
|
||||
})
|
||||
|
||||
it('leaves a young lock alone even when the marker names a dead process', async () => {
|
||||
// A marker outlives its lock, so a foreign lock can appear after our death.
|
||||
// Age is the only thing separating our wreckage from somebody's live lock.
|
||||
const commonDir = await gitCommonDir()
|
||||
const { lock, marker } = paths(commonDir)
|
||||
await writeFile(marker, JSON.stringify({ pid: DEAD_PID }))
|
||||
await writeFile(lock, 'a different git process, started just now')
|
||||
|
||||
await expect(new PackRefsLockOwnership(commonDir).claim()).resolves.toMatchObject({ ok: false })
|
||||
await expect(exists(lock)).resolves.toBe(true)
|
||||
})
|
||||
|
||||
it('does not wedge a repository forever when the recorded pid was recycled', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
const { lock, marker } = paths(commonDir)
|
||||
await writeFile(lock, 'abandoned mid-rewrite')
|
||||
// Our own pid stands in for a recycled one: alive, but not the process that wrote this.
|
||||
await writeFile(marker, JSON.stringify({ pid: process.pid }))
|
||||
|
||||
await expect(
|
||||
new PackRefsLockOwnership(commonDir).claim(laterBy(ABANDONED_LOCK_AGE_MS + 1))
|
||||
).resolves.toMatchObject({ ok: false })
|
||||
|
||||
await expect(
|
||||
new PackRefsLockOwnership(commonDir).claim(laterBy(PID_REUSE_HORIZON_MS + 1))
|
||||
).resolves.toEqual({ ok: true })
|
||||
await expect(exists(lock)).resolves.toBe(false)
|
||||
})
|
||||
|
||||
it('refuses a lock whose marker is unreadable rather than guessing', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
const { lock, marker } = paths(commonDir)
|
||||
await writeFile(lock, 'in progress')
|
||||
await writeFile(marker, 'not json')
|
||||
|
||||
await expect(
|
||||
new PackRefsLockOwnership(commonDir).claim(laterBy(PID_REUSE_HORIZON_MS + 1))
|
||||
).resolves.toMatchObject({ ok: false })
|
||||
await expect(exists(lock)).resolves.toBe(true)
|
||||
})
|
||||
|
||||
it('claims cleanly when a marker outlived its lock', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
await writeFile(paths(commonDir).marker, JSON.stringify({ pid: DEAD_PID }))
|
||||
|
||||
await expect(new PackRefsLockOwnership(commonDir).claim()).resolves.toEqual({ ok: true })
|
||||
})
|
||||
})
|
||||
|
||||
describe('stranded per-ref locks', () => {
|
||||
it('clears the empty refs/**/*.lock files its own dead process left behind', async () => {
|
||||
// `tempfile.c` opens the lock O_EXCL before linking it into the list the
|
||||
// signal handler walks, so a kill in that window leaves a 0-byte file that
|
||||
// Git never clears -- and `update-ref -d` on that ref then fails forever.
|
||||
const commonDir = await gitCommonDir()
|
||||
const namespace = join(commonDir, 'refs', 'remotes', 'origin')
|
||||
await mkdir(namespace, { recursive: true })
|
||||
await writeFile(join(namespace, 'main.lock'), '')
|
||||
await writeFile(join(namespace, 'main'), 'a'.repeat(40))
|
||||
await writeFile(paths(commonDir).marker, JSON.stringify({ pid: DEAD_PID }))
|
||||
|
||||
await new PackRefsLockOwnership(commonDir).claim(laterBy(ABANDONED_LOCK_AGE_MS + 1))
|
||||
|
||||
await expect(exists(join(namespace, 'main.lock'))).resolves.toBe(false)
|
||||
// The ref itself is untouched.
|
||||
await expect(exists(join(namespace, 'main'))).resolves.toBe(true)
|
||||
})
|
||||
|
||||
it('leaves a non-empty ref lock alone, because a live writer is mid-write', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
const namespace = join(commonDir, 'refs', 'heads')
|
||||
await mkdir(namespace, { recursive: true })
|
||||
await writeFile(join(namespace, 'busy.lock'), 'b'.repeat(40))
|
||||
await writeFile(paths(commonDir).marker, JSON.stringify({ pid: DEAD_PID }))
|
||||
|
||||
await new PackRefsLockOwnership(commonDir).claim(laterBy(ABANDONED_LOCK_AGE_MS + 1))
|
||||
|
||||
await expect(exists(join(namespace, 'busy.lock'))).resolves.toBe(true)
|
||||
})
|
||||
|
||||
it('leaves ref locks alone when there is no marker naming a dead process', async () => {
|
||||
const commonDir = await gitCommonDir()
|
||||
const namespace = join(commonDir, 'refs', 'heads')
|
||||
await mkdir(namespace, { recursive: true })
|
||||
await writeFile(join(namespace, 'other.lock'), '')
|
||||
|
||||
await new PackRefsLockOwnership(commonDir).claim(laterBy(ABANDONED_LOCK_AGE_MS + 1))
|
||||
|
||||
await expect(exists(join(namespace, 'other.lock'))).resolves.toBe(true)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,202 @@
|
||||
import { readFile, readdir, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import { posix, win32 } from 'node:path'
|
||||
import { isWindowsAbsolutePathLike } from '../../shared/cross-platform-path'
|
||||
import {
|
||||
PACK_REFS_TIMEOUT_MS,
|
||||
PACKED_REFS_LOCK_POLL_MS
|
||||
} from '../../shared/repo-ref-maintenance-policy'
|
||||
|
||||
/** No legitimate `pack-refs` outlives its own deadline, so an older lock is abandoned. */
|
||||
const ABANDONED_LOCK_AGE_MS = PACK_REFS_TIMEOUT_MS
|
||||
|
||||
/** Beyond this a recorded pid may have been recycled, so it stops being evidence of life. */
|
||||
const PID_REUSE_HORIZON_MS = 24 * 60 * 60_000
|
||||
|
||||
/** The ref tree is wide but shallow; this only stops a pathological walk. */
|
||||
const REF_LOCK_SCAN_CEILING = 4096
|
||||
|
||||
/**
|
||||
* Makes a `packed-refs.lock` Orca left behind attributable, and only that one.
|
||||
*
|
||||
* Git registers signal handlers that clean the lock up, but SIGKILL and power
|
||||
* loss bypass them, and Git never removes a stale `packed-refs.lock` on its own
|
||||
* -- every later ref deletion in that repository fails until someone deletes a
|
||||
* file they have never heard of. Recording our pid beside the lock lets a later
|
||||
* run recognise its own wreckage.
|
||||
*
|
||||
* Three independent conditions must all hold before anything is unlinked,
|
||||
* because deleting a lock somebody else is holding is far worse than declining
|
||||
* to pack: a marker must exist at all, the lock must be older than any
|
||||
* `pack-refs` could legitimately run for, and the recorded process must be gone.
|
||||
* A marker can outlive its lock, so age is what separates "our wreckage" from a
|
||||
* foreign lock that happened to appear afterwards.
|
||||
*/
|
||||
export class PackRefsLockOwnership {
|
||||
private readonly lockPath: string
|
||||
private readonly markerPath: string
|
||||
|
||||
constructor(gitCommonDir: string) {
|
||||
const path = isWindowsAbsolutePathLike(gitCommonDir) ? win32 : posix
|
||||
this.lockPath = path.join(gitCommonDir, 'packed-refs.lock')
|
||||
this.markerPath = path.join(gitCommonDir, 'packed-refs.orca-owner')
|
||||
}
|
||||
|
||||
/** Refused when the lock belongs to something we cannot prove is our own wreckage. */
|
||||
async claim(now = Date.now()): Promise<PackRefsLockClaim> {
|
||||
const reclaim = await this.reclaimAbandonedLock(now)
|
||||
if (!reclaim.ok) {
|
||||
return reclaim
|
||||
}
|
||||
// Per-ref strands outlive their pack and are invisible to Git, which never
|
||||
// clears a `refs/**\/*.lock` it did not create in this process.
|
||||
await this.reclaimStrandedRefLocks(now)
|
||||
try {
|
||||
await writeFile(this.markerPath, JSON.stringify({ pid: process.pid }), 'utf-8')
|
||||
} catch {
|
||||
// Losing the marker only costs attribution on the next run, never correctness.
|
||||
}
|
||||
return { ok: true }
|
||||
}
|
||||
|
||||
/**
|
||||
* Poll `packed-refs.lock` so the scheduler knows when the exclusive rewrite
|
||||
* window opens and closes. Cheap: one `stat` on a fixed path.
|
||||
*/
|
||||
watchLock(report: (held: boolean) => void): { stop: () => void } {
|
||||
let stopped = false
|
||||
let last = false
|
||||
const tick = async (): Promise<void> => {
|
||||
if (stopped) {
|
||||
return
|
||||
}
|
||||
const held = (await fileAgeMs(this.lockPath, Date.now())) !== null
|
||||
if (!stopped && held !== last) {
|
||||
last = held
|
||||
report(held)
|
||||
}
|
||||
}
|
||||
const timer = setInterval(() => void tick(), PACKED_REFS_LOCK_POLL_MS)
|
||||
timer.unref?.()
|
||||
void tick()
|
||||
return {
|
||||
stop: () => {
|
||||
stopped = true
|
||||
clearInterval(timer)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async release(): Promise<void> {
|
||||
await rm(this.markerPath, { force: true }).catch(() => {})
|
||||
}
|
||||
|
||||
private async reclaimAbandonedLock(now: number): Promise<PackRefsLockClaim> {
|
||||
const lockAgeMs = await fileAgeMs(this.lockPath, now)
|
||||
if (lockAgeMs === null) {
|
||||
return { ok: true }
|
||||
}
|
||||
// No marker means the lock is not ours to reason about, let alone remove.
|
||||
const marker = await readOwnerMarker(this.markerPath)
|
||||
if (marker === null) {
|
||||
return { ok: false, reason: 'held by another process' }
|
||||
}
|
||||
if (lockAgeMs < ABANDONED_LOCK_AGE_MS) {
|
||||
// Ours, but too young to be certain the writer is gone. Worth retrying soon.
|
||||
return { ok: false, reason: 'our own lock, not yet old enough to reclaim' }
|
||||
}
|
||||
// Past the pid-reuse horizon the pid proves nothing, and a lock this old is
|
||||
// abandoned whoever wrote it -- otherwise a recycled pid would wedge the
|
||||
// repository permanently.
|
||||
if (isProcessAlive(marker.pid) && lockAgeMs < PID_REUSE_HORIZON_MS) {
|
||||
return { ok: false, reason: 'the recorded owner is still running' }
|
||||
}
|
||||
await rm(this.lockPath, { force: true }).catch(() => {})
|
||||
await rm(this.markerPath, { force: true }).catch(() => {})
|
||||
return { ok: true }
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear `refs/**\/*.lock` files a dead pack of ours left behind.
|
||||
*
|
||||
* `tempfile.c` opens the lock `O_EXCL` before `activate_tempfile()` links it
|
||||
* into the list the signal handler walks, so a kill inside that window leaves
|
||||
* a 0-byte file. Afterwards `update-ref -d` and any fetch touching that ref
|
||||
* fail with `cannot lock ref ... File exists`, forever. Same three conditions
|
||||
* as the packed-refs lock, plus a size check: a live writer's lock is not empty.
|
||||
*/
|
||||
private async reclaimStrandedRefLocks(now: number): Promise<void> {
|
||||
const marker = await readOwnerMarker(this.markerPath)
|
||||
if (marker === null || isProcessAlive(marker.pid)) {
|
||||
return
|
||||
}
|
||||
const markerAgeMs = await fileAgeMs(this.markerPath, now)
|
||||
if (markerAgeMs === null || markerAgeMs < ABANDONED_LOCK_AGE_MS) {
|
||||
return
|
||||
}
|
||||
const path = isWindowsAbsolutePathLike(this.markerPath) ? win32 : posix
|
||||
const pending = [path.join(path.dirname(this.markerPath), 'refs')]
|
||||
let visited = 0
|
||||
while (pending.length > 0) {
|
||||
const directory = pending.pop()
|
||||
if (directory === undefined || (visited += 1) > REF_LOCK_SCAN_CEILING) {
|
||||
return
|
||||
}
|
||||
let entries: { name: string; isDirectory: () => boolean }[]
|
||||
try {
|
||||
entries = await readdir(directory, { withFileTypes: true })
|
||||
} catch {
|
||||
continue
|
||||
}
|
||||
for (const entry of entries) {
|
||||
const full = path.join(directory, entry.name)
|
||||
if (entry.isDirectory()) {
|
||||
pending.push(full)
|
||||
} else if (entry.name.endsWith('.lock') && (await isEmptyFile(full))) {
|
||||
await rm(full, { force: true }).catch(() => {})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export type PackRefsLockClaim = { ok: true } | { ok: false; reason: string }
|
||||
|
||||
/** A strand from the `O_EXCL` window is 0 bytes; a live writer's lock is not. */
|
||||
async function isEmptyFile(path: string): Promise<boolean> {
|
||||
try {
|
||||
return (await stat(path)).size === 0
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
async function readOwnerMarker(path: string): Promise<{ pid: number } | null> {
|
||||
try {
|
||||
const raw = (await readFile(path, 'utf-8')).slice(0, 256)
|
||||
const pid = (JSON.parse(raw) as { pid?: unknown }).pid
|
||||
return typeof pid === 'number' && Number.isInteger(pid) && pid > 0 ? { pid } : null
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/** Null when the file does not exist. Uses stat: the lock holds a whole packed-refs. */
|
||||
async function fileAgeMs(path: string, now: number): Promise<number | null> {
|
||||
try {
|
||||
return Math.max(0, now - (await stat(path)).mtimeMs)
|
||||
} catch (error) {
|
||||
return (error as NodeJS.ErrnoException).code === 'ENOENT' ? null : 0
|
||||
}
|
||||
}
|
||||
|
||||
function isProcessAlive(pid: number): boolean {
|
||||
if (pid === process.pid) {
|
||||
return true
|
||||
}
|
||||
try {
|
||||
process.kill(pid, 0)
|
||||
return true
|
||||
} catch (error) {
|
||||
return (error as NodeJS.ErrnoException).code !== 'ESRCH'
|
||||
}
|
||||
}
|
||||
+36
-20
@@ -7,6 +7,10 @@ import { gitRefTargetsBranchOnRemote } from '../../shared/git-remote-branch-name
|
||||
import type { GitPushTarget } from '../../shared/worktree/types'
|
||||
import type { GitRuntimeOptions } from './git-runtime-options'
|
||||
import { gitOptionsForWorktree } from './git-runtime-options'
|
||||
import {
|
||||
postponeRepoRefMaintenance,
|
||||
withRepoRefMaintenancePaused
|
||||
} from './local-repo-ref-maintenance'
|
||||
import { validateGitPushTarget } from './push-target-validation'
|
||||
import { gitExecFileAsync } from './runner'
|
||||
import { fetchForkRemoteWithStaleRefspecRepair } from './fork-remote-stale-branch-refspec'
|
||||
@@ -260,8 +264,11 @@ export async function gitPull(
|
||||
// Why: plain `git pull` uses the user's configured pull strategy (merge by
|
||||
// default) so diverged branches reconcile instead of erroring out. Conflicts
|
||||
// surface through the existing conflict-resolution flow.
|
||||
await runWithGitWorktreeOperationLock(worktreePath, options.signal, () =>
|
||||
runWithGitReadCacheInvalidation(() => gitPullWithArgs(worktreePath, [], pushTarget, options))
|
||||
postponeRepoRefMaintenance()
|
||||
await withRepoRefMaintenancePaused('git-pull', () =>
|
||||
runWithGitWorktreeOperationLock(worktreePath, options.signal, () =>
|
||||
runWithGitReadCacheInvalidation(() => gitPullWithArgs(worktreePath, [], pushTarget, options))
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
@@ -270,9 +277,12 @@ export async function gitFastForward(
|
||||
pushTarget?: GitPushTarget,
|
||||
options: GitRuntimeOptions = {}
|
||||
): Promise<void> {
|
||||
await runWithGitWorktreeOperationLock(worktreePath, options.signal, () =>
|
||||
runWithGitReadCacheInvalidation(() =>
|
||||
gitPullWithArgs(worktreePath, ['--ff-only'], pushTarget, options)
|
||||
postponeRepoRefMaintenance()
|
||||
await withRepoRefMaintenancePaused('git-fast-forward', () =>
|
||||
runWithGitWorktreeOperationLock(worktreePath, options.signal, () =>
|
||||
runWithGitReadCacheInvalidation(() =>
|
||||
gitPullWithArgs(worktreePath, ['--ff-only'], pushTarget, options)
|
||||
)
|
||||
)
|
||||
)
|
||||
}
|
||||
@@ -282,22 +292,28 @@ export async function gitFetch(
|
||||
pushTarget?: GitPushTarget,
|
||||
options: GitRuntimeOptions = {}
|
||||
): Promise<void> {
|
||||
// `--prune` deletes remote-tracking refs, which needs the `packed-refs` lock a
|
||||
// running idle pack holds while it rewrites -- ~1.4s at most. This is the user
|
||||
// clicking Fetch, so wait that window out rather than letting it fail on the lock.
|
||||
postponeRepoRefMaintenance()
|
||||
try {
|
||||
if (pushTarget) {
|
||||
const target = await validateGitPushTarget(worktreePath, pushTarget, options)
|
||||
const runtimeOptions = gitOptionsForWorktree(worktreePath, options)
|
||||
await fetchForkRemoteWithStaleRefspecRepair(
|
||||
(args, cwd) => gitExecFileAsync(args, { ...runtimeOptions, cwd }),
|
||||
worktreePath,
|
||||
target.remoteName,
|
||||
() =>
|
||||
gitExecFileAsync(['fetch', '--prune', target.remoteName], runtimeOptions).then(
|
||||
() => undefined
|
||||
)
|
||||
)
|
||||
return
|
||||
}
|
||||
await gitExecFileAsync(['fetch', '--prune'], gitOptionsForWorktree(worktreePath, options))
|
||||
await withRepoRefMaintenancePaused('git-fetch', async () => {
|
||||
if (pushTarget) {
|
||||
const target = await validateGitPushTarget(worktreePath, pushTarget, options)
|
||||
const runtimeOptions = gitOptionsForWorktree(worktreePath, options)
|
||||
await fetchForkRemoteWithStaleRefspecRepair(
|
||||
(args, cwd) => gitExecFileAsync(args, { ...runtimeOptions, cwd }),
|
||||
worktreePath,
|
||||
target.remoteName,
|
||||
() =>
|
||||
gitExecFileAsync(['fetch', '--prune', target.remoteName], runtimeOptions).then(
|
||||
() => undefined
|
||||
)
|
||||
)
|
||||
return
|
||||
}
|
||||
await gitExecFileAsync(['fetch', '--prune'], gitOptionsForWorktree(worktreePath, options))
|
||||
})
|
||||
} catch (error) {
|
||||
throw new Error(normalizeGitErrorMessage(error, 'fetch'))
|
||||
}
|
||||
|
||||
@@ -0,0 +1,290 @@
|
||||
import { execFileSync } from 'node:child_process'
|
||||
import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { countLooseRefs } from '../../shared/loose-ref-count'
|
||||
import { RepoRefMaintenance } from '../../shared/repo-ref-maintenance'
|
||||
import {
|
||||
_resetLocalRepoRefMaintenanceForTests,
|
||||
createLocalRepoRefMaintenanceTarget,
|
||||
getLocalRepoRefMaintenance,
|
||||
setRepoMaintenanceActivityProbe
|
||||
} from './local-repo-ref-maintenance'
|
||||
import { forceDeleteLocalBranch } from './worktree-branch-removal'
|
||||
|
||||
const roots: string[] = []
|
||||
// Large enough that the deferral ladder (1x, 2x, 4x ... capped at 8x) outlasts
|
||||
// three real `pack-refs` runs before the deferral budget is spent.
|
||||
const QUIET_MS = 25
|
||||
const THRESHOLD = 20
|
||||
|
||||
function git(cwd: string, args: string[]): string {
|
||||
return execFileSync('git', args, {
|
||||
cwd,
|
||||
encoding: 'utf8',
|
||||
stdio: ['pipe', 'pipe', 'pipe']
|
||||
}).trim()
|
||||
}
|
||||
|
||||
/** A repo whose only loose-ref backlog is the one the test asks for. */
|
||||
async function createRepo(looseRefs: number): Promise<{ repoPath: string; refsDir: string }> {
|
||||
const root = await mkdtemp(join(tmpdir(), 'orca-ref-maintenance-git-'))
|
||||
roots.push(root)
|
||||
const repoPath = join(root, 'repo')
|
||||
execFileSync('git', ['init', '--quiet', repoPath])
|
||||
git(repoPath, ['symbolic-ref', 'HEAD', 'refs/heads/main'])
|
||||
git(repoPath, ['config', 'user.email', 'test@example.com'])
|
||||
git(repoPath, ['config', 'user.name', 'Test User'])
|
||||
await writeFile(join(repoPath, 'file.txt'), 'one\n')
|
||||
git(repoPath, ['add', 'file.txt'])
|
||||
git(repoPath, ['commit', '--quiet', '-m', 'initial'])
|
||||
const head = git(repoPath, ['rev-parse', 'HEAD'])
|
||||
// Written directly: `update-ref` for thousands of refs is the slow part of the fixture.
|
||||
const namespace = join(repoPath, '.git', 'refs', 'remotes', 'origin')
|
||||
await mkdir(namespace, { recursive: true })
|
||||
for (let index = 0; index < looseRefs; index += 1) {
|
||||
await writeFile(join(namespace, `branch-${index}`), `${head}\n`)
|
||||
}
|
||||
return { repoPath, refsDir: join(repoPath, '.git', 'refs') }
|
||||
}
|
||||
|
||||
function createMaintenance(onPackRefs: () => void = () => {}): {
|
||||
maintenance: RepoRefMaintenance
|
||||
arm: (repoPath: string) => void
|
||||
} {
|
||||
const maintenance = new RepoRefMaintenance({
|
||||
quietPeriodMs: QUIET_MS,
|
||||
looseRefThreshold: THRESHOLD
|
||||
})
|
||||
return {
|
||||
maintenance,
|
||||
arm: (repoPath: string) => {
|
||||
const target = createLocalRepoRefMaintenanceTarget({
|
||||
key: `local::${repoPath}`,
|
||||
repoPath
|
||||
})
|
||||
maintenance.arm({
|
||||
...target,
|
||||
packRefs: async (signal) => {
|
||||
onPackRefs()
|
||||
await target.packRefs(signal)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function settle(maintenance: RepoRefMaintenance): Promise<void> {
|
||||
await new Promise((resolve) => setTimeout(resolve, QUIET_MS * 4))
|
||||
await maintenance.whenAttemptSettled()
|
||||
}
|
||||
|
||||
/** Deferred repos re-arm for another quiet period, so drain rather than count rounds. */
|
||||
async function settleUntil(
|
||||
maintenance: RepoRefMaintenance,
|
||||
done: () => Promise<boolean>
|
||||
): Promise<void> {
|
||||
for (let round = 0; round < 100; round += 1) {
|
||||
if (await done()) {
|
||||
return
|
||||
}
|
||||
await settle(maintenance)
|
||||
}
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
_resetLocalRepoRefMaintenanceForTests()
|
||||
await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true })))
|
||||
})
|
||||
|
||||
describe('idle ref maintenance against real Git', () => {
|
||||
it('packs a backlogged repository down to zero loose refs', async () => {
|
||||
const { repoPath, refsDir } = await createRepo(THRESHOLD + 30)
|
||||
const { maintenance, arm } = createMaintenance()
|
||||
|
||||
await expect(countLooseRefs(refsDir, 10_000)).resolves.toMatchObject({
|
||||
count: THRESHOLD + 31
|
||||
})
|
||||
|
||||
arm(repoPath)
|
||||
await settle(maintenance)
|
||||
maintenance.dispose()
|
||||
|
||||
await expect(countLooseRefs(refsDir, 10_000)).resolves.toEqual({ count: 0, saturated: false })
|
||||
// The refs survived the move into packed-refs; nothing was lost.
|
||||
expect(git(repoPath, ['for-each-ref', '--format=%(refname)']).split('\n')).toHaveLength(
|
||||
THRESHOLD + 31
|
||||
)
|
||||
expect(git(repoPath, ['rev-parse', '--verify', 'refs/remotes/origin/branch-0'])).toMatch(
|
||||
/^[0-9a-f]{40}$/
|
||||
)
|
||||
}, 30_000)
|
||||
|
||||
it('leaves a healthy repository untouched', async () => {
|
||||
const { repoPath, refsDir } = await createRepo(2)
|
||||
let packed = 0
|
||||
const { maintenance, arm } = createMaintenance(() => {
|
||||
packed += 1
|
||||
})
|
||||
|
||||
arm(repoPath)
|
||||
await settle(maintenance)
|
||||
maintenance.dispose()
|
||||
|
||||
expect(packed).toBe(0)
|
||||
await expect(countLooseRefs(refsDir, 10_000)).resolves.toMatchObject({ count: 3 })
|
||||
}, 30_000)
|
||||
|
||||
it('honours maintenance.auto=false in the repository config', async () => {
|
||||
const { repoPath, refsDir } = await createRepo(THRESHOLD + 30)
|
||||
git(repoPath, ['config', 'maintenance.auto', 'false'])
|
||||
let packed = 0
|
||||
const { maintenance, arm } = createMaintenance(() => {
|
||||
packed += 1
|
||||
})
|
||||
|
||||
arm(repoPath)
|
||||
await settle(maintenance)
|
||||
maintenance.dispose()
|
||||
|
||||
expect(packed).toBe(0)
|
||||
await expect(countLooseRefs(refsDir, 10_000)).resolves.toMatchObject({
|
||||
count: THRESHOLD + 31
|
||||
})
|
||||
}, 30_000)
|
||||
|
||||
it('runs one repository at a time even when several go quiet together', async () => {
|
||||
const repos = await Promise.all([
|
||||
createRepo(THRESHOLD + 5),
|
||||
createRepo(THRESHOLD + 5),
|
||||
createRepo(THRESHOLD + 5)
|
||||
])
|
||||
let concurrent = 0
|
||||
let peak = 0
|
||||
const maintenance = new RepoRefMaintenance({
|
||||
quietPeriodMs: QUIET_MS,
|
||||
looseRefThreshold: THRESHOLD
|
||||
})
|
||||
for (const { repoPath } of repos) {
|
||||
const target = createLocalRepoRefMaintenanceTarget({
|
||||
key: `local::${repoPath}`,
|
||||
repoPath
|
||||
})
|
||||
maintenance.arm({
|
||||
...target,
|
||||
packRefs: async (signal) => {
|
||||
concurrent += 1
|
||||
peak = Math.max(peak, concurrent)
|
||||
try {
|
||||
await target.packRefs(signal)
|
||||
} finally {
|
||||
concurrent -= 1
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
const allPacked = async (): Promise<boolean> => {
|
||||
const counts = await Promise.all(repos.map(({ refsDir }) => countLooseRefs(refsDir, 10_000)))
|
||||
return counts.every((scan) => scan.count === 0)
|
||||
}
|
||||
await settleUntil(maintenance, allPacked)
|
||||
maintenance.dispose()
|
||||
|
||||
expect(peak).toBe(1)
|
||||
for (const { refsDir } of repos) {
|
||||
await expect(countLooseRefs(refsDir, 10_000)).resolves.toEqual({
|
||||
count: 0,
|
||||
saturated: false
|
||||
})
|
||||
}
|
||||
}, 60_000)
|
||||
})
|
||||
|
||||
describe('yielding the repository to work that deletes refs', () => {
|
||||
it('waits for the packed-refs lock and succeeds while the prune continues', async () => {
|
||||
// The pack is never killed. `packed-refs.lock` is held for ~1.4s of a 30s
|
||||
// run; the rest is the prune, during which a concurrent `update-ref -d`
|
||||
// succeeds on its own because per-ref locks last microseconds. Signalling
|
||||
// the child there strands a `refs/**` lock Git never clears.
|
||||
const { repoPath } = await createRepo(0)
|
||||
git(repoPath, ['branch', 'doomed'])
|
||||
const head = git(repoPath, ['rev-parse', 'refs/heads/doomed'])
|
||||
|
||||
let packing = false
|
||||
let releaseLock: (() => void) | undefined
|
||||
_resetLocalRepoRefMaintenanceForTests({ quietPeriodMs: QUIET_MS, looseRefThreshold: 1 })
|
||||
setRepoMaintenanceActivityProbe(() => false)
|
||||
getLocalRepoRefMaintenance().arm({
|
||||
key: `local::${repoPath}`,
|
||||
resolveRefsDirectory: async () => join(repoPath, '.git', 'refs'),
|
||||
packRefs: async (lock) => {
|
||||
packing = true
|
||||
lock.setHeld(true)
|
||||
// Stands in for the rewrite window, then the long prune that follows it.
|
||||
await new Promise<void>((resolve) => {
|
||||
releaseLock = () => {
|
||||
lock.setHeld(false)
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
}
|
||||
})
|
||||
for (let attempt = 0; attempt < 200 && !packing; attempt += 1) {
|
||||
await new Promise((resolve) => setTimeout(resolve, QUIET_MS))
|
||||
}
|
||||
expect(packing).toBe(true)
|
||||
|
||||
// The real deletion path, which routes through withRepoRefMaintenancePaused.
|
||||
let deleted = false
|
||||
const deletion = forceDeleteLocalBranch(repoPath, 'doomed', head).then(() => {
|
||||
deleted = true
|
||||
})
|
||||
|
||||
// It must still be waiting: the rewrite window is open.
|
||||
await new Promise((resolve) => setTimeout(resolve, QUIET_MS * 4))
|
||||
expect(deleted).toBe(false)
|
||||
expect(git(repoPath, ['branch', '--list', 'doomed'])).toContain('doomed')
|
||||
|
||||
// Releasing the window is enough -- the pack is never cancelled.
|
||||
releaseLock?.()
|
||||
await deletion
|
||||
expect(deleted).toBe(true)
|
||||
expect(git(repoPath, ['branch', '--list', 'doomed'])).toBe('')
|
||||
}, 30_000)
|
||||
|
||||
it('does not block the caller once the rewrite window has closed', async () => {
|
||||
// The prune phase is concurrency-safe, so a caller arriving during it pays
|
||||
// nothing at all.
|
||||
const { repoPath } = await createRepo(0)
|
||||
git(repoPath, ['branch', 'doomed'])
|
||||
const head = git(repoPath, ['rev-parse', 'refs/heads/doomed'])
|
||||
|
||||
let pruning = false
|
||||
let finishPrune: (() => void) | undefined
|
||||
_resetLocalRepoRefMaintenanceForTests({ quietPeriodMs: QUIET_MS, looseRefThreshold: 1 })
|
||||
setRepoMaintenanceActivityProbe(() => false)
|
||||
getLocalRepoRefMaintenance().arm({
|
||||
key: `local::${repoPath}`,
|
||||
resolveRefsDirectory: async () => join(repoPath, '.git', 'refs'),
|
||||
packRefs: async (lock) => {
|
||||
lock.setHeld(true)
|
||||
lock.setHeld(false)
|
||||
pruning = true
|
||||
await new Promise<void>((resolve) => {
|
||||
finishPrune = resolve
|
||||
})
|
||||
}
|
||||
})
|
||||
for (let attempt = 0; attempt < 200 && !pruning; attempt += 1) {
|
||||
await new Promise((resolve) => setTimeout(resolve, QUIET_MS))
|
||||
}
|
||||
|
||||
const startedAt = Date.now()
|
||||
await expect(forceDeleteLocalBranch(repoPath, 'doomed', head)).resolves.toBeUndefined()
|
||||
expect(Date.now() - startedAt).toBeLessThan(2_000)
|
||||
|
||||
finishPrune?.()
|
||||
}, 30_000)
|
||||
})
|
||||
@@ -4,6 +4,7 @@ import type {
|
||||
LocalBaseRefUpdateSuggestion
|
||||
} from '../../shared/worktree/base-ref-drift-types'
|
||||
import { windowsLongPathGitArgs } from '../../shared/windows-long-path-git-args'
|
||||
import { withRepoRefMaintenancePaused } from './local-repo-ref-maintenance'
|
||||
import { gitExecFileAsync } from './runner'
|
||||
import { runWithGitReadCacheInvalidation } from './status'
|
||||
import { invalidateWslLinkedWorktreeGitRouting } from './wsl-linked-worktree-git-routing'
|
||||
@@ -149,15 +150,17 @@ export async function addWorktree(
|
||||
options: AddWorktreeOptions = {}
|
||||
): Promise<AddWorktreeResult> {
|
||||
try {
|
||||
return await runWithGitReadCacheInvalidation(() =>
|
||||
performAddWorktree(
|
||||
repoPath,
|
||||
worktreePath,
|
||||
branch,
|
||||
baseBranch,
|
||||
refreshLocalBaseRef,
|
||||
noCheckout,
|
||||
options
|
||||
return await withRepoRefMaintenancePaused('worktree-add', () =>
|
||||
runWithGitReadCacheInvalidation(() =>
|
||||
performAddWorktree(
|
||||
repoPath,
|
||||
worktreePath,
|
||||
branch,
|
||||
baseBranch,
|
||||
refreshLocalBaseRef,
|
||||
noCheckout,
|
||||
options
|
||||
)
|
||||
)
|
||||
)
|
||||
} finally {
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
} from '../../shared/git-branch-cleanup'
|
||||
import type { RemoveWorktreeResult } from '../../shared/worktree/create-types'
|
||||
import { withLocalGitCapabilityCacheForExecution } from './git-capability-state'
|
||||
import { withRepoRefMaintenancePaused } from './local-repo-ref-maintenance'
|
||||
import { gitExecFileAsync } from './runner'
|
||||
import { parseWorktreeList } from './worktree-list-parser'
|
||||
import type { GitWorktreeExecOptions, RemoveWorktreeOptions } from './worktree-operation-options'
|
||||
@@ -152,7 +153,11 @@ export async function forceDeleteLocalBranch(
|
||||
}
|
||||
// Why: stale toast actions must not delete a branch that moved; `update-ref -d` deletes only if the ref still == expectedHead.
|
||||
try {
|
||||
await runGit(['update-ref', '-d', `refs/heads/${branchName}`, expectedHead], repoPath)
|
||||
// `update-ref -d` needs the packed-refs lock a running idle pack holds while
|
||||
// it rewrites; waits it out rather than cancelling the pack.
|
||||
await withRepoRefMaintenancePaused('branch-delete', () =>
|
||||
runGit(['update-ref', '-d', `refs/heads/${branchName}`, expectedHead], repoPath)
|
||||
)
|
||||
} catch {
|
||||
throw new Error(
|
||||
`Local branch "${branchName}" changed after the workspace was deleted. Review it before deleting it.`
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
WORKTREE_REMOVAL_REGISTRATION_TIMEOUT_MS
|
||||
} from './worktree'
|
||||
import { hasWorktreeBaseCommitRef } from './worktree-base-ref-probe'
|
||||
import { withRepoRefMaintenancePaused } from './local-repo-ref-maintenance'
|
||||
import { gitExecFileAsync } from './runner'
|
||||
import { runWithGitReadCacheInvalidation } from './status'
|
||||
import { invalidateWslLinkedWorktreeGitRouting } from './wsl-linked-worktree-git-routing'
|
||||
@@ -69,46 +70,48 @@ export async function prepareWorktreeCreateCheckout(
|
||||
options: GitWorktreeExecOptions = {}
|
||||
): Promise<void> {
|
||||
try {
|
||||
await runWithGitReadCacheInvalidation(async () => {
|
||||
const effectiveBase = await resolveWorktreeAddBaseRef(baseBranch, (qualifiedRef) =>
|
||||
hasWorktreeBaseCommitRef(repoPath, qualifiedRef, options)
|
||||
)
|
||||
try {
|
||||
await gitExecFileAsync(
|
||||
[
|
||||
...windowsLongPathGitArgs(repoPath),
|
||||
'worktree',
|
||||
'add',
|
||||
'--detach',
|
||||
'--no-checkout',
|
||||
worktreePath,
|
||||
effectiveBase
|
||||
],
|
||||
{ ...gitExecOptions(repoPath, options), timeout: resolveWorktreeAddTimeoutMs() }
|
||||
await withRepoRefMaintenancePaused('worktree-prepare', () =>
|
||||
runWithGitReadCacheInvalidation(async () => {
|
||||
const effectiveBase = await resolveWorktreeAddBaseRef(baseBranch, (qualifiedRef) =>
|
||||
hasWorktreeBaseCommitRef(repoPath, qualifiedRef, options)
|
||||
)
|
||||
// The add just wrote the marker; drop any pre-create route before the reset routes Git.
|
||||
invalidateWslLinkedWorktreeGitRouting(worktreePath)
|
||||
// Why: reset materializes files without running user post-checkout hooks before submit.
|
||||
await gitExecFileAsync(
|
||||
[...windowsLongPathGitArgs(worktreePath), 'reset', '--hard', effectiveBase],
|
||||
{ ...gitExecOptions(worktreePath, options), timeout: resolveWorktreeAddTimeoutMs() }
|
||||
)
|
||||
await gitExecFileAsync(
|
||||
[
|
||||
...windowsLongPathGitArgs(repoPath),
|
||||
'worktree',
|
||||
'lock',
|
||||
'--reason',
|
||||
lockReason,
|
||||
worktreePath
|
||||
],
|
||||
{ ...gitExecOptions(repoPath, options), timeout: resolveWorktreeAddTimeoutMs() }
|
||||
)
|
||||
} catch (error) {
|
||||
await performDiscardPreparedWorktree(repoPath, worktreePath, options).catch(() => {})
|
||||
throw error
|
||||
}
|
||||
})
|
||||
try {
|
||||
await gitExecFileAsync(
|
||||
[
|
||||
...windowsLongPathGitArgs(repoPath),
|
||||
'worktree',
|
||||
'add',
|
||||
'--detach',
|
||||
'--no-checkout',
|
||||
worktreePath,
|
||||
effectiveBase
|
||||
],
|
||||
{ ...gitExecOptions(repoPath, options), timeout: resolveWorktreeAddTimeoutMs() }
|
||||
)
|
||||
// The add just wrote the marker; drop any pre-create route before the reset routes Git.
|
||||
invalidateWslLinkedWorktreeGitRouting(worktreePath)
|
||||
// Why: reset materializes files without running user post-checkout hooks before submit.
|
||||
await gitExecFileAsync(
|
||||
[...windowsLongPathGitArgs(worktreePath), 'reset', '--hard', effectiveBase],
|
||||
{ ...gitExecOptions(worktreePath, options), timeout: resolveWorktreeAddTimeoutMs() }
|
||||
)
|
||||
await gitExecFileAsync(
|
||||
[
|
||||
...windowsLongPathGitArgs(repoPath),
|
||||
'worktree',
|
||||
'lock',
|
||||
'--reason',
|
||||
lockReason,
|
||||
worktreePath
|
||||
],
|
||||
{ ...gitExecOptions(repoPath, options), timeout: resolveWorktreeAddTimeoutMs() }
|
||||
)
|
||||
} catch (error) {
|
||||
await performDiscardPreparedWorktree(repoPath, worktreePath, options).catch(() => {})
|
||||
throw error
|
||||
}
|
||||
})
|
||||
)
|
||||
} finally {
|
||||
notifyPreparedWorktreeMutation(repoPath)
|
||||
}
|
||||
|
||||
@@ -22,6 +22,7 @@ import {
|
||||
} from './worktree-operation-options'
|
||||
import { areWorktreePathsEqual } from './worktree-path-comparison'
|
||||
import { assertWorktreeCleanForRemoval } from './worktree-removal-preflight'
|
||||
import { withRepoRefMaintenancePaused } from './local-repo-ref-maintenance'
|
||||
import { bumpWorktreeScanGeneration, listWorktrees } from './worktree-scan-cache'
|
||||
import { invalidateSparseCheckoutState } from './worktree-sparse-checkout-cache'
|
||||
|
||||
@@ -36,8 +37,13 @@ export async function removeWorktree(
|
||||
options: RemoveWorktreeOptions = {}
|
||||
): Promise<RemoveWorktreeResult> {
|
||||
try {
|
||||
return await runWithGitReadCacheInvalidation(() =>
|
||||
performRemoveWorktree(repoPath, worktreePath, force, options)
|
||||
// Removal deletes branches, and a ref deletion needs the packed-refs lock a
|
||||
// running idle pack holds while it rewrites. Waits that window out; the
|
||||
// prune phase that follows it is concurrency-safe and is left to finish.
|
||||
return await withRepoRefMaintenancePaused('worktree-remove', () =>
|
||||
runWithGitReadCacheInvalidation(() =>
|
||||
performRemoveWorktree(repoPath, worktreePath, force, options)
|
||||
)
|
||||
)
|
||||
} finally {
|
||||
invalidateWslLinkedWorktreeGitRouting(worktreePath)
|
||||
|
||||
@@ -61,8 +61,11 @@ vi.mock('fs/promises', () => ({
|
||||
rm: rmMock
|
||||
}))
|
||||
|
||||
// `availableParallelism` is read at module load by the git admission scheduler,
|
||||
// which this module graph reaches; a partial `os` mock breaks that import.
|
||||
vi.mock('os', () => ({
|
||||
homedir: homedirMock
|
||||
homedir: homedirMock,
|
||||
availableParallelism: () => 8
|
||||
}))
|
||||
|
||||
vi.mock('../git/runner', () => ({
|
||||
|
||||
@@ -17,7 +17,10 @@ import { registerSparseCheckoutCacheInvalidation } from './worktrees/listing/reg
|
||||
import { registerWorktreeMetadataHandlers } from './worktrees/metadata/register-worktree-metadata-handlers'
|
||||
import { registerWorktreeForgetHandlers } from './worktrees/removal/register-worktree-forget-handlers'
|
||||
import { registerWorktreeRemovalHandlers } from './worktrees/removal/register-worktree-removal-handlers'
|
||||
import type { WorktreeIpcContext } from './worktrees/worktree-ipc-context'
|
||||
import {
|
||||
createWorktreeRemovalRegistry,
|
||||
type WorktreeIpcContext
|
||||
} from './worktrees/worktree-ipc-context'
|
||||
|
||||
registerDetectedWorktreeScanInvalidation()
|
||||
|
||||
@@ -66,7 +69,7 @@ export function registerWorktreeHandlers(
|
||||
runtime,
|
||||
...(options ? { options } : {}),
|
||||
detectedWorktreeCancellations: createSenderScopedRequestCancellations(),
|
||||
worktreeRemovalsInFlight: new Map()
|
||||
worktreeRemovalsInFlight: createWorktreeRemovalRegistry()
|
||||
}
|
||||
|
||||
// Remove all stale registrations before installing any replacement handler.
|
||||
|
||||
@@ -14,3 +14,18 @@ export type WorktreeIpcContext = {
|
||||
detectedWorktreeCancellations: SenderScopedRequestCancellations
|
||||
worktreeRemovalsInFlight: Map<string, WorktreeRemovalInFlight>
|
||||
}
|
||||
|
||||
// Why: removal and forget both delete refs, and a ref deletion has to take the
|
||||
// `packed-refs` lock. Idle ref maintenance needs a process-wide view of that
|
||||
// registry so it never packs while one is running.
|
||||
let activeWorktreeRemovals: ReadonlyMap<string, WorktreeRemovalInFlight> | null = null
|
||||
|
||||
export function createWorktreeRemovalRegistry(): Map<string, WorktreeRemovalInFlight> {
|
||||
const registry = new Map<string, WorktreeRemovalInFlight>()
|
||||
activeWorktreeRemovals = registry
|
||||
return registry
|
||||
}
|
||||
|
||||
export function hasWorktreeRemovalsInFlight(): boolean {
|
||||
return (activeWorktreeRemovals?.size ?? 0) > 0
|
||||
}
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const isOnBatteryPowerMock = vi.hoisted(() => vi.fn(() => false))
|
||||
const hasPendingPreparationsMock = vi.hoisted(() => vi.fn(() => false))
|
||||
const hasRemovalsInFlightMock = vi.hoisted(() => vi.fn(() => false))
|
||||
const setProbeMock = vi.hoisted(() => vi.fn())
|
||||
const disposeMock = vi.hoisted(() => vi.fn(async () => {}))
|
||||
const postponeMock = vi.hoisted(() => vi.fn())
|
||||
const powerListeners = vi.hoisted(() => new Map<string, () => void>())
|
||||
const appListeners = vi.hoisted(() => new Map<string, () => void>())
|
||||
|
||||
vi.mock('electron', () => ({
|
||||
app: {
|
||||
on: (event: string, listener: () => void) => appListeners.set(event, listener),
|
||||
off: (event: string) => appListeners.delete(event)
|
||||
},
|
||||
powerMonitor: {
|
||||
isOnBatteryPower: isOnBatteryPowerMock,
|
||||
on: (event: string, listener: () => void) => powerListeners.set(event, listener),
|
||||
off: (event: string) => powerListeners.delete(event)
|
||||
}
|
||||
}))
|
||||
|
||||
vi.mock('./worktree-create-preparation', () => ({
|
||||
hasPendingWorktreeCreatePreparations: hasPendingPreparationsMock
|
||||
}))
|
||||
|
||||
vi.mock('./ipc/worktrees/worktree-ipc-context', () => ({
|
||||
hasWorktreeRemovalsInFlight: hasRemovalsInFlightMock
|
||||
}))
|
||||
|
||||
vi.mock('./git/local-repo-ref-maintenance', () => ({
|
||||
setRepoMaintenanceActivityProbe: setProbeMock,
|
||||
disposeLocalRepoRefMaintenance: disposeMock,
|
||||
postponeRepoRefMaintenance: postponeMock
|
||||
}))
|
||||
|
||||
import { installRepoMaintenanceIdleGate } from './repo-maintenance-idle-gate'
|
||||
|
||||
function installProbe(
|
||||
overrides: Partial<{ isQuitting: () => boolean; getWorkingAgentCount: () => number }> = {}
|
||||
): { probe: () => boolean; uninstall: () => Promise<void> } {
|
||||
const uninstall = installRepoMaintenanceIdleGate({
|
||||
isQuitting: () => false,
|
||||
getWorkingAgentCount: () => 0,
|
||||
...overrides
|
||||
})
|
||||
return { probe: setProbeMock.mock.calls.at(-1)?.[0] as () => boolean, uninstall }
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
isOnBatteryPowerMock.mockReturnValue(false)
|
||||
hasPendingPreparationsMock.mockReturnValue(false)
|
||||
hasRemovalsInFlightMock.mockReturnValue(false)
|
||||
postponeMock.mockClear()
|
||||
powerListeners.clear()
|
||||
appListeners.clear()
|
||||
setProbeMock.mockClear()
|
||||
disposeMock.mockClear()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
describe('repo maintenance idle gate', () => {
|
||||
it('reports idle when nothing is happening', () => {
|
||||
expect(installProbe().probe()).toBe(false)
|
||||
})
|
||||
|
||||
it('vetoes while an agent is working', () => {
|
||||
expect(installProbe({ getWorkingAgentCount: () => 1 }).probe()).toBe(true)
|
||||
})
|
||||
|
||||
it('vetoes while a worktree create is prepared or in flight', () => {
|
||||
hasPendingPreparationsMock.mockReturnValue(true)
|
||||
|
||||
expect(installProbe().probe()).toBe(true)
|
||||
})
|
||||
|
||||
it('vetoes while a worktree removal is deleting refs', () => {
|
||||
// Removal deletes branches, and a ref deletion needs the same packed-refs lock.
|
||||
hasRemovalsInFlightMock.mockReturnValue(true)
|
||||
|
||||
expect(installProbe().probe()).toBe(true)
|
||||
})
|
||||
|
||||
it('vetoes on battery power', () => {
|
||||
isOnBatteryPowerMock.mockReturnValue(true)
|
||||
|
||||
expect(installProbe().probe()).toBe(true)
|
||||
})
|
||||
|
||||
it('vetoes during shutdown', () => {
|
||||
expect(installProbe({ isQuitting: () => true }).probe()).toBe(true)
|
||||
})
|
||||
|
||||
it('treats an unavailable power API as not-on-battery', () => {
|
||||
isOnBatteryPowerMock.mockImplementation(() => {
|
||||
throw new Error('unsupported')
|
||||
})
|
||||
|
||||
expect(installProbe().probe()).toBe(false)
|
||||
})
|
||||
|
||||
it('pushes the next attempt out when the machine drops onto battery', () => {
|
||||
// Do-not-start, never stop-what-is-running: killing a pack to honour a
|
||||
// battery change would strand a ref lock to save a little unlinking.
|
||||
installProbe()
|
||||
|
||||
powerListeners.get('on-battery')?.()
|
||||
|
||||
expect(postponeMock).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('pushes the next attempt out when the user comes back to the window', () => {
|
||||
// A focus transition, not focus itself: a window left focused while the user
|
||||
// walks away fires no event and blocks nothing.
|
||||
installProbe()
|
||||
|
||||
appListeners.get('browser-window-focus')?.()
|
||||
|
||||
expect(postponeMock).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('cancels armed timers, unsubscribes both sources, and clears the probe when uninstalled', async () => {
|
||||
await installProbe().uninstall()
|
||||
|
||||
expect(disposeMock).toHaveBeenCalledTimes(1)
|
||||
expect(powerListeners.has('on-battery')).toBe(false)
|
||||
expect(appListeners.has('browser-window-focus')).toBe(false)
|
||||
expect(setProbeMock).toHaveBeenLastCalledWith(null)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,67 @@
|
||||
import { app, powerMonitor } from 'electron'
|
||||
import {
|
||||
disposeLocalRepoRefMaintenance,
|
||||
postponeRepoRefMaintenance,
|
||||
setRepoMaintenanceActivityProbe
|
||||
} from './git/local-repo-ref-maintenance'
|
||||
import { hasWorktreeRemovalsInFlight } from './ipc/worktrees/worktree-ipc-context'
|
||||
import { hasPendingWorktreeCreatePreparations } from './worktree-create-preparation'
|
||||
|
||||
/**
|
||||
* The app-wide "not now" answer for idle repo maintenance.
|
||||
*
|
||||
* `pack-refs` holds a general git admission slot for its whole run, which on a
|
||||
* large backlog is minutes, and takes the `packed-refs` lock while it writes.
|
||||
* Any ref deletion needs that same lock and gives up after
|
||||
* `core.packedRefsTimeout` (1s), so worktree removal in particular has to veto
|
||||
* this -- as does a create in flight, an agent mid-run, and shutdown. Battery is
|
||||
* a veto too: this is work the user did not ask for, and a plugged-in quiet
|
||||
* window always comes along later.
|
||||
*/
|
||||
export type RepoMaintenanceIdleInputs = {
|
||||
isQuitting: () => boolean
|
||||
getWorkingAgentCount: () => number
|
||||
}
|
||||
|
||||
export function installRepoMaintenanceIdleGate(
|
||||
inputs: RepoMaintenanceIdleInputs
|
||||
): () => Promise<void> {
|
||||
setRepoMaintenanceActivityProbe(
|
||||
() =>
|
||||
inputs.isQuitting() ||
|
||||
inputs.getWorkingAgentCount() > 0 ||
|
||||
hasPendingWorktreeCreatePreparations() ||
|
||||
hasWorktreeRemovalsInFlight() ||
|
||||
isOnBatteryPower()
|
||||
)
|
||||
// Do-not-start, never stop-what-is-running. Killing a pack to honour a battery
|
||||
// or focus change would strand a ref lock roughly one time in five to save at
|
||||
// most a couple of minutes of background unlinking; pushing the next attempt
|
||||
// out costs nothing and risks nothing.
|
||||
const onBattery = (): void => {
|
||||
postponeRepoRefMaintenance()
|
||||
}
|
||||
const onFocus = (): void => {
|
||||
postponeRepoRefMaintenance()
|
||||
}
|
||||
powerMonitor.on('on-battery', onBattery)
|
||||
app.on('browser-window-focus', onFocus)
|
||||
return () => {
|
||||
app.off('browser-window-focus', onFocus)
|
||||
powerMonitor.off('on-battery', onBattery)
|
||||
// Order matters: clearing the probe alone would leave armed timers running
|
||||
// against a gate that can no longer see agents, creates, or shutdown.
|
||||
const stopped = disposeLocalRepoRefMaintenance()
|
||||
setRepoMaintenanceActivityProbe(null)
|
||||
return stopped
|
||||
}
|
||||
}
|
||||
|
||||
function isOnBatteryPower(): boolean {
|
||||
try {
|
||||
return powerMonitor.isOnBatteryPower()
|
||||
} catch {
|
||||
// Absence of the API is not evidence of battery; desktops answer false anyway.
|
||||
return false
|
||||
}
|
||||
}
|
||||
@@ -133,9 +133,11 @@ describe('OrcaRuntimeService.fetchRemoteWithCache', () => {
|
||||
const first = runtime.fetchRemoteWithCache('/repo/c', 'origin')
|
||||
const second = runtime.fetchRemoteWithCache('/repo/c', 'origin')
|
||||
|
||||
// Allow both callers to register before we resolve.
|
||||
await Promise.resolve()
|
||||
await Promise.resolve()
|
||||
// Allow both callers to register before we resolve. Each canonicalizes the
|
||||
// repo key first, so the dispatch lands several microtasks in.
|
||||
for (let tick = 0; tick < 8; tick += 1) {
|
||||
await Promise.resolve()
|
||||
}
|
||||
expect(fetchCallCount()).toBe(1)
|
||||
|
||||
resolveFetch()
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
import { GIT_FETCH_SKIP_AUTO_MAINTENANCE_CONFIG_ARGS } from '../../shared/git-fetch-auto-maintenance'
|
||||
import { getCanonicalRepoKey } from '../git/canonical-repo-key'
|
||||
import {
|
||||
armLocalRepoRefMaintenance,
|
||||
setRepoRefMaintenanceBusyProbe
|
||||
} from '../git/local-repo-ref-maintenance'
|
||||
import { gitExecFileAsync } from '../git/runner'
|
||||
import { setBoundedMapEntry } from './runtime-async-boundaries'
|
||||
|
||||
@@ -33,6 +38,11 @@ export class RuntimeRemoteFetchController {
|
||||
return this.fetchLastCompletedAt
|
||||
}
|
||||
|
||||
/** `${runtimeKey}::${gitCommonDir}` -- one repo on one execution host. */
|
||||
async getCanonicalRepoKey(repoPath: string, gitOptions: GitOptions = {}): Promise<string> {
|
||||
return getCanonicalRepoKey(repoPath, gitOptions)
|
||||
}
|
||||
|
||||
async getCanonicalFetchKey(
|
||||
repoPath: string,
|
||||
remote: string,
|
||||
@@ -45,23 +55,41 @@ export class RuntimeRemoteFetchController {
|
||||
setBoundedMapEntry(this.canonicalFetchKeyCache, cacheKey, cached, REMOTE_FETCH_CACHE_MAX)
|
||||
return cached
|
||||
}
|
||||
let resolved = cacheKey
|
||||
try {
|
||||
const { stdout } = await gitExecFileAsync(
|
||||
['rev-parse', '--path-format=absolute', '--git-common-dir'],
|
||||
{ cwd: repoPath, ...gitOptions }
|
||||
)
|
||||
const commonDir = stdout.trim()
|
||||
if (commonDir) {
|
||||
resolved = `${runtimeKey}::${commonDir}::${remote}`
|
||||
}
|
||||
} catch {
|
||||
// The caller path remains a safe serialization key when canonicalization fails.
|
||||
}
|
||||
const resolved = `${await this.getCanonicalRepoKey(repoPath, gitOptions)}::${remote}`
|
||||
setBoundedMapEntry(this.canonicalFetchKeyCache, cacheKey, resolved, REMOTE_FETCH_CACHE_MAX)
|
||||
return resolved
|
||||
}
|
||||
|
||||
/**
|
||||
* Orca strips git's auto-maintenance off these fetches, so every one of them
|
||||
* adds to a loose-ref backlog nothing else will ever pack. Arm the idle sweep
|
||||
* that pays it back; each fetch pushes the attempt a further quiet period out.
|
||||
*/
|
||||
private armRefMaintenance(repoPath: string, gitOptions: GitOptions): void {
|
||||
void this.getCanonicalRepoKey(repoPath, gitOptions)
|
||||
.then((key) => {
|
||||
setRepoRefMaintenanceBusyProbe(key, () => this.hasInflightFetchForRepo(key))
|
||||
armLocalRepoRefMaintenance({
|
||||
key,
|
||||
repoPath,
|
||||
...(gitOptions.wslDistro ? { wslDistro: gitOptions.wslDistro } : {})
|
||||
})
|
||||
})
|
||||
.catch(() => {
|
||||
// Maintenance is best effort; a repo we cannot name is a repo we skip.
|
||||
})
|
||||
}
|
||||
|
||||
private hasInflightFetchForRepo(repoKey: string): boolean {
|
||||
const prefix = `${repoKey}::`
|
||||
for (const key of this.fetchInflight.keys()) {
|
||||
if (key.startsWith(prefix)) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private enqueueRemoteFetch(
|
||||
remoteKey: string,
|
||||
runFetch: () => Promise<RemoteFetchResult>
|
||||
@@ -123,6 +151,7 @@ export class RuntimeRemoteFetchController {
|
||||
})
|
||||
).finally(() => {
|
||||
this.fetchInflight.delete(key)
|
||||
this.armRefMaintenance(repoPath, gitOptions)
|
||||
})
|
||||
this.fetchInflight.set(key, promise)
|
||||
return promise
|
||||
@@ -178,6 +207,7 @@ export class RuntimeRemoteFetchController {
|
||||
})
|
||||
}).finally(() => {
|
||||
this.fetchInflight.delete(key)
|
||||
this.armRefMaintenance(repoPath, gitOptions)
|
||||
})
|
||||
this.fetchInflight.set(key, promise)
|
||||
return promise
|
||||
|
||||
@@ -0,0 +1,145 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
// Why: Orca's fetches are what create the loose-ref backlog (they suppress
|
||||
// git's auto-maintenance), so the fetch controller is where the idle sweep has
|
||||
// to be armed. These tests pin that wiring and the per-repo busy signal it
|
||||
// hands the sweep.
|
||||
|
||||
const gitExecFileAsyncMock = vi.hoisted(() => vi.fn())
|
||||
const armMock = vi.hoisted(() => vi.fn())
|
||||
const busyProbeMock = vi.hoisted(() => vi.fn())
|
||||
|
||||
vi.mock('../git/runner', async (importOriginal) => ({
|
||||
...((await importOriginal()) as Record<string, unknown>),
|
||||
gitExecFileAsync: gitExecFileAsyncMock
|
||||
}))
|
||||
|
||||
vi.mock('../git/local-repo-ref-maintenance', async (importOriginal) => ({
|
||||
...((await importOriginal()) as Record<string, unknown>),
|
||||
armLocalRepoRefMaintenance: armMock,
|
||||
setRepoRefMaintenanceBusyProbe: busyProbeMock
|
||||
}))
|
||||
|
||||
import { _resetCanonicalRepoKeyCacheForTests } from '../git/canonical-repo-key'
|
||||
import { RuntimeRemoteFetchController } from './runtime-remote-fetch-controller'
|
||||
|
||||
function armedTargets(): { key: string }[] {
|
||||
return armMock.mock.calls.map(([args]) => args as { key: string })
|
||||
}
|
||||
|
||||
/** The per-repo "a fetch is in flight" answer the controller registers for a key. */
|
||||
function busyProbeFor(key: string): (() => boolean) | undefined {
|
||||
return busyProbeMock.mock.calls.findLast(([registered]) => registered === key)?.[1] as
|
||||
| (() => boolean)
|
||||
| undefined
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
_resetCanonicalRepoKeyCacheForTests()
|
||||
gitExecFileAsyncMock.mockReset()
|
||||
armMock.mockReset()
|
||||
busyProbeMock.mockReset()
|
||||
gitExecFileAsyncMock.mockImplementation(async (argv: string[]) =>
|
||||
argv[0] === 'rev-parse' ? { stdout: '/repo/.git\n', stderr: '' } : { stdout: '', stderr: '' }
|
||||
)
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
describe('fetch-armed ref maintenance', () => {
|
||||
it('arms the sweep for the repo after a remote fetch, keyed by common dir', async () => {
|
||||
const controller = new RuntimeRemoteFetchController()
|
||||
|
||||
await controller.getOrStartRemoteFetch('/repo/worktrees/a', 'origin')
|
||||
|
||||
expect(armedTargets().map((target) => target.key)).toEqual(['local::/repo/.git'])
|
||||
})
|
||||
|
||||
it('gives every worktree of one repo the same maintenance key', async () => {
|
||||
const controller = new RuntimeRemoteFetchController()
|
||||
|
||||
await controller.getOrStartRemoteFetch('/repo/worktrees/a', 'origin')
|
||||
await controller.getOrStartRemoteTrackingBaseRefresh('/repo/worktrees/b', {
|
||||
remote: 'origin',
|
||||
branch: 'main',
|
||||
ref: 'refs/remotes/origin/main',
|
||||
base: 'origin/main'
|
||||
})
|
||||
|
||||
const keys = new Set(armedTargets().map((target) => target.key))
|
||||
expect(keys).toEqual(new Set(['local::/repo/.git']))
|
||||
})
|
||||
|
||||
it('scopes the key to the WSL distro that executes the repo', async () => {
|
||||
const controller = new RuntimeRemoteFetchController()
|
||||
|
||||
await controller.getOrStartRemoteFetch('//wsl$/Ubuntu/repo', 'origin', {
|
||||
wslDistro: 'Ubuntu'
|
||||
})
|
||||
|
||||
expect(armedTargets()[0]?.key).toBe('wsl:Ubuntu::/repo/.git')
|
||||
})
|
||||
|
||||
it('does not collapse every repo onto one key on Git older than 2.31', async () => {
|
||||
// Old Git echoes the unrecognized `--path-format` flag, exits 0, and prints a
|
||||
// relative `.git`; taking that raw would name every repository identically.
|
||||
gitExecFileAsyncMock.mockImplementation(async (argv: string[]) =>
|
||||
argv[0] === 'rev-parse'
|
||||
? { stdout: '--path-format=absolute\n.git\n', stderr: '' }
|
||||
: { stdout: '', stderr: '' }
|
||||
)
|
||||
const controller = new RuntimeRemoteFetchController()
|
||||
|
||||
await controller.getOrStartRemoteFetch('/repo/one', 'origin')
|
||||
await controller.getOrStartRemoteFetch('/repo/two', 'origin')
|
||||
|
||||
expect(armedTargets().map((entry) => entry.key)).toEqual([
|
||||
'local::/repo/one/.git',
|
||||
'local::/repo/two/.git'
|
||||
])
|
||||
})
|
||||
|
||||
it('reports the repo as busy while another fetch on it is in flight', async () => {
|
||||
const controller = new RuntimeRemoteFetchController()
|
||||
await controller.getOrStartRemoteFetch('/repo', 'first')
|
||||
const isBusy = busyProbeFor('local::/repo/.git')
|
||||
expect(isBusy?.()).toBe(false)
|
||||
|
||||
let releaseFetch: (() => void) | undefined
|
||||
gitExecFileAsyncMock.mockImplementation(async (argv: string[]) => {
|
||||
if (argv[0] === 'rev-parse') {
|
||||
return { stdout: '/repo/.git\n', stderr: '' }
|
||||
}
|
||||
await new Promise<void>((resolve) => {
|
||||
releaseFetch = resolve
|
||||
})
|
||||
return { stdout: '', stderr: '' }
|
||||
})
|
||||
const second = controller.getOrStartRemoteFetch('/repo', 'second')
|
||||
await vi.waitFor(() => expect(releaseFetch).toBeDefined())
|
||||
expect(isBusy?.()).toBe(true)
|
||||
|
||||
releaseFetch?.()
|
||||
await second
|
||||
expect(isBusy?.()).toBe(false)
|
||||
})
|
||||
|
||||
it('arms even when the fetch fails, because a partial fetch still writes refs', async () => {
|
||||
const controller = new RuntimeRemoteFetchController()
|
||||
gitExecFileAsyncMock.mockImplementation(async (argv: string[]) => {
|
||||
if (argv[0] === 'rev-parse') {
|
||||
return { stdout: '/repo/.git\n', stderr: '' }
|
||||
}
|
||||
throw new Error('network is unreachable')
|
||||
})
|
||||
vi.spyOn(console, 'warn').mockImplementation(() => {})
|
||||
|
||||
await expect(controller.getOrStartRemoteFetch('/repo', 'origin')).resolves.toEqual({
|
||||
ok: false,
|
||||
errorKind: 'git_error'
|
||||
})
|
||||
expect(armedTargets()).toHaveLength(1)
|
||||
})
|
||||
})
|
||||
@@ -18,6 +18,7 @@ import { AgentSessionTransitionRecorder } from '../stats/agent-session-transitio
|
||||
import { ClaudeUsageStore } from '../claude-usage/store'
|
||||
import { CodexUsageStore } from '../codex-usage/store'
|
||||
import { OpenCodeUsageStore } from '../opencode-usage/store'
|
||||
import { installRepoMaintenanceIdleGate } from '../repo-maintenance-idle-gate'
|
||||
import { mainProcessState as state } from './main-process-state'
|
||||
|
||||
export function initializeMainProcessObservers(): void {
|
||||
@@ -35,6 +36,10 @@ export function initializeMainProcessObservers(): void {
|
||||
)
|
||||
// Why: start from empty — disk-hydrated status rows are UI continuity only; only this runtime's hook events keep the computer awake.
|
||||
state.agentAwakeService.setStatuses([])
|
||||
state.uninstallRepoMaintenanceIdleGate = installRepoMaintenanceIdleGate({
|
||||
isQuitting: () => state.isQuitting,
|
||||
getWorkingAgentCount: () => state.agentAwakeService?.getWorkingAgentCount() ?? 0
|
||||
})
|
||||
const collectChangedProviderSessionWorktrees = createHookProviderSessionInvalidator()
|
||||
const publishProviderSessionChanges = (identities: AgentHookProviderSessionIdentity[]): void => {
|
||||
const ownedIdentities = identities.map((identity) => ({
|
||||
|
||||
@@ -14,6 +14,7 @@ import { clearRuntimeMetadataIfOwned } from '../runtime/runtime-metadata'
|
||||
import { shutdownPairedRuntimeBrowserClientHosts } from '../browser/paired-runtime-browser-client-host-runtime'
|
||||
import { browserManager } from '../browser/browser-manager'
|
||||
import { stopCodexStateDbBackfillRecoveries } from '../codex/codex-state-db-backfill-recovery'
|
||||
import { awaitPackedRefsLockRelease } from '../git/local-repo-ref-maintenance'
|
||||
import { settleTeardownWithinDeadline, settleWithinMs } from '../quit-teardown-deadline'
|
||||
import { quitTeardownStartGate } from '../quit-teardown-start-gate'
|
||||
import { setUnreadDockBadgeCount } from '../dock/unread-badge'
|
||||
@@ -33,6 +34,8 @@ let daemonDisconnectDone = false
|
||||
let watcherShutdownPromise: Promise<void> | null = null
|
||||
// Why 2s: a config delete is best-effort, not durable state.
|
||||
const GROK_HOOK_CLEANUP_DEADLINE_MS = 2_000
|
||||
// Why 2s: long enough for a `pack-refs` child to take SIGTERM and unlink its lock.
|
||||
const REF_MAINTENANCE_QUIT_DEADLINE_MS = 2_000
|
||||
|
||||
function shutdownWatchersOnce(): Promise<void> {
|
||||
if (state.watcherShutdownDone) {
|
||||
@@ -73,6 +76,10 @@ function installBeforeQuitHandler(): void {
|
||||
state.unsubscribeAgentAwakeStatusChanges = null
|
||||
state.agentAwakeService?.dispose()
|
||||
state.agentAwakeService = null
|
||||
// Why wait but not uninstall: a renderer beforeunload can still veto this
|
||||
// quit, and tearing the sweep down here would kill it for the rest of the
|
||||
// session. `isQuitting` already vetoes new attempts; will-quit does the teardown.
|
||||
state.repoMaintenanceShutdown = awaitPackedRefsLockRelease()
|
||||
// Why: defer PTY cleanup to will-quit so the renderer captures scrollback before PTY-exit events unmount TerminalPane (dropping its capture callbacks).
|
||||
state.rateLimits?.stop()
|
||||
})
|
||||
@@ -123,6 +130,16 @@ function installWillQuitHandler(): void {
|
||||
const structuredAgentSessionShutdown = stopStructuredAgentSessionRuntime()
|
||||
state.pluginService = null
|
||||
setUnreadDockBadgeCount(0)
|
||||
// Why wait rather than kill: the child finishes fine orphaned, and signalling
|
||||
// it mid-prune strands a ref lock Git never clears. The wait is only for the
|
||||
// short rewrite window, and is bounded so a quit can never hang on it.
|
||||
const refMaintenanceShutdown = settleWithinMs(
|
||||
Promise.all([state.repoMaintenanceShutdown, state.uninstallRepoMaintenanceIdleGate?.()]).then(
|
||||
() => {}
|
||||
),
|
||||
REF_MAINTENANCE_QUIT_DEADLINE_MS
|
||||
).then(() => {})
|
||||
state.uninstallRepoMaintenanceIdleGate = null
|
||||
agentHookServer.stop()
|
||||
// Why Windows only: POSIX hooks short-circuit on ORCA_PANE_KEY, while Windows must register a
|
||||
// bare script path that cannot express the guard and would otherwise keep spawning after quit.
|
||||
@@ -219,6 +236,7 @@ function installWillQuitHandler(): void {
|
||||
{ name: 'plugin-hosts', promise: pluginHostShutdown },
|
||||
{ name: 'skill-uploads', promise: skillUploadShutdown },
|
||||
{ name: 'grok-hooks', promise: grokHookCleanup },
|
||||
{ name: 'ref-maintenance', promise: refMaintenanceShutdown },
|
||||
{ name: 'codex-backfill-recovery', promise: codexBackfillRecoveryShutdown },
|
||||
{ name: 'structured-agent-session', promise: structuredAgentSessionShutdown },
|
||||
{ name: 'usage-cache', promise: usageCacheFlush },
|
||||
|
||||
@@ -71,6 +71,8 @@ export const mainProcessState = {
|
||||
headlessBrowserDisplayAvailable: false,
|
||||
starNag: null as StarNagService | null,
|
||||
agentAwakeService: null as AgentAwakeService | null,
|
||||
uninstallRepoMaintenanceIdleGate: null as (() => Promise<void>) | null,
|
||||
repoMaintenanceShutdown: Promise.resolve() as Promise<void>,
|
||||
crashReports: null as CrashReportStore | null,
|
||||
unsubscribeAgentAwakeStatusChanges: null as (() => void) | null,
|
||||
publishProviderSessionChanges: null as
|
||||
|
||||
@@ -61,6 +61,11 @@ type ConsumePreparedWorktreeArgs = {
|
||||
|
||||
const preparations = new Map<string, PreparationEntry>()
|
||||
|
||||
/** A prepared checkout is a create that is either in flight or imminent. */
|
||||
export function hasPendingWorktreeCreatePreparations(): boolean {
|
||||
return preparations.size > 0 || staleCleanupInFlight.size > 0
|
||||
}
|
||||
|
||||
function pathOps(path: string): Pick<typeof posix, 'dirname' | 'join' | 'normalize'> {
|
||||
return isWindowsAbsolutePathLike(path) ? win32 : posix
|
||||
}
|
||||
|
||||
@@ -277,6 +277,39 @@ describeBinaryCompatibility('real Git binary compatibility', () => {
|
||||
).rejects.toMatchObject({ code: 1 })
|
||||
})
|
||||
|
||||
it('packs loose refs and reads the maintenance opt-out at the baseline', async () => {
|
||||
// Why: idle ref maintenance runs `pack-refs --all --prune` on every supported
|
||||
// Git rather than the 2.45+ `--auto` form, and reads `maintenance.auto` to
|
||||
// honour a user who disabled Git's own auto-maintenance. Both must work at 2.25.
|
||||
const head = (await runGit(['rev-parse', 'HEAD'])).stdout.trim()
|
||||
const packedRef = 'refs/remotes/origin/compat-pack-refs'
|
||||
await runGit(['update-ref', packedRef, head])
|
||||
await expect(readFile(join(repoPath, '.git', packedRef), 'utf-8')).resolves.toContain(head)
|
||||
|
||||
await expect(runGit(['pack-refs', '--all', '--prune'])).resolves.toBeDefined()
|
||||
|
||||
// The loose file is gone and the ref still resolves through packed-refs.
|
||||
await expect(readFile(join(repoPath, '.git', packedRef), 'utf-8')).rejects.toMatchObject({
|
||||
code: 'ENOENT'
|
||||
})
|
||||
await expect(runGit(['rev-parse', '--verify', packedRef])).resolves.toMatchObject({
|
||||
stdout: `${head}\n`
|
||||
})
|
||||
await expect(readFile(join(repoPath, '.git', 'packed-refs'), 'utf-8')).resolves.toContain(
|
||||
packedRef
|
||||
)
|
||||
|
||||
// `--get` exits 1 on an unset key; that absence must read as consent, not opt-out.
|
||||
await expect(runGit(['config', '--bool', '--get', 'maintenance.auto'])).rejects.toMatchObject({
|
||||
code: 1
|
||||
})
|
||||
await runGit(['config', 'maintenance.auto', 'false'])
|
||||
await expect(runGit(['config', '--bool', '--get', 'maintenance.auto'])).resolves.toMatchObject({
|
||||
stdout: 'false\n'
|
||||
})
|
||||
await runGit(['config', '--unset', 'maintenance.auto'])
|
||||
})
|
||||
|
||||
it('fetches hosted review heads into dedicated refs', async () => {
|
||||
const head = (await runGit(['rev-parse', 'HEAD'])).stdout.trim()
|
||||
await runGit(['update-ref', 'refs/pull/42/head', head])
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
import { mkdir, mkdtemp, rm, symlink, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
// Wraps the real `readdir` so the walk's concurrency is observable without
|
||||
// changing what it reads.
|
||||
const readdirCalls = vi.hoisted(() => ({ outstanding: 0, peak: 0, count: 0 }))
|
||||
|
||||
vi.mock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<Record<string, unknown>>()
|
||||
const realReaddir = actual.readdir as (...args: unknown[]) => Promise<never>
|
||||
return {
|
||||
...actual,
|
||||
readdir: async (...args: unknown[]) => {
|
||||
readdirCalls.outstanding += 1
|
||||
readdirCalls.count += 1
|
||||
readdirCalls.peak = Math.max(readdirCalls.peak, readdirCalls.outstanding)
|
||||
try {
|
||||
return await realReaddir(...args)
|
||||
} finally {
|
||||
readdirCalls.outstanding -= 1
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
import { countLooseRefs } from './loose-ref-count'
|
||||
|
||||
const roots: string[] = []
|
||||
|
||||
async function makeRefsTree(counts: Record<string, number>): Promise<string> {
|
||||
const root = await mkdtemp(join(tmpdir(), 'orca-loose-refs-'))
|
||||
roots.push(root)
|
||||
const refs = join(root, 'refs')
|
||||
for (const [namespace, count] of Object.entries(counts)) {
|
||||
const directory = join(refs, namespace)
|
||||
await mkdir(directory, { recursive: true })
|
||||
for (let index = 0; index < count; index += 1) {
|
||||
await writeFile(join(directory, `ref-${index}`), 'a'.repeat(40))
|
||||
}
|
||||
}
|
||||
await mkdir(refs, { recursive: true })
|
||||
return refs
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
vi.restoreAllMocks()
|
||||
await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true })))
|
||||
})
|
||||
|
||||
describe('countLooseRefs', () => {
|
||||
it('counts files across nested namespaces', async () => {
|
||||
const refs = await makeRefsTree({ heads: 3, 'remotes/origin': 4, 'remotes/fork/deep': 2 })
|
||||
|
||||
await expect(countLooseRefs(refs, 100)).resolves.toEqual({ count: 9, saturated: false })
|
||||
})
|
||||
|
||||
it('stops at the budget instead of walking the whole backlog', async () => {
|
||||
const refs = await makeRefsTree({ 'remotes/origin': 500 })
|
||||
|
||||
const result = await countLooseRefs(refs, 10)
|
||||
|
||||
expect(result).toEqual({ count: 10, saturated: true })
|
||||
})
|
||||
|
||||
it('reports zero for a repository with no refs directory', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'orca-loose-refs-missing-'))
|
||||
roots.push(root)
|
||||
|
||||
await expect(countLooseRefs(join(root, 'refs'), 100)).resolves.toEqual({
|
||||
count: 0,
|
||||
saturated: false
|
||||
})
|
||||
})
|
||||
|
||||
it('never has more than one directory read outstanding', async () => {
|
||||
// libuv's filesystem thread pool has four slots shared with the whole main
|
||||
// process. A probe that fanned out would stall unrelated fs work, so this
|
||||
// pins the walk as strictly sequential rather than merely bounded.
|
||||
const refs = await makeRefsTree({
|
||||
'remotes/a': 3,
|
||||
'remotes/b': 3,
|
||||
'remotes/c': 3,
|
||||
'remotes/d': 3,
|
||||
'remotes/e/deep': 3
|
||||
})
|
||||
readdirCalls.peak = 0
|
||||
readdirCalls.count = 0
|
||||
|
||||
await countLooseRefs(refs, 1000)
|
||||
|
||||
expect(readdirCalls.count).toBeGreaterThan(1)
|
||||
expect(readdirCalls.peak).toBe(1)
|
||||
})
|
||||
|
||||
it('reads each directory once rather than streaming it in batches', async () => {
|
||||
// One thread-pool round trip per directory is what makes the probe ~8x
|
||||
// cheaper than the streaming form on a real degraded repository.
|
||||
const refs = await makeRefsTree({ 'remotes/origin': 400 })
|
||||
readdirCalls.count = 0
|
||||
|
||||
await countLooseRefs(refs, 1000)
|
||||
|
||||
// refs/ plus refs/remotes plus refs/remotes/origin.
|
||||
expect(readdirCalls.count).toBe(3)
|
||||
})
|
||||
|
||||
it('does not follow directory symlinks into a loop', async () => {
|
||||
const refs = await makeRefsTree({ heads: 2 })
|
||||
await symlink(refs, join(refs, 'loop'), 'dir')
|
||||
|
||||
const result = await countLooseRefs(refs, 100)
|
||||
|
||||
expect(result.saturated).toBe(false)
|
||||
// The symlink is one dirent, never a second traversal of the tree.
|
||||
expect(result.count).toBe(3)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,80 @@
|
||||
import { readdir } from 'node:fs/promises'
|
||||
import { join } from 'node:path'
|
||||
|
||||
export type LooseRefCount = {
|
||||
/** Loose ref files seen, never above `budget`. */
|
||||
count: number
|
||||
/** The walk stopped early, so `count` is a floor rather than the total. */
|
||||
saturated: boolean
|
||||
}
|
||||
|
||||
// Why: a ref tree is shallow and wide; this bounds both the directories visited
|
||||
// and the queue holding those still to visit, so neither a symlink loop nor a
|
||||
// pathological repo turns a gate probe into an unbounded walk.
|
||||
const DIRECTORY_VISIT_CEILING = 4096
|
||||
|
||||
/**
|
||||
* Count loose refs under a repository's `refs/` directory, stopping at `budget`.
|
||||
*
|
||||
* Deliberately budgeted: callers use this as an admission gate, so the cost has
|
||||
* to be bounded by the threshold being tested and not by the size of the
|
||||
* backlog it is testing for.
|
||||
*
|
||||
* One `readdir` per directory, dirents only -- no `stat` per entry, and no
|
||||
* `opendir` streaming. Measured against a real 36,600-loose-ref repository, the
|
||||
* batched form is ~8x faster (23ms vs 177ms median to reach a 1000 threshold)
|
||||
* and holds the event loop for less than half as long, because streaming issues
|
||||
* a thread-pool round trip every 32 entries where this issues one per
|
||||
* directory. The cost is holding one directory's dirents at a time, which is
|
||||
* bounded by the widest ref namespace rather than by the size of the tree.
|
||||
*
|
||||
* Strictly sequential on purpose: it awaits one directory before opening the
|
||||
* next, so it can never occupy more than one of libuv's four thread-pool slots
|
||||
* and cannot stall unrelated main-process filesystem work.
|
||||
*
|
||||
* `signal` stops the walk between directories. A single hung `readdir` is not
|
||||
* interruptible, but it holds no Git lock, so it delays only maintenance.
|
||||
*/
|
||||
export async function countLooseRefs(
|
||||
refsDirectory: string,
|
||||
budget: number,
|
||||
signal?: AbortSignal
|
||||
): Promise<LooseRefCount> {
|
||||
const pending = [refsDirectory]
|
||||
let count = 0
|
||||
let visited = 0
|
||||
while (pending.length > 0) {
|
||||
const directory = pending.pop()
|
||||
if (directory === undefined) {
|
||||
break
|
||||
}
|
||||
visited += 1
|
||||
// A cancelled walk reports what it saw as a floor rather than throwing; callers
|
||||
// already have to treat a saturated result as "not known to be clean".
|
||||
if (
|
||||
signal?.aborted === true ||
|
||||
visited > DIRECTORY_VISIT_CEILING ||
|
||||
pending.length > DIRECTORY_VISIT_CEILING
|
||||
) {
|
||||
return { count, saturated: true }
|
||||
}
|
||||
let entries: { name: string; isDirectory: () => boolean }[]
|
||||
try {
|
||||
entries = await readdir(directory, { withFileTypes: true })
|
||||
} catch {
|
||||
// A missing or unreadable namespace contributes nothing to the count.
|
||||
continue
|
||||
}
|
||||
for (const entry of entries) {
|
||||
if (entry.isDirectory()) {
|
||||
pending.push(join(directory, entry.name))
|
||||
continue
|
||||
}
|
||||
count += 1
|
||||
if (count >= budget) {
|
||||
return { count, saturated: true }
|
||||
}
|
||||
}
|
||||
}
|
||||
return { count, saturated: false }
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
/**
|
||||
* Tracks the one window in a pack that actually excludes anybody: the
|
||||
* `packed-refs` rewrite. Callers about to touch refs wait this out rather than
|
||||
* killing the child, because a signal delivered into the prune phase strands a
|
||||
* `refs/**\/*.lock` roughly one time in five and Git never clears those.
|
||||
*/
|
||||
export class PackedRefsLockGate {
|
||||
private held = false
|
||||
private waiters: (() => void)[] = []
|
||||
|
||||
setHeld(held: boolean): void {
|
||||
this.held = held
|
||||
if (held) {
|
||||
return
|
||||
}
|
||||
const waiting = this.waiters
|
||||
this.waiters = []
|
||||
for (const resolve of waiting) {
|
||||
resolve()
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolves on release, or on `timeoutMs` -- past which Git's own retry is the better bet. */
|
||||
whenReleased(timeoutMs: number): Promise<void> {
|
||||
if (!this.held) {
|
||||
return Promise.resolve()
|
||||
}
|
||||
return new Promise<void>((resolve) => {
|
||||
let settled = false
|
||||
const finish = (): void => {
|
||||
if (settled) {
|
||||
return
|
||||
}
|
||||
settled = true
|
||||
clearTimeout(timer)
|
||||
resolve()
|
||||
}
|
||||
const timer = setTimeout(finish, timeoutMs)
|
||||
timer.unref?.()
|
||||
this.waiters.push(finish)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,166 @@
|
||||
/**
|
||||
* Idle-time loose-ref packing for repositories Orca itself degrades.
|
||||
*
|
||||
* Orca strips git's auto-maintenance off its own frequent fetches
|
||||
* (`GIT_FETCH_SKIP_AUTO_MAINTENANCE_CONFIG_ARGS`) and never compensated, so an
|
||||
* Orca-driven checkout accumulates loose refs forever and every ref
|
||||
* enumeration -- `show-ref`, `for-each-ref`, worktree create -- pays for them.
|
||||
* This is the compensation: after a repo goes quiet, probe it, and pack only
|
||||
* when the backlog is real.
|
||||
*
|
||||
* The engine is host-agnostic on purpose. The execution host owns everything
|
||||
* that touches execution, so each host supplies its own target (which git to
|
||||
* run, which filesystem to walk) and all state here is keyed per host.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Below this, ref enumeration is already fast and `pack-refs` would cost more
|
||||
* than it saves.
|
||||
*
|
||||
* Git's own files-backend auto heuristic (2.47+) packs at
|
||||
* `max(16, log2(packed_refs_bytes / 100) * 5)` loose refs -- about 76 for the
|
||||
* 4.1 MB `packed-refs` that motivated this work. A flat 1000 is roughly an
|
||||
* order of magnitude more conservative on purpose: this runs unasked against a
|
||||
* real checkout, and being late is cheap where being wrong is not.
|
||||
*/
|
||||
export const LOOSE_REF_PACK_THRESHOLD = 1000
|
||||
|
||||
/** No fetch, create, or other tracked write on the repo for this long. */
|
||||
export const REF_MAINTENANCE_QUIET_PERIOD_MS = 10 * 60_000
|
||||
|
||||
/** Packing empties the backlog; there is nothing to do again for a long while. */
|
||||
export const REF_MAINTENANCE_PACKED_COOLDOWN_MS = 12 * 60 * 60_000
|
||||
|
||||
/** A healthy or unresolvable repo should not be re-probed on every quiet window. */
|
||||
export const REF_MAINTENANCE_CLEAN_COOLDOWN_MS = 6 * 60 * 60_000
|
||||
|
||||
/** A failing repo (permissions, stale lock) must not be retried in a loop. */
|
||||
export const REF_MAINTENANCE_FAILURE_COOLDOWN_MS = 6 * 60 * 60_000
|
||||
|
||||
/**
|
||||
* A repository whose `packed-refs.lock` is a strand from our own dead process
|
||||
* becomes reclaimable at `PACK_REFS_TIMEOUT_MS`, so retry near that rather than
|
||||
* serving the full failure cooldown -- otherwise a Windows force-kill leaves
|
||||
* every ref deletion in that repo failing for six hours instead of thirty
|
||||
* minutes.
|
||||
*/
|
||||
export const REF_MAINTENANCE_LOCKED_COOLDOWN_MS = 30 * 60_000
|
||||
|
||||
/**
|
||||
* `pack-refs` holds `packed-refs.lock` only while it rewrites the file --
|
||||
* measured at 0.03-1.37s of a 23-32s run, the other ~95% being the prune phase
|
||||
* unlinking loose refs. A caller about to touch refs waits out that window
|
||||
* instead of killing the pack.
|
||||
*/
|
||||
export const PACKED_REFS_LOCK_POLL_MS = 50
|
||||
|
||||
/**
|
||||
* Ceiling on that wait. Past this we stop blocking the user and let Git's own
|
||||
* retry (`core.filesRefLockTimeout`, `core.packedRefsTimeout`) handle it, which
|
||||
* is what happens today without any of this.
|
||||
*/
|
||||
export const PACKED_REFS_LOCK_WAIT_MS = 5_000
|
||||
|
||||
/**
|
||||
* `pack-refs --prune` unlinks one file per loose ref. Paying off a 36k-ref
|
||||
* backlog measured at ~83s on APFS, so the deadline has to clear a cold repo on
|
||||
* a slow disk by a wide margin. A kill mid-run is safe -- git renames
|
||||
* `packed-refs` into place atomically and the surviving loose refs stay
|
||||
* authoritative -- but it wastes the work.
|
||||
*/
|
||||
export const PACK_REFS_TIMEOUT_MS = 15 * 60_000
|
||||
|
||||
/**
|
||||
* Ancient, safe on the Git 2.25 baseline, and does exactly one thing.
|
||||
*
|
||||
* Not `pack-refs --auto`: that arrived in 2.45 and unconditionally rewrote
|
||||
* `packed-refs` on the files backend until 2.47, so it is both unavailable at
|
||||
* our baseline and wrong on two shipped releases. Not `git maintenance run`
|
||||
* either -- newer, and it pulls in commit-graph and repack work we did not ask
|
||||
* for. `--all` is required because the backlog is `refs/heads` and
|
||||
* `refs/remotes`, which a bare `pack-refs` leaves alone.
|
||||
*/
|
||||
export const PACK_REFS_ARGS = ['pack-refs', '--all', '--prune'] as const
|
||||
|
||||
/**
|
||||
* Backstop on a whole attempt: aborts it, rather than abandoning it. Every Git
|
||||
* child is already deadlined, but an admission wait is not, and the whole app
|
||||
* shares one maintenance slot. Abandoning would release that slot while a pack
|
||||
* that may still hold `packed-refs.lock` runs on, so the deadline cancels the
|
||||
* work instead and the slot is held until it really stops.
|
||||
*/
|
||||
export const REF_MAINTENANCE_ATTEMPT_DEADLINE_MS = PACK_REFS_TIMEOUT_MS + 5 * 60_000
|
||||
|
||||
export type RefMaintenanceOutcome =
|
||||
| 'packed'
|
||||
| 'below_threshold'
|
||||
| 'unresolved'
|
||||
| 'opted_out'
|
||||
| 'deferred'
|
||||
| 'interrupted'
|
||||
| 'locked'
|
||||
| 'timed_out'
|
||||
| 'failed'
|
||||
|
||||
/** Structurally satisfied by the tracer's `ActiveSpan`. */
|
||||
export type RefMaintenanceSpan = {
|
||||
setAttribute(key: string, value: unknown): void
|
||||
}
|
||||
|
||||
export type RepoRefMaintenanceTarget = {
|
||||
/** Repo identity scoped to its execution host; all state here is keyed by it. */
|
||||
readonly key: string
|
||||
/** Absolute `refs/` path *on the host that runs the walk*, or undefined if unresolvable. */
|
||||
resolveRefsDirectory(signal: AbortSignal): Promise<string | undefined>
|
||||
/** A user who told Git not to auto-maintain this repo has told Orca too. */
|
||||
isOptedOut?(signal: AbortSignal): Promise<boolean>
|
||||
/** True while work on *this repo* is in flight -- a fetch, a create, a removal. */
|
||||
isBusy?(): boolean
|
||||
/**
|
||||
* Runs `pack-refs` to completion. Deliberately takes no abort signal: killing
|
||||
* a pack is measurably worse than waiting for it (see `PACKED_REFS_LOCK_*`).
|
||||
* It must report `packed-refs.lock` transitions through `lock` so callers can
|
||||
* wait for the short window that actually blocks them.
|
||||
*/
|
||||
packRefs(lock: PackedRefsLockReporter): Promise<void>
|
||||
}
|
||||
|
||||
/** How `packRefs` tells the scheduler whether the exclusive write window is open. */
|
||||
export type PackedRefsLockReporter = {
|
||||
setHeld(held: boolean): void
|
||||
}
|
||||
|
||||
export type RepoRefMaintenanceOptions = {
|
||||
now?: () => number
|
||||
/** True while app-wide work this must not race is in flight (create, live agent, battery, quit). */
|
||||
isBusy?: () => boolean
|
||||
/** Wraps one attempt so a host can trace it; must invoke and await `attempt`. */
|
||||
observe?: (attempt: (span: RefMaintenanceSpan) => Promise<void>) => Promise<void>
|
||||
quietPeriodMs?: number
|
||||
looseRefThreshold?: number
|
||||
onError?: (error: unknown) => void
|
||||
}
|
||||
|
||||
/** Marks an abort Orca asked for, so the attempt is retried rather than blamed on the repo. */
|
||||
export class RefMaintenanceInterrupted extends Error {
|
||||
constructor(
|
||||
reason: string,
|
||||
/** True when the attempt ran out of time rather than yielding to real work. */
|
||||
readonly deadline = false
|
||||
) {
|
||||
super(`Ref maintenance interrupted: ${reason}`)
|
||||
this.name = 'RefMaintenanceInterrupted'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The repository's `packed-refs.lock` is held by something we must not touch.
|
||||
* Distinct from a failure so a strand our own dead process left can be retried
|
||||
* once it ages into reclaimability, rather than parked for six hours.
|
||||
*/
|
||||
export class RefMaintenanceRepoLocked extends Error {
|
||||
constructor(detail: string) {
|
||||
super(`packed-refs.lock is held: ${detail}`)
|
||||
this.name = 'RefMaintenanceRepoLocked'
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,530 @@
|
||||
import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { RepoRefMaintenance } from './repo-ref-maintenance'
|
||||
import {
|
||||
RefMaintenanceRepoLocked,
|
||||
REF_MAINTENANCE_PACKED_COOLDOWN_MS,
|
||||
PACKED_REFS_LOCK_WAIT_MS,
|
||||
type PackedRefsLockReporter,
|
||||
type RefMaintenanceSpan,
|
||||
type RepoRefMaintenanceOptions,
|
||||
type RepoRefMaintenanceTarget
|
||||
} from './repo-ref-maintenance-policy'
|
||||
|
||||
const QUIET_MS = 1000
|
||||
const THRESHOLD = 5
|
||||
const roots: string[] = []
|
||||
|
||||
async function refsDirectoryWith(looseRefs: number): Promise<string> {
|
||||
const root = await mkdtemp(join(tmpdir(), 'orca-ref-maintenance-'))
|
||||
roots.push(root)
|
||||
const refs = join(root, 'refs', 'remotes', 'origin')
|
||||
await mkdir(refs, { recursive: true })
|
||||
for (let index = 0; index < looseRefs; index += 1) {
|
||||
await writeFile(join(refs, `ref-${index}`), 'a')
|
||||
}
|
||||
return join(root, 'refs')
|
||||
}
|
||||
|
||||
/** More directories than `countLooseRefs` will visit, but very few files. */
|
||||
async function saturatingRefsDirectory(): Promise<string> {
|
||||
const root = await mkdtemp(join(tmpdir(), 'orca-ref-maintenance-wide-'))
|
||||
roots.push(root)
|
||||
const refs = join(root, 'refs')
|
||||
for (let index = 0; index < 4200; index += 1) {
|
||||
await mkdir(join(refs, `ns-${index}`), { recursive: true })
|
||||
}
|
||||
return refs
|
||||
}
|
||||
|
||||
/** Stands in for a pack that moved the refs into packed-refs before erroring. */
|
||||
async function emptyRefsDirectory(refs: string): Promise<void> {
|
||||
await rm(refs, { recursive: true, force: true })
|
||||
}
|
||||
|
||||
function attributesOf(span: RefMaintenanceSpan): Record<string, unknown> {
|
||||
return (span as unknown as { recorded: Record<string, unknown> }).recorded
|
||||
}
|
||||
|
||||
function recordingSpan(): RefMaintenanceSpan {
|
||||
const recorded: Record<string, unknown> = {}
|
||||
return {
|
||||
recorded,
|
||||
setAttribute(key: string, value: unknown) {
|
||||
recorded[key] = value
|
||||
}
|
||||
} as unknown as RefMaintenanceSpan
|
||||
}
|
||||
|
||||
type Harness = {
|
||||
maintenance: RepoRefMaintenance
|
||||
spans: RefMaintenanceSpan[]
|
||||
packRefs: ((lock: PackedRefsLockReporter) => Promise<void>) & { mock: { calls: unknown[] } }
|
||||
}
|
||||
|
||||
function createHarness(
|
||||
overrides: Partial<RepoRefMaintenanceOptions> & {
|
||||
packRefs?: (lock: PackedRefsLockReporter) => Promise<void>
|
||||
} = {}
|
||||
): Harness {
|
||||
const spans: RefMaintenanceSpan[] = []
|
||||
const packRefs = vi.fn<(lock: PackedRefsLockReporter) => Promise<void>>(
|
||||
overrides.packRefs ?? (async () => {})
|
||||
)
|
||||
const maintenance = new RepoRefMaintenance({
|
||||
quietPeriodMs: QUIET_MS,
|
||||
looseRefThreshold: THRESHOLD,
|
||||
now: () => Date.now(),
|
||||
observe: (attempt) => {
|
||||
const span = recordingSpan()
|
||||
spans.push(span)
|
||||
return attempt(span)
|
||||
},
|
||||
...overrides
|
||||
})
|
||||
return { maintenance, spans, packRefs }
|
||||
}
|
||||
|
||||
function target(
|
||||
key: string,
|
||||
refsDirectory: string,
|
||||
packRefs: (lock: PackedRefsLockReporter) => Promise<void>,
|
||||
extra: Partial<RepoRefMaintenanceTarget> = {}
|
||||
): RepoRefMaintenanceTarget {
|
||||
return {
|
||||
key,
|
||||
resolveRefsDirectory: async () => refsDirectory,
|
||||
packRefs,
|
||||
...extra
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolves the first time the pack starts, so tests never race real filesystem I/O. */
|
||||
function packStartSignal(): {
|
||||
started: Promise<PackedRefsLockReporter>
|
||||
onStart: (lock: PackedRefsLockReporter) => void
|
||||
} {
|
||||
let onStart: (lock: PackedRefsLockReporter) => void = () => {}
|
||||
const started = new Promise<PackedRefsLockReporter>((resolve) => {
|
||||
onStart = resolve
|
||||
})
|
||||
return { started, onStart }
|
||||
}
|
||||
|
||||
function yieldToIo(): Promise<void> {
|
||||
return new Promise<void>((resolve) => setImmediate(resolve))
|
||||
}
|
||||
|
||||
/**
|
||||
* Spins the real event loop until `predicate` holds, so filesystem completions
|
||||
* can land while `setTimeout` is faked. Bounded by wall clock rather than by a
|
||||
* turn count: a loaded CI runner exhausts a fixed number of turns long before
|
||||
* the I/O finishes, which fails as a confusing assertion somewhere else.
|
||||
*/
|
||||
async function until(predicate: () => boolean, what: string): Promise<void> {
|
||||
const deadline = Date.now() + 10_000
|
||||
while (!predicate() && Date.now() < deadline) {
|
||||
await yieldToIo()
|
||||
}
|
||||
if (!predicate()) {
|
||||
throw new Error(`timed out after 10s waiting for ${what}`)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Like `until`, but for conditions that also need a scheduled retry to fire:
|
||||
* spinning the real loop alone can never satisfy them, because `setTimeout` is
|
||||
* faked. Alternates advancing the fake clock with yielding to real I/O.
|
||||
*/
|
||||
async function untilWithTimers(predicate: () => boolean, what: string): Promise<void> {
|
||||
const deadline = Date.now() + 10_000
|
||||
while (!predicate() && Date.now() < deadline) {
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS)
|
||||
await yieldToIo()
|
||||
}
|
||||
if (!predicate()) {
|
||||
throw new Error(`timed out after 10s waiting for ${what}`)
|
||||
}
|
||||
}
|
||||
|
||||
/** Fires the quiet-period timer and waits for the attempt it starts. */
|
||||
async function elapseQuietPeriod(maintenance: RepoRefMaintenance, periods = 1): Promise<void> {
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS * periods)
|
||||
await maintenance.whenAttemptSettled()
|
||||
}
|
||||
|
||||
/** Only the quiet-period timer is faked; real filesystem I/O still has to complete. */
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] })
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
vi.useRealTimers()
|
||||
await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true })))
|
||||
})
|
||||
|
||||
describe('RepoRefMaintenance gating', () => {
|
||||
it('packs only after the repo has been quiet for the full period', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, spans, packRefs } = createHarness()
|
||||
const repo = target('local::/repo/.git', refs, packRefs)
|
||||
|
||||
maintenance.arm(repo)
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS - 1)
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
|
||||
// A second write restarts the countdown rather than shortening it.
|
||||
maintenance.arm(repo)
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS - 1)
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
|
||||
await elapseQuietPeriod(maintenance)
|
||||
expect(packRefs).toHaveBeenCalledTimes(1)
|
||||
expect(attributesOf(spans[0])).toMatchObject({
|
||||
'repo.maintenance_outcome': 'packed',
|
||||
'repo.maintenance_key': 'local::/repo/.git',
|
||||
'git.loose_ref_count': THRESHOLD + 1
|
||||
})
|
||||
})
|
||||
|
||||
it('leaves a healthy repository alone', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD - 1)
|
||||
const { maintenance, spans, packRefs } = createHarness()
|
||||
|
||||
maintenance.arm(target('local::/healthy/.git', refs, packRefs))
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
expect(attributesOf(spans[0])).toMatchObject({
|
||||
'repo.maintenance_outcome': 'below_threshold',
|
||||
'git.loose_ref_count': THRESHOLD - 1
|
||||
})
|
||||
})
|
||||
|
||||
it('does not run while the app is busy', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
let busy = true
|
||||
const { maintenance, packRefs } = createHarness({ isBusy: () => busy })
|
||||
|
||||
maintenance.arm(target('local::/busy/.git', refs, packRefs))
|
||||
await elapseQuietPeriod(maintenance)
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
|
||||
// The deferral re-arms on a backed-off delay, so the next window picks it up.
|
||||
busy = false
|
||||
await elapseQuietPeriod(maintenance, 2)
|
||||
expect(packRefs).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('does not run while the repo itself has work in flight', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, packRefs } = createHarness()
|
||||
|
||||
maintenance.arm(target('local::/fetching/.git', refs, packRefs, { isBusy: () => true }))
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('honours a user who disabled Git auto-maintenance for the repo', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, spans, packRefs } = createHarness()
|
||||
|
||||
maintenance.arm(
|
||||
target('local::/opted-out/.git', refs, packRefs, { isOptedOut: async () => true })
|
||||
)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
expect(attributesOf(spans[0])['repo.maintenance_outcome']).toBe('opted_out')
|
||||
})
|
||||
|
||||
it('never reads a truncated walk as a clean repository', async () => {
|
||||
const { maintenance, spans, packRefs } = createHarness()
|
||||
|
||||
// A walk that stopped early reports a floor, so a low count is not evidence of health.
|
||||
maintenance.arm({
|
||||
key: 'local::/saturated/.git',
|
||||
resolveRefsDirectory: async () => saturatingRefsDirectory(),
|
||||
packRefs
|
||||
})
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(packRefs).toHaveBeenCalledTimes(1)
|
||||
expect(attributesOf(spans[0])['repo.maintenance_outcome']).toBe('packed')
|
||||
})
|
||||
|
||||
it('records a repo whose packed-refs lock is held, and retries sooner than a failure', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, spans } = createHarness()
|
||||
|
||||
maintenance.arm(
|
||||
target('local::/locked/.git', refs, async () => {
|
||||
throw new RefMaintenanceRepoLocked('our own lock, not yet old enough to reclaim')
|
||||
})
|
||||
)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(attributesOf(spans[0])['repo.maintenance_outcome']).toBe('locked')
|
||||
})
|
||||
|
||||
it('skips a repository whose common dir cannot be resolved', async () => {
|
||||
const { maintenance, spans, packRefs } = createHarness()
|
||||
|
||||
maintenance.arm({
|
||||
key: 'local::/gone/.git',
|
||||
resolveRefsDirectory: async () => undefined,
|
||||
packRefs
|
||||
})
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
expect(attributesOf(spans[0])['repo.maintenance_outcome']).toBe('unresolved')
|
||||
})
|
||||
})
|
||||
|
||||
describe('RepoRefMaintenance single-flight and backoff', () => {
|
||||
it('runs one repository at a time', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
let concurrent = 0
|
||||
let peak = 0
|
||||
const releases: (() => void)[] = []
|
||||
const { maintenance } = createHarness()
|
||||
const slowPack = async (): Promise<void> => {
|
||||
concurrent += 1
|
||||
peak = Math.max(peak, concurrent)
|
||||
await new Promise<void>((resolve) => releases.push(resolve))
|
||||
concurrent -= 1
|
||||
}
|
||||
|
||||
maintenance.arm(target('local::/a/.git', refs, slowPack))
|
||||
maintenance.arm(target('local::/b/.git', refs, slowPack))
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS)
|
||||
await until(() => concurrent === 1, 'a pack to start')
|
||||
expect(concurrent).toBe(1)
|
||||
|
||||
releases.shift()?.()
|
||||
await maintenance.whenAttemptSettled()
|
||||
// The second repo was deferred behind the first, so its retry is on a timer.
|
||||
await untilWithTimers(() => concurrent === 1, 'the second repo to start')
|
||||
releases.shift()?.()
|
||||
await maintenance.whenAttemptSettled()
|
||||
|
||||
expect(peak).toBe(1)
|
||||
expect(concurrent).toBe(0)
|
||||
})
|
||||
|
||||
it('waits out the rewrite window instead of killing the pack', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
let finished = false
|
||||
let release: (() => void) | undefined
|
||||
const { maintenance } = createHarness()
|
||||
const started = packStartSignal()
|
||||
|
||||
maintenance.arm(
|
||||
target('local::/yield/.git', refs, async (lock) => {
|
||||
lock.setHeld(true)
|
||||
started.onStart(lock)
|
||||
await new Promise<void>((resolve) => {
|
||||
release = () => {
|
||||
lock.setHeld(false)
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
finished = true
|
||||
})
|
||||
)
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS)
|
||||
await started.started
|
||||
|
||||
let paused = false
|
||||
void maintenance.pause('worktree-remove').then(() => {
|
||||
paused = true
|
||||
})
|
||||
await vi.advanceTimersByTimeAsync(1)
|
||||
// Blocked while the rewrite window is open...
|
||||
expect(paused).toBe(false)
|
||||
expect(finished).toBe(false)
|
||||
|
||||
release?.()
|
||||
await until(() => paused, 'pause() to resolve')
|
||||
// ...and released without the pack ever being cancelled.
|
||||
expect(paused).toBe(true)
|
||||
expect(finished).toBe(true)
|
||||
})
|
||||
|
||||
it('gives up waiting on the lock rather than blocking the user indefinitely', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance } = createHarness()
|
||||
const started = packStartSignal()
|
||||
|
||||
maintenance.arm(
|
||||
target('local::/stuck-lock/.git', refs, async (lock) => {
|
||||
lock.setHeld(true)
|
||||
started.onStart(lock)
|
||||
await new Promise<void>(() => {})
|
||||
})
|
||||
)
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS)
|
||||
await started.started
|
||||
|
||||
let paused = false
|
||||
void maintenance.pause('git-fetch').then(() => {
|
||||
paused = true
|
||||
})
|
||||
await vi.advanceTimersByTimeAsync(PACKED_REFS_LOCK_WAIT_MS)
|
||||
await until(() => paused, 'pause() to resolve')
|
||||
|
||||
expect(paused).toBe(true)
|
||||
})
|
||||
|
||||
it('reopens the window only when the last overlapping caller releases', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, packRefs } = createHarness()
|
||||
|
||||
const outer = await maintenance.pause('worktree-add')
|
||||
const inner = await maintenance.pause('git-fetch')
|
||||
maintenance.arm(target('local::/nested/.git', refs, packRefs))
|
||||
|
||||
await elapseQuietPeriod(maintenance, 8)
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
|
||||
inner()
|
||||
await elapseQuietPeriod(maintenance, 8)
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
|
||||
outer()
|
||||
await elapseQuietPeriod(maintenance, 8)
|
||||
expect(packRefs).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('restarts every armed countdown when the user does ref work themselves', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, packRefs } = createHarness()
|
||||
|
||||
const firstPack = packStartSignal()
|
||||
const observed = target('local::/a/.git', refs, async (lock) => {
|
||||
firstPack.onStart(lock)
|
||||
await packRefs(lock)
|
||||
})
|
||||
maintenance.arm(observed)
|
||||
maintenance.arm(target('local::/b/.git', refs, packRefs))
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS - 1)
|
||||
|
||||
// A manual fetch says the user is at the keyboard, so nothing may fire yet.
|
||||
maintenance.postponeAll()
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS - 1)
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
|
||||
await vi.advanceTimersByTimeAsync(QUIET_MS)
|
||||
await firstPack.started
|
||||
expect(packRefs).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('costs nothing when no pack is running', async () => {
|
||||
const { maintenance } = createHarness()
|
||||
|
||||
const release = await maintenance.pause('git-fetch')
|
||||
release()
|
||||
// Releasing twice must not leave the window wedged shut.
|
||||
release()
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { packRefs } = createHarness()
|
||||
maintenance.arm(target('local::/free/.git', refs, packRefs))
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(packRefs).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('does not re-pack a repository inside its cooldown', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
let clock = 0
|
||||
const { maintenance, packRefs } = createHarness({ now: () => clock })
|
||||
const repo = target('local::/cooldown/.git', refs, packRefs)
|
||||
|
||||
maintenance.arm(repo)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
expect(packRefs).toHaveBeenCalledTimes(1)
|
||||
|
||||
clock = REF_MAINTENANCE_PACKED_COOLDOWN_MS - 1
|
||||
maintenance.arm(repo)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
expect(packRefs).toHaveBeenCalledTimes(1)
|
||||
|
||||
clock = REF_MAINTENANCE_PACKED_COOLDOWN_MS + 1
|
||||
maintenance.arm(repo)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
expect(packRefs).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('counts a pack that could not lock every ref as a success', async () => {
|
||||
// Field-observed on a machine running several Orca sessions: a branch moved
|
||||
// mid-pack, Git reported an error, and 36,688 loose refs still became 3.
|
||||
// Retrying that aggressively would be wrong -- the backlog is gone.
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, spans } = createHarness()
|
||||
const repo = target('local::/raced/.git', refs, async () => {
|
||||
await emptyRefsDirectory(refs)
|
||||
throw new Error("error: cannot lock ref 'refs/heads/moved'")
|
||||
})
|
||||
|
||||
maintenance.arm(repo)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(attributesOf(spans[0])).toMatchObject({
|
||||
'repo.maintenance_outcome': 'packed',
|
||||
'git.pack_refs_partial': true,
|
||||
'git.loose_ref_count_after': 0
|
||||
})
|
||||
|
||||
// And it serves the full post-pack cooldown rather than retrying.
|
||||
maintenance.arm(repo)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
expect(spans).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('records a failure when the pack left the backlog in place', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, spans } = createHarness({
|
||||
packRefs: async () => {
|
||||
throw new Error('permission denied')
|
||||
}
|
||||
})
|
||||
|
||||
maintenance.arm(target('local::/denied/.git', refs, () => Promise.reject(new Error('denied'))))
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(attributesOf(spans[0])['repo.maintenance_outcome']).toBe('failed')
|
||||
})
|
||||
|
||||
it('records a failure instead of throwing, and backs off', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, spans, packRefs } = createHarness({
|
||||
packRefs: async () => {
|
||||
throw new Error('packed-refs.lock exists')
|
||||
}
|
||||
})
|
||||
const repo = target('local::/failing/.git', refs, packRefs)
|
||||
|
||||
maintenance.arm(repo)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
expect(attributesOf(spans[0])['repo.maintenance_outcome']).toBe('failed')
|
||||
|
||||
maintenance.arm(repo)
|
||||
await elapseQuietPeriod(maintenance)
|
||||
expect(packRefs).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('stops scheduling once disposed', async () => {
|
||||
const refs = await refsDirectoryWith(THRESHOLD + 2)
|
||||
const { maintenance, packRefs } = createHarness()
|
||||
|
||||
maintenance.arm(target('local::/disposed/.git', refs, packRefs))
|
||||
maintenance.dispose()
|
||||
await elapseQuietPeriod(maintenance)
|
||||
|
||||
expect(packRefs).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,366 @@
|
||||
import { countLooseRefs } from './loose-ref-count'
|
||||
import { PackedRefsLockGate } from './packed-refs-lock-gate'
|
||||
import {
|
||||
LOOSE_REF_PACK_THRESHOLD,
|
||||
REF_MAINTENANCE_ATTEMPT_DEADLINE_MS,
|
||||
REF_MAINTENANCE_CLEAN_COOLDOWN_MS,
|
||||
REF_MAINTENANCE_FAILURE_COOLDOWN_MS,
|
||||
REF_MAINTENANCE_PACKED_COOLDOWN_MS,
|
||||
REF_MAINTENANCE_QUIET_PERIOD_MS,
|
||||
PACKED_REFS_LOCK_WAIT_MS,
|
||||
REF_MAINTENANCE_LOCKED_COOLDOWN_MS,
|
||||
RefMaintenanceInterrupted,
|
||||
RefMaintenanceRepoLocked,
|
||||
type RefMaintenanceOutcome,
|
||||
type RefMaintenanceSpan,
|
||||
type RepoRefMaintenanceOptions,
|
||||
type RepoRefMaintenanceTarget
|
||||
} from './repo-ref-maintenance-policy'
|
||||
|
||||
/**
|
||||
* The scheduler half of idle loose-ref packing: when to probe, when to pack,
|
||||
* when to stand down. The thresholds and the host contract it works against
|
||||
* live in `./repo-ref-maintenance-policy`.
|
||||
*/
|
||||
|
||||
/** Give up until the next real activity rather than re-arming forever. */
|
||||
const MAX_DEFERRALS = 6
|
||||
/** Each deferral doubles the wait, so a busy app is retried rarely, not hammered. */
|
||||
const MAX_DEFERRAL_BACKOFF_MULTIPLIER = 8
|
||||
/** Armed repos are evicted oldest-first past this; the next write on one re-arms it. */
|
||||
const MAX_TRACKED_REPOS = 64
|
||||
|
||||
type TrackedRepo = {
|
||||
target: RepoRefMaintenanceTarget
|
||||
timer: ReturnType<typeof setTimeout> | null
|
||||
deferrals: number
|
||||
}
|
||||
|
||||
const noopSpan: RefMaintenanceSpan = { setAttribute: () => {} }
|
||||
|
||||
/** A deadline means something is stuck: back off instead of retrying straight away. */
|
||||
function hitDeadline(signal: AbortSignal): boolean {
|
||||
return signal.reason instanceof RefMaintenanceInterrupted && signal.reason.deadline
|
||||
}
|
||||
|
||||
export class RepoRefMaintenance {
|
||||
private readonly tracked = new Map<string, TrackedRepo>()
|
||||
private readonly cooldownUntil = new Map<string, number>()
|
||||
private readonly now: () => number
|
||||
private readonly isAppBusy: () => boolean
|
||||
private readonly observe: NonNullable<RepoRefMaintenanceOptions['observe']>
|
||||
private readonly quietPeriodMs: number
|
||||
private readonly looseRefThreshold: number
|
||||
private readonly onError: (error: unknown) => void
|
||||
// Why: at most one pack-refs anywhere. It holds a general git admission slot
|
||||
// for its whole run, and two at once would halve git throughput on a small host.
|
||||
// The slot is never released while a pack that could hold `packed-refs.lock`
|
||||
// is still running -- an interrupt cancels the work and waits for it to stop.
|
||||
private inFlight: Promise<void> | null = null
|
||||
private inFlightAbort: AbortController | null = null
|
||||
private readonly lockGate = new PackedRefsLockGate()
|
||||
// Why a count, not a flag: several ref-touching operations overlap routinely
|
||||
// (a create's fetch inside a create), and the last one out reopens the window.
|
||||
private suspensions = 0
|
||||
private lastAttempt: Promise<void> = Promise.resolve()
|
||||
private disposed = false
|
||||
|
||||
constructor(options: RepoRefMaintenanceOptions = {}) {
|
||||
this.now = options.now ?? Date.now
|
||||
this.isAppBusy = options.isBusy ?? (() => false)
|
||||
this.observe = options.observe ?? ((attempt) => attempt(noopSpan))
|
||||
this.quietPeriodMs = options.quietPeriodMs ?? REF_MAINTENANCE_QUIET_PERIOD_MS
|
||||
this.looseRefThreshold = options.looseRefThreshold ?? LOOSE_REF_PACK_THRESHOLD
|
||||
this.onError = options.onError ?? (() => {})
|
||||
}
|
||||
|
||||
/**
|
||||
* Record a write to `target`'s repo and (re)start its quiet-period countdown.
|
||||
* Every call pushes the attempt further out, so a burst of fetches or a
|
||||
* worktree create can never be interrupted by maintenance it triggered.
|
||||
*/
|
||||
arm(target: RepoRefMaintenanceTarget): void {
|
||||
if (this.disposed) {
|
||||
return
|
||||
}
|
||||
const existing = this.tracked.get(target.key)
|
||||
if (existing?.timer) {
|
||||
clearTimeout(existing.timer)
|
||||
}
|
||||
const tracked: TrackedRepo = { target, timer: null, deferrals: existing?.deferrals ?? 0 }
|
||||
this.tracked.delete(target.key)
|
||||
this.evictOldestBeyondCap()
|
||||
this.tracked.set(target.key, tracked)
|
||||
this.schedule(target.key, tracked)
|
||||
}
|
||||
|
||||
/** Resolves once the attempt started by the most recent timer has settled. */
|
||||
whenAttemptSettled(): Promise<void> {
|
||||
return this.lastAttempt
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait out the `packed-refs` rewrite, if one is in progress.
|
||||
*
|
||||
* Deliberately not a kill. The lock is held for 0.03-1.37s of a 23-32s pack;
|
||||
* the rest is the prune phase, during which a concurrent `fetch --prune`,
|
||||
* `branch -D` or `update-ref` measurably succeeds because per-ref locks last
|
||||
* microseconds and Git retries for `core.filesRefLockTimeout`. Signalling the
|
||||
* child there buys nothing and strands a lock file about one time in five.
|
||||
*
|
||||
* Free when no pack is running, which is almost always.
|
||||
*/
|
||||
awaitPackedRefsLockRelease(): Promise<void> {
|
||||
return this.lockGate.whenReleased(PACKED_REFS_LOCK_WAIT_MS)
|
||||
}
|
||||
|
||||
/**
|
||||
* Push every armed repository's attempt out by a full quiet period.
|
||||
*
|
||||
* User-initiated ref work is evidence the user is active in the app, not just
|
||||
* in one repo, and it is free -- no key to resolve, no subprocess, nothing at
|
||||
* all when nothing is armed.
|
||||
*/
|
||||
postponeAll(): void {
|
||||
if (this.disposed) {
|
||||
return
|
||||
}
|
||||
for (const [key, tracked] of this.tracked) {
|
||||
if (tracked.timer) {
|
||||
clearTimeout(tracked.timer)
|
||||
}
|
||||
tracked.deferrals = 0
|
||||
this.schedule(key, tracked)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hold the repository open for work that is about to touch refs.
|
||||
*
|
||||
* Two things at once: no *new* attempt can start for any repository until the
|
||||
* returned release is called, and the caller waits out any `packed-refs`
|
||||
* rewrite already in progress. A prune already running is left alone to
|
||||
* finish -- it does not block the caller.
|
||||
*/
|
||||
async pause(_reason: string): Promise<() => void> {
|
||||
this.suspensions += 1
|
||||
let released = false
|
||||
try {
|
||||
await this.awaitPackedRefsLockRelease()
|
||||
} catch {
|
||||
// The wait cannot reject, but a release must exist even if it did.
|
||||
}
|
||||
return () => {
|
||||
if (!released) {
|
||||
released = true
|
||||
this.suspensions -= 1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dispose(): void {
|
||||
this.disposed = true
|
||||
this.inFlightAbort?.abort(new RefMaintenanceInterrupted('disposed'))
|
||||
for (const tracked of this.tracked.values()) {
|
||||
if (tracked.timer) {
|
||||
clearTimeout(tracked.timer)
|
||||
}
|
||||
}
|
||||
this.tracked.clear()
|
||||
this.cooldownUntil.clear()
|
||||
}
|
||||
|
||||
private isBusy(tracked: TrackedRepo): boolean {
|
||||
return this.isAppBusy() || (tracked.target.isBusy?.() ?? false)
|
||||
}
|
||||
|
||||
private schedule(key: string, tracked: TrackedRepo, delayMs = this.quietPeriodMs): void {
|
||||
const timer = setTimeout(() => {
|
||||
tracked.timer = null
|
||||
this.lastAttempt = this.attempt(key).catch((error) => this.onError(error))
|
||||
}, delayMs)
|
||||
// Never hold the process open for maintenance.
|
||||
timer.unref?.()
|
||||
tracked.timer = timer
|
||||
}
|
||||
|
||||
private evictOldestBeyondCap(): void {
|
||||
while (this.tracked.size >= MAX_TRACKED_REPOS) {
|
||||
const oldest = this.tracked.keys().next()
|
||||
if (oldest.done) {
|
||||
return
|
||||
}
|
||||
const evicted = this.tracked.get(oldest.value)
|
||||
if (evicted?.timer) {
|
||||
clearTimeout(evicted.timer)
|
||||
}
|
||||
this.tracked.delete(oldest.value)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* `counted` spends the give-up budget. Waiting behind another repository's
|
||||
* pack, or yielding to work Orca asked us to yield to, does not: both end on
|
||||
* their own, so charging for them would let a busy machine starve a repo
|
||||
* until its next fetch. Only "the app is busy" is charged.
|
||||
*/
|
||||
private defer(key: string, tracked: TrackedRepo, counted: boolean): void {
|
||||
// A fetch that landed while this attempt was probing already re-armed the
|
||||
// repo; that entry is fresher, so the deferral must not overwrite it.
|
||||
if (this.disposed || this.tracked.has(key)) {
|
||||
return
|
||||
}
|
||||
if (counted) {
|
||||
if (tracked.deferrals >= MAX_DEFERRALS) {
|
||||
return
|
||||
}
|
||||
tracked.deferrals += 1
|
||||
}
|
||||
this.tracked.set(key, tracked)
|
||||
const multiplier = Math.min(2 ** tracked.deferrals, MAX_DEFERRAL_BACKOFF_MULTIPLIER)
|
||||
this.schedule(key, tracked, this.quietPeriodMs * multiplier)
|
||||
}
|
||||
|
||||
private async attempt(key: string): Promise<void> {
|
||||
const tracked = this.tracked.get(key)
|
||||
if (!tracked || this.disposed) {
|
||||
return
|
||||
}
|
||||
this.tracked.delete(key)
|
||||
const cooldownUntil = this.cooldownUntil.get(key)
|
||||
if (cooldownUntil !== undefined && this.now() < cooldownUntil) {
|
||||
return
|
||||
}
|
||||
if (this.inFlight !== null) {
|
||||
this.defer(key, tracked, false)
|
||||
return
|
||||
}
|
||||
if (this.suspensions > 0 || this.isBusy(tracked)) {
|
||||
this.defer(key, tracked, true)
|
||||
return
|
||||
}
|
||||
const abort = new AbortController()
|
||||
const deadline = setTimeout(
|
||||
() => abort.abort(new RefMaintenanceInterrupted('attempt deadline', true)),
|
||||
REF_MAINTENANCE_ATTEMPT_DEADLINE_MS
|
||||
)
|
||||
deadline.unref?.()
|
||||
const run = this.observe((span) => this.packIfNeeded(key, tracked, span, abort.signal))
|
||||
this.inFlight = run
|
||||
this.inFlightAbort = abort
|
||||
try {
|
||||
await run
|
||||
} finally {
|
||||
clearTimeout(deadline)
|
||||
if (this.inFlight === run) {
|
||||
this.inFlight = null
|
||||
this.inFlightAbort = null
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async packIfNeeded(
|
||||
key: string,
|
||||
tracked: TrackedRepo,
|
||||
span: RefMaintenanceSpan,
|
||||
signal: AbortSignal
|
||||
): Promise<void> {
|
||||
span.setAttribute('repo.maintenance_key', key)
|
||||
// Every await below carries the signal, so a caller waiting in `pause()` is
|
||||
// never stuck behind a probe that has already been told to stop.
|
||||
if (await tracked.target.isOptedOut?.(signal)) {
|
||||
this.settle(key, span, 'opted_out', REF_MAINTENANCE_CLEAN_COOLDOWN_MS)
|
||||
return
|
||||
}
|
||||
if (signal.aborted) {
|
||||
this.yieldTo(key, tracked, span, signal)
|
||||
return
|
||||
}
|
||||
const refsDirectory = await tracked.target.resolveRefsDirectory(signal)
|
||||
if (!refsDirectory) {
|
||||
this.settle(key, span, 'unresolved', REF_MAINTENANCE_CLEAN_COOLDOWN_MS)
|
||||
return
|
||||
}
|
||||
const budget = this.looseRefThreshold + 1
|
||||
const before = await countLooseRefs(refsDirectory, budget, signal)
|
||||
if (signal.aborted) {
|
||||
this.yieldTo(key, tracked, span, signal)
|
||||
return
|
||||
}
|
||||
span.setAttribute('git.loose_ref_count', before.count)
|
||||
span.setAttribute('git.loose_ref_threshold', this.looseRefThreshold)
|
||||
// A saturated walk stopped early, so `count` is a floor -- never read it as "clean".
|
||||
if (!before.saturated && before.count < this.looseRefThreshold) {
|
||||
this.settle(key, span, 'below_threshold', REF_MAINTENANCE_CLEAN_COOLDOWN_MS)
|
||||
return
|
||||
}
|
||||
// The quiet window can close while the probe walks; re-check before spending a git slot.
|
||||
if (this.suspensions > 0 || this.isBusy(tracked)) {
|
||||
span.setAttribute('repo.maintenance_outcome', 'deferred' satisfies RefMaintenanceOutcome)
|
||||
this.defer(key, tracked, true)
|
||||
return
|
||||
}
|
||||
const startedAt = this.now()
|
||||
let partial = false
|
||||
try {
|
||||
// No signal: the pack runs to completion. Callers that need the refs wait
|
||||
// out the rewrite window through `pause()` instead of killing it.
|
||||
await tracked.target.packRefs(this.lockGate)
|
||||
} catch (error) {
|
||||
span.setAttribute('repo.maintenance_error', String(error))
|
||||
if (error instanceof RefMaintenanceRepoLocked) {
|
||||
this.settle(key, span, 'locked', REF_MAINTENANCE_LOCKED_COOLDOWN_MS)
|
||||
return
|
||||
}
|
||||
partial = true
|
||||
} finally {
|
||||
this.lockGate.setHeld(false)
|
||||
}
|
||||
span.setAttribute('git.pack_refs_ms', this.now() - startedAt)
|
||||
// Judge by the backlog, not by the exit code. On a machine running several
|
||||
// Orca sessions a branch moving mid-pack is the normal case, and Git's
|
||||
// response -- leave that one ref loose, pack the rest -- is the correct one.
|
||||
// Measured in the field: 36,688 loose refs down to 3, reported as an error.
|
||||
const after = await countLooseRefs(refsDirectory, budget, signal)
|
||||
span.setAttribute('git.loose_ref_count_after', after.count)
|
||||
if (partial && (after.saturated || after.count >= this.looseRefThreshold)) {
|
||||
this.settle(key, span, 'failed', REF_MAINTENANCE_FAILURE_COOLDOWN_MS)
|
||||
return
|
||||
}
|
||||
span.setAttribute('git.pack_refs_partial', partial)
|
||||
this.settle(key, span, 'packed', REF_MAINTENANCE_PACKED_COOLDOWN_MS)
|
||||
}
|
||||
|
||||
/** Record an aborted attempt: retry soon if Orca yielded, back off if it stalled. */
|
||||
private yieldTo(
|
||||
key: string,
|
||||
tracked: TrackedRepo,
|
||||
span: RefMaintenanceSpan,
|
||||
signal: AbortSignal
|
||||
): void {
|
||||
if (hitDeadline(signal)) {
|
||||
this.settle(key, span, 'timed_out', REF_MAINTENANCE_FAILURE_COOLDOWN_MS)
|
||||
return
|
||||
}
|
||||
span.setAttribute('repo.maintenance_outcome', 'interrupted' satisfies RefMaintenanceOutcome)
|
||||
this.defer(key, tracked, false)
|
||||
}
|
||||
|
||||
private settle(
|
||||
key: string,
|
||||
span: RefMaintenanceSpan,
|
||||
outcome: RefMaintenanceOutcome,
|
||||
cooldownMs: number
|
||||
): void {
|
||||
span.setAttribute('repo.maintenance_outcome', outcome)
|
||||
// Re-insert so Map order stays newest-last and the eviction below drops the oldest.
|
||||
this.cooldownUntil.delete(key)
|
||||
this.cooldownUntil.set(key, this.now() + cooldownMs)
|
||||
if (this.cooldownUntil.size > MAX_TRACKED_REPOS * 4) {
|
||||
const oldest = this.cooldownUntil.keys().next()
|
||||
if (!oldest.done) {
|
||||
this.cooldownUntil.delete(oldest.value)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user