Files
orca/src/main/cli/cli-installer.ts
T
Neil d7123591ce perf(git): pack the loose refs Orca's own fetches leave behind (#17857)
* perf(git): pack the loose refs Orca's own fetches leave behind

Orca strips git's auto-maintenance off every fetch it issues
(GIT_FETCH_SKIP_AUTO_MAINTENANCE_CONFIG_ARGS) and never compensated, so
nothing in an Orca-driven checkout ever packs refs. One real machine
reached 36,574 loose refs, where `git show-ref -- main` costs 5.2s and
every worktree create pays for it.

Add an idle-time, per-repo `git pack-refs --all --prune`, armed by the
fetches that create the debt. It runs only after ten minutes of quiet on
that repo, only above 1000 loose refs (probed with a walk bounded by that
threshold, not by the backlog), one at a time across the whole app, at
the background admission tier, and never while an agent is working, a
create is prepared or in flight, a worktree removal is deleting refs, the
app is quitting, or the machine is on battery. A user who set
`maintenance.auto=false` or `gc.auto=0` has opted out.

Measured on a 36,001-loose-ref fixture (macOS/APFS, git 2.44):
`show-ref` 5.5-12.2s -> 30-49ms, `for-each-ref` 4.0-10.8s -> 43-48ms.

Also fixes a pre-existing bug the split exposed: `--path-format=absolute`
is ignored before git 2.31, and taking rev-parse's stdout raw collapsed
every repo on such a host onto one fetch-serialization key.

Refs #17828

* perf(git): make idle ref maintenance preemptible and cheaper to probe

The idle veto was one-directional: it stopped a pack from starting during
a create, removal, or agent work, but nothing stopped those from starting
during a pack. A user-clicked Fetch, a branch delete, or a worktree
removal that needed `packed-refs.lock` mid-rewrite could fail with
`unable to create packed-refs.lock` -- a git error with no visible cause.

Make the pack cancellable end to end. An AbortSignal now reaches the
`pack-refs` child and both pre-pack probes, and `pause()` aborts what is
running, waits for it to actually stop, and holds a suspension count so
nothing new starts until the caller releases. Every entry point that
deletes a ref takes that pause: gitFetch, gitPull, gitFastForward,
removeWorktree, forceDeleteLocalBranch, prepareWorktreeCreateCheckout,
addWorktree. Five more triggers close the rest of the window: battery
drop, window focus, quit, the attempt deadline, and any other git command
queueing for an admission slot.

Judge a pack by re-probing the backlog rather than by the child's exit
code. Measured in the field: another Orca session moved a branch
mid-pack, git reported `cannot lock ref`, skipped that ref and packed the
rest -- 36,688 loose refs down to 3. On a machine running several
sessions that is the normal case, and retrying it would be wrong.

Probe with one batched `readdir` per directory instead of streaming
`opendir`, which issues a thread-pool round trip every 32 entries: 177ms
-> 23ms on a real 36,600-ref repository, with half the event-loop lag.
The walk stays strictly sequential so it can never occupy more than one
of libuv's four filesystem threads.

`PackRefsLockOwnership` makes a lock left by SIGKILL attributable, and
only reclaims one when a marker exists, the lock is older than any
pack-refs could run for, and the recorded process is gone.

Refs #17828

* fix(git): wait out the packed-refs lock instead of killing the pack

Measured on Git 2.55/APFS with 37k loose refs: a full `pack-refs --all
--prune` takes 23-32s but holds `packed-refs.lock` for only 0.03-1.37s of
it. The other ~95% is the prune phase, during which a concurrent `fetch
--prune`, `branch -D` or `update-ref` succeeds every time -- per-ref locks
last microseconds and git retries for `core.filesRefLockTimeout`.

So the abort-on-everything design was strictly harmful. SIGTERM into the
prune loop strands an empty `refs/**/*.lock` about one time in five
(9/30, 5/40, 6/30 kills): `tempfile.c` opens the lock O_EXCL before
`activate_tempfile()` links it into the list the signal handler walks,
and a pack does ~36k lock cycles. Afterwards `update-ref -d` on that ref
fails with `cannot lock ref ... File exists`, permanently. On Windows
`taskkill /f` never runs git's handlers at all, so an abort inside the
rewrite strands `packed-refs.lock` every time.

Never signal the child. `packRefs` no longer takes an abort signal; it
polls `packed-refs.lock` and reports the window through a
`PackedRefsLockReporter`. `pause()` resolves when the lock is released --
bounded, and free during the prune -- while the suspension counter still
blocks new attempts. Battery and window-focus become do-not-start rather
than stop-what-is-running, and quit waits for the lock and lets the child
finish orphaned.

For strands that already exist, `PackRefsLockOwnership` now also reclaims
`refs/**/*.lock` under the same three conditions plus a 0-byte check, and
a lock carrying our own not-yet-reclaimable marker records `locked` with
a 30min retry instead of the 6h failure cooldown -- so a Windows strand
self-heals in half an hour rather than six.

Reverts the git admission-scheduler event bus, which existed only to
drive the abort this removes.

Refs #17828

* test(git): make the ref-maintenance waits survive a loaded runner

CI shard 4/8 failed on `restarts every armed countdown when the user does
ref work themselves`, which passes locally. The `until()` helper spun a
fixed 200 event-loop turns and then returned silently, so on a contended
runner the filesystem probe had not finished and the assertion that
followed failed with an unrelated message.

Bound the wait by wall clock instead and throw a named error, which
immediately exposed a second latent bug: the single-flight test's second
wait could never succeed, because the deferred repo's retry is on a faked
`setTimeout` that spinning the real loop never advances. It had been
passing only because the old helper gave up quietly. Add a timer-aware
variant for those, and have the countdown test await a signal the fake
pack resolves rather than polling at all.

Verified stable across five sequential runs and once under load average
32 with six concurrent suites.

Refs #17828
2026-09-01 19:06:44 -07:00

219 lines
8.7 KiB
TypeScript

import { mkdir, unlink } from 'node:fs/promises'
import { dirname } from 'node:path'
import type { CliInstallStatus } from '../../shared/cli-install-types'
import {
pruneAppImageExtractedRoots,
removeAppImageInstalledPayloads
} from './appimage-extraction-pruning'
import { withAppImageRegistrationLock } from './appimage-registration-lock'
import {
isAppImageInstalledLauncherCurrent,
isAppImageInstalledLauncherOwnedBySibling,
resolveAppImageNamespacePath
} from './appimage-extracted-root'
import { isAppImageStableLauncherReady } from './appimage-stable-launcher'
import { CliPathRegistration } from './cli-path-registration'
export class CliInstaller extends CliPathRegistration {
isAppImageRegistrationOwnedBySibling(status: CliInstallStatus): boolean {
if (
status.currentTarget !== status.launcherPath ||
!isAppImageStableLauncherReady(this.appImageCacheRootPath)
) {
return false
}
const extractionOptions = this.appImageExtractionOptions()
return Boolean(
extractionOptions && isAppImageInstalledLauncherOwnedBySibling(extractionOptions)
)
}
async getStatus(): Promise<CliInstallStatus> {
const defaultSpec = this.resolveInstallSpec()
if (!defaultSpec) {
return {
platform: this.platform,
commandName: this.commandName,
commandPath: null,
pathDirectory: null,
pathConfigured: false,
launcherPath: null,
installMethod: null,
supported: false,
state: 'unsupported',
currentTarget: null,
unsupportedReason: 'platform_not_supported',
detail: 'CLI registration is not implemented on this platform.'
}
}
const launcherPath = await this.resolveLauncherPath()
if (!launcherPath) {
const detail = this.hasUnverifiedAppImageRuntime
? 'Orca could not verify the inherited AppImage runtime identity, so CLI registration is unavailable.'
: this.isLinuxAppImage() && this.appImagePath
? `The AppImage file at ${this.appImagePath} is missing. Move it back or re-run CLI registration from the current AppImage location.`
: this.isPackaged
? 'The bundled CLI launcher is missing from this Orca build.'
: 'Development mode uses a generated launcher for validation only.'
return {
platform: this.platform,
commandName: this.commandName,
commandPath: defaultSpec.commandPath,
pathDirectory: dirname(defaultSpec.commandPath),
pathConfigured: false,
launcherPath: null,
installMethod: defaultSpec.installMethod,
supported: false,
state: 'unsupported',
currentTarget: null,
unsupportedReason: this.isPackaged ? 'launcher_missing' : 'launch_mode_unavailable',
detail
}
}
return this.getStatusForLauncher(launcherPath)
}
private async getStatusForLauncher(launcherPath: string): Promise<CliInstallStatus> {
const defaultSpec = this.resolveInstallSpec()
if (!defaultSpec) {
throw new Error('CLI registration is not implemented on this platform.')
}
const spec = await this.resolveActiveInstallSpec(defaultSpec, launcherPath)
const inspectedStatus =
spec.installMethod === 'symlink'
? await this.inspectSymlink(spec.commandPath, launcherPath)
: await this.inspectWindowsWrapper(spec.commandPath, launcherPath)
const extractionOptions = this.appImageExtractionOptions()
const baseStatus =
inspectedStatus.state === 'installed' &&
extractionOptions &&
!isAppImageInstalledLauncherCurrent(extractionOptions)
? {
...inspectedStatus,
state: 'stale' as const,
detail: `${spec.commandPath} does not point to the current Orca AppImage payload.`
}
: inspectedStatus
const pathDirectory = dirname(spec.commandPath)
const pathProbe = await this.probePathConfiguration(pathDirectory)
return this.withPathInfo(baseStatus, pathDirectory, pathProbe)
}
async install(): Promise<CliInstallStatus> {
return this.runAppImageRegistrationOperation(() => this.installUnlocked())
}
private async installUnlocked(): Promise<CliInstallStatus> {
const initialStatus = await this.getStatus()
if (
!initialStatus.supported ||
!initialStatus.commandPath ||
!initialStatus.launcherPath ||
!initialStatus.installMethod
) {
throw new Error(initialStatus.detail ?? 'CLI registration is unavailable on this build.')
}
if (initialStatus.state === 'conflict') {
throw new Error(
`Refusing to replace non-Orca command at ${initialStatus.commandPath}. Remove it and register again if it is no longer needed.`
)
}
const extractedRoot = await this.ensureLinuxAppImagePayload()
const status = extractedRoot
? await this.getStatusForLauncher(extractedRoot.stableLauncherPath)
: initialStatus
if (!status.supported || !status.commandPath || !status.launcherPath || !status.installMethod) {
throw new Error(status.detail ?? 'CLI registration is unavailable on this build.')
}
if (status.state === 'conflict') {
throw new Error(
`Refusing to replace non-Orca command at ${status.commandPath}. Remove it and register again if it is no longer needed.`
)
}
// eslint-disable-next-line unicorn/prefer-ternary -- Why: the install path performs async side effects and is easier to audit as an explicit branch than as an awaited ternary.
if (status.installMethod === 'symlink') {
await this.installSymlink(status)
await this.removeLegacyLinuxCommandIfManaged(status.launcherPath)
} else if (this.isWindowsPackagedBundledCommand(status.commandPath, status.launcherPath)) {
// Why: packaged Windows already ships resources/bin/orca.exe; registration only owns the PATH entry.
} else {
// Why: the Windows wrapper dir is user-writable (%LOCALAPPDATA%), so mkdir here can't hit EACCES.
await mkdir(dirname(status.commandPath), { recursive: true })
await this.installWindowsWrapper(status.commandPath, status.launcherPath)
}
if (this.platform === 'win32') {
// Why: Windows shells find commands via user PATH, so the installer owns that entry, not the desktop installer.
await this.ensureWindowsPathEntry(dirname(status.commandPath))
}
if (extractedRoot) {
await pruneAppImageExtractedRoots(extractedRoot.rootPath)
}
return extractedRoot
? this.getStatusForLauncher(extractedRoot.stableLauncherPath)
: this.getStatus()
}
async remove(): Promise<CliInstallStatus> {
return this.runAppImageRegistrationOperation(() => this.removeUnlocked())
}
private async removeUnlocked(): Promise<CliInstallStatus> {
const status = await this.getStatus()
if (!status.supported || !status.commandPath || !status.launcherPath || !status.installMethod) {
await this.removeLinuxAppImagePayloads()
return status
}
if (status.state === 'not_installed') {
await this.removeLegacyLinuxCommandIfManaged(status.launcherPath)
if (this.platform === 'win32') {
await this.removeWindowsPathEntry(dirname(status.commandPath))
return this.getStatus()
}
await this.removeLinuxAppImagePayloads()
return status
}
if (status.state === 'conflict') {
throw new Error(`Refusing to remove non-Orca command at ${status.commandPath}.`)
}
if (status.state === 'stale' && status.installMethod !== 'symlink') {
throw new Error(`Refusing to remove a command not owned by Orca at ${status.commandPath}.`)
}
if (status.state === 'stale' && this.isAppImageRegistrationOwnedBySibling(status)) {
await this.removeLinuxAppImagePayloads()
return this.getStatus()
}
if (status.installMethod === 'symlink') {
await this.removeSymlink(status.commandPath)
await this.removeLegacyLinuxCommandIfManaged(status.launcherPath)
} else if (this.isWindowsPackagedBundledCommand(status.commandPath, status.launcherPath)) {
await this.removeWindowsPathEntry(dirname(status.commandPath))
} else {
await unlink(status.commandPath)
await this.removeWindowsPathEntry(dirname(status.commandPath))
}
await this.removeLinuxAppImagePayloads()
return this.getStatus()
}
private async removeLinuxAppImagePayloads(): Promise<void> {
const extractionOptions = this.appImageExtractionOptions()
if (this.isLinuxAppImage() && extractionOptions) {
await removeAppImageInstalledPayloads(resolveAppImageNamespacePath(extractionOptions))
}
}
private runAppImageRegistrationOperation<T>(operation: () => Promise<T>): Promise<T> {
return this.isLinuxAppImage()
? withAppImageRegistrationLock(this.appImageCacheRootPath, operation)
: operation()
}
}