mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 08:02:28 +00:00
* perf(git): pack the loose refs Orca's own fetches leave behind Orca strips git's auto-maintenance off every fetch it issues (GIT_FETCH_SKIP_AUTO_MAINTENANCE_CONFIG_ARGS) and never compensated, so nothing in an Orca-driven checkout ever packs refs. One real machine reached 36,574 loose refs, where `git show-ref -- main` costs 5.2s and every worktree create pays for it. Add an idle-time, per-repo `git pack-refs --all --prune`, armed by the fetches that create the debt. It runs only after ten minutes of quiet on that repo, only above 1000 loose refs (probed with a walk bounded by that threshold, not by the backlog), one at a time across the whole app, at the background admission tier, and never while an agent is working, a create is prepared or in flight, a worktree removal is deleting refs, the app is quitting, or the machine is on battery. A user who set `maintenance.auto=false` or `gc.auto=0` has opted out. Measured on a 36,001-loose-ref fixture (macOS/APFS, git 2.44): `show-ref` 5.5-12.2s -> 30-49ms, `for-each-ref` 4.0-10.8s -> 43-48ms. Also fixes a pre-existing bug the split exposed: `--path-format=absolute` is ignored before git 2.31, and taking rev-parse's stdout raw collapsed every repo on such a host onto one fetch-serialization key. Refs #17828 * perf(git): make idle ref maintenance preemptible and cheaper to probe The idle veto was one-directional: it stopped a pack from starting during a create, removal, or agent work, but nothing stopped those from starting during a pack. A user-clicked Fetch, a branch delete, or a worktree removal that needed `packed-refs.lock` mid-rewrite could fail with `unable to create packed-refs.lock` -- a git error with no visible cause. Make the pack cancellable end to end. An AbortSignal now reaches the `pack-refs` child and both pre-pack probes, and `pause()` aborts what is running, waits for it to actually stop, and holds a suspension count so nothing new starts until the caller releases. Every entry point that deletes a ref takes that pause: gitFetch, gitPull, gitFastForward, removeWorktree, forceDeleteLocalBranch, prepareWorktreeCreateCheckout, addWorktree. Five more triggers close the rest of the window: battery drop, window focus, quit, the attempt deadline, and any other git command queueing for an admission slot. Judge a pack by re-probing the backlog rather than by the child's exit code. Measured in the field: another Orca session moved a branch mid-pack, git reported `cannot lock ref`, skipped that ref and packed the rest -- 36,688 loose refs down to 3. On a machine running several sessions that is the normal case, and retrying it would be wrong. Probe with one batched `readdir` per directory instead of streaming `opendir`, which issues a thread-pool round trip every 32 entries: 177ms -> 23ms on a real 36,600-ref repository, with half the event-loop lag. The walk stays strictly sequential so it can never occupy more than one of libuv's four filesystem threads. `PackRefsLockOwnership` makes a lock left by SIGKILL attributable, and only reclaims one when a marker exists, the lock is older than any pack-refs could run for, and the recorded process is gone. Refs #17828 * fix(git): wait out the packed-refs lock instead of killing the pack Measured on Git 2.55/APFS with 37k loose refs: a full `pack-refs --all --prune` takes 23-32s but holds `packed-refs.lock` for only 0.03-1.37s of it. The other ~95% is the prune phase, during which a concurrent `fetch --prune`, `branch -D` or `update-ref` succeeds every time -- per-ref locks last microseconds and git retries for `core.filesRefLockTimeout`. So the abort-on-everything design was strictly harmful. SIGTERM into the prune loop strands an empty `refs/**/*.lock` about one time in five (9/30, 5/40, 6/30 kills): `tempfile.c` opens the lock O_EXCL before `activate_tempfile()` links it into the list the signal handler walks, and a pack does ~36k lock cycles. Afterwards `update-ref -d` on that ref fails with `cannot lock ref ... File exists`, permanently. On Windows `taskkill /f` never runs git's handlers at all, so an abort inside the rewrite strands `packed-refs.lock` every time. Never signal the child. `packRefs` no longer takes an abort signal; it polls `packed-refs.lock` and reports the window through a `PackedRefsLockReporter`. `pause()` resolves when the lock is released -- bounded, and free during the prune -- while the suspension counter still blocks new attempts. Battery and window-focus become do-not-start rather than stop-what-is-running, and quit waits for the lock and lets the child finish orphaned. For strands that already exist, `PackRefsLockOwnership` now also reclaims `refs/**/*.lock` under the same three conditions plus a 0-byte check, and a lock carrying our own not-yet-reclaimable marker records `locked` with a 30min retry instead of the 6h failure cooldown -- so a Windows strand self-heals in half an hour rather than six. Reverts the git admission-scheduler event bus, which existed only to drive the abort this removes. Refs #17828 * test(git): make the ref-maintenance waits survive a loaded runner CI shard 4/8 failed on `restarts every armed countdown when the user does ref work themselves`, which passes locally. The `until()` helper spun a fixed 200 event-loop turns and then returned silently, so on a contended runner the filesystem probe had not finished and the assertion that followed failed with an unrelated message. Bound the wait by wall clock instead and throw a named error, which immediately exposed a second latent bug: the single-flight test's second wait could never succeed, because the deferred repo's retry is on a faked `setTimeout` that spinning the real loop never advances. It had been passing only because the old helper gave up quietly. Add a timer-aware variant for those, and have the countdown test await a signal the fake pack resolves rather than polling at all. Verified stable across five sequential runs and once under load average 32 with six concurrent suites. Refs #17828
327 lines
11 KiB
TypeScript
327 lines
11 KiB
TypeScript
import { link, readlink, rmdir, symlink, unlink, writeFile } from 'node:fs/promises'
|
|
import { basename, dirname, join, resolve } from 'node:path'
|
|
import type { CliInstallStatus } from '../../shared/cli-install-types'
|
|
import {
|
|
ensureAppImageExtractedRoot,
|
|
isAppImageExtractedLauncherPath,
|
|
type AppImageExtractedRoot
|
|
} from './appimage-extracted-root'
|
|
import { CliCommandInspection } from './cli-command-inspection'
|
|
import {
|
|
buildMacPrivilegedSymlinkTransaction,
|
|
capturedExpectedEntry,
|
|
hasSameIdentity,
|
|
hasSameSnapshot,
|
|
inspectStableCommand,
|
|
quarantineCommandPath,
|
|
readEntrySnapshot,
|
|
type CommandQuarantine,
|
|
type StableCommandInspection
|
|
} from './cli-command-filesystem-transaction'
|
|
import { DEV_LAUNCHER_DIR, LEGACY_LINUX_COMMAND_NAME } from './cli-install-constants'
|
|
import { buildWindowsForwarder } from './cli-dev-launcher'
|
|
import { isMissingError, isPermissionError } from './cli-install-errors'
|
|
import { isPathInsideOrEqual } from './cli-install-path-format'
|
|
|
|
const STABLE_LEGACY_INSPECTION_ATTEMPTS = 3
|
|
|
|
export class CliCommandInstallation extends CliCommandInspection {
|
|
protected async installSymlink(status: CliInstallStatus): Promise<void> {
|
|
const commandPath = status.commandPath
|
|
const launcherPath = status.launcherPath
|
|
if (!commandPath || !launcherPath || status.state === 'installed') {
|
|
return
|
|
}
|
|
|
|
const inspected = await this.inspectStableSymlink(commandPath, launcherPath)
|
|
if (inspected.status.state === 'conflict') {
|
|
throw new Error(
|
|
`Refusing to replace non-Orca command at ${commandPath}. Remove it and register again if it is no longer needed.`
|
|
)
|
|
}
|
|
if (inspected.status.state === 'installed') {
|
|
return
|
|
}
|
|
|
|
let quarantine: CommandQuarantine
|
|
try {
|
|
quarantine = await this.quarantineCommandPath(commandPath)
|
|
} catch (error) {
|
|
if (this.platform !== 'darwin' || !isPermissionError(error)) {
|
|
throw error
|
|
}
|
|
await this.installSymlinkWithPrivileges(commandPath, launcherPath, inspected)
|
|
return
|
|
}
|
|
|
|
if (!(await capturedExpectedEntry(quarantine, inspected))) {
|
|
await this.restoreQuarantinedCommand(quarantine, commandPath)
|
|
throw new Error(
|
|
`Refusing to replace non-Orca command at ${commandPath}. Remove it and register again if it is no longer needed.`
|
|
)
|
|
}
|
|
|
|
try {
|
|
await symlink(launcherPath, commandPath)
|
|
} catch (error) {
|
|
await this.restoreQuarantinedCommand(quarantine, commandPath)
|
|
throw error
|
|
}
|
|
await this.discardQuarantinedCommand(quarantine)
|
|
}
|
|
|
|
protected async removeSymlink(commandPath: string): Promise<void> {
|
|
const launcherPath = await this.resolveLauncherPath()
|
|
if (!launcherPath) {
|
|
throw new Error('The Orca CLI launcher is no longer available.')
|
|
}
|
|
const inspected = await this.inspectStableSymlink(commandPath, launcherPath)
|
|
if (inspected.status.state === 'not_installed') {
|
|
return
|
|
}
|
|
if (inspected.status.state === 'conflict') {
|
|
throw new Error(`Refusing to remove non-Orca command at ${commandPath}.`)
|
|
}
|
|
|
|
let quarantine: CommandQuarantine
|
|
try {
|
|
quarantine = await this.quarantineCommandPath(commandPath)
|
|
} catch (error) {
|
|
if (this.platform !== 'darwin' || !isPermissionError(error)) {
|
|
throw error
|
|
}
|
|
await this.removeSymlinkWithPrivileges(commandPath, inspected)
|
|
return
|
|
}
|
|
if (!(await capturedExpectedEntry(quarantine, inspected))) {
|
|
await this.restoreQuarantinedCommand(quarantine, commandPath)
|
|
throw new Error(`Refusing to remove non-Orca command at ${commandPath}.`)
|
|
}
|
|
await this.discardQuarantinedCommand(quarantine)
|
|
}
|
|
|
|
protected async removeLegacyLinuxCommandIfManaged(launcherPath: string | null): Promise<void> {
|
|
if (this.platform !== 'linux' || this.commandPathOverride || !launcherPath) {
|
|
return
|
|
}
|
|
|
|
const commandPath = join(this.homePath, '.local', 'bin', LEGACY_LINUX_COMMAND_NAME)
|
|
try {
|
|
const inspected = await this.inspectStableLegacyCommand(commandPath, launcherPath)
|
|
if (!inspected?.managed) {
|
|
return
|
|
}
|
|
const quarantine = await this.quarantineCommandPath(commandPath)
|
|
if (!(await capturedExpectedEntry(quarantine, inspected))) {
|
|
await this.restoreQuarantinedCommand(quarantine, commandPath)
|
|
return
|
|
}
|
|
await this.discardQuarantinedCommand(quarantine)
|
|
} catch (error) {
|
|
// Why: the new command is already registered; leave legacy cleanup for a later attempt.
|
|
console.warn(
|
|
`[cli] Could not remove the legacy command at ${commandPath}:`,
|
|
error instanceof Error ? error.message : String(error)
|
|
)
|
|
}
|
|
}
|
|
|
|
protected async quarantineCommandPath(commandPath: string): Promise<CommandQuarantine> {
|
|
return quarantineCommandPath(commandPath)
|
|
}
|
|
|
|
protected async linkQuarantinedCommand(heldPath: string, commandPath: string): Promise<void> {
|
|
await link(heldPath, commandPath)
|
|
}
|
|
|
|
protected isManagedLegacyLinuxTarget(resolvedTarget: string, launcherPath: string): boolean {
|
|
const legacyLauncherPath = resolve(dirname(launcherPath), LEGACY_LINUX_COMMAND_NAME)
|
|
if (resolvedTarget === legacyLauncherPath) {
|
|
return true
|
|
}
|
|
|
|
if (basename(resolvedTarget) !== LEGACY_LINUX_COMMAND_NAME) {
|
|
return false
|
|
}
|
|
|
|
if (this.isPackagedLinuxLauncherTarget(resolvedTarget, LEGACY_LINUX_COMMAND_NAME)) {
|
|
return true
|
|
}
|
|
|
|
const devLauncherDir = resolve(this.userDataPath, ...DEV_LAUNCHER_DIR)
|
|
if (isPathInsideOrEqual(devLauncherDir, resolvedTarget)) {
|
|
return true
|
|
}
|
|
|
|
const extractionOptions = this.appImageExtractionOptions()
|
|
return extractionOptions
|
|
? isAppImageExtractedLauncherPath(
|
|
extractionOptions,
|
|
resolvedTarget,
|
|
LEGACY_LINUX_COMMAND_NAME
|
|
)
|
|
: false
|
|
}
|
|
|
|
protected async installWindowsWrapper(commandPath: string, launcherPath: string): Promise<void> {
|
|
await writeFile(commandPath, buildWindowsForwarder(launcherPath), 'utf8')
|
|
}
|
|
|
|
protected async ensureLinuxAppImagePayload(): Promise<AppImageExtractedRoot | null> {
|
|
const extractionOptions = this.appImageExtractionOptions()
|
|
if (!this.isLinuxAppImage() || !extractionOptions) {
|
|
return null
|
|
}
|
|
const extractedRoot = await ensureAppImageExtractedRoot(extractionOptions)
|
|
if (!extractedRoot) {
|
|
throw new Error(
|
|
`Could not extract the Orca AppImage at ${this.appImagePath}. Check that it is executable and that ${this.appImageCacheRootPath} has free space.`
|
|
)
|
|
}
|
|
return extractedRoot
|
|
}
|
|
|
|
private async inspectStableSymlink(
|
|
commandPath: string,
|
|
launcherPath: string
|
|
): Promise<StableCommandInspection> {
|
|
return inspectStableCommand(commandPath, () => this.inspectSymlink(commandPath, launcherPath))
|
|
}
|
|
|
|
private async inspectStableLegacyCommand(
|
|
commandPath: string,
|
|
launcherPath: string
|
|
): Promise<
|
|
| (Pick<StableCommandInspection, 'fileSha256' | 'rawSymlinkTarget'> & {
|
|
snapshot: NonNullable<StableCommandInspection['snapshot']>
|
|
managed: boolean
|
|
})
|
|
| null
|
|
> {
|
|
for (let attempt = 0; attempt < STABLE_LEGACY_INSPECTION_ATTEMPTS; attempt += 1) {
|
|
const before = await readEntrySnapshot(commandPath)
|
|
if (!before) {
|
|
return null
|
|
}
|
|
let target: string | null = null
|
|
try {
|
|
target = before.isSymbolicLink ? await readlink(commandPath) : null
|
|
} catch (error) {
|
|
if (isMissingError(error)) {
|
|
continue
|
|
}
|
|
throw error
|
|
}
|
|
const after = await readEntrySnapshot(commandPath)
|
|
if (after && hasSameSnapshot(before, after)) {
|
|
const resolvedTarget = target ? resolve(dirname(commandPath), target) : null
|
|
return {
|
|
fileSha256: null,
|
|
rawSymlinkTarget: target,
|
|
snapshot: after,
|
|
managed: Boolean(
|
|
resolvedTarget && this.isManagedLegacyLinuxTarget(resolvedTarget, launcherPath)
|
|
)
|
|
}
|
|
}
|
|
}
|
|
throw new Error(`The command at ${commandPath} changed while Orca inspected it.`)
|
|
}
|
|
|
|
private async restoreQuarantinedCommand(
|
|
quarantine: CommandQuarantine,
|
|
commandPath: string
|
|
): Promise<void> {
|
|
if (!quarantine.snapshot) {
|
|
await rmdir(quarantine.directoryPath)
|
|
return
|
|
}
|
|
await this.assertHeldIdentity(quarantine)
|
|
try {
|
|
await (quarantine.snapshot.isSymbolicLink
|
|
? symlink(await readlink(quarantine.heldPath), commandPath)
|
|
: this.linkQuarantinedCommand(quarantine.heldPath, commandPath))
|
|
const restored = await readEntrySnapshot(commandPath)
|
|
const restoredSymlink =
|
|
restored?.isSymbolicLink && quarantine.snapshot.isSymbolicLink
|
|
? (await readlink(commandPath)) === (await readlink(quarantine.heldPath))
|
|
: false
|
|
if (
|
|
!restored ||
|
|
(!restoredSymlink && !hasSameIdentity(restored.identity, quarantine.snapshot.identity))
|
|
) {
|
|
throw new Error('The restored command identity could not be verified.')
|
|
}
|
|
await this.discardQuarantinedCommand(quarantine, quarantine.snapshot.isSymbolicLink)
|
|
} catch (error) {
|
|
throw new Error(
|
|
`The displaced entry is preserved at ${quarantine.heldPath}; ${commandPath} could not be restored without overwriting another entry.`,
|
|
{ cause: error }
|
|
)
|
|
}
|
|
}
|
|
|
|
private async discardQuarantinedCommand(
|
|
quarantine: CommandQuarantine,
|
|
requireStableMetadata = true
|
|
): Promise<void> {
|
|
if (quarantine.snapshot) {
|
|
await this.assertHeldIdentity(quarantine, requireStableMetadata)
|
|
await unlink(quarantine.heldPath)
|
|
}
|
|
await rmdir(quarantine.directoryPath)
|
|
}
|
|
|
|
private async assertHeldIdentity(
|
|
quarantine: CommandQuarantine,
|
|
requireStableMetadata = true
|
|
): Promise<void> {
|
|
const current = await readEntrySnapshot(quarantine.heldPath)
|
|
if (
|
|
!current ||
|
|
!quarantine.snapshot ||
|
|
!(requireStableMetadata
|
|
? hasSameSnapshot(current, quarantine.snapshot)
|
|
: hasSameIdentity(current.identity, quarantine.snapshot.identity))
|
|
) {
|
|
throw new Error(`The quarantined command changed at ${quarantine.heldPath}.`)
|
|
}
|
|
}
|
|
|
|
private async installSymlinkWithPrivileges(
|
|
commandPath: string,
|
|
launcherPath: string,
|
|
inspected: StableCommandInspection
|
|
): Promise<void> {
|
|
await this.privilegedRunner(
|
|
buildMacPrivilegedSymlinkTransaction({
|
|
action: 'install',
|
|
commandPath,
|
|
launcherPath,
|
|
expected: inspected.snapshot?.identity ?? null,
|
|
expectedFileSha256: inspected.fileSha256,
|
|
expectedRawSymlinkTarget: inspected.rawSymlinkTarget
|
|
})
|
|
)
|
|
const installed = await this.inspectStableSymlink(commandPath, launcherPath)
|
|
if (installed.status.state !== 'installed') {
|
|
throw new Error(`Could not register the Orca command at ${commandPath}.`)
|
|
}
|
|
}
|
|
|
|
private async removeSymlinkWithPrivileges(
|
|
commandPath: string,
|
|
inspected: StableCommandInspection
|
|
): Promise<void> {
|
|
await this.privilegedRunner(
|
|
buildMacPrivilegedSymlinkTransaction({
|
|
action: 'remove',
|
|
commandPath,
|
|
expected: inspected.snapshot?.identity ?? null,
|
|
expectedFileSha256: inspected.fileSha256,
|
|
expectedRawSymlinkTarget: inspected.rawSymlinkTarget
|
|
})
|
|
)
|
|
}
|
|
}
|