mirror of
https://github.com/stablyai/orca.git
synced 2026-10-30 16:47:34 +00:00
* perf(git): pack the loose refs Orca's own fetches leave behind Orca strips git's auto-maintenance off every fetch it issues (GIT_FETCH_SKIP_AUTO_MAINTENANCE_CONFIG_ARGS) and never compensated, so nothing in an Orca-driven checkout ever packs refs. One real machine reached 36,574 loose refs, where `git show-ref -- main` costs 5.2s and every worktree create pays for it. Add an idle-time, per-repo `git pack-refs --all --prune`, armed by the fetches that create the debt. It runs only after ten minutes of quiet on that repo, only above 1000 loose refs (probed with a walk bounded by that threshold, not by the backlog), one at a time across the whole app, at the background admission tier, and never while an agent is working, a create is prepared or in flight, a worktree removal is deleting refs, the app is quitting, or the machine is on battery. A user who set `maintenance.auto=false` or `gc.auto=0` has opted out. Measured on a 36,001-loose-ref fixture (macOS/APFS, git 2.44): `show-ref` 5.5-12.2s -> 30-49ms, `for-each-ref` 4.0-10.8s -> 43-48ms. Also fixes a pre-existing bug the split exposed: `--path-format=absolute` is ignored before git 2.31, and taking rev-parse's stdout raw collapsed every repo on such a host onto one fetch-serialization key. Refs #17828 * perf(git): make idle ref maintenance preemptible and cheaper to probe The idle veto was one-directional: it stopped a pack from starting during a create, removal, or agent work, but nothing stopped those from starting during a pack. A user-clicked Fetch, a branch delete, or a worktree removal that needed `packed-refs.lock` mid-rewrite could fail with `unable to create packed-refs.lock` -- a git error with no visible cause. Make the pack cancellable end to end. An AbortSignal now reaches the `pack-refs` child and both pre-pack probes, and `pause()` aborts what is running, waits for it to actually stop, and holds a suspension count so nothing new starts until the caller releases. Every entry point that deletes a ref takes that pause: gitFetch, gitPull, gitFastForward, removeWorktree, forceDeleteLocalBranch, prepareWorktreeCreateCheckout, addWorktree. Five more triggers close the rest of the window: battery drop, window focus, quit, the attempt deadline, and any other git command queueing for an admission slot. Judge a pack by re-probing the backlog rather than by the child's exit code. Measured in the field: another Orca session moved a branch mid-pack, git reported `cannot lock ref`, skipped that ref and packed the rest -- 36,688 loose refs down to 3. On a machine running several sessions that is the normal case, and retrying it would be wrong. Probe with one batched `readdir` per directory instead of streaming `opendir`, which issues a thread-pool round trip every 32 entries: 177ms -> 23ms on a real 36,600-ref repository, with half the event-loop lag. The walk stays strictly sequential so it can never occupy more than one of libuv's four filesystem threads. `PackRefsLockOwnership` makes a lock left by SIGKILL attributable, and only reclaims one when a marker exists, the lock is older than any pack-refs could run for, and the recorded process is gone. Refs #17828 * fix(git): wait out the packed-refs lock instead of killing the pack Measured on Git 2.55/APFS with 37k loose refs: a full `pack-refs --all --prune` takes 23-32s but holds `packed-refs.lock` for only 0.03-1.37s of it. The other ~95% is the prune phase, during which a concurrent `fetch --prune`, `branch -D` or `update-ref` succeeds every time -- per-ref locks last microseconds and git retries for `core.filesRefLockTimeout`. So the abort-on-everything design was strictly harmful. SIGTERM into the prune loop strands an empty `refs/**/*.lock` about one time in five (9/30, 5/40, 6/30 kills): `tempfile.c` opens the lock O_EXCL before `activate_tempfile()` links it into the list the signal handler walks, and a pack does ~36k lock cycles. Afterwards `update-ref -d` on that ref fails with `cannot lock ref ... File exists`, permanently. On Windows `taskkill /f` never runs git's handlers at all, so an abort inside the rewrite strands `packed-refs.lock` every time. Never signal the child. `packRefs` no longer takes an abort signal; it polls `packed-refs.lock` and reports the window through a `PackedRefsLockReporter`. `pause()` resolves when the lock is released -- bounded, and free during the prune -- while the suspension counter still blocks new attempts. Battery and window-focus become do-not-start rather than stop-what-is-running, and quit waits for the lock and lets the child finish orphaned. For strands that already exist, `PackRefsLockOwnership` now also reclaims `refs/**/*.lock` under the same three conditions plus a 0-byte check, and a lock carrying our own not-yet-reclaimable marker records `locked` with a 30min retry instead of the 6h failure cooldown -- so a Windows strand self-heals in half an hour rather than six. Reverts the git admission-scheduler event bus, which existed only to drive the abort this removes. Refs #17828 * test(git): make the ref-maintenance waits survive a loaded runner CI shard 4/8 failed on `restarts every armed countdown when the user does ref work themselves`, which passes locally. The `until()` helper spun a fixed 200 event-loop turns and then returned silently, so on a contended runner the filesystem probe had not finished and the assertion that followed failed with an unrelated message. Bound the wait by wall clock instead and throw a named error, which immediately exposed a second latent bug: the single-flight test's second wait could never succeed, because the deferred repo's retry is on a faked `setTimeout` that spinning the real loop never advances. It had been passing only because the old helper gave up quietly. Add a timer-aware variant for those, and have the countdown test await a signal the fake pack resolves rather than polling at all. Verified stable across five sequential runs and once under load average 32 with six concurrent suites. Refs #17828
184 lines
6.8 KiB
TypeScript
184 lines
6.8 KiB
TypeScript
import type { RemoveWorktreeResult } from '../../shared/worktree/create-types'
|
|
import { assertWorktreeUnlockedForRemoval } from '../../shared/worktree/removal'
|
|
import { isSubmoduleWorktreeRemovalRefusal } from '../../shared/worktree/submodule-removal'
|
|
import { withWorktreeRemoveStageSpan } from '../observability/instrumentation'
|
|
import { withSpan } from '../observability/tracer'
|
|
import {
|
|
moveWorktreeDirectoryToTrash,
|
|
restoreWorktreeDirectoryFromTrash,
|
|
scheduleWorktreeTrashDeletion
|
|
} from '../worktree-trash'
|
|
import { parseWslPath } from '../wsl'
|
|
import { gitExecFileAsync } from './runner'
|
|
import { runWithGitReadCacheInvalidation } from './status'
|
|
import { deleteBranchAfterWorktreeRemoval } from './worktree-branch-removal'
|
|
import { listWorktreesStrict } from './worktree-listing'
|
|
import { invalidateWslLinkedWorktreeGitRouting } from './wsl-linked-worktree-git-routing'
|
|
import type { RemoveWorktreeOptions } from './worktree-operation-options'
|
|
import {
|
|
WORKTREE_REMOVAL_REGISTRATION_TIMEOUT_MS,
|
|
gitExecOptions,
|
|
normalizeLocalBranchRef
|
|
} from './worktree-operation-options'
|
|
import { areWorktreePathsEqual } from './worktree-path-comparison'
|
|
import { assertWorktreeCleanForRemoval } from './worktree-removal-preflight'
|
|
import { withRepoRefMaintenancePaused } from './local-repo-ref-maintenance'
|
|
import { bumpWorktreeScanGeneration, listWorktrees } from './worktree-scan-cache'
|
|
import { invalidateSparseCheckoutState } from './worktree-sparse-checkout-cache'
|
|
|
|
/**
|
|
* Remove a worktree.
|
|
*/
|
|
export async function removeWorktree(
|
|
repoPath: string,
|
|
worktreePath: string,
|
|
force = false,
|
|
// forceBranchDelete: for failed-creation rollback (fresh branch, no user work); user deletes leave it false so unmerged commits survive.
|
|
options: RemoveWorktreeOptions = {}
|
|
): Promise<RemoveWorktreeResult> {
|
|
try {
|
|
// Removal deletes branches, and a ref deletion needs the packed-refs lock a
|
|
// running idle pack holds while it rewrites. Waits that window out; the
|
|
// prune phase that follows it is concurrency-safe and is left to finish.
|
|
return await withRepoRefMaintenancePaused('worktree-remove', () =>
|
|
runWithGitReadCacheInvalidation(() =>
|
|
performRemoveWorktree(repoPath, worktreePath, force, options)
|
|
)
|
|
)
|
|
} finally {
|
|
invalidateWslLinkedWorktreeGitRouting(worktreePath)
|
|
invalidateSparseCheckoutState(repoPath, worktreePath)
|
|
bumpWorktreeScanGeneration(repoPath)
|
|
}
|
|
}
|
|
|
|
async function performRemoveWorktree(
|
|
repoPath: string,
|
|
worktreePath: string,
|
|
force = false,
|
|
options: RemoveWorktreeOptions = {}
|
|
): Promise<RemoveWorktreeResult> {
|
|
const removedWorktree =
|
|
options.knownRemovedWorktree ??
|
|
(await listWorktrees(repoPath, options)).find((worktree) =>
|
|
areWorktreePathsEqual(worktree.path, worktreePath)
|
|
)
|
|
const branchName = normalizeLocalBranchRef(removedWorktree?.branch ?? '')
|
|
const branchHead = removedWorktree?.head ?? ''
|
|
|
|
// Why: callers outside the IPC/runtime preflight must not bypass Git's lock contract or rely on localized stderr after side effects.
|
|
assertWorktreeUnlockedForRemoval(removedWorktree)
|
|
|
|
if (
|
|
!(await tryRemoveWorktreeWithDeferredDirectoryDeletion(repoPath, worktreePath, force, options))
|
|
) {
|
|
const args = ['worktree', 'remove']
|
|
if (force) {
|
|
args.push('--force')
|
|
}
|
|
args.push(worktreePath)
|
|
try {
|
|
await gitExecFileAsync(args, gitExecOptions(repoPath, options))
|
|
} catch (error) {
|
|
if (force || !isSubmoduleWorktreeRemovalRefusal(error)) {
|
|
throw error
|
|
}
|
|
// Why: Git refuses non-force removal of a worktree with an initialised submodule even when clean; re-prove cleanliness, then --force.
|
|
await assertWorktreeCleanForRemoval(worktreePath, false, options)
|
|
await gitExecFileAsync(
|
|
['worktree', 'remove', '--force', worktreePath],
|
|
gitExecOptions(repoPath, options)
|
|
)
|
|
}
|
|
}
|
|
|
|
if (!branchName) {
|
|
return {}
|
|
}
|
|
if (options.deleteBranch === false) {
|
|
return {}
|
|
}
|
|
|
|
// Why its own span: branch cleanup can reach the network (`fetch --prune`), so a stall here reads as
|
|
// `git worktree remove` being slow unless it is timed separately.
|
|
return withSpan('worktree.remove.branch_delete', () =>
|
|
deleteBranchAfterWorktreeRemoval(repoPath, branchName, branchHead, options)
|
|
)
|
|
}
|
|
|
|
/**
|
|
* Rename the checkout into a sibling trash directory and clear Git's registration for the
|
|
* now-missing path, so the multi-GB recursive delete runs after this removal has returned.
|
|
* Returns false when the caller must let `git worktree remove` delete the directory inline.
|
|
*/
|
|
async function tryRemoveWorktreeWithDeferredDirectoryDeletion(
|
|
repoPath: string,
|
|
worktreePath: string,
|
|
force: boolean,
|
|
options: RemoveWorktreeOptions
|
|
): Promise<boolean> {
|
|
// Why: WSL-owned checkouts are deleted inside the distro, so Node on Windows must not rename them.
|
|
if (options.wslDistro || parseWslPath(worktreePath)) {
|
|
return false
|
|
}
|
|
if (!force) {
|
|
try {
|
|
// Why: `git worktree remove` re-checks cleanliness as it removes; prove the same thing here or leave removal to Git.
|
|
await assertWorktreeCleanForRemoval(worktreePath, false, options)
|
|
} catch {
|
|
return false
|
|
}
|
|
}
|
|
|
|
const trashPath = await withWorktreeRemoveStageSpan('trash_rename', 'local', () =>
|
|
moveWorktreeDirectoryToTrash(worktreePath)
|
|
)
|
|
if (!trashPath) {
|
|
return false
|
|
}
|
|
try {
|
|
await clearGitRegistrationForMissingWorktree(repoPath, worktreePath, options)
|
|
} catch (error) {
|
|
// Why: put the checkout back so the in-place removal below still sees the worktree Git registered.
|
|
if (await restoreWorktreeDirectoryFromTrash(trashPath, worktreePath)) {
|
|
return false
|
|
}
|
|
throw error
|
|
}
|
|
scheduleWorktreeTrashDeletion(trashPath)
|
|
return true
|
|
}
|
|
|
|
async function clearGitRegistrationForMissingWorktree(
|
|
repoPath: string,
|
|
worktreePath: string,
|
|
options: RemoveWorktreeOptions
|
|
): Promise<void> {
|
|
const registrationOptions = {
|
|
...options,
|
|
timeout: options.timeout ?? WORKTREE_REMOVAL_REGISTRATION_TIMEOUT_MS
|
|
}
|
|
try {
|
|
// Removing an already-missing directory is accepted back to the Git 2.25 baseline and touches only this entry.
|
|
await gitExecFileAsync(
|
|
['worktree', 'remove', '--force', worktreePath],
|
|
gitExecOptions(repoPath, registrationOptions)
|
|
)
|
|
return
|
|
} catch (error) {
|
|
console.warn(
|
|
`[git] Failed to deregister the moved worktree "${worktreePath}"; pruning instead`,
|
|
error
|
|
)
|
|
}
|
|
|
|
await gitExecFileAsync(['worktree', 'prune'], gitExecOptions(repoPath, registrationOptions))
|
|
// Strict (not the shared scan): an unreadable repo must not read as proof that the row is gone.
|
|
const stillRegistered = (await listWorktreesStrict(repoPath, registrationOptions)).some(
|
|
(worktree) => areWorktreePathsEqual(worktree.path, worktreePath)
|
|
)
|
|
if (stillRegistered) {
|
|
throw new Error(`Git still reports a registration for "${worktreePath}" after pruning it.`)
|
|
}
|
|
}
|