Files
orca/src/main/ipc/worktree-apfs-clone.ts
T
Brennan Benson b31e9bb03d fix(worktree): bound the .worktreeinclude copy so a huge include can't freeze workspace creation (#10540)
* fix(worktree): bound the .worktreeinclude copy so a huge include can't freeze creation

`.worktreeinclude` copying was bounded in entry count (1000) but unbounded in
bytes and files, and awaited inline during worktree creation. A repo listing
`node_modules` froze creation for minutes behind the create dialog on Linux and
Windows, where the fallback is a full `fs.cp` (macOS gets a cheap APFS clone).

Measure each copy-mode source against a cumulative budget (2 GB / 50k files)
before the first byte is written, and refuse the entries that bust it. Refused
entries ride the existing `CreateWorktreeResult.warning` channel so a workspace
never silently comes up missing its included files.

Pre-measurement rather than mid-copy abort: `fs.cp` ignores its `signal`
option, so a started copy cannot be cancelled and would strand a partial tree.
Refusing up front means there is no partial state to clean up.

* fix(worktree): don't charge bytes for copy-on-write clones, and bound the sizing walk

Two defects in the copy budget, both found by review:

- The byte limit was applied on macOS, where the copy is an APFS clonefile.
  Measured: a 2.7 GB tree clones in 22 ms and consumes no disk. Refusing it on
  a 2 GB byte ceiling denied work that was already free — a regression on the
  one platform this bound was never meant to touch. Bytes are now charged only
  when a byte-for-byte copy will actually run; the volume probe that decides
  this is the same cached df+diskutil pair the clone runs, and writes nothing,
  so the "refuse before the first byte" invariant holds. The entry limit still
  applies everywhere: inodes are real work even on the clone path.

- A refused entry consumed no budget, so a `.worktreeinclude` listing many
  over-budget directories paid a fresh full-limit walk for each one — up to
  1000 x 50,000 lstat calls, re-creating the stall this bounds. The walk is now
  charged against its own ceiling whatever the verdict.

Also documents that `admit()` must be awaited sequentially (CodeRabbit).

* fix(worktree): give the sizing walk headroom so one huge entry can't starve the rest

The walk ceiling added in the previous commit was seeded with maxEntries, the
same number the entry limit uses. Sizing an entry that busts the file-count
limit walks maxEntries + 1, driving the ceiling negative, so every later
`.worktreeinclude` entry was refused without being measured at all.

That regressed the common case: a repo listing `node_modules` plus `.env` used
to get `.env`; it silently got nothing. Reproduced, and now covered by a test
that fails when the headroom is removed.

The walk now gets 5x the entry budget, so total sizing work stays bounded
(<=250k lstat per materialization, vs the 1000 x 50k this ceiling exists to
prevent) while ordinary lists never reach it. Entries refused because earlier
ones exhausted the walk report a distinct 'sizing' reason, so the warning stops
quoting size limits at a 4-byte file that was never measured.

* fix(worktree): bill a failed clone's bytes, and blame the right ceiling

Two follow-on defects from the copy-on-write fix:

- A predicted APFS clone that then failed mid-copy (EPERM, ENOSPC) fell through
  to a real `fs.cp` whose bytes were never charged, because the entry had been
  admitted on the premise that cloning is free. That reopened the unbounded
  copy on macOS. The measured size is already known, so the fallback now bills
  it and refuses if it no longer fits, reporting the entry as skipped instead
  of silently copying gigabytes. A clone that was never viable
  (ApfsCloneUnavailableError) was already charged as a real copy, so that path
  keeps falling back as before.

- The walk ceiling is also applied inside the measurement via
  min(remainingEntries, remainingWalk), and when the walk term bound, the
  refusal was still reported as 'entries' — telling the user a 3-file directory
  busted a 4-file limit. It now attributes to whichever ceiling actually bound.

Also fixes the singular warning text, which said "entry X was not copied ...
copying them would exceed ... Copy them in manually".

* fix(worktree): flag a partial clone leftover, cap the warning, cover two branches

- A clone that fails partway only removes an *empty* reservation, so leftovers
  can survive at the target. Reporting that entry as simply "not copied" sent
  the user to copy it in manually, straight into a half-populated directory.
  Those skips now carry mayBePartial and the warning says to check the path
  first. Cleaning up the leftovers stays the deferred follow-up it already was.

- The warning enumerated every skipped path. `.worktreeinclude` allows 1000
  entries and all of them can be skipped, so it now names five and counts the
  rest — an unbounded string is a poor look in a PR about bounds.

- Two load-bearing branches had no test, both proven by surviving mutants: the
  `bytesAreCopied` short-circuit (reachable when a wedged df/diskutil makes the
  volume probe answer "no clone", so bytes are charged up front and must not be
  billed twice), and chargeBytes actually consuming budget for later entries.

* fix(worktree): only flag directory clones as partial, and cap that list too

- mayBePartial was set for every refused clone fallback, but only a *directory*
  clone can leave anything behind: the file path clones into a temp name and
  publishes with link(2), so a failure leaves nothing at the target. Sending
  the user to inspect a path that does not exist is its own small lie.

- The partial-copy sentence sliced to five names without the "and N more" that
  the other sentence appends, so entries past the fifth were surfaced nowhere.
  Both sentences now share one nameList helper.
2026-07-25 05:11:03 -07:00

217 lines
7.0 KiB
TypeScript

import { execFile, type ExecFileOptions } from 'node:child_process'
import { randomUUID } from 'node:crypto'
import { mkdir, stat, rm, link, rmdir, chmod } from 'node:fs/promises'
import { dirname, resolve, sep } from 'node:path'
import { promisify } from 'node:util'
type ExecFileAsync = (
file: string,
args: readonly string[],
options?: Pick<ExecFileOptions, 'timeout'>
) => Promise<{ stdout: string; stderr: string }>
const execFileAsync = promisify(execFile) as ExecFileAsync
// Why: bound the df/diskutil volume probes so a wedged mount can't stall worktree creation.
const APFS_FILESYSTEM_PROBE_TIMEOUT_MS = 5_000
export type ApfsCloneDeps = {
execFileAsync: ExecFileAsync
randomUUID: () => string
}
export const defaultApfsCloneDeps: ApfsCloneDeps = {
execFileAsync,
randomUUID
}
type DarwinFilesystemInfo = {
device: string
filesystemName: string
}
/** Per-materialization cache keyed by `stat().dev`. Copying N `.worktreeinclude`
* paths would otherwise re-run df+diskutil per path (4 subprocesses each) even
* though source and worktree almost always share one volume; caching collapses
* that to one probe per distinct volume. */
export type DarwinFilesystemCache = Map<number, Promise<DarwinFilesystemInfo>>
export class ApfsCloneUnavailableError extends Error {
constructor(message: string) {
super(message)
this.name = 'ApfsCloneUnavailableError'
}
}
export class WorktreeLinkedPathTargetExistsError extends Error {
constructor(target: string) {
super(`Worktree linked path target already exists: ${target}`)
this.name = 'WorktreeLinkedPathTargetExistsError'
}
}
function isAlreadyExistsError(error: unknown): boolean {
return (error as { code?: unknown })?.code === 'EEXIST'
}
async function getDarwinFilesystemInfo(
path: string,
deps: ApfsCloneDeps
): Promise<DarwinFilesystemInfo> {
const { stdout: dfOutput } = await deps.execFileAsync('/bin/df', ['-P', path], {
timeout: APFS_FILESYSTEM_PROBE_TIMEOUT_MS
})
const device = dfOutput.trim().split(/\r?\n/)[1]?.trim().split(/\s+/)[0]
if (!device) {
throw new Error(`Could not resolve filesystem device for ${path}`)
}
const { stdout: diskutilOutput } = await deps.execFileAsync(
'/usr/sbin/diskutil',
['info', '-plist', device],
{ timeout: APFS_FILESYSTEM_PROBE_TIMEOUT_MS }
)
const filesystemNameMatch = /<key>FilesystemName<\/key>\s*<string>([^<]+)<\/string>/u.exec(
diskutilOutput
)
return {
device,
filesystemName: filesystemNameMatch?.[1] ?? ''
}
}
async function getCachedDarwinFilesystemInfo(
path: string,
deps: ApfsCloneDeps,
cache: DarwinFilesystemCache
): Promise<DarwinFilesystemInfo> {
const deviceId = (await stat(path)).dev
const cached = cache.get(deviceId)
if (cached) {
return cached
}
// Why: cache the pending (or rejected) probe so every path on this volume
// reuses one df+diskutil pair instead of respawning them per copy.
const pending = getDarwinFilesystemInfo(path, deps)
cache.set(deviceId, pending)
return pending
}
async function isSameApfsVolume(
source: string,
targetDirectory: string,
deps: ApfsCloneDeps,
cache: DarwinFilesystemCache
): Promise<boolean> {
const [sourceInfo, targetInfo] = await Promise.all([
getCachedDarwinFilesystemInfo(source, deps, cache),
getCachedDarwinFilesystemInfo(targetDirectory, deps, cache)
])
return (
sourceInfo.device === targetInfo.device &&
sourceInfo.filesystemName === 'APFS' &&
targetInfo.filesystemName === 'APFS'
)
}
async function assertSameApfsVolume(
source: string,
target: string,
deps: ApfsCloneDeps,
cache: DarwinFilesystemCache
): Promise<void> {
if (!(await isSameApfsVolume(source, dirname(target), deps, cache))) {
throw new ApfsCloneUnavailableError(
'APFS clone-copy requires source and target on the same APFS volume'
)
}
}
/** Whether copying `source` into `targetDirectory` would take the clonefile
* path. Pure probe: it reuses the cached df+diskutil pair the clone itself
* runs and writes nothing, so a caller can size the work before any bytes
* land. A failed probe answers "no", matching the clone's own fallback to a
* real copy. */
export async function canCloneWithApfs(
source: string,
targetDirectory: string,
deps: ApfsCloneDeps = defaultApfsCloneDeps,
filesystemCache: DarwinFilesystemCache = new Map()
): Promise<boolean> {
try {
return await isSameApfsVolume(source, targetDirectory, deps, filesystemCache)
} catch {
return false
}
}
async function cloneFileWithApfs(
source: string,
target: string,
deps: ApfsCloneDeps
): Promise<void> {
const tempTarget = resolve(dirname(target), `.orca-apfs-clone-${deps.randomUUID()}`)
try {
await deps.execFileAsync('/bin/cp', ['-c', source, tempTarget])
try {
// Why: link(2) is an atomic no-clobber publish for files; rename(2) can
// overwrite a target that appeared after the earlier existence check.
await link(tempTarget, target)
} catch (error) {
if (isAlreadyExistsError(error)) {
throw new WorktreeLinkedPathTargetExistsError(target)
}
throw error
}
} finally {
await rm(tempTarget, { force: true }).catch(() => undefined)
}
}
async function cloneDirectoryWithApfs(
source: string,
target: string,
deps: ApfsCloneDeps
): Promise<void> {
const sourceMode = (await stat(source)).mode & 0o777
try {
// Why: reserve the final directory path before copying into it so a raced
// user-created directory cannot be replaced by a final rename.
await mkdir(target)
} catch (error) {
if (isAlreadyExistsError(error)) {
throw new WorktreeLinkedPathTargetExistsError(target)
}
throw error
}
try {
// Why: the top-level directory is reserved before cp runs, so use `-n`
// to keep a raced nested file from being overwritten during the copy.
// Why: copy `source/.` into the reserved target so contents land at the
// requested path even when the source is a symlinked directory.
await deps.execFileAsync('/bin/cp', ['-n', '-c', '-R', `${source}${sep}.`, target])
await chmod(target, sourceMode)
} catch (error) {
// Why: remove only the empty reservation. If cp wrote anything, or another
// process raced files into the directory, leave it for Git/user review.
await rmdir(target).catch(() => undefined)
throw error
}
}
export async function cloneWorktreePathWithApfs(
source: string,
target: string,
sourceIsDirectory: boolean,
deps: ApfsCloneDeps = defaultApfsCloneDeps,
filesystemCache: DarwinFilesystemCache = new Map()
): Promise<void> {
await mkdir(dirname(target), { recursive: true })
await assertSameApfsVolume(source, target, deps, filesystemCache)
// Why: Node's COPYFILE_FICLONE_FORCE returns ENOSYS on macOS in our runtime,
// while Darwin's cp exposes APFS clonefile via -c. Preflight the volume so
// cp's non-APFS full-copy fallback cannot surprise users.
await (sourceIsDirectory
? cloneDirectoryWithApfs(source, target, deps)
: cloneFileWithApfs(source, target, deps))
}