Files
orca/src/shared/git-object-quarantine.test.ts
T
Brennan BensonandClaude 0fea878680 fix(git): keep merge-tree's throwaway trees out of the real object store (#22356)
* fix(git): keep merge-tree's throwaway trees out of the real object store

The PR conflict summary and the squash-merged branch cleanup both run
`git merge-tree --write-tree` only to read its output, but every divergent
run left unreachable loose objects behind. Run it with a per-call scratch
object directory (inside objects/, alternates pointing at the real store)
that is deleted afterwards, on native, WSL and SSH relay hosts alike.

* fix(git): keep partial-clone downloads and clean up stranded scratch stores

- A partial clone fetches missing blobs during merge-tree; those packs landed in the
  scratch store and were deleted, so every check re-downloaded them. Move fetched packs
  (index last) into the real store before removing the scratch dir.
- Sweep this app's own scratch dirs older than an hour, once per objects dir per process,
  so a crash or failed delete cannot strand one forever in repos where gc never runs.
- Remove with Windows transient-lock retries; the shared core now owns all filesystem
  work and host adapters only resolve the objects dir.

* fix(git): quote Windows alternates paths containing ';' and log scratch cleanup failures

* fix(git): sweep a merge-tree scratch dir only when its owner is provably gone

Another Orca process on the same repo (a second instance, dev and release
builds) could delete a scratch object dir still in use once its mtime passed
an hour. Each scratch dir now records its owner (pid, hostname, platform,
Linux pid namespace); the sweep removes it only when that owner shares this
pid space and has exited, or, with no checkable owner, at Git's own two-week
tmp_ expiry.

Also:
- A fetched pack whose later rename fails is rolled back, so the real store
  never keeps an index-less pack; a pack already installed is left alone.
- WSL Git gets the /mnt/<drive> spelling of a drive-path common dir (Git
  < 2.31 prints it relative, resolved against the C:\ repo path).
- conflict-summary tests no longer create scratch dirs in the cwd.

* fix(git): cap every merge-tree scratch dir at two weeks, live-looking owner included

A reused pid held by a long-lived process could keep a leftover scratch dir
forever in a repo where git gc never runs. The two-week age (git gc's own
tmp_ expiry) now removes any scratch dir; a younger one is still swept only
when its owner shares this pid space and has exited.

* fix(git): sweep merge-tree scratch dirs by age alone, off the check's path

- The stale sweep now removes any scratch dir at least two weeks old, Git's
  own expiry for its objects/tmp_* dirs; no merge-tree runs that long. The
  owner file and pid-space probe are gone, and with them the cases where a
  sweeper misjudged another process's pid.
- The sweep no longer delays the user's check: it starts without being
  awaited and can only touch dirs far older than the current run's.
- A fetched pack's .keep never moves into the real store, so a transient one
  from an aborted lazy fetch cannot stop gc from repacking that pack.
- Tests: the legacy merge-tree fallback gets the scratch env; the local
  quarantine's WSL decisions (distro from a UNC path, drvfs spelling,
  host-Git linked-worktree skip, WSLENV hand-off).

* fix(git): let quarantined merge-tree see exactly the object stores Git sees without it

The scratch object store must not change what merge-tree can read, or the
conflict list and the squash-merged branch cleanup differ from main:

- An inherited GIT_ALTERNATE_OBJECT_DIRECTORIES (from Orca's or the SSH
  relay's environment) was replaced; it is now kept and the real store
  appended after it, as Git's own temporary object stores do.
- An inherited GIT_OBJECT_DIRECTORY is the store Git reads and writes, so the
  command now runs unquarantined. On the WSL route a Windows value only
  counts if WSLENV forwards it, and then the quarantine is skipped too.
- The real store's own info/alternates entries are listed before it, so an
  alternates chain is linked at the same depth as without the quarantine
  (a 6-deep chain no longer hits Git's nesting limit). An alternates file
  that cannot be read or holds a quoted entry runs unquarantined.

* fix(git): run merge-tree unquarantined when the repo's store has its own alternates

Mirroring Git's alternates resolution could not match it in every shape: a
path back to the real store (mutual alternates, or an inherited alternates
value naming the repo itself) still linked its chain one level deeper, and a
NUL byte in info/alternates broke the spawn. When objects/info/alternates
exists, or cannot be checked, the command now runs exactly as before. Repos
cloned with --reference or --shared keep writing throwaway objects, as they
always have. Inherited alternates are still appended to, and an inherited
GIT_OBJECT_DIRECTORY still skips the quarantine.

* refactor(git): drop the stale-folder sweep and pack-move rollback from merge-tree's scratch store

A scratch folder left by a crash is a tmp_* dir inside objects/, which
Git's own gc prunes after gc.pruneExpire, as it does Git's own leftovers,
so Orca's 14-day sweep and its test hooks go.

Packs a partial clone fetched into the scratch store are moved in Git's
migration order: pack- files only, index last, existing names left in
place, stopping at the first failed rename with no rollback. A failed
move is logged and never stops the scratch dir's removal or the result.

Adds a real-Git test that a partial clone lists its conflicts offline
once an earlier check downloaded the contents.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(git): quarantine merge-tree only once the host is known to support --write-tree

Git before 2.35 cannot prune a leftover scratch folder under objects/tmp_*, so
a crash in the moment one exists makes every later `git gc` warn and stops
automatic cleanup. On Git before 2.38 merge-tree --write-tree is rejected, yet
the first check per host per session still made and removed a folder around it.

Branch cleanup and the PR conflict list now use the scratch object store only
when that host's capability cache already knows merge-tree --write-tree works.
Until then the command runs exactly as it does on main: on a Git that
rejects it no folder is ever created, and every Git that does get one is 2.38+,
whose gc prunes it. Also reword two comments in the quarantine module.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(git): learn merge-tree support from a conflicting branch-cleanup check

Branch cleanup quarantines merge-tree only once the host's capability cache
knows --write-tree works. A conflicting merge-tree exits 1, so the check threw
and support was never recorded; on a host whose only checks conflict (likely
over SSH, where the relay has no conflict summary to teach its cache) every
check kept writing loose objects into the real store.

A failure whose stdout starts with a tree ID proves --write-tree ran (old Git
that rejects the option prints nothing there), so it now resolves to the same
"not merged" result and lets the cache record support. Every other failure is
rethrown unchanged. Also shorten two comments and soften a third.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(git): put WSL Git's scratch object store on its command line

A WSL login profile that set GIT_ALTERNATE_OBJECT_DIRECTORIES replaced the
WSLENV-forwarded value, so a quarantined merge-tree lost the real store and
failed where main succeeds. The WSL runner now passes both variables as
command-line assignments after the profile, appending to its alternates.

* test(git): drop quarantine tests the real-Git and compatibility tests already pin

The cold-capability gating, scratch-folder lifecycle and WSL path spelling
are covered end to end by merge-tree-object-quarantine-real-git.test.ts, the
real-binary compatibility contract (Git 2.25/2.38/2.49) and the
localGitObjectsDirectory tests, so the unit copies only mirrored them.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-15 15:23:18 -07:00

213 lines
7.1 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import {
chmodSync,
existsSync,
mkdirSync,
mkdtempSync,
readdirSync,
readFileSync,
rmSync,
writeFileSync
} from 'node:fs'
import type * as NodeFsPromises from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { basename, join } from 'node:path'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import {
createGitObjectQuarantine,
GIT_OBJECT_QUARANTINE_DIR_PREFIX,
type GitObjectQuarantineEnv
} from './git-object-quarantine'
const { failedRenameTargets, renamedTargets } = vi.hoisted(() => {
const renamedTargets: string[] = []
return { failedRenameTargets: new Set<string>(), renamedTargets }
})
vi.mock('node:fs/promises', async (importOriginal) => {
const actual = await importOriginal<typeof NodeFsPromises>()
return {
...actual,
rename: async (from: string, to: string) => {
if (failedRenameTargets.has(basename(to))) {
throw Object.assign(new Error('EBUSY: resource busy or locked'), { code: 'EBUSY' })
}
await actual.rename(from, to)
renamedTargets.push(basename(to))
}
}
})
describe('createGitObjectQuarantine', () => {
let root: string
let objects: string
beforeEach(() => {
root = mkdtempSync(join(tmpdir(), 'orca-object-quarantine-'))
objects = join(root, 'objects')
mkdirSync(join(objects, 'pack'), { recursive: true })
})
afterEach(() => {
failedRenameTargets.clear()
renamedTargets.length = 0
vi.restoreAllMocks()
rmSync(root, { recursive: true, force: true })
})
const resolveNative = () => async () => ({ hostPath: objects, gitPath: objects })
const scratchDirs = (): string[] =>
readdirSync(objects).filter((entry) => entry.startsWith(GIT_OBJECT_QUARANTINE_DIR_PREFIX))
it.each([
['the objects dir is unknown', async () => undefined],
[
'resolving the objects dir fails',
async () => {
throw new Error('x')
}
],
[
'the scratch dir cannot be created',
async () => ({ hostPath: '/no/such/objects', gitPath: '/no/such/objects' })
]
])('runs the command unquarantined when %s', async (_label, resolve) => {
const command = vi.fn(async () => 'ok')
await expect(createGitObjectQuarantine(resolve).run(command)).resolves.toBe('ok')
expect(command).toHaveBeenCalledWith(undefined)
})
it.each([
['/plain/objects', '/plain/objects'],
['/odd:dir/objects', '"/odd:dir/objects"'],
['/quo"te\\dir/objects', '"/quo\\"te\\\\dir/objects"'],
['C:\\repo\\.git\\objects', 'C:\\repo\\.git\\objects'],
['C:\\a;b\\.git\\objects', '"C:\\\\a;b\\\\.git\\\\objects"']
])('spells alternates for %s so Git reads one entry', async (gitPath, expected) => {
let seen: GitObjectQuarantineEnv | undefined
await createGitObjectQuarantine(async () => ({ hostPath: objects, gitPath })).run(
async (env) => {
seen = env
}
)
expect(seen?.GIT_ALTERNATE_OBJECT_DIRECTORIES).toBe(expected)
})
it.each([
['/repo/.git/objects', '/shared/objects:/more', '/shared/objects:/more:/repo/.git/objects'],
['C:\\repo\\.git\\objects', 'D:\\shared', 'D:\\shared;C:\\repo\\.git\\objects'],
[
'\\\\server\\share\\repo\\.git\\objects',
'\\\\server\\shared',
'\\\\server\\shared;\\\\server\\share\\repo\\.git\\objects'
]
])(
'appends the real store to inherited alternates for %s, as Git’s own temporary stores do',
async (gitPath, inheritedAlternates, expected) => {
let seen: GitObjectQuarantineEnv | undefined
await createGitObjectQuarantine(async () => ({
hostPath: objects,
gitPath,
inheritedAlternates
})).run(async (env) => {
seen = env
})
expect(seen?.GIT_ALTERNATE_OBJECT_DIRECTORIES).toBe(expected)
}
)
it.each([
['has an alternates file', (info: string) => writeFileSync(join(info, 'alternates'), '')],
[
'has an alternates file this process cannot inspect',
(info: string) => {
writeFileSync(join(info, 'alternates'), '/elsewhere/objects\n')
chmodSync(info, 0o000)
}
]
])('runs unquarantined when the real store %s', async (_label, prepare) => {
const info = join(objects, 'info')
mkdirSync(info)
prepare(info)
const command = vi.fn(async () => 'ok')
try {
await expect(createGitObjectQuarantine(resolveNative()).run(command)).resolves.toBe('ok')
} finally {
chmodSync(info, 0o755)
}
expect(command).toHaveBeenCalledWith(undefined)
expect(scratchDirs()).toEqual([])
})
it('moves packs Git fetched into the scratch dir to the real store, index last', async () => {
await createGitObjectQuarantine(resolveNative()).run(async (env) => {
const scratchPack = join(env?.GIT_OBJECT_DIRECTORY ?? '', 'pack')
mkdirSync(scratchPack)
for (const file of ['pack-abc1.idx', 'pack-abc1.pack', 'pack-abc1.promisor']) {
writeFileSync(join(scratchPack, file), file)
}
// A pack without its index is invisible to Git, so moving it is harmless.
writeFileSync(join(scratchPack, 'pack-def2.pack'), 'partial')
writeFileSync(join(scratchPack, 'tmp_pack_XYZ'), 'partial')
mkdirSync(join(env?.GIT_OBJECT_DIRECTORY ?? '', 'ab'))
writeFileSync(join(env?.GIT_OBJECT_DIRECTORY ?? '', 'ab', 'cdef'), 'merge-tree output')
})
expect(readdirSync(join(objects, 'pack')).sort()).toEqual([
'pack-abc1.idx',
'pack-abc1.pack',
'pack-abc1.promisor',
'pack-def2.pack'
])
expect(renamedTargets.at(-1)).toBe('pack-abc1.idx')
expect(existsSync(join(objects, 'ab'))).toBe(false)
expect(scratchDirs()).toEqual([])
})
it('leaves a pack the real store already has untouched', async () => {
writeFileSync(join(objects, 'pack', 'pack-abc1.pack'), 'installed')
writeFileSync(join(objects, 'pack', 'pack-abc1.idx'), 'installed')
await createGitObjectQuarantine(resolveNative()).run(async (env) => {
const scratchPack = join(env?.GIT_OBJECT_DIRECTORY ?? '', 'pack')
mkdirSync(scratchPack)
writeFileSync(join(scratchPack, 'pack-abc1.pack'), 'refetched')
writeFileSync(join(scratchPack, 'pack-abc1.idx'), 'refetched')
})
expect(readFileSync(join(objects, 'pack', 'pack-abc1.pack'), 'utf8')).toBe('installed')
expect(readFileSync(join(objects, 'pack', 'pack-abc1.idx'), 'utf8')).toBe('installed')
})
it('stops at a failed pack move, still removing the scratch dir and returning the result', async () => {
failedRenameTargets.add('pack-abc1.pack')
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {})
await expect(
createGitObjectQuarantine(resolveNative()).run(async (env) => {
const scratchPack = join(env?.GIT_OBJECT_DIRECTORY ?? '', 'pack')
mkdirSync(scratchPack)
for (const file of ['pack-abc1.idx', 'pack-abc1.pack']) {
writeFileSync(join(scratchPack, file), file)
}
return 'ok'
})
).resolves.toBe('ok')
// The index never lands without its pack.
expect(readdirSync(join(objects, 'pack'))).toEqual([])
expect(scratchDirs()).toEqual([])
expect(warn).toHaveBeenCalledWith(
'[git-object-quarantine] could not keep a fetched pack',
expect.any(Error)
)
})
})