mirror of
https://github.com/stablyai/orca.git
synced 2026-09-28 16:02:45 +00:00
* perf(source-control): stop blocking main on four sync git-dir probes per status poll detectConflictOperation ran four existsSync calls against the git dir on every status poll. On a `\\wsl.localhost\...` worktree each one is a 9p round trip, and being synchronous they landed on the Electron main thread back to back. Replace them with concurrent fs/promises access probes: same "any failure reads as absent" semantics existsSync had, one wave instead of four serialized blocking calls. The outer try/catch went with them -- neither resolveGitDir nor the probes can throw now, so it was unreachable. Part of #15036 (source-control latency). * perf(wsl): let git reads take the shell-free route from a cwd-derived distro shouldAttemptWslDirectGit required options.wslDistro, so a `\\wsl.localhost\...` worktree without a resolved WSL project runtime never qualified -- even though the distro is right there in the cwd and wslDistroForCommand already knew how to read it. Every `git show` behind a diff therefore ran through the user's login shell, executing their rc once per blob read. Three changes: - Derive the distro from the cwd when no override was supplied. This is the fix; the routing decision now depends on where the repo actually lives. - Wait, bounded, for a cold read-environment probe instead of resolving without it. The probe is one wsl.exe call shared per distro, so the wait is paid at most once, and past WSL_GIT_READ_ENVIRONMENT_WAIT_MS the shell route runs exactly as before. It returns null rather than a settled promise when there is nothing to wait for, so a non-WSL git call is not pushed into a later microtask. - Opt the blob reads into preferWslDirectGit via gitReadOptionsForWorktree (renamed from gitStatusReadOptionsForWorktree; it was never status-specific). Belt-and- braces only: `show`, `config --get-regexp`, `ls-files` and `rev-parse` were all already matched by isWslDirectGitReadCommand, so this changes no routing today -- it just stops the diff path depending on a heuristic it knows the answer to. git-blob-read also gains a `failed` flag distinguishing "git ran and reported the path absent" (exit 128) from "the read never got an answer"; nothing consumes it yet, the settled diff cache does. Part of #15036 (source-control latency). * perf(source-control): give diff reads a settled cache keyed on stamped git state gitDiffReadDedupe coalesces only while a read is in flight, so every file selection re-ran the whole read: a `git config --file .gitmodules` spawn, one or two `git show` spawns, and a working-tree stat+read. On a WSL/UNC worktree each git spawn is a wsl.exe invocation, which is the ">3s Loading diff..." in #15036. Correctness first -- a stale diff is worse than a slow one. The cache never expires on a clock and there is no TTL to tune. Instead: - worktree-diff-stamp.ts takes a subprocess-free stamp of exactly the inputs a file diff is built from: HEAD (by resolved tip *content*, so a commit is visible even though HEAD's own bytes never move), `.git/index` (mtime+size), `.gitmodules` (submodule routing), and the working-tree file. A linked worktree's commondir and the packed-refs/reftable fallback are handled; an unborn branch is caught by recording "no loose ref" rather than only the packed stamps. - The stamp is captured BEFORE the read and stored with the result. Anything that moves during or after the read leaves the stored stamp behind, so the next lookup misses. That, not a freshness window, is why a stale diff cannot be served. - A store is refused unless the stamp was taken a full mtime bucket (2s, FAT's granularity) after its newest component. Below that, a second write inside the same bucket would be invisible -- git's own racy-index rule. - `null` stamp means "cannot prove" and never caches: a folder workspace, a repo whose layout cannot be read, or a filesystem reporting no usable mtime. - Submodule routes and reads that failed rather than proved absence are not reusable. A wsl.exe hiccup produces the same empty left side a new file does, and pinning that would persist a wrong diff. - invalidateGitReadCaches clears it and bumps a generation, so a read that started pre-mutation cannot store its result post-mutation. `ino` is deliberately optional in the working-tree component: Windows reports 0 for it on the redirector behind `\\wsl.localhost`, and requiring an unstable 0 to match would make the cache silently never hit on the exact host it exists for. Cache counters are exposed for the same reason -- a miss storm and a cold start otherwise look identical. Also drops gitDiffReadDedupe.clear() from getStatus. A status poll is a read; all it did was destroy a live coalescing entry so a concurrent identical request started duplicate git work. Mutations still invalidate through the shared point. Memory is bounded by retained characters, not entry count -- one diff result can legitimately hold megabytes. Fixes the source-control half of #15036. * perf(source-control): reuse BoundedMap and stop the WSL probe wait from outliving its answer Review follow-ups on the settled-diff-cache work: - SettledDiffCache now sits on the shared BoundedMap instead of hand-rolling the same Map + character ledger + evict-oldest loop. - pendingWslDirectGitReadEnvironment returns null once the probe has settled either way, so a distro whose direct route was disabled no longer pays for a 1.5s timer and two microtask hops on every git read. - That wait now honours the read's abort signal and goes through withTimeout, so an aborted read is not held for the full bound and a probe rejection can never surface as a read failure. - The settled-cache generation fence is taken before the stamp read, so a mutation that lands entirely inside the stamp's stats can no longer store an entry whose stamp is torn across it. - The cache counters are folded into the main-thread churn probe report, which is what tells a permanently-cold cache apart from a cold start in the field. * fix(source-control): tell WSL clock skew apart from a genuinely fresh write The racy-write margin compares two clocks: capturedAtMs is this host's, while the component mtimes come from whatever wrote the files. On a \\wsl.localhost worktree the guest sets them, so a guest running ahead pushes every recently-touched file past the margin and the cache refuses to store — for as long as the skew lasts, on exactly the platform this cache exists for. Nothing was wrong with the refusal; it was invisible. racyWrites alone cannot distinguish "the repo was just edited" from "the clocks disagree and this will never resolve on its own", so a permanently cold cache looked like a cold start. isDiffStampClockSkewed flags the one thing no local write can produce — an mtime in this host's future — and the cache counts those separately as clockSkewedWrites. A nonzero count is the signal that the cache is off for a reason idling will not fix. Found by review of #16600; behavior is unchanged, only observability.
This commit is contained in:
@@ -6,7 +6,7 @@ import {
|
||||
} from '../../../shared/git-effective-upstream'
|
||||
import { createGitConfigSnapshotRunner } from '../../../shared/git-config-snapshot-runner'
|
||||
import type { GitRuntimeOptions } from '../git-runtime-options'
|
||||
import { gitStatusReadOptionsForWorktree } from '../git-runtime-options'
|
||||
import { gitReadOptionsForWorktree } from '../git-runtime-options'
|
||||
import { gitExecFileAsync } from '../runner'
|
||||
import {
|
||||
MAX_EFFECTIVE_UPSTREAM_NEGATIVE_CACHE_ENTRIES,
|
||||
@@ -90,7 +90,7 @@ async function probeOrRevalidateEffectiveUpstreamStatus(
|
||||
} else if (cached) {
|
||||
try {
|
||||
const status = await getGitUpstreamStatusForUpstreamName(
|
||||
(args) => gitExecFileAsync(args, gitStatusReadOptionsForWorktree(worktreePath, options)),
|
||||
(args) => gitExecFileAsync(args, gitReadOptionsForWorktree(worktreePath, options)),
|
||||
cached.upstreamName
|
||||
)
|
||||
return { status, probedSameNameOriginRef: false }
|
||||
@@ -127,7 +127,7 @@ async function probeEffectiveUpstreamStatus(
|
||||
): Promise<{ status: GitUpstreamStatus; probedSameNameOriginRef: boolean }> {
|
||||
let probedSameNameOriginRef = false
|
||||
const snapshotRunner = createGitConfigSnapshotRunner((args) =>
|
||||
gitExecFileAsync(args, gitStatusReadOptionsForWorktree(worktreePath, options))
|
||||
gitExecFileAsync(args, gitReadOptionsForWorktree(worktreePath, options))
|
||||
)
|
||||
const status = await getEffectiveGitUpstreamStatus((args) => {
|
||||
if (args[0] === 'rev-parse' && args.includes(`refs/remotes/origin/${branchName}`)) {
|
||||
|
||||
@@ -3,7 +3,8 @@ import type { GitDiffResult } from '../../../shared/git-diff-compare-types'
|
||||
import { stableInFlightKey } from '../../../shared/in-flight-promise-dedupe'
|
||||
import type { GitRuntimeOptions } from '../git-runtime-options'
|
||||
import { gitRuntimeOptionsKey } from './git-runtime-options-cache-key'
|
||||
import { gitDiffReadDedupe } from './git-read-cache-invalidation'
|
||||
import { gitDiffReadDedupe, settledDiffCache } from './git-read-cache-invalidation'
|
||||
import { readWorktreeDiffStamp } from './worktree-diff-stamp'
|
||||
import { buildDiffResult } from './diff-result'
|
||||
import {
|
||||
readGitBlobAtIndexPath,
|
||||
@@ -33,27 +34,74 @@ export async function getDiff(
|
||||
compareAgainstHead = false,
|
||||
options: GitRuntimeOptions = {}
|
||||
): Promise<GitDiffResult> {
|
||||
// Why: register the dedupe synchronously (before any await) so concurrent identical reads coalesce.
|
||||
return gitDiffReadDedupe.run(
|
||||
stableInFlightKey([
|
||||
'diff',
|
||||
const readKey = stableInFlightKey([
|
||||
'diff',
|
||||
worktreePath,
|
||||
filePath,
|
||||
staged,
|
||||
compareAgainstHead,
|
||||
...gitRuntimeOptionsKey(options)
|
||||
])
|
||||
// Why: register the dedupe synchronously (before any await) so concurrent identical reads
|
||||
// coalesce — including on the settled-cache lookup, which is itself I/O.
|
||||
return gitDiffReadDedupe.run(readKey, () =>
|
||||
loadDiffThroughSettledCache(
|
||||
readKey,
|
||||
worktreePath,
|
||||
filePath,
|
||||
staged,
|
||||
compareAgainstHead,
|
||||
...gitRuntimeOptionsKey(options)
|
||||
]),
|
||||
() => loadDiff(worktreePath, filePath, staged, compareAgainstHead, options)
|
||||
options
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Serve a settled diff when the git state it was built from is provably
|
||||
* unchanged, otherwise read and — only if the read proved everything it touched
|
||||
* — record it under the stamp taken *before* the read.
|
||||
*
|
||||
* Stamping first is what makes staleness impossible: anything that moves during
|
||||
* or after the read leaves the stored stamp behind, so the next lookup misses.
|
||||
*/
|
||||
async function loadDiffThroughSettledCache(
|
||||
readKey: string,
|
||||
worktreePath: string,
|
||||
filePath: string,
|
||||
staged: boolean,
|
||||
compareAgainstHead: boolean,
|
||||
options: GitRuntimeOptions
|
||||
): Promise<GitDiffResult> {
|
||||
// Why before the stamp read: the stamp is itself several awaited stats, and a mutation that
|
||||
// lands entirely inside that window would otherwise leave the fence covering only the git read.
|
||||
const readGeneration = settledDiffCache.beginRead()
|
||||
// A staged diff compares HEAD to the index, so the working tree is not one of its inputs.
|
||||
const stamp = await readWorktreeDiffStamp(worktreePath, filePath, !staged)
|
||||
const cached = settledDiffCache.get(readKey, stamp)
|
||||
if (cached) {
|
||||
return cached
|
||||
}
|
||||
const loaded = await loadDiff(worktreePath, filePath, staged, compareAgainstHead, options)
|
||||
if (loaded.reusable) {
|
||||
settledDiffCache.set(readKey, stamp, loaded.result, readGeneration)
|
||||
}
|
||||
return loaded.result
|
||||
}
|
||||
|
||||
/**
|
||||
* `reusable` is false when the result cannot be proven to describe the stamped
|
||||
* state: a submodule route, whose inputs live in another repo and are stamped by
|
||||
* that repo's own read, or a blob read that failed rather than proving absence.
|
||||
*/
|
||||
type LoadedDiff = { result: GitDiffResult; reusable: boolean }
|
||||
|
||||
async function loadDiff(
|
||||
worktreePath: string,
|
||||
filePath: string,
|
||||
staged: boolean,
|
||||
compareAgainstHead: boolean,
|
||||
options: GitRuntimeOptions
|
||||
): Promise<GitDiffResult> {
|
||||
): Promise<LoadedDiff> {
|
||||
// Why: gitlink paths can't be read as blobs, so route submodule diffs explicitly (root → pointer, inner → recurse).
|
||||
const submodulePaths = await listSubmodulePaths(worktreePath, options)
|
||||
if (submodulePaths.length > 0) {
|
||||
@@ -63,13 +111,15 @@ async function loadDiff(
|
||||
const submoduleWorktreePath = resolveSubmoduleWorktreePath(worktreePath, matchedSubmodule)
|
||||
const normalizedFilePath = filePath.replace(/\\/g, '/').replace(/\/+$/, '')
|
||||
if (normalizedFilePath === matchedSubmodule) {
|
||||
return buildSubmodulePointerDiff(
|
||||
worktreePath,
|
||||
matchedSubmodule,
|
||||
staged,
|
||||
compareAgainstHead,
|
||||
options,
|
||||
submoduleWorktreePath
|
||||
return notReusable(
|
||||
await buildSubmodulePointerDiff(
|
||||
worktreePath,
|
||||
matchedSubmodule,
|
||||
staged,
|
||||
compareAgainstHead,
|
||||
options,
|
||||
submoduleWorktreePath
|
||||
)
|
||||
)
|
||||
}
|
||||
const innerPath = normalizedFilePath.slice(matchedSubmodule.length + 1)
|
||||
@@ -82,15 +132,20 @@ async function loadDiff(
|
||||
: await readWorkingSubmoduleHead(submoduleWorktreePath, options)
|
||||
// Why: a moved gitlink with a clean submodule worktree means the change is committed — diff the two commits.
|
||||
if (fromOid && toOid && fromOid !== toOid) {
|
||||
return buildSubmoduleInnerCommitRangeDiff(
|
||||
submoduleWorktreePath,
|
||||
innerPath,
|
||||
fromOid,
|
||||
toOid,
|
||||
options
|
||||
return notReusable(
|
||||
await buildSubmoduleInnerCommitRangeDiff(
|
||||
submoduleWorktreePath,
|
||||
innerPath,
|
||||
fromOid,
|
||||
toOid,
|
||||
options
|
||||
)
|
||||
)
|
||||
}
|
||||
return getDiff(submoduleWorktreePath, innerPath, staged, compareAgainstHead, options)
|
||||
// The inner read stamps and caches against the submodule's own repo state.
|
||||
return notReusable(
|
||||
await getDiff(submoduleWorktreePath, innerPath, staged, compareAgainstHead, options)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -99,6 +154,7 @@ async function loadDiff(
|
||||
let originalIsBinary = false
|
||||
let modifiedIsBinary = false
|
||||
let modifiedDeleted = false
|
||||
let readFailed = false
|
||||
|
||||
try {
|
||||
if (staged) {
|
||||
@@ -113,6 +169,7 @@ async function loadDiff(
|
||||
modifiedContent = rightBlob.content
|
||||
modifiedIsBinary = rightBlob.isBinary
|
||||
modifiedDeleted = !rightBlob.exists
|
||||
readFailed = leftBlob.failed === true || rightBlob.failed === true
|
||||
} else {
|
||||
// The left chain (index→HEAD) is sequential within itself, but the working
|
||||
// tree read is a plain fs read that does not depend on it.
|
||||
@@ -127,9 +184,11 @@ async function loadDiff(
|
||||
modifiedContent = workingTreeBlob.content
|
||||
modifiedIsBinary = workingTreeBlob.isBinary
|
||||
modifiedDeleted = !workingTreeBlob.exists
|
||||
readFailed = leftBlob.failed === true || workingTreeBlob.failed === true
|
||||
}
|
||||
} catch {
|
||||
// Fallback
|
||||
readFailed = true
|
||||
}
|
||||
|
||||
const result = buildDiffResult(
|
||||
@@ -141,7 +200,11 @@ async function loadDiff(
|
||||
)
|
||||
// Why: mark a proven deletion so previewers don't mistake a read failure's empty side for one.
|
||||
if (result.kind === 'binary' && modifiedDeleted) {
|
||||
return { ...result, modifiedDeleted: true }
|
||||
return { result: { ...result, modifiedDeleted: true }, reusable: !readFailed }
|
||||
}
|
||||
return result
|
||||
return { result, reusable: !readFailed }
|
||||
}
|
||||
|
||||
function notReusable(result: GitDiffResult): LoadedDiff {
|
||||
return { result, reusable: false }
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ import { readFile, stat } from 'node:fs/promises'
|
||||
import * as path from 'node:path'
|
||||
import { isBinaryBuffer } from '../../../shared/binary-buffer'
|
||||
import type { GitRuntimeOptions } from '../git-runtime-options'
|
||||
import { gitOptionsForWorktree } from '../git-runtime-options'
|
||||
import { gitReadOptionsForWorktree } from '../git-runtime-options'
|
||||
import { gitExecFileAsyncBuffer } from '../runner'
|
||||
import { isMaxBufferOverflowError } from '../max-buffer-overflow'
|
||||
import { MAX_GIT_SHOW_BYTES } from './git-show-max-bytes'
|
||||
@@ -12,6 +12,21 @@ export type GitBlobReadResult = {
|
||||
content: string
|
||||
isBinary: boolean
|
||||
exists: boolean
|
||||
/**
|
||||
* The read did not complete: the blob is neither known-present nor proven
|
||||
* absent. Callers must not persist a diff built on one, because the empty side
|
||||
* it produces is indistinguishable from a genuinely new file.
|
||||
*/
|
||||
failed?: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Tell "Git ran and said the path is not there" apart from "the read never got
|
||||
* an answer". Git exits 128 for a missing path in a tree or the index; a WSL
|
||||
* relay that never reached Git exits with anything else, or with a spawn errno.
|
||||
*/
|
||||
function isProvenAbsentError(error: unknown): boolean {
|
||||
return (error as { code?: unknown } | null)?.code === 128
|
||||
}
|
||||
|
||||
export async function readUnstagedLeftBlob(
|
||||
@@ -24,7 +39,9 @@ export async function readUnstagedLeftBlob(
|
||||
return indexBlob
|
||||
}
|
||||
|
||||
return readGitBlobAtOidPath(worktreePath, 'HEAD', filePath, options)
|
||||
const headBlob = await readGitBlobAtOidPath(worktreePath, 'HEAD', filePath, options)
|
||||
// Why: if the index read never got an answer, falling back to HEAD is a guess, not a proof.
|
||||
return indexBlob.failed ? { ...headBlob, failed: true } : headBlob
|
||||
}
|
||||
|
||||
export async function readGitBlobAtIndexPath(
|
||||
@@ -36,7 +53,7 @@ export async function readGitBlobAtIndexPath(
|
||||
const gitPath = filePath.replace(/\\/g, '/')
|
||||
try {
|
||||
const { stdout } = await gitExecFileAsyncBuffer(['show', `:${gitPath}`], {
|
||||
...gitOptionsForWorktree(worktreePath, options),
|
||||
...gitReadOptionsForWorktree(worktreePath, options),
|
||||
maxBuffer: MAX_GIT_SHOW_BYTES
|
||||
})
|
||||
|
||||
@@ -45,7 +62,7 @@ export async function readGitBlobAtIndexPath(
|
||||
if (isMaxBufferOverflowError(error)) {
|
||||
return { content: '', isBinary: true, exists: true }
|
||||
}
|
||||
return { content: '', isBinary: false, exists: false }
|
||||
return { content: '', isBinary: false, exists: false, failed: !isProvenAbsentError(error) }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -61,7 +78,7 @@ export async function readGitBlobAtOidPath(
|
||||
const { stdout } = await gitExecFileAsyncBuffer(
|
||||
['show', '--end-of-options', `${oid}:${gitPath}`],
|
||||
{
|
||||
...gitOptionsForWorktree(worktreePath, options),
|
||||
...gitReadOptionsForWorktree(worktreePath, options),
|
||||
maxBuffer: MAX_GIT_SHOW_BYTES
|
||||
}
|
||||
)
|
||||
@@ -71,7 +88,7 @@ export async function readGitBlobAtOidPath(
|
||||
if (isMaxBufferOverflowError(error)) {
|
||||
return { content: '', isBinary: true, exists: true }
|
||||
}
|
||||
return { content: '', isBinary: false, exists: false }
|
||||
return { content: '', isBinary: false, exists: false, failed: !isProvenAbsentError(error) }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -81,11 +98,8 @@ export async function readWorkingTreeFile(filePath: string): Promise<GitBlobRead
|
||||
fileStat = await stat(filePath)
|
||||
} catch (error) {
|
||||
// Why: only ENOENT is a real deletion; other stat errors are read failures, not absence.
|
||||
return {
|
||||
content: '',
|
||||
isBinary: false,
|
||||
exists: (error as NodeJS.ErrnoException)?.code !== 'ENOENT'
|
||||
}
|
||||
const missing = (error as NodeJS.ErrnoException)?.code === 'ENOENT'
|
||||
return { content: '', isBinary: false, exists: !missing, ...(missing ? {} : { failed: true }) }
|
||||
}
|
||||
if (!fileStat.isFile()) {
|
||||
return { content: '', isBinary: false, exists: false }
|
||||
@@ -99,7 +113,7 @@ export async function readWorkingTreeFile(filePath: string): Promise<GitBlobRead
|
||||
return bufferToBlob(buffer, filePath)
|
||||
} catch {
|
||||
// Why: the file exists but could not be read — a read failure, not a deletion.
|
||||
return { content: '', isBinary: false, exists: true }
|
||||
return { content: '', isBinary: false, exists: true, failed: true }
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { existsSync } from 'node:fs'
|
||||
import { access } from 'node:fs/promises'
|
||||
import * as path from 'node:path'
|
||||
import type { GitConflictOperation } from '../../../shared/git-status-types'
|
||||
import type { GitRuntimeOptions } from '../git-runtime-options'
|
||||
@@ -16,17 +16,12 @@ export async function detectConflictOperation(worktreePath: string): Promise<Git
|
||||
const rebaseMergeDir = path.join(gitDir, 'rebase-merge')
|
||||
const rebaseApplyDir = path.join(gitDir, 'rebase-apply')
|
||||
|
||||
let hasMergeHead = false
|
||||
let hasCherryPickHead = false
|
||||
let hasRebaseDir = false
|
||||
|
||||
try {
|
||||
hasMergeHead = existsSync(mergeHead)
|
||||
hasCherryPickHead = existsSync(cherryPickHead)
|
||||
hasRebaseDir = existsSync(rebaseMergeDir) || existsSync(rebaseApplyDir)
|
||||
} catch {
|
||||
return 'unknown'
|
||||
}
|
||||
// Why async and concurrent: this runs on every status poll, and on a WSL/UNC git dir each
|
||||
// probe is a 9p round trip — four of them synchronously blocked the Electron main thread.
|
||||
const [hasMergeHead, hasCherryPickHead, hasRebaseMergeDir, hasRebaseApplyDir] = await Promise.all(
|
||||
[mergeHead, cherryPickHead, rebaseMergeDir, rebaseApplyDir].map(pathExists)
|
||||
)
|
||||
const hasRebaseDir = hasRebaseMergeDir || hasRebaseApplyDir
|
||||
|
||||
if (hasMergeHead) {
|
||||
return 'merge'
|
||||
@@ -40,6 +35,16 @@ export async function detectConflictOperation(worktreePath: string): Promise<Git
|
||||
return 'unknown'
|
||||
}
|
||||
|
||||
/** Mirrors existsSync: any failure to reach the path reads as absent, never as a throw. */
|
||||
async function pathExists(target: string): Promise<boolean> {
|
||||
try {
|
||||
await access(target)
|
||||
return true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
export async function abortMerge(
|
||||
worktreePath: string,
|
||||
options: GitRuntimeOptions = {}
|
||||
|
||||
@@ -7,15 +7,20 @@ import { GitStatusReadLeaseOwner } from '../git-status-read-lease-owner'
|
||||
import { invalidateGitUpstreamStatusReads } from '../upstream'
|
||||
import { clearSubmodulePathsCache } from './submodule-paths'
|
||||
import { resolvedUpstreamNameCache } from './resolved-upstream-name-cache'
|
||||
import { SettledDiffCache } from './settled-diff-cache'
|
||||
|
||||
export const gitDiffReadDedupe = new InFlightPromiseDedupe<GitDiffResult>()
|
||||
|
||||
/** Settled diff results, valid only while their stamped git state holds. */
|
||||
export const settledDiffCache = new SettledDiffCache()
|
||||
|
||||
export const statusReadLeaseOwner = new GitStatusReadLeaseOwner<GitStatusResult>()
|
||||
|
||||
// Why: clear every in-flight git read cache; clearing only some would let a post-mutation
|
||||
// getStatus() join a pre-mutation read and publish it as current.
|
||||
export function invalidateGitReadCaches(): void {
|
||||
gitDiffReadDedupe.clear()
|
||||
settledDiffCache.clear()
|
||||
statusReadLeaseOwner.invalidate()
|
||||
invalidateGitBranchLineTotalInFlight()
|
||||
invalidateGitUpstreamStatusReads()
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
import { BoundedMap } from '../../../shared/bounded-map'
|
||||
import type { GitDiffResult } from '../../../shared/git-diff-compare-types'
|
||||
import {
|
||||
canProveUnchangedByStamp,
|
||||
isDiffStampClockSkewed,
|
||||
type WorktreeDiffStamp
|
||||
} from './worktree-diff-stamp'
|
||||
|
||||
/**
|
||||
* Diff results that survive their read, guarded by a stamp of the git state they
|
||||
* were built from.
|
||||
*
|
||||
* Why this is not a TTL: nothing here expires on a clock, so no window exists in
|
||||
* which a stale diff can be served. An entry is returned only when a freshly
|
||||
* taken stamp equals the one captured *before* the read that produced it, which
|
||||
* means no input moved from then until now. Everything else — an unprovable
|
||||
* stamp, a write too recent for its mtime to be conclusive, a read that failed
|
||||
* partway, a mutation that ran while the read was in flight — declines to cache
|
||||
* rather than risk it.
|
||||
*/
|
||||
|
||||
// A diff result carries whole file contents on both sides, so an entry count alone bounds
|
||||
// nothing: the real budget is characters, and one entry can legitimately be huge.
|
||||
export const MAX_SETTLED_DIFF_CACHE_ENTRIES = 32
|
||||
export const MAX_SETTLED_DIFF_CACHE_RESULT_CHARACTERS = 1_000_000
|
||||
export const MAX_SETTLED_DIFF_CACHE_TOTAL_CHARACTERS = 8_000_000
|
||||
|
||||
export type SettledDiffCacheStats = {
|
||||
hits: number
|
||||
/** Stamp was taken but no entry matched it. */
|
||||
misses: number
|
||||
/** No stamp could be taken, so the read could never be cached. */
|
||||
unprovable: number
|
||||
stores: number
|
||||
/** Store declined because a write was too recent for its mtime to be conclusive. */
|
||||
racyWrites: number
|
||||
/**
|
||||
* Subset of `racyWrites` where a component's mtime was in this host's future, so
|
||||
* the clocks disagree and the refusal will persist until they converge. A nonzero
|
||||
* count here means the cache is off for a reason no amount of idling will fix.
|
||||
*/
|
||||
clockSkewedWrites: number
|
||||
/** Store declined because a mutation invalidated the cache while the read ran. */
|
||||
invalidatedDuringRead: number
|
||||
entries: number
|
||||
retainedCharacters: number
|
||||
}
|
||||
|
||||
type CacheEntry = { stamp: string; result: GitDiffResult; characters: number }
|
||||
|
||||
export class SettledDiffCache {
|
||||
private readonly entries = new BoundedMap<string, CacheEntry>({
|
||||
maxEntries: MAX_SETTLED_DIFF_CACHE_ENTRIES,
|
||||
maxBytes: MAX_SETTLED_DIFF_CACHE_TOTAL_CHARACTERS,
|
||||
maxEntryBytes: MAX_SETTLED_DIFF_CACHE_RESULT_CHARACTERS,
|
||||
sizeOf: (entry) => entry.characters
|
||||
})
|
||||
private generation = 0
|
||||
private hits = 0
|
||||
private misses = 0
|
||||
private unprovable = 0
|
||||
private stores = 0
|
||||
private racyWrites = 0
|
||||
private clockSkewedWrites = 0
|
||||
private invalidatedDuringRead = 0
|
||||
|
||||
/**
|
||||
* Take before starting a read; hand back to `set`. A mutation that lands while
|
||||
* the read is in flight bumps the generation, and the store is refused — the
|
||||
* result describes pre-mutation state and must not outlive it.
|
||||
*/
|
||||
beginRead(): number {
|
||||
return this.generation
|
||||
}
|
||||
|
||||
get(key: string, stamp: WorktreeDiffStamp | null): GitDiffResult | undefined {
|
||||
if (!stamp) {
|
||||
this.unprovable += 1
|
||||
return undefined
|
||||
}
|
||||
// BoundedMap.get() already refreshes the LRU position.
|
||||
const entry = this.entries.get(key)
|
||||
if (!entry || entry.stamp !== stamp.value) {
|
||||
this.misses += 1
|
||||
return undefined
|
||||
}
|
||||
this.hits += 1
|
||||
return entry.result
|
||||
}
|
||||
|
||||
set(
|
||||
key: string,
|
||||
stamp: WorktreeDiffStamp | null,
|
||||
result: GitDiffResult,
|
||||
readGeneration: number
|
||||
): void {
|
||||
if (!stamp) {
|
||||
return
|
||||
}
|
||||
if (readGeneration !== this.generation) {
|
||||
this.invalidatedDuringRead += 1
|
||||
return
|
||||
}
|
||||
if (!canProveUnchangedByStamp(stamp)) {
|
||||
this.racyWrites += 1
|
||||
if (isDiffStampClockSkewed(stamp)) {
|
||||
this.clockSkewedWrites += 1
|
||||
}
|
||||
return
|
||||
}
|
||||
const characters = resultCharacterCount(result)
|
||||
if (this.entries.set(key, { stamp: stamp.value, result, characters })) {
|
||||
this.stores += 1
|
||||
}
|
||||
}
|
||||
|
||||
clear(): void {
|
||||
this.entries.clear()
|
||||
// Why: bump so a read that started pre-mutation can't repopulate the invalidated cache.
|
||||
this.generation += 1
|
||||
}
|
||||
|
||||
/**
|
||||
* Why exposed: a stamp that can never match — an inode or mtime the filesystem
|
||||
* reports unstably — looks exactly like having no cache at all. These counters
|
||||
* are what tells a miss storm apart from a cold start.
|
||||
*/
|
||||
stats(): SettledDiffCacheStats {
|
||||
return {
|
||||
hits: this.hits,
|
||||
misses: this.misses,
|
||||
unprovable: this.unprovable,
|
||||
stores: this.stores,
|
||||
racyWrites: this.racyWrites,
|
||||
clockSkewedWrites: this.clockSkewedWrites,
|
||||
invalidatedDuringRead: this.invalidatedDuringRead,
|
||||
entries: this.entries.size,
|
||||
retainedCharacters: this.entries.retainedBytes
|
||||
}
|
||||
}
|
||||
|
||||
resetStatsForTests(): void {
|
||||
this.hits = 0
|
||||
this.misses = 0
|
||||
this.unprovable = 0
|
||||
this.stores = 0
|
||||
this.racyWrites = 0
|
||||
this.clockSkewedWrites = 0
|
||||
this.invalidatedDuringRead = 0
|
||||
}
|
||||
}
|
||||
|
||||
function resultCharacterCount(result: GitDiffResult): number {
|
||||
return result.originalContent.length + result.modifiedContent.length
|
||||
}
|
||||
@@ -6,7 +6,7 @@ import {
|
||||
type GitBranchLineTotal
|
||||
} from '../../../shared/git-branch-line-total'
|
||||
import type { GitRuntimeOptions } from '../git-runtime-options'
|
||||
import { gitStatusReadOptionsForWorktree } from '../git-runtime-options'
|
||||
import { gitReadOptionsForWorktree } from '../git-runtime-options'
|
||||
import { gitExecFileAsync, gitOptionalLocksDisabledEnv } from '../runner'
|
||||
import type { GetStatusOptions } from './get-status-options'
|
||||
|
||||
@@ -36,7 +36,7 @@ export function createBranchLineTotalInput(
|
||||
.map((entry) => entry.path),
|
||||
runDiffNumstat: (args, signal) =>
|
||||
gitExecFileAsync(args, {
|
||||
...gitStatusReadOptionsForWorktree(worktreePath, options),
|
||||
...gitReadOptionsForWorktree(worktreePath, options),
|
||||
// Why: after the spread, so the shared lease signal wins over this caller's own.
|
||||
signal,
|
||||
env: gitOptionalLocksDisabledEnv(),
|
||||
|
||||
@@ -6,7 +6,7 @@ import {
|
||||
type GitLineStats
|
||||
} from '../../../shared/git-uncommitted-line-stats'
|
||||
import type { GitRuntimeOptions } from '../git-runtime-options'
|
||||
import { gitStatusReadOptionsForWorktree } from '../git-runtime-options'
|
||||
import { gitReadOptionsForWorktree } from '../git-runtime-options'
|
||||
import { gitExecFileAsync, gitOptionalLocksDisabledEnv } from '../runner'
|
||||
|
||||
async function runNumstat(
|
||||
@@ -26,7 +26,7 @@ async function runNumstat(
|
||||
'-M'
|
||||
],
|
||||
{
|
||||
...gitStatusReadOptionsForWorktree(worktreePath, options),
|
||||
...gitReadOptionsForWorktree(worktreePath, options),
|
||||
env: gitOptionalLocksDisabledEnv()
|
||||
}
|
||||
)
|
||||
|
||||
@@ -15,7 +15,7 @@ import {
|
||||
import { gitOptionalLocksDisabledEnv, gitStreamStdout } from '../runner'
|
||||
import { findExistingWorktreeSymlinkPaths } from '../worktree-symlink-detection'
|
||||
import type { GetStatusOptions } from './get-status-options'
|
||||
import { gitDiffReadDedupe, statusReadLeaseOwner } from './git-read-cache-invalidation'
|
||||
import { statusReadLeaseOwner } from './git-read-cache-invalidation'
|
||||
import { detectConflictOperation } from './git-conflict-operation'
|
||||
import { parseUnmergedEntry } from './status-conflict-entries'
|
||||
import { getEffectiveUpstreamStatusCacheKey } from './effective-upstream-status-cache'
|
||||
@@ -37,8 +37,10 @@ export async function getStatus(
|
||||
worktreePath: string,
|
||||
options: GetStatusOptions = {}
|
||||
): Promise<GitStatusResult> {
|
||||
gitDiffReadDedupe.clear()
|
||||
// Why: dedupe only concurrent identical reads; after settle, callers must run a fresh read.
|
||||
// Why nothing is cleared here: a status poll is a read. Dropping the in-flight diff entry
|
||||
// mid-read only made a concurrent identical request start duplicate git work, and the
|
||||
// settled diff cache is keyed on stamped git state, which a read cannot change anyway.
|
||||
// Mutations invalidate both, through invalidateGitReadCaches.
|
||||
const cacheKey = getStatusReadKey(worktreePath, options)
|
||||
return statusReadLeaseOwner.lease(cacheKey, options.signal, (sharedSignal) =>
|
||||
runGetStatus(worktreePath, { ...options, signal: sharedSignal })
|
||||
|
||||
@@ -0,0 +1,226 @@
|
||||
import { readFile, stat } from 'node:fs/promises'
|
||||
import * as path from 'node:path'
|
||||
import { resolveGitDir } from './resolve-git-dir'
|
||||
|
||||
/**
|
||||
* Subprocess-free proof of every input one working-tree file diff is built from:
|
||||
* the HEAD tree, the index, `.gitmodules` (which decides submodule routing) and
|
||||
* the working-tree file itself.
|
||||
*
|
||||
* Two equal stamps mean `git show HEAD:<path>`, `git show :<path>` and the
|
||||
* working-tree bytes would still return what they returned when the stamp was
|
||||
* taken, so a caller may reuse a settled diff instead of respawning Git — on a
|
||||
* WSL/UNC worktree that trades two `wsl.exe` spawns for a handful of 9p stats.
|
||||
*
|
||||
* `null` means "cannot prove unchanged" — not a repo, an unreadable layout, or a
|
||||
* filesystem that reports no usable mtime — and callers must not cache.
|
||||
*/
|
||||
export type WorktreeDiffStamp = {
|
||||
/** Opaque; compare for equality only. */
|
||||
value: string
|
||||
/** Newest mtime any component reported, or -Infinity when every one was absent. */
|
||||
newestMtimeMs: number
|
||||
capturedAtMs: number
|
||||
}
|
||||
|
||||
/**
|
||||
* How long after a component's mtime the stamp has to be taken before a later
|
||||
* write is guaranteed to move that mtime.
|
||||
*
|
||||
* Why 2s: FAT/exFAT truncate mtime to a 2s bucket, the coarsest granularity a
|
||||
* repo can realistically sit on. Below the margin a second write inside the same
|
||||
* bucket would leave the stamp identical, so the read stays uncached instead.
|
||||
*/
|
||||
export const DIFF_STAMP_RACY_WRITE_MARGIN_MS = 2_000
|
||||
|
||||
const MISSING = '-'
|
||||
const ABSENT: StampComponent = { text: MISSING, mtimeMs: Number.NEGATIVE_INFINITY }
|
||||
|
||||
type StampComponent = { text: string; mtimeMs: number }
|
||||
|
||||
/**
|
||||
* A write that lands in the same timestamp bucket as the stamp is invisible, so
|
||||
* only a stamp taken a full bucket after its newest component can be trusted.
|
||||
*
|
||||
* The margin compares two clocks: `capturedAtMs` is this host's, while the mtimes
|
||||
* come from whatever wrote the files. On a `\\wsl.localhost` worktree the guest
|
||||
* sets them, and a guest running ahead pushes every recently-touched file past the
|
||||
* margin — silently, and for as long as the skew lasts. `isDiffStampClockSkewed`
|
||||
* separates that from an honestly-too-fresh file so the caller can count it.
|
||||
*/
|
||||
export function canProveUnchangedByStamp(stamp: WorktreeDiffStamp): boolean {
|
||||
return stamp.capturedAtMs - stamp.newestMtimeMs >= DIFF_STAMP_RACY_WRITE_MARGIN_MS
|
||||
}
|
||||
|
||||
/**
|
||||
* True when a component's mtime is in this host's future, which no local write can
|
||||
* produce — so the margin above is measuring skew, not freshness, and will keep
|
||||
* refusing to store until the clocks converge.
|
||||
*/
|
||||
export function isDiffStampClockSkewed(stamp: WorktreeDiffStamp): boolean {
|
||||
return Number.isFinite(stamp.newestMtimeMs) && stamp.newestMtimeMs > stamp.capturedAtMs
|
||||
}
|
||||
|
||||
/**
|
||||
* Stamp the inputs of one file diff. Pass `includeWorkingTree: false` for a
|
||||
* staged diff, which compares HEAD to the index and never reads the working tree.
|
||||
*/
|
||||
export async function readWorktreeDiffStamp(
|
||||
worktreePath: string,
|
||||
filePath: string,
|
||||
includeWorkingTree: boolean
|
||||
): Promise<WorktreeDiffStamp | null> {
|
||||
const capturedAtMs = Date.now()
|
||||
try {
|
||||
const gitDir = await resolveGitDir(worktreePath)
|
||||
const [head, index, gitmodules, workingTree] = await Promise.all([
|
||||
readHeadComponent(gitDir),
|
||||
// Over-invalidates on purpose: git run outside Orca (a terminal `git status`/`git add`)
|
||||
// can refresh a stat-dirty index and rewrite this file without changing a single blob,
|
||||
// which costs one re-read. That is the safe direction — do not "fix" it by dropping the
|
||||
// index from the stamp, because `git add` then becomes invisible and the cache serves a
|
||||
// pre-staging diff.
|
||||
readFileStampComponent(path.join(gitDir, 'index')),
|
||||
readFileStampComponent(path.join(worktreePath, '.gitmodules')),
|
||||
includeWorkingTree
|
||||
? readWorkingTreeComponent(path.join(worktreePath, filePath))
|
||||
: Promise.resolve(ABSENT)
|
||||
])
|
||||
if (!head) {
|
||||
return null
|
||||
}
|
||||
const components = [head, index, gitmodules, workingTree]
|
||||
return {
|
||||
// Why JSON: a path or a ref can contain any separator character, and an ambiguous
|
||||
// join is a stamp collision — two different states that compare equal.
|
||||
value: JSON.stringify([
|
||||
worktreePath,
|
||||
filePath,
|
||||
...components.map((component) => component.text)
|
||||
]),
|
||||
newestMtimeMs: Math.max(...components.map((component) => component.mtimeMs)),
|
||||
capturedAtMs
|
||||
}
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Identify the commit HEAD resolves to. Reading the tip is what makes a plain
|
||||
* commit visible: it rewrites `refs/heads/<branch>` and leaves HEAD untouched.
|
||||
*
|
||||
* A loose tip is recorded by content, so it needs no mtime margin. Only when no
|
||||
* loose ref exists at all — packed refs, the reftable backend, or an unborn
|
||||
* branch — does this fall back to the coarser stamps of the files a packed tip
|
||||
* moves, and the recorded "no loose ref" marker still catches the ref appearing.
|
||||
*/
|
||||
async function readHeadComponent(gitDir: string): Promise<StampComponent | null> {
|
||||
const [head, commonDirEntry] = await Promise.all([
|
||||
readTrimmedFile(path.join(gitDir, 'HEAD')),
|
||||
readTrimmedFile(path.join(gitDir, 'commondir'))
|
||||
])
|
||||
if (!head) {
|
||||
return null
|
||||
}
|
||||
const commonDir = commonDirEntry ? path.resolve(gitDir, commonDirEntry) : gitDir
|
||||
const refName = head.match(/^ref:\s*(.+?)\s*$/)?.[1]
|
||||
if (!refName || !isSafeRefName(refName)) {
|
||||
// Detached HEAD already holds the object id; an unrecognized HEAD is covered by its own text.
|
||||
return { text: head, mtimeMs: Number.NEGATIVE_INFINITY }
|
||||
}
|
||||
// Per-worktree refs (`refs/bisect`, `refs/worktree`) live beside the checkout; branches are shared.
|
||||
const [perWorktreeTip, sharedTip] = await Promise.all([
|
||||
readTrimmedFile(path.join(gitDir, refName)),
|
||||
commonDir === gitDir ? Promise.resolve(null) : readTrimmedFile(path.join(commonDir, refName))
|
||||
])
|
||||
const looseTip = perWorktreeTip ?? sharedTip
|
||||
if (looseTip) {
|
||||
// A loose ref shadows any packed entry, so its bytes settle the tip on their own.
|
||||
return {
|
||||
text: JSON.stringify([head, 'loose', perWorktreeTip ? 'worktree' : 'common', looseTip]),
|
||||
mtimeMs: Number.NEGATIVE_INFINITY
|
||||
}
|
||||
}
|
||||
const [packedRefs, reftable] = await Promise.all([
|
||||
readFileStampComponent(path.join(commonDir, 'packed-refs')),
|
||||
readFileStampComponent(path.join(commonDir, 'reftable'))
|
||||
])
|
||||
return {
|
||||
text: JSON.stringify([head, 'packed', packedRefs.text, reftable.text]),
|
||||
mtimeMs: Math.max(packedRefs.mtimeMs, reftable.mtimeMs)
|
||||
}
|
||||
}
|
||||
|
||||
/** Keep a hand-edited HEAD from steering the stamp outside the repo's ref store. */
|
||||
function isSafeRefName(refName: string): boolean {
|
||||
const segments = refName.split(/[\\/]/)
|
||||
return (
|
||||
segments[0] === 'refs' &&
|
||||
segments.length > 1 &&
|
||||
segments.every((segment) => segment.length > 0 && segment !== '.' && segment !== '..') &&
|
||||
!path.isAbsolute(refName)
|
||||
)
|
||||
}
|
||||
|
||||
async function readTrimmedFile(filePath: string): Promise<string | null> {
|
||||
try {
|
||||
const trimmed = (await readFile(filePath, 'utf-8')).trim()
|
||||
return trimmed.length > 0 ? trimmed : null
|
||||
} catch (error) {
|
||||
if (isMissingEntryError(error)) {
|
||||
return null
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async function readFileStampComponent(filePath: string): Promise<StampComponent> {
|
||||
try {
|
||||
const stats = await stat(filePath)
|
||||
return { text: `${requireMtimeMs(stats.mtimeMs)}:${stats.size}`, mtimeMs: stats.mtimeMs }
|
||||
} catch (error) {
|
||||
if (isMissingEntryError(error)) {
|
||||
return ABSENT
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async function readWorkingTreeComponent(filePath: string): Promise<StampComponent> {
|
||||
try {
|
||||
const stats = await stat(filePath)
|
||||
// Why ino is optional: an atomic-rename save can keep both the size and the mtime bucket,
|
||||
// so a changed inode is extra proof — but Windows reports 0 for it on the network
|
||||
// redirector behind `\\wsl.localhost`, and requiring an unstable 0 to match would make
|
||||
// the stamp never hit on exactly the host this cache exists for. Fold it in only when
|
||||
// the filesystem gives a real one.
|
||||
const inode = isUsableInode(stats.ino) ? String(stats.ino) : MISSING
|
||||
return {
|
||||
text: `${requireMtimeMs(stats.mtimeMs)}:${stats.size}:${inode}`,
|
||||
mtimeMs: stats.mtimeMs
|
||||
}
|
||||
} catch (error) {
|
||||
if (isMissingEntryError(error)) {
|
||||
return ABSENT
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
function isUsableInode(ino: unknown): boolean {
|
||||
return typeof ino === 'number' && Number.isFinite(ino) && ino !== 0
|
||||
}
|
||||
|
||||
/** A filesystem (or a stub) that reports no usable mtime cannot prove anything unchanged. */
|
||||
function requireMtimeMs(mtimeMs: unknown): number {
|
||||
if (typeof mtimeMs !== 'number' || !Number.isFinite(mtimeMs)) {
|
||||
throw new Error('stat reported no usable mtime')
|
||||
}
|
||||
return mtimeMs
|
||||
}
|
||||
|
||||
function isMissingEntryError(error: unknown): boolean {
|
||||
const code = (error as NodeJS.ErrnoException | null)?.code
|
||||
return code === 'ENOENT' || code === 'ENOTDIR'
|
||||
}
|
||||
Reference in New Issue
Block a user