fix: retire obsolete GitLab host cache generations (#21136)

Co-authored-by: m4air <m4air@Mac.localdomain>
This commit is contained in:
OrcaWin
2026-09-17 20:12:08 -07:00
committed by GitHub
co-authored by m4air
parent f90370fb6b
commit 51f809aa82
9 changed files with 813 additions and 11 deletions
@@ -0,0 +1,39 @@
# Retire obsolete GitLab known-host generations
Each successful `getGlabKnownHosts` probe previously stored a host-list array under a connection ID plus its SSH provider generation. Reconnecting under the same ID created a new entry while every earlier successful generation remained cached until an explicit preflight reset. The cache now keeps one successful generation per observed execution identity.
Async publication also uses the existing coalescer's `ownsKey()` and checks the current SSH generation. A result completing after reconnect, explicit reset, or replacement by a newer probe cannot recreate retired cache state. Original callers can still receive their own completed result. Explicitly remembered hosts, native/WSL separation, command routing and existing probe timeouts are preserved.
## Evidence
The runner bundles the actual cache, coalescer and parser. Only command-result and SSH-generation ports are controlled; it opens no SSH connection and runs no GitLab command. It reverses `fix.patch` in memory, verifies the original source hash, and compares that baseline against the unmodified current product source. Reports include product, dependency, regression-test and fixture hashes.
| Control | Original | Fixed |
| --- | --- | --- |
| Successful result arrays retained after 128 generations | 128 | 1 current array |
| Remembered result arrays retained after 16 generations | 16 | 1 current array |
| Delayed old-generation result after a successor answers | Still retained | Collectable; successor preserved |
| Explicit reset followed by old completion | Old result repopulates cache | Next lookup executes a fresh probe |
| Abandoned probe finishes after its replacement | Old host added to replacement cache | Replacement remains unchanged |
| Explicit reset after retention exercise | 0 original arrays retained | 0 original arrays retained |
Both phases preserve remembered-host updates while probes succeed or fail and isolate native, Ubuntu WSL, Debian WSL and two connection IDs. `results.json` records Node26.6; `electron-results.json` records installed Electron43.7 / Node24.21 running without an app window. Both runs pass all controls. This is compatibility evidence, not a historical packaged-binary reproduction.
```sh
ORCA_BACKGROUND_LAUNCH=1 node --expose-gc docs/audits/gitlab-known-host-retirement/reproduce.cjs
ORCA_BACKGROUND_LAUNCH=1 node node_modules/vitest/vitest.mjs run --config config/vitest.config.ts src/main/gitlab/gitlab-known-host-retirement.test.ts src/main/gitlab/gitlab-known-host-probe.test.ts src/main/gitlab/gitlab-known-host-probe-wsl-fallback.test.ts src/main/git/coalesced-probe.test.ts src/main/gitlab/client-mr-auth-rate-limit.test.ts
```
For the installed macOS Electron binary:
```sh
ELECTRON_RUN_AS_NODE=1 ORCA_BACKGROUND_LAUNCH=1 node_modules/electron/dist/Electron.app/Contents/MacOS/Electron --expose-gc docs/audits/gitlab-known-host-retirement/reproduce.cjs docs/audits/gitlab-known-host-retirement/electron-results.json
```
The runner is portable; that executable path is macOS-specific. Thirty-two focused tests pass, including eight new retention/lifecycle/scope controls. Running those eight against the original source produces six failures and two passing controls. Node typecheck, focused lint (including artifact type-aware/casting scans), and the changed-code quality gate pass. The original product module and reused coalescer match named main `291b4ddd6f1c1af480169885e0fda7f9c78ff053`.
## Limits and incident mapping
This removes small metadata retained across SSH generations. Distinct historical execution identities may still keep one entry each until reset; this change does not impose a new cache cap or alter connection/provider lifetime. One generation's host list remains input-sized.
The demonstrated accumulation requires changing SSH provider generations, so it cannot explain [#19831](https://github.com/stablyai/orca/issues/19831)'s reported all-local session. No affected-host observation ties it to another OOM report. The proof measures reachable result arrays, not RSS or gigabytes of incident memory.
@@ -0,0 +1,89 @@
{
"sourceHashes": {
"src/main/gitlab/gitlab-known-host-probe.ts": {
"baseline": "4f9651c5a383438aa968aec082c200a3371e2f4317372125aa11aa6938793c25",
"fixed": "ecf0d67f68cf4b7b1bc7d6ff19a1815a8b95fc9721962d3cd873702f98831bad"
},
"src/main/git/coalesced-probe.ts": "e5a13820a7d8b5f501a3804961ea26526bd6ad9144ecf597d5d83a70d81885e1",
"src/main/git/remote-ref-probe-cache.ts": "d5cbfd97b30e72b03c0d28d8b9e75a2ae1f9e3efc9f5b5570c78e0742c0582dd",
"src/main/gitlab/project-ref-parser.ts": "0f8b6758e6a162a58f436ca4213addc42bf6ceb3fcc2e63e8c7402c844af9303",
"src/main/gitlab/gitlab-known-host-retirement.test.ts": "c6dedbac0462cae22903805d0a89be397a2b9d122a29640b6a07a2d274f8e98e"
},
"proofHashes": {
"reproduce.cjs": "19f49406685923b050a99fbc92b02b9f6f5f8d8f308dcc14336415a0988679cc",
"sources.cjs": "2ee8ac0f295d65f16489da2b0c03800b9db4e88c83ecfb63ae7be1ec3ea6c148",
"fix.patch": "695c929087006b3406ce2c30a7ab783d88af3f9675f4ea320dc7f98b8476be9f",
"original-source-hashes.json": "f9770d9f0a87afc9231eef6af99e8ded33348fdc21ba46a70b1d5d3388874b97"
},
"runtime": {
"node": "24.21.0",
"acorn": "8.18.0",
"ada": "4.0.0",
"amaro": "1.1.11",
"ares": "1.34.8",
"brotli": "1.2.0",
"cldr": "48.0",
"icu": "78.2",
"llhttp": "9.4.3",
"merve": "1.2.2",
"modules": "148",
"napi": "10",
"nbytes": "0.1.4",
"ncrypto": "0.0.1",
"nghttp2": "1.70.0",
"nghttp3": "",
"ngtcp2": "",
"openssl": "0.0.0",
"simdjson": "4.6.7",
"simdutf": "7.7.0",
"sqlite": "3.53.4",
"tz": "2025c",
"undici": "7.29.1",
"unicode": "17.0",
"uv": "1.52.1",
"uvwasi": "0.0.23",
"v8": "15.0.245.31-electron.0",
"zlib": "1.3.2.1-motley",
"zstd": "1.6.0",
"electron": "43.7.0",
"chrome": "150.0.7871.250"
},
"phases": {
"baseline": {
"retained": {
"retained": 128,
"afterReset": 0
},
"rememberedGenerationsRetained": 16,
"oldGeneration": {
"oldResultRetained": true,
"replacementHostsPreserved": true
},
"reset": {
"hosts": ["gitlab.com", "old-before-reset.test"],
"calls": 1
},
"abandoned": ["gitlab.com", "replacement.test", "abandoned.test"],
"rememberedSuccessAndFailure": "passed",
"nativeWslConnectionIsolation": "passed"
},
"fixed": {
"retained": {
"retained": 1,
"afterReset": 0
},
"rememberedGenerationsRetained": 1,
"oldGeneration": {
"oldResultRetained": false,
"replacementHostsPreserved": true
},
"reset": {
"hosts": ["gitlab.com", "fresh-after-reset.test"],
"calls": 2
},
"abandoned": ["gitlab.com", "replacement.test"],
"rememberedSuccessAndFailure": "passed",
"nativeWslConnectionIsolation": "passed"
}
}
}
@@ -0,0 +1,103 @@
diff --git a/src/main/gitlab/gitlab-known-host-probe.ts b/src/main/gitlab/gitlab-known-host-probe.ts
index 752e1b0291..d328cd38ca 100644
--- a/src/main/gitlab/gitlab-known-host-probe.ts
+++ b/src/main/gitlab/gitlab-known-host-probe.ts
@@ -12,7 +12,10 @@ export type LocalGitExecOptions = {
const GLAB_KNOWN_HOSTS_TIMEOUT_MS = 10_000
const UNAUTHENTICATED_HOSTS_MAX_ENTRIES = 128
-const knownHostsCacheByExecutionContext = new Map<string, readonly string[]>()
+const knownHostsCacheByExecutionContext = new Map<
+ string,
+ { key: string; hosts: readonly string[] }
+>()
const knownHostsInFlightByExecutionContext: CoalescedProbes<readonly string[]> = new Map()
const unauthenticatedHostExpiries = new Map<string, number>()
@@ -27,6 +30,19 @@ function knownHostsExecutionKey(
return localGitOptions.wslDistro ? `wsl:${localGitOptions.wslDistro}` : 'native'
}
+function knownHostsCacheContext(
+ connectionId?: string | null,
+ localGitOptions: LocalGitExecOptions = {}
+): { key: string; cacheKey: string } {
+ const key = knownHostsExecutionKey(connectionId, localGitOptions)
+ const cacheKey = connectionId ? `connection:${connectionId}` : key
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)
+ if (cached && cached.key !== key) {
+ knownHostsCacheByExecutionContext.delete(cacheKey)
+ }
+ return { key, cacheKey }
+}
+
/** @internal - exposed for tests only */
export function _resetKnownHostsCache(): void {
knownHostsCacheByExecutionContext.clear()
@@ -103,8 +119,8 @@ export function rememberGlabKnownHosts(
connectionId?: string | null,
localGitOptions: LocalGitExecOptions = {}
): void {
- const key = knownHostsExecutionKey(connectionId, localGitOptions)
- const cached = knownHostsCacheByExecutionContext.get(key) ?? DEFAULT_GITLAB_HOSTS
+ const { key, cacheKey } = knownHostsCacheContext(connectionId, localGitOptions)
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)?.hosts ?? DEFAULT_GITLAB_HOSTS
const seen = new Set(cached.map(normalizeGitLabHost))
const additions: string[] = []
for (const host of hosts) {
@@ -121,27 +137,29 @@ export function rememberGlabKnownHosts(
if (additions.length === 0) {
return
}
- knownHostsCacheByExecutionContext.set(key, [...cached, ...additions])
+ knownHostsCacheByExecutionContext.set(cacheKey, { key, hosts: [...cached, ...additions] })
}
export async function getGlabKnownHosts(
connectionId?: string | null,
localGitOptions: LocalGitExecOptions = {}
): Promise<readonly string[]> {
- const key = knownHostsExecutionKey(connectionId, localGitOptions)
- const cached = knownHostsCacheByExecutionContext.get(key)
+ const { key, cacheKey } = knownHostsCacheContext(connectionId, localGitOptions)
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)?.hosts
if (cached) {
return cached
}
// Why: only join a probe still young enough to answer, so a wedged one cannot
// pin every later retry for the life of the process (P1-D).
- return runCoalescedProbe(knownHostsInFlightByExecutionContext, key, () =>
- probeGlabKnownHosts(key, connectionId, localGitOptions)
+ return runCoalescedProbe(knownHostsInFlightByExecutionContext, key, (ownsKey) =>
+ probeGlabKnownHosts(key, cacheKey, ownsKey, connectionId, localGitOptions)
)
}
async function probeGlabKnownHosts(
key: string,
+ cacheKey: string,
+ ownsKey: () => boolean,
connectionId?: string | null,
localGitOptions: LocalGitExecOptions = {}
): Promise<readonly string[]> {
@@ -160,13 +178,17 @@ async function probeGlabKnownHosts(
...(localGitOptions.admissionTier ? { admissionTier: localGitOptions.admissionTier } : {})
})
const hosts = parseGlabAuthStatusHosts(`${stdout}\n${stderr}`)
- const remembered = knownHostsCacheByExecutionContext.get(key) ?? []
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)
+ const remembered = cached?.key === key ? cached.hosts : []
const merged = Array.from(new Set([...DEFAULT_GITLAB_HOSTS, ...remembered, ...hosts]))
- knownHostsCacheByExecutionContext.set(key, merged)
+ if (ownsKey() && knownHostsExecutionKey(connectionId, localGitOptions) === key) {
+ knownHostsCacheByExecutionContext.set(cacheKey, { key, hosts: merged })
+ }
return merged
} catch {
// Keep failures uncached so auth or tunnel recovery is discovered later.
- return knownHostsCacheByExecutionContext.get(key) ?? [...DEFAULT_GITLAB_HOSTS]
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)
+ return cached?.key === key ? cached.hosts : [...DEFAULT_GITLAB_HOSTS]
}
}
@@ -0,0 +1,3 @@
{
"src/main/gitlab/gitlab-known-host-probe.ts": "4f9651c5a383438aa968aec082c200a3371e2f4317372125aa11aa6938793c25"
}
@@ -0,0 +1,257 @@
const assert = require('node:assert/strict')
const fs = require('node:fs')
const path = require('node:path')
const Module = require('node:module')
const esbuild = require('esbuild')
const { sourcePath, baseline, fixed, sourceHashes, hash } = require('./sources.cjs')
assert.equal(process.env.ORCA_BACKGROUND_LAUNCH, '1')
assert.equal(typeof global.gc, 'function')
const symbol = Symbol.for('orca-known-host-comparison')
const context = { generation: 1, calls: 0, runner: null }
globalThis[symbol] = context
const resultFor = (host) => ({ stdout: `Logged in to ${host} as user`, stderr: '' })
async function load(phase) {
const source = phase === 'baseline' ? baseline : fixed
const built = await esbuild.build({
entryPoints: [sourcePath],
bundle: true,
platform: 'node',
format: 'cjs',
write: false,
plugins: [
{
name: 'actual-cache-with-fixture-ports',
setup(build) {
build.onLoad({ filter: /gitlab-known-host-probe\.ts$/ }, () => ({
contents: source,
loader: 'ts'
}))
build.onResolve({ filter: /\/(runner|ssh-git-dispatch)$/ }, (args) => ({
path: path.basename(args.path),
namespace: 'ports'
}))
build.onLoad({ filter: /.*/, namespace: 'ports' }, (args) => ({
loader: 'js',
contents: `const context=globalThis[Symbol.for('orca-known-host-comparison')];${
args.path === 'runner'
? `exports.glabExecFileAsync=(...args)=>{context.calls++;return context.runner(...args)};`
: `exports.getSshGitProviderGeneration=()=>context.generation;`
}`
}))
}
}
]
})
const loaded = new Module(sourcePath, module)
loaded.filename = sourcePath
loaded.paths = module.paths
loaded._compile(built.outputFiles[0].text, sourcePath)
return loaded.exports
}
async function collect() {
for (let index = 0; index < 6; index++) {
await new Promise((resolve) => setImmediate(resolve))
global.gc()
}
await new Promise((resolve) => setImmediate(resolve))
}
async function rememberResult(api) {
const hosts = await api.getGlabKnownHosts('same-connection')
assert.deepEqual(hosts, ['gitlab.com', `host${context.generation}.test`])
return new WeakRef(hosts)
}
async function retention(api) {
api._resetKnownHostsCache()
context.calls = 0
context.runner = async () => resultFor(`host${context.generation}.test`)
const refs = []
for (let generation = 1; generation <= 128; generation++) {
context.generation = generation
refs.push(await rememberResult(api))
}
await collect()
const retained = refs.filter((ref) => ref.deref() !== undefined).length
await rememberResult(api)
assert.equal(context.calls, 128)
api._resetKnownHostsCache()
await collect()
const afterReset = refs.filter((ref) => ref.deref() !== undefined).length
assert.equal(afterReset, 0)
return { retained, afterReset }
}
async function afterReset(api) {
api._resetKnownHostsCache()
context.calls = 0
const pending = Promise.withResolvers()
context.runner = () => pending.promise
const old = api.getGlabKnownHosts()
api._resetKnownHostsCache()
context.runner = async () => resultFor('fresh-after-reset.test')
pending.resolve(resultFor('old-before-reset.test'))
assert.deepEqual(await old, ['gitlab.com', 'old-before-reset.test'])
return { hosts: await api.getGlabKnownHosts(), calls: context.calls }
}
async function weakResult(promise) {
return new WeakRef(await promise)
}
async function lateGeneration(api) {
api._resetKnownHostsCache()
context.generation = 1
const pending = Promise.withResolvers()
context.runner = () => pending.promise
let old = api.getGlabKnownHosts('same-connection')
context.generation = 2
context.runner = async () => resultFor('replacement-generation.test')
assert.deepEqual(await api.getGlabKnownHosts('same-connection'), [
'gitlab.com',
'replacement-generation.test'
])
pending.resolve(resultFor('retired-generation.test'))
const oldResult = await weakResult(old)
old = null
await collect()
const retained = oldResult.deref() !== undefined
assert.deepEqual(await api.getGlabKnownHosts('same-connection'), [
'gitlab.com',
'replacement-generation.test'
])
return { oldResultRetained: retained, replacementHostsPreserved: true }
}
async function rememberGeneration(api) {
api._resetKnownHostsCache()
context.runner = () => {
throw new Error('remembered hosts must not probe')
}
const refs = []
for (let generation = 1; generation <= 16; generation++) {
context.generation = generation
api.rememberGlabKnownHost(`host${generation}.test`, 'same-connection')
refs.push(await rememberResult(api))
}
await collect()
return refs.filter((ref) => ref.deref() !== undefined).length
}
async function abandonedProbe(api) {
api._resetKnownHostsCache()
const originalNow = Date.now
let now = 1000
Date.now = () => now
try {
const pending = Promise.withResolvers()
context.runner = () => pending.promise
const old = api.getGlabKnownHosts()
now += 60_001
context.runner = async () => resultFor('replacement.test')
assert.deepEqual(await api.getGlabKnownHosts(), ['gitlab.com', 'replacement.test'])
pending.resolve(resultFor('abandoned.test'))
await old
return await api.getGlabKnownHosts()
} finally {
Date.now = originalNow
}
}
async function rememberWhilePending(api, fail) {
api._resetKnownHostsCache()
const pending = Promise.withResolvers()
context.runner = () => pending.promise
const old = api.getGlabKnownHosts()
api.rememberGlabKnownHosts(['Remembered.TEST', ' remembered.test '])
if (fail) {
pending.reject(new Error('controlled auth failure'))
} else {
pending.resolve(resultFor('gitlab.com'))
}
assert.deepEqual(await old, ['gitlab.com', 'remembered.test'])
assert.deepEqual(await api.getGlabKnownHosts(), ['gitlab.com', 'remembered.test'])
}
async function scopeIsolation(api) {
api._resetKnownHostsCache()
const contexts = [
[undefined, {}],
[undefined, { wslDistro: 'Ubuntu' }],
[undefined, { wslDistro: 'Debian' }],
['connection-a', {}],
['connection-b', {}]
]
for (let index = 0; index < contexts.length; index++) {
context.runner = async () => resultFor(`scope${index}.test`)
assert.deepEqual(await api.getGlabKnownHosts(...contexts[index]), [
'gitlab.com',
`scope${index}.test`
])
}
context.runner = () => {
throw new Error('cached contexts must not probe')
}
for (let index = 0; index < contexts.length; index++) {
assert.deepEqual(await api.getGlabKnownHosts(...contexts[index]), [
'gitlab.com',
`scope${index}.test`
])
}
}
async function main() {
const proofHashes = Object.fromEntries(
['reproduce.cjs', 'sources.cjs', 'fix.patch', 'original-source-hashes.json'].map((file) => [
file,
hash(fs.readFileSync(path.join(__dirname, file)))
])
)
const report = { sourceHashes, proofHashes, runtime: process.versions, phases: {} }
for (const phase of ['baseline', 'fixed']) {
const api = await load(phase)
const retained = await retention(api)
const rememberedGenerationsRetained = await rememberGeneration(api)
const oldGeneration = await lateGeneration(api)
const reset = await afterReset(api)
const abandoned = await abandonedProbe(api)
await rememberWhilePending(api, false)
await rememberWhilePending(api, true)
await scopeIsolation(api)
assert.equal(retained.retained, phase === 'baseline' ? 128 : 1)
assert.equal(rememberedGenerationsRetained, phase === 'baseline' ? 16 : 1)
assert.equal(oldGeneration.oldResultRetained, phase === 'baseline')
assert.deepEqual(
reset.hosts,
phase === 'baseline'
? ['gitlab.com', 'old-before-reset.test']
: ['gitlab.com', 'fresh-after-reset.test']
)
assert.deepEqual(
abandoned,
phase === 'baseline'
? ['gitlab.com', 'replacement.test', 'abandoned.test']
: ['gitlab.com', 'replacement.test']
)
report.phases[phase] = {
retained,
rememberedGenerationsRetained,
oldGeneration,
reset,
abandoned,
rememberedSuccessAndFailure: 'passed',
nativeWslConnectionIsolation: 'passed'
}
api._resetKnownHostsCache()
}
fs.writeFileSync(
process.argv[2] || path.join(__dirname, 'results.json'),
`${JSON.stringify(report, null, 2)}\n`
)
console.log(JSON.stringify(report.phases, null, 2))
}
main()
.catch((error) => {
console.error(error)
process.exitCode = 1
})
.finally(() => {
delete globalThis[symbol]
})
setTimeout(() => {
console.error('fixture deadline')
process.exit(2)
}, 10000).unref()
@@ -0,0 +1,88 @@
{
"sourceHashes": {
"src/main/gitlab/gitlab-known-host-probe.ts": {
"baseline": "4f9651c5a383438aa968aec082c200a3371e2f4317372125aa11aa6938793c25",
"fixed": "ecf0d67f68cf4b7b1bc7d6ff19a1815a8b95fc9721962d3cd873702f98831bad"
},
"src/main/git/coalesced-probe.ts": "e5a13820a7d8b5f501a3804961ea26526bd6ad9144ecf597d5d83a70d81885e1",
"src/main/git/remote-ref-probe-cache.ts": "d5cbfd97b30e72b03c0d28d8b9e75a2ae1f9e3efc9f5b5570c78e0742c0582dd",
"src/main/gitlab/project-ref-parser.ts": "0f8b6758e6a162a58f436ca4213addc42bf6ceb3fcc2e63e8c7402c844af9303",
"src/main/gitlab/gitlab-known-host-retirement.test.ts": "c6dedbac0462cae22903805d0a89be397a2b9d122a29640b6a07a2d274f8e98e"
},
"proofHashes": {
"reproduce.cjs": "19f49406685923b050a99fbc92b02b9f6f5f8d8f308dcc14336415a0988679cc",
"sources.cjs": "2ee8ac0f295d65f16489da2b0c03800b9db4e88c83ecfb63ae7be1ec3ea6c148",
"fix.patch": "695c929087006b3406ce2c30a7ab783d88af3f9675f4ea320dc7f98b8476be9f",
"original-source-hashes.json": "f9770d9f0a87afc9231eef6af99e8ded33348fdc21ba46a70b1d5d3388874b97"
},
"runtime": {
"node": "26.6.0",
"acorn": "8.17.0",
"ada": "4.0.0",
"amaro": "1.1.11",
"ares": "1.34.8",
"brotli": "1.2.0",
"cldr": "48.0",
"icu": "78.3",
"libffi": "3.7.1",
"llhttp": "9.4.3",
"merve": "1.2.2",
"modules": "147",
"napi": "10",
"nbytes": "0.1.4",
"ncrypto": "0.0.1",
"nghttp2": "1.70.0",
"nghttp3": "",
"ngtcp2": "",
"openssl": "3.6.3",
"simdjson": "4.6.6",
"simdutf": "7.7.0",
"sqlite": "3.53.4",
"tz": "2026a",
"undici": "8.9.0",
"unicode": "17.0",
"uv": "1.52.1",
"uvwasi": "0.0.23",
"v8": "14.6.202.34-node.26",
"zlib": "1.2.12",
"zstd": "1.5.7"
},
"phases": {
"baseline": {
"retained": {
"retained": 128,
"afterReset": 0
},
"rememberedGenerationsRetained": 16,
"oldGeneration": {
"oldResultRetained": true,
"replacementHostsPreserved": true
},
"reset": {
"hosts": ["gitlab.com", "old-before-reset.test"],
"calls": 1
},
"abandoned": ["gitlab.com", "replacement.test", "abandoned.test"],
"rememberedSuccessAndFailure": "passed",
"nativeWslConnectionIsolation": "passed"
},
"fixed": {
"retained": {
"retained": 1,
"afterReset": 0
},
"rememberedGenerationsRetained": 1,
"oldGeneration": {
"oldResultRetained": false,
"replacementHostsPreserved": true
},
"reset": {
"hosts": ["gitlab.com", "fresh-after-reset.test"],
"calls": 2
},
"abandoned": ["gitlab.com", "replacement.test"],
"rememberedSuccessAndFailure": "passed",
"nativeWslConnectionIsolation": "passed"
}
}
}
@@ -0,0 +1,30 @@
const assert = require('node:assert/strict')
const fs = require('node:fs')
const path = require('node:path')
const crypto = require('node:crypto')
const { parsePatch, reversePatch, applyPatch } = require('diff')
const root = path.resolve(__dirname, '../../..')
const relativePath = 'src/main/gitlab/gitlab-known-host-probe.ts'
const sourcePath = path.join(root, relativePath)
const fixed = fs.readFileSync(sourcePath, 'utf8')
const patches = parsePatch(fs.readFileSync(path.join(__dirname, 'fix.patch'), 'utf8'))
assert.equal(patches.length, 1)
assert.equal(patches[0].newFileName, `b/${relativePath}`)
const baseline = applyPatch(fixed, reversePatch(patches[0]))
assert.notEqual(baseline, false, 'Current source must reverse exactly to the original cache')
const hash = (value) => crypto.createHash('sha256').update(value).digest('hex')
assert.equal(hash(baseline), require('./original-source-hashes.json')[relativePath])
const sourceHashes = {
[relativePath]: { baseline: hash(baseline), fixed: hash(fixed) },
...Object.fromEntries(
[
'src/main/git/coalesced-probe.ts',
'src/main/git/remote-ref-probe-cache.ts',
'src/main/gitlab/project-ref-parser.ts',
'src/main/gitlab/gitlab-known-host-retirement.test.ts'
].map((file) => [file, hash(fs.readFileSync(path.join(root, file)))])
)
}
module.exports = { root, sourcePath, baseline, fixed, sourceHashes, hash }
+33 -11
View File
@@ -12,7 +12,10 @@ export type LocalGitExecOptions = {
const GLAB_KNOWN_HOSTS_TIMEOUT_MS = 10_000
const UNAUTHENTICATED_HOSTS_MAX_ENTRIES = 128
const knownHostsCacheByExecutionContext = new Map<string, readonly string[]>()
const knownHostsCacheByExecutionContext = new Map<
string,
{ key: string; hosts: readonly string[] }
>()
const knownHostsInFlightByExecutionContext: CoalescedProbes<readonly string[]> = new Map()
const unauthenticatedHostExpiries = new Map<string, number>()
@@ -27,6 +30,19 @@ function knownHostsExecutionKey(
return localGitOptions.wslDistro ? `wsl:${localGitOptions.wslDistro}` : 'native'
}
function knownHostsCacheContext(
connectionId?: string | null,
localGitOptions: LocalGitExecOptions = {}
): { key: string; cacheKey: string } {
const key = knownHostsExecutionKey(connectionId, localGitOptions)
const cacheKey = connectionId ? `connection:${connectionId}` : key
const cached = knownHostsCacheByExecutionContext.get(cacheKey)
if (cached && cached.key !== key) {
knownHostsCacheByExecutionContext.delete(cacheKey)
}
return { key, cacheKey }
}
/** @internal - exposed for tests only */
export function _resetKnownHostsCache(): void {
knownHostsCacheByExecutionContext.clear()
@@ -103,8 +119,8 @@ export function rememberGlabKnownHosts(
connectionId?: string | null,
localGitOptions: LocalGitExecOptions = {}
): void {
const key = knownHostsExecutionKey(connectionId, localGitOptions)
const cached = knownHostsCacheByExecutionContext.get(key) ?? DEFAULT_GITLAB_HOSTS
const { key, cacheKey } = knownHostsCacheContext(connectionId, localGitOptions)
const cached = knownHostsCacheByExecutionContext.get(cacheKey)?.hosts ?? DEFAULT_GITLAB_HOSTS
const seen = new Set(cached.map(normalizeGitLabHost))
const additions: string[] = []
for (const host of hosts) {
@@ -121,27 +137,29 @@ export function rememberGlabKnownHosts(
if (additions.length === 0) {
return
}
knownHostsCacheByExecutionContext.set(key, [...cached, ...additions])
knownHostsCacheByExecutionContext.set(cacheKey, { key, hosts: [...cached, ...additions] })
}
export async function getGlabKnownHosts(
connectionId?: string | null,
localGitOptions: LocalGitExecOptions = {}
): Promise<readonly string[]> {
const key = knownHostsExecutionKey(connectionId, localGitOptions)
const cached = knownHostsCacheByExecutionContext.get(key)
const { key, cacheKey } = knownHostsCacheContext(connectionId, localGitOptions)
const cached = knownHostsCacheByExecutionContext.get(cacheKey)?.hosts
if (cached) {
return cached
}
// Why: only join a probe still young enough to answer, so a wedged one cannot
// pin every later retry for the life of the process (P1-D).
return runCoalescedProbe(knownHostsInFlightByExecutionContext, key, () =>
probeGlabKnownHosts(key, connectionId, localGitOptions)
return runCoalescedProbe(knownHostsInFlightByExecutionContext, key, (ownsKey) =>
probeGlabKnownHosts(key, cacheKey, ownsKey, connectionId, localGitOptions)
)
}
async function probeGlabKnownHosts(
key: string,
cacheKey: string,
ownsKey: () => boolean,
connectionId?: string | null,
localGitOptions: LocalGitExecOptions = {}
): Promise<readonly string[]> {
@@ -160,13 +178,17 @@ async function probeGlabKnownHosts(
...(localGitOptions.admissionTier ? { admissionTier: localGitOptions.admissionTier } : {})
})
const hosts = parseGlabAuthStatusHosts(`${stdout}\n${stderr}`)
const remembered = knownHostsCacheByExecutionContext.get(key) ?? []
const cached = knownHostsCacheByExecutionContext.get(cacheKey)
const remembered = cached?.key === key ? cached.hosts : []
const merged = Array.from(new Set([...DEFAULT_GITLAB_HOSTS, ...remembered, ...hosts]))
knownHostsCacheByExecutionContext.set(key, merged)
if (ownsKey() && knownHostsExecutionKey(connectionId, localGitOptions) === key) {
knownHostsCacheByExecutionContext.set(cacheKey, { key, hosts: merged })
}
return merged
} catch {
// Keep failures uncached so auth or tunnel recovery is discovered later.
return knownHostsCacheByExecutionContext.get(key) ?? [...DEFAULT_GITLAB_HOSTS]
const cached = knownHostsCacheByExecutionContext.get(cacheKey)
return cached?.key === key ? cached.hosts : [...DEFAULT_GITLAB_HOSTS]
}
}
@@ -0,0 +1,171 @@
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
const { execute, generations } = vi.hoisted(() => ({
execute: vi.fn(),
generations: new Map<string, number>()
}))
vi.mock('../git/runner', () => ({ glabExecFileAsync: execute }))
vi.mock('../providers/ssh-git-dispatch', () => ({
getSshGitProviderGeneration: (connectionId: string) => generations.get(connectionId) ?? 0
}))
import {
_resetKnownHostsCache,
getGlabKnownHosts,
rememberGlabKnownHost
} from './gitlab-known-host-probe'
import { PROBE_COALESCE_STALE_MS } from '../git/coalesced-probe'
const response = (host: string) => ({ stdout: `Logged in to ${host} as user`, stderr: '' })
const deferred = () => Promise.withResolvers<ReturnType<typeof response>>()
async function collect(): Promise<void> {
if (typeof globalThis.gc !== 'function') {
throw new Error('Run with the repository Vitest --expose-gc config')
}
for (let index = 0; index < 6; index++) {
await new Promise<void>((resolve) => setImmediate(resolve))
globalThis.gc()
}
await new Promise<void>((resolve) => setImmediate(resolve))
}
async function weakResult(connectionId: string): Promise<WeakRef<readonly string[]>> {
return new WeakRef(await getGlabKnownHosts(connectionId))
}
beforeEach(() => {
_resetKnownHostsCache()
generations.clear()
execute.mockReset()
})
afterEach(() => {
_resetKnownHostsCache()
vi.restoreAllMocks()
})
it('releases successful host arrays from superseded SSH generations', async () => {
const results: WeakRef<readonly string[]>[] = []
for (let generation = 1; generation <= 32; generation++) {
generations.set('connection', generation)
execute.mockResolvedValue(response(`host${generation}.test`))
results.push(await weakResult('connection'))
}
await collect()
expect(results.filter((result) => result.deref() !== undefined)).toHaveLength(1)
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'host32.test'])
expect(execute).toHaveBeenCalledTimes(32)
})
it('retires remembered generations without requiring an auth-status probe', async () => {
const results: WeakRef<readonly string[]>[] = []
for (let generation = 1; generation <= 16; generation++) {
generations.set('connection', generation)
rememberGlabKnownHost(`host${generation}.test`, 'connection')
results.push(await weakResult('connection'))
}
await collect()
expect(results.filter((result) => result.deref() !== undefined)).toHaveLength(1)
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'host16.test'])
expect(execute).not.toHaveBeenCalled()
})
it('does not retain a delayed old-generation result after a replacement answers', async () => {
const old = deferred()
generations.set('connection', 1)
execute.mockReturnValueOnce(old.promise)
const oldResult = weakResult('connection')
generations.set('connection', 2)
execute.mockResolvedValueOnce(response('replacement.test'))
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'replacement.test'])
old.resolve(response('retired.test'))
const reference = await oldResult
await collect()
expect(reference.deref()).toBeUndefined()
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'replacement.test'])
expect(execute).toHaveBeenCalledTimes(2)
})
it('does not repopulate an explicitly reset cache from an earlier probe', async () => {
const old = deferred()
execute.mockReturnValueOnce(old.promise)
const oldResult = getGlabKnownHosts()
_resetKnownHostsCache()
old.resolve(response('before-reset.test'))
await expect(oldResult).resolves.toEqual(['gitlab.com', 'before-reset.test'])
execute.mockResolvedValueOnce(response('after-reset.test'))
await expect(getGlabKnownHosts()).resolves.toEqual(['gitlab.com', 'after-reset.test'])
expect(execute).toHaveBeenCalledTimes(2)
})
it('keeps a post-reset successor joinable when the old probe settles first', async () => {
const old = deferred()
const next = deferred()
execute.mockReturnValueOnce(old.promise).mockReturnValueOnce(next.promise)
const oldResult = getGlabKnownHosts()
_resetKnownHostsCache()
const nextResult = getGlabKnownHosts()
old.resolve(response('before-reset.test'))
await oldResult
let joinedSettled = false
const joined = getGlabKnownHosts().then((hosts) => {
joinedSettled = true
return hosts
})
await Promise.resolve()
expect(joinedSettled).toBe(false)
next.resolve(response('after-reset.test'))
await expect(nextResult).resolves.toEqual(['gitlab.com', 'after-reset.test'])
await expect(joined).resolves.toEqual(['gitlab.com', 'after-reset.test'])
expect(execute).toHaveBeenCalledTimes(2)
})
it('prevents an abandoned same-generation probe from publishing over its successor', async () => {
const clock = vi.spyOn(Date, 'now').mockReturnValue(1000)
const old = deferred()
execute.mockReturnValueOnce(old.promise)
const oldResult = getGlabKnownHosts('connection')
clock.mockReturnValue(1000 + PROBE_COALESCE_STALE_MS + 1)
execute.mockResolvedValueOnce(response('replacement.test'))
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'replacement.test'])
old.resolve(response('abandoned.test'))
await oldResult
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'replacement.test'])
expect(execute).toHaveBeenCalledTimes(2)
})
it('keeps native, WSL and other connection caches when one generation changes', async () => {
execute
.mockResolvedValueOnce(response('native.test'))
.mockResolvedValueOnce(response('ubuntu.test'))
.mockResolvedValueOnce(response('debian.test'))
.mockResolvedValueOnce(response('connection-a.test'))
.mockResolvedValueOnce(response('connection-b.test'))
const native = await getGlabKnownHosts()
const ubuntu = await getGlabKnownHosts(undefined, { wslDistro: 'Ubuntu' })
const debian = await getGlabKnownHosts(undefined, { wslDistro: 'Debian' })
await getGlabKnownHosts('connection-a')
const other = await getGlabKnownHosts('connection-b')
generations.set('connection-a', 1)
rememberGlabKnownHost('replacement.test', 'connection-a')
await expect(getGlabKnownHosts('connection-a')).resolves.toEqual([
'gitlab.com',
'replacement.test'
])
await expect(getGlabKnownHosts()).resolves.toBe(native)
await expect(getGlabKnownHosts(undefined, { wslDistro: 'Ubuntu' })).resolves.toBe(ubuntu)
await expect(getGlabKnownHosts(undefined, { wslDistro: 'Debian' })).resolves.toBe(debian)
await expect(getGlabKnownHosts('connection-b')).resolves.toBe(other)
expect(execute).toHaveBeenCalledTimes(5)
})
it('does not serve a retired generation after the current probe fails', async () => {
execute.mockResolvedValueOnce(response('retired.test'))
await getGlabKnownHosts('connection')
generations.set('connection', 1)
execute.mockRejectedValueOnce(new Error('current host unavailable'))
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com'])
execute.mockResolvedValueOnce(response('current.test'))
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'current.test'])
expect(execute).toHaveBeenCalledTimes(3)
})