mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 00:02:31 +00:00
fix: retire obsolete GitLab host cache generations (#21136)
Co-authored-by: m4air <m4air@Mac.localdomain>
This commit is contained in:
@@ -0,0 +1,39 @@
|
||||
# Retire obsolete GitLab known-host generations
|
||||
|
||||
Each successful `getGlabKnownHosts` probe previously stored a host-list array under a connection ID plus its SSH provider generation. Reconnecting under the same ID created a new entry while every earlier successful generation remained cached until an explicit preflight reset. The cache now keeps one successful generation per observed execution identity.
|
||||
|
||||
Async publication also uses the existing coalescer's `ownsKey()` and checks the current SSH generation. A result completing after reconnect, explicit reset, or replacement by a newer probe cannot recreate retired cache state. Original callers can still receive their own completed result. Explicitly remembered hosts, native/WSL separation, command routing and existing probe timeouts are preserved.
|
||||
|
||||
## Evidence
|
||||
|
||||
The runner bundles the actual cache, coalescer and parser. Only command-result and SSH-generation ports are controlled; it opens no SSH connection and runs no GitLab command. It reverses `fix.patch` in memory, verifies the original source hash, and compares that baseline against the unmodified current product source. Reports include product, dependency, regression-test and fixture hashes.
|
||||
|
||||
| Control | Original | Fixed |
|
||||
| --- | --- | --- |
|
||||
| Successful result arrays retained after 128 generations | 128 | 1 current array |
|
||||
| Remembered result arrays retained after 16 generations | 16 | 1 current array |
|
||||
| Delayed old-generation result after a successor answers | Still retained | Collectable; successor preserved |
|
||||
| Explicit reset followed by old completion | Old result repopulates cache | Next lookup executes a fresh probe |
|
||||
| Abandoned probe finishes after its replacement | Old host added to replacement cache | Replacement remains unchanged |
|
||||
| Explicit reset after retention exercise | 0 original arrays retained | 0 original arrays retained |
|
||||
|
||||
Both phases preserve remembered-host updates while probes succeed or fail and isolate native, Ubuntu WSL, Debian WSL and two connection IDs. `results.json` records Node26.6; `electron-results.json` records installed Electron43.7 / Node24.21 running without an app window. Both runs pass all controls. This is compatibility evidence, not a historical packaged-binary reproduction.
|
||||
|
||||
```sh
|
||||
ORCA_BACKGROUND_LAUNCH=1 node --expose-gc docs/audits/gitlab-known-host-retirement/reproduce.cjs
|
||||
ORCA_BACKGROUND_LAUNCH=1 node node_modules/vitest/vitest.mjs run --config config/vitest.config.ts src/main/gitlab/gitlab-known-host-retirement.test.ts src/main/gitlab/gitlab-known-host-probe.test.ts src/main/gitlab/gitlab-known-host-probe-wsl-fallback.test.ts src/main/git/coalesced-probe.test.ts src/main/gitlab/client-mr-auth-rate-limit.test.ts
|
||||
```
|
||||
|
||||
For the installed macOS Electron binary:
|
||||
|
||||
```sh
|
||||
ELECTRON_RUN_AS_NODE=1 ORCA_BACKGROUND_LAUNCH=1 node_modules/electron/dist/Electron.app/Contents/MacOS/Electron --expose-gc docs/audits/gitlab-known-host-retirement/reproduce.cjs docs/audits/gitlab-known-host-retirement/electron-results.json
|
||||
```
|
||||
|
||||
The runner is portable; that executable path is macOS-specific. Thirty-two focused tests pass, including eight new retention/lifecycle/scope controls. Running those eight against the original source produces six failures and two passing controls. Node typecheck, focused lint (including artifact type-aware/casting scans), and the changed-code quality gate pass. The original product module and reused coalescer match named main `291b4ddd6f1c1af480169885e0fda7f9c78ff053`.
|
||||
|
||||
## Limits and incident mapping
|
||||
|
||||
This removes small metadata retained across SSH generations. Distinct historical execution identities may still keep one entry each until reset; this change does not impose a new cache cap or alter connection/provider lifetime. One generation's host list remains input-sized.
|
||||
|
||||
The demonstrated accumulation requires changing SSH provider generations, so it cannot explain [#19831](https://github.com/stablyai/orca/issues/19831)'s reported all-local session. No affected-host observation ties it to another OOM report. The proof measures reachable result arrays, not RSS or gigabytes of incident memory.
|
||||
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"sourceHashes": {
|
||||
"src/main/gitlab/gitlab-known-host-probe.ts": {
|
||||
"baseline": "4f9651c5a383438aa968aec082c200a3371e2f4317372125aa11aa6938793c25",
|
||||
"fixed": "ecf0d67f68cf4b7b1bc7d6ff19a1815a8b95fc9721962d3cd873702f98831bad"
|
||||
},
|
||||
"src/main/git/coalesced-probe.ts": "e5a13820a7d8b5f501a3804961ea26526bd6ad9144ecf597d5d83a70d81885e1",
|
||||
"src/main/git/remote-ref-probe-cache.ts": "d5cbfd97b30e72b03c0d28d8b9e75a2ae1f9e3efc9f5b5570c78e0742c0582dd",
|
||||
"src/main/gitlab/project-ref-parser.ts": "0f8b6758e6a162a58f436ca4213addc42bf6ceb3fcc2e63e8c7402c844af9303",
|
||||
"src/main/gitlab/gitlab-known-host-retirement.test.ts": "c6dedbac0462cae22903805d0a89be397a2b9d122a29640b6a07a2d274f8e98e"
|
||||
},
|
||||
"proofHashes": {
|
||||
"reproduce.cjs": "19f49406685923b050a99fbc92b02b9f6f5f8d8f308dcc14336415a0988679cc",
|
||||
"sources.cjs": "2ee8ac0f295d65f16489da2b0c03800b9db4e88c83ecfb63ae7be1ec3ea6c148",
|
||||
"fix.patch": "695c929087006b3406ce2c30a7ab783d88af3f9675f4ea320dc7f98b8476be9f",
|
||||
"original-source-hashes.json": "f9770d9f0a87afc9231eef6af99e8ded33348fdc21ba46a70b1d5d3388874b97"
|
||||
},
|
||||
"runtime": {
|
||||
"node": "24.21.0",
|
||||
"acorn": "8.18.0",
|
||||
"ada": "4.0.0",
|
||||
"amaro": "1.1.11",
|
||||
"ares": "1.34.8",
|
||||
"brotli": "1.2.0",
|
||||
"cldr": "48.0",
|
||||
"icu": "78.2",
|
||||
"llhttp": "9.4.3",
|
||||
"merve": "1.2.2",
|
||||
"modules": "148",
|
||||
"napi": "10",
|
||||
"nbytes": "0.1.4",
|
||||
"ncrypto": "0.0.1",
|
||||
"nghttp2": "1.70.0",
|
||||
"nghttp3": "",
|
||||
"ngtcp2": "",
|
||||
"openssl": "0.0.0",
|
||||
"simdjson": "4.6.7",
|
||||
"simdutf": "7.7.0",
|
||||
"sqlite": "3.53.4",
|
||||
"tz": "2025c",
|
||||
"undici": "7.29.1",
|
||||
"unicode": "17.0",
|
||||
"uv": "1.52.1",
|
||||
"uvwasi": "0.0.23",
|
||||
"v8": "15.0.245.31-electron.0",
|
||||
"zlib": "1.3.2.1-motley",
|
||||
"zstd": "1.6.0",
|
||||
"electron": "43.7.0",
|
||||
"chrome": "150.0.7871.250"
|
||||
},
|
||||
"phases": {
|
||||
"baseline": {
|
||||
"retained": {
|
||||
"retained": 128,
|
||||
"afterReset": 0
|
||||
},
|
||||
"rememberedGenerationsRetained": 16,
|
||||
"oldGeneration": {
|
||||
"oldResultRetained": true,
|
||||
"replacementHostsPreserved": true
|
||||
},
|
||||
"reset": {
|
||||
"hosts": ["gitlab.com", "old-before-reset.test"],
|
||||
"calls": 1
|
||||
},
|
||||
"abandoned": ["gitlab.com", "replacement.test", "abandoned.test"],
|
||||
"rememberedSuccessAndFailure": "passed",
|
||||
"nativeWslConnectionIsolation": "passed"
|
||||
},
|
||||
"fixed": {
|
||||
"retained": {
|
||||
"retained": 1,
|
||||
"afterReset": 0
|
||||
},
|
||||
"rememberedGenerationsRetained": 1,
|
||||
"oldGeneration": {
|
||||
"oldResultRetained": false,
|
||||
"replacementHostsPreserved": true
|
||||
},
|
||||
"reset": {
|
||||
"hosts": ["gitlab.com", "fresh-after-reset.test"],
|
||||
"calls": 2
|
||||
},
|
||||
"abandoned": ["gitlab.com", "replacement.test"],
|
||||
"rememberedSuccessAndFailure": "passed",
|
||||
"nativeWslConnectionIsolation": "passed"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
diff --git a/src/main/gitlab/gitlab-known-host-probe.ts b/src/main/gitlab/gitlab-known-host-probe.ts
|
||||
index 752e1b0291..d328cd38ca 100644
|
||||
--- a/src/main/gitlab/gitlab-known-host-probe.ts
|
||||
+++ b/src/main/gitlab/gitlab-known-host-probe.ts
|
||||
@@ -12,7 +12,10 @@ export type LocalGitExecOptions = {
|
||||
|
||||
const GLAB_KNOWN_HOSTS_TIMEOUT_MS = 10_000
|
||||
const UNAUTHENTICATED_HOSTS_MAX_ENTRIES = 128
|
||||
-const knownHostsCacheByExecutionContext = new Map<string, readonly string[]>()
|
||||
+const knownHostsCacheByExecutionContext = new Map<
|
||||
+ string,
|
||||
+ { key: string; hosts: readonly string[] }
|
||||
+>()
|
||||
const knownHostsInFlightByExecutionContext: CoalescedProbes<readonly string[]> = new Map()
|
||||
const unauthenticatedHostExpiries = new Map<string, number>()
|
||||
|
||||
@@ -27,6 +30,19 @@ function knownHostsExecutionKey(
|
||||
return localGitOptions.wslDistro ? `wsl:${localGitOptions.wslDistro}` : 'native'
|
||||
}
|
||||
|
||||
+function knownHostsCacheContext(
|
||||
+ connectionId?: string | null,
|
||||
+ localGitOptions: LocalGitExecOptions = {}
|
||||
+): { key: string; cacheKey: string } {
|
||||
+ const key = knownHostsExecutionKey(connectionId, localGitOptions)
|
||||
+ const cacheKey = connectionId ? `connection:${connectionId}` : key
|
||||
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)
|
||||
+ if (cached && cached.key !== key) {
|
||||
+ knownHostsCacheByExecutionContext.delete(cacheKey)
|
||||
+ }
|
||||
+ return { key, cacheKey }
|
||||
+}
|
||||
+
|
||||
/** @internal - exposed for tests only */
|
||||
export function _resetKnownHostsCache(): void {
|
||||
knownHostsCacheByExecutionContext.clear()
|
||||
@@ -103,8 +119,8 @@ export function rememberGlabKnownHosts(
|
||||
connectionId?: string | null,
|
||||
localGitOptions: LocalGitExecOptions = {}
|
||||
): void {
|
||||
- const key = knownHostsExecutionKey(connectionId, localGitOptions)
|
||||
- const cached = knownHostsCacheByExecutionContext.get(key) ?? DEFAULT_GITLAB_HOSTS
|
||||
+ const { key, cacheKey } = knownHostsCacheContext(connectionId, localGitOptions)
|
||||
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)?.hosts ?? DEFAULT_GITLAB_HOSTS
|
||||
const seen = new Set(cached.map(normalizeGitLabHost))
|
||||
const additions: string[] = []
|
||||
for (const host of hosts) {
|
||||
@@ -121,27 +137,29 @@ export function rememberGlabKnownHosts(
|
||||
if (additions.length === 0) {
|
||||
return
|
||||
}
|
||||
- knownHostsCacheByExecutionContext.set(key, [...cached, ...additions])
|
||||
+ knownHostsCacheByExecutionContext.set(cacheKey, { key, hosts: [...cached, ...additions] })
|
||||
}
|
||||
|
||||
export async function getGlabKnownHosts(
|
||||
connectionId?: string | null,
|
||||
localGitOptions: LocalGitExecOptions = {}
|
||||
): Promise<readonly string[]> {
|
||||
- const key = knownHostsExecutionKey(connectionId, localGitOptions)
|
||||
- const cached = knownHostsCacheByExecutionContext.get(key)
|
||||
+ const { key, cacheKey } = knownHostsCacheContext(connectionId, localGitOptions)
|
||||
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)?.hosts
|
||||
if (cached) {
|
||||
return cached
|
||||
}
|
||||
// Why: only join a probe still young enough to answer, so a wedged one cannot
|
||||
// pin every later retry for the life of the process (P1-D).
|
||||
- return runCoalescedProbe(knownHostsInFlightByExecutionContext, key, () =>
|
||||
- probeGlabKnownHosts(key, connectionId, localGitOptions)
|
||||
+ return runCoalescedProbe(knownHostsInFlightByExecutionContext, key, (ownsKey) =>
|
||||
+ probeGlabKnownHosts(key, cacheKey, ownsKey, connectionId, localGitOptions)
|
||||
)
|
||||
}
|
||||
|
||||
async function probeGlabKnownHosts(
|
||||
key: string,
|
||||
+ cacheKey: string,
|
||||
+ ownsKey: () => boolean,
|
||||
connectionId?: string | null,
|
||||
localGitOptions: LocalGitExecOptions = {}
|
||||
): Promise<readonly string[]> {
|
||||
@@ -160,13 +178,17 @@ async function probeGlabKnownHosts(
|
||||
...(localGitOptions.admissionTier ? { admissionTier: localGitOptions.admissionTier } : {})
|
||||
})
|
||||
const hosts = parseGlabAuthStatusHosts(`${stdout}\n${stderr}`)
|
||||
- const remembered = knownHostsCacheByExecutionContext.get(key) ?? []
|
||||
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)
|
||||
+ const remembered = cached?.key === key ? cached.hosts : []
|
||||
const merged = Array.from(new Set([...DEFAULT_GITLAB_HOSTS, ...remembered, ...hosts]))
|
||||
- knownHostsCacheByExecutionContext.set(key, merged)
|
||||
+ if (ownsKey() && knownHostsExecutionKey(connectionId, localGitOptions) === key) {
|
||||
+ knownHostsCacheByExecutionContext.set(cacheKey, { key, hosts: merged })
|
||||
+ }
|
||||
return merged
|
||||
} catch {
|
||||
// Keep failures uncached so auth or tunnel recovery is discovered later.
|
||||
- return knownHostsCacheByExecutionContext.get(key) ?? [...DEFAULT_GITLAB_HOSTS]
|
||||
+ const cached = knownHostsCacheByExecutionContext.get(cacheKey)
|
||||
+ return cached?.key === key ? cached.hosts : [...DEFAULT_GITLAB_HOSTS]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"src/main/gitlab/gitlab-known-host-probe.ts": "4f9651c5a383438aa968aec082c200a3371e2f4317372125aa11aa6938793c25"
|
||||
}
|
||||
@@ -0,0 +1,257 @@
|
||||
const assert = require('node:assert/strict')
|
||||
const fs = require('node:fs')
|
||||
const path = require('node:path')
|
||||
const Module = require('node:module')
|
||||
const esbuild = require('esbuild')
|
||||
const { sourcePath, baseline, fixed, sourceHashes, hash } = require('./sources.cjs')
|
||||
|
||||
assert.equal(process.env.ORCA_BACKGROUND_LAUNCH, '1')
|
||||
assert.equal(typeof global.gc, 'function')
|
||||
const symbol = Symbol.for('orca-known-host-comparison')
|
||||
const context = { generation: 1, calls: 0, runner: null }
|
||||
globalThis[symbol] = context
|
||||
const resultFor = (host) => ({ stdout: `Logged in to ${host} as user`, stderr: '' })
|
||||
|
||||
async function load(phase) {
|
||||
const source = phase === 'baseline' ? baseline : fixed
|
||||
const built = await esbuild.build({
|
||||
entryPoints: [sourcePath],
|
||||
bundle: true,
|
||||
platform: 'node',
|
||||
format: 'cjs',
|
||||
write: false,
|
||||
plugins: [
|
||||
{
|
||||
name: 'actual-cache-with-fixture-ports',
|
||||
setup(build) {
|
||||
build.onLoad({ filter: /gitlab-known-host-probe\.ts$/ }, () => ({
|
||||
contents: source,
|
||||
loader: 'ts'
|
||||
}))
|
||||
build.onResolve({ filter: /\/(runner|ssh-git-dispatch)$/ }, (args) => ({
|
||||
path: path.basename(args.path),
|
||||
namespace: 'ports'
|
||||
}))
|
||||
build.onLoad({ filter: /.*/, namespace: 'ports' }, (args) => ({
|
||||
loader: 'js',
|
||||
contents: `const context=globalThis[Symbol.for('orca-known-host-comparison')];${
|
||||
args.path === 'runner'
|
||||
? `exports.glabExecFileAsync=(...args)=>{context.calls++;return context.runner(...args)};`
|
||||
: `exports.getSshGitProviderGeneration=()=>context.generation;`
|
||||
}`
|
||||
}))
|
||||
}
|
||||
}
|
||||
]
|
||||
})
|
||||
const loaded = new Module(sourcePath, module)
|
||||
loaded.filename = sourcePath
|
||||
loaded.paths = module.paths
|
||||
loaded._compile(built.outputFiles[0].text, sourcePath)
|
||||
return loaded.exports
|
||||
}
|
||||
|
||||
async function collect() {
|
||||
for (let index = 0; index < 6; index++) {
|
||||
await new Promise((resolve) => setImmediate(resolve))
|
||||
global.gc()
|
||||
}
|
||||
await new Promise((resolve) => setImmediate(resolve))
|
||||
}
|
||||
async function rememberResult(api) {
|
||||
const hosts = await api.getGlabKnownHosts('same-connection')
|
||||
assert.deepEqual(hosts, ['gitlab.com', `host${context.generation}.test`])
|
||||
return new WeakRef(hosts)
|
||||
}
|
||||
async function retention(api) {
|
||||
api._resetKnownHostsCache()
|
||||
context.calls = 0
|
||||
context.runner = async () => resultFor(`host${context.generation}.test`)
|
||||
const refs = []
|
||||
for (let generation = 1; generation <= 128; generation++) {
|
||||
context.generation = generation
|
||||
refs.push(await rememberResult(api))
|
||||
}
|
||||
await collect()
|
||||
const retained = refs.filter((ref) => ref.deref() !== undefined).length
|
||||
await rememberResult(api)
|
||||
assert.equal(context.calls, 128)
|
||||
api._resetKnownHostsCache()
|
||||
await collect()
|
||||
const afterReset = refs.filter((ref) => ref.deref() !== undefined).length
|
||||
assert.equal(afterReset, 0)
|
||||
return { retained, afterReset }
|
||||
}
|
||||
async function afterReset(api) {
|
||||
api._resetKnownHostsCache()
|
||||
context.calls = 0
|
||||
const pending = Promise.withResolvers()
|
||||
context.runner = () => pending.promise
|
||||
const old = api.getGlabKnownHosts()
|
||||
api._resetKnownHostsCache()
|
||||
context.runner = async () => resultFor('fresh-after-reset.test')
|
||||
pending.resolve(resultFor('old-before-reset.test'))
|
||||
assert.deepEqual(await old, ['gitlab.com', 'old-before-reset.test'])
|
||||
return { hosts: await api.getGlabKnownHosts(), calls: context.calls }
|
||||
}
|
||||
async function weakResult(promise) {
|
||||
return new WeakRef(await promise)
|
||||
}
|
||||
async function lateGeneration(api) {
|
||||
api._resetKnownHostsCache()
|
||||
context.generation = 1
|
||||
const pending = Promise.withResolvers()
|
||||
context.runner = () => pending.promise
|
||||
let old = api.getGlabKnownHosts('same-connection')
|
||||
context.generation = 2
|
||||
context.runner = async () => resultFor('replacement-generation.test')
|
||||
assert.deepEqual(await api.getGlabKnownHosts('same-connection'), [
|
||||
'gitlab.com',
|
||||
'replacement-generation.test'
|
||||
])
|
||||
pending.resolve(resultFor('retired-generation.test'))
|
||||
const oldResult = await weakResult(old)
|
||||
old = null
|
||||
await collect()
|
||||
const retained = oldResult.deref() !== undefined
|
||||
assert.deepEqual(await api.getGlabKnownHosts('same-connection'), [
|
||||
'gitlab.com',
|
||||
'replacement-generation.test'
|
||||
])
|
||||
return { oldResultRetained: retained, replacementHostsPreserved: true }
|
||||
}
|
||||
async function rememberGeneration(api) {
|
||||
api._resetKnownHostsCache()
|
||||
context.runner = () => {
|
||||
throw new Error('remembered hosts must not probe')
|
||||
}
|
||||
const refs = []
|
||||
for (let generation = 1; generation <= 16; generation++) {
|
||||
context.generation = generation
|
||||
api.rememberGlabKnownHost(`host${generation}.test`, 'same-connection')
|
||||
refs.push(await rememberResult(api))
|
||||
}
|
||||
await collect()
|
||||
return refs.filter((ref) => ref.deref() !== undefined).length
|
||||
}
|
||||
async function abandonedProbe(api) {
|
||||
api._resetKnownHostsCache()
|
||||
const originalNow = Date.now
|
||||
let now = 1000
|
||||
Date.now = () => now
|
||||
try {
|
||||
const pending = Promise.withResolvers()
|
||||
context.runner = () => pending.promise
|
||||
const old = api.getGlabKnownHosts()
|
||||
now += 60_001
|
||||
context.runner = async () => resultFor('replacement.test')
|
||||
assert.deepEqual(await api.getGlabKnownHosts(), ['gitlab.com', 'replacement.test'])
|
||||
pending.resolve(resultFor('abandoned.test'))
|
||||
await old
|
||||
return await api.getGlabKnownHosts()
|
||||
} finally {
|
||||
Date.now = originalNow
|
||||
}
|
||||
}
|
||||
async function rememberWhilePending(api, fail) {
|
||||
api._resetKnownHostsCache()
|
||||
const pending = Promise.withResolvers()
|
||||
context.runner = () => pending.promise
|
||||
const old = api.getGlabKnownHosts()
|
||||
api.rememberGlabKnownHosts(['Remembered.TEST', ' remembered.test '])
|
||||
if (fail) {
|
||||
pending.reject(new Error('controlled auth failure'))
|
||||
} else {
|
||||
pending.resolve(resultFor('gitlab.com'))
|
||||
}
|
||||
assert.deepEqual(await old, ['gitlab.com', 'remembered.test'])
|
||||
assert.deepEqual(await api.getGlabKnownHosts(), ['gitlab.com', 'remembered.test'])
|
||||
}
|
||||
async function scopeIsolation(api) {
|
||||
api._resetKnownHostsCache()
|
||||
const contexts = [
|
||||
[undefined, {}],
|
||||
[undefined, { wslDistro: 'Ubuntu' }],
|
||||
[undefined, { wslDistro: 'Debian' }],
|
||||
['connection-a', {}],
|
||||
['connection-b', {}]
|
||||
]
|
||||
for (let index = 0; index < contexts.length; index++) {
|
||||
context.runner = async () => resultFor(`scope${index}.test`)
|
||||
assert.deepEqual(await api.getGlabKnownHosts(...contexts[index]), [
|
||||
'gitlab.com',
|
||||
`scope${index}.test`
|
||||
])
|
||||
}
|
||||
context.runner = () => {
|
||||
throw new Error('cached contexts must not probe')
|
||||
}
|
||||
for (let index = 0; index < contexts.length; index++) {
|
||||
assert.deepEqual(await api.getGlabKnownHosts(...contexts[index]), [
|
||||
'gitlab.com',
|
||||
`scope${index}.test`
|
||||
])
|
||||
}
|
||||
}
|
||||
async function main() {
|
||||
const proofHashes = Object.fromEntries(
|
||||
['reproduce.cjs', 'sources.cjs', 'fix.patch', 'original-source-hashes.json'].map((file) => [
|
||||
file,
|
||||
hash(fs.readFileSync(path.join(__dirname, file)))
|
||||
])
|
||||
)
|
||||
const report = { sourceHashes, proofHashes, runtime: process.versions, phases: {} }
|
||||
for (const phase of ['baseline', 'fixed']) {
|
||||
const api = await load(phase)
|
||||
const retained = await retention(api)
|
||||
const rememberedGenerationsRetained = await rememberGeneration(api)
|
||||
const oldGeneration = await lateGeneration(api)
|
||||
const reset = await afterReset(api)
|
||||
const abandoned = await abandonedProbe(api)
|
||||
await rememberWhilePending(api, false)
|
||||
await rememberWhilePending(api, true)
|
||||
await scopeIsolation(api)
|
||||
assert.equal(retained.retained, phase === 'baseline' ? 128 : 1)
|
||||
assert.equal(rememberedGenerationsRetained, phase === 'baseline' ? 16 : 1)
|
||||
assert.equal(oldGeneration.oldResultRetained, phase === 'baseline')
|
||||
assert.deepEqual(
|
||||
reset.hosts,
|
||||
phase === 'baseline'
|
||||
? ['gitlab.com', 'old-before-reset.test']
|
||||
: ['gitlab.com', 'fresh-after-reset.test']
|
||||
)
|
||||
assert.deepEqual(
|
||||
abandoned,
|
||||
phase === 'baseline'
|
||||
? ['gitlab.com', 'replacement.test', 'abandoned.test']
|
||||
: ['gitlab.com', 'replacement.test']
|
||||
)
|
||||
report.phases[phase] = {
|
||||
retained,
|
||||
rememberedGenerationsRetained,
|
||||
oldGeneration,
|
||||
reset,
|
||||
abandoned,
|
||||
rememberedSuccessAndFailure: 'passed',
|
||||
nativeWslConnectionIsolation: 'passed'
|
||||
}
|
||||
api._resetKnownHostsCache()
|
||||
}
|
||||
fs.writeFileSync(
|
||||
process.argv[2] || path.join(__dirname, 'results.json'),
|
||||
`${JSON.stringify(report, null, 2)}\n`
|
||||
)
|
||||
console.log(JSON.stringify(report.phases, null, 2))
|
||||
}
|
||||
main()
|
||||
.catch((error) => {
|
||||
console.error(error)
|
||||
process.exitCode = 1
|
||||
})
|
||||
.finally(() => {
|
||||
delete globalThis[symbol]
|
||||
})
|
||||
setTimeout(() => {
|
||||
console.error('fixture deadline')
|
||||
process.exit(2)
|
||||
}, 10000).unref()
|
||||
@@ -0,0 +1,88 @@
|
||||
{
|
||||
"sourceHashes": {
|
||||
"src/main/gitlab/gitlab-known-host-probe.ts": {
|
||||
"baseline": "4f9651c5a383438aa968aec082c200a3371e2f4317372125aa11aa6938793c25",
|
||||
"fixed": "ecf0d67f68cf4b7b1bc7d6ff19a1815a8b95fc9721962d3cd873702f98831bad"
|
||||
},
|
||||
"src/main/git/coalesced-probe.ts": "e5a13820a7d8b5f501a3804961ea26526bd6ad9144ecf597d5d83a70d81885e1",
|
||||
"src/main/git/remote-ref-probe-cache.ts": "d5cbfd97b30e72b03c0d28d8b9e75a2ae1f9e3efc9f5b5570c78e0742c0582dd",
|
||||
"src/main/gitlab/project-ref-parser.ts": "0f8b6758e6a162a58f436ca4213addc42bf6ceb3fcc2e63e8c7402c844af9303",
|
||||
"src/main/gitlab/gitlab-known-host-retirement.test.ts": "c6dedbac0462cae22903805d0a89be397a2b9d122a29640b6a07a2d274f8e98e"
|
||||
},
|
||||
"proofHashes": {
|
||||
"reproduce.cjs": "19f49406685923b050a99fbc92b02b9f6f5f8d8f308dcc14336415a0988679cc",
|
||||
"sources.cjs": "2ee8ac0f295d65f16489da2b0c03800b9db4e88c83ecfb63ae7be1ec3ea6c148",
|
||||
"fix.patch": "695c929087006b3406ce2c30a7ab783d88af3f9675f4ea320dc7f98b8476be9f",
|
||||
"original-source-hashes.json": "f9770d9f0a87afc9231eef6af99e8ded33348fdc21ba46a70b1d5d3388874b97"
|
||||
},
|
||||
"runtime": {
|
||||
"node": "26.6.0",
|
||||
"acorn": "8.17.0",
|
||||
"ada": "4.0.0",
|
||||
"amaro": "1.1.11",
|
||||
"ares": "1.34.8",
|
||||
"brotli": "1.2.0",
|
||||
"cldr": "48.0",
|
||||
"icu": "78.3",
|
||||
"libffi": "3.7.1",
|
||||
"llhttp": "9.4.3",
|
||||
"merve": "1.2.2",
|
||||
"modules": "147",
|
||||
"napi": "10",
|
||||
"nbytes": "0.1.4",
|
||||
"ncrypto": "0.0.1",
|
||||
"nghttp2": "1.70.0",
|
||||
"nghttp3": "",
|
||||
"ngtcp2": "",
|
||||
"openssl": "3.6.3",
|
||||
"simdjson": "4.6.6",
|
||||
"simdutf": "7.7.0",
|
||||
"sqlite": "3.53.4",
|
||||
"tz": "2026a",
|
||||
"undici": "8.9.0",
|
||||
"unicode": "17.0",
|
||||
"uv": "1.52.1",
|
||||
"uvwasi": "0.0.23",
|
||||
"v8": "14.6.202.34-node.26",
|
||||
"zlib": "1.2.12",
|
||||
"zstd": "1.5.7"
|
||||
},
|
||||
"phases": {
|
||||
"baseline": {
|
||||
"retained": {
|
||||
"retained": 128,
|
||||
"afterReset": 0
|
||||
},
|
||||
"rememberedGenerationsRetained": 16,
|
||||
"oldGeneration": {
|
||||
"oldResultRetained": true,
|
||||
"replacementHostsPreserved": true
|
||||
},
|
||||
"reset": {
|
||||
"hosts": ["gitlab.com", "old-before-reset.test"],
|
||||
"calls": 1
|
||||
},
|
||||
"abandoned": ["gitlab.com", "replacement.test", "abandoned.test"],
|
||||
"rememberedSuccessAndFailure": "passed",
|
||||
"nativeWslConnectionIsolation": "passed"
|
||||
},
|
||||
"fixed": {
|
||||
"retained": {
|
||||
"retained": 1,
|
||||
"afterReset": 0
|
||||
},
|
||||
"rememberedGenerationsRetained": 1,
|
||||
"oldGeneration": {
|
||||
"oldResultRetained": false,
|
||||
"replacementHostsPreserved": true
|
||||
},
|
||||
"reset": {
|
||||
"hosts": ["gitlab.com", "fresh-after-reset.test"],
|
||||
"calls": 2
|
||||
},
|
||||
"abandoned": ["gitlab.com", "replacement.test"],
|
||||
"rememberedSuccessAndFailure": "passed",
|
||||
"nativeWslConnectionIsolation": "passed"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
const assert = require('node:assert/strict')
|
||||
const fs = require('node:fs')
|
||||
const path = require('node:path')
|
||||
const crypto = require('node:crypto')
|
||||
const { parsePatch, reversePatch, applyPatch } = require('diff')
|
||||
|
||||
const root = path.resolve(__dirname, '../../..')
|
||||
const relativePath = 'src/main/gitlab/gitlab-known-host-probe.ts'
|
||||
const sourcePath = path.join(root, relativePath)
|
||||
const fixed = fs.readFileSync(sourcePath, 'utf8')
|
||||
const patches = parsePatch(fs.readFileSync(path.join(__dirname, 'fix.patch'), 'utf8'))
|
||||
assert.equal(patches.length, 1)
|
||||
assert.equal(patches[0].newFileName, `b/${relativePath}`)
|
||||
const baseline = applyPatch(fixed, reversePatch(patches[0]))
|
||||
assert.notEqual(baseline, false, 'Current source must reverse exactly to the original cache')
|
||||
const hash = (value) => crypto.createHash('sha256').update(value).digest('hex')
|
||||
assert.equal(hash(baseline), require('./original-source-hashes.json')[relativePath])
|
||||
const sourceHashes = {
|
||||
[relativePath]: { baseline: hash(baseline), fixed: hash(fixed) },
|
||||
...Object.fromEntries(
|
||||
[
|
||||
'src/main/git/coalesced-probe.ts',
|
||||
'src/main/git/remote-ref-probe-cache.ts',
|
||||
'src/main/gitlab/project-ref-parser.ts',
|
||||
'src/main/gitlab/gitlab-known-host-retirement.test.ts'
|
||||
].map((file) => [file, hash(fs.readFileSync(path.join(root, file)))])
|
||||
)
|
||||
}
|
||||
|
||||
module.exports = { root, sourcePath, baseline, fixed, sourceHashes, hash }
|
||||
@@ -12,7 +12,10 @@ export type LocalGitExecOptions = {
|
||||
|
||||
const GLAB_KNOWN_HOSTS_TIMEOUT_MS = 10_000
|
||||
const UNAUTHENTICATED_HOSTS_MAX_ENTRIES = 128
|
||||
const knownHostsCacheByExecutionContext = new Map<string, readonly string[]>()
|
||||
const knownHostsCacheByExecutionContext = new Map<
|
||||
string,
|
||||
{ key: string; hosts: readonly string[] }
|
||||
>()
|
||||
const knownHostsInFlightByExecutionContext: CoalescedProbes<readonly string[]> = new Map()
|
||||
const unauthenticatedHostExpiries = new Map<string, number>()
|
||||
|
||||
@@ -27,6 +30,19 @@ function knownHostsExecutionKey(
|
||||
return localGitOptions.wslDistro ? `wsl:${localGitOptions.wslDistro}` : 'native'
|
||||
}
|
||||
|
||||
function knownHostsCacheContext(
|
||||
connectionId?: string | null,
|
||||
localGitOptions: LocalGitExecOptions = {}
|
||||
): { key: string; cacheKey: string } {
|
||||
const key = knownHostsExecutionKey(connectionId, localGitOptions)
|
||||
const cacheKey = connectionId ? `connection:${connectionId}` : key
|
||||
const cached = knownHostsCacheByExecutionContext.get(cacheKey)
|
||||
if (cached && cached.key !== key) {
|
||||
knownHostsCacheByExecutionContext.delete(cacheKey)
|
||||
}
|
||||
return { key, cacheKey }
|
||||
}
|
||||
|
||||
/** @internal - exposed for tests only */
|
||||
export function _resetKnownHostsCache(): void {
|
||||
knownHostsCacheByExecutionContext.clear()
|
||||
@@ -103,8 +119,8 @@ export function rememberGlabKnownHosts(
|
||||
connectionId?: string | null,
|
||||
localGitOptions: LocalGitExecOptions = {}
|
||||
): void {
|
||||
const key = knownHostsExecutionKey(connectionId, localGitOptions)
|
||||
const cached = knownHostsCacheByExecutionContext.get(key) ?? DEFAULT_GITLAB_HOSTS
|
||||
const { key, cacheKey } = knownHostsCacheContext(connectionId, localGitOptions)
|
||||
const cached = knownHostsCacheByExecutionContext.get(cacheKey)?.hosts ?? DEFAULT_GITLAB_HOSTS
|
||||
const seen = new Set(cached.map(normalizeGitLabHost))
|
||||
const additions: string[] = []
|
||||
for (const host of hosts) {
|
||||
@@ -121,27 +137,29 @@ export function rememberGlabKnownHosts(
|
||||
if (additions.length === 0) {
|
||||
return
|
||||
}
|
||||
knownHostsCacheByExecutionContext.set(key, [...cached, ...additions])
|
||||
knownHostsCacheByExecutionContext.set(cacheKey, { key, hosts: [...cached, ...additions] })
|
||||
}
|
||||
|
||||
export async function getGlabKnownHosts(
|
||||
connectionId?: string | null,
|
||||
localGitOptions: LocalGitExecOptions = {}
|
||||
): Promise<readonly string[]> {
|
||||
const key = knownHostsExecutionKey(connectionId, localGitOptions)
|
||||
const cached = knownHostsCacheByExecutionContext.get(key)
|
||||
const { key, cacheKey } = knownHostsCacheContext(connectionId, localGitOptions)
|
||||
const cached = knownHostsCacheByExecutionContext.get(cacheKey)?.hosts
|
||||
if (cached) {
|
||||
return cached
|
||||
}
|
||||
// Why: only join a probe still young enough to answer, so a wedged one cannot
|
||||
// pin every later retry for the life of the process (P1-D).
|
||||
return runCoalescedProbe(knownHostsInFlightByExecutionContext, key, () =>
|
||||
probeGlabKnownHosts(key, connectionId, localGitOptions)
|
||||
return runCoalescedProbe(knownHostsInFlightByExecutionContext, key, (ownsKey) =>
|
||||
probeGlabKnownHosts(key, cacheKey, ownsKey, connectionId, localGitOptions)
|
||||
)
|
||||
}
|
||||
|
||||
async function probeGlabKnownHosts(
|
||||
key: string,
|
||||
cacheKey: string,
|
||||
ownsKey: () => boolean,
|
||||
connectionId?: string | null,
|
||||
localGitOptions: LocalGitExecOptions = {}
|
||||
): Promise<readonly string[]> {
|
||||
@@ -160,13 +178,17 @@ async function probeGlabKnownHosts(
|
||||
...(localGitOptions.admissionTier ? { admissionTier: localGitOptions.admissionTier } : {})
|
||||
})
|
||||
const hosts = parseGlabAuthStatusHosts(`${stdout}\n${stderr}`)
|
||||
const remembered = knownHostsCacheByExecutionContext.get(key) ?? []
|
||||
const cached = knownHostsCacheByExecutionContext.get(cacheKey)
|
||||
const remembered = cached?.key === key ? cached.hosts : []
|
||||
const merged = Array.from(new Set([...DEFAULT_GITLAB_HOSTS, ...remembered, ...hosts]))
|
||||
knownHostsCacheByExecutionContext.set(key, merged)
|
||||
if (ownsKey() && knownHostsExecutionKey(connectionId, localGitOptions) === key) {
|
||||
knownHostsCacheByExecutionContext.set(cacheKey, { key, hosts: merged })
|
||||
}
|
||||
return merged
|
||||
} catch {
|
||||
// Keep failures uncached so auth or tunnel recovery is discovered later.
|
||||
return knownHostsCacheByExecutionContext.get(key) ?? [...DEFAULT_GITLAB_HOSTS]
|
||||
const cached = knownHostsCacheByExecutionContext.get(cacheKey)
|
||||
return cached?.key === key ? cached.hosts : [...DEFAULT_GITLAB_HOSTS]
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
|
||||
|
||||
const { execute, generations } = vi.hoisted(() => ({
|
||||
execute: vi.fn(),
|
||||
generations: new Map<string, number>()
|
||||
}))
|
||||
vi.mock('../git/runner', () => ({ glabExecFileAsync: execute }))
|
||||
vi.mock('../providers/ssh-git-dispatch', () => ({
|
||||
getSshGitProviderGeneration: (connectionId: string) => generations.get(connectionId) ?? 0
|
||||
}))
|
||||
|
||||
import {
|
||||
_resetKnownHostsCache,
|
||||
getGlabKnownHosts,
|
||||
rememberGlabKnownHost
|
||||
} from './gitlab-known-host-probe'
|
||||
import { PROBE_COALESCE_STALE_MS } from '../git/coalesced-probe'
|
||||
|
||||
const response = (host: string) => ({ stdout: `Logged in to ${host} as user`, stderr: '' })
|
||||
const deferred = () => Promise.withResolvers<ReturnType<typeof response>>()
|
||||
|
||||
async function collect(): Promise<void> {
|
||||
if (typeof globalThis.gc !== 'function') {
|
||||
throw new Error('Run with the repository Vitest --expose-gc config')
|
||||
}
|
||||
for (let index = 0; index < 6; index++) {
|
||||
await new Promise<void>((resolve) => setImmediate(resolve))
|
||||
globalThis.gc()
|
||||
}
|
||||
await new Promise<void>((resolve) => setImmediate(resolve))
|
||||
}
|
||||
|
||||
async function weakResult(connectionId: string): Promise<WeakRef<readonly string[]>> {
|
||||
return new WeakRef(await getGlabKnownHosts(connectionId))
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
_resetKnownHostsCache()
|
||||
generations.clear()
|
||||
execute.mockReset()
|
||||
})
|
||||
afterEach(() => {
|
||||
_resetKnownHostsCache()
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
it('releases successful host arrays from superseded SSH generations', async () => {
|
||||
const results: WeakRef<readonly string[]>[] = []
|
||||
for (let generation = 1; generation <= 32; generation++) {
|
||||
generations.set('connection', generation)
|
||||
execute.mockResolvedValue(response(`host${generation}.test`))
|
||||
results.push(await weakResult('connection'))
|
||||
}
|
||||
await collect()
|
||||
expect(results.filter((result) => result.deref() !== undefined)).toHaveLength(1)
|
||||
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'host32.test'])
|
||||
expect(execute).toHaveBeenCalledTimes(32)
|
||||
})
|
||||
|
||||
it('retires remembered generations without requiring an auth-status probe', async () => {
|
||||
const results: WeakRef<readonly string[]>[] = []
|
||||
for (let generation = 1; generation <= 16; generation++) {
|
||||
generations.set('connection', generation)
|
||||
rememberGlabKnownHost(`host${generation}.test`, 'connection')
|
||||
results.push(await weakResult('connection'))
|
||||
}
|
||||
await collect()
|
||||
expect(results.filter((result) => result.deref() !== undefined)).toHaveLength(1)
|
||||
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'host16.test'])
|
||||
expect(execute).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('does not retain a delayed old-generation result after a replacement answers', async () => {
|
||||
const old = deferred()
|
||||
generations.set('connection', 1)
|
||||
execute.mockReturnValueOnce(old.promise)
|
||||
const oldResult = weakResult('connection')
|
||||
generations.set('connection', 2)
|
||||
execute.mockResolvedValueOnce(response('replacement.test'))
|
||||
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'replacement.test'])
|
||||
old.resolve(response('retired.test'))
|
||||
const reference = await oldResult
|
||||
await collect()
|
||||
expect(reference.deref()).toBeUndefined()
|
||||
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'replacement.test'])
|
||||
expect(execute).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('does not repopulate an explicitly reset cache from an earlier probe', async () => {
|
||||
const old = deferred()
|
||||
execute.mockReturnValueOnce(old.promise)
|
||||
const oldResult = getGlabKnownHosts()
|
||||
_resetKnownHostsCache()
|
||||
old.resolve(response('before-reset.test'))
|
||||
await expect(oldResult).resolves.toEqual(['gitlab.com', 'before-reset.test'])
|
||||
execute.mockResolvedValueOnce(response('after-reset.test'))
|
||||
await expect(getGlabKnownHosts()).resolves.toEqual(['gitlab.com', 'after-reset.test'])
|
||||
expect(execute).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('keeps a post-reset successor joinable when the old probe settles first', async () => {
|
||||
const old = deferred()
|
||||
const next = deferred()
|
||||
execute.mockReturnValueOnce(old.promise).mockReturnValueOnce(next.promise)
|
||||
const oldResult = getGlabKnownHosts()
|
||||
_resetKnownHostsCache()
|
||||
const nextResult = getGlabKnownHosts()
|
||||
old.resolve(response('before-reset.test'))
|
||||
await oldResult
|
||||
let joinedSettled = false
|
||||
const joined = getGlabKnownHosts().then((hosts) => {
|
||||
joinedSettled = true
|
||||
return hosts
|
||||
})
|
||||
await Promise.resolve()
|
||||
expect(joinedSettled).toBe(false)
|
||||
next.resolve(response('after-reset.test'))
|
||||
await expect(nextResult).resolves.toEqual(['gitlab.com', 'after-reset.test'])
|
||||
await expect(joined).resolves.toEqual(['gitlab.com', 'after-reset.test'])
|
||||
expect(execute).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('prevents an abandoned same-generation probe from publishing over its successor', async () => {
|
||||
const clock = vi.spyOn(Date, 'now').mockReturnValue(1000)
|
||||
const old = deferred()
|
||||
execute.mockReturnValueOnce(old.promise)
|
||||
const oldResult = getGlabKnownHosts('connection')
|
||||
clock.mockReturnValue(1000 + PROBE_COALESCE_STALE_MS + 1)
|
||||
execute.mockResolvedValueOnce(response('replacement.test'))
|
||||
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'replacement.test'])
|
||||
old.resolve(response('abandoned.test'))
|
||||
await oldResult
|
||||
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'replacement.test'])
|
||||
expect(execute).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('keeps native, WSL and other connection caches when one generation changes', async () => {
|
||||
execute
|
||||
.mockResolvedValueOnce(response('native.test'))
|
||||
.mockResolvedValueOnce(response('ubuntu.test'))
|
||||
.mockResolvedValueOnce(response('debian.test'))
|
||||
.mockResolvedValueOnce(response('connection-a.test'))
|
||||
.mockResolvedValueOnce(response('connection-b.test'))
|
||||
const native = await getGlabKnownHosts()
|
||||
const ubuntu = await getGlabKnownHosts(undefined, { wslDistro: 'Ubuntu' })
|
||||
const debian = await getGlabKnownHosts(undefined, { wslDistro: 'Debian' })
|
||||
await getGlabKnownHosts('connection-a')
|
||||
const other = await getGlabKnownHosts('connection-b')
|
||||
generations.set('connection-a', 1)
|
||||
rememberGlabKnownHost('replacement.test', 'connection-a')
|
||||
await expect(getGlabKnownHosts('connection-a')).resolves.toEqual([
|
||||
'gitlab.com',
|
||||
'replacement.test'
|
||||
])
|
||||
await expect(getGlabKnownHosts()).resolves.toBe(native)
|
||||
await expect(getGlabKnownHosts(undefined, { wslDistro: 'Ubuntu' })).resolves.toBe(ubuntu)
|
||||
await expect(getGlabKnownHosts(undefined, { wslDistro: 'Debian' })).resolves.toBe(debian)
|
||||
await expect(getGlabKnownHosts('connection-b')).resolves.toBe(other)
|
||||
expect(execute).toHaveBeenCalledTimes(5)
|
||||
})
|
||||
|
||||
it('does not serve a retired generation after the current probe fails', async () => {
|
||||
execute.mockResolvedValueOnce(response('retired.test'))
|
||||
await getGlabKnownHosts('connection')
|
||||
generations.set('connection', 1)
|
||||
execute.mockRejectedValueOnce(new Error('current host unavailable'))
|
||||
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com'])
|
||||
execute.mockResolvedValueOnce(response('current.test'))
|
||||
await expect(getGlabKnownHosts('connection')).resolves.toEqual(['gitlab.com', 'current.test'])
|
||||
expect(execute).toHaveBeenCalledTimes(3)
|
||||
})
|
||||
Reference in New Issue
Block a user