Files
orca/src/main/daemon/daemon-pid-record-quarantine.ts
T
Brennan BensonandBrennan Benson 7abdf037d6 Fix Windows daemon host pruning on unverifiable liveness (#16908)
* Fix Windows daemon host prune liveness contract

* Scope host prune evidence per version

* Drop redundant default cases from exhaustive liveness switches

All three switches consume ProcessLivenessVerdict/ProcessSignalEvidence values
constructed in-process by inspectProcessSignal/inspectProcessLiveness; the union
is never deserialized from a wire, RPC, or persisted record, so the defaults are
genuinely unreachable.

* Cover prune liveness gates and quarantine corrupt pid records

* Make the prune delete-gate fail safe and refuse truncated pid salvage

The prune switch shared #16900's delete-gate shape: an unhandled future
verdict status fell through into rmSync, protected only by the lint
exhaustiveness rule. Deletion is now opted into by a positively matched
'exited' via reclaimUnownedDaemonHostDir; a pinning test feeds an
out-of-contract verdict and asserts the host dir survives.

Pid salvage from corrupt records now requires the digit run to be
terminated by a following non-digit byte. A tear inside the digits leaves
a truncated prefix that is a different pid: probing it either quarantined
a record on an unrelated process's death or, when the prefix collided
with an immortal pid (Windows System pid 4), re-created the permanent
prune veto for that record. Unterminated digits mean the writer died
mid-write, so the record quarantines without consulting any probe.

* fix(daemon): stop an in-flight pid publish from being read as a dead version

publishDaemonPidFile creates the record before writing it (writeFileSync with
flag 'wx'), so a concurrent launch can read a live daemon's record as empty. A
two-process probe observed the empty window on 7 of 2273 reads.

An empty record was not treated as corrupt at all: the parser's legacy
bare-integer fallback coerces it to pid 0 (Number('') === 0) with appVersion
null, so the scan skipped it as a pre-relocation daemon, left its version
unpinned, and the prune reclaimed a running daemon's host image -- the exact
destructive outcome this change exists to prevent, reached without any
'unverifiable' verdict. A pid that is not a positive integer names no process
(process.kill(0, 0) probes the caller's own process group), so it is now a
veto rather than a skip.

Quarantine additionally refuses any record written in the last minute: an
in-flight publish is by definition fresh, while a record left corrupt by a
dead writer ages past the floor and is quarantined on a later launch. Fixed
locally rather than in parseDaemonPidFile, whose null result also drives an
unlink in daemon-stale-kill.

Each gate is pinned by a test that fails individually when it is reverted.

---------

Co-authored-by: Brennan Benson <brennanb2025@users.noreply.github.com>
2026-08-28 15:54:21 -07:00

62 lines
2.9 KiB
TypeScript

import { renameSync, statSync } from 'node:fs'
import { join } from 'node:path'
import { salvagePidFromCorruptDaemonRecord } from './daemon-pid-file-parse'
import { inspectProcessLiveness } from './daemon-process-inspection'
/**
* A record that was just written is never quarantined: publishDaemonPidFile creates the record
* before writing it (writeFileSync with flag 'wx'), so a concurrent launch can read a LIVE
* daemon's record as empty or torn. Renaming it aside would strand that daemon's record, and the
* next launch — seeing a complete listing with no record for its version — would reclaim the
* running daemon's host image. An in-flight publish is by definition fresh; a record left corrupt
* by a dead writer ages past this floor and is quarantined on a later launch.
*/
const QUARANTINE_MIN_RECORD_AGE_MS = 60_000
/**
* A pid record that parses to nothing would otherwise veto daemon-host pruning on every future
* launch: nothing ever rewrites a retired protocol version's pid file, so the veto never expires.
* Quarantine the record (rename in place, bytes kept for diagnosis) so the next launch scans a
* complete listing again — unless a process still answers for a pid salvaged from the corrupt
* bytes, in which case the record may belong to a live daemon and keeps its conservative veto
* until that pid exits. Returns the unverifiable reason; every branch is logged because this
* state suppresses pruning.
*/
export function quarantineCorruptDaemonPidRecord(
runtimeDir: string,
name: string,
contents: string
): string {
const salvagedPid = salvagePidFromCorruptDaemonRecord(contents)
if (salvagedPid !== null && inspectProcessLiveness(salvagedPid).status !== 'exited') {
const reason = `the daemon pid file could not be parsed and salvaged pid ${salvagedPid} may still be running: ${name}`
console.warn(`[daemon] Keeping corrupt daemon pid record: ${reason}`)
return reason
}
const recordPath = join(runtimeDir, name)
let modifiedAtMs: number
try {
modifiedAtMs = statSync(recordPath).mtimeMs
} catch {
const reason = `the daemon pid file could not be parsed or aged: ${name}`
console.warn(`[daemon] ${reason}`)
return reason
}
// A future mtime (clock adjustment) reads as negative age and is treated as fresh.
if (Date.now() - modifiedAtMs < QUARANTINE_MIN_RECORD_AGE_MS) {
const reason = `the daemon pid file could not be parsed and was written too recently to quarantine: ${name}`
console.warn(`[daemon] Keeping corrupt daemon pid record: ${reason}`)
return reason
}
try {
renameSync(recordPath, join(runtimeDir, `${name}.corrupt`))
} catch {
const reason = `the daemon pid file could not be parsed or quarantined: ${name}`
console.warn(`[daemon] ${reason}`)
return reason
}
const reason = `the daemon pid file could not be parsed and was quarantined: ${name}`
console.warn(`[daemon] ${reason}`)
return reason
}