fix(daemon): ask the tolerant question while the clock can still fund the answer

The launcher asked the cheap question first and the patient one last. That is
backwards. This path is only reached because a 3s health check timed out, so
every 4s probe re-asks on a stricter budget than the one that triaged the daemon
here — it can only ever agree. The one ask that could disagree ran last, by which
point the clock could fund its handshake but not its answer, and a daemon that
answered in 12s was read as dead and replaced along with its agents.

Three changes, one idea: never make an ask you cannot afford to hear out.

- The patient ask goes first, with every millisecond the answer does not need.
- Cheap retries follow it, and stop once the clock cannot fund both halves.
  Funded to knock but not to listen is not an ask.
- The adoption connect is capped. It is not a classification step — it acquires
  a lease preserveDaemon() re-establishes anyway — but on a daemon that accepts
  the socket and never completes hello it would spend the entire classification
  budget, leaving nothing for the probes that protect that daemon's sessions.

The test double now forwards the connect budget instead of dropping it, so what
the launcher was willing to wait for is observable at all.
This commit is contained in:
Neil
2026-08-12 13:45:17 -07:00
parent b4a4b13c5a
commit 7256ea2876
2 changed files with 28 additions and 29 deletions
+9 -7
View File
@@ -315,13 +315,15 @@ vi.mock('./daemon-health', () => ({
vi.mock('./client', () => ({
// Mirror ensureConnected onto the bounded variant so every existing double keeps its
// behaviour — including the ones whose whole point is that connecting throws.
// behaviour — including the ones whose whole point is that connecting throws. The budget is
// forwarded rather than dropped so a test can see what the launcher was willing to wait.
DaemonClient: function DaemonClientDouble(...args: unknown[]) {
const instance = (daemonClientMock as unknown as (...a: unknown[]) => Record<string, unknown>)(
...args
)
if (instance && typeof instance === 'object' && !('ensureConnectedWithin' in instance)) {
instance.ensureConnectedWithin = instance.ensureConnected
instance.ensureConnectedWithin = (budgetMs?: number) =>
(instance.ensureConnected as (ms?: number) => unknown)(budgetMs)
}
return instance
}
@@ -3195,11 +3197,11 @@ describe('daemon-init: runRestartDaemon (7-step sequence)', () => {
'/fake/token'
)
expect(forkMock).toHaveBeenCalled()
// 1 adoption + 1 initial probe + the retries + 1 patient probe + 1 post-fork adoption.
// The patient probe is the point: retrying a four-second handshake never reaches a
// daemon that consistently needs longer, so the launcher spends what is left of the
// clock on a single tolerant ask before it is willing to replace anything.
expect(daemonClientMock).toHaveBeenCalledTimes(4 + WEDGED_DAEMON_GRACE_RETRIES)
// 1 adoption + 1 patient probe + the cheap retries + 1 post-fork adoption. The patient
// probe comes first and is the point: retrying a four-second handshake never reaches a
// daemon that consistently needs longer, and spending the clock on the cheap asks first
// left the tolerant one unable to fund an answer.
expect(daemonClientMock).toHaveBeenCalledTimes(3 + WEDGED_DAEMON_GRACE_RETRIES)
// Why: this replace path used to kill the daemon with no log, so a post-hoc
// reader could not tell it apart from an adoption; the verdict must be recorded.
expect(warnSpy).toHaveBeenCalledWith(
+19 -22
View File
@@ -510,7 +510,12 @@ function createOutOfProcessLauncher(
const classificationRemainingMs = (): number => Math.max(0, classificationDeadline - Date.now())
try {
// Why: acquire the full pair before control-only probes so an expired inherited deadline can't fire in the probe-to-adoption gap.
await adoptionClient.ensureConnectedWithin(classificationRemainingMs())
// Capped: this only acquires a lease that preserveDaemon() re-establishes anyway, and a
// daemon that accepts the socket but never answers hello would otherwise spend the whole
// classification clock here, leaving nothing for the probes that protect its sessions.
await adoptionClient.ensureConnectedWithin(
Math.min(OCCUPANCY_CONNECT_BUDGET_MS, classificationRemainingMs())
)
await reconcileDaemonPidOwnership(adoptionClient, pidPath)
} catch {
adoptionClient.disconnect()
@@ -641,12 +646,19 @@ function createOutOfProcessLauncher(
budgetMs: probeBudgetMs(),
connectBudgetMs
})
// Why grace at all: a wedged-but-connectable daemon (a Windows update relaunch) may
// still own live sessions and simply need a moment. A permanent wedge (#8689) spends
// the window, and 'rejected' skips it — a refused handshake is never going to be
// adopted. Retries stop when the clock can no longer fund a handshake and an answer,
// so the count is a ceiling rather than the budget.
let occupancy = await askDaemonWhatItHosts(OCCUPANCY_CONNECT_BUDGET_MS)
// Why the tolerant question goes first: this path is only reached because a 3s health
// check timed out, so a cheap probe re-asks on a stricter budget than the one that
// triaged the daemon here and can only ever agree with it. The patient ask is the only
// one that can disagree, and asking it last meant asking it with the clock already
// spent. It gets every millisecond the answer itself does not still need, and never
// less than the cheap ask would have had.
const patientConnectBudgetMs = (): number =>
Math.max(OCCUPANCY_CONNECT_BUDGET_MS, probeBudgetMs() - OCCUPANCY_REQUEST_BUDGET_MS)
let occupancy = await askDaemonWhatItHosts(patientConnectBudgetMs())
// Why retry after it: a daemon that recovers on its own answers a cheap question, and
// recovering is a different failure from being slow. The patient ask cannot see it,
// because it asked before the recovery. Retries stop once the clock can no longer fund
// both halves of an ask — funded to knock but not to listen is no ask at all.
let graceRetry = 0
while (
occupancy.state === 'unknown' &&
@@ -658,21 +670,6 @@ function createOutOfProcessLauncher(
occupancy = await askDaemonWhatItHosts(OCCUPANCY_CONNECT_BUDGET_MS)
graceRetry++
}
// Why one patient ask after the impatient ones: retrying a four-second handshake never
// reaches a daemon that consistently needs longer, and this path is only reached
// because a three-second health check already timed out — so every probe so far has
// judged it on a stricter budget than the one that triaged it here. Spending what is
// left on a single tolerant attempt is the only question that can disagree.
if (
occupancy.state === 'unknown' &&
health !== 'rejected' &&
probeBudgetMs() > 0 &&
// Nothing to be patient with when the endpoint is provably gone — a cold start
// reaches here too, and has no daemon to wait for.
!endpointIsProvenDead(await probeSocketConnect(socketPath))
) {
occupancy = await askDaemonWhatItHosts(probeBudgetMs())
}
// The evidence that separates a wedged daemon still hosting terminals from one with
// nothing left to lose (#8689). Read once IPC has had its full chance, and only when
// it never answered — both because it costs a process scan, and because identity has