fix(relay): bound the live-broker wait over a chain of superseding reconciles

The budget bounded the armed-retry chain only. A waiter that kept following
fresh reconciles — which the previous two commits made it correctly do — rode
that chain with no deadline check at all.

MEASURED, on a fake clock: budget 1,000 ms, reconciles landing every 200 ms,
opens taking 400 ms and rejecting. Pre-fix the waiter NEVER SETTLED within a
300,000 ms horizon. Post-fix it settles at exactly 1,000 ms, the budget the
caller asked for.

Reachable in production because `withTransientDemand` reconciles on BOTH acquire
and release, so a busy host supersedes a waiter's reconcile faster than opens
settle, and the caller rides the chain indefinitely while holding its demand ref.

The deadline check is gated on `joinedReconcile` so the contract that matters is
preserved: the one open the waiter ARRIVED on is still never cut short, because
cutting a slow-but-succeeding open short would fail a pairing that was about to
work. The existing "slow first open past the budget still wins" test passes
unchanged, which is the assertion that proves the gate.

This is the third defect in this loop and all three share one root cause:
`await pending` had no escape once the reconcile the waiter captured stopped
being the one that mattered.
This commit is contained in:
Neil
2026-09-10 17:28:00 -07:00
parent 6cfc1f2a4a
commit 1673716c6d
2 changed files with 32 additions and 0 deletions
@@ -229,4 +229,28 @@ describe('relay live-broker wait under interleaving', () => {
expect(openBroker).toHaveBeenCalledTimes(1)
coordinator.stop()
})
it('keeps the budget over a chain of superseding reconciles, not just over armed retries', async () => {
// Every transient-demand acquire and release reconciles, so a busy host can
// supersede a waiter's reconcile faster than opens settle. The budget has to
// survive that or a caller rides the chain indefinitely holding its demand ref.
vi.useFakeTimers()
const coordinator = new RelayAuthCoordinator({
readContext: async () => context,
openBroker: () =>
new Promise<CoordinatedRelayBroker>((_resolve, reject) =>
setTimeout(() => reject(new Error('slow open')), 400)
),
onStatus: vi.fn(),
random: () => 0.5
})
coordinator.reconcile()
const seen = observe(coordinator.waitForLiveBrokerResult(1_000))
const churn = setInterval(() => coordinator.reconcile(), 200)
await vi.advanceTimersByTimeAsync(2_000)
clearInterval(churn)
expect(seen()).not.toBe('unsettled')
coordinator.stop()
})
})
@@ -30,13 +30,21 @@ export async function runLiveBrokerWait(
budgetMs: number
): Promise<LiveBrokerWaitResult> {
const deadline = Date.now() + budgetMs
let joinedReconcile = false
while (!source.stopped()) {
const broker = source.liveBroker()
if (broker) {
return { broker }
}
// Why the budget applies only once a reconcile has been joined: the one open
// the waiter arrived on is never cut short, but a chain of superseding opens
// must not outlive the budget the caller asked for.
if (joinedReconcile && Date.now() >= deadline) {
return settledResult(source)
}
const pending = source.reconcile()
const superseded = source.authorityChange()
joinedReconcile = true
// Why unbounded on `pending`: a reconcile always settles (opens carry HTTP
// deadlines), and cutting a slow-but-succeeding open short would fail a
// pairing that was about to work.