Files
archived-orca/cloud/apps/sandbox-control/test/settled-cache.test.ts
T
Neil fc5c502bb5 fix(cloud): WORKER-XREQ-PROMISE module caches hold settled values, never another request's in-flight promise
Egress proxy secret routes (custom-base-URL keys, OPENAI_BASE_URL) hung until the client gave up
(dev, proxy mode, 2026-11-15). The Worker was invoked (tail: `canceled`, wall 65 s, CPU 3 ms, no log
line) and was waiting, never reaching its upstream fetch. GitHub routes on the same sandbox answered.

An isolate serves many requests, and a promise one request starts is bound to that request's I/O. If
that request is cancelled while the promise waits on D1, KMS or fetch, the promise never settles, and
every later request awaiting the same cached promise hangs. Several module-level caches shared
in-flight promises: the KMS keyring (forever, per isolate), decrypted provider keys and account
credentials (per minute), the relay's ID token mint, the App's pending-request owners, create timing
and share-link lookups.

They now cache settled values only (settled-cache.ts and the same pattern inline). A request that
misses loads for itself; a cold isolate may run a few duplicate loads.
2026-11-15 14:20:14 -08:00

29 lines
1.2 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { describe, expect, it } from 'vitest'
import { settledCache } from '../src/settled-cache'
describe('settledCache (WORKER-XREQ-PROMISE)', () => {
it('never hands a request another request’s in-flight load; shares the value once settled', async () => {
const cache = settledCache<number>(1000)
let loads = 0
let release: (value: number) => void = () => {}
const slow = () => new Promise<number>((resolve) => ((loads++), (release = resolve)))
const first = cache.get('k', 0, slow)
// A second request while the first is loading loads for itself.
const second = cache.get('k', 1, async () => (loads++, 2))
expect(await second).toBe(2)
release(1)
expect(await first).toBe(1)
expect(loads).toBe(2)
// Settled now: fresh hits reuse the value, stale ones load again.
expect(await cache.get('k', 10, async () => (loads++, 3))).toBe(1)
expect(await cache.get('k', 5000, async () => (loads++, 4))).toBe(4)
expect(loads).toBe(3)
})
it('a failed load is not cached', async () => {
const cache = settledCache<number>(1000)
await expect(cache.get('k', 0, async () => Promise.reject(new Error('down')))).rejects.toThrow('down')
expect(await cache.get('k', 1, async () => 7)).toBe(7)
})
})