mirror of
https://github.com/stablyai/orca.git
synced 2026-11-17 19:34:29 +00:00
Egress proxy secret routes (custom-base-URL keys, OPENAI_BASE_URL) hung until the client gave up (dev, proxy mode, 2026-11-15). The Worker was invoked (tail: `canceled`, wall 65 s, CPU 3 ms, no log line) and was waiting, never reaching its upstream fetch. GitHub routes on the same sandbox answered. An isolate serves many requests, and a promise one request starts is bound to that request's I/O. If that request is cancelled while the promise waits on D1, KMS or fetch, the promise never settles, and every later request awaiting the same cached promise hangs. Several module-level caches shared in-flight promises: the KMS keyring (forever, per isolate), decrypted provider keys and account credentials (per minute), the relay's ID token mint, the App's pending-request owners, create timing and share-link lookups. They now cache settled values only (settled-cache.ts and the same pattern inline). A request that misses loads for itself; a cold isolate may run a few duplicate loads.