mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-09 08:03:50 +00:00
Ruben FiszelandClaude Opus 5
785277e0bb
* feat(nativets): bound fetch on a peer that never answers
deno_fetch applies no deadline of any kind. A peer that completes the TCP
handshake, accepts the request and then goes silent leaves `await fetch(...)`
pending indefinitely, holding its worker slot until the *job* timeout -- which
on self-hosted defaults to DEFAULT_SELFHOSTED_TIMEOUT, i.e. 7 days.
Nothing else catches this. Zombie-job detection keys off a stale
v2_job_runtime.ping, and a worker blocked inside a pending fetch keeps pinging
normally throughout: the worker is alive and healthy, only the work is dead.
What this bounds is the wait for a response to begin, and it stops there:
- a peer that never answers -> rejected after N seconds
- a peer slow to answer, but under N -> unaffected
- a body that then streams for an hour,
or is read slowly by the caller -> unaffected, always
That last line rules out the obvious implementation: AbortSignal.timeout(N)
around every fetch would bound the hang and break every streaming response and
long download. This is a hang detector, not a latency budget.
Default 300s via WINDMILL_FETCH_RESPONSE_TIMEOUT_SECS (0 disables), with a
per-script `//fetch_response_timeout <seconds>` annotation alongside the
existing //useragent and //proxy. Both nativets paths inherit it, since
eval_fetch_timeout and the dedicated-worker path in bun_executor both funnel
through create_nativets_runtime.
The ms value is clamped to i32::MAX: deno_web's setTimeout runs its delay
through webidl.converters.long, a 32-bit conversion that *wraps*, so a setting
past ~24.8 days would come out negative and fire immediately -- turning an
over-generous timeout into an instant one on every fetch.
The window covers connect, TLS and request upload as well as server think
time, so a very slow large upload is bounded by it too; the error message says
so rather than claiming the connection went silent.
Not covered: a body that stalls midway. Reaching that needs the response's
InnerBody, which deno_fetch keeps module-private, and every way to wrap it
from outside changes observable Response semantics (locking, bodyUsed,
double-consume errors). Left for a follow-up in deno_fetch itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb
* test(nativets): cover the instance-wide response-timeout env var
A typo in WINDMILL_FETCH_RESPONSE_TIMEOUT_SECS would compile, pass every
other test, and silently hand every operator the 300s default -- the same
class of silent-default failure the timeout itself exists to prevent. Its
own test binary, since a LazyLock resolves the value once per process.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb
* fix(nativets): inherit the caller's RequestInit, and clear the long-poll ceiling
Two problems with the first cut, both found in review.
`{ ...init, signal }` copied only own enumerable properties, but RequestInit is
a WebIDL dictionary whose members deno_fetch reads with plain property gets
that walk the prototype chain. Anything inherited or non-enumerable was
dropped: `fetch(url, Object.create({method: "POST"}))` silently became a GET.
Worse, a non-object init went from a loud TypeError to a silent GET, because
spreading "POST" yields {0:"P",1:"O",...} -- a valid dictionary with ignored
keys. Now the init is inherited from rather than copied, and a non-dictionary
is handed straight back to deno_fetch for its own TypeError.
The 300s default also sat at half of TIMEOUT_WAIT_RESULT (600s), which
run_wait_result long-polls against with no response headers. A script running
another job synchronously for 300-600s would have timed out client-side while
the server was still legitimately holding the request open -- the long-poll
risk class, instantiated inside the product and reachable without writing a
raw fetch. Default raised to 900s, with the constraint recorded where someone
would break it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb
* fix(nativets): hand the caller's RequestInit to Request untouched
Carrying a WebIDL dictionary across by hand has no safe form, and both
previous attempts were wrong in opposite directions. Spreading a copy drops
inherited and non-enumerable members, and turns a non-object init from a loud
TypeError into a silent GET. Inheriting from it via Object.create fixes those
but makes the child the receiver, so an accessor on the original runs against
an object that lacks its private-field brand:
Cannot read private member #body from an object whose class did not
declare it
So don't carry it at all. fetch()'s own first act is `new Request(input,
init)`; doing that here hands the init to the same constructor, read exactly
as it would be without this wrapper, and our signal travels in an init we own.
`req.signal` is then deno's own resolution of init.signal over an input
Request's signal, which removes the hand-rolled version of that rule too.
The Request is built twice as a result, once here and once inside fetch. That
is cheap: cloneInnerRequest carries method, headers, redirect mode, clientRid
and blob entry, and a body is proxied rather than buffered -- a static body is
a shallow {body, consumed} copy sharing its bytes, a stream gets a one-chunk
pass-through.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb
* fix(nativets): keep an aborted fetch settling in the same tick
deno_fetch keeps its outer fetch non-async on purpose: "WPT has a test that
aborted fetch is settled in the same tick. This means we cannot wrap the
promise if it is already settled" (26_fetch.js). An `async` wrapper adopts
that promise through another one, so a rejection that used to land before any
microtask queued after the call now lands after it.
Made the wrapper non-async, with an early return that hands deno's settled
rejection straight back for an already-aborted signal, and no timer armed
there since there is no response to wait for. Construction still has to reject
rather than throw, so it is caught and returned as a rejection, which is what
the `async` was buying.
The comment claiming this matched deno_fetch's own `async function fetch` was
wrong on two counts -- that function is not async, and the wrapper was not
matching it. Replaced with the constraint that actually holds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb
* fix(nativets): keep fetch's observable shape and reach intrinsics safely
Three ways the wrapper was distinguishable from the fetch it replaces, all
observable from a script sharing the isolate.
`.then` was an ordinary property lookup, so `Promise.prototype.then =
undefined` broke fetch after the request had already gone out. deno's own
modules reach intrinsics through primordials, and this file already captured
setTimeout, clearTimeout and Promise.reject for exactly that reason, so the
lookup was the odd one out. Now captured alongside them.
Declaring `init` without a default made `fetch.length` 2 where the standard
says 1. And the empty-call branch forwarded two explicit `undefined`s, so
deno's required-argument check saw two arguments and raised "Invalid URL:
'undefined'" instead of "1 argument required". Forwarding through
ReflectApply preserves the count.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb
* docs: clarify fetch timeout restart requirements
* fix: capture native fetch abort helpers
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
38 lines
1.1 KiB
Rust
38 lines
1.1 KiB
Rust
//! `//fetch_response_timeout` parsing. Cheap (no V8), so it stays out of the
|
|
//! e2e file.
|
|
|
|
use windmill_runtime_nativets::get_annotation;
|
|
|
|
#[test]
|
|
fn a_value_is_parsed_and_zero_stays_distinct_from_absent() {
|
|
// Collapsing `Some(0)` to `None` would silently reinstate the default on a
|
|
// script that explicitly asked for no limit.
|
|
assert_eq!(
|
|
get_annotation("//native\n//fetch_response_timeout 30\n").fetch_response_timeout_secs,
|
|
Some(30)
|
|
);
|
|
assert_eq!(
|
|
get_annotation("//fetch_response_timeout 0\n").fetch_response_timeout_secs,
|
|
Some(0)
|
|
);
|
|
assert_eq!(
|
|
get_annotation("//native\n").fetch_response_timeout_secs,
|
|
None
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn a_malformed_value_falls_back_to_the_default_not_to_no_timeout() {
|
|
for src in [
|
|
"//fetch_response_timeout abc\n",
|
|
"//fetch_response_timeout\n",
|
|
"//fetch_response_timeout -5\n",
|
|
] {
|
|
assert_eq!(
|
|
get_annotation(src).fetch_response_timeout_secs,
|
|
None,
|
|
"{src:?} should leave the default in force"
|
|
);
|
|
}
|
|
}
|