Commit Graph
120 Commits
Author SHA1 Message Date
ldm0 feb019b3c6 fix(fetch): enforce CORP before redirect and response delivery
fix(fetch): enforce CORP before redirect and response delivery.

Extracted from development commit db59e77d3c5 and adapted to main.

Validation (32 build jobs and test threads): cargo fmt --all; cargo clippy --workspace --all-targets --all-features -- -D warnings; cargo nextest run --no-fail-fast.
     Summary [ 132.611s] 22172 tests run: 22172 passed (9 slow), 20 skipped
2026-11-03 07:18:55 +08:00
ldm0 d9290e87ad feat(modules): support native text module imports
Support type: text through the shared module map, Window, child Documents and workers. Export UTF-8 decoded strings without executing their contents, preserve typed-module identity, and carry text request metadata through network, preload and Service Worker loaders. Regressions cover static/dynamic import sharing, BOM and invalid UTF-8, CORS, integrity, worker environments and unsupported-type rejection.

Extracted from development commit a6de018613.
Base: main (62e527cd34).

Validation (32 build jobs and test threads): cargo fmt --all; cargo clippy --workspace --all-targets --all-features -- -D warnings; cargo nextest run --no-fail-fast.
     Summary [ 146.817s] 22203 tests run: 22203 passed (9 slow), 20 skipped
2026-11-03 06:30:06 +08:00
ldm0 fe15e4b73e fix(fetch): preserve raw Location bytes across redirect paths
fix(fetch): preserve raw Location bytes across redirect paths.

Extracted from development commit 7e5a9cbf8191b2ff2aa8afd9d6a298eb86e1aab1 and adapted to main.

Validation (32 build jobs and test threads): cargo fmt --all; cargo clippy --workspace --all-targets --all-features -- -D warnings; cargo nextest run --no-fail-fast.
     Summary [ 179.354s] 22167 tests run: 22167 passed (9 slow), 20 skipped
2026-11-02 02:03:43 +08:00
ldm0 5e6482996e feat(storage): open Cookie and localStorage databases on first use 2026-11-01 04:03:38 +08:00
ldm0 1f0dc4e011 chore(browser-profile): update default Chrome version to 156.0.8078.74 2026-11-01 00:58:17 +08:00
ldm0 59b0aa9754 fix(worker): send typed Fetch Metadata for worker main scripts
Set SameOrigin mode and native Worker/SharedWorker destinations on dedicated, nested and shared worker main-script requests. Retain CORS mode for module imports and honor nested worker script-kind credentials.

Validation (32 build jobs / test threads, zero retries):

- `cargo fmt --all`
- `cargo clippy --workspace --all-targets --all-features -- -D warnings`
- `cargo nextest run --no-fail-fast`: Summary [ 133.978s] 22032 tests run: 22032 passed (9 slow), 19 skipped
2026-10-31 21:23:14 +08:00
ldm0 75a0e87674 fix(fetch): restrict automatic Sec-Fetch-User to navigation
A retained top-level cookie context must not mark a non-navigation Fetch
as user-activated. Restore the navigation-mode guard at the shared emitter.

Cover absent initiators across CORS modes, restore HTTP and HTTPS Origin
redirect cases, assert exact redirect site metadata, and preserve explicit
embedder headers without duplication.
2026-10-31 21:23:14 +08:00
ldm0 3516e967a5 fix(fetch): gate Metadata headers on each trustworthy destination
Use one Fetch Metadata emission path for browser navigations and subresources. Suppress generated Sec-Fetch headers when the current destination is not potentially trustworthy, including after redirects, while retaining loopback and localhost HTTP support.

Validation (32 build jobs / test threads, zero retries):

- `cargo fmt --all`
- `cargo clippy --workspace --all-targets --all-features -- -D warnings`
- `cargo nextest run --no-fail-fast`: Summary [ 152.548s] 22026 tests run: 22026 passed (9 slow), 19 skipped
2026-10-31 21:23:14 +08:00
ldm0 2dd72ca458 fix(performance): distinguish cached and validated resource transfers
The existing from_cache flag collapses fresh local responses and revalidated 304 responses into the same state, producing incorrect Resource Timing transferSize.

Carry explicit cache state through buffered and streamed fetches, navigation, subresource snapshots and response interception. Resource Timing reports zero transfer for local hits, validation overhead for merged 304 responses, and body bytes plus header overhead for network responses. Cover disk and memory cache state in the HTTP timing fixture and native response tests.
2026-10-31 20:06:00 +08:00
ldm0 5d65e521c5 refactor(protocol): supervise dedicated worker closure and command recovery
Run dedicated-worker close cleanup as an ordinary async workflow in the
existing in-flight entry. Join the original session disposal, reject new
attachments during retirement, and bind cleanup to the original worker and
session lifetimes. Preserve earlier captured auto-attach facts while retiring
any original binding they publish during close.

Return command contexts, typed events, exact renderer boundaries and actual
terminals to the original scope on cancellation or discarded results. Resume
admitted cleanup rather than restarting it, and finish precise resource cleanup
after a workflow panic. Settle pending worker calls at close admission so the
original Target-detached terminal stays before the close acknowledgement.

Preserve the existing typed FetchCancelled cause when cancellation ends curl
through its write callback, keeping the original transport error and metadata.

Validation: cargo fmt --all; full workspace/all-targets/all-features Clippy;
root nextest 21,007 passed /19 skipped; five state-access doctests; 70 focused
cleanup tests; 32 stress rounds for raw/Digest and HTML typed cancellation.
Final-source CLI checks passed: worker close 3/3 with two sessions, held promise
and peer progress; CDP 12/12, captured/auth 7/7, child navigation, Playwright, and
debugger-held activation/Page.close/Target.close. All 4,805 Rust/Cargo input
hashes remained identical across final checks, CLI build and staging.
2026-10-29 17:39:23 +08:00
ldm0 9268deff09 refactor(protocol): own teardown and command completion workflows
Move session disposal, frontend teardown, target activation and closure waits
out of connection borrows into supervised workflows. Retain exact binding
identities and command output ownership through cancellation and discarded
ready results.

Complete Native and finite query results synchronously and run Input
download and file chooser completion as owned async work. Preserve response
and renderer publication order, and release the actor while Tab auto-attach
joins an existing initial-document installation.

Make the meta-refresh shutdown regression wait for its successor document
before asserting exact DOM output.

Validation: cargo fmt --all; full workspace Clippy with warnings denied;
cargo nextest run --no-fail-fast (20,973 passed, 19 skipped); real CLI,
Playwright, paused-renderer, Fetch-close and Input checks.
2026-10-29 17:39:23 +08:00
ldm0 0380308056 refactor(protocol): own Fetch navigation and cleanup workflows
Keep intercepted document requests, authentication, captured responses and native
commit/adoption in ordinary async workflows with exact navigation ownership.
Move child lifecycle waits and Fetch.disable cleanup out of actor completion,
retain source response obligations through cancellation, and preserve ACK timing.

Prepare owned session policy/script/worker cleanup and a synchronous state owner
with weak handles and compile-time borrowing constraints. Retain explicit legacy
inline drivers until disposal, frontend control and adapter paths migrate.
Preserve typed error contexts and share the inline Fetch driver with BiDi adapters
so they cannot wait on their own unserviced state mailbox.

Validation: cargo fmt --all; complete workspace Clippy with all targets/features
and warnings denied; root nextest 20,941 passed, 19 skipped; protocol doctests
1 positive and 4 compile-fail cases passed.
2026-10-29 17:39:23 +08:00
ldm0 5d34b770d1 fix(fetch): scope redirect authorization to credential origins
Filter configured and request Authorization headers against the initial
transfer origin at the shared outgoing-header boundary. Remove the
unconditional Basic auth fallback that bypassed the origin guard.

Bind native server credentials when supplied and enforce that scope for
Basic and libcurl-managed authentication, preserving proxy credentials
and explicit auth responses to later redirect targets.

Cover all three HTTP transports, mutable caller-managed redirect URLs,
same-origin and foreign Digest challenges, and fresh destination grants.
2026-10-28 00:24:55 +08:00
ldm0 638fa6dc7e fix(fetch): gate HTTP cache on effective literal cookies
Share literal Cookie selection between header emission and cache eligibility.
Unbound data, out-of-scope hosts and credentials-suppressed literals no longer
disable the disk cache. Requests that actually send cookies still bypass
lookup and storage, preserving any earlier public entry.

Cover raw and HTML streams with fresh network runtimes and both request-level
and configured Cookie data, including credentials omit and cross-origin
same-origin requests.
2026-10-28 00:24:55 +08:00
ldm0 ded7899eda fix(fetch): give configured cookie data a default host scope
Keep serve --cookie DATA and bind its default host at the first HTTP(S)
top-level navigation. Share the binding across cloned configurations and
network runtimes so frames, workers, later pages and browser contexts cannot
claim or expand it. Subrequests use host-only matching; scoped top-level
transfers retain curl-compatible redirect propagation.

Keep literal data separate from the Cookie Store and preserve credentials
checks, imported cookie scopes and request-level CLI cookie behavior.

Bind at navigation entry before Service Worker interception as well as at
HTTP runtime admission, so synthetic responses do not delay the scope choice.
2026-10-28 00:24:55 +08:00
ldm0 b6d1f46486 fix(fetch): honor credentials mode for literal cookies
Apply the Cookie Store credentials decision to both configured and
request-specific literal Cookie data. This excludes browser omit requests
and cross-origin same-origin requests, including credentialless OPTIONS
preflights, while CLI fetches retain their Include credentials policy.

Cover both configuration sources, every fetch transport and the actual
renderer preflight followed by a credentialed request.
2026-10-28 00:24:55 +08:00
ldm0 4b87174bc7 fix(fetch): scope explicit cookies to the initial redirect origin
Filter CLI Cookie headers at the shared outgoing-header boundary using the
initial URL hostname, scheme and effective port. Keep the configured header
list intact so returning to the initial origin restores curl behavior.
Literal Cookie data and per-target Cookie Store selection stay separate.

Cover default and per-request headers, casing and duplicates, redirected
header restoration, all fetch transports and CLI cookie combinations.
2026-10-28 00:24:55 +08:00
ldm0 daee1e9eaa fix(fetch): preserve explicit cookie headers in CLI requests 2026-10-28 00:24:55 +08:00
ldm0 580cde7fa2 fix(fetch): complete cookie inputs and unify export finalization 2026-10-28 00:24:55 +08:00
ldm0 ce7f503f28 fix(fetch): match Chromium for bare-CR header EOF 2026-10-22 02:33:44 +08:00
ldm0 510a5cbf3e fix(fetch): parse bare CR response fields like Chromium
Enable the curl fork's opt-in browser HTTP/1 response-field parser before
framing and callbacks. Cover CR-separated/folded fields, leading/repeated
CR, observations, length/chunked bodies and malformed framing across all
three fetch transports. Consume lexmount/curl-rust#3 at 79310cbd939.

Extracted from 3c1dfe9e8a8b69a72cb50ec53c57df7e249c5a67 onto main
8d57e02b3c, retaining its XML fork pin.

Validated cargo fmt --all, strict workspace/all-targets/all-features
Clippy and cargo nextest run --no-fail-fast: 20,490 passed, 18 skipped.
Live CDP checks of six raw-header responses match Chromium exactly in
status, response headers and body. The curl fork's 75 local tests pass.
2026-10-21 19:34:39 +08:00
ldm0 d75c82c30a perf(fetch): share immutable request bodies with Bytes 2026-10-19 11:22:56 +08:00
ldm0 b6a1e93e89 refactor(fetch): clarify transport redirects and cancellation 2026-10-19 05:24:53 +08:00
ldm0 1275fb7016 chore(deps): upgrade curl bindings and bundle libcurl 8.23
Pin the fork rebased onto upstream curl-rust 0.4.51, including the native
pre-request API and libcurl 8.23 with the upstream connect-only eviction fix.
Enable the upstream apple-sectrust feature explicitly in moli-curl.

Keep Moli proxy endpoints explicit while adapting transport comparisons to
libcurl's implicit HTTP port and hostname validation. Update form wire
request assertions to preserve the empty query already in navigation URLs.

Validated workspace formatting, warnings-as-errors Clippy, and all 20,342
nextest cases (18 skipped). The curl fork passed Linux/macOS/Windows CI.
2026-10-19 03:53:12 +08:00
ldm0 012bd10983 refactor(fetch): use curl-rust pre-request callbacks
Pin curl and curl-sys to the native callback API from curl-rust#1 and replace
the handwritten callback registration with Handler::prereq / Easy2::prereq.
Keep each collector's captured timing token and re-enable the callback after
reset; the dependency owns userdata lifetime and C panic handling.
2026-10-19 03:00:24 +08:00
ldm0 4c92f8049c perf(fetch): observe request start without verbose
Record request start with CURLOPT_PREREQFUNCTION using the handler already
owned by Easy2. Reinstall the callback on each attempt and read the
collector's writer token across moves, reset and progress reselection.

Enable verbose only for requested header observations or proxy CONNECT
diagnostics. Cover keepalive GET/HEAD/POST reuse, callback ownership,
HTTP/2 multiplexing and gated proxy tunnels.
2026-10-19 03:00:24 +08:00
ldm0 de2a56a39c perf(fetch): compact declared body lengths with NonZeroUsize
Keep zero-length completion local to header processing and retain only
positive body-length targets in the HTML and raw streaming collectors.
Preserve HTML redirect eligibility before committing an empty response.

On the same x86_64 dev build, the field shrinks from 16 to 8 bytes,
HTML collectors from 2752 to 2744 bytes, and raw collectors from 2728
to 2720 bytes. The enclosing transfer handler also shrinks by 8 bytes.

Cover accepted empty responses, unknown lengths and followed empty redirects
without changing their completion/cancellation behavior.
2026-10-19 03:00:24 +08:00
ldm0 0cf1cc09a9 fix(fetch): recover HTML response channels for cached redirects
Redirects and retries retain start/body senders in the reused easy handle.
Return those channels to the HTML job before publishing a final cache hit,
and use the retained easy when an attempt fails before reconfiguration.

Cover network-to-cache redirects with discarded interim/body observations,
and preserve the original error instead of reporting a closed start channel.
2026-10-19 03:00:24 +08:00
ldm0 bb1cf493c6 perf(fetch): record response body presence once per transfer
Latch the first body observation in the transfer handler and reset it whenever
the easy handle is reused. Record nonempty writes before declared completion,
while retaining progress-based detection for compressed empty payloads.

Avoid repeated timing locks without skipping cancellation checks.
2026-10-19 03:00:24 +08:00
ldm0 85df60cc74 perf(fetch): share transfer progress state in one allocation
Keep cancellation arbitration and timing in one Arc<ResponseProgressState>.
This removes one allocation per progress generation and one reference-count
update per clone, reducing the writer token from 16 to 8 bytes on x86_64.
Preserve transfer isolation and the existing atomic completion/cancel protocol.
2026-10-19 03:00:24 +08:00
ldm0 5fc2edc671 fix(fetch): classify 101 as final in response timing
Reuse the existing interim status predicate in buffered, HTML and raw
collectors so Switching Protocols starts the final response timeline.
Cover 101 alone and after 103 across all collectors, plus HTML header completion.
2026-10-19 03:00:24 +08:00
ldm0 703765dcd7 fix(fetch): record native terminal timing for HTML streams
Record response end before publishing native success or failure, including
cancellation, header-only responses and declared identity body completion.
Keep cache availability separate from the end of the cached body reader.

Cover delayed consumers, chunked bodies, empty responses, truncated transfers,
cancellation, cache hits and encoded Content-Length comparison boundaries.

Observe delivered body bytes before publishing declared completion.
2026-10-19 03:00:24 +08:00
ldm0 87efd22bc6 fix(fetch): isolate HTML timing for each transfer attempt
Select a fresh ResponseProgress for every HTML attempt and pass that exact
writer token to the collector, matching raw streaming. Redirects and retries
must not retain observations from a discarded hop.

Add a gated loopback 103 -> 302 with body -> 200 without body regression.
2026-10-19 03:00:24 +08:00
ldm0 029d2d0889 feat(fetch): observe native request and interim response timing 2026-10-18 23:20:44 +08:00
ldm0 a8def2344e feat(fetch): preserve native response timing observations 2026-10-18 23:20:44 +08:00
ldm0 496bc5c848 fix(fetch): revalidate disk cache for request control headers 2026-10-18 20:11:53 +08:00
ldm0 05a1c73fa6 refactor(fetch): unify redirect handling state 2026-10-17 01:25:36 +08:00
ldm0 91544b6694 fix(fetch): preserve cancellation across response selection
Delegate subrequest cancellation to Tokio CancellationToken while retaining
transfer-local completion arbitration and cancellation-safe adoption.

Commit buffered final response progress before delivery and materialize CORS
follow responses after dropping intermediate redirect and preflight bodies.
Cover cancellation/selection races and buffered response completion.
2026-10-17 01:25:36 +08:00
ldm0 73e12a28f4 fix(fetch): bind response progress to individual transfers
Capture a fixed progress token in each transport job and collector. Share the accepted response token with the logical request and arbitrate completion against consumer cancellation with one atomic state.

Isolate preflight and redirect cancellation, discard unused bodies after headers, and apply conditional cancellation to EventSource close. Add concurrent progress and loopback HTTP regressions.

Validation: cargo fmt --all; workspace Clippy for all targets and features with warnings denied; cargo nextest run --no-fail-fast (20,185 passed, 18 skipped).
2026-10-17 01:25:36 +08:00
ldm0 be1a37082f feat(fetch): isolate subrequest cancellation and response progress
Add hierarchical cancellation authority for redirect/preflight subrequests: parent abort reaches children, while a child abort leaves siblings and the parent running. The accepted final response can transfer completion facts to its owner without retaining the child cancellation state. Resetting a logical transfer detaches old progress. Native unit tests cover cancellation inheritance, isolation and completion selection before/after terminal notification. Integration of every browser redirect producer is separate work.
2026-10-17 01:25:36 +08:00
ldm0 6f0e9631ef fix(fetch): bound authentication retries per logical request 2026-10-14 15:59:24 +08:00
ldm0 99c24fd8de fix(fetch): retain referrer selection across redirect hops
Retain the referrer selected for each completed hop before applying the redirect response policy. A later permissive policy cannot restore an omitted referrer or stripped path. Apply this to buffered, HTML streaming, raw streaming and cached redirects, with native HTTP regressions for policy changes, manual redirects and request identity preservation.
2026-10-12 04:51:02 +08:00
ldm0 cfc97efe51 fix(fetch): preserve received HTTP status text
Carry received HTTP reason phrases from the parser through buffered and streaming responses, disk and memory caches, page responses, Fetch, and XHR. Preserve custom and empty HTTP/1 phrases and report empty text for HTTP/2 and HTTP/3. Synthesized responses retain canonical fallback. Regressions cover whitespace, non-ASCII bytes, and cache round trips.
2026-10-11 21:19:08 +08:00
ldm0 0487827c0b feat(fetch): retain explicit referrer state independently of request identity
Add native Fetch referrer selection and inheritance without changing request origin, cookie initiator or network partition. Apply the effective policy to the selected referrer when building outgoing headers, preserve explicit omission and about:client fallback, and copy policy to derived requests without copying integrity. Unit tests cover header selection, sanitization and request identity.
2026-10-11 06:05:43 +08:00
ldm0 af86134376 fix(xhr): preserve upload-listener preflight requirements 2026-10-10 19:03:01 +08:00
ldm0 bde57a391a refactor(fetch): share effective referrer policy selection 2026-10-07 18:49:51 +08:00
ldm0 6ef92ff113 test(fetch): accept stable and nightly owner panic frame formats 2026-10-07 18:39:33 +08:00
ldm0 e10d2a7713 test(fetch): assert empty request body framing 2026-10-05 05:19:04 +08:00
ldm0 0c9f4775d1 test(fetch): cover opaque request origins across sandbox and CORS 2026-10-05 04:24:18 +08:00
ldm0 3d504a198e refactor(fetch): share quoted HTTP header list splitting 2026-10-05 03:42:50 +08:00