Commit Graph
217 Commits
Author SHA1 Message Date
ldm0 cb907cd5ac test(cdp): report smoke progress 2026-08-21 11:23:44 +08:00
ldm0 3ad7dfda80 refactor(renderer): make lifecycle document ids opaque 2026-08-21 11:23:44 +08:00
ldm0 9925bb12f3 refactor(renderer): identify document runtime incarnations exactly 2026-08-21 11:23:44 +08:00
ldm0 b6f798dfa9 refactor(renderer): bind document write loads to exact owners 2026-08-21 11:23:44 +08:00
ldm0 527b408ead refactor(renderer): guard module evaluation by document owner 2026-08-21 11:23:44 +08:00
ldm0 66fd45517c refactor(renderer): snapshot script execution by document owner 2026-08-21 11:23:44 +08:00
ldm0 80bb2e6509 refactor(renderer): coalesce continuations by document owner 2026-08-21 11:23:44 +08:00
ldm0 1a33394011 refactor(renderer): target module reactions by document owner 2026-08-21 11:23:44 +08:00
ldm0 9e04a203d7 refactor(renderer): bind prepared scripts to exact document owners 2026-08-21 11:23:44 +08:00
ldm0 27b86343f8 refactor(renderer): authorize document tasks by exact owner 2026-08-21 11:23:44 +08:00
ldm0 530e9ae592 refactor(renderer): bind host scripts at document admission 2026-08-21 11:23:44 +08:00
ldm0 8164cfd94d refactor(navigation): allocate opaque document history ids 2026-08-21 11:23:44 +08:00
ldm0 cde08f79e3 refactor(renderer): bind service worker callbacks to window owners 2026-08-21 11:23:44 +08:00
ldm0 213634ba62 refactor(renderer): key DOM wrappers by exact handles 2026-08-21 11:23:44 +08:00
ldm0 b97719d675 refactor(protocol): harden transient identity allocation 2026-08-21 11:23:44 +08:00
ldm0 e6994b21b9 refactor(renderer): signal dialog openings directly 2026-08-21 11:23:44 +08:00
ldm0 4f75442c7f refactor(renderer): bind page storage work to exact task ids 2026-08-21 11:23:44 +08:00
ldm0 ff26e9cfb7 simplify indexeddb storage bucket integration 2026-08-21 11:23:44 +08:00
ldm0 cca374e0bf bind service worker callbacks to exact run identity 2026-08-21 11:23:44 +08:00
ldm0 40b1d0ce02 bind classic DOM references to exact page residence 2026-08-21 11:23:44 +08:00
ldm0 7733dcd843 Anchor scrollable overflow at the padding box 2026-08-21 03:42:28 +08:00
ldm0 2ae43dcc63 fix(docs): slimmer webfetch skill 2026-08-21 03:42:02 +08:00
Athul Nambiar 777ae9af36 fix(cli): enforce --obey-robots against robots.txt
`--obey-robots` parsed into `FetchConfig` and stopped there. Its only
consumer was the network trace, which reported the flag's own value back
out; no robots.txt was ever fetched or honored. The webfetch skill
meanwhile told agents to enable it for crawl workloads, so a crawl could
report that it obeyed robots.txt while ignoring it.

Add `moli-robots`, a dependency-light RFC 9309 parser and matcher:

- group parsing, including merged repeat groups and `User-agent` runs;
- user-agent selection by specificity with `*` as the fallback group;
- `Allow`/`Disallow` with `*` and `$`, longest match winning and `Allow`
  taking ties;
- percent-encoding normalization so a pattern and a request target
  compare in one space;
- the RFC 9309 section 2.3.1 response rules, where 4xx publishes no rules
  and 5xx or a transport failure disallows the origin.

Wire it into `moli fetch`. The check runs before the browser starts, so a
refused URL costs one robots.txt request and the target is never
contacted.

Scope is the top-level navigation. Subresource and protocol-server
navigations run through NavigationEngine, whose redirect, service-worker,
and CDP paths need a design decision of their own; `moli serve
--obey-robots` now warns that it is not enforced instead of appearing to
work.

Fixes #79
2026-08-21 02:34:26 +08:00
ldm0 a142d011c0 refactor: extract HTML parser input stack 2026-08-20 23:39:45 +08:00
ldm0 cf33d7dca5 Bump version to 1.0.2 v1.0.2 2026-08-20 22:19:44 +08:00
ldm0 a46043736e fix: preserve nested parser input order 2026-08-20 22:17:23 +08:00
allen 622848ff40 Rename 'Test data' section to 'Benchmark' 2026-08-20 20:57:47 +08:00
ldm0 84f70612b8 fix: Fix parser early finish 2026-08-20 20:39:24 +08:00
ldm0 2c186e3c4a fix(devtools): preserve nested Main response order 2026-08-20 14:12:15 +08:00
ldm0 a74431d226 fix(devtools): unify synchronous Page IO responses 2026-08-20 14:12:15 +08:00
ldm0 912101e9dd test(cdp): stress synchronous IO response order 2026-08-19 19:58:31 +08:00
ldm0 d28c20ef87 refactor(devtools): align inspector session routing with Chromium 2026-08-19 19:58:31 +08:00
ldm0 897db5964c test(cdp): cover Worker interrupt routing 2026-08-19 19:58:31 +08:00
ldm0 2416e731f6 refactor(devtools): unify Worker inspector task queue 2026-08-19 19:58:31 +08:00
ldm0 1c4a91e09a test(cdp): gate inspector routing by default 2026-08-19 19:58:31 +08:00
ldm0 3456ec7c38 fix(devtools): split Worker inspector execution paths 2026-08-19 19:58:31 +08:00
ldm0 f63a5640ab fix(devtools): make Page crash terminal IO control 2026-08-19 19:58:31 +08:00
ldm0 f90729edf3 fix(devtools): unify Page IO agent ingress 2026-08-19 19:58:31 +08:00
Donough Liu a906f0d57a Reuse CSSOM side-entry snapshots (#128) 2026-08-19 16:38:50 +08:00
Eric 3d3a90bb5e Add Lexbench-Headless-Browser benchmark results 2026-08-19 15:25:36 +08:00
ldm0 9800a2b091 Fix CDP execution context replay on reconnect 2026-08-19 07:05:43 +08:00
ldm0 509b70f370 docs: add dark mode banner 2026-08-19 02:49:45 +08:00
ldm0 107edf1680 Simplify repository license metadata 2026-08-19 00:14:42 +08:00
ldm0 4637e6df95 Skip Rust checks for non-Rust changes 2026-08-19 00:14:42 +08:00
ldm0 c020b81cec Bump version to 1.0.1 v1.0.1 2026-08-18 17:56:26 +08:00
ldm0 595a538bcf Refactor network fetch errors as anyhow context 2026-08-18 17:38:12 +08:00
ldm0 1e5fdc5c84 bind storage bucket work to persistent identity 2026-08-18 03:53:58 +08:00
XDLCS 4352193ff3 opt: remove manual flock on Windows 2026-08-18 03:28:48 +08:00
ldm0 b1275af7f4 refactor(protocol): remove speculative cookie subscription state 2026-08-18 02:26:17 +08:00
ldm0 763ba96d6e refactor(renderer): store document input stream state directly 2026-08-18 02:26:17 +08:00