Read Chrome localStorage snapshots through moli-leveldb-parser with
zerocopy layouts. Keep the LevelDB engine in test dependencies and port
Chromium reader scenarios with reproducible native fixtures.
Share WAL-aware SQLite snapshots and merge localStorage by origin and
UTF-16 key. Read all sources and required destination files before
replacing cookie or localStorage data.
Validation:
- cargo fmt --all
- cargo clippy --workspace --all-targets --all-features -- -D warnings
- cargo nextest run --no-fail-fast (17573 passed, 13 skipped)
Move Fontconfig matching, substitution policy, and the Fontique lookup cache into moli-system-fonts. Keep CSS family adaptation in moli-layout, migrate tests and fixtures, and retain the Skia notice in source comments.
`--obey-robots` parsed into `FetchConfig` and stopped there. Its only
consumer was the network trace, which reported the flag's own value back
out; no robots.txt was ever fetched or honored. The webfetch skill
meanwhile told agents to enable it for crawl workloads, so a crawl could
report that it obeyed robots.txt while ignoring it.
Add `moli-robots`, a dependency-light RFC 9309 parser and matcher:
- group parsing, including merged repeat groups and `User-agent` runs;
- user-agent selection by specificity with `*` as the fallback group;
- `Allow`/`Disallow` with `*` and `$`, longest match winning and `Allow`
taking ties;
- percent-encoding normalization so a pattern and a request target
compare in one space;
- the RFC 9309 section 2.3.1 response rules, where 4xx publishes no rules
and 5xx or a transport failure disallows the origin.
Wire it into `moli fetch`. The check runs before the browser starts, so a
refused URL costs one robots.txt request and the target is never
contacted.
Scope is the top-level navigation. Subresource and protocol-server
navigations run through NavigationEngine, whose redirect, service-worker,
and CDP paths need a design decision of their own; `moli serve
--obey-robots` now warns that it is not enforced instead of appearing to
work.
Fixes#79
ldm0
SpringCorel
Athul Nambiar