Every option except `--header` was long-form only, so ordinary invocations
carried a lot of typing and boolean flags could not be grouped.
Assign short flags to the options that appear most often, leaving the rest
long-form. `CommonArgs` is flattened into both subcommands, so each letter
has to be unique across the fetch union and the serve union rather than
within one struct:
fetch -d dump, -w wait-until, -s wait-selector, -t timeout,
-n noscript, -H header (unchanged)
serve -p port, -t timeout
common -l layout, -r resource, -i image, -f font, -a audio, -v video,
-m media, -T text-track, -c cookie-file, -P profile-dir,
-A user-agent, -L log-level
The whole optional-resource family gets a letter so it clusters, which is
the shape the request asked for: `moli serve --layout --resource` is now
`moli serve -lr`, and `-ifavmT` selects every optional family.
`-P` rather than `-p` for `--profile-dir` keeps `-p` for the serve port.
`-A` for `--user-agent` follows curl. `-T` and `-L` are uppercase only
because `-t` and `-l` are taken. `--host` keeps no short so `-H` is not
overloaded, since it already means `--header` under fetch.
Two call sites match flags by literal string and needed the short
spellings as well, or they would have silently ignored them:
- `normalize_dump_flag`, so a bare `-d` still defaults to `html`;
- `FETCH_INFER_FLAGS` and `SERVE_INFER_FLAGS`, so `moli -d markdown URL`
still infers the fetch subcommand. Each short is mirrored beside the
long flag it aliases, so both spellings infer identically.
Guard the letters with `Cli::command().debug_assert()`, which is clap's
own duplicate-short check and runs over each subcommand's full union.
Fixes#121
Embeds the light/dark SVGs published by lexmount/moli-metrics, which
regenerates them hourly from the stargazer timeline and commits only when
the count actually changed. Added to all six language READMEs, using the
same <picture> theme-switching pattern as the existing benchmark charts.
`--obey-robots` parsed into `FetchConfig` and stopped there. Its only
consumer was the network trace, which reported the flag's own value back
out; no robots.txt was ever fetched or honored. The webfetch skill
meanwhile told agents to enable it for crawl workloads, so a crawl could
report that it obeyed robots.txt while ignoring it.
Add `moli-robots`, a dependency-light RFC 9309 parser and matcher:
- group parsing, including merged repeat groups and `User-agent` runs;
- user-agent selection by specificity with `*` as the fallback group;
- `Allow`/`Disallow` with `*` and `$`, longest match winning and `Allow`
taking ties;
- percent-encoding normalization so a pattern and a request target
compare in one space;
- the RFC 9309 section 2.3.1 response rules, where 4xx publishes no rules
and 5xx or a transport failure disallows the origin.
Wire it into `moli fetch`. The check runs before the browser starts, so a
refused URL costs one robots.txt request and the target is never
contacted.
Scope is the top-level navigation. Subresource and protocol-server
navigations run through NavigationEngine, whose redirect, service-worker,
and CDP paths need a design decision of their own; `moli serve
--obey-robots` now warns that it is not enforced instead of appearing to
work.
Fixes#79
ldm0
Athul Nambiar
allen
Donough Liu
Eric