mirror of
https://github.com/daijro/camoufox.git
synced 2026-10-03 16:00:19 +00:00
main
34
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  
|
f36390a19e |
feat(geoip): make GeoIP AIO the default source, deprecate GeoLite2 (#820)
The default GeoIP source was MaxMind GeoLite2 via sapics/ip-location-db, whose URLs kept serving the 2026-06-17 build after that project moved to GitHub Releases (found in #815). GeoIP AIO (daijro/geoip-all-in-one) resolves timezones more accurately on real proxy IPs and is rebuilt weekly. - repos.yml: AIO is the default; GeoLite2 is `deprecated: true`, with the Releases URLs from #815 so it still works when picked by name. - A cache holding a deprecated source it was not explicitly given (`camoufox set --geoip` or the GUI) moves to the default and drops the old database. An explicit choice is kept, with a FutureWarning. - needs_update() reads the database's build date instead of the file age: refresh once the build is over 8 days old, re-checking at most daily, and warn when a fresh download is over 30 days old (a frozen source). - get_geolocation(geoip_db=...) now reads that source's own database rather than the active one's, and no longer makes it the active one. - tests/test_geoip_sources.py (from #815) downloads every non-deprecated source and fails when its build is stale; tests.yml installs the geoip extra so it runs, and so gates every release. - TypeScript twin updated to match; goldens answer in both layouts. Co-authored-by: lp177 <57773165+lp177@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Jake WriterandClaude Opus 5.5
|
11969fa44a |
Prerelease on every tested merge, promote by tag, and pair each library release with its browser (#810)
* Pair each library release with the browser build it was tested with
Nothing tied a library release to a browser build: `camoufox fetch` took the
newest build in a channel, and a launch used whatever config.json marked
active, so an upgraded library could run a browser it was never tested with,
and an old library would pick up a newer, incompatible browser.
A released package now carries browser-pin.json, naming the browser release
built from the same sources. With it, and no explicit choice by the user:
- fetch installs exactly that build (no prerelease prompt: it is the build
this release was tested with, prerelease or not);
- a launch uses exactly that build, whatever else is installed or active,
and reports it as not installed rather than falling back to another;
- the fetcher's automatic install (TypeScript's first run) takes only it.
An explicit `camoufox set` still wins, with a one-time warning at launch;
`camoufox set --release` returns to the paired build, and `camoufox active`
says which is in use. The checked-in pin is `{}`, so development checkouts
follow their channel as before.
Also: prerelease library versions (0.5.8b1, 0.5.8-beta.1) parse as their
release; they were read as 0.5.0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Release a prerelease on every tested merge; promote to stable by tag
Every merge to main whose tests pass now publishes a prerelease of all
three artifacts, and pushing vX.Y.Z on a tested main commit promotes it.
- Build and Release runs after Tests on main. It builds the browser only
when its sources changed (ci.browser_inputs.source_digest: every browser
input, not counting the release number). Each build gets the next unused
beta.N on a release commit beside main -- main is protected -- and is
published as a GitHub prerelease, not a draft, with its source digest in
the notes.
- Publish to pypi follows it: <next>bN on PyPI, then Publish to npm puts
<next>-beta.N under the `next` dist-tag. Both are stamped with the browser
release built from the same sources.
- A vX.Y.Z tag is refused unless the commit is on main and `All tests
passed` succeeded on it. The paired browser prerelease then becomes the
stable, latest release (no rebuild, so users get the tested binaries), and
X.Y.Z goes to PyPI and npm `latest`.
The tested commit travels between workflows as an artifact: a workflow_run
is told main's head, so two quick merges would otherwise publish the second,
untested one. ci/release.py holds the planning, stamping and promotion,
unit-tested in ci/tests/test_release.py.
Also fixes two checks that failed the manual release already: vermin
targeted Python 3.8 exactly, against a package that declares ^3.10 and a
code base that needs 3.9, and check-pack compared npm and PyPI prerelease
versions as strings, although each registry spells them differently.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test driver-only pull requests against the release paired with their sources
The scope step matched the release tag named by upstream.sh. With release
numbers now allocated per build, that number is a floor, not a release, so
driver-only pull requests would nearly always rebuild, or fetch a build other
than the one their sources produce. It now asks `ci.release paired` for the
release built from exactly this tree's browser sources, and fetch-browser
installs it through the same pin a released package carries.
Documents the release flow in ci/README.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* pythonlib: replace asyncio.to_thread so the 3.8 vermin gate passes
publish-pypi.yml checks the package with
`vermin . --eval-annotations --target=3.8 --violations camoufox/`, and
asyncio.to_thread (Python 3.9+) in _resolve_proxy_geo failed it, stopping
the 0.5.7 release. loop.run_in_executor does the same off-loop lookup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Pair with releases cut before the digest marker, and test the pairing against the step
The scope step now asks ci.release paired, which only knew releases whose notes
carry a source digest. None does yet: v156.0.1-beta.32, the release built from
main's sources, predates the marker. So every driver-only pull request would have
rebuilt the browser, the first merge would have cut a duplicate beta.33, and the
two scope tests in ci/tests/test_ci.py -- which ran the step in a scratch repo
where ci.release did not import -- failed.
find_paired falls back to the tag upstream.sh names when that release is
published (a prerelease counts; a draft does not) and no browser source changed
since, listing the files that did when they have. browser-plan and promote use
the same lookup. paired takes --root and --releases so the tests run the
workflow's own step against a scratch repo and a fixed release list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Release from one workflow, with trusted publishing
The release chain was four workflows linked by workflow_run, with the
tested commit carried between them as an artifact; a browser release
number committed beside main; pairing data in HTML comments in release
notes; a stored PyPI token; and packages rebuilt at each publish.
release.yml now does all of it with `needs`:
- On a push to main it calls tests.yml on the pushed commit (tests.yml
loses its own push trigger), then builds the browser only when its
sources changed, and publishes a library prerelease only when something
a package ships changed. Docs- and CI-only merges publish nothing.
- A browser release's number lives only in its tag, which points at the
tested main commit; `ci.release set-build` writes it into the build's
working tree. Nothing is committed.
- Each browser release carries a manifest.json asset (source digest,
commit), which is what a library pairs by. Builds are attested with
actions/attest-build-provenance.
- Both packages are built once, in build-library, and the publish jobs
upload exactly those files. PyPI and npm use trusted publishing; no
credential is stored.
- A vX.Y.Z tag builds and checks both packages before promoting the
paired browser and publishing.
- Every published library version is tagged (vX.Y.ZbN for a prerelease),
which is how the next merge tells whether the library changed.
- A failed publish is retried with "Re-run failed jobs"; the retry-only
workflow_dispatch path is gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
 coffeegrind123andClaude Opus 5.5
|
0cb8c60bb9 |
Pace humanized moves on their own schedule, and write toggle prefs on every launch (#808)
* fix(juggler): pace a humanized move on its own schedule sendTrajectoryAcked waited each point's full pause after the previous point's ack, so every ack's round trip was added to the move: it took its plan plus one round trip per point. On a page whose main thread is busy 19ms in every 20, moves capped at 0.5s took 0.54-0.71s. Each point is now due at its planned offset from the start of the move, so a late ack delays only its own point and the move ends on its planned time. tests/patches/humanize-pacing.py times eleven capped moves on such a page and checks their median against the cap. It failed 3/3 before (medians 587-607ms) and passes after (484-487ms). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(launch): write toggle prefs on every launch, on or off block_webrtc, block_images and disable_coop wrote their pref only when switched on. A persistent profile keeps a user.js pref in prefs.js, so removing the flag later left the setting in force. A real profile launched once with block_webrtc still had media.peerconnection.enabled false long after, and BrowserScan reported WebRTC disabled. Each pref is now written every launch, with the stock value when its flag is off. A caller's own firefox_user_prefs entry still wins. The same change is made in the TypeScript launcher, with its tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Jake WriterandClaude Opus 5.5
|
12a1bb4fc6 |
fix(fingerprints): a NewContext Linux identity is Linux in platform and oscpu too
fpgen's Linux pool now and then pairs a Linux user agent with
navigator.platform Win32 and a Windows oscpu. launch_options() corrects
that with fix_navigator_arch(); generate_context_fingerprint() never
called it, so 24 of 1,500 Linux NewContext identities (1.6%) said Win32
under a Linux UA -- and because that path reads the OS for fonts and
voices from the platform, they drew Windows fonts and voices as well.
It now applies the same fix right after the fpgen draw, in pythonlib and
in the TypeScript twin. The fix draws nothing, so the parity goldens are
unchanged.
The new tests feed the context path a real Linux draw with the Windows
platform and oscpu, and fail without the fix ('Win32' != 'Linux x86_64')
in both ports. After it: 0 of 1,500 sampled contexts mismatch.
pythonlib 412 passed; typescript 585 passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Jake WriterandClaude Opus 5.5
|
8081061156 |
fix(juggler): never enter Responsive Design Mode, and warn about is_mobile
#798 left RDM on for isMobile, as Playwright's Juggler does. RDM matches no real browser: Firefox for Android never runs it, and under touch emulation it drops a mouse click's pointer events, which no device does. Camoufox only has desktop identities, so an is_mobile context was a desktop UA, platform, fonts and GPU with devtools' mobile mode on top. Juggler now keeps inRDMPane off for every page, is_mobile included, and the isMobile plumbing #798 added is gone again. viewport-no-rdm requires is_mobile=True to keep the platform's scrollbars too. Both launchers warn instead (warnings.yml is_mobile): on new_page() / new_context(is_mobile=True) of a Camoufox browser, and on is_mobile passed to launch_options() for a persistent context. has_touch, device_scale_factor and viewport keep working without RDM. Upstream playwright-python skips its isMobile tests on Firefox in 1.61-1.63, so no skiplist entries are needed. On beta.31 with this Juggler in omni.ja, viewport-no-rdm passes: 12/12 px of scrollbar with no viewport, with a viewport, and with is_mobile, and a has_touch click fires pointerdown and pointerup. pythonlib 411 passed; typescript 584 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Jake WriterandClaude Opus 5.5
|
8823d399b8 |
fix(fpgen): keep a values.dat the TypeScript launcher decompressed from the pin
CAMOUFOX_FPGEN_DATA may point the TS launcher at pythonlib's fpgen data/, and it decompresses values.dat there. ensure_fpgen_model() refused to run whenever values.dat existed, so sharing the directory broke every Python launch. The pin now carries values.dat's sha256 (all three twins): a matching values.dat is kept, any other is removed, since fpgen reads it in preference to the verified archive. The files `fpgen decompress` leaves still fail loudly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
coffeegrind123
|
c2817c1cae |
fix(coherence): let macOS draw Intel Mac GPUs, checked as machines
coherence rejected "Intel(R) HD Graphics 400" and "Radeon R9 200
Series" on macOS as a Braswell Atom IGP and a desktop PC card. They are
Firefox's sanitized buckets, not devices: SanitizeRenderer
(FIREFOX_152_0_4_RELEASE) maps an Intel Mac's UHD 630 to the first and
a Radeon Pro 5300M to the second, per Firefox's own TestCiMac and
TestMacAmd. fpgen's pinned model records both from Firefox on macOS at
1.14% each. The rule kept every generated Mac off them, removed 24
real macOS presets in #779 (restored here, 9 + 15), and dropped either
GPU from a caller's Mac preset. ANGLE and llvmpipe stay rejected:
Firefox 152 has no ANGLE-on-Metal path (Bug 2046027 came later).
Intel Macs are now checked as machines instead (intel-mac-hardware),
for every non-Apple GPU on macOS:
- a core count some Intel Mac with that GPU reports. Firefox reports
physical cores where kern.tcsm_available is set and logical ones
otherwise, so either counts: 2-8 physical / 4-16 logical for the
IGP, up to the 2019 Mac Pro for a discrete GPU;
- a screen that is not a notched MacBook's or the 24" iMac's. 45.5% of
fpgen's Firefox macOS screens are one, and the draw already put an
Intel or AMD bucket behind 4.9% of them.
The WebGL draw applies the same check, given the core count. The preset
GPU drop moves next to the WebGL draw, after the host core count and
the display clamp: before, a preset's GPU was judged against cores the
launch then replaced. The TypeScript launcher gets the same changes,
and the goldens cover the new check and the narrowed draws.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
(cherry picked from commit
|
||
|
coffeegrind123
|
7f0e52e792 |
Install the pinned fpgen model before fpgen is imported
pythonlib left the model to fpgen, whose first import downloads the
first release the GitHub API lists: model-4/2025, whose WebGL records
have no vendor or renderer. Every generated launch on a fresh install
failed with KeyError: 'vendor'. scripts/pin-fpgen-model.py pinned the
model for CI only, and fpgen's five-week refresh replaced even that
pin, under a stamp the script's --check still trusted.
fpgen is now imported only through fpgen_model.load_fpgen(), which
installs the release named by the pin, checks the archive and each
file against their sha256, and dates the files past fpgen's refresh.
It writes the layout and stamp the script and the TypeScript launcher
use, verifies an install by hashing it, and leaves FPGEN_MODEL_URL to
fpgen. `camoufox fetch` installs the model as well. The pin gains each
file's sha256; the package carries a copy of it, and the script now
calls the module.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
(cherry picked from commit
|
||
|
Jake WriterandClaude Opus 5.5
|
180e6553cc |
Docs that match the code, one AGENTS.md, dead code removed, and the audit's bug fixes (#787)
* chore: remove build tooling nothing uses - The developer UI (scripts/developer.py, `make edits`). It depended on easygui, which no requirements file declares, and every action it offered is a Makefile target: patch, unpatch, workspace, revert, diff. Its two helpers in scripts/_mixin.py (is_bootstrap_patch, patch) had no other callers. - legacy/, the Go launcher deprecated in 2024-11. Nothing built or shipped it. Its Makefile targets and scripts/run-pw.py go with it, and so does Go from every dependency list and workflow. - jsonvv/ and settings/camoucfg.jvv. Nothing read the .jvv schema: config is validated against settings/properties.json, and the two had already drifted. The jsonvv package stays on PyPI. - Scripts with no caller: bootstrap.py, moztree, setup-wasi-linux.sh, package-helper.sh, install-local-build.sh, mozfetch.sh (copied into lw/ but never packaged), examples/. - The pre-ESM Juggler copies JugglerFrameParent.jsm and JugglerFrameChild.jsm, and hidden-scrollbars.css. Juggler loads the .sys.mjs actors and deliberately no stylesheet, but jar.mn still packaged all three. - patches/librewolf/*.opt, which list_patches() never picks up; the roverfox second pass in patch.py, whose directory no longer exists; the unread --no-settings-pane option. - The CAMOUFOX_PASSWD secret passed to `make fetch` and closedsrc_rev in upstream.sh, which nothing reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(python): remove dead helpers and a stale dependency None of these had a caller: - pkgman: is_supported_path, extract_zip, cleanup and set_version, left over from the single-directory install. cleanup() would have deleted every installed browser version. - multiversion.get_cached_repo_names, CONSTRAINTS.as_range, fingerprints._load_os_voices, utils._clean_locals, and unused imports. Also: - The "Apify Fingerprints" row in `camoufox version`, which has read "?" since fpgen replaced BrowserForge. - lxml is no longer a dependency; nothing imports it. - The geoip extra now names maxminddb, the module geolocation.py actually imports, rather than getting it transitively through geoip2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: show the cursor paths humanize=True actually produces The README's cursor video showed the Bezier generator Camoufox replaced with Cursory's recorded trajectories. scripts/cursor-demo.py drives a real build with humanize=True and records every mousemove event the page receives. It writes assets/humanize-cursor.svg, an animated replay at the recorded speed, so what the figure shows is what a site sees. The script cannot change the binary, so ci/browser_inputs.py lists it as non-native and editing it does not invalidate the cached browser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(python): stop naming BrowserForge in user-facing text fpgen replaced BrowserForge, but two LeakWarnings, the NonFirefoxFingerprint message and the fingerprint_preset docstring still named it. One warning also linked to a README anchor that no longer exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: one AGENTS.md for every agent, a roadmap, and docs that match the code - AGENTS.md holds the engineering rules for any coding agent, plus the repo map, build, patch and test commands that CLAUDE.md used to carry. CLAUDE.md now only imports it, so there is one set of rules. ci/tribal-rules.yml is the record of settled decisions it points to. - ROADMAP.md lists planned work, each item linked to its issue. - README: - fpgen and the coherence check replace BrowserForge; - the patch workflow uses the make targets instead of the removed developer UI; - letter-spacing noise is described as off by default, as it is. - docs/: - beta-testing-ff146.md removed; - patch-upgrading-guide rewritten around the make targets; - per-context-patches without the canvas patch that no longer exists, and with measured preset counts; - playwright-maintenance without the JSM wrapper that does not exist; - smaller fixes in MEDIA-DEVICES, input-dispatch and FONTS. - ci/README: every job, and the real shard, skiplist and entry-point lists. - pythonlib, tester and patch-dependency READMEs corrected against the code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(pythonlib): handle headless='virtual' in launch_server launch_server() is documented to take the same arguments as Camoufox(), but passed headless='virtual' straight to launch_options(), so the server launched with no Xvfb display. Start a VirtualDisplay the way Camoufox() does, launch headful on it, and kill it when the server process exits or the launch fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(python): remove fontprobe, which nothing called fontprobe listed the fonts installed on the host, for a `camoufox fonts` command that was never added. It has nothing to do with the font bundle Camoufox serves to pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(license): the Python launcher is MIT; the browser stays MPL-2.0 The Python package has always been published to PyPI as MIT (#727), but pythonlib/ shipped no licence file, and the repo's LICENSE is the browser's MPL-2.0. MPL is copyleft per file. It covers the modified Firefox sources, not a separate launcher that drives the browser over Playwright. So pythonlib/LICENSE now carries the MIT text its metadata already declares, and a Licensing section in the README says which part is which. Closes #727. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python): fingerprint_preset=False no longer turns presets on launch_options checked `fingerprint_preset is not None`, so passing False drew a random bundled preset, the opposite of what was asked. It now uses a truthiness check, and a test proves that None and False never draw a preset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: bring the release workflow in line with the tests build job build.yml had drifted from tests.yml. It ran actions at v1/v2 on a retired Node runtime, prepared the source tree with bare make calls that fail the whole release on one dropped connection, and built with a different Python than every pull request is tested with. - Pin every action by commit SHA, at the major versions tests.yml uses (checkout v4, setup-python v5, upload/download-artifact v4, the same remove-unwanted-software SHA), and action-gh-release v2. The release job holds contents: write, so it should not follow a movable tag. - Prepare the tree with `python3 -m ci.run_prepare`, as the tests build job does. BUILD_TARGET is set from the matrix so `make dir` writes the right mozconfig and Rust targets; multibuild.py then finds _READY and builds without re-patching. mach's toolchain bootstrap ignores the mozconfig, so running it after `dir` bootstraps the same toolchains. - Build with Python 3.12, the version the tests build job compiles with. - Default the workflow to no permissions; the build job gets contents: read and the release job keeps contents: write. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python): NewContext looks up a proxy's exit IP through the right URL, or fails NewContext derives the context's WebRTC IP and timezone from the proxy's exit IP. That lookup had two defects, and both left the context showing the host's values while its traffic went through the proxy: - It built its own proxy URL with urlparse, which reads a scheme-less server such as "1.2.3.4:8080" (a form Playwright accepts) as scheme "1.2.3.4" with no host. urllib could not use a SOCKS proxy at all. - Any failure was swallowed, and the context opened without the values. The URL is now built with Proxy.as_string(), which the geoip launch path already uses (scheme-less means http). The lookup goes through requests, which handles SOCKS, and a failed lookup raises InvalidIP, naming the two options that skip it. The tests cover scheme-less, http and socks5 servers with credentials, both failure modes, and the case where no lookup is needed, for NewContext and AsyncNewContext. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: stop generating a canvas seed, and drop config keys nothing reads The browser has not noised the canvas since #528, and no patch reads canvas:seed (#721). The launcher still drew one on every launch and sent it through CAMOU_CONFIG, and NewContext called a setCanvasSeed that does not exist. They no longer do. For users this changes nothing on any browser since #528: the value was ignored. A config that still passes canvas:seed gets the usual "Skipping unknown patch" notice instead of silence. On a browser from before #528, the launcher no longer turns canvas noise on, which is the behaviour #528 chose. The same audit found more keys declared in settings/properties.json that no patch or Juggler file reads, so setting them did nothing: - canvas:aaOffset, canvas:aaCapOffset - memorysaver, pdfViewerEnabled, webrtc:localipv4/6 - navigator.onLine, navigator.cookieEnabled, navigator.languages - navigator.appCodeName, appName, product, productSub. Firefox reports these constants itself, so fpgen.yml no longer maps them. - webGl:parameters:blockIfNotDefined and its WebGL2 twin test_config_schema now checks this direction too: every declared key must be read by the browser, unless it is listed with a reason. Three are listed: locale:script and navigator.doNotTrack, which the launcher applies itself, and navigator.buildID (#780). The build-tester grading followed the same wrong premise. It tracked canvas collisions as an unfixed per-context leak. A canvas that is rendered rather than noised follows the fonts and GPU, as it does on real machines, so canvas collisions are now counted with the other device-level values. The tribal rule that recorded it as an open question is now a settled one, canvas-is-not-noised, with an automated check. Closes #721. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python): repair devicePixelRatio the same way on every launch The DPR repair snaps an off-grid ratio to the nearest real scaling step and keeps the first of two equally near steps. The steps were frozenset literals, and a frozenset literal iterates in one order when the module is compiled from source and another when it is loaded back from a .pyc. So a midpoint such as 1.125 became 1.25 on the first launch after an install and 1 on every launch after it: the same pinned identity presented two different devicePixelRatio values. The steps are now ascending tuples, so a tie always goes to the lower step. The test runs the repair in two fresh interpreters that share a bytecode cache, compiling in the first and loading in the second. It failed before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: remove the glyph-spacing seed from the browser and the launcher anti-font-fingerprinting.patch added a seeded amount to every glyph advance, so that text widths differed per context. No real machine produces those widths: the same font on the same OS measures the same everywhere. So the noise was itself a fingerprint, measured in #779 at +1 px per ~100 glyphs plus fractional deltas on every measureText. #779 defaulted the seed to 0 and kept it as an opt-in, but an opt-in whose only effect is to become detectable is not worth carrying. Removed: - The browser side: - FontSpacingSeedManager and window.setFontSpacingSeed; - the HarfBuzz hook; - the plumbing that existed only to carry the context id down to the shaper: the userContextId on gfxTextRun, gfxShapedWord and the word-cache key, and the extra MakeTextRun argument in nsTextFrame, nsFontMetrics, MathML and canvas. The font group keeps its userContextId, which font-list-spoofing.patch uses to apply the per-context font list. Text is now shaped exactly as stock Firefox shapes it. - The fonts:spacing_seed key. The launcher had been sending 0 on every launch, plus a setFontSpacingSeed(0) call in every context's init script. - tests/patches/config-overrides.py, which tested only the spacing override. A pythonlib test now covers config_overrides with another key. timezone-spoofing, webrtc-ip-spoofing and window-setter-seal change only in context lines and the setter seal list. Every patch applies cleanly to a fresh tree, and the result builds. The settled decision is recorded as no-glyph-spacing-noise, with an automated check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: stock animations and speech by default; drop config keys that freeze live values Three behaviours a page could detect, changed in one breaking release: - **Animations run on stock timing.** no-css-animations.patch finished every finite animation at once by default, and any page could read it: `el.animate(frames, 1000).effect.getComputedTiming().duration` was 0, and a 500ms transition reported 0. Measured on v152.0.4-beta.31. The speedup is now an opt-in, `instantAnimations: True`, which raises a LeakWarning. disableInstantAnimations is gone. - **speak() on a spoofed voice works like a real voice.** It fired `error` after 3ms unless voices:fakeCompletion was set, and then start and end in the same tick. It now starts and ends after the text's duration at ~150 words per minute. Both voices:fakeCompletion keys are gone, and so is a debug line printed to stderr on every call. - **Keys removed:** - battery:* and window.scrollMinX/Y: Firefox keeps getBattery() and scrollMin* chrome-only, so no page could read them. - window.scrollMaxX/Y, screen.pageXOffset/pageYOffset, window.history.length and document.body.client*: each pinned a live value to a constant, so scrolling, navigating or re-laying out never changed it. fpgen.yml mapped pageYOffset, so about 15% of identities froze window.scrollY at a non-zero value. - The body keys' role as an undocumented alias for window.innerWidth/Height in browser-init and in the launcher. - MaskConfig::GetInt32Rect, which only the body keys used. New guards, both of which fail on v152.0.4-beta.31: tests/patches/animation-timing.py and tests/patches/spoofed-voice-speaks.py. The decisions are recorded as animations-run-on-stock-timing and spoofed-voices-speak. Every patch applies cleanly to a fresh tree, and the result builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python)!: remove dead public API and make `list all --path` work Breaking changes: - Remove the exceptions UnknownProperty, InvalidDebugPort and MissingDebugPort. Nothing in the package raises them, so code catching them was catching nothing. - Remove the legacy `allow_webgl` keyword of launch_options(). Use `block_webgl=True`. The keyword now reaches Playwright as an unknown launch option and fails there instead of being silently consumed. `camoufox list all --path` accepted the flag and ignored it. It now prints the install path beside each installed build, as `camoufox list --path` already does for the installed tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(python)!: drop data the package never draws from voices.json shipped in the wheel, but no code in the package reads it: the voice draw uses voice-manifests.json and voice-uris.json. Its only readers are the TypeScript port's golden-fixture generator and data-sync script (typescript/scripts/golden/identity_golden.py, typescript/scripts/sync-identity-data.py), which live on another branch and will need a new source; the last copy is at 676fb3f:pythonlib/camoufox/voices.json. docs/per-context-patches.md described it as runtime data and now describes the files that are. webgl_data.db held two rows with zero weight on every OS ("Intel(R) HD Graphics 400, or similar" from "Intel Inc." and "Radeon R9 200 Series, or similar" from "ATI Technologies Inc."), left behind when their impossible macOS weights were zeroed. No draw can reach them. They are deleted with secure_delete so their blobs do not linger in free pages; the file is not vacuumed, so the other pages are unchanged. A new test requires every row to be drawable on at least one OS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python): warn whenever an identity falls back to a substitute value Several draws swallowed their failure and used something else, so an identity could ship with values the rest of it was not drawn to match and nobody would hear about it: - from_preset(): a failed font or voice draw used the preset's recorded list, or nothing, on any exception. - generate_context_fingerprint(): a failed font, voice or WebGL draw was `except Exception: pass`, leaving the browser's launch-time values. - _load_font_groups() / _load_font_bases(): an unreadable file became {}, i.e. no font additions or no OS-version base. - launch_options(): a failed font draw used every font in fonts.json, a failed voice draw used no voices, and a preset GPU missing from webgl_data.db was silently swapped for a drawn one (36 of the 397 bundled presets). Each site now catches only the errors its data can raise (OSError and ValueError for an unreadable or corrupt file, KeyError for a manifest with no entry for the OS, sqlite3.Error for the WebGL database) and emits a FallbackWarning. The text names what failed and what the identity uses instead, then gives a block to paste into an issue (camoufox, browser, OS and Python versions, the error, and the identity's user agent or GPU), asking the user to report it on GitHub. It shares LeakWarning's caller-frame attribution and its template lives in warnings.yml. The broad excepts had also been hiding a broken fixture: test_launch_environment's font and voice stubs did not accept `seed`, so every draw there raised and was swallowed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python): give NewContext identities the browser's Firefox version NewContext() and AsyncNewContext() passed ff_version=None through to generate_context_fingerprint(), so a context's user agent kept the version fpgen drew (e.g. Firefox/146) while the browser underneath was 152. They now default ff_version to the major version of Playwright's Browser.version, which Juggler reports from MOZ_APP_VERSION_DISPLAY, so the UA always names the browser the page is actually talking to. An explicit ff_version still wins. The docstrings said each context gets "its own real fingerprint preset"; the default has been an fpgen draw, with a preset only when one is passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python): send an IPv6 WebRTC address to setWebRTCIPv6 The per-context init script passed every webrtc_ip, IPv6 included, to window.setWebRTCIPv4(), and never called setWebRTCIPv6(). An IPv6 address (given directly, or resolved as a proxy's exit IP) was stored as the context's IPv4 value and the IPv6 slot stayed empty. The script now picks the setter by address family, and an address that is neither raises InvalidIP instead of being passed through. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python): stop pinning the page's scroll offset from fpgen fpgen.yml mapped the drawn window.pageYOffset (e.g. 528) to screen.pageYOffset, and the browser returns that value from scrollY on every read, so a page saw one scroll position forever whatever the user did. Real scroll offsets are live page state, not part of a device's fingerprint, so neither offset is mapped any more. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(python): stop checking a config key that no longer exists warn_manual_config() looked for navigator.languages, which was removed from settings/properties.json; validate_config() rejects it before the check could matter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(python): close the WebGL database connection on every path sample_webgl raised its not-found and wrong-OS errors before reaching conn.close(), leaking a sqlite connection each time a preset named a GPU the database does not hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(data): drop the 23 presets whose GPU has no WebGL data A preset records only its GPU's name. The WebGL parameters, extensions and shader precision behind it have to come from somewhere, and for these 23 nothing Camoufox has describes the GPU: fpgen has never seen Firefox report it on that OS. So each launch paired the name with another device's parameters, a mismatch any WebGL fingerprinter can see. They were: - Windows on ARM (Adreno 650); - Direct3D 10-level GPUs (vs_4_0/vs_4_1); - "Generic Renderer"; - 945GM and GTX 480 on macOS; - nouveau/Mesa buckets on Linux; - one Linux preset pairing NVIDIA's proprietary vendor string with the nouveau renderer name. scripts/clean-fingerprint-data.py now applies the rule, via a shared fingerprints.firefox_gpus(), and test_shipped_data asserts it. 374 presets remain, and every OS keeps its presets. ROADMAP.md lists capturing WebGL data for these GPUs, which would bring them back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(python): draw every identity's WebGL from fpgen WebGL vendor, renderer, context attributes, extensions, parameters and shader precisions, for WebGL1 and WebGL2, now come from fpgen's recorded Firefox devices instead of webgl_data.db, which is deleted with the camoufox/webgl/ package. camoufox/webgl.py: - webgl_for_gpu() traces `webgl` given Firefox, the OS and the GPU, then `webgl2` given the chosen `webgl` too, and draws each with one seeded random.Random. The GPU and the webgl value are pinned by their fpgen lookup index: a dict condition is flattened into leaves that overwrite each other, so only the renderer applied and Linux "Mesa" and "AMD" Radeon HD 3200 devices came back mixed. - sample_webgl_for_screen() draws the GPU of a generated identity from fpgen's per-OS weights, filtering out software rasterisers, GPUs the OS cannot report, discrete GPUs behind a netbook screen and the resistFingerprinting "Mozilla" mask before the weighted choice, so there is no rejection loop. An empty pool raises. - The draft/host-dependent extension filter moves over unchanged. A preset's GPU and a caller's webgl_config pair are looked up as given; a pair fpgen has never seen from Firefox on that OS raises instead of falling back to another GPU. generate_context_fingerprint no longer falls back to the host GPU when the draw fails. For 10 of the 15 (GPU, OS) pairs the two sources share, one of fpgen's records converts to exactly the database row on every value the browser reads. The other five rows (Linux R9 200 and Radeon HD 3200, macOS Intel HD, and two software rasterisers) are devices fpgen does not carry; those GPUs now present fpgen's recorded devices instead. The Linux GTX 980 row is kept as a test fixture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: say where WebGL comes from now that the database is gone The per-context guide, the fpgen.yml header and coherence's comments still named webgl_data.db and sample_webgl(). They now point at camoufox/webgl.py and fpgen. The guide also claimed presets carry WebGL parameters; they record only the vendor and renderer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(patches): a host's missing speech daemon no longer errors spoofed speech On a Linux host where speech-dispatcher cannot start, Firefox broadcasts synth-voices-error, and SpeechSynthesis answers it by firing `error` on every queued utterance. So a spoofed Windows voice errored about 11ms into speak() on any host without the daemon: the CI runners, and most servers. It passed only where the daemon runs. While Camoufox manages the voice list, the registry no longer forwards a host backend's error. The spoofed voices do not depend on the host's engine, and a Windows or macOS identity never raises one. The guard now makes the daemon unreachable itself, so it tests this case on every machine; on the previous build it fails every time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: stop skipping the two click tests that stock animation timing fixed test_wait_for_stable_position and test_timeout_waiting_for_stable_position were skipped with humanized travel time as the reason. The real cause was instant animations. Every finite animation finished at once, so the button Playwright waits on to stop moving never moved, and the click landed where upstream does not expect. With animations on stock timing both pass, and the skiplist audit flagged them as no longer failing. The entries go, and the counts in ci/README.md drop from 14 to 12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(build-tester): accept 18 and 22 cores, as real hardware reports plausibleHWC's list of common core counts lacked 18 and 22 -- Intel Meteor Lake laptops (Core Ultra 5 125H, Core Ultra 7 155H), and 22 is in 8 recorded presets. build-tester draws random presets, so a run that picked one of the two Linux presets reporting 22 failed: about one run in eleven, on any pull request. A CI self-test now fails if the list rejects any core count pythonlib can present (the presets and PLAUSIBLE_CORE_COUNTS). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(guards): judge the query-cost probes on a median, not one sample stock-parity-probes timed each getter once. On a shared runner one GC pause or CPU-steal spike decided the verdict: navigator.hardwareConcurrency took 77 ms against a 50 ms allowance on the same restored build that passed the run before. Each pair is now timed five times, interleaved, and compared by median. The regressions these catch (a sync IPC per read, ~240 ms over the loop) cost extra on every read, so they move the median; verified by giving the getter a constant ~4 us of extra work per read -- 86 ms median, FAIL -- while the healthy build passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(native): compare the whole fingerprint when two launches must differ test_two_browsers_get_different_fingerprints compared seven coarse values: UA, platform, screen size, core count, timezone and language. CI pins the timezone and language, and real machines share the rest: two draws of a common Mac (Firefox 152, MacIntel, 2560x1440, 8 cores) matched, and the test failed on a correct browser. It now reads the whole fingerprint a site computes, from a script in the page: - navigator values, screen and window geometry, device pixel ratio, timezone; - WebGL vendor, renderer, limits and extensions; - installed fonts, measured by width against the generic fallbacks; - voices, media-device counts, and an OfflineAudioContext hash. The page is served from an https URL Playwright fulfils locally, because mediaDevices exists only in a secure context. The page computes the result itself because the isolated world may not read audio sample data. The test then requires the fingerprints to differ, and the audio hash to differ on its own, since its noise is seeded per identity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Update README to remove warning, camoufox is now actively maintained Camoufox will now be actively maintained and improved for the foreseeable future --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Jake WriterandClaude Opus 5
|
6daae88dbc |
Stock-Firefox parity for native identities: input, fonts, locale, WebGL, WebRTC, media, timing, launcher (#779)
* feat(humanize): replay recorded human mouse movements (Cursory) humanize=True used to walk a Bezier curve through two random knots and emit a point every 10 ms. Both halves are tells: an analytic curve sampled at a fixed rate has velocity and jerk profiles that separate cleanly from a hand's, and every movement accelerated through the same easing function. Juggler now picks one of Cursory's 2357 recorded human movements whose direction, distance and wander suit the move, morphs it onto the requested endpoints and replays it with the recording's own timing. The generator is cursory-js (a bit-exact TypeScript port of Vinyzu/cursory) vendored under additions/juggler/input/cursory/; it is LGPLv3-or-later, not MPL-2.0, and ships its LICENSE and NOTICE inside juggler.jar. MouseTrajectories.hpp and ChromeUtils.camouGetMouseTrajectory are removed. sendTrajectoryAcked takes per-step pauses, drops points on the pixel the last dispatch left the cursor on (a zero-displacement move is never acked), and the humanize guards are updated for the new path shape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(patches): shared helpers for binary resolution, a private Xvfb and Marionette resolve_binary() honours the runner's CAMOUFOX_EXECUTABLE_PATH before falling back to a Linux objdir (search-service-init and touchscreen-digitizer ignored it and ran the newest objdir, which after a macOS cross build is an arm64 Mach-O), hidden_display() gives a guard its own Xvfb so nothing ever opens on the user's display, and a minimal chrome-context Marionette client lets guards inspect browser UI state. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(juggler): synthesized input carries what real mouse and keyboard input carries - pointerType was "" for every Playwright mouse event: juggler dispatched with MOZ_SOURCE_UNKNOWN. It now passes MOZ_SOURCE_MOUSE (#776). - keyboard.type() never pressed Shift: a shifted character now arrives bracketed by ShiftLeft keydown/keyup (location 1) with shiftKey set. - After the pointer was parked off content, pointerover/enter re-entered with buttons=1 and pressure 0.5; the tracked position is now forgotten on park. - A Windows identity gets contextmenu after mouseup with buttons=0, as Windows does; GTK/macOS keep it on press. - Wheel events are sent as line deltas (DOMMouseScroll.detail 3 per notch instead of the pixel count). - The browser rect is measured after the APZ flush await, so a chrome height change during the wait cannot put a y==0 dispatch one row above content. - ci/run_sundial.py moves and clicks the mouse so input vectors have data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(juggler): evaluate() no longer grants user activation Upstream Playwright runs every evaluate() as handling user input and notifies a user-gesture activation. Init scripts go through that path at load, so every page started with navigator.userActivation.hasBeenActive === true, autoplay allowed and popups permitted before any input. Activation now only comes from juggler's trusted input events, as in a stock browser. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(juggler): stop hiding scrollbars in headless The headless agent sheet set scrollbar-width: none !important, which a page reads back from getComputedStyle and from overflow:scroll gutters. Scrollbar appearance is left to the platform look-and-feel (the launcher sets ui.useOverlayScrollbars per claimed OS). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ui): no visible automation cues in the browser window - Every Playwright context was a public container, so the URL bar showed a "JUGGLER <id>" label and a container colour. Contexts are now non-public identities (tabbrowser renders public identities only); startup cleanup still removes persisted leftovers. - showcursor defaulted to true, drawing a red dot that followed the mouse. It is now opt-in. tests/patches/visible-automation-cues.py checks both on a private Xvfb. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fonts): web fonts, local(), per-character fallback and native bundles - FontFace / @font-face were answered from the font allowlist by the FontFace's own family name, so every url() web font failed with NS_ERROR_FAILURE and never rendered, local() of an allowed font failed, and a miss rejected with an XPCOM code instead of NetworkError (#759). Stock FontFace/FontFaceImpl are restored; local() is filtered by the RESOLVED family in gfxUserFontSet. - GlobalFontFallback forced the cmap scan, which skips families whose charmap is not loaded yet, so any character outside Gecko's script-based common-fallback table rendered as the primary family's .notdef (U+1E9E on macOS). The platform fallback chooses again, and its choice is held to the mask. - For a native macOS/Windows identity the bundled font sets are not activated: a bundled face of a family the system also has (Papyrus, Helvetica) won the lookup with different metrics. On Windows the enumerator still keeps Twemoji Mozilla, the emoji font stock Firefox ships (flag emoji drew nothing). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fonts): CSS2 system fonts and system-ui follow the claimed OS - The host's own OS gets no system-ui override (macOS resolved system-ui to Helvetica instead of -apple-system). - CSS2 system font keywords use per-keyword faces and sizes; a Linux identity reports the Ubuntu desktop font; Windows form controls (-moz-button/field/list) answer "MS Shell Dlg 2" as Windows does, not Segoe UI. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(fonts): per-OS font model and fontconfig parity fonts.json is now generated from the bundle by scripts/gen-fonts-json.py (fc-scan + aliases + scan-time families, intersected with the per-OS manifest in scripts/data/font-manifests.json) so every reportable family is renderable; scripts/verify-fonts.py checks that invariant, the generics and the reject globs. font-groups.json lets the draw keep co-shipped families together. Linux fontconfig: stock metric aliases (Arial -> Liberation Sans, ...), 49-sansserif, urw-base35 and the non-Latin rule files in stock conf.d order, generics resolving like a stock Ubuntu (Noto Sans / Noto Serif / DejaVu Sans Mono / Z003), hintslight so advances are not pinned to whole pixels, and weak <prefer> lists instead of strongly-bound generic pins so lang can promote a script face. Windows fontconfig: GDI substitution aliases, MS Shell Dlg 2, cursive/fantasy generics, duplicate-face rejects and Sitka / Segoe UI Variable optical-size families. NOTE: generated against a ~3.9 GB target font bundle that is not part of this change (one file is over GitHub's 100 MB limit); see docs/FONTS.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(locale): localize browser strings with the spoofed locale; stop rewriting explicit locales - With locale="fr-FR", Intl/number/date went French but input.validationMessage and XML parse errors stayed English, a mix no real Firefox produces. Official language packs are now baked in as packaged locales (scripts/fetch-langpacks.py, scripts/inject-locales.py, called by package.py, fetched on demand) and the launcher selects the UI locale through intl.locale.requested. A langpack add-on cannot do this: the parent pre-creates those string bundles first. - locale-spoofing.patch overrode Language/Script/Region on every intl::Locale, so new Intl.DisplayNames(['en'],{type:'region'}).of('DE') returned the spoofed region's name and Intl.Locale('ja-Jpan-JP').minimize() returned the spoofed tag. Only the OS/default locale is spoofed now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(media): enumerate, capture and label the identity's media devices coherently The fake media engine now enumerates the identity's microphones, cameras and speakers (labels and group ids from new mediaDevices:*Labels/*Groups config keys), MediaManager uses it whenever mediaDevices:enabled, and stock exposure rules apply: before a grant one device per input kind, no outputs, no labels; after a grant OS-style labels, distinct deviceIds, shared groupIds. So enumerateDevices(), getUserMedia() tracks and getSettings() ids agree, and a claimed camera captures instead of throwing NotFoundError. Fixes the content-process crash on an identity with a camera and no microphone (InsertElementAt on an empty array). docs/MEDIA-DEVICES.md; guard tests/patches/media-devices-coherence.py. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(navigator): globalPrivacyControl agrees between window and workers (#760) The main-thread Navigator getter ignored the config key that WorkerNavigator::GlobalPrivacyControl honours, so a page read false in the window and true in a worker. Both read the key the same way now; the launcher also mirrors it into privacy.globalprivacycontrol.enabled so the Sec-GPC header agrees. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(timezone): apply the launch-level timezone from the first read (#773) The timezone config key was only applied lazily from a navigator getter, so Intl and Date reported the host zone until a page happened to touch navigator. It is now applied eagerly in every process (nsJSContext::EnsureStatics) and per realm when a new inner window is created, entering that window's realm rather than whichever one triggered the navigation. window.setTimezone() still takes precedence per context. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(screen): the CSS color media feature follows the spoofed colorDepth screen.colorDepth was spoofed at the WebIDL level only, so on a 10-bit panel a 24-bit identity reported 24 with (color: 10), a pair Gecko cannot produce. Gecko_MediaFeatures_GetColorDepth now resolves the depth in the same order as nsScreen::PixelDepth. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(webgl): pass live state through instead of answering it from the table getParameter answered everything from the sampled table, so state a page had just set read back wrong (lineWidth(5) read 1, VIEWPORT/SCISSOR_BOX stayed 300x150 on a 64x64 canvas), extension parameters were null (anisotropy, draw buffers), COMPRESSED_TEXTURE_FORMATS was null instead of [], and getContextAttributes() ignored the attributes requested ({antialias:false} still reported 4 samples). Identity and limits still come from the table; live state and context attributes are real. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(webrtc): ICE gathering completes behind a proxy (#774) With Playwright's per-context proxy and media.peerconnection.ice.proxy_only_if_behind_proxy, ICE failed before gathering started and iceGatheringState stayed "new" forever, where stock Firefox completes with host candidates. When that happens around the fabricated candidates the new -> gathering -> complete state walk is replayed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(patches): refresh window-setter-seal.patch offsets Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(windows): embed Firefox's application manifest in camoufox.exe config/rules.mk embeds <program>.manifest and browser/app only ships firefox.exe.manifest, so --with-app-name=camoufox produced an exe with no manifest. Without the Windows 10 supportedOS GUID the process and its children run as a pre-Windows-10 application and Gecko's Windows-10-gated paths switch off (MediaCapabilities.decodingInfo powerEfficient false for H.264/VP9 where stock is true). The new patch adds a byte-for-byte copy as camoufox.exe.manifest; the old rename hunk in windows-theming-bug-modified.patch is dropped. Guard: tests/patches/windows-exe-manifest.py. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): stock values for page-observable prefs; launcher prefs at startup Page-observable defaults that no stock Firefox has, restored: - the forced built-in dark theme (it also removed the 1 px nav-bar separator, and was applied ~1 s after startup, resizing the viewport) and ui.systemUsesDarkTheme (prefers-color-scheme disagreed with the desktop); - focus rings off, autoplay allowed, popup blocker off; - gfx.color_management.mode=0 (Playwright's test pref: ICC-tagged images were drawn unconverted, readable from a canvas pixel); - ui.use_standins_for_native_colors (non-native system colours); - GMP updates off (Widevine/OpenH264 never available); - storage.estimate() quota derived from the raw disk instead of the stock cap. The HardwareAcceleration:false enterprise policy is removed: it locked software WebRender with no hardware video decoding on every OS (guard tests/patches/hardware-acceleration-policy.py). The minimal-theme chrome.css is emptied: its ~55 px chrome made outerHeight - innerHeight impossible. Playwright's non-persistent launch writes no user.js, so launcher prefs only arrived through juggler after startup and anything Gecko reads while starting raced (on Windows the UI locale lost 3 of 4 launches). camoufox.cfg now applies the launcher's CAMOU_PREFS_1..N env chunks as default prefs at startup (guard tests/patches/startup-prefs.py). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(branding): chrome://branding assets match stock Firefox chrome://branding/content/ is content-accessible. The wordmark SVGs had different intrinsic sizes (336x48 / 172x48 vs 300x67) and document.ico, document_pdf.svg and the private-browsing about logos were missing, all measurable from a page with an <img>. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(pythonlib): identity draws that match real machines and stay stable Launcher-side fixes found by comparing camoufox against stock Firefox 152.0.4 on Linux, Windows 11 and macOS hosts: - DNT / GPC: BrowserForge draws doNotTrack "1" on most Firefox samples, but a stock Firefox 152 reports "unspecified" and globalPrivacyControl false; the stock defaults are used unless the caller sets them, and both are applied as prefs so the API, the worker and the DNT / Sec-GPC headers agree (#760). - Timezone and geolocation: the timezone is passed to the browser, and a configured position sets permissions.default.geo so permissions.query agrees with the auto-grant (#769, #773). - hardwareConcurrency: the reported count is the fingerprint's and the browser is pinned to that many cores (cpu_affinity.py, Linux/Windows), so worker timing agrees with it; otherwise the host count snapped into the core counts real machines ship with (never 2, Firefox's resistFingerprinting value). - Fonts: the OS base is always present in full, OS-version variants are drawn all-or-nothing, co-shipped groups stay together, Cascadia is never claimed off Windows, a native macOS/Windows identity claims only the real OS base, and gfx.font_rendering.fallback.async is off on Linux so per-character fallback does not depend on cmap-load timing. - Speech voices: a per-OS installed-voice model (voice-manifests.json) with the voiceURI formats each backend really produces (voice-uris.json); no default voice where stock has none. - WebGL: extensions a release Firefox never exposes are filtered, but OVR_multiview2 stays for Windows D3D11 renderers, which expose it. - Media devices: a seeded draw of common per-OS devices with OS-style labels. - Windows scrollbars follow the drawn Windows version (overlay on 11). - Glyph-advance perturbation (fonts:spacing_seed) defaults to off: it moved every measureText width off the value the same font gives on a real machine. - Launcher prefs are also exported as CAMOU_PREFS_1..N so camoufox.cfg applies them at startup, and the browser UI locale follows the spoofed locale. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(pythonlib): per-identity salt for seeded draws; core pinning under concurrency Found in review of the previous commits: - identity_seed() hashed only the UA, platform, screen size and core count. Those take a handful of values per OS, so over 500 launches the seed took 12-30 distinct values and every install drew its fonts, voices, GPU, media devices and canvas/audio noise seeds from that same short list. The seed now mixes in identity_salt(): derived from what the caller pinned the identity with (a Fingerprint, a preset dict, a config naming the UA) so relaunching that identity reproduces every draw, and random otherwise. A pinned preset now reproduces its noise seeds too; seeds the caller sets are kept. - Concurrent AsyncNewBrowser launches on one driver interleaved pin/restore: one browser inherited the other's mask and the driver could stay pinned. pin -> launch -> restore is serialized per driver. - Every pinned browser landed on cores 0..N-1; pins now take N adjacent cores from a random start. - A pinnable host with 1-3 cores reported 1, 2 or 3 (2 is the resistFingerprinting value); the table floor of 4 applies as on other hosts. - launch_options() callers that launch the browser themselves (launch_server, direct use) kept the drawn core count although nothing pins the browser; only Camoufox/AsyncCamoufox pass pin_cpu_cores=True now, everyone else reports the host's snapped count. - PLAUSIBLE_CORE_COUNTS gains 18, 22, 28 and 32, all recorded in the -v150 corpus. - The Windows voice list was drawn before the locale was resolved, so an fr-FR identity got en-US voices; it is drawn after locale/geoip now. - macOS "Alex" gets its com.apple.speech.synthesis.voice identifier. - CAMOU_PREFS env chunks are ASCII-only JSON (Windows getenv goes through the ANSI code page). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fonts): canvas accepts CSS2 system-font keywords; local() works on macOS - GetSpoofedSystemFontForRFP's per-OS branches returned before marking the result a system font. ComputeSystemFont copies that flag into FontFamilyList::is_system_font, and without it the canvas font setter could not serialize the value: ctx.font = 'caption' (or icon, menu, message-box, small-caption, status-bar) was silently ignored and read back '10px sans-serif' where stock reads back the keyword. - CoreTextFontList::LookupLocalFont builds a CTFontEntry with no family name, and local() sources are held to the spoofed font list by the resolved family, so on macOS every local() face (Helvetica, Menlo, Arial...) failed with NetworkError, installed and allowed or not. The entry now carries the family CoreText resolved. A blocked lookup's entry is released instead of leaked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(webgl): only device limits come from the spoofed table getParameter still answered ~100 state pnames from the table, so state the page had just changed read back wrong: UNPACK_FLIP_Y_WEBGL / PREMULTIPLY_ALPHA / COLORSPACE_CONVERSION after pixelStorei, FRAGMENT_SHADER_DERIVATIVE_HINT after hint(), DRAW_BUFFERi after drawBuffers(), RED/ALPHA/DEPTH/STENCIL_BITS and IMPLEMENTATION_COLOR_READ_* for the bound framebuffer, and COMPRESSED_TEXTURE_ FORMATS after enabling an extension. UNMASKED_VENDOR/RENDERER_WEBGL came back without the extension enabled, where stock returns null with INVALID_ENUM. The table now answers only the MAX_*/ALIASED_*/SUBPIXEL_BITS limits, WebGL 2 limits on WebGL 2 contexts only, and extension limits (anisotropy, draw buffers, OVR multiview) only once that extension is enabled; everything else is the real context's answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(webrtc): fabricate candidates only where a real gather would have them With webrtc:ipv4/ipv6 set (every geoip launch), new RTCPeerConnection() with no iceServers produced a srflx carrying the spoofed IP, and after end-of-candidates a second host set with a different mDNS name. getStats() exposed that srflx as id 'camou-srflx' and rewrote every candidate address, including .local host names and the remote peer's candidates. A srflx is now fabricated only when the page configured an ICE server, host candidates only when none reached the page (sharing the real UDP host's port otherwise), the synthetic stats id has the shape real candidate ids have (8 hex digits, fixed per connection), and only this side's non-mDNS addresses are rewritten. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(media): honour mediaDevices:enabled=false; page fake:true behaves as stock - MaskConfig::GetBool returns std::optional<bool>, and the checks tested its presence: "mediaDevices:enabled": false still enabled the fake devices. - media.navigator.permission.fake=true was page-readable: a page's own getUserMedia({video: true, fake: true}) prompted and never resolved, where stock resolves at once with its generic fake device. The pref is off again; the identity's devices count as real hardware in the capturing checks instead (prompt, sharing indicator, post-grant labels), and a page's fake:true request gets stock's generic devices rather than the identity's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(storage): per-context values are session state, and misses are cached - Values lived on the user pref branch, which a persistent profile writes to prefs.js: relaunching with a different timezone (or navigator values) kept reporting the previous session's in the page, iframes and workers. They now live on the default branch, which is never saved, and reads ignore user values an older build left behind. - A read of an unset key did a synchronous IPC to the parent every time, and in a launch without per-context values every read is unset: navigator.hardwareConcurrency, screen.* and (color) media queries measured ~20x slower than stock. A miss is now cached per key until a pref change or a local put clears it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(timezone): no per-realm override for the process-wide zone; cache DateTimeInfo With a launch-level timezone every new document and worker got a per-realm override of the zone the process already reported. Setting one releases all JIT code in the runtime (hot code after adding an iframe ran ~4x slower), and the realm rebuilt its DateTimeInfo on every call (getHours() ~40x slower than stock). The override is applied only when the zone differs from the one JS::SetTimeZoneOverride applied process-wide, and a realm keeps its DateTimeInfo until its override changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(juggler): wheel scrolls in native notches; Shift leads the key it modifies - A wheel notch now reaches the page as its own 3-line event carrying one native tick (new WHEEL_EVENT_NATIVE_NOTCHES option in patches/wheel-native-ticks.patch), so wheelDelta is -120 per notch as with a physical wheel; it was -396, and a multi-notch scroll arrived as one event. Several notches are spaced a few tens of ms apart. - Auto-Shift pressed Shift 0 ms before the character's keydown and released it 0 ms after its keyup; it now leads and trails by a drawn human-scale delay, and a failing keydown no longer leaves Shift latched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): stock cookie partitioning, preconnect, popup notification; about dialog CSS - network.cookie.cookieBehavior 4 (Playwright's) -> Firefox's default 5. With 4 a cross-site iframe (captcha and anti-bot widgets are exactly that) sees document.hasStorageAccess() true and its first-party cookies and localStorage, where stock partitions them. Playwright set 4 so storageState need not carry thirdPartyCookie^ permissions. - network.http.speculative-parallel-limit 0 turned <link rel=preconnect> into a no-op, visible in Resource Timing. - privacy.popups.showBrowserMessage false: stock shows a notification bar for a blocked popup, which shrinks the viewport and fires resize. - chrome://branding/content/aboutDialog.css is page-loadable and was empty; it is the official branding's now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(patches): stock-parity probes for the leaks found in review One launch (plus two persistent relaunches) checks page-observable invariants stock Firefox 152 holds: canvas CSS2 system-font keywords, WebGL state readback and UNMASKED_RENDERER without the extension, no srflx without ICE servers and no 'camou' stats id, getUserMedia({fake: true}), cross-site storage partitioning, wheel notches, per-read cost of (color)/hardwareConcurrency and of local Date getters under a launch timezone, and a persistent profile's timezone after relaunch (page and worker). Run against the build before these fixes it fails on every one of them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(humanize): judge cadence by distinct values and spread, not a share of the gap count The page clock is clamped to 1 ms and Cursory's recorded steps mostly sit between 12 and 20 ms, so the number of distinct gap values cannot grow with the number of gaps. Requiring len(gaps) // 4 made the guard fail on visibly uneven runs whenever event delivery was steady (3 of 4 runs once the per-read sync IPC jitter was gone). A fixed 10 ms cadence yields about three values within a few ms of each other, which the new rule (>= 6 values, >= 20 ms spread) still fails. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(juggler): restore the popup, wheel and history contracts The Playwright suite went red on this branch, and five of its six shards ran out their 40-minute budget before reporting, so ~470 sync tests were never run at all. Three causes, all ours: - window.open() from page.evaluate() returned null. The popup blocker being ON (Firefox's default) and evaluate() no longer granting user activation are each defensible alone; together they block every gesture-less popup. ~35 tests, each burning 30s x 4 attempts x 2 worlds, which is what exhausted the shards. The blocker goes back to Playwright's and geckodriver's value. Reading it costs a detector a popup window the user sees, so it is not a check an anti-bot script in the page runs -- unlike navigator.userActivation.hasBeenActive, which is one property read, and which is why the activation half stays. - mouse.wheel(0, 100) delivered deltaY 114 (or 132, depending on the host's font metrics) in deltaMode 1. Quantising into native wheel notches is what a physical wheel does, but it changes the number the caller asked for, so it now rides behind humanize= with the rest of the humanized input. Default is the exact requested delta in deltaMode 0. - page.go_back() did nothing after history.pushState(). canGoBack is the BACK BUTTON's answer: under browser.navigation.requireUserInteraction it reports false when every entry behind this one was pushed without the user touching the page, which is now every entry, because evaluate() grants no activation. goBack() itself does not skip those entries and neither does history.back(), so ask canGoBackIgnoringUserInteraction, as Marionette does. Two keyboard tests are skiplisted rather than fixed: auto-Shift means typing "!" emits the Shift a US keyboard requires, and upstream asserts the character's three events with shiftKey false throughout. The character's own key/code/keyCode are unchanged; what upstream asserts is the absence of a Shift no real typist could omit. Full suite against the fixed build: 2223 passed, 6 failed -- the two keyboard tests above, and four client-certificate tests that fail only on this machine (Node/OpenSSL rejects the fixture server) and pass in CI. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(pythonlib): stop overriding the corpus on core counts; pinning is opt-in Two findings from auditing the sweep's fixes against one bar: a difference is a leak only if a page's JavaScript can actually read it. hardwareConcurrency 2 was excluded from PLAUSIBLE_CORE_COUNTS because "2 is what Firefox reports under resistFingerprinting". That has not been true for years: RuntimeService::ClampedHardwareConcurrency hardcodes 4, and 8 on macOS, both of which are already in the table. The exclusion protected against nothing and cost every genuinely dual-core machine -- 20% of the macOS presets in the recorded corpus, 4.2% of Linux draws. It also made the small-host tail worse: a 3-core host reported 4, which cannot be pinned, so a page measured 3 while being told 4. At 2 the pin succeeds. pin_cpu_cores now defaults to False. What it buys is defence against a page timing N parallel workers; what it costs is a browser-wide CPU cap, a per-driver launch lock, and nothing at all on macOS. Unpinned, the host's own snapped count is reported, so reported and measurable still agree -- the identity just loses one drawn value. Callers who want the draw kept can still ask for it. The WebGL sampler keeps rejecting software rasterisers, and its docstring now says so: it described the opposite of what the code does. llvmpipe as the presented GPU is a live check on a string every fingerprint script reads, which is worth ~1.5% of corpus fidelity. 251 pythonlib tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(pythonlib): keep 2 out of the core table -- an Apple M1 is never dual-core Reverts the PLAUSIBLE_CORE_COUNTS half of |
||
|
Jake WriterandClaude Opus 5
|
52d6746a4a |
ci: a repo-wide test pipeline, and the one check that gates merge on it (#772)
* ci: a repo-wide test pipeline, and the suites Camoufox was missing
Nothing checked a pull request before this. `build.yml` runs on tags and takes
about forty minutes, and `lint.yml` ran a single static script, so a change
could reach main having had no browser suite run against it at all.
This adds one pipeline, driven identically from a pull request, a push to main,
and -- through `workflow_call` -- any caller that needs to test a specific
browser version, so there is exactly one definition of "the tests pass".
resolve ──┬─ static ────────── tribal rules, skiplist, self-tests
├─ pythonlib ─────── the package's own tests
└─ build ──┬─ playwright upstream × 6 shards (conformance)
├─ playwright vendored (regression)
├─ native ───────────── leaks, contexts
├─ patch guards ─────── one per spoofing patch
├─ build-tester ─────── 8 fingerprint profiles
└─ sundial ──────────── stealth grade (off, see below)
│
summary ──► one comment on the PR
Two Playwright suites, because they answer different questions. `tests/` is a
frozen ~v1.55-era fork carrying roughly 1800 lines of Camoufox adaptations, so
every test in it has a known prior outcome: that is the regression check. The
upstream suite is fetched fresh at the tag `ci/versions.py` resolves and runs
unmodified, which is the conformance check -- `ci/pw_camoufox_plugin.py` adapts
the environment around it rather than editing it, hooking BrowserType at the
_impl layer so upstream can refactor its fixtures freely.
`native-tests/` covers what neither can ask about: that resource cost does not
scale with launch count (the shape an FD or socket leak actually has), that two
contexts in one browser get different fingerprints while two pages in one
context get the same one (get this wrong and per-context injection silently
degrades to process-global, which passes every single-context test there is),
and that decisions already made stay made -- `ci/tribal-rules.yml` lists them
with the issue or PR that settled each.
Cost is tiered so a two-second lint failure never reaches a build, and a
driver-only pull request never builds at all: it fetches the published release
and tests against the build users are actually running, a minute instead of
seventy. Merges gate on one required check, `All tests passed`, so the
branch-protection list does not need editing every time a suite is added or
resharded; `ci/branch-protection.json` holds the settings so they are reviewable
rather than lore.
**The stealth check ships disabled** (`ci/sundial.yml: enabled: false`). It
drives a private detection suite, and the deployment it talks to predates that
suite's score mode; an older one ignores `?score=1` and posts the entire report
-- every vector's id, name, brief, source and value -- to whatever collector
asked. Receiving that on a public runner and discarding it afterwards is not the
same guarantee as never being sent it, so while the flag is false the job is not
scheduled, no credential enters a runner, and `run_sundial.py` refuses a hand-run
too. When it is enabled, `redact()` publishes a grade and counts against a
runtime whitelist and refuses anything that is not already aggregated.
Also included: the fixes these suites exposed on a clean runner -- build-tester
hashing canvas pixels rather than a prefix of the data URL, the virtdisplay
cleanup when Xvfb has already died, a juggler sandbox released on frame destroy
rather than only on navigation, and the pythonlib geometry and version-floor
corrections. `lint.yml` is removed because the static job absorbed its one check.
Verified locally: ci/tests 68 passed, tribal rules 24 passed, pythonlib 209
passed, input-dispatch clean, `ci.versions` resolves 152.0.4/beta.31 against
playwright v1.61.0, and `ci.summarize` folds a run to "all suites passed".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1UY3f8gm2jA1J23C3ew9s
* ci: make result files survive the trip from job to summary
The first full run failed, and the summary could not say why: five suites came
back "required, but produced no result", including two whose jobs had passed.
Three separate plumbing bugs, none of them in a test.
**Hidden files.** `actions/upload-artifact@v4` excludes dotfiles unless told
otherwise, and every result we write lives under `.ci-work`. The jobs whose
`path:` was a list containing a glob uploaded nothing at all -- the Playwright
suites and the leak suite each wrote their evidence and then had it silently
dropped:
evidence -> .../.ci-work/results/playwright_vendored.json (fail, 1203 tests)
##[warning]No files were found with the provided path: .ci-work/results/
.ci-work/junit-*.xml. No artifacts will be uploaded.
**Common root.** Where a list did upload, the second entry moved
upload-artifact's common root from `.ci-work/results/` up to `.ci-work/`, so the
JSON arrived at `results/build_tester.json` instead of the artifact root. The
summary merges every `results-*` into one directory and `load_all()` globs a
single level, so the file was there and invisible. build_tester passed and was
reported missing.
Every `results-*` artifact now uploads exactly `.ci-work/results/`, with
diagnostics (junit XML, the build-tester graded tree) split into their own
`diagnostics-*` artifacts that the summary's `results-*` pattern ignores.
`include-hidden-files: true` everywhere that touches `.ci-work`.
**A required name nothing writes.** `static` was in the required list, but it is
a job, not a suite -- no runner writes a result by that name, so summarize
reported it missing on every run including a wholly green one. The suites that
job runs are the pipeline self-tests, which write no result, and native_rules,
which is required by name. The job is already covered: the gate fails on any job
that is not success.
Three guards, each verified by reintroducing the bug it catches:
- results-* artifacts upload exactly one path, so nothing nests
- anything touching .ci-work sets include-hidden-files
- every required name is one some runner can actually write
This changes no test. The real failures the first run found -- 6 in the vendored
suite, plus upstream shards 1 and 5 and the leak suite -- were masked by the
above and should now be reported rather than swallowed.
ci/tests 71 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1UY3f8gm2jA1J23C3ew9s
* test: two failures that were the tests' fault, not the browser's
**The leak check waited on the wrong set of processes.**
`test_a_single_launch_leaves_nothing` failed with Gecko's GPU probe still alive:
1 process(es) this test started are still alive: glxtest(2887, now ppid=1)
`settle()` polled `children(recursive=True)`, but `survivors()` judges the
sampled PID set -- deliberately, so that a process reparented to init cannot
hide a leak. Those two sets differ exactly when a process outlives its parent:
it stops being our child, `settle()` sees nothing left and returns at once, and
anything still winding down is reported as leaked. `glxtest` does this on every
launch; it is spawned by Gecko, its parent exits first, and it needs a moment.
So settle on the set the assertion actually uses. This is a grace period, not an
exemption -- a process that is still there when the timeout expires fails the
test exactly as before, and no name is special-cased.
**Playwright renamed a protocol method the tracing tests spelled out.**
`Page.waitForEventInfo` is `Page.__waitInfo__` in newer versions, so two tracing
assertions failed on a name, not on behaviour. The suite is pinned to a range
(`playwright<1.63`), not a version, so hard-coding either spelling is wrong.
Normalised in `get_trace_actions()`, next to the comment about the last time
Playwright moved this data -- the tests care which actions ran and in what
order, not what Playwright calls them this month.
Neither of these was Camoufox misbehaving.
Still failing, and genuinely about the browser or by design -- triaged next:
navigation popup load state, locator handler visibility, clock pause off by 1ms,
websocket close reason, and the three upstream ones (request headers, worker
locale, screencast viewport) which all look like deliberate spoofing divergence
and probably belong in the skiplist with a stated reason.
ci/tests 71 passed, tribal rules 24 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1UY3f8gm2jA1J23C3ew9s
* fix: the failures the new pipeline found, and the flake that hid them
Eleven gates were red on PR #9. Each one is now either a fixed defect or an
entry that says why the test cannot apply here -- nothing is silenced.
One real browser bug, found by the conformance suite:
The compositor-backed screencast added in
|
||
|
Jake Writer
|
0169975638 |
fix(config): declare media:spoof_codecs, and guard the whole class
PR #562 added a `media:spoof_codecs` read on the C++ side -- MaskConfig::GetBool("media:spoof_codecs") in MP4Decoder and MatroskaDecoder -- but never declared the key in settings/. Since validate_config() drops any key it does not recognise, the documented usage was inert: AsyncCamoufox(config={"media:spoof_codecs": True}) -> "Skipping unknown patch media:spoof_codecs : True" The key never reached the browser, so the feature could not be turned on through the supported path at all. Declared in both properties.json and camoucfg.jvv (bool, beside mediaDevices:enabled). The new test is the general form rather than a check for this one key: it scans patches/ and additions/ for MaskConfig::Get*/Has*("key") reads and fails when a key is not declared in settings/properties.json. A patch and its schema entry are two halves of one change, and shipping only one half is a mistake this project has now made in both directions -- canvas:seed (#721) and navigator.maxTouchPoints (#696) were declared but unconsumed; this one was consumed but undeclared. Across the tree the scan finds 63 reads against 109 declared keys, and media:spoof_codecs was the only gap. Note the runtime reads properties.json from the *installed browser bundle*, not the repo, so this fix only takes effect for a build packaged after it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GQgHHGRXNp29jr4xQjK7iv (cherry picked from commit 375b0fca4529a722220022c7993c030b83439db1) |
||
 Anton Poliatskii
|
fff2c730be |
fix(pythonlib): derive navigator.appVersion from the preset's user agent
from_preset() set userAgent, platform and oscpu from the captured device
but never appVersion. Firefox reports appVersion as "5.0 (<OS tokens>)",
so leaving it unset let the host's own value through — and a page reading
two properties saw them disagree.
Measured on 152.0.4-beta.29, macOS host, os="linux", fingerprint_preset:
navigator.platform Linux x86_64
navigator.appVersion 5.0 (Macintosh) <- the host
The value is derived from the user agent rather than from the platform,
because 20 of the 65 bundled Linux presets carry a distro token
("X11; Ubuntu") that a platform lookup would flatten to "X11" — a smaller
mismatch than the host leaking, but the same kind. Firefox builds
appVersion from the same OS tokens as the UA, minus the architecture and
the Gecko revision, with Windows collapsed to its family name; checked
against 800 browserforge fingerprints, the derivation is exact on every
one, including Android and the Ubuntu variant.
A preset that ships its own appVersion keeps it, and a user agent the
rule cannot parse leaves the key unset rather than inventing a value.
(cherry picked from commit
|
||
 luandhgt
|
b2d842177a |
Verify sha256 of downloaded release assets before extracting
check_asset() already reads the asset's digest from the GitHub API and
stores it as installed_sha256, and AvailableVersion carries a sha256
field through to version.json. Nothing compared either against the
bytes that were downloaded: every sha256 equality check in the package
compares metadata to metadata when selecting an installed version, and
hashlib appeared only in utils.py to key a config cache.
So the archive that gets extracted over the install directory, and then
chmod 755'd and executed, was accepted on transport security alone. The
digest needed to catch a substituted or truncated asset was already in
hand and unused.
Add verify_sha256() and call it between download and extraction on both
install paths -- install_versioned() for the CLI and InstallWorker for
the GUI. It hashes in 1 MiB blocks so a multi-hundred-megabyte asset
does not have to be held in memory, and rewinds the buffer afterwards
so unzip() still reads from the start.
When no digest is published the install proceeds with a warning rather
than failing: some sources publish no digest, and refusing to install
from them would be a regression, not a fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit
|
||
 codechrl
|
6b8b08646d |
fix(addons): re-download addons with a missing manifest
maybe_download_addons() treated an addon as already downloaded whenever its
directory existed. A download that fails partway leaves an empty directory
behind, which is then trusted on every later launch, so confirm_paths()
raises InvalidAddonPath: manifest.json is missing and never recovers. Gate
the check on manifest.json presence and rmtree the partial directory on
failure. Closes #308.
(cherry picked from commit
|
||
|
Jake WriterandClaude Opus 5
|
8cb7914328 |
feat(python): warn when a supplied binary predates the Playwright in use
A managed install below the version floor is upgraded by pkgman, but
executable_path deliberately bypasses that -- the caller supplied the binary,
so we neither replace it nor download another. That left one pairing nothing
checked: an old build driven by Playwright >= 1.61, which sends viewport fields
the older Juggler schema rejects. The user saw a bare
Protocol error (Browser.setDefaultViewport)
with nothing naming the cause.
Warn rather than raise, because the pairing is not always fatal. Camoufox
defaults to no_viewport when it spoofs window dimensions (sync_api), and
Playwright then never sends Browser.setDefaultViewport -- so the default path
works fine on an old build. Measured against a real beta.29 binary on
Playwright 1.62:
default path WORKS
new_context(viewport=...) BREAKS
new_context(no_viewport=False) BREAKS
new_context(viewport=..., is_mobile=False) BREAKS
Refusing to launch would break the setups in the first row. A build with no
version.json beside it -- an unpackaged objdir build -- tells us nothing, so it
is left alone rather than nagged about.
Verified end to end: warns on the real beta.29 build under Playwright 1.62,
silent on beta.30.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Jake WriterandClaude Opus 5
|
fc3392e427 |
fix(python): resolve the bundle from executable_path, not the managed install
get_env_vars() and _generate_fontconfig() read the bundled fontconfig and fonts through get_path(), i.e. the managed install, even when the caller supplied their own binary. _load_properties() already honours executable_path for properties.json; these two did not. Before the floor could reject anything this silently mixed one build's fonts into another build's launch. Once the floor is live it becomes fatal: every launch raises UnsupportedVersion while the caller is holding a perfectly good binary, because resolving the bundle drags in the managed install and that is what gets version-checked. Thread executable_path through both, matching _load_properties. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Jake WriterandClaude Opus 5
|
b1fe7227fa |
fix(python): key the browser floor on Playwright instead of a flat minimum
The incompatibility is two-dimensional -- it needs both a Playwright >= 1.61
and a browser < beta.30 -- but MIN_VERSION only knows about the browser. To
stay safe a flat floor has to assume the worst Playwright, which means:
* every 0.5.6 user re-downloads the browser, including the majority on
<1.61 who are in no danger;
* installs pinned to an older build lose the pin, and prerelease/alpha users
are moved off their channel, since every alpha sorts below beta.30;
* the library cannot run at all until the matching browser release is
published, making the PyPI-after-release ordering load-bearing.
Key it on the resolved Playwright instead. Measured: 1.60 works on beta.29 and
beta.30; 1.61 and 1.62 fail on beta.29 and pass on beta.30.
playwright <1.61 -> floor alpha.1 -> every install kept
playwright >=1.61 -> floor beta.30 -> below-beta.30 installs upgraded
version unreadable -> floor alpha.1 -> kept; a spurious forced re-download is
worse than leaving a working install
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Jake WriterandClaude Opus 5
|
ce87cf7dab |
fix(python): report an unsatisfiable version floor instead of recursing
camoufox_path() ended in `return camoufox_path()` after a fetch. When the
newest published build is still below CONSTRAINTS.MIN_VERSION, install() is a
no-op ("already installed") and that tail recursed ~1000 times -- each
iteration firing another GitHub API call, which exhausts the unauthenticated
rate limit (60/hr) long before the RecursionError lands.
That is precisely the state a library published ahead of its browser release
puts every user in, and it is reachable now that the floor is raised. It also
hits permanently for anyone using a repos.yml source that does not carry the
required build.
Re-check after the fetch instead, and raise UnsupportedVersion naming the
required minimum.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Jake WriterandClaude Opus 5
|
da67775257 |
fix(python): reach the fetch path when the installed build is below the floor
Raising CONSTRAINTS.MIN_VERSION is how this library has always forced a browser upgrade (beta.12 -> beta.15 -> beta.17 -> beta.18 -> beta.19); the floor only became 'alpha.1' incidentally, in an unrelated PR. That left the branch dead, and it had rotted: camoufox_path() probed INSTALL_DIR/version.json, which only the pre-multiversion flat layout ever wrote. With a versioned install below the floor it raised FileNotFoundError instead of falling through to a fetch, so raising the floor would have crashed every existing user rather than upgrading them. Treat a missing root version.json as "no legacy install here" so the caller falls through to CamoufoxFetcher().install() as intended. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
 Jake WriterandD Yiapanis
|
d6a806e2b5 |
fix(fingerprint): keep the WebGL renderer coherent with the screen (#729)
BrowserForge picks navigator/screen; the GPU is drawn separately from webgl_data.db weighted only by OS. Nothing ties the two together, so the synthetic path emits pairs no real machine ships -- a discrete desktop GPU behind a 1024x600 panel. Consistency checks (Pixelscan, Fingerprint.com) read that as masking even though every individual value is plausible on its own. Builds on @dyiapanis's #730, which identified the problem and the GPU-class thresholds, with three changes: * Constrain the GPU to the screen rather than the screen to the GPU. sample_webgl_for_screen does rejection sampling, so the GPU keeps webgl_data.db's real OS-weighted distribution and the geometry -- already reconciled against the real display and the window box by clamp_screen_to_display / fix_screen_no_taskbar / clamp_window_dimensions / clamp_window_position -- is left alone. * Where no coherent GPU exists at all (BrowserForge still carries netbook-era geometry, and nothing in the pool drives a sub-1366x768 panel), raise_screen_to_gpu_floor lifts the screen instead. It measures the screen-to-avail gap BEFORE mutating -- #730 computed it after overwriting screen.height, which turned a 1024x600 -> 1080 bump into a 520px "taskbar", a fresh impossible-geometry tell -- and it runs BEFORE clamp_screen_to_display so a genuinely small monitor still wins and a headful window cannot be pushed back off its own display (#499). * No Apple-M Retina floor. Apple silicon also ships in the Mac mini and Mac Studio, which drive whatever external monitor is attached, so pinning it to 2560x1600 would reject real hardware and shrink the pool for nothing. Measured over 300 synthetic fingerprints, incoherent GPU/screen pairs fall from 54.3% to 0%, with avail <= screen and availHeight < height holding in every trial. The screen floor is a no-op for the Linux and Windows pools (0/400 draws below it) and fires on 3.5% of macOS draws, so the entropy cost is confined to the implausible tail it exists to remove. Co-authored-by: D Yiapanis <d@yiapanis.co> |
||
|
Jake Writer
|
160c806ad1 |
fix(stealth): make spoofed speech voices fail closed (#731)
Firefox registers the host's speech-dispatcher / SAPI / NSSpeech voices
unless something stops it, and nsSynthVoiceRegistry only stopped it when the
explicit `voices:blockIfNotDefined` flag was set. Nothing set that flag, so
the host was suppressed only as a side effect of a non-empty spoofed list --
and the Python layer built that list inside a bare `except Exception: pass`.
Any path that left the list empty or unset therefore fell through to the host
backend. On a stock Linux box that exposes 14805 espeak-ng voices to the page
under a fingerprint claiming macOS or Windows, which both leaks the real host
OS and contradicts the rest of the profile. Reproduced on 152.0.4-beta.29:
config voices exposed
generation raises 14805 (all host speechd)
{"voices": []} 14805 (all host speechd)
valid list 115 (correct)
Three changes, so the failure is closed at both layers:
* nsSynthVoiceRegistry::AddVoice now also blocks when MaskConfig carries a
`voices` array at all -- including an empty one, or one whose entries were
all rejected as malformed. An empty spoofed list must mean "no voices",
never "all of the host's". With no `voices` key the browser still behaves
like stock Firefox, so a bare binary is unaffected.
* launch_options pins `voices:blockIfNotDefined` (via set_into, so an
explicit caller value still wins) and degrades a generation failure to an
empty list rather than leaving the key unset. It also passes the spoofed
navigator.language through, so the default voice matches the locale.
* validate_voices rejects the shapes MaskConfig::MVoices() silently drops --
bare "Name:lang:type" strings and half-filled objects -- before launch
instead of letting them degrade into a host-voice leak.
Both failure paths now expose 0 voices; the normal path still exposes 115.
|
||
|
Jake WriterandClaude Opus 5
|
2834a463d1 |
test(virtdisplay): assert the real post-condition of kill()
`VirtualDisplay.kill()` reaps the Xvfb child and then clears `self.proc`, so asserting `vd.proc.poll() is not None` afterwards raises AttributeError on None. Two tests failed this way on main, unrelated to any of the merged PRs. Assert `proc is None or proc.poll() is not None` -- reaped-and-cleared is the success path, and a surviving handle must still report an exit code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
 Cloudymap1e
|
e61642aaf5 | fix(server): close browser when launcher exits | ||
 Pratyush Sharma
|
a5afa46cfa |
Apply screen constraints on Windows and macOS
get_screen_cons() was gated on DISPLAY being set, which only ever happens on Linux, so headful runs on Windows and macOS generated fingerprints with no monitor bound at all. Fixes #425 |
||
|
Pratyush Sharma
|
22c6ffbdda |
Probe the host monitor in CSS pixels
screeninfo makes the process per-monitor DPI aware, so it reports physical pixels, while Firefox lays windows out in CSS pixels. At 150% Windows scaling a 1920x1080 panel is 1280x720 CSS px, so bounding the fingerprint by the physical size lets the window open 1.5x larger than the screen. Refs #425 |
||
|
Pratyush Sharma
|
fbafbcf9f0 |
Exclude virtual displays from the display clamp
headless='virtual' reaches launch_options as headless=False with virtual_display set (async_api rewrites it), so the headful gate fired and clamped the fingerprint to Xvfb's 1x1 stub. fix_screen_no_taskbar then drove availHeight to -39 and validate_config rejected the launch outright. |
||
|
Pratyush Sharma
|
2266f27501 |
Clamp headful window geometry to the real display
get_screen_cons() bounds the generated fingerprint to the monitor, but BrowserForge honours a Screen constraint only when its pool has a match: FingerprintGenerator.partial_csp catches the filtering failure and deletes the constraint unless strict=True. So a 1366x768 laptop routinely gets a 2560x1440 fingerprint with window.outerWidth 1920, and browser-init resizes the real chrome window to it -- rendering past the edge of the monitor. Re-apply the bound after generation instead of trusting BrowserForge with it, and pull screenX/screenY back inside the shrunken screen. Headful only. headless has no window to overflow, and headless='virtual' runs a 1x1 Xvfb whose "monitor" would otherwise shrink the fingerprint to 1x1. Fixes #499 |
||
|
Cloudymap1e
|
a8ad6285d6 | fix(linux): prepare required runtime directory | ||
|
Cloudymap1e
|
9e74311572 | fix(python): preserve humanize duration types | ||
|
Cloudymap1e
|
3efa5e2182 | fix(python): isolate per-launch environments | ||
|
Jake Writer
|
0e4151f820 |
fix: page-recycle hang under spoofed window dims, and the unmerged halves of #637-#647
Fixes the new_page() hang from #666, and restores the pythonlib/ + settings/ halves of #637-#647 that were dropped when those PRs were consolidated into #666 (that PR only carried patches/ + additions/, so these never actually landed). ## new_page() hangs when window.outer* is spoofed (#666) The outer-size hijack in browser-init.patch pinned the chrome documentElement to the spoofed size. That caps .browserStack, which caps the content viewport, so the content window can never reach the size Juggler asks for in updateViewportSize() -- and awaitViewportDimensions awaits exact equality with no timeout, so it deadlocks rather than erroring. The second new_page() hung forever and took the context with it. The pin was never load-bearing: GetOuterWidth/GetOuterHeight already consult MaskConfig unconditionally (fingerprint-injection.patch), so window.outerWidth is spoofed in C++ regardless of the real chrome window size. Resizing is enough. Measured on the official v152.0.4-beta.26 build (headless): config before after none pass pass inner pass pass outer HANG pass both HANG pass (iw:360 ih:740 ow:360 oh:800 -- exact) This corrects the diagnosis in #666, which blamed the inner+outer combination and the `!(outerWidth || outerHeight)` guard. outer* ALONE is sufficient to hang, and dropping inner* does not help, so that guard is not the culprit. Also fixed driver-side: Playwright's implicit 1280x720 viewport is what asks for the impossible size, so the driver now defaults to no_viewport when the config spoofs any window dimension. That fixes the hang on already-released builds without a rebuild. An explicit viewport=/no_viewport= from the caller wins. ## WebRTC ICE prefs (#538) #666 merged the C++ half of the WebRTC fix but not the prefs, so the shipped build still has no_host=true and none of the proxy_only prefs. proxy_only_if_behind_proxy is the pref that actually stops the real-IP leak: it prevents a UDP STUN request routing around a TCP proxy. no_host=false keeps the stock two-candidate shape, which obfuscate_host_addresses makes leak-free. ## Also restored from the consolidation - fix(proxy): dom.security.https_first rewrote http:// before the launch-arg proxy filter saw it, breaking CONNECT-only proxies (#638). - fix(stealth): speech-voice spoofing + stop leaking host voices (#646). - fix(stealth): clamp inner <= outer <= avail <= screen; BrowserForge can emit impossible geometries that leak as tells (#647). Refs: https://github.com/daijro/camoufox/pull/666 Refs: https://github.com/daijro/camoufox/issues/538 |
||
|
Pratyush Sharma
|
ab20eca72d |
Add regression tests for camoufox server
Cover both failure modes from #656 and pin the driver entrypoint contract, so a future Playwright reshuffle fails in CI rather than in a user's terminal. No browser download or launch, so they run anywhere. Refs #656 |
||
 Jake Writer
|
8f9ff07b48 |
fix(virtdisplay): atomically claim X11 display via Xvfb -displayfd (#597)
Replace the userspace lock-file scan + random-jitter retry loop with Xvfb's own -displayfd mechanism. Xvfb scans up from :0 and atomically binds the first free X11 socket (kernel-mediated, no userspace race), then writes the chosen display number back through an inherited pipe. This eliminates the duplicate-display race that occurred when many camoufox processes started concurrently and all observed the same set of free display numbers before any of them bound. Adds a 10s read timeout on the displayfd pipe so a hung Xvfb fails fast instead of blocking forever, and adds tests covering single launch, idempotent get(), 50 concurrent reservations with uniqueness, and post-kill display reuse. |