starred/camoufox
1
0
Fork 0
mirror of https://github.com/daijro/camoufox.git synced 2026-10-04 00:00:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

Official TypeScript/JavaScript launcher at parity with pythonlib, published to npm (#785)

Browse Source 
* feat(ts): import the TypeScript launcher port from feat/captchakrakenAndJSSupport

CAPTCHA support is left out; this branch is the JS/TS driver only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(ts): drop the CAPTCHA wiring left behind by the import

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ts): port fpgen to TypeScript, on the same pinned model

fpgen is not on npm. The port reads scripts/data/fpgen-model.json and checks
its sha256 with TLS on, never fpgen's own first-release download. Everything
that does not depend on the random draw is identical to Python (network,
value lookups, trace probabilities, conditions, errors); the draws are held to
Python's distributions by chi-square tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ts): identity layer at parity with pythonlib

A bit-exact port of CPython's random.Random, numpy's PCG64 choice and orjson's
serialisation, so identity_salt/identity_seed and every seeded draw (fonts,
voices, media devices, WebGL, noise seeds) come out identical to Python for the
same identity. coherence.py, presets and screen/window fixes are ported, and
golden fixtures recorded from pythonlib hold all of it to exact equality.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ts): launcher at parity with pythonlib's launch_options

launch_options() now produces pythonlib's output byte for byte (CAMOU_CONFIG,
CAMOU_PREFS_N, prefs, env, fontconfig, warnings) over 89 recorded scenarios.
Ports core pinning, geolocation, locales, fontprobe, the async API, and the
pkgman/multiversion integrity checks. An opt-in e2e suite launches a real
build through both launchers and compares what a page sees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: test the TypeScript package, and publish it to npm like pypi

- ci/run_typescript.py writes the `typescript` gate (typecheck, lint, vitest
  with the pythonlib golden tests) and, with --browser, `typescript_browser`
  (the e2e suite against the browser under test). Both are required by the gate.
- publish-npm.yml mirrors publish-pypi.yml: workflow_dispatch, checks, build,
  scripts/check-pack.mjs (version == pythonlib, every data file shipped, the
  tarball installs and imports), then publish via npm trusted publishing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): e2e that holds on any browser, NewContext, and the package on every PR

- Parity (TS == Python on the same binary) stays strict everywhere; whether the
  browser honours the config is asserted only on a binary whose properties.json
  knows every key the launcher sets, and otherwise skips naming the missing keys.
  A driver-only pull request is tested against the published release, which
  lags the launcher (beta.30 predates #779), so this is what makes the suite
  meaningful there instead of red on skew it cannot fix.
- New: NewContext in a real browser -- a per-context identity that differs from
  the launch identity and from a sibling context, and equals Python's.
- python_probe.py keeps stdout for its JSON (pythonlib prints "Skipping unknown
  patch" there), and a non-JSON reply now fails fast instead of hanging 240 s.
- The typescript gate builds the package and runs scripts/check-pack.mjs, so a
  packaging mistake fails the pull request that makes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): commit the launch fixtures, and fetch the browser from its real directory

- The root .gitignore ignores every path named `launch` (local build output),
  which silently dropped typescript/tests/fixtures/launch/ -- the
  launch_options() goldens -- from the branch. Re-included in
  typescript/.gitignore.
- fetch-browser read camoufox-bin from `camoufox path`, the cache ROOT, but
  multiversion installs each build under browsers/<channel>/<version>/, so the
  job has failed on every driver-only pull request since #772. It now resolves
  the active build as the launcher does (pkgman.camoufox_path).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(ts): give the typescript job pythonlib, so the cross-language checks run

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): fpgen model install is safe across processes

Processes installing into an empty cache at once each downloaded the model,
and one's install deleted the values.dat another had just decompressed, which
then failed its next lookup with ENOENT. Seen with vitest's parallel files on a
cold cache; a worker pool on a fresh machine would hit it too.

- ensureModel() installs under a cross-process lock (an atomic mkdir, stale
  after 10 min) and re-checks what is installed once it holds it.
- values.dat is only removed when the model is actually being replaced.
- The model keeps values.dat open, instead of reopening it on every lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: make local runs and CI see the same test suite

Three ways the suite passed here and not on the runner, each fixed at its cause:

- The root .gitignore's bare `launch` rule (for the Go launcher binary) ignored
  every path segment named launch, so tests/fixtures/launch/ never reached git.
  Anchored to /launch; and ci/run_typescript.py now fails when any file under
  typescript/{src,tests,scripts} is git-ignored, which would have caught it on
  the machine that wrote the fixtures.
- A missing prerequisite (fpgen model, pythonlib venv, fontTools, Xvfb, a font
  directory) skipped its tests, and a skip reads as green. tests/prereq.ts now
  fails them under CI unless the job names the gap in
  CAMOUFOX_TEST_ALLOW_MISSING. The typescript job installs all of them. The
  font-name check read one developer's local browser bundle; it now reads
  /usr/share/fonts (or CAMOUFOX_TEST_FONT_DIR), and CI installs a .ttc set.
- The fpgen install race surfaced only on a cold cache, by accident. It now has
  deterministic tests: a same-model reinstall keeps values.dat (verified to fail
  on the old code), the lock admits one holder and releases on error, and a
  stale lock is reclaimed.

Also: the browser gate runs only the e2e file, and the e2e probe and the
virtual-display test time-box each await, so a hang names its step instead of
reporting a bare 240 s timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): hold headless="virtual" to Python's, not to headless

On the runner (no media hardware), published beta.31 never settles
enumerateDevices() in a headful window while headless answers -- the named
timeout in the probe caught it. That is a browser property, so like the other
page-vs-config checks it moves to a test that runs on a binary current with
the launcher; the virtual-display test now requires the same page as Python's
headless="virtual" on the same binary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(build-tester): accept 18 and 22 cores, as real hardware reports

plausibleHWC's list of common core counts lacked 18 and 22 -- Intel Meteor
Lake laptops (Core Ultra 5 125H, Core Ultra 7 155H), and 22 is in 8 recorded
presets. build-tester draws random presets, so a run that picked one of the two
Linux presets reporting 22 failed: about one run in eleven, on any pull
request. A CI self-test now fails if the list rejects any core count pythonlib
can present (the presets and PLAUSIBLE_CORE_COUNTS).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): test on the published release only when it matches this tree

A pull request that did not touch the browser was always tested against the
published release. The patch guards and suites come from the checkout, so once
a browser change was merged but not yet released (#779, on top of beta.31),
every driver-only pull request ran #779's guards against a browser without
#779 -- eight guards failed on #785, which changes no browser source.

resolve now also compares the tree's browser sources with the tag the release
was cut from (v<version>-<release> from upstream.sh), and builds when they
differ or the tag does not exist. Building restores the base branch's cached
browser when its compiled half matches -- main's #779 build, here -- so the
extra cost is a cache restore, not a compile. Self-tests run the workflow's own
scope step in a scratch repo for the four cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): say when CI's browser cannot present the configured locale

With #785 finally tested on a browser current with the tree (main's cached
#779 build), every TS-vs-Python parity check passed and the page-vs-config
check failed: a de-DE/fr-FR identity presented en-US. Python presents the same
on that binary. CI tests the build job's unpackaged dist/bin, whose
res/multilocale.txt lists en-US only -- scripts/package.py injects the
langpacks, and CI never packages. So no CI suite had ever run a non-English
locale on a browser that has one.

The e2e locale assertions now run when the binary under test packages the
configured locale (read from res/multilocale.txt, loose or in omni.ja), and
otherwise go through prerequisite("packaged-locales"), which fails in CI
unless the job names the gap. The typescript (browser) job names it, with the
reason; the rest of the page-vs-config check stays strict. On a packaged #779
build all of it, locale included, passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(guards): judge the query-cost probes on a median, not one sample

stock-parity-probes timed each getter once. On a shared runner one GC pause or
CPU-steal spike decided the verdict: navigator.hardwareConcurrency took 77 ms
against a 50 ms allowance on the same restored build that passed the run
before. Each pair is now timed five times, interleaved, and compared by median.
The regressions these catch (a sync IPC per read, ~240 ms over the loop) cost
extra on every read, so they move the median; verified by giving the getter a
constant ~4 us of extra work per read -- 86 ms median, FAIL -- while the
healthy build passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): give the headful e2e page focus before probing it

enumerateDevices() intermittently never settled in the headless="virtual"
test on CI (passed one run, timed out the next, same build). Firefox defers
device enumeration until the document has focus -- LEAKS row 57 recorded the
same for a background tab -- and headless mode fakes focus while a headful
window on a bare Xvfb, with no window manager, only sometimes receives it. A
user's window has focus, so both launchers' virtual-display probes now bring
the page to the front first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: remove build tooling nothing uses

- The developer UI (scripts/developer.py, `make edits`). It depended on
  easygui, which no requirements file declares, and every action it offered is
  a Makefile target: patch, unpatch, workspace, revert, diff. Its two helpers in
  scripts/_mixin.py (is_bootstrap_patch, patch) had no other callers.
- legacy/, the Go launcher deprecated in 2024-11. Nothing built or shipped it.
  Its Makefile targets and scripts/run-pw.py go with it, and so does Go from
  every dependency list and workflow.
- jsonvv/ and settings/camoucfg.jvv. Nothing read the .jvv schema: config is
  validated against settings/properties.json, and the two had already drifted.
  The jsonvv package stays on PyPI.
- Scripts with no caller: bootstrap.py, moztree, setup-wasi-linux.sh,
  package-helper.sh, install-local-build.sh, mozfetch.sh (copied into lw/ but
  never packaged), examples/.
- The pre-ESM Juggler copies JugglerFrameParent.jsm and JugglerFrameChild.jsm,
  and hidden-scrollbars.css. Juggler loads the .sys.mjs actors and deliberately
  no stylesheet, but jar.mn still packaged all three.
- patches/librewolf/*.opt, which list_patches() never picks up; the roverfox
  second pass in patch.py, whose directory no longer exists; the unread
  --no-settings-pane option.
- The CAMOUFOX_PASSWD secret passed to `make fetch` and closedsrc_rev in
  upstream.sh, which nothing reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(python): remove dead helpers and a stale dependency

None of these had a caller:
- pkgman: is_supported_path, extract_zip, cleanup and set_version, left over
  from the single-directory install. cleanup() would have deleted every
  installed browser version.
- multiversion.get_cached_repo_names, CONSTRAINTS.as_range,
  fingerprints._load_os_voices, utils._clean_locals, and unused imports.

Also:
- The "Apify Fingerprints" row in `camoufox version`, which has read "?"
  since fpgen replaced BrowserForge.
- lxml is no longer a dependency; nothing imports it.
- The geoip extra now names maxminddb, the module geolocation.py actually
  imports, rather than getting it transitively through geoip2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: show the cursor paths humanize=True actually produces

The README's cursor video showed the Bezier generator Camoufox replaced
with Cursory's recorded trajectories. scripts/cursor-demo.py drives a real
build with humanize=True and records every mousemove event the page
receives. It writes assets/humanize-cursor.svg, an animated replay at the
recorded speed, so what the figure shows is what a site sees.

The script cannot change the binary, so ci/browser_inputs.py lists it as
non-native and editing it does not invalidate the cached browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(python): stop naming BrowserForge in user-facing text

fpgen replaced BrowserForge, but two LeakWarnings, the NonFirefoxFingerprint
message and the fingerprint_preset docstring still named it. One warning also
linked to a README anchor that no longer exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: one AGENTS.md for every agent, a roadmap, and docs that match the code

- AGENTS.md holds the engineering rules for any coding agent, plus the
  repo map, build, patch and test commands that CLAUDE.md used to carry.
  CLAUDE.md now only imports it, so there is one set of rules.
  ci/tribal-rules.yml is the record of settled decisions it points to.
- ROADMAP.md lists planned work, each item linked to its issue.
- README:
  - fpgen and the coherence check replace BrowserForge;
  - the patch workflow uses the make targets instead of the removed
    developer UI;
  - letter-spacing noise is described as off by default, as it is.
- docs/:
  - beta-testing-ff146.md removed;
  - patch-upgrading-guide rewritten around the make targets;
  - per-context-patches without the canvas patch that no longer exists,
    and with measured preset counts;
  - playwright-maintenance without the JSM wrapper that does not exist;
  - smaller fixes in MEDIA-DEVICES, input-dispatch and FONTS.
- ci/README: every job, and the real shard, skiplist and entry-point lists.
- pythonlib, tester and patch-dependency READMEs corrected against the code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pythonlib): handle headless='virtual' in launch_server

launch_server() is documented to take the same arguments as Camoufox(),
but passed headless='virtual' straight to launch_options(), so the server
launched with no Xvfb display. Start a VirtualDisplay the way Camoufox()
does, launch headful on it, and kill it when the server process exits or
the launch fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(python): remove fontprobe, which nothing called

fontprobe listed the fonts installed on the host, for a `camoufox fonts`
command that was never added. It has nothing to do with the font bundle
Camoufox serves to pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(license): the Python launcher is MIT; the browser stays MPL-2.0

The Python package has always been published to PyPI as MIT (#727), but
pythonlib/ shipped no licence file, and the repo's LICENSE is the browser's
MPL-2.0. MPL is copyleft per file. It covers the modified Firefox sources, not
a separate launcher that drives the browser over Playwright. So
pythonlib/LICENSE now carries the MIT text its metadata already declares, and
a Licensing section in the README says which part is which.

Closes #727.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python): fingerprint_preset=False no longer turns presets on

launch_options checked `fingerprint_preset is not None`, so passing False
drew a random bundled preset, the opposite of what was asked. It now uses a
truthiness check, and a test proves that None and False never draw a preset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: bring the release workflow in line with the tests build job

build.yml had drifted from tests.yml. It ran actions at v1/v2 on a
retired Node runtime, prepared the source tree with bare make calls
that fail the whole release on one dropped connection, and built with
a different Python than every pull request is tested with.

- Pin every action by commit SHA, at the major versions tests.yml uses
  (checkout v4, setup-python v5, upload/download-artifact v4, the same
  remove-unwanted-software SHA), and action-gh-release v2. The release
  job holds contents: write, so it should not follow a movable tag.
- Prepare the tree with `python3 -m ci.run_prepare`, as the tests build
  job does. BUILD_TARGET is set from the matrix so `make dir` writes the
  right mozconfig and Rust targets; multibuild.py then finds _READY and
  builds without re-patching. mach's toolchain bootstrap ignores the
  mozconfig, so running it after `dir` bootstraps the same toolchains.
- Build with Python 3.12, the version the tests build job compiles with.
- Default the workflow to no permissions; the build job gets
  contents: read and the release job keeps contents: write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python): NewContext looks up a proxy's exit IP through the right URL, or fails

NewContext derives the context's WebRTC IP and timezone from the proxy's exit
IP. That lookup had two defects, and both left the context showing the
host's values while its traffic went through the proxy:

- It built its own proxy URL with urlparse, which reads a scheme-less server
  such as "1.2.3.4:8080" (a form Playwright accepts) as scheme "1.2.3.4" with
  no host. urllib could not use a SOCKS proxy at all.
- Any failure was swallowed, and the context opened without the values.

The URL is now built with Proxy.as_string(), which the geoip launch path
already uses (scheme-less means http). The lookup goes through requests,
which handles SOCKS, and a failed lookup raises InvalidIP, naming the two
options that skip it. The tests cover scheme-less, http and socks5 servers
with credentials, both failure modes, and the case where no lookup is needed,
for NewContext and AsyncNewContext.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: stop generating a canvas seed, and drop config keys nothing reads

The browser has not noised the canvas since #528, and no patch reads
canvas:seed (#721). The launcher still drew one on every launch and sent it
through CAMOU_CONFIG, and NewContext called a setCanvasSeed that does not
exist. They no longer do.

For users this changes nothing on any browser since #528: the value was
ignored. A config that still passes canvas:seed gets the usual "Skipping
unknown patch" notice instead of silence. On a browser from before #528, the
launcher no longer turns canvas noise on, which is the behaviour #528 chose.

The same audit found more keys declared in settings/properties.json that no
patch or Juggler file reads, so setting them did nothing:
- canvas:aaOffset, canvas:aaCapOffset
- memorysaver, pdfViewerEnabled, webrtc:localipv4/6
- navigator.onLine, navigator.cookieEnabled, navigator.languages
- navigator.appCodeName, appName, product, productSub. Firefox reports these
  constants itself, so fpgen.yml no longer maps them.
- webGl:parameters:blockIfNotDefined and its WebGL2 twin

test_config_schema now checks this direction too: every declared key must be
read by the browser, unless it is listed with a reason. Three are listed:
locale:script and navigator.doNotTrack, which the launcher applies itself,
and navigator.buildID (#780).

The build-tester grading followed the same wrong premise. It tracked canvas
collisions as an unfixed per-context leak. A canvas that is rendered rather
than noised follows the fonts and GPU, as it does on real machines, so canvas
collisions are now counted with the other device-level values. The tribal rule
that recorded it as an open question is now a settled one,
canvas-is-not-noised, with an automated check.

Closes #721.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(ts): remove dead helpers

None of these had a caller:
- pkgman: isSupportedPath, extractZip, cleanup and setVersion, left over
  from the single-directory install. cleanup() would have deleted every
  installed browser version.
- multiversion getCachedRepoNames and getCachedVersions, CONSTRAINTS.asRange,
  removeMmdb (Python keeps its twins for the GUI) and pycompat pySorted.
- The "Apify Fingerprints" row in `camoufox version`, which read "?".

utils.ts now calls noiseSeedsFromIdentity instead of repeating its two
formulas inline, so the tested function is the one that runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(ts): remove fontprobe, which nothing called

fontprobe.ts listed the fonts installed on the host, for a `camoufox fonts`
command neither launcher has. It has nothing to do with the font bundle
Camoufox serves to pages. Its parity test goes with it, and so do the CI
prerequisites only that test needed: fonttools and the extra font packages.
(The Python twin is removed in #787.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(ts): license the launcher MIT, with third-party notices

The TypeScript launcher is a port of pythonlib, which has always been
published to PyPI as MIT (#727); the MPL-2.0 of the browser covers the
modified Firefox sources, not a launcher that drives it over Playwright.

THIRD_PARTY_NOTICES.md ships in the npm package with the notices for the
code the port translates: fpgen (Apache-2.0), CPython's random (the MT19937
BSD notice and the PSF licence), and NumPy's SeedSequence and PCG64 (BSD-3
and MIT).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: document the TypeScript package outside typescript/

The README, CONTRIBUTING, ci/README and the issue templates did not mention
the npm package or its two CI gates. ci/README also still said driver-only
pull requests never build. Since the scope step started comparing browser
sources against the release tag, they build whenever the release is behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python): repair devicePixelRatio the same way on every launch

The DPR repair snaps an off-grid ratio to the nearest real scaling step and
keeps the first of two equally near steps. The steps were frozenset
literals, and a frozenset literal iterates in one order when the module is
compiled from source and another when it is loaded back from a .pyc. So a
midpoint such as 1.125 became 1.25 on the first launch after an install and
1 on every launch after it: the same pinned identity presented two different
devicePixelRatio values.

The steps are now ascending tuples, so a tie always goes to the lower step.
The test runs the repair in two fresh interpreters that share a bytecode
cache, compiling in the first and loading in the second. It failed before
this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): mirror #787's pythonlib fixes

The TypeScript side of the behaviour #787 changes in pythonlib, so the port
stays at parity:
- fingerprint_preset=false no longer draws a preset.
- NewContext builds the proxy URL with ProxyHelper.asString() (scheme-less
  means http), looks up the exit IP through impit, and throws InvalidIP when
  the lookup fails instead of opening the context with the host's values.
- No canvas seed is generated or sent (#721). noiseSeedsFromIdentity becomes
  audioSeedFromIdentity, and fpgen's constant navigator fields are no longer
  mapped.
- The devicePixelRatio steps are ascending, so a tie goes to the lower step.
- The two LeakWarning texts that named BrowserForge.
- The README's note that Python's launch_server() ignored headless='virtual'
  is gone, because it no longer does.

The golden fixtures are regenerated from #787's pythonlib. The generator now
masks the fontconfig file name the way the test already did. The name hashes
content that embeds the checkout path, so every regeneration from a different
checkout used to rewrite 76 fixtures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: remove the glyph-spacing seed from the browser and the launcher

anti-font-fingerprinting.patch added a seeded amount to every glyph advance,
so that text widths differed per context. No real machine produces those
widths: the same font on the same OS measures the same everywhere. So the
noise was itself a fingerprint, measured in #779 at +1 px per ~100 glyphs
plus fractional deltas on every measureText. #779 defaulted the seed to 0
and kept it as an opt-in, but an opt-in whose only effect is to become
detectable is not worth carrying. Removed:

- The browser side:
  - FontSpacingSeedManager and window.setFontSpacingSeed;
  - the HarfBuzz hook;
  - the plumbing that existed only to carry the context id down to the
    shaper: the userContextId on gfxTextRun, gfxShapedWord and the word-cache
    key, and the extra MakeTextRun argument in nsTextFrame, nsFontMetrics,
    MathML and canvas.
  The font group keeps its userContextId, which font-list-spoofing.patch
  uses to apply the per-context font list. Text is now shaped exactly as
  stock Firefox shapes it.
- The fonts:spacing_seed key. The launcher had been sending 0 on every
  launch, plus a setFontSpacingSeed(0) call in every context's init script.
- tests/patches/config-overrides.py, which tested only the spacing override.
  A pythonlib test now covers config_overrides with another key.

timezone-spoofing, webrtc-ip-spoofing and window-setter-seal change only in
context lines and the setter seal list. Every patch applies cleanly to a
fresh tree, and the result builds. The settled decision is recorded as
no-glyph-spacing-noise, with an automated check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: stock animations and speech by default; drop config keys that freeze live values

Three behaviours a page could detect, changed in one breaking release:

- **Animations run on stock timing.** no-css-animations.patch finished every
  finite animation at once by default, and any page could read it:
  `el.animate(frames, 1000).effect.getComputedTiming().duration` was 0, and a
  500ms transition reported 0. Measured on v152.0.4-beta.31. The speedup is
  now an opt-in, `instantAnimations: True`, which raises a LeakWarning.
  disableInstantAnimations is gone.
- **speak() on a spoofed voice works like a real voice.** It fired `error`
  after 3ms unless voices:fakeCompletion was set, and then start and end in
  the same tick. It now starts and ends after the text's duration at ~150
  words per minute. Both voices:fakeCompletion keys are gone, and so is a
  debug line printed to stderr on every call.
- **Keys removed:**
  - battery:* and window.scrollMinX/Y: Firefox keeps getBattery() and
    scrollMin* chrome-only, so no page could read them.
  - window.scrollMaxX/Y, screen.pageXOffset/pageYOffset,
    window.history.length and document.body.client*: each pinned a live value
    to a constant, so scrolling, navigating or re-laying out never changed it.
    fpgen.yml mapped pageYOffset, so about 15% of identities froze
    window.scrollY at a non-zero value.
  - The body keys' role as an undocumented alias for window.innerWidth/Height
    in browser-init and in the launcher.
  - MaskConfig::GetInt32Rect, which only the body keys used.

New guards, both of which fail on v152.0.4-beta.31:
tests/patches/animation-timing.py and tests/patches/spoofed-voice-speaks.py.
The decisions are recorded as animations-run-on-stock-timing and
spoofed-voices-speak. Every patch applies cleanly to a fresh tree, and the
result builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python)!: remove dead public API and make `list all --path` work

Breaking changes:

- Remove the exceptions UnknownProperty, InvalidDebugPort and
  MissingDebugPort. Nothing in the package raises them, so code catching
  them was catching nothing.
- Remove the legacy `allow_webgl` keyword of launch_options(). Use
  `block_webgl=True`. The keyword now reaches Playwright as an unknown
  launch option and fails there instead of being silently consumed.

`camoufox list all --path` accepted the flag and ignored it. It now prints
the install path beside each installed build, as `camoufox list --path`
already does for the installed tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(python)!: drop data the package never draws from

voices.json shipped in the wheel, but no code in the package reads it: the
voice draw uses voice-manifests.json and voice-uris.json. Its only readers
are the TypeScript port's golden-fixture generator and data-sync script
(typescript/scripts/golden/identity_golden.py,
typescript/scripts/sync-identity-data.py), which live on another branch and
will need a new source; the last copy is at
676fb3f:pythonlib/camoufox/voices.json. docs/per-context-patches.md
described it as runtime data and now describes the files that are.

webgl_data.db held two rows with zero weight on every OS ("Intel(R) HD
Graphics 400, or similar" from "Intel Inc." and "Radeon R9 200 Series, or
similar" from "ATI Technologies Inc."), left behind when their impossible
macOS weights were zeroed. No draw can reach them. They are deleted with
secure_delete so their blobs do not linger in free pages; the file is not
vacuumed, so the other pages are unchanged. A new test requires every row
to be drawable on at least one OS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python): warn whenever an identity falls back to a substitute value

Several draws swallowed their failure and used something else, so an
identity could ship with values the rest of it was not drawn to match and
nobody would hear about it:

- from_preset(): a failed font or voice draw used the preset's recorded
  list, or nothing, on any exception.
- generate_context_fingerprint(): a failed font, voice or WebGL draw was
  `except Exception: pass`, leaving the browser's launch-time values.
- _load_font_groups() / _load_font_bases(): an unreadable file became {},
  i.e. no font additions or no OS-version base.
- launch_options(): a failed font draw used every font in fonts.json, a
  failed voice draw used no voices, and a preset GPU missing from
  webgl_data.db was silently swapped for a drawn one (36 of the 397
  bundled presets).

Each site now catches only the errors its data can raise (OSError and
ValueError for an unreadable or corrupt file, KeyError for a manifest with
no entry for the OS, sqlite3.Error for the WebGL database) and emits a
FallbackWarning. The text names what failed and what the identity uses
instead, then gives a block to paste into an issue (camoufox, browser, OS
and Python versions, the error, and the identity's user agent or GPU),
asking the user to report it on GitHub. It shares LeakWarning's
caller-frame attribution and its template lives in warnings.yml.

The broad excepts had also been hiding a broken fixture:
test_launch_environment's font and voice stubs did not accept `seed`, so
every draw there raised and was swallowed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python): give NewContext identities the browser's Firefox version

NewContext() and AsyncNewContext() passed ff_version=None through to
generate_context_fingerprint(), so a context's user agent kept the version
fpgen drew (e.g. Firefox/146) while the browser underneath was 152. They
now default ff_version to the major version of Playwright's
Browser.version, which Juggler reports from MOZ_APP_VERSION_DISPLAY, so the
UA always names the browser the page is actually talking to. An explicit
ff_version still wins.

The docstrings said each context gets "its own real fingerprint preset";
the default has been an fpgen draw, with a preset only when one is passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python): send an IPv6 WebRTC address to setWebRTCIPv6

The per-context init script passed every webrtc_ip, IPv6 included, to
window.setWebRTCIPv4(), and never called setWebRTCIPv6(). An IPv6 address
(given directly, or resolved as a proxy's exit IP) was stored as the
context's IPv4 value and the IPv6 slot stayed empty. The script now picks
the setter by address family, and an address that is neither raises
InvalidIP instead of being passed through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python): stop pinning the page's scroll offset from fpgen

fpgen.yml mapped the drawn window.pageYOffset (e.g. 528) to
screen.pageYOffset, and the browser returns that value from scrollY on
every read, so a page saw one scroll position forever whatever the user
did. Real scroll offsets are live page state, not part of a device's
fingerprint, so neither offset is mapped any more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(python): stop checking a config key that no longer exists

warn_manual_config() looked for navigator.languages, which was removed
from settings/properties.json; validate_config() rejects it before the
check could matter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(python): close the WebGL database connection on every path

sample_webgl raised its not-found and wrong-OS errors before reaching
conn.close(), leaking a sqlite connection each time a preset named a GPU the
database does not hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(data): drop the 23 presets whose GPU has no WebGL data

A preset records only its GPU's name. The WebGL parameters, extensions and
shader precision behind it have to come from somewhere, and for these 23
nothing Camoufox has describes the GPU: fpgen has never seen Firefox report it
on that OS. So each launch paired the name with another device's parameters,
a mismatch any WebGL fingerprinter can see. They were:

- Windows on ARM (Adreno 650);
- Direct3D 10-level GPUs (vs_4_0/vs_4_1);
- "Generic Renderer";
- 945GM and GTX 480 on macOS;
- nouveau/Mesa buckets on Linux;
- one Linux preset pairing NVIDIA's proprietary vendor string with the
  nouveau renderer name.

scripts/clean-fingerprint-data.py now applies the rule, via a shared
fingerprints.firefox_gpus(), and test_shipped_data asserts it. 374 presets
remain, and every OS keeps its presets. ROADMAP.md lists capturing WebGL data
for these GPUs, which would bring them back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): stop generating the glyph-spacing seed

Mirrors 676fb3f: the browser no longer has glyph-spacing noise, so the
launcher sends no fonts:spacing_seed and the per-context init script no
longer calls setFontSpacingSeed. config_overrides is now tested with
audio:seed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): warn on instantAnimations; stop treating body keys as window size

Mirrors the launcher half of fb21b2e: instantAnimations raises the
instant_animations LeakWarning (warnings.yml copied from pythonlib), the
document.body.client* keys no longer count as window dimensions, and fpgen's
pageYOffset is no longer mapped, so no identity freezes window.scrollY.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts)!: remove dead public API and make `list all --path` work

Mirrors 4616aa5: drop the UnknownProperty, InvalidDebugPort and
MissingDebugPort exceptions (nothing raises them) and the legacy allow_webgl
option (use block_webgl; allow_webgl now passes through to Playwright like
any unknown option). `camoufox list all --path` prints each installed
build's path, as `list --path` already did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(ts)!: drop voices.json, which nothing draws from

Mirrors 9a42b6a. The voice draw reads voice-manifests.json and
voice-uris.json; voices.json was only read by the golden generator and the
data-sync script. The voice-URI golden now hashes the URI of every entry in
voice-manifests.json, the list the draw actually picks from.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): warn whenever an identity falls back to a substitute value

Mirrors 165e68f for the font and voice draws. Each fallback site in
fromPreset(), generateContextFingerprint(), loadFontGroups(),
loadFontBases() and launchOptions() now catches only the errors its data can
raise and emits a FallbackWarning naming what failed, what the identity uses
instead, and a block to paste into an issue (camoufox, browser, OS and Node
versions, the error, the identity). The message is warnings.yml's
`fallback` template, shared with pythonlib.

Python's except clauses name builtin classes JavaScript lacks, so pycompat
gains OSError, ValueError and KeyError twins and isPyError(): a Node system
error counts as an OSError and JSON.parse's SyntaxError as a ValueError, as
json.JSONDecodeError is. The voice draw now throws ValueError for a malformed
entry and KeyError when the manifest has no macOS entry, as Python does.

The WebGL fallback sites are left for the change that replaces the TS WebGL
source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): give NewContext identities the browser's Firefox version

Mirrors 46e5c8d: without an explicit ff_version, NewContext() kept the
Firefox version fpgen drew, so a context's UA could name 146 on a 152
browser. It now defaults to the major version of Browser.version(). The
option docs now say the default identity is an fpgen draw.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): send an IPv6 WebRTC address to setWebRTCIPv6

Mirrors 0bbd152: the per-context init script passed every WebRTC IP to
setWebRTCIPv4(), IPv6 included. It now picks the setter by address family
and raises InvalidIP for an address that is neither. The init-script golden
gains an IPv6 case.

Also ports 7b43112's regression test: a drawn pageXOffset/pageYOffset is
not carried into the config (the mapping went in fb21b2e's mirror).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(ts): stop checking a config key that no longer exists

Mirrors 480789a: warnManualConfig() looked for navigator.languages, which
settings/properties.json no longer has; validateConfig() rejects it first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(ts): sync the pruned presets and the properties.json fixture

Copies fingerprint-presets*.json from pythonlib (40edebb dropped the 23
presets whose GPU has no WebGL data) and refreshes the launch fixture's copy
of settings/properties.json, which lost the keys removed in 676fb3f and
fb21b2e.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(python): draw every identity's WebGL from fpgen

WebGL vendor, renderer, context attributes, extensions, parameters and
shader precisions, for WebGL1 and WebGL2, now come from fpgen's recorded
Firefox devices instead of webgl_data.db, which is deleted with the
camoufox/webgl/ package.

camoufox/webgl.py:
- webgl_for_gpu() traces `webgl` given Firefox, the OS and the GPU, then
  `webgl2` given the chosen `webgl` too, and draws each with one seeded
  random.Random. The GPU and the webgl value are pinned by their fpgen
  lookup index: a dict condition is flattened into leaves that overwrite
  each other, so only the renderer applied and Linux "Mesa" and "AMD"
  Radeon HD 3200 devices came back mixed.
- sample_webgl_for_screen() draws the GPU of a generated identity from
  fpgen's per-OS weights, filtering out software rasterisers, GPUs the OS
  cannot report, discrete GPUs behind a netbook screen and the
  resistFingerprinting "Mozilla" mask before the weighted choice, so there
  is no rejection loop. An empty pool raises.
- The draft/host-dependent extension filter moves over unchanged.

A preset's GPU and a caller's webgl_config pair are looked up as given;
a pair fpgen has never seen from Firefox on that OS raises instead of
falling back to another GPU. generate_context_fingerprint no longer
falls back to the host GPU when the draw fails.

For 10 of the 15 (GPU, OS) pairs the two sources share, one of fpgen's
records converts to exactly the database row on every value the browser
reads. The other five rows (Linux R9 200 and Radeon HD 3200, macOS
Intel HD, and two software rasterisers) are devices fpgen does not carry;
those GPUs now present fpgen's recorded devices instead. The Linux
GTX 980 row is kept as a test fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: say where WebGL comes from now that the database is gone

The per-context guide, the fpgen.yml header and coherence's comments still
named webgl_data.db and sample_webgl(). They now point at camoufox/webgl.py
and fpgen. The guide also claimed presets carry WebGL parameters; they
record only the vendor and renderer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): regenerate the golden fixtures for the mirrored pythonlib changes

Regenerates identity_golden.py and launch_golden.py output from this tree's
pythonlib. launch_golden.py drops fonts:spacing_seed and
document.body.clientWidth from its inputs, adds an instantAnimations
scenario, and masks a FallbackWarning's report block in both launchers, since
it names the host and the runtime.

Two launch scenarios still differ: preset_windows_unknown_gpu and
config_webgl_unknown_pair expect the FallbackWarning pythonlib now raises when
a preset's GPU is missing from the WebGL data. That site belongs to the
change replacing the TS WebGL source; the rest of each scenario matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ts): draw every identity's WebGL from fpgen

Mirrors 0d859b3. src/webgl.ts is the twin of camoufox/webgl.py:
webglForGpu() traces fpgen's webgl node given Firefox, the OS and the GPU,
then webgl2 given the chosen webgl too; sampleWebglForScreen() first draws
the GPU from fpgen's per-OS weights, filtered (no software rasteriser, no
resistFingerprinting mask, a GPU the OS can report, no discrete GPU behind a
netbook screen) before one weighted choice. Every draw is PyRandom.choices on
one seeded instance, in Python's order. The GPU and webgl value are pinned by
their fpgen lookup index, found from the value's stored JSON, which
TraceResult now carries: re-serialising a parsed value would spell 2**64
differently from orjson.

A preset's GPU and a caller's webgl_config are looked up as given and raise
when fpgen has never seen them, instead of falling back to another GPU;
generateContextFingerprint no longer swallows a failed draw. Removed with
the old source: webgl/sample.ts, the numpy default_rng port
(webgl/nprandom.ts), data-files/webgl_data.json and its export in
sync-identity-data.py. The NumPy notice now covers the pairwise sum in
locales.ts, the one NumPy port left.

launchOptions now throws the pycompat ValueError where Python raises
ValueError. Tests port test_webgl.py and the shipped-data check that every
preset GPU has WebGL data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): pin the fpgen WebGL draws against Python bit for bit

identity_golden.py now records camoufox.webgl's output as sha256 of the
exact orjson bytes (key order, int versus float and all): 1272 screen draws
over every OS and seven screens (60 seeds each, plus four large seeds),
webgl_for_gpu for every GPU fpgen records and every bundled preset GPU, the
unknown-GPU and unknown-OS errors, and to_config's extension filter. The
numpy golden keeps only np.sum, now checked against locales.ts. The renderer
list the coherence golden walks comes from fpgen's traces.

launch_golden.py takes its WebGL pairs from firefox_gpus(), and the
unknown-GPU preset input is a GPU nobody records. The launch goldens are
regenerated; the preset_windows_unknown_gpu and config_webgl_unknown_pair
scenarios now expect Python's ValueError. The golden test no longer maps
ValueError to Error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(patches): a host's missing speech daemon no longer errors spoofed speech

On a Linux host where speech-dispatcher cannot start, Firefox broadcasts
synth-voices-error, and SpeechSynthesis answers it by firing `error` on every
queued utterance. So a spoofed Windows voice errored about 11ms into speak()
on any host without the daemon: the CI runners, and most servers. It passed
only where the daemon runs.

While Camoufox manages the voice list, the registry no longer forwards a host
backend's error. The spoofed voices do not depend on the host's engine, and a
Windows or macOS identity never raises one. The guard now makes the daemon
unreachable itself, so it tests this case on every machine; on the previous
build it fails every time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: stop skipping the two click tests that stock animation timing fixed

test_wait_for_stable_position and test_timeout_waiting_for_stable_position
were skipped with humanized travel time as the reason. The real cause was
instant animations. Every finite animation finished at once, so the button
Playwright waits on to stop moving never moved, and the click landed where
upstream does not expect. With animations on stock timing both pass, and the
skiplist audit flagged them as no longer failing. The entries go, and the
counts in ci/README.md drop from 14 to 12.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(build-tester): accept 18 and 22 cores, as real hardware reports

plausibleHWC's list of common core counts lacked 18 and 22 -- Intel Meteor
Lake laptops (Core Ultra 5 125H, Core Ultra 7 155H), and 22 is in 8 recorded
presets. build-tester draws random presets, so a run that picked one of the two
Linux presets reporting 22 failed: about one run in eleven, on any pull
request. A CI self-test now fails if the list rejects any core count pythonlib
can present (the presets and PLAUSIBLE_CORE_COUNTS).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(guards): judge the query-cost probes on a median, not one sample

stock-parity-probes timed each getter once. On a shared runner one GC pause or
CPU-steal spike decided the verdict: navigator.hardwareConcurrency took 77 ms
against a 50 ms allowance on the same restored build that passed the run
before. Each pair is now timed five times, interleaved, and compared by median.
The regressions these catch (a sync IPC per read, ~240 ms over the loop) cost
extra on every read, so they move the median; verified by giving the getter a
constant ~4 us of extra work per read -- 86 ms median, FAIL -- while the
healthy build passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(native): compare the whole fingerprint when two launches must differ

test_two_browsers_get_different_fingerprints compared seven coarse values:
UA, platform, screen size, core count, timezone and language. CI pins the
timezone and language, and real machines share the rest: two draws of a
common Mac (Firefox 152, MacIntel, 2560x1440, 8 cores) matched, and the test
failed on a correct browser.

It now reads the whole fingerprint a site computes, from a script in the page:
- navigator values, screen and window geometry, device pixel ratio, timezone;
- WebGL vendor, renderer, limits and extensions;
- installed fonts, measured by width against the generic fallbacks;
- voices, media-device counts, and an OfflineAudioContext hash.
The page is served from an https URL Playwright fulfils locally, because
mediaDevices exists only in a secure context. The page computes the result
itself because the isolated world may not read audio sample data. The test
then requires the fingerprints to differ, and the audio hash to differ on its
own, since its noise is seeded per identity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Update README to remove warning, camoufox is now actively maintained

Camoufox will now be actively maintained and improved for the foreseeable future

* ci(ts): hold the golden tests to live pythonlib, not a snapshot of it

The typescript job ran the golden tests against the fixtures committed in
typescript/tests/fixtures/. A pythonlib change that typescript/ did not
mirror left those fixtures untouched, so the tests kept passing -- the
opposite of what the job's comment promised.

`ci.run_typescript --regenerate-golden` now rewrites the fixtures from the
checkout's pythonlib before vitest runs, and CI passes it. The job runs
Python 3.14 because pySum() reproduces sum() as 3.14 computes it; on 3.12
one crafted mixed int/float case differs in its last bit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(ts): do not redraw fpgen's stats fixture on every run

stats.json is thousands of random fpgen draws that the TS tests compare
statistically. It changes with the pinned model, not with pythonlib, and
redrawing it took eight of the typescript job's eleven minutes on a
runner. The deterministic fpgen fixtures are still regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(native): measure crash growth from a warmed-up parent

test_the_parent_stays_flat_across_content_crashes took its baseline right
after launch. The parent's first context costs it 150-200 MB with no
crash at all, so the warm-up counted as crash growth: 330-370 MB of the
400 MB allowance locally, and 469 MB on a CI runner, failing a PR that
changes nothing in the browser. The baseline now follows one clean
context; each crash still has to stay within the same allowance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ts): read pythonlib's data files instead of copying them

typescript/src/data-files/ held byte-identical copies of eleven pythonlib
data files (23k lines), kept in step by a sync script and a test that
failed when a copy drifted. The TS launcher now reads them from
pythonlib/camoufox/ when it runs from the repo, and `pnpm build` copies
them into dist/data-files/ for the npm tarball, so what users install is
unchanged. DATA_FILES in src/paths.ts is the one list; check-pack.mjs
checks each is in the tarball.

The essential-font lists were the one large table both ports hard-coded
(~170 lines of Python, ~770 of TS). They move to
pythonlib/camoufox/essential-fonts.json, which fingerprints.py and
fingerprints.ts both read; gen-fonts-json.py --print-bases writes it and
verify-fonts.py checks it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): record the golden fixtures from pythonlib on every run

The goldens were pythonlib's output committed as 14k lines of fixtures,
most of it the same input fingerprint pasted into ~80 launch scenarios,
and CI already rewrote them from pythonlib before each run. They are now
recorded by a vitest globalSetup (tests/golden-setup.ts) from the repo's
.venv or $CAMOUFOX_PYTHON, in about 6 seconds, and git-ignored. A
pythonlib change that typescript/ does not mirror fails `pnpm test`
locally as well as in CI, and ci.run_typescript no longer needs
--regenerate-golden.

Committed inputs stay: launch/inputs.json, the bundle stubs, the addon,
e2e/probe.js, and fpgen/stats.json (random draws tested statistically,
which change with the pinned model, not with pythonlib, and take minutes
to redraw). The one Python-version-sensitive case, sum() over mixed ints
and floats, skips with a named prerequisite when the goldens come from
Python < 3.14.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(npm): give the publish job the pythonlib its tests record goldens from

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): NewContext hands Playwright the identity's user agent, DPR and timezone

generateContextFingerprint already returns Playwright's JS option names;
NewContext re-cased them, and camelCase() lowercases first, so userAgent,
deviceScaleFactor and timezoneId became keys Playwright silently drops.
navigator.userAgent was still spoofed at the C++ level, so the page probe
matched Python's, but the HTTP User-Agent, the DPR and the timezone did
not. The options now pass through as generated.

NewContext also awaits ensureModel(): a browser from connect() or a
custom executable never went through launchOptions(), which fetches the
fpgen model, so an fpgen draw threw ModelNotInstalled where Python works.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): a failed browser download rejects instead of killing the process

The download's file stream had no 'error' listener, so a failed write
(disk full) was an uncaught 'error' event: Node exited before
installVersioned()'s catch could remove the partial install and its temp
directory, and the caller had nothing to catch. finished() now listens
from the moment the stream is created, and webdl() surfaces an errored
stream instead of writing into it.

webdl() also waits for 'drain': it ignored write()'s return value, so on
a slow disk the whole archive queued in memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): concurrent launches no longer read or inherit each other's CPU pin

playwright-core spawns the browser from this process, so pin_cpu_cores
narrows this process's own mask while a launch's browser starts. Python
pins a separate driver and never sees its own mask change. Here, a
second launch started in that window:

- read the pinned mask as the host's cores (pinnedCoreCount, and the
  identity's hardwareConcurrency via availableParallelism()), and
- if it did not pin, spawned its browser without the lock, inheriting the
  first launch's pin while reporting more cores.

The host's core count is now read once, before this process first pins
itself (cpu_affinity.hostCoreCount), and unpinned launches, launchServer
included, take the pin lock once any launch in the process has pinned.
With pin_cpu_cores off (the default) nothing waits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ts): an Xvfb that fails to start throws CannotExecuteXvfb

A spawn that fails after spawn() returns (EACCES, ENOENT) is an 'error'
event on the child process, which had no listener: Node treated it as
uncaught and exited instead of get() throwing. The child now always has
a listener, readDisplayNumber() rejects with CannotExecuteXvfb on it, and
the display pipe keeps an error listener after the read settles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(scripts): gen-fonts-json refuses to write an incomplete essential-fonts.json

An OS with no bases in the manifest was skipped, and the file written
without its key; fingerprints.py and fingerprints.ts read every OS's list
at import, so `import camoufox` then failed with a KeyError. The script
now exits and keeps the existing file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(release): pythonlib and the npm package to 0.5.7

@camoufox/camoufox 0.5.6 went to npm before the review fixes above, so
they ship as 0.5.7; the two launchers are versioned in lockstep, and
main already carries pythonlib changes from #787 that 0.5.6 on PyPI
does not have.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): NewContext's HTTP User-Agent must match navigator.userAgent

The e2e parity check read only navigator.userAgent, which the browser
spoofs itself, so a context that dropped Playwright's userAgent option
still matched Python. The probe server now records the request's
User-Agent header. Against the NewContext before the fix, a context
whose navigator said Windows sent the launch identity's Linux UA.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): draw the NewContext option-name test's preset from the v150 bundle

getRandomPreset() without a Firefox version draws from the older bundle,
where some Windows presets carry no devicePixelRatio, so the test failed
on some draws in CI. Every v150 preset has one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ts): wait for the headful e2e page to have focus, not one bringToFront()

On a bare Xvfb, with no window manager, a single bringToFront() before
goto() sometimes left the window unfocused, and Firefox holds
enumerateDevices() until the document has focus, so the virtual-display
probe timed out on some runs. Both launchers' probes now navigate first,
then bring the page to the front until document.hasFocus() is true, and
fail with that reason if it never is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Jake Writer
2026-09-27 03:28:12 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 180e6553cc
commit 0c6cc0a397
132 changed files with 26085 additions and 208 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/ISSUE_TEMPLATE/bug-report.md
+1 -1
View File
@@ -21,4 +21,4 @@ Provide steps or a code snippet that reproduces the bug.
### Version:
Run `python -m camoufox version` in your terminal and paste the output here.
Run `python -m camoufox version` (or `npx camoufox version` for the npm package) and paste the output here.
.github/ISSUE_TEMPLATE/camoufox-detected.md
+1 -1
View File
@@ -41,4 +41,4 @@ These questions will help me diagnose the issue:
### Version:
Run `python -m camoufox version` in your terminal and paste the output here.
Run `python -m camoufox version` (or `npx camoufox version` for the npm package) and paste the output here.
.github/workflows/publish-npm.yml
+87
View File
@@ -0,0 +1,87 @@
name: Publish to npm
# The npm twin of publish-pypi.yml: dispatched by hand, checks, builds, verifies
# the tarball, publishes. The package version is typescript/package.json's and
# must equal pythonlib's -- scripts/check-pack.mjs refuses to go on otherwise.
#
# Authentication is npm trusted publishing (OIDC): no token is stored anywhere.
# npm accepts this workflow's identity because the package's settings on
# npmjs.com name this repository and this file (publish-npm.yml). Renaming the
# file, or publishing from a fork, is rejected by the registry. The first
# version of a new package has to be published by hand, since the trusted
# publisher is configured on a package that already exists.
on:
workflow_dispatch:
inputs:
dry_run:
description: "Run every check and `npm publish --dry-run`, upload nothing"
type: boolean
default: false
permissions:
contents: read
jobs:
publish:
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # trusted publishing, and the provenance attestation
defaults:
run:
working-directory: typescript
env:
CAMOUFOX_FPGEN_DATA: ${{ github.workspace }}/.fpgen
steps:
- name: Check out repository
uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.14"
- name: Install pythonlib
# tests/golden-setup.ts records the golden fixtures from it, and the
# build copies its data files into the package.
working-directory: .
run: |
python3 -m venv .venv
.venv/bin/pip install -r ci/requirements.txt -e pythonlib
.venv/bin/python scripts/pin-fpgen-model.py
- name: Set up pnpm
uses: pnpm/action-setup@v4
with:
package_json_file: typescript/package.json
- name: Set up Node
uses: actions/setup-node@v6
with:
# Trusted publishing needs npm >= 11.5.1, which Node 24 ships.
node-version: "24"
registry-url: https://registry.npmjs.org
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Type check and lint
run: |
pnpm typecheck
pnpm check
- name: Test
run: pnpm test
- name: Build package
run: pnpm build
- name: Check package
run: node scripts/check-pack.mjs
- name: Publish to npm
run: npm publish --access public ${{ inputs.dry_run && '--dry-run' || '' }}
.github/workflows/tests.yml
+167 -9
View File
@@ -150,22 +150,48 @@ jobs:
# Only a change that can alter the binary justifies compiling one. A
# pull request that touches pythonlib/ or ci/ is a driver change: it
# still gets the full browser suite, but against the published build its
# users are running, which takes a minute instead of seventy.
# users are running, which takes a minute instead of seventy -- as long
# as that build matches this tree's browser sources (see below).
run: |
if [ "${{ github.event_name }}" != "pull_request" ]; then
echo "browser_changed=true" >> "$GITHUB_OUTPUT"
echo "::notice::Not a pull request -- building, which also refreshes the shared ccache."
exit 0
fi
sources='^(patches/|additions/|settings/|assets/|upstream\.sh|Makefile|scripts/)'
base="${{ github.event.pull_request.base.sha }}"
changed=$(git diff --name-only "$base"...HEAD || echo "")
echo "changed files:"; echo "$changed" | sed 's/^/ /'
if echo "$changed" | grep -qE '^(patches/|additions/|settings/|assets/|upstream\.sh|Makefile|scripts/)'; then
if echo "$changed" | grep -qE "$sources"; then
echo "browser_changed=true" >> "$GITHUB_OUTPUT"
echo "::notice::Browser sources changed -- rebuilding from source."
exit 0
fi
# The pull request leaves the browser alone -- but the published
# release is only the right browser to test it on if it was built from
# the SAME browser sources as this tree. The patch guards and suites
# come from this checkout, so when the base branch has moved past the
# release (a merged browser change that is not published yet), testing
# the release pairs new guards with an old browser, and every guard for
# the unreleased change fails on a pull request that never touched it.
# Compare against the tag the release was cut from; if they differ,
# build -- which restores the base branch's cached browser when its
# compiled half matches, so it costs minutes, not the full build.
. ./upstream.sh
tag="v${version}-${release}"
if ! git rev-parse -q --verify "refs/tags/$tag" >/dev/null; then
echo "browser_changed=true" >> "$GITHUB_OUTPUT"
echo "::notice::No release tag $tag -- the release this tree targets is not published; building."
exit 0
fi
ahead=$(git diff --name-only "$tag" HEAD | grep -E "$sources" || true)
if [ -n "$ahead" ]; then
echo "browser sources that differ from $tag:"; echo "$ahead" | sed 's/^/ /'
echo "browser_changed=true" >> "$GITHUB_OUTPUT"
echo "::notice::The base branch's browser sources are ahead of the published $tag ($(echo "$ahead" | wc -l) files) -- building (a cache hit when the base branch already built it)."
else
echo "browser_changed=false" >> "$GITHUB_OUTPUT"
echo "::notice::No browser sources changed -- testing against the published release."
echo "::notice::No browser sources differ from the published $tag -- testing against it."
fi
- name: May the stealth check run?
@@ -307,6 +333,73 @@ jobs:
include-hidden-files: true
if-no-files-found: warn
# ---------------------------------------------------------------------------
typescript:
name: typescript
# Tier 1, beside pythonlib. The npm package's type check, lint and vitest
# suite, including the golden tests that hold it to pythonlib's output byte
# for byte -- so a pythonlib change that typescript/ does not mirror fails
# here, in a minute, not after the build.
needs: [resolve, static]
runs-on: ubuntu-24.04
permissions:
contents: read
env:
# The fpgen model the TS port downloads (sha256-pinned by
# scripts/data/fpgen-model.json). Kept in the workspace so it can be cached.
CAMOUFOX_FPGEN_DATA: ${{ github.workspace }}/.ci-work/fpgen
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.ref || github.ref }}
- uses: actions/setup-python@v5
with:
# Not PYTHON_VERSION: the golden fixtures are regenerated from this
# interpreter, and pycompat.ts's pySum() reproduces sum() as 3.14
# computes it (3.12/3.13 round mixed int/float sums differently in
# the last bit).
python-version: "3.14"
- run: |
# A venv at the repo root: tests/golden-setup.ts records the golden
# fixtures from its pythonlib before the suite runs.
python3 -m venv .venv
.venv/bin/pip install -r ci/requirements.txt -e pythonlib
# fpgen downloads its model on first import with TLS verification
# OFF and no checksum, and its release picker can only ever reach the
# April-2025 model. Install the pinned one first: see
# scripts/pin-fpgen-model.py.
.venv/bin/python scripts/pin-fpgen-model.py
- name: Test prerequisites
# Everything tests/prereq.ts may ask for. In CI a missing prerequisite
# FAILS its tests rather than skipping them, so this list is the job's
# contract. xvfb: the virtual-display lifecycle.
run: |
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends xvfb
- uses: pnpm/action-setup@v4
with:
package_json_file: typescript/package.json
- uses: actions/setup-node@v4
with:
node-version: "22"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- uses: actions/cache@v4
with:
path: .ci-work/fpgen
key: fpgen-model-${{ hashFiles('scripts/data/fpgen-model.json') }}
- name: Run
env:
GITHUB_TOKEN: ${{ github.token }}
run: .venv/bin/python -m ci.run_typescript
- uses: actions/upload-artifact@v4
if: always()
with:
name: results-typescript
path: .ci-work/results/
include-hidden-files: true
if-no-files-found: warn
# ---------------------------------------------------------------------------
build:
name: Build (linux x86_64)
@@ -595,7 +688,11 @@ jobs:
run: |
set -euo pipefail
python -m camoufox fetch
install_dir="$(python -m camoufox path)"
# The ACTIVE build's directory, resolved the way the launcher resolves
# it. `camoufox path` prints the cache root, and multiversion installs
# each build under browsers/<channel>/<version>/ -- so reading the
# binary from the root failed every driver-only run since #772.
install_dir="$(python -c 'from camoufox.pkgman import camoufox_path; print(camoufox_path(download_if_missing=False))')"
echo "install dir: $install_dir"
# Which browser did we actually get? This path does not build, it
# downloads the current release -- correct for a driver change, since
@@ -833,6 +930,67 @@ jobs:
include-hidden-files: true
if-no-files-found: ignore
# ---------------------------------------------------------------------------
typescript-browser:
name: typescript (browser)
# Tier 3a. Launches the browser under test through the TS API -- headless,
# persistent context, and launchServer -- and checks that a page sees the
# same identity pythonlib's launch of it shows.
needs: [resolve, build, fetch-browser, typescript]
if: >-
always() && needs.typescript.result == 'success' &&
(needs.build.result == 'success' || needs.fetch-browser.result == 'success')
runs-on: ubuntu-24.04
timeout-minutes: 45
permissions:
contents: read
env:
CAMOUFOX_FPGEN_DATA: ${{ github.workspace }}/.ci-work/fpgen
# The browser under test is the build job's unpackaged dist/bin, which
# packages en-US only: scripts/package.py adds the langpacks, and CI never
# runs it. A de-DE/fr-FR identity therefore presents en-US here though a
# packaged release presents de-DE. The e2e locale assertions skip on that
# (named) gap; everything else in the page-vs-config checks still runs.
# Remove this once the test artifact carries the langpacks.
CAMOUFOX_TEST_ALLOW_MISSING: packaged-locales
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.ref || github.ref }}
- uses: ./.github/actions/prepare-browser
with:
python-version: ${{ env.PYTHON_VERSION }}
- run: |
pip install -e pythonlib
# fpgen downloads its model on first import with TLS verification
# OFF and no checksum, and its release picker can only ever reach the
# April-2025 model. Install the pinned one first: see
# scripts/pin-fpgen-model.py.
python3 scripts/pin-fpgen-model.py
- uses: pnpm/action-setup@v4
with:
package_json_file: typescript/package.json
- uses: actions/setup-node@v4
with:
node-version: "22"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- uses: actions/cache@v4
with:
path: .ci-work/fpgen
key: fpgen-model-${{ hashFiles('scripts/data/fpgen-model.json') }}
- name: Run
env:
GITHUB_TOKEN: ${{ github.token }}
run: xvfb-run -a python3 -m ci.run_typescript --browser "$CAMOUFOX_BINARY"
- uses: actions/upload-artifact@v4
if: always()
with:
name: results-typescript-browser
path: .ci-work/results/
include-hidden-files: true
if-no-files-found: warn
# ---------------------------------------------------------------------------
growth:
name: Memory growth (scheduled)
@@ -927,8 +1085,8 @@ jobs:
# ---------------------------------------------------------------------------
summary:
name: Summary
needs: [resolve, static, pythonlib, build, fetch-browser, playwright,
patch-guards, build-tester, native, sundial]
needs: [resolve, static, pythonlib, typescript, build, fetch-browser, playwright,
patch-guards, build-tester, native, typescript-browser, sundial]
if: always() && needs.resolve.result == 'success'
runs-on: ubuntu-24.04
permissions:
@@ -968,7 +1126,7 @@ jobs:
# summarize report "produced no result file" and fail the gate on
# every pull request that did not touch the browser. Which is most of
# them, and exactly the cheap path this pipeline advertises.
required="pythonlib native_rules patch_guards skiplist_audit build_tester playwright native_browser"
required="pythonlib typescript typescript_browser native_rules patch_guards skiplist_audit build_tester playwright native_browser"
if [ "${{ needs.resolve.outputs.browser_changed }}" = "true" ]; then
required="$required build"
fi
@@ -1033,8 +1191,8 @@ jobs:
# that is not `success`, including `skipped`, fails the gate: a suite that
# did not run has not passed, and silently skipping one is the cheapest way
# to a green tick.
needs: [resolve, static, pythonlib, build, fetch-browser, playwright,
patch-guards, build-tester, native, sundial, summary]
needs: [resolve, static, pythonlib, typescript, build, fetch-browser, playwright,
patch-guards, build-tester, native, typescript-browser, sundial, summary]
if: always()
runs-on: ubuntu-24.04
permissions:
CONTRIBUTING.md
+2 -1
View File
@@ -38,7 +38,7 @@ The engineering rules in [`AGENTS.md`](AGENTS.md) apply to every change, whether
## Testing Requirements
**CI runs everything, on every pull request.** [`.github/workflows/tests.yml`](.github/workflows/tests.yml) builds the browser from your branch when you touch browser sources (and tests against the published release when you do not), then runs the Python package tests, the patch guards, build-tester, the upstream Playwright suite, the leak suite and the stealth check. Branch protection requires exactly one check, **`All tests passed`**, which is green only when every applicable suite is.
**CI runs everything, on every pull request.** [`.github/workflows/tests.yml`](.github/workflows/tests.yml) builds the browser from your branch when you touch browser sources (and tests against the published release when you do not), then runs the Python and TypeScript package tests, the patch guards, build-tester, the upstream Playwright suite, the leak suite and the stealth check. Branch protection requires exactly one check, **`All tests passed`**, which is green only when every applicable suite is.
So there is nothing to attach to the pull request by hand. The old process — run the suites locally, screenshot the output, paste it in — was unenforceable: nothing checked that the browser in the screenshot was built from the branch under review. If you want a report in the description anyway, CI leaves one as a comment on the pull request.
@@ -50,6 +50,7 @@ python3 -m ci.run_build_tester --binary /path/to/camoufox-bin
python3 -m ci.run_playwright --binary /path/to/camoufox-bin # or --shard 3/6
python3 -m ci.run_skiplist_audit --binary /path/to/camoufox-bin
python3 -m ci.run_pythonlib # pythonlib/
python3 -m ci.run_typescript # typescript/
python3 -m pytest ci/tests -q # the pipeline's own tests
```
README.md
+22 -11
View File
@@ -80,7 +80,7 @@
Processing tens of billions of requests per month for thousands of customers, Byteful powers browser-based AI agents, automation systems, and data workflows. It is a member of the Internet Watch Foundation and the Ethical Web Data Collection Initiative.
Get 10% off Byteful Residential Proxies with the code: CAMOUFOX10
Get 10% off Byteful Residential Bandwidth with the code: CAMOUFOX10
</td>
</tr>
<tr>
@@ -338,7 +338,7 @@ Camoufox is a Firefox fork engineered for web scraping and AI agents. It is head
* **Built for AI agents** 🤖
* Minimal, debloated Firefox - fast to launch, cheap to run
* Drop-in Playwright compatibility via Python interface
* Drop-in Playwright compatibility from Python and JavaScript/TypeScript
* Invisible to anti-bot systems so you can run your agent cluster locally or in the cloud without being flagged
- **Undetectable by design** 🎭
@@ -362,7 +362,7 @@ Camoufox is a Firefox fork engineered for web scraping and AI agents. It is head
* Optional instant animations (`instantAnimations`), so Playwright never waits on one 💨
- Debloated & optimized for memory efficiency ⚡
- [PyPI package](https://pypi.org/project/camoufox/) for updates & auto fingerprint injection 📦
- [PyPI](https://pypi.org/project/camoufox/) and npm packages for updates & auto fingerprint injection 📦
- Stays up to date with the latest Firefox version 🕓
---
@@ -371,7 +371,7 @@ Camoufox is a Firefox fork engineered for web scraping and AI agents. It is head
In Camoufox, data is intercepted at the C++ implementation level, making the changes undetectable through JavaScript inspection.
To spoof individual fingerprint properties, pass a JSON containing properties to spoof to the [Python interface](pythonlib/):
To spoof individual fingerprint properties, pass a JSON containing properties to spoof to the [Python](pythonlib/) or [TypeScript](typescript/) interface:
```py
>>> with Camoufox(config={"property": "value"}) as browser:
@@ -383,11 +383,11 @@ Config data not set by the user is populated from [fpgen](https://github.com/scr
---
## Python Usage
## Usage
Camoufox is compatible with your existing Playwright code. You only have to change your browser initialization.
**Sync API**
**Python, sync API**
```python
from camoufox.sync_api import Camoufox
@@ -397,7 +397,7 @@ with Camoufox() as browser:
page.goto("https://example.com")
```
**Async API**
**Python, async API**
```python
from camoufox.async_api import AsyncCamoufox
@@ -407,7 +407,18 @@ async with AsyncCamoufox() as browser:
await page.goto("https://example.com")
```
[[Installation & usage](https://camoufox.com/python/)]
**JavaScript / TypeScript**
```javascript
import { Camoufox } from "@camoufox/camoufox";
const browser = await Camoufox({ headless: true });
const page = await browser.newPage();
await page.goto("https://example.com");
await browser.close();
```
[[Python installation & usage](https://camoufox.com/python/)] · [[TypeScript package](typescript/README.md)]
---
@@ -472,7 +483,7 @@ Below is a list of patches and features implemented in Camoufox.
- Addons are automatically pinned to the toolbar
- Fixes DNS leaks with uBO prefetching
### Python Interface
### Python & TypeScript Interfaces
- Automatically generates & injects unique device characteristics into Camoufox based on their real-world distribution
- WebGL fingerprint injection & rotation
@@ -568,7 +579,7 @@ Additionally, Juggler sends its inputs directly through the Firefox's original u
<h1 align="center">Build System</h1>
> [!WARNING]
> The content below is intended for those interested in building & debugging Camoufox. For usage instructions, see [pythonlib](pythonlib/).
> The content below is intended for those interested in building & debugging Camoufox. For usage instructions, see [pythonlib](pythonlib/) or [typescript](typescript/).
### Overview
@@ -793,7 +804,7 @@ flowchart TD
## Licensing
- **The browser** (`patches/`, `additions/`, `settings/`, and the build system) is [MPL-2.0](LICENSE), the licence of the Firefox source it modifies. The vendored Cursory trajectories are LGPLv3-or-later (`additions/juggler/input/cursory/NOTICE`).
- **The Python launcher** is MIT ([`pythonlib/LICENSE`](pythonlib/LICENSE)), as it has always been declared on PyPI.
- **The launchers** are MIT: the Python package ([`pythonlib/LICENSE`](pythonlib/LICENSE)) and the TypeScript package ([`typescript/LICENSE`](typescript/LICENSE)). The TypeScript package also contains ports of fpgen, CPython's `random` and NumPy's random generators; their notices are in [`typescript/THIRD_PARTY_NOTICES.md`](typescript/THIRD_PARTY_NOTICES.md).
---
ci/README.md
+16 -6
View File
@@ -7,9 +7,11 @@ pass", not two.
```
resolve ── static ─────────────── lint, tribal rules, skiplist, self-tests (seconds)
├─ typescript ────── type check, lint, vitest, golden parity
└─ pythonlib ─────── the package's own tests (a minute)
└─ build or fetch ─┬─ patch guards ─────── one per spoofing patch, + skiplist audit
├─ build-tester ─────── 8 fingerprint profiles
├─ typescript-browser ─ the npm launcher end to end
└─ once guards and build-tester pass:
├─ playwright × 6 shards (conformance + our own)
├─ native ───────── leaks, contexts, crash recovery
@@ -537,17 +539,23 @@ Each tier gates the next, so a two-second lint failure never reaches the build:
```
0 static lint, self-tests, settled decisions seconds
1 unit pythonlib ~1 min
1 unit pythonlib, typescript ~1 min
2 browser build (patches/additions/settings/assets/upstream.sh/Makefile/scripts changed)
fetch (anything else -- driver changes test against the published release)
3a smoke patch guards, skiplist audit, build-tester ~15 min
fetch (anything else, when the published release has this tree's browser sources)
3a smoke patch guards, skiplist audit, build-tester,
typescript-browser ~15 min
3b full Playwright x6, leaks, stealth ~40 min
4 gate the required check
```
**Driver-only pull requests never build.** There is nothing new to compile, so
`fetch-browser` downloads the published release and the browser suites run
against the build users are actually on — a minute instead of seventy.
**Driver-only pull requests test the published release, when it matches.**
There is nothing new to compile, so `fetch-browser` downloads the published
release and the browser suites run against the build users are actually on — a
minute instead of seventy. That is only right while the release was built from
this tree's browser sources: once a browser change has merged but not been
released, the guards in the checkout would judge an older browser. So the scope
step compares the browser sources against the release tag, and when they
differ it builds instead, which restores the base branch's cached browser.
**Changing Juggler's JavaScript does not rebuild the browser.** Measured on a
real build: ccache reported a **98.63%** hit rate, so almost none of those 24
@@ -622,6 +630,8 @@ way, so a tagged build gets the same hardening.
python3 -m ci.run_prepare # make setup-minimal, dir, mozbootstrap
python3 -m ci.run_build
python3 -m ci.run_pythonlib # no browser needed
python3 -m ci.run_typescript # no browser needed
python3 -m ci.run_typescript --browser path/to/camoufox-bin
python3 -m ci.run_patch_guards --binary path/to/camoufox-bin
python3 -m ci.run_build_tester --binary path/to/camoufox-bin
python3 -m ci.run_skiplist_audit --binary path/to/camoufox-bin
ci/run_typescript.py
+176
View File
@@ -0,0 +1,176 @@
#!/usr/bin/env python3
"""typescript gate: the npm package's own checks and test suite.
Two modes, two gates:
typescript (default) type check, lint, and the vitest suite. No
browser. Includes the golden tests that hold the TS
launcher to byte-for-byte parity with pythonlib, so a
pythonlib change that is not mirrored in typescript/
fails here, in tier 1, rather than after the build.
typescript_browser (--browser BINARY) the opt-in end-to-end suite: launches
the browser under test through the TS API and compares
what a page sees with what pythonlib's launch of the same
identity shows.
Run:
python3 -m ci.run_typescript
python3 -m ci.run_typescript --browser path/to/camoufox-bin
"""
from __future__ import annotations
import argparse
import os
import sys
import xml.etree.ElementTree as ET
from pathlib import Path
from typing import Dict, List, Optional
from . import results as evidence
from ._util import EVIDENCE_DIR, REPO_ROOT, WORK_DIR, run
TYPESCRIPT = REPO_ROOT / "typescript"
def parse_vitest_junit(path: Path) -> Dict[str, str]:
"""vitest junit XML -> {"tests/file.test.ts::suite > test": outcome}.
vitest puts the test file in `classname` and the describe path in `name`,
which already make a stable identity; pytest's dotted-module trimming in
ci/_pytest.py would mangle a `.test.ts` path.
"""
if not path.exists():
return {}
outcomes: Dict[str, str] = {}
for case in ET.parse(path).getroot().iter("testcase"):
tid = f"{case.get('classname', '')}::{case.get('name', '')}"
if case.find("error") is not None:
outcome = evidence.ERROR
elif case.find("failure") is not None:
outcome = evidence.FAIL
elif case.find("skipped") is not None:
outcome = evidence.SKIP
else:
outcome = evidence.PASS
if outcomes.get(tid) == evidence.PASS:
continue
outcomes[tid] = outcome
return outcomes
def main(argv: Optional[List[str]] = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--evidence-dir", type=Path, default=EVIDENCE_DIR)
parser.add_argument("--browser", type=Path, help="camoufox-bin for the end-to-end suite")
parser.add_argument("--python", type=Path, default=Path(sys.executable),
help="interpreter with pythonlib installed: the golden fixtures are "
"recorded from it, and the e2e suite compares against it")
parser.add_argument("--timeout", type=int, default=1800)
args = parser.parse_args(argv)
gate = "typescript_browser" if args.browser else "typescript"
result = evidence.GateResult(gate=gate)
if not (TYPESCRIPT / "package.json").is_file():
result.note("typescript/package.json does not exist")
result.finish(evidence.ERROR).save(args.evidence_dir)
return 1
env = dict(os.environ)
# tests/golden-setup.ts records the golden fixtures from this interpreter.
env["CAMOUFOX_PYTHON"] = str(args.python.absolute())
if args.browser:
env.update(
CAMOUFOX_E2E="1",
CAMOUFOX_EXECUTABLE=str(args.browser.resolve()),
CAMOUFOX_E2E_PYTHON=str(args.python.absolute()),
)
# Every test input must be something a checkout gets. A git-ignored file
# under src/, tests/ or scripts/ exists on the machine that made it and
# nowhere else, so the suite passes there and fails in CI -- which is how
# tests/fixtures/launch/ once went missing from a branch (an unanchored
# `launch` rule in the root .gitignore).
ignored = run(
["git", "ls-files", "--others", "--ignored", "--exclude-standard", "--directory",
"--", "typescript/src", "typescript/tests", "typescript/scripts"],
cwd=REPO_ROOT,
)
# The goldens tests/golden-setup.ts records are ignored on purpose, by
# typescript/tests/fixtures/.gitignore; anything another rule hides is stray.
candidates = [p for p in ignored.stdout.split() if "__pycache__" not in p]
rules = run(["git", "check-ignore", "--verbose", "--no-index", *candidates], cwd=REPO_ROOT) \
if candidates else None
generated = {
line.split("\t", 1)[1] for line in (rules.stdout.splitlines() if rules else [])
if line.startswith("typescript/tests/fixtures/.gitignore:")
}
stray = [p for p in candidates if p not in generated]
for path in stray:
result.note(f"git-ignored test input: {path}")
result.record("no test input is git-ignored", evidence.FAIL if stray else evidence.PASS)
install = run(["pnpm", "install", "--frozen-lockfile"], cwd=TYPESCRIPT, env=env,
timeout=600, tee=True, capture=False)
if not install.ok:
result.note(f"pnpm install exited {install.code}")
result.finish(evidence.ERROR).save(args.evidence_dir)
return 1
# Static checks are recorded as tests of their own, so the summary names
# which one failed instead of reporting a bare non-zero exit.
if not args.browser:
for script in ("typecheck", "check"):
proc = run(["pnpm", script], cwd=TYPESCRIPT, env=env, timeout=600, tee=True, capture=False)
result.record(f"pnpm {script}", evidence.PASS if proc.ok else evidence.FAIL)
# The tarball a user would install: builds, ships every data file, installs
# and imports in an empty project, and its CLI starts. publish-npm.yml runs
# the same check before uploading; running it here means a packaging mistake
# is caught on the pull request that makes it, not on release day.
if not args.browser:
build = run(["pnpm", "build"], cwd=TYPESCRIPT, env=env, timeout=600, tee=True, capture=False)
pack = build.ok and run(["node", "scripts/check-pack.mjs"], cwd=TYPESCRIPT, env=env,
timeout=900, tee=True, capture=False).ok
result.record("npm package (scripts/check-pack.mjs)", evidence.PASS if pack else evidence.FAIL)
junit = WORK_DIR / f"junit-{gate}.xml"
junit.parent.mkdir(parents=True, exist_ok=True)
# The browser gate runs the e2e file alone: the unit suite already ran in
# tier 1, and running it again here would need that job's prerequisites.
files = ["tests/e2e.test.ts"] if args.browser else []
proc = run(
["pnpm", "exec", "vitest", "run", "--config", "tests/vitest.config.ts",
"--reporter=default", "--reporter=junit", f"--outputFile.junit={junit}", *files],
cwd=TYPESCRIPT, env=env, timeout=args.timeout, tee=True, capture=False,
)
outcomes = parse_vitest_junit(junit)
if not outcomes:
result.note(f"vitest exited {proc.code} with no junit output; the suite did not run")
result.finish(evidence.ERROR).save(args.evidence_dir)
return 1
for tid, outcome in outcomes.items():
result.record(tid, outcome)
tally = result.tally()
result.artifacts.append(junit.name)
result.metrics["exit_code"] = proc.code
result.note(
f"{tally.get('pass', 0)} passed, {tally.get('fail', 0)} failed, "
f"{tally.get('error', 0)} errored, {tally.get('skip', 0)} skipped "
f"({tally.get('total', 0)} collected)"
)
failing = tally.get("fail", 0) + tally.get("error", 0)
e2e_passed = sum(1 for t, o in result.tests.items() if "e2e" in t and o == evidence.PASS)
result.metrics["e2e_passed"] = e2e_passed
if args.browser and e2e_passed == 0:
# An e2e run where every browser test skipped proved nothing.
result.note("no end-to-end test ran; CAMOUFOX_E2E did not take effect")
failing += 1
status = evidence.PASS if failing == 0 else evidence.FAIL
result.finish(status).save(args.evidence_dir)
return 0 if status == evidence.PASS else 1
if __name__ == "__main__":
sys.exit(main())
ci/summarize.py
+2 -2
View File
@@ -52,8 +52,8 @@ _UNIONED_METRICS = frozenset({"main_world_fallbacks"})
# Presented in this order; anything unexpected is appended.
_ORDER = [
"native_rules", "pythonlib", "patches_apply", "build", "patch_guards",
"skiplist_audit", "native_browser", "build_tester", "playwright", "sundial",
"native_rules", "pythonlib", "typescript", "patches_apply", "build", "patch_guards",
"skiplist_audit", "native_browser", "build_tester", "playwright", "typescript_browser", "sundial",
]
ci/tests/test_ci.py
+88 -2
View File
@@ -952,7 +952,7 @@ def test_required_suites_are_names_a_runner_actually_writes():
producible = {
"build", "build_tester", "patch_guards", "pythonlib", "sundial",
"native", "native_rules", "native_browser", "native_growth",
"playwright", "skiplist_audit",
"playwright", "skiplist_audit", "typescript", "typescript_browser",
}
unknown = required - producible
assert not unknown, (
@@ -2260,7 +2260,7 @@ def test_the_native_inputs_cover_everything_that_can_change_the_binary():
from ci.browser_inputs import BROWSER_DIRS, BROWSER_FILES
text = WORKFLOW.read_text(encoding="utf-8")
scope = re.search(r"grep -qE '\^\(([^)]*)\)'", text)
scope = re.search(r"sources='\^\(([^)]*)\)'", text)
assert scope, "the browser_changed grep is gone or was reshaped"
considered = {
part.replace("\\", "").rstrip("/") for part in scope.group(1).split("|") if part
@@ -2623,3 +2623,89 @@ def test_build_tester_accepts_every_core_count_pythonlib_presents():
f"build-tester's plausibleHWC rejects core counts pythonlib presents: {missing}. "
"Add them to the list in build-tester/src/lib/checks/extended.ts."
)
# ---------------------------------------------------------------------------
# build-or-fetch: the published release is used only when it matches the tree
# ---------------------------------------------------------------------------
def _scope(repo: pathlib.Path, base: str) -> tuple[str, str]:
"""Run the workflow's own "Does this change the browser?" step in `repo`."""
import subprocess
text = WORKFLOW.read_text(encoding="utf-8")
start = text.index(" run: |\n if [ \"${{ github.event_name }}\" != \"pull_request\" ]")
end = text.index(" - name: May the stealth check run?", start)
lines = text[start:end].splitlines()[1:]
block = "\n".join(line[10:] if line.startswith(" " * 10) else line.strip() for line in lines)
block = block.replace("${{ github.event_name }}", "pull_request")
block = block.replace("${{ github.event.pull_request.base.sha }}", base)
out = repo / "out.txt"
proc = subprocess.run(
["bash", "-e", "-c", block], cwd=repo, capture_output=True, text=True,
env={**os.environ, "GITHUB_OUTPUT": str(out)},
)
assert proc.returncode == 0, proc.stderr
return out.read_text().strip(), proc.stdout
def _git(repo: pathlib.Path, *args: str) -> str:
import subprocess
return subprocess.run(
["git", "-c", "user.email=ci@test", "-c", "user.name=ci", *args],
cwd=repo, check=True, capture_output=True, text=True,
).stdout.strip()
@pytest.fixture
def release_repo(tmp_path):
"""A repo whose v1.0-beta.1 tag is the published release, and a main after it."""
repo = tmp_path / "repo"
repo.mkdir()
_git(repo, "init", "-q", "-b", "main")
(repo / "upstream.sh").write_text("version=1.0\nrelease=beta.1\n")
(repo / "patches").mkdir()
(repo / "patches" / "a.patch").write_text("a\n")
(repo / "typescript").mkdir()
(repo / "typescript" / "x.ts").write_text("x\n")
_git(repo, "add", "-A")
_git(repo, "commit", "-qm", "release")
_git(repo, "tag", "v1.0-beta.1")
return repo
def test_driver_pr_on_the_released_sources_tests_the_release(release_repo):
base = _git(release_repo, "rev-parse", "HEAD")
(release_repo / "typescript" / "x.ts").write_text("y\n")
_git(release_repo, "commit", "-qam", "driver change")
assert _scope(release_repo, base)[0] == "browser_changed=false"
def test_driver_pr_on_unreleased_browser_sources_builds(release_repo):
"""#785: main had merged browser patches (#779) that no release carried yet,
so a TS-only pull request fetched beta.31 and ran #779's patch guards on it."""
(release_repo / "patches" / "a.patch").write_text("merged but unreleased\n")
_git(release_repo, "commit", "-qam", "browser change merged to main")
base = _git(release_repo, "rev-parse", "HEAD")
(release_repo / "typescript" / "x.ts").write_text("y\n")
_git(release_repo, "commit", "-qam", "driver change")
result, log = _scope(release_repo, base)
assert result == "browser_changed=true"
assert "patches/a.patch" in log
def test_browser_pr_builds(release_repo):
base = _git(release_repo, "rev-parse", "HEAD")
(release_repo / "patches" / "a.patch").write_text("b\n")
_git(release_repo, "commit", "-qam", "browser change")
assert _scope(release_repo, base)[0] == "browser_changed=true"
def test_unpublished_release_tag_builds(release_repo):
_git(release_repo, "tag", "-d", "v1.0-beta.1")
base = _git(release_repo, "rev-parse", "HEAD")
(release_repo / "typescript" / "x.ts").write_text("y\n")
_git(release_repo, "commit", "-qam", "driver change")
assert _scope(release_repo, base)[0] == "browser_changed=true"
native-tests/test_crash_recovery.py
+6
View File
@@ -637,6 +637,12 @@ async def test_the_parent_stays_flat_across_content_crashes(binary, psutil_mod,
manager, browser = await open_browser(binary)
parents = wait_for_process(psutil_mod, BROWSER)
parent = parents[0]
# The baseline follows one clean context. The parent's first context costs
# it 150-200 MB with no crash at all, and a baseline taken before that
# counted the warm-up as crash growth: 330-370 MB of the 400 MB allowance
# locally, and over it on a CI runner.
context, _ = await a_page(browser)
await close_bounded(context, 30, "the warm-up context")
baseline = parent.memory_info().rss
for _ in range(3):
native-tests/test_tribal_rules.py
+4 -4
View File
@@ -577,10 +577,10 @@ def test_no_canvas_seed_is_declared_or_sent():
for entry in json.loads((REPO_ROOT / "settings" / "properties.json").read_text())
}
assert not {k for k in declared if k.startswith("canvas:")}, explain("canvas-is-not-noised")
fingerprints = REPO_ROOT / "pythonlib" / "camoufox" / "fingerprints.py"
assert "setCanvasSeed" not in fingerprints.read_text(encoding="utf-8"), (
"fingerprints.py still calls setCanvasSeed" + explain("canvas-is-not-noised")
)
for launcher in ("pythonlib/camoufox/fingerprints.py", "typescript/src/fingerprints.ts"):
assert "setCanvasSeed" not in (REPO_ROOT / launcher).read_text(encoding="utf-8"), (
f"{launcher} still calls setCanvasSeed" + explain("canvas-is-not-noised")
)
def test_the_canvas_check_hashes_pixels_rather_than_a_data_url_prefix():
pythonlib/README.md
+1 -1
View File
@@ -238,7 +238,7 @@ Display the Python package version, active browser version, channel, and update
```bash
> camoufox version
Python Packages
Camoufox v0.5.6
Camoufox v0.5.7
fpgen v1.3.0
Playwright v1.62.0
Browser
pythonlib/camoufox/essential-fonts.json
+5
View File
File diff suppressed because one or more lines are too long
pythonlib/camoufox/fingerprints.py
+8 -165
View File
@@ -130,177 +130,20 @@ def _load_os_fonts() -> Dict[str, List[str]]:
# the stock CJK families; macOS Sonoma; Ubuntu and its Mint variant),
# intersected with fonts.json so only names the bundle can
# render are listed (Sonoma's PingFang / Kefa / Hiragino families are in the real
# base but not bundled, so they are absent here). Regenerate together with
# fonts.json: `python3 scripts/gen-fonts-json.py --print-bases`.
# base but not bundled, so they are absent here). They live in essential-fonts.json,
# which the TypeScript launcher reads too; regenerate it together with fonts.json:
# `python3 scripts/gen-fonts-json.py --print-bases`.
#
# Windows: the seven GDI-substitution names (Courier, Helvetica, MS Sans Serif,
# MS Serif, Roman, Small Fonts, Times) and the six Light/Semilight names have no
# file of their own; bundle/fontconfig/windows/fonts.conf rewrites each to its
# bundled target unconditionally, so they MUST stay in this always-reported set
# (an identity that did not report Helvetica would still render it otherwise).
_ESSENTIAL_FONTS_MACOS = [
'Academy Engraved LET', 'Al Bayan', 'Al Nile', 'Al Tarikh', 'American Typewriter', 'American Typewriter Semibold',
'Andale Mono', 'Apple Braille', 'Apple Chancery', 'Apple Color Emoji', 'Apple SD Gothic Neo',
'Apple SD Gothic Neo ExtraBold', 'Apple Symbols', 'AppleGothic', 'AppleMyungjo', 'Arial',
'Arial Black', 'Arial Hebrew', 'Arial Hebrew Scholar', 'Arial Narrow', 'Arial Rounded MT Bold',
'Arial Unicode MS', 'Athelas', 'Avenir', 'Avenir Black', 'Avenir Black Oblique', 'Avenir Book',
'Avenir Heavy', 'Avenir Light', 'Avenir Medium', 'Avenir Next', 'Avenir Next Demi Bold',
'Avenir Next Heavy', 'Avenir Next Medium', 'Avenir Next Ultra Light', 'Ayuthaya', 'Baghdad',
'Bangla MN', 'Bangla Sangam MN', 'Baskerville', 'Beirut', 'Big Caslon', 'Bodoni 72', 'Bodoni 72 Oldstyle',
'Bodoni 72 Smallcaps', 'Bodoni Ornaments', 'Bradley Hand', 'Brush Script MT', 'Chalkboard',
'Chalkboard SE', 'Chalkduster', 'Charter', 'Charter Black', 'Cochin', 'Comic Sans MS',
'Copperplate', 'Corsiva Hebrew', 'Courier', 'Courier New', 'DIN Alternate', 'DIN Condensed',
'Damascus', 'DecoType Naskh', 'Devanagari MT', 'Devanagari Sangam MN', 'Didot', 'Diwan Kufi',
'Diwan Thuluth', 'Euphemia UCAS', 'Farah', 'Farisi', 'Futura', 'Futura Bold', 'GB18030 Bitmap',
'Galvji', 'Geeza Pro', 'Geneva', 'Georgia', 'Gill Sans', 'Grantha Sangam MN', 'Gujarati MT',
'Gujarati Sangam MN', 'Gurmukhi MN', 'Gurmukhi MT', 'Gurmukhi Sangam MN', 'Heiti SC', 'Heiti TC',
'Helvetica', 'Helvetica Neue', 'Hiragino Kaku Gothic Pro', 'Hiragino Kaku Gothic Std',
'Hiragino Kaku Gothic StdN', 'Hiragino Maru Gothic Pro', 'Hiragino Maru Gothic ProN', 'Hiragino Maru Gothic ProN W4',
'Hiragino Mincho Pro', 'Hiragino Mincho ProN', 'Hiragino Mincho ProN W3', 'Hiragino Mincho ProN W6',
'Hiragino Sans', 'Hiragino Sans GB', 'Hiragino Sans GB W3', 'Hiragino Sans GB W6', 'Hiragino Sans W0',
'Hiragino Sans W1', 'Hiragino Sans W2', 'Hiragino Sans W3', 'Hiragino Sans W4', 'Hiragino Sans W5',
'Hiragino Sans W6', 'Hiragino Sans W7', 'Hiragino Sans W8', 'Hiragino Sans W9', 'Hoefler Text',
'Hoefler Text Ornaments', 'ITF Devanagari', 'ITF Devanagari Marathi', 'Impact', 'InaiMathi',
'InaiMathi Bold', 'Iowan Old Style', 'Kailasa', 'Kannada MN', 'Kannada Sangam MN', 'Khmer MN',
'Khmer Sangam MN', 'Kohinoor Bangla', 'Kohinoor Devanagari', 'Kohinoor Devanagari Medium',
'Kohinoor Gujarati', 'Kohinoor Telugu', 'Kokonor', 'Krungthep', 'KufiStandardGK', 'Lao MN',
'Lao Sangam MN', 'Lucida Grande', 'Luminari', 'Malayalam MN', 'Malayalam Sangam MN', 'Marker Felt',
'Menlo', 'Microsoft Sans Serif', 'Mishafi', 'Mishafi Gold', 'Monaco', 'Mshtakan', 'MuktaMahee Bold',
'MuktaMahee ExtraBold', 'MuktaMahee ExtraLight', 'MuktaMahee Light', 'MuktaMahee Medium',
'MuktaMahee Regular', 'MuktaMahee SemiBold', 'Muna', 'Myanmar MN', 'Myanmar Sangam MN',
'Nadeem', 'New Peninim MT', 'Noteworthy', 'Noto Nastaliq Urdu', 'Noto Sans Adlam', 'Noto Sans Armenian',
'Noto Sans Armenian Blk', 'Noto Sans Armenian ExtBd', 'Noto Sans Armenian ExtLt', 'Noto Sans Armenian Light',
'Noto Sans Armenian Med', 'Noto Sans Armenian SemBd', 'Noto Sans Armenian Thin', 'Noto Sans Avestan',
'Noto Sans Bamum', 'Noto Sans Bassa Vah', 'Noto Sans Batak', 'Noto Sans Bhaiksuki', 'Noto Sans Buginese',
'Noto Sans Buhid', 'Noto Sans Canadian Aboriginal Regular', 'Noto Sans Carian', 'Noto Sans CaucAlban',
'Noto Sans Chakma', 'Noto Sans Cham', 'Noto Sans Coptic', 'Noto Sans Cuneiform', 'Noto Sans Cypriot',
'Noto Sans Duployan', 'Noto Sans EgyptHiero', 'Noto Sans Elbasan', 'Noto Sans Glagolitic',
'Noto Sans Gothic', 'Noto Sans Gunjala Gondi', 'Noto Sans HanifiRohg', 'Noto Sans Hanunoo',
'Noto Sans Hatran', 'Noto Sans ImpAramaic', 'Noto Sans InsPahlavi', 'Noto Sans InsParthi',
'Noto Sans Javanese', 'Noto Sans Kaithi', 'Noto Sans Kannada', 'Noto Sans Kannada Black',
'Noto Sans Kannada ExtraBold', 'Noto Sans Kannada ExtraLight', 'Noto Sans Kannada Light',
'Noto Sans Kannada Medium', 'Noto Sans Kannada SemiBold', 'Noto Sans Kannada Thin', 'Noto Sans Kayah Li',
'Noto Sans Kharoshthi', 'Noto Sans Khojki', 'Noto Sans Khudawadi', 'Noto Sans Lepcha',
'Noto Sans Limbu', 'Noto Sans Linear A', 'Noto Sans Linear B', 'Noto Sans Lisu', 'Noto Sans Lycian',
'Noto Sans Lydian', 'Noto Sans Mahajani', 'Noto Sans Mandaic', 'Noto Sans Manichaean',
'Noto Sans Marchen', 'Noto Sans Masaram Gondi', 'Noto Sans Mende Kikakui', 'Noto Sans Meroitic',
'Noto Sans Miao', 'Noto Sans Modi', 'Noto Sans Mongolian', 'Noto Sans Mro', 'Noto Sans Multani',
'Noto Sans Myanmar', 'Noto Sans Myanmar Blk', 'Noto Sans Myanmar ExtBd', 'Noto Sans Myanmar ExtLt',
'Noto Sans Myanmar Light', 'Noto Sans Myanmar Med', 'Noto Sans Myanmar SemBd', 'Noto Sans Myanmar Thin',
'Noto Sans NKo', 'Noto Sans Nabataean', 'Noto Sans Newa', 'Noto Sans Ol Chiki', 'Noto Sans Old Italic',
'Noto Sans Old Permic', 'Noto Sans Old Turkic', 'Noto Sans OldHung', 'Noto Sans OldNorArab',
'Noto Sans OldSouArab', 'Noto Sans Oriya', 'Noto Sans Osage', 'Noto Sans Osmanya', 'Noto Sans Pahawh Hmong',
'Noto Sans Palmyrene', 'Noto Sans PhagsPa', 'Noto Sans Phoenician', 'Noto Sans PsaPahlavi',
'Noto Sans Rejang', 'Noto Sans Samaritan', 'Noto Sans Saurashtra', 'Noto Sans Sharada',
'Noto Sans Siddham', 'Noto Sans SoraSomp', 'Noto Sans Sundanese', 'Noto Sans Syloti Nagri',
'Noto Sans Syriac', 'Noto Sans Tagalog', 'Noto Sans Tagbanwa', 'Noto Sans Tai Le', 'Noto Sans Tai Tham',
'Noto Sans Tai Viet', 'Noto Sans Takri', 'Noto Sans Thaana', 'Noto Sans Tifinagh', 'Noto Sans Tirhuta',
'Noto Sans Ugaritic', 'Noto Sans Vai', 'Noto Sans Wancho', 'Noto Sans Yi', 'Noto Sans Zawgyi',
'Noto Sans Zawgyi Blk', 'Noto Sans Zawgyi ExtBd', 'Noto Sans Zawgyi ExtLt', 'Noto Sans Zawgyi Light',
'Noto Sans Zawgyi Med', 'Noto Sans Zawgyi SemBd', 'Noto Sans Zawgyi Thin', 'Noto Serif Ahom',
'Noto Serif Balinese', 'Noto Serif Hmong Nyiakeng', 'Noto Serif Myanmar', 'Noto Serif Myanmar Blk',
'Noto Serif Myanmar ExtBd', 'Noto Serif Myanmar ExtLt', 'Noto Serif Myanmar Light', 'Noto Serif Myanmar Med',
'Noto Serif Myanmar SemBd', 'Noto Serif Myanmar Thin', 'Noto Serif Yezidi', 'Optima', 'Oriya MN',
'Oriya Sangam MN', 'PT Mono', 'PT Sans', 'PT Sans Caption', 'PT Sans Narrow', 'PT Serif',
'PT Serif Caption', 'Palatino', 'Papyrus', 'Party LET', 'Phosphate', 'PingFang HK', 'PingFang SC',
'PingFang TC', 'Plantagenet Cherokee', 'Raanana', 'Rockwell', 'STIX Two Math', 'STIX Two Math Regular',
'STIX Two Text', 'STIX Two Text Regular', 'STIXGeneral', 'STIXIntegralsD', 'STIXIntegralsSm',
'STIXIntegralsUp', 'STIXIntegralsUpD', 'STIXIntegralsUpSm', 'STIXNonUnicode', 'STIXSizeFiveSym',
'STIXSizeFourSym', 'STIXSizeOneSym', 'STIXSizeThreeSym', 'STIXSizeTwoSym', 'STIXVariants',
'STSong', 'Sana', 'Sathu', 'Savoye LET', 'Shree Devanagari 714', 'SignPainter-HouseScript',
'Silom', 'Sinhala MN', 'Sinhala Sangam MN', 'Skia', 'Snell Roundhand', 'Songti SC', 'Songti TC',
'Sukhumvit Set', 'Superclarendon', 'Symbol', 'System Font', 'Tahoma', 'Tamil MN', 'Tamil Sangam MN',
'Telugu MN', 'Telugu Sangam MN', 'Thonburi', 'Times', 'Times New Roman', 'Trattatello',
'Trebuchet MS', 'Verdana', 'Waseem', 'Webdings', 'Wingdings', 'Wingdings 2', 'Wingdings 3',
'Zapf Dingbats', 'Zapfino',
]
_ESSENTIAL_FONTS_WINDOWS = [
'Arial', 'Arial Black', 'Bahnschrift', 'Calibri', 'Calibri Light', 'Cambria', 'Cambria Math',
'Candara', 'Candara Light', 'Comic Sans MS', 'Consolas', 'Constantia', 'Corbel', 'Corbel Light',
'Courier', 'Courier New', 'Ebrima', 'Franklin Gothic Medium', 'Gabriola', 'Gadugi', 'Georgia',
'Helvetica', 'Impact', 'Ink Free', 'Javanese Text', 'Leelawadee UI', 'Leelawadee UI Semilight',
'Lucida Console', 'Lucida Sans Unicode', 'MS Gothic', 'MS PGothic', 'MS Sans Serif', 'MS Serif',
'MS UI Gothic', 'MV Boli', 'Malgun Gothic', 'Malgun Gothic Semilight', 'Marlett', 'Microsoft Himalaya',
'Microsoft JhengHei', 'Microsoft JhengHei Light', 'Microsoft JhengHei UI', 'Microsoft JhengHei UI Light',
'Microsoft New Tai Lue', 'Microsoft PhagsPa', 'Microsoft Sans Serif', 'Microsoft Tai Le',
'Microsoft YaHei', 'Microsoft YaHei Light', 'Microsoft YaHei UI', 'Microsoft YaHei UI Light',
'Microsoft Yi Baiti', 'MingLiU-ExtB', 'MingLiU_HKSCS-ExtB', 'MingLiU_MSCS-ExtB', 'Mongolian Baiti',
'Myanmar Text', 'NSimSun', 'Nirmala Text', 'Nirmala Text Semilight', 'Nirmala UI', 'Nirmala UI Semilight',
'PMingLiU-ExtB', 'Palatino Linotype', 'Roman', 'Sans Serif Collection', 'Segoe Fluent Icons',
'Segoe MDL2 Assets', 'Segoe Print', 'Segoe Script', 'Segoe UI', 'Segoe UI Black', 'Segoe UI Emoji',
'Segoe UI Historic', 'Segoe UI Light', 'Segoe UI Semibold', 'Segoe UI Semilight', 'Segoe UI Symbol',
'Segoe UI Variable', 'Segoe UI Variable Display', 'Segoe UI Variable Small', 'Segoe UI Variable Text',
'SimSun', 'SimSun-ExtB', 'Sitka Banner', 'Sitka Display', 'Sitka Heading', 'Sitka Small',
'Sitka Subheading', 'Sitka Text', 'Small Fonts', 'Sylfaen', 'Symbol', 'Tahoma', 'Times',
'Times New Roman', 'Trebuchet MS', 'Twemoji Mozilla', 'Verdana', 'Webdings', 'Wingdings',
'Yu Gothic', 'Yu Gothic Light', 'Yu Gothic Medium', 'Yu Gothic UI', 'Yu Gothic UI Light',
'Yu Gothic UI Semibold', 'Yu Gothic UI Semilight', '宋体', '微軟正黑體', '微軟正黑體 Light', '微软雅黑',
'微软雅黑 Light', '新宋体', '新細明體-ExtB', '游ゴシック', '游ゴシック Light', '游ゴシック Medium', '細明體-ExtB', '細明體_HKSCS-ExtB',
'細明體_MSCS-ExtB', '맑은 고딕', '맑은 고딕 Semilight', 'ＭＳ ゴシック', 'ＭＳ Ｐゴシック',
]
_ESSENTIAL_FONTS_LINUX = [
'AR PL UKai CN', 'AR PL UKai HK', 'AR PL UKai TW', 'AR PL UKai TW MBE', 'AR PL UMing CN',
'AR PL UMing HK', 'AR PL UMing TW', 'AR PL UMing TW MBE', 'Arial', 'Arial Narrow', 'Avant Garde',
'Bookman Old Style', 'C059', 'Calibri', 'Cambria', 'Century Schoolbook', 'Courier', 'Courier New',
'D050000L', 'DejaVu Sans', 'DejaVu Sans Mono', 'DejaVu Serif', 'Droid Sans Fallback', 'Helvetica',
'Helvetica Narrow', 'Liberation Mono', 'Liberation Sans', 'Liberation Sans Narrow', 'Liberation Serif',
'Nimbus Mono PS', 'Nimbus Roman', 'Nimbus Sans', 'Nimbus Sans Narrow', 'Noto Color Emoji',
'Noto Kufi Arabic', 'Noto Looped Lao', 'Noto Looped Lao Bold', 'Noto Looped Lao Regular',
'Noto Looped Thai', 'Noto Looped Thai Bold', 'Noto Looped Thai Regular', 'Noto Mono', 'Noto Music',
'Noto Naskh Arabic', 'Noto Nastaliq Urdu', 'Noto Rashi Hebrew', 'Noto Sans', 'Noto Sans Adlam',
'Noto Sans Adlam Unjoined', 'Noto Sans AnatoHiero', 'Noto Sans Anatolian Hieroglyphs',
'Noto Sans Arabic', 'Noto Sans Armenian', 'Noto Sans Avestan', 'Noto Sans Balinese', 'Noto Sans Bamum',
'Noto Sans Bassa Vah', 'Noto Sans Batak', 'Noto Sans Bengali', 'Noto Sans Bhaiksuki', 'Noto Sans Brahmi',
'Noto Sans Buginese', 'Noto Sans Buhid', 'Noto Sans CJK HK', 'Noto Sans CJK JP', 'Noto Sans CJK KR',
'Noto Sans CJK SC', 'Noto Sans CJK TC', 'Noto Sans CanAborig', 'Noto Sans Canadian Aboriginal',
'Noto Sans Carian', 'Noto Sans CaucAlban', 'Noto Sans Caucasian Albanian', 'Noto Sans Chakma',
'Noto Sans Cham', 'Noto Sans Cherokee', 'Noto Sans Coptic', 'Noto Sans Cuneiform', 'Noto Sans Cypriot',
'Noto Sans Deseret', 'Noto Sans Devanagari', 'Noto Sans Display', 'Noto Sans Duployan',
'Noto Sans EgyptHiero', 'Noto Sans Egyptian Hieroglyphs', 'Noto Sans Elbasan', 'Noto Sans Elymaic',
'Noto Sans Ethiopic', 'Noto Sans Georgian', 'Noto Sans Glagolitic', 'Noto Sans Gothic',
'Noto Sans Grantha', 'Noto Sans Gujarati', 'Noto Sans Gunjala Gondi', 'Noto Sans Gurmukhi',
'Noto Sans Hanifi Rohingya', 'Noto Sans Hanunoo', 'Noto Sans Hatran', 'Noto Sans Hebrew',
'Noto Sans ImpAramaic', 'Noto Sans Imperial Aramaic', 'Noto Sans Indic Siyaq Numbers',
'Noto Sans InsPahlavi', 'Noto Sans InsParthi', 'Noto Sans Inscriptional Pahlavi', 'Noto Sans Inscriptional Parthian',
'Noto Sans Javanese', 'Noto Sans Kaithi', 'Noto Sans Kannada', 'Noto Sans Kayah Li', 'Noto Sans Kharoshthi',
'Noto Sans Khmer', 'Noto Sans Khojki', 'Noto Sans Khudawadi', 'Noto Sans Lao', 'Noto Sans Lepcha',
'Noto Sans Limbu', 'Noto Sans Linear A', 'Noto Sans Linear B', 'Noto Sans Lisu', 'Noto Sans Lycian',
'Noto Sans Lydian', 'Noto Sans Mahajani', 'Noto Sans Malayalam', 'Noto Sans Mandaic', 'Noto Sans Manichaean',
'Noto Sans Marchen', 'Noto Sans Masaram Gondi', 'Noto Sans Math', 'Noto Sans Mayan Numerals',
'Noto Sans Medefaidrin', 'Noto Sans Meetei Mayek', 'Noto Sans Mende Kikakui', 'Noto Sans Meroitic',
'Noto Sans Miao', 'Noto Sans Modi', 'Noto Sans Mongolian', 'Noto Sans Mono', 'Noto Sans Mono CJK HK',
'Noto Sans Mono CJK JP', 'Noto Sans Mono CJK KR', 'Noto Sans Mono CJK SC', 'Noto Sans Mono CJK TC',
'Noto Sans Mro', 'Noto Sans Multani', 'Noto Sans Myanmar', 'Noto Sans NKo', 'Noto Sans Nabataean',
'Noto Sans New Tai Lue', 'Noto Sans Newa', 'Noto Sans Nushu', 'Noto Sans Ogham', 'Noto Sans Ol Chiki',
'Noto Sans Old Hungarian', 'Noto Sans Old Italic', 'Noto Sans Old North Arabian', 'Noto Sans Old Permic',
'Noto Sans Old Persian', 'Noto Sans Old Sogdian', 'Noto Sans Old South Arabian', 'Noto Sans Old Turkic',
'Noto Sans OldHung', 'Noto Sans OldNorArab', 'Noto Sans OldSouArab', 'Noto Sans Oriya',
'Noto Sans Osage', 'Noto Sans Osmanya', 'Noto Sans Pahawh Hmong', 'Noto Sans Palmyrene',
'Noto Sans Pau Cin Hau', 'Noto Sans PhagsPa', 'Noto Sans Phoenician', 'Noto Sans PsaPahlavi',
'Noto Sans Psalter Pahlavi', 'Noto Sans Rejang', 'Noto Sans Runic', 'Noto Sans Samaritan',
'Noto Sans Saurashtra', 'Noto Sans Sharada', 'Noto Sans Shavian', 'Noto Sans Siddham',
'Noto Sans SignWrit', 'Noto Sans SignWriting', 'Noto Sans Sinhala', 'Noto Sans Sogdian',
'Noto Sans Sora Sompeng', 'Noto Sans Soyombo', 'Noto Sans Sundanese', 'Noto Sans Syloti Nagri',
'Noto Sans Symbols', 'Noto Sans Symbols2', 'Noto Sans Syriac', 'Noto Sans Tagalog', 'Noto Sans Tagbanwa',
'Noto Sans Tai Le', 'Noto Sans Tai Tham', 'Noto Sans Tai Viet', 'Noto Sans Takri', 'Noto Sans Tamil',
'Noto Sans Tamil Supplement', 'Noto Sans Telugu', 'Noto Sans Thaana', 'Noto Sans Thai',
'Noto Sans Tifinagh', 'Noto Sans Tifinagh APT', 'Noto Sans Tifinagh Adrar', 'Noto Sans Tifinagh Agraw Imazighen',
'Noto Sans Tifinagh Ahaggar', 'Noto Sans Tifinagh Air', 'Noto Sans Tifinagh Azawagh', 'Noto Sans Tifinagh Ghat',
'Noto Sans Tifinagh Hawad', 'Noto Sans Tifinagh Rhissa Ixa', 'Noto Sans Tifinagh SIL',
'Noto Sans Tifinagh Tawellemmet', 'Noto Sans Tirhuta', 'Noto Sans Ugaritic', 'Noto Sans Vai',
'Noto Sans Wancho', 'Noto Sans Warang Citi', 'Noto Sans Yi', 'Noto Sans Zanabazar', 'Noto Sans Zanabazar Square',
'Noto Serif', 'Noto Serif Ahom', 'Noto Serif Armenian', 'Noto Serif Balinese', 'Noto Serif Bengali',
'Noto Serif CJK HK', 'Noto Serif CJK JP', 'Noto Serif CJK KR', 'Noto Serif CJK SC', 'Noto Serif CJK TC',
'Noto Serif Devanagari', 'Noto Serif Display', 'Noto Serif Dogra', 'Noto Serif Ethiopic',
'Noto Serif Georgian', 'Noto Serif Grantha', 'Noto Serif Gujarati', 'Noto Serif Gurmukhi',
'Noto Serif Hebrew', 'Noto Serif Hmong Nyiakeng', 'Noto Serif Kannada', 'Noto Serif Khmer',
'Noto Serif Khojki', 'Noto Serif Lao', 'Noto Serif Malayalam', 'Noto Serif Myanmar', 'Noto Serif Sinhala',
'Noto Serif Tamil', 'Noto Serif Tamil Slanted', 'Noto Serif Tangut', 'Noto Serif Telugu',
'Noto Serif Thai', 'Noto Serif Tibetan', 'Noto Serif Yezidi', 'Noto Traditional Nushu',
'OpenSymbol', 'P052', 'Palatino', 'Palatino Linotype', 'Standard Symbols PS', 'Symbol',
'Times', 'Times New Roman', 'URW Bookman', 'URW Gothic', 'Ubuntu', 'Ubuntu Mono', 'Ubuntu Sans',
'Ubuntu Sans Mono', 'Z003', 'Zapf Chancery',
]
with open(os.path.join(os.path.dirname(__file__), 'essential-fonts.json'), 'rb') as _f:
_ESSENTIAL = json.loads(_f.read())
_ESSENTIAL_FONTS_WINDOWS: List[str] = _ESSENTIAL['win']
_ESSENTIAL_FONTS_MACOS: List[str] = _ESSENTIAL['mac']
_ESSENTIAL_FONTS_LINUX: List[str] = _ESSENTIAL['lin']
# OS-version variants of the base, drawn ALL-OR-NOTHING on top of the essential
# core with the real-world share of that version (the manifest's base weights). A
pythonlib/pyproject.toml
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
[tool.poetry]
name = "camoufox"
version = "0.5.6"
version = "0.5.7"
description = "Wrapper around Playwright to help launch Camoufox"
authors = ["daijro <daijro.dev@gmail.com>"]
license = "MIT"
scripts/gen-fonts-json.py
+16 -4
View File
@@ -35,9 +35,9 @@ Usage (build machine, after the bundle changed):
The manifest lists, per OS, the base font sets of each OS version (with their
real-world share) and the optional additions (Office, LibreOffice, developer
and web fonts) with their install probability, per OS. --print-bases prints
the OS base lists (intersected with the result) as Python literals for the
_ESSENTIAL_FONTS_* constants in pythonlib/camoufox/fingerprints.py, which must
be kept in step with this file.
the OS base lists (intersected with the result) and writes the essential floor
to pythonlib/camoufox/essential-fonts.json, which both launchers read and which
must be kept in step with this file.
"""
import argparse
import json
@@ -321,6 +321,7 @@ def main():
# onto every identity -- which is what made a macOS 26 identity keep
# claiming 131 Sonoma-only families -- and it must still carry the alias
# names fonts.conf rewrites unconditionally, which always render.
essential = {}
for os_key in OSDIRS:
bases = list(bases_out[os_key].values())
if not bases:
@@ -333,7 +334,18 @@ def main():
suffix = {'win': 'WINDOWS', 'mac': 'MACOS', 'lin': 'LINUX'}[os_key]
print(f'# _ESSENTIAL_FONTS_{suffix}: {len(common)} families '
f'(intersection of {len(bases)} base(s) + aliases)')
print(json.dumps(sorted(common), ensure_ascii=False))
essential[os_key] = sorted(common)
path = os.path.join(REPO, 'pythonlib', 'camoufox', 'essential-fonts.json')
# Both launchers read every OS's list at import, so a file missing one
# breaks `import camoufox`. Keep the old file rather than write that.
missing = [k for k in OSDIRS if k not in essential]
if missing:
sys.exit(f'not writing {path}: no OS bases for {", ".join(missing)} '
f'in the manifest')
with open(path, 'w', encoding='utf-8') as f:
f.write('{\n' + ',\n'.join(f' {json.dumps(k)}: {json.dumps(v, ensure_ascii=False)}'
for k, v in essential.items()) + '\n}\n')
print(f'wrote {path}')
if __name__ == '__main__':
scripts/verify-fonts.py
+5
View File
@@ -70,6 +70,11 @@ def load_constants():
name = node.targets[0].id
if 'FONTS' in name and name.startswith('_'):
consts[name] = ast.literal_eval(node.value)
# The OS bases live in essential-fonts.json, shared with the TS launcher.
with open(os.path.join(REPO, 'pythonlib', 'camoufox', 'essential-fonts.json'), encoding='utf-8') as f:
for key, fonts in json.load(f).items():
suffix = {'win': 'WINDOWS', 'mac': 'MACOS', 'lin': 'LINUX'}[key]
consts[f'_ESSENTIAL_FONTS_{suffix}'] = fonts
return consts
typescript/.gitignore
+6
View File
@@ -0,0 +1,6 @@
node_modules/
yarn.lock
.yarn
.npmrc
dist/
*.mmdb
typescript/LICENSE
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2024-2026 daijro and the Camoufox contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
typescript/README.md
+180
View File
@@ -0,0 +1,180 @@
# camoufox (TypeScript)
This is the JavaScript/TypeScript client for Camoufox. It is a port of the
Python wrapper in [`../pythonlib`](../pythonlib) — it does **not** shell out
to Python.
The two launchers are twins: they read the same `properties.json`, write the
same chunked `CAMOU_CONFIG`, share the same browser install directory, and
ship the same fingerprint presets, font/voice lists and GeoIP configuration.
They draw the same identities too: fingerprints and WebGL devices come from a
TypeScript port of [fpgen](https://github.com/scrapfly/fingerprint-generator)
using the same pinned model, and the per-identity draws (fonts, voices, GPU,
media devices, noise seeds) use a bit-exact port of CPython's `random`, so a
pinned identity presents identically from either language.
## Installation
```bash
npm install @camoufox/camoufox playwright-core
# then download the browser
npx camoufox fetch
```
`playwright-core` is a peer dependency — bring your own version (`<1.63`,
the same ceiling as the Python package). Node 22.15 or newer is required.
## Usage
```javascript
import { Camoufox } from "@camoufox/camoufox";
const browser = await Camoufox({
// any Camoufox option, plus any Playwright Firefox launch option
headless: true,
os: "windows",
geoip: true,
});
const page = await browser.newPage(); // a Playwright Page
await page.goto("https://example.com");
await browser.close();
```
### Persistent profiles
```javascript
const context = await Camoufox({ user_data_dir: "./profiles/alice" });
const page = await context.newPage();
```
### Per-context identities
`NewContext()` gives each context its own fingerprint — real preset or
fpgen-synthesised — with its own audio noise seed. The
values are applied through `addInitScript`, so the setters self-destruct before
any page script runs.
```javascript
import { Camoufox, NewContext } from "@camoufox/camoufox";
const browser = await Camoufox({ headless: true });
const context = await NewContext(browser, {
os: "macos",
proxy: { server: "http://proxy:8080", username: "u", password: "p" },
});
```
When a `proxy` is given and no `webrtc_ip`/`timezoneId` is, both are resolved
from the proxy's exit IP. If that lookup fails, `NewContext()` throws
`InvalidIP` rather than open a context that would show the host's values.
### Server mode
```javascript
import { launchServer } from "@camoufox/camoufox";
const server = await launchServer({ headless: true, port: 9222 });
console.log(server.wsEndpoint());
```
Persistent contexts are not servable — Playwright's `launchServer` can only
expose a pre-launched `Browser`.
### Building launch options yourself
```javascript
import { launchOptions } from "@camoufox/camoufox";
import { firefox } from "playwright-core";
const browser = await firefox.launch(await launchOptions({ os: "linux" }));
```
## Options
Every option from the Python `launch_options()` is supported, with the same
snake_case names: `os`, `config`, `block_images`, `block_webrtc`,
`block_webgl`, `disable_coop`, `webgl_config`, `geoip`, `geoip_db`, `humanize`,
`locale`, `addons`, `fonts`, `custom_fonts_only`, `exclude_addons`, `screen`,
`window`, `fingerprint`, `fingerprint_preset`, `ff_version`, `headless`,
`main_world_eval`, `allow_addon_new_tab`, `executable_path`, `browser`,
`firefox_user_prefs`, `proxy`, `enable_cache`, `args`, `env`,
`i_know_what_im_doing`, `debug`, `virtual_display`, `pin_cpu_cores`. Anything else is passed
straight through to Playwright.
The returned launch options use Playwright's camelCase keys
(`executablePath`, `firefoxUserPrefs`) rather than Python's snake_case. As in
Python, `headless: "virtual"` is handled by `Camoufox()`, `NewBrowser()` and
`launchServer()`, not by `launchOptions()`.
## CLI
```
camoufox sync # refresh the version catalogue
camoufox fetch [version] # install the active or a specific version
camoufox set [specifier] # pin a version or channel; no specifier opens a picker
camoufox set --geoip # pick a GeoIP source
camoufox list [installed|all] # list versions
camoufox remove [version] # remove one version, or everything (--select to pick)
camoufox active # print the active version
camoufox path # print the install directory
camoufox version # version / storage info
camoufox test [url] # open the Playwright inspector
camoufox server # launch a Playwright server
```
The commands and pickers match the Python CLI. The one exception is `gui`, a
PySide6 desktop app that only the Python package provides.
## Development
The tests need a Python with pythonlib next to them, at the repo root (or
point `CAMOUFOX_PYTHON` at one):
```bash
python3.14 -m venv .venv # repo root
.venv/bin/pip install -r ci/requirements.txt -e pythonlib
.venv/bin/python scripts/pin-fpgen-model.py
cd typescript
pnpm install
pnpm build # tsc -> dist/, then copy pythonlib's data files into dist/data-files
pnpm test # records the golden fixtures from pythonlib, then vitest
pnpm check # biome lint + format
pnpm typecheck # tsc --noEmit
```
The data files (presets, fonts, voices, territoryInfo.xml, ...) are read from
`pythonlib/camoufox/`, the only copy in the repo; `DATA_FILES` in
`src/paths.ts` lists them, and the build copies them into the package.
### Parity with pythonlib
The golden tests are what keep the two launchers twins. Before the suite runs,
`tests/golden-setup.ts` runs the scripts under `scripts/golden/`, which put the
Python code through hundreds of fixed inputs and record its output in
`tests/fixtures/` (git-ignored); the TS tests must reproduce it exactly --
`launch_options()` byte for byte, including the `CAMOU_CONFIG` blob. A
pythonlib change that is not mirrored here fails `pnpm test`. Use Python 3.14:
`pySum()` follows its `sum()`, and on 3.12/3.13 that one test skips.
The end-to-end suite launches a real browser through both launchers and compares
what a page sees:
```bash
CAMOUFOX_E2E=1 CAMOUFOX_EXECUTABLE=/path/to/camoufox-bin pnpm test tests/e2e.test.ts
```
## Releasing
`.github/workflows/publish-npm.yml` is dispatched by hand, like the PyPI
workflow. It type-checks, lints, tests, builds, runs `scripts/check-pack.mjs`
(the version must equal pythonlib's; every data file must be in the tarball;
the tarball must install and import in an empty project), then publishes with
npm trusted publishing -- no token is stored.
## Licence
MIT, like the Python package ([`LICENSE`](LICENSE)); the browser itself is
MPL-2.0. The package contains ports of fpgen (Apache-2.0), CPython's `random`
and NumPy's pairwise summation; their notices are in
[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md), which ships with it.
typescript/THIRD_PARTY_NOTICES.md
+158
View File
@@ -0,0 +1,158 @@
# Third-party notices
The Camoufox TypeScript launcher is MIT-licensed (see `LICENSE`). It contains
ports of the third-party code below, each under its own licence.
## fpgen — `src/fpgen/`
A TypeScript port of [fpgen](https://github.com/scrapfly/fingerprint-generator)
1.3.0, licensed under the Apache License 2.0. See `src/fpgen/NOTICE`, which
ships with the package.
## CPython `random` — `src/pyrandom.ts`
A port of CPython's `random.Random`. The MT19937 core follows
`Modules/_randommodule.c`:
```
Copyright (C) 1997 - 2002, Makoto Matsumoto and Takuji Nishimura,
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The names of its contributors may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
```
The methods follow `Lib/random.py`, which is distributed under the Python
Software Foundation License:
```
PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
--------------------------------------------
1. This LICENSE AGREEMENT is between the Python Software Foundation
("PSF"), and the Individual or Organization ("Licensee") accessing and
otherwise using this software ("Python") in source or binary form and
its associated documentation.
2. Subject to the terms and conditions of this License Agreement, PSF hereby
grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
analyze, test, perform and/or display publicly, prepare derivative works,
distribute, and otherwise use Python alone or in any derivative version,
provided, however, that PSF's License Agreement and PSF's notice of copyright,
i.e., "Copyright (c) 2001-2024 Python Software Foundation; All Rights Reserved"
are retained in Python alone or in any derivative version prepared by Licensee.
3. In the event Licensee prepares a derivative work that is based on
or incorporates Python or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python.
4. PSF is making Python available to Licensee on an "AS IS"
basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.
5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.
7. Nothing in this License Agreement shall be deemed to create any
relationship of agency, partnership, or joint venture between PSF and
Licensee. This License Agreement does not grant permission to use PSF
trademarks or trade name in a trademark sense to endorse or promote
products or services of Licensee, or any third party.
8. By copying, installing or otherwise using Python, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.
```
## NumPy — `src/locales.ts`
A port of NumPy's pairwise float64 summation (`np.sum`):
```
Copyright (c) 2005-2025, NumPy Developers.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided
with the distribution.
* Neither the name of the NumPy Developers nor the names of any
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
```
The PCG64 bit generator it drives:
```
PCG Random Number Generation for C.
Copyright 2014 Melissa O'Neill <oneill@pcg-random.org>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
```
typescript/biome.json
+40
View File
@@ -0,0 +1,40 @@
{
"$schema": "https://biomejs.dev/schemas/2.4.10/schema.json",
"vcs": {
"enabled": false,
"clientKind": "git",
"useIgnoreFile": false
},
"files": {
"ignoreUnknown": false,
"includes": ["src/**/*.ts", "tests/*.ts"]
},
"formatter": {
"enabled": true,
"indentStyle": "tab"
},
"linter": {
"enabled": true,
"rules": {
"recommended": true,
"suspicious": {
"noDocumentCookie": "off",
"noExplicitAny": "off",
"noImplicitAnyLet": "off"
}
}
},
"javascript": {
"formatter": {
"quoteStyle": "double"
}
},
"assist": {
"enabled": true,
"actions": {
"source": {
"organizeImports": "on"
}
}
}
}
typescript/package.json
+88
View File
@@ -0,0 +1,88 @@
{
"name": "@camoufox/camoufox",
"version": "0.5.7",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"scripts": {
"build": "rimraf dist && tsc && pnpm copy-files",
"copy-files": "node scripts/copy-files.mjs",
"test": "vitest run --config tests/vitest.config.ts",
"test:watch": "vitest --config tests/vitest.config.ts",
"check": "biome check",
"typecheck": "tsc --noEmit",
"prepare": "pnpm build",
"check:dist": "node scripts/check-dist-fresh.mjs",
"check:pack": "node scripts/check-pack.mjs"
},
"bin": {
"camoufox": "dist/__main__.js"
},
"files": [
"dist",
"THIRD_PARTY_NOTICES.md"
],
"type": "module",
"keywords": [
"camoufox",
"firefox",
"playwright",
"fingerprint",
"anti-detect",
"scraping"
],
"author": "",
"repository": {
"type": "git",
"url": "git+https://github.com/daijro/camoufox.git",
"directory": "typescript"
},
"bugs": {
"url": "https://github.com/daijro/camoufox/issues"
},
"engines": {
"node": ">= 22.15"
},
"license": "MIT",
"description": "Camoufox: TypeScript launcher for the Camoufox anti-detect Firefox browser. A port of the Python wrapper.",
"//playwright-core": "Range mirrors pythonlib/pyproject.toml's `playwright = \"<1.63\"`: every Playwright minor is free to change Juggler, so the ceiling is bumped deliberately, with a run of `make tests`. The dev pin (1.62.0) is the version the Python venv resolves, so the goldens compare like with like.",
"packageManager": "pnpm@10.33.4",
"dependencies": {
"adm-zip": "^0.5.16",
"cli-progress": "^3.12.0",
"commander": "^14.0.0",
"impit": "^0.14.1",
"language-tags": "^2.0.1",
"maxmind": "^5.0.0",
"pretty-bytes": "^7.1.0",
"ua-parser-js": "^2.0.2",
"xml2js": "^0.6.2",
"yaml": "^2.9.1"
},
"devDependencies": {
"@biomejs/biome": "2.4.10",
"@types/adm-zip": "^0.5.7",
"@types/cli-progress": "^3.11.6",
"@types/language-tags": "^1.0.4",
"@types/node": "^24.0.0",
"@types/xml2js": "^0.4.14",
"playwright-core": "1.62.0",
"rimraf": "^6.0.1",
"typescript": "^5.8.3",
"vitest": "^4.0.0"
},
"peerDependencies": {
"playwright-core": "<1.63"
},
"devEngines": {
"packageManager": {
"name": "pnpm",
"version": "10.33.4",
"onFail": "warn"
}
},
"//engines": "22.15 for node:zlib zstd, which the fpgen model archive needs.",
"publishConfig": {
"access": "public",
"provenance": true
}
}
typescript/pnpm-lock.yaml
Generated
+1267
View File
File diff suppressed because it is too large Load Diff
typescript/pnpm-workspace.yaml
+20
View File
@@ -0,0 +1,20 @@
# 24h quarantine on anything freshly published, as supply-chain cover.
minimumReleaseAge: 1440
# captchakraken is exempt because it is OURS -- JWriter20/CaptchaKraken, the
# same hands that publish this package. The window exists to put distance
# between a compromised third-party release and our lockfile; it buys nothing
# against a repo we control, and it does cost something: the camoufox bridge and
# the solver ship in step, so a same-day captchakraken release is the normal
# case here, not a red flag. Scoped to this one name, never widened.
minimumReleaseAgeExclude:
- captchakraken
onlyBuiltDependencies:
- esbuild
- playwright-core
nodeLinker: hoisted
linkWorkspacePackages: true
preferWorkspacePackages: true
publicHoistPattern:
- "*"
typescript/renovate.json
+7
View File
@@ -0,0 +1,7 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended"],
"dependencyDashboard": false,
"minimumReleaseAge": "1 day",
"internalChecksFilter": "strict"
}
typescript/scripts/check-dist-fresh.mjs
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env node
/**
* Fail if `dist/` is missing or older than `src/`.
*
* `src/captcha.ts` was added and then sat uncompiled: `dist/captcha.js` did not
* exist at all, so `import { solveCaptcha } from "camoufox"` resolved to a build
* that had never heard of it. Nothing complained — `dist/` is gitignored, the
* package still imported, and only the missing export gave it away at runtime,
* as `CaptchaSolverUnavailable`, which reads like a missing optional dependency
* rather than a stale build.
*
* `prepare` now rebuilds on install, which covers the common path. This is the
* guard for the rest: run it in CI or a pre-commit hook and a stale build is a
* red check instead of a confusing runtime error.
*/
import { readdirSync, statSync, existsSync } from "node:fs";
import { join, relative } from "node:path";
import { fileURLToPath } from "node:url";
const root = fileURLToPath(new URL("..", import.meta.url));
const src = join(root, "src");
const dist = join(root, "dist");
if (!existsSync(dist)) {
console.error("dist/ does not exist — run `pnpm build`.");
process.exit(1);
}
/** Every file under `dir`, recursively. */
function walk(dir) {
const out = [];
for (const entry of readdirSync(dir, { withFileTypes: true })) {
const full = join(dir, entry.name);
if (entry.isDirectory()) out.push(...walk(full));
else out.push(full);
}
return out;
}
const newestDist = Math.max(...walk(dist).map((f) => statSync(f).mtimeMs));
// Sources newer than the newest build output. Anything here is not in dist.
const stale = walk(src)
.filter((f) => statSync(f).mtimeMs > newestDist)
.map((f) => relative(root, f));
// A .ts source with no corresponding .js in dist was never compiled at all —
// the captcha.ts case, and the one that actually bit.
const missing = walk(src)
.filter((f) => f.endsWith(".ts") && !f.endsWith(".d.ts"))
.filter((f) => !existsSync(join(dist, relative(src, f).replace(/\.ts$/, ".js"))))
.map((f) => relative(root, f));
if (missing.length || stale.length) {
if (missing.length) {
console.error("Never compiled into dist/:");
for (const f of missing) console.error(` ${f}`);
}
if (stale.length) {
console.error("Newer than dist/:");
for (const f of stale) console.error(` ${f}`);
}
console.error("\nRun `pnpm build`.");
process.exit(1);
}
console.log("dist/ is up to date with src/");
typescript/scripts/check-pack.mjs
+102
View File
@@ -0,0 +1,102 @@
#!/usr/bin/env node
/**
* The npm twin of `twine check`: prove the tarball `npm publish` would upload is
* a working package before it goes anywhere.
*
* 1. its version equals pythonlib's (the two launchers ship in lockstep, and
* a user comparing `camoufox version` across them should see one number);
* 2. it carries every file src/ reads at runtime -- the DATA_FILES it takes
* from pythonlib and any non-TS file under src/ -- since a file missing
* only shows up on a user's machine, as an ENOENT from inside dist/;
* 3. installed into an empty project, it imports and exposes its entry
* points, and its CLI starts.
*
* Run after `pnpm build`: node scripts/check-pack.mjs
*/
import { execFileSync } from "node:child_process";
import { mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, relative } from "node:path";
import { fileURLToPath } from "node:url";
const root = fileURLToPath(new URL("..", import.meta.url));
const pkg = JSON.parse(readFileSync(join(root, "package.json"), "utf8"));
const problems = [];
// 1. version lockstep with pythonlib
const pyproject = readFileSync(join(root, "..", "pythonlib", "pyproject.toml"), "utf8");
const pyVersion = pyproject.match(/^version\s*=\s*"([^"]+)"/m)?.[1];
if (pyVersion !== pkg.version) {
problems.push(`package.json version ${pkg.version} != pythonlib ${pyVersion}`);
}
if (pkg.private) problems.push("package.json is private: npm will refuse to publish it");
// 2. every runtime data file is in the tarball
// npm 10 still runs `prepare` (the build) on pack despite --ignore-scripts, and
// its banner lands on stdout ahead of the JSON.
const packOut = execFileSync("npm", ["pack", "--dry-run", "--json", "--ignore-scripts"], {
cwd: root,
encoding: "utf8",
});
const packed = JSON.parse(packOut.slice(packOut.search(/^\[/m)))[0];
const inTarball = new Set(packed.files.map((f) => f.path));
function walk(dir) {
return readdirSync(dir, { withFileTypes: true }).flatMap((e) =>
e.isDirectory() ? walk(join(dir, e.name)) : [join(dir, e.name)],
);
}
const src = join(root, "src");
for (const file of walk(src)) {
const rel = relative(src, file);
if (/\.ts$/.test(rel)) {
const js = `dist/${rel.replace(/\.ts$/, ".js")}`;
if (!rel.endsWith(".d.ts") && !inTarball.has(js)) problems.push(`missing ${js}`);
} else if (!inTarball.has(`dist/${rel}`)) {
problems.push(`missing dist/${rel} (a non-TS file under src/ that copy-files does not ship)`);
}
}
const { DATA_FILES } = await import(join(root, "dist", "paths.js"));
for (const name of DATA_FILES) {
if (!inTarball.has(`dist/data-files/${name}`)) problems.push(`missing dist/data-files/${name}`);
}
console.log(`${packed.filename}: ${packed.entryCount} files, ${(packed.size / 1e6).toFixed(1)} MB packed`);
// 3. installs and imports in a clean project
if (problems.length === 0) {
const tmp = mkdtempSync(join(tmpdir(), "camoufox-pack-"));
try {
const tgz = execFileSync("npm", ["pack", "--ignore-scripts", "--pack-destination", tmp], {
cwd: root,
encoding: "utf8",
})
.trim()
.split("\n")
.pop();
writeFileSync(join(tmp, "package.json"), '{"name":"pack-check","private":true,"type":"module"}');
execFileSync("npm", ["install", "--no-audit", "--no-fund", join(tmp, tgz), `playwright-core@${pkg.peerDependencies["playwright-core"]}`], {
cwd: tmp,
stdio: "inherit",
});
const probe = `
const m = await import(${JSON.stringify(pkg.name)});
for (const name of ["Camoufox", "NewBrowser", "launchOptions"]) {
if (typeof m[name] !== "function") throw new Error("missing export " + name);
}
console.log("exports:", Object.keys(m).length);
`;
execFileSync("node", ["--input-type=module", "-e", probe], { cwd: tmp, stdio: "inherit" });
for (const bin of Object.keys(pkg.bin ?? {})) {
execFileSync("npx", ["--no-install", bin, "--help"], { cwd: tmp, stdio: "ignore" });
}
} catch (err) {
problems.push(`clean install failed: ${err.message}`);
} finally {
rmSync(tmp, { recursive: true, force: true });
}
}
if (problems.length) {
console.error(problems.map((p) => ` - ${p}`).join("\n"));
process.exit(1);
}
console.log("pack check OK");
typescript/scripts/copy-files.mjs
+25
View File
@@ -0,0 +1,25 @@
// Ship every non-TS file the package needs at runtime into dist/:
// DATA_FILES (src/paths.ts) from pythonlib/camoufox/ -> dist/data-files/
// (presets, fonts, voices, territoryInfo.xml, repos.yml, warnings.yml, ...)
// src/fpgen/NOTICE -> dist/fpgen/NOTICE (Apache-2.0 attribution for the port)
// Runs after tsc, so the list comes from the compiled paths.js.
import * as fs from "node:fs";
import * as path from "node:path";
import { fileURLToPath } from "node:url";
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const PYTHONLIB = path.resolve(ROOT, "..", "pythonlib", "camoufox");
const DATA = path.join(ROOT, "dist", "data-files");
fs.rmSync(DATA, { recursive: true, force: true });
const { DATA_FILES } = await import(path.join(ROOT, "dist", "paths.js"));
fs.mkdirSync(DATA, { recursive: true });
for (const name of DATA_FILES) {
fs.copyFileSync(path.join(PYTHONLIB, name), path.join(DATA, name));
}
console.log(`copied ${DATA_FILES.length} data files from pythonlib -> dist/data-files`);
const notice = path.join(ROOT, "dist", "fpgen", "NOTICE");
fs.mkdirSync(path.dirname(notice), { recursive: true });
fs.copyFileSync(path.join(ROOT, "src", "fpgen", "NOTICE"), notice);
console.log("copied src/fpgen/NOTICE -> dist/fpgen/NOTICE");
typescript/scripts/e2e/python_probe.py
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env python3
"""Launch the Python camoufox with a fixed identity and run the shared page probe.
Used by tests/e2e.test.ts to compare the TypeScript launcher against Python on
the same binary and the same identity:
echo '{"mode": "headless", "url": "...", "kwargs": {...}}' | \
.venv/bin/python typescript/scripts/e2e/python_probe.py
Prints one JSON object: {"probe": <probe result>, "config": <CAMOU_CONFIG>}.
`mode` is "config-only" (launch_options() alone), "headless" (Camoufox(...) -> new_page),
"persistent" (Camoufox(persistent_context=True, user_data_dir=...)), "virtual"
(Camoufox(headless="virtual")) or "context"
(Camoufox(...) -> NewContext(browser, preset=req["preset"]) -> new_page).
stdout carries the JSON and nothing else: pythonlib prints to stdout (e.g.
"Skipping unknown patch" when the binary predates a config key), so everything
the launch prints is sent to stderr instead.
"""
import contextlib
import json
import sys
import tempfile
import warnings
from pathlib import Path
from camoufox.sync_api import Camoufox, NewContext
from camoufox.utils import launch_options
PROBE = (Path(__file__).resolve().parent.parent.parent / 'tests' / 'fixtures' / 'e2e' / 'probe.js').read_text()
def config_of(options):
env = options['env']
chunks = sorted((int(k.rsplit('_', 1)[1]), v) for k, v in env.items() if k.startswith('CAMOU_CONFIG_'))
return json.loads(''.join(v for _, v in chunks))
def main():
req = json.loads(sys.stdin.read())
with contextlib.redirect_stdout(sys.stderr):
out = run(req)
json.dump(out, sys.stdout)
def run(req):
kwargs = req['kwargs']
warnings.simplefilter('ignore')
config = config_of(launch_options(**kwargs))
if req['mode'] == 'config-only':
return {'config': config}
if req['mode'] == 'context':
with Camoufox(**kwargs) as browser:
context = NewContext(browser, preset=req['preset'])
page = context.new_page()
page.goto(req['url'])
return {'probe': page.evaluate(PROBE), 'config': config}
with tempfile.TemporaryDirectory() as profile:
extra = {'persistent_context': True, 'user_data_dir': profile} if req['mode'] == 'persistent' else {}
if req['mode'] == 'virtual':
extra = {'headless': 'virtual'}
kwargs = {k: v for k, v in kwargs.items() if k != 'headless'}
with Camoufox(**kwargs, **extra) as browser:
page = browser.new_page()
page.goto(req['url'])
if req['mode'] == 'virtual':
# As the TS side does: Firefox defers enumerateDevices() until
# the document has focus, which a headful window on a bare Xvfb
# only sometimes gets. Wait for it, as a user's window has it.
for _ in range(50):
page.bring_to_front()
if page.evaluate('document.hasFocus()'):
break
page.wait_for_timeout(200)
else:
raise RuntimeError('the headful page never got focus on Xvfb')
probe = page.evaluate(PROBE)
return {'probe': probe, 'config': config}
if __name__ == '__main__':
main()
typescript/scripts/golden/fpgen_golden.py
+459
View File
@@ -0,0 +1,459 @@
#!/usr/bin/env python3
"""Golden fixtures for the TypeScript fpgen port (typescript/src/fpgen/).
Run from the repo root with the worktree venv, which has fpgen and the PINNED
model installed (scripts/pin-fpgen-model.py):
.venv/bin/python typescript/scripts/golden/fpgen_golden.py
Writes typescript/tests/fixtures/fpgen/*.json. Everything except stats.json is
deterministic -- network structure, value lookups, beam-search distributions,
condition -> evidence, query() -- and the TS tests compare against it exactly.
stats.json holds marginal counts from real Python draws; the TS test compares
its own draws against them statistically, because Python samples with
`random.random()` and cannot be matched draw-for-draw.
The CASES tables below are mirrored by name in typescript/tests/fpgen*.test.ts;
predicates are named because a lambda cannot be serialised.
"""
import hashlib
import json
import os
import random
import re
import struct
import sys
import time
from collections import Counter
from multiprocessing import Pool
from pathlib import Path
HERE = Path(__file__).resolve().parent
TS_ROOT = HERE.parent.parent
REPO = TS_ROOT.parent
OUT = TS_ROOT / 'tests' / 'fixtures' / 'fpgen'
PIN = json.loads((REPO / 'scripts' / 'data' / 'fpgen-model.json').read_text())
STATS_N = int(os.environ.get('FPGEN_GOLDEN_STATS_N', '3000'))
def _guard_model():
"""Refuse to run against anything but the pinned model, and keep fpgen from
'refreshing' it: fpgen re-downloads (unverified, wrong release) when its
files look older than five weeks, so bump their mtime before importing."""
import importlib.util
spec = importlib.util.find_spec('fpgen')
if spec is None or not spec.origin:
sys.exit('fpgen is not installed in this interpreter')
data = Path(spec.origin).parent / 'data'
stamp = data / '.pinned-model'
if not stamp.exists() or stamp.read_text().strip() != PIN['sha256']:
sys.exit(f'fpgen model in {data} is not pinned to {PIN["tag"]}; '
'run scripts/pin-fpgen-model.py first')
now = time.time()
for f in PIN['files']:
os.utime(data / f, (now, now))
_guard_model()
import fpgen # noqa: E402
import fpgen.bayesian_network as bn # noqa: E402
from fpgen import Generator, query, trace # noqa: E402
from fpgen.unpacker import VALUE_PAIRS, base85_to_int, lookup_value_list # noqa: E402
from fpgen.utils import NETWORK, build_evidence # noqa: E402
# Count beam prunings so the fixture can prove the exact-match traces cover them.
_PRUNES = [0]
_nlargest = bn.heapq.nlargest
def _counting_nlargest(*a, **k):
_PRUNES[0] += 1
return _nlargest(*a, **k)
bn.heapq.nlargest = _counting_nlargest
# Predicates, by name. The TS test defines the same names.
PREDICATES = {
'screen_width_1280_1920': lambda w: isinstance(w, int) and 1280 <= w <= 1920,
'os_linux_or_windows': lambda v: v in {'linux', 'windows'},
'hc_at_least_8': lambda v: isinstance(v, int) and v >= 8,
'ua_rv146': lambda v: 'rv:146' in v,
'never': lambda v: False,
}
class Pred:
def __init__(self, name):
self.name = name
class Alt:
"""A tuple of alternatives (a Set in TS)."""
def __init__(self, *values):
self.values = values
def to_py(cond):
if isinstance(cond, Pred):
return PREDICATES[cond.name]
if isinstance(cond, Alt):
return tuple(to_py(v) for v in cond.values)
if isinstance(cond, dict):
return {k: to_py(v) for k, v in cond.items()}
return cond
def to_fixture(cond):
if isinstance(cond, Pred):
return {'$pred': cond.name}
if isinstance(cond, Alt):
return {'$alt': [to_fixture(v) for v in cond.values]}
if isinstance(cond, dict):
return {k: to_fixture(v) for k, v in cond.items()}
return cond
def chrome_ua():
uas = query('navigator.userAgent')
return next(u for u in uas if 'Chrome/' in u and 'Firefox' not in u)
CONDITION_CASES = [
('empty', {}),
('firefox', {'browser': 'Firefox'}),
('firefox_windows', {'browser': 'Firefox', 'os': 'Windows'}),
('firefox_linux', {'browser': 'Firefox', 'os': 'Linux'}),
('firefox_macos', {'browser': 'Firefox', 'os': 'macOS'}),
('casefolded_keys_values', {'Browser': 'firefox', 'OS': 'WINDOWS'}),
('nested_dict', {'browser': 'Firefox', 'screen': {'width': 1920, 'height': 1080}}),
('dotted_key', {'browser': 'Firefox', 'screen.width': 2560}),
('alt_os', {'browser': 'Firefox', 'os': Alt('Linux', 'Windows')}),
('alt_nested', {'browser': 'Firefox', 'screen.width': Alt(1920, 2560)}),
('pred_screen', {'browser': 'Firefox', 'os': 'Windows',
'screen.width': Pred('screen_width_1280_1920')}),
('pred_os', {'browser': 'Firefox', 'os': Pred('os_linux_or_windows')}),
('pred_hc', {'browser': 'Firefox', 'navigator.hardwareConcurrency': Pred('hc_at_least_8')}),
('pred_ua', {'navigator.userAgent': Pred('ua_rv146')}),
('list_value', {'navigator.languages': ['en-US', 'en']}),
('nested_string', {'browser': 'Firefox', 'gpu': {'vendor': 'Apple'}}),
('device_memory', {'navigator.deviceMemory': 8}),
('hc_int', {'browser': 'Firefox', 'os': 'Linux', 'navigator.hardwareConcurrency': 16}),
]
ERROR_CASES = [
('bad_value', {'os': 'Plan9'}, True),
('bad_node', {'nosuch.node': 1}, True),
('bad_root_node', {'nosuch': 1}, True),
('empty_key', {'': 1}, True),
('bad_nested_value', {'screen.width': 12345}, True),
('bad_nested_path', {'screen.nosuchfield': 1}, True),
('pred_never', {'os': Pred('never')}, True),
('pred_never_nested', {'screen.width': Pred('never')}, True),
('restrictive', {'browser': 'Firefox', 'navigator.userAgent': '$CHROME_UA'}, True),
# Not strict: fpgen drops the FIRST key and does not re-validate.
('restrictive_relaxed', {'browser': 'Firefox', 'navigator.userAgent': '$CHROME_UA'}, False),
]
TRACE_TARGETS = [
'os', 'navigator.userAgent', 'screen', 'navigator.hardwareConcurrency', 'gpu', 'gpuInfo',
]
# Deep nodes with big supports, traced for a few cases only to keep the fixture small.
EXTRA_TRACE_TARGETS = ['allFonts', 'window']
EXTRA_TRACE_CASES = {'empty', 'firefox_linux', 'list_value', 'pred_hc'}
def subst(cond, ua):
if cond == '$CHROME_UA':
return ua
if isinstance(cond, dict):
return {k: subst(v, ua) for k, v in cond.items()}
return cond
def evidence_fixture(evidence):
return [[k, sorted(v, key=base85_to_int)] for k, v in evidence.items()]
def dist_fixture(dist):
return [[k, p] for k, p in dist.items()]
def float_hex(x):
return struct.pack('>d', float(x)).hex()
def cpt_canonical(o):
if isinstance(o, dict):
return '{' + ','.join(json.dumps(k) + ':' + cpt_canonical(v) for k, v in o.items()) + '}'
return float_hex(o)
def sha(s):
return hashlib.sha256(s.encode('utf-8')).hexdigest()
def write(name, data):
OUT.mkdir(parents=True, exist_ok=True)
path = OUT / name
path.write_text(json.dumps(data, separators=(',', ':'), ensure_ascii=False) + '\n')
print(f'{path.relative_to(REPO)}: {path.stat().st_size} bytes', file=sys.stderr)
def all_ids():
ids = set()
for node in NETWORK.nodes_in_sampling_order:
ids.update(node.possible_values)
return sorted(ids, key=base85_to_int)
def gen_structure():
nodes = []
for node in NETWORK.nodes_in_sampling_order:
nodes.append({
'name': node.name,
'parentNames': node.parent_names,
'possibleValues': node.possible_values,
'cptSha256': sha(cpt_canonical(node.probabilities)),
'ancestors': sorted(NETWORK.get_all_ancestors(node.name)),
})
ids = all_ids()
return {
'pin': PIN['sha256'],
'fpgenVersion': '1.3.0',
'nodeNames': list(NETWORK.node_names),
'nodes': nodes,
'valuePairs': {
'count': len(VALUE_PAIRS),
'sha256': sha(''.join(f'{int(o, 16)}:{n};' for o, n in VALUE_PAIRS)),
},
'base85': {
'count': len(ids),
'sha256': sha(''.join(f'{i}={base85_to_int(i)};' for i in ids)),
'samples': {i: base85_to_int(i) for i in ids[:: max(1, len(ids) // 200)]},
},
}
def gen_values():
ids = all_ids()
rng = random.Random(1234)
sample = sorted(rng.sample(ids, 300), key=base85_to_int)
texts = lookup_value_list(sample)
samples = []
for i, t in zip(sample, texts):
entry = {'id': i, 'length': len(t.encode()), 'sha256': sha(t)}
if len(t) <= 160:
entry['text'] = t
samples.append(entry)
# Digest of every referenced value, in id order, in chunks.
h = hashlib.sha256()
for n in range(0, len(ids), 500):
for t in lookup_value_list(ids[n:n + 500]):
h.update(sha(t).encode())
return {'samples': samples, 'allIdsDigest': h.hexdigest(), 'idCount': len(ids)}
def gen_conditions(ua):
out = []
for name, cond in CONDITION_CASES:
evidence = {}
build_evidence(to_py(cond), evidence)
traces = {}
before = _PRUNES[0]
targets = TRACE_TARGETS + (EXTRA_TRACE_TARGETS if name in EXTRA_TRACE_CASES else [])
for target in targets:
traces[target] = dist_fixture(NETWORK.trace(target, evidence))
out.append({
'name': name,
'conditions': to_fixture(cond),
'evidence': evidence_fixture(evidence),
'traces': traces,
'pruned': _PRUNES[0] - before,
})
errors = []
for name, cond, strict in ERROR_CASES:
cond = subst(cond, ua)
evidence = {}
entry = {'name': name, 'conditions': to_fixture(cond), 'strict': strict}
try:
build_evidence(to_py(cond), evidence, strict=strict)
entry['evidence'] = evidence_fixture(evidence)
except Exception as exc: # noqa: BLE001 -- recording the class is the point
entry['error'] = type(exc).__name__
entry['message'] = str(exc)
errors.append(entry)
return {'cases': out, 'errors': errors}
def trace_results(res):
if isinstance(res, list):
return [{'value': r.value, 'probability': r.probability} for r in res]
return {k: trace_results(v) for k, v in res.items()}
def gen_api():
"""Public-API outputs: trace(), Generator.trace, query(), deterministic generate()."""
traces = []
for name, target, cond, kw in [
('os_firefox', 'os', {'browser': 'Firefox'}, {}),
('hc_firefox_linux', 'navigator.hardwareConcurrency',
{'browser': 'Firefox', 'os': 'Linux'}, {}),
('screen_firefox_macos', 'screen', {'browser': 'Firefox', 'os': 'macOS'}, {}),
('inside_node', 'screen.width', {'browser': 'Firefox', 'os': 'Linux'}, {}),
('two_targets_nested', ['navigator.platform', 'navigator.oscpu'], {'browser': 'Firefox'}, {}),
('two_targets_flat', ['navigator.platform', 'navigator.oscpu'], {'browser': 'Firefox'},
{'flatten': True}),
('prefix_target', 'headers.sec-fetch', {'browser': 'Firefox'}, {}),
('pred', 'os', {'browser': 'Firefox', 'os': Pred('os_linux_or_windows')}, {}),
]:
traces.append({
'name': name, 'target': target, 'conditions': to_fixture(cond), 'options': kw,
'result': trace_results(trace(target, to_py(cond), **kw)),
})
gen_trace = trace_results(Generator(browser='Firefox', os='Windows').trace('navigator.platform'))
queries = []
for target, kw in [
('os', {}), ('os', {'sort': True}),
('navigator.hardwareConcurrency', {}), ('navigator.hardwareConcurrency', {'sort': True}),
('navigator.deviceMemory', {'sort': True}),
('screen', {'sort': True}), ('screen', {'sort': True, 'flatten': True}),
('screen.width', {'sort': True}),
('window', {'sort': True}),
('windowComponents', {}),
('navigator', {'sort': True}), ('navigator', {'sort': True, 'flatten': True}),
('matchmedia', {'sort': True}),
('headers', {'sort': True, 'flatten': True}),
('permissions.geolocation', {}),
# The only node whose merged lists mix ints and integral floats (grouping by type).
('audio.values', {'sort': True}),
]:
queries.append({'target': target, 'options': kw, 'result': query(target, **kw)})
query_errors = []
for target in ['nosuch', 'screen.nosuchfield', 'navigator.nosuch', 'matchMedia']:
try:
query(target)
query_errors.append({'target': target, 'error': None})
except Exception as exc: # noqa: BLE001
query_errors.append({'target': target, 'error': type(exc).__name__, 'message': str(exc)})
# generate() outputs that do not depend on the draw (checked by repetition).
deterministic = []
for name, cond, kw in [
('appName', {'browser': 'Firefox'}, {'target': 'navigator.appName'}),
('two', {'browser': 'Firefox'}, {'target': ['navigator.appCodeName', 'navigator.productSub']}),
('platform_win', {'browser': 'Firefox', 'os': 'Windows'}, {'target': 'navigator.platform'}),
('platform_mac', {'browser': 'Firefox', 'os': 'macOS'}, {'target': 'navigator.platform'}),
('casefold_target', {'browser': 'Firefox', 'os': 'Windows'}, {'target': 'NAVIGATOR.PLATFORM'}),
('flat_target', {'browser': 'Firefox', 'os': 'Windows'},
{'target': ['navigator.appName', 'navigator.platform'], 'flatten': True}),
]:
outs = [Generator().generate(to_py(cond), **kw) for _ in range(20)]
if any(o != outs[0] for o in outs):
sys.exit(f'deterministic case {name} is not deterministic')
deterministic.append({'name': name, 'conditions': cond, 'options': kw, 'result': outs[0]})
g = Generator(browser='Firefox')
shapes = {
'topLevelKeys': sorted(g.generate().keys()),
'navigatorKeys': sorted(g.generate(target='navigator').keys()),
'flatKeysSample': sorted(k for k in g.generate(flatten=True) if k.startswith('navigator.')),
}
return {'traces': traces, 'generatorTrace': gen_trace, 'queries': queries,
'queryErrors': query_errors, 'deterministic': deterministic, 'shapes': shapes}
# ---- statistics -----------------------------------------------------------
STATS_SCENARIOS = [
('firefox_any', {'browser': 'Firefox'}),
('firefox_windows', {'browser': 'Firefox', 'os': 'Windows'}),
('firefox_linux', {'browser': 'Firefox', 'os': 'Linux'}),
('firefox_macos', {'browser': 'Firefox', 'os': 'macOS'}),
('firefox_windows_screen_bound', {'browser': 'Firefox', 'os': 'Windows',
'screen.width': Pred('screen_width_1280_1920')}),
]
def fields(fp):
"""The marginals compared. Mirrored by fields() in fpgen-stats.test.ts."""
ua = fp['navigator']['userAgent']
m = re.search(r'\(([^)]*)\)', ua)
platform = m.group(1).split('; rv:')[0] if m else ''
rv = re.search(r'rv:(\d+)', ua)
return {
'os': fp['os'],
'uaPlatform': platform,
'firefoxMajor': rv.group(1) if rv else '',
'screen': f"{fp['screen']['width']}x{fp['screen']['height']}",
'hardwareConcurrency': str(fp['navigator']['hardwareConcurrency']),
'gpuVendor': fp['gpu']['vendor'],
}
_WORKER_GEN = None
def _draw(args):
global _WORKER_GEN
scenario, n, seed = args
random.seed(seed)
if _WORKER_GEN is None:
_WORKER_GEN = Generator()
cond = to_py(dict(STATS_SCENARIOS)[scenario])
counts = {}
for _ in range(n):
for k, v in fields(_WORKER_GEN.generate(cond)).items():
counts.setdefault(k, Counter())[v] += 1
return scenario, counts
def gen_stats():
chunk = 100
jobs = []
for s, (name, _) in enumerate(STATS_SCENARIOS):
for c in range(STATS_N // chunk):
jobs.append((name, chunk, s * 100000 + c))
merged = {name: {} for name, _ in STATS_SCENARIOS}
with Pool(max(1, (os.cpu_count() or 2) - 1)) as pool:
for scenario, counts in pool.imap_unordered(_draw, jobs):
for k, c in counts.items():
merged[scenario].setdefault(k, Counter()).update(c)
return {
'n': STATS_N,
'scenarios': [
{'name': name, 'conditions': to_fixture(cond),
'counts': {k: dict(sorted(v.items())) for k, v in merged[name].items()}}
for name, cond in STATS_SCENARIOS
],
}
def main():
only = set(sys.argv[1:])
def want(x):
return not only or x in only
t = time.time()
if want('structure'):
write('structure.json', gen_structure())
if want('values'):
write('values.json', gen_values())
if want('conditions'):
write('conditions.json', gen_conditions(chrome_ua()))
if want('api'):
write('api.json', gen_api())
if want('stats'):
write('stats.json', gen_stats())
print(f'done in {time.time() - t:.1f}s (fpgen {fpgen.__name__})', file=sys.stderr)
if __name__ == '__main__':
main()
typescript/scripts/golden/identity_golden.py
+728
View File
@@ -0,0 +1,728 @@
#!/usr/bin/env python3
"""Record the Python identity layer's outputs as goldens for the TypeScript port.
.venv/bin/python typescript/scripts/golden/identity_golden.py
Writes typescript/tests/fixtures/identity/*.json (and one .json.gz holding full
fpgen fingerprints). tests/identity-golden.test.ts replays every case through
the TypeScript functions and requires the identical result.
Everything recorded is a pure function of its recorded inputs: seeded draws
take their seed from the case, the unseeded module-level `random` is re-seeded
per case (the TS side seeds its `pyRandom` the same way), and the host probes
fix_hardware_concurrency reads are patched. Large outputs (font / voice lists,
WebGL parameter blobs, preset configs) are recorded as a short hash:
sha256(orjson.dumps(canon(value), OPT_SORT_KEYS)).hexdigest()[:20]
where canon() turns integral floats below 2**53 into ints and ints from 2**53
up into floats (JavaScript cannot tell 1.0 from 1 once parsed, nor keep an int
past 2**53 exact, so the TS side hashes the same canonical form).
"""
import copy
import gzip
import hashlib
import json
import math
import random
import sys
import zlib
from pathlib import Path
from unittest import mock
import numpy as np
import orjson
HERE = Path(__file__).resolve().parent
TS_ROOT = HERE.parent.parent
FIXTURES = TS_ROOT / 'tests' / 'fixtures' / 'identity'
sys.path.insert(0, str(HERE))
import pyrandom_cases # noqa: E402
from camoufox import coherence # noqa: E402
from camoufox import cpu_affinity # noqa: E402
from camoufox import fingerprints as fp # noqa: E402
from camoufox import webgl # noqa: E402
OS_NAMES = ('windows', 'macos', 'linux')
OS_KEYS = ('win', 'mac', 'lin')
def canon(o):
if isinstance(o, bool) or o is None:
return o
if isinstance(o, float) and o.is_integer() and abs(o) < 2**53:
return int(o)
if isinstance(o, int) and abs(o) >= 2**53:
return float(o) # a JavaScript number past 2**53 is a float either way
if isinstance(o, dict):
return {str(k): canon(v) for k, v in o.items()}
if isinstance(o, (list, tuple)):
return [canon(v) for v in o]
return o
def h(o) -> str:
return hashlib.sha256(orjson.dumps(canon(o), option=orjson.OPT_SORT_KEYS)).hexdigest()[:20]
def write(name, data, compress=True):
"""Write a fixture; the large ones gzipped (mtime 0, so reruns are byte-stable)."""
FIXTURES.mkdir(parents=True, exist_ok=True)
text = json.dumps(data, ensure_ascii=False, separators=(',', ':')) + '\n'
path = FIXTURES / (name + '.gz' if compress else name)
if compress:
path.write_bytes(gzip.compress(text.encode('utf-8'), compresslevel=9, mtime=0))
else:
path.write_text(text, encoding='utf-8')
print(f'{path.relative_to(TS_ROOT)}: {path.stat().st_size // 1024} KiB')
def hashed_lists(config):
"""A config with its (long) font and voice lists replaced by their hashes."""
out = dict(config)
for key in ('fonts', 'voices'):
if key in out:
out[key] = {'hash': h(out[key]), 'len': len(out[key])}
return out
def err(fn):
try:
return {'ok': fn()}
except Exception as exc: # noqa: BLE001
return {'error': type(exc).__name__, 'message': str(exc)}
# ---------------------------------------------------------------------------
# pyrandom + numpy + python-compat primitives
# ---------------------------------------------------------------------------
def record_pyrandom():
write('pyrandom.json', {'cases': pyrandom_cases.cases()})
def record_numpy():
"""np.sum over a float64 array, which the locale selector normalises by."""
rng = random.Random(20260924)
sums = []
for n in (0, 1, 2, 5, 7, 8, 9, 15, 16, 17, 31, 64, 100, 127, 128, 129, 200, 255, 256, 257, 600):
vals = [rng.uniform(-1, 1) * 10 ** rng.randint(-8, 8) for _ in range(n)]
sums.append({'values': vals, 'sum': float(np.array(vals, dtype=np.float64).sum())})
write('numpy.json', {'sums': sums})
SALT_OBJECTS = [
{},
{'a': 1, 'b': 2},
{'b': 2, 'a': 1},
{'nested': {'z': [1, 2.5, None, True, False], 'a': 'x'}, 'k': -0.0},
{'floats': [1.0, 0.1, 1e16, 1e15, 1.5e-7, 1e-5, 1e-6, 3.4028234663852886e38, -2.5e-310, 123456.789, 1e21]},
{'big': 18446744073709551615, 'neg': -9223372036854775808, 'safe': 9007199254740993},
{'unicode': 'é😀
\x00\x1f"\\/', 'é': 1, '😀': 2, '￿': 3, 'Z': 4, 'a': 5},
{'navigator.userAgent': 'Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0',
'screen.width': 1920, 'screen.height': 1080, 'window.devicePixelRatio': 1.0},
['list', 'at', 'top', 1, 2.0],
'a plain string',
12345,
1.0,
None,
]
def record_pycompat():
rng = random.Random(1717)
floats = [0.0, -0.0, 1.0, -1.0, 0.1, 0.5, 1e16, 1e15, 9999999999999998.0, 1.0000000000000002e16,
123456789012345678.0, 1.5e-7, 1e-5, 1.5e-5, 9.99e-6, 1e-4, 1e-6, 0.000123, 1e21, 1e22, 1e100,
3.4028234663852886e38, 5e-324, 1.7976931348623157e308, 12345.678, 9007199254740992.0,
9223372034707292000.0, 1.8446744073709552e19, 2.5, 100.0, 1e-7, 1.234e-300]
for _ in range(300):
floats.append(rng.uniform(-1, 1) * 10 ** rng.randint(-30, 30))
for _ in range(60):
floats.append(float(rng.randint(-10**18, 10**18)))
float_cases = [[x, orjson.dumps(x).decode(), repr(x)] for x in floats]
sums = []
for _ in range(150):
n = rng.randint(0, 25)
items = []
for _ in range(n):
kind = rng.random()
if kind < 0.25:
items.append(rng.randint(-100, 100))
elif kind < 0.6:
items.append(rng.uniform(-1, 1) * 10 ** rng.randint(-20, 20))
else:
items.append(rng.random())
result = sum(items)
sums.append({
'items': [{'i': x} if isinstance(x, int) else {'f': x} for x in items],
'result': {'i': result} if isinstance(result, int) else {'f': result},
})
float_sums = []
for _ in range(150):
vals = [rng.random() * rng.choice([1, 1e-3, 1e10, 1e-17, 3]) for _ in range(rng.randint(1, 40))]
float_sums.append({'values': vals, 'sum': sum(vals)})
crc = [[s, zlib.crc32(s.encode('utf-8'))] for s in ('', 'a', 'hello world', 'é😀', 'x' * 1000)]
salts = [{'value': o, 'salt': str(fp.identity_salt(o))} for o in SALT_OBJECTS]
salts.append({'screen': [1, 2, 3, 4], 'salt': str(fp.identity_salt(fp.Screen(1, 2, 3, 4)))})
salts.append({'screen': [None, 1920, None, 1080], 'salt': str(fp.identity_salt(fp.Screen(None, 1920, None, 1080)))})
seeds = []
configs = [
{},
{'navigator.userAgent': 'x', 'navigator.platform': 'Win32', 'screen.width': 1920,
'screen.height': 1080, 'navigator.hardwareConcurrency': 8},
{'navigator.userAgent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:152.0) Gecko/20100101 Firefox/152.0',
'navigator.platform': 'MacIntel', 'screen.width': 1512.5, 'screen.height': None,
'navigator.hardwareConcurrency': True},
{'navigator.userAgent': 'é😀', 'screen.width': 0, 'screen.height': ''},
{'navigator.platform': 1e-5, 'screen.width': 1e16, 'navigator.hardwareConcurrency': -3},
]
for c in configs:
for salt in (0, 1, 2**31, 2**64 - 1, 12345678901234567890):
seeds.append({'config': c, 'salt': str(salt), 'seed': fp.identity_seed(c, salt)})
reprs = [[s, repr(s)] for s in ('', 'abc', "it's", 'say "hi"', 'both \' and "', 'tab\there', 'nl\n',
'back\\slash', '\x00\x7f', 'é😀', '​', '
', 'Apple M1, or similar')]
write('pycompat.json', {'floats': float_cases, 'sums': sums, 'floatSums': float_sums, 'crc32': crc,
'salts': salts, 'seeds': seeds, 'reprs': reprs,
'python': list(sys.version_info[:2])})
def record_fpgen_salts():
"""Full fpgen fingerprints (one per OS, plus one carrying an int > 2**53)."""
found = {}
tries = 0
while len(found) < 4 and tries < 200:
tries += 1
os_name = OS_NAMES[tries % 3]
f = fp.generate_fingerprint(os=os_name)
big = isinstance(((f.get('webgl2') or {}).get('params') or {}).get('37137', {}).get('value'), int)
if os_name not in found:
found[os_name] = f
elif big and 'bigint' not in found:
found['bigint'] = f
cases = [{'label': k, 'fingerprint': v, 'salt': str(fp.identity_salt(v))} for k, v in found.items()]
FIXTURES.mkdir(parents=True, exist_ok=True)
path = FIXTURES / 'fpgen-salts.json.gz'
path.write_bytes(gzip.compress(json.dumps(cases, ensure_ascii=False).encode('utf-8'), mtime=0))
print(f'{path.relative_to(TS_ROOT)}: {path.stat().st_size // 1024} KiB')
# ---------------------------------------------------------------------------
# per-identity draws
# ---------------------------------------------------------------------------
def record_fonts():
cases = []
for os_name in OS_NAMES + ('plan9',):
for seed in range(150):
locale = None if seed < 100 else ('zh-TW' if seed < 125 else 'en-US')
fonts = fp._generate_random_font_subset(os_name, seed=seed, locale=locale)
case = {'os': os_name, 'seed': seed, 'locale': locale, 'hash': h(fonts), 'len': len(fonts)}
if seed < 2:
case['fonts'] = fonts
cases.append(case)
for seed in (2**32 - 1, 2**40 + 3):
fonts = fp._generate_random_font_subset(os_name, seed=seed)
cases.append({'os': os_name, 'seed': str(seed), 'locale': None, 'hash': h(fonts), 'len': len(fonts)})
fonts = fp._generate_random_font_subset(os_name, seed=0, native=True)
cases.append({'os': os_name, 'seed': 0, 'native': True, 'locale': None, 'hash': h(fonts),
'len': len(fonts), 'fonts': fonts})
write('fonts.json', {'cases': cases})
VOICE_LOCALES = (None, 'en-US', 'en-GB', 'de-DE', 'fr-FR', 'zh-TW', 'ja-JP', 'pt-BR', 'es', 'xx-YY')
def manifest_voice_entries(node):
"""Every "Name:lang:type" entry in a voice manifest, in document order."""
if isinstance(node, dict):
return [e for value in node.values() for e in manifest_voice_entries(value)]
if isinstance(node, list):
return [e for item in node
for e in ([item] if isinstance(item, str) and item.count(':') >= 2
else manifest_voice_entries(item))]
return []
def record_voices():
cases = []
for os_name in OS_NAMES + ('plan9',):
for locale in VOICE_LOCALES:
for seed in range(20):
voices = fp._generate_random_voice_subset(os_name, locale, seed=seed)
case = {'os': os_name, 'locale': locale, 'seed': seed, 'hash': h(voices), 'len': len(voices)}
if seed == 0 and locale in (None, 'de-DE'):
case['voices'] = voices
cases.append(case)
uris = []
uri_hashes = {}
for os_key, manifest in fp._load_voice_manifests().items():
entries = manifest_voice_entries(manifest)
per = []
for entry in entries:
name, lang, _ = entry.rsplit(':', 2)
per.append(fp._voice_uri(os_key, name, lang))
uri_hashes[os_key] = h(per)
for entry in entries[:: max(1, len(entries) // 15)]:
name, lang, _ = entry.rsplit(':', 2)
uris.append([os_key, name, lang, fp._voice_uri(os_key, name, lang)])
for os_key in ('mac', 'win', 'lin', 'xx'):
for name, lang in (('Albert', 'en-US'), ('Eddy', 'de-DE'), ('Zoë Ünïcode', 'fr-FR'),
('Some Voice (Enhanced)', 'en-GB'), ('日本語', 'ja-JP'), ('.Dots.', 'en')):
uris.append([os_key, name, lang, fp._voice_uri(os_key, name, lang)])
normalized = []
for fname in ('fingerprint-presets.json', 'fingerprint-presets-v150.json'):
presets = json.loads((Path(fp.__file__).parent / fname).read_text())['presets']
for os_name, entries in presets.items():
for i, preset in enumerate(entries):
if preset.get('speechVoices'):
out = fp._normalize_preset_voices(preset['speechVoices'], os_name)
normalized.append({'file': fname, 'os': os_name, 'index': i, 'hash': h(out), 'len': len(out)})
extra = ['Albert:en-US:local', 'bad', 'x:y', ':en:local', 'Name::local', 'A:b:c:remote',
{'name': 'Obj', 'lang': 'en', 'voiceUri': 'u', 'isDefault': False, 'isLocalService': True}]
normalized.append({'extra': extra, 'os': 'macos', 'out': fp._normalize_preset_voices(extra, 'macos')})
write('voices.json', {'cases': cases, 'uris': uris, 'uriHashes': uri_hashes, 'normalized': normalized})
def record_media():
cases = []
for os_key in OS_KEYS + ('xx',):
for seed in range(150):
out = fp.draw_media_devices(os_key, seed)
case = {'os': os_key, 'seed': seed, 'hash': h(out)}
if seed < 4:
case['out'] = out
cases.append(case)
defaults = []
for i in range(60):
plat = ('Win32', 'MacIntel', 'Linux x86_64', '', None)[i % 5]
config = {'navigator.userAgent': f'ua{i}', 'screen.width': 1920}
if plat is not None:
config['navigator.platform'] = plat
salt = (0, 7, 2**64 - 1)[i % 3]
before = copy.deepcopy(config)
fp.set_media_devices_defaults(config, salt)
defaults.append({'config': before, 'salt': str(salt), 'hash': h(config)})
preset_mix = {'mediaDevices:webcams': 5, 'navigator.platform': 'Win32'}
before = copy.deepcopy(preset_mix)
fp.set_media_devices_defaults(preset_mix)
defaults.append({'config': before, 'salt': '0', 'hash': h(preset_mix)})
write('media.json', {'cases': cases, 'defaults': defaults})
def exact(o) -> str:
"""sha256 of the exact orjson bytes: key order, int vs float and all."""
return hashlib.sha256(orjson.dumps(o)).hexdigest()[:20]
WEBGL_SCREENS = [(1024, 600), (800, 480), (1366, 768), (1280, 800), (1920, 1080), (2560, 1440), (None, None)]
def record_webgl():
gpus = {os_key: [[r.value['vendor'], r.value['renderer']] for r in webgl._trace('gpu', os_key)]
for os_key in OS_KEYS}
for_screen = []
for os_key in OS_KEYS:
for w, hh in WEBGL_SCREENS:
for seed in range(60):
out = webgl.sample_webgl_for_screen(os_key, w, hh, seed=seed)
for_screen.append({'os': os_key, 'w': w, 'h': hh, 'seed': str(seed),
'renderer': out['webGl:renderer'], 'hash': exact(out)})
for seed in (2**32 - 1, 2**40, 2**64 + 1, 12345678901234567890123):
out = webgl.sample_webgl_for_screen(os_key, 1920, 1080, seed=seed)
for_screen.append({'os': os_key, 'w': 1920, 'h': 1080, 'seed': str(seed),
'renderer': out['webGl:renderer'], 'hash': exact(out)})
# Every GPU fpgen records for each OS, and every bundled preset's GPU.
for_gpu = []
targets = {(os_key, v, r) for os_key in OS_KEYS for v, r in gpus[os_key]}
for fname in ('fingerprint-presets.json', 'fingerprint-presets-v150.json'):
presets = json.loads((Path(fp.__file__).parent / fname).read_text())['presets']
for os_name, entries in presets.items():
os_key = {'windows': 'win', 'macos': 'mac', 'linux': 'lin'}[os_name]
for preset in entries:
targets.add((os_key, preset['webgl']['unmaskedVendor'], preset['webgl']['unmaskedRenderer']))
for os_key, v, r in sorted(targets):
for seed in range(5):
for_gpu.append({'os': os_key, 'vendor': v, 'renderer': r, 'seed': seed,
**err(lambda: exact(webgl.webgl_for_gpu(os_key, v, r, seed=seed)))})
for os_key, v, r in (('win', 'Nope', 'Nope GPU'), ('lin', 'Apple', 'Apple M1, or similar')):
for_gpu.append({'os': os_key, 'vendor': v, 'renderer': r, 'seed': 0,
**err(lambda: webgl.webgl_for_gpu(os_key, v, r, seed=0))})
errors = {
'unknownOs': err(lambda: webgl.sample_webgl_for_screen('bsd', 1920, 1080, seed=0)),
}
recorded = {
'vendor': 'v', 'renderer': 'r', 'contextAttributes': {'alpha': True}, 'params': {'3379': {'value': 1.0}},
'shaderPrecisionFormats': [{'shaderType': 35633, 'precisionType': 36336,
'shaderPrecisionFormat': {'rangeMin': 127, 'rangeMax': 127, 'precision': 23}}],
'supportedExtensions': ['ANGLE_instanced_arrays', 'WEBGL_multi_draw', 'OVR_multiview2',
'WEBGL_compressed_texture_etc1'],
}
recorded2 = {**recorded, 'supportedExtensions': ['EXT_texture_norm16', 'WEBGL_clip_cull_distance',
'OVR_multiview2', 'EXT_color_buffer_float']}
converted = [{'os': os_key, 'webgl2': w2, 'hash': exact(webgl.to_config(recorded, w2, os_key))}
for os_key in OS_KEYS for w2 in (recorded2, [])]
write('webgl.json', {'gpus': gpus, 'forScreen': for_screen, 'forGpu': for_gpu, 'errors': errors,
'recorded': recorded, 'recorded2': recorded2, 'converted': converted})
# ---------------------------------------------------------------------------
# geometry fixes + coherence
# ---------------------------------------------------------------------------
def maybe(rng, value, p_missing=0.15, p_none=0.03):
roll = rng.random()
if roll < p_missing:
return '__missing__'
if roll < p_missing + p_none:
return None
return value
def random_geometry(rng, p_none=0.03):
sw = rng.choice([800, 1024, 1280, 1366, 1440, 1536, 1600, 1920, 2560, 3440, 3840, 736, 1080])
sh = rng.choice([480, 600, 720, 768, 800, 864, 900, 1024, 1080, 1440, 1600, 2160, 414, 1920])
aw = sw - rng.choice([0, 0, 0, 40, 60, -30])
ah = sh - rng.choice([0, 0, 25, 27, 40, 48, -20, 900])
ow = rng.choice([sw, sw - 100, sw + 200, 1280, 800, 0])
oh = rng.choice([sh, ah, sh - 40, sh + 100, 720, 1000, 0])
iw = ow - rng.choice([0, 16, -20, 200, 0])
ih = oh - rng.choice([0, 74, 86, 90, 120, -10, 2000])
sx = rng.choice([0, 0, 8, -8, 60, 250, -200, 5000])
sy = rng.choice([0, 0, 20, 281, -30, 900])
c = {}
for key, value in (('screen.width', sw), ('screen.height', sh), ('screen.availWidth', aw),
('screen.availHeight', ah), ('window.outerWidth', ow), ('window.outerHeight', oh),
('window.innerWidth', iw), ('window.innerHeight', ih), ('window.screenX', sx),
('window.screenY', sy)):
v = maybe(rng, value, p_none=p_none)
if v != '__missing__':
c[key] = v
return c
UAS = [
'Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0',
'Mozilla/5.0 (X11; Linux i686; rv:152.0) Gecko/20100101 Firefox/152.0',
'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:152.0) Gecko/20100101 Firefox/152.0',
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:152.0) Gecko/20100101 Firefox/152.0',
'Mozilla/5.0 (Android 16; Mobile; rv:152.0) Gecko/152.0 Firefox/152.0',
'Mozilla/5.0 (X11; Linux aarch64; rv:152.0) Gecko/20100101 Firefox/152.0',
'',
]
PLATFORMS = ['Linux x86_64', 'Linux armv81', 'Linux i686', 'Win32', 'MacIntel', 'Linux aarch64', '']
def record_geometry():
rng = random.Random(4242)
cases = []
for i in range(500):
c = random_geometry(rng)
target = OS_KEYS[i % 3]
cap_w = rng.choice([None, 1366, 1920, 1280, 2560, 0])
cap_h = rng.choice([None, 768, 1080, 720, 1440, 0])
out = {}
for name, fn in (
('fixScreenNoTaskbar', lambda d: fp.fix_screen_no_taskbar(d, target)),
('clampWindowDimensions', fp.clamp_window_dimensions),
('clampScreenToDisplay', lambda d: fp.clamp_screen_to_display(d, cap_w, cap_h)),
('clampWindowPosition', fp.clamp_window_position),
('raiseScreenToModernFloor', fp.raise_screen_to_modern_floor),
('repairScreenOrientation', coherence.repair_screen_orientation),
):
d = copy.deepcopy(c)
ret = fn(d)
out[name] = {'config': d, 'ret': ret} if i < 15 else {'hash': h(d), 'keys': h(list(d)), 'ret': ret}
# the launch_options order
d = copy.deepcopy(c)
if coherence.screen_is_implausible(d):
coherence.repair_screen_orientation(d)
fp.raise_screen_to_modern_floor(d)
fp.raise_screen_to_modern_floor(d)
fp.clamp_screen_to_display(d, cap_w, cap_h)
fp.fix_screen_no_taskbar(d, target)
fp.clamp_window_dimensions(d)
fp.clamp_window_position(d)
out['pipeline'] = {'config': d} if i < 15 else {'hash': h(d), 'keys': h(list(d))}
cases.append({'input': c, 'os': target, 'capW': cap_w, 'capH': cap_h, 'out': out})
arch = []
for ua in UAS:
for plat in PLATFORMS + [None]:
for oscpu in ('Linux armv81', 'Linux x86_64', None):
for target in OS_KEYS:
c = {'navigator.userAgent': ua}
if plat is not None:
c['navigator.platform'] = plat
if oscpu is not None:
c['navigator.oscpu'] = oscpu
d = copy.deepcopy(c)
fp.fix_navigator_arch(d, target)
arch.append([ua, plat, oscpu, target, d.get('navigator.platform'), d.get('navigator.oscpu')])
hc = []
for host in (None, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 16, 22, 24, 26, 32, 64, 128):
for supported in (True, False):
for can_pin in (None, True, False):
for drawn in (None, 1, 2, 3, 4, 5, 7, 8, 9, 11, 13, 15, 16, 18, 24, 32, 48, True, '8', 8.5):
c = {} if drawn is None else {'navigator.hardwareConcurrency': drawn}
d = copy.deepcopy(c)
with mock.patch.object(fp, 'host_cpu_count', lambda n=host: n), \
mock.patch.object(cpu_affinity, 'supported', lambda s=supported: s):
fp.fix_hardware_concurrency(d, can_pin=can_pin)
hc.append([drawn, host, supported, can_pin, d.get('navigator.hardwareConcurrency', '__missing__')])
write('geometry.json', {'cases': cases, 'arch': arch, 'hardwareConcurrency': hc})
def fpgen_renderers():
"""Every renderer fpgen records from Firefox, once each, in trace order."""
renderers = [r.value['renderer'] for os_key in OS_KEYS for r in webgl._trace('gpu', os_key)]
return list(dict.fromkeys(renderers))
EXTRA_RENDERERS = [
'ANGLE (NVIDIA, NVIDIA GeForce GTX 980 Direct3D11 vs_5_0 ps_5_0), or similar',
'NVIDIA GeForce GTX 980/PCIe/SSE2', 'GeForce GTX 980, or similar',
'ANGLE (AMD, Radeon HD 3200 Graphics Direct3D11 vs_5_0 ps_5_0), or similar',
'Radeon HD 3200 Graphics, or similar', 'ANGLE (Samsung Xclipse 920) on Vulkan',
'ANGLE (Intel, Intel(R) HD Graphics Direct3D11 vs_5_0 ps_5_0), or similar', 'Apple M1, or similar',
'llvmpipe, or similar', 'ANGLE (Microsoft, Microsoft Basic Render Driver Direct3D11 vs_5_0 ps_5_0)',
'ANGLE (Google, Vulkan 1.3.0 (SwiftShader Device (Subzero)), SwiftShader driver)', 'Generic Renderer',
'ANGLE (NVIDIA, NVIDIA GeForce RTX 3070 Direct3D11 vs_5_0 ps_5_0)', 'Intel(R) HD Graphics 400, or similar',
'Radeon R9 200 Series, or similar', 'NVIDIA GeForce GTX 480/PCI/SSE2', 'GeForce 8800 GTX', '', 'Mozilla',
]
def record_coherence():
renderers = fpgen_renderers() + EXTRA_RENDERERS
rng = random.Random(9001)
fits = [[r, os_key, coherence.gpu_fits_os(r, os_key)] for r in renderers + [None] for os_key in OS_KEYS + ('bsd',)]
gpu = []
for r in renderers + [None]:
for w, hh in ((1024, 600), (800, 480), (1024, 768), (1366, 768), (1920, 1080), (None, None), (0, 600)):
gpu.append([r, w, hh, fp.gpu_screen_is_plausible(r, w, hh), fp.is_software_renderer(r),
fp._renderer_bucket(r) if r is not None else None])
cases = []
for i in range(900):
c = random_geometry(rng, p_none=0) if rng.random() < 0.7 else {}
fields = {
'webGl:renderer': rng.choice(renderers + [None]),
'navigator.hardwareConcurrency': rng.choice([2, 4, 6, 8, 9, 10, 11, 12, 13, 16, 24, 33, 64, True, '8']),
'screen.colorDepth': rng.choice([24, 30, 32, 16, 48, True, '24']),
'navigator.maxTouchPoints': rng.choice([0, 1, 2, 5, 10, 11, 40, 256, -1, 1.5, True, '5']),
'window.devicePixelRatio': rng.choice([1, 1.25, 1.5, 1.75, 2, 2.5, 3, 1.125, 1.1, 1.818181818181818,
1.09, 3.5, 0.9, 2.25, 1.625, 2.75, '2']),
'screen.pixelDepth': rng.choice([24, 30, 32]),
'navigator.userAgent': rng.choice(UAS),
'navigator.platform': rng.choice(PLATFORMS),
'navigator.oscpu': rng.choice(PLATFORMS),
}
for key, value in fields.items():
if rng.random() < 0.6:
c[key] = value
target = rng.choice(OS_KEYS + ('bsd',))
v = [list(x) for x in coherence.validate(c, target)]
applied = copy.deepcopy(c)
left = [list(x) for x in coherence.apply(applied, target)]
dropped_cfg = copy.deepcopy(c)
dropped = [list(x) for x in coherence.drop_incoherent_source_values(dropped_cfg, target)]
case = {'input': c, 'os': target, 'rules': [x[0] for x in v], 'validate': h(v),
'apply': [x[0] for x in left], 'applyHash': h(left), 'applied': h(applied),
'dropped': dropped, 'droppedConfig': h(dropped_cfg),
'implausible': coherence.screen_is_implausible(c)}
if i < 40:
case.update(validateFull=v, appliedFull=applied)
cases.append(case)
write('coherence.json', {'fits': fits, 'gpuScreen': gpu, 'cases': cases})
# ---------------------------------------------------------------------------
# fpgen -> config, presets
# ---------------------------------------------------------------------------
def trimmed(f):
return {k: f[k] for k in ('navigator', 'screen', 'window', 'headers') if k in f}
def record_from_fpgen_inputs():
"""from_fpgen goldens over stored inputs (generated once, trimmed)."""
inputs = []
for os_name in OS_NAMES:
for _ in range(25):
inputs.append(trimmed(fp.generate_fingerprint(os=os_name)))
# Hand-made edge cases: windowed screenX, negatives, header lists, empties.
inputs.append({'navigator': {'userAgent': 'Mozilla/5.0 (X11; Linux x86_64; rv:135.0) Gecko/20100101 Firefox/135.0',
'platform': 'Linux x86_64', 'hardwareConcurrency': 0, 'maxTouchPoints': 0,
'oscpu': ''},
'screen': {'width': 1920, 'height': 1080, 'availHeight': 1040, 'availLeft': -5, 'availTop': 0},
'window': {'screenX': 300, 'outerHeight': 900, 'outerWidth': 1200},
'headers': {'accept-encoding': ['gzip, deflate, br', 'gzip'], 'accept-language': ['en']}})
inputs.append({'screen': {'availHeight': 700}, 'window': {'screenX': -120, 'outerHeight': 900},
'headers': {'accept-encoding': ['gzip, deflate, br, zstd']}})
inputs.append({'window': {'screenX': 51}, 'screen': {}})
inputs.append({'window': {'screenX': 50, 'screenY': 3}})
inputs.append({'navigator': {'userAgent': 'Firefox/115.0 rv:115.0 1115.0 115.01 Firefox/99.0'}})
inputs.append({})
cases = []
for i, f in enumerate(inputs):
for ffv in (None, '152'):
random.seed(1000 + i)
config = fp.from_fpgen(copy.deepcopy(f), ffv)
cases.append({'input': i, 'ffVersion': ffv, 'moduleSeed': 1000 + i, 'config': config,
'configKeys': list(config.keys()),
'identitySeed': fp.identity_seed(config, 12345678901234567890)})
windows = []
for i, f in enumerate(inputs[:20]):
for w, hh in ((1280, 720), (800, 600), (1920, 1080)):
d = copy.deepcopy(f)
fp.handle_window_size(d, w, hh)
windows.append({'input': i, 'w': w, 'h': hh, 'out': d})
screens = []
for bounds in ((None, None, None, None), (100, 2000, None, None), (None, 1920, None, 1080), (1366, 1366, 768, 768)):
s = fp.Screen(*bounds)
conds = s.as_conditions()
probes = [800, 1366, 1920, 2560, 1080.0, '1920', None]
screens.append({'bounds': list(bounds), 'keys': sorted(conds),
'width': [conds['screen.width'](p) for p in probes] if 'screen.width' in conds else None,
'height': [conds['screen.height'](p) for p in probes] if 'screen.height' in conds else None})
write('from-fpgen.json', {'inputs': inputs, 'cases': cases, 'windowSize': windows, 'screens': screens})
def record_presets():
base = Path(fp.__file__).parent
cases = []
full = []
for fname, ffv in (('fingerprint-presets.json', None), ('fingerprint-presets-v150.json', '152')):
presets = json.loads((base / fname).read_text())['presets']
for os_name, entries in presets.items():
for i, preset in enumerate(entries):
for salt in (0, 12345678901234567890):
random.seed(i * 7 + salt % 1000)
config = fp.from_preset(copy.deepcopy(preset), ffv, salt=salt)
key = {'macos': 'mac', 'windows': 'win', 'linux': 'lin'}[os_name]
cases.append({'file': fname, 'os': os_name, 'index': i, 'ffVersion': ffv, 'salt': str(salt),
'moduleSeed': i * 7 + salt % 1000, 'hash': h(config),
'keys': h(list(config.keys())),
'validate': [x.rule for x in coherence.validate(config, key)]})
if i < 2 and salt == 0:
full.append({'file': fname, 'os': os_name, 'index': i, 'config': hashed_lists(config)})
# Synthetic presets: derived oscpu / appVersion, fallbacks.
synthetic = [
{'navigator': {'platform': 'Linux x86_64', 'userAgent': UAS[0]}},
{'navigator': {'platform': 'Win32', 'userAgent': UAS[3], 'appVersion': '5.0 (Windows NT 10.0; Win64; x64)'}},
{'navigator': {'platform': 'iPhone', 'userAgent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) Gecko/20100101'}},
{'navigator': {'platform': 'Win32', 'userAgent': 'not a user agent', 'maxTouchPoints': 0}},
{'navigator': {'platform': 'MacIntel', 'hardwareConcurrency': 0}, 'screen': {'width': 1440, 'colorDepth': 30},
'webgl': {'unmaskedVendor': 'Apple', 'unmaskedRenderer': 'Apple M1, or similar'}, 'timezone': 'Europe/Paris'},
{},
]
synth = []
for i, preset in enumerate(synthetic):
for ffv in (None, '152'):
random.seed(5000 + i)
config = fp.from_preset(copy.deepcopy(preset), ffv, salt=99)
synth.append({'preset': preset, 'ffVersion': ffv, 'moduleSeed': 5000 + i, 'config': hashed_lists(config)})
rand = []
for os_arg in (None, 'windows', 'mac', 'lin', ['linux', 'macos'], 'bsd'):
for ffv in (None, '152', '148', 'abc'):
for seed in range(10):
random.seed(seed)
preset = fp.get_random_preset(os=os_arg, ff_version=ffv)
rand.append({'os': os_arg, 'ffVersion': ffv, 'moduleSeed': seed,
'hash': None if preset is None else h(preset)})
app_versions = []
uas = set(UAS)
for fname in ('fingerprint-presets.json', 'fingerprint-presets-v150.json'):
for entries in json.loads((base / fname).read_text())['presets'].values():
for preset in entries:
ua = (preset.get('navigator') or {}).get('userAgent')
if ua:
uas.add(ua)
uas |= {'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0',
'Mozilla/5.0 (X11; Linux x86_64)', 'Mozilla/5.0 (rv:1.0)', 'Mozilla/5.0 ()', None}
for ua in sorted(uas, key=lambda x: x or ''):
app_versions.append([ua, fp._app_version_from_user_agent(ua)])
write('presets.json', {'cases': cases, 'full': full, 'synthetic': synth, 'random': rand,
'appVersions': app_versions,
'presetsFile': {str(v): Path(fp._select_presets_file(v)).name
for v in (None, '148', '149', '150.0.2', 'abc', 152, '', ' 150')}})
def record_constants():
write('constants.json', compress=False, data={
'fpgenData': fp.FPGEN_DATA,
'essentialMacos': fp._ESSENTIAL_FONTS_MACOS,
'essentialWindows': fp._ESSENTIAL_FONTS_WINDOWS,
'essentialLinux': fp._ESSENTIAL_FONTS_LINUX,
'markers': {'macos': fp._MACOS_MARKER_FONTS, 'windows': fp._WINDOWS_MARKER_FONTS,
'linux': fp._LINUX_MARKER_FONTS},
'windows11Markers': sorted(fp.WINDOWS_11_MARKER_FONTS),
'plausibleCoreCounts': list(fp.PLAUSIBLE_CORE_COUNTS),
'modernScreenFloor': list(fp.MODERN_SCREEN_FLOOR),
'appleSiliconCores': sorted(coherence.APPLE_SILICON_CORES),
'plausibleDpr': {k: list(v) for k, v in coherence.PLAUSIBLE_DPR.items()},
'plausibleColorDepth': sorted(coherence.PLAUSIBLE_COLOR_DEPTH),
'maxTouchPoints': coherence.MAX_PLAUSIBLE_TOUCH_POINTS,
'browserChromeHeight': coherence.BROWSER_CHROME_HEIGHT,
'rules': [r.name for r in coherence.RULES],
'macNovelty': sorted(fp._MAC_NOVELTY_VOICES),
'macEloquence': sorted(fp._MAC_ELOQUENCE_VOICES),
'presetsV150MinFf': fp.PRESETS_V150_MIN_FF,
})
def record_init_script():
cases = [
{},
{'audioFingerprintSeed': 123,
'navigatorPlatform': 'Win32', 'navigatorOscpu': 'Windows NT 10.0; Win64; x64',
'navigatorUserAgent': UAS[3], 'hardwareConcurrency': 8, 'webglVendor': 'Google Inc. (Intel)',
'webglRenderer': 'ANGLE (Intel, "quoted" é)', 'screenWidth': 1920, 'screenHeight': 1080,
'screenColorDepth': 24, 'timezone': 'Europe/Paris', 'fontList': ['Arial', '微软雅黑', 'Segoe UI'],
'speechVoices': [{'name': 'Microsoft David'}, 'Plain Name'], 'webrtcIP': '1.2.3.4'},
{'screenWidth': 1920, 'screenHeight': None, 'screenColorDepth': 24, 'webrtcIP': '', 'fontList': [],
'speechVoices': [], 'timezone': ''},
{'navigatorUserAgent': 'emoji 😀 
 \x7f tab\t', 'hardwareConcurrency': None},
{'webrtcIP': '2001:db8::7'},
]
out = [{'values': v, 'script': fp._build_init_script(v)} for v in cases]
write('init-script.json', {'cases': out}, compress=False)
def main():
record_constants()
record_pyrandom()
record_numpy()
record_pycompat()
record_fpgen_salts()
record_fonts()
record_voices()
record_media()
record_webgl()
record_geometry()
record_coherence()
record_from_fpgen_inputs()
record_presets()
record_init_script()
if __name__ == '__main__':
main()
typescript/scripts/golden/launch_golden.py
+529
View File
@@ -0,0 +1,529 @@
#!/usr/bin/env python3
"""Record Python launch_options() outputs as goldens for the TypeScript port.
.venv/bin/python typescript/scripts/golden/launch_golden.py
Writes typescript/tests/fixtures/launch/*.json, one file per scenario, and
typescript/tests/fixtures/launch/inputs.json (the fixed fingerprints / presets
the scenarios are built from). tests/launch-golden.test.ts replays every
scenario through the TypeScript launchOptions() and requires the same result.
Every source of nondeterminism is pinned, and every host probe is replaced, so
the goldens are a function of the code alone:
* identity salt: scenarios pass a fixed fingerprint / preset dict (salt is a
hash of it); the few that exercise the generate / random-preset paths patch
the fresh salt, generate_fingerprint and get_random_preset.
* host: XDG_CACHE_HOME points the camoufox cache at a scratch dir; the stock
profile disk capacity, the host OS key, the host CPU count, the monitor
probe and the public-IP lookup are patched; the GeoIP reader is a fake
maxminddb module over a fixed table; numpy's weighted locale choice uses a
fixed uniform draw (same algorithm as numpy, so the TS port can mirror it).
* paths: the browser bundle is tests/fixtures/launch/bundle*, the cache the
scratch dir; both are written back as <BUNDLE>/<CACHE>/... placeholders.
Recorded on Linux: OS_NAME affects the chunk size and the fontconfig, and the
TS test only replays on Linux.
"""
import copy
import io
import json
import os
import re
import sys
import tempfile
import types
import warnings
from contextlib import redirect_stdout
from pathlib import Path
HERE = Path(__file__).resolve().parent
TS_ROOT = HERE.parent.parent
FIXTURES = TS_ROOT / 'tests' / 'fixtures' / 'launch'
# Must precede every camoufox import: INSTALL_DIR is computed at import time.
SCRATCH = Path(tempfile.mkdtemp(prefix='camoufox-golden-'))
os.environ['XDG_CACHE_HOME'] = str(SCRATCH / 'xdg-cache')
CACHE = SCRATCH / 'xdg-cache' / 'camoufox'
HOME = SCRATCH / 'home'
HOME.mkdir(parents=True)
# ---------------------------------------------------------------- fake GeoIP
GEO_TABLE = {
'8.8.8.8': {'country_code': 'US', 'longitude': -97.822, 'latitude': 37.751, 'timezone': 'America/Chicago'},
'81.2.69.160': {'country_code': 'GB', 'longitude': -0.0931, 'latitude': 51.5142, 'timezone': 'Europe/London'},
'2a01:4f8::1': {'country_code': 'DE', 'longitude': 9.491, 'latitude': 51.2993, 'timezone': 'Europe/Berlin'},
'203.0.113.7': {'country_code': 'JP', 'longitude': 139.6899, 'latitude': 35.6893, 'timezone': 'Asia/Tokyo'},
}
_fake_mmdb = types.ModuleType('maxminddb')
class _FakeReader:
def __init__(self, path):
self.path = path
def get(self, ip):
return copy.deepcopy(GEO_TABLE.get(ip))
def __enter__(self):
return self
def __exit__(self, *a):
return False
_fake_mmdb.open_database = lambda path: _FakeReader(path)
sys.modules['maxminddb'] = _fake_mmdb
import numpy as np # noqa: E402
import orjson # noqa: E402
import camoufox.fingerprints as fingerprints # noqa: E402
from camoufox import geolocation, locales, utils # noqa: E402
from camoufox.addons import DefaultAddons # noqa: E402
from camoufox.fingerprints import Screen # noqa: E402
from camoufox.utils import launch_options # noqa: E402
assert str(utils.INSTALL_DIR) == str(CACHE), (utils.INSTALL_DIR, CACHE)
# The mmdb files only have to exist and be fresh; the fake reader answers.
for name in ('maxmind geolite2-ipv4.mmdb', 'maxmind geolite2-ipv6.mmdb'):
p = geolocation.MMDB_DIR / name
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(b'')
# The default addon, already "downloaded", so nothing is fetched.
(CACHE / 'addons' / 'UBO').mkdir(parents=True)
(CACHE / 'addons' / 'UBO' / 'manifest.json').write_text('{}')
# ------------------------------------------------------------ host probes
HOST = {
'cpu_count': 16,
'disk_capacity_kb': 250_000_000,
'host_os_key': 'lin',
'display': [1600, 900],
'public_ip': '81.2.69.160',
'locale_uniform': 0.5,
'fresh_salt': 1234567890123456789,
}
fingerprints.host_cpu_count = lambda: HOST['cpu_count']
utils._stock_profile_disk_capacity_kb = lambda: HOST['disk_capacity_kb']
utils._host_os_key = lambda: HOST['host_os_key']
def _display():
from camoufox.display import DisplaySize
return DisplaySize(*HOST['display'])
utils.largest_display = _display
utils.public_ip = lambda proxy=None: HOST['public_ip']
_real_salt = utils.identity_salt
utils.identity_salt = lambda pinned=None: _real_salt(pinned) if pinned is not None else HOST['fresh_salt']
def _fixed_choice(a, p=None):
# numpy.random.RandomState.choice(a, p=p), with the uniform draw pinned.
cdf = np.cumsum(p)
cdf /= cdf[-1]
return a[int(cdf.searchsorted(HOST['locale_uniform'], side='right'))]
locales.np.random.choice = _fixed_choice
# ------------------------------------------------------------------ inputs
FPGEN_KEYS = ('navigator', 'screen', 'window', 'headers')
def js_equivalent(value):
"""What the value is once it has been through JSON in JavaScript: an
integral float becomes an int, and a dict's array-index keys move to the
front in ascending order (JS object key order)."""
if isinstance(value, float) and value.is_integer() and abs(value) < 2**53:
return int(value)
if isinstance(value, dict):
idx = sorted((k for k in value if isinstance(k, str) and _is_index(k)), key=int)
rest = [k for k in value if k not in idx]
return {k: js_equivalent(value[k]) for k in idx + rest}
if isinstance(value, (list, tuple)):
return [js_equivalent(v) for v in value]
return value
def _is_index(k):
return k.isdigit() and (k == '0' or not k.startswith('0')) and int(k) < 2**32 - 1
def make_fingerprint(os_name):
fp = fingerprints._generator().generate(browser='Firefox', os=fingerprints._FPGEN_OS[os_name])
fp = {k: fp[k] for k in FPGEN_KEYS if k in fp}
# handle_screenXY draws screenY with randrange outside [-50, 50]
fp.setdefault('window', {})['screenX'] = 0
return js_equivalent(fp)
def load_inputs():
path = FIXTURES / 'inputs.json'
if path.exists() and '--regen-inputs' not in sys.argv:
return json.loads(path.read_text())
presets = orjson.loads((Path(fingerprints.__file__).parent / 'fingerprint-presets-v150.json').read_bytes())['presets']
from camoufox.webgl import firefox_gpus
pairs = {os_key: sorted(firefox_gpus(os_key)) for os_key in ('win', 'mac', 'lin')}
known = {x for v in pairs.values() for x in v}
def pick(os_key):
for p in presets[os_key]:
gl = p.get('webgl') or {}
if (gl.get('unmaskedVendor'), gl.get('unmaskedRenderer')) in known:
return p
raise SystemExit(f'no preset for {os_key} with a recorded GPU')
inputs = {
'fingerprints': {o: make_fingerprint(o) for o in ('linux', 'windows', 'macos')},
'presets': {
'windows': pick('windows'),
'macos': pick('macos'),
'linux': pick('linux'),
# A GPU fpgen has never seen, which launch_options must refuse.
'windows_unknown_gpu': {**pick('windows'), 'webgl': {
'unmaskedVendor': 'Google Inc. (Acme)',
'unmaskedRenderer': 'ANGLE (Acme, Acme GPU 9000 Direct3D11 vs_5_0 ps_5_0)'}},
},
'webgl_pairs': {k: [list(x) for x in v[:2]] for k, v in pairs.items()},
}
inputs = js_equivalent(inputs)
path.write_text(json.dumps(inputs, indent=1, ensure_ascii=False) + '\n')
return inputs
INPUTS = load_inputs()
# ---------------------------------------------------------------- scenarios
BUNDLE = FIXTURES / 'bundle'
BUNDLE_OLD = FIXTURES / 'bundle-old'
ADDON = FIXTURES / 'addons' / 'example-addon'
PLACEHOLDERS = [
(str(BUNDLE_OLD), '<BUNDLE_OLD>'),
(str(BUNDLE), '<BUNDLE>'),
(str(ADDON), '<ADDON>'),
(str(CACHE), '<CACHE>'),
(str(HOME), '<HOME>'),
]
def fill(value):
"""Scenario kwargs -> real values."""
if isinstance(value, str):
for real, ph in PLACEHOLDERS:
value = value.replace(ph, real)
return value
if isinstance(value, dict):
return {k: fill(v) for k, v in value.items()}
if isinstance(value, list):
return [fill(v) for v in value]
return value
def mask(value):
if isinstance(value, str):
for real, ph in PLACEHOLDERS:
value = value.replace(real, ph)
# A FallbackWarning's report block names the host and the runtime
# (python/node), so neither launcher can reproduce the other's.
value = re.sub(r'(and include:\n\n)( .*(\n|$))+', r'\1<REPORT>', value)
# The name hashes the fonts.conf content, which embeds the checkout path;
# the TS test masks it the same way and checks the hash itself.
return re.sub(r'fonts-[0-9a-f]{12}\.conf', 'fonts-<HASH>.conf', value)
if isinstance(value, dict):
return {mask(k): mask(v) for k, v in value.items()}
if isinstance(value, (list, tuple)):
return [mask(v) for v in value]
return value
BASE = {'executable_path': '<BUNDLE>/camoufox-bin', 'env': {'HOME': '<HOME>'}}
FP = INPUTS['fingerprints']
PR = INPUTS['presets']
S = {}
def scenario(name, _special=None, **kwargs):
assert name not in S, name
S[name] = {'kwargs': {**BASE, **kwargs}, 'special': _special or {}}
# fpgen fingerprints, one per target OS
for _os in ('linux', 'windows', 'macos'):
scenario(f'fpgen_{_os}', fingerprint=FP[_os], os=_os)
scenario('fpgen_linux_ikwid', fingerprint=FP['linux'], os='linux', i_know_what_im_doing=True)
scenario('fpgen_no_os', fingerprint=FP['windows'])
scenario('fpgen_os_list', fingerprint=FP['macos'], os=['macos', 'windows'])
# presets
for _os in ('windows', 'macos', 'linux'):
scenario(f'preset_{_os}', fingerprint_preset=PR[_os])
scenario('preset_windows_unknown_gpu', fingerprint_preset=PR['windows_unknown_gpu'])
scenario('preset_random', {'random_preset': 'macos'}, fingerprint_preset=True, os='macos')
# generation path (generate_fingerprint patched to a fixed fingerprint)
scenario('generate_default', {'generate': 'linux'})
scenario('generate_window', {'generate': 'windows'}, window=[1280, 720], os='windows')
scenario('generate_screen', {'generate': 'macos'}, screen={'max_width': 1920, 'max_height': 1080}, os='macos')
scenario('generate_headful_display', {'generate': 'linux'}, headless=False,
env={'HOME': '<HOME>', 'DISPLAY': ':0'})
scenario('generate_headless_display', {'generate': 'linux'}, headless=True,
env={'HOME': '<HOME>', 'DISPLAY': ':0'})
# headless / display handling on a fixed identity
scenario('headful_display_clamp', fingerprint=FP['windows'], os='windows', headless=False,
env={'HOME': '<HOME>', 'DISPLAY': ':0'})
scenario('headless_true', fingerprint=FP['linux'], os='linux', headless=True)
scenario('virtual_display', fingerprint=FP['linux'], os='linux', virtual_display=':99',
env={'HOME': '<HOME>', 'WAYLAND_DISPLAY': 'wayland-0', 'GDK_BACKEND': 'wayland', 'KEEP': '1'})
# locale
scenario('locale_full', fingerprint=FP['linux'], os='linux', locale='fr-FR')
scenario('locale_script', fingerprint=FP['linux'], os='linux', locale='zh-Hans-CN')
scenario('locale_language_only', fingerprint=FP['windows'], os='windows', locale='de')
scenario('locale_region_only', fingerprint=FP['windows'], os='windows', locale='CA')
scenario('locale_list', fingerprint=FP['macos'], os='macos', locale=['en-US', 'fr-FR', 'de', 'en-US'])
scenario('locale_string_list', fingerprint=FP['macos'], os='macos', locale='ja-JP, en')
scenario('locale_invalid', fingerprint=FP['linux'], os='linux', locale='xx-invalid-tag-!!')
# geoip / proxy
scenario('geoip_ipv4', fingerprint=FP['windows'], os='windows', geoip='8.8.8.8')
scenario('geoip_ipv6', fingerprint=FP['windows'], os='windows', geoip='2a01:4f8::1')
scenario('geoip_true_proxy', fingerprint=FP['linux'], os='linux', geoip=True,
proxy={'server': 'http://proxy.example:8080', 'username': 'u', 'password': 'p'})
scenario('geoip_true_no_proxy', fingerprint=FP['linux'], os='linux', geoip=True)
scenario('geoip_block_webrtc', fingerprint=FP['macos'], os='macos', geoip='203.0.113.7', block_webrtc=True)
scenario('geoip_with_locale', fingerprint=FP['macos'], os='macos', geoip='8.8.8.8', locale='es-MX')
scenario('geoip_manual_timezone', fingerprint=FP['macos'], os='macos', geoip='8.8.8.8',
config={'timezone': 'Europe/Paris', 'locale:language': 'fr', 'locale:region': 'FR'})
scenario('geoip_db_named', fingerprint=FP['linux'], os='linux', geoip='8.8.8.8', geoip_db='MaxMind GeoLite2')
scenario('geoip_unknown_ip', fingerprint=FP['linux'], os='linux', geoip='192.0.2.1')
scenario('geoip_invalid_ip', fingerprint=FP['linux'], os='linux', geoip='not-an-ip')
scenario('proxy_without_geoip', fingerprint=FP['linux'], os='linux', proxy={'server': 'http://proxy.example:8080'})
scenario('proxy_localhost', fingerprint=FP['linux'], os='linux', proxy={'server': 'http://localhost:8080'})
scenario('proxy_manual_geolocation', fingerprint=FP['linux'], os='linux', proxy={'server': 'socks5://1.2.3.4:1080'},
config={'geolocation:latitude': 10.5, 'geolocation:longitude': 20.25})
# humanize
scenario('humanize_true', fingerprint=FP['linux'], os='linux', humanize=True)
scenario('humanize_float', fingerprint=FP['linux'], os='linux', humanize=1.5)
scenario('humanize_int', fingerprint=FP['linux'], os='linux', humanize=2)
scenario('humanize_false', fingerprint=FP['linux'], os='linux', humanize=False)
# block_* and friends
scenario('block_all', fingerprint=FP['windows'], os='windows', block_images=True, block_webrtc=True,
block_webgl=True, disable_coop=True)
scenario('block_all_ikwid', fingerprint=FP['windows'], os='windows', block_images=True, block_webrtc=True,
block_webgl=True, disable_coop=True, i_know_what_im_doing=True)
scenario('allow_webgl_false', fingerprint=FP['linux'], os='linux', allow_webgl=False)
scenario('flags', fingerprint=FP['macos'], os='macos', main_world_eval=True, allow_addon_new_tab=True,
enable_cache=True, args=['--foo', '--bar=1'], firefox_user_prefs={'my.pref': 'x', 'ui.useOverlayScrollbars': 0,
'intl.locale.requested': 'de-DE'})
scenario('ff_version', fingerprint=FP['windows'], os='windows', ff_version=140)
scenario('pin_cpu_cores', fingerprint=FP['linux'], os='linux', pin_cpu_cores=True)
# addons
scenario('addons_custom', fingerprint=FP['linux'], os='linux', addons=['<ADDON>'])
scenario('addons_exclude_default', fingerprint=FP['linux'], os='linux', exclude_addons=['UBO'])
scenario('addons_only_custom', fingerprint=FP['linux'], os='linux', addons=['<ADDON>'], exclude_addons=['UBO'])
scenario('addons_invalid', fingerprint=FP['linux'], os='linux', addons=['<BUNDLE>/fonts'])
# webgl
scenario('webgl_config_windows', fingerprint=FP['windows'], os='windows', webgl_config=INPUTS['webgl_pairs']['win'][0])
scenario('webgl_config_macos', fingerprint=FP['macos'], os='macos', webgl_config=INPUTS['webgl_pairs']['mac'][0])
scenario('webgl_config_no_os', fingerprint=FP['linux'], webgl_config=INPUTS['webgl_pairs']['lin'][0])
scenario('webgl_config_unknown', fingerprint=FP['linux'], os='linux', webgl_config=['Nope', 'Nope GPU'])
# fonts / voices
scenario('fonts_custom', fingerprint=FP['linux'], os='linux', fonts=['Arial', 'Helvetica', 'Comic Sans MS'])
scenario('fonts_custom_only', fingerprint=FP['windows'], os='windows', fonts=['Arial'], custom_fonts_only=True)
scenario('fonts_custom_only_missing', fingerprint=FP['windows'], os='windows', custom_fonts_only=True)
scenario('fonts_config', fingerprint=FP['macos'], os='macos', config={'fonts': ['Helvetica', 'Menlo']})
scenario('fonts_config_empty', fingerprint=FP['macos'], os='macos', config={'fonts': []})
scenario('voices_config', fingerprint=FP['macos'], os='macos', config={'voices': [
{'lang': 'en-US', 'name': 'Samantha', 'voiceUri': 'com.apple.voice.compact.en-US.Samantha', 'isDefault': True,
'isLocalService': True}]})
scenario('voices_config_bad', fingerprint=FP['macos'], os='macos', config={'voices': ['Samantha:en-US:local']})
scenario('voices_config_missing_field', fingerprint=FP['macos'], os='macos', config={'voices': [{'lang': 'en-US'}]})
# config overrides and the warnings they emit
scenario('config_navigator', fingerprint=FP['windows'], os='windows',
config={'navigator.userAgent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:152.0) Gecko/20100101 Firefox/152.0',
'navigator.platform': 'Win32', 'navigator.hardwareConcurrency': 6})
scenario('config_ua_only', {'generate': 'macos'}, config={
'navigator.userAgent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:152.0) Gecko/20100101 Firefox/152.0'})
scenario('config_ua_ikwid', {'generate': 'linux'}, i_know_what_im_doing=True, config={
'navigator.userAgent': 'Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0'})
scenario('config_header_ua', fingerprint=FP['linux'], os='linux', config={'headers.User-Agent': 'x'})
scenario('config_locale_keys', fingerprint=FP['linux'], os='linux',
config={'locale:language': 'pt', 'locale:region': 'BR', 'headers.Accept-Language': 'pt-BR'})
scenario('config_screen', fingerprint=FP['macos'], os='macos',
config={'screen.width': 1440, 'screen.height': 900, 'window.outerWidth': 1200})
scenario('config_touch', fingerprint=FP['windows'], os='windows', config={'navigator.maxTouchPoints': 5})
scenario('config_dnt_gpc', fingerprint=FP['windows'], os='windows',
config={'navigator.doNotTrack': '1', 'navigator.globalPrivacyControl': True})
scenario('config_accept_encoding', fingerprint=FP['linux'], os='linux', config={'headers.Accept-Encoding': 'gzip'})
scenario('config_seeds', fingerprint=FP['linux'], os='linux', config={'audio:seed': 42})
scenario('config_instant_animations', fingerprint=FP['linux'], os='linux', config={'instantAnimations': True})
scenario('config_media_devices', fingerprint=FP['linux'], os='linux', config={'mediaDevices:micros': 0})
scenario('config_webgl_pair', fingerprint=FP['linux'], os='linux',
config={'webGl:vendor': INPUTS['webgl_pairs']['lin'][1][0], 'webGl:renderer': INPUTS['webgl_pairs']['lin'][1][1]})
scenario('config_webgl_unknown_pair', fingerprint=FP['linux'], os='linux',
config={'webGl:vendor': 'Nope', 'webGl:renderer': 'Nope GPU'})
scenario('config_unknown_key', fingerprint=FP['linux'], os='linux',
config={'not.a.real.property': {'a': [1, 'b', None, True]}, 'also.unknown': 'x'})
scenario('config_bad_type', fingerprint=FP['linux'], os='linux', config={'navigator.buildID': 5})
scenario('config_float_int', fingerprint=FP['linux'], os='linux', config={'screen.width': 1920.0,
'window.devicePixelRatio': 2})
scenario('config_non_ascii', fingerprint=FP['linux'], os='linux', config={'timezone': 'America/São_Paulo'},
firefox_user_prefs={'my.unicode.pref': 'héllo ✓ \U0001F600'})
# errors / validation
scenario('invalid_os_case', fingerprint=FP['linux'], os='Linux')
scenario('invalid_os_name', fingerprint=FP['linux'], os='beos')
scenario('non_firefox_fingerprint', fingerprint={**FP['linux'], 'navigator': {
**FP['linux']['navigator'], 'userAgent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0 Safari/537.36'}})
# persistent context / passthrough Playwright options
scenario('persistent_passthrough', fingerprint=FP['linux'], os='linux', user_data_dir='<HOME>/profile',
slow_mo=50, timeout=12345)
# executable_path handling
scenario('executable_old_build', fingerprint=FP['linux'], os='linux', executable_path='<BUNDLE_OLD>/camoufox-bin')
scenario('executable_from_env', {'process_env': {'CAMOUFOX_EXECUTABLE_PATH': '<BUNDLE>/camoufox-bin'}},
fingerprint=FP['linux'], os='linux', executable_path=None)
# chunking: a config bigger than one env var chunk
scenario('config_large', fingerprint=FP['linux'], os='linux', i_know_what_im_doing=True,
config={'fonts': [f'Font Family {i:05d}' for i in range(2600)]})
# ------------------------------------------------------------------ running
def run(name, spec):
kwargs = fill(copy.deepcopy(spec['kwargs']))
special = spec['special']
calls = []
restore = []
if kwargs.get('executable_path') is None:
kwargs.pop('executable_path', None)
if 'screen' in kwargs:
kwargs['screen'] = Screen(**kwargs['screen'])
if 'exclude_addons' in kwargs:
kwargs['exclude_addons'] = [DefaultAddons[x] for x in kwargs['exclude_addons']]
if 'webgl_config' in kwargs:
kwargs['webgl_config'] = tuple(kwargs['webgl_config'])
if 'window' in kwargs:
kwargs['window'] = tuple(kwargs['window'])
if 'generate' in special:
base_fp = FP[special['generate']]
def fake_generate(window=None, screen=None, os=None, **conditions):
calls.append({'fn': 'generate_fingerprint', 'window': list(window) if window else None,
'screen': None if screen is None else {k: getattr(screen, k) for k in (
'min_width', 'max_width', 'min_height', 'max_height')},
'os': os})
fp = copy.deepcopy(base_fp)
if window:
fingerprints.handle_window_size(fp, *window)
return fp
restore.append(('generate_fingerprint', utils.generate_fingerprint))
utils.generate_fingerprint = fake_generate
if 'random_preset' in special:
preset = PR[special['random_preset']]
def fake_random_preset(os=None, ff_version=None):
calls.append({'fn': 'get_random_preset', 'os': os, 'ff_version': ff_version})
return copy.deepcopy(preset)
restore.append(('get_random_preset', utils.get_random_preset))
utils.get_random_preset = fake_random_preset
env_backup = {}
for k, v in special.get('process_env', {}).items():
env_backup[k] = os.environ.get(k)
os.environ[k] = fill(v)
out = io.StringIO()
record = {'name': name, 'kwargs': spec['kwargs'], 'special': special}
try:
with warnings.catch_warnings(record=True) as caught, redirect_stdout(out):
warnings.simplefilter('always')
try:
result = launch_options(**kwargs)
except Exception as exc: # recorded, not raised
record['error'] = {'type': type(exc).__name__, 'message': mask(str(exc))}
result = None
record['warnings'] = [{'category': w.category.__name__, 'message': mask(str(w.message))} for w in caught]
finally:
for attr, value in restore:
setattr(utils, attr, value)
for k, v in env_backup.items():
if v is None:
os.environ.pop(k, None)
else:
os.environ[k] = v
record['stdout'] = mask(out.getvalue())
record['calls'] = mask(calls)
if result is not None:
record['result'] = describe(result)
return record
def describe(result):
env = dict(result['env'])
config_chunks = sorted(((int(k.rsplit('_', 1)[1]), k) for k in env if k.startswith('CAMOU_CONFIG_')))
pref_chunks = sorted(((int(k.rsplit('_', 1)[1]), k) for k in env if k.startswith('CAMOU_PREFS_')))
config_blob = ''.join(env[k] for _, k in config_chunks)
prefs_blob = ''.join(env[k] for _, k in pref_chunks)
fontconfig = None
if 'FONTCONFIG_FILE' in env:
fontconfig = {'path': mask(env['FONTCONFIG_FILE']), 'content': mask(Path(env['FONTCONFIG_FILE']).read_text())}
other_env = {k: v for k, v in env.items() if not k.startswith(('CAMOU_CONFIG_', 'CAMOU_PREFS_'))}
rest = {k: v for k, v in result.items() if k not in ('env',)}
config = orjson.loads(config_blob)
js_blob = mask(orjson.dumps(js_equivalent(config)).decode())
return {
'options': mask(js_equivalent(rest)),
'env': mask(other_env),
'config_chunks': [len(env[k]) for _, k in config_chunks],
'prefs_chunks': [len(env[k]) for _, k in pref_chunks],
# The config as JavaScript serializes the same data (see js_equivalent);
# the raw Python blob too, only when it differs from that.
'config_blob_js': js_blob,
**({'config_blob_raw': mask(config_blob)} if mask(config_blob) != js_blob else {}),
'prefs_blob': prefs_blob,
'fontconfig': fontconfig,
}
def main():
only = [a for a in sys.argv[1:] if not a.startswith('--')]
for old in FIXTURES.glob('scenario-*.json'):
if not only:
old.unlink()
summary = {'host': HOST, 'geo_table': GEO_TABLE, 'scenarios': []}
for name, spec in S.items():
if only and name not in only:
continue
rec = run(name, spec)
(FIXTURES / f'scenario-{name}.json').write_text(json.dumps(rec, indent=1, ensure_ascii=False) + '\n')
summary['scenarios'].append(name)
status = rec.get('error', {}).get('type', 'ok')
print(f'{name:32s} {status:24s} warnings={len(rec["warnings"])}', file=sys.stderr)
if not only:
(FIXTURES / 'host.json').write_text(json.dumps(summary, indent=1) + '\n')
if __name__ == '__main__':
main()
typescript/scripts/golden/pyrandom_cases.py
+63
View File
@@ -0,0 +1,63 @@
"""random.Random sequences for tests/pyrandom.test.ts (stdlib only).
Imported by identity_golden.py, and runnable on its own under any CPython to
check that the recorded sequences do not depend on the Python version:
python3.10 typescript/scripts/golden/pyrandom_cases.py > /tmp/310.json
python3.12 typescript/scripts/golden/pyrandom_cases.py > /tmp/312.json
.venv/bin/python typescript/scripts/golden/pyrandom_cases.py > /tmp/314.json
"""
import json
import sys
from random import Random
SEEDS = [0, 1, 2, 7, 42, 99, 12345, 2**31 - 1, 2**32 - 1, 2**32, 2**32 + 1, 2**53 + 3,
2**64 - 1, 2**64 + 5, 123456789012345678901234567890, -7, -(2**40)]
STR_SEEDS = ['', 'hello', 'camoufox', 'héllo wörld 😀']
def seed_json(seed):
return {'int': str(seed)} if isinstance(seed, int) else {'str': seed}
def cases():
out = []
for seed in SEEDS + STR_SEEDS:
r = Random(seed)
case = {'seed': seed_json(seed)}
case['random'] = [r.random() for _ in range(8)]
case['getrandbits'] = [[k, str(r.getrandbits(k))] for k in (1, 2, 5, 8, 16, 31, 32, 33, 40, 53, 64, 65, 100, 0)]
case['randbelow'] = [[n, r._randbelow(n)] for n in (1, 2, 3, 10, 100, 1000, 2**31, 2**32 - 1, 2**32 + 7, 10**15)]
case['randrange'] = [
[[10], r.randrange(10)],
[[5, 15], r.randrange(5, 15)],
[[-20, -3], r.randrange(-20, -3)],
[[0, 100, 7], r.randrange(0, 100, 7)],
[[100, 0, -3], r.randrange(100, 0, -3)],
]
case['randint'] = [[a, b, r.randint(a, b)] for a, b in ((1, 6), (0, 0), (-5, 5), (1, 4294967295), (0, 2**40))]
pop = list('abcdefghijklmnopqrstuvwxyz')
case['choice'] = [r.choice(pop) for _ in range(5)]
case['choices'] = {
'plain': r.choices(pop, k=6),
'weights': r.choices(pop[:5], weights=[0.1, 0.5, 2.0, 1.25, 0.15], k=6),
'cum_weights': r.choices(pop[:4], cum_weights=[1, 3, 6, 10], k=6),
}
shuffled = list(range(20))
r.shuffle(shuffled)
case['shuffle'] = shuffled
# sample: the pool branch (n <= setsize) and the set branch (n > setsize)
case['sample'] = [
[n, k, r.sample(range(n), k)]
for n, k in ((5, 5), (10, 3), (21, 6), (22, 6), (30, 5), (40, 10), (100, 4), (100, 6),
(500, 30), (1000, 7), (60, 50), (1, 0), (300, 300))
]
case['uniform'] = [r.uniform(-3.5, 10.25) for _ in range(3)]
case['after'] = r.random()
out.append(case)
return out
if __name__ == '__main__':
json.dump({'python': sys.version.split()[0], 'cases': cases()}, sys.stdout, sort_keys=True)
typescript/src/__main__.ts
+1345
View File
File diff suppressed because it is too large Load Diff
typescript/src/__version__.ts
+40
View File
@@ -0,0 +1,40 @@
/**
* Camoufox version constants.
*
* TypeScript twin of pythonlib/camoufox/__version__.py.
*/
// biome-ignore lint/complexity/noStaticOnlyClass: mirrors the Python twin's CONSTRAINTS class so both launchers read the same
export class CONSTRAINTS {
/**
* The minimum and maximum supported versions of the Camoufox browser.
*/
static readonly MIN_VERSION: string = "alpha.1";
static readonly MAX_VERSION: string = "1";
/**
* The browser floor is conditional on the resolved Playwright, not fixed.
*
* Each entry is [playwrightVersion, requiredBrowserBuild]: from that
* Playwright on, the browser must be at least that build. 1.61 began
* sending viewport isMobile/screenSize in Browser.setDefaultViewport and
* Page.setViewportSize; beta.30 is the first build whose Protocol.js schema
* accepts them. Below that pairing every newContext() dies with
* "Protocol error (Browser.setDefaultViewport)". Measured: 1.60 works on
* beta.29 and beta.30; 1.61 and 1.62 fail on beta.29 and pass on beta.30.
*
* A flat MIN_VERSION cannot express this. It only knows about the browser,
* so to stay safe it has to assume the worst Playwright and force *every*
* user to re-download -- including the majority on <1.61, who are in no
* danger -- and it leaves the library unusable until the matching browser
* release is published. Keyed on Playwright, only the users who would
* actually break get moved.
*/
static readonly PLAYWRIGHT_BROWSER_FLOORS: ReadonlyArray<
readonly [readonly number[], string]
> = [[[1, 61], "beta.30"]];
}
/** Version of this launcher library. Kept in step with package.json and
* pythonlib's pyproject.toml. */
export const LIBRARY_VERSION = "0.5.7";
typescript/src/addons.ts
+109
View File
@@ -0,0 +1,109 @@
/**
* Default Firefox addon download/extraction.
*
* TypeScript twin of pythonlib/camoufox/addons.py.
*/
import * as fs from "node:fs";
import * as path from "node:path";
import { InvalidAddonPath } from "./exceptions.js";
import { INSTALL_DIR, unzip, webdl } from "./pkgman.js";
/**
* Default addons to be downloaded.
*/
export const DefaultAddons = {
UBO: "https://addons.mozilla.org/firefox/downloads/latest/ublock-origin/latest.xpi",
} as const;
export type DefaultAddon = keyof typeof DefaultAddons;
// Addons are stored in a shared folder, not per-browser version
export const ADDONS_DIR: string = path.join(INSTALL_DIR, "addons");
/**
* Confirms that the addon paths are valid.
*/
export function confirmPaths(paths: string[]): void {
for (const addonPath of paths) {
if (!fs.existsSync(addonPath) || !fs.statSync(addonPath).isDirectory()) {
throw new InvalidAddonPath(addonPath);
}
if (!fs.existsSync(path.join(addonPath, "manifest.json"))) {
throw new InvalidAddonPath(
"manifest.json is missing. Addon path must be a path to an extracted addon.",
);
}
}
}
/**
* Adds default addons, minus any specified in excludeList, to addonsList.
*/
export async function addDefaultAddons(
addonsList: string[],
excludeList: DefaultAddon[] = [],
): Promise<void> {
const addons: Record<string, string> = {};
for (const [name, url] of Object.entries(DefaultAddons)) {
if (!excludeList.includes(name as DefaultAddon)) {
addons[name] = url;
}
}
await maybeDownloadAddons(addons, addonsList);
}
/**
* Downloads and extracts an addon from a given URL to a specified path.
*/
export async function downloadAndExtract(
url: string,
extractPath: string,
name: string,
): Promise<void> {
const buffer = await webdl(url, `Downloading addon (${name})`, false);
unzip(buffer, extractPath, `Extracting addon (${name})`, false);
}
/**
* Returns a path to the addon in the shared addons folder.
*/
export function getAddonPath(addonName: string): string {
return path.join(ADDONS_DIR, addonName);
}
/** Seams the Python tests reach with monkeypatch. */
export const addonsDeps = {
downloadAndExtract: (url: string, extractPath: string, name: string) =>
downloadAndExtract(url, extractPath, name),
};
/**
* Downloads and extracts addons from a given map into the given list.
* Skips downloading if the addon is already downloaded.
*/
export async function maybeDownloadAddons(
addons: Record<string, string>,
addonsList?: string[],
): Promise<void> {
for (const [addonName, url] of Object.entries(addons)) {
const addonPath = getAddonPath(addonName);
// Check if the addon is already extracted. A bare directory is not
// enough: a failed download leaves an empty dir behind, so require the
// manifest that confirmPaths() looks for.
if (fs.existsSync(path.join(addonPath, "manifest.json"))) {
addonsList?.push(addonPath);
continue;
}
try {
fs.mkdirSync(addonPath, { recursive: true });
await addonsDeps.downloadAndExtract(url, addonPath, addonName);
addonsList?.push(addonPath);
} catch (e) {
// Drop the partial directory so the next run re-downloads.
fs.rmSync(addonPath, { recursive: true, force: true });
console.log(`Failed to download and extract ${addonName}: ${e}`);
}
}
}
typescript/src/async_api.ts
+15
View File
@@ -0,0 +1,15 @@
/**
* TypeScript twin of pythonlib/camoufox/async_api.py.
*
* playwright-core has a single, promise-based API, so the async entry points
* are the same functions as sync_api.ts under Python's async names. The
* behaviours async_api.py adds over sync_api.py (the per-driver pin lock) live
* there already.
*/
export {
Camoufox as AsyncCamoufox,
NewBrowser as AsyncNewBrowser,
type NewBrowserOptions,
NewContext as AsyncNewContext,
type NewContextOptions,
} from "./sync_api.js";
typescript/src/coherence.ts
+382
View File
@@ -0,0 +1,382 @@
/**
* Whole-identity coherence: the checks that look at more than one field.
*
* TypeScript twin of pythonlib/camoufox/coherence.py -- see there for the
* measurements behind each rule. Camoufox assembles an identity from several
* independently sampled pools, so a combination no machine has ever had can
* be built out of individually plausible parts; every identity passes through
* here, whatever it was built from.
*
* `validate()` reports what is still broken; `apply()` repairs what it can.
*/
import { isPyInt, num, pyRepr, pyStr, pyTruthy } from "./pycompat.js";
type Config = Record<string, any>;
/** Core counts Apple Silicon actually ships. */
export const APPLE_SILICON_CORES: ReadonlySet<number> = new Set([
8, 10, 11, 12, 14, 16, 20, 24, 28, 32,
]);
/**
* devicePixelRatio by platform, ascending as in coherence.py: the
* nearest-step repair keeps the first of two equally near steps, so a tie
* goes to the lower one.
*/
export const PLAUSIBLE_DPR: Readonly<Record<string, readonly number[]>> = {
win: [1, 1.25, 1.5, 1.75, 2, 2.5, 3],
mac: [1, 2],
lin: [1, 1.25, 1.5, 1.75, 2],
};
/** colorDepth: Firefox reports 24, or 30 on a deep-colour display. */
export const PLAUSIBLE_COLOR_DEPTH: ReadonlySet<number> = new Set([24, 30]);
/** maxTouchPoints: consumer digitisers top out at 10 contacts. */
export const MAX_PLAUSIBLE_TOUCH_POINTS = 10;
/** The browser's own chrome height, in CSS pixels (a property of the binary). */
export const BROWSER_CHROME_HEIGHT = 86;
/** GPU strings that are not possible on macOS. */
const NOT_A_MAC_GPU = [
"ANGLE",
"Intel(R) HD Graphics 400",
"Radeon R9 200 Series",
"llvmpipe",
];
export interface Violation {
rule: string;
detail: string;
}
export interface Rule {
name: string;
/** Returns a description of the breakage, or null when the identity holds. */
check: (config: Config, targetOs: string) => string | null;
/** Repairs the identity in place. null where no correct value is determined. */
repair: ((config: Config, targetOs: string) => void) | null;
}
function isNone(value: unknown): boolean {
return value === null || value === undefined;
}
function renderer(config: Config): string {
const r = config["webGl:renderer"];
return pyTruthy(r) ? pyStr(r) : "";
}
function isAppleSilicon(config: Config): boolean {
return renderer(config).includes("Apple M");
}
function checkAppleSiliconCores(
config: Config,
_targetOs: string,
): string | null {
if (!isAppleSilicon(config)) return null;
const cores = config["navigator.hardwareConcurrency"];
if (isPyInt(cores) && !APPLE_SILICON_CORES.has(Number(cores))) {
return `${pyRepr(renderer(config))} with hardwareConcurrency ${pyStr(cores)}; Apple Silicon starts at 8`;
}
return null;
}
function repairAppleSiliconCores(config: Config, _targetOs: string): void {
const cores = config["navigator.hardwareConcurrency"];
if (!isPyInt(cores)) return;
const n = Number(cores);
const sorted = [...APPLE_SILICON_CORES].sort((a, b) => a - b);
config["navigator.hardwareConcurrency"] =
sorted.find((c) => c >= n) ?? Math.max(...sorted);
}
/**
* Whether this renderer string is one the OS can report. Used both to check
* a finished identity and to filter the WebGL pool before sampling.
*/
export function gpuFitsOs(
rendererString: string | null | undefined,
targetOs: string,
): boolean {
const r = pyTruthy(rendererString) ? pyStr(rendererString) : "";
if (!r) return true;
if (targetOs === "mac") return !NOT_A_MAC_GPU.some((bad) => r.includes(bad));
if (targetOs === "win") return r.startsWith("ANGLE");
if (targetOs === "lin") return !r.includes("ANGLE") && !r.includes("Apple M");
return true;
}
function checkGpuMatchesOs(config: Config, targetOs: string): string | null {
const r = renderer(config);
if (!r || gpuFitsOs(r, targetOs)) return null;
if (targetOs === "mac")
return `macOS identity with ${pyRepr(r)}, which no Mac reports`;
if (targetOs === "win") {
return `Windows identity with ${pyRepr(r)}; Firefox on Windows renders through ANGLE`;
}
return `Linux identity with ${pyRepr(r)}`;
}
function inColorDepths(depth: unknown): boolean {
return typeof depth !== "boolean" && PLAUSIBLE_COLOR_DEPTH.has(num(depth));
}
function checkColorDepth(config: Config, targetOs: string): string | null {
const depth = config["screen.colorDepth"];
if (isNone(depth)) return null;
if (!inColorDepths(depth))
return `screen.colorDepth ${pyStr(depth)}; Firefox reports 24 or 30`;
if (targetOs === "mac" && isAppleSilicon(config) && num(depth) !== 30) {
return `Apple Silicon Mac with colorDepth ${pyStr(depth)}; deep colour is the macOS default`;
}
return null;
}
function repairColorDepth(config: Config, targetOs: string): void {
const depth = config["screen.colorDepth"];
if (!isNone(depth) && !inColorDepths(depth)) config["screen.colorDepth"] = 24;
if (targetOs === "mac" && isAppleSilicon(config))
config["screen.colorDepth"] = 30;
// pixelDepth is the same number in every browser that reports both.
if ("screen.pixelDepth" in config || "screen.colorDepth" in config) {
config["screen.pixelDepth"] =
"screen.colorDepth" in config ? config["screen.colorDepth"] : 24;
}
}
function checkTouchPoints(config: Config, targetOs: string): string | null {
const touch = config["navigator.maxTouchPoints"];
if (isNone(touch)) return null;
if (
!isPyInt(touch) ||
Number(touch) < 0 ||
Number(touch) > MAX_PLAUSIBLE_TOUCH_POINTS
) {
return `navigator.maxTouchPoints ${pyStr(touch)}; a digitiser reports at most ${MAX_PLAUSIBLE_TOUCH_POINTS}`;
}
if (targetOs === "mac" && pyTruthy(touch)) {
return `macOS identity with maxTouchPoints ${pyStr(touch)}; no Mac has a touchscreen`;
}
return null;
}
function repairTouchPoints(config: Config, targetOs: string): void {
const touch = config["navigator.maxTouchPoints"];
const tooMany = isPyInt(touch) && Number(touch) > MAX_PLAUSIBLE_TOUCH_POINTS;
if (targetOs === "mac" || tooMany || (!isNone(touch) && !isPyInt(touch))) {
config["navigator.maxTouchPoints"] = 0;
}
}
function checkDevicePixelRatio(
config: Config,
targetOs: string,
): string | null {
const dpr = config["window.devicePixelRatio"];
if (isNone(dpr)) return null;
const allowed = PLAUSIBLE_DPR[targetOs];
if (allowed?.length && !allowed.includes(Number(num(dpr)))) {
return `window.devicePixelRatio ${pyStr(dpr)} is not a display mode ${targetOs} offers`;
}
return null;
}
function repairDevicePixelRatio(config: Config, targetOs: string): void {
const dpr = config["window.devicePixelRatio"];
const allowed = PLAUSIBLE_DPR[targetOs];
if (isNone(dpr) || !allowed?.length) return;
// Nearest real scaling step; min() keeps the first of equal distances.
const x = Number(num(dpr));
let best = allowed[0];
for (const v of allowed.slice(1)) {
if (Math.abs(v - x) < Math.abs(best - x)) best = v;
}
config["window.devicePixelRatio"] = best;
}
function checkWindowChrome(config: Config, _targetOs: string): string | null {
const inner = config["window.innerHeight"];
const outer = config["window.outerHeight"];
if (!pyTruthy(inner) || !pyTruthy(outer)) return null;
const chrome = num(outer) - num(inner);
if (chrome < BROWSER_CHROME_HEIGHT) {
return (
`window.outerHeight ${pyStr(outer)} - innerHeight ${pyStr(inner)} = ${pyStr(chrome)}, less than the ` +
`${BROWSER_CHROME_HEIGHT}px of chrome the window actually has; the bottom ` +
`${pyStr(BROWSER_CHROME_HEIGHT - chrome)}px of the claimed viewport cannot receive input`
);
}
return null;
}
function repairWindowChrome(config: Config, _targetOs: string): void {
const inner = config["window.innerHeight"];
const outer = config["window.outerHeight"];
if (!pyTruthy(inner) || !pyTruthy(outer)) return;
const avail = pyTruthy(config["screen.availHeight"])
? config["screen.availHeight"]
: config["screen.height"];
// Prefer growing the window, which keeps the viewport the identity drew.
if (!pyTruthy(avail) || num(inner) + BROWSER_CHROME_HEIGHT <= num(avail)) {
config["window.outerHeight"] = num(inner) + BROWSER_CHROME_HEIGHT;
return;
}
// No room on the claimed screen: shrink the viewport instead.
config["window.innerHeight"] = Math.max(
num(outer) - BROWSER_CHROME_HEIGHT,
1,
);
}
function checkScreenShape(config: Config, _targetOs: string): string | null {
const width = config["screen.width"];
const height = config["screen.height"];
if (!pyTruthy(width) || !pyTruthy(height)) return null;
if (num(height) > num(width)) {
return `portrait screen ${pyStr(width)}x${pyStr(height)}; desktop panels are landscape`;
}
if (num(width) < 1024) {
return `screen ${pyStr(width)}x${pyStr(height)} is smaller than any current desktop panel`;
}
return null;
}
function checkAvailBounds(config: Config, _targetOs: string): string | null {
const width = config["screen.width"];
const height = config["screen.height"];
const availW = config["screen.availWidth"];
const availH = config["screen.availHeight"];
if (pyTruthy(width) && pyTruthy(availW) && num(availW) > num(width)) {
return `screen.availWidth ${pyStr(availW)} exceeds screen.width ${pyStr(width)}`;
}
if (pyTruthy(height) && pyTruthy(availH) && num(availH) > num(height)) {
return `screen.availHeight ${pyStr(availH)} exceeds screen.height ${pyStr(height)}`;
}
return null;
}
function repairAvailBounds(config: Config, _targetOs: string): void {
const width = config["screen.width"];
const height = config["screen.height"];
const availW =
"screen.availWidth" in config ? config["screen.availWidth"] : 0;
const availH =
"screen.availHeight" in config ? config["screen.availHeight"] : 0;
if (pyTruthy(width) && num(availW) > num(width))
config["screen.availWidth"] = width;
if (pyTruthy(height) && num(availH) > num(height))
config["screen.availHeight"] = height;
}
function checkArchAgreement(config: Config, _targetOs: string): string | null {
const str = (key: string) =>
pyTruthy(config[key]) ? pyStr(config[key]) : "";
const ua = str("navigator.userAgent");
const platform = str("navigator.platform");
const oscpu = str("navigator.oscpu");
if (!ua) return null;
if (
ua.includes("x86_64") &&
(platform.includes("armv") || oscpu.includes("armv"))
) {
return `user agent claims x86_64 while platform/oscpu say ${pyRepr(platform)}/${pyRepr(oscpu)}`;
}
return null;
}
export const RULES: readonly Rule[] = [
{
name: "apple-silicon-cores",
check: checkAppleSiliconCores,
repair: repairAppleSiliconCores,
},
{ name: "gpu-matches-os", check: checkGpuMatchesOs, repair: null },
{ name: "color-depth", check: checkColorDepth, repair: repairColorDepth },
{ name: "touch-points", check: checkTouchPoints, repair: repairTouchPoints },
{
name: "device-pixel-ratio",
check: checkDevicePixelRatio,
repair: repairDevicePixelRatio,
},
{
name: "window-chrome",
check: checkWindowChrome,
repair: repairWindowChrome,
},
{ name: "screen-shape", check: checkScreenShape, repair: null },
{ name: "avail-bounds", check: checkAvailBounds, repair: repairAvailBounds },
{ name: "arch-agreement", check: checkArchAgreement, repair: null },
];
/** Whether the screen is one no desktop reports (portrait, or tiny). */
export function screenIsImplausible(config: Config): boolean {
return checkScreenShape(config, "") !== null;
}
/**
* Turn a portrait screen landscape, keeping the panel's own dimensions. Run
* before the window clamps, which then bound the window to the new screen.
*/
export function repairScreenOrientation(config: Config): boolean {
const width = config["screen.width"];
const height = config["screen.height"];
if (!pyTruthy(width) || !pyTruthy(height) || num(height) <= num(width))
return false;
config["screen.width"] = height;
config["screen.height"] = width;
const availW = config["screen.availWidth"];
const availH = config["screen.availHeight"];
if (pyTruthy(availW) && pyTruthy(availH)) {
config["screen.availWidth"] = availH;
config["screen.availHeight"] = availW;
}
return true;
}
/**
* Discard values a source supplied that this identity cannot keep (a preset's
* GPU pair its OS cannot report), so the normal WebGL sampling draws a
* coherent one instead. Only values another pool can replace are dropped.
*/
export function dropIncoherentSourceValues(
config: Config,
targetOs: string,
): Violation[] {
const dropped: Violation[] = [];
const r = renderer(config);
if (r && !gpuFitsOs(r, targetOs)) {
delete config["webGl:renderer"];
delete config["webGl:vendor"];
dropped.push({
rule: "gpu-matches-os",
detail: `dropped ${pyRepr(r)} for a ${targetOs} identity`,
});
}
return dropped;
}
/** Every invariant this identity breaks. Empty means coherent. */
export function validate(config: Config, targetOs: string): Violation[] {
const violations: Violation[] = [];
for (const rule of RULES) {
const detail = rule.check(config, targetOs);
if (detail) violations.push({ rule: rule.name, detail });
}
return violations;
}
/**
* Repair what is determined, and report what is left. A rule with no repair
* cannot be corrected without inventing a machine, so it is returned for the
* caller to decide about.
*/
export function apply(config: Config, targetOs: string): Violation[] {
for (const rule of RULES) {
if (rule.repair && rule.check(config, targetOs))
rule.repair(config, targetOs);
}
return validate(config, targetOs);
}
typescript/src/cpu_affinity.ts
+235
View File
@@ -0,0 +1,235 @@
/**
* Pin the browser to as many CPU cores as the identity reports.
*
* TypeScript twin of pythonlib/camoufox/cpu_affinity.py.
*
* navigator.hardwareConcurrency is spoofed by the browser, but the number of
* cores a page can *measure* (timing N parallel workers) is the number the OS
* lets the browser run on. Reporting the fingerprint's value and pinning the
* browser's CPU affinity to that many cores makes the two agree, so the drawn
* value survives instead of being replaced by the host count.
*
* Python pins the Playwright driver (a separate Node process) right before the
* launch; here the driver IS this process -- playwright-core spawns the browser
* from the main thread -- so the pin is applied to this process's main thread
* and lifted again afterwards. Child processes inherit the affinity mask on
* Linux and Windows, so the browser and every content/GPU process it spawns
* run on the pinned set. macOS has no process affinity API, so nothing can be
* pinned there and the launcher falls back to reporting the host's (snapped)
* count.
*
* Node has no sched_setaffinity binding, so Linux goes through util-linux's
* `taskset` (which, like os.sched_setaffinity(pid), sets the thread whose TID
* is `pid` -- the main thread) and Windows through PowerShell's
* Process.ProcessorAffinity.
*/
import { execFileSync } from "node:child_process";
import * as fs from "node:fs";
import * as os from "node:os";
function run(command: string, args: string[]): string | null {
try {
return execFileSync(command, args, {
encoding: "utf-8",
stdio: ["ignore", "pipe", "ignore"],
timeout: 10_000,
windowsHide: true,
});
} catch {
return null;
}
}
let tasksetAvailable: boolean | undefined;
/** Whether this host can constrain a process to a subset of its cores. */
export function supported(): boolean {
if (process.platform === "linux") {
// Python checks hasattr(os, 'sched_setaffinity'), which is always true on
// Linux. The equivalent capability here is the taskset binary.
tasksetAvailable ??= run("taskset", ["-V"]) !== null;
return tasksetAvailable;
}
return process.platform === "win32";
}
/** Parse a Linux cpu list ("0-3,8,10-11") into sorted core numbers. */
export function parseCpuList(list: string): number[] {
const cores = new Set<number>();
for (const part of list.trim().split(",")) {
if (!part) continue;
const [lo, hi] = part.split("-").map((n) => Number.parseInt(n, 10));
if (Number.isNaN(lo)) continue;
const top = hi === undefined || Number.isNaN(hi) ? lo : hi;
for (let c = lo; c <= top; c++) {
cores.add(c);
}
}
return [...cores].sort((a, b) => a - b);
}
function linuxGetAffinity(pid: number): number[] | null {
try {
const status = fs.readFileSync(`/proc/${pid}/status`, "utf-8");
const match = status.match(/^Cpus_allowed_list:\s*(.+)$/m);
if (match) {
const cores = parseCpuList(match[1]);
if (cores.length) return cores;
}
} catch {
// fall through
}
return null;
}
function linuxSetAffinity(pid: number, cores: Iterable<number>): boolean {
const list = [...cores].sort((a, b) => a - b).join(",");
return run("taskset", ["-p", "-c", list, String(pid)]) !== null;
}
/** The cores this process may run on, in order. */
export function hostCores(): number[] | null {
if (process.platform === "linux") {
const cores = linuxGetAffinity(process.pid);
if (cores) return cores;
}
if (process.platform === "win32") {
const mask = winGetMask(process.pid);
if (mask) return maskToCores(mask);
}
const n = os.cpus().length;
return n ? Array.from({ length: n }, (_, i) => i) : null;
}
let hostCountSnapshot: number | undefined;
/**
* How many cores the host lets this process use, as read before this process
* first pinned itself. playwright-core spawns the browser from this process,
* so pin() narrows this process's own mask during a launch; a live reading
* taken then by a concurrent launch (hostCores(), or Node's
* availableParallelism()) would see the pinned count and pass it off as the
* host's. Python pins a separate driver process and never has this problem.
*/
export function hostCoreCount(): number {
hostCountSnapshot ??=
hostCores()?.length || os.availableParallelism() || os.cpus().length;
return hostCountSnapshot;
}
/**
* `count` adjacent cores from a random starting point (wrapping). Always
* taking the first `count` stacked every browser on one host onto cores
* 0..count-1, so concurrent browsers measured far less parallelism than they
* report; adjacent cores keep the SMT topology a real machine of that size
* would have.
*/
export function pick(cores: readonly number[], count: number): number[] {
const start = Math.floor(Math.random() * cores.length);
return [...cores.slice(start), ...cores.slice(0, start)]
.slice(0, count)
.sort((a, b) => a - b);
}
/**
* Restrict `pid` to `count` of its cores. Returns the previous set so it can
* be handed back to `restore()`, or null if nothing was changed.
*
* The caller must not pin the same process for two launches at once: the
* browser inherits whatever mask the driver has when it is spawned.
*/
export function pin(pid: number, count: number): number[] | null {
if (count < 1 || !supported()) return null;
if (pid === process.pid) hostCoreCount(); // read the host before narrowing it
if (process.platform === "linux") {
const before = linuxGetAffinity(pid);
if (!before || count >= before.length) return null;
return linuxSetAffinity(pid, pick(before, count)) ? before : null;
}
if (process.platform === "win32") {
const beforeMask = winGetMask(pid);
if (!beforeMask) return null;
const before = maskToCores(beforeMask);
if (count >= before.length) return null;
return winSetMask(pid, coresToMask(pick(before, count))) ? before : null;
}
return null;
}
/** Give `pid` back the cores it had before `pin()`. */
export function restore(pid: number, previous: readonly number[] | null): void {
if (!previous?.length) return;
if (process.platform === "linux") {
linuxSetAffinity(pid, previous);
} else if (process.platform === "win32") {
winSetMask(pid, coresToMask(previous));
}
}
// -- Windows ---------------------------------------------------------------
export function maskToCores(mask: bigint): number[] {
const cores: number[] = [];
for (let i = 0; mask >> BigInt(i) > 0n; i++) {
if ((mask >> BigInt(i)) & 1n) cores.push(i);
}
return cores;
}
export function coresToMask(cores: Iterable<number>): bigint {
let mask = 0n;
for (const c of cores) mask |= 1n << BigInt(c);
return mask;
}
function winGetMask(pid: number): bigint {
const out = run("powershell", [
"-NoProfile",
"-Command",
`[int64](Get-Process -Id ${pid}).ProcessorAffinity`,
]);
if (!out) return 0n;
try {
return BigInt(out.trim());
} catch {
return 0n;
}
}
function winSetMask(pid: number, mask: bigint): boolean {
return (
run("powershell", [
"-NoProfile",
"-Command",
`(Get-Process -Id ${pid}).ProcessorAffinity = [IntPtr][int64]${mask}`,
]) !== null
);
}
// -- launch serialisation --------------------------------------------------
let pinChain: Promise<unknown> = Promise.resolve();
let everPinned = false;
/**
* Run `fn` while holding the process-wide pin lock. The browser inherits the
* driver's mask at spawn, so two concurrent launches must not interleave
* pin/restore: the second pin would land on the first browser, and the first
* restore would leave the driver pinned. (Python: async_api._pin_lock.)
*/
export function withPinLock<T>(fn: () => Promise<T>): Promise<T> {
everPinned = true;
const result = pinChain.then(fn, fn);
pinChain = result.catch(() => undefined);
return result;
}
/**
* Run an UNPINNED launch. It spawns a browser too, which inherits whatever
* mask this process has at that moment, so once any launch in this process
* has pinned, it waits for the lock rather than spawning into another
* launch's pin. Until then (pin_cpu_cores is opt-in) it runs straight away.
*/
export function withUnpinnedLaunch<T>(fn: () => Promise<T>): Promise<T> {
return everPinned ? withPinLock(fn) : fn();
}
typescript/src/display.ts
+151
View File
@@ -0,0 +1,151 @@
/**
* Host display geometry, in the units Firefox lays its windows out in.
*
* TypeScript twin of pythonlib/camoufox/display.py.
*
* Firefox sizes windows in **CSS pixels**. Python uses `screeninfo`, which
* marks the process per-monitor DPI aware and therefore reports **physical**
* pixels; where Windows display scaling is enabled the two differ by the scale
* factor (a 1920x1080 panel at 150% is only 1280x720 CSS px), so deriving a
* window size from the physical numbers opens it partly off-screen
* (daijro/camoufox#425).
*
* There is no dependency-free `screeninfo` equivalent on npm, so the probes
* below shell out per platform and are deliberately best-effort: any failure
* returns null, exactly as the Python twin does when enumeration fails, and
* the caller simply skips the screen constraint.
*
* X11 (xrandr) and macOS already report CSS pixels; only Windows needs the
* scale correction, and there the DPI-aware value is what PowerShell reports.
*/
import { execFileSync } from "node:child_process";
import { OS_NAME } from "./pkgman.js";
/** Size of a monitor in CSS pixels. */
export interface DisplaySize {
width: number;
height: number;
}
/**
* Whether the host has a desktop session for Camoufox's window to open on.
*
* DISPLAY / WAYLAND_DISPLAY only ever exist on Linux, so they cannot be the
* sole probe: keying off DISPLAY alone skips the screen constraints entirely on
* Windows and macOS, where a session is always present.
*/
export function hasDisplay(
env: Record<string, string | number | boolean | undefined>,
): boolean {
if (OS_NAME !== "lin") {
return true;
}
return Boolean(env.DISPLAY || env.WAYLAND_DISPLAY);
}
function run(command: string, args: string[]): string | null {
try {
return execFileSync(command, args, {
encoding: "utf-8",
stdio: ["ignore", "pipe", "ignore"],
timeout: 5000,
});
} catch {
return null;
}
}
/** Every connected monitor's resolution, or [] when it can't be probed. */
function enumerateMonitors(): DisplaySize[] {
if (OS_NAME === "lin") return enumerateLinux();
if (OS_NAME === "mac") return enumerateMac();
return enumerateWindows();
}
function enumerateLinux(): DisplaySize[] {
// `xrandr --current` avoids a mode probe and is safe to call repeatedly.
// Connected outputs carry a "<w>x<h>+<x>+<y>" geometry token.
const out = run("xrandr", ["--current"]);
if (!out) return [];
const monitors: DisplaySize[] = [];
for (const line of out.split("\n")) {
if (!/\bconnected\b/.test(line)) continue;
const match = line.match(/\b(\d+)x(\d+)\+\d+\+\d+/);
if (!match) continue;
monitors.push({
width: Number.parseInt(match[1], 10),
height: Number.parseInt(match[2], 10),
});
}
return monitors;
}
function enumerateMac(): DisplaySize[] {
const out = run("system_profiler", ["-json", "SPDisplaysDataType"]);
if (!out) return [];
try {
const data = JSON.parse(out);
const monitors: DisplaySize[] = [];
for (const gpu of data.SPDisplaysDataType ?? []) {
for (const display of gpu.spdisplays_ndrvs ?? []) {
// e.g. "2560 x 1440" or "2560 x 1440 @ 60.00Hz"
const raw: string =
display._spdisplays_resolution ?? display.spdisplays_resolution ?? "";
const match = raw.match(/(\d+)\s*x\s*(\d+)/);
if (!match) continue;
monitors.push({
width: Number.parseInt(match[1], 10),
height: Number.parseInt(match[2], 10),
});
}
}
return monitors;
} catch {
return [];
}
}
function enumerateWindows(): DisplaySize[] {
// Screen.AllScreens reports DPI-*unaware* bounds for a non-manifested
// process, which is exactly the CSS-pixel figure Firefox lays out in --
// so unlike the Python twin no scale-factor correction is needed here.
const script =
"Add-Type -AssemblyName System.Windows.Forms; " +
"[System.Windows.Forms.Screen]::AllScreens | " +
'ForEach-Object { "$($_.Bounds.Width)x$($_.Bounds.Height)" }';
const out = run("powershell", ["-NoProfile", "-Command", script]);
if (!out) return [];
const monitors: DisplaySize[] = [];
for (const line of out.split("\n")) {
const match = line.trim().match(/^(\d+)x(\d+)$/);
if (!match) continue;
monitors.push({
width: Number.parseInt(match[1], 10),
height: Number.parseInt(match[2], 10),
});
}
return monitors;
}
/**
* Size of the roomiest attached monitor in CSS pixels, or null when the display
* cannot be probed (no monitors, or enumeration failed).
*/
export function largestDisplay(): DisplaySize | null {
let monitors: DisplaySize[];
try {
monitors = enumerateMonitors();
} catch {
return null;
}
if (!monitors.length) return null;
// max() keeps the FIRST of equally large monitors, as Python's max() does.
const monitor = monitors.reduce((prev, curr) =>
curr.width * curr.height > prev.width * prev.height ? curr : prev,
);
return {
width: Math.max(1, Math.trunc(monitor.width)),
height: Math.max(1, Math.trunc(monitor.height)),
};
}
typescript/src/exceptions.ts
+194
View File
@@ -0,0 +1,194 @@
/**
* TypeScript twin of python/src/exceptions.py.
*
* The Python hierarchy leans on builtin bases (FileNotFoundError, ValueError,
* ImportError) that have no JS analogue; those become plain Error subclasses
* here. Every relationship that callers actually catch on -- LocaleError,
* VirtualDisplayError -- is preserved.
*/
/** The Python twin relies on the builtin FileNotFoundError; JS has no such
* class, so version-lookup misses raise this instead. */
export class FileNotFoundError extends Error {
constructor(message?: string) {
super(message ?? "File couldn't be found.");
this.name = "FileNotFoundError";
}
}
export class UnsupportedVersion extends Error {
constructor(message?: string) {
super(message ?? "The Camoufox executable is outdated.");
this.name = "UnsupportedVersion";
}
}
export class MissingRelease extends Error {
constructor(message?: string) {
super(message ?? "A required GitHub release asset is missing.");
this.name = "MissingRelease";
}
}
/** Raised when a downloaded asset does not match its expected sha256 digest. */
export class CorruptedDownload extends Error {
constructor(message?: string) {
super(
message ??
"A downloaded asset does not match its expected sha256 digest.",
);
this.name = "CorruptedDownload";
}
}
export class UnsupportedArchitecture extends Error {
constructor(message?: string) {
super(message ?? "The architecture is not supported.");
this.name = "UnsupportedArchitecture";
}
}
export class UnsupportedOS extends Error {
constructor(message?: string) {
super(message ?? "The OS is not supported.");
this.name = "UnsupportedOS";
}
}
export class InvalidPropertyType extends Error {
constructor(message?: string) {
super(message ?? "The property type is invalid.");
this.name = "InvalidPropertyType";
}
}
export class InvalidAddonPath extends FileNotFoundError {
constructor(message?: string) {
super(message ?? "The addon path is invalid.");
this.name = "InvalidAddonPath";
}
}
export class LocaleError extends Error {
constructor(message?: string) {
super(message ?? "The locale is invalid.");
this.name = "LocaleError";
}
}
export class InvalidIP extends Error {
constructor(message?: string, options?: ErrorOptions) {
super(message ?? "An IP address is invalid.", options);
this.name = "InvalidIP";
}
}
export class InvalidProxy extends Error {
constructor(message?: string, options?: ErrorOptions) {
super(message ?? "A proxy is invalid.", options);
this.name = "InvalidProxy";
}
}
export class UnknownIPLocation extends LocaleError {
constructor(message?: string) {
super(message ?? "The location of an IP is unknown.");
this.name = "UnknownIPLocation";
}
}
export class InvalidLocale extends LocaleError {
constructor(message?: string) {
super(message ?? "The locale input is invalid.");
this.name = "InvalidLocale";
}
static invalidInput(locale: string): InvalidLocale {
return new InvalidLocale(
`Invalid locale: '${locale}'. Must be either a region, language, language-region, or language-script-region.`,
);
}
}
export class UnknownTerritory extends InvalidLocale {
constructor(message?: string) {
super(message ?? "The territory is unknown.");
this.name = "UnknownTerritory";
}
}
export class UnknownLanguage extends InvalidLocale {
constructor(message?: string) {
super(message ?? "The language is unknown.");
this.name = "UnknownLanguage";
}
}
export class NotInstalledGeoIPExtra extends Error {
constructor(message?: string) {
super(message ?? "The GeoIP database reader is not available.");
this.name = "NotInstalledGeoIPExtra";
}
}
export class NonFirefoxFingerprint extends Error {
constructor(message?: string) {
super(message ?? "A passed fingerprint is not a Firefox fingerprint.");
this.name = "NonFirefoxFingerprint";
}
}
export class InvalidOS extends Error {
constructor(message?: string) {
super(message ?? "The target OS is invalid.");
this.name = "InvalidOS";
}
}
export class VirtualDisplayError extends Error {
constructor(message?: string) {
super(message ?? "There is an error with the virtual display.");
this.name = "VirtualDisplayError";
}
}
export class CannotFindXvfb extends VirtualDisplayError {
constructor(message?: string) {
super(message ?? "Xvfb cannot be found.");
this.name = "CannotFindXvfb";
}
}
export class CannotExecuteXvfb extends VirtualDisplayError {
constructor(message?: string) {
super(message ?? "Xvfb cannot be executed.");
this.name = "CannotExecuteXvfb";
}
}
export class VirtualDisplayNotSupported extends VirtualDisplayError {
constructor(message?: string) {
super(
message ?? "The user tried to use a virtual display on a non-Linux OS.",
);
this.name = "VirtualDisplayNotSupported";
}
}
export class CamoufoxNotInstalled extends FileNotFoundError {
constructor(message?: string) {
super(message ?? "Camoufox is not installed.");
this.name = "CamoufoxNotInstalled";
}
}
/** Raised when Camoufox's required runtime directory cannot be prepared. */
export class ProfileDirectoryError extends Error {
constructor(message?: string, options?: ErrorOptions) {
super(
message ?? "Camoufox's runtime directory could not be prepared.",
options,
);
this.name = "ProfileDirectoryError";
}
}
typescript/src/fingerprints.ts
+2076
View File
File diff suppressed because it is too large Load Diff
typescript/src/fpgen/NOTICE
+42
View File
@@ -0,0 +1,42 @@
Camoufox fpgen (TypeScript)
===========================
The files in this directory are a TypeScript port of fpgen 1.3.0
("fingerprint-generator", https://github.com/scrapfly/fingerprint-generator),
the browser fingerprint generator by daijro and Scrapfly.
fpgen is licensed under the Apache License, Version 2.0. This port is a
Derivative Work of it and is distributed under the same license:
Licensed under the Apache License, Version 2.0 (the "License");
you may not use these files except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
The upstream project ships no NOTICE file of its own.
Changes made in the port (Apache-2.0 section 4(b)):
- Translated from Python to TypeScript. The generator, Bayesian network
(beam-search trace), condition handling, query and trace follow the
Python line for line; see the header of each file for any deliberate
deviation.
- The model is NOT fetched the way fpgen fetches it (unverified TLS, no
checksum, first-listed GitHub release). It is downloaded from the release
pinned in Camoufox's scripts/data/fpgen-model.json with TLS verification
on, and its sha256 is checked before anything is extracted (model.ts).
- values.dat is decompressed once with Node's built-in zstd and cached on
disk, instead of being read through indexed_zstd.
- The deprecated get_shared_possibilities / get_distribution_for_node
helpers and the `python -m fpgen` CLI are not ported.
The fingerprint model data itself (fingerprint-network.json, values.json,
values.dat) is not part of this directory; it is downloaded at run time from
the upstream release and remains under its upstream terms.
typescript/src/fpgen/bayesian-network.ts
+389
View File
@@ -0,0 +1,389 @@
/**
* Ported from fpgen/bayesian_network.py (scrapfly/fingerprint-generator,
* Apache-2.0; see ./NOTICE).
*
* The inference is a line-for-line port: the same beam search, the same
* BEAM_WIDTH, the same multiplication order, the same stable top-k pruning.
* `trace()` therefore returns bit-identical probabilities to Python (the tests
* compare them). Only the draws differ, because Python uses `random.random()`
* and this uses `Math.random()`.
*
* Kept on purpose, because they change results:
* - probability tables are Maps in document order (see pyjson.ts);
* - validate_evidence passes a single *string* as the allowed values, and
* Python's `value in "abc"` is a substring test. `isAllowed` reproduces it,
* so the same condition sets are rejected in both languages.
*
* Not ported: get_distribution_for_node, get_shared_possibilities,
* _intersect_parents and collect_parents. Nothing in fpgen calls them (the
* docstring marks get_shared_possibilities deprecated since 1.3.0).
*/
import { RestrictiveConstraints } from "./exceptions.js";
import { casefold } from "./pyjson.js";
/** Width for beam search. Cuts off values that are way too low or contaminated. */
export const BEAM_WIDTH = 1000;
/** A conditional probability table: parent value -> ... -> value -> p. */
export type CPT = Map<string, CPT | number>;
/** A distribution over value ids, in insertion order (Python dict order). */
export type Distribution = Map<string, number>;
/** Allowed value ids for a node. A bare string is Python's `str` (substring `in`). */
export type Allowed = ReadonlySet<string> | string;
export type Evidence = ReadonlyMap<string, Allowed>;
/** Anything that can turn value ids into their stored JSON text. */
export interface ValueLookup {
lookupValueList(indexList: Iterable<string>): string[];
}
function isAllowed(allowed: Allowed, value: string): boolean {
return typeof allowed === "string"
? allowed.includes(value)
: allowed.has(value);
}
function sumValues(dist: Distribution): number {
let total = 0;
for (const p of dist.values()) total += p;
return total;
}
/** A single node in the network, with its conditional probability table. */
export class BayesianNode {
readonly name: string;
readonly parentNames: readonly string[];
readonly possibleValues: readonly string[];
readonly probabilities: CPT;
constructor(
readonly nodeDefinition: Map<string, unknown>,
readonly index: number,
) {
this.name = nodeDefinition.get("name") as string;
this.parentNames = nodeDefinition.get("parentNames") as string[];
this.possibleValues = nodeDefinition.get("possibleValues") as string[];
this.probabilities = nodeDefinition.get("conditionalProbabilities") as CPT;
}
/** This node's value probabilities given its parents' values. */
getProbabilitiesGivenKnownValues(
parentValues: ReadonlyMap<string, string>,
): Distribution {
let probabilities: CPT = this.probabilities;
for (const parentName of this.parentNames) {
const parentValue = parentValues.get(parentName) as string;
const next = probabilities.get(parentValue);
probabilities = next instanceof Map ? next : new Map();
}
return probabilities as Distribution;
}
}
/** Map with casefolded string keys (structs.CaseInsensitiveDict). */
export class CaseInsensitiveMap<V> extends Map<string, V> {
override get(key: string): V | undefined {
return super.get(casefold(key));
}
override set(key: string, value: V): this {
return super.set(casefold(key), value);
}
override has(key: string): boolean {
return super.has(casefold(key));
}
override delete(key: string): boolean {
return super.delete(casefold(key));
}
}
interface BeamEntry {
/** Assigned value ids, aligned with the trace's ordered node list. */
readonly values: string[];
readonly prob: number;
}
export class BayesianNetwork {
readonly nodesInSamplingOrder: BayesianNode[];
readonly nodesByName: CaseInsensitiveMap<BayesianNode>;
/** The original (cased) node names, in sampling order. */
readonly nodeNames: readonly string[];
readonly ancestorsByName = new Map<string, Set<string>>();
constructor(
networkDefinition: unknown,
readonly values: ValueLookup,
) {
const nodes = (networkDefinition as Map<string, unknown>).get(
"nodes",
) as Map<string, unknown>[];
this.nodesInSamplingOrder = nodes.map(
(def, index) => new BayesianNode(def, index),
);
// dict comprehension: a later duplicate name replaces the node but keeps
// the first one's position.
const byName = new Map<string, BayesianNode>();
for (const node of this.nodesInSamplingOrder) byName.set(node.name, node);
this.nodesByName = new CaseInsensitiveMap<BayesianNode>();
for (const [name, node] of byName) this.nodesByName.set(name, node);
this.nodeNames = [...byName.keys()];
for (const node of this.nodesInSamplingOrder) {
this.getAllAncestors(node.name);
}
}
/** Look up a node by (case-insensitive) name; KeyError -> Error. */
node(name: string): BayesianNode {
const node = this.nodesByName.get(name);
if (!node) throw new Error(`KeyError: '${casefold(name)}'`);
return node;
}
/** Generate a full sample from the network. */
generateConsistentSample(
evidence: ReadonlyMap<string, ReadonlySet<string>>,
): Map<string, string> | null {
const result = new Map<string, string>();
// A working copy of the evidence, updated in place.
const currentEvidence = new Map<string, Allowed>();
for (const [k, v] of evidence) currentEvidence.set(k, new Set(v));
for (const node of this.nodesInSamplingOrder) {
const nodeName = node.name;
let sampledValue: string;
const allowedValues = currentEvidence.get(nodeName);
if (allowedValues !== undefined) {
// Explicit evidence: leave the node itself out of the beam search.
const searchEvidence = new Map(currentEvidence);
searchEvidence.delete(nodeName);
const distribution = this.trace(nodeName, searchEvidence);
// Filter the distribution to allowed values and renormalize.
let filtered: Distribution = new Map();
for (const [k, v] of distribution) {
if (isAllowed(allowedValues, k)) filtered.set(k, v);
}
if (filtered.size === 0 || sumValues(filtered) <= 0) {
// Only ever a Set here: it came from build_evidence or from
// a sampled value.
const allowed = allowedValues as ReadonlySet<string>;
const uniform = 1.0 / allowed.size;
filtered = new Map([...allowed].map((v) => [v, uniform]));
} else {
const total = sumValues(filtered);
for (const [k, v] of filtered) filtered.set(k, v / total);
}
sampledValue = this.sampleValueFromDistribution(filtered);
} else {
// Unconstrained node: use all current evidence.
const distribution = this.trace(nodeName, currentEvidence);
sampledValue = this.sampleValueFromDistribution(distribution);
}
result.set(nodeName, sampledValue);
// Update current evidence with the newly sampled value.
currentEvidence.set(nodeName, new Set([sampledValue]));
}
return result;
}
/** Generate values for target nodes given conditions. */
generateCertainNodes(
evidence: ReadonlyMap<string, ReadonlySet<string>>,
targets?: Iterable<string> | null,
): Map<string, string> | null {
// If no target specified, generate full sample
if (targets == null) return this.generateConsistentSample(evidence);
const result = new Map<string, string>();
for (const targetNode of targets) {
let distribution = this.trace(targetNode, evidence);
// Handle multi-value conditions for the target
const allowedValues = evidence.get(targetNode);
if (allowedValues !== undefined) {
const filtered: Distribution = new Map();
for (const [k, v] of distribution) {
if (allowedValues.has(k)) filtered.set(k, v);
}
if (filtered.size === 0 || sumValues(filtered) <= 0) {
throw new RestrictiveConstraints(
`Cannot generate fingerprint: No valid values for ${targetNode} with current conditions.`,
);
}
const total = sumValues(filtered);
for (const [k, v] of filtered) filtered.set(k, v / total);
distribution = filtered;
}
if (distribution.size) {
result.set(targetNode, this.sampleValueFromDistribution(distribution));
} else {
throw new RestrictiveConstraints(
`Cannot generate fingerprint: Empty distribution for ${targetNode}.`,
);
}
}
return result;
}
/**
* Validate that the evidence is mutually compatible given the network
* structure. Throws RestrictiveConstraints when it isn't.
*/
validateEvidence(evidence: ReadonlyMap<string, ReadonlySet<string>>): void {
// Skip validation for single constraint
if (evidence.size <= 1) return;
for (const [nodeName, allowedValues] of evidence) {
// The other conditions pinned to a single value.
const fixed = new Map<string, string>();
for (const [k, v] of evidence) {
if (k !== nodeName && v.size === 1) fixed.set(k, [...v][0]);
}
if (!fixed.size) continue;
const dist = this.trace(nodeName, fixed);
if (
dist.size &&
[...allowedValues].every((val) => (dist.get(val) ?? 0) <= 0)
) {
const allowed = [...allowedValues];
let valuesStr = this.values
.lookupValueList(allowed.slice(0, 5))
.join(", ");
if (allowed.length > 5) valuesStr += ", ...";
const constraintValues = this.values.lookupValueList(fixed.values());
const constraintsStr = [...fixed.keys()]
.map((k, i) => `${k}=${constraintValues[i]}`)
.join(", ");
throw new RestrictiveConstraints(
`Cannot generate fingerprint: ${nodeName}=(${valuesStr}) ` +
`is impossible with constraint: ${constraintsStr}`,
);
}
}
}
/** All ancestors of a node (the nodes that can influence its value). */
getAllAncestors(nodeName: string): Set<string> {
const cached = this.ancestorsByName.get(nodeName);
if (cached) return cached;
const node = this.node(nodeName);
const ancestors = new Set<string>();
for (const parent of node.parentNames) {
ancestors.add(parent);
for (const a of this.getAllAncestors(parent)) ancestors.add(a);
}
this.ancestorsByName.set(nodeName, ancestors);
return ancestors;
}
/**
* The conditional distribution of `target` given `evidence`, by beam
* search. Empty when the evidence admits nothing.
*/
trace(target: string, evidence: Evidence): Distribution {
// The actual target name, and the nodes that matter for it.
const targetName = this.node(target).name;
const relevant = new Set(this.getAllAncestors(targetName));
relevant.add(targetName);
for (const evNode of evidence.keys()) {
if (this.nodesByName.has(evNode)) {
relevant.add(evNode);
for (const a of this.getAllAncestors(evNode)) relevant.add(a);
}
}
// Relevant nodes in sampling order, with each parent's slot.
const ordered = this.nodesInSamplingOrder.filter((n) =>
relevant.has(n.name),
);
const slot = new Map<string, number>();
for (const [i, n] of ordered.entries()) slot.set(n.name, i);
let beam: BeamEntry[] = [{ values: [], prob: 1.0 }];
for (let i = 0; i < ordered.length; i++) {
const node = ordered[i];
const nodeName = node.name;
const allowedValues = evidence.get(nodeName);
const parentSlots = node.parentNames.map((p) => {
const s = slot.get(p);
if (s === undefined) throw new Error(`KeyError: '${p}'`);
return s;
});
const newBeam: BeamEntry[] = [];
// Python keys its cache by (node, parent values); one per node here.
const cptCache = new Map<string, Distribution>();
for (const { values, prob } of beam) {
const parentValues = parentSlots.map((s) => values[s]);
const cacheKey = parentValues.join("\u0000");
let cpt = cptCache.get(cacheKey);
if (cpt === undefined) {
const known = new Map<string, string>();
node.parentNames.forEach((p, j) => {
known.set(p, parentValues[j]);
});
cpt = node.getProbabilitiesGivenKnownValues(known);
// Use a uniform distribution if the table has no row.
if (!cpt.size && node.possibleValues.length) {
const uniform = 1.0 / node.possibleValues.length;
cpt = new Map(node.possibleValues.map((v) => [v, uniform]));
}
cptCache.set(cacheKey, cpt);
}
// Expand the beam with new assignments
for (const [value, p] of cpt) {
if (
(allowedValues === undefined || isAllowed(allowedValues, value)) &&
p > 0
) {
const next = values.slice();
next.push(value);
newBeam.push({ values: next, prob: prob * p });
}
}
}
if (!newBeam.length) return new Map();
if (newBeam.length > BEAM_WIDTH) {
// heapq.nlargest(k, ..., key=prob) == sorted(reverse=True)[:k],
// which keeps equal-probability entries in their original order.
// Array.prototype.sort is stable, so this is the same list.
newBeam.sort((a, b) => b.prob - a.prob);
newBeam.length = BEAM_WIDTH;
}
beam = newBeam;
}
// Extract the target distribution
const targetSlot = slot.get(targetName) as number;
const targetDist: Distribution = new Map();
let totalProb = 0.0;
for (const { values, prob } of beam) {
const value = values[targetSlot];
targetDist.set(value, (targetDist.get(value) ?? 0) + prob);
totalProb += prob;
}
if (totalProb > 0) {
for (const [v, p] of targetDist) targetDist.set(v, p / totalProb);
return targetDist;
}
return new Map();
}
/** Draw a value from a distribution (cumulative, falls back to the first). */
sampleValueFromDistribution(distribution: Distribution): string {
const anchor = Math.random();
let cumulative = 0.0;
for (const [value, probability] of distribution) {
cumulative += probability;
if (anchor < cumulative) return value;
}
return distribution.keys().next().value as string;
}
}
typescript/src/fpgen/exceptions.ts
+80
View File
@@ -0,0 +1,80 @@
/**
* Ported from fpgen/exceptions.py (scrapfly/fingerprint-generator, Apache-2.0).
* See ./NOTICE.
*
* The hierarchy and the class names are kept: the Python launcher tells a
* constraint that matched nothing apart from one that was too restrictive by
* `type(exc).__name__`, so `name` is set to the exact Python class name.
* Python's builtin ValueError has no JS twin; it becomes a plain Error subclass
* with the same name, and NetworkError still derives from it.
*/
export class ValueError extends Error {
constructor(message?: string) {
super(message);
this.name = "ValueError";
}
}
/** Error with the network. */
export class NetworkError extends ValueError {
constructor(message?: string) {
super(message);
this.name = "NetworkError";
}
}
/** Raised when a constraint isn't possible. */
export class InvalidConstraints extends NetworkError {
constructor(message?: string) {
super(message);
this.name = "InvalidConstraints";
}
}
/** Raised when the passed constraints are too restrictive. */
export class RestrictiveConstraints extends InvalidConstraints {
constructor(message?: string) {
super(message);
this.name = "RestrictiveConstraints";
}
}
/** Raised when a node doesn't exist. */
export class InvalidNode extends NetworkError {
constructor(message?: string) {
super(message);
this.name = "InvalidNode";
}
}
/** Raised when a key path doesn't exist. The message is the missing key. */
export class NodePathError extends InvalidNode {
constructor(message?: string) {
super(message);
this.name = "NodePathError";
}
}
/**
* The pinned model is not on disk. Python fpgen downloads at import time; the
* TypeScript API is synchronous, so the download is a separate, awaited step
* (`ensureModel()`), and the synchronous calls throw this until it has run.
*/
export class ModelNotInstalled extends Error {
constructor(message?: string) {
super(
message ??
"The fpgen model is not installed. Call `await ensureModel()` first.",
);
this.name = "ModelNotInstalled";
}
}
/** The downloaded model failed its size / sha256 check, or is malformed. */
export class ModelVerificationError extends Error {
constructor(message?: string) {
super(message);
this.name = "ModelVerificationError";
}
}
typescript/src/fpgen/generator.ts
+192
View File
@@ -0,0 +1,192 @@
/**
* Ported from fpgen/generator.py (scrapfly/fingerprint-generator, Apache-2.0;
* see ./NOTICE).
*
* Python keyword arguments become two objects: the conditions, then the
* options (`strict`, `flatten`, `target`). `Generator(conditions, options)`,
* `generator.generate(conditions, options)`, and so on.
*
* The model must be on disk before any of this runs: `await ensureModel()`.
*/
import { RestrictiveConstraints } from "./exceptions.js";
import { getModel } from "./model.js";
import {
type TraceOptions,
type TraceResult,
type TraceResultDict,
traceWithEvidence,
} from "./trace.js";
import {
assertConditions,
buildEvidence,
type Conditions,
type EvidenceMap,
findRoots,
makeOutputDict,
maybeFlatten,
reassembleTargets,
} from "./utils.js";
export interface GeneratorOptions {
/** Throw when the conditions are too strict (default true). */
strict?: boolean;
/** Flatten the output object into dotted keys (default false). */
flatten?: boolean;
}
export interface GenerateOptions {
/** Overrides the Generator's `strict`. */
strict?: boolean;
/** Overrides the Generator's `flatten`. */
flatten?: boolean;
/** Only generate specific value(s): a node, a path inside one, or a prefix. */
target?: string | readonly string[];
}
/** A generated fingerprint (nested unless `flatten`). */
export type Fingerprint = Record<string, any>;
function hasConditions(conditions: Conditions | null | undefined): boolean {
return !!conditions && Object.keys(conditions).length > 0;
}
/** Generates realistic browser fingerprints. */
export class Generator {
strict: boolean;
flatten: boolean;
readonly evidence: EvidenceMap = new Map();
/**
* Conditions and options given here are inherited by every generate().
*/
constructor(
conditions?: Conditions | null,
{ strict = true, flatten = false }: GeneratorOptions = {},
) {
assertConditions(conditions);
this.strict = strict;
this.flatten = flatten;
if (hasConditions(conditions)) {
// fpgen passes no `strict` here, so construction is always strict.
buildEvidence(conditions as Conditions, this.evidence);
}
}
/** Generate a fingerprint, or just `target` when given. */
generate(
conditions: Conditions | null | undefined,
options: GenerateOptions & { target: string },
): any;
generate(
conditions?: Conditions | null,
options?: GenerateOptions,
): Fingerprint;
generate(conditions?: Conditions | null, options: GenerateOptions = {}): any {
assertConditions(conditions);
const { network } = getModel();
const strict = options.strict ?? this.strict;
const flatten = options.flatten ?? this.flatten;
const { target } = options;
// Inherit the evidence from the instance
const evidence: EvidenceMap = new Map(this.evidence);
if (hasConditions(conditions)) {
buildEvidence(conditions as Conditions, evidence, strict);
}
const targetTup =
target === undefined
? null
: typeof target === "string"
? [target]
: [...target];
const hasTarget = !!target && (targetTup as string[]).length > 0;
const targetRoots = hasTarget
? new Set(findRoots(targetTup as string[]))
: null;
let fingerprint: Map<string, string> | null;
while (true) {
fingerprint = targetRoots?.size
? network.generateCertainNodes(evidence, targetRoots)
: network.generateConsistentSample(evidence);
if (fingerprint !== null) break;
if (strict) {
throw new RestrictiveConstraints(
"Cannot generate fingerprint. Constraints are too restrictive.",
);
}
// Relax the evidence until something can be generated
evidence.delete(evidence.keys().next().value as string);
}
if (hasTarget) {
// Don't flatten yet
let output: any = reassembleTargets(
targetTup as string[],
makeOutputDict(fingerprint, false),
);
if (typeof target === "string") output = output[target];
return maybeFlatten(flatten, output);
}
return makeOutputDict(fingerprint, flatten);
}
/** Generate a specific target. Shortcut for `generate(..., {target})`. */
generateTarget(
target: string,
conditions?: Conditions | null,
options: Omit<GenerateOptions, "target"> = {},
): any {
return this.generate(conditions, { ...options, target });
}
/**
* The probability distribution(s) of a target given conditions, on top of
* this Generator's conditions.
*/
trace(
target: string | readonly string[],
conditions?: Conditions | null,
options: TraceOptions = {},
): TraceResult[] | TraceResultDict {
return traceWithEvidence(
target,
conditions,
options,
new Map(this.evidence),
);
}
}
/*
* A global generate() for callers that don't build a Generator.
*/
let GLOBAL_GENERATOR: Generator | null = null;
/** Generate a fingerprint with a shared, condition-less Generator. */
export function generate(
conditions: Conditions | null | undefined,
options: GenerateOptions & { target: string },
): any;
export function generate(
conditions?: Conditions | null,
options?: GenerateOptions,
): Fingerprint;
export function generate(
conditions?: Conditions | null,
options?: GenerateOptions,
): any {
GLOBAL_GENERATOR ??= new Generator();
return GLOBAL_GENERATOR.generate(conditions, options);
}
/** Generate a specific target. Shortcut for `generate(..., {target})`. */
export function generateTarget(
target: string,
conditions?: Conditions | null,
options: Omit<GenerateOptions, "target"> = {},
): any {
return generate(conditions, { ...options, target });
}
typescript/src/fpgen/index.ts
+58
View File
@@ -0,0 +1,58 @@
/**
* TypeScript port of fpgen 1.3.0 (scrapfly/fingerprint-generator), the
* fingerprint generator the Python launcher uses. Apache-2.0; see ./NOTICE.
*
* import { ensureModel, Generator } from "./fpgen/index.js";
* await ensureModel(); // pinned model, sha256-checked
* const fp = new Generator().generate({ browser: "Firefox", os: "Windows" });
*
* Everything after ensureModel() is synchronous, like the Python API.
*/
export {
BayesianNetwork,
BayesianNode,
BEAM_WIDTH,
type Distribution,
} from "./bayesian-network.js";
export {
InvalidConstraints,
InvalidNode,
ModelNotInstalled,
ModelVerificationError,
NetworkError,
NodePathError,
RestrictiveConstraints,
ValueError,
} from "./exceptions.js";
export {
type Fingerprint,
type GenerateOptions,
Generator,
type GeneratorOptions,
generate,
generateTarget,
} from "./generator.js";
export {
downloadArchive,
type EnsureModelOptions,
ensureModel,
FpgenModel,
getModel,
installArchive,
isModelInstalled,
modelDir,
resetModelCache,
verifyArchive,
} from "./model.js";
export { MODEL_PIN, type ModelPin } from "./pin.js";
export {
type TraceOptions,
TraceResult,
type TraceResultDict,
trace,
} from "./trace.js";
export {
type Conditions,
type Predicate,
query,
} from "./utils.js";
typescript/src/fpgen/model.ts
+423
View File
@@ -0,0 +1,423 @@
/**
* Model files for the fpgen port: where they live, how they get there, and the
* loaded model. Replaces fpgen/pkgman.py and the file half of fpgen/unpacker.py
* (scrapfly/fingerprint-generator, Apache-2.0; see ./NOTICE).
*
* Deliberately NOT like fpgen:
* - the archive comes from the release pinned in MODEL_PIN (the twin of
* scripts/data/fpgen-model.json), never "the first release the API lists";
* - TLS is verified (Node's fetch always does), and the archive's size and
* sha256 are checked before a byte of it is extracted;
* - only the three expected members are written, by name, so a hostile
* archive cannot place files outside the data directory;
* - there is no five-week re-download: the model changes when the pin does.
*
* Layout of the data directory (compatible with scripts/pin-fpgen-model.py, so
* CAMOUFOX_FPGEN_DATA may point at a pinned Python fpgen `data/` directory):
*
* fingerprint-network.json.zst
* values.json.zst
* values.dat.zst
* values.dat decompressed once on first load (~210 MB), then read
* by offset -- fpgen does the same when values.dat exists
* .pinned-model sha256 of the verified archive; written last
*/
import { createHash } from "node:crypto";
import * as fs from "node:fs";
import * as path from "node:path";
import { pipeline } from "node:stream/promises";
import * as zlib from "node:zlib";
import AdmZip from "adm-zip";
import { INSTALL_DIR } from "../paths.js";
import { BayesianNetwork } from "./bayesian-network.js";
import { ModelNotInstalled, ModelVerificationError } from "./exceptions.js";
import { MODEL_PIN, type ModelPin } from "./pin.js";
import { base85ToInt, parseOrdered } from "./pyjson.js";
export const STAMP_FILE = ".pinned-model";
export const NETWORK_ZST = "fingerprint-network.json.zst";
export const VALUES_JSON_ZST = "values.json.zst";
export const VALUES_DAT_ZST = "values.dat.zst";
export const VALUES_DAT = "values.dat";
/**
* The model directory: $CAMOUFOX_FPGEN_DATA when set, otherwise `fpgen/` in
* the Camoufox cache (next to the browsers, addons and fontconfig cache).
*/
export function modelDir(): string {
const override = process.env.CAMOUFOX_FPGEN_DATA;
if (override) return path.resolve(override);
return path.join(INSTALL_DIR, "fpgen");
}
/** True when `dir` holds the pinned model (stamp matches, members present). */
export function isModelInstalled(
dir: string = modelDir(),
pin: ModelPin = MODEL_PIN,
): boolean {
let stamp: string;
try {
stamp = fs.readFileSync(path.join(dir, STAMP_FILE), "utf-8").trim();
} catch {
return false;
}
if (stamp !== pin.sha256) return false;
return pin.files.every((f) => fs.existsSync(path.join(dir, f)));
}
export interface EnsureModelOptions {
/** Directory to install into / load from. Defaults to `modelDir()`. */
dir?: string;
/** Re-download even when the pinned model is already present. */
force?: boolean;
/** Override the fetch implementation (tests). */
fetchImpl?: typeof fetch;
}
const inflight = new Map<string, Promise<string>>();
export const LOCK_DIR = ".install.lock";
/** A lock older than this was left by a process that died holding it. */
export const STALE_LOCK_MS = 10 * 60 * 1000;
/**
* Run `fn` holding `dir`'s install lock, across processes. `inflight` only
* dedupes within one process; several processes installing into an empty cache
* at once (a worker pool on a fresh machine, or vitest's parallel files) would
* otherwise each download the model, and one's install deleted the values.dat
* another had just decompressed and was about to read. mkdir is atomic on every
* platform, so the lock is a directory.
*/
export async function withInstallLock<T>(
dir: string,
fn: () => Promise<T>,
): Promise<T> {
fs.mkdirSync(dir, { recursive: true });
const lock = path.join(dir, LOCK_DIR);
for (;;) {
try {
fs.mkdirSync(lock);
break;
} catch (e) {
if ((e as NodeJS.ErrnoException).code !== "EEXIST") throw e;
try {
if (Date.now() - fs.statSync(lock).mtimeMs > STALE_LOCK_MS) {
fs.rmSync(lock, { recursive: true, force: true });
continue;
}
} catch {
continue; // released between the mkdir and the stat
}
await new Promise((resolve) => setTimeout(resolve, 200));
}
}
try {
return await fn();
} finally {
fs.rmSync(lock, { recursive: true, force: true });
}
}
/**
* Make sure the pinned model is installed and values.dat is decompressed.
* Downloads (TLS on, sha256-checked) only when it is missing. Returns the
* model directory. Safe to call repeatedly and concurrently.
*/
export function ensureModel(options: EnsureModelOptions = {}): Promise<string> {
const dir = path.resolve(options.dir ?? modelDir());
const running = inflight.get(dir);
if (running && !options.force) return running;
const job = (async () => {
// The unlocked check is the fast path for an installed model; the
// locked one decides, since another process may have just installed it.
if (!options.force && isModelInstalled(dir) && datIsReady(dir)) return dir;
await withInstallLock(dir, async () => {
if (options.force || !isModelInstalled(dir)) {
const archive = await downloadArchive(MODEL_PIN, options.fetchImpl);
installArchive(archive, dir, MODEL_PIN);
}
await decompressValuesDat(dir);
});
return dir;
})();
inflight.set(dir, job);
job.catch(() => inflight.delete(dir));
return job;
}
/** Download the pinned archive and verify its size and sha256. */
export async function downloadArchive(
pin: ModelPin = MODEL_PIN,
fetchImpl: typeof fetch = fetch,
retries = 3,
): Promise<Buffer> {
let lastError: unknown;
for (let attempt = 0; attempt < retries; attempt++) {
let response: Response;
try {
response = await fetchImpl(pin.url, {
headers: { "User-Agent": "camoufox-js" },
redirect: "follow",
signal: AbortSignal.timeout(120_000),
});
} catch (e) {
lastError = e;
continue;
}
if (!response.ok) {
lastError = new Error(
`fpgen model download failed: HTTP ${response.status} for ${pin.url}`,
);
// 4xx will not get better by retrying.
if (response.status >= 400 && response.status < 500) break;
continue;
}
const buf = Buffer.from(await response.arrayBuffer());
verifyArchive(buf, pin);
return buf;
}
throw lastError instanceof Error
? lastError
: new Error(`fpgen model download failed: ${String(lastError)}`);
}
/** Throws ModelVerificationError unless `buf` is exactly the pinned archive. */
export function verifyArchive(buf: Buffer, pin: ModelPin = MODEL_PIN): void {
if (pin.size && buf.length !== pin.size) {
throw new ModelVerificationError(
`fpgen model size mismatch: got ${buf.length}, expected ${pin.size}`,
);
}
const got = createHash("sha256").update(buf).digest("hex");
if (got !== pin.sha256) {
throw new ModelVerificationError(
`fpgen model sha256 mismatch:\n got ${got}\n expected ${pin.sha256}`,
);
}
}
/** Write the pinned members of a verified archive into `dir`, stamp last. */
export function installArchive(
archive: Buffer,
dir: string,
pin: ModelPin = MODEL_PIN,
): void {
verifyArchive(archive, pin);
fs.mkdirSync(dir, { recursive: true });
let previous = "";
try {
previous = fs.readFileSync(path.join(dir, STAMP_FILE), "utf-8").trim();
} catch {}
// An interrupted install must not leave a stamp over a partial model.
fs.rmSync(path.join(dir, STAMP_FILE), { force: true });
// A values.dat decompressed from a DIFFERENT model would be trusted (its
// size is all that is checked), so drop it -- but only then: reinstalling
// the same model must not pull the file out from under a process reading it.
if (previous !== pin.sha256) {
fs.rmSync(path.join(dir, VALUES_DAT), { force: true });
}
const zip = new AdmZip(archive);
for (const name of pin.files) {
if (path.basename(name) !== name) {
throw new ModelVerificationError(
`unexpected member path in pin: ${name}`,
);
}
const entry = zip.getEntry(name);
if (!entry || entry.isDirectory) {
throw new ModelVerificationError(
`fpgen model archive is missing ${name}`,
);
}
atomicWrite(path.join(dir, name), entry.getData());
}
fs.writeFileSync(path.join(dir, STAMP_FILE), `${pin.sha256}\n`);
}
function atomicWrite(target: string, data: Uint8Array): void {
const tmp = `${target}.${process.pid}.${Date.now()}.tmp`;
fs.writeFileSync(tmp, data);
fs.renameSync(tmp, target);
}
/** Byte length values.dat must have: the end of the furthest slice. */
function expectedDatSize(valuePairs: readonly ValuePair[]): number {
let end = 0;
for (const [offset, length] of valuePairs) {
if (offset + length > end) end = offset + length;
}
return end;
}
function datIsReady(dir: string): boolean {
try {
return datIsComplete(path.join(dir, VALUES_DAT), readValuePairs(dir));
} catch {
return false;
}
}
function datIsComplete(
datPath: string,
valuePairs: readonly ValuePair[],
): boolean {
try {
return fs.statSync(datPath).size === expectedDatSize(valuePairs);
} catch {
return false;
}
}
/** Stream-decompress values.dat.zst -> values.dat if it isn't there yet. */
async function decompressValuesDat(dir: string): Promise<void> {
const datPath = path.join(dir, VALUES_DAT);
const pairs = readValuePairs(dir);
if (datIsComplete(datPath, pairs)) return;
const tmp = `${datPath}.${process.pid}.${Date.now()}.tmp`;
try {
await pipeline(
fs.createReadStream(path.join(dir, VALUES_DAT_ZST)),
zlib.createZstdDecompress(),
fs.createWriteStream(tmp),
);
fs.renameSync(tmp, datPath);
} finally {
fs.rmSync(tmp, { force: true });
}
if (!datIsComplete(datPath, pairs)) {
throw new ModelVerificationError(
`decompressed ${datPath} does not match values.json`,
);
}
}
function decompressValuesDatSync(
dir: string,
pairs: readonly ValuePair[],
): void {
const datPath = path.join(dir, VALUES_DAT);
if (datIsComplete(datPath, pairs)) return;
const data = zlib.zstdDecompressSync(
fs.readFileSync(path.join(dir, VALUES_DAT_ZST)),
);
atomicWrite(datPath, data);
if (!datIsComplete(datPath, pairs)) {
throw new ModelVerificationError(
`decompressed ${datPath} does not match values.json`,
);
}
}
function readZstJson(file: string): unknown {
return parseOrdered(
zlib.zstdDecompressSync(fs.readFileSync(file)).toString("utf-8"),
);
}
/** [byte offset, byte length] of each value in values.dat, by value id. */
export type ValuePair = readonly [offset: number, length: number];
function readValuePairs(dir: string): ValuePair[] {
const json = readZstJson(path.join(dir, VALUES_JSON_ZST));
if (!(json instanceof Map)) {
throw new ModelVerificationError("values.json is not an object");
}
// VALUE_PAIRS = list(values_json.items()) -- document order is the id.
const pairs: ValuePair[] = [];
for (const [hexOffset, length] of json as Map<string, number>) {
pairs.push([Number.parseInt(hexOffset, 16), length]);
}
return pairs;
}
/**
* The loaded model: the Bayesian network plus the value store it indexes into.
* One per data directory, loaded on first use (fpgen loads it at import).
*/
export class FpgenModel {
readonly network: BayesianNetwork;
readonly valuePairs: readonly ValuePair[];
readonly datPath: string;
/** Held open for the model's lifetime: one open per lookup was a syscall
* per call, and a path reopened each time fails if the file is replaced. */
private fd: number | null = null;
constructor(readonly dir: string) {
this.valuePairs = readValuePairs(dir);
this.datPath = path.join(dir, VALUES_DAT);
decompressValuesDatSync(dir, this.valuePairs);
this.network = new BayesianNetwork(
readZstJson(path.join(dir, NETWORK_ZST)),
this,
);
}
/** unpacker.lookup_value: the raw JSON text stored for a value id. */
lookupValue(index: string): string {
return this.lookupValueList([index])[0];
}
/**
* unpacker.lookup_value_list: raw JSON text for each id, in input order.
* Reads in ascending offset order, as fpgen does.
*/
lookupValueList(indexList: Iterable<string>): string[] {
const ids = [...indexList];
const out = new Array<string>(ids.length);
const sorted = ids
.map((id, n) => [base85ToInt(id), n] as const)
.sort((a, b) => a[0] - b[0] || a[1] - b[1]);
if (this.fd === null) this.fd = fs.openSync(this.datPath, "r");
const fd = this.fd;
for (const [index, n] of sorted) {
const pair = this.valuePairs[index];
if (pair === undefined) {
throw new RangeError(`list index out of range: value id ${index}`);
}
const [offset, length] = pair;
const buf = Buffer.allocUnsafe(length);
let read = 0;
while (read < length) {
const got = fs.readSync(fd, buf, read, length - read, offset + read);
if (got === 0) break;
read += got;
}
out[n] = buf.toString("utf-8", 0, read);
}
return out;
}
/** Release the values.dat handle. The model reopens it on next use. */
close(): void {
if (this.fd !== null) fs.closeSync(this.fd);
this.fd = null;
}
}
const loaded = new Map<string, FpgenModel>();
/**
* The model for `dir` (default `modelDir()`), loading it on first call.
* Synchronous: throws ModelNotInstalled when the pinned model is not on disk --
* `await ensureModel()` first.
*/
export function getModel(dir: string = modelDir()): FpgenModel {
const key = path.resolve(dir);
let model = loaded.get(key);
if (model) return model;
if (!isModelInstalled(key)) {
throw new ModelNotInstalled(
`The fpgen model (${MODEL_PIN.tag}) is not installed in ${key}. ` +
"Call `await ensureModel()` (camoufox/fpgen) before generating fingerprints.",
);
}
model = new FpgenModel(key);
loaded.set(key, model);
return model;
}
/** Drop loaded models (tests, or after changing CAMOUFOX_FPGEN_DATA). */
export function resetModelCache(): void {
for (const model of loaded.values()) model.close();
loaded.clear();
}
typescript/src/fpgen/pin.ts
+32
View File
@@ -0,0 +1,32 @@
/**
* The pinned fpgen model: a copy of scripts/data/fpgen-model.json.
*
* That file is the single source of truth for the Python build; this constant
* is its twin for the npm package, which does not ship the repo's scripts/.
* tests/fpgen-model.test.ts fails if the two ever disagree, so bumping the
* model means editing both (the sha256 is the gate in each).
*
* See scripts/pin-fpgen-model.py for why fpgen's own downloader is not used:
* it disables TLS verification, never checks a digest, and its "first listed
* release" rule cannot reach this tag.
*/
export interface ModelPin {
readonly tag: string;
readonly asset: string;
readonly size: number;
readonly sha256: string;
readonly url: string;
readonly repo: string;
readonly files: readonly string[];
}
export const MODEL_PIN: ModelPin = {
tag: "model-2/2026",
asset: "model-release.zip",
size: 1564571,
sha256: "6530b8322cdaa4ec042921c8d9a0369a0e6e0269ba636c01a7203e4a2f109936",
url: "https://github.com/scrapfly/fingerprint-generator/releases/download/model-2/2026/model-release.zip",
repo: "scrapfly/fingerprint-generator",
files: ["fingerprint-network.json.zst", "values.dat.zst", "values.json.zst"],
};
typescript/src/fpgen/pyjson.ts
+240
View File
@@ -0,0 +1,240 @@
/**
* Python-semantics helpers for the fpgen port (scrapfly/fingerprint-generator,
* Apache-2.0; see ./NOTICE).
*
* fpgen leans on three Python behaviours JavaScript does not share, and each
* one changes results if it is dropped:
*
* 1. dict order. Every probability table is a dict keyed by base85 value ids,
* and 5.5k of those ids ("0", "12", ...) look like array indices, which a JS
* object silently moves to the front. The order decides which value a
* cumulative draw lands on and how the beam breaks ties, so the network is
* parsed into Maps (`parseOrdered`).
* 2. int vs float. `query()` groups values by Python type and sorts each group,
* so 1.0 and 1 land in different places. `parsePyTyped` keeps floats
* wrapped as PyFloat until the grouping is done.
* 3. `==`. Nested conditions compare parsed values with Python equality, where
* 1 == 1.0 == True and dicts ignore key order (`pyEquals`).
*/
import { ValueError } from "./exceptions.js";
/** Marks a Python float through `query()`'s type grouping. */
export class PyFloat {
constructor(readonly value: number) {}
}
const KEY_TOKEN = /"(?:[^"\\]|\\.)*"(\s*:)?/g;
/**
* JSON.parse, but every object becomes a Map whose iteration order is the
* document order (the order Python's dict would have).
*/
export function parseOrdered(text: string): unknown {
// Prefix every object key with U+0001 (written as its JSON escape: a raw
// control character is not valid inside a JSON string) so none is integer-like, which makes
// the native parser keep insertion order; the reviver strips it again. The
// regex consumes whole string tokens, so a quote or colon inside a string
// can never be mistaken for a key.
const marked = text.replace(KEY_TOKEN, (tok, colon: string | undefined) =>
colon === undefined ? tok : `"\\u0001${tok.slice(1)}`,
);
return JSON.parse(marked, (_key, value) => {
if (value === null || typeof value !== "object" || Array.isArray(value)) {
return value;
}
const map = new Map<string, unknown>();
for (const k of Object.keys(value)) {
map.set(k.slice(1), value[k]);
}
return map;
});
}
const TYPED_TOKEN =
/"(?:[^"\\]|\\.)*"|-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+-]?\d+)?/g;
const FLOAT_MARK = "\u0001pyfloat";
const FLOAT_MARK_JSON = "\\u0001pyfloat";
/**
* JSON.parse that keeps Python's int/float distinction: a number token written
* with a fraction or exponent (what Python's json/orjson emit for a float) comes
* back as a PyFloat. Use `unwrapPy` to get plain values back.
*/
export function parsePyTyped(text: string): unknown {
const marked = text.replace(TYPED_TOKEN, (tok) =>
tok[0] === '"' || !/[.eE]/.test(tok)
? tok
: `{"${FLOAT_MARK_JSON}":${tok}}`,
);
return JSON.parse(marked, (_key, value) => {
if (
value !== null &&
typeof value === "object" &&
!Array.isArray(value) &&
FLOAT_MARK in value
) {
return new PyFloat(value[FLOAT_MARK]);
}
return value;
});
}
/** Replace every PyFloat in a structure with its number. */
export function unwrapPy(value: unknown): unknown {
if (value instanceof PyFloat) return value.value;
if (Array.isArray(value)) return value.map(unwrapPy);
if (isPlainObject(value)) {
const out: Record<string, unknown> = {};
for (const [k, v] of Object.entries(value)) out[k] = unwrapPy(v);
return out;
}
return value;
}
export function isPlainObject(value: unknown): value is Record<string, any> {
if (value === null || typeof value !== "object") return false;
const proto = Object.getPrototypeOf(value);
return proto === Object.prototype || proto === null;
}
function pyNumber(value: unknown): number | undefined {
if (typeof value === "number") return value;
if (typeof value === "boolean") return value ? 1 : 0;
if (value instanceof PyFloat) return value.value;
return undefined;
}
/** Python `==` over JSON-shaped values (1 == 1.0 == True, dicts unordered). */
export function pyEquals(a: unknown, b: unknown): boolean {
const na = pyNumber(a);
const nb = pyNumber(b);
if (na !== undefined || nb !== undefined) return na === nb;
if (a === null || b === null) return a === b;
if (typeof a === "string" || typeof b === "string") return a === b;
if (Array.isArray(a) || Array.isArray(b)) {
if (!Array.isArray(a) || !Array.isArray(b) || a.length !== b.length) {
return false;
}
return a.every((v, i) => pyEquals(v, b[i]));
}
if (isPlainObject(a) && isPlainObject(b)) {
const ka = Object.keys(a);
if (ka.length !== Object.keys(b).length) return false;
return ka.every((k) => Object.hasOwn(b, k) && pyEquals(a[k], b[k]));
}
return a === b;
}
/**
* Python's str.casefold(). toLowerCase() agrees with it everywhere except the
* handful of characters whose full case folding expands or differs; those are
* mapped explicitly. (Every value in the model is ASCII, so this only matters
* for what a caller types.)
*/
export function casefold(s: string): string {
const lower = s.toLowerCase();
// biome-ignore lint/suspicious/noControlCharactersInRegex: ASCII fast path
if (/^[\x00-\x7f]*$/.test(lower)) return lower;
return lower
.replace(/ß|ẞ/g, "ss")
.replace(/ς/g, "σ")
.replace(/ſ/g, "s")
.replace(/ff/g, "ff")
.replace(/fi/g, "fi")
.replace(/fl/g, "fl")
.replace(/ffi/g, "ffi")
.replace(/ffl/g, "ffl")
.replace(/ſt|st/g, "st");
}
/**
* orjson.dumps(value).decode() for a condition value. JSON.stringify writes
* the same compact form; the one thing it cannot write is `1.0`, because JS
* has no separate float (build_evidence compensates, see utils.ts).
*/
export function pyDumps(value: unknown): string {
if (typeof value === "function") {
// orjson.JSONDecodeError is a TypeError subclass.
throw new TypeError("Type is not JSON serializable: function");
}
const out = JSON.stringify(value);
if (out === undefined) {
throw new TypeError(`Type is not JSON serializable: ${typeof value}`);
}
return out;
}
const B85_ALPHABET =
"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz!#$%&()*+-;<=>?@^_`{|}~";
const B85_DECODE = new Map<string, number>(
[...B85_ALPHABET].map((c, i) => [c, i]),
);
/** int.from_bytes(base64.b85decode(s), 'big') -- unpacker.base85_to_int. */
export function base85ToInt(s: string): number {
const padding = (5 - (s.length % 5)) % 5;
const padded = s + "~".repeat(padding);
const bytes: number[] = [];
for (let i = 0; i < padded.length; i += 5) {
let acc = 0;
for (let j = i; j < i + 5; j++) {
const d = B85_DECODE.get(padded[j]);
if (d === undefined) {
throw new ValueError(`bad base85 character at position ${j}`);
}
acc = acc * 85 + d;
}
if (acc > 0xffffffff) {
throw new ValueError(`base85 overflow in hunk starting at byte ${i}`);
}
bytes.push(
(acc >>> 24) & 0xff,
(acc >>> 16) & 0xff,
(acc >>> 8) & 0xff,
acc & 0xff,
);
}
if (padding) bytes.length -= padding;
let n = 0;
for (const b of bytes) {
n = n * 256 + b;
if (n > Number.MAX_SAFE_INTEGER) {
throw new ValueError(`base85 value too large: ${s}`);
}
}
return n;
}
/** Python's default ordering for a homogeneous group (str by code point). */
export function pyCompare(a: unknown, b: unknown): number {
const na = pyNumber(a);
const nb = pyNumber(b);
if (na !== undefined && nb !== undefined) return na - nb;
if (typeof a === "string" && typeof b === "string") {
// Code-point order, not UTF-16 code-unit order.
const ia = a[Symbol.iterator]();
const ib = b[Symbol.iterator]();
for (;;) {
const x = ia.next();
const y = ib.next();
if (x.done || y.done) return x.done ? (y.done ? 0 : -1) : 1;
const cx = x.value.codePointAt(0) as number;
const cy = y.value.codePointAt(0) as number;
if (cx !== cy) return cx - cy;
}
}
return 0;
}
/** Python's `type(x).__name__` for a (PyFloat-tagged) JSON value. */
export function pyTypeName(value: unknown): string {
if (value === null) return "NoneType";
if (value instanceof PyFloat) return "float";
if (typeof value === "boolean") return "bool";
if (typeof value === "number")
return Number.isInteger(value) ? "int" : "float";
if (typeof value === "string") return "str";
if (Array.isArray(value)) return "list";
return "dict";
}
typescript/src/fpgen/trace.ts
+129
View File
@@ -0,0 +1,129 @@
/**
* Ported from fpgen/trace.py (scrapfly/fingerprint-generator, Apache-2.0; see
* ./NOTICE): the probability distribution of a target given conditions.
*/
import { RestrictiveConstraints, ValueError } from "./exceptions.js";
import { getModel } from "./model.js";
import {
assertConditions,
buildEvidence,
type Conditions,
type EvidenceMap,
findRoots,
} from "./utils.js";
export class TraceResult {
constructor(
readonly value: any,
readonly probability: number,
/**
* The value's JSON as the model stores it (a TypeScript addition). It
* is the key of the node's lookup index, and JSON.parse cannot always
* give it back: 2**64 re-serialises as 18446744073709552000, where
* the store holds orjson's 1.8446744073709552e19.
*/
readonly text: string,
) {}
toString(): string {
return `<${typeof this.value === "string" ? this.value : JSON.stringify(this.value)}: ${(this.probability * 100).toFixed(5)}%>`;
}
}
/** Recursive result type for several targets. */
export interface TraceResultDict {
[key: string]: TraceResult[] | TraceResultDict;
}
export interface TraceOptions {
/** Return a flat {node: results} object for several targets. */
flatten?: boolean;
}
/**
* The probability distribution(s) of a target given conditions.
*
* One target resolving to one node -> TraceResult[] (most likely first);
* otherwise an object of them.
*/
export function trace(
target: string,
conditions?: Conditions | null,
options?: TraceOptions,
): TraceResult[] | TraceResultDict;
export function trace(
target: readonly string[],
conditions?: Conditions | null,
options?: TraceOptions,
): TraceResultDict | TraceResult[];
export function trace(
target: string | readonly string[],
conditions?: Conditions | null,
options: TraceOptions = {},
): TraceResult[] | TraceResultDict {
return traceWithEvidence(target, conditions, options, new Map());
}
/** trace() with evidence inherited from a Generator (Python's __evidence__). */
export function traceWithEvidence(
target: string | readonly string[],
conditions: Conditions | null | undefined,
{ flatten = false }: TraceOptions,
evidence: EvidenceMap,
): TraceResult[] | TraceResultDict {
assertConditions(conditions);
getModel();
if (conditions && Object.keys(conditions).length) {
buildEvidence(conditions, evidence);
}
const targetTup = typeof target === "string" ? [target] : [...target];
const targetRoots = findRoots(targetTup);
if (!targetTup.length) {
throw new ValueError("Please pass at least one valid target.");
}
// One target: return its results directly
if (targetRoots.length === 1) return pullTarget(targetRoots[0], evidence);
if (flatten) {
const out: TraceResultDict = {};
for (const root of targetRoots) out[root] = pullTarget(root, evidence);
return out;
}
// NOTE: fpgen builds the intermediate dicts but then assigns every leaf at
// the TOP level (`output[parts[-1]] = ...`, not `d[parts[-1]]`). Kept as-is
// so the output has the same shape in both languages.
const output: TraceResultDict = {};
for (const root of targetRoots) {
const parts = root.split(".");
let d: TraceResultDict = output;
for (const part of parts.slice(0, -1)) {
if (!Object.hasOwn(d, part)) d[part] = {};
d = d[part] as TraceResultDict;
}
output[parts[parts.length - 1]] = pullTarget(root, evidence);
}
return output;
}
function pullTarget(target: string, evidence: EvidenceMap): TraceResult[] {
const model = getModel();
const possibilities = model.network.trace(target, evidence);
if (!possibilities.size) {
throw new RestrictiveConstraints(
`Restraints are too restrictive. No possible values for ${target}.`,
);
}
const data = model.lookupValueList(possibilities.keys());
const probs = [...possibilities.values()];
const resp = data.map(
(text, i) => new TraceResult(JSON.parse(text), probs[i], text),
);
// list.sort is stable, as is Array.prototype.sort.
resp.sort((a, b) => b.probability - a.probability);
return resp;
}
typescript/src/fpgen/utils.ts
+571
View File
@@ -0,0 +1,571 @@
/**
* Ported from fpgen/utils.py (scrapfly/fingerprint-generator, Apache-2.0; see
* ./NOTICE): condition parsing (build_evidence), query(), and the helpers that
* walk node names up and down the network and assemble output.
*
* Conditions, in JS terms (Python's kwargs become one object):
* - a plain object is flattened into dotted keys, like a Python dict;
* - a function is a predicate. NOTE, as in Python: it is called with the
* value parsed from the CASEFOLDED JSON, so strings arrive lowercased
* (`os: (v) => v === "windows"`, not "Windows");
* - a Set is a list of alternatives (Python's set/tuple). An array is one
* value, as a Python list is;
* - anything else is one JSON value; `undefined` means "no condition".
*
* One deliberate deviation: Python matches a plain value by comparing
* orjson.dumps(value) with the stored JSON text, so `1.0` only matches "1.0".
* JS cannot write 1.0, so when the exact text finds nothing the lookup falls
* back to comparing the re-serialised stored value. That only ever turns a
* Python InvalidConstraints into a match on the numerically equal value.
*/
import type { BayesianNetwork } from "./bayesian-network.js";
import {
InvalidConstraints,
InvalidNode,
NodePathError,
RestrictiveConstraints,
ValueError,
} from "./exceptions.js";
import { type FpgenModel, getModel } from "./model.js";
import {
casefold,
isPlainObject,
parsePyTyped,
pyCompare,
pyDumps,
pyEquals,
pyTypeName,
unwrapPy,
} from "./pyjson.js";
/** A predicate over a (casefolded, parsed) node value. */
export type Predicate = (value: any) => unknown;
export type ConditionValue = unknown;
/** Conditions for generation / tracing. See the module header. */
export type Conditions = Record<string, ConditionValue>;
/** Evidence: node name -> allowed value ids, in insertion order. */
export type EvidenceMap = Map<string, Set<string>>;
export function model(): FpgenModel {
return getModel();
}
function network(): BayesianNetwork {
return getModel().network;
}
/**
* Query the possible values of a target.
*
* @param target a node, a path inside a node's value, or a prefix of nodes
* @param options.flatten flatten the output dictionary
* @param options.sort sort the output arrays
*/
export function query(
target: string,
{ flatten = false, sort = false }: { flatten?: boolean; sort?: boolean } = {},
): Record<string, any> | any[] {
getModel();
return unwrapPy(queryTyped(target, flatten, sort)) as
| Record<string, any>
| any[];
}
function queryTyped(target: string, flatten: boolean, sort: boolean): unknown {
// Check node list first
const values = lookupPossibilities(target, false);
if (values?.size) {
const output = [...values.keys()].map(parsePyTyped);
// Merge dicts if the data is all dicts, else return a deduped list
if (output.every(isPlainObject)) {
return maybeFlatten(flatten, mergeDicts(output, sort));
}
return dedupe(output, sort);
}
// Target is within a node: look up the tree
const nestedKeys: string[] = [];
const rootData = lookupRootPossibilities(target, {
nestedKeys,
noneIfMissing: true,
casefold: false,
});
if (rootData !== null) {
const output = [...rootData[1].keys()].map((d) =>
atPath(parsePyTyped(d), nestedKeys),
);
if (output.every(isPlainObject)) {
return maybeFlatten(flatten, mergeDicts(output, sort));
}
return dedupe(output, sort);
}
// Search down the tree
const data = searchDownward(target);
const resp = new Map<string, unknown[]>();
for (const key of data) {
const stripped = key.startsWith(`${target}.`)
? key.slice(target.length + 1)
: key;
resp.set(
stripped,
[...(lookupPossibilities(key, false)?.keys() ?? [])].map(parsePyTyped),
);
}
if (flatten) {
const deduped: Record<string, unknown> = {};
for (const [node, vals] of resp) deduped[node] = dedupe(vals, sort);
return flattenDict(deduped);
}
return unflatten(resp, sort);
}
/* Helpers for searching for nodes up/down the network */
/** The value at a nested path (_at_path). Throws NodePathError(key). */
export function atPath(
data: unknown,
pathKeys: Iterable<string>,
{ casefold: fold = false }: { casefold?: boolean } = {},
): any {
let cur: any = data;
for (const key of pathKeys) {
if (!isPlainObject(cur)) throw new NodePathError(key);
if (fold) {
// CaseInsensitiveDict(data)[key]: last colliding key wins.
const want = casefold(key);
let found = false;
let value: unknown;
for (const k of Object.keys(cur)) {
if (casefold(k) === want) {
found = true;
value = cur[k];
}
}
if (!found) throw new NodePathError(key);
cur = value;
} else {
if (!Object.hasOwn(cur, key)) throw new NodePathError(key);
cur = cur[key];
}
}
return cur;
}
/**
* Find the first node that is a prefix of `key` and return [node, its
* possibilities]. `nestedKeys` receives the remaining path.
*/
export function lookupRootPossibilities(
key: string,
{
nestedKeys,
casefold: fold = true,
noneIfMissing = false,
}: {
nestedKeys?: string[];
casefold?: boolean;
noneIfMissing?: boolean;
} = {},
): [string, Map<string, string>] | null {
if (!key) throw new InvalidNode("Key cannot be empty.");
let possibleValues: Map<string, string> | null = null;
while (key) {
const cut = key.lastIndexOf(".");
// Ran out of keys to parse
if (cut === -1) {
if (noneIfMissing) return null;
throw new InvalidNode(`${key} is not a valid node`);
}
const sliced = key.slice(cut + 1);
key = key.slice(0, cut);
nestedKeys?.push(sliced);
possibleValues = lookupPossibilities(key, fold);
if (possibleValues !== null) break;
}
if (possibleValues === null) {
if (noneIfMissing) return null;
throw new InvalidNode(`${key} is not a valid node`);
}
nestedKeys?.reverse();
return [key, possibleValues];
}
/**
* The possible values of a node as {value JSON text: value id}, or null when
* the node doesn't exist. `fold` casefolds the text (the default, as in fpgen).
*/
export function lookupPossibilities(
nodeName: string,
fold = true,
): Map<string, string> | null {
const node = network().nodesByName.get(nodeName);
if (!node) return null;
const lookupValues = node.possibleValues;
const actual = model().lookupValueList(lookupValues);
const out = new Map<string, string>();
actual.forEach((text, i) => {
out.set(fold ? casefold(text) : text, lookupValues[i]);
});
return out;
}
/** All (original-cased) node names under `domain`. Throws when there are none. */
export function searchDownward(domain: string): string[] {
const net = network();
const found: string[] = [];
let i = 0;
for (const node of net.nodesByName.keys()) {
const n = i++;
if (!node.startsWith(domain)) continue;
if (node.length > domain.length && node[domain.length] !== ".") continue;
found.push(net.nodeNames[n]);
}
if (!found.length) throw new InvalidNode(`Unknown node: "${domain}"`);
return found;
}
/** The nodes that make up each target's data (_find_roots). */
export function findRoots(targets: Iterable<string>): string[] {
const net = network();
const out: string[] = [];
for (const t of targets) {
let target = casefold(t);
while (true) {
// Found a valid target
if (net.nodesByName.has(target)) {
out.push(target);
break;
}
const cut = target.lastIndexOf(".");
if (cut !== -1) {
target = target.slice(0, cut);
continue;
}
// At the root key: find nodes below it before giving up
out.push(...searchDownward(target));
break;
}
}
return out;
}
export function reassembleTargets(
targets: readonly string[],
fingerprint: Record<string, any>,
): Record<string, any> {
const result: Record<string, any> = {};
for (const target of targets) {
try {
result[target] = atPath(fingerprint, target.split("."), {
casefold: true,
});
} catch (e) {
if (e instanceof NodePathError) {
throw new InvalidNode(
`'${target}' is not a valid key path (missing ${e.message}).`,
);
}
throw e;
}
}
return result;
}
/* Miscellaneous list/dict helpers */
/** Group items by Python type, dedupe each group, order groups by type name. */
export function dedupe(list: Iterable<unknown>, sort: boolean): unknown[] {
const groups = new Map<string, unknown[]>();
for (const item of list) {
const t = pyTypeName(item);
let group = groups.get(t);
if (!group) {
group = [];
groups.set(t, group);
}
if (!group.some((g) => pyEquals(g, item))) group.push(item);
}
const result: unknown[] = [];
for (const t of [...groups.keys()].sort(pyCompare)) {
const items = groups.get(t) as unknown[];
if (!sort || t === "list" || t === "dict") result.push(...items);
else result.push(...[...items].sort(pyCompare));
}
return result;
}
function unflatten(
dictionary: Map<string, unknown>,
sort: boolean,
): Record<string, any> {
const result: Record<string, any> = {};
for (const [key, raw] of dictionary) {
const parts = key.split(".");
let d = result;
for (const part of parts.slice(0, -1)) {
if (!Object.hasOwn(d, part)) d[part] = {};
d = d[part];
}
d[parts[parts.length - 1]] = Array.isArray(raw) ? dedupe(raw, sort) : raw;
}
return result;
}
/** Turn a nested dictionary into a flattened one (dotted keys). */
export function flattenDict(
dictionary: Record<string, any>,
parentKey = "",
): Record<string, any> {
const items: Record<string, any> = {};
for (const [key, value] of Object.entries(dictionary)) {
const newKey = parentKey ? `${parentKey}.${key}` : key;
if (isPlainObject(value)) Object.assign(items, flattenDict(value, newKey));
else items[newKey] = value;
}
return items;
}
export function maybeFlatten(flatten: boolean | undefined, data: any): any {
if (!isPlainObject(data)) return data;
return flatten ? flattenDict(data) : data;
}
/**
* Merge a list of dicts: dict values merge recursively, list values are
* concatenated and deduped, anything else is deduped.
*/
function mergeDicts(
dictList: Record<string, unknown>[],
sort: boolean,
): Record<string, any> {
if (!dictList.length) return {};
const merged: Record<string, any> = {};
const allKeys = new Set<string>();
for (const d of dictList) for (const k of Object.keys(d)) allKeys.add(k);
for (const key of allKeys) {
const values = dictList
.filter((d) => Object.hasOwn(d, key))
.map((d) => d[key]);
if (values.every(isPlainObject)) {
merged[key] = mergeDicts(values as Record<string, unknown>[], sort);
} else if (values.every(Array.isArray)) {
merged[key] = dedupe((values as unknown[][]).flat(), sort);
} else {
merged[key] = dedupe(values, sort);
}
}
return merged;
}
/* Parse user input */
/** A flattened condition: one JSON text, a predicate, or alternatives. */
type FlatCondition = string | Predicate | readonly (string | Predicate)[];
/** Flatten nested conditions into dotted keys (_flatten_conditions). */
export function flattenConditions(
dictionary: Record<string, unknown>,
parentKey = "",
fold = false,
): Map<string, FlatCondition> {
const items = new Map<string, FlatCondition>();
for (const [key, value] of Object.entries(dictionary)) {
if (value === undefined) continue;
let newKey = parentKey ? `${parentKey}.${key}` : key;
if (isPlainObject(value)) {
// As in Python, the recursion does not casefold.
for (const [k, v] of flattenConditions(value, newKey)) items.set(k, v);
continue;
}
let flat: FlatCondition;
if (value instanceof Set) {
// A set (Python: set/tuple) is a list of alternatives.
flat = [...value].map((v) => pyDumps(v));
} else if (typeof value === "function") {
flat = value as Predicate;
} else {
flat = pyDumps(value);
}
if (fold) newKey = casefold(newKey);
items.set(newKey, flat);
}
return items;
}
function describe(val: unknown): string {
if (typeof val === "function") {
return `<function ${val.name || "<anonymous>"}>`;
}
return String(val);
}
/**
* Turn user conditions into evidence (node -> allowed value ids), validating
* them against the network. Mutates `evidence`.
*/
export function buildEvidence(
conditions: Record<string, unknown>,
evidence: EvidenceMap,
strict?: boolean | null,
): void {
if (strict == null) strict = true;
const net = network();
// Flatten to match the format of the fingerprint network
const flat = flattenConditions(conditions, "", true);
for (let [key, value] of flat) {
let possibleValues = lookupPossibilities(key);
// Handle nested keys
let nestedKeys: string[] = [];
if (possibleValues === null) {
[key, possibleValues] = lookupRootPossibilities(key, {
nestedKeys,
}) as [string, Map<string, string>];
}
// Get the real name for the key
key = net.node(key).name;
const allowed = new Set<string>();
evidence.set(key, allowed);
const alternatives: readonly (string | Predicate)[] =
typeof value === "string" || typeof value === "function"
? [value]
: value;
for (const valueCon of alternatives) {
// Read the passed value
const val: unknown =
typeof valueCon === "function"
? valueCon
: JSON.parse(casefold(valueCon));
// Nested keys: keep the possible values whose value at the nested
// path matches.
if (nestedKeys.length) {
nestedKeys = nestedKeys.map(casefold);
for (const [possValue, lookupIndex] of possibleValues) {
const outputtedPossible = JSON.parse(possValue);
let targetValue: unknown;
try {
targetValue = atPath(outputtedPossible, nestedKeys);
} catch (e) {
if (e instanceof NodePathError) continue; // bad data
throw e;
}
if (typeof val === "function" && val(targetValue)) {
allowed.add(lookupIndex);
} else if (pyEquals(targetValue, val)) {
allowed.add(lookupIndex);
}
}
if (!allowed.size) {
if (typeof val === "function") {
throw new InvalidConstraints(
`The passed function (${describe(val)}) yielded no possible values for "${key}" ` +
`at "${nestedKeys.join(".")}"`,
);
}
throw new InvalidConstraints(
`${describe(valueCon)} is not a possible value for "${key}" ` +
`at "${nestedKeys.join(".")}"`,
);
}
continue;
}
// ===== NON NESTED VALUE HANDLING =====
if (typeof val === "function") {
let found = false;
for (const [possibleVal, lookupIndex] of possibleValues) {
if (val(JSON.parse(possibleVal))) {
allowed.add(lookupIndex);
found = true;
}
}
if (!found) {
throw new InvalidConstraints(
`The passed function (${describe(val)}) yielded no possible values for "${key}"`,
);
}
continue;
}
// Non nested values: look the JSON text up directly
const wanted = casefold(valueCon as string);
let lookupIndex = possibleValues.get(wanted);
if (lookupIndex === undefined) {
// JS has no 1.0; compare against the re-serialised stored value.
for (const [possibleVal, idx] of possibleValues) {
if (JSON.stringify(JSON.parse(possibleVal)) === wanted) {
lookupIndex = idx;
break;
}
}
}
if (lookupIndex === undefined) {
throw new InvalidConstraints(
`${describe(valueCon)} is not a possible value for "${key}"`,
);
}
allowed.add(lookupIndex);
}
}
// Validate the evidence (or, when not strict, relax it once). fpgen drops
// the FIRST key and does not re-validate; so does this.
try {
net.validateEvidence(evidence);
} catch (e) {
if (strict || !(e instanceof RestrictiveConstraints)) {
throw e;
}
const first = evidence.keys().next();
if (!first.done) evidence.delete(first.value);
}
}
/** Conditions must be a plain object when given (_assert_dict_xor_kwargs). */
export function assertConditions(conditions: unknown): void {
if (conditions == null) return;
if (!isPlainObject(conditions)) {
throw new ValueError(
"Invalid argument. Constraints must be passed as kwargs or as a dictionary.",
);
}
}
/* Convert network output to human readable output */
/** Unflatten (or flatten) a sampled {node: value id} into the output object. */
export function makeOutputDict(
data: Map<string, string>,
flatten: boolean | undefined,
): Record<string, any> {
const keys = [...data.keys()];
const values = model().lookupValueList(data.values());
if (flatten) {
const flat: Record<string, any> = {};
keys.forEach((k, i) => {
flat[k] = JSON.parse(values[i]);
});
// Flatten node values that themselves are dicts
return flattenDict(flat);
}
const result: Record<string, any> = {};
keys.forEach((key, i) => {
const parts = key.split(".");
let d = result;
for (const part of parts.slice(0, -1)) {
if (!Object.hasOwn(d, part)) d[part] = {};
d = d[part];
}
d[parts[parts.length - 1]] = JSON.parse(values[i]);
});
return result;
}
typescript/src/geolocation.ts
+335
View File
@@ -0,0 +1,335 @@
/**
* Helpers to fetch geolocation, timezone, and locale data given an IP.
*
* TypeScript twin of pythonlib/camoufox/geolocation.py. The on-disk layout
* (geoip/mmdb/<name>-<ipver>.mmdb and geoip/config.yml under the camoufox
* cache dir) is the Python package's, so both launchers share one database.
*/
import * as fs from "node:fs";
import { createRequire } from "node:module";
import * as os from "node:os";
import * as path from "node:path";
import { parse as parseYaml, stringify as stringifyYaml } from "yaml";
import { NotInstalledGeoIPExtra, UnknownIPLocation } from "./exceptions.js";
import { validateIP } from "./ip.js";
import { Geolocation, SELECTOR } from "./locales.js";
import { INSTALL_DIR, LOCAL_DATA } from "./paths.js";
export const GEOIP_DIR: string = path.join(INSTALL_DIR, "geoip");
export const MMDB_DIR: string = path.join(GEOIP_DIR, "mmdb");
export const GEOIP_CONFIG: string = path.join(GEOIP_DIR, "config.yml");
export interface GeoIPRepo {
name: string;
urls: Record<string, string | string[]>;
paths: Record<string, string>;
extract?: boolean;
[key: string]: any;
}
/** A reader over an mmdb file: maxminddb.Reader's `get`. */
export interface MmdbReader {
get(ip: string): any;
close?(): void;
}
const require_ = createRequire(import.meta.url);
/**
* Whether the mmdb reader is available. Python gates this on the optional
* `maxminddb` import (`pip install camoufox[geoip]`); here it is the optional
* `maxmind` package.
*/
export function allowGeoip(): boolean {
try {
require_.resolve("maxmind");
return true;
} catch {
return false;
}
}
/**
* Injection points (tests replace them): opening an mmdb, and downloading.
*/
export const geoipDeps = {
async openDatabase(mmdbPath: string): Promise<MmdbReader> {
const maxmind = (await import("maxmind")).default;
const buffer = fs.readFileSync(mmdbPath);
return new maxmind.Reader<any>(buffer);
},
downloadMmdb: (source?: string) => downloadMmdb(source),
};
/**
* Resolve a dotted path in a nested object.
*/
function findIn(data: any, key: string): any {
for (const part of key.split(".")) {
if (typeof data !== "object" || data === null || Array.isArray(data)) {
return null;
}
data = data[part];
if (data === undefined || data === null) {
return null;
}
}
return data;
}
/**
* Load GeoIP repos and default name from repos.yml.
*/
function loadGeoipRepos(): [GeoIPRepo[], string] {
const data =
(parseYaml(
fs.readFileSync(path.join(LOCAL_DATA, "repos.yml"), "utf-8"),
) as Record<string, any>) ?? {};
const geoipRepos: GeoIPRepo[] = data.geoip ?? [];
const defaultName: string = data.default?.geoip ?? "GeoLite2";
return [geoipRepos, defaultName];
}
/**
* Get GeoIP config by name from repos.yml. If omitted, uses the default.
*/
export function getGeoipConfigByName(name?: string | null): GeoIPRepo {
const [repos, defaultName] = loadGeoipRepos();
const targetName = name || defaultName;
const validateRepo = (repo: GeoIPRepo): GeoIPRepo => {
const raw = repo as Record<string, any>;
if (!("urls" in raw)) {
throw new Error(`GeoIP repo '${raw.name}' missing required urls`);
}
if (!("paths" in raw)) {
throw new Error(`GeoIP repo '${raw.name}' missing required paths`);
}
return repo;
};
for (const repo of repos) {
if ((repo.name ?? "").toLowerCase() === targetName.toLowerCase()) {
return validateRepo(repo);
}
}
if (name) {
const available = repos.map((r) => `'${r.name ?? "Unknown"}'`);
throw new Error(
`GeoIP database '${name}' not found. Available: [${available.join(", ")}]`,
);
}
if (repos.length) {
return validateRepo(repos[0]);
}
throw new Error("No GeoIP repos configured in repos.yml");
}
/**
* Load the active GeoIP config from disk, falling back to the repos.yml default.
*/
export function loadGeoipConfig(): GeoIPRepo {
if (fs.existsSync(GEOIP_CONFIG)) {
const saved =
(parseYaml(fs.readFileSync(GEOIP_CONFIG, "utf-8")) as Record<
string,
any
>) ?? {};
try {
return getGeoipConfigByName(saved.name);
} catch {
return saved as GeoIPRepo;
}
}
return getGeoipConfigByName(undefined);
}
/**
* Save the active GeoIP source name to disk.
*/
export function saveGeoipConfig(config: GeoIPRepo): void {
fs.mkdirSync(GEOIP_DIR, { recursive: true });
fs.writeFileSync(GEOIP_CONFIG, stringifyYaml({ name: config.name }));
}
/**
* Get the path to the mmdb file for the specified IP version.
*/
export function getMmdbPath(
ipVersion: string = "ipv4",
config?: GeoIPRepo,
): string {
const cfg = config ?? loadGeoipConfig();
const name = (cfg.name ?? "geolite2").toLowerCase();
const urls = cfg.urls ?? {};
if ("combined" in urls) {
return path.join(MMDB_DIR, `${name}-combined.mmdb`);
}
return path.join(MMDB_DIR, `${name}-${ipVersion}.mmdb`);
}
/**
* Checks that the mmdb reader is available.
*/
export function geoipAllowed(): void {
if (!allowGeoip()) {
throw new NotInstalledGeoIPExtra(
"Please install the geoip extra to use this feature: npm install maxmind",
);
}
}
/**
* Downloads the GeoIP database(s) to geoip/mmdb/.
*/
export async function downloadMmdb(
source?: string,
progressCallback?: (downloaded: number, total: number) => void,
): Promise<void> {
geoipAllowed();
const { unzip, webdl } = await import("./pkgman.js");
const config = source ? getGeoipConfigByName(source) : loadGeoipConfig();
const urls = config.urls;
const name = config.name.toLowerCase();
fs.mkdirSync(MMDB_DIR, { recursive: true });
const extract = config.extract ?? false;
const isCombined = "combined" in urls;
for (const [ipVer, rawList] of Object.entries(urls)) {
const suffix = isCombined ? "" : ` (${ipVer})`;
let dlDesc = `Downloading ${config.name}${suffix}`;
let exDesc = `Extracting ${config.name}${suffix}`;
const maxLen = Math.max(dlDesc.length, exDesc.length);
dlDesc = dlDesc.padEnd(maxLen);
exDesc = exDesc.padEnd(maxLen);
const mmdbPath = path.join(MMDB_DIR, `${name}-${ipVer}.mmdb`);
const urlList = typeof rawList === "string" ? [rawList] : rawList;
let lastError: unknown;
let done = false;
for (const url of urlList) {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "camoufox-geoip-"));
try {
const buffer = await webdl(
url,
dlDesc,
progressCallback === undefined,
null,
{ progressCallback },
);
if (extract) {
unzip(buffer, tmpDir, exDesc, progressCallback === undefined);
const found = findFirstMmdb(tmpDir);
if (!found) {
throw new Error("No .mmdb file found in archive");
}
fs.renameSync(found, mmdbPath);
} else {
fs.writeFileSync(mmdbPath, buffer);
}
done = true;
break;
} catch (error) {
lastError = error;
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
}
if (!done) {
throw lastError ?? new Error(`Failed to download ${ipVer}`);
}
}
saveGeoipConfig(config);
}
/** Path(tmpdir).rglob('*.mmdb')[0] */
function findFirstMmdb(dir: string): string | null {
const entries = fs.readdirSync(dir, { withFileTypes: true, recursive: true });
for (const entry of entries) {
if (entry.isFile() && entry.name.endsWith(".mmdb")) {
return path.join(entry.parentPath, entry.name);
}
}
return null;
}
/**
* Check if the GeoIP database needs an update (older than 30 days).
*/
export function needsUpdate(config?: GeoIPRepo): boolean {
const cfg = config ?? loadGeoipConfig();
const updateDays = 30;
const ipv4Path = getMmdbPath("ipv4", cfg);
if (!fs.existsSync(ipv4Path)) {
return true;
}
const age = Date.now() - fs.statSync(ipv4Path).mtimeMs;
return age > updateDays * 24 * 60 * 60 * 1000;
}
/** float(x) for a value read out of the database. */
function pyFloat(value: any): number {
if (value === null || value === undefined) {
throw new TypeError(
"float() argument must be a string or a real number, not 'NoneType'",
);
}
const n = Number(value);
if (Number.isNaN(n)) {
throw new Error(`could not convert string to float: '${value}'`);
}
return n;
}
/**
* Gets the geolocation for an IP address.
*/
export async function getGeolocation(
ip: string,
geoipDb?: string,
): Promise<Geolocation> {
validateIP(ip);
const ipVersion = ip.includes(":") ? "ipv6" : "ipv4";
let mmdbPath = getMmdbPath(ipVersion);
if (!fs.existsSync(mmdbPath) || needsUpdate()) {
await geoipDeps.downloadMmdb();
mmdbPath = getMmdbPath(ipVersion);
}
const config = geoipDb ? getGeoipConfigByName(geoipDb) : loadGeoipConfig();
const paths = config.paths;
const reader = await geoipDeps.openDatabase(mmdbPath);
try {
const resp = reader.get(ip);
if (!resp) {
throw new UnknownIPLocation(`IP not found in database: ${ip}`);
}
const isoCode = findIn(resp, paths.iso_code);
const longitude = findIn(resp, paths.longitude);
const latitude = findIn(resp, paths.latitude);
const timezone = findIn(resp, paths.timezone);
const iso = String(isoCode ?? "None").toUpperCase();
const locale = await SELECTOR.fromRegion(iso);
return new Geolocation(
locale,
pyFloat(longitude),
pyFloat(latitude),
String(timezone ?? "None"),
);
} finally {
reader.close?.();
}
}
typescript/src/index.ts
+46
View File
@@ -0,0 +1,46 @@
/**
* Public API. Mirrors pythonlib/camoufox/__init__.py (Camoufox, NewBrowser,
* NewContext, their Async* twins, DefaultAddons, launch_options), plus the
* package-management and server helpers the TS port has always exported.
*/
export type { DefaultAddon } from "./addons.js";
export { DefaultAddons } from "./addons.js";
export {
AsyncCamoufox,
AsyncNewBrowser,
AsyncNewContext,
} from "./async_api.js";
export {
generateContextFingerprint,
getRandomPreset,
loadPresets,
Screen,
} from "./fingerprints.js";
export {
findInstalledVersion,
listInstalled,
printTree,
} from "./multiversion.js";
export {
CamoufoxFetcher,
INSTALL_DIR,
installedVerStr,
OS_NAME,
RepoConfig,
} from "./pkgman.js";
export { type LaunchServerOptions, launchServer } from "./server.js";
export {
Camoufox,
type Headless,
NewBrowser,
type NewBrowserOptions,
NewContext,
type NewContextOptions,
} from "./sync_api.js";
export {
type LaunchOptions,
launchOptions,
launchOptions as launch_options,
} from "./utils.js";
export { VirtualDisplay } from "./virtdisplay.js";
export { FallbackWarning, LeakWarning } from "./warnings.js";
typescript/src/ip.ts
+207
View File
@@ -0,0 +1,207 @@
/**
* Helpers to find the user's public IP address for geolocation.
*
* TypeScript twin of pythonlib/camoufox/ip.py.
*/
import { Impit } from "impit";
import { InvalidIP, InvalidProxy } from "./exceptions.js";
export interface ProxyConfig {
server: string;
username?: string;
password?: string;
bypass?: string;
}
// biome-ignore lint/complexity/noStaticOnlyClass: these are the Python Proxy dataclass's methods; the data itself is ProxyConfig
export class ProxyHelper {
/**
* Parses the proxy server string.
*/
static parseServer(server: string): {
schema: string;
url: string;
port?: string;
} {
const proxyMatch = server.match(/^(?:(\w+):\/\/)?(.*?)(?::(\d+))?$/);
if (!proxyMatch) {
throw new InvalidProxy(`Invalid proxy server: ${server}`);
}
return {
schema: proxyMatch[1] || "http",
url: proxyMatch[2],
port: proxyMatch[3],
};
}
static asString(proxy: ProxyConfig): string {
const { schema, url, port } = ProxyHelper.parseServer(proxy.server);
let result = `${schema}://`;
if (proxy.username) {
result += proxy.username;
if (proxy.password) {
result += `:${proxy.password}`;
}
result += "@";
}
result += url;
if (port) {
result += `:${port}`;
}
return result;
}
}
export function validIPv4(ip: string | false): boolean {
if (!ip) {
return false;
}
return /^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$/.test(ip);
}
export function validIPv6(ip: string | false): boolean {
if (!ip) {
return false;
}
return /^(([0-9a-fA-F]{0,4}:){1,7}[0-9a-fA-F]{0,4})$/.test(ip);
}
export function validateIP(ip: string): void {
if (!validIPv4(ip) && !validIPv6(ip)) {
throw new InvalidIP(`Invalid IP address: ${ip}`);
}
}
// Impit has no close/dispose API: each instance's native client (Tokio
// runtime resources, one UDP resolver socket) is only reclaimed when the
// JS wrapper is GC'd -- and V8 rarely collects the tiny wrappers, so
// per-call instances leak fds in long-running processes. Reuse instances
// via a small LRU keyed by proxy URL; evicted entries are reclaimed by GC.
const IMPIT_CACHE_MAX = 8;
const impitCache = new Map<string, Impit>();
function getImpit(proxy?: string): Impit {
const key = proxy ?? "";
const cached = impitCache.get(key);
if (cached) {
impitCache.delete(key);
impitCache.set(key, cached);
return cached;
}
// Certificates are verified (the Python twin's verify=True): an IP lookup
// is what geoip trusts for the whole identity, so a MITM must not pick it.
const impit = new Impit({
proxyUrl: proxy,
timeout: 5000,
ignoreTlsErrors: false,
});
impitCache.set(key, impit);
if (impitCache.size > IMPIT_CACHE_MAX) {
impitCache.delete(impitCache.keys().next().value as string);
}
return impit;
}
export const PROXY_LOOKUP_FAILED =
"Could not look up the proxy's exit IP and timezone. Pass webrtc_ip and " +
"timezone_id explicitly to skip the lookup";
/**
* The exit IP of `proxy` and that IP's timezone, looked up through the proxy.
* Throws InvalidIP when the lookup fails: a context that silently kept the
* host's WebRTC IP and timezone behind a proxy would be a leak.
*/
export async function proxyExitGeo(proxy: string): Promise<[string, string]> {
let data: {
status?: string;
message?: string;
query?: string;
timezone?: string;
};
try {
const response = await getImpit(proxy).fetch(
"http://ip-api.com/json?fields=status,message,query,timezone",
);
if (!response.ok) {
throw new Error(`${response.status} Error: ${response.statusText}`);
}
data = await response.json();
} catch (error) {
throw new InvalidIP(`${PROXY_LOOKUP_FAILED}: ${error}`);
}
if (data.status !== "success" || !data.query || !data.timezone) {
throw new InvalidIP(
`${PROXY_LOOKUP_FAILED}: ${data.message ?? JSON.stringify(data)}`,
);
}
validateIP(data.query);
return [data.query, data.timezone];
}
const PUBLIC_IP_URLS = [
// Prefers IPv4
"https://api.ipify.org",
"https://checkip.amazonaws.com",
"https://ipinfo.io/ip",
// IPv4 & IPv6
"https://icanhazip.com",
"https://ifconfig.co/ip",
"https://ipecho.net/plain",
];
// The Python twin memoizes public_ip() with lru_cache; mirror that so repeated
// launches through the same proxy don't re-hit the API endpoints.
const publicIPCache = new Map<string, Promise<string>>();
/**
* Sends a request to a public IP api.
*/
export function publicIP(proxy?: string): Promise<string> {
const key = proxy ?? "";
const cached = publicIPCache.get(key);
if (cached) {
return cached;
}
const pending = resolvePublicIP(proxy).catch((error) => {
// Never cache a failure: the next launch should retry.
publicIPCache.delete(key);
throw error;
});
publicIPCache.set(key, pending);
return pending;
}
async function resolvePublicIP(proxy?: string): Promise<string> {
let endException: unknown;
for (const url of PUBLIC_IP_URLS) {
try {
const response = await getImpit(proxy).fetch(url);
if (!response.ok) {
// requests' raise_for_status()
throw new Error(
`${response.status} Error: ${response.statusText} for url: ${url}`,
);
}
const ip = (await response.text()).trim();
validateIP(ip);
return ip;
} catch (error) {
endException = error;
if (process.env.CAMOUFOX_DEBUG) {
console.warn(
new InvalidProxy(
`camoufox(warn): Failed to fetch public IP from ${url}, retrying with another URL...`,
{ cause: error },
),
);
}
}
}
const detail =
endException instanceof Error ? endException.message : String(endException);
throw new InvalidIP(`Failed to get IP address: ${detail}`, {
cause: endException,
});
}
typescript/src/locale.ts
+8
View File
@@ -0,0 +1,8 @@
/**
* Locale + geolocation handling: re-exports locales.ts and geolocation.ts,
* the twins of pythonlib/camoufox/locales.py and geolocation.py.
*
* Kept so existing `./locale.js` imports keep working.
*/
export * from "./geolocation.js";
export * from "./locales.js";
typescript/src/locales.ts
+415
View File
@@ -0,0 +1,415 @@
/**
* Locale data structures, validation and the statistical locale selector.
*
* TypeScript twin of pythonlib/camoufox/locales.py.
*/
import * as fs from "node:fs";
import * as path from "node:path";
import tags from "language-tags";
import xml2js from "xml2js";
import {
InvalidLocale,
UnknownLanguage,
UnknownTerritory,
} from "./exceptions.js";
import { LOCAL_DATA } from "./paths.js";
import { LeakWarning } from "./warnings.js";
/*
* Data structures for locale and geolocation info
*/
/**
* Stores locale, region, and script information.
*/
export class Locale {
constructor(
public language: string,
public region?: string,
public script?: string,
) {}
get asString(): string {
if (this.region) {
return `${this.language}-${this.region}`;
}
return this.language;
}
/**
* Converts the locale to an intl config object.
*/
asConfig(): Record<string, string> {
if (!this.region) {
throw new Error("AssertionError: Locale.asConfig() needs a region");
}
const data: Record<string, string> = {
"locale:region": this.region,
"locale:language": this.language,
};
if (this.script) {
data["locale:script"] = this.script;
}
return data;
}
}
/**
* Stores geolocation information.
*/
export class Geolocation {
constructor(
public readonly locale: Locale,
public readonly longitude: number,
public readonly latitude: number,
public readonly timezone: string,
public readonly accuracy?: number,
) {}
/**
* Converts the geolocation to a config object.
*/
asConfig(): Record<string, any> {
const data: Record<string, any> = {
"geolocation:longitude": this.longitude,
"geolocation:latitude": this.latitude,
timezone: this.timezone,
...this.locale.asConfig(),
};
if (this.accuracy) {
data["geolocation:accuracy"] = this.accuracy;
}
return data;
}
}
/*
* Helpers to validate and normalize locales
*/
/**
* Verifies that a locale is valid. Takes either language-region or language.
*/
export function verifyLocale(loc: string): void {
if (tags.check(loc)) {
return;
}
throw InvalidLocale.invalidInput(loc);
}
/**
* Normalizes and validates a locale code.
*
* The script is the LANGUAGE's Suppress-Script ("Latn" for "en"), exactly as
* pythonlib does -- not the tag's explicit script subtag. Parity note: that
* makes "en-US" come out as en/Latn/US, and "zh-Hans-CN" as zh/(none)/CN.
*/
export function normalizeLocale(locale: string): Locale {
verifyLocale(locale);
// Parse the locale
const parser = tags(locale);
const region = parser.region();
if (!region) {
throw InvalidLocale.invalidInput(locale);
}
const language = parser.language();
if (!language) {
throw InvalidLocale.invalidInput(locale);
}
const suppressScript = language.script();
// Return a formatted locale object
return new Locale(
language.format(),
region.format(),
suppressScript ? suppressScript.format() : undefined,
);
}
/**
* Handles a locale input, normalizing it if necessary.
*/
export async function handleLocale(
locale: string,
ignoreRegion: boolean = false,
): Promise<Locale> {
// If the caller passed `language-region` or `language-script-region`, normalize it.
if (locale.length > 3) {
return normalizeLocale(locale);
}
// Case: caller passed a `region` and needs a full locale
try {
return await SELECTOR.fromRegion(locale);
} catch (e) {
if (!(e instanceof UnknownTerritory)) throw e;
}
// Case: caller passed a `language`, and doesn't care about the region
if (ignoreRegion) {
verifyLocale(locale);
return new Locale(locale);
}
// Case: caller passed a `language` and wants a region
let language: Locale | undefined;
try {
language = await SELECTOR.fromLanguage(locale);
} catch (e) {
if (!(e instanceof UnknownLanguage)) throw e;
}
if (language) {
LeakWarning.warn("no_region");
return language;
}
// Locale is not in a valid format.
throw InvalidLocale.invalidInput(locale);
}
/**
* Handles a list of locales, writing them into the config.
*/
export async function handleLocales(
locales: string | string[],
config: Record<string, any>,
): Promise<void> {
const list =
typeof locales === "string"
? locales.split(",").map((loc) => loc.trim())
: locales;
// First, handle the first locale. This will be used for the intl api.
const intlLocale = await handleLocale(list[0]);
Object.assign(config, intlLocale.asConfig());
if (list.length < 2) {
return;
}
// If additional locales were passed, validate them.
// Note: in this case, we do not need the region.
const resolved: string[] = [];
for (const locale of list) {
resolved.push((await handleLocale(locale, true)).asString);
}
config["locale:all"] = joinUnique(resolved);
}
/**
* Joins a sequence of strings without duplicates.
*/
function joinUnique(seq: string[]): string {
return [...new Set(seq)].join(", ");
}
/*
* Gets a random language based on the territory code.
*/
interface TerritoryElement {
$: Record<string, string>;
languagePopulation?: Array<{ $: Record<string, string> }>;
}
let unicodeInfo: Promise<TerritoryElement[]> | null = null;
/**
* Fetches supplemental data from the territoryInfo.xml file.
* Source: https://raw.githubusercontent.com/unicode-org/cldr/master/common/supplemental/supplementalData.xml
*/
export function getUnicodeInfo(): Promise<TerritoryElement[]> {
if (!unicodeInfo) {
unicodeInfo = (async () => {
const data = await fs.promises.readFile(
path.join(LOCAL_DATA, "territoryInfo.xml"),
);
const parsed = await new xml2js.Parser().parseStringPromise(data);
const territories = parsed?.territoryInfo?.territory;
if (!territories) {
throw new Error("Failed to load territoryInfo.xml");
}
return territories as TerritoryElement[];
})();
}
return unicodeInfo;
}
/** float(element.get(attr, 0)) */
function asFloat(attrs: Record<string, string>, attr: string): number {
const raw = attrs[attr];
if (raw === undefined) return 0;
const value = Number(raw.trim());
if (Number.isNaN(value)) {
throw new Error(`could not convert string to float: '${raw}'`);
}
return value;
}
/**
* numpy's pairwise summation (np.sum over a 1-D float64 array), so the
* normalised probabilities are bit-identical to the Python selector's.
*/
export function pairwiseSum(values: readonly number[]): number {
const sum = (lo: number, n: number): number => {
if (n < 8) {
let res = 0;
for (let i = 0; i < n; i++) res += values[lo + i];
return res;
}
if (n <= 128) {
const r = values.slice(lo, lo + 8);
let i = 8;
for (; i < n - (n % 8); i += 8) {
for (let j = 0; j < 8; j++) r[j] += values[lo + i + j];
}
let res = r[0] + r[1] + (r[2] + r[3]) + (r[4] + r[5] + (r[6] + r[7]));
for (; i < n; i++) res += values[lo + i];
return res;
}
let n2 = Math.floor(n / 2);
n2 -= n2 % 8;
return sum(lo, n2) + sum(lo + n2, n - n2);
};
return sum(0, values.length);
}
/**
* Injection point for the selector's one random draw (tests pin it).
* `random()` plays numpy's RandomState.random_sample().
*/
export const localeDeps = {
random: (): number => Math.random(),
};
/**
* numpy.random.choice(items, p=probabilities): searchsorted(right) of one
* uniform draw in the renormalised CDF.
*/
export function weightedChoice<T>(
items: readonly T[],
p: readonly number[],
): T {
const cdf: number[] = [];
let acc = 0;
for (const x of p) {
acc += x;
cdf.push(acc);
}
const last = cdf[cdf.length - 1];
for (let i = 0; i < cdf.length; i++) cdf[i] /= last;
const u = localeDeps.random();
let idx = 0;
while (idx < cdf.length && cdf[idx] <= u) idx++;
return items[Math.min(idx, items.length - 1)];
}
/**
* Selects a random locale based on statistical data. Takes either a territory
* code or a language code, and generates a Locale.
*/
export class StatisticalLocaleSelector {
/**
* Calculates a random language based on the territory code, based on the
* probability that a person speaks the language in the territory.
*/
private async loadTerritoryData(
isoCode: string,
): Promise<[string[], number[]]> {
const territories = await getUnicodeInfo();
const territory = territories.find((t) => t.$?.type === isoCode);
if (!territory) {
throw new UnknownTerritory(`Unknown territory: ${isoCode}`);
}
const langPopulations = territory.languagePopulation;
if (!langPopulations?.length) {
throw new Error(`No language data found for region: ${isoCode}`);
}
const languages = langPopulations.map((lang) => lang.$.type);
const percentages = langPopulations.map((lang) =>
asFloat(lang.$, "populationPercent"),
);
return this.normalizeProbabilities(languages, percentages);
}
/**
* Calculates a random region for a language based on the total speakers of
* the language in that region.
*/
private async loadLanguageData(
language: string,
): Promise<[string[], number[]]> {
const territories = (await getUnicodeInfo()).filter((t) =>
t.languagePopulation?.some((lp) => lp.$?.type === language),
);
if (!territories.length) {
throw new UnknownLanguage(
`No region data found for language: ${language}`,
);
}
const regions: string[] = [];
const percentages: number[] = [];
for (const terr of territories) {
const region = terr.$?.type;
if (region === undefined) continue; // Skip if region is not found
const langPop = terr.languagePopulation?.find(
(lp) => lp.$?.type === language,
);
if (!langPop) continue;
regions.push(region);
percentages.push(
((asFloat(langPop.$, "populationPercent") *
asFloat(terr.$, "literacyPercent")) /
10_000) *
asFloat(terr.$, "population"),
);
}
if (!regions.length) {
throw new Error(`No valid region data found for language: ${language}`);
}
return this.normalizeProbabilities(regions, percentages);
}
/** Normalize probabilities. */
normalizeProbabilities(
items: string[],
freq: number[],
): [string[], number[]] {
const total = pairwiseSum(freq);
return [items, freq.map((f) => f / total)];
}
/**
* Get a random locale based on the territory ISO code.
*/
async fromRegion(region: string): Promise<Locale> {
const [languages, probabilities] = await this.loadTerritoryData(region);
const language = weightedChoice(languages, probabilities).replaceAll(
"_",
"-",
);
return normalizeLocale(`${language}-${region}`);
}
/**
* Get a random locale based on the language.
*/
async fromLanguage(language: string): Promise<Locale> {
const [regions, probabilities] = await this.loadLanguageData(language);
const region = weightedChoice(regions, probabilities);
return normalizeLocale(`${language}-${region}`);
}
}
export const SELECTOR = new StatisticalLocaleSelector();
typescript/src/multiversion.ts
+657
View File
@@ -0,0 +1,657 @@
/**
* Manager for handling multiple Camoufox versions side by side.
*
* TypeScript twin of pythonlib/camoufox/multiversion.py.
*/
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { finished } from "node:stream/promises";
import { INSTALL_DIR, OS_NAME, rprint } from "./paths.js";
// pkgman and multiversion are mutually dependent, exactly as the Python twin's
// function-local imports are. Every use of these sits inside a function body,
// so the ESM cycle resolves before any binding is read.
import {
type AvailableVersion,
type CamoufoxFetcher,
cmpStr,
formatAssetDate,
makeExecutable,
RepoConfig,
unzip,
Version,
verifySha256,
} from "./pkgman.js";
export const BROWSERS_DIR: string = path.join(INSTALL_DIR, "browsers");
export const CONFIG_FILE: string = path.join(INSTALL_DIR, "config.json");
export const REPO_CACHE_FILE: string = path.join(
INSTALL_DIR,
"repo_cache.json",
);
export const COMPAT_FLAG: string = path.join(INSTALL_DIR, ".0.5_FLAG");
export interface CamoufoxConfig {
active_version?: string | null;
channel?: string;
pinned?: string;
pinned_sha?: string;
geoip?: string;
[key: string]: unknown;
}
/**
* Load user config from disk, or return an empty object.
*/
export function loadConfig(): CamoufoxConfig {
if (fs.existsSync(CONFIG_FILE)) {
try {
return JSON.parse(fs.readFileSync(CONFIG_FILE, "utf-8"));
} catch {
// Corrupt config: treat as absent, exactly as the Python twin does.
}
}
return {};
}
/**
* Get the default repo's stable channel string (like official/stable).
*/
export function getDefaultChannel(): string {
return `${RepoConfig.getDefaultName().toLowerCase()}/stable`;
}
/**
* Save user config to disk.
*/
export function saveConfig(config: CamoufoxConfig): void {
fs.mkdirSync(INSTALL_DIR, { recursive: true });
fs.writeFileSync(CONFIG_FILE, JSON.stringify(config, null, 2));
}
export interface CachedVersion {
version: string;
build: string;
url: string;
is_prerelease?: boolean;
asset_id?: number;
asset_size?: number;
asset_updated_at?: string;
sha256?: string;
created_at?: string;
}
export interface RepoCache {
repos?: Array<{ name: string; versions?: CachedVersion[] }>;
[key: string]: unknown;
}
/**
* Load cached repo data from disk.
*/
export function loadRepoCache(): RepoCache {
if (fs.existsSync(REPO_CACHE_FILE)) {
try {
return JSON.parse(fs.readFileSync(REPO_CACHE_FILE, "utf-8"));
} catch {
// Corrupt cache: treat as absent.
}
}
return {};
}
/**
* Save repo cache to disk.
*/
export function saveRepoCache(cache: RepoCache): void {
fs.mkdirSync(INSTALL_DIR, { recursive: true });
fs.writeFileSync(REPO_CACHE_FILE, JSON.stringify(cache, null, 2));
}
/**
* Keep one cache entry per version-build, the newest by created_at.
*/
export function latestPerBuild(versions: CachedVersion[]): CachedVersion[] {
const best = new Map<string, CachedVersion>();
for (const v of versions) {
const key = `${v.version}-${v.build}`;
const cur = best.get(key);
if (!cur || (v.created_at ?? "") > (cur.created_at ?? "")) {
best.set(key, v);
}
}
return [...best.values()].sort((a, b) => {
const byVersion = cmpStr(b.version, a.version);
if (byVersion !== 0) return byVersion;
return cmpStr(b.created_at ?? "", a.created_at ?? "");
});
}
/**
* Get the display name for a repo from the repo config, lowercased.
*/
export function getRepoName(githubRepo: string): string {
for (const repo of RepoConfig.loadRepos()) {
if (repo.repos.includes(githubRepo)) {
return repo.name.toLowerCase();
}
}
return githubRepo.split("/")[0].toLowerCase();
}
/**
* Information about an installed Camoufox version.
*/
export class InstalledVersion {
repoName: string;
version: Version;
path: string;
isActive: boolean;
isPrerelease: boolean;
assetId?: number;
assetSize?: number;
assetUpdatedAt?: string;
sha256?: string;
createdAt?: string;
constructor(init: {
repoName: string;
version: Version;
path: string;
isActive?: boolean;
isPrerelease?: boolean;
assetId?: number;
assetSize?: number;
assetUpdatedAt?: string;
sha256?: string;
createdAt?: string;
}) {
this.repoName = init.repoName;
this.version = init.version;
this.path = init.path;
this.isActive = init.isActive ?? false;
this.isPrerelease = init.isPrerelease ?? false;
this.assetId = init.assetId;
this.assetSize = init.assetSize;
this.assetUpdatedAt = init.assetUpdatedAt;
this.sha256 = init.sha256;
this.createdAt = init.createdAt;
}
/** Folder name, e.g. 150.0.2-beta.25-8020db3b. */
get folderName(): string {
return path.basename(this.path);
}
/** Path relative to INSTALL_DIR, e.g. browsers/official/150.0.2-beta.25. */
get relativePath(): string {
return `browsers/${this.repoName}/${this.folderName}`;
}
/** Channel display string (like official/stable/134.0.2-beta.20). */
get channelPath(): string {
const ctype = this.isPrerelease ? "prerelease" : "stable";
return `${this.repoName}/${ctype}/${this.version.fullString}`;
}
/**
* Compare with an available version and return change indicators.
*/
getChanges(available: AvailableVersion): string[] {
const changes: string[] = [];
if (this.isPrerelease && !available.isPrerelease) {
changes.push("prerelease -> stable");
} else if (!this.isPrerelease && available.isPrerelease) {
changes.push("stable -> prerelease");
}
if (this.assetUpdatedAt && available.assetUpdatedAt) {
if (this.assetUpdatedAt !== available.assetUpdatedAt) {
changes.push("asset updated");
}
} else if (this.assetSize && available.assetSize) {
if (this.assetSize !== available.assetSize) {
changes.push("asset updated");
}
}
return changes;
}
}
/**
* Install folder name with an optional sha8 suffix.
*/
export function versionFolderName(
version: string,
build: string,
sha8: string = "",
): string {
const base = `${version}-${build}`;
return sha8 ? `${base}-${sha8}` : base;
}
/**
* Get the installed folder for a catalog item. Falls back to version-build/
* (without the sha8) for backwards compatibility.
*/
function matchInstall(
full: string,
sha256: string | undefined,
byFolder: Map<string, InstalledVersion>,
count: number,
): InstalledVersion | null {
const sha8 = (sha256 ?? "").slice(0, 8);
if (sha8) {
const exact = byFolder.get(`${full}-${sha8}`);
if (exact) return exact;
}
const legacy = byFolder.get(full);
if (!legacy) return null;
if (legacy.sha256) {
return legacy.sha256 === sha256 ? legacy : null;
}
return count <= 1 ? legacy : null;
}
/**
* Match each catalog item to an install folder.
* Returns the matches and any orphaned leftovers.
*/
export function classifyInstalls(
versions: AvailableVersion[],
installed: InstalledVersion[],
): [Array<InstalledVersion | null>, Array<[InstalledVersion, string]>] {
const counts = new Map<string, number>();
for (const v of versions) {
const key = v.version.fullString;
counts.set(key, (counts.get(key) ?? 0) + 1);
}
const byFolder = new Map(installed.map((iv) => [iv.folderName, iv]));
const rowInst: Array<InstalledVersion | null> = [];
const matched = new Set<string>();
for (const v of versions) {
const full = v.version.fullString;
const inst = matchInstall(full, v.sha256, byFolder, counts.get(full) ?? 0);
rowInst.push(inst);
if (inst) matched.add(inst.folderName);
}
const extras: Array<[InstalledVersion, string]> = [];
for (const iv of installed) {
if (matched.has(iv.folderName)) continue;
const inCatalog = (counts.get(iv.version.fullString) ?? 0) > 0;
const note = inCatalog && !iv.sha256 ? "date unknown" : "unavailable";
extras.push([iv, note]);
}
return [rowInst, extras];
}
/**
* Installed version for a single version-build and sha; legacy folder allowed.
*/
export function findInstall(
versionBuild: string,
sha256: string | undefined,
installed: InstalledVersion[],
count: number = 1,
): InstalledVersion | null {
return matchInstall(
versionBuild,
sha256,
new Map(installed.map((iv) => [iv.folderName, iv])),
count,
);
}
/**
* Find an installed version by its build string.
*/
export function findInstalledByBuild(
build: string,
repoName?: string,
): InstalledVersion | null {
for (const v of listInstalled()) {
if (v.version.build === build && (!repoName || v.repoName === repoName)) {
return v;
}
}
return null;
}
/**
* Scan browsers/ for installed versions, sorted by repo then version descending.
*/
export function listInstalled(): InstalledVersion[] {
const installed: InstalledVersion[] = [];
const active = loadConfig().active_version;
if (!fs.existsSync(BROWSERS_DIR)) return installed;
for (const repoEntry of fs.readdirSync(BROWSERS_DIR, {
withFileTypes: true,
})) {
if (!repoEntry.isDirectory() || repoEntry.name.startsWith(".")) continue;
const repoDir = path.join(BROWSERS_DIR, repoEntry.name);
for (const versionEntry of fs.readdirSync(repoDir, {
withFileTypes: true,
})) {
if (!versionEntry.isDirectory()) continue;
const versionDir = path.join(repoDir, versionEntry.name);
const versionJson = path.join(versionDir, "version.json");
if (!fs.existsSync(versionJson)) continue;
try {
const ver = Version.fromPath(versionDir);
const versionData = JSON.parse(fs.readFileSync(versionJson, "utf-8"));
const relPath = `browsers/${repoEntry.name}/${versionEntry.name}`;
installed.push(
new InstalledVersion({
repoName: repoEntry.name,
version: ver,
path: versionDir,
isActive: relPath === active,
isPrerelease: versionData.prerelease ?? false,
assetId: versionData.asset_id,
assetSize: versionData.asset_size,
assetUpdatedAt: versionData.asset_updated_at,
sha256: versionData.sha256,
createdAt: versionData.created_at,
}),
);
} catch {
// Missing/corrupt version.json: not an install we can use.
}
}
}
installed.sort((a, b) => {
const byRepo = cmpStr(b.repoName, a.repoName);
if (byRepo !== 0) return byRepo;
return b.version.compare(a.version);
});
return installed;
}
/**
* Get the path to the active version, or null if no version is active.
*/
export function getActivePath(): string | null {
const config = loadConfig();
const active = config.active_version;
if (active) {
const activePath = path.join(INSTALL_DIR, active);
if (
fs.existsSync(activePath) &&
fs.existsSync(path.join(activePath, "version.json"))
) {
return activePath;
}
}
// Only auto-select if the user didn't set a channel or pin
if (!config.channel && !config.pinned) {
const installed = listInstalled();
if (installed.length) {
config.active_version = installed[0].relativePath;
saveConfig(config);
return installed[0].path;
}
}
return null;
}
/**
* Set the active version by its relative path.
*/
export function setActive(relativePath: string): void {
const config = loadConfig();
config.active_version = relativePath;
saveConfig(config);
}
/**
* Find an installed version by path, build, full version, or repo/build.
*/
export function findInstalledVersion(specifier: string): string | null {
const installed = listInstalled();
if (!installed.length) return null;
const lower = specifier.toLowerCase();
for (const v of installed) {
if (
v.relativePath === specifier ||
v.relativePath === `browsers/${specifier}`
) {
return v.path;
}
if (
`browsers/${v.repoName}/${v.version.fullString}`.endsWith(specifier) ||
`${v.repoName}/${v.version.build}`.toLowerCase() === lower ||
v.version.build.toLowerCase() === lower ||
v.version.fullString.toLowerCase() === lower ||
v.version.version?.toLowerCase() === lower
) {
return v.path;
}
}
return null;
}
/**
* Install to browsers/{repoName}/{version}-{build}-{sha8}; the suffix is
* omitted when the release carries no sha.
*/
export async function installVersioned(
fetcher: CamoufoxFetcher,
replace: boolean = false,
): Promise<boolean> {
const repoName = getRepoName(fetcher.githubRepo);
const sha8 = fetcher._selectedVersion?.sha256
? fetcher._selectedVersion.sha8
: fetcher.installedSha8;
const versionFolder = versionFolderName(fetcher.version, fetcher.build, sha8);
const installPath = path.join(BROWSERS_DIR, repoName, versionFolder);
if (
fs.existsSync(installPath) &&
fs.existsSync(path.join(installPath, "version.json"))
) {
if (!replace) {
const installedV = findInstalledByBuild(fetcher.build, repoName);
let changeMsg = "";
if (installedV && fetcher._selectedVersion) {
const changes = installedV.getChanges(fetcher._selectedVersion);
if (changes.length) changeMsg = ` (${changes.join(", ")})`;
}
rprint(
`Version v${fetcher.verstr} already installed${changeMsg}.`,
"yellow",
);
rprint(
changeMsg
? "Use --replace to update with the new release."
: "Use --replace to reinstall.",
"yellow",
);
if (!loadConfig().active_version) {
setActive(`browsers/${repoName}/${versionFolder}`);
}
return false;
}
rprint(`Replacing: ${installPath}`, "yellow");
fs.rmSync(installPath, { recursive: true, force: true });
}
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "camoufox-"));
const tempFilePath = path.join(tempDir, "camoufox.zip");
try {
fs.mkdirSync(installPath, { recursive: true });
const tempFileStream = fs.createWriteStream(tempFilePath);
// finished() listens for the stream's 'error' from the start. Without a
// listener, a failed write (disk full) is an uncaught 'error' event that
// kills the process before the catch below can remove the partial
// install.
const written = finished(tempFileStream);
written.catch(() => {}); // awaited below; a download error wins
try {
// Through the instance's class, as Python's fetcher.download_file(),
// so a subclass can override it.
await (fetcher.constructor as typeof CamoufoxFetcher).downloadFile(
tempFileStream,
fetcher.url,
);
} finally {
tempFileStream.end();
}
await written;
const expectedSha = fetcher._selectedVersion
? fetcher._selectedVersion.sha256
: fetcher.installedSha256;
verifySha256(tempFilePath, expectedSha, `Camoufox v${fetcher.verstr}`);
rprint(`Extracting Camoufox: ${installPath}`);
unzip(tempFilePath, installPath);
const metadata = fetcher._selectedVersion
? fetcher._selectedVersion.toMetadata()
: {
version: fetcher.version,
build: fetcher.build,
prerelease: fetcher.isPrerelease,
sha256: fetcher.installedSha256 ?? null,
created_at: fetcher.installedCreatedAt ?? null,
};
fs.writeFileSync(
path.join(installPath, "version.json"),
JSON.stringify(metadata),
);
if (OS_NAME !== "win") {
makeExecutable(installPath);
}
setActive(`browsers/${repoName}/${versionFolder}`);
// Mark the install dir as compatible with this version
fs.writeFileSync(COMPAT_FLAG, "");
rprint(`\nCamoufox v${fetcher.verstr} installed.`, "green");
rprint(`Path: ${installPath}`, "green");
return true;
} catch (error) {
rprint(`Error: ${error}`, "red");
if (fs.existsSync(installPath)) {
fs.rmSync(installPath, { recursive: true, force: true });
}
throw error;
} finally {
fs.rmSync(tempDir, { recursive: true, force: true });
}
}
/**
* Remove a specific version installation.
*/
export function removeVersion(versionPath: string): boolean {
if (!fs.existsSync(versionPath)) return false;
rprint(`Removing: ${versionPath}`);
fs.rmSync(versionPath, { recursive: true, force: true });
const parent = path.dirname(versionPath);
if (
fs.existsSync(parent) &&
parent !== BROWSERS_DIR &&
fs.readdirSync(parent).length === 0
) {
fs.rmdirSync(parent);
}
if (
fs.existsSync(BROWSERS_DIR) &&
fs.readdirSync(BROWSERS_DIR).length === 0
) {
fs.rmdirSync(BROWSERS_DIR);
}
const config = loadConfig();
const relPath = path.relative(INSTALL_DIR, versionPath);
if (!relPath.startsWith("..") && config.active_version === relPath) {
const remaining = listInstalled();
config.active_version = remaining.length ? remaining[0].relativePath : null;
saveConfig(config);
}
return true;
}
/**
* Short tag to tell coexisting installs apart: date, else sha8.
*/
export function installedLabel(iv: InstalledVersion): string {
if (iv.createdAt) {
const date = formatAssetDate(iv.createdAt);
if (date) return date;
}
return (iv.sha256 ?? "").slice(0, 8);
}
/**
* Print installed versions as a tree.
*/
export function printTree(
showHeader: boolean = true,
showPaths: boolean = false,
): void {
const installed = listInstalled();
if (!installed.length) {
rprint("No versions installed.", "yellow");
rprint("Run `camoufox fetch` to install.", "yellow");
return;
}
if (showHeader) {
rprint("Installed versions:\n", "yellow");
}
let currentRepo: string | null = null;
for (let i = 0; i < installed.length; i++) {
const v = installed[i];
const isLast =
i === installed.length - 1 || installed[i + 1].repoName !== v.repoName;
if (v.repoName !== currentRepo) {
currentRepo = v.repoName;
rprint(`${currentRepo}/`, "cyan", false);
if (showPaths) {
rprint(` -> ${path.join(BROWSERS_DIR, currentRepo)}`, "bright_black");
} else {
rprint("");
}
}
rprint(` ${isLast ? "└── " : "├── "}`, undefined, false);
rprint(`v${v.version.fullString}`, v.isActive ? "green" : undefined, false);
rprint(
v.isPrerelease ? " (prerelease)" : " (stable)",
v.isPrerelease ? "yellow" : "blue",
false,
);
const tag = installedLabel(v);
if (tag) rprint(` (${tag})`, "bright_black", false);
if (v.isActive) rprint(" (active)", "green", false);
rprint("");
}
}
typescript/src/paths.ts
+129
View File
@@ -0,0 +1,129 @@
/**
* Platform constants and install-directory paths.
*
* These are the leaves of the module graph: pkgman.ts and multiversion.ts are
* mutually dependent (as their Python twins are, via function-local imports),
* and both need these values *at module-evaluation time*. Keeping them here
* means the cycle only ever involves function bodies, which ESM resolves
* cleanly. Everything is re-exported from pkgman.ts, which stays the public
* entry point for them.
*/
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { fileURLToPath } from "node:url";
import { UnsupportedOS } from "./exceptions.js";
export const ARCH_MAP: Record<string, string> = {
x64: "x86_64",
amd64: "x86_64",
x86: "x86_64",
ia32: "i686",
i686: "i686",
i386: "i686",
arm64: "arm64",
aarch64: "arm64",
arm: "arm64",
};
export const OS_MAP: Record<string, "mac" | "win" | "lin"> = {
darwin: "mac",
linux: "lin",
win32: "win",
};
if (!(process.platform in OS_MAP)) {
throw new UnsupportedOS(`OS ${process.platform} is not supported`);
}
export const OS_NAME: "mac" | "win" | "lin" = OS_MAP[process.platform];
const currentDir =
import.meta.dirname ?? path.dirname(fileURLToPath(import.meta.url));
/**
* platformdirs' user_cache_dir(appName), reimplemented so the TS and Python
* launchers share one install directory. Hardcoding ~/.cache would diverge on
* hosts that set XDG_CACHE_HOME, and on macOS/Windows entirely.
*/
export function userCacheDir(appName: string): string {
if (OS_NAME === "win") {
const localAppData = process.env.LOCALAPPDATA;
const base =
localAppData && path.isAbsolute(localAppData)
? localAppData
: path.join(os.homedir(), "AppData", "Local");
return path.join(base, appName, appName, "Cache");
}
if (OS_NAME === "mac") {
return path.join(os.homedir(), "Library", "Caches", appName);
}
// platformdirs: any non-blank XDG_CACHE_HOME is taken as-is.
const xdg = process.env.XDG_CACHE_HOME ?? "";
const base = xdg.trim() ? xdg : path.join(os.homedir(), ".cache");
return path.join(base, appName);
}
export const INSTALL_DIR: string = userCacheDir("camoufox");
/**
* The data files both launchers read. They live in pythonlib/camoufox/, the one
* copy in the repo; `pnpm build` copies them into dist/data-files/ for the npm
* tarball, so one seed draws one identity in either launcher.
*/
export const DATA_FILES: readonly string[] = [
"essential-fonts.json",
"fingerprint-presets.json",
"fingerprint-presets-v150.json",
"font-bases.json",
"font-groups.json",
"fonts.json",
"media-devices.json",
"repos.yml",
"territoryInfo.xml",
"voice-manifests.json",
"voice-uris.json",
"warnings.yml",
];
/**
* Where DATA_FILES are read from: dist/data-files/ in a built package, else
* pythonlib's copies when running from src/ in the repo.
*/
export const LOCAL_DATA: string = fs.existsSync(
path.join(currentDir, "data-files"),
)
? path.join(currentDir, "data-files")
: path.resolve(currentDir, "..", "..", "pythonlib", "camoufox");
export const OS_ARCH_MATRIX: Record<string, string[]> = {
win: ["x86_64", "i686"],
mac: ["x86_64", "arm64"],
lin: ["x86_64", "arm64", "i686"],
};
export const LAUNCH_FILE: Record<string, string> = {
win: "camoufox.exe",
mac: "../MacOS/camoufox",
lin: "camoufox-bin",
};
const COLORS: Record<string, string> = {
red: "\x1b[31m",
green: "\x1b[32m",
yellow: "\x1b[33m",
blue: "\x1b[34m",
cyan: "\x1b[36m",
bright_black: "\x1b[90m",
};
/**
* Print a styled message. The Python twin uses `rich`; keeping the same helper
* name means the ported call sites read identically.
*/
export function rprint(msg: string, fg?: string, nl: boolean = true): void {
const useColor = process.stdout.isTTY && !process.env.NO_COLOR;
const prefix = useColor ? `\x1b[1m${(fg && COLORS[fg]) || ""}` : "";
const suffix = useColor ? "\x1b[0m" : "";
process.stdout.write(`${prefix}${msg}${suffix}${nl ? "\n" : ""}`);
}
typescript/src/pkgman.ts
+1281
View File
File diff suppressed because it is too large Load Diff
typescript/src/pycompat.ts
+413
View File
@@ -0,0 +1,413 @@
/**
* The handful of CPython / orjson behaviours the identity layer has to
* reproduce bit-for-bit so that the same inputs yield the same identity in the
* Python and TypeScript launchers: orjson's JSON bytes (hashed by
* identity_salt), str() of a value (hashed by identity_seed), zlib.crc32,
* code-point string ordering, and the float summation builtin sum() uses.
*
* JavaScript has one number type where Python has two, so `1.0` and `1` are
* the same value here. Where the difference reaches a hash, wrap the float in
* {@link PyFloat}, or read the JSON with {@link parsePyJson}, which keeps the
* distinction (and keeps integers above 2**53 exact as bigint).
*/
import { FileNotFoundError } from "./exceptions.js";
/** A Python float whose value happens to be integral (1.0, 3.4e38...). */
export class PyFloat {
constructor(public readonly value: number) {}
valueOf(): number {
return this.value;
}
toJSON(): number {
return this.value;
}
toString(): string {
return formatPyFloatRepr(this.value);
}
}
/** Number of a value that may be a PyFloat. */
export function num(value: unknown): number {
return value instanceof PyFloat ? value.value : (value as number);
}
/** Whether `value` is a Python int (bool counts, as in Python). */
export function isPyInt(value: unknown): boolean {
return (
typeof value === "bigint" ||
typeof value === "boolean" ||
(typeof value === "number" && Number.isSafeInteger(value))
);
}
/**
* JSON.parse that keeps what Python's json.loads would: `1.0` stays a float
* (a {@link PyFloat} when integral) and an integer beyond 2**53 stays exact
* (a bigint).
*/
export function parsePyJson(text: string): any {
return JSON.parse(text, function (
this: any,
_key: string,
value: any,
context?: { source?: string },
) {
if (typeof value !== "number" || !context?.source) return value;
const src = context.source;
const isFloat = /[.eE]/.test(src);
if (!isFloat) {
return Number.isSafeInteger(value) ? value : BigInt(src);
}
return Number.isSafeInteger(value) ? new PyFloat(value) : value;
} as any);
}
/** Shortest round-trip digits and decimal exponent of a finite, non-zero x. */
function shortestDigits(x: number): { digits: string; exp: number } {
const [mantissa, exponent] = Math.abs(x).toExponential().split("e");
return { digits: mantissa.replace(".", ""), exp: Number(exponent) };
}
function fixedNotation(digits: string, exp: number): string {
if (exp >= 0) {
if (digits.length <= exp + 1) {
return `${digits}${"0".repeat(exp + 1 - digits.length)}.0`;
}
return `${digits.slice(0, exp + 1)}.${digits.slice(exp + 1)}`;
}
return `0.${"0".repeat(-exp - 1)}${digits}`;
}
function sciMantissa(digits: string): string {
return digits.length === 1 ? digits : `${digits[0]}.${digits.slice(1)}`;
}
/** A float exactly as orjson serializes it. */
export function formatOrjsonFloat(x: number): string {
if (!Number.isFinite(x)) return "null";
if (x === 0) return Object.is(x, -0) ? "-0.0" : "0.0";
const sign = x < 0 ? "-" : "";
const { digits, exp } = shortestDigits(x);
if (exp >= -5 && exp < 16) return sign + fixedNotation(digits, exp);
return `${sign}${sciMantissa(digits)}e${exp < 0 ? "-" : "+"}${Math.abs(exp)}`;
}
/** repr(float) / str(float) in CPython. */
export function formatPyFloatRepr(x: number): string {
if (Number.isNaN(x)) return "nan";
if (!Number.isFinite(x)) return x > 0 ? "inf" : "-inf";
if (x === 0) return Object.is(x, -0) ? "-0.0" : "0.0";
const sign = x < 0 ? "-" : "";
const { digits, exp } = shortestDigits(x);
if (exp >= -4 && exp < 16) return sign + fixedNotation(digits, exp);
const e = Math.abs(exp).toString().padStart(2, "0");
return `${sign}${sciMantissa(digits)}e${exp < 0 ? "-" : "+"}${e}`;
}
/** Compare two strings by code point, as Python (and UTF-8 bytes) order them. */
export function comparePyStr(a: string, b: string): number {
if (a === b) return 0;
const n = Math.min(a.length, b.length);
for (let i = 0; i < n; i++) {
const ca = a.charCodeAt(i);
const cb = b.charCodeAt(i);
if (ca === cb) continue;
// Surrogates (0xD800-0xDFFF) encode code points above every other
// BMP unit, so only the surrogate-vs-high-BMP case reorders.
const sa = ca >= 0xd800 && ca <= 0xdfff;
const sb = cb >= 0xd800 && cb <= 0xdfff;
if (sa !== sb) {
if (sa) return cb >= 0xe000 ? 1 : ca - cb;
return ca >= 0xe000 ? -1 : ca - cb;
}
return ca - cb;
}
return a.length - b.length;
}
/** Python truthiness of a JSON-like value. */
export function pyTruthy(value: unknown): boolean {
if (value === null || value === undefined) return false;
if (value instanceof PyFloat) return value.value !== 0;
if (typeof value === "number") return value !== 0;
if (typeof value === "bigint") return value !== 0n;
if (typeof value === "string") return value.length > 0;
if (typeof value === "boolean") return value;
if (Array.isArray(value)) return value.length > 0;
if (value instanceof Map || value instanceof Set) return value.size > 0;
if (typeof value === "object") return Object.keys(value).length > 0;
return true;
}
const NON_PRINTABLE = /[\p{C}\p{Z}]/u;
/** repr() of a str. */
export function pyStrRepr(s: string): string {
const quote = s.includes("'") && !s.includes('"') ? '"' : "'";
let out = quote;
for (const ch of s) {
const cp = ch.codePointAt(0) as number;
if (ch === quote || ch === "\\") out += `\\${ch}`;
else if (ch === "\t") out += "\\t";
else if (ch === "\n") out += "\\n";
else if (ch === "\r") out += "\\r";
else if (cp < 0x20 || cp === 0x7f)
out += `\\x${cp.toString(16).padStart(2, "0")}`;
else if (ch !== " " && NON_PRINTABLE.test(ch)) {
if (cp <= 0xff) out += `\\x${cp.toString(16).padStart(2, "0")}`;
else if (cp <= 0xffff) out += `\\u${cp.toString(16).padStart(4, "0")}`;
else out += `\\U${cp.toString(16).padStart(8, "0")}`;
} else out += ch;
}
return out + quote;
}
function pyNumberStr(value: number): string {
if (Number.isSafeInteger(value)) return String(value);
return formatPyFloatRepr(value);
}
/** repr() of a JSON-like value. */
export function pyRepr(value: unknown): string {
if (typeof value === "string") return pyStrRepr(value);
if (Array.isArray(value)) return `[${value.map(pyRepr).join(", ")}]`;
if (value instanceof PyFloat) return formatPyFloatRepr(value.value);
if (value !== null && typeof value === "object") {
const entries =
value instanceof Map ? [...value.entries()] : Object.entries(value);
return `{${entries.map(([k, v]) => `${pyRepr(k)}: ${pyRepr(v)}`).join(", ")}}`;
}
return pyStr(value);
}
/** str() of a JSON-like value. */
export function pyStr(value: unknown): string {
if (value === null || value === undefined) return "None";
if (value === true) return "True";
if (value === false) return "False";
if (typeof value === "string") return value;
if (typeof value === "bigint") return value.toString();
if (typeof value === "number") return pyNumberStr(value);
if (value instanceof PyFloat) return formatPyFloatRepr(value.value);
return pyRepr(value);
}
function orjsonString(s: string): string {
let out = '"';
for (let i = 0; i < s.length; i++) {
const c = s.charCodeAt(i);
if (c === 0x22) out += '\\"';
else if (c === 0x5c) out += "\\\\";
else if (c < 0x20) {
switch (c) {
case 0x08:
out += "\\b";
break;
case 0x09:
out += "\\t";
break;
case 0x0a:
out += "\\n";
break;
case 0x0c:
out += "\\f";
break;
case 0x0d:
out += "\\r";
break;
default:
out += `\\u${c.toString(16).padStart(4, "0")}`;
}
} else out += s[i];
}
return `${out}"`;
}
function orjsonKey(key: unknown): string {
if (typeof key === "string") return key;
if (key === null || key === undefined) return "null";
if (typeof key === "boolean") return key ? "true" : "false";
if (typeof key === "bigint") return key.toString();
if (key instanceof PyFloat) return formatOrjsonFloat(key.value);
if (typeof key === "number") {
return Number.isSafeInteger(key) ? String(key) : formatOrjsonFloat(key);
}
return pyStr(key);
}
/**
* orjson.dumps(value, option=OPT_SORT_KEYS | OPT_NON_STR_KEYS, default=str),
* as a string (its UTF-8 encoding is orjson's bytes).
*
* Plain numbers that are safe integers serialize as Python ints, every other
* number as a float; use {@link PyFloat} for an integral float and bigint for
* an integer beyond 2**53. An object with a `toPyDict()` method (the TS twin
* of a dataclass) serializes as that dict.
*/
export function orjsonDumps(value: unknown, sortKeys = true): string {
if (value === null || value === undefined) return "null";
if (value === true) return "true";
if (value === false) return "false";
if (typeof value === "string") return orjsonString(value);
if (typeof value === "bigint") return value.toString();
if (typeof value === "number") {
return Number.isSafeInteger(value)
? String(value)
: formatOrjsonFloat(value);
}
if (value instanceof PyFloat) return formatOrjsonFloat(value.value);
if (Array.isArray(value)) {
return `[${value.map((v) => orjsonDumps(v, sortKeys)).join(",")}]`;
}
if (typeof value === "object") {
const withDict = value as { toPyDict?: () => unknown };
if (typeof withDict.toPyDict === "function") {
return orjsonDumps(withDict.toPyDict(), sortKeys);
}
if (value instanceof Set) {
return orjsonString(pyStr([...value]));
}
let entries: Array<[string, unknown]>;
if (value instanceof Map) {
entries = [...value.entries()].map(([k, v]) => [orjsonKey(k), v]);
} else if (
Object.getPrototypeOf(value) === Object.prototype ||
Object.getPrototypeOf(value) === null
) {
entries = Object.entries(value);
} else {
// default=str
return orjsonString(String(value));
}
if (sortKeys) entries.sort((a, b) => comparePyStr(a[0], b[0]));
return `{${entries.map(([k, v]) => `${orjsonString(k)}:${orjsonDumps(v, sortKeys)}`).join(",")}}`;
}
return orjsonString(String(value));
}
/**
* sum() over floats, as CPython 3.12+ computes it (Neumaier-compensated).
* CPython 3.10/3.11 fold left without compensation; the two differ only in
* the last bit, and only for sums that lose precision.
*/
export function pySumFloats(values: Iterable<number>): number {
let started = false;
let f = 0;
let c = 0;
for (const x of values) {
if (!started) {
// int 0 start + first float: a plain add
f = 0 + x;
started = true;
continue;
}
const t = f + x;
if (Math.abs(f) >= Math.abs(x)) c += f - t + x;
else c += x - t + f;
f = t;
}
if (c !== 0 && Number.isFinite(c)) f += c;
return f;
}
/**
* sum() over a mix of ints and floats, as CPython 3.14 computes it: ints add
* exactly until the first float, which is added plainly; after that every item
* is Neumaier-compensated. (3.12/3.13 add those later ints without
* compensation and 3.10/3.11 compensate nothing, so a sum that loses
* precision can differ in its last bit there.) A safe
* integer number counts as an int here, anything else (or a PyFloat) as a
* float. Returns a number, or a bigint when every item was an int that only
* a bigint holds exactly.
*/
export function pySum(values: Iterable<unknown>): number | bigint {
let i: bigint | null = 0n;
let f = 0;
let c = 0;
for (const v of values) {
const isInt =
typeof v === "bigint" ||
typeof v === "boolean" ||
(typeof v === "number" && Number.isSafeInteger(v));
if (i !== null) {
if (isInt) {
i += typeof v === "bigint" ? v : BigInt(Number(v));
continue;
}
f = Number(i) + num(v);
i = null;
continue;
}
const x = isInt ? Number(v) : num(v);
const t = f + x;
if (Math.abs(f) >= Math.abs(x)) c += f - t + x;
else c += x - t + f;
f = t;
}
if (i !== null) return Number.isSafeInteger(Number(i)) ? Number(i) : i;
if (c !== 0 && Number.isFinite(c)) f += c;
return f;
}
let CRC_TABLE: Uint32Array | null = null;
/** zlib.crc32. */
export function crc32(data: Uint8Array | string): number {
if (!CRC_TABLE) {
CRC_TABLE = new Uint32Array(256);
for (let n = 0; n < 256; n++) {
let c = n;
for (let k = 0; k < 8; k++) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
CRC_TABLE[n] = c >>> 0;
}
}
const bytes =
typeof data === "string" ? new TextEncoder().encode(data) : data;
let crc = 0xffffffff;
for (let i = 0; i < bytes.length; i++) {
crc = CRC_TABLE[(crc ^ bytes[i]) & 0xff] ^ (crc >>> 8);
}
return (crc ^ 0xffffffff) >>> 0;
}
/** Python's OSError, for failures Node does not raise as a system error. */
export class OSError extends Error {
name = "OSError";
}
/** Python's ValueError. */
export class ValueError extends Error {
name = "ValueError";
}
/** Python's KeyError. */
export class KeyError extends Error {
name = "KeyError";
}
const PY_ERROR_TESTS = {
// Node's system errors (fs, net) carry a string code and a numeric errno;
// FileNotFoundError is an OSError in Python.
OSError: (e: unknown) =>
e instanceof OSError ||
e instanceof FileNotFoundError ||
(e instanceof Error &&
typeof (e as NodeJS.ErrnoException).code === "string" &&
typeof (e as NodeJS.ErrnoException).errno === "number"),
// JSON.parse's SyntaxError is json.JSONDecodeError, a ValueError.
ValueError: (e: unknown) =>
e instanceof ValueError || e instanceof SyntaxError,
KeyError: (e: unknown) => e instanceof KeyError,
};
/** Whether `except (<kinds>)` in the Python twin would catch `error`. */
export function isPyError(
error: unknown,
...kinds: Array<keyof typeof PY_ERROR_TESTS>
): boolean {
return kinds.some((kind) => PY_ERROR_TESTS[kind](error));
}
typescript/src/pyrandom.ts
+323
View File
@@ -0,0 +1,323 @@
/**
* A bit-exact port of CPython's `random.Random` (MT19937).
*
* The Python launcher seeds every per-identity draw (fonts, voices, media
* devices) with `random.Random(seed)`, so the same identity must draw the same
* values here. Seeding follows `random_seed()` in Modules/_randommodule.c
* (init_by_array over the 32-bit words of abs(seed)); the methods follow
* Lib/random.py. Every method below returns the same values on CPython 3.10
* through 3.14 (checked by the golden fixtures).
*/
import { createHash, randomBytes } from "node:crypto";
const N = 624;
const M = 397;
const MATRIX_A = 0x9908b0df;
const UPPER_MASK = 0x80000000;
const LOWER_MASK = 0x7fffffff;
export type PySeed = number | bigint | string | null | undefined;
function seedToKey(seed: number | bigint): Uint32Array {
let n = typeof seed === "bigint" ? seed : BigInt(seed);
if (n < 0n) n = -n;
const words: number[] = [];
if (n === 0n) words.push(0);
while (n > 0n) {
words.push(Number(n & 0xffffffffn));
n >>= 32n;
}
return Uint32Array.from(words);
}
export class PyRandom {
private mt = new Uint32Array(N);
private mti = N + 1;
constructor(seed?: PySeed) {
this.seed(seed);
}
/**
* random.seed(a): an int (number/bigint), a str (version 2: the str's
* UTF-8 bytes followed by their SHA-512), or None for OS entropy.
*/
seed(a?: PySeed): void {
if (a === null || a === undefined) {
const bytes = randomBytes(N * 4);
const key = new Uint32Array(N);
for (let i = 0; i < N; i++) key[i] = bytes.readUInt32LE(i * 4);
this.initByArray(key);
return;
}
if (typeof a === "string") {
const utf8 = Buffer.from(a, "utf-8");
const digest = createHash("sha512").update(utf8).digest();
const all = Buffer.concat([utf8, digest]);
a = all.length ? BigInt(`0x${all.toString("hex")}`) : 0n;
}
if (typeof a === "number" && !Number.isInteger(a)) {
throw new TypeError("PyRandom only seeds from integers and strings");
}
this.initByArray(seedToKey(a));
}
private initGenrand(s: number): void {
const mt = this.mt;
mt[0] = s >>> 0;
for (let i = 1; i < N; i++) {
const prev = mt[i - 1] ^ (mt[i - 1] >>> 30);
mt[i] = (Math.imul(1812433253, prev) + i) >>> 0;
}
this.mti = N;
}
private initByArray(key: Uint32Array): void {
const mt = this.mt;
this.initGenrand(19650218);
let i = 1;
let j = 0;
const keyLength = key.length;
for (let k = Math.max(N, keyLength); k; k--) {
const prev = mt[i - 1] ^ (mt[i - 1] >>> 30);
mt[i] = ((mt[i] ^ Math.imul(prev, 1664525)) + key[j] + j) >>> 0;
i++;
j++;
if (i >= N) {
mt[0] = mt[N - 1];
i = 1;
}
if (j >= keyLength) j = 0;
}
for (let k = N - 1; k; k--) {
const prev = mt[i - 1] ^ (mt[i - 1] >>> 30);
mt[i] = ((mt[i] ^ Math.imul(prev, 1566083941)) - i) >>> 0;
i++;
if (i >= N) {
mt[0] = mt[N - 1];
i = 1;
}
}
mt[0] = 0x80000000;
this.mti = N;
}
/** genrand_uint32 */
genrandUint32(): number {
const mt = this.mt;
let y: number;
if (this.mti >= N) {
let kk = 0;
for (; kk < N - M; kk++) {
y = (mt[kk] & UPPER_MASK) | (mt[kk + 1] & LOWER_MASK);
mt[kk] = mt[kk + M] ^ (y >>> 1) ^ (y & 1 ? MATRIX_A : 0);
}
for (; kk < N - 1; kk++) {
y = (mt[kk] & UPPER_MASK) | (mt[kk + 1] & LOWER_MASK);
mt[kk] = mt[kk + (M - N)] ^ (y >>> 1) ^ (y & 1 ? MATRIX_A : 0);
}
y = (mt[N - 1] & UPPER_MASK) | (mt[0] & LOWER_MASK);
mt[N - 1] = mt[M - 1] ^ (y >>> 1) ^ (y & 1 ? MATRIX_A : 0);
this.mti = 0;
}
y = mt[this.mti++];
y ^= y >>> 11;
y ^= (y << 7) & 0x9d2c5680;
y ^= (y << 15) & 0xefc60000;
y ^= y >>> 18;
return y >>> 0;
}
/** random.random(): a float in [0, 1) with 53 random bits. */
random(): number {
const a = this.genrandUint32() >>> 5;
const b = this.genrandUint32() >>> 6;
return (a * 67108864.0 + b) * (1.0 / 9007199254740992.0);
}
/** random.getrandbits(k) as a bigint (any k >= 0). */
getrandbitsBig(k: number): bigint {
if (k < 0) throw new RangeError("number of bits must be non-negative");
if (k === 0) return 0n;
if (k <= 32) return BigInt(this.genrandUint32() >>> (32 - k));
let result = 0n;
let shift = 0n;
for (let left = k; left > 0; left -= 32) {
let r = this.genrandUint32();
if (left < 32) r >>>= 32 - left;
result |= BigInt(r) << shift;
shift += 32n;
}
return result;
}
/** random.getrandbits(k) as a number (k <= 53). */
getrandbits(k: number): number {
if (k < 0) throw new RangeError("number of bits must be non-negative");
if (k === 0) return 0;
if (k <= 32) return this.genrandUint32() >>> (32 - k);
if (k > 53)
throw new RangeError("use getrandbitsBig for more than 53 bits");
const lo = this.genrandUint32();
const hi = this.genrandUint32() >>> (64 - k);
return hi * 4294967296 + lo;
}
/** random._randbelow(n): an int in [0, n). */
randbelow(n: number): number {
if (n <= 0) return 0;
const k = bitLength(n);
let r = this.getrandbits(k);
while (r >= n) r = this.getrandbits(k);
return r;
}
/** random.randrange(start[, stop[, step]]) */
randrange(start: number, stop?: number, step = 1): number {
if (
!Number.isInteger(start) ||
(stop !== undefined && !Number.isInteger(stop)) ||
!Number.isInteger(step)
) {
throw new TypeError("randrange() arguments must be integers");
}
if (stop === undefined) {
if (step !== 1) throw new TypeError("Missing a non-None stop argument");
if (start > 0) return this.randbelow(start);
throw new RangeError("empty range for randrange()");
}
const width = stop - start;
if (step === 1) {
if (width > 0) return start + this.randbelow(width);
throw new RangeError(`empty range in randrange(${start}, ${stop})`);
}
let n: number;
if (step > 0) n = Math.floor((width + step - 1) / step);
else if (step < 0) n = Math.floor((width + step + 1) / step);
else throw new RangeError("zero step for randrange()");
if (n <= 0)
throw new RangeError(
`empty range in randrange(${start}, ${stop}, ${step})`,
);
return start + step * this.randbelow(n);
}
/** random.randint(a, b): an int in [a, b]. */
randint(a: number, b: number): number {
return this.randrange(a, b + 1);
}
/** random.choice(seq) */
choice<T>(seq: ArrayLike<T>): T {
if (!seq.length)
throw new RangeError("Cannot choose from an empty sequence");
return seq[this.randbelow(seq.length)];
}
/** random.choices(population, weights=None, *, cum_weights=None, k=1) */
choices<T>(
population: ArrayLike<T>,
{
weights,
cumWeights,
k = 1,
}: { weights?: number[]; cumWeights?: number[]; k?: number } = {},
): T[] {
const n = population.length;
const out: T[] = [];
if (!cumWeights) {
if (!weights) {
for (let i = 0; i < k; i++)
out.push(population[Math.floor(this.random() * n)]);
return out;
}
cumWeights = [];
let acc = 0;
weights.forEach((w, i) => {
acc = i === 0 ? w : acc + w;
(cumWeights as number[]).push(acc);
});
} else if (weights) {
throw new TypeError("Cannot specify both weights and cumulative weights");
}
if (cumWeights.length !== n) {
throw new RangeError(
"The number of weights does not match the population",
);
}
const total = cumWeights[n - 1] + 0.0;
if (total <= 0.0)
throw new RangeError("Total of weights must be greater than zero");
if (!Number.isFinite(total))
throw new RangeError("Total of weights must be finite");
const hi = n - 1;
for (let i = 0; i < k; i++) {
out.push(
population[bisectRight(cumWeights, this.random() * total, 0, hi)],
);
}
return out;
}
/** random.shuffle(x), in place. */
shuffle<T>(x: T[]): void {
for (let i = x.length - 1; i > 0; i--) {
const j = this.randbelow(i + 1);
[x[i], x[j]] = [x[j], x[i]];
}
}
/** random.sample(population, k), including its pool-vs-set branch. */
sample<T>(population: ArrayLike<T>, k: number): T[] {
const n = population.length;
if (!(k >= 0 && k <= n)) {
throw new RangeError("Sample larger than population or is negative");
}
const result: T[] = new Array(k);
let setsize = 21;
if (k > 5) setsize += 4 ** Math.ceil(Math.log(k * 3) / Math.log(4));
if (n <= setsize) {
const pool = Array.from(population);
for (let i = 0; i < k; i++) {
const j = this.randbelow(n - i);
result[i] = pool[j];
pool[j] = pool[n - i - 1];
}
} else {
const selected = new Set<number>();
for (let i = 0; i < k; i++) {
let j = this.randbelow(n);
while (selected.has(j)) j = this.randbelow(n);
selected.add(j);
result[i] = population[j];
}
}
return result;
}
/** random.uniform(a, b) */
uniform(a: number, b: number): number {
return a + (b - a) * this.random();
}
}
function bitLength(n: number): number {
return n === 0 ? 0 : Math.trunc(Math.abs(n)).toString(2).length;
}
function bisectRight(a: number[], x: number, lo: number, hi: number): number {
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (x < a[mid]) hi = mid;
else lo = mid + 1;
}
return lo;
}
/**
* The module-level generator: Python's `random.random()`, `random.choice()`
* and friends all draw from one shared `Random` instance, and so do the TS
* functions that mirror an unseeded Python draw. Seed it (`pyRandom.seed(n)`)
* to reproduce a Python run that called `random.seed(n)`.
*/
export const pyRandom = new PyRandom();
typescript/src/server.ts
+94
View File
@@ -0,0 +1,94 @@
/**
* Playwright server mode.
*
* TypeScript twin of pythonlib/camoufox/server.py. Python has to shell out to
* the Node runtime bundled with its Playwright driver (and hand it a base64
* config frame over stdin, via launchServer.js) because there is no Python
* binding for BrowserServer. Here we already are that runtime, so this calls
* playwright-core's launchServer() directly with the same options Python
* would send: launchOptions() with every top-level key camelCased.
*/
import { type BrowserServer, firefox } from "playwright-core";
import { withUnpinnedLaunch } from "./cpu_affinity.js";
import { camelCase } from "./sync_api.js";
import { type LaunchOptions, launchOptions } from "./utils.js";
import { VirtualDisplay } from "./virtdisplay.js";
export interface LaunchServerOptions extends Omit<LaunchOptions, "headless"> {
/** Port to listen on. Defaults to a random free port. */
port?: number;
/** Path of the websocket endpoint. Defaults to a random path. */
ws_path?: string;
/** Whether to run the browser headless. `"virtual"` spawns an Xvfb display
* (a TS extension: Python's launch_server passes headless through). */
headless?: boolean | "virtual";
}
/**
* Convert a dictionary's keys to camelCase (server.to_camel_case_dict). Keys
* without an underscore are already JS names and are kept as they are.
*/
export function toCamelCaseDict(
data: Record<string, any>,
): Record<string, any> {
const out: Record<string, any> = {};
for (const [key, value] of Object.entries(data)) {
out[key.includes("_") ? camelCase(key) : key] = value;
}
return out;
}
/**
* Launch a Playwright server. Takes the same options as `Camoufox()`.
*
* Note: persistent contexts are not servable. Playwright's `launchServer`
* routes through `BrowserType.launch()`, and its `PlaywrightServer` only
* accepts a pre-launched Browser -- there is no way to expose a persistent
* BrowserContext over a websocket endpoint. Reject those options up front
* rather than accepting them and silently launching a throwaway profile.
*/
export async function launchServer({
headless,
...options
}: LaunchServerOptions = {}): Promise<BrowserServer> {
for (const unsupported of ["persistent_context", "user_data_dir"] as const) {
if (options[unsupported]) {
throw new Error(
`launch_server() does not support '${unsupported}': Playwright cannot ` +
"serve a persistent context over a websocket endpoint. Use " +
"Camoufox(persistent_context=True, ...) in-process instead.",
);
}
delete options[unsupported];
}
let virtualDisplay: VirtualDisplay | null = null;
let headlessBool: boolean | undefined;
if (headless === "virtual") {
virtualDisplay = new VirtualDisplay(options.debug ?? false);
options.virtual_display = await virtualDisplay.get();
headlessBool = false;
} else {
headlessBool = headless;
}
try {
const config = await launchOptions({ ...options, headless: headlessBool });
// The server's browser is spawned from this process too, so it must not
// start inside another launch's CPU pin.
const server = await withUnpinnedLaunch(() =>
firefox.launchServer(toCamelCaseDict(config)),
);
if (virtualDisplay) {
// BrowserServer has no "disconnected" event; "close" fires on shutdown.
const display = virtualDisplay;
server.on("close", () => display.kill());
}
return server;
} catch (error) {
virtualDisplay?.kill();
throw error;
}
}
typescript/src/sync_api.ts
+296
View File
@@ -0,0 +1,296 @@
/**
* The launcher entry points: Camoufox(), NewBrowser(), NewContext().
*
* TypeScript twin of pythonlib/camoufox/sync_api.py (and async_api.py, whose
* names async_api.ts re-exports). The Python library ships a sync and an
* async variant because Playwright-Python has two APIs; playwright-core has
* only one (promise-based), so the behaviours of both live here: the
* no-viewport default, the stock media defaults, CPU-core pinning with its
* launch lock (async_api), and virtual-display teardown.
*/
import {
type Browser,
type BrowserContext,
type BrowserType,
firefox,
} from "playwright-core";
import * as cpuAffinity from "./cpu_affinity.js";
import { generateContextFingerprint } from "./fingerprints.js";
import { ensureModel } from "./fpgen/index.js";
import { type ProxyConfig, ProxyHelper, proxyExitGeo } from "./ip.js";
import {
applyNoViewport,
attachNoViewportDefault,
attachStockMediaDefaults,
attachVirtualDisplay,
driverPid,
type LaunchOptions,
launchOptions,
pinnedCoreCount,
STOCK_MEDIA_DEFAULTS,
spoofsWindowDimensions,
} from "./utils.js";
import { VirtualDisplay } from "./virtdisplay.js";
export type Headless = boolean | "virtual";
export interface NewBrowserOptions extends Omit<LaunchOptions, "headless"> {
/** Whether to run the browser headless. On Linux, `"virtual"` runs a real
* (non-headless) browser on an Xvfb display instead. */
headless?: Headless;
/** A set of launch options generated by `launchOptions()` to use as-is. */
from_options?: Record<string, any>;
/** Whether to use a persistent context. */
persistent_context?: boolean;
/** Directory for the persistent profile (Playwright's user_data_dir). */
user_data_dir?: string;
}
/**
* Launches a Camoufox browser and returns it (Python: `with Camoufox(...) as
* browser`). Close it with `browser.close()`; a virtual display, if one was
* spawned, is torn down with it.
*/
export async function Camoufox(
options: NewBrowserOptions & { persistent_context: true },
): Promise<BrowserContext>;
export async function Camoufox(options?: NewBrowserOptions): Promise<Browser>;
export async function Camoufox(
options: NewBrowserOptions = {},
): Promise<Browser | BrowserContext> {
return NewBrowser(firefox, options);
}
/**
* Launches a new browser instance for Camoufox given a set of launch options.
*
* `from_options`: a set of launch options generated by `launchOptions()` to
* use. `persistent_context`: whether to use a persistent context. Every other
* option is passed to `launchOptions()`.
*/
export async function NewBrowser(
playwright: BrowserType,
options: NewBrowserOptions & { persistent_context: true },
): Promise<BrowserContext>;
export async function NewBrowser(
playwright: BrowserType,
options?: NewBrowserOptions,
): Promise<Browser>;
export async function NewBrowser(
playwright: BrowserType,
{
headless,
from_options,
persistent_context,
debug,
...kwargs
}: NewBrowserOptions = {},
): Promise<Browser | BrowserContext> {
let virtualDisplay: VirtualDisplay | null = null;
let headlessBool: boolean | undefined;
if (headless === "virtual") {
virtualDisplay = new VirtualDisplay(debug ?? false);
kwargs.virtual_display = await virtualDisplay.get();
headlessBool = false;
} else {
headlessBool = headless;
}
let fromOptions = from_options;
try {
if (!fromOptions || !Object.keys(fromOptions).length) {
// Opt-in (2026-09-17). Pinning keeps the identity's core count by
// constraining the browser to that many cores; it costs real CPU,
// needs a launch lock, and does nothing on macOS. Off, the host's own
// snapped count is reported, so reported and measurable still agree.
kwargs.pin_cpu_cores ??= false;
fromOptions = await launchOptions({
...kwargs,
headless: headlessBool,
debug,
});
}
} catch (error) {
// A failed launch must not leave the Xvfb process behind.
virtualDisplay?.kill();
throw error;
}
// Playwright's default viewport deadlocks Juggler when the window is spoofed
// to a different size (daijro/camoufox#666), so default to no viewport.
const noViewportDefault = spoofsWindowDimensions(fromOptions);
// Pin the driver (and so the browser it is about to spawn) to as many cores
// as the identity reports, so measurable parallelism matches
// navigator.hardwareConcurrency; the driver gets its cores back afterwards.
const pinTo = pinnedCoreCount(fromOptions);
const launch = () =>
launchWith(
playwright,
fromOptions as Record<string, any>,
Boolean(persistent_context),
noViewportDefault,
virtualDisplay,
);
try {
if (!pinTo) {
return await cpuAffinity.withUnpinnedLaunch(launch);
}
const pid = driverPid();
// The browser inherits the driver's mask at spawn, so two concurrent
// launches must not interleave pin/restore.
return await cpuAffinity.withPinLock(async () => {
const previous = cpuAffinity.pin(pid, pinTo);
try {
return await launch();
} finally {
cpuAffinity.restore(pid, previous);
}
});
} catch (error) {
virtualDisplay?.kill();
throw error;
}
}
async function launchWith(
playwright: BrowserType,
fromOptions: Record<string, any>,
persistentContext: boolean,
noViewportDefault: boolean,
virtualDisplay: VirtualDisplay | null,
): Promise<Browser | BrowserContext> {
// Persistent context. Python passes user_data_dir inside the options; the
// JS API takes it positionally. A user_data_dir alone also selects it.
const userDataDir = fromOptions.user_data_dir ?? fromOptions.userDataDir;
if (persistentContext || userDataDir !== undefined) {
let options: Record<string, any> = { ...fromOptions };
delete options.user_data_dir;
delete options.userDataDir;
if (
noViewportDefault &&
!("viewport" in options || "noViewport" in options)
) {
options = applyNoViewport(options);
} else if ("noViewport" in options) {
options = applyNoViewport(options);
}
// The persistent context is created by the launch itself, so its media
// features come from these options rather than from newContext().
for (const [key, value] of Object.entries(STOCK_MEDIA_DEFAULTS)) {
if (!(key in options)) options[key] = value;
}
const context = await playwright.launchPersistentContext(
userDataDir ?? "",
options,
);
return attachVirtualDisplay(context, virtualDisplay);
}
// Browser
const browser = await playwright.launch(fromOptions);
if (noViewportDefault) {
attachNoViewportDefault(browser);
}
attachStockMediaDefaults(browser);
return attachVirtualDisplay(browser, virtualDisplay);
}
export interface NewContextOptions extends Record<string, any> {
/** A fingerprint preset to use. If omitted, fpgen draws a new identity. */
preset?: Record<string, any>;
/** Target OS for the drawn identity ("windows", "macos", "linux"). */
os?: string;
/** Firefox major version to claim in the UA. Defaults to the browser's own. */
ff_version?: string;
/** IPv4 or IPv6 address to spoof for WebRTC ICE candidates. */
webrtc_ip?: string;
/** Per-context proxy, in Playwright's format. */
proxy?: ProxyConfig;
/** Per-context geolocation. */
geolocation?: { latitude: number; longitude: number; accuracy?: number };
}
/** Injection point for the proxy exit-IP lookup (tests replace it). */
export const contextDeps = {
resolveProxyGeo: (proxy: ProxyConfig) =>
proxyExitGeo(ProxyHelper.asString(proxy)),
};
/** snake_case -> camelCase, as camoufox.server.camel_case does. */
export function camelCase(snake: string): string {
if (snake.length < 2) return snake;
const parts = snake.toLowerCase().split("_");
const joined = parts
.map((x) => (x ? x[0].toUpperCase() + x.slice(1) : ""))
.join("");
return (
(snake[0] === "_" ? "_" : "") + joined[0].toLowerCase() + joined.slice(1)
);
}
/**
* Creates a new browser context with a unique fingerprint identity.
*
* Each context gets its own identity (navigator, screen, WebGL, fonts,
* voices), drawn by fpgen unless a preset is given, with its own audio noise
* seed. All values are applied via addInitScript so they self-destruct before
* page scripts can detect them.
*/
export async function NewContext(
browser: Browser,
{
preset,
os,
ff_version,
webrtc_ip,
proxy,
geolocation,
...contextOptions
}: NewContextOptions = {},
): Promise<BrowserContext> {
// Python's fpgen loads its model on import; here it is fetched on first use.
// launchOptions() awaits it, but a browser from connect() or a custom
// executable never went through launchOptions().
await ensureModel();
// The drawn UA carries fpgen's Firefox version, which must not disagree with
// the browser the page is actually talking to.
const ffVersion = ff_version || browser.version().split(".", 1)[0];
// Auto-derive the WebRTC IP and timezone from the proxy's exit IP when they
// aren't explicitly provided.
let webrtcIp = webrtc_ip;
if (proxy && (!webrtcIp || !("timezoneId" in contextOptions))) {
const [exitIp, timezone] = await contextDeps.resolveProxyGeo(proxy);
webrtcIp ||= exitIp;
if (!("timezoneId" in contextOptions)) contextOptions.timezoneId = timezone;
}
const fp = generateContextFingerprint({
preset: preset as any,
os,
ff_version: ffVersion,
webrtc_ip: webrtcIp,
});
// Merge the generated context options with user overrides (user wins). They
// are already Playwright's JS names (userAgent, deviceScaleFactor,
// timezoneId); re-casing them turned userAgent into `useragent`, which
// Playwright ignores, so the HTTP User-Agent contradicted navigator's.
const opts: Record<string, any> = {
...fp.context_options,
...contextOptions,
};
if (proxy) opts.proxy = proxy;
if (geolocation) {
opts.geolocation = geolocation;
opts.permissions ??= ["geolocation"];
}
const context = await browser.newContext(opts);
await context.addInitScript(
(fp as any).initScript ?? (fp as any).init_script,
);
return context;
}
typescript/src/utils.ts
+1812
View File
File diff suppressed because it is too large Load Diff
typescript/src/virtdisplay.ts
+320
View File
@@ -0,0 +1,320 @@
/**
* A minimal virtual display implementation for Linux.
*
* TypeScript twin of pythonlib/camoufox/virtdisplay.py.
*/
import { type ChildProcess, execFileSync, spawn } from "node:child_process";
import * as fs from "node:fs";
import {
CannotExecuteXvfb,
CannotFindXvfb,
VirtualDisplayNotSupported,
} from "./exceptions.js";
import { OS_NAME } from "./pkgman.js";
/** Safe timeout for Xvfb writing the display num; prevents an infinite hang. */
const DISPLAYFD_READ_TIMEOUT_MS = 10_000;
/**
* Xvfb screen geometry for headless="virtual".
*
* 1x1x24 is Camoufox's long-standing default and stays the default. The root
* window size is not observable as a fingerprint: screen.* comes from the
* generated fingerprint, applied per context in the browser, and
* clampScreenToDisplay() is skipped entirely for virtual displays (see the
* `!virtual_display` guard in utils.ts), so a 1x1 root never clamps a generated
* screen down to 1x1.
*
* Override with CAMOUFOX_VIRTUAL_DISPLAY_SIZE="<width>x<height>x<depth>", e.g.
* "1920x1080x24", for the cases that do want a real framebuffer to draw into.
* Depth may be omitted.
*/
export const DEFAULT_SCREEN = "1x1x24";
export const SCREEN_ENV_VAR = "CAMOUFOX_VIRTUAL_DISPLAY_SIZE";
/**
* The Composite extension, disabled by default (Xvfb's `-extension COMPOSITE`).
*
* This was briefly enabled by default on the theory that #93 (no video under
* headless="virtual") was caused by disabling it. It was not: #93 was a juggler
* bug, fixed by capturing the screencast from the compositor instead of from
* libwebrtc's X11 window capturer. Both states were measured before that fix:
*
* composite off, record_video_dir -> a valid .webm of 24 pure-white frames
* composite ON, record_video_dir -> browser dies with SIGSEGV, no video
* composite ON, no recording -> fine
*
* The segfault was inside the X11 capturer, which the browser no longer uses,
* so enabling Composite is no longer dangerous -- but it is also no longer good
* for anything, since recording never touches X11 window capture now. Leave it
* off (Camoufox's long-standing default) and keep the escape hatch:
* CAMOUFOX_VIRTUAL_DISPLAY_COMPOSITE=1 enables it.
*/
export const COMPOSITE_ENV_VAR = "CAMOUFOX_VIRTUAL_DISPLAY_COMPOSITE";
/** Screen geometry for Xvfb's -screen argument. */
function resolveScreen(): string {
const value = (process.env[SCREEN_ENV_VAR] ?? "").trim();
if (!value) return DEFAULT_SCREEN;
const parts = value.toLowerCase().split("x");
const valid =
(parts.length === 2 || parts.length === 3) &&
parts.every((p) => /^\d+$/.test(p) && Number.parseInt(p, 10) > 0);
if (!valid) {
throw new VirtualDisplayNotSupported(
`${SCREEN_ENV_VAR} must look like '1920x1080' or '1920x1080x24', got '${value}'`,
);
}
if (parts.length === 2) parts.push("24");
return parts.join("x");
}
export class VirtualDisplay {
debug: boolean;
screen: string;
composite: boolean;
proc: ChildProcess | null = null;
private _display: number | null = null;
constructor(debug: boolean = false, screen?: string, composite?: boolean) {
this.debug = debug;
this.screen = screen || resolveScreen();
this.composite =
composite ??
["1", "true"].includes((process.env[COMPOSITE_ENV_VAR] ?? "0").trim());
}
get xvfbArgs(): string[] {
return [
"-screen",
"0",
this.screen,
"-ac",
"-nolisten",
"tcp",
"-extension",
"RENDER",
"+extension",
"GLX",
this.composite ? "+extension" : "-extension",
"COMPOSITE",
"-extension",
"XVideo",
"-extension",
"XVideo-MotionCompensation",
"-extension",
"XINERAMA",
"-fp",
"built-ins",
"-nocursor",
"-br",
];
}
get xvfbPath(): string {
let resolved: string;
try {
resolved = execFileSync("which", ["Xvfb"], {
encoding: "utf-8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
} catch {
throw new CannotFindXvfb("Please install Xvfb to use headless mode.");
}
if (!resolved) {
throw new CannotFindXvfb("Please install Xvfb to use headless mode.");
}
try {
fs.accessSync(resolved, fs.constants.X_OK);
} catch {
throw new CannotExecuteXvfb(
`I do not have permission to execute Xvfb: ${resolved}`,
);
}
return resolved;
}
/**
* Spawn Xvfb (if it isn't already running) and return its ":N" display.
*
* Uses `-displayfd` so Xvfb itself picks a free display number atomically
* and reports it back, avoiding userspace races. Python's `pass_fds` keeps
* the fd at its parent number; Node's `stdio` array renumbers extra pipes
* from 3 upward, so we hand Xvfb fd 3 and read the pipe at index 3.
*/
async get(): Promise<string> {
VirtualDisplay.assertLinux();
if (this.proc === null) {
const cmd = [this.xvfbPath, "-displayfd", "3", ...this.xvfbArgs];
if (this.debug) {
console.log("Starting virtual display:", cmd.join(" "));
}
this.proc = spawn(cmd[0], cmd.slice(1), {
stdio: [
"ignore",
this.debug ? "inherit" : "ignore",
this.debug ? "inherit" : "ignore",
"pipe",
],
detached: true,
env: {
...process.env,
// Force Mesa software GLX; we don't use the GPU anyway.
__GLX_VENDOR_LIBRARY_NAME: "mesa",
LIBGL_ALWAYS_SOFTWARE: "1",
},
});
// A spawn that fails after spawn() returns (EACCES, ENOEXEC, a noexec
// mount) is an 'error' event on the child; with no listener Node
// treats it as uncaught and exits. readDisplayNumber() turns it into
// CannotExecuteXvfb; this keeps any later one from killing the process.
this.proc.on("error", () => {});
const displayFd = this.proc.stdio[3] as NodeJS.ReadableStream | null;
if (!displayFd) {
this.kill();
throw new CannotExecuteXvfb("Could not open Xvfb's -displayfd pipe");
}
const raw = await this.readDisplayNumber(displayFd);
const parsed = Number.parseInt(raw.trim(), 10);
if (Number.isNaN(parsed)) {
this.kill();
throw new CannotExecuteXvfb(`Xvfb wrote non-integer display: '${raw}'`);
}
this._display = parsed;
} else if (this.debug) {
console.log(`Using virtual display: ${this._display}`);
}
return `:${this._display}`;
}
private readDisplayNumber(stream: NodeJS.ReadableStream): Promise<string> {
return new Promise<string>((resolve, reject) => {
let buf = "";
let settled = false;
const finish = (fn: () => void) => {
if (settled) return;
settled = true;
clearTimeout(timer);
stream.removeAllListeners("data");
stream.removeAllListeners("end");
stream.removeAllListeners("error");
stream.on("error", () => {}); // a late pipe error is not fatal
proc?.removeListener("error", onSpawnError);
fn();
};
const proc = this.proc;
const onSpawnError = (error: Error) =>
finish(() => {
this.kill();
reject(new CannotExecuteXvfb(`Could not start Xvfb: ${error}`));
});
proc?.once("error", onSpawnError);
const timer = setTimeout(() => {
finish(() => {
this.kill();
reject(
new CannotExecuteXvfb(
`Xvfb did not report a display within ${DISPLAYFD_READ_TIMEOUT_MS}ms`,
),
);
});
}, DISPLAYFD_READ_TIMEOUT_MS);
stream.on("data", (chunk) => {
buf += chunk.toString();
if (buf.includes("\n")) {
finish(() => resolve(buf));
}
});
stream.on("end", () => {
finish(() => {
if (buf.includes("\n")) {
resolve(buf);
return;
}
const exit = this.proc?.exitCode;
this.kill();
reject(
new CannotExecuteXvfb(
`Xvfb did not report a display (got '${buf}', exit=${exit})`,
),
);
});
});
stream.on("error", (error) => {
finish(() => {
this.kill();
reject(
new CannotExecuteXvfb(`Failed to read Xvfb's display: ${error}`),
);
});
});
});
}
/**
* Stop Xvfb if it is running, and remove its lock and socket either way.
*
* The cleanup deliberately does NOT depend on whether we did the killing.
* It used to: the whole body sat behind "is the process still running", so
* a display whose Xvfb had already died -- crashed, OOM-killed, or reaped
* with the browser's process group -- was never cleaned up at all.
*
* That is backwards. A SIGKILLed Xvfb never gets to remove its own socket,
* so the crash path is precisely the one where /tmp/.X11-unix/X<n> is left
* behind. Those accumulate, and because -displayfd scans upward for a free
* number, every stranded socket pushes the next display higher until a
* long-running host stops being able to allocate one.
*
* Python waits for the killed process before unlinking; kill() is
* synchronous here, but a SIGKILLed Xvfb can no longer recreate either
* file, so unlinking right after the signal leaves the same end state.
*/
kill(): void {
if (!this.proc) return;
if (this.proc.exitCode === null && this.proc.signalCode === null) {
if (this.debug) {
console.log("Terminating virtual display:", this._display);
}
try {
this.proc.kill("SIGKILL");
} catch {
// Already gone.
}
} else if (this.debug) {
console.log("Virtual display already exited:", this._display);
}
for (const stale of [
`/tmp/.X${this._display}-lock`,
`/tmp/.X11-unix/X${this._display}`,
]) {
try {
fs.unlinkSync(stale);
} catch {
// Missing is the normal case; anything else (a permission error
// from a number another user has since claimed) must not take
// down a teardown path.
}
}
this.proc = null;
}
static assertLinux(): void {
if (OS_NAME !== "lin") {
throw new VirtualDisplayNotSupported(
"Virtual display is only supported on Linux.",
);
}
}
}
typescript/src/warnings.ts
+142
View File
@@ -0,0 +1,142 @@
/**
* Leak and fallback warnings, and the warning channel the launcher reports through.
*
* TypeScript twin of pythonlib/camoufox/_warnings.py. The messages are read
* from the same warnings.yml the Python package ships, so both launchers say
* the same thing.
*
* Python routes these through the `warnings` module; the Node equivalent is
* `process.emitWarning`, which prints `(node:<pid>) <Category>: <message>` to
* stderr and fires `process.on("warning")`. Python's
* `warnings.catch_warnings(record=True)` is `recordWarnings()` here.
*/
import { AsyncLocalStorage } from "node:async_hooks";
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { parse as parseYaml } from "yaml";
import { LIBRARY_VERSION } from "./__version__.js";
import { CamoufoxNotInstalled } from "./exceptions.js";
import { LOCAL_DATA } from "./paths.js";
import { installedVerStr } from "./pkgman.js";
let warningsData: Record<string, string> | undefined;
/** warnings.yml, loaded once. */
export function loadWarnings(): Record<string, string> {
warningsData ??= parseYaml(
fs.readFileSync(path.join(LOCAL_DATA, "warnings.yml"), "utf-8"),
) as Record<string, string>;
return warningsData;
}
export interface RecordedWarning {
category: string;
message: string;
}
const recorder = new AsyncLocalStorage<RecordedWarning[]>();
/**
* Emit a warning. While a `recordWarnings()` block is active the warning is
* captured instead of printed, as Python's catch_warnings(record=True) does.
*/
export function warn(message: string, category = "RuntimeWarning"): void {
const captured = recorder.getStore();
if (captured) {
captured.push({ category, message });
return;
}
process.emitWarning(message, { type: category });
}
/**
* Run `fn`, capturing every warning it emits (sync or async) instead of
* printing it. Scoped by AsyncLocalStorage, so concurrent work outside the
* block is not captured; nested blocks capture into the innermost one.
*/
export async function recordWarnings<T>(
fn: () => T | Promise<T>,
): Promise<{ result?: T; error?: unknown; warnings: RecordedWarning[] }> {
const captured: RecordedWarning[] = [];
try {
return { result: await recorder.run(captured, fn), warnings: captured };
} catch (error) {
return { error, warnings: captured };
}
}
/**
* Emitted when a caller has a setting enabled that can cause detection.
*/
export class LeakWarning extends Error {
constructor(message: string) {
super(message);
this.name = "LeakWarning";
}
/**
* Warns the caller if a passed parameter can cause leaks.
*/
static warn(warningKey: string, iKnowWhatImDoing?: boolean): void {
let warning = loadWarnings()[warningKey];
if (iKnowWhatImDoing) {
return;
}
if (iKnowWhatImDoing !== undefined) {
warning += "\nIf this is intentional, pass `i_know_what_im_doing=True`.";
}
warn(warning, "LeakWarning");
}
}
function browserVersion(): string {
try {
return installedVerStr();
} catch (error) {
if (error instanceof CamoufoxNotInstalled) return "not installed";
throw error;
}
}
/**
* Emitted when part of an identity could not be drawn and a substitute was used.
*/
export class FallbackWarning extends Error {
constructor(message: string) {
super(message);
this.name = "FallbackWarning";
}
/**
* Warns that `what` failed with `error` and the identity uses `instead`,
* with a block of versions and the error for the user to paste into an issue.
*/
static warn(
what: string,
instead: string,
error: unknown,
identity?: string | null,
): void {
const lines = [
`camoufox: ${LIBRARY_VERSION} (npm)`,
`browser: ${browserVersion()}`,
`os: ${os.type()}-${os.release()}-${os.arch()}`,
`node: ${process.versions.node}`,
`error: ${error instanceof Error ? `${error.name}: ${error.message}` : String(error)}`,
];
if (identity) lines.push(`identity: ${identity}`);
const values: Record<string, string> = {
what,
instead,
report: lines.map((line) => ` ${line}`).join("\n"),
};
warn(
loadWarnings().fallback.replace(
/\{(what|instead|report)\}/g,
(_, key: string) => values[key],
),
"FallbackWarning",
);
}
}
typescript/src/webgl.ts
+261
View File
@@ -0,0 +1,261 @@
/**
* WebGL identities, drawn from the Firefox devices fpgen has recorded.
*
* TypeScript twin of pythonlib/camoufox/webgl.py. Everything a page can read
* from WebGL -- vendor, renderer, context attributes, extensions, parameters
* and shader precisions, for WebGL1 and WebGL2 -- comes from one recorded
* device. fpgen's `webgl` node is conditioned on the GPU, and `webgl2` on the
* GPU and the `webgl` chosen for it, so a WebGL2 limit never contradicts its
* WebGL1 counterpart.
*
* Every draw takes one seeded PyRandom, in a fixed order (GPU, then webgl,
* then webgl2), so a seeded identity presents the same device in both
* launchers.
*/
import { gpuFitsOs } from "./coherence.js";
import {
FPGEN_OS,
gpuScreenIsPlausible,
isSoftwareRenderer,
} from "./fingerprints.js";
import { type TraceResult, traceWithEvidence } from "./fpgen/trace.js";
import { lookupPossibilities } from "./fpgen/utils.js";
import {
comparePyStr,
KeyError,
orjsonDumps,
parsePyJson,
pyStr,
pyStrRepr,
ValueError,
} from "./pycompat.js";
import { PyRandom, type PySeed } from "./pyrandom.js";
export type TargetOS = "win" | "mac" | "lin";
/** The config keys a WebGL identity sets. */
export interface WebGLData {
"webGl:vendor": string;
"webGl:renderer": string;
webGl2Enabled?: boolean;
[key: string]: any;
}
// Extensions a release Firefox never exposes (draft extensions behind
// webgl.enable-draft-extensions, or mobile-only): fpgen's corpus carries some
// of them, and a spoofed list that names one is a tell on its own.
const NEVER_EXPOSED_EXTENSIONS: ReadonlySet<string> = new Set([
"WEBGL_multi_draw",
"WEBGL_clip_cull_distance",
"EXT_texture_norm16",
"WEBGL_compressed_texture_etc1",
]);
// OVR_multiview2 depends on the graphics backend: ANGLE's D3D11 backend has it
// on every Windows GPU, a Linux or macOS host driver may not, and the browser
// answers from the spoofed list without asking the host.
const HOST_DEPENDENT_EXTENSIONS: ReadonlySet<string> = new Set([
"OVR_multiview2",
]);
// What Firefox reports under privacy.resistFingerprinting, which a Camoufox
// identity otherwise does not present.
const RFP_RENDERER = "Mozilla";
function filteredExtensions(targetOs: string): ReadonlySet<string> {
if (targetOs === "win") return NEVER_EXPOSED_EXTENSIONS;
return new Set([...NEVER_EXPOSED_EXTENSIONS, ...HOST_DEPENDENT_EXTENSIONS]);
}
function fpgenOs(targetOs: string): string {
const name = FPGEN_OS[targetOs];
if (name === undefined) throw new KeyError(pyStrRepr(targetOs));
return name;
}
type Pin = readonly [node: string, index: string];
/**
* Evidence fixing `node` to exactly the value stored as `text`.
*
* A dict passed to fpgen as a condition is flattened into one condition per
* leaf, and each leaf replaces the node's evidence, so only the last one
* applies. Pinning the value's own lookup index is exact.
*/
function pin(node: string, text: string): Pin {
const index = lookupPossibilities(node, false)?.get(text);
if (index === undefined) throw new KeyError(pyStrRepr(text));
return [node, index];
}
const traceCache = new Map<string, readonly TraceResult[]>();
/** fpgen's distribution of `target` for Firefox on `targetOs`, in its order. */
function trace(
target: string,
targetOs: string,
pinned: readonly Pin[] = [],
): readonly TraceResult[] {
const key = JSON.stringify([target, targetOs, pinned]);
let results = traceCache.get(key);
if (!results) {
results = traceWithEvidence(
target,
{ browser: "Firefox", os: fpgenOs(targetOs) },
{},
new Map(pinned.map(([node, index]) => [node, new Set([index])])),
) as TraceResult[];
traceCache.set(key, results);
}
return results;
}
function choose(rng: PyRandom, results: readonly TraceResult[]): TraceResult {
return rng.choices(results, {
weights: results.map((result) => result.probability),
})[0];
}
/**
* Every (vendor, renderer) that fpgen has seen Firefox report on this OS.
*
* A GPU outside this set has no recorded WebGL parameters behind it, so an
* identity naming it could only borrow another device's.
*/
export function firefoxGpus(targetOs: string): Array<[string, string]> {
return trace("gpu", targetOs.toLowerCase()).map((result) => [
result.value.vendor,
result.value.renderer,
]);
}
function contextConfig(
prefix: string,
webgl: Record<string, any>,
targetOs: string,
): Record<string, any> {
const blocked = filteredExtensions(targetOs);
return {
[`${prefix}:contextAttributes`]: webgl.contextAttributes,
[`${prefix}:supportedExtensions`]: (
webgl.supportedExtensions as string[]
).filter((extension) => !blocked.has(extension)),
[`${prefix}:parameters`]: Object.fromEntries(
Object.entries(webgl.params as Record<string, any>).map(
([pname, param]) => [pname, param.value],
),
),
[`${prefix}:shaderPrecisionFormats`]: Object.fromEntries(
(webgl.shaderPrecisionFormats as Array<Record<string, any>>).map(
(entry) => [
`${entry.shaderType},${entry.precisionType}`,
entry.shaderPrecisionFormat,
],
),
),
};
}
/**
* fpgen's `webgl` and `webgl2` values as Camoufox config keys. `webgl2` is
* `[]` for a device without WebGL2.
*
* Read the values with parsePyJson, as Python reads them, so an integral
* float and an integer past 2**53 reach CAMOU_CONFIG as the same digits.
*/
export function toConfig(
webgl: Record<string, any>,
webgl2: any,
targetOs: string,
): WebGLData {
const hasWebgl2 = Array.isArray(webgl2)
? webgl2.length > 0
: Object.keys(webgl2 ?? {}).length > 0;
const config: Record<string, any> = {
"webGl:vendor": webgl.vendor,
"webGl:renderer": webgl.renderer,
...contextConfig("webGl", webgl, targetOs),
webGl2Enabled: hasWebgl2,
};
if (hasWebgl2)
Object.assign(config, contextConfig("webGl2", webgl2, targetOs));
// The values are the trace's cached objects; the caller gets its own copy.
return parsePyJson(orjsonDumps(config, false));
}
function webglConfig(
targetOs: string,
gpuText: string,
rng: PyRandom,
): WebGLData {
const gpuPin = pin("gpu", gpuText);
const webgl = choose(rng, trace("webgl", targetOs, [gpuPin]));
const webgl2 = choose(
rng,
trace("webgl2", targetOs, [gpuPin, pin("webgl", webgl.text)]),
);
return toConfig(parsePyJson(webgl.text), parsePyJson(webgl2.text), targetOs);
}
/**
* The WebGL config of a device with this GPU, as Firefox on `targetOs`
* reports it.
*
* @throws ValueError for a GPU fpgen has never seen Firefox report on that OS:
* it has no recorded parameters, and another device's would contradict it.
*/
export function webglForGpu(
targetOs: string,
vendor: string,
renderer: string,
seed?: PySeed,
): WebGLData {
const gpus = firefoxGpus(targetOs);
if (!gpus.some(([v, r]) => v === vendor && r === renderer)) {
const pairs = [...gpus]
.sort((a, b) => comparePyStr(a[0], b[0]) || comparePyStr(a[1], b[1]))
.map(([v, r]) => `(${pyStrRepr(v)}, ${pyStrRepr(r)})`);
throw new ValueError(
`No recorded WebGL data for vendor ${pyStrRepr(vendor)} and renderer ${pyStrRepr(renderer)} ` +
`from Firefox on ${fpgenOs(targetOs)}. Possible pairs: [${pairs.join(", ")}]`,
);
}
return webglConfig(
targetOs,
orjsonDumps({ vendor, renderer }, false),
new PyRandom(seed),
);
}
/**
* Draw a GPU for a synthetic identity, weighted as fpgen records Firefox on
* `targetOs`, and its WebGL config.
*
* Only GPUs the rest of the identity can stand beside are drawn: never a
* software rasteriser, a GPU the OS cannot report, the resistFingerprinting
* mask, or a discrete GPU behind a netbook panel. The screen is left alone:
* it has already been reconciled with the real display and the window (#499).
*
* @throws ValueError when no recorded GPU fits.
*/
export function sampleWebglForScreen(
targetOs: string,
width?: number | null,
height?: number | null,
seed?: PySeed,
): WebGLData {
const candidates = trace("gpu", targetOs).filter(
(result) =>
!isSoftwareRenderer(result.value.renderer) &&
result.value.renderer !== RFP_RENDERER &&
gpuFitsOs(result.value.renderer, targetOs) &&
gpuScreenIsPlausible(result.value.renderer, width, height),
);
if (!candidates.length) {
throw new ValueError(
`No recorded ${targetOs} GPU fits a ${pyStr(width)}x${pyStr(height)} screen`,
);
}
const rng = new PyRandom(seed);
return webglConfig(targetOs, choose(rng, candidates).text, rng);
}
typescript/tests/addons.test.ts
+83
View File
@@ -0,0 +1,83 @@
/**
* Mirrors pythonlib/tests/test_addons.py: regression guard for #308, where a
* failed first download left an empty addon directory that was trusted
* forever afterwards.
*/
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
type Addons = typeof import("../src/addons.js");
let tmp: string;
let savedXdg: string | undefined;
let addons: Addons;
let ubo: string;
beforeEach(async () => {
tmp = fs.mkdtempSync(path.join(os.tmpdir(), "camoufox-addons-"));
savedXdg = process.env.XDG_CACHE_HOME;
// Point the addon store at a throwaway dir so no real cache is touched.
process.env.XDG_CACHE_HOME = tmp;
vi.resetModules();
addons = await import("../src/addons.js");
ubo = addons.getAddonPath("UBO");
expect(ubo.startsWith(tmp)).toBe(true);
});
afterEach(() => {
if (savedXdg === undefined) delete process.env.XDG_CACHE_HOME;
else process.env.XDG_CACHE_HOME = savedXdg;
fs.rmSync(tmp, { recursive: true, force: true });
});
function writeManifest(extractPath: string) {
fs.mkdirSync(extractPath, { recursive: true });
fs.writeFileSync(path.join(extractPath, "manifest.json"), "{}");
}
it("re-downloads a partial (manifest-less) directory", async () => {
fs.mkdirSync(ubo, { recursive: true });
const calls: string[] = [];
addons.addonsDeps.downloadAndExtract = async (_url, extractPath, name) => {
calls.push(name);
writeManifest(extractPath);
};
const out: string[] = [];
await addons.maybeDownloadAddons({ ...addons.DefaultAddons }, out);
expect(calls).toEqual(["UBO"]);
expect(fs.existsSync(path.join(ubo, "manifest.json"))).toBe(true);
expect(out).toEqual([ubo]);
});
it("does not re-download an extracted addon", async () => {
writeManifest(ubo);
addons.addonsDeps.downloadAndExtract = async () => {
throw new Error("must not re-download an already-extracted addon");
};
const out: string[] = [];
await addons.maybeDownloadAddons({ ...addons.DefaultAddons }, out);
expect(out).toEqual([ubo]);
});
it("removes the partial directory when a download fails", async () => {
addons.addonsDeps.downloadAndExtract = async (_url, extractPath) => {
fs.mkdirSync(extractPath, { recursive: true }); // partial write, then die
throw new Error("network died mid-download");
};
const log = vi.spyOn(console, "log").mockImplementation(() => {});
const out: string[] = [];
await addons.maybeDownloadAddons({ ...addons.DefaultAddons }, out);
expect(fs.existsSync(ubo)).toBe(false);
expect(out).toEqual([]);
expect(log).toHaveBeenCalledWith(
"Failed to download and extract UBO: Error: network died mid-download",
);
log.mockRestore();
});
typescript/tests/cli-list.test.ts
+69
View File
@@ -0,0 +1,69 @@
/**
* Port of pythonlib/tests/test_cli_list.py: `camoufox list --path` shows
* install paths in both listing modes.
*/
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
let tmp: string;
let savedXdg: string | undefined;
let savedArgv: string[];
beforeEach(() => {
tmp = fs.mkdtempSync(path.join(os.tmpdir(), "camoufox-cli-"));
savedXdg = process.env.XDG_CACHE_HOME;
savedArgv = process.argv;
process.env.XDG_CACHE_HOME = tmp;
vi.resetModules();
});
afterEach(() => {
if (savedXdg === undefined) delete process.env.XDG_CACHE_HOME;
else process.env.XDG_CACHE_HOME = savedXdg;
process.argv = savedArgv;
vi.doUnmock("../src/multiversion.js");
vi.restoreAllMocks();
fs.rmSync(tmp, { recursive: true, force: true });
});
it("list all --path shows the path of an installed build", async () => {
const install = "/cache/browsers/official/152.0.4-beta.30";
const actual = await import("../src/multiversion.js");
const { Version } = await import("../src/pkgman.js");
fs.mkdirSync(path.dirname(actual.REPO_CACHE_FILE), { recursive: true });
fs.writeFileSync(
actual.REPO_CACHE_FILE,
JSON.stringify({
repos: [
{
name: "official",
versions: [{ version: "152.0.4", build: "beta.30" }],
},
],
}),
);
const installed = new actual.InstalledVersion({
repoName: "official",
version: new Version("beta.30", "152.0.4"),
path: install,
});
vi.doMock("../src/multiversion.js", () => ({
...actual,
listInstalled: () => [installed],
}));
let output = "";
vi.spyOn(process.stdout, "write").mockImplementation((chunk) => {
output += String(chunk);
return true;
});
process.argv = ["node", "camoufox", "list", "all", "--path"];
await import("../src/__main__.js");
await new Promise((resolve) => setTimeout(resolve, 0));
vi.mocked(process.stdout.write).mockRestore();
expect(process.exitCode ?? 0).toBe(0);
expect(output).toContain(install);
});
typescript/tests/coherence.test.ts
+290
View File
@@ -0,0 +1,290 @@
/**
* Port of pythonlib/tests/test_coherence.py (the rule-level half; the
* launch-level half lives with the launcher tests) and test_shipped_data.py.
*
* Every identity Camoufox can produce has to be a machine that could exist:
* the pools are sampled independently, so an incoherent identity is assembled
* rather than inherited, and cleaning the pools cannot prevent it.
*/
import * as fs from "node:fs";
import * as path from "node:path";
import { describe, expect, it } from "vitest";
import * as coherence from "../src/coherence.js";
import { fromPreset } from "../src/fingerprints.js";
import { LOCAL_DATA } from "../src/pkgman.js";
import { firefoxGpus } from "../src/webgl.js";
import { MODEL } from "./fpgen-setup.js";
const rules = (config: Record<string, any>, os: string) =>
coherence.validate(config, os).map((v) => v.rule);
describe("rules", () => {
it("Apple Silicon never has fewer than eight cores", () => {
const config = {
"webGl:renderer": "Apple M1, or similar",
"navigator.hardwareConcurrency": 2,
};
expect(rules(config, "mac")).toEqual(["apple-silicon-cores"]);
expect(coherence.apply(config, "mac")).toEqual([]);
expect(config["navigator.hardwareConcurrency"]).toBe(8);
});
it("a Mac cannot report a Braswell Atom IGP", () => {
expect(
rules(
{ "webGl:renderer": "Intel(R) HD Graphics 400, or similar" },
"mac",
),
).toEqual(["gpu-matches-os"]);
});
it("a Mac cannot report an ANGLE renderer", () => {
expect(
rules(
{
"webGl:renderer":
"ANGLE (Intel, Intel(R) HD Graphics Direct3D11 vs_5_0), or similar",
},
"mac",
),
).toEqual(["gpu-matches-os"]);
});
it("Windows renders through ANGLE", () => {
expect(
coherence.gpuFitsOs(
"ANGLE (NVIDIA, NVIDIA GeForce GTX 980 Direct3D11), or similar",
"win",
),
).toBe(true);
expect(coherence.gpuFitsOs("Apple M1, or similar", "win")).toBe(false);
});
it("Apple Silicon reports deep colour", () => {
const config: Record<string, any> = {
"webGl:renderer": "Apple M1, or similar",
"screen.colorDepth": 24,
};
expect(rules(config, "mac")).toEqual(["color-depth"]);
expect(coherence.apply(config, "mac")).toEqual([]);
expect(config["screen.colorDepth"]).toBe(30);
expect(config["screen.pixelDepth"]).toBe(30);
});
it("colour depth is 24 or 30", () => {
const config = { "screen.colorDepth": 32 };
expect(rules(config, "lin")).toEqual(["color-depth"]);
expect(coherence.apply(config, "lin")).toEqual([]);
expect(config["screen.colorDepth"]).toBe(24);
});
it("touch points are a digitiser count", () => {
const config = { "navigator.maxTouchPoints": 256 };
expect(rules(config, "win")).toEqual(["touch-points"]);
expect(coherence.apply(config, "win")).toEqual([]);
expect(config["navigator.maxTouchPoints"]).toBe(0);
for (const real of [0, 1, 2, 5, 10]) {
expect(
coherence.validate({ "navigator.maxTouchPoints": real }, "win"),
).toEqual([]);
}
});
it("a Mac has no touchscreen", () => {
expect(rules({ "navigator.maxTouchPoints": 5 }, "mac")).toEqual([
"touch-points",
]);
});
it("devicePixelRatio is a real display mode", () => {
const config = { "window.devicePixelRatio": 1.8181818181818181 };
expect(rules(config, "win")).toEqual(["device-pixel-ratio"]);
expect(coherence.apply(config, "win")).toEqual([]);
expect(config["window.devicePixelRatio"]).toBe(1.75);
expect(coherence.validate({ "window.devicePixelRatio": 1 }, "lin")).toEqual(
[],
);
expect(
coherence.validate({ "window.devicePixelRatio": 2.5 }, "win"),
).toEqual([]);
expect(coherence.validate({ "window.devicePixelRatio": 2 }, "mac")).toEqual(
[],
);
expect(rules({ "window.devicePixelRatio": 1.5 }, "mac")).toEqual([
"device-pixel-ratio",
]);
});
it("desktop screens are landscape", () => {
const config = { "screen.width": 1440, "screen.height": 2560 };
expect(rules(config, "win")).toEqual(["screen-shape"]);
expect(coherence.repairScreenOrientation(config)).toBe(true);
expect([config["screen.width"], config["screen.height"]]).toEqual([
2560, 1440,
]);
});
it("a phone viewport is not a desktop screen", () => {
expect(rules({ "screen.width": 736, "screen.height": 414 }, "mac")).toEqual(
["screen-shape"],
);
});
it("avail never exceeds the screen", () => {
const config = {
"screen.width": 1920,
"screen.height": 1080,
"screen.availWidth": 2000,
};
expect(rules(config, "lin")).toEqual(["avail-bounds"]);
expect(coherence.apply(config, "lin")).toEqual([]);
expect(config["screen.availWidth"]).toBe(1920);
});
it("the platform agrees with the user agent arch", () => {
expect(
rules(
{
"navigator.userAgent":
"Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0",
"navigator.platform": "Linux armv81",
},
"lin",
),
).toEqual(["arch-agreement"]);
});
it("the viewport leaves room for the browser chrome", () => {
const config = {
"window.outerHeight": 801,
"window.innerHeight": 717,
"screen.availHeight": 1040,
};
expect(rules(config, "lin")).toEqual(["window-chrome"]);
expect(coherence.apply(config, "lin")).toEqual([]);
expect(config["window.outerHeight"]).toBe(
717 + coherence.BROWSER_CHROME_HEIGHT,
);
});
it("the real machines pass", () => {
const real: Array<[string, Record<string, any>]> = [
[
"lin",
{
"navigator.userAgent":
"Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0",
"navigator.platform": "Linux x86_64",
"navigator.hardwareConcurrency": 16,
"navigator.maxTouchPoints": 0,
"screen.width": 1920,
"screen.height": 1080,
"screen.colorDepth": 24,
"webGl:renderer": "Radeon HD 3200 Graphics, or similar",
},
],
[
"win",
{
"navigator.platform": "Win32",
"navigator.hardwareConcurrency": 16,
"navigator.maxTouchPoints": 5,
"screen.width": 1382,
"screen.height": 864,
"screen.colorDepth": 24,
"webGl:renderer":
"ANGLE (Intel, Intel(R) HD Graphics Direct3D11 vs_5_0 ps_5_0), or similar",
},
],
[
"mac",
{
"navigator.platform": "MacIntel",
"navigator.hardwareConcurrency": 10,
"navigator.maxTouchPoints": 0,
"screen.width": 2560,
"screen.height": 1440,
"screen.colorDepth": 30,
"webGl:renderer": "Apple M1, or similar",
},
],
];
for (const [os, config] of real) {
expect(coherence.validate(config, os), os).toEqual([]);
}
});
it("drops a source GPU the OS cannot report, and only that", () => {
const config: Record<string, any> = {
"webGl:vendor": "Intel",
"webGl:renderer": "Intel(R) HD Graphics 400, or similar",
"screen.width": 1920,
};
const dropped = coherence.dropIncoherentSourceValues(config, "mac");
expect(dropped.map((v) => v.rule)).toEqual(["gpu-matches-os"]);
expect(config).toEqual({ "screen.width": 1920 });
});
});
describe("shipped data (test_shipped_data.py)", () => {
const OS_KEY: Record<string, string> = {
macos: "mac",
windows: "win",
linux: "lin",
};
for (const file of [
"fingerprint-presets.json",
"fingerprint-presets-v150.json",
]) {
it(`every preset in ${file} is coherent as stored`, () => {
const presets = JSON.parse(
fs.readFileSync(path.join(LOCAL_DATA, file), "utf-8"),
).presets as Record<string, any[]>;
for (const [os, entries] of Object.entries(presets)) {
expect(
entries.length,
`${file}/${os} has no presets left`,
).toBeGreaterThan(0);
entries.forEach((preset, i) => {
const config = fromPreset(preset, "152", 0);
const violations = coherence.validate(config, OS_KEY[os]);
expect(
violations,
`${file} ${os}[${i}]: ${violations.map((v) => v.detail).join("; ")}`,
).toEqual([]);
});
}
});
}
for (const file of [
"fingerprint-presets.json",
"fingerprint-presets-v150.json",
]) {
// A preset records only its GPU's name; the WebGL parameters behind it
// come from fpgen, and a GPU fpgen never saw from Firefox on that OS has
// none, so launching it would borrow another device's.
it.skipIf(!MODEL.ok)(`every preset GPU in ${file} has WebGL data`, () => {
const presets = JSON.parse(
fs.readFileSync(path.join(LOCAL_DATA, file), "utf-8"),
).presets as Record<string, any[]>;
for (const [os, entries] of Object.entries(presets)) {
const known = new Set(
firefoxGpus(
os === "windows" ? "win" : os === "macos" ? "mac" : "lin",
).map((gpu) => JSON.stringify(gpu)),
);
entries.forEach((preset, i) => {
const gpu = [
preset.webgl.unmaskedVendor,
preset.webgl.unmaskedRenderer,
];
expect(
known.has(JSON.stringify(gpu)),
`${file} ${os}[${i}]: ${gpu[1]} has no WebGL data`,
).toBe(true);
});
}
});
}
});
typescript/tests/download-integrity.test.ts
+86
View File
@@ -0,0 +1,86 @@
/**
* Mirrors pythonlib/tests/test_download_integrity.py: a release asset whose
* bytes do not match the GitHub-published sha256 digest must abort the
* install before extraction.
*/
import { createHash } from "node:crypto";
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { describe, expect, it } from "vitest";
import { CorruptedDownload } from "../src/exceptions.js";
import { verifySha256 } from "../src/pkgman.js";
// Large enough to span several read() blocks, so a single-shot read
// regression cannot pass by accident.
const PAYLOAD = Buffer.from("camoufox release asset".repeat(100_000));
const DIGEST = createHash("sha256").update(PAYLOAD).digest("hex");
describe("verifySha256", () => {
it("accepts a matching digest", () => {
expect(() => verifySha256(PAYLOAD, DIGEST, "asset")).not.toThrow();
});
it("compares digests case-insensitively", () => {
expect(() =>
verifySha256(PAYLOAD, DIGEST.toUpperCase(), "asset"),
).not.toThrow();
});
const mutations: Record<string, (b: Buffer) => Buffer> = {
"first-byte-flipped": (b) =>
Buffer.concat([Buffer.from([b[0] ^ 0xff]), b.subarray(1)]),
"last-bit-flipped": (b) =>
Buffer.concat([b.subarray(0, -1), Buffer.from([b[b.length - 1] ^ 1])]),
truncated: (b) => b.subarray(0, -1),
appended: (b) => Buffer.concat([b, Buffer.from([0])]),
empty: () => Buffer.alloc(0),
};
for (const [name, mutate] of Object.entries(mutations)) {
it(`aborts the install on a tampered payload (${name})`, () => {
expect(() => verifySha256(mutate(PAYLOAD), DIGEST, "asset")).toThrow(
CorruptedDownload,
);
});
}
it("names both digests in the error", () => {
let message = "";
try {
verifySha256(Buffer.from("wrong"), DIGEST, "Camoufox v1.2.3");
} catch (e) {
expect(e).toBeInstanceOf(CorruptedDownload);
message = (e as Error).message;
}
expect(message).toContain("Camoufox v1.2.3");
expect(message).toContain(DIGEST);
expect(message).toContain(
createHash("sha256").update("wrong").digest("hex"),
);
});
for (const absent of [null, undefined, ""]) {
it(`does not block the install without a digest (${JSON.stringify(absent)})`, () => {
expect(() => verifySha256(PAYLOAD, absent, "asset")).not.toThrow();
});
}
it("verifies a real temporary file, as the install path does", () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "camoufox-sha-"));
try {
const file = path.join(dir, "asset.zip");
fs.writeFileSync(file, PAYLOAD);
expect(() => verifySha256(file, DIGEST, "asset")).not.toThrow();
fs.appendFileSync(file, "x");
expect(() => verifySha256(file, DIGEST, "asset")).toThrow(
CorruptedDownload,
);
// The file is untouched for the extraction that follows.
expect(fs.readFileSync(file).subarray(0, PAYLOAD.length)).toEqual(
PAYLOAD,
);
} finally {
fs.rmSync(dir, { recursive: true, force: true });
}
});
});
typescript/tests/download-stream.test.ts
+101
View File
@@ -0,0 +1,101 @@
/**
* The browser download writes through a file stream. A failed write (disk
* full) must come back as an ordinary rejection that installVersioned() cleans
* up after -- an 'error' event with no listener kills the process instead --
* and a slow disk must hold the download back rather than queue it in memory.
*/
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { Writable } from "node:stream";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
let tmp: string;
let savedXdg: string | undefined;
beforeEach(() => {
tmp = fs.mkdtempSync(path.join(os.tmpdir(), "camoufox-dl-"));
savedXdg = process.env.XDG_CACHE_HOME;
process.env.XDG_CACHE_HOME = tmp;
vi.resetModules();
});
afterEach(() => {
if (savedXdg === undefined) delete process.env.XDG_CACHE_HOME;
else process.env.XDG_CACHE_HOME = savedXdg;
vi.unstubAllGlobals();
fs.rmSync(tmp, { recursive: true, force: true });
});
it("a failed write rejects, and the partial install is removed", async () => {
const { installVersioned, BROWSERS_DIR } = await import(
"../src/multiversion.js"
);
const { CamoufoxFetcher } = await import("../src/pkgman.js");
class DiskFull extends CamoufoxFetcher {
static override async downloadFile(file: Writable): Promise<void> {
file.write(Buffer.alloc(1024));
file.destroy(
Object.assign(new Error("ENOSPC: no space left"), { code: "ENOSPC" }),
);
await new Promise((r) => setTimeout(r, 20));
}
}
// Skip the constructor: it would query GitHub.
const fetcher = Object.create(DiskFull.prototype);
const own = (value: unknown) => ({ value, configurable: true });
Object.defineProperties(fetcher, {
githubRepo: own("example/camoufox"),
version: own("152.0.4"),
build: own("beta.99"),
verstr: own("152.0.4-beta.99"),
url: own("https://example.invalid/camoufox.zip"),
_selectedVersion: own(null),
});
await expect(installVersioned(fetcher)).rejects.toThrow(/ENOSPC/);
expect(
fs.existsSync(path.join(BROWSERS_DIR, "example", "152.0.4-beta.99")),
).toBe(false);
});
it("waits for a slow stream to drain instead of queueing the download", async () => {
const { webdl } = await import("../src/pkgman.js");
const chunks = 64;
const size = 1024;
vi.stubGlobal(
"fetch",
async () =>
new Response(
new ReadableStream({
start(controller) {
for (let i = 0; i < chunks; i++)
controller.enqueue(new Uint8Array(size));
controller.close();
},
}),
{ headers: { "content-length": String(chunks * size) } },
),
);
let peak = 0;
let received = 0;
const slow: Writable = new Writable({
highWaterMark: size,
write(chunk, _enc, done) {
received += chunk.length;
setTimeout(() => {
// What is still queued behind this chunk once the disk catches up.
peak = Math.max(peak, slow.writableLength);
done();
}, 2);
},
});
await webdl("https://example.invalid/x.zip", "x", false, slow, {
progressCallback: () => {},
});
await new Promise<void>((r) => slow.end(r));
expect(received).toBe(chunks * size);
// Without waiting for 'drain' all 64 KiB sit in the stream's buffer.
expect(peak).toBeLessThanOrEqual(2 * size);
});
typescript/tests/e2e.test.ts
+626
View File
@@ -0,0 +1,626 @@
/**
* End-to-end: launch a real Camoufox build through the TypeScript API and
* check what a page actually sees.
*
* Opt-in -- it needs a browser binary:
*
* CAMOUFOX_E2E=1 CAMOUFOX_EXECUTABLE=/path/to/camoufox-bin pnpm test tests/e2e.test.ts
*
* Two kinds of assertion, deliberately kept apart:
*
* - PARITY, always enforced: for each fixed identity it launches headless, as a
* persistent context, through launchServer() and through NewContext(), runs
* tests/fixtures/e2e/probe.js on a local page, then launches the PYTHON
* camoufox on the same binary with the same identity
* (scripts/e2e/python_probe.py, run with $CAMOUFOX_E2E_PYTHON or the repo's
* .venv) and requires the same CAMOU_CONFIG and the same probe result. This is
* the claim this package makes, and it holds on any binary.
*
* - THE BROWSER HONOURS THE CONFIG: navigator/screen/window/fonts/timezone/
* voices/WebGL on the page equal what the launcher sent. That is the
* browser's half of the contract, so it is only asserted on a binary that
* knows every key the launcher sets (its properties.json lists them). An
* older binary -- the published release, on a driver-only pull request, while
* the launcher is ahead of it -- ignores the keys it does not know, exactly
* as pythonlib's "Skipping unknown patch" says, and those tests skip naming
* the missing keys rather than failing on the skew.
*/
import { spawn } from "node:child_process";
import * as fs from "node:fs";
import * as http from "node:http";
import type { AddressInfo } from "node:net";
import * as os from "node:os";
import * as path from "node:path";
import { fileURLToPath } from "node:url";
import AdmZip from "adm-zip";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { PYTHON as GOLDEN_PYTHON } from "./golden-setup.js";
import { prerequisite } from "./prereq.js";
const HERE = path.dirname(fileURLToPath(import.meta.url));
const REPO = path.resolve(HERE, "..", "..");
const EXECUTABLE = process.env.CAMOUFOX_EXECUTABLE ?? "";
const ENABLED = process.env.CAMOUFOX_E2E === "1" && Boolean(EXECUTABLE);
const PYTHON = process.env.CAMOUFOX_E2E_PYTHON ?? GOLDEN_PYTHON;
const PROBE = fs.readFileSync(
path.join(HERE, "fixtures", "e2e", "probe.js"),
"utf-8",
);
const INPUTS = JSON.parse(
fs.readFileSync(
path.join(HERE, "fixtures", "launch", "inputs.json"),
"utf-8",
),
);
/** The fixed identities: an fpgen fingerprint and a cross-OS preset. */
const IDENTITIES: Record<string, Record<string, any>> = {
fpgen_linux_de: {
fingerprint: INPUTS.fingerprints.linux,
os: "linux",
locale: "de-DE",
config: { timezone: "Asia/Tokyo" },
},
preset_windows_fr: {
fingerprint_preset: INPUTS.presets.windows,
locale: "fr-FR",
config: { timezone: "Europe/Paris" },
},
};
/** Config keys the binary under test does not know (see the header). */
function unknownToBinary(config: Record<string, any>): string[] {
const dir = path.dirname(EXECUTABLE);
const candidates = [
path.join(dir, "properties.json"),
path.join(dir, "..", "Resources", "properties.json"), // macOS bundle
];
const file = candidates.find((f) => fs.existsSync(f));
if (!file) return ["<no properties.json beside the binary>"];
const known = new Set(
JSON.parse(fs.readFileSync(file, "utf-8")).map((p: any) => p.property),
);
return Object.keys(config).filter((k) => !known.has(k));
}
/**
* The locales the binary under test packages (res/multilocale.txt): loose in
* an unpackaged dist/bin, inside omni.ja in a packaged build. A spoofed locale
* the binary does not package falls back to en-US -- scripts/package.py adds the
* langpacks, so a packaged release has them and CI's unpackaged dist/bin does
* not. null when it cannot be read.
*/
function packagedLocales(): string[] | null {
const dir = path.dirname(EXECUTABLE);
try {
const loose = path.join(dir, "res", "multilocale.txt");
const text = fs.existsSync(loose)
? fs.readFileSync(loose, "utf-8")
: new AdmZip(path.join(dir, "omni.ja"))
.getEntry("res/multilocale.txt")
?.getData()
.toString("utf-8");
return text
? text
.split(",")
.map((l) => l.trim())
.filter(Boolean)
: null;
} catch {
return null;
}
}
let server: http.Server;
let url = "";
/** The User-Agent header of the last request the probe page served. */
let lastRequestUserAgent = "";
beforeAll(async () => {
if (!ENABLED) return;
server = http.createServer((req, res) => {
lastRequestUserAgent = req.headers["user-agent"] ?? "";
res.writeHead(200, { "content-type": "text/html; charset=utf-8" });
res.end(
"<!doctype html><html><head><title>probe</title></head><body>probe</body></html>",
);
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
url = `http://127.0.0.1:${(server.address() as AddressInfo).port}/`;
});
afterAll(async () => {
if (server) await new Promise((resolve) => server.close(resolve));
});
function configOf(options: Record<string, any>): Record<string, any> {
const env = options.env as Record<string, string>;
return JSON.parse(
Object.keys(env)
.filter((k) => k.startsWith("CAMOU_CONFIG_"))
.sort((a, b) => Number(a.split("_").pop()) - Number(b.split("_").pop()))
.map((k) => env[k])
.join(""),
);
}
function kwargsFor(identity: string): Record<string, any> {
return {
...structuredClone(IDENTITIES[identity]),
executable_path: EXECUTABLE,
headless: true,
i_know_what_im_doing: true,
};
}
/**
* Give a headful page focus and wait until it has it. One bringToFront() on a
* bare Xvfb, with no window manager, sometimes left the window unfocused, and
* Firefox holds enumerateDevices() until the document has focus.
*/
async function focusPage(page: any): Promise<void> {
for (let i = 0; i < 50; i++) {
await page.bringToFront();
if (await page.evaluate("document.hasFocus()")) return;
await new Promise((r) => setTimeout(r, 200));
}
throw new Error("the headful page never got focus on Xvfb");
}
async function probePage(page: any): Promise<any> {
await page.goto(url);
// Python's evaluate() calls a function-source string; JS's evaluates it as
// an expression, so call it explicitly.
return page.evaluate(`(${PROBE})()`);
}
/**
* Run the Python side. Asynchronously: the probe page is served from this
* process, so blocking the event loop would hang Python's page.goto().
*/
function pythonProbe(
mode: string,
kwargs: Record<string, any>,
extra: Record<string, any> = {},
): Promise<any> {
return new Promise((resolve, reject) => {
const child = spawn(
PYTHON,
[path.join(REPO, "typescript", "scripts", "e2e", "python_probe.py")],
{
stdio: ["pipe", "pipe", "pipe"],
},
);
let stdout = "";
let stderr = "";
child.stdout.on("data", (d) => {
stdout += d;
});
child.stderr.on("data", (d) => {
stderr += d;
});
child.on("error", reject);
child.on("close", (code) => {
if (code !== 0) {
reject(new Error(`python probe failed (${code}): ${stderr}`));
return;
}
try {
resolve(JSON.parse(stdout));
} catch (err) {
reject(
new Error(
`python probe printed non-JSON (${err}): ${stdout.slice(0, 500)}`,
),
);
}
});
child.stdin.end(JSON.stringify({ mode, url, kwargs, ...extra }));
});
}
/**
* The probe minus the canvas hash. The canvas readback differs between two
* launches of the SAME config -- measured with this build,
* TS and Python alike -- so it is compared by its size only.
*/
/**
* `promise`, or a rejection naming the step after `ms`. A hung launch otherwise
* surfaces only as vitest's whole-test timeout, which says nothing about where.
*/
function step<T>(what: string, promise: Promise<T>, ms = 60_000): Promise<T> {
let timer: NodeJS.Timeout | undefined;
return Promise.race([
promise,
new Promise<never>((_, reject) => {
timer = setTimeout(
() => reject(new Error(`${what}: did not complete in ${ms / 1000} s`)),
ms,
);
}),
]).finally(() => clearTimeout(timer));
}
function stable(probe: any): any {
const { canvasHash, ...rest } = probe;
return { ...rest, canvasSize: String(canvasHash).split(":")[1] };
}
/** What the page must report for a config the launcher sent. */
function expectMatchesConfig(
probe: any,
config: Record<string, any>,
headful = false,
) {
expect(probe.navigator.userAgent).toBe(config["navigator.userAgent"]);
expect(probe.navigator.platform).toBe(config["navigator.platform"]);
if ("navigator.oscpu" in config)
expect(probe.navigator.oscpu).toBe(config["navigator.oscpu"]);
expect(probe.navigator.hardwareConcurrency).toBe(
config["navigator.hardwareConcurrency"],
);
expect(probe.navigator.webdriver).toBe(false);
// DNT / GPC follow stock Firefox unless the caller set them.
expect(probe.navigator.doNotTrack).toBe("unspecified");
expect(probe.navigator.globalPrivacyControl).toBe(false);
for (const key of [
"width",
"height",
"availWidth",
"availHeight",
"colorDepth",
]) {
if (`screen.${key}` in config)
expect(probe.screen[key], `screen.${key}`).toBe(config[`screen.${key}`]);
}
for (const key of ["outerWidth", "outerHeight"]) {
if (`window.${key}` in config)
expect(probe.window[key], `window.${key}`).toBe(config[`window.${key}`]);
}
expect(probe.intl.timeZone).toBe(config.timezone);
const lang = [config["locale:language"], config["locale:region"]].join("-");
const packaged = packagedLocales();
const hasLocale =
packaged === null ||
packaged.includes(lang) ||
packaged.includes(config["locale:language"]);
if (
prerequisite(
"packaged-locales",
hasLocale,
`the binary packages ${packaged?.join(",")}, not ${lang}`,
)
) {
expect(probe.navigator.language).toBe(lang);
expect(probe.intl.locale).toBe(lang);
}
// Fonts: every probed family the identity claims resolves; the others don't.
const claimed = new Set<string>(config.fonts);
for (const [family, present] of Object.entries(probe.fonts)) {
if (!claimed.has(family)) expect(present, `${family} leaked`).toBe(false);
}
// Voices: exactly the identity's list, never the host's.
expect(probe.voices.map((v: string) => v.split("|")[0]).sort()).toEqual(
config.voices.map((v: any) => v.name).sort(),
);
if (probe.webgl) {
expect(probe.webgl.vendor).toBe(config["webGl:vendor"]);
expect(probe.webgl.renderer).toBe(config["webGl:renderer"]);
}
// Playwright's media emulation is off, so the host answers: headless is
// light; headful follows the desktop theme.
if (!headful) expect(probe.media.colorScheme).toBe("light");
// Stock Firefox's max_entries: the history starts at 1.
expect(probe.historyLength).toBe(1);
// The storage quota is the stock-profile disk's, not the temp dir's.
expect(typeof probe.storageQuota).toBe("number");
expect(probe.mediaDevices.length).toBeGreaterThan(0);
}
describe.runIf(ENABLED)("e2e: the TS launcher drives a real Camoufox", () => {
let mods: {
sync: typeof import("../src/sync_api.js");
server: typeof import("../src/server.js");
utils: typeof import("../src/utils.js");
warnings: typeof import("../src/warnings.js");
};
const results: Record<string, any> = {};
beforeAll(async () => {
mods = {
sync: await import("../src/sync_api.js"),
server: await import("../src/server.js"),
utils: await import("../src/utils.js"),
warnings: await import("../src/warnings.js"),
};
});
for (const identity of Object.keys(IDENTITIES)) {
describe(identity, () => {
it("builds the same CAMOU_CONFIG as Python", async () => {
const { result } = await mods.warnings.recordWarnings(() =>
mods.utils.launchOptions(kwargsFor(identity)),
);
const config = configOf(result as Record<string, any>);
results[`${identity}:config`] = config;
const py = await pythonProbe("config-only", kwargsFor(identity));
expect(config).toEqual(py.config);
}, 240_000);
it("headless: the page sees the configured identity, same as Python", async () => {
const { result: browser } = await mods.warnings.recordWarnings(() =>
mods.sync.Camoufox(kwargsFor(identity)),
);
let probe: any;
try {
const page = await (browser as any).newPage();
probe = await probePage(page);
} finally {
await (browser as any).close();
}
results[`${identity}:headless`] = probe;
// A second launch of the same identity presents the same device.
const { result: again } = await mods.warnings.recordWarnings(() =>
mods.sync.Camoufox(kwargsFor(identity)),
);
try {
expect(
stable(await probePage(await (again as any).newPage())),
).toEqual(stable(probe));
} finally {
await (again as any).close();
}
const py = await pythonProbe("headless", kwargsFor(identity));
expect(stable(probe)).toEqual(stable(py.probe));
}, 240_000);
it("persistent context: same identity, same as Python", async () => {
const profile = fs.mkdtempSync(
path.join(os.tmpdir(), "camoufox-e2e-profile-"),
);
let probe: any;
try {
const { result: context } = await mods.warnings.recordWarnings(() =>
mods.sync.Camoufox({
...kwargsFor(identity),
persistent_context: true,
user_data_dir: profile,
}),
);
try {
const page = await (context as any).newPage();
probe = await probePage(page);
} finally {
await (context as any).close();
}
} finally {
fs.rmSync(profile, { recursive: true, force: true });
}
results[`${identity}:persistent`] = probe;
const py = await pythonProbe("persistent", kwargsFor(identity));
expect(stable(probe)).toEqual(stable(py.probe));
}, 240_000);
it("launchServer: a connected client sees the same identity", async () => {
const { firefox } = await import("playwright-core");
const { result: bs } = await mods.warnings.recordWarnings(() =>
mods.server.launchServer(kwargsFor(identity)),
);
const browserServer = bs as import("playwright-core").BrowserServer;
let probe: any;
try {
const browser = await firefox.connect(browserServer.wsEndpoint());
try {
// A remote client gets plain Playwright: no Camoufox defaults,
// so ask for them explicitly (as a Python client would).
const config = results[`${identity}:config`];
const spoofsWindow = Object.keys(config).some((k) =>
/^(window\.(outer|inner)|document\.body\.client)/.test(k),
);
const page = await browser.newPage({
...(spoofsWindow ? { viewport: null } : {}),
...mods.utils.STOCK_MEDIA_DEFAULTS,
});
probe = await probePage(page);
} finally {
await browser.close();
}
} finally {
await browserServer.close();
}
results[`${identity}:server`] = probe;
expect(stable(probe)).toEqual(stable(results[`${identity}:headless`]));
}, 240_000);
it("NewContext: a per-context identity, same as Python", async () => {
// A different identity from the launch one, so a context that
// inherited the browser's identity would be caught.
const preset =
identity === "fpgen_linux_de"
? INPUTS.presets.windows
: INPUTS.presets.macos;
const { result: browser } = await mods.warnings.recordWarnings(() =>
mods.sync.Camoufox(kwargsFor(identity)),
);
let probe: any;
let second: any;
try {
const context = await mods.sync.NewContext(browser as any, {
preset,
});
probe = await probePage(await context.newPage());
// The HTTP header must tell the same story as navigator: a
// context that dropped its userAgent option sent the
// browser's own UA here while navigator was spoofed.
expect(lastRequestUserAgent).toBe(probe.navigator.userAgent);
// Contexts are isolated: another preset, another device.
const other = await mods.sync.NewContext(browser as any, {
preset: INPUTS.presets.linux,
});
second = await probePage(await other.newPage());
} finally {
await (browser as any).close();
}
const launch = results[`${identity}:headless`];
expect(probe.navigator.userAgent).not.toBe(launch.navigator.userAgent);
expect(second.navigator.userAgent).not.toBe(probe.navigator.userAgent);
// Per-context noise seeds are drawn fresh in both launchers, so
// compare the identity itself, not the noise.
const identityOf = (p: any) => ({
navigator: p.navigator,
screen: p.screen,
timeZone: p.intl.timeZone,
webgl: p.webgl,
});
const py = await pythonProbe("context", kwargsFor(identity), {
preset,
});
expect(identityOf(probe)).toEqual(identityOf(py.probe));
}, 240_000);
it("the browser honours the config (headless, persistent, launchServer)", async (ctx) => {
const config = results[`${identity}:config`];
const missing = unknownToBinary(config);
if (missing.length)
ctx.skip(
`binary predates the launcher; it does not know: ${missing.join(", ")}`,
);
for (const mode of ["headless", "persistent", "server"]) {
expect(results[`${identity}:${mode}`], mode).toBeTruthy();
expectMatchesConfig(results[`${identity}:${mode}`], config);
}
// A persistent profile presents the same device as a throwaway one.
// (beta.30 did not: its storage quota was Firefox's pinned 10 GiB
// headless and the disk's in a profile -- LEAKS row 107.)
expect(stable(results[`${identity}:persistent`])).toEqual(
stable(results[`${identity}:headless`]),
);
});
it.runIf(identity === "fpgen_linux_de" && process.platform === "linux")(
"pin_cpu_cores: the browser runs on as many cores as it reports",
async () => {
const selfCores = () =>
fs
.readFileSync("/proc/self/status", "utf-8")
.match(/Cpus_allowed_list:\s*(.+)/)?.[1];
const before = selfCores();
const { result: browser } = await mods.warnings.recordWarnings(() =>
mods.sync.Camoufox({ ...kwargsFor(identity), pin_cpu_cores: true }),
);
try {
const page = await (browser as any).newPage();
await page.goto(url);
const reported = await page.evaluate(
"navigator.hardwareConcurrency",
);
// The launching process got its cores back...
expect(selfCores()).toBe(before);
// ...and the browser it spawned runs on exactly `reported` cores.
const { parseCpuList } = await import("../src/cpu_affinity.js");
const children = fs
.readFileSync(
`/proc/${process.pid}/task/${process.pid}/children`,
"utf-8",
)
.trim()
.split(/\s+/)
.filter((pid) => {
try {
return (
fs.readFileSync(`/proc/${pid}/comm`, "utf-8").trim() ===
"camoufox-bin"
);
} catch {
return false;
}
});
expect(children.length).toBeGreaterThan(0);
for (const pid of children) {
const list =
fs
.readFileSync(`/proc/${pid}/status`, "utf-8")
.match(/Cpus_allowed_list:\s*(.+)/)?.[1] ?? "";
expect(parseCpuList(list)).toHaveLength(reported);
}
} finally {
await (browser as any).close();
}
},
240_000,
);
it.runIf(identity === "fpgen_linux_de" && process.platform === "linux")(
"headless: 'virtual' runs headful on a private Xvfb and tears it down",
async () => {
const { result: browser } = await step(
"virtual: launch (Xvfb + headful browser)",
mods.warnings.recordWarnings(() =>
mods.sync.Camoufox({
...kwargsFor(identity),
headless: "virtual",
}),
),
);
const display = (browser as any)._virtualDisplay;
expect(display).toBeTruthy();
let probe: any;
try {
const page = await step(
"virtual: newPage",
(browser as any).newPage(),
);
// Firefox defers enumerateDevices() until the document has
// focus (LEAKS row 57). Headless fakes focus; a headful window
// on a bare Xvfb only sometimes gets it, so the probe hung on
// some runs. A user's window has focus: give it one.
await step("virtual: goto", page.goto(url));
await step("virtual: focus", focusPage(page));
probe = await step(
"virtual: probe the page",
page.evaluate(`(${PROBE})()`),
);
} finally {
await step("virtual: close", (browser as any).close());
}
// close() killed the Xvfb it spawned.
expect(display.proc ?? null).toBeNull();
results[`${identity}:virtual`] = probe;
// Parity: Python's headless="virtual" on the same binary.
const py = await pythonProbe("virtual", kwargsFor(identity));
expect(stable(probe)).toEqual(stable(py.probe));
},
240_000,
);
it.runIf(identity === "fpgen_linux_de" && process.platform === "linux")(
"the browser honours the config headful, as it does headless",
async (ctx) => {
const config = results[`${identity}:config`];
const missing = unknownToBinary(config);
if (missing.length)
ctx.skip(
`binary predates the launcher; it does not know: ${missing.join(", ")}`,
);
const probe = results[`${identity}:virtual`];
expect(probe, "the virtual-display test ran").toBeTruthy();
expectMatchesConfig(probe, config, true);
// A headful window presents the same device as headless. On a
// runner with no media hardware, published beta.31 never settled
// enumerateDevices() headful while headless answered.
const { media: _m, ...headful } = stable(probe);
const { media: _h, ...headless } = stable(
results[`${identity}:headless`],
);
expect(headful).toEqual(headless);
},
240_000,
);
});
}
});
typescript/tests/fallback-warnings.test.ts
+136
View File
@@ -0,0 +1,136 @@
/**
* Port of pythonlib/tests/test_fallback_warnings.py (identity half; the launch
* half is in launch.test.ts): every place an identity falls back to a
* substitute value says so, with a report block to paste into an issue.
*
* Python breaks the draws with monkeypatch; here they read a copy of the data
* files with one file removed or corrupted, which fails the same way.
*/
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { DATA_FILES, LOCAL_DATA } from "../src/paths.js";
import { prerequisite } from "./prereq.js";
const REPORT =
"Please report this at https://github.com/daijro/camoufox/issues/new";
let modelReady = true;
try {
await (await import("../src/fpgen/index.js")).ensureModel();
} catch (e) {
modelReady = prerequisite("fpgen-model", false, String(e));
}
let tmp: string;
beforeEach(() => {
tmp = fs.mkdtempSync(path.join(os.tmpdir(), "camoufox-fallback-"));
vi.resetModules();
});
afterEach(() => {
vi.doUnmock("../src/paths.js");
fs.rmSync(tmp, { recursive: true, force: true });
});
const missing = (file: string) => (dir: string) =>
fs.rmSync(path.join(dir, file));
const corrupt = (file: string) => (dir: string) =>
fs.writeFileSync(path.join(dir, file), "{not json");
/** fingerprints.ts and warnings.ts reading a data directory `broken` has altered. */
async function withData(broken: (dir: string) => void) {
const data = path.join(tmp, "data-files");
fs.mkdirSync(data, { recursive: true });
for (const name of DATA_FILES)
fs.copyFileSync(path.join(LOCAL_DATA, name), path.join(data, name));
broken(data);
vi.doMock("../src/paths.js", async (importOriginal) => ({
...(await importOriginal<typeof import("../src/paths.js")>()),
LOCAL_DATA: data,
}));
await (await import("../src/fpgen/index.js")).ensureModel();
return {
fp: await import("../src/fingerprints.js"),
warnings: await import("../src/warnings.js"),
};
}
/** The one FallbackWarning `fn` emitted, checked for its report block. */
async function report(
warnings: typeof import("../src/warnings.js"),
fn: () => unknown,
): Promise<{ result: any; text: string }> {
const { result, error, warnings: caught } = await warnings.recordWarnings(fn);
if (error) throw error;
const fallbacks = caught.filter((w) => w.category === "FallbackWarning");
expect(fallbacks).toHaveLength(1);
const text = fallbacks[0].message;
for (const part of [REPORT, "camoufox:", "node:", "os:", "error:"]) {
expect(text).toContain(part);
}
return { result, text };
}
function preset(fp: typeof import("../src/fingerprints.js")) {
const first = fp.loadPresets("152")?.presets.windows[0];
return { ...first, fonts: ["Arial"] };
}
it("a failed preset font draw falls back to the preset's fonts", async () => {
const { fp, warnings } = await withData(missing("fonts.json"));
const { result: config, text } = await report(warnings, () =>
fp.fromPreset(preset(fp), "152"),
);
expect(text).toContain("ENOENT");
expect(text).toContain("the preset's recorded fonts");
expect(config.fonts).toContain("Arial");
});
it("a failed preset voice draw warns", async () => {
const { fp, warnings } = await withData(corrupt("voice-manifests.json"));
const { text } = await report(warnings, () =>
fp.fromPreset(preset(fp), "152"),
);
expect(text).toContain("error: SyntaxError:");
});
describe.skipIf(!modelReady)("context draws", () => {
it.each([
["fonts.json", missing("fonts.json"), "fonts", "ENOENT"],
[
"voice-manifests.json",
corrupt("voice-manifests.json"),
"voices",
"SyntaxError",
],
])("an unreadable %s leaves the launch-time value", async (_file, broken, key, error) => {
const { fp, warnings } = await withData(broken);
const { result: context, text } = await report(warnings, () =>
fp.generateContextFingerprint({ os: "linux" }),
);
expect(text).toContain(error);
expect(key in context.config).toBe(false);
});
});
it.each([
"font-groups.json",
"font-bases.json",
])("an unreadable %s warns and is read as empty", async (file) => {
const { fp, warnings } = await withData(missing(file));
const { result: fonts, text } = await report(warnings, () =>
fp.generateRandomFontSubset("windows", 1),
);
expect(text).toContain(`Reading ${file}`);
expect(fonts.length).toBeGreaterThan(0);
});
it("an error the data cannot raise is not swallowed", async () => {
const { fp } = await withData((dir) =>
fs.writeFileSync(path.join(dir, "fonts.json"), "null"),
);
expect(() => fp.fromPreset(preset(fp), "152")).toThrow(TypeError);
});
typescript/tests/fingerprints.test.ts
+890
View File
@@ -0,0 +1,890 @@
/**
* The identity layer's behaviour: ports of pythonlib/tests/
* test_fingerprint_fixes.py, test_preset_appversion.py, test_voices.py,
* test_font_distribution.py and the unit half of test_identity_salt.py.
* Exact parity with Python is pinned separately in identity-golden.test.ts.
*/
import * as fs from "node:fs";
import * as path from "node:path";
import { describe, expect, it } from "vitest";
import { InvalidIP } from "../src/exceptions.js";
import {
appVersionFromUserAgent,
audioSeedFromIdentity,
buildInitScript,
clampScreenToDisplay,
clampWindowDimensions,
clampWindowPosition,
drawMediaDevices,
fixHardwareConcurrency,
fixNavigatorArch,
fixScreenNoTaskbar,
fromFpgen,
fromPreset,
generateContextFingerprint,
generateFingerprint,
generateRandomFontSubset,
generateRandomVoiceSubset,
getRandomPreset,
identitySalt,
identitySeed,
normalizePresetVoices,
PLAUSIBLE_CORE_COUNTS,
type Preset,
Screen,
setMediaDevicesDefaults,
WINDOWS_11_MARKER_FONTS,
} from "../src/fingerprints.js";
import { LOCAL_DATA } from "../src/pkgman.js";
import { MODEL } from "./fpgen-setup.js";
function data(name: string): any {
return JSON.parse(fs.readFileSync(path.join(LOCAL_DATA, name), "utf-8"));
}
describe("fixNavigatorArch", () => {
it("corrects armv81 to the UA arch", () => {
const c: Record<string, any> = {
"navigator.userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:135.0) ...",
"navigator.platform": "Linux armv81",
"navigator.oscpu": "Linux armv81",
};
fixNavigatorArch(c, "lin");
expect(c["navigator.platform"]).toBe("Linux x86_64");
expect(c["navigator.oscpu"]).toBe("Linux x86_64");
});
it("only runs on Linux, and only with a UA", () => {
const mac = {
"navigator.userAgent": "... Macintosh ...",
"navigator.platform": "MacIntel",
};
fixNavigatorArch(mac, "mac");
expect(mac["navigator.platform"]).toBe("MacIntel");
const noUa = { "navigator.platform": "Linux armv81" };
fixNavigatorArch(noUa, "lin");
expect(noUa["navigator.platform"]).toBe("Linux armv81");
});
});
describe("fixScreenNoTaskbar", () => {
it("subtracts the taskbar when avail equals the screen", () => {
const c: Record<string, number> = {
"screen.width": 1920,
"screen.height": 1080,
"screen.availWidth": 1920,
"screen.availHeight": 1080,
"window.outerHeight": 1080,
"window.innerHeight": 1040,
};
fixScreenNoTaskbar(c, "lin");
expect(c["screen.availHeight"]).toBe(1080 - 27);
expect(c["window.outerHeight"]).toBe(1053);
expect(c["window.innerHeight"]).toBe(1053 - 40);
});
it("uses each OS's taskbar height", () => {
for (const [os, px] of [
["win", 40],
["mac", 25],
["lin", 27],
] as const) {
const c: Record<string, number> = {
"screen.width": 1920,
"screen.height": 1080,
"screen.availWidth": 1920,
"screen.availHeight": 1080,
};
fixScreenNoTaskbar(c, os);
expect(c["screen.availHeight"]).toBe(1080 - px);
}
});
it("is a no-op when avail is already below the screen", () => {
const c = {
"screen.width": 1920,
"screen.height": 1080,
"screen.availWidth": 1920,
"screen.availHeight": 1040,
};
fixScreenNoTaskbar(c, "lin");
expect(c["screen.availHeight"]).toBe(1040);
});
});
describe("clampWindowDimensions", () => {
it("clamps impossible geometry on both axes", () => {
const c: Record<string, number> = {
"screen.width": 1920,
"screen.height": 1080,
"screen.availWidth": 2000,
"window.outerWidth": 2200,
"window.innerWidth": 2100,
};
clampWindowDimensions(c);
expect(c["screen.availWidth"]).toBe(1920);
expect(c["window.outerWidth"]).toBe(1920);
expect(c["window.innerWidth"]).toBeLessThanOrEqual(c["window.outerWidth"]);
});
it("preserves the chrome delta", () => {
const c: Record<string, number> = {
"screen.width": 1000,
"window.outerWidth": 1200,
"window.innerWidth": 1180,
};
clampWindowDimensions(c);
expect(c["window.outerWidth"]).toBe(1000);
expect(c["window.innerWidth"]).toBe(980);
});
it("leaves a valid hierarchy alone", () => {
const c = {
"screen.width": 1920,
"screen.availWidth": 1920,
"window.outerWidth": 1280,
"window.innerWidth": 1264,
};
clampWindowDimensions(c);
expect(c["window.outerWidth"]).toBe(1280);
expect(c["window.innerWidth"]).toBe(1264);
});
});
describe("clampScreenToDisplay", () => {
it("shrinks the screen to the display, keeping the taskbar delta", () => {
const c: Record<string, number> = {
"screen.width": 2560,
"screen.height": 1440,
"screen.availWidth": 2560,
"screen.availHeight": 1400,
};
clampScreenToDisplay(c, 1366, 768);
expect(c).toEqual({
"screen.width": 1366,
"screen.height": 768,
"screen.availWidth": 1366,
"screen.availHeight": 728,
});
});
it("ignores unset bounds and never drops avail below one", () => {
const c = { "screen.width": 2560, "screen.height": 1440 };
clampScreenToDisplay(c, null, null);
expect(c).toEqual({ "screen.width": 2560, "screen.height": 1440 });
const tall = { "screen.height": 2000, "screen.availHeight": 100 };
clampScreenToDisplay(tall, null, 768);
expect(tall["screen.availHeight"]).toBeGreaterThanOrEqual(1);
});
it("survives the clampWindowDimensions cascade", () => {
const c: Record<string, number> = {
"screen.width": 2560,
"screen.height": 1440,
"screen.availWidth": 2560,
"screen.availHeight": 1400,
"window.outerWidth": 1920,
"window.outerHeight": 1055,
"window.innerWidth": 1920,
"window.innerHeight": 1000,
};
clampScreenToDisplay(c, 1366, 768);
clampWindowDimensions(c);
expect(c["window.outerWidth"]).toBeLessThanOrEqual(c["screen.availWidth"]);
expect(c["screen.availWidth"]).toBeLessThanOrEqual(1366);
expect(c["window.outerHeight"]).toBeLessThanOrEqual(
c["screen.availHeight"],
);
expect(c["window.innerHeight"]).toBeLessThanOrEqual(
c["window.outerHeight"],
);
});
});
describe("clampWindowPosition", () => {
it("pulls the window back inside the screen, never negative", () => {
const c: Record<string, number> = {
"screen.width": 1366,
"screen.height": 768,
"window.outerWidth": 1366,
"window.outerHeight": 728,
"window.screenX": 250,
"window.screenY": 281,
};
clampWindowPosition(c);
expect([c["window.screenX"], c["window.screenY"]]).toEqual([0, 40]);
const wide: Record<string, number> = {
"screen.width": 800,
"window.outerWidth": 1000,
"window.screenX": 50,
};
clampWindowPosition(wide);
expect(wide["window.screenX"]).toBe(0);
});
});
describe("media devices", () => {
it("draws common desktop devices, seeded by the identity", () => {
const c: Record<string, any> = {
"navigator.userAgent": "ua",
"navigator.platform": "Win32",
};
setMediaDevicesDefaults(c);
expect(c["mediaDevices:enabled"]).toBe(true);
for (const [kind, key] of [
["micros", "microphone"],
["webcams", "webcam"],
["speakers", "speaker"],
]) {
const n = c[`mediaDevices:${kind}`];
expect(c[`mediaDevices:${key}Labels`]).toHaveLength(n);
expect(c[`mediaDevices:${key}Groups`]).toHaveLength(n);
}
expect(c["mediaDevices:speakers"]).toBeGreaterThanOrEqual(1);
expect(
c["mediaDevices:speakerLabels"].every((s: string) => s.includes("(")),
).toBe(true);
const again: Record<string, any> = {
"navigator.userAgent": "ua",
"navigator.platform": "Win32",
};
setMediaDevicesDefaults(again);
expect(again).toEqual(c);
let mics = 0;
let cams = 0;
for (let i = 0; i < 400; i++) {
const d: Record<string, any> = {
"navigator.userAgent": `ua${i}`,
"navigator.platform": "Win32",
};
setMediaDevicesDefaults(d);
mics += d["mediaDevices:micros"] > 0 ? 1 : 0;
cams += d["mediaDevices:webcams"] > 0 ? 1 : 0;
}
expect(mics / 400).toBeGreaterThan(0.8);
expect(mics / 400).toBeLessThan(1.0);
expect(cams / 400).toBeGreaterThan(0.55);
expect(cams / 400).toBeLessThan(0.95);
});
it("labels devices in each OS's style", () => {
const mac = drawMediaDevices("mac", 7);
expect(
mac["mediaDevices:microphoneLabels"].some((s: string) =>
s.startsWith("Microphone ("),
),
).toBe(false);
const lin = drawMediaDevices("lin", 7);
for (const o of lin["mediaDevices:speakerLabels"]) {
expect(lin["mediaDevices:microphoneLabels"]).toContain(`Monitor of ${o}`);
}
const win = drawMediaDevices("win", 11);
if (win["mediaDevices:micros"] && win["mediaDevices:speakers"]) {
expect(win["mediaDevices:microphoneGroups"][0]).toBe(
win["mediaDevices:speakerGroups"][0],
);
}
for (const os of ["win", "mac", "lin"]) {
for (let seed = 0; seed < 50; seed++) {
const d = drawMediaDevices(os, seed);
const m: string[] = d["mediaDevices:microphoneLabels"];
const cams: string[] = d["mediaDevices:webcamLabels"];
expect(m).not.toContain("Default Audio Device");
expect(cams).not.toContain("Default Video Device");
expect(new Set(m).size).toBe(m.length);
expect(new Set(cams).size).toBe(cams.length);
}
}
});
it("respects caller-set mediaDevices keys", () => {
const c = { "mediaDevices:webcams": 5 };
setMediaDevicesDefaults(c);
expect(c).toEqual({ "mediaDevices:webcams": 5 });
});
});
describe("fixHardwareConcurrency", () => {
it("keeps a plausible draw the host can be pinned to", () => {
for (const drawn of [4, 6, 8, 10, 12, 14, 16]) {
const c = { "navigator.hardwareConcurrency": drawn };
fixHardwareConcurrency(c, undefined, { cpuCount: 16, canPinHost: true });
expect(c["navigator.hardwareConcurrency"]).toBe(drawn);
}
});
it("snaps implausible draws down into the table", () => {
for (const [drawn, want] of [
[1, 4],
[2, 4],
[3, 4],
[5, 4],
[7, 6],
[9, 8],
[11, 10],
[13, 12],
[15, 14],
[32, 16],
]) {
const c = { "navigator.hardwareConcurrency": drawn };
fixHardwareConcurrency(c, undefined, { cpuCount: 16, canPinHost: true });
expect(c["navigator.hardwareConcurrency"], String(drawn)).toBe(want);
}
const c = { "navigator.hardwareConcurrency": 2 };
fixHardwareConcurrency(c, undefined, { cpuCount: 4, canPinHost: true });
expect(c["navigator.hardwareConcurrency"]).toBe(4);
});
it("snaps host parallelism when it cannot pin", () => {
for (const [host, want] of [
[16, 16],
[10, 10],
[24, 24],
[26, 24],
[32, 32],
[64, 32],
[22, 22],
[7, 6],
[5, 4],
[2, 4],
[9, 8],
]) {
const c = { "navigator.hardwareConcurrency": 2 };
fixHardwareConcurrency(c, undefined, {
cpuCount: host,
canPinHost: false,
});
expect(c["navigator.hardwareConcurrency"], String(host)).toBe(want);
}
const c = { "navigator.hardwareConcurrency": 32 };
fixHardwareConcurrency(c, undefined, { cpuCount: 8, canPinHost: true });
expect(c["navigator.hardwareConcurrency"]).toBe(8);
});
it("reports the host when the launch will not pin", () => {
const c = { "navigator.hardwareConcurrency": 8 };
fixHardwareConcurrency(c, false, { cpuCount: 16, canPinHost: true });
expect(c["navigator.hardwareConcurrency"]).toBe(16);
});
it("reports the table floor on a small pinnable host", () => {
for (const host of [1, 2, 3]) {
for (const drawn of [1, 2, 3, 8]) {
const c = { "navigator.hardwareConcurrency": drawn };
fixHardwareConcurrency(c, undefined, {
cpuCount: host,
canPinHost: true,
});
expect(c["navigator.hardwareConcurrency"]).toBe(4);
}
}
});
it("leaves the draw without a host count", () => {
const c = { "navigator.hardwareConcurrency": 8 };
fixHardwareConcurrency(c, undefined, { cpuCount: null });
expect(c["navigator.hardwareConcurrency"]).toBe(8);
});
it("keeps the recorded counts in the table", () => {
for (const n of [18, 22, 28, 32])
expect(PLAUSIBLE_CORE_COUNTS).toContain(n);
});
});
describe("preset appVersion (test_preset_appversion.py)", () => {
const preset = (platform: string, userAgent: string) => ({
navigator: { platform, userAgent },
});
it("follows the preset platform", () => {
for (const [platform, ua, want] of [
[
"Linux x86_64",
"Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0",
"5.0 (X11)",
],
[
"Win32",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:152.0) Gecko/20100101 Firefox/152.0",
"5.0 (Windows)",
],
[
"MacIntel",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:152.0) Gecko/20100101 Firefox/152.0",
"5.0 (Macintosh)",
],
]) {
expect(
fromPreset(preset(platform, ua), null, 0)["navigator.appVersion"],
).toBe(want);
}
});
it("keeps a captured appVersion, follows an unknown platform's UA", () => {
const p: any = preset(
"Win32",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:152.0) Firefox/152.0",
);
p.navigator.appVersion = "5.0 (Windows NT 10.0; Win64; x64)";
expect(fromPreset(p, null, 0)["navigator.appVersion"]).toBe(
"5.0 (Windows NT 10.0; Win64; x64)",
);
expect(
fromPreset(
preset(
"iPhone",
"Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) Gecko/20100101",
),
null,
0,
)["navigator.appVersion"],
).toBe("5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X)");
});
it("matches what Firefox reports", () => {
for (const [ua, want] of [
[
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0",
"5.0 (X11; Ubuntu)",
],
[
"Mozilla/5.0 (Android 16; Mobile; rv:152.0) Gecko/152.0 Firefox/152.0",
"5.0 (Android 16)",
],
]) {
expect(appVersionFromUserAgent(ua)).toBe(want);
}
});
it("leaves a user agent it cannot read alone", () => {
expect(
"navigator.appVersion" in
fromPreset(preset("Win32", "not a user agent"), null, 0),
).toBe(false);
});
it("rewrites the Firefox version when asked", () => {
const c = fromPreset(
preset(
"Linux x86_64",
"Mozilla/5.0 (X11; Linux x86_64; rv:135.0) Gecko/20100101 Firefox/135.0",
),
"152",
0,
);
expect(c["navigator.userAgent"]).toBe(
"Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0",
);
});
});
const REQUIRED_VOICE_FIELDS = [
"lang",
"name",
"voiceUri",
"isDefault",
"isLocalService",
];
describe("voices (test_voices.py)", () => {
for (const os of ["macos", "windows", "linux"]) {
it(`${os}: non-empty full objects, none marked default`, () => {
const voices = generateRandomVoiceSubset(os, "en-US");
expect(voices.length).toBeGreaterThan(0);
for (const v of voices) {
for (const f of REQUIRED_VOICE_FIELDS) expect(v).toHaveProperty(f);
}
expect(voices.some((v) => v.isDefault)).toBe(false);
});
}
it("uses the Windows display language pack", () => {
expect(generateRandomVoiceSubset("windows", "de-DE", 1)[0].lang).toBe(
"de-DE",
);
expect(
generateRandomVoiceSubset("windows", "en-US", 1)
.slice(0, 3)
.map((v) => v.name),
).toEqual([
"Microsoft David - English (United States)",
"Microsoft Mark - English (United States)",
"Microsoft Zira - English (United States)",
]);
});
it("builds Linux speechd URIs like SpeechDispatcherService.cpp", () => {
const lin = generateRandomVoiceSubset("linux", "en-US");
for (const v of lin) {
expect(v.voiceUri.startsWith("urn:moz-tts:speechd:")).toBe(true);
expect(v.voiceUri.endsWith(`?${v.lang}`)).toBe(true);
expect(v.isLocalService).toBe(true);
}
const gb = lin.find((v) => v.name === "English (Great Britain)");
expect(gb?.voiceUri).toBe(
"urn:moz-tts:speechd:English%20(Great%20Britain)?en-GB",
);
});
it("is deterministic per seed and falls back to macOS for an unknown OS", () => {
expect(generateRandomVoiceSubset("windows", "fr-FR", 5)).toEqual(
generateRandomVoiceSubset("windows", "fr-FR", 5),
);
expect(generateRandomVoiceSubset("plan9", "en-US").length).toBeGreaterThan(
0,
);
});
it("normalizes preset voices", () => {
const out = normalizePresetVoices(
["Albert:en-US:local", "Alice:it-IT:local"],
"macos",
);
for (const v of out) {
for (const f of REQUIRED_VOICE_FIELDS) expect(v).toHaveProperty(f);
}
expect([out[0].name, out[0].lang]).toEqual(["Albert", "en-US"]);
expect(out.filter((v) => v.isDefault)).toHaveLength(1);
const obj = {
name: "Alex",
lang: "en-US",
voiceUri: "urn:moz-tts:osx:alex",
isDefault: true,
isLocalService: true,
};
expect(normalizePresetVoices([obj], "macos")).toEqual([obj]);
});
});
describe("font distribution (test_font_distribution.py)", () => {
const OS_KEYS: Record<string, string> = {
windows: "win",
macos: "mac",
linux: "lin",
};
const N = 1500;
const BASES = data("font-bases.json");
const GROUPS = data("font-groups.json");
const REPORTABLE = data("fonts.json");
const samples: Record<string, Set<string>[]> = {};
for (const os of Object.keys(OS_KEYS)) {
samples[os] = Array.from(
{ length: N },
(_, i) => new Set(generateRandomFontSubset(os, i)),
);
}
const tolerance = (p: number, n = N, sigmas = 5.0, floor = 0.03) =>
Math.max(floor, sigmas * Math.sqrt(Math.max(p * (1 - p), 1e-6) / n));
const union = (sets: Iterable<string>[]) => {
const out = new Set<string>();
for (const s of sets) for (const x of s) out.add(x);
return out;
};
const unitExclusive = (osKey: string, unit: any) => {
const others = union(
GROUPS[osKey]
.filter((u: any) => u.id !== unit.id)
.map((u: any) => u.fonts),
);
const inBase = union(BASES[osKey].map((b: any) => b.fonts));
return new Set<string>(
unit.fonts.filter((f: string) => !others.has(f) && !inBase.has(f)),
);
};
for (const os of Object.keys(OS_KEYS)) {
const key = OS_KEYS[os];
it(`${os}: every draw contains one complete base`, () => {
const bases = BASES[key].map((b: any) => new Set<string>(b.fonts));
samples[os].forEach((fonts, i) => {
const ok = bases.some((b: Set<string>) =>
[...b].every((f) => fonts.has(f)),
);
expect(ok, `draw #${i}`).toBe(true);
});
});
it(`${os}: base weights and unit probabilities match the manifest`, () => {
const bases = BASES[key];
if (bases.length >= 2) {
for (const b of bases) {
const others = union(
bases.filter((o: any) => o.id !== b.id).map((o: any) => o.fonts),
);
const ex = b.fonts.filter((f: string) => !others.has(f));
if (!ex.length) continue;
const seen =
samples[os].filter((fonts) => ex.every((f: string) => fonts.has(f)))
.length / N;
expect(Math.abs(seen - b.weight), `base ${b.id}`).toBeLessThanOrEqual(
tolerance(b.weight),
);
}
}
let checked = 0;
for (const unit of GROUPS[key]) {
if (unit.requiresLocale) continue;
const ex = unitExclusive(key, unit);
if (!ex.size) continue;
const hits = samples[os].filter((fonts) =>
[...ex].some((f) => fonts.has(f)),
).length;
checked++;
expect(
Math.abs(hits / N - unit.prob),
`unit ${unit.id}`,
).toBeLessThanOrEqual(tolerance(unit.prob));
}
expect(checked).toBeGreaterThan(0);
});
it(`${os}: bundles are all-or-nothing, a-la-carte units piecemeal`, () => {
let alacarte = 0;
for (const unit of GROUPS[key]) {
const ex = unitExclusive(key, unit);
if (unit.kind === "bundle" && ex.size >= 2) {
for (const fonts of samples[os]) {
const present = [...ex].filter((f) => fonts.has(f)).length;
expect(
present === 0 || present === ex.size,
`bundle ${unit.id}`,
).toBe(true);
}
}
if (unit.kind === "alacarte" && ex.size >= 4) {
alacarte++;
const counts = new Set(
samples[os].map(
(fonts) => [...ex].filter((f) => fonts.has(f)).length,
),
);
expect(
[...counts].some((c) => c > 0 && c < ex.size),
`alacarte ${unit.id}`,
).toBe(true);
}
}
expect(alacarte).toBeGreaterThan(0);
});
it(`${os}: draws vary, stay renderable, repeat per seed, never duplicate`, () => {
const lists = samples[os].map((s) => [...s].sort().join("\n"));
const counts = new Map<string, number>();
for (const l of lists) counts.set(l, (counts.get(l) ?? 0) + 1);
expect(counts.size).toBeGreaterThanOrEqual(50);
expect(Math.max(...counts.values()) / N).toBeLessThanOrEqual(0.5);
expect(
new Set(samples[os].map((s) => s.size)).size,
).toBeGreaterThanOrEqual(5);
const pool = new Set<string>(REPORTABLE[key]);
for (const fonts of samples[os]) {
for (const f of fonts) expect(pool.has(f), f).toBe(true);
}
for (const seed of [1, 7, 99]) {
expect(generateRandomFontSubset(os, seed)).toEqual(
generateRandomFontSubset(os, seed),
);
}
for (let i = 0; i < 200; i++) {
const list = generateRandomFontSubset(os, i);
expect(new Set(list).size).toBe(list.length);
}
});
}
it("every Windows identity presents Windows 11", () => {
for (let i = 0; i < 50; i++) {
const fonts = new Set(generateRandomFontSubset("windows", i));
for (const f of WINDOWS_11_MARKER_FONTS) expect(fonts.has(f)).toBe(true);
}
});
it("a native identity claims only the OS base", () => {
const native = generateRandomFontSubset("linux", 0, true);
expect(native).toEqual(generateRandomFontSubset("linux", 99, true));
});
});
describe("identity salt and seed (test_identity_salt.py)", () => {
it("unpinned salts differ", () => {
const config = {
"navigator.userAgent": "x",
"navigator.platform": "Win32",
"screen.width": 1920,
"screen.height": 1080,
"navigator.hardwareConcurrency": 8,
};
const seeds = new Set(
Array.from({ length: 200 }, () => identitySeed(config, identitySalt())),
);
expect(seeds.size).toBe(200);
});
it("the salt of equal objects is equal", () => {
expect(identitySalt({ a: 1, b: 2 })).toBe(identitySalt({ b: 2, a: 1 }));
const s = new Screen({ maxWidth: 1920, maxHeight: 1080 });
expect(identitySalt(s)).toBe(
identitySalt(new Screen({ maxWidth: 1920, maxHeight: 1080 })),
);
});
it("derives the audio seed without losing precision", () => {
// (ident * 2654435761 + 97) & 0xFFFFFFFF, checked with Python ints
expect(audioSeedFromIdentity(4294967295)).toBe(1640531632);
expect(audioSeedFromIdentity(0)).toBe(97);
});
it("a pinned preset reproduces its draws", () => {
const preset = getRandomPreset("windows", "150") as Preset;
expect(preset).not.toBeNull();
const salt = identitySalt(preset);
const a = fromPreset(structuredClone(preset), "150", salt);
const b = fromPreset(structuredClone(preset), "150", salt);
expect(a.fonts).toEqual(b.fonts);
expect(a.voices).toEqual(b.voices);
});
});
describe("buildInitScript", () => {
it("guards every setter and always clears the WebRTC IP", () => {
const script = buildInitScript({
navigatorPlatform: "Win32",
hardwareConcurrency: 8,
screenWidth: 1920,
screenHeight: 1080,
fontList: ["Arial"],
speechVoices: [{ name: "A" } as any],
});
for (const line of script.split("\n").slice(2, -1)) {
expect(line).toMatch(
/^ {2}if \(typeof w\.\w+ === "function"\) w\.\w+\(.*\);$/,
);
}
expect(script).toContain('w.setWebRTCIPv4("")');
expect(buildInitScript({})).not.toContain("setFontList");
});
it("hands an IPv4 WebRTC address to the IPv4 setter", () => {
const script = buildInitScript({ webrtcIP: "203.0.113.7" });
expect(script).toContain('w.setWebRTCIPv4("203.0.113.7")');
expect(script).not.toContain("setWebRTCIPv6(");
});
it("hands an IPv6 WebRTC address to the IPv6 setter", () => {
const script = buildInitScript({ webrtcIP: "2001:db8::7" });
expect(script).toContain('w.setWebRTCIPv6("2001:db8::7")');
expect(script.split("setWebRTCIPv6")[0]).not.toContain("2001:db8::7");
});
it("refuses an invalid WebRTC address", () => {
expect(() => buildInitScript({ webrtcIP: "not-an-ip" })).toThrow(InvalidIP);
});
});
describe("fromFpgen", () => {
it("maps navigator/screen/window/headers and rewrites the Firefox version", () => {
const config = fromFpgen(
{
navigator: {
userAgent:
"Mozilla/5.0 (X11; Linux x86_64; rv:146.0) Gecko/20100101 Firefox/146.0",
platform: "Linux x86_64",
hardwareConcurrency: 8,
deviceMemory: "undefined",
},
screen: { width: 1920, height: 1080, availLeft: -3 },
window: { screenX: 10, innerWidth: 1900 },
headers: { "accept-encoding": ["gzip, deflate, br, zstd"] },
},
"152",
);
expect(config).toEqual({
"navigator.userAgent":
"Mozilla/5.0 (X11; Linux x86_64; rv:152.0) Gecko/20100101 Firefox/152.0",
"navigator.platform": "Linux x86_64",
"navigator.hardwareConcurrency": 8,
"screen.width": 1920,
"screen.height": 1080,
"screen.availLeft": 0,
"window.screenX": 10,
"window.screenY": 10,
"headers.Accept-Encoding": "gzip, deflate, br, zstd",
});
});
});
describe.skipIf(!MODEL.ok)("fpgen generation (needs the model)", () => {
it("generates a Firefox fingerprint for each OS", () => {
for (const [os, platform] of [
["windows", "Win32"],
["macos", "MacIntel"],
["linux", "Linux"],
]) {
const f = generateFingerprint({ os });
expect(String(f.navigator.platform)).toContain(platform);
expect(String(f.navigator.userAgent)).toContain("Firefox");
}
});
it("does not carry a drawn scroll offset into the config", () => {
// The browser returns screen.pageYOffset from scrollY on every read, so
// a drawn value froze the page at one scroll position.
const fingerprint = generateFingerprint({ os: "windows" });
fingerprint.window = {
...fingerprint.window,
pageXOffset: 17,
pageYOffset: 528,
};
const config = fromFpgen(fingerprint, "152");
expect("screen.pageXOffset" in config).toBe(false);
expect("screen.pageYOffset" in config).toBe(false);
});
it("honours the screen bound, best-effort", () => {
const f = generateFingerprint({ screen: new Screen({ maxWidth: 1400 }) });
expect(f.screen.width).toBeLessThanOrEqual(1400);
// A bound the pool cannot meet falls back to an unbounded draw.
expect(() =>
generateFingerprint({ screen: new Screen({ maxWidth: 1 }) }),
).not.toThrow();
});
it("applies a custom window size, centred", () => {
const f = generateFingerprint({ os: "linux", window: [800, 600] });
expect([f.window.outerWidth, f.window.outerHeight]).toEqual([800, 600]);
});
it("accepts a list of OSes (Python raises InvalidConstraints here)", () => {
// Divergence by design: fpgen passes predicates the casefolded value,
// so Python's `v in {'Linux', 'Windows'}` never matches; TS compares
// casefolded.
const platforms = new Set<string>();
for (let i = 0; i < 20; i++) {
const f = generateFingerprint({ os: ["linux", "windows"] });
const p = String(f.navigator.platform);
expect(p.includes("Linux") || p === "Win32", p).toBe(true);
platforms.add(p === "Win32" ? "win" : "lin");
}
expect(platforms.size).toBe(2);
});
it("rejects an unknown OS", () => {
expect(() => generateFingerprint({ os: "plan9" })).toThrow(/Unknown OS/);
});
it("builds a context fingerprint from fpgen and from a preset", () => {
const gen = generateContextFingerprint({ os: "windows", locale: "de-DE" });
expect(gen.config.fonts.length).toBeGreaterThan(0);
expect(gen.context_options.locale).toBe("de-DE");
expect(gen.init_script).toContain("setNavigatorUserAgent");
const preset = getRandomPreset("macos", "150");
const fromP = generateContextFingerprint({
preset,
timezone: "Europe/Paris",
config_overrides: { "audio:seed": 7 },
});
expect(fromP.context_options.timezoneId).toBe("Europe/Paris");
expect(fromP.config["navigator.platform"]).toBe("MacIntel");
});
});
typescript/tests/fixtures/.gitignore
+6
View File
@@ -0,0 +1,6 @@
# Recorded from pythonlib by tests/golden-setup.ts on every test run.
/fpgen/*
!/fpgen/stats.json
/identity/
/launch/scenario-*.json
/launch/host.json
typescript/tests/fixtures/e2e/probe.js
Vendored
+132
View File
@@ -0,0 +1,132 @@
async () => {
const out = {};
// An await that never settles would hang the whole test with no clue which
// one; this records "timeout: <what>" in the result instead.
const within = (promise, what, ms = 10000) =>
Promise.race([
promise,
new Promise((_, reject) =>
setTimeout(() => reject(new Error(`timeout: ${what} (${ms} ms)`)), ms),
),
]);
const nav = navigator;
out.navigator = {
userAgent: nav.userAgent,
platform: nav.platform,
oscpu: nav.oscpu,
appVersion: nav.appVersion,
hardwareConcurrency: nav.hardwareConcurrency,
maxTouchPoints: nav.maxTouchPoints,
language: nav.language,
languages: [...nav.languages],
doNotTrack: nav.doNotTrack,
globalPrivacyControl: nav.globalPrivacyControl,
webdriver: nav.webdriver,
};
out.screen = {
width: screen.width,
height: screen.height,
availWidth: screen.availWidth,
availHeight: screen.availHeight,
colorDepth: screen.colorDepth,
pixelDepth: screen.pixelDepth,
};
out.window = {
outerWidth: window.outerWidth,
outerHeight: window.outerHeight,
devicePixelRatio: window.devicePixelRatio,
screenX: window.screenX,
screenY: window.screenY,
};
const ro = Intl.DateTimeFormat().resolvedOptions();
out.intl = {
timeZone: ro.timeZone,
locale: ro.locale,
offset: new Date(Date.UTC(2026, 0, 15, 12)).getTimezoneOffset(),
number: new Intl.NumberFormat().format(1234567.891),
};
const fonts = [
"Arial", "Helvetica", "Times New Roman", "Courier New", "Segoe UI", "Calibri",
"DejaVu Sans", "Liberation Sans", "Noto Sans", "Ubuntu", "Menlo", "Monaco",
"Comic Sans MS", "Verdana", "Georgia", "Cantarell", "Tahoma", "Impact",
];
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
const width = (family) => {
ctx.font = `32px "${family}", monospace`;
return ctx.measureText("mmmmmmmmmmlli1WQ@#").width;
};
const base = width("__no_such_font__");
out.fonts = Object.fromEntries(fonts.map((f) => [f, width(f) !== base]));
out.fontWidths = Object.fromEntries(fonts.map((f) => [f, width(f)]));
canvas.width = 220;
canvas.height = 40;
ctx.textBaseline = "top";
ctx.font = "16px Arial";
ctx.fillStyle = "#f60";
ctx.fillRect(10, 1, 62, 20);
ctx.fillStyle = "#069";
ctx.fillText("Camoufox parity <canvas> 1.0", 2, 15);
ctx.beginPath();
ctx.arc(180, 20, 12, 0, Math.PI * 1.5);
ctx.strokeStyle = "rgba(10, 120, 40, 0.7)";
ctx.stroke();
const data = canvas.toDataURL();
// FNV-1a over the data URL: the probe runs in Camoufox's isolated world,
// where handing TypedArrays across the Xray boundary is forbidden.
let h = 0x811c9dc5;
for (let i = 0; i < data.length; i++) {
h ^= data.charCodeAt(i);
h = Math.imul(h, 0x01000193) >>> 0;
}
out.canvasHash = `${h.toString(16)}:${data.length}`;
const gl = document.createElement("canvas").getContext("webgl");
if (gl) {
const dbg = gl.getExtension("WEBGL_debug_renderer_info");
out.webgl = {
vendor: dbg ? gl.getParameter(dbg.UNMASKED_VENDOR_WEBGL) : null,
renderer: dbg ? gl.getParameter(dbg.UNMASKED_RENDERER_WEBGL) : null,
maxTextureSize: gl.getParameter(gl.MAX_TEXTURE_SIZE),
extensions: gl.getSupportedExtensions(),
};
} else {
out.webgl = null;
}
const voices = await new Promise((resolve) => {
const got = speechSynthesis.getVoices();
if (got.length) return resolve(got);
const t = setTimeout(() => resolve(speechSynthesis.getVoices()), 1500);
speechSynthesis.onvoiceschanged = () => {
clearTimeout(t);
resolve(speechSynthesis.getVoices());
};
});
out.voices = voices.map((v) => `${v.name}|${v.lang}|${v.default}|${v.localService}`);
try {
out.storageQuota = (await within(navigator.storage.estimate(), "storage.estimate")).quota;
} catch (e) {
out.storageQuota = `error: ${e}`;
}
out.media = {
colorScheme: matchMedia("(prefers-color-scheme: dark)").matches ? "dark" : "light",
reducedMotion: matchMedia("(prefers-reduced-motion: reduce)").matches,
};
try {
const devices = await within(
navigator.mediaDevices.enumerateDevices(),
"enumerateDevices",
);
out.mediaDevices = devices.map((d) => d.kind).sort();
} catch (e) {
out.mediaDevices = `error: ${e}`;
}
try {
out.permissionsGeo = (
await within(navigator.permissions.query({ name: "geolocation" }), "permissions.query")
).state;
} catch (e) {
out.permissionsGeo = `error: ${e}`;
}
out.historyLength = history.length;
return out;
}
typescript/tests/fixtures/fpgen/stats.json
Vendored
+1
View File
File diff suppressed because one or more lines are too long
typescript/tests/fixtures/launch/addons/example-addon/manifest.json
Vendored
+1
View File
@@ -0,0 +1 @@
{"manifest_version": 2, "name": "example", "version": "1.0"}
typescript/tests/fixtures/launch/bundle-old/application.ini
Vendored
+5
View File
@@ -0,0 +1,5 @@
[App]
Vendor=Mozilla
Name=Camoufox
Version=152.0.4
BuildID=20260914000000
typescript/tests/fixtures/launch/bundle-old/camoufox-bin
Vendored Executable
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
# placeholder executable for launch goldens
exit 1
typescript/tests/fixtures/launch/bundle-old/fontconfigs/linux/fonts.conf
Vendored
+7
View File
@@ -0,0 +1,7 @@
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<!-- Minimal stand-in for the bundled fonts.conf (linux); only the dir line matters. -->
<fontconfig>
<dir prefix="cwd">fonts</dir>
<cachedir prefix="xdg">fontconfig</cachedir>
</fontconfig>
typescript/tests/fixtures/launch/bundle-old/fontconfigs/macos/fonts.conf
Vendored
+7
View File
@@ -0,0 +1,7 @@
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<!-- Minimal stand-in for the bundled fonts.conf (macos); only the dir line matters. -->
<fontconfig>
<dir prefix="cwd">fonts</dir>
<cachedir prefix="xdg">fontconfig</cachedir>
</fontconfig>
typescript/tests/fixtures/launch/bundle-old/fontconfigs/windows/fonts.conf
Vendored
+7
View File
@@ -0,0 +1,7 @@
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<!-- Minimal stand-in for the bundled fonts.conf (windows); only the dir line matters. -->
<fontconfig>
<dir prefix="cwd">fonts</dir>
<cachedir prefix="xdg">fontconfig</cachedir>
</fontconfig>
typescript/tests/fixtures/launch/bundle-old/fonts/linux/.gitkeep
Vendored
View File
typescript/tests/fixtures/launch/bundle-old/fonts/macos/.gitkeep
Vendored
View File
typescript/tests/fixtures/launch/bundle-old/fonts/windows/.gitkeep
Vendored
View File
typescript/tests/fixtures/launch/bundle-old/properties.json
Vendored
+119
View File
@@ -0,0 +1,119 @@
[
{ "property": "navigator.userAgent", "type": "str" },
{ "property": "navigator.doNotTrack", "type": "str" },
{ "property": "navigator.appCodeName", "type": "str" },
{ "property": "navigator.appName", "type": "str" },
{ "property": "navigator.appVersion", "type": "str" },
{ "property": "navigator.oscpu", "type": "str" },
{ "property": "navigator.language", "type": "str" },
{ "property": "navigator.languages", "type": "array" },
{ "property": "navigator.platform", "type": "str" },
{ "property": "navigator.hardwareConcurrency", "type": "uint" },
{ "property": "navigator.product", "type": "str" },
{ "property": "navigator.productSub", "type": "str" },
{ "property": "navigator.maxTouchPoints", "type": "uint" },
{ "property": "navigator.cookieEnabled", "type": "bool" },
{ "property": "navigator.globalPrivacyControl", "type": "bool" },
{ "property": "navigator.buildID", "type": "str" },
{ "property": "navigator.onLine", "type": "bool" },
{ "property": "screen.availHeight", "type": "uint" },
{ "property": "screen.availWidth", "type": "uint" },
{ "property": "screen.availTop", "type": "uint" },
{ "property": "screen.availLeft", "type": "uint" },
{ "property": "screen.height", "type": "uint" },
{ "property": "screen.width", "type": "uint" },
{ "property": "screen.colorDepth", "type": "uint" },
{ "property": "screen.pixelDepth", "type": "uint" },
{ "property": "screen.pageXOffset", "type": "double" },
{ "property": "screen.pageYOffset", "type": "double" },
{ "property": "window.scrollMinX", "type": "int" },
{ "property": "window.scrollMinY", "type": "int" },
{ "property": "window.scrollMaxX", "type": "int" },
{ "property": "window.scrollMaxY", "type": "int" },
{ "property": "window.outerHeight", "type": "uint" },
{ "property": "window.outerWidth", "type": "uint" },
{ "property": "window.innerHeight", "type": "uint" },
{ "property": "window.innerWidth", "type": "uint" },
{ "property": "window.screenX", "type": "int" },
{ "property": "window.screenY", "type": "int" },
{ "property": "window.history.length", "type": "uint" },
{ "property": "window.devicePixelRatio", "type": "double" },
{ "property": "document.body.clientWidth", "type": "uint" },
{ "property": "document.body.clientHeight", "type": "uint" },
{ "property": "document.body.clientTop", "type": "uint" },
{ "property": "document.body.clientLeft", "type": "uint" },
{ "property": "headers.User-Agent", "type": "str" },
{ "property": "headers.Accept-Language", "type": "str" },
{ "property": "headers.Accept-Encoding", "type": "str" },
{ "property": "webrtc:ipv4", "type": "str" },
{ "property": "webrtc:ipv6", "type": "str" },
{ "property": "webrtc:localipv4", "type": "str" },
{ "property": "webrtc:localipv6", "type": "str" },
{ "property": "pdfViewerEnabled", "type": "bool" },
{ "property": "battery:charging", "type": "bool" },
{ "property": "battery:chargingTime", "type": "double" },
{ "property": "battery:dischargingTime", "type": "double" },
{ "property": "battery:level", "type": "double" },
{ "property": "fonts", "type": "array" },
{ "property": "fonts:spacing_seed", "type": "uint" },
{ "property": "audio:seed", "type": "uint" },
{ "property": "canvas:seed", "type": "uint" },
{ "property": "geolocation:latitude", "type": "double" },
{ "property": "geolocation:longitude", "type": "double" },
{ "property": "geolocation:accuracy", "type": "double" },
{ "property": "timezone", "type": "str" },
{ "property": "locale:language", "type": "str" },
{ "property": "locale:region", "type": "str" },
{ "property": "locale:script", "type": "str" },
{ "property": "locale:all", "type": "str" },
{ "property": "humanize", "type": "bool" },
{ "property": "humanize:maxTime", "type": "double" },
{ "property": "humanize:minTime", "type": "double" },
{ "property": "showcursor", "type": "bool" },
{ "property": "AudioContext:sampleRate", "type": "uint" },
{ "property": "AudioContext:outputLatency", "type": "double" },
{ "property": "AudioContext:maxChannelCount", "type": "uint" },
{ "property": "webGl:renderer", "type": "str" },
{ "property": "webGl:vendor", "type": "str" },
{ "property": "webGl:supportedExtensions", "type": "array" },
{ "property": "webGl2:supportedExtensions", "type": "array" },
{ "property": "webGl:parameters", "type": "dict" },
{ "property": "webGl:parameters:blockIfNotDefined", "type": "bool" },
{ "property": "webGl2:parameters", "type": "dict" },
{ "property": "webGl2:parameters:blockIfNotDefined", "type": "bool" },
{ "property": "webGl:shaderPrecisionFormats", "type": "dict" },
{ "property": "webGl:shaderPrecisionFormats:blockIfNotDefined", "type": "bool" },
{ "property": "webGl2:shaderPrecisionFormats", "type": "dict" },
{ "property": "webGl2:shaderPrecisionFormats:blockIfNotDefined", "type": "bool" },
{ "property": "webGl:contextAttributes", "type": "dict" },
{ "property": "webGl2:contextAttributes", "type": "dict" },
{ "property": "canvas:aaOffset", "type": "int" },
{ "property": "canvas:aaCapOffset", "type": "bool" },
{ "property": "voices", "type": "array" },
{ "property": "voices:blockIfNotDefined", "type": "bool" },
{ "property": "voices:fakeCompletion", "type": "bool" },
{ "property": "voices:fakeCompletion:charsPerSecond", "type": "double" },
{ "property": "mediaDevices:micros", "type": "uint" },
{ "property": "mediaDevices:webcams", "type": "uint" },
{ "property": "mediaDevices:speakers", "type": "uint" },
{ "property": "mediaDevices:enabled", "type": "bool" },
{ "property": "mediaDevices:microphoneLabels", "type": "array" },
{ "property": "mediaDevices:microphoneGroups", "type": "array" },
{ "property": "mediaDevices:webcamLabels", "type": "array" },
{ "property": "mediaDevices:webcamGroups", "type": "array" },
{ "property": "mediaDevices:speakerLabels", "type": "array" },
{ "property": "mediaDevices:speakerGroups", "type": "array" },
{ "property": "media:spoof_codecs", "type": "bool" },
{ "property": "allowMainWorld", "type": "bool" },
{ "property": "disableWorldIsolation", "type": "bool" },
{ "property": "allowAddonNewtab", "type": "bool" },
{ "property": "forceScopeAccess", "type": "bool" },
{ "property": "disableTheming", "type": "bool" },
{ "property": "disableInstantAnimations", "type": "bool" },
{ "property": "memorysaver", "type": "bool" },
{ "property": "addons", "type": "array" },
{ "property": "certificatePaths", "type": "array" },
{ "property": "certificates", "type": "array" },
{ "property": "debug", "type": "bool" }
]
typescript/tests/fixtures/launch/bundle-old/version.json
Vendored
+1
View File
@@ -0,0 +1 @@
{"version": "152.0.4", "build": "beta.29"}
typescript/tests/fixtures/launch/bundle/application.ini
Vendored
+5
View File
@@ -0,0 +1,5 @@
[App]
Vendor=Mozilla
Name=Camoufox
Version=152.0.4
BuildID=20260914000000
typescript/tests/fixtures/launch/bundle/camoufox-bin
Vendored Executable
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
# placeholder executable for launch goldens
exit 1
typescript/tests/fixtures/launch/bundle/fontconfig/linux/fonts.conf
Vendored
+7
View File
@@ -0,0 +1,7 @@
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<!-- Minimal stand-in for the bundled fonts.conf (linux); only the dir line matters. -->
<fontconfig>
<dir prefix="cwd">fonts</dir>
<cachedir prefix="xdg">fontconfig</cachedir>
</fontconfig>
typescript/tests/fixtures/launch/bundle/fontconfig/macos/fonts.conf
Vendored
+7
View File
@@ -0,0 +1,7 @@
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<!-- Minimal stand-in for the bundled fonts.conf (macos); only the dir line matters. -->
<fontconfig>
<dir prefix="cwd">fonts</dir>
<cachedir prefix="xdg">fontconfig</cachedir>
</fontconfig>
typescript/tests/fixtures/launch/bundle/fontconfig/windows/fonts.conf
Vendored
+7
View File
@@ -0,0 +1,7 @@
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<!-- Minimal stand-in for the bundled fonts.conf (windows); only the dir line matters. -->
<fontconfig>
<dir prefix="cwd">fonts</dir>
<cachedir prefix="xdg">fontconfig</cachedir>
</fontconfig>
typescript/tests/fixtures/launch/bundle/fonts/L/.gitkeep
Vendored
View File
typescript/tests/fixtures/launch/bundle/fonts/LM/.gitkeep
Vendored
View File
typescript/tests/fixtures/launch/bundle/fonts/LMW/.gitkeep
Vendored
View File

Some files were not shown because too many files have changed in this diff Show More