Conflict: pythonlib/camoufox/utils.py — both sides fixed the fontconfig
cache dir independently (#654 here, #712 upstream). The two spellings
resolve to the same path, since pkgman's INSTALL_DIR is
platformdirs.user_cache_dir("camoufox"). Kept INSTALL_DIR so the module
has one name for that directory, dropped the now-unused platformdirs
import, and kept the comment explaining why the dir must sit outside the
read-only browser bundle.
_generate_fontconfig hardcoded ~/.cache/camoufox/fontconfig instead
of respecting XDG_CACHE_HOME. On systems where ~/.cache is read-only
(e.g. containerized environments), this causes OSError on browser launch.
Replaced os.path.join(os.path.expanduser('~'), '.cache', 'camoufox',
'fontconfig') with os.path.join(platformdirs.user_cache_dir('camoufox'),
'fontconfig'). platformdirs is already a declared dependency and
respects XDG_CACHE_HOME on Linux, ~/Library/Caches on macOS, and
%LOCALAPPDATA% on Windows.
Closes#654
Reverts the default half of 4b20b77. That commit raised Xvfb's root window
from 1x1x24 to 1920x1080x24 for #458, on the reasoning that a 1x1 root
"breaks anything that measures the screen".
That reasoning does not hold here:
- screen.* never comes from the root window. It comes from the generated
fingerprint, applied per context in the browser, and
clamp_screen_to_display() is skipped outright for virtual displays (the
`not virtual_display` guard in utils.py), so a 1x1 root cannot clamp a
generated screen down to 1x1.
- #458's actual symptom -- blank/dark screenshots -- does not reproduce on
152.0.4-beta.28. Measured at both geometries on the same build, same page:
Xvfb 1x1x24 493 distinct colours, 57.5% dominant -> renders
Xvfb 1920x1080x24 493 distinct colours, 55.8% dominant -> renders
Identical. Firefox composites offscreen, so the root window size does not
gate rendering. A full-page screenshot of example.com under 1x1x24 is
pixel-correct.
1x1x24 is Camoufox's long-standing default and has run that way for years.
CAMOUFOX_VIRTUAL_DISPLAY_SIZE is kept as an escape hatch for anyone who does
want a real framebuffer, and still validates its input.
The Composite half of 4b20b77 was already reverted separately in 75d09a3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
After the screencastFrameAck/timestamp fix, recording worked headless but still
produced nothing usable anywhere else: `headless="virtual"` and plain headful
both emitted a valid .webm containing 24 pure-white frames -- Playwright's
filler for a screencast that never delivered a frame.
nsScreencastService only has a working source when the browser is headless
(HeadlessWindowCapturer). Outside headless, CreateWindowCapturer falls through
to libwebrtc's X11 window capturer, which fails three different ways:
* no XComposite -> startVideoRecording() succeeds and then never delivers a
frame. This is Camoufox's own Xvfb configuration, which passes
`-extension COMPOSITE`;
* XComposite enabled -> the browser segfaults during capture (reproduced on
the shipped 152.0.4-beta.28 as well, so it is not specific to this branch);
* Wayland -> nsWindow::GetNativeData(NS_NATIVE_WINDOW_WEBRTC_DEVICE_ID) is
documented as unhandled and returns null, so the service throws
NS_ERROR_FAILURE ("Failed to get native window id") and no capture starts.
Capture from the compositor instead when not headless, via
WindowGlobalParent.drawSnapshot() -- the same call Page.screenshot already
uses, which is why screenshots have always worked in every mode. It renders
page content directly and does not care about the windowing system.
The tick is ack-driven, mirroring nsScreencastService's kMaxFramesInFlight = 1,
so a slow consumer throttles capture rather than queueing JPEGs. Headless keeps
the native C++ capturer, which is cheaper and already correct.
Measured on the packaged Linux build, 3s recording of an animated page, frames
decoded to PNG and inspected rather than trusting file existence:
before after
headless 100 frames, real unchanged, real
headless="virtual" 24 frames, all white 100 frames, real
headful (Xvfb, X11) 24 frames, all white 99 frames, real
headful (Wayland env) no capture at all 99 frames, real
tests/async/test_video.py passes 5/5 both headless and headful. Enabling
Composite no longer crashes either, since X11 window capture is now unused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
9654452 enabled Xvfb's Composite extension on the theory that #93 (no video
under headless="virtual") was caused by disabling it. Measurement disproves it:
composite off + record_video_dir -> valid .webm, 24 pure-white frames
composite ON + record_video_dir -> browser dies with SIGSEGV, no video
composite ON + no recording -> fine
So compositing does not fix#93, and defaulting it on turns a blank recording
into a crash for anyone recording under a virtual display. The segfault
reproduces on the shipped 152.0.4-beta.28 too, so it is a pre-existing fault in
the screencast capture path rather than something this branch introduced -- but
that is exactly why it should not be reached by default.
Kept as an opt-in (CAMOUFOX_VIRTUAL_DISPLAY_COMPOSITE=1) for hosts with real
GL, where it may behave differently. The real-screen-size half of 9654452 is
unaffected and stays.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two hardcoded Xvfb arguments, both verified against a live Xvfb with xdpyinfo.
#458 -- `-screen 0 1x1x24`. A 1x1 root window is not a plausible desktop: it
breaks anything that measures the screen, and it is the reason
clamp_screen_to_display() has to special-case virtual displays (a generated
fingerprint would otherwise be clamped to 1x1). Default to 1920x1080x24;
the framebuffer cost is ~8MB. Overridable per-run with
CAMOUFOX_VIRTUAL_DISPLAY_SIZE="1920x1080[x24]", which is validated and rejects
malformed values rather than passing them to Xvfb.
#93 -- `-extension COMPOSITE`. Offscreen rendering needs Composite, which is
what Playwright's video recording uses, so disabling it silently broke
record_video_dir under headless="virtual". A real X server has the extension,
so enabling it is also the more faithful default. Set
CAMOUFOX_VIRTUAL_DISPLAY_COMPOSITE=0 to restore the old behaviour.
Verified with xdpyinfo against real Xvfb instances:
default -> dimensions 1920x1080, Composite present
screen="800x600x24", composite=False -> dimensions 800x600, Composite absent
CAMOUFOX_VIRTUAL_DISPLAY_SIZE=2560x1440 -> resolves to 2560x1440x24
CAMOUFOX_VIRTUAL_DISPLAY_SIZE=bogus -> VirtualDisplayNotSupported
xvfb_args becomes a property so the two settings can vary per instance; the
existing VirtualDisplay(debug=...) call sites are unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`VirtualDisplay.kill()` reaps the Xvfb child and then clears `self.proc`, so
asserting `vd.proc.poll() is not None` afterwards raises AttributeError on
None. Two tests failed this way on main, unrelated to any of the merged PRs.
Assert `proc is None or proc.poll() is not None` -- reaped-and-cleared is the
success path, and a surviving handle must still report an exit code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PR #398 added `persistent_context` / `user_data_dir` to `launch_options()` and
emitted `_user_data_dir` in the result, on the assumption that Playwright's
`browserServerImpl` consumes it. It does not.
`launchServer()` spreads its options into `BrowserType.launch()`, which passes
`undefined` as the userDataDir and never reads `options._userDataDir` (only
`browser._userDataDirForTest` is ever assigned, after the fact). Verified
against the bundled playwright-core 1.53.1: launching a server with
`user_data_dir=/tmp/...` starts cleanly and leaves the directory empty.
Serving a persistent context is not merely unimplemented, it is outside
Playwright's server model: `launchPersistentContext` returns a BrowserContext
while `PlaywrightServer` only accepts a `preLaunchedBrowser`.
So keep #398's genuinely-correct `camel_case` fix -- it lets any underscore-
prefixed private option reach the driver -- and drop the two options that would
otherwise be accepted, validated, and silently ignored. `launch_server()` now
fails loudly and points at the in-process API instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
get_screen_cons() was gated on DISPLAY being set, which only ever happens on
Linux, so headful runs on Windows and macOS generated fingerprints with no
monitor bound at all.
Fixes#425
screeninfo makes the process per-monitor DPI aware, so it reports physical
pixels, while Firefox lays windows out in CSS pixels. At 150% Windows scaling a
1920x1080 panel is 1280x720 CSS px, so bounding the fingerprint by the physical
size lets the window open 1.5x larger than the screen.
Refs #425
headless='virtual' reaches launch_options as headless=False with
virtual_display set (async_api rewrites it), so the headful gate fired and
clamped the fingerprint to Xvfb's 1x1 stub. fix_screen_no_taskbar then drove
availHeight to -39 and validate_config rejected the launch outright.
get_screen_cons() bounds the generated fingerprint to the monitor, but
BrowserForge honours a Screen constraint only when its pool has a match:
FingerprintGenerator.partial_csp catches the filtering failure and deletes the
constraint unless strict=True. So a 1366x768 laptop routinely gets a 2560x1440
fingerprint with window.outerWidth 1920, and browser-init resizes the real
chrome window to it -- rendering past the edge of the monitor.
Re-apply the bound after generation instead of trusting BrowserForge with it,
and pull screenX/screenY back inside the shrunken screen.
Headful only. headless has no window to overflow, and headless='virtual' runs a
1x1 Xvfb whose "monitor" would otherwise shrink the fingerprint to 1x1.
Fixes#499
PR #678 made the fontconfig cache XDG-aware, but `get_path('fontconfig')`
resolves inside the versioned browser install directory
(.../browsers/official/<version>-<hash>/fontconfig/), which already holds the
bundled linux/ macos/ windows/ trees and is read-only in the common
"bake the browser into the image as root, run as non-root" deployment.
Use INSTALL_DIR / 'fontconfig' instead: still XDG-aware, but outside the
bundle. This is byte-identical to the pre-#678 path when XDG_CACHE_HOME is
unset, so existing caches are reused and no migration is needed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fixes the new_page() hang from #666, and restores the pythonlib/ + settings/
halves of #637-#647 that were dropped when those PRs were consolidated into #666
(that PR only carried patches/ + additions/, so these never actually landed).
## new_page() hangs when window.outer* is spoofed (#666)
The outer-size hijack in browser-init.patch pinned the chrome documentElement to
the spoofed size. That caps .browserStack, which caps the content viewport, so
the content window can never reach the size Juggler asks for in
updateViewportSize() -- and awaitViewportDimensions awaits exact equality with
no timeout, so it deadlocks rather than erroring. The second new_page() hung
forever and took the context with it.
The pin was never load-bearing: GetOuterWidth/GetOuterHeight already consult
MaskConfig unconditionally (fingerprint-injection.patch), so window.outerWidth is
spoofed in C++ regardless of the real chrome window size. Resizing is enough.
Measured on the official v152.0.4-beta.26 build (headless):
config before after
none pass pass
inner pass pass
outer HANG pass
both HANG pass (iw:360 ih:740 ow:360 oh:800 -- exact)
This corrects the diagnosis in #666, which blamed the inner+outer combination and
the `!(outerWidth || outerHeight)` guard. outer* ALONE is sufficient to hang, and
dropping inner* does not help, so that guard is not the culprit.
Also fixed driver-side: Playwright's implicit 1280x720 viewport is what asks for
the impossible size, so the driver now defaults to no_viewport when the config
spoofs any window dimension. That fixes the hang on already-released builds
without a rebuild. An explicit viewport=/no_viewport= from the caller wins.
## WebRTC ICE prefs (#538)
#666 merged the C++ half of the WebRTC fix but not the prefs, so the shipped
build still has no_host=true and none of the proxy_only prefs.
proxy_only_if_behind_proxy is the pref that actually stops the real-IP leak: it
prevents a UDP STUN request routing around a TCP proxy. no_host=false keeps the
stock two-candidate shape, which obfuscate_host_addresses makes leak-free.
## Also restored from the consolidation
- fix(proxy): dom.security.https_first rewrote http:// before the launch-arg
proxy filter saw it, breaking CONNECT-only proxies (#638).
- fix(stealth): speech-voice spoofing + stop leaking host voices (#646).
- fix(stealth): clamp inner <= outer <= avail <= screen; BrowserForge can emit
impossible geometries that leak as tells (#647).
Refs: https://github.com/daijro/camoufox/pull/666
Refs: https://github.com/daijro/camoufox/issues/538
Cover both failure modes from #656 and pin the driver entrypoint
contract, so a future Playwright reshuffle fails in CI rather than in a
user's terminal. No browser download or launch, so they run anywhere.
Refs #656
When the node server exits early, writing its config to the dead stdin
raised BrokenPipeError (EINVAL on Windows), burying the real cause.
communicate() ignores both, so the underlying failure stays visible.
Refs #656
Playwright 1.60 bundled its internals and removed the private
lib/browserServerImpl.js that launchServer.js required, so
`python -m camoufox server` died with MODULE_NOT_FOUND. Load the
driver's package entrypoint instead, which is a bundled playwright-core
and exposes launchServer as public API.
The driver path is now passed explicitly rather than inferred from
process.cwd().
Fixes#656
Fix for JWriter20 repo having several releases with the same version and build name, which broke the manager
- Installs browsers as version-build-sha256/ rather than version-build/ since assets can share a version-build combo
- Doesn't break existing downloads
- Show date column in gui and selector tui
Replace the terminate→wait→kill fallback in VirtualDisplay.kill() with a
direct SIGKILL to prevent zombie Xvfb processes. After the process exits,
remove the stale /tmp/.X{n}-lock and /tmp/.X11-unix/X{n} files so future
display allocations are not blocked. Also set self.proc = None to mark the
display as fully cleaned up.
* fix v150 patches
* screen related patch fixes
* fix juggler issues with 150
* Update grading.py
improved build tester scoring
* fix windows build for v150
- scripts/_mixin.py: switch moz_target from x86_64-pc-mingw32 (no longer
supported in FF150) to x86_64-pc-windows-msvc
- additions/juggler/screencast/HeadlessWindowCapturer.h: typedef pid_t
on XP_WIN; libwebrtc headers (video_capture.h, desktop_capturer.h)
reference pid_t which is POSIX-only
- patches/anti-font-fingerprinting.patch: include mozilla/dom/Document.h
in gfxTextRun.cpp; on Windows it is not transitively included so
doc->GetInnerWindow() failed with "incomplete type 'Document'"
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* make service test use local binary
* updated ff fingerprint versions
* Update README.md
---------
Co-authored-by: Ubuntu <ubuntu@ip-172-31-15-96.us-east-2.compute.internal>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the userspace lock-file scan + random-jitter retry loop with
Xvfb's own -displayfd mechanism. Xvfb scans up from :0 and atomically
binds the first free X11 socket (kernel-mediated, no userspace race),
then writes the chosen display number back through an inherited pipe.
This eliminates the duplicate-display race that occurred when many
camoufox processes started concurrently and all observed the same set
of free display numbers before any of them bound.
Adds a 10s read timeout on the displayfd pipe so a hung Xvfb fails
fast instead of blocking forever, and adds tests covering single
launch, idempotent get(), 50 concurrent reservations with uniqueness,
and post-kill display reuse.
* Add system-ui font spoofing patch
GetSystemUIFontFamilies() now reads navigator.platform from MaskConfig
and returns the appropriate system font (Helvetica for macOS, Segoe UI
for Windows) before falling through to LookAndFeel::GetFont. Fixes the
CreepJS headless.systemFonts "Sans:Linux" leak when spoofing macOS from
Linux.
* Add patch verification test for system-ui font spoofing
Tests macOS and Windows presets: launches Camoufox with a fingerprint,
measures canvas text width with unquoted system-ui, asserts it matches
the expected system font (Helvetica / Segoe UI).
* Rename test to match patch naming convention
tests/patches/ files should mirror patch names, not use dated prefixes.
* Add config_overrides param to generate_context_fingerprint()
There was no clean way to override config values (like disabling font
spacing perturbation with fonts:spacing_seed=0) because init_script is
rendered inside generate_context_fingerprint() — by the time the caller
gets the config dict back, the init_script string is already baked.
config_overrides is applied after all config building (preset/seeds/
timezone/locale) but before init_script rendering, giving callers a
clean override point without touching the preset (which represents
real device data, not perturbation config).
The longer-term fix is separating config building from init_script
rendering so callers can modify config and re-render. config_overrides
is the minimal API addition that unblocks the use case without that
refactor.
* Rename test to match patch naming convention
When using Xvfb-backed virtual display, override Wayland env vars
(GDK_BACKEND, WAYLAND_DISPLAY, MOZ_ENABLE_WAYLAND) so Firefox/GTK
honors DISPLAY and reliably uses the X11 virtual screen.
closes#575
Without FONTCONFIG_FILE, fontconfig loads the system's /etc/fonts/fonts.conf
which includes system font directories. This means system fonts leak into the
rendering pipeline for fallback glyphs (emoji, CJK, etc.), even though the
font enumeration whitelist hides them from JavaScript. The result is
environment-dependent font metrics that differ between machines with different
system font packages installed.
Generate a runtime fontconfig at ~/.cache/camoufox/fontconfig/ that resolves
the bundled font directory absolutely (the bundled fonts.conf uses
prefix="cwd" relative paths which break under Playwright), and point
FONTCONFIG_FILE at it.
The else branch in _build_init_script() reads the system timezone via
Intl.DateTimeFormat().resolvedOptions().timeZone and stores it via
setTimezone(). When geoip resolves a different timezone (set later via
CAMOU_CONFIG/launch_options), the storage value from the init_script
takes precedence — workers read the wrong timezone and the C++ MaskConfig
fallback (PR #546) is never reached.
Fix: only call setTimezone() when an explicit timezone value is provided.
When omitted, timezone propagation is handled entirely by the C++ side
(SetNewDocument + TimezoneManager::GetTimezone MaskConfig fallback).
Companion to #546 (Fix worker timezone leak when using geoip/proxy).
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Allow callers to inject pre-resolved timezone/locale into the
fingerprint before init_script generation. When provided, these
take priority over preset data.
In launch_options(), use setdefault for geoip timezone/locale keys
so that values pre-set via generate_context_fingerprint() aren't
overwritten by geoip resolution.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* example files
* contributing guides
* simple service test
* run tests in sync
* update pr template
* pip updates
* Update README.md
* typo fixes
* undo pip package update lol
* upgraded service test
* undo injections
* test with proxies
* auto set timezone and proxy url
* delete checks bundle
* split up service tests
* split up build tests
* rename service tests to service tester
* Update CONTRIBUTING.md
* fix entry vs exit ip
* allow alpha versions
* fix patch issues on macos
* bidirectional patch
* Add note on experimental pip package
Jake Writer
daijro
D Yiapanis
Aleksandr
Piero Rolando
Cloudymap1e
Pratyush Sharma
Andrei Zhemaituk
daijro
Hunter Boyd
Praveen Senpai
oyaah
LY
icepaq
Jake Writer
Ruben Vereecken