mirror of
https://github.com/daijro/camoufox.git
synced 2026-11-19 00:07:28 +00:00
Merge remote-tracking branch 'upstream/main' into pr/windows-identity
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+58
-2
@@ -167,7 +167,59 @@ host's only when the identity claims the host's GPU. Otherwise
|
||||
|---|---|---|
|
||||
| `False` (default) | A normal window | Needs a display. |
|
||||
| `True` | Firefox headless | Camoufox patches the headless tells it knows about. |
|
||||
| `"virtual"` | A headed browser on a private Xvfb display | Linux only, needs `Xvfb` and `setpriv` (util-linux). Handled by `Camoufox`, `AsyncCamoufox`, `NewBrowser` and `launch_server`. The display is 1x1 unless `CAMOUFOX_VIRTUAL_DISPLAY_SIZE` (e.g. `1920x1080x24`) says otherwise. |
|
||||
| `"virtual"` | A headed browser on a private Xvfb display | Linux only, needs `Xvfb` and `setpriv` (util-linux). Handled by `Camoufox`, `AsyncCamoufox`, `NewBrowser` and `launch_server`. The display is 1x1 unless `CAMOUFOX_VIRTUAL_DISPLAY_SIZE` (e.g. `1920x1080x24`) says otherwise. Xvfb has no GPU: canvas and WebGL render in software. |
|
||||
| `"virtual-gpu"` | A headed browser on a private, headless Xorg server bound to one GPU | Linux only, needs Xorg and the card's driver. Canvas and WebGL render on the card. Handled like `"virtual"`. See [The GPU display](#the-gpu-display). |
|
||||
|
||||
#### The GPU display
|
||||
|
||||
```python
|
||||
from camoufox.sync_api import Camoufox
|
||||
|
||||
with Camoufox(headless="virtual-gpu", gpu="0000:01:00.0", os="linux") as browser:
|
||||
page = browser.new_page()
|
||||
```
|
||||
|
||||
- **The card is named, never guessed.** `gpu=` (or `CAMOUFOX_GPU`) takes a PCI
|
||||
address, which is the stable choice, or an index in PCI address order, a
|
||||
render or card node (`renderD128`, `/dev/dri/card1`), `vendor:device` ids
|
||||
(`10de:2c02`) or a vendor name (`nvidia`). With several GPUs and no selector
|
||||
the launch raises `AmbiguousGpuSelection`; `GpuNotFound` and `NoGpuFound`
|
||||
mean what they say. `python -m camoufox gpu-xorg-config` lists the GPUs when
|
||||
the choice is ambiguous.
|
||||
- **Without root, set the machine up once per card.** Xorg then runs through
|
||||
the setuid `Xorg.wrap`, which takes a config only from `/etc/X11` and needs
|
||||
`allowed_users=anybody` and `needs_root_rights=yes` in
|
||||
`/etc/X11/Xwrapper.config`. Stock Debian and Ubuntu ship
|
||||
`allowed_users=console`, which refuses a launch from ssh, systemd or a
|
||||
container. One command installs Xorg and the wrapper (with `apt-get`;
|
||||
elsewhere it says what to install), sets those two lines and installs the
|
||||
card's config. It is safe to run again:
|
||||
```bash
|
||||
python -m camoufox gpu-setup 0000:01:00.0 | sudo sh
|
||||
```
|
||||
A launch that fails for want of any of this ends its error with that
|
||||
command. The settings let any local user start a root X server.
|
||||
`gpu-xorg-config` prints just the card's config. As root, nothing is
|
||||
installed: a temporary config is used instead.
|
||||
- **It leaves your desktop alone.** The server drives no monitor, shares the
|
||||
current VT and exits when the browser closes. If the active VT changes while
|
||||
it starts, the launch switches back, stops the server and raises
|
||||
`GpuVtTakeover`.
|
||||
- **The launch checks the browser renders on the card.** A page draws WebGL
|
||||
and the launch waits up to 30 s (`CAMOUFOX_GPU_WAIT_S`) for the kernel to
|
||||
account GPU work to the browser. If it never does, the browser is closed and
|
||||
`GpuSoftwareFallback` is raised; `require_gpu=False` warns instead. There is
|
||||
no fallback to Xvfb.
|
||||
- **The identity claims that card.** On the host's OS the identity claims the
|
||||
GPU Firefox reports, read on this display, so it is the card's and not the
|
||||
desktop's (see [The canvas on another OS](#the-canvas-on-another-os)).
|
||||
- The browser gets `DISPLAY` and the X11 backend, and nothing that chooses a
|
||||
GL driver. These prefs default on, and `firefox_user_prefs` wins:
|
||||
`gfx.webrender.all`, `layers.acceleration.force-enabled`,
|
||||
`gfx.x11-egl.enabled`, `gfx.canvas.accelerated` and
|
||||
`gfx.canvas.accelerated.force-enabled`.
|
||||
|
||||
Every error here is a `VirtualDisplayError`.
|
||||
|
||||
### Remote server
|
||||
|
||||
@@ -213,7 +265,9 @@ passed to Playwright's `firefox.launch()` (or `launch_persistent_context()`).
|
||||
| Option | Type | Default | Effect |
|
||||
|---|---|---|---|
|
||||
| `os` | `str` or `list[str]` | fpgen's real-world mix | `"windows"`, `"macos"`, `"linux"`, or a list to pick from. |
|
||||
| `headless` | `bool` or `"virtual"` | `False` | See [Headless modes](#headless-modes). |
|
||||
| `headless` | `bool`, `"virtual"` or `"virtual-gpu"` | `False` | See [Headless modes](#headless-modes). |
|
||||
| `gpu` | `str` or `int` | `CAMOUFOX_GPU`, or the only GPU | The card for `headless="virtual-gpu"`. See [The GPU display](#the-gpu-display). Not taken by `launch_options`. |
|
||||
| `require_gpu` | `bool` | `True` | With `headless="virtual-gpu"`, fail the launch when the browser does not render on the card; `False` warns. Not taken by `launch_options`. |
|
||||
| `proxy` | `dict` | none | Playwright proxy: `server`, `username`, `password`, `bypass`. |
|
||||
| `geoip` | `bool` or `str` | none | `True` finds the public IP (through `proxy` if set); a string uses that IP. Sets timezone, locale, geolocation and the WebRTC IP. Without a proxy, also decides whether IPv6 stays on. Needs the `geoip` extra. |
|
||||
| `geoip_db` | `str` | the `camoufox set --geoip` choice | GeoIP database name, e.g. `"GeoIP AIO by daijro"`. |
|
||||
@@ -267,6 +321,8 @@ passed to Playwright's `firefox.launch()` (or `launch_persistent_context()`).
|
||||
| `path` | | Print the data directory (`~/.cache/camoufox` on Linux). |
|
||||
| `test [URL]` | `--executable-path PATH` | Open a headed browser with the Playwright inspector. |
|
||||
| `server` | | Start a Playwright server with default options and print its websocket endpoint. |
|
||||
| `gpu-setup [GPU]` | | Print the root script that sets this machine up for `headless="virtual-gpu"` on the card `GPU` names (or `CAMOUFOX_GPU`). Run it with `\| sudo sh`. |
|
||||
| `gpu-xorg-config [GPU]` | | Print the Xorg config `headless="virtual-gpu"` needs for the card `GPU` names (or `CAMOUFOX_GPU`). |
|
||||
|
||||
`set` and `fetch` read builds from these repositories (`repos.yml`):
|
||||
|
||||
|
||||
@@ -880,6 +880,36 @@ def server():
|
||||
launch_server()
|
||||
|
||||
|
||||
@cli.command(name="gpu-xorg-config")
|
||||
@click.argument("gpu", required=False)
|
||||
def gpu_xorg_config(gpu: Optional[str] = None) -> None:
|
||||
"""
|
||||
Print the Xorg config headless="virtual-gpu" needs for GPU
|
||||
"""
|
||||
from .exceptions import GpuDisplayError
|
||||
from .gpudisplay import select_gpu, xorg_config
|
||||
|
||||
try:
|
||||
click.echo(xorg_config(select_gpu(gpu)), nl=False)
|
||||
except GpuDisplayError as e:
|
||||
raise click.ClickException(str(e)) from e
|
||||
|
||||
|
||||
@cli.command(name="gpu-setup")
|
||||
@click.argument("gpu", required=False)
|
||||
def gpu_setup(gpu: Optional[str] = None) -> None:
|
||||
"""
|
||||
Print a root script that sets this machine up for headless="virtual-gpu" on GPU
|
||||
"""
|
||||
from .exceptions import GpuDisplayError
|
||||
from .gpudisplay import select_gpu, setup_script
|
||||
|
||||
try:
|
||||
click.echo(setup_script(select_gpu(gpu)), nl=False)
|
||||
except GpuDisplayError as e:
|
||||
raise click.ClickException(str(e)) from e
|
||||
|
||||
|
||||
class VersionInfo:
|
||||
def __init__(self):
|
||||
from rich.table import Table
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
from enum import Enum
|
||||
from multiprocessing import Lock
|
||||
from threading import Lock
|
||||
from typing import List, Optional
|
||||
|
||||
from .exceptions import InvalidAddonPath
|
||||
@@ -10,6 +11,11 @@ from .pkgman import INSTALL_DIR, unzip, webdl
|
||||
# Addons are stored in a shared folder, not per-browser version
|
||||
ADDONS_DIR = INSTALL_DIR / "addons"
|
||||
|
||||
# One lock for the module: `with Lock():` on a fresh multiprocessing.Lock per call
|
||||
# guarded nothing, so concurrent launches in one process raced the same
|
||||
# check-then-extract.
|
||||
_EXTRACT_LOCK = Lock()
|
||||
|
||||
|
||||
class DefaultAddons(Enum):
|
||||
"""
|
||||
@@ -44,17 +50,35 @@ def add_default_addons(
|
||||
|
||||
addons = [addon for addon in DefaultAddons if addon not in exclude_list]
|
||||
|
||||
with Lock():
|
||||
with _EXTRACT_LOCK:
|
||||
maybe_download_addons(addons, addons_list)
|
||||
|
||||
|
||||
def download_and_extract(url: str, extract_path: str, name: str) -> None:
|
||||
"""
|
||||
Downloads and extracts an addon from a given URL to a specified path
|
||||
Downloads and extracts an addon from a given URL to a specified path.
|
||||
|
||||
Extracts into a sibling temp dir and renames it into place last, so a
|
||||
concurrent launch (another process, which the lock cannot see) never finds
|
||||
a manifest.json in a directory whose other files are still being written.
|
||||
"""
|
||||
# Create a temporary file to store the downloaded zip
|
||||
buffer = webdl(url, desc=f"Downloading addon ({name})", bar=False)
|
||||
unzip(buffer, extract_path, f"Extracting addon ({name})", bar=False)
|
||||
parent = os.path.dirname(extract_path)
|
||||
tmp_dir = tempfile.mkdtemp(prefix=f".{name}.tmp-", dir=parent)
|
||||
try:
|
||||
unzip(buffer, tmp_dir, f"Extracting addon ({name})", bar=False)
|
||||
try:
|
||||
os.replace(tmp_dir, extract_path)
|
||||
except OSError:
|
||||
# The target is not empty (or exists at all, on Windows). A complete
|
||||
# copy came from another process and may be loading in its browser,
|
||||
# so keep it; only an incomplete one is replaced.
|
||||
if os.path.exists(os.path.join(extract_path, 'manifest.json')):
|
||||
return
|
||||
shutil.rmtree(extract_path)
|
||||
os.replace(tmp_dir, extract_path)
|
||||
finally:
|
||||
shutil.rmtree(tmp_dir, ignore_errors=True)
|
||||
|
||||
|
||||
def get_addon_path(addon_name: str) -> str:
|
||||
@@ -92,6 +116,8 @@ def maybe_download_addons(
|
||||
if addons_list is not None:
|
||||
addons_list.append(addon_path)
|
||||
except Exception as e:
|
||||
# Drop the partial directory so the next run re-downloads.
|
||||
shutil.rmtree(addon_path, ignore_errors=True)
|
||||
# Drop the partial directory so the next run re-downloads. A complete
|
||||
# one came from another process and may be loading in its browser.
|
||||
if not os.path.exists(os.path.join(addon_path, 'manifest.json')):
|
||||
shutil.rmtree(addon_path, ignore_errors=True)
|
||||
print(f"Failed to download and extract {addon.name}: {e}")
|
||||
|
||||
@@ -23,6 +23,7 @@ from .utils import (
|
||||
new_context_options,
|
||||
persistent_context_options,
|
||||
spoofs_window_dimensions,
|
||||
start_virtual_display,
|
||||
)
|
||||
|
||||
|
||||
@@ -82,7 +83,7 @@ async def AsyncNewBrowser(
|
||||
async def AsyncNewBrowser(
|
||||
playwright: Playwright,
|
||||
*,
|
||||
headless: Optional[Union[bool, Literal['virtual']]] = None,
|
||||
headless: Optional[Union[bool, Literal['virtual', 'virtual-gpu']]] = None,
|
||||
from_options: Optional[Dict[str, Any]] = None,
|
||||
persistent_context: bool = False,
|
||||
debug: Optional[bool] = None,
|
||||
@@ -92,6 +93,11 @@ async def AsyncNewBrowser(
|
||||
Launches a new browser instance for Camoufox given a set of launch options.
|
||||
|
||||
Parameters:
|
||||
headless (Optional[Union[bool, Literal['virtual', 'virtual-gpu']]]):
|
||||
True for Firefox headless. On Linux, 'virtual' runs a headed browser on
|
||||
an Xvfb display, and 'virtual-gpu' on a headless Xorg server bound to
|
||||
the GPU `gpu` (or CAMOUFOX_GPU) names; `require_gpu=False` warns instead
|
||||
of failing when the browser does not render on it.
|
||||
from_options (Dict[str, Any]):
|
||||
A set of launch options generated by `launch_options()` to use
|
||||
persistent_context (bool):
|
||||
@@ -99,13 +105,32 @@ async def AsyncNewBrowser(
|
||||
**kwargs:
|
||||
All other keyword arguments passed to `launch_options()`.
|
||||
"""
|
||||
if headless == 'virtual':
|
||||
virtual_display = VirtualDisplay(debug=debug)
|
||||
kwargs['virtual_display'] = virtual_display.get()
|
||||
headless = False
|
||||
else:
|
||||
virtual_display = None
|
||||
virtual_display, headless = await asyncio.get_running_loop().run_in_executor(
|
||||
None, start_virtual_display, headless, debug, kwargs
|
||||
)
|
||||
# Everything after the display exists can fail (bad proxy, missing binary,
|
||||
# unsatisfiable constraints) before sync/async_attach_vd wires kill() into
|
||||
# close, which used to strand one Xvfb and its /tmp/.X<n> lock per failed
|
||||
# launch.
|
||||
try:
|
||||
return await _new_browser(
|
||||
playwright, virtual_display, headless, from_options, persistent_context, debug, kwargs
|
||||
)
|
||||
except BaseException:
|
||||
if virtual_display:
|
||||
virtual_display.kill()
|
||||
raise
|
||||
|
||||
|
||||
async def _new_browser(
|
||||
playwright: Playwright,
|
||||
virtual_display: Optional[VirtualDisplay],
|
||||
headless: Any,
|
||||
from_options: Optional[Dict[str, Any]],
|
||||
persistent_context: bool,
|
||||
debug: Optional[bool],
|
||||
kwargs: Dict[str, Any],
|
||||
) -> Union[Browser, BrowserContext]:
|
||||
# Options the caller built leave its contexts to the caller (host_identity).
|
||||
canvas_noise: Optional[bool] = False
|
||||
if not from_options:
|
||||
|
||||
@@ -172,6 +172,70 @@ class VirtualDisplayNotSupported(VirtualDisplayError):
|
||||
...
|
||||
|
||||
|
||||
class GpuDisplayError(VirtualDisplayError):
|
||||
"""
|
||||
Raised when the headless='virtual-gpu' display cannot be used.
|
||||
"""
|
||||
|
||||
...
|
||||
|
||||
|
||||
class NoGpuFound(GpuDisplayError):
|
||||
"""
|
||||
Raised when no GPU render node with a PCI address is found.
|
||||
"""
|
||||
|
||||
...
|
||||
|
||||
|
||||
class AmbiguousGpuSelection(GpuDisplayError):
|
||||
"""
|
||||
Raised when several GPUs match, or several exist and none was named.
|
||||
"""
|
||||
|
||||
...
|
||||
|
||||
|
||||
class GpuNotFound(GpuDisplayError):
|
||||
"""
|
||||
Raised when the GPU selector matches no GPU.
|
||||
"""
|
||||
|
||||
...
|
||||
|
||||
|
||||
class CannotFindXorg(GpuDisplayError):
|
||||
"""
|
||||
Raised when Xorg cannot be found.
|
||||
"""
|
||||
|
||||
...
|
||||
|
||||
|
||||
class CannotExecuteXorg(GpuDisplayError):
|
||||
"""
|
||||
Raised when Xorg cannot be started, or exits right after starting.
|
||||
"""
|
||||
|
||||
...
|
||||
|
||||
|
||||
class GpuVtTakeover(GpuDisplayError):
|
||||
"""
|
||||
Raised when the Xorg server switched the active console VT.
|
||||
"""
|
||||
|
||||
...
|
||||
|
||||
|
||||
class GpuSoftwareFallback(GpuDisplayError):
|
||||
"""
|
||||
Raised when the browser on a GPU display never rendered on the GPU.
|
||||
"""
|
||||
|
||||
...
|
||||
|
||||
|
||||
class CamoufoxNotInstalled(FileNotFoundError):
|
||||
"""
|
||||
Raised when camoufox is not installed.
|
||||
|
||||
@@ -0,0 +1,632 @@
|
||||
"""headless='virtual-gpu': a headless Xorg server pinned to one GPU.
|
||||
|
||||
Xvfb has no GPU path, so a browser on it renders WebGL and canvas with Mesa's
|
||||
software rasterizer. This display is a real Xorg server bound to one card by
|
||||
its PCI address, drawing a virtual screen with no monitor attached, so the
|
||||
browser renders on that card. The card is always named, never guessed: the
|
||||
identity claims the GPU that renders, and two cards render differently.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import signal
|
||||
import socket
|
||||
import subprocess # nosec
|
||||
import tempfile
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from shutil import which
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
from ._warnings import LeakWarning
|
||||
from .exceptions import (
|
||||
AmbiguousGpuSelection,
|
||||
CannotExecuteXorg,
|
||||
CannotFindXorg,
|
||||
GpuNotFound,
|
||||
GpuSoftwareFallback,
|
||||
GpuVtTakeover,
|
||||
NoGpuFound,
|
||||
)
|
||||
from .virtdisplay import VirtualDisplay
|
||||
|
||||
# Names the card when the `gpu` launch option is not given.
|
||||
GPU_ENV_VAR = 'CAMOUFOX_GPU'
|
||||
|
||||
# How long the browser has to render on the GPU after the probe page starts
|
||||
# drawing. A card busy with other work took 5-10s in Holo's measurements.
|
||||
GPU_WAIT_S = 30.0
|
||||
GPU_WAIT_ENV_VAR = 'CAMOUFOX_GPU_WAIT_S'
|
||||
|
||||
# The root window is not observable (screen.* is spoofed per context), but the
|
||||
# GPU compositor needs a real surface, so it is not 1x1 like Xvfb's.
|
||||
GPU_SCREEN = '1920x1080x24'
|
||||
|
||||
# Where Xorg looks up a relative -config when the setuid wrapper elevates it.
|
||||
SYSTEM_CONFIG_DIR = Path('/etc/X11')
|
||||
|
||||
# Without root, Xorg.wrap must let any user start Xorg (a launch from ssh,
|
||||
# systemd or a container is not a console login) and keep it root, which a
|
||||
# server with no VT of its own needs to open the card. Stock Debian and Ubuntu
|
||||
# ship allowed_users=console and needs_root_rights=auto.
|
||||
XWRAPPER_CONFIG = Path('/etc/X11/Xwrapper.config')
|
||||
XWRAPPER_SETTINGS = {'allowed_users': 'anybody', 'needs_root_rights': 'yes'}
|
||||
XORG_WRAPPERS = (Path('/usr/lib/xorg/Xorg.wrap'), Path('/usr/libexec/Xorg.wrap'))
|
||||
|
||||
# -terminate: the server exits this many seconds after its last client leaves.
|
||||
# -sharevts: never take a VT of its own, which blanks a live desktop.
|
||||
SERVER_FLAGS = ('-terminate', '2', '-nolisten', 'tcp', '-sharevts', '-novtswitch')
|
||||
|
||||
# Firefox's GPU paths, on by default for this display only; the caller's prefs
|
||||
# win. webgl.force-enabled is set by every launch already. Forcing the
|
||||
# accelerated canvas made 2D canvas match Windows Firefox on the same card.
|
||||
GPU_PREFS: Dict[str, Any] = {
|
||||
'gfx.webrender.all': True,
|
||||
'layers.acceleration.force-enabled': True,
|
||||
'gfx.x11-egl.enabled': True,
|
||||
'gfx.canvas.accelerated': True,
|
||||
'gfx.canvas.accelerated.force-enabled': True,
|
||||
}
|
||||
|
||||
# Draws one WebGL frame per animation frame: Firefox opens the GPU only when
|
||||
# something renders.
|
||||
PROBE_PAGE = (
|
||||
"data:text/html,<canvas id=c width=32 height=32></canvas><script>"
|
||||
"const gl=c.getContext('webgl2')||c.getContext('webgl');"
|
||||
"function f(){if(gl){gl.clear(gl.COLOR_BUFFER_BIT);gl.flush();}requestAnimationFrame(f);}f();"
|
||||
"</script>"
|
||||
)
|
||||
|
||||
ACTIVE_VT = Path('/sys/class/tty/tty0/active')
|
||||
# A crash right after startup (a driver fault under VRAM pressure) and a VT
|
||||
# switch both land just after the display is reported.
|
||||
SETTLE_S = 1.0
|
||||
|
||||
# An X11 connection setup request: little-endian, protocol 11.0, no auth.
|
||||
X11_SETUP = bytes([0x6C, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0])
|
||||
|
||||
VENDORS = {'0x10de': 'NVIDIA', '0x1002': 'AMD', '0x8086': 'Intel'}
|
||||
|
||||
_PCI_ADDRESS = re.compile(r'^[0-9a-f]{4}:[0-9a-f]{2}:[0-9a-f]{2}\.[0-9a-f]$')
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GpuDevice:
|
||||
"""A GPU with a DRM render node, as sysfs describes it."""
|
||||
|
||||
pci_address: str
|
||||
render_node: str
|
||||
card_node: Optional[str]
|
||||
vendor_id: str
|
||||
device_id: str
|
||||
kernel_driver: str
|
||||
# /dev/nvidia<minor> for the NVIDIA driver, which renders through it.
|
||||
nvidia_node: Optional[str] = None
|
||||
|
||||
@property
|
||||
def drm_nodes(self) -> List[str]:
|
||||
"""The card's render node and its primary node. Firefox's GPU work is
|
||||
accounted to whichever it was handed: a second launch from one process
|
||||
on an RX 7900 XTX drew through /dev/dri/card0, not renderD128."""
|
||||
return [node for node in (self.render_node, self.card_node) if node]
|
||||
|
||||
@property
|
||||
def vendor(self) -> str:
|
||||
return VENDORS.get(self.vendor_id, self.vendor_id)
|
||||
|
||||
@property
|
||||
def bus_id(self) -> str:
|
||||
"""Xorg's BusID: decimal bus:device:function, without the domain."""
|
||||
_, bus, dev_fn = self.pci_address.split(':')
|
||||
dev, fn = dev_fn.split('.')
|
||||
return f'PCI:{int(bus, 16)}:{int(dev, 16)}:{int(fn, 16)}'
|
||||
|
||||
@property
|
||||
def xorg_driver(self) -> str:
|
||||
# modesetting drives any KMS device with glamor; NVIDIA needs its own.
|
||||
return 'nvidia' if self.kernel_driver == 'nvidia' else 'modesetting'
|
||||
|
||||
def __str__(self) -> str:
|
||||
return (
|
||||
f'{self.pci_address} {self.vendor} [{self.vendor_id}:{self.device_id}] '
|
||||
f'driver={self.kernel_driver} {self.render_node}'
|
||||
)
|
||||
|
||||
|
||||
def drew_on_gpu(fdinfo: str) -> bool:
|
||||
"""Whether a DRM file's usage stats (its fdinfo) show work on a GPU engine.
|
||||
|
||||
Drivers report busy time (drm-engine-<engine>: <n> ns; amdgpu, i915) or busy
|
||||
cycles (drm-cycles-<engine>: <n>; xe reports only these). drm-total-cycles
|
||||
counts the GPU's clock whether this client used it or not, so it is not work.
|
||||
"""
|
||||
busy = re.findall(r'^drm-(?:engine-[\w-]+:\s*(\d+) ns|cycles-[\w-]+:\s*(\d+))$', fdinfo, re.M)
|
||||
return any(int(ns or cycles) > 0 for ns, cycles in busy)
|
||||
|
||||
|
||||
def _read(path: Path) -> str:
|
||||
try:
|
||||
return path.read_text().strip()
|
||||
except OSError:
|
||||
return ''
|
||||
|
||||
|
||||
def _uevent(path: Path) -> Dict[str, str]:
|
||||
return dict(line.split('=', 1) for line in _read(path).splitlines() if '=' in line)
|
||||
|
||||
|
||||
def _nvidia_minor(root: Path, pci_address: str) -> Optional[str]:
|
||||
info = _read(root / 'proc/driver/nvidia/gpus' / pci_address / 'information')
|
||||
match = re.search(r'^Device Minor:\s*(\d+)', info, re.M)
|
||||
return f'/dev/nvidia{match.group(1)}' if match else None
|
||||
|
||||
|
||||
def enumerate_gpus(root: Union[str, Path] = '/') -> List[GpuDevice]:
|
||||
"""The GPUs with a render node and a PCI address, in PCI address order.
|
||||
|
||||
Ordered by PCI address, not render node number: render nodes are numbered
|
||||
in the order the drivers loaded, which can change from one boot to the next.
|
||||
"""
|
||||
root = Path(root)
|
||||
drm = root / 'sys/class/drm'
|
||||
cards = {}
|
||||
for card in drm.glob('card*'):
|
||||
if re.fullmatch(r'card\d+', card.name):
|
||||
try:
|
||||
cards[(card / 'device').resolve(strict=True)] = f'/dev/dri/{card.name}'
|
||||
except OSError:
|
||||
continue
|
||||
|
||||
gpus = []
|
||||
for node in (root / 'dev/dri').glob('renderD*'):
|
||||
device = drm / node.name / 'device'
|
||||
try:
|
||||
real = device.resolve(strict=True)
|
||||
except OSError:
|
||||
continue
|
||||
if not _PCI_ADDRESS.match(real.name):
|
||||
continue
|
||||
driver = _uevent(device / 'uevent').get('DRIVER', '')
|
||||
gpus.append(
|
||||
GpuDevice(
|
||||
pci_address=real.name,
|
||||
render_node=f'/dev/dri/{node.name}',
|
||||
card_node=cards.get(real),
|
||||
vendor_id=_read(device / 'vendor').lower(),
|
||||
device_id=_read(device / 'device').lower(),
|
||||
kernel_driver=driver,
|
||||
nvidia_node=_nvidia_minor(root, real.name) if driver == 'nvidia' else None,
|
||||
)
|
||||
)
|
||||
return sorted(gpus, key=lambda gpu: gpu.pci_address)
|
||||
|
||||
|
||||
def _matches(index: int, gpu: GpuDevice, selector: str) -> bool:
|
||||
if selector.isdigit():
|
||||
return int(selector) == index
|
||||
ids = f"{gpu.vendor_id}:{gpu.device_id}".replace('0x', '')
|
||||
names = {gpu.render_node, os.path.basename(gpu.render_node), gpu.vendor.lower(), ids}
|
||||
if gpu.card_node:
|
||||
names |= {gpu.card_node, os.path.basename(gpu.card_node)}
|
||||
return selector.replace('0x', '') in {name.lower() for name in names} or selector in gpu.pci_address
|
||||
|
||||
|
||||
def _listing(gpus: List[GpuDevice], shown: Optional[List[GpuDevice]] = None) -> str:
|
||||
return '\n'.join(f' [{i}] {gpu}' for i, gpu in enumerate(gpus) if shown is None or gpu in shown)
|
||||
|
||||
|
||||
def select_gpu(selector: Optional[Union[str, int]] = None, gpus: Optional[List[GpuDevice]] = None) -> GpuDevice:
|
||||
"""The one GPU `selector` (or CAMOUFOX_GPU) names.
|
||||
|
||||
A selector is an index into enumerate_gpus(), a render or card node (name or
|
||||
/dev/dri path), a PCI address or part of one, vendor:device ids, or a vendor
|
||||
name. Without one, the host must have exactly one GPU.
|
||||
"""
|
||||
if gpus is None:
|
||||
gpus = enumerate_gpus()
|
||||
if not gpus:
|
||||
raise NoGpuFound(
|
||||
'virtual-gpu: no GPU render node with a PCI address. Is a GPU present, is its driver '
|
||||
'loaded, and are /dev/dri and /sys visible (--device /dev/dri in a container)?'
|
||||
)
|
||||
if selector is None:
|
||||
selector = os.environ.get(GPU_ENV_VAR, '')
|
||||
selector = str(selector).strip().lower()
|
||||
if not selector:
|
||||
if len(gpus) == 1:
|
||||
return gpus[0]
|
||||
raise AmbiguousGpuSelection(
|
||||
f'virtual-gpu: {len(gpus)} GPUs found. Name one with gpu= or {GPU_ENV_VAR}, '
|
||||
f'preferably by PCI address:\n{_listing(gpus)}'
|
||||
)
|
||||
matched = [gpu for i, gpu in enumerate(gpus) if _matches(i, gpu, selector)]
|
||||
if not matched:
|
||||
raise GpuNotFound(f'virtual-gpu: no GPU matches {selector!r}. Available:\n{_listing(gpus)}')
|
||||
if len(matched) > 1:
|
||||
raise AmbiguousGpuSelection(
|
||||
f'virtual-gpu: {selector!r} matches {len(matched)} GPUs:\n{_listing(gpus, matched)}'
|
||||
)
|
||||
return matched[0]
|
||||
|
||||
|
||||
def xorg_config(gpu: GpuDevice, screen: str = GPU_SCREEN) -> str:
|
||||
"""An xorg.conf that drives `gpu` with a virtual screen and no outputs.
|
||||
|
||||
`UseDisplayDevice "None"` keeps the NVIDIA driver off every connector, so it
|
||||
needs no DRM master and coexists with a desktop on the same card.
|
||||
"""
|
||||
width, height, depth = screen.split('x')
|
||||
offscreen = '\tOption "UseDisplayDevice" "None"\n' if gpu.xorg_driver == 'nvidia' else ''
|
||||
return f'''# Generated by camoufox for headless="virtual-gpu".
|
||||
# GPU: {gpu.pci_address} {gpu.vendor} [{gpu.vendor_id}:{gpu.device_id}] driver={gpu.kernel_driver}
|
||||
Section "ServerFlags"
|
||||
\tOption "AutoAddDevices" "false"
|
||||
\tOption "AutoAddGPU" "false"
|
||||
\tOption "DontVTSwitch" "true"
|
||||
\tOption "AllowMouseOpenFail" "true"
|
||||
EndSection
|
||||
|
||||
Section "Device"
|
||||
\tIdentifier "camoufox-gpu"
|
||||
\tDriver "{gpu.xorg_driver}"
|
||||
\tBusID "{gpu.bus_id}"
|
||||
{offscreen}\tOption "AllowEmptyInitialConfiguration" "true"
|
||||
EndSection
|
||||
|
||||
Section "Monitor"
|
||||
\tIdentifier "camoufox-monitor"
|
||||
\tHorizSync 5.0 - 1000.0
|
||||
\tVertRefresh 5.0 - 200.0
|
||||
EndSection
|
||||
|
||||
Section "Screen"
|
||||
\tIdentifier "camoufox-screen"
|
||||
\tDevice "camoufox-gpu"
|
||||
\tMonitor "camoufox-monitor"
|
||||
\tDefaultDepth {depth}
|
||||
\tSubSection "Display"
|
||||
\t\tDepth {depth}
|
||||
\t\tVirtual {width} {height}
|
||||
\tEndSubSection
|
||||
EndSection
|
||||
|
||||
Section "ServerLayout"
|
||||
\tIdentifier "camoufox-layout"
|
||||
\tScreen "camoufox-screen"
|
||||
EndSection
|
||||
'''
|
||||
|
||||
|
||||
def config_name(gpu: GpuDevice) -> str:
|
||||
"""The config's file name in /etc/X11, keyed by PCI address."""
|
||||
return f"camoufox-gpu-{re.sub(r'[:.]', '_', gpu.pci_address)}.conf"
|
||||
|
||||
|
||||
def setup_command(gpu: GpuDevice) -> str:
|
||||
"""The one command that sets this machine up for `gpu`."""
|
||||
sudo = '' if os.geteuid() == 0 else 'sudo '
|
||||
return f'python -m camoufox gpu-setup {gpu.pci_address} | {sudo}sh'
|
||||
|
||||
|
||||
def setup_script(gpu: GpuDevice) -> str:
|
||||
"""A root shell script that sets this machine up for headless="virtual-gpu"
|
||||
on `gpu`: Xorg and its setuid wrapper, the wrapper's settings, and the
|
||||
card's config. Safe to run again."""
|
||||
config = SYSTEM_CONFIG_DIR / config_name(gpu)
|
||||
xwrapper = XWRAPPER_CONFIG
|
||||
settings = ''.join(f'{key}={value}\\n' for key, value in XWRAPPER_SETTINGS.items())
|
||||
deletes = ' '.join(
|
||||
f"-e '/^[[:space:]]*{key}[[:space:]]*=/d'" for key in XWRAPPER_SETTINGS
|
||||
)
|
||||
nvidia = (
|
||||
'\n'
|
||||
'if ! find /usr/lib /usr/lib64 -name nvidia_drv.so 2>/dev/null | grep -q .; then\n'
|
||||
"\techo 'camoufox gpu-setup: no nvidia_drv.so found. Install the NVIDIA driver'\\''s Xorg "
|
||||
"module, matching the kernel driver (on Ubuntu, xserver-xorg-video-nvidia-<version>).' >&2\n"
|
||||
'fi\n'
|
||||
if gpu.xorg_driver == 'nvidia'
|
||||
else ''
|
||||
)
|
||||
return f'''#!/bin/sh
|
||||
# Generated by camoufox gpu-setup for headless="virtual-gpu".
|
||||
# GPU: {gpu}
|
||||
# Run once as root: {setup_command(gpu)}
|
||||
set -e
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
\techo 'camoufox gpu-setup: run this as root (pipe it to sudo sh).' >&2
|
||||
\texit 1
|
||||
fi
|
||||
|
||||
# 1. The Xorg server and its setuid wrapper, Xorg.wrap.
|
||||
if command -v apt-get >/dev/null 2>&1; then
|
||||
\tdpkg -s xserver-xorg-core xserver-xorg-legacy >/dev/null 2>&1 \\
|
||||
\t\t|| DEBIAN_FRONTEND=noninteractive apt-get install -y xserver-xorg-core xserver-xorg-legacy
|
||||
elif ! command -v Xorg >/dev/null 2>&1; then
|
||||
\techo 'camoufox gpu-setup: install the Xorg server and its setuid wrapper (Xorg.wrap), then run this again.' >&2
|
||||
\texit 1
|
||||
fi
|
||||
{nvidia}
|
||||
# 2. Let Xorg.wrap start Xorg as root for any user. A launch from ssh, systemd
|
||||
# or a container is not a console login. Any local user can then start a
|
||||
# root X server.
|
||||
touch {xwrapper}
|
||||
sed -i {deletes} {xwrapper}
|
||||
printf '{settings}' >> {xwrapper}
|
||||
|
||||
# 3. The card's config, bound by PCI address.
|
||||
cat > {config} <<'CAMOUFOX_XORG_CONF'
|
||||
{xorg_config(gpu)}CAMOUFOX_XORG_CONF
|
||||
|
||||
echo 'headless="virtual-gpu" is set up for {gpu.pci_address}.'
|
||||
'''
|
||||
|
||||
|
||||
def setup_problems() -> List[str]:
|
||||
"""What keeps Xorg.wrap from starting Xorg for this user, if anything."""
|
||||
problems = []
|
||||
if not any(path.exists() for path in XORG_WRAPPERS):
|
||||
problems.append(f"no setuid Xorg.wrap ({' or '.join(map(str, XORG_WRAPPERS))})")
|
||||
settings = {}
|
||||
for line in _read(XWRAPPER_CONFIG).splitlines():
|
||||
key, _, value = line.partition('=')
|
||||
settings[key.strip()] = value.strip()
|
||||
for key, wanted in XWRAPPER_SETTINGS.items():
|
||||
have = settings.get(key) or 'unset'
|
||||
if have != wanted:
|
||||
problems.append(f'{XWRAPPER_CONFIG} has {key}={have}, needs {key}={wanted}')
|
||||
return problems
|
||||
|
||||
|
||||
def _session_on_vt(vt: str) -> Optional[str]:
|
||||
"""The logind session on VT `vt`, which its user may activate without root."""
|
||||
try:
|
||||
sessions = subprocess.run( # nosec
|
||||
['loginctl', 'list-sessions', '--no-legend'], capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
except (OSError, subprocess.CalledProcessError):
|
||||
return None
|
||||
for line in sessions.splitlines():
|
||||
session = line.split()[0]
|
||||
number = subprocess.run( # nosec
|
||||
['loginctl', 'show-session', session, '-p', 'VTNr', '--value'],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout.strip()
|
||||
if number == vt:
|
||||
return session
|
||||
return None
|
||||
|
||||
|
||||
class GpuDisplay(VirtualDisplay):
|
||||
"""A headless Xorg server on one GPU, for headless='virtual-gpu'.
|
||||
|
||||
As root, Xorg takes a temporary config and log. Otherwise it runs through
|
||||
the setuid Xorg.wrap, which accepts only a config name relative to
|
||||
/etc/X11 and no -logfile, so the config is installed there once.
|
||||
"""
|
||||
|
||||
_name = 'Xorg'
|
||||
|
||||
def _error(self, message: str) -> CannotExecuteXorg: # type: ignore[override]
|
||||
"""A failed start, with what the machine is missing when that is why."""
|
||||
if os.geteuid() != 0:
|
||||
problems = setup_problems()
|
||||
if problems:
|
||||
message += (
|
||||
'\nheadless="virtual-gpu" without root needs Xorg.wrap to start Xorg as root:\n'
|
||||
+ ''.join(f' - {problem}\n' for problem in problems)
|
||||
+ f'Set this machine up with:\n {setup_command(self.gpu)}'
|
||||
)
|
||||
return CannotExecuteXorg(message)
|
||||
|
||||
def __init__(self, gpu: GpuDevice, debug: bool = False, require_gpu: bool = True) -> None:
|
||||
super().__init__(debug=debug, screen=GPU_SCREEN)
|
||||
self.gpu = gpu
|
||||
self.require_gpu = require_gpu
|
||||
self._client: Optional[socket.socket] = None
|
||||
self._temp_files: List[str] = []
|
||||
self._vt: Optional[str] = None
|
||||
|
||||
@property
|
||||
def xorg_path(self) -> str:
|
||||
path = which('Xorg')
|
||||
if not path:
|
||||
raise CannotFindXorg(
|
||||
'headless="virtual-gpu" needs the Xorg server. Set this machine up with:\n'
|
||||
f' {setup_command(self.gpu)}'
|
||||
)
|
||||
return path
|
||||
|
||||
def _command(self) -> List[str]:
|
||||
if os.geteuid() == 0:
|
||||
fd, config = tempfile.mkstemp(prefix='camoufox-xorg-', suffix='.conf')
|
||||
with os.fdopen(fd, 'w') as f:
|
||||
f.write(xorg_config(self.gpu, self.screen))
|
||||
log = config[: -len('.conf')] + '.log'
|
||||
self._temp_files = [config, log]
|
||||
files = ['-config', config, '-logfile', log]
|
||||
else:
|
||||
name = config_name(self.gpu)
|
||||
if not (SYSTEM_CONFIG_DIR / name).is_file():
|
||||
raise CannotExecuteXorg(
|
||||
f'headless="virtual-gpu" without root needs {SYSTEM_CONFIG_DIR / name}. '
|
||||
f'Set this machine up with:\n {setup_command(self.gpu)}'
|
||||
)
|
||||
files = ['-config', name]
|
||||
return [self.xorg_path, *files, *SERVER_FLAGS]
|
||||
|
||||
def _wrapper(self) -> List[str]:
|
||||
# No parent-death signal: the setuid Xorg.wrap drops it on exec. The
|
||||
# held client and -terminate end the server instead (_ready).
|
||||
return []
|
||||
|
||||
def _server_env(self) -> Optional[Dict[str, str]]:
|
||||
return None
|
||||
|
||||
def _log(self) -> str:
|
||||
return self._temp_files[1] if self._temp_files else f'/var/log/Xorg.{self._display}.log'
|
||||
|
||||
def get(self) -> str:
|
||||
self._vt = _read(ACTIVE_VT) or None
|
||||
return super().get()
|
||||
|
||||
def _starting(self) -> None:
|
||||
if self._vt and _read(ACTIVE_VT) != self._vt:
|
||||
self._vt_taken()
|
||||
|
||||
def _ready(self) -> None:
|
||||
deadline = time.monotonic() + SETTLE_S
|
||||
while time.monotonic() < deadline:
|
||||
self._starting()
|
||||
if self.proc is not None and self.proc.poll() is not None:
|
||||
error = self._error(
|
||||
f'Xorg exited ({self.proc.returncode}) right after starting. See {self._log()}.'
|
||||
)
|
||||
self.kill()
|
||||
raise error
|
||||
time.sleep(0.1)
|
||||
# The server exits at its last client's disconnect (-terminate), and a
|
||||
# server no client ever reached never does. Holding one client from here
|
||||
# on means it always has a last client: this one, closed by kill() or by
|
||||
# the kernel when this process dies. An unprivileged launcher may not be
|
||||
# able to signal a root Xorg, so this is the teardown that always works.
|
||||
self._client = socket.socket(socket.AF_UNIX)
|
||||
try:
|
||||
self._client.connect(f'/tmp/.X11-unix/X{self._display}')
|
||||
self._client.sendall(X11_SETUP)
|
||||
except OSError as e:
|
||||
self.kill()
|
||||
raise self._error(f'Could not connect to the Xorg display it reported: {e}') from e
|
||||
|
||||
def _vt_taken(self) -> None:
|
||||
vt = self._vt or ''
|
||||
number = vt.removeprefix('tty')
|
||||
session = _session_on_vt(number)
|
||||
if session:
|
||||
subprocess.run(['loginctl', 'activate', session], capture_output=True) # nosec
|
||||
if _read(ACTIVE_VT) != vt:
|
||||
subprocess.run(['chvt', number], capture_output=True) # nosec
|
||||
restored = _read(ACTIVE_VT) == vt
|
||||
self.kill()
|
||||
raise GpuVtTakeover(
|
||||
f'Xorg on {self.gpu} switched the active VT away from {vt}. '
|
||||
+ ('It was switched back.' if restored else f'Switch back with `sudo chvt {number}`.')
|
||||
)
|
||||
|
||||
def _display_processes(self) -> List[int]:
|
||||
"""Every process started with DISPLAY set to this display, except Xorg and us."""
|
||||
needle = f'\0DISPLAY=:{self._display}\0'.encode()
|
||||
skip = {os.getpid(), self.proc.pid if self.proc else None}
|
||||
pids = []
|
||||
for entry in Path('/proc').iterdir():
|
||||
if not entry.name.isdigit() or int(entry.name) in skip:
|
||||
continue
|
||||
try:
|
||||
if needle in b'\0' + (entry / 'environ').read_bytes():
|
||||
pids.append(int(entry.name))
|
||||
except OSError:
|
||||
continue
|
||||
return pids
|
||||
|
||||
def uses_gpu(self) -> bool:
|
||||
"""Whether a process on this display renders on the GPU.
|
||||
|
||||
Firefox opens the card's DRM nodes for DMABuf even when it renders in
|
||||
software, so an open render node proves nothing. Rendering is the NVIDIA
|
||||
driver's /dev/nvidia<minor> held open, or work the kernel accounted to a
|
||||
GPU engine on one of the card's DRM nodes.
|
||||
"""
|
||||
nodes = self.gpu.drm_nodes
|
||||
for pid in self._display_processes():
|
||||
try:
|
||||
fds = list(Path(f'/proc/{pid}/fd').iterdir())
|
||||
except OSError:
|
||||
continue
|
||||
for fd in fds:
|
||||
try:
|
||||
target = os.readlink(fd)
|
||||
if target == self.gpu.nvidia_node or (
|
||||
target in nodes
|
||||
and drew_on_gpu(Path(f'/proc/{pid}/fdinfo/{fd.name}').read_text())
|
||||
):
|
||||
return True
|
||||
except OSError:
|
||||
continue
|
||||
return False
|
||||
|
||||
def wait_for_gpu(self) -> bool:
|
||||
deadline = time.monotonic() + float(os.environ.get(GPU_WAIT_ENV_VAR) or GPU_WAIT_S)
|
||||
while time.monotonic() < deadline:
|
||||
if self.uses_gpu():
|
||||
return True
|
||||
time.sleep(0.25)
|
||||
return False
|
||||
|
||||
def check_gpu(self, active: bool) -> None:
|
||||
"""Raise, or warn with require_gpu=False, when the browser did not render on the GPU."""
|
||||
if active:
|
||||
return
|
||||
if not self.require_gpu:
|
||||
LeakWarning.warn('gpu_software_rendering')
|
||||
return
|
||||
raise GpuSoftwareFallback(
|
||||
f'The browser on :{self._display} never rendered on {self.gpu}, so it renders in software. '
|
||||
f'Firefox may have blocklisted the driver or failed to create a GPU context. '
|
||||
f'See {self._log()}.'
|
||||
)
|
||||
|
||||
def kill(self) -> None:
|
||||
if self._client:
|
||||
self._client.close()
|
||||
self._client = None
|
||||
if self._display is not None:
|
||||
# The browser's helpers (dbus, at-spi) are not in its process group.
|
||||
for sig in (signal.SIGTERM, signal.SIGKILL):
|
||||
for pid in self._display_processes():
|
||||
try:
|
||||
os.kill(pid, sig)
|
||||
except OSError:
|
||||
pass
|
||||
deadline = time.monotonic() + 1.5
|
||||
while self._display_processes() and time.monotonic() < deadline:
|
||||
time.sleep(0.05)
|
||||
# SIGTERM lets Xorg remove its root-owned socket, which this process
|
||||
# cannot; base kill() SIGKILLs only a server still running after that.
|
||||
if self.proc is not None and self.proc.poll() is None:
|
||||
try:
|
||||
self.proc.terminate()
|
||||
self.proc.wait(timeout=10)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
pass
|
||||
super().kill()
|
||||
for path in self._temp_files:
|
||||
try:
|
||||
os.remove(path)
|
||||
except OSError:
|
||||
pass
|
||||
self._temp_files = []
|
||||
self._display = None
|
||||
|
||||
|
||||
def prove_gpu(target: Any, display: GpuDisplay) -> None:
|
||||
"""Draw in a page of `target` and check the browser rendered on the GPU."""
|
||||
page = target.new_page()
|
||||
try:
|
||||
page.goto(PROBE_PAGE)
|
||||
active = display.wait_for_gpu()
|
||||
finally:
|
||||
page.close()
|
||||
display.check_gpu(active)
|
||||
|
||||
|
||||
async def async_prove_gpu(target: Any, display: GpuDisplay) -> None:
|
||||
import asyncio
|
||||
|
||||
page = await target.new_page()
|
||||
try:
|
||||
await page.goto(PROBE_PAGE)
|
||||
active = await asyncio.get_running_loop().run_in_executor(None, display.wait_for_gpu)
|
||||
finally:
|
||||
await page.close()
|
||||
display.check_gpu(active)
|
||||
@@ -16,6 +16,7 @@ from functools import lru_cache
|
||||
from typing import Any, Dict, Mapping, Optional, Tuple
|
||||
|
||||
from .fingerprints import is_software_renderer
|
||||
from .gpudisplay import GPU_PREFS
|
||||
|
||||
BASELINE_OVERRIDES_PREF = 'privacy.baselineFingerprintingProtection.overrides'
|
||||
|
||||
@@ -76,22 +77,31 @@ def set_webgpu(prefs: Dict[str, Any], target_os: str, gpu: Gpu, host: Optional[G
|
||||
prefs.setdefault(WEBGPU_BLOCKLIST_PREF, FEATURE_STATUS_OK if own_adapter else FEATURE_BLOCKED_DEVICE)
|
||||
|
||||
|
||||
def host_gpu(executable_path: str, headless: bool, env: Mapping[str, Any]) -> Optional[Gpu]:
|
||||
def host_gpu(
|
||||
executable_path: str, headless: bool, env: Mapping[str, Any], prefs: Mapping[str, Any]
|
||||
) -> Optional[Gpu]:
|
||||
"""The (vendor, renderer) this machine's Firefox reports, or None without WebGL.
|
||||
|
||||
Read from the binary about to launch, with no identity applied, under the
|
||||
same display: headless, Xvfb and a real display can each render on a
|
||||
different device. The strings are the ones Firefox sanitizes itself.
|
||||
same display and the same rendering prefs: headless, Xvfb, a GPU display
|
||||
and a real display can each render on a different device. The strings are
|
||||
the ones Firefox sanitizes itself.
|
||||
"""
|
||||
return _host_gpu(
|
||||
str(executable_path),
|
||||
bool(headless),
|
||||
tuple(sorted((k, str(v)) for k, v in env.items() if not k.startswith(_LAUNCHER_ENV))),
|
||||
tuple(sorted((k, prefs[k]) for k in GPU_PREFS if k in prefs)),
|
||||
)
|
||||
|
||||
|
||||
@lru_cache(maxsize=None)
|
||||
def _host_gpu(executable_path: str, headless: bool, env: Tuple[Tuple[str, str], ...]) -> Optional[Gpu]:
|
||||
def _host_gpu(
|
||||
executable_path: str,
|
||||
headless: bool,
|
||||
env: Tuple[Tuple[str, str], ...],
|
||||
prefs: Tuple[Tuple[str, Any], ...],
|
||||
) -> Optional[Gpu]:
|
||||
def read() -> Optional[Gpu]:
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
@@ -101,7 +111,7 @@ def _host_gpu(executable_path: str, headless: bool, env: Tuple[Tuple[str, str],
|
||||
headless=headless,
|
||||
env=dict(env),
|
||||
# The launch forces WebGL on, which can choose another device.
|
||||
firefox_user_prefs={'webgl.force-enabled': True},
|
||||
firefox_user_prefs={**dict(prefs), 'webgl.force-enabled': True},
|
||||
)
|
||||
try:
|
||||
gpu = browser.new_page().evaluate(_READ_GPU)
|
||||
|
||||
@@ -85,6 +85,13 @@ def validate_ip(ip: str) -> None:
|
||||
raise InvalidIP(f"Invalid IP address: {ip}")
|
||||
|
||||
|
||||
# Probing the exit IP through a proxy adds a hop and a TLS handshake behind it; 5 s
|
||||
# is not enough for a proxy that routes the long way round, and every URL timing out
|
||||
# fails the whole launch. Direct probes keep 5 s.
|
||||
DIRECT_IP_TIMEOUT = 5
|
||||
PROXIED_IP_TIMEOUT = 15
|
||||
|
||||
|
||||
def proxy_exit_geo(proxy: str) -> Tuple[str, str]:
|
||||
"""
|
||||
The exit IP of `proxy` and that IP's timezone, looked up through the proxy.
|
||||
@@ -95,7 +102,7 @@ def proxy_exit_geo(proxy: str) -> Tuple[str, str]:
|
||||
resp = requests.get(
|
||||
"http://ip-api.com/json?fields=status,message,query,timezone",
|
||||
proxies=Proxy.as_requests_proxy(proxy),
|
||||
timeout=10,
|
||||
timeout=PROXIED_IP_TIMEOUT,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
@@ -121,7 +128,7 @@ def public_ipv6() -> Optional[str]:
|
||||
"""
|
||||
for url in ("https://api6.ipify.org", "https://v6.ident.me"):
|
||||
try:
|
||||
resp = requests.get(url, timeout=5, verify=True)
|
||||
resp = requests.get(url, timeout=DIRECT_IP_TIMEOUT, verify=True)
|
||||
resp.raise_for_status()
|
||||
except requests.RequestException:
|
||||
continue
|
||||
@@ -153,7 +160,7 @@ def public_ip(proxy: Optional[str] = None) -> str:
|
||||
resp = requests.get(
|
||||
url,
|
||||
proxies=Proxy.as_requests_proxy(proxy) if proxy else None,
|
||||
timeout=5,
|
||||
timeout=PROXIED_IP_TIMEOUT if proxy else DIRECT_IP_TIMEOUT,
|
||||
verify=True,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
|
||||
@@ -1,13 +1,19 @@
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, NoReturn, Tuple, Union
|
||||
from typing import Any, Dict, NoReturn, Optional, Tuple, Union
|
||||
|
||||
import base64
|
||||
import orjson
|
||||
from playwright._impl._driver import compute_driver_executable
|
||||
|
||||
from camoufox.pkgman import LOCAL_DATA
|
||||
from camoufox.utils import launch_options
|
||||
from camoufox.gpudisplay import GpuDisplay, prove_gpu
|
||||
from camoufox.utils import launch_options, start_virtual_display
|
||||
from camoufox.virtdisplay import VirtualDisplay
|
||||
|
||||
LAUNCH_SCRIPT: Path = LOCAL_DATA / "launchServer.js"
|
||||
@@ -61,21 +67,46 @@ def launch_server(**kwargs) -> NoReturn:
|
||||
)
|
||||
kwargs.pop(unsupported, None)
|
||||
|
||||
virtual_display = None
|
||||
if kwargs.get('headless') == 'virtual':
|
||||
virtual_display = VirtualDisplay(debug=kwargs.get('debug'))
|
||||
kwargs['virtual_display'] = virtual_display.get()
|
||||
kwargs['headless'] = False
|
||||
virtual_display, kwargs['headless'] = start_virtual_display(
|
||||
kwargs.get('headless'), kwargs.get('debug'), kwargs
|
||||
)
|
||||
try:
|
||||
_serve(launch_options(**kwargs))
|
||||
_serve(launch_options(**kwargs), virtual_display)
|
||||
finally:
|
||||
if virtual_display:
|
||||
virtual_display.kill()
|
||||
|
||||
|
||||
def _serve(config: Dict[str, Any]) -> NoReturn:
|
||||
def _prove_served_gpu(process: subprocess.Popen, display: GpuDisplay) -> None:
|
||||
"""Pass the server's output through, and once it prints its endpoint, check
|
||||
that its browser renders on the GPU."""
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
def prove(endpoint: str) -> None:
|
||||
with sync_playwright() as playwright:
|
||||
browser = playwright.firefox.connect(endpoint)
|
||||
try:
|
||||
prove_gpu(browser, display)
|
||||
finally:
|
||||
browser.close()
|
||||
|
||||
assert process.stdout is not None
|
||||
for line in process.stdout:
|
||||
sys.stdout.write(line)
|
||||
sys.stdout.flush()
|
||||
endpoint = re.search(r'ws://\S+', line)
|
||||
if endpoint:
|
||||
# The sync API refuses to start inside a running asyncio loop.
|
||||
with ThreadPoolExecutor(max_workers=1) as pool:
|
||||
pool.submit(prove, endpoint.group(0)).result()
|
||||
break
|
||||
threading.Thread(target=shutil.copyfileobj, args=(process.stdout, sys.stdout), daemon=True).start()
|
||||
|
||||
|
||||
def _serve(config: Dict[str, Any], virtual_display: Optional[VirtualDisplay] = None) -> NoReturn:
|
||||
"""Run launchServer.js with `config` until the Node process exits."""
|
||||
nodejs = get_nodejs()
|
||||
gpu_display = virtual_display if isinstance(virtual_display, GpuDisplay) else None
|
||||
|
||||
data = orjson.dumps(to_camel_case_dict(config))
|
||||
|
||||
@@ -90,6 +121,7 @@ def _serve(config: Dict[str, Any]) -> NoReturn:
|
||||
],
|
||||
cwd=driver_package,
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE if gpu_display else None,
|
||||
text=True,
|
||||
)
|
||||
# Send one newline-delimited configuration frame, then keep the pipe open.
|
||||
@@ -126,6 +158,8 @@ def _serve(config: Dict[str, Any]) -> NoReturn:
|
||||
try:
|
||||
stdin.write(encoded_config)
|
||||
stdin.flush()
|
||||
if gpu_display:
|
||||
_prove_served_gpu(process, gpu_display)
|
||||
process.wait()
|
||||
except OSError as error:
|
||||
# If Node failed before reading the frame, preserve its exit status
|
||||
|
||||
@@ -20,6 +20,7 @@ from .utils import (
|
||||
new_context_options,
|
||||
persistent_context_options,
|
||||
spoofs_window_dimensions,
|
||||
start_virtual_display,
|
||||
sync_attach_vd,
|
||||
)
|
||||
|
||||
@@ -84,7 +85,7 @@ def NewBrowser(
|
||||
def NewBrowser(
|
||||
playwright: Playwright,
|
||||
*,
|
||||
headless: Optional[Union[bool, Literal['virtual']]] = None,
|
||||
headless: Optional[Union[bool, Literal['virtual', 'virtual-gpu']]] = None,
|
||||
from_options: Optional[Dict[str, Any]] = None,
|
||||
persistent_context: bool = False,
|
||||
debug: Optional[bool] = None,
|
||||
@@ -94,6 +95,11 @@ def NewBrowser(
|
||||
Launches a new browser instance for Camoufox given a set of launch options.
|
||||
|
||||
Parameters:
|
||||
headless (Optional[Union[bool, Literal['virtual', 'virtual-gpu']]]):
|
||||
True for Firefox headless. On Linux, 'virtual' runs a headed browser on
|
||||
an Xvfb display, and 'virtual-gpu' on a headless Xorg server bound to
|
||||
the GPU `gpu` (or CAMOUFOX_GPU) names; `require_gpu=False` warns instead
|
||||
of failing when the browser does not render on it.
|
||||
from_options (Dict[str, Any]):
|
||||
A set of launch options generated by `launch_options()` to use
|
||||
persistent_context (bool):
|
||||
@@ -101,13 +107,28 @@ def NewBrowser(
|
||||
**kwargs:
|
||||
All other keyword arguments passed to `launch_options()`.
|
||||
"""
|
||||
if headless == 'virtual':
|
||||
virtual_display = VirtualDisplay(debug=debug)
|
||||
kwargs['virtual_display'] = virtual_display.get()
|
||||
headless = False
|
||||
else:
|
||||
virtual_display = None
|
||||
virtual_display, headless = start_virtual_display(headless, debug, kwargs)
|
||||
# Everything after the display exists can fail (bad proxy, missing binary,
|
||||
# unsatisfiable constraints) before sync/async_attach_vd wires kill() into
|
||||
# close, which used to strand one Xvfb and its /tmp/.X<n> lock per failed
|
||||
# launch.
|
||||
try:
|
||||
return _launch(playwright, virtual_display, headless, from_options, persistent_context, debug, kwargs)
|
||||
except BaseException:
|
||||
if virtual_display:
|
||||
virtual_display.kill()
|
||||
raise
|
||||
|
||||
|
||||
def _launch(
|
||||
playwright: Playwright,
|
||||
virtual_display: Optional[VirtualDisplay],
|
||||
headless: Any,
|
||||
from_options: Optional[Dict[str, Any]],
|
||||
persistent_context: bool,
|
||||
debug: Optional[bool],
|
||||
kwargs: Dict[str, Any],
|
||||
) -> Union[Browser, BrowserContext]:
|
||||
# Options the caller built leave its contexts to the caller (host_identity).
|
||||
canvas_noise: Optional[bool] = False
|
||||
if not from_options:
|
||||
|
||||
@@ -43,6 +43,7 @@ from .pkgman import (
|
||||
selected_launch_path,
|
||||
warn_if_package_outdated,
|
||||
)
|
||||
from .gpudisplay import GPU_PREFS, GpuDisplay, async_prove_gpu, prove_gpu, select_gpu
|
||||
from .virtdisplay import VirtualDisplay
|
||||
from ._warnings import FallbackWarning, LeakWarning
|
||||
from .host_rendering import Gpu, add_canvas_placeholder, has_canvas_placeholder, host_gpu, renders_on_hardware, set_webgpu
|
||||
@@ -796,7 +797,7 @@ def host_identity(from_options: Dict[str, Any], canvas_noise: Optional[bool]) ->
|
||||
host_os = _host_os_key()
|
||||
covered, gpu = plan_rendering(
|
||||
host_os, None, False, bool(prefs.get('webgl.disabled')),
|
||||
lambda: host_gpu(from_options['executable_path'], from_options['headless'], from_options['env']),
|
||||
lambda: host_gpu(from_options['executable_path'], from_options['headless'], from_options['env'], prefs),
|
||||
)
|
||||
return None if covered else (host_os, gpu)
|
||||
|
||||
@@ -846,11 +847,37 @@ def new_context_options(
|
||||
return opts
|
||||
|
||||
|
||||
def start_virtual_display(
|
||||
headless: Any, debug: Optional[bool], kwargs: Dict[str, Any]
|
||||
) -> Tuple[Optional[VirtualDisplay], Any]:
|
||||
"""Start the display headless='virtual' or 'virtual-gpu' asks for.
|
||||
|
||||
Pops the display's own options from `kwargs` and points the launch at the
|
||||
display. Returns the display, or None, and the headless value to launch with.
|
||||
"""
|
||||
gpu = kwargs.pop('gpu', None)
|
||||
require_gpu = kwargs.pop('require_gpu', None)
|
||||
if headless == 'virtual-gpu':
|
||||
display: VirtualDisplay = GpuDisplay(
|
||||
select_gpu(gpu), debug=bool(debug), require_gpu=require_gpu is not False
|
||||
)
|
||||
kwargs['firefox_user_prefs'] = {**GPU_PREFS, **(kwargs.get('firefox_user_prefs') or {})}
|
||||
elif gpu is not None or require_gpu is not None:
|
||||
raise ValueError("gpu and require_gpu apply to headless='virtual-gpu' only")
|
||||
elif headless == 'virtual':
|
||||
display = VirtualDisplay(debug=debug)
|
||||
else:
|
||||
return None, headless
|
||||
kwargs['virtual_display'] = display.get()
|
||||
return display, False
|
||||
|
||||
|
||||
async def async_attach_vd(
|
||||
browser: Any, virtual_display: Optional[VirtualDisplay] = None
|
||||
) -> Any: # type: ignore
|
||||
"""
|
||||
Attaches the virtual display to the async browser cleanup
|
||||
Attaches the virtual display to the async browser cleanup, and on a GPU
|
||||
display checks the browser renders on the GPU.
|
||||
"""
|
||||
if not virtual_display: # Skip if no virtual display is provided
|
||||
return browser
|
||||
@@ -865,6 +892,13 @@ async def async_attach_vd(
|
||||
|
||||
browser.close = new_close
|
||||
|
||||
if isinstance(virtual_display, GpuDisplay):
|
||||
try:
|
||||
await async_prove_gpu(browser, virtual_display)
|
||||
except BaseException:
|
||||
await browser.close()
|
||||
raise
|
||||
|
||||
return browser
|
||||
|
||||
|
||||
@@ -872,7 +906,8 @@ def sync_attach_vd(
|
||||
browser: Any, virtual_display: Optional[VirtualDisplay] = None
|
||||
) -> Any: # type: ignore
|
||||
"""
|
||||
Attaches the virtual display to the sync browser cleanup
|
||||
Attaches the virtual display to the sync browser cleanup, and on a GPU
|
||||
display checks the browser renders on the GPU.
|
||||
"""
|
||||
if not virtual_display: # Skip if no virtual display is provided
|
||||
return browser
|
||||
@@ -887,6 +922,13 @@ def sync_attach_vd(
|
||||
|
||||
browser.close = new_close
|
||||
|
||||
if isinstance(virtual_display, GpuDisplay):
|
||||
try:
|
||||
prove_gpu(browser, virtual_display)
|
||||
except BaseException:
|
||||
browser.close()
|
||||
raise
|
||||
|
||||
return browser
|
||||
|
||||
|
||||
@@ -1002,7 +1044,8 @@ def launch_options(
|
||||
headless (Optional[bool]):
|
||||
Whether to run the browser in headless mode. Defaults to False.
|
||||
Note: If you are running linux, passing headless='virtual' to Camoufox, AsyncCamoufox
|
||||
or launch_server will use Xvfb.
|
||||
or launch_server will use Xvfb, and headless='virtual-gpu' a headless Xorg
|
||||
server bound to one GPU.
|
||||
main_world_eval (Optional[bool]):
|
||||
Whether to enable running scripts in the main world.
|
||||
To use this, prepend "mw:" to the script: page.evaluate("mw:" + script).
|
||||
@@ -1576,7 +1619,7 @@ def launch_options(
|
||||
canvas_noise,
|
||||
bool(webgl_config or (config.get('webGl:vendor') and config.get('webGl:renderer'))),
|
||||
bool(block_webgl),
|
||||
lambda: host_gpu(browser_binary, headless, env),
|
||||
lambda: host_gpu(browser_binary, headless, env, firefox_user_prefs),
|
||||
)
|
||||
if cover_canvas:
|
||||
add_canvas_placeholder(firefox_user_prefs)
|
||||
|
||||
@@ -4,7 +4,7 @@ import select
|
||||
import subprocess # nosec
|
||||
import time
|
||||
from shutil import which
|
||||
from typing import Optional
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
from camoufox.exceptions import (
|
||||
CannotExecuteXvfb,
|
||||
@@ -13,8 +13,10 @@ from camoufox.exceptions import (
|
||||
)
|
||||
from camoufox.pkgman import OS_NAME
|
||||
|
||||
# Safe timeout for Xvfb writing display num, prevents infinite hang.
|
||||
DISPLAYFD_READ_TIMEOUT_S = 10.0
|
||||
# Safe timeout for Xvfb writing display num, prevents infinite hang. A cold first
|
||||
# Xvfb on a fresh host took 0.6-8.4 s to report (lang315/camoufox#157), so 10 s
|
||||
# failed launches that would have succeeded.
|
||||
DISPLAYFD_READ_TIMEOUT_S = 30.0
|
||||
|
||||
# Xvfb screen geometry for headless="virtual". The root window size is not
|
||||
# observable (see virtual-display-default-1x1x24 in ci/tribal-rules.yml);
|
||||
@@ -94,24 +96,43 @@ class VirtualDisplay:
|
||||
raise CannotFindXvfb("Please install util-linux (for setpriv) to use headless mode.")
|
||||
return path
|
||||
|
||||
# What get() calls the server in its errors, and what it raises.
|
||||
_name = "Xvfb"
|
||||
_error = CannotExecuteXvfb
|
||||
|
||||
def _wrapper(self) -> List[str]:
|
||||
"""What execs the server. setpriv's parent-death signal stops Xvfb when
|
||||
this process dies without reaching kill(), even by SIGKILL; SIGTERM lets
|
||||
Xvfb remove its own lock and socket. See xvfb-dies-with-its-launcher in
|
||||
ci/tribal-rules.yml."""
|
||||
return [self.setpriv_path, "--pdeathsig", "TERM"]
|
||||
|
||||
def _command(self) -> List[str]:
|
||||
"""The server's argv, less -displayfd, which get() adds."""
|
||||
return [self.xvfb_path, *self.xvfb_args]
|
||||
|
||||
def _server_env(self) -> Optional[Dict[str, str]]:
|
||||
# Force Mesa software GLX; we don't use the GPU anyway.
|
||||
return {**os.environ, "__GLX_VENDOR_LIBRARY_NAME": "mesa", "LIBGL_ALWAYS_SOFTWARE": "1"}
|
||||
|
||||
def _starting(self) -> None:
|
||||
"""Called about every 100ms while the server has not reported a display."""
|
||||
|
||||
def _ready(self) -> None:
|
||||
"""Called once the server has reported its display."""
|
||||
|
||||
def get(self) -> str:
|
||||
self._assert_linux()
|
||||
|
||||
if self.proc is None:
|
||||
# Launch Xvfb with -displayfd so Xvfb itself picks a free display
|
||||
# Launch the server with -displayfd so it picks a free display
|
||||
# number atomically and reports it back. Avoids userspace races.
|
||||
# subprocess.Popen's pass_fds keeps an fd at its parent number in
|
||||
# the child (unlike Node's `stdio: [..., 'pipe']` which renumbers
|
||||
# to 3), so we tell Xvfb that exact number.
|
||||
# to 3), so we tell the server that exact number.
|
||||
program, *args = self._command()
|
||||
read_fd, write_fd = os.pipe()
|
||||
# setpriv's parent-death signal stops Xvfb when this process dies
|
||||
# without reaching kill(), even by SIGKILL; SIGTERM lets Xvfb remove
|
||||
# its own lock and socket. See xvfb-dies-with-its-launcher in
|
||||
# ci/tribal-rules.yml.
|
||||
cmd = [
|
||||
self.setpriv_path, "--pdeathsig", "TERM",
|
||||
self.xvfb_path, "-displayfd", str(write_fd), *self.xvfb_args,
|
||||
]
|
||||
cmd = [*self._wrapper(), program, "-displayfd", str(write_fd), *args]
|
||||
if self.debug:
|
||||
print("Starting virtual display:", " ".join(cmd))
|
||||
self.proc = subprocess.Popen( # nosec
|
||||
@@ -121,31 +142,29 @@ class VirtualDisplay:
|
||||
stderr=None if self.debug else subprocess.DEVNULL,
|
||||
start_new_session=True,
|
||||
pass_fds=(write_fd,),
|
||||
env={
|
||||
**os.environ,
|
||||
# Force Mesa software GLX; we don't use the GPU anyway.
|
||||
"__GLX_VENDOR_LIBRARY_NAME": "mesa",
|
||||
"LIBGL_ALWAYS_SOFTWARE": "1",
|
||||
},
|
||||
env=self._server_env(),
|
||||
)
|
||||
os.close(write_fd) # so the read end EOFs when Xvfb closes its end
|
||||
os.close(write_fd) # so the read end EOFs when the server closes its end
|
||||
|
||||
buf = b""
|
||||
deadline = time.monotonic() + DISPLAYFD_READ_TIMEOUT_S
|
||||
try:
|
||||
while b"\n" not in buf:
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0 or not select.select([read_fd], [], [], remaining)[0]:
|
||||
if remaining <= 0:
|
||||
self.kill()
|
||||
raise CannotExecuteXvfb(
|
||||
f"Xvfb did not report a display within "
|
||||
raise self._error(
|
||||
f"{self._name} did not report a display within "
|
||||
f"{int(DISPLAYFD_READ_TIMEOUT_S * 1000)}ms"
|
||||
)
|
||||
if not select.select([read_fd], [], [], min(remaining, 0.1))[0]:
|
||||
self._starting()
|
||||
continue
|
||||
chunk = os.read(read_fd, 64)
|
||||
if not chunk:
|
||||
self.kill()
|
||||
raise CannotExecuteXvfb(
|
||||
f"Xvfb did not report a display "
|
||||
raise self._error(
|
||||
f"{self._name} did not report a display "
|
||||
f"(got {buf!r}, exit={self.proc.poll()})"
|
||||
)
|
||||
buf += chunk
|
||||
@@ -156,7 +175,8 @@ class VirtualDisplay:
|
||||
self._display = int(buf.strip())
|
||||
except ValueError:
|
||||
self.kill()
|
||||
raise CannotExecuteXvfb(f"Xvfb wrote non-integer display: {buf!r}")
|
||||
raise self._error(f"{self._name} wrote non-integer display: {buf!r}")
|
||||
self._ready()
|
||||
elif self.debug:
|
||||
print(f"Using virtual display: {self._display}")
|
||||
|
||||
|
||||
@@ -63,6 +63,11 @@ canvas_noise: >-
|
||||
the identity claims another OS or GPU than this machine's, whose canvas would
|
||||
show the real one. Pass `canvas_noise=False` to keep stock canvas behaviour.
|
||||
|
||||
gpu_software_rendering: >-
|
||||
The browser on the headless="virtual-gpu" display never rendered on the GPU, so it
|
||||
renders canvas and WebGL in software, which a page can tell from the GPU the
|
||||
identity claims. Pass `require_gpu=True` (the default) to fail the launch instead.
|
||||
|
||||
block_images: >-
|
||||
Blocking image requests has been reported to cause detection issues on major WAFs.
|
||||
|
||||
|
||||
@@ -51,6 +51,13 @@ _HOST_DEPENDENT_EXTENSIONS = frozenset({'OVR_multiview2'})
|
||||
# unconditionally on D3D11 (renderer11_utils.cpp); 146's ANGLE did not.
|
||||
_BACKEND_EXTENSIONS = {'win': ('EXT_depth_clamp',)}
|
||||
|
||||
# Firefox 158 (bug 1854003) exposes EXT_polygon_offset_clamp wherever the GL
|
||||
# backend clamps polygon offsets: ANGLE's D3D11 backend from feature level 10_0,
|
||||
# which WebGL2 needs, and Linux desktop GL drivers (ARB_polygon_offset_clamp,
|
||||
# core in GL 4.6), whose recordings have EXT_depth_clamp. macOS release builds
|
||||
# run WebGL on CGL's GL 4.1, which lacks it.
|
||||
_POLYGON_OFFSET_CLAMP = 'EXT_polygon_offset_clamp'
|
||||
|
||||
_ALIASED_LINE_WIDTH_RANGE = '33902'
|
||||
|
||||
# What Firefox reports under privacy.resistFingerprinting. A Camoufox identity
|
||||
@@ -111,10 +118,22 @@ def firefox_gpus(target_os: str) -> FrozenSet[Tuple[str, str]]:
|
||||
)
|
||||
|
||||
|
||||
def _context_config(prefix: str, webgl: Dict[str, Any], target_os: str) -> Dict[str, Any]:
|
||||
def _backend_extensions(target_os: str, webgl: Dict[str, Any], webgl2: Any) -> Tuple[str, ...]:
|
||||
"""The extensions the shipped Firefox adds on this device's backend."""
|
||||
extensions = _BACKEND_EXTENSIONS.get(target_os, ())
|
||||
if (target_os == 'win' and webgl2) or (
|
||||
target_os == 'lin' and 'EXT_depth_clamp' in webgl['supportedExtensions']
|
||||
):
|
||||
extensions += (_POLYGON_OFFSET_CLAMP,)
|
||||
return extensions
|
||||
|
||||
|
||||
def _context_config(
|
||||
prefix: str, webgl: Dict[str, Any], target_os: str, backend: Tuple[str, ...]
|
||||
) -> Dict[str, Any]:
|
||||
blocked = _filtered_extensions(target_os)
|
||||
extensions = [extension for extension in webgl['supportedExtensions'] if extension not in blocked]
|
||||
for extension in _BACKEND_EXTENSIONS.get(target_os, ()):
|
||||
for extension in backend:
|
||||
if extension not in extensions:
|
||||
# Firefox lists them in WebGLExtensionID order: alphabetical, ignoring case.
|
||||
at = next((i for i, other in enumerate(extensions) if other.casefold() > extension.casefold()), None)
|
||||
@@ -134,14 +153,15 @@ def to_config(webgl: Dict[str, Any], webgl2: Any, target_os: str) -> Dict[str, A
|
||||
|
||||
`webgl2` is `[]` for a device without WebGL2.
|
||||
"""
|
||||
backend = _backend_extensions(target_os, webgl, webgl2)
|
||||
config = {
|
||||
'webGl:vendor': webgl['vendor'],
|
||||
'webGl:renderer': webgl['renderer'],
|
||||
**_context_config('webGl', webgl, target_os),
|
||||
**_context_config('webGl', webgl, target_os, backend),
|
||||
'webGl2Enabled': bool(webgl2),
|
||||
}
|
||||
if webgl2:
|
||||
config.update(_context_config('webGl2', webgl2, target_os))
|
||||
config.update(_context_config('webGl2', webgl2, target_os, backend))
|
||||
# Firefox 156 creates WebGL1 on the same GL profile as WebGL2, so both
|
||||
# report one line width range. 146 forced WebGL1 onto a compatibility
|
||||
# profile, and fpgen recorded that profile's driver range.
|
||||
|
||||
@@ -12,11 +12,13 @@ Run with:
|
||||
"""
|
||||
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from camoufox import addons as addons_mod
|
||||
from camoufox.addons import DefaultAddons, maybe_download_addons
|
||||
from camoufox.addons import DefaultAddons, add_default_addons, maybe_download_addons
|
||||
|
||||
UBO = DefaultAddons.UBO.name
|
||||
|
||||
@@ -89,3 +91,89 @@ def test_failed_download_removes_partial_dir(addons_dir, monkeypatch):
|
||||
# trusting an addon that has no manifest.json.
|
||||
assert not path.exists()
|
||||
assert out == []
|
||||
|
||||
|
||||
def test_concurrent_launches_download_once(addons_dir, monkeypatch):
|
||||
# Two launches in one process share the check-then-extract. A lock created
|
||||
# per call guarded nothing, so both saw no manifest and both downloaded.
|
||||
calls = []
|
||||
|
||||
def slow(url, extract_path, name):
|
||||
calls.append(name)
|
||||
time.sleep(0.2)
|
||||
_write_manifest(url, extract_path, name)
|
||||
|
||||
monkeypatch.setattr(addons_mod, "download_and_extract", slow)
|
||||
|
||||
outs = [[], []]
|
||||
threads = [threading.Thread(target=add_default_addons, args=(out,)) for out in outs]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
assert calls == [UBO]
|
||||
assert outs == [[str(addons_dir / UBO)]] * 2
|
||||
|
||||
|
||||
def test_extraction_appears_complete_or_not_at_all(tmp_path, monkeypatch):
|
||||
# Another process checks for manifest.json without the lock, so the addon
|
||||
# directory must never hold a manifest while its other files are missing.
|
||||
target = tmp_path / UBO
|
||||
target.mkdir()
|
||||
seen = []
|
||||
|
||||
def fake_unzip(buffer, extract_path, desc=None, bar=True):
|
||||
seen.append((target / "manifest.json").exists())
|
||||
_write_manifest(None, extract_path, UBO)
|
||||
with open(os.path.join(extract_path, "background.js"), "w") as f:
|
||||
f.write("")
|
||||
seen.append((target / "manifest.json").exists())
|
||||
|
||||
monkeypatch.setattr(addons_mod, "webdl", lambda *a, **k: b"")
|
||||
monkeypatch.setattr(addons_mod, "unzip", fake_unzip)
|
||||
|
||||
addons_mod.download_and_extract("url", str(target), UBO)
|
||||
|
||||
assert seen == [False, False]
|
||||
assert sorted(os.listdir(target)) == ["background.js", "manifest.json"]
|
||||
assert os.listdir(tmp_path) == [UBO]
|
||||
|
||||
|
||||
def test_complete_addon_installed_meanwhile_is_kept(tmp_path, monkeypatch):
|
||||
# Another process can finish installing the addon while this one extracts.
|
||||
# Its copy may already be loading in a browser, so it must not be deleted.
|
||||
target = tmp_path / UBO
|
||||
target.mkdir()
|
||||
|
||||
def fake_unzip(buffer, extract_path, desc=None, bar=True):
|
||||
_write_manifest(None, extract_path, UBO)
|
||||
_write_manifest(None, str(target), UBO)
|
||||
with open(target / "theirs.js", "w") as f:
|
||||
f.write("")
|
||||
|
||||
monkeypatch.setattr(addons_mod, "webdl", lambda *a, **k: b"")
|
||||
monkeypatch.setattr(addons_mod, "unzip", fake_unzip)
|
||||
|
||||
addons_mod.download_and_extract("url", str(target), UBO)
|
||||
|
||||
assert sorted(os.listdir(target)) == ["manifest.json", "theirs.js"]
|
||||
assert os.listdir(tmp_path) == [UBO]
|
||||
|
||||
|
||||
def test_failed_download_keeps_complete_addon(addons_dir, monkeypatch):
|
||||
# This launch's download fails after another process installed the addon.
|
||||
# That copy may already be loading in a browser, so it must not be deleted.
|
||||
path = addons_dir / UBO
|
||||
|
||||
def fail(url, extract_path, name):
|
||||
_write_manifest(url, str(path), name)
|
||||
raise RuntimeError("network died mid-download")
|
||||
|
||||
monkeypatch.setattr(addons_mod, "download_and_extract", fail)
|
||||
|
||||
out = []
|
||||
maybe_download_addons([DefaultAddons.UBO], out)
|
||||
|
||||
assert (path / "manifest.json").exists()
|
||||
assert out == []
|
||||
|
||||
@@ -0,0 +1,632 @@
|
||||
"""
|
||||
Tests for camoufox.gpudisplay (headless='virtual-gpu') that need no GPU:
|
||||
enumeration and selection on a fake sysfs tree, the generated xorg.conf, the
|
||||
Xorg argv, the browser's environment and prefs, the identity's GPU probe, the
|
||||
VT guard, the GPU-use check, and teardown.
|
||||
|
||||
The real display is browser/tests/native/test_gpu_display.py.
|
||||
|
||||
Mirrors typescript/tests/gpudisplay.test.ts.
|
||||
"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
from camoufox import gpudisplay, host_rendering, sync_api, utils
|
||||
from camoufox._warnings import LeakWarning
|
||||
from camoufox.exceptions import (
|
||||
AmbiguousGpuSelection,
|
||||
CannotExecuteXorg,
|
||||
CannotFindXorg,
|
||||
GpuNotFound,
|
||||
GpuSoftwareFallback,
|
||||
GpuVtTakeover,
|
||||
NoGpuFound,
|
||||
VirtualDisplayError,
|
||||
)
|
||||
from camoufox.gpudisplay import (
|
||||
GPU_PREFS,
|
||||
GpuDevice,
|
||||
GpuDisplay,
|
||||
config_name,
|
||||
enumerate_gpus,
|
||||
select_gpu,
|
||||
xorg_config,
|
||||
)
|
||||
|
||||
pytestmark = pytest.mark.skipif(sys.platform != "linux", reason="virtual-gpu is Linux-only")
|
||||
|
||||
NVIDIA = GpuDevice(
|
||||
pci_address='0000:01:00.0',
|
||||
render_node='/dev/dri/renderD129',
|
||||
card_node='/dev/dri/card1',
|
||||
vendor_id='0x10de',
|
||||
device_id='0x2c02',
|
||||
kernel_driver='nvidia',
|
||||
nvidia_node='/dev/nvidia0',
|
||||
)
|
||||
AMD = GpuDevice(
|
||||
pci_address='0000:6f:00.0',
|
||||
render_node='/dev/dri/renderD128',
|
||||
card_node='/dev/dri/card0',
|
||||
vendor_id='0x1002',
|
||||
device_id='0x744c',
|
||||
kernel_driver='amdgpu',
|
||||
)
|
||||
AMD_IGPU = GpuDevice(
|
||||
pci_address='0000:70:00.0',
|
||||
render_node='/dev/dri/renderD130',
|
||||
card_node='/dev/dri/card3',
|
||||
vendor_id='0x1002',
|
||||
device_id='0x13c0',
|
||||
kernel_driver='amdgpu',
|
||||
)
|
||||
HOST = [NVIDIA, AMD, AMD_IGPU]
|
||||
|
||||
|
||||
def _device(root: Path, pci: str, vendor: str, device: str, driver: str) -> Path:
|
||||
path = root / 'sys/devices/pci0000:00' / pci
|
||||
path.mkdir(parents=True)
|
||||
(path / 'vendor').write_text(vendor + '\n')
|
||||
(path / 'device').write_text(device + '\n')
|
||||
(path / 'uevent').write_text(f'DRIVER={driver}\nPCI_SLOT_NAME={pci}\n')
|
||||
return path
|
||||
|
||||
|
||||
def _node(root: Path, name: str, device: Path) -> None:
|
||||
(root / 'dev/dri' / name).touch()
|
||||
(root / 'sys/class/drm' / name).mkdir()
|
||||
(root / 'sys/class/drm' / name / 'device').symlink_to(device)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host(tmp_path):
|
||||
"""This machine's layout: the render nodes are numbered in driver load order,
|
||||
so the AMD card is renderD128 though the NVIDIA one has the lower address."""
|
||||
(tmp_path / 'dev/dri').mkdir(parents=True)
|
||||
(tmp_path / 'sys/class/drm').mkdir(parents=True)
|
||||
nvidia = _device(tmp_path, '0000:01:00.0', '0x10de', '0x2c02', 'nvidia')
|
||||
amd = _device(tmp_path, '0000:6f:00.0', '0x1002', '0x744C', 'amdgpu')
|
||||
igpu = _device(tmp_path, '0000:70:00.0', '0x1002', '0x13c0', 'amdgpu')
|
||||
for name, device in (
|
||||
('renderD128', amd), ('card0', amd),
|
||||
('renderD129', nvidia), ('card1', nvidia),
|
||||
('renderD130', igpu), ('card3', igpu),
|
||||
):
|
||||
_node(tmp_path, name, device)
|
||||
# A connector is a card*-named entry that is not a card.
|
||||
(tmp_path / 'sys/class/drm/card1-DP-1').mkdir()
|
||||
# A render node with no PCI device cannot be bound by BusID.
|
||||
platform = tmp_path / 'sys/devices/platform/vgem'
|
||||
platform.mkdir(parents=True)
|
||||
_node(tmp_path, 'renderD131', platform)
|
||||
info = tmp_path / 'proc/driver/nvidia/gpus/0000:01:00.0'
|
||||
info.mkdir(parents=True)
|
||||
(info / 'information').write_text('Model: \t NVIDIA GeForce RTX 5080\nDevice Minor: \t 0\n')
|
||||
return tmp_path
|
||||
|
||||
|
||||
class TestEnumeration:
|
||||
def test_lists_pci_gpus_in_address_order_not_render_node_order(self, host):
|
||||
assert enumerate_gpus(host) == HOST
|
||||
|
||||
def test_no_dri_directory_is_no_gpus(self, tmp_path):
|
||||
assert enumerate_gpus(tmp_path) == []
|
||||
|
||||
def test_bus_id_is_decimal_without_the_domain(self):
|
||||
assert NVIDIA.bus_id == 'PCI:1:0:0'
|
||||
assert AMD.bus_id == 'PCI:111:0:0'
|
||||
|
||||
def test_nvidia_gets_its_own_driver_and_everything_else_modesetting(self):
|
||||
assert NVIDIA.xorg_driver == 'nvidia'
|
||||
assert AMD.xorg_driver == 'modesetting'
|
||||
|
||||
def test_the_nvidia_driver_device_is_found_by_pci_address(self, host):
|
||||
assert [gpu.nvidia_node for gpu in enumerate_gpus(host)] == ['/dev/nvidia0', None, None]
|
||||
|
||||
|
||||
class TestSelection:
|
||||
@pytest.mark.parametrize(
|
||||
'selector',
|
||||
[0, '0', 'renderD129', '/dev/dri/renderD129', 'card1', '/dev/dri/card1',
|
||||
'01:00.0', '0000:01:00.0', '10de:2c02', '0x10de:0x2c02', 'nvidia', 'NVIDIA'],
|
||||
)
|
||||
def test_every_kind_of_selector_names_the_nvidia_card(self, selector):
|
||||
assert select_gpu(selector, HOST) == NVIDIA
|
||||
|
||||
def test_a_pci_address_names_the_amd_card(self):
|
||||
assert select_gpu('6f:00.0', HOST) == AMD
|
||||
|
||||
def test_the_environment_names_the_card_when_the_option_does_not(self, monkeypatch):
|
||||
monkeypatch.setenv('CAMOUFOX_GPU', '6f:00.0')
|
||||
assert select_gpu(None, HOST) == AMD
|
||||
|
||||
def test_the_option_wins_over_the_environment(self, monkeypatch):
|
||||
monkeypatch.setenv('CAMOUFOX_GPU', '6f:00.0')
|
||||
assert select_gpu('01:00.0', HOST) == NVIDIA
|
||||
|
||||
def test_a_lone_gpu_needs_no_selector(self, monkeypatch):
|
||||
monkeypatch.delenv('CAMOUFOX_GPU', raising=False)
|
||||
assert select_gpu(None, [AMD]) == AMD
|
||||
|
||||
def test_several_gpus_and_no_selector_is_ambiguous(self, monkeypatch):
|
||||
monkeypatch.delenv('CAMOUFOX_GPU', raising=False)
|
||||
with pytest.raises(AmbiguousGpuSelection, match='3 GPUs'):
|
||||
select_gpu(None, HOST)
|
||||
|
||||
def test_a_selector_matching_two_gpus_is_ambiguous(self):
|
||||
with pytest.raises(AmbiguousGpuSelection, match=r"(?s)\[1\].*\[2\]"):
|
||||
select_gpu('amd', HOST)
|
||||
|
||||
def test_a_selector_matching_nothing_fails(self):
|
||||
with pytest.raises(GpuNotFound, match='intel'):
|
||||
select_gpu('intel', HOST)
|
||||
|
||||
def test_an_index_past_the_end_fails(self):
|
||||
with pytest.raises(GpuNotFound):
|
||||
select_gpu(3, HOST)
|
||||
|
||||
def test_no_gpus_fails(self):
|
||||
with pytest.raises(NoGpuFound):
|
||||
select_gpu('0', [])
|
||||
|
||||
def test_every_failure_is_a_virtual_display_error(self):
|
||||
with pytest.raises(VirtualDisplayError):
|
||||
select_gpu('intel', HOST)
|
||||
|
||||
|
||||
class TestXorgConfig:
|
||||
def test_nvidia(self):
|
||||
config = xorg_config(NVIDIA)
|
||||
for line in (
|
||||
'Option "AutoAddDevices" "false"',
|
||||
'Option "AutoAddGPU" "false"',
|
||||
'Option "DontVTSwitch" "true"',
|
||||
'Option "AllowMouseOpenFail" "true"',
|
||||
'Driver "nvidia"',
|
||||
'BusID "PCI:1:0:0"',
|
||||
'Option "UseDisplayDevice" "None"',
|
||||
'Option "AllowEmptyInitialConfiguration" "true"',
|
||||
'HorizSync 5.0 - 1000.0',
|
||||
'VertRefresh 5.0 - 200.0',
|
||||
'DefaultDepth 24',
|
||||
'Virtual 1920 1080',
|
||||
):
|
||||
assert line in config, line
|
||||
|
||||
def test_other_cards_use_modesetting_without_the_nvidia_option(self):
|
||||
config = xorg_config(AMD)
|
||||
assert 'Driver "modesetting"' in config
|
||||
assert 'BusID "PCI:111:0:0"' in config
|
||||
assert 'UseDisplayDevice' not in config
|
||||
|
||||
def test_names_no_render_node_which_can_change_between_boots(self):
|
||||
assert 'renderD' not in xorg_config(NVIDIA)
|
||||
|
||||
def test_the_file_is_named_by_pci_address(self):
|
||||
assert config_name(NVIDIA) == 'camoufox-gpu-0000_01_00_0.conf'
|
||||
|
||||
def test_the_cli_prints_it(self, monkeypatch, capsys):
|
||||
from camoufox.__main__ import cli
|
||||
|
||||
monkeypatch.setattr(gpudisplay, 'enumerate_gpus', lambda: HOST)
|
||||
with pytest.raises(SystemExit) as exit:
|
||||
cli(['gpu-xorg-config', '01:00.0'])
|
||||
assert exit.value.code == 0
|
||||
assert capsys.readouterr().out == xorg_config(NVIDIA)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def xwrapper(monkeypatch, tmp_path):
|
||||
config = tmp_path / 'Xwrapper.config'
|
||||
wrapper = tmp_path / 'Xorg.wrap'
|
||||
wrapper.touch()
|
||||
monkeypatch.setattr(gpudisplay, 'XWRAPPER_CONFIG', config)
|
||||
monkeypatch.setattr(gpudisplay, 'XORG_WRAPPERS', (wrapper,))
|
||||
monkeypatch.setattr(os, 'geteuid', lambda: 1000)
|
||||
return config
|
||||
|
||||
|
||||
class TestSetup:
|
||||
"""One command sets a machine up, and every failure it would fix names it."""
|
||||
|
||||
def test_a_stock_wrapper_is_named_with_the_fix(self, xwrapper):
|
||||
# What Debian and Ubuntu ship.
|
||||
xwrapper.write_text('# Xwrapper.config\nallowed_users=console\n')
|
||||
error = GpuDisplay(AMD)._error('Xorg exited (1) right after starting.')
|
||||
assert isinstance(error, CannotExecuteXorg)
|
||||
message = str(error)
|
||||
assert f'{xwrapper} has allowed_users=console, needs allowed_users=anybody' in message
|
||||
assert f'{xwrapper} has needs_root_rights=unset, needs needs_root_rights=yes' in message
|
||||
assert message.endswith('python -m camoufox gpu-setup 0000:6f:00.0 | sudo sh')
|
||||
|
||||
def test_a_missing_wrapper_is_named(self, xwrapper, monkeypatch, tmp_path):
|
||||
xwrapper.write_text('allowed_users=anybody\nneeds_root_rights=yes\n')
|
||||
monkeypatch.setattr(gpudisplay, 'XORG_WRAPPERS', (tmp_path / 'none',))
|
||||
assert gpudisplay.setup_problems() == [f'no setuid Xorg.wrap ({tmp_path / "none"})']
|
||||
|
||||
def test_a_set_up_machine_adds_nothing_to_the_error(self, xwrapper):
|
||||
xwrapper.write_text('allowed_users = anybody\nneeds_root_rights = yes\n')
|
||||
assert str(GpuDisplay(AMD)._error('Xorg exited (1).')) == 'Xorg exited (1).'
|
||||
|
||||
def test_as_root_the_wrapper_is_not_involved(self, xwrapper, monkeypatch):
|
||||
monkeypatch.setattr(os, 'geteuid', lambda: 0)
|
||||
assert str(GpuDisplay(AMD)._error('Xorg exited (1).')) == 'Xorg exited (1).'
|
||||
|
||||
@pytest.mark.parametrize('gpu', [NVIDIA, AMD], ids=['nvidia', 'amd'])
|
||||
def test_the_script_is_valid_sh_and_installs_the_card_config(self, gpu):
|
||||
script = gpudisplay.setup_script(gpu)
|
||||
assert subprocess.run(['sh', '-n'], input=script, text=True).returncode == 0
|
||||
config = gpudisplay.SYSTEM_CONFIG_DIR / config_name(gpu)
|
||||
assert f"cat > {config} <<'CAMOUFOX_XORG_CONF'\n{xorg_config(gpu)}CAMOUFOX_XORG_CONF\n" in script
|
||||
assert "printf 'allowed_users=anybody\\nneeds_root_rights=yes\\n'" in script
|
||||
assert ('nvidia_drv.so' in script) is (gpu.xorg_driver == 'nvidia')
|
||||
|
||||
def test_the_cli_prints_it(self, monkeypatch, capsys):
|
||||
from camoufox.__main__ import cli
|
||||
|
||||
monkeypatch.setattr(gpudisplay, 'enumerate_gpus', lambda: HOST)
|
||||
with pytest.raises(SystemExit) as exit:
|
||||
cli(['gpu-setup', '01:00.0'])
|
||||
assert exit.value.code == 0
|
||||
assert capsys.readouterr().out == gpudisplay.setup_script(NVIDIA)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def xorg(monkeypatch):
|
||||
monkeypatch.setattr(gpudisplay, 'which', lambda name: f'/usr/bin/{name}')
|
||||
|
||||
|
||||
class TestCommand:
|
||||
def test_without_root_the_installed_config_is_named(self, xorg, monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(os, 'geteuid', lambda: 1000)
|
||||
monkeypatch.setattr(gpudisplay, 'SYSTEM_CONFIG_DIR', tmp_path)
|
||||
(tmp_path / config_name(NVIDIA)).write_text(xorg_config(NVIDIA))
|
||||
|
||||
assert GpuDisplay(NVIDIA)._command() == [
|
||||
'/usr/bin/Xorg', '-config', 'camoufox-gpu-0000_01_00_0.conf',
|
||||
'-terminate', '2', '-nolisten', 'tcp', '-sharevts', '-novtswitch',
|
||||
]
|
||||
|
||||
def test_without_root_a_missing_config_says_how_to_install_it(self, xorg, monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(os, 'geteuid', lambda: 1000)
|
||||
monkeypatch.setattr(gpudisplay, 'SYSTEM_CONFIG_DIR', tmp_path)
|
||||
|
||||
with pytest.raises(CannotExecuteXorg) as error:
|
||||
GpuDisplay(NVIDIA)._command()
|
||||
assert str(error.value).endswith('python -m camoufox gpu-setup 0000:01:00.0 | sudo sh')
|
||||
|
||||
def test_a_missing_xorg_says_how_to_set_the_machine_up(self, monkeypatch):
|
||||
monkeypatch.setattr(gpudisplay, 'which', lambda name: None)
|
||||
monkeypatch.setattr(os, 'geteuid', lambda: 0)
|
||||
with pytest.raises(CannotFindXorg) as error:
|
||||
GpuDisplay(AMD).xorg_path
|
||||
# As root the script needs no sudo.
|
||||
assert str(error.value).endswith('python -m camoufox gpu-setup 0000:6f:00.0 | sh')
|
||||
|
||||
def test_as_root_a_temporary_config_and_log_are_used_and_removed(self, xorg, monkeypatch):
|
||||
monkeypatch.setattr(os, 'geteuid', lambda: 0)
|
||||
display = GpuDisplay(AMD)
|
||||
command = display._command()
|
||||
|
||||
config = command[command.index('-config') + 1]
|
||||
log = command[command.index('-logfile') + 1]
|
||||
assert os.path.isabs(config)
|
||||
assert Path(config).read_text() == xorg_config(AMD)
|
||||
assert command[-6:] == ['-terminate', '2', '-nolisten', 'tcp', '-sharevts', '-novtswitch']
|
||||
|
||||
Path(log).touch()
|
||||
display.kill()
|
||||
assert not os.path.exists(config) and not os.path.exists(log)
|
||||
|
||||
def test_xorg_inherits_the_environment_untouched(self):
|
||||
# Xvfb's server is forced onto Mesa's software GLX; this one must not be.
|
||||
assert GpuDisplay(NVIDIA)._server_env() is None
|
||||
|
||||
|
||||
class TestLaunch:
|
||||
@pytest.fixture
|
||||
def started(self, monkeypatch):
|
||||
monkeypatch.setattr(utils, 'select_gpu', lambda selector: NVIDIA)
|
||||
monkeypatch.setattr(GpuDisplay, 'get', lambda self: ':77')
|
||||
|
||||
def test_points_the_launch_at_the_display_with_the_gpu_prefs(self, started):
|
||||
kwargs = {'gpu': '01:00.0', 'firefox_user_prefs': {'gfx.webrender.all': False}}
|
||||
display, headless = utils.start_virtual_display('virtual-gpu', False, kwargs)
|
||||
|
||||
assert isinstance(display, GpuDisplay) and display.require_gpu
|
||||
assert headless is False
|
||||
assert kwargs == {
|
||||
'virtual_display': ':77',
|
||||
# The caller's prefs win.
|
||||
'firefox_user_prefs': {**GPU_PREFS, 'gfx.webrender.all': False},
|
||||
}
|
||||
|
||||
def test_require_gpu_false_only_warns(self, started):
|
||||
display, _ = utils.start_virtual_display('virtual-gpu', False, {'require_gpu': False})
|
||||
assert not display.require_gpu
|
||||
|
||||
@pytest.mark.parametrize('option', ['gpu', 'require_gpu'])
|
||||
def test_gpu_options_without_virtual_gpu_are_refused(self, option):
|
||||
with pytest.raises(ValueError, match="virtual-gpu"):
|
||||
utils.start_virtual_display('virtual', False, {option: '0'})
|
||||
|
||||
def test_the_browser_gets_only_x11_settings_no_gpu_overrides(self, started):
|
||||
kwargs = {'gpu': '0'}
|
||||
utils.start_virtual_display('virtual-gpu', False, kwargs)
|
||||
with mock.patch.object(utils, 'installed_verstr', lambda: '150.0.2'), mock.patch.object(
|
||||
utils, 'launch_path', lambda **kw: '/nonexistent/camoufox'
|
||||
):
|
||||
env = utils.launch_options(headless=False, i_know_what_im_doing=True, env={}, **kwargs)['env']
|
||||
|
||||
assert env['DISPLAY'] == ':77'
|
||||
overrides = [
|
||||
key for key in env
|
||||
if key.startswith(('MESA_', '__GLX_', 'LIBGL_', 'DRI_PRIME', '__NV_'))
|
||||
]
|
||||
assert overrides == []
|
||||
|
||||
def test_the_identity_probes_the_gpu_display_with_its_prefs(self, started):
|
||||
"""The identity claims the GPU Firefox reports on the GPU display, not
|
||||
the one the desktop renders on."""
|
||||
probes = []
|
||||
|
||||
def probe(executable_path, headless, env, prefs):
|
||||
probes.append((env.get('DISPLAY'), {k: prefs.get(k) for k in GPU_PREFS}))
|
||||
return ('NVIDIA Corporation', 'NVIDIA GeForce GTX 980, or similar')
|
||||
|
||||
kwargs = {'gpu': '0'}
|
||||
utils.start_virtual_display('virtual-gpu', False, kwargs)
|
||||
with mock.patch.object(utils, '_host_os_key', lambda: 'lin'), mock.patch.object(
|
||||
utils, 'host_gpu', probe
|
||||
), mock.patch.object(utils, 'installed_verstr', lambda: '150.0.2'), mock.patch.object(
|
||||
utils, 'launch_path', lambda **kw: '/nonexistent/camoufox'
|
||||
):
|
||||
options = utils.launch_options(os='linux', headless=False, i_know_what_im_doing=True, **kwargs)
|
||||
|
||||
assert probes == [(':77', GPU_PREFS)]
|
||||
prefs = options['firefox_user_prefs']
|
||||
assert not host_rendering.has_canvas_placeholder(prefs)
|
||||
|
||||
def test_the_probe_reads_with_the_launch_rendering_prefs_only(self, monkeypatch):
|
||||
calls = []
|
||||
monkeypatch.setattr(host_rendering, '_host_gpu', lambda *args: calls.append(args))
|
||||
host_rendering.host_gpu('/x/camoufox', False, {'DISPLAY': ':77'}, {**GPU_PREFS, 'intl.locale.requested': 'en-US'})
|
||||
|
||||
assert calls == [('/x/camoufox', False, (('DISPLAY', ':77'),), tuple(sorted(GPU_PREFS.items())))]
|
||||
|
||||
def test_a_failed_launch_tears_the_display_down(self, monkeypatch):
|
||||
display = mock.Mock()
|
||||
monkeypatch.setattr(sync_api, 'start_virtual_display', lambda headless, debug, kwargs: (display, False))
|
||||
monkeypatch.setattr(sync_api, 'launch_options', mock.Mock(side_effect=RuntimeError('boom')))
|
||||
|
||||
with pytest.raises(RuntimeError, match='boom'):
|
||||
sync_api.NewBrowser(mock.Mock(), headless='virtual-gpu')
|
||||
display.kill.assert_called_once()
|
||||
|
||||
|
||||
class FakePage:
|
||||
def __init__(self):
|
||||
self.url = None
|
||||
self.closed = False
|
||||
|
||||
def goto(self, url):
|
||||
self.url = url
|
||||
|
||||
def close(self):
|
||||
self.closed = True
|
||||
|
||||
|
||||
class FakeBrowser:
|
||||
def __init__(self):
|
||||
self.page = FakePage()
|
||||
self.closed = False
|
||||
|
||||
def new_page(self):
|
||||
return self.page
|
||||
|
||||
def close(self):
|
||||
self.closed = True
|
||||
|
||||
|
||||
class TestGpuProof:
|
||||
def test_a_browser_that_opened_the_gpu_is_kept(self, monkeypatch):
|
||||
display = GpuDisplay(NVIDIA)
|
||||
monkeypatch.setattr(display, 'wait_for_gpu', lambda: True)
|
||||
browser = FakeBrowser()
|
||||
|
||||
assert utils.sync_attach_vd(browser, display) is browser
|
||||
assert browser.page.url == gpudisplay.PROBE_PAGE and browser.page.closed
|
||||
assert not browser.closed
|
||||
|
||||
def test_a_browser_in_software_is_closed_and_the_launch_fails(self, monkeypatch):
|
||||
display = GpuDisplay(NVIDIA)
|
||||
monkeypatch.setattr(display, 'wait_for_gpu', lambda: False)
|
||||
kill = mock.Mock()
|
||||
monkeypatch.setattr(display, 'kill', kill)
|
||||
browser = FakeBrowser()
|
||||
|
||||
with pytest.raises(GpuSoftwareFallback):
|
||||
utils.sync_attach_vd(browser, display)
|
||||
assert browser.closed
|
||||
kill.assert_called()
|
||||
|
||||
def test_require_gpu_false_warns_instead(self, monkeypatch):
|
||||
display = GpuDisplay(NVIDIA, require_gpu=False)
|
||||
monkeypatch.setattr(display, 'wait_for_gpu', lambda: False)
|
||||
|
||||
with warnings.catch_warnings(record=True) as caught:
|
||||
warnings.simplefilter('always')
|
||||
utils.sync_attach_vd(FakeBrowser(), display)
|
||||
assert any(issubclass(w.category, LeakWarning) and 'software' in str(w.message) for w in caught)
|
||||
|
||||
def test_the_wait_is_overridable(self, monkeypatch):
|
||||
display = GpuDisplay(NVIDIA)
|
||||
monkeypatch.setattr(display, 'uses_gpu', lambda: False)
|
||||
monkeypatch.setenv('CAMOUFOX_GPU_WAIT_S', '0.3')
|
||||
start = time.monotonic()
|
||||
assert display.wait_for_gpu() is False
|
||||
assert time.monotonic() - start < 5
|
||||
|
||||
|
||||
def _holding(path: str, display: str) -> subprocess.Popen:
|
||||
"""A process on `display` that holds `path` open, as a browser holds the GPU."""
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, '-c', f'f = open({path!r}); print(1, flush=True); import time; time.sleep(60)'],
|
||||
env={**os.environ, 'DISPLAY': display},
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
proc.stdout.readline()
|
||||
return proc
|
||||
|
||||
|
||||
# /proc/<pid>/fdinfo of Firefox's render node on an RX 7900 XTX, trimmed.
|
||||
AMDGPU_FDINFO = """pos:\t0
|
||||
flags:\t02100002
|
||||
drm-driver:\tamdgpu
|
||||
drm-client-id:\t5873
|
||||
drm-total-vram:\t159812 KiB
|
||||
drm-engine-gfx:\t{gfx} ns
|
||||
drm-engine-compute:\t0 ns
|
||||
drm-engine-capacity-gfx:\t1
|
||||
"""
|
||||
|
||||
# From the kernel's Xe usage stats documentation.
|
||||
XE_FDINFO = """pos:\t0
|
||||
flags:\t0100002
|
||||
drm-driver:\txe
|
||||
drm-client-id:\t21
|
||||
drm-cycles-rcs:\t{rcs}
|
||||
drm-total-cycles-rcs:\t7655183225
|
||||
drm-cycles-bcs:\t0
|
||||
drm-total-cycles-bcs:\t7655183225
|
||||
drm-engine-capacity-vcs:\t2
|
||||
"""
|
||||
|
||||
|
||||
class TestGpuUse:
|
||||
"""Rendering is read from what the kernel accounts to the browser, never
|
||||
from the renderer string, which the identity spoofs."""
|
||||
|
||||
def test_engine_time_on_the_render_node_is_rendering(self):
|
||||
assert gpudisplay.drew_on_gpu(AMDGPU_FDINFO.format(gfx=39907610))
|
||||
|
||||
def test_xe_busy_cycles_count(self):
|
||||
# xe reports cycles only; its total-cycles line grows on an idle client too.
|
||||
assert gpudisplay.drew_on_gpu(XE_FDINFO.format(rcs=28257900))
|
||||
assert not gpudisplay.drew_on_gpu(XE_FDINFO.format(rcs=0))
|
||||
|
||||
def test_an_idle_render_node_is_not(self):
|
||||
# Firefox opens it for DMABuf in software too; the capacity line is not time.
|
||||
assert not gpudisplay.drew_on_gpu(AMDGPU_FDINFO.format(gfx=0))
|
||||
assert not gpudisplay.drew_on_gpu('pos:\t0\nflags:\t02\n')
|
||||
|
||||
@pytest.fixture
|
||||
def card(self, tmp_path):
|
||||
for name in ('renderD200', 'nvidia7'):
|
||||
(tmp_path / name).touch()
|
||||
return GpuDevice(
|
||||
'0000:01:00.0', str(tmp_path / 'renderD200'), None, '0x10de', '0x2c02', 'nvidia',
|
||||
str(tmp_path / 'nvidia7'),
|
||||
)
|
||||
|
||||
@pytest.fixture
|
||||
def display(self, card):
|
||||
display = GpuDisplay(card)
|
||||
display._display = 98765
|
||||
return display
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'held, on, rendering',
|
||||
[
|
||||
('nvidia_node', ':98765', True),
|
||||
('nvidia_node', ':987650', False),
|
||||
# An open render node with no engine time is software rendering.
|
||||
('render_node', ':98765', False),
|
||||
],
|
||||
)
|
||||
def test_what_a_process_on_the_display_holds(self, card, display, held, on, rendering):
|
||||
proc = _holding(getattr(card, held), on)
|
||||
try:
|
||||
assert display.uses_gpu() is rendering
|
||||
finally:
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
|
||||
@pytest.mark.parametrize('held', ['render_node', 'card_node'])
|
||||
def test_engine_work_on_either_drm_node_of_the_card_is_rendering(
|
||||
self, tmp_path, monkeypatch, held
|
||||
):
|
||||
# On an RX 7900 XTX, a second launch from one process drew through
|
||||
# /dev/dri/card0 and left renderD128 idle.
|
||||
for name in ('renderD200', 'card9', 'other'):
|
||||
(tmp_path / name).touch()
|
||||
gpu = GpuDevice(
|
||||
'0000:6f:00.0', str(tmp_path / 'renderD200'), str(tmp_path / 'card9'),
|
||||
'0x1002', '0x744c', 'amdgpu',
|
||||
)
|
||||
display = GpuDisplay(gpu)
|
||||
display._display = 98765
|
||||
monkeypatch.setattr(gpudisplay, 'drew_on_gpu', lambda fdinfo: True)
|
||||
for path, rendering in ((getattr(gpu, held), True), (str(tmp_path / 'other'), False)):
|
||||
proc = _holding(path, ':98765')
|
||||
try:
|
||||
assert display.uses_gpu() is rendering
|
||||
finally:
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
|
||||
def test_kill_sweeps_every_process_on_the_display(self, card, display):
|
||||
proc = _holding(card.render_node, ':98765')
|
||||
try:
|
||||
display.kill()
|
||||
assert proc.wait(timeout=5) is not None
|
||||
finally:
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
|
||||
|
||||
class TestVtGuard:
|
||||
@pytest.fixture
|
||||
def vt(self, monkeypatch, tmp_path):
|
||||
active = tmp_path / 'active'
|
||||
active.write_text('tty2\n')
|
||||
monkeypatch.setattr(gpudisplay, 'ACTIVE_VT', active)
|
||||
return active
|
||||
|
||||
def test_a_vt_switch_is_undone_and_fails_the_launch(self, vt, monkeypatch):
|
||||
commands = []
|
||||
|
||||
def run(command, **kwargs):
|
||||
commands.append(command)
|
||||
if command[:2] == ['loginctl', 'list-sessions']:
|
||||
return subprocess.CompletedProcess(command, 0, '3 1000 user seat0 tty2\n', '')
|
||||
if command[:2] == ['loginctl', 'show-session']:
|
||||
return subprocess.CompletedProcess(command, 0, '2\n', '')
|
||||
if command[:2] == ['loginctl', 'activate']:
|
||||
vt.write_text('tty2\n')
|
||||
return subprocess.CompletedProcess(command, 0, '', '')
|
||||
|
||||
monkeypatch.setattr(gpudisplay.subprocess, 'run', run)
|
||||
display = GpuDisplay(NVIDIA)
|
||||
kill = mock.Mock()
|
||||
monkeypatch.setattr(display, 'kill', kill)
|
||||
display._vt = 'tty2'
|
||||
vt.write_text('tty7\n')
|
||||
|
||||
with pytest.raises(GpuVtTakeover, match='switched back'):
|
||||
display._starting()
|
||||
assert ['loginctl', 'activate', '3'] in commands
|
||||
kill.assert_called_once()
|
||||
|
||||
def test_an_unchanged_vt_is_left_alone(self, vt):
|
||||
display = GpuDisplay(NVIDIA)
|
||||
display._vt = 'tty2'
|
||||
display._starting()
|
||||
@@ -45,7 +45,7 @@ def on_host(gpu, os_key="lin"):
|
||||
"""A host whose Firefox renders on `gpu`; records every probe."""
|
||||
probes = []
|
||||
|
||||
def probe(executable_path, headless, env):
|
||||
def probe(executable_path, headless, env, prefs):
|
||||
probes.append((executable_path, headless))
|
||||
return gpu
|
||||
|
||||
@@ -234,13 +234,17 @@ class TestWebGpu:
|
||||
|
||||
|
||||
def test_the_probe_ignores_what_the_launcher_adds_to_the_environment():
|
||||
"""host_identity probes with the launch's env, which carries
|
||||
CAMOU_* and FONTCONFIG_FILE; that must hit launch_options' cached probe."""
|
||||
"""host_identity probes with the launch's env and prefs, which carry
|
||||
CAMOU_*, FONTCONFIG_FILE and prefs that choose no device; that must hit
|
||||
launch_options' cached probe."""
|
||||
from camoufox import host_rendering
|
||||
|
||||
with mock.patch.object(host_rendering, "_host_gpu", return_value=HOST_GPU) as probe:
|
||||
host_gpu("/bin/camoufox", True, {"DISPLAY": ":0"})
|
||||
host_gpu("/bin/camoufox", True, {"DISPLAY": ":0", "CAMOU_CONFIG_1": "{}", "FONTCONFIG_FILE": "/x"})
|
||||
host_gpu("/bin/camoufox", True, {"DISPLAY": ":0"}, {})
|
||||
host_gpu(
|
||||
"/bin/camoufox", True, {"DISPLAY": ":0", "CAMOU_CONFIG_1": "{}", "FONTCONFIG_FILE": "/x"},
|
||||
{"webgl.force-enabled": True},
|
||||
)
|
||||
|
||||
assert probe.call_args_list[0] == probe.call_args_list[1]
|
||||
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
"""headless="virtual" must not strand its Xvfb when the launch fails.
|
||||
|
||||
Everything after VirtualDisplay() -- launch_options(), the Playwright launch --
|
||||
can raise before sync/async_attach_vd wires kill() into close, which used to
|
||||
leave one Xvfb and its /tmp/.X<n> lock per failed launch.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from camoufox import async_api, sync_api, utils
|
||||
|
||||
|
||||
class FakeDisplay:
|
||||
instances = []
|
||||
|
||||
def __init__(self, debug=None):
|
||||
self.killed = False
|
||||
FakeDisplay.instances.append(self)
|
||||
|
||||
def get(self):
|
||||
return ":99"
|
||||
|
||||
def kill(self):
|
||||
self.killed = True
|
||||
|
||||
|
||||
class Boom(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _raise(*args, **kwargs):
|
||||
raise Boom()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def fake_display(monkeypatch):
|
||||
FakeDisplay.instances.clear()
|
||||
# NewBrowser and AsyncNewBrowser start the display through
|
||||
# utils.start_virtual_display.
|
||||
monkeypatch.setattr(utils, "VirtualDisplay", FakeDisplay)
|
||||
|
||||
|
||||
def test_sync_kills_display_when_launch_options_fails(monkeypatch):
|
||||
monkeypatch.setattr(sync_api, "launch_options", _raise)
|
||||
with pytest.raises(Boom):
|
||||
sync_api.NewBrowser(object(), headless="virtual")
|
||||
assert FakeDisplay.instances[0].killed
|
||||
|
||||
|
||||
def test_async_kills_display_when_launch_options_fails(monkeypatch):
|
||||
monkeypatch.setattr(async_api, "launch_options", _raise)
|
||||
with pytest.raises(Boom):
|
||||
asyncio.run(async_api.AsyncNewBrowser(object(), headless="virtual"))
|
||||
assert FakeDisplay.instances[0].killed
|
||||
|
||||
|
||||
def test_sync_kills_display_when_browser_launch_fails(monkeypatch):
|
||||
monkeypatch.setattr(sync_api, "launch_options", lambda **kw: {})
|
||||
monkeypatch.setattr(sync_api, "host_identity", lambda *a: None)
|
||||
|
||||
class Firefox:
|
||||
launch = staticmethod(_raise)
|
||||
|
||||
class Playwright:
|
||||
firefox = Firefox()
|
||||
|
||||
with pytest.raises(Boom):
|
||||
sync_api.NewBrowser(Playwright(), headless="virtual")
|
||||
assert FakeDisplay.instances[0].killed
|
||||
@@ -107,3 +107,29 @@ def test_no_lookup_when_both_values_are_given(api):
|
||||
with mock.patch.object(ip.requests, "get") as get:
|
||||
_new_context(api, {"server": "1.2.3.4:8080"}, webrtc_ip="198.51.100.1", timezone_id="UTC")
|
||||
get.assert_not_called()
|
||||
|
||||
|
||||
def test_lookups_through_a_proxy_get_longer_than_direct_ones(monkeypatch):
|
||||
# A proxied probe adds a hop and a TLS handshake; 5 s failed whole launches
|
||||
# behind proxies that route the long way round.
|
||||
timeouts = []
|
||||
|
||||
def get(url, **kwargs):
|
||||
timeouts.append((url, kwargs["timeout"], bool(kwargs.get("proxies"))))
|
||||
response = mock.MagicMock()
|
||||
response.text = "203.0.113.7"
|
||||
response.json.return_value = EXIT
|
||||
return response
|
||||
|
||||
monkeypatch.setattr(ip.requests, "get", get)
|
||||
ip.public_ip.cache_clear()
|
||||
ip.public_ip()
|
||||
ip.public_ip("http://u:p@proxy.example.com:8080")
|
||||
ip.proxy_exit_geo("http://u:p@proxy.example.com:8080")
|
||||
ip.public_ip.cache_clear()
|
||||
assert [(t, proxied) for _, t, proxied in timeouts] == [
|
||||
(ip.DIRECT_IP_TIMEOUT, False),
|
||||
(ip.PROXIED_IP_TIMEOUT, True),
|
||||
(ip.PROXIED_IP_TIMEOUT, True),
|
||||
]
|
||||
assert ip.PROXIED_IP_TIMEOUT > ip.DIRECT_IP_TIMEOUT
|
||||
|
||||
@@ -25,7 +25,7 @@ from pathlib import Path
|
||||
import orjson
|
||||
import pytest
|
||||
|
||||
from camoufox import server
|
||||
from camoufox import server, utils
|
||||
from camoufox.server import get_nodejs
|
||||
|
||||
# Anything on the driver's private lib/ path is fair game for Playwright to
|
||||
@@ -303,7 +303,7 @@ class _ExitedProcess:
|
||||
@pytest.fixture
|
||||
def fake_virtual_display(monkeypatch):
|
||||
_FakeVirtualDisplay.instances = []
|
||||
monkeypatch.setattr(server, "VirtualDisplay", _FakeVirtualDisplay, raising=False)
|
||||
monkeypatch.setattr(utils, "VirtualDisplay", _FakeVirtualDisplay)
|
||||
monkeypatch.setattr(server, "get_nodejs", lambda: "/node")
|
||||
return _FakeVirtualDisplay.instances
|
||||
|
||||
@@ -347,3 +347,49 @@ def test_launch_server_kills_virtual_display_when_launch_fails(
|
||||
|
||||
assert len(fake_virtual_display) == 1
|
||||
assert fake_virtual_display[0].killed
|
||||
|
||||
|
||||
def test_a_served_gpu_display_is_proved_on_the_printed_endpoint(monkeypatch, capsys):
|
||||
# The server's browser runs in Node, so the GPU check connects to the
|
||||
# endpoint launchServer.js prints, and the output still reaches the user.
|
||||
import io
|
||||
|
||||
import playwright.sync_api
|
||||
|
||||
connected = []
|
||||
|
||||
class FakePlaywright:
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *args):
|
||||
pass
|
||||
|
||||
@property
|
||||
def firefox(self):
|
||||
return self
|
||||
|
||||
def connect(self, endpoint):
|
||||
connected.append(endpoint)
|
||||
return self
|
||||
|
||||
def close(self):
|
||||
connected.append("closed")
|
||||
|
||||
proved = []
|
||||
monkeypatch.setattr(playwright.sync_api, "sync_playwright", FakePlaywright)
|
||||
monkeypatch.setattr(server, "prove_gpu", lambda browser, display: proved.append(display))
|
||||
|
||||
class Process:
|
||||
stdout = io.StringIO(
|
||||
"Launching server...\n"
|
||||
"Websocket endpoint:\x1b[93m ws://localhost:4242/abc \x1b[0m\n"
|
||||
"after\n"
|
||||
)
|
||||
|
||||
display = object()
|
||||
server._prove_served_gpu(Process(), display)
|
||||
|
||||
assert connected == ["ws://localhost:4242/abc", "closed"]
|
||||
assert proved == [display]
|
||||
assert "ws://localhost:4242/abc" in capsys.readouterr().out
|
||||
|
||||
@@ -42,7 +42,9 @@ def test_converter_reproduces_the_recorded_device():
|
||||
value, less the UNMASKED_* strings, which the browser takes from
|
||||
webGl:vendor/renderer rather than the table. Keys since marked removed
|
||||
(contextAttributes) are no longer sent. WebGL1's line width range is
|
||||
WebGL2's since Firefox 156 (test_webgl1_line_width_range_is_webgl2s)."""
|
||||
WebGL2's since Firefox 156 (test_webgl1_line_width_range_is_webgl2s), and
|
||||
Firefox 158 adds EXT_polygon_offset_clamp on this desktop GL device
|
||||
(test_polygon_offset_clamp_follows_the_backend)."""
|
||||
removed = {
|
||||
entry["property"]
|
||||
for entry in json.loads(PROPERTIES.read_text())
|
||||
@@ -51,6 +53,8 @@ def test_converter_reproduces_the_recorded_device():
|
||||
old = json.loads((Path(__file__).parent / "data" / "webgl-gtx980-linux.json").read_text())
|
||||
old = {key: value for key, value in old.items() if key not in removed}
|
||||
old["webGl:parameters"]["33902"] = old["webGl2:parameters"]["33902"]
|
||||
for key in ("webGl:supportedExtensions", "webGl2:supportedExtensions"):
|
||||
old[key] = sorted([*old[key], "EXT_polygon_offset_clamp"], key=str.casefold)
|
||||
new = webgl_for_gpu("lin", *_GTX_980_LINUX, seed=0)
|
||||
|
||||
assert new.keys() == old.keys()
|
||||
@@ -196,7 +200,9 @@ def test_draft_extensions_filtered_on_every_os():
|
||||
webgl2 = {**recorded, "supportedExtensions": ["EXT_texture_norm16", "WEBGL_clip_cull_distance", "OVR_multiview2"]}
|
||||
for target_os in OSES:
|
||||
config = webgl.to_config(recorded, webgl2, target_os)
|
||||
backend = ["EXT_depth_clamp"] if target_os == "win" else []
|
||||
# A device with WebGL2 is D3D11 feature level 10_0 or above, where ANGLE
|
||||
# also clamps polygon offsets; the recording carries no desktop GL mark.
|
||||
backend = ["EXT_depth_clamp", "EXT_polygon_offset_clamp"] if target_os == "win" else []
|
||||
assert config["webGl:supportedExtensions"] == ["ANGLE_instanced_arrays", *backend]
|
||||
assert config["webGl2:supportedExtensions"] == (backend + ["OVR_multiview2"] if target_os == "win" else [])
|
||||
|
||||
@@ -208,6 +214,21 @@ def test_windows_exposes_ext_depth_clamp_on_every_device():
|
||||
assert all("EXT_depth_clamp" in exts for exts in _extensions("win", key) if exts), key
|
||||
|
||||
|
||||
def test_polygon_offset_clamp_follows_the_backend():
|
||||
"""Firefox 158 exposes EXT_polygon_offset_clamp on ANGLE's D3D11 backend
|
||||
from feature level 10_0 (every device with WebGL2) and on Linux desktop GL
|
||||
(the recordings with EXT_depth_clamp), never on macOS's GL 4.1; fpgen's
|
||||
Firefox 146 records predate it."""
|
||||
for target_os in OSES:
|
||||
for s in range(100):
|
||||
config = sample_webgl_for_screen(target_os, 1920, 1080, s)
|
||||
desktop_gl = "EXT_depth_clamp" in config["webGl:supportedExtensions"]
|
||||
want = {"win": config["webGl2Enabled"], "lin": desktop_gl, "mac": False}[target_os]
|
||||
for key in ("webGl:supportedExtensions", "webGl2:supportedExtensions"):
|
||||
if key in config:
|
||||
assert ("EXT_polygon_offset_clamp" in config[key]) == want, (target_os, s, key)
|
||||
|
||||
|
||||
def test_extensions_are_in_firefoxs_order():
|
||||
"""getSupportedExtensions lists in WebGLExtensionID order: alphabetical,
|
||||
ignoring case. An extension the backend adds takes its place there."""
|
||||
|
||||
Reference in New Issue
Block a user