Files
AntonandClaude Opus 5.5 e48193ad72 Upgrade to v159 (#986)
* Pin v159.0

Camoufox beta.35 is built on Firefox 159.0 (was 158.0.1), with the PGO profiles Mozilla trained for that release. Linux arm64 is still built with the x86_64 Linux profile in 159's taskcluster/kinds/build.

* Port the Playwright bootstrap patch to Firefox 159

Firefox 159 gives each BrowsingContext field its CanSet policy inside the FIELD macro and removes the per-field CanSet() overloads. PrefersContrastOverride's old CanSet() returned IsTop(), which is {.mTopOnly = true, .mCanSet = CanSet::Unrestricted}, the policy 159 gives its sibling ForcedColorsOverride. The hunk that only shortened two mozilla::dom:: qualifiers is dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Port the SessionStore popup-close guard to Firefox 159

Firefox 159 keys SessionStore's windows by a private #windowIds WeakMap instead of the aWindow.__SSi expando. maybeDontRestoreTabs looks the window up the same way onClose now does, and still returns early for a window SessionStore never registered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Refresh two patches' context for Firefox 159

Both applied with fuzz 2. font-hijacker.patch's DWrite hunk sits where 159 moved the factory into the bundled-fonts branch, and wheel-native-ticks.patch's hunk now sits before 159's new mIsSynthesizedForTests line in SendWheelEvent. Only context lines change. The wheel flag matches what jugglerSendMouseEvent already sets for mouse input (isDOMEventSynthesized), and camoufox.cfg zeroes the user-input delay it would bypass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Drop remote-security-settings.patch; Firefox 159 ships the OneCRL import

Bug 2084311's startup import of the bundled OneCRL list is in Firefox 159's RemoteSecuritySettings.sys.mjs, so the backport's hunk reads as already applied. The certificate-revocations guard passes on 159 without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Record stock Firefox 159.0's content-accessible chrome files

Regenerated with gen-contentaccessible-manifest.py from the stock 159.0 Linux x86_64 release (build 20261022150525). Regenerating from stock 158.0.1 reproduces the previous manifest byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Import jsdebugger from its Firefox 159 location in Juggler

Firefox 159 moved jsdebugger.sys.mjs to moz-src:///devtools/platform/. The old resource://gre/modules/ URL no longer resolves, Runtime.js throws on load in every content process, and Playwright's launch never completes. Every other module Juggler imports still exists in 159's omni.ja.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Grant the Marionette guards system access the way Firefox 159 requires

Firefox 159 removed --remote-allow-system-access; RemoteAgent reads MOZ_REMOTE_ALLOW_SYSTEM_ACCESS=1 instead, and Marionette:SetContext refuses the chrome context without it. helpers.marionette_env() supplies it to every guard that drives Marionette.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Expose EXT_texture_norm16 on macOS and Linux WebGL2, as Firefox 159 does

Bug 1735401 took EXT_texture_norm16 out from behind webgl.enable-draft-extensions. It is exposed on WebGL2 wherever the GL has 16-bit normalized textures, core in desktop GL 3.1: every macOS (CGL GL 4.1) and x86_64 Linux WebGL2 context. Measured on stock 159.0 on M1 and on Linux llvmpipe; 158.0.1 and 146.0 do not list it. stock-gpu-parity failed on macOS on exactly this extension and passes with the change. Windows is unchanged: ANGLE decides per device from its D3D11 format support, which no recording carries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Run make clean's revert from browser/, not the source tree

The cd into the source tree persisted into the next command, so $(MAKE) revert ran inside the Firefox tree, found no such target, and left the patched tree in place. The step-by-step patch upgrade workflow starts with make clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Expose EXT_clip_control and EXT_texture_mirror_clamp_to_edge where Firefox 159 does

Both extensions are new in Firefox 159 and reach WebGL1 and WebGL2 wherever the GL backend has them. Stock 159.0 on Linux llvmpipe (x86_64 and aarch64) lists both where 158.0.1 lists neither, and stock 159.0 on Apple M1 lists neither: CGL's GL 4.1 lacks clip control (GL 4.5) and mirror clamp (GL 4.4). ANGLE's D3D11 backend sets textureMirrorClampToEdgeEXT on every device and clipControlEXT from feature level 10_0, the level WebGL2 needs, so Windows follows the rule EXT_polygon_offset_clamp already uses. Linux keys on the desktop GL mark (EXT_depth_clamp) as that rule does. stock-gpu-parity failed on Linux on exactly these two extensions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 09:22:13 -08:00

139 lines
5.1 KiB
Python

#!/usr/bin/env python3
"""
The PGO profile a build uses: the one Mozilla trained for the stock release of
the same Firefox, so Camoufox's unpatched code is optimized as stock's is.
Mozilla trains one profile per shippable platform (taskcluster kind
generate-profile) and builds the release with it. data/pgo-profiles.json pins
each one by Taskcluster task and sha256 for the Firefox in upstream.sh.
Run:
python3 scripts/pgo.py pin # after a Firefox version bump
python3 scripts/pgo.py fetch linux x86_64
"""
import hashlib
import json
import os
import sys
import tarfile
import urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
PIN = os.path.join(HERE, 'data', 'pgo-profiles.json')
TASKCLUSTER = 'https://firefox-ci-tc.services.mozilla.com/api'
ARTIFACT = 'public/build/profdata.tar.xz'
MEMBERS = ('merged.profdata', 'en-US.log')
# Mozilla's profile platform for each Camoufox target, as `use-pgo` assigns
# them in Firefox 159's taskcluster/kinds/build: Linux arm64 is built with the
# x86_64 Linux profile.
PROFILE_FOR = {
('linux', 'x86_64'): 'linux64',
('linux', 'arm64'): 'linux64',
('windows', 'x86_64'): 'win64',
('windows', 'i686'): 'win32',
('macos', 'x86_64'): 'macosx64',
('macos', 'arm64'): 'macosx64-aarch64',
}
def digest(path):
h = hashlib.sha256()
with open(path, 'rb') as fh:
for block in iter(lambda: fh.read(1 << 20), b''):
h.update(block)
return h.hexdigest()
def load_pin(version):
with open(PIN, encoding='utf-8') as fh:
pin = json.load(fh)
if pin['firefox'] != version:
sys.exit(f'{PIN} pins the profiles of Firefox {pin["firefox"]}, not {version}. '
f'Run: python3 scripts/pgo.py pin')
return pin
def cache_dir():
return os.path.join(os.environ.get('MOZBUILD_STATE_PATH', os.path.expanduser('~/.mozbuild')),
'camoufox-pgo')
def fetch(target, arch, version):
"""Directory holding merged.profdata and en-US.log for this target."""
entry = load_pin(version)['profiles'][PROFILE_FOR[(target, arch)]]
dest = os.path.join(cache_dir(), entry['sha256'])
if all(os.path.isfile(os.path.join(dest, m)) for m in MEMBERS):
return dest
os.makedirs(cache_dir(), exist_ok=True)
archive = dest + '.tar.xz'
if not os.path.isfile(archive) or digest(archive) != entry['sha256']:
url = f'{TASKCLUSTER}/queue/v1/task/{entry["task"]}/artifacts/{ARTIFACT}'
print(f'fetching {url}', file=sys.stderr)
with urllib.request.urlopen(url, timeout=300) as resp, open(archive + '.part', 'wb') as out:
while block := resp.read(1 << 20):
out.write(block)
os.replace(archive + '.part', archive)
if digest(archive) != entry['sha256']:
os.remove(archive)
sys.exit(f'{url}: sha256 is not the pinned {entry["sha256"]}')
tmp = dest + '.part'
os.makedirs(tmp, exist_ok=True)
with tarfile.open(archive, 'r:xz') as tf:
for name in MEMBERS:
member = tf.getmember(name)
if not member.isfile():
sys.exit(f'{archive}: {name} is not a regular file')
with tf.extractfile(member) as src, open(os.path.join(tmp, name), 'wb') as out:
out.write(src.read())
os.replace(tmp, dest)
os.remove(archive)
return dest
def _get_json(url):
with urllib.request.urlopen(url, timeout=60) as resp:
return json.load(resp)
def release_commit(version):
"""The commit Mozilla released this Firefox from, by its release tag."""
tag = 'FIREFOX_' + version.replace('.', '_') + '_RELEASE'
obj = _get_json(f'https://api.github.com/repos/mozilla-firefox/firefox/git/ref/tags/{tag}')['object']
if obj['type'] == 'tag':
obj = _get_json(obj['url'])['object']
return obj['sha']
def pin(version):
"""Pin the profile of every platform for this Firefox, by the sha256 its
task recorded in its chain of trust."""
commit = release_commit(version)
profiles = {}
for platform in sorted(set(PROFILE_FOR.values())):
index = f'gecko.v2.mozilla-release.revision.{commit}.firefox.{platform}-profile'
task = _get_json(f'{TASKCLUSTER}/index/v1/task/{index}')['taskId']
cot = _get_json(f'{TASKCLUSTER}/queue/v1/task/{task}/artifacts/public/chain-of-trust.json')
profiles[platform] = {'task': task, 'sha256': cot['artifacts'][ARTIFACT]['sha256']}
with open(PIN, 'w', encoding='utf-8') as fh:
json.dump({'firefox': version, 'commit': commit, 'profiles': profiles}, fh, indent=2)
fh.write('\n')
print(f'pinned the profiles of Firefox {version} ({commit[:12]}) in {PIN}')
def upstream_version():
with open(os.path.join(HERE, '..', 'upstream.sh'), encoding='utf-8') as fh:
return next(line.split('=', 1)[1].strip() for line in fh if line.startswith('version='))
if __name__ == '__main__':
if sys.argv[1:2] == ['pin']:
pin(upstream_version())
elif sys.argv[1:2] == ['fetch'] and len(sys.argv) == 4:
print(fetch(sys.argv[2], sys.argv[3], upstream_version()))
else:
sys.exit(__doc__)