Files
Jake WriterandClaude Fable 5.2 340cf16170 CI: build pull requests with the release's command, and preflight its other targets (#1055)
Four prerelease cuts failed on commits whose tests had passed (runs
41280578561, 43352392915, 43395652803, 43631788944), each in a step only
release.yml ran: midl.exe under Wine (twice), verify-fonts.py, and
make set-target. tests.yml built with `make build`; release.yml built with
multibuild.py after installing its own packages. A step only a release runs
is run for the first time by a release.

- make set-target (scripts/patch.py --mozconfig-only) writes the mozconfig
  and nothing else. It used to reset the tree and copy the additions over
  it, which in a tree without a git repository undid any patch hunk to an
  addition while leaving _READY in place, so the build went on with the
  reverted files. The mozconfig it writes is byte for byte the same.
- ci.run_build runs multibuild.py for one target. tests.yml's build job runs
  it for linux x86_64 and release.yml's for every target, so the pull
  request retargets, builds and packages exactly as a release does.
- multibuild.py --preflight (make preflight, scripts/preflight.py) checks a
  target without building it: configure, the pre-export, export and
  pre-compile tiers, then the object of every source file a patch or an
  addition touches. tests.yml preflights the five targets only a release
  builds, beside the build; the gate requires them. A target that passed is
  not run again for the same compiled inputs, except by the scheduled run.
- .github/actions/build-host frees the disk, installs the packages and adds
  the swap for all three jobs, so no build depends on a package only one of
  them installs.
- browser/multibuild.py is a browser input: changing it rebuilds.

The preflight does not link or package, so a link error on a cross-compiled
target is still first seen by a release.

Tests: ci/tests/test_set_target.py fails on main ('as added' != 'as patched');
test_preflight.py covers the source-to-object mapping and that an object left
by an earlier run is compiled again; test_ci.py holds the preflight matrix to
release.yml's and all three jobs to build-host; the tribal rule
one-build-entry-point holds all three to ci.run_build.

Run locally on Firefox 159 with CI's tree (make setup-minimal, make dir):
- All five preflights pass, in 2 to 6.5 minutes each on 16 cores.
- With the four additions as run 43631788944's tree had them, the linux
  arm64 preflight fails with that run's error,
  SkBitmapProcState_opts.cpp:41:13: use of undeclared identifier
  'sk_cpu_emulates_x86'.
- python3 -m ci.run_build --target linux --arch arm64, the release's
  command for a target that run never linked, builds and packages
  camoufox-159.0-beta.35-lin.arm64.zip in 41 minutes.
No workflow here has run on a GitHub runner yet; this pull request's run is
the first.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-15 06:54:41 +00:00

187 lines
5.3 KiB
Python

"""
Easy build CLI for Camoufox
options:
-h, --help show this help message and exit
--target {linux,windows,macos} [{linux,windows,macos} ...]
Target platforms to build
--arch {x86_64,arm64,i686} [{x86_64,arm64,i686} ...]
Target architectures to build for each platform
--bootstrap Bootstrap the build system
--clean Clean the build directory before starting
--preflight Check that each target builds, without building it
Example:
$ python3 multibuild.py --target linux windows macos --arch x86_64 arm64
Since Camoufox is NOT meant to be used as a daily driver, no installers are provided.
"""
import argparse
import glob
import os
from pathlib import Path
import sys
from dataclasses import dataclass
from typing import List
import shutil
# Constants
AVAILABLE_TARGETS = ["linux", "windows", "macos"]
AVAILABLE_ARCHS = ["x86_64", "arm64", "i686"]
def setup_linux_sysroots():
"""
Set up symlinks required for Linux cross-compilation.
The sysroots may be missing the libsqlite3.so symlink needed for linking.
"""
mozbuild = Path.home() / '.mozbuild'
sysroots = [
('sysroot-aarch64-linux-gnu', 'aarch64-linux-gnu'),
('sysroot-x86_64-linux-gnu', 'x86_64-linux-gnu'),
]
for sysroot_name, lib_arch in sysroots:
sysroot_lib = mozbuild / sysroot_name / 'usr' / 'lib' / lib_arch
sqlite_so = sysroot_lib / 'libsqlite3.so'
sqlite_so_0 = sysroot_lib / 'libsqlite3.so.0'
if sysroot_lib.exists() and sqlite_so_0.exists() and not sqlite_so.exists():
print(f"Creating libsqlite3.so symlink in {sysroot_lib}")
sqlite_so.symlink_to('libsqlite3.so.0')
def run(cmd):
print(f'\n------------\n{cmd}\n------------\n')
if os.system(cmd) != 0:
sys.exit(f"fatal error: command '{cmd}' failed")
@dataclass
class BSYS:
target: str
arch: str
@staticmethod
def bootstrap():
"""Bootstrap the build system"""
run('make bootstrap')
@staticmethod
def generate_assets_car():
"""Generate Assets.car for macOS builds"""
run('make generate-assets-car')
def build(self):
"""Build the Camoufox source code"""
os.environ['BUILD_TARGET'] = f'{self.target},{self.arch}'
run('make build')
def preflight(self):
"""Configure, run the export tiers and compile what the patches touch"""
os.environ['BUILD_TARGET'] = f'{self.target},{self.arch}'
run('make preflight')
def package(self):
"""Package the Camoufox source code"""
run(f'make package-{self.target} arch={self.arch}')
def update_target(self):
"""Change the build target"""
os.environ['BUILD_TARGET'] = f'{self.target},{self.arch}'
run('make set-target')
@property
def assets(self) -> List[str]:
"""Get the list of assets"""
package_pattern = f'camoufox-*-{self.target[:3]}.{self.arch}.zip'
return glob.glob(package_pattern)
@staticmethod
def clean():
"""Clean the Camoufox directory"""
run('make clean')
def run_build(target, arch, preflight=False):
"""
Run the build for the given target and architecture
"""
builder = BSYS(target, arch)
builder.update_target()
if preflight:
builder.preflight()
return
# Run build
builder.build()
# Run package
builder.package()
# Move assets to dist
print('Assets:', ', '.join(builder.assets))
for asset in builder.assets:
shutil.move(asset, f'dist/{asset}')
def main():
parser = argparse.ArgumentParser(description="Easy build CLI for Camoufox")
parser.add_argument(
"--target",
choices=AVAILABLE_TARGETS,
nargs='+',
required=True,
help="Target platform for the build",
)
parser.add_argument(
"--arch",
choices=AVAILABLE_ARCHS,
nargs='+',
required=True,
help="Target architecture for the build",
)
parser.add_argument("--bootstrap", action="store_true", help="Bootstrap the build system")
parser.add_argument(
"--clean", action="store_true", help="Clean the build directory before starting"
)
parser.add_argument(
"--preflight",
action="store_true",
help="Check that each target builds, without building it",
)
args = parser.parse_args()
# Run bootstrap if requested
if args.bootstrap:
BSYS.bootstrap()
# Clean if requested
if args.clean:
BSYS.clean()
# Generate Assets.car if building for macOS
if 'macos' in args.target:
BSYS.generate_assets_car()
run('make setup-macos-sdk')
# Ensure dist directory exists
os.makedirs('dist', exist_ok=True)
# Set up Linux sysroot symlinks if needed
if 'linux' in args.target:
setup_linux_sysroots()
# Run build
for target in args.target:
for arch in args.arch:
if (target, arch) in [
("windows", "arm64"),
("macos", "i686"),
("linux", "i686"),
]:
print(f"Skipping {target} {arch}: Unsupported architecture.")
continue
run_build(target, arch, args.preflight)
if __name__ == "__main__":
main()