3 Commits
Author SHA1 Message Date
Jake WriterandClaude Fable 5.2 340cf16170 CI: build pull requests with the release's command, and preflight its other targets (#1055)
Four prerelease cuts failed on commits whose tests had passed (runs
41280578561, 43352392915, 43395652803, 43631788944), each in a step only
release.yml ran: midl.exe under Wine (twice), verify-fonts.py, and
make set-target. tests.yml built with `make build`; release.yml built with
multibuild.py after installing its own packages. A step only a release runs
is run for the first time by a release.

- make set-target (scripts/patch.py --mozconfig-only) writes the mozconfig
  and nothing else. It used to reset the tree and copy the additions over
  it, which in a tree without a git repository undid any patch hunk to an
  addition while leaving _READY in place, so the build went on with the
  reverted files. The mozconfig it writes is byte for byte the same.
- ci.run_build runs multibuild.py for one target. tests.yml's build job runs
  it for linux x86_64 and release.yml's for every target, so the pull
  request retargets, builds and packages exactly as a release does.
- multibuild.py --preflight (make preflight, scripts/preflight.py) checks a
  target without building it: configure, the pre-export, export and
  pre-compile tiers, then the object of every source file a patch or an
  addition touches. tests.yml preflights the five targets only a release
  builds, beside the build; the gate requires them. A target that passed is
  not run again for the same compiled inputs, except by the scheduled run.
- .github/actions/build-host frees the disk, installs the packages and adds
  the swap for all three jobs, so no build depends on a package only one of
  them installs.
- browser/multibuild.py is a browser input: changing it rebuilds.

The preflight does not link or package, so a link error on a cross-compiled
target is still first seen by a release.

Tests: ci/tests/test_set_target.py fails on main ('as added' != 'as patched');
test_preflight.py covers the source-to-object mapping and that an object left
by an earlier run is compiled again; test_ci.py holds the preflight matrix to
release.yml's and all three jobs to build-host; the tribal rule
one-build-entry-point holds all three to ci.run_build.

Run locally on Firefox 159 with CI's tree (make setup-minimal, make dir):
- All five preflights pass, in 2 to 6.5 minutes each on 16 cores.
- With the four additions as run 43631788944's tree had them, the linux
  arm64 preflight fails with that run's error,
  SkBitmapProcState_opts.cpp:41:13: use of undeclared identifier
  'sk_cpu_emulates_x86'.
- python3 -m ci.run_build --target linux --arch arm64, the release's
  command for a target that run never linked, builds and packages
  camoufox-159.0-beta.35-lin.arm64.zip in 41 minutes.
No workflow here has run on a GitHub runner yet; this pull request's run is
the first.

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-15 06:54:41 +00:00
6e9a1de13b Build with cross-language LTO and PGO, as stock Firefox does (#886)
* Build with cross-language LTO and PGO, as stock Firefox does

Without them Gecko's C++ ran 1.3-1.8x stock while JIT-compiled script
did not, and a page can time that ratio (JSON.parse over a regexp
replace read ~3.3 here, ~2.2 in stock). Stock's recipe is two stages:
an instrumented build trains on Mozilla's own pages, then the release
links with --enable-lto=cross and --enable-profile-use=cross.

The "PGO profile" workflow is stage one: it publishes the merged
profile as a release asset, pinned by sha256 in
browser/assets/optimized-build.json. ci/pgo.py exports the switches
base.mozconfig reads; a local build without them is unchanged. The CI
build job pins Rust, checks the pairing, and builds LTO-only with a
warning while no profile is pinned for the tree's Firefox. None is
pinned for 156.0.1 yet.

Rust is pinned to 1.95.0. Cross-language LTO hands Rust bitcode to lld,
so rustc's LLVM may not be newer than the bootstrapped clang's. Firefox
156 bootstraps clang 22 (linux64-clang in toolchain/clang.yml) and
Mozilla builds it with Rust 1.95.0 (linux64-rust in toolchain/rust.yml),
whose LLVM is 22. 1.90.0, the pin for 152, is still 156's minimum but
no longer what stock compiles with.

ccache is restored and saved in separate steps, so a build that fails
late keeps what it compiled, under opt- keys that include the profile.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Link every build with cross-language ThinLTO, from the one mozconfig

LTO had entered only the tests.yml build, through an environment
variable, so a release would have been tested on an LTO Linux binary and
shipped Linux, Windows and macOS binaries built without it. base.mozconfig
now has --enable-lto=cross and --enable-rust-simd unconditionally, and
every build copies it: the tests.yml build, multibuild.py and the
release matrix.

Firefox 158's lto-pgo.configure makes --enable-lto=cross full LTO only
when PGO and MOZ_AUTOMATION are on and the target is x86_64 Linux or
Windows, so without a profile it is ThinLTO across C++ and Rust on every
target. That is what stock links outside those two, and PGO is the
remaining difference. ./mach configure on 158 with the generated mozconfig
gives MOZ_LTO_RUST_CROSS=thin, -Clinker-plugin-lto and an lld linker for
linux x86_64/arm64, windows x86_64/i686 (clang-cl, lld-link) and macOS
x86_64/arm64.

lld cannot read Rust bitcode from an LLVM newer than its own, and the
runner's stable Rust can be. scripts/patch.py now installs the Rust release
taskcluster/kinds/toolchain/rust.yml aliases as linux64-rust (1.95.0,
LLVM 22, for 158's clang 22) and writes RUSTUP_TOOLCHAIN into the
mozconfig, so the pin follows each Firefox bump by itself. The build jobs
no longer install a system LLVM 18 from apt.llvm.org: mach bootstrap
already fetches the clang and lld the tree builds with.

The PGO training workflow, ci/pgo.py, the profile pin and the separate
ccache keys are removed; PGO needs a profile to pin first. The commented
out mozconfig lines go too, and linux.mozconfig, which held only one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Document the LTO build: what it costs and where its Rust comes from

AGENTS.md and README.md said an incremental build takes about five
minutes; with LTO every build, incremental ones included, relinks
libxul across C++ and Rust, and ccache cannot cache that link. ci/README.md
lists it under cost control. On CI the libxul ThinLTO link took about
nine minutes, and the cold build step 57 minutes against 54-99 for
recent cold non-LTO builds, since rustc no longer runs its own fat LTO
over gkrust.

mozilla_rust_version's docstring now says what the pairing is: Rust's
LLVM has the bootstrapped clang's major version (1.95.0 has LLVM 22.1.2,
158's clang is 22.1.8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Build with stock's PGO profile, and full LTO where stock uses it

Stock builds every desktop release with a profile Mozilla trains on that
release (taskcluster kind generate-profile), and with full cross-language LTO
on x86_64 Linux and Windows. LTO alone left the remaining difference from stock
in how Gecko's C++ is optimized, which a page can time against script.

scripts/pgo.py fetches, for each target, the profile Mozilla trained for the
stock release of the same Firefox: Taskcluster indexes the generate-profile
task by the release commit, and the task's chain of trust records the
archive's sha256. data/pgo-profiles.json pins all five (Linux arm64 uses the
x86_64 Linux profile, as stock's does). `make dir` fetches and verifies the
profile, and base.mozconfig builds with it, with full LTO on x86_64 Linux and
Windows and ThinLTO elsewhere, as stock's configure picks under MOZ_AUTOMATION.
A function a patch changes loses its profile data; configure makes that a
warning. A pin for another Firefox fails `make dir` and says to run
`scripts/pgo.py pin`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jake Writer <jake_writer@jobharvest.com>
2026-10-31 06:20:35 +00:00
Jake Writer 658806eb23 chore: major code cleanup/refactor 2026-10-06 19:10:22 -06:00