test(mobile): make the bridged recording-corpus replay a CI gate and pin the C1 page closure (OTA phase C, C1.6) (#21533)

* test(mobile): the bridged corpus replay is a gate, not an opt-in flag (OTA phase C, C1.6)

`rpc-recording-through-bridge.test.ts` replays all 787 goldens through the page
bridge and pins how far that bridge is from byte-identical. It only ran when
`RPC_FOUNDATION_BRIDGE=1` was set, which CI did in a step of its own. A gate
whose CI job has to remember to set a variable is opt-in, and a branch that
widened the divergence with that step edited away would have been measured by
nobody.

The suite now runs by default and `RPC_FOUNDATION_BRIDGE=0` skips it, for a
local run that does not want the three minutes. `BRIDGED_PARITY_OFF` names the
one value that skips, so an unset or mistyped variable still runs the gate.

The dedicated CI step goes with it: `pnpm test` collects the file already, so
keeping the step would have run the same 788 tests twice in one job. Vitest
gives the file a worker beside the rest of the suite, so the marginal wall time
is a fraction of the ~3 min it takes alone.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): pin the bridged verdict of every C1-page-closure golden by id (OTA phase C, C1.6)

The bridged replay certifies the corpus in counts over 787 goldens, and a count
is the wrong instrument for the claim C1 needs. C1 moves one domain to the web:
`app/h/_layout.tsx`, `app/h/[hostId]/index.tsx` and their import closure. A
golden recorded at a call site in there that stopped replaying byte-identically
is paid for by any of the other 684 that started, and every existing assertion
still passes.

`c1-page-closure.ts` names the 22 families and 103 goldens of that closure and
pins each one to the verdict it gives: 54 byte-identical, 49 in four of the five
classes the suite already excludes, all of them recorder observation artifacts
whose wire bytes C0.5 and C0.8 proved identical. Membership is checked per
family, not against the flat id list, so a golden newly derived into a family
this domain owns arrives as a finding instead of being missed for never having
been pinned. A closure golden may only be excluded into a class that carries a
reason in `BRIDGED_PARITY_EXCLUSIONS`.

Two full-corpus assertions go with it. `identical` was a floor; it is now the
exact 787 minus the excluded classes. And the run's own size is pinned to the
corpus: every class is an upper bound, so without that a corpus that lost
goldens outside the identical set satisfied all of them.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): move a closure golden to a verdict it is not already pinned to (OTA phase C, C1.6)

The drift test took whichever golden the pin lists first and moved it to
`params-undefined`. Nothing said that golden was not pinned to that class
already, and the day one is, the test asserts a change it never made. Pick the
first golden whose verdict is something else, and assert the pair of verdicts
the line reports rather than just the new one.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): pin `identical` to its baseline, not to the run's own exclusions (OTA phase C, C1.6)

`identical: pinned - excludedCount` took `excludedCount` from the run, and the sum
assertion above it already forces `total(counts) === excludedCount`, so the pair
reduced to `corpus === pinned`: only the size of the corpus was pinned. With every
per-class check an upper bound, a golden moving out of an excluded class into
`identical` satisfied all of them.

`bridgedParityTallyDrift` pins every number to `BRIDGED_PARITY_BASELINE` exactly,
`identical` among them, and its own tests encode the mutation: one
`result-absent-settlement` golden reported `identical` is two drift lines, where
before it was a green run.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* docs(mobile): the observation class counts 3, not 7 (OTA phase C, C1.6)

The number in the prose predates the baseline it describes; `result-absent-observation`
is 3 in `BRIDGED_PARITY_BASELINE` and in the run, and the line above it already says
341 / 3 / 6 / 33 / 8.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* docs(mobile): name the checks that are still there, not the bound that is gone (OTA phase C, C1.6)

Three comments described the per-class upper bound the tally replaced. What the
exact pin is exact *against* is now `unclassified`, the exclusion sum and the
membership pins, so say those.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): drop the closure-exclusion check the cheap suite already makes (OTA phase C, C1.6)

`c1PageClosureExclusions()` reads `C1_PAGE_CLOSURE` and nothing the run produced, so
asserting on it inside the three-minute replay bought nothing the cheap suite next
door does not already buy. `c1-page-closure.test.ts` makes the same assertion with a
presence precondition on top, which is the stronger of the two.

The comment beside the tally counted two checks above it; there are three, and a
golden that moved out of an excluded class into `identical` is invisible to all of
them: `result-absent-settlement` is past the nameable cap, so membership drift never
looks at it, and lowering it lowers the exclusion sum with it.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb
This commit is contained in:
Jinwoo Hong
2026-09-18 19:56:27 -04:00
committed by GitHub
parent 09073086a8
commit f2fd18c820
6 changed files with 532 additions and 47 deletions
+6 -13
View File
@@ -103,22 +103,15 @@ jobs:
- name: Typecheck tests (ratchet)
run: pnpm run check:tests-typecheck
# This includes the bridged replay of the whole recording corpus, which used to be a second
# step of its own behind RPC_FOUNDATION_BRIDGE=1. A gate nobody can forget to set is the point:
# it fails when a divergence class grows, when a divergence lands in no class at all, or when
# one of the 103 goldens inside the C1 page closure changes the verdict it is pinned to. It is
# ~3 min of test time on its own, and Vitest runs it on a worker beside the rest of the suite,
# so folding it in costs a fraction of that in wall time and one step less to skip.
- name: Test
run: pnpm test
# Why a second run of the same corpus: `pnpm test` leaves this suite off, because it replays
# every golden through the page bridge and the per-class counts it pins are the only thing
# that says how far that bridge is from byte-identical. It fails when a class grows or when a
# divergence lands in no class at all, so a change that widens the gap cannot land quietly.
# ~2.5 min locally, because a golden that diverges is replayed a second time with `_meta`
# supplied and that counterfactual is what separates the reader's share of the gap from the
# rest. Ungated on purpose: unlike the pin guard's reproduce, this verdict moves on any change
# to the bridge, which no path filter on the corpus would catch.
- name: Replay the recording corpus through the page bridge
env:
RPC_FOUNDATION_BRIDGE: '1'
run: pnpm exec vitest run src/test-support/rpc-recording/rpc-recording-through-bridge.test.ts
- name: Test iOS release version resolution
run: ruby fastlane/ios_release_version_test.rb