mirror of
https://github.com/stablyai/orca.git
synced 2026-09-21 16:02:20 +00:00
test(mobile): make the bridged recording-corpus replay a CI gate and pin the C1 page closure (OTA phase C, C1.6) (#21533)
* test(mobile): the bridged corpus replay is a gate, not an opt-in flag (OTA phase C, C1.6) `rpc-recording-through-bridge.test.ts` replays all 787 goldens through the page bridge and pins how far that bridge is from byte-identical. It only ran when `RPC_FOUNDATION_BRIDGE=1` was set, which CI did in a step of its own. A gate whose CI job has to remember to set a variable is opt-in, and a branch that widened the divergence with that step edited away would have been measured by nobody. The suite now runs by default and `RPC_FOUNDATION_BRIDGE=0` skips it, for a local run that does not want the three minutes. `BRIDGED_PARITY_OFF` names the one value that skips, so an unset or mistyped variable still runs the gate. The dedicated CI step goes with it: `pnpm test` collects the file already, so keeping the step would have run the same 788 tests twice in one job. Vitest gives the file a worker beside the rest of the suite, so the marginal wall time is a fraction of the ~3 min it takes alone. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): pin the bridged verdict of every C1-page-closure golden by id (OTA phase C, C1.6) The bridged replay certifies the corpus in counts over 787 goldens, and a count is the wrong instrument for the claim C1 needs. C1 moves one domain to the web: `app/h/_layout.tsx`, `app/h/[hostId]/index.tsx` and their import closure. A golden recorded at a call site in there that stopped replaying byte-identically is paid for by any of the other 684 that started, and every existing assertion still passes. `c1-page-closure.ts` names the 22 families and 103 goldens of that closure and pins each one to the verdict it gives: 54 byte-identical, 49 in four of the five classes the suite already excludes, all of them recorder observation artifacts whose wire bytes C0.5 and C0.8 proved identical. Membership is checked per family, not against the flat id list, so a golden newly derived into a family this domain owns arrives as a finding instead of being missed for never having been pinned. A closure golden may only be excluded into a class that carries a reason in `BRIDGED_PARITY_EXCLUSIONS`. Two full-corpus assertions go with it. `identical` was a floor; it is now the exact 787 minus the excluded classes. And the run's own size is pinned to the corpus: every class is an upper bound, so without that a corpus that lost goldens outside the identical set satisfied all of them. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): move a closure golden to a verdict it is not already pinned to (OTA phase C, C1.6) The drift test took whichever golden the pin lists first and moved it to `params-undefined`. Nothing said that golden was not pinned to that class already, and the day one is, the test asserts a change it never made. Pick the first golden whose verdict is something else, and assert the pair of verdicts the line reports rather than just the new one. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): pin `identical` to its baseline, not to the run's own exclusions (OTA phase C, C1.6) `identical: pinned - excludedCount` took `excludedCount` from the run, and the sum assertion above it already forces `total(counts) === excludedCount`, so the pair reduced to `corpus === pinned`: only the size of the corpus was pinned. With every per-class check an upper bound, a golden moving out of an excluded class into `identical` satisfied all of them. `bridgedParityTallyDrift` pins every number to `BRIDGED_PARITY_BASELINE` exactly, `identical` among them, and its own tests encode the mutation: one `result-absent-settlement` golden reported `identical` is two drift lines, where before it was a green run. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * docs(mobile): the observation class counts 3, not 7 (OTA phase C, C1.6) The number in the prose predates the baseline it describes; `result-absent-observation` is 3 in `BRIDGED_PARITY_BASELINE` and in the run, and the line above it already says 341 / 3 / 6 / 33 / 8. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * docs(mobile): name the checks that are still there, not the bound that is gone (OTA phase C, C1.6) Three comments described the per-class upper bound the tally replaced. What the exact pin is exact *against* is now `unclassified`, the exclusion sum and the membership pins, so say those. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): drop the closure-exclusion check the cheap suite already makes (OTA phase C, C1.6) `c1PageClosureExclusions()` reads `C1_PAGE_CLOSURE` and nothing the run produced, so asserting on it inside the three-minute replay bought nothing the cheap suite next door does not already buy. `c1-page-closure.test.ts` makes the same assertion with a presence precondition on top, which is the stronger of the two. The comment beside the tally counted two checks above it; there are three, and a golden that moved out of an excluded class into `identical` is invisible to all of them: `result-absent-settlement` is past the nameable cap, so membership drift never looks at it, and lowering it lowers the exclusion sum with it. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb
This commit is contained in:
@@ -103,22 +103,15 @@ jobs:
|
||||
- name: Typecheck tests (ratchet)
|
||||
run: pnpm run check:tests-typecheck
|
||||
|
||||
# This includes the bridged replay of the whole recording corpus, which used to be a second
|
||||
# step of its own behind RPC_FOUNDATION_BRIDGE=1. A gate nobody can forget to set is the point:
|
||||
# it fails when a divergence class grows, when a divergence lands in no class at all, or when
|
||||
# one of the 103 goldens inside the C1 page closure changes the verdict it is pinned to. It is
|
||||
# ~3 min of test time on its own, and Vitest runs it on a worker beside the rest of the suite,
|
||||
# so folding it in costs a fraction of that in wall time and one step less to skip.
|
||||
- name: Test
|
||||
run: pnpm test
|
||||
|
||||
# Why a second run of the same corpus: `pnpm test` leaves this suite off, because it replays
|
||||
# every golden through the page bridge and the per-class counts it pins are the only thing
|
||||
# that says how far that bridge is from byte-identical. It fails when a class grows or when a
|
||||
# divergence lands in no class at all, so a change that widens the gap cannot land quietly.
|
||||
# ~2.5 min locally, because a golden that diverges is replayed a second time with `_meta`
|
||||
# supplied and that counterfactual is what separates the reader's share of the gap from the
|
||||
# rest. Ungated on purpose: unlike the pin guard's reproduce, this verdict moves on any change
|
||||
# to the bridge, which no path filter on the corpus would catch.
|
||||
- name: Replay the recording corpus through the page bridge
|
||||
env:
|
||||
RPC_FOUNDATION_BRIDGE: '1'
|
||||
run: pnpm exec vitest run src/test-support/rpc-recording/rpc-recording-through-bridge.test.ts
|
||||
|
||||
- name: Test iOS release version resolution
|
||||
run: ruby fastlane/ios_release_version_test.rb
|
||||
|
||||
|
||||
Reference in New Issue
Block a user