mirror of
https://github.com/stablyai/orca.git
synced 2026-10-01 08:01:56 +00:00
Brennan Benson
e594cb06af
* test(mobile): add rpc:diff to decode what a recording change moved
The RPC recording goldens are content-addressed JSON, so their raw git diff is
pool hashes. `pnpm --dir mobile rpc:diff [<base>]` decodes both sides and prints,
per golden, the checkpoint, field and JSON path that moved with both values,
grouped across checkpoints, plus added and removed goldens. `--summary <file>`
appends a Markdown report capped for GitHub's step-summary limit.
It reads any pooled format, so it can prove the next commit's format change
moves no recorded value. Checkpoints are matched by occurrence because an id can
repeat within one golden.
This commit adds files under the recorder directory, which moves the header
digest every golden pins; the next commit removes that header.
* test(mobile): record RPC goldens without a pinned commit or input digests
Every golden carried a pinned `baseline` commit plus digests of the recorder,
its mount adapter and its scenario, and the record script refused to run unless
the product tree matched the pin. So every behaviour change repinned to its own
branch commit and rewrote all ~790 files, the squash made that commit
unreachable, and main's pin job stayed red until a hand-made repin pull request
landed (22 of them in 12 days). The digests could only fail when an input moved
and the recording did not, which is exactly the change that carries no
information; every run already re-derives each golden from the current tree and
compares it.
Format 6 keeps the format version, operation, family, named deltas, the value
pool and the recording. Removed: the pin and fence, the three digest modules and
their test, the pin guard and its CI job, and the dead scenario `version` field
(the manifest reader now refuses `baseline` and `version` with a message).
- `pnpm --dir mobile rpc:record [<golden-id>...] [--prune]` records all or some
goldens; orphans are listed, and deleted only with `--prune`. Every derived
test title now starts with its golden id so an id selects it.
- `compareGolden` reports every difference in one failure (identity fields by
name, the checkpoint list, each checkpoint/field/path grouped), keeps the
final byte compare, and ends with the command to re-record that golden.
- `unhandled-recording.test.ts` now drives a detached rejection through
`runRecording` into a checkpoint and the cleanup checkpoint; no golden carries
one, and disconnecting the capture passed every suite before.
- Seam rules that existed only to keep a digest honest are gone; the
mutant-reachability, register-completeness and one-exposure rules stay.
- CI: `Mobile tests on main` runs the whole mobile suite on every merge that
touches mobile/, src/shared/, the root lockfile or the host RPC paths, since
`verify` never runs on main. A new `Mobile RPC Recording Replay` workflow
replays the recordings on pull requests that touch src/shared/ or the root
lockfile without touching mobile/. `verify` writes the `rpc:diff` report to
the job summary.
Proof: `rpc:diff` against the parent reports no recorded behaviour moved; each
golden only loses its ten header lines.
* test(mobile): check every recorded request against the host's params contract
The goldens script the host's replies, so a scenario could record a success
for a request the real host would refuse, and a desktop change that tightens a
params schema moved no golden at all.
`recorded-request-params.test.ts` parses every distinct request the corpus puts
on the wire with the host dispatcher's own `parseRpcRequestParams` and the
schema `rpc-params-catalog.generated.ts` binds to that method. It fails on a
method the host lacks, params it refuses, params sent to a method that takes
none (the dispatcher never reads them), and keys the schema silently strips
unless an inventory entry gives the reason; a stale entry fails too. Each rule
is also shown firing on a made-up request, since the corpus has no instance of
three of them. It imports the desktop dispatcher, so it sits beside the other
Node-side tests outside the RN test program, and the params-contract boundary
now exempts test files, which are never bundled.
It found twelve requests the host would refuse, all from invented fixture
values, not product code, fixed at their source:
- git.branchDiff sent `base-oid`/`head-oid`/`merge-base` where the host needs
full object ids (diff-review and source-control adapters, and the branch
compare replies in the manifest that feed them);
- an iOS push registration without `apnsEnvironment`, which a real iOS token
always carries (`push-token.ts`); the adapter now defaults to `production`;
- `settings.update` given Linear's `assigned` filter as a GitHub preset, which
the product type forbids; the scenario now picks `my-issues`;
- GitLab `projectRef` as a string where the host and the product type take
`{ host, path }` (7 methods, 5 adapters and the manifest).
46 goldens move, and a decoded comparison of every one of them shows no change
other than those substitutions; `rpc:diff` lists them.
* ci(mobile): detect a mobile change without a SIGPIPE-prone grep pipe
Under the runner's pipefail, grep -q exiting on its first match SIGPIPEs git
diff on a long file list, so a large pull request touching mobile/ read as
uncovered and replayed the recordings a second time.
* test(mobile): drop comments that still describe the golden header and digests
Eleven adapters justified an import rule by the header a golden no longer
carries, and that rule's test is gone. The census failure now names the
rpc:record and --prune commands.
* test(mobile): refuse a golden that keeps a key no recording writes
Decoding dropped unknown top-level keys, so an old header left behind by a
hand-resolved merge conflict passed every compare unseen.
* ci(mobile): summarize RPC recording changes after a failed test step too
* test(mobile): stream rpc:record output instead of capturing it
A captured run stayed silent for its whole duration and clipped its tail,
where the failure summary sits, past 8 MB.
* test(ci): let the Ruby-gate contract skip the always-run RPC summary step
fef088d8f4 gave the summary step an `if: ${{ !cancelled() }}`, and this test lists every gated step in `verify` and expects each to be gated on the Ruby scope.
* test(mobile): replay only a golden file that is exactly what rpc:record writes
Replay compared two re-encodings of decoded values, so anything decoding drops (a leftover header
key, a hand edit) sat in the committed file uncompared; a key allow-list covered one case of that.
Replay now passes only if the file text equals the formatted golden for the run, sharing one
formatter with writeGolden, and keeps the field-level report as the failure message. The allow-list
goes; the value-based compareGolden stays for the bridged run, which has no file.
* test(mobile): end a corrupt or hand-edited golden's failure with the re-record command
A hand edit to a pooled value failed in decode with only "Golden value <hash> does not hash to its
pool key": no golden id and no command to fix it. readGolden now prefixes parse and decode failures
with the golden id and ends them with the rpc:record command. The rpc:diff header also said it
always exits 0; it exits non-zero when git or a golden cannot be read, and now says so.
* ci(mobile): run Mobile Checks on every src/shared and root lockfile change
Replaces the replay-only workflow: mobile imports hundreds of shared modules, so a shared edit can
move a golden or break mobile's typecheck, and the full job catches both before merge. A root
lockfile-only change skips the Ruby release checks, which read no root Node dependency.
* test(mobile): list or prune orphaned goldens even when the recording run fails
Orphans come from the manifest, not the run, so a failed or timed-out rpc:record still reports
them; the exit code stays non-zero. README: say what a failed replay reports (first differing
path per field, capped groups) and what rpc:diff compares with and without a base.
* test(mobile): pin the RPC recording goldens to LF so a CRLF checkout still replays
Replay now requires the committed golden text to equal exactly what rpc:record writes, which is
LF. A Windows checkout with core.autocrlf=true converted every golden to CRLF and failed all 790
with "holds the same recording but is not the file rpc:record writes for it".