From 6729f3a0b0c2f66db054f8420b8530ebd56c17ea Mon Sep 17 00:00:00 2001 From: Jinwoo Hong <73622457+Jinwoo-H@users.noreply.github.com> Date: Mon, 7 Sep 2026 13:28:47 -0400 Subject: [PATCH] tools: add a phone-vantage relay connect benchmark (#19251) * tools: add a phone-vantage relay connect benchmark Connect-speed work on the phone had no way to attribute latency to a hop. Timing the mobile app end to end only says "connect is slow", and a synthetic WebSocket probe does not exercise the credential check, the E2EE handshake, or the RPCs the phone blocks on before it publishes connected. This replays the shipped mobile wire sequence from Node against a real desktop over the production relay, so each phase gets its own number. The handshake is a plain-JS port of the mobile client session, which is only trustworthy if it stays byte-identical to what ships; a parity test runs it against the real desktop responder in the normal unit suite so drift in the transcript encoding, key schedule, or frame layout fails there rather than producing a bench that measures a handshake nobody uses. Adds a foreground mode for the resume-after-background question the phone lanes need: connect, go silent past the relay's client silence watchdog, then report whether the retained socket still answers and what the fallback redial costs. The bench writes a resume-credential bundle at runtime. That file carries a live device token for a real paired desktop, so the directory ignores it outright. * tools: make the relay bench name its target and opt in to dialing The supporting scripts carried production defaults: the director origin was hardcoded in both, and the hop-latency probe defaulted to a named production cell. Running either with no arguments sent live traffic at production, and the region probe did it on import, before any argument was read. A default like that is the wrong shape for a bench, because the operator never states what they are measuring against and a stray invocation is indistinguishable from an intended one. Every script now refuses to open a socket unless ORCA_RELAY_BENCH_LIVE=1 is set, and the director comes from --director or ORCA_RELAY_BENCH_DIRECTOR with no fallback. The cell origin is a required argument. Refusals print one line of usage and exit 2, so an accidental run is inert rather than live. The remaining host-id default is an id no desktop owns, which is the point of that probe: it measures the cell hop without reaching a desktop at all. * tools(relay-bench): type refuse() as never so origins are strings * tools(relay-bench): fail closed on hostile input and bounded arguments Review found the harness trusted whatever it was handed: the DevTools port and the director-supplied probe origins went straight into a URL, http origins were accepted, repeat counts came from a bare Number() cast, and the state file kept its existing mode. - Validate the DevTools port as a 1-65535 integer, so '80@attacker.example' cannot move the fetch off loopback via URL userinfo. - Require https for every origin, and refuse loopback, link-local, private, and multicast destinations. Region probe origins and the cell URL the director returns go through the same check, so a compromised director cannot aim the harness at the operator's own network. - Bound --runs, --rounds, runs, --gap, and --hold as whole numbers, so 'Infinity' exits 2 instead of looping forever against the relay. - Report a region as UNREACHABLE when every probe fails, rather than letting Math.min([]) spread into NaN and read as ok. - Bound the director /v1/resolve and /v1/regions fetches and report timeouts. - Return null openMs when the socket never opened, and clear dial, cell, and RPC timers on the first terminal event so Node exits promptly. - Default handle.rpc() to RPC_TIMEOUT_MS, not DIAL_TIMEOUT_MS. - Write the state file through a helper that creates the parent directory, refuses a symlink, and forces 0600 on an existing file; refuse to read one that is readable beyond the operator. - Read the pairing link from stdin or a 0600 file, never argv. - Reject missing and invalid positionals with usage and exit 2. Adds unit tests for the pure guards: argument parsing, bounded integers, port and origin classification, DNS vetting, state-file modes and symlinks, region verdicts, and pairing-link decoding. None opens a socket, and every network path stays gated on ORCA_RELAY_BENCH_LIVE=1. * tools(relay-bench): settle in-flight rpcs and guard an empty region catalog Follow-up to the review fixes. Clearing a pending rpc timer without a resolution swapped a 15 s timeout for an await that never returns, so the teardown paths now settle each waiter with a closed result. A director that answers /v1/regions with no regions now reports that and exits 1 instead of printing an empty round. * tools(relay-bench): attach the origin-vetting doc to the function it describes * fix(tools): resolve a director-named cell through DNS and fail cdp-eval clearly --- tests/tools/relay-bench/.gitignore | 4 + tests/tools/relay-bench/README.md | 209 ++++++ tests/tools/relay-bench/cdp-eval.mjs | 54 ++ .../phone-e2ee-desktop-parity.test.mjs | 69 ++ .../relay-bench/phone-e2ee-v2-session.mjs | 192 ++++++ .../tools/relay-bench/region-probe-replay.mjs | 134 ++++ .../relay-bench/region-probe-replay.test.mjs | 122 ++++ .../relay-bench/relay-bench-invocation.mjs | 294 ++++++++ .../relay-bench-invocation.test.mjs | 244 +++++++ .../relay-bench/relay-bench-state-file.mjs | 76 +++ .../relay-bench-state-file.test.mjs | 100 +++ tests/tools/relay-bench/relay-hop-latency.mjs | 119 ++++ .../relay-bench/relay-phone-connect-bench.mjs | 625 ++++++++++++++++++ .../relay-phone-connect-bench.test.mjs | 59 ++ 14 files changed, 2301 insertions(+) create mode 100644 tests/tools/relay-bench/.gitignore create mode 100644 tests/tools/relay-bench/README.md create mode 100644 tests/tools/relay-bench/cdp-eval.mjs create mode 100644 tests/tools/relay-bench/phone-e2ee-desktop-parity.test.mjs create mode 100644 tests/tools/relay-bench/phone-e2ee-v2-session.mjs create mode 100644 tests/tools/relay-bench/region-probe-replay.mjs create mode 100644 tests/tools/relay-bench/region-probe-replay.test.mjs create mode 100644 tests/tools/relay-bench/relay-bench-invocation.mjs create mode 100644 tests/tools/relay-bench/relay-bench-invocation.test.mjs create mode 100644 tests/tools/relay-bench/relay-bench-state-file.mjs create mode 100644 tests/tools/relay-bench/relay-bench-state-file.test.mjs create mode 100644 tests/tools/relay-bench/relay-hop-latency.mjs create mode 100644 tests/tools/relay-bench/relay-phone-connect-bench.mjs create mode 100644 tests/tools/relay-bench/relay-phone-connect-bench.test.mjs diff --git a/tests/tools/relay-bench/.gitignore b/tests/tools/relay-bench/.gitignore new file mode 100644 index 00000000000..c4959241eb8 --- /dev/null +++ b/tests/tools/relay-bench/.gitignore @@ -0,0 +1,4 @@ +# The bench writes a resume-credential bundle here. It carries a live device token and +# resume token for a real paired desktop; it must never reach the repo. +*.json +state* diff --git a/tests/tools/relay-bench/README.md b/tests/tools/relay-bench/README.md new file mode 100644 index 00000000000..58d2b8e6a65 --- /dev/null +++ b/tests/tools/relay-bench/README.md @@ -0,0 +1,209 @@ +# relay-bench + +Measures how long a phone takes to reach a usable connection with a desktop over the production +relay, without building or instrumenting the mobile app. + +`relay-phone-connect-bench.mjs` replays the shipped mobile wire sequence: the relay auth frame, +the E2EE v2 handshake with the same transcript encoding and HKDF key schedule the app uses, then +the RPCs the phone issues before it publishes `connected`. Because it is the real sequence against +a real desktop, the per-phase numbers attribute latency to a specific hop rather than to "connect". + +The handshake itself lives in `phone-e2ee-v2-session.mjs`, a plain-JS port of the mobile client +session so it runs outside the React Native bundle. +`phone-e2ee-desktop-parity.test.mjs` pins that port to the desktop responder +in `src/main/runtime/rpc/mobile-e2ee-v2-desktop-session.ts`. It runs in the normal unit suite, so +a change to the transcript encoding, key schedule, or frame layout fails there instead of leaving +a bench that quietly measures a handshake nobody ships. The four other `*.test.mjs` files in this +directory cover the invocation guards, the state file, the region verdicts, and pairing-link +decoding, and none of them opens a socket. + +## Security rules + +- The pairing link contains a live invite token and a device token. Treat it as a credential. `pair` + reads it from stdin, or from a file named by `--pairing-url-file`, so it never reaches your shell + history or the process argument list. Passing it as an argument is refused. +- `state.json` holds the resume token and device token for a real paired desktop. Never commit it, + paste it, or attach it to an issue. The `.gitignore` in this directory blocks `*.json` and + `state*`, but do not rely on that alone. +- Revoke the bench device when you are done. See "Cleaning up" below. +- Do not point the bench at a desktop you do not own. + +No script here has a production default. Every one of them refuses to open a socket unless +`ORCA_RELAY_BENCH_LIVE=1` is set, and the two that talk to the director require its origin from +`--director=` or `ORCA_RELAY_BENCH_DIRECTOR`. Without those, they print usage and exit 2. +That keeps an accidental or automated invocation inert instead of live traffic. + +The guards are in `relay-bench-invocation.mjs` and `relay-bench-state-file.mjs`, and +`relay-bench-invocation.test.mjs` / `relay-bench-state-file.test.mjs` pin them: + +| Guard | What it stops | +| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| https-only origins | An `http:` director or cell, where an on-path observer reads bench credentials | +| Public-destination check | A director aiming the harness at your loopback, link-local, or private network, by literal address or by a name that resolves there | +| Bounded integer arguments | `--runs=Infinity` and friends, which loop forever and generate relay traffic | +| `0600` state file | An existing state file staying group- or world-readable, or being a symlink | + +A director you name also _supplies_ URLs: the region catalog's probe origins and the cell URL from +`/v1/resolve`. Those go through the same public-https check as an origin you typed, so a compromised +or spoofed director cannot turn the harness into a probe of your own network. Region entries whose +probe origins are all refused report `REFUSED (no allowed probe origin)` rather than being sampled. +Hostnames are also resolved and checked, which narrows but does not close the DNS rebinding window, +because `fetch()` resolves again. + +State-file handling creates the parent directory before writing, refuses a symlink, and forces +`0600` on an existing file. The first of those matters most: `pair` writes only after the desktop +has already provisioned the resume credential, so a failed write loses it. + +## Requirements + +`ws` and `tweetnacl` resolve from the repo root `node_modules`. Measured against `ws` 8.21.3 and +`tweetnacl` 1.0.3. Run every command from the repo root. + +Syntax check after editing: + +```bash +for f in tests/tools/relay-bench/*.mjs; do node --check "$f"; done +npx vitest run --config config/vitest.config.ts tests/tools/relay-bench +``` + +## Getting a pairing link + +Start a relay-enabled dev app hidden, with remote debugging on: + +```bash +ORCA_BACKGROUND_LAUNCH=1 \ +REMOTE_DEBUGGING_PORT=9222 \ +ORCA_CLOUD_API_URL=https://login.onorca.dev \ +ORCA_CLOUD_CLIENT_ID=orca-desktop \ +ORCA_DEV_USER_DATA_PATH=/tmp/orca-relay-bench-profile \ +ORCA_RELAY_REGION_OVERRIDE=us-central1 \ +pnpm run dev +``` + +`ORCA_DEV_USER_DATA_PATH` keeps the bench pairing out of your real profile. +`ORCA_RELAY_REGION_OVERRIDE` pins the cell region, which is what you want when comparing a change +rather than comparing regions. Both are optional. + +Sign in, then read the pairing offer out of the hidden renderer: + +```bash +node tests/tools/relay-bench/cdp-eval.mjs 9222 'window.api.mobile.getPairingQR({})' +``` + +The `orca://pair?code=...` value in that output is the pairing link. + +## Commands + +```bash +export ORCA_RELAY_BENCH_LIVE=1 +BENCH=tests/tools/relay-bench/relay-phone-connect-bench.mjs + +# One-time: dial the invite, provision a resume credential, save the bundle. The pairing link +# comes in on stdin so it stays out of your shell history and out of `ps`. +pbpaste | node $BENCH pair /tmp/relay-bench/state.json + +# Or from a file you protect yourself, which `pair` requires to be mode 0600: +umask 077 && printf '%s' '' > /tmp/relay-bench/pair.txt +node $BENCH pair /tmp/relay-bench/state.json --pairing-url-file=/tmp/relay-bench/pair.txt +rm /tmp/relay-bench/pair.txt + +# Steady-state foreground reconnect, 10 times, 2 s apart, re-resolving the cell each time. +node $BENCH run /tmp/relay-bench/state.json 10 --resolve --gap=2000 + +# Resume after background: connect, idle 45 s, then probe the retained socket. +node $BENCH foreground /tmp/relay-bench/state.json --hold=45000 + +# Same, but crossing the relay's ~105 s client silence watchdog. +node $BENCH foreground /tmp/relay-bench/state.json --hold=120000 +``` + +On Linux or Windows, replace `pbpaste` with whatever prints the link to stdout, or use +`--pairing-url-file`. Every count and duration is a whole number: `runs` and `--rounds` are 1-1000, +`--gap` and `--hold` are 0-3600000 ms, and anything else exits 2 rather than running unbounded. + +The bench reads the director and cell for a resume dial out of `state.json`, which the pairing +offer supplied, so it takes no `--director`. + +`run` prints one JSON row per iteration plus a `SUMMARY` line with medians. + +`foreground` prints a single JSON row. Flags: + +| Flag | Default | Meaning | +| ---------------- | ------- | -------------------------------------------------------------- | +| `--hold=ms` | `45000` | Idle time with no application traffic after reaching connected | +| `--force-redial` | off | Redial even when the retained socket answered | +| `--resolve` | off | Re-resolve the cell through the director before each dial | + +It adds two fields to the per-phase shape. `retainedAnswerMs` is how long the held-open socket took +to answer `status.get`, or `null` if it could not. `redialMs` is the wall clock for a full resume +redial through the same connected sequence, measured on failure or with `--force-redial`. +`closedDuringHold` carries the close code if the relay dropped the socket while it was idle. + +Note that the WebSocket library answers protocol-level pings automatically, exactly as the phone's +socket does. The silence watchdog counts application traffic, not pongs. + +Two supporting scripts: + +- `relay-hop-latency.mjs --cell= --director= [--host=] [--runs=N]` + measures the infrastructure floor with a throwaway credential: director `/v1/resolve` plus cell + WebSocket open to `relay-hello`. It needs no pairing, because a cell answers a bogus credential + without reaching a desktop. `--host` defaults to an id no desktop owns. `openMs` is `null` when + the socket never opened, and a director that stalls is reported as a resolve timeout rather than + hanging the run loop. +- `region-probe-replay.mjs --director= [--rounds=N]` replays the desktop's region + selection with the same probe, sample count, and spread rule, and prints why each region passed + or failed. A region whose every probe fails reports `UNREACHABLE`, not `ok`. + +Both take the director from `--director` or `ORCA_RELAY_BENCH_DIRECTOR`, and both need +`ORCA_RELAY_BENCH_LIVE=1`: + +```bash +ORCA_RELAY_BENCH_LIVE=1 ORCA_RELAY_BENCH_DIRECTOR= \ + node tests/tools/relay-bench/region-probe-replay.mjs --rounds=3 +``` + +## What each phase means + +| Phase | Measures | +| ------------------- | ------------------------------------------------------------------------------------ | +| `wsOpen` | DNS, TCP, and TLS to the cell, up to the WebSocket upgrade | +| `relayHello` | Cell-side credential validation and the desktop-side attach, ending at `relay-hello` | +| `e2eeReady` | Desktop's `e2ee_ready`, so one relay round trip plus the desktop's key generation | +| `e2eeAuthenticated` | Device-token check on the desktop, ending the handshake | +| `confirm` | `pairing.getEndpoints` with the resume confirm id, which settles the credential | +| `capabilities` | The client capability advisory the phone sends before publishing connected | +| `status.get` | The first RPC the UI gate blocks on | +| `worktree.ps` | The worktree catalog, and the largest payload in the sequence | +| `session.tabs.list` | Per-worktree tab list for the first worktree | +| `terminal.list` | Per-worktree terminal list for the first worktree | + +`totalToConnectedMs` is `e2eeAuthenticated` plus `confirm` plus `capabilities`. +`totalToFirstTerminalListMs` is the whole sequence. + +## Reference numbers + +Measured 2026-09-07 from a US-East vantage, same desktop and identical sequence, differing only in +which cell region served the connection. The vantage matters: these are not what a phone next to +the desktop would see. + +| Cell region | To connected | `relayHello` | `confirm` | +| ----------- | ------------ | ------------ | --------- | +| Asia | 10.5 s | 5.8 s | 3.4 s | +| US | 0.63 s | 0.29 s | 0.14 s | + +## Cleaning up + +Revoke the bench device from the desktop that granted it: + +```bash +node tests/tools/relay-bench/cdp-eval.mjs 9222 'window.api.mobile.revokeDevice({ deviceId: "" })' +``` + +If you do not know the id, list the paired devices first: + +```bash +node tests/tools/relay-bench/cdp-eval.mjs 9222 'window.api.mobile.listDevices()' +``` + +Then delete `state.json`. If you used +`ORCA_DEV_USER_DATA_PATH`, removing that directory drops the pairing with it. diff --git a/tests/tools/relay-bench/cdp-eval.mjs b/tests/tools/relay-bench/cdp-eval.mjs new file mode 100644 index 00000000000..23c96ab5a2c --- /dev/null +++ b/tests/tools/relay-bench/cdp-eval.mjs @@ -0,0 +1,54 @@ +// usage: node cdp-eval.mjs +import WebSocket from 'ws' +import { requirePort } from './relay-bench-invocation.mjs' + +const USAGE = 'node cdp-eval.mjs ' +const RENDERER_ORIGIN = 'http://localhost:5173' +const OPEN_TIMEOUT_MS = 5_000 + +function findRendererPage(list) { + return list.find((p) => p.type === 'page' && p.url.startsWith(RENDERER_ORIGIN)) +} + +function describePages(list) { + return list.length ? list.map((p) => `${p.type} ${p.url}`).join(', ') : 'none' +} +const [rawPort, expr] = process.argv.slice(2) +// Why not interpolate directly: URL parsing reads '80@attacker.example' as userinfo, so the +// fetch would leave the loopback DevTools endpoint for an attacker-named host. +const port = requirePort(rawPort, 'devtools port', USAGE) +const list = await (await fetch(`http://127.0.0.1:${port}/json/list`)).json() +const page = findRendererPage(list) +if (!page) { + console.error( + `no renderer page at ${RENDERER_ORIGIN} on devtools port ${port}; pages: ${describePages(list)}` + ) + process.exit(1) +} +const ws = new WebSocket(page.webSocketDebuggerUrl) +await new Promise((resolve, reject) => { + ws.once('open', resolve) + ws.once('error', reject) + setTimeout( + () => reject(new Error(`devtools socket did not open within ${OPEN_TIMEOUT_MS} ms`)), + OPEN_TIMEOUT_MS + ).unref() +}) +ws.on('error', (err) => { + console.error(`devtools socket error: ${err.message}`) + process.exit(1) +}) +ws.send( + JSON.stringify({ + id: 1, + method: 'Runtime.evaluate', + params: { expression: expr, awaitPromise: true, returnByValue: true } + }) +) +ws.on('message', (m) => { + const d = JSON.parse(m.toString()) + if (d.id === 1) { + console.log(JSON.stringify(d.result?.result?.value ?? d.result ?? d.error)) + ws.close() + } +}) diff --git a/tests/tools/relay-bench/phone-e2ee-desktop-parity.test.mjs b/tests/tools/relay-bench/phone-e2ee-desktop-parity.test.mjs new file mode 100644 index 00000000000..6400498796a --- /dev/null +++ b/tests/tools/relay-bench/phone-e2ee-desktop-parity.test.mjs @@ -0,0 +1,69 @@ +// Why: the bench hand-rolls the mobile E2EE v2 client in plain JS so it can run outside the +// React Native bundle. This pins it to the real desktop responder, so a change to the transcript +// encoding, key schedule, or frame layout fails here instead of silently producing a bench that +// no longer measures the shipped handshake. +import nacl from 'tweetnacl' +import { describe, expect, it } from 'vitest' +import { DesktopMobileE2EEV2Session } from '../../../src/main/runtime/rpc/mobile-e2ee-v2-desktop-session' +import { PhoneE2EE } from './phone-e2ee-v2-session.mjs' + +const RELAY_HOST_ID = 'AAAAAAAAAAAAAAAA' + +function handshake() { + const desktopKeys = nacl.box.keyPair() + const phone = new PhoneE2EE(Buffer.from(desktopKeys.publicKey).toString('base64'), RELAY_HOST_ID) + const desktop = DesktopMobileE2EEV2Session.create({ + hello: phone.hello, + serverSecretKey: desktopKeys.secretKey, + expectedContext: { transport: 'relay', relayHostId: RELAY_HOST_ID } + }) + return { phone, desktop } +} + +describe('bench PhoneE2EE against the desktop E2EE v2 responder', () => { + it('derives the same transcript hash from the shipped hello', () => { + const { phone, desktop } = handshake() + expect(desktop).not.toBeNull() + phone.acceptReady(desktop.ready) + expect(phone.transcriptHashB64).toBe(desktop.transcriptHashB64) + }) + + it('round-trips the e2ee_auth frame the bench sends', () => { + const { phone, desktop } = handshake() + phone.acceptReady(desktop.ready) + const auth = JSON.stringify({ + type: 'e2ee_auth', + v: 2, + transcriptHashB64: phone.transcriptHashB64, + deviceToken: 'device-token' + }) + expect(desktop.openText(phone.sealText(auth))).toBe(auth) + }) + + it('opens the desktop reply and keeps counters in step across frames', () => { + const { phone, desktop } = handshake() + phone.acceptReady(desktop.ready) + expect(phone.openText(desktop.sealText('{"type":"e2ee_authenticated"}'))).toBe( + '{"type":"e2ee_authenticated"}' + ) + expect(phone.openText(desktop.sealText('{"id":"b-1","ok":true}'))).toBe( + '{"id":"b-1","ok":true}' + ) + const binary = new Uint8Array([1, 2, 3, 4]) + expect(Array.from(phone.open(desktop.sealBinary(binary), 1))).toEqual([1, 2, 3, 4]) + expect(phone.openText(desktop.sealText('{"id":"b-2","ok":true}'))).toBe( + '{"id":"b-2","ok":true}' + ) + }) + + it('rejects a desktop key it did not pin', () => { + const { phone, desktop } = handshake() + const impostor = nacl.box.keyPair() + expect(() => + phone.acceptReady({ + ...desktop.ready, + desktopPublicKeyB64: Buffer.from(impostor.publicKey).toString('base64') + }) + ).toThrow(/desktop key mismatch/) + }) +}) diff --git a/tests/tools/relay-bench/phone-e2ee-v2-session.mjs b/tests/tools/relay-bench/phone-e2ee-v2-session.mjs new file mode 100644 index 00000000000..ebd2a03f86e --- /dev/null +++ b/tests/tools/relay-bench/phone-e2ee-v2-session.mjs @@ -0,0 +1,192 @@ +// The mobile E2EE v2 client handshake, re-implemented in plain JS so the relay bench can run +// outside the React Native bundle. Mirrors mobile/src/transport/mobile-e2ee-v2-client-session.ts +// plus the encodings in src/shared/mobile-e2ee-v2-contract.ts and mobile-e2ee-v2-framing.ts. +// phone-e2ee-desktop-parity.test.mjs pins it to the real desktop responder. +import { createHash, hkdfSync } from 'node:crypto' +import { createRequire } from 'node:module' + +const nacl = createRequire(import.meta.url)('tweetnacl') + +const TRANSCRIPT_DOMAIN = 'orca-mobile-e2ee/v2/transcript' +const SALT_LABEL = utf8('orca-mobile-e2ee/v2/salt\0') +const INFO_LABEL = utf8('orca-mobile-e2ee/v2/session\0') +const NONCE_LENGTH = 24 +const SESSION_ID_LENGTH = 32 +const HEADER_LENGTH = SESSION_ID_LENGTH + 1 + 1 + 8 + +// ---------- byte helpers ---------- +export function utf8(value) { + return new TextEncoder().encode(value) +} +function uint32(value) { + const bytes = new Uint8Array(4) + new DataView(bytes.buffer).setUint32(0, value) + return bytes +} +function concat(parts) { + const out = new Uint8Array(parts.reduce((total, part) => total + part.length, 0)) + let offset = 0 + for (const part of parts) { + out.set(part, offset) + offset += part.length + } + return out +} +export function sha256(bytes) { + return new Uint8Array(createHash('sha256').update(bytes).digest()) +} +function b64(bytes) { + return Buffer.from(bytes).toString('base64') +} +function unb64(value) { + return new Uint8Array(Buffer.from(value, 'base64')) +} +export function b64url(bytes) { + return Buffer.from(bytes).toString('base64url') +} +function writeU64(target, offset, value) { + new DataView(target.buffer, target.byteOffset).setBigUint64(offset, value) +} +// Transcript list encodings must stay byte-identical to encodeMobileE2EEV2Transcript in +// src/shared/mobile-e2ee-v2-contract.ts, or the derived key schedule diverges silently. +function encodeStringList(items) { + return concat([ + uint32(items.length), + ...items.map((value) => concat([uint32(value.length), value])) + ]) +} +function encodeNumberList(items) { + return concat([uint32(items.length), ...items.map(uint32)]) +} + +// ---------- E2EE v2 (mirrors mobile/src/transport/mobile-e2ee-v2-client-session.ts) ---------- +export class PhoneE2EE { + constructor(desktopPublicKeyB64, relayHostId) { + this.keys = nacl.box.keyPair() + this.desktopPublicKey = unb64(desktopPublicKeyB64) + this.clientNonce = nacl.randomBytes(32) + this.hello = { + type: 'e2ee_hello', + v: 2, + clientPublicKeyB64: b64(this.keys.publicKey), + clientNonceB64: b64(this.clientNonce), + capabilities: { framing: [2], payloadKinds: ['text', 'binary'] }, + context: { + protocol: 'orca-mobile-e2ee', + initiator: 'mobile', + responder: 'desktop', + transport: 'relay', + relayHostId + } + } + this.inbound = 0n + this.outbound = 0n + } + + acceptReady(ready) { + if (ready?.type !== 'e2ee_ready' || ready.v !== 2) { + throw new Error('bad e2ee_ready') + } + const desktopPublicKey = unb64(ready.desktopPublicKeyB64) + if (!nacl.verify(desktopPublicKey, this.desktopPublicKey)) { + throw new Error('desktop key mismatch') + } + const desktopNonce = unb64(ready.desktopNonceB64) + const hello = this.hello + const fields = [ + ['domain', utf8(TRANSCRIPT_DOMAIN)], + ['mobile-to-desktop.type', utf8(hello.type)], + ['mobile-to-desktop.version', uint32(hello.v)], + ['mobile-to-desktop.client-public-key', this.keys.publicKey], + ['mobile-to-desktop.client-nonce', this.clientNonce], + ['mobile-to-desktop.capabilities.framing', encodeNumberList(hello.capabilities.framing)], + [ + 'mobile-to-desktop.capabilities.payload-kinds', + encodeStringList(hello.capabilities.payloadKinds.map(utf8)) + ], + ['mobile-to-desktop.context.protocol', utf8(hello.context.protocol)], + ['mobile-to-desktop.context.initiator', utf8(hello.context.initiator)], + ['mobile-to-desktop.context.responder', utf8(hello.context.responder)], + ['mobile-to-desktop.context.transport', utf8(hello.context.transport)], + ['mobile-to-desktop.context.relay-host-id', utf8(hello.context.relayHostId ?? '')], + ['desktop-to-mobile.type', utf8(ready.type)], + ['desktop-to-mobile.version', uint32(ready.v)], + ['desktop-to-mobile.desktop-public-key', desktopPublicKey], + ['desktop-to-mobile.client-nonce-echo', this.clientNonce], + ['desktop-to-mobile.desktop-nonce', desktopNonce], + ['desktop-to-mobile.selection.framing', uint32(ready.selection.framing)], + [ + 'desktop-to-mobile.selection.payload-kinds', + encodeStringList(ready.selection.payloadKinds.map(utf8)) + ], + ['desktop-to-mobile.context.protocol', utf8(ready.context.protocol)], + ['desktop-to-mobile.context.initiator', utf8(ready.context.initiator)], + ['desktop-to-mobile.context.responder', utf8(ready.context.responder)], + ['desktop-to-mobile.context.transport', utf8(ready.context.transport)], + ['desktop-to-mobile.context.relay-host-id', utf8(ready.context.relayHostId ?? '')] + ] + const transcript = concat( + fields.map(([name, value]) => + concat([uint32(utf8(name).length), utf8(name), uint32(value.length), value]) + ) + ) + const shared = nacl.box.before(this.desktopPublicKey, this.keys.secretKey) + const transcriptHash = sha256(transcript) + const salt = sha256(concat([SALT_LABEL, this.clientNonce, desktopNonce])) + const info = concat([INFO_LABEL, transcriptHash]) + const expanded = new Uint8Array(hkdfSync('sha256', shared, salt, info, 96)) + this.m2d = expanded.slice(0, 32) + this.d2m = expanded.slice(32, 64) + this.sessionId = expanded.slice(64, 96) + this.transcriptHashB64 = b64(transcriptHash) + } + + frameNonce(direction, kind, counter) { + const nonce = new Uint8Array(NONCE_LENGTH) + nonce.set(this.sessionId.subarray(0, 12), 0) + nonce[12] = 2 + nonce[13] = direction + nonce[14] = kind + nonce[15] = 0 + writeU64(nonce, 16, counter) + return nonce + } + + frameHeader(direction, kind, counter) { + const header = new Uint8Array(HEADER_LENGTH) + header.set(this.sessionId, 0) + header[SESSION_ID_LENGTH] = direction + header[SESSION_ID_LENGTH + 1] = kind + writeU64(header, SESSION_ID_LENGTH + 2, counter) + return header + } + + sealText(plaintext) { + const counter = this.outbound++ + const nonce = this.frameNonce(0, 0, counter) + const body = concat([this.frameHeader(0, 0, counter), utf8(plaintext)]) + return b64(concat([nonce, nacl.secretbox(body, nonce, this.m2d)])) + } + + // The inbound counter is shared across text and binary, so every inbound frame must be + // consumed here even when the caller discards it, or the next open() nonce is off by one. + open(frame, kind) { + const counter = this.inbound++ + const nonce = this.frameNonce(1, kind, counter) + if (!nacl.verify(frame.subarray(0, NONCE_LENGTH), nonce)) { + throw new Error('nonce mismatch') + } + const plain = nacl.secretbox.open(frame.subarray(NONCE_LENGTH), nonce, this.d2m) + if (!plain) { + throw new Error('open failed') + } + if (!nacl.verify(plain.subarray(0, HEADER_LENGTH), this.frameHeader(1, kind, counter))) { + throw new Error('header mismatch') + } + return plain.slice(HEADER_LENGTH) + } + + openText(frameB64) { + return new TextDecoder().decode(this.open(unb64(frameB64), 0)) + } +} diff --git a/tests/tools/relay-bench/region-probe-replay.mjs b/tests/tools/relay-bench/region-probe-replay.mjs new file mode 100644 index 00000000000..9f97b13deef --- /dev/null +++ b/tests/tools/relay-bench/region-probe-replay.mjs @@ -0,0 +1,134 @@ +// Replays the desktop's region selection (relay-region-preference.ts) with the same probe, +// sample count, spread rule, and Node fetch, and prints why each region passed or failed. +import { pathToFileURL } from 'node:url' +import { + classifyPublicHttpsOrigin, + LIVE_ENV_VAR, + parseArgs, + requireBoundedInteger, + requireDirector, + requireLiveRun, + resolvesToPublicAddress +} from './relay-bench-invocation.mjs' + +const USAGE = `${LIVE_ENV_VAR}=1 node region-probe-replay.mjs --director= [--rounds=N]` +const SAMPLES = 3 +const PROBE_TIMEOUT_MS = 1500 +const CATALOG_TIMEOUT_MS = 10_000 +const MAX_ROUNDS = 1000 + +const probe = async (origin) => { + const started = performance.now() + try { + const res = await fetch(`${origin}/health`, { + cache: 'no-store', + redirect: 'error', + signal: AbortSignal.timeout(PROBE_TIMEOUT_MS) + }) + await res.arrayBuffer() + return res.ok ? performance.now() - started : null + } catch { + return null + } +} + +// The catalog names the destinations, so a compromised or spoofed director would otherwise get to +// aim this harness at the operator's loopback and private networks. redirect: 'error' above only +// constrains where a probe may go next, never where the first request goes. +export async function vetProbeOrigins(entry, deps) { + const allowed = [] + const refused = [] + for (const origin of entry.probeOrigins ?? []) { + const verdict = classifyPublicHttpsOrigin(origin) + if (!verdict.ok) { + refused.push(verdict.reason) + continue + } + const resolved = await resolvesToPublicAddress(verdict.origin, deps) + if (!resolved.ok) { + refused.push(resolved.reason) + continue + } + allowed.push(verdict.origin) + } + return { allowed, refused } +} + +export async function sampleRegion(entry, deps) { + const { allowed, refused } = await vetProbeOrigins(entry, deps) + const base = { region: entry.region, samples: [], median: null, spread: null } + if (!allowed.length) { + return { ...base, refusedOrigins: refused, verdict: 'REFUSED (no allowed probe origin)' } + } + const samples = [] + for (let index = 0; index < SAMPLES; index++) { + const latencies = (await Promise.all(allowed.map(deps?.probe ?? probe))).filter( + (value) => value !== null + ) + // Math.min of nothing is Infinity, which would spread into NaN and read as a passing region. + if (!latencies.length) { + return { + ...base, + samples: samples.map(Math.round), + verdict: 'UNREACHABLE (every probe failed)' + } + } + samples.push(Math.min(...latencies)) + } + const raw = samples.map((value) => Math.round(value)) + samples.sort((a, b) => a - b) + const median = samples[1] + const spread = samples[2] - samples[0] + return { + region: entry.region, + samples: raw, + median: Math.round(median), + spread: Math.round(spread), + ...(refused.length ? { refusedOrigins: refused } : {}), + // The shipped rule: a wide spread means the samples are untrustworthy, not that the + // region is far, so the region is dropped rather than ranked. + verdict: spread > Math.max(20, median * 0.5) ? 'REJECTED (spread)' : 'ok' + } +} + +async function main() { + const { options } = parseArgs(process.argv.slice(2)) + requireLiveRun(USAGE) + const director = requireDirector(options, USAGE) + const rounds = requireBoundedInteger(options.get('--rounds'), '--rounds', USAGE, { + min: 1, + max: MAX_ROUNDS, + fallback: 3 + }) + + let catalog + try { + const res = await fetch(`${director}/v1/regions`, { + signal: AbortSignal.timeout(CATALOG_TIMEOUT_MS) + }) + catalog = await res.json() + } catch (err) { + const timedOut = err.name === 'TimeoutError' || err.cause?.name === 'TimeoutError' + console.error( + timedOut + ? `director ${director}/v1/regions did not answer within ${CATALOG_TIMEOUT_MS} ms` + : `director ${director}/v1/regions failed: ${err.message}` + ) + process.exitCode = 1 + return + } + if (!Array.isArray(catalog?.regions) || catalog.regions.length === 0) { + console.error(`director ${director}/v1/regions returned no regions`) + process.exitCode = 1 + return + } + for (let round = 0; round < rounds; round++) { + console.log( + JSON.stringify(await Promise.all(catalog.regions.map((entry) => sampleRegion(entry)))) + ) + } +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + await main() +} diff --git a/tests/tools/relay-bench/region-probe-replay.test.mjs b/tests/tools/relay-bench/region-probe-replay.test.mjs new file mode 100644 index 00000000000..89c56d8bcb5 --- /dev/null +++ b/tests/tools/relay-bench/region-probe-replay.test.mjs @@ -0,0 +1,122 @@ +// Why: the region catalog comes from the director, so it names the destinations this harness +// fetches. Without vetting, a compromised or spoofed director aims the operator's own host at +// loopback and private networks, and `redirect: 'error'` never constrains the first request. +// The all-probes-failed case is here because Math.min of nothing is Infinity, which spread into +// NaN and made an unreachable region report 'ok'. +import { createServer } from 'node:http' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { sampleRegion, vetProbeOrigins } from './region-probe-replay.mjs' + +const servers = [] + +afterEach(async () => { + await Promise.all(servers.splice(0).map((server) => new Promise((res) => server.close(res)))) +}) + +/** A real listener, so "no request reached it" is observed rather than assumed. */ +async function loopbackListener() { + const received = [] + const server = createServer((req, res) => { + received.push(req.url) + res.end('ok') + }) + servers.push(server) + await new Promise((res) => server.listen(0, '127.0.0.1', res)) + return { port: server.address().port, received } +} + +describe('vetProbeOrigins', () => { + it('refuses every non-https and non-public origin the director offers', async () => { + const { allowed, refused } = await vetProbeOrigins({ + region: 'test', + probeOrigins: [ + 'http://relay.example', + 'https://127.0.0.1:8443', + 'https://localhost:8443', + 'https://[::1]:8443', + 'https://169.254.169.254', + 'https://10.0.0.4' + ] + }) + expect(allowed).toEqual([]) + expect(refused).toHaveLength(6) + }) + + it('keeps a public https origin and consults DNS for a name', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '8.8.8.8', family: 4 }]) + const { allowed, refused } = await vetProbeOrigins( + { region: 'test', probeOrigins: ['https://relay.example/health'] }, + { lookup } + ) + expect(allowed).toEqual(['https://relay.example']) + expect(refused).toEqual([]) + expect(lookup).toHaveBeenCalledWith('relay.example', { all: true }) + }) + + it('refuses a public-looking name that resolves into the operator network', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '127.0.0.1', family: 4 }]) + const { allowed } = await vetProbeOrigins( + { region: 'test', probeOrigins: ['https://rebound.example'] }, + { lookup } + ) + expect(allowed).toEqual([]) + }) + + it('tolerates a region with no probe origins', async () => { + expect(await vetProbeOrigins({ region: 'test' })).toEqual({ allowed: [], refused: [] }) + }) +}) + +describe('sampleRegion', () => { + it('sends no request to a loopback listener the director named', async () => { + const listener = await loopbackListener() + const result = await sampleRegion({ + region: 'evil', + probeOrigins: [`http://127.0.0.1:${listener.port}`, `https://127.0.0.1:${listener.port}`] + }) + expect(listener.received).toEqual([]) + expect(result.verdict).toBe('REFUSED (no allowed probe origin)') + expect(result.median).toBeNull() + }) + + it('reports unreachable instead of ok when every probe fails', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '8.8.8.8', family: 4 }]) + const probe = vi.fn().mockResolvedValue(null) + const result = await sampleRegion( + { region: 'far', probeOrigins: ['https://relay.example'] }, + { lookup, probe } + ) + expect(result.verdict).toBe('UNREACHABLE (every probe failed)') + expect(result.median).toBeNull() + expect(result.spread).toBeNull() + expect(Number.isFinite(result.median)).toBe(false) + }) + + it('ranks a region whose probes answer consistently', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '8.8.8.8', family: 4 }]) + const latencies = [30, 31, 32] + const probe = vi.fn(() => Promise.resolve(latencies.shift())) + const result = await sampleRegion( + { region: 'near', probeOrigins: ['https://relay.example'] }, + { lookup, probe } + ) + expect(result).toMatchObject({ + region: 'near', + samples: [30, 31, 32], + median: 31, + spread: 2, + verdict: 'ok' + }) + }) + + it('applies the shipped spread rule to an inconsistent region', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '8.8.8.8', family: 4 }]) + const latencies = [10, 500, 12] + const probe = vi.fn(() => Promise.resolve(latencies.shift())) + const result = await sampleRegion( + { region: 'jittery', probeOrigins: ['https://relay.example'] }, + { lookup, probe } + ) + expect(result.verdict).toBe('REJECTED (spread)') + }) +}) diff --git a/tests/tools/relay-bench/relay-bench-invocation.mjs b/tests/tools/relay-bench/relay-bench-invocation.mjs new file mode 100644 index 00000000000..117ea4036b8 --- /dev/null +++ b/tests/tools/relay-bench/relay-bench-invocation.mjs @@ -0,0 +1,294 @@ +// Argument parsing and the guards every script in this directory runs before it opens a socket. +// Why: these benches dial real relay infrastructure with real credentials, so nothing here carries +// a production default. The operator names the target and opts in explicitly, which makes an +// accidental or automated run inert rather than live traffic against production. The destination +// guards below exist because a director the operator names also *supplies* URLs (probe origins, +// resolved cell URLs); without them a compromised or spoofed director could aim this harness at +// the operator's own loopback and private networks. +import { lookup as dnsLookup } from 'node:dns/promises' + +export const LIVE_ENV_VAR = 'ORCA_RELAY_BENCH_LIVE' +export const DIRECTOR_ENV_VAR = 'ORCA_RELAY_BENCH_DIRECTOR' + +export function parseArgs(argv) { + const flags = new Set() + const options = new Map() + const positional = [] + for (const arg of argv) { + if (!arg.startsWith('--')) { + positional.push(arg) + continue + } + const equals = arg.indexOf('=') + if (equals === -1) { + flags.add(arg) + } else { + options.set(arg.slice(0, equals), arg.slice(equals + 1)) + } + } + return { flags, options, positional } +} + +/** @returns {never} */ +export function refuse(message) { + console.error(message) + process.exit(2) +} + +export function requireLiveRun(usage) { + if (process.env[LIVE_ENV_VAR] !== '1') { + refuse(`refusing to dial the relay: set ${LIVE_ENV_VAR}=1 to opt in. usage: ${usage}`) + } +} + +// ---------- numeric arguments ---------- +// Why: a bare Number() cast accepts 'Infinity' (loops forever, unbounded relay traffic), '' and +// 'abc' (NaN, a silent no-op run that still reports success), and negatives. +export function parseBoundedInteger(value, { min, max }) { + if (typeof value !== 'string') { + return null + } + const text = value.trim() + if (!/^\d+$/.test(text)) { + return null + } + const parsed = Number(text) + if (!Number.isSafeInteger(parsed) || parsed < min || parsed > max) { + return null + } + return parsed +} + +export function requireBoundedInteger(value, label, usage, { min, max, fallback }) { + if (value === undefined || value === null) { + return fallback + } + const parsed = parseBoundedInteger(value, { min, max }) + if (parsed === null) { + refuse(`${label} must be a whole number ${min}-${max}, got ${value}. usage: ${usage}`) + } + return parsed +} + +/** Rejects '80@attacker.example', which URL parsing would read as userinfo, not a port. */ +export function parsePort(value) { + return parseBoundedInteger(value, { min: 1, max: 65_535 }) +} + +export function requirePort(value, label, usage) { + const parsed = parsePort(value) + if (parsed === null) { + refuse(`${label} must be a port 1-65535, got ${value}. usage: ${usage}`) + } + return parsed +} + +// ---------- destinations ---------- +const BLOCKED_IPV4_RANGES = [ + ['0.0.0.0', 8], + ['10.0.0.0', 8], + ['100.64.0.0', 10], + ['127.0.0.0', 8], + ['169.254.0.0', 16], + ['172.16.0.0', 12], + ['192.0.0.0', 24], + ['192.168.0.0', 16], + ['198.18.0.0', 15], + ['224.0.0.0', 4], + ['240.0.0.0', 4] +] + +function ipv4ToInt(text) { + const parts = text.split('.') + if (parts.length !== 4) { + return null + } + let value = 0 + for (const part of parts) { + if (!/^\d{1,3}$/.test(part)) { + return null + } + const octet = Number(part) + if (octet > 255) { + return null + } + value = value * 256 + octet + } + return value +} + +function isPublicIpv4(value) { + return !BLOCKED_IPV4_RANGES.some(([base, bits]) => { + const mask = bits === 0 ? 0 : (-1 << (32 - bits)) >>> 0 + return (value & mask) >>> 0 === (ipv4ToInt(base) & mask) >>> 0 + }) +} + +function ipv6ToBytes(host) { + let text = host.toLowerCase() + const zone = text.indexOf('%') + if (zone !== -1) { + text = text.slice(0, zone) + } + if (!text.includes(':')) { + return null + } + const lastColon = text.lastIndexOf(':') + const tail = text.slice(lastColon + 1) + if (tail.includes('.')) { + // ::ffff:127.0.0.1 and ::127.0.0.1 embed a v4 address in the last two groups. + const embedded = ipv4ToInt(tail) + if (embedded === null) { + return null + } + const high = ((embedded >>> 16) & 0xffff).toString(16) + const low = (embedded & 0xffff).toString(16) + text = `${text.slice(0, lastColon + 1)}${high}:${low}` + } + const halves = text.split('::') + if (halves.length > 2) { + return null + } + const head = halves[0] ? halves[0].split(':') : [] + const rest = halves.length === 2 && halves[1] ? halves[1].split(':') : [] + const missing = 8 - head.length - rest.length + if ( + missing < 0 || + (halves.length === 1 && missing !== 0) || + (halves.length === 2 && missing < 1) + ) { + return null + } + const zeros = Array.from({ length: halves.length === 2 ? missing : 0 }, () => '0') + const groups = [...head, ...zeros, ...rest] + const bytes = [] + for (const group of groups) { + if (!/^[0-9a-f]{1,4}$/.test(group)) { + return null + } + const parsed = Number.parseInt(group, 16) + bytes.push((parsed >> 8) & 0xff, parsed & 0xff) + } + return bytes +} + +function isPublicIpv6(bytes) { + const leadingZeros = bytes.slice(0, 10).every((byte) => byte === 0) + if (leadingZeros && bytes[10] === 0xff && bytes[11] === 0xff) { + return isPublicIpv4( + ((bytes[12] << 24) >>> 0) + (bytes[13] << 16) + (bytes[14] << 8) + bytes[15] + ) + } + if (leadingZeros && bytes[10] === 0 && bytes[11] === 0) { + // Covers :: and ::1 as well as the deprecated v4-compatible form. + return false + } + if ((bytes[0] & 0xfe) === 0xfc || bytes[0] === 0xff) { + return false + } + if (bytes[0] === 0xfe && (bytes[1] & 0xc0) === 0x80) { + return false + } + return true +} + +/** true/false for an IP literal, null when the hostname is a DNS name. */ +export function isPublicIpAddress(host) { + const v4 = ipv4ToInt(host) + if (v4 !== null) { + return isPublicIpv4(v4) + } + const v6 = ipv6ToBytes(host) + if (v6 !== null) { + return isPublicIpv6(v6) + } + return null +} + +// WHATWG keeps the brackets on an IPv6 hostname, and a trailing dot is the same name. +function normalizeHostname(hostname) { + return hostname + .toLowerCase() + .replace(/^\[|\]$/g, '') + .replace(/\.$/, '') +} + +/** + * Literal-address vetting for a URL this harness is about to fetch. Returns the normalized origin + * or the reason it is refused. A DNS name still needs resolvesToPublicAddress(). + */ +export function classifyPublicHttpsOrigin(value) { + if (typeof value !== 'string' || !value) { + return { ok: false, reason: 'missing origin' } + } + let parsed + try { + parsed = new URL(value) + } catch { + return { ok: false, reason: `not a URL: ${value}` } + } + if (parsed.protocol !== 'https:') { + return { ok: false, reason: `must be an https origin: ${value}` } + } + if (parsed.username || parsed.password) { + return { ok: false, reason: `must not carry credentials: ${value}` } + } + const host = normalizeHostname(parsed.hostname) + if (host === 'localhost' || host.endsWith('.localhost')) { + return { ok: false, reason: `refusing a loopback destination: ${value}` } + } + if (isPublicIpAddress(host) === false) { + return { + ok: false, + reason: `refusing a loopback, link-local, or private destination: ${value}` + } + } + return { ok: true, origin: parsed.origin } +} + +/** + * Second layer for DNS names: a director could hand back a public-looking name that resolves into + * the operator's network. fetch() resolves again, so this narrows the window rather than closing + * it; the literal check above is what makes the obvious cases impossible. + */ +export async function resolvesToPublicAddress(origin, { lookup = dnsLookup } = {}) { + const host = normalizeHostname(new URL(origin).hostname) + if (isPublicIpAddress(host) !== null) { + return { ok: true } + } + let addresses + try { + addresses = await lookup(host, { all: true }) + } catch (err) { + return { ok: false, reason: `cannot resolve ${host}: ${err.message}` } + } + if (!addresses.length) { + return { ok: false, reason: `cannot resolve ${host}` } + } + const blocked = addresses.find((entry) => isPublicIpAddress(entry.address) === false) + if (blocked) { + return { ok: false, reason: `${host} resolves to a private address ${blocked.address}` } + } + return { ok: true } +} + +export function requireOrigin(value, label, usage) { + if (!value) { + refuse(`missing ${label}. usage: ${usage}`) + } + // https only: these origins carry bench credentials, and http would let an on-path observer + // read or rewrite them. + const verdict = classifyPublicHttpsOrigin(value) + if (!verdict.ok) { + refuse(`${label} ${verdict.reason}. usage: ${usage}`) + } + return verdict.origin +} + +export function requireDirector(options, usage) { + return requireOrigin( + options.get('--director') ?? process.env[DIRECTOR_ENV_VAR], + `director origin (--director= or ${DIRECTOR_ENV_VAR})`, + usage + ) +} diff --git a/tests/tools/relay-bench/relay-bench-invocation.test.mjs b/tests/tools/relay-bench/relay-bench-invocation.test.mjs new file mode 100644 index 00000000000..b450695fc99 --- /dev/null +++ b/tests/tools/relay-bench/relay-bench-invocation.test.mjs @@ -0,0 +1,244 @@ +// Why: every guard in relay-bench-invocation.mjs is the only thing standing between an operator +// typo (or a director that hands back a hostile URL) and live traffic from the operator's host. +// These are the cases that previously slipped through a bare Number() cast or a URL constructor. +import { describe, expect, it, vi } from 'vitest' +import { + classifyPublicHttpsOrigin, + isPublicIpAddress, + parseArgs, + parseBoundedInteger, + parsePort, + requireBoundedInteger, + requireDirector, + requireOrigin, + requirePort, + resolvesToPublicAddress +} from './relay-bench-invocation.mjs' + +/** refuse() exits the process; make that observable instead of killing the test worker. */ +function captureRefusal(run) { + const exit = vi.spyOn(process, 'exit').mockImplementation((code) => { + throw new Error(`exit:${code}`) + }) + const error = vi.spyOn(console, 'error').mockImplementation(() => {}) + try { + run() + return null + } catch (err) { + if (!err.message.startsWith('exit:')) { + throw err + } + return { code: Number(err.message.slice('exit:'.length)), message: error.mock.calls[0]?.[0] } + } finally { + exit.mockRestore() + error.mockRestore() + } +} + +describe('parseArgs', () => { + it('splits flags, options, and positionals', () => { + const { flags, options, positional } = parseArgs(['run', 'state.json', '--resolve', '--gap=20']) + expect([...flags]).toEqual(['--resolve']) + expect(options.get('--gap')).toBe('20') + expect(positional).toEqual(['run', 'state.json']) + }) + + it('keeps an equals sign inside an option value', () => { + const { options } = parseArgs(['--director=https://a.example/?x=1']) + expect(options.get('--director')).toBe('https://a.example/?x=1') + }) +}) + +describe('parseBoundedInteger', () => { + it.each(['5', ' 5 ', '0'])('accepts the whole number %s', (value) => { + expect(parseBoundedInteger(value, { min: 0, max: 10 })).toBe(Number(value.trim())) + }) + + // 'Infinity' is the one that mattered: Number('Infinity') made the run loops never terminate. + it.each(['Infinity', '-Infinity', 'NaN', '', ' ', 'abc', '1e3', '-1', '1.5', '0x10', '+2'])( + 'rejects %j', + (value) => { + expect(parseBoundedInteger(value, { min: 0, max: 10 })).toBeNull() + } + ) + + it('rejects values outside the bounds', () => { + expect(parseBoundedInteger('11', { min: 0, max: 10 })).toBeNull() + expect(parseBoundedInteger('0', { min: 1, max: 10 })).toBeNull() + }) + + it('rejects a non-string', () => { + expect(parseBoundedInteger(undefined, { min: 0, max: 10 })).toBeNull() + expect(parseBoundedInteger(5, { min: 0, max: 10 })).toBeNull() + }) +}) + +describe('requireBoundedInteger', () => { + it('falls back when the option is absent', () => { + expect( + requireBoundedInteger(undefined, '--runs', 'usage', { min: 1, max: 10, fallback: 5 }) + ).toBe(5) + }) + + it('exits 2 on Infinity rather than looping forever', () => { + const refusal = captureRefusal(() => + requireBoundedInteger('Infinity', '--runs', 'usage', { min: 1, max: 10, fallback: 5 }) + ) + expect(refusal?.code).toBe(2) + expect(refusal?.message).toContain('--runs must be a whole number 1-10') + }) +}) + +describe('parsePort', () => { + it('accepts a decimal port', () => { + expect(parsePort('9222')).toBe(9222) + }) + + // WHATWG URL reads '80@attacker.example' as userinfo, so the fetch would leave loopback. + it.each(['80@attacker.example', '0', '65536', '9222 9223', 'Infinity', ''])( + 'rejects %j', + (value) => { + expect(parsePort(value)).toBeNull() + } + ) + + it('exits 2 through requirePort', () => { + expect( + captureRefusal(() => requirePort('80@attacker.example', 'devtools port', 'usage'))?.code + ).toBe(2) + }) +}) + +describe('isPublicIpAddress', () => { + it.each([ + '127.0.0.1', + '127.1.2.3', + '0.0.0.0', + '10.0.0.1', + '172.16.0.1', + '172.31.255.255', + '192.168.1.1', + '169.254.169.254', + '100.64.0.1', + '224.0.0.1', + '255.255.255.255', + '::1', + '::', + '::ffff:127.0.0.1', + 'fe80::1', + 'fc00::1', + 'fd12:3456::1', + 'ff02::1' + ])('refuses %s', (host) => { + expect(isPublicIpAddress(host)).toBe(false) + }) + + it.each(['8.8.8.8', '172.32.0.1', '172.15.0.1', '1.1.1.1', '2001:db8::1', '::ffff:8.8.8.8'])( + 'allows %s', + (host) => { + expect(isPublicIpAddress(host)).toBe(true) + } + ) + + it('reports null for a DNS name', () => { + expect(isPublicIpAddress('relay.example')).toBeNull() + }) +}) + +describe('classifyPublicHttpsOrigin', () => { + it('normalizes an accepted origin', () => { + expect(classifyPublicHttpsOrigin('https://relay.example/health?x=1')).toEqual({ + ok: true, + origin: 'https://relay.example' + }) + }) + + it.each([ + ['http://relay.example', 'must be an https origin'], + ['wss://relay.example', 'must be an https origin'], + ['https://user:pass@relay.example', 'must not carry credentials'], + ['https://localhost:9222', 'loopback'], + ['https://app.localhost', 'loopback'], + ['https://127.0.0.1:8080', 'loopback, link-local, or private'], + ['https://[::1]/', 'loopback, link-local, or private'], + ['https://[::ffff:127.0.0.1]/', 'loopback, link-local, or private'], + ['https://169.254.169.254/latest/meta-data', 'loopback, link-local, or private'], + ['https://10.1.2.3', 'loopback, link-local, or private'], + ['not a url', 'not a URL'], + ['', 'missing origin'] + ])('refuses %s', (value, reason) => { + const verdict = classifyPublicHttpsOrigin(value) + expect(verdict.ok).toBe(false) + expect(verdict.reason).toContain(reason) + }) +}) + +describe('resolvesToPublicAddress', () => { + it('skips the lookup for a literal address', async () => { + const lookup = vi.fn() + await expect(resolvesToPublicAddress('https://8.8.8.8', { lookup })).resolves.toEqual({ + ok: true + }) + expect(lookup).not.toHaveBeenCalled() + }) + + it('refuses a name that resolves into the operator network', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '127.0.0.1', family: 4 }]) + const verdict = await resolvesToPublicAddress('https://relay.example', { lookup }) + expect(verdict.ok).toBe(false) + expect(verdict.reason).toContain('127.0.0.1') + }) + + it('refuses when any resolved address is private', async () => { + const lookup = vi.fn().mockResolvedValue([ + { address: '8.8.8.8', family: 4 }, + { address: '10.0.0.5', family: 4 } + ]) + expect((await resolvesToPublicAddress('https://relay.example', { lookup })).ok).toBe(false) + }) + + it('accepts a name that resolves publicly', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '8.8.8.8', family: 4 }]) + expect(await resolvesToPublicAddress('https://relay.example', { lookup })).toEqual({ ok: true }) + }) + + it('refuses when resolution fails', async () => { + const lookup = vi.fn().mockRejectedValue(new Error('ENOTFOUND')) + expect((await resolvesToPublicAddress('https://relay.example', { lookup })).ok).toBe(false) + }) +}) + +describe('requireOrigin and requireDirector', () => { + it('returns the origin for an https target', () => { + expect(requireOrigin('https://relay.example/x', 'cell origin', 'usage')).toBe( + 'https://relay.example' + ) + }) + + // http would let an on-path observer read or rewrite the credentials these origins carry. + it('exits 2 for an http origin', () => { + const refusal = captureRefusal(() => + requireOrigin('http://relay.example', 'cell origin', 'usage') + ) + expect(refusal?.code).toBe(2) + expect(refusal?.message).toContain('must be an https origin') + }) + + it('exits 2 when the director origin is missing', () => { + const previous = process.env.ORCA_RELAY_BENCH_DIRECTOR + delete process.env.ORCA_RELAY_BENCH_DIRECTOR + try { + expect(captureRefusal(() => requireDirector(new Map(), 'usage'))?.code).toBe(2) + } finally { + if (previous !== undefined) { + process.env.ORCA_RELAY_BENCH_DIRECTOR = previous + } + } + }) + + it('reads the director from the flag ahead of the environment', () => { + expect(requireDirector(new Map([['--director', 'https://d.example']]), 'usage')).toBe( + 'https://d.example' + ) + }) +}) diff --git a/tests/tools/relay-bench/relay-bench-state-file.mjs b/tests/tools/relay-bench/relay-bench-state-file.mjs new file mode 100644 index 00000000000..ac2ade00343 --- /dev/null +++ b/tests/tools/relay-bench/relay-bench-state-file.mjs @@ -0,0 +1,76 @@ +// Reads and writes the bench state bundle, which holds a live resume token and device token for a +// real paired desktop. Why this is not a bare writeFileSync: `mode` only applies when the file is +// created, so an existing world-readable state.json would keep its mode; and the default path +// lives under a directory the operator may not have created yet, so the write would throw ENOENT +// *after* the desktop already provisioned the credential, losing it. +import { + chmodSync, + closeSync, + constants, + fstatSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + writeFileSync +} from 'node:fs' +import { dirname } from 'node:path' + +export const SECRET_FILE_MODE = 0o600 +const GROUP_AND_OTHER_BITS = 0o077 +// O_NOFOLLOW is POSIX-only; on Windows the lstat check below is the whole guard. +const NOFOLLOW = constants.O_NOFOLLOW ?? 0 + +function refuseSymlink(path) { + let stats + try { + stats = lstatSync(path) + } catch { + return + } + if (!stats.isFile()) { + throw new Error( + `refusing to use ${path}: it is a symlink or a special file, not a regular file` + ) + } +} + +export function writeSecretFile(path, contents) { + mkdirSync(dirname(path), { recursive: true }) + refuseSymlink(path) + let fd + try { + fd = openSync( + path, + constants.O_WRONLY | constants.O_CREAT | constants.O_TRUNC | NOFOLLOW, + SECRET_FILE_MODE + ) + } catch (err) { + if (err.code === 'ELOOP') { + throw new Error(`refusing to use ${path}: it is a symlink, not a regular file`) + } + throw err + } + try { + if (!fstatSync(fd).isFile()) { + throw new Error(`refusing to write ${path}: not a regular file`) + } + writeFileSync(fd, contents) + } finally { + closeSync(fd) + } + // Fail closed rather than silently leaving a pre-existing 0644 file readable. + chmodSync(path, SECRET_FILE_MODE) +} + +export function readSecretFile(path) { + refuseSymlink(path) + const stats = lstatSync(path) + // Windows fs modes do not express POSIX permissions, so the check would always fail there. + if (process.platform !== 'win32' && (stats.mode & GROUP_AND_OTHER_BITS) !== 0) { + throw new Error( + `refusing to read ${path}: mode ${(stats.mode & 0o777).toString(8)} is readable beyond you. run: chmod 600 ${path}` + ) + } + return readFileSync(path, 'utf8') +} diff --git a/tests/tools/relay-bench/relay-bench-state-file.test.mjs b/tests/tools/relay-bench/relay-bench-state-file.test.mjs new file mode 100644 index 00000000000..f4293cd36ee --- /dev/null +++ b/tests/tools/relay-bench/relay-bench-state-file.test.mjs @@ -0,0 +1,100 @@ +// Why: the bench state file holds a live resume token and device token for a real paired desktop. +// A plain writeFileSync with `mode` leaves an existing 0644 file world-readable, follows a symlink +// into someone else's tree, and throws ENOENT on the default path after the desktop has already +// burned the provision request, losing the credential. +import { + chmodSync, + existsSync, + lstatSync, + mkdtempSync, + readFileSync, + rmSync, + symlinkSync, + writeFileSync +} from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { readSecretFile, writeSecretFile } from './relay-bench-state-file.mjs' + +const posix = process.platform !== 'win32' +let dir + +beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), 'relay-bench-state-')) +}) + +afterEach(() => { + rmSync(dir, { recursive: true, force: true }) +}) + +const modeOf = (path) => lstatSync(path).mode & 0o777 + +describe('writeSecretFile', () => { + it('creates a missing parent directory instead of throwing ENOENT', () => { + const path = join(dir, 'nested', 'deeper', 'state.json') + writeSecretFile(path, '{"resumeToken":"secret"}') + expect(readFileSync(path, 'utf8')).toBe('{"resumeToken":"secret"}') + }) + + it.runIf(posix)('forces 0600 on a file that already exists as 0644', () => { + const path = join(dir, 'state.json') + writeFileSync(path, 'old') + chmodSync(path, 0o644) + writeSecretFile(path, 'new') + expect(modeOf(path)).toBe(0o600) + expect(readFileSync(path, 'utf8')).toBe('new') + }) + + it.runIf(posix)('creates the file as 0600', () => { + const path = join(dir, 'state.json') + writeSecretFile(path, 'new') + expect(modeOf(path)).toBe(0o600) + }) + + it.runIf(posix)('refuses to follow a symlink and leaves the target untouched', () => { + const target = join(dir, 'target.json') + const link = join(dir, 'state.json') + writeFileSync(target, 'target contents') + symlinkSync(target, link) + expect(() => writeSecretFile(link, 'secret')).toThrow(/symlink/) + expect(readFileSync(target, 'utf8')).toBe('target contents') + }) + + it('truncates rather than appending to a longer previous file', () => { + const path = join(dir, 'state.json') + writeSecretFile(path, '{"a":"aaaaaaaaaaaaaaaaaaaa"}') + writeSecretFile(path, '{"b":1}') + expect(readFileSync(path, 'utf8')).toBe('{"b":1}') + }) +}) + +describe('readSecretFile', () => { + it('reads a file it wrote', () => { + const path = join(dir, 'state.json') + writeSecretFile(path, '{"resumeToken":"secret"}') + expect(readSecretFile(path)).toBe('{"resumeToken":"secret"}') + }) + + it.runIf(posix)('refuses a state file other users can read', () => { + const path = join(dir, 'state.json') + writeFileSync(path, 'secret') + chmodSync(path, 0o644) + expect(() => readSecretFile(path)).toThrow(/chmod 600/) + }) + + it.runIf(posix)('refuses to read through a symlink', () => { + const target = join(dir, 'target.json') + const link = join(dir, 'state.json') + writeFileSync(target, 'secret') + chmodSync(target, 0o600) + symlinkSync(target, link) + expect(() => readSecretFile(link)).toThrow(/symlink/) + }) + + it('reports a missing file rather than returning empty text', () => { + const path = join(dir, 'absent.json') + expect(existsSync(path)).toBe(false) + expect(() => readSecretFile(path)).toThrow(/ENOENT/) + }) +}) diff --git a/tests/tools/relay-bench/relay-hop-latency.mjs b/tests/tools/relay-bench/relay-hop-latency.mjs new file mode 100644 index 00000000000..66ebc8b8bd1 --- /dev/null +++ b/tests/tools/relay-bench/relay-hop-latency.mjs @@ -0,0 +1,119 @@ +// Measures the infrastructure floor of a phone→relay connect with throwaway credentials: +// director /v1/resolve (DB lookup path) and cell WebSocket open → relay-hello. Needs no pairing, +// because a cell answers a bogus credential without ever reaching a desktop. +import { createRequire } from 'node:module' +import { performance } from 'node:perf_hooks' +import { pathToFileURL } from 'node:url' +import { + LIVE_ENV_VAR, + parseArgs, + requireBoundedInteger, + requireDirector, + requireLiveRun, + requireOrigin +} from './relay-bench-invocation.mjs' + +const require = createRequire(import.meta.url) +const WebSocket = require('ws') + +const USAGE = `${LIVE_ENV_VAR}=1 node relay-hop-latency.mjs --cell= --director= [--host=] [--runs=N]` + +// A 16-character base64url id that no desktop owns, so the probe stops at the cell. +const UNROUTABLE_HOST_ID = 'AAAAAAAAAAAAAAAA' +const BOGUS_CREDENTIAL = 'A'.repeat(43) +const CELL_TIMEOUT_MS = 15_000 +// Without this a director that accepts the connection and never answers stalls the whole run loop. +const RESOLVE_TIMEOUT_MS = 10_000 +const MAX_RUNS = 1000 + +async function timeResolve(director, relayHostId) { + const started = performance.now() + try { + const res = await fetch(`${director}/v1/resolve`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ v: 1, relayHostId, resumeToken: BOGUS_CREDENTIAL }), + signal: AbortSignal.timeout(RESOLVE_TIMEOUT_MS) + }) + const body = await res.text() + return { + ms: Math.round(performance.now() - started), + status: res.status, + body: body.slice(0, 80) + } + } catch (err) { + const timedOut = err.name === 'TimeoutError' || err.cause?.name === 'TimeoutError' + return { + ms: Math.round(performance.now() - started), + status: null, + error: timedOut ? `timeout after ${RESOLVE_TIMEOUT_MS} ms` : err.message + } + } +} + +function timeCellHello(cell, relayHostId) { + return new Promise((resolve) => { + const started = performance.now() + let openedAt = 0 + const url = new URL(cell) + url.protocol = 'wss:' + url.pathname = `/v1/connect/${encodeURIComponent(relayHostId)}` + const ws = new WebSocket(url.toString(), { perMessageDeflate: false }) + let settled = false + const done = (extra) => { + // One-shot: a socket normally emits close after error, and an uncleared timer keeps Node + // alive for the full CELL_TIMEOUT_MS after the last run. + if (settled) { + return + } + settled = true + clearTimeout(timer) + ws.terminate() + resolve({ + // openedAt stays 0 when error or close beat open; reporting the difference would be a + // large negative number, not a measurement. + openMs: openedAt === 0 ? null : Math.round(openedAt - started), + totalMs: Math.round(performance.now() - started), + ...extra + }) + } + const timer = setTimeout(() => done({ error: 'timeout' }), CELL_TIMEOUT_MS) + ws.on('open', () => { + openedAt = performance.now() + ws.send( + JSON.stringify({ + type: 'relay-auth', + v: 1, + mode: 'connect', + credential: BOGUS_CREDENTIAL + }) + ) + }) + ws.on('message', (message) => done({ hello: message.toString().slice(0, 80) })) + ws.on('close', (code, reason) => done({ close: code, reason: reason.toString() })) + ws.on('error', (err) => done({ error: err.message })) + }) +} + +async function main() { + const { options } = parseArgs(process.argv.slice(2)) + requireLiveRun(USAGE) + const director = requireDirector(options, USAGE) + const cell = requireOrigin(options.get('--cell'), 'cell origin (--cell=)', USAGE) + const relayHostId = options.get('--host') ?? UNROUTABLE_HOST_ID + const runs = requireBoundedInteger(options.get('--runs'), '--runs', USAGE, { + min: 1, + max: MAX_RUNS, + fallback: 5 + }) + + for (let run = 0; run < runs; run++) { + const resolve = await timeResolve(director, relayHostId) + const cellHello = await timeCellHello(cell, relayHostId) + console.log(JSON.stringify({ run, resolve, cell: cellHello })) + } +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + await main() +} diff --git a/tests/tools/relay-bench/relay-phone-connect-bench.mjs b/tests/tools/relay-bench/relay-phone-connect-bench.mjs new file mode 100644 index 00000000000..9ad44739949 --- /dev/null +++ b/tests/tools/relay-bench/relay-phone-connect-bench.mjs @@ -0,0 +1,625 @@ +// Phone-side relay connect benchmark. Replays the shipped mobile wire sequence against a real +// desktop through the production relay and prints per-phase timings, so a connect-speed change +// can be measured from the phone's vantage without building and instrumenting the mobile app. +// +// pair: node relay-phone-connect-bench.mjs pair [state.json] [--pairing-url-file=] +// Reads the orca://pair link from stdin, or from a 0600 file, so the live invite +// token never lands in shell history or the process argument list. Dials the invite, +// runs E2EE, pairing.provisionRelay + pairing.getEndpoints, and persists the resume +// credential bundle to state.json (mode 0600, never commit it). +// run: node relay-phone-connect-bench.mjs run [state.json] [runs] [--resolve] [--gap=ms] +// Steady-state resume dial N times (what a foreground reconnect does today). +// foreground: node relay-phone-connect-bench.mjs foreground [state.json] [--hold=ms] +// [--resolve] [--force-redial] +// Connect, idle the socket like a backgrounded phone, then measure whether the +// retained socket still answers and what a full resume redial costs. +// +// See README.md for the dev-app recipe. Run from the repo root so `ws` / `tweetnacl` resolve. +import { createRequire } from 'node:module' +import { performance } from 'node:perf_hooks' +import { pathToFileURL } from 'node:url' +import { b64url, PhoneE2EE, sha256, utf8 } from './phone-e2ee-v2-session.mjs' +import { + classifyPublicHttpsOrigin, + LIVE_ENV_VAR, + parseArgs, + refuse, + requireBoundedInteger, + requireLiveRun, + resolvesToPublicAddress +} from './relay-bench-invocation.mjs' +import { readSecretFile, writeSecretFile } from './relay-bench-state-file.mjs' + +const require = createRequire(import.meta.url) +const WebSocket = require('ws') +const nacl = require('tweetnacl') + +const CAPABILITY_METHOD = 'runtime.clientCapabilities.update' +const DIAL_TIMEOUT_MS = 30_000 +const RPC_TIMEOUT_MS = 15_000 +// Without this a director that accepts the connection and never answers blocks the benchmark +// before any dial or RPC deadline has started. +const RESOLVE_TIMEOUT_MS = 10_000 +const DEFAULT_HOLD_MS = 45_000 +const DEFAULT_STATE_PATH = '/tmp/relay-bench/state.json' +const MAX_RUNS = 1000 +const MAX_DELAY_MS = 3_600_000 + +// ---------- one relay dial, phone-shaped ---------- +// Resolves once e2ee_authenticated lands, with timings and an rpc() bound to the live socket. +export function dialRelay({ + cellUrl, + relayHostId, + credential, + expectedKind, + deviceToken, + desktopPublicKeyB64 +}) { + return new Promise((resolve, reject) => { + const timings = { start: performance.now() } + const mark = (name) => (timings[name] = Math.round(performance.now() - timings.start)) + const url = new URL(cellUrl) + url.protocol = 'wss:' + url.pathname = `/v1/connect/${encodeURIComponent(relayHostId)}` + const ws = new WebSocket(url.toString(), { perMessageDeflate: false }) + const e2ee = new PhoneE2EE(desktopPublicKeyB64, relayHostId) + const handle = { timings, hello: null, closed: null } + let stage = 'awaiting-hello' + const pending = new Map() + let nextId = 0 + let settled = false + // Cleared on both outcomes: an uncleared 30 s timer keeps Node alive long after the last dial. + const dialTimer = setTimeout(() => fail(new Error('dial timeout 30s')), DIAL_TIMEOUT_MS) + // Settle, not just clear: an in-flight rpc() whose timer is dropped without a resolution + // would await forever, which is exactly the hang the rpc timeout exists to prevent. + const settlePending = (code) => { + for (const waiter of pending.values()) { + clearTimeout(waiter.timer) + waiter.res({ ok: false, error: { code } }) + } + pending.clear() + } + const fail = (err) => { + if (settled) { + return + } + settled = true + clearTimeout(dialTimer) + settlePending('dial-failed') + try { + ws.terminate() + } catch { + // already gone + } + reject(Object.assign(err, { timings, stage })) + } + handle.rpc = (method, params, timeoutMs = RPC_TIMEOUT_MS) => + new Promise((res, rej) => { + // Without this the send would only surface as a 15 s rpc timeout, which would be + // indistinguishable from a slow desktop in the foreground-hold measurement. + if (ws.readyState !== WebSocket.OPEN) { + rej(new Error(`socket not open (readyState ${ws.readyState})`)) + return + } + const id = `b-${++nextId}` + const timer = setTimeout(() => { + pending.delete(id) + rej(new Error(`rpc timeout ${method}`)) + }, timeoutMs) + pending.set(id, { res, timer }) + ws.send(e2ee.sealText(JSON.stringify({ id, method, params }))) + }) + handle.close = () => { + clearTimeout(dialTimer) + settlePending('closed') + ws.terminate() + } + handle.socket = ws + ws.on('open', () => { + mark('wsOpen') + ws.send(JSON.stringify({ type: 'relay-auth', v: 1, mode: 'connect', credential })) + mark('relayAuthSent') + }) + ws.on('message', (raw, isBinary) => { + try { + if (stage === 'awaiting-hello') { + const hello = JSON.parse(raw.toString()) + handle.hello = hello + mark('relayHello') + if (!hello.ok) { + throw new Error(`relay-hello rejected code=${hello.code}`) + } + if (hello.credentialKind !== expectedKind) { + throw new Error(`credentialKind ${hello.credentialKind} != ${expectedKind}`) + } + stage = 'awaiting-ready' + ws.send(JSON.stringify(e2ee.hello)) + mark('e2eeHelloSent') + return + } + if (stage === 'awaiting-ready') { + e2ee.acceptReady(JSON.parse(raw.toString())) + mark('e2eeReady') + stage = 'awaiting-authenticated' + ws.send( + e2ee.sealText( + JSON.stringify({ + type: 'e2ee_auth', + v: 2, + transcriptHashB64: e2ee.transcriptHashB64, + deviceToken + }) + ) + ) + mark('e2eeAuthSent') + return + } + if (isBinary) { + e2ee.open(new Uint8Array(raw), 1) + return + } + const text = e2ee.openText(raw.toString()) + if (stage === 'awaiting-authenticated') { + const msg = JSON.parse(text) + if (msg.type !== 'e2ee_authenticated') { + throw new Error(`auth rejected: ${text.slice(0, 120)}`) + } + mark('e2eeAuthenticated') + stage = 'ready' + settled = true + clearTimeout(dialTimer) + resolve(handle) + return + } + const msg = JSON.parse(text) + const waiter = msg.id && pending.get(msg.id) + if (waiter) { + clearTimeout(waiter.timer) + pending.delete(msg.id) + waiter.res(msg) + } + } catch (err) { + fail(err) + } + }) + ws.on('close', (code, reason) => { + handle.closed = { + code, + reason: reason.toString(), + atMs: Math.round(performance.now() - timings.start) + } + if (!settled) { + fail(new Error(`closed ${code} ${reason.toString()}`)) + return + } + clearTimeout(dialTimer) + settlePending('closed') + }) + ws.on('error', (err) => fail(err)) + }) +} + +/** Parses the pairing link. Every failure here is operator input, so say which part was wrong. */ +export function decodeOffer(pairingUrl) { + if (typeof pairingUrl !== 'string' || !pairingUrl.startsWith('orca://pair')) { + throw new Error('pairing link must look like orca://pair?code=') + } + const marker = pairingUrl.indexOf('code=') + if (marker === -1) { + throw new Error('pairing link has no code= parameter') + } + const code = pairingUrl + .slice(marker + 'code='.length) + .split('&')[0] + .trim() + if (!/^[A-Za-z0-9_-]+$/.test(code)) { + throw new Error('pairing link code is not base64url') + } + let offer + try { + offer = JSON.parse(Buffer.from(code, 'base64url').toString('utf8')) + } catch { + throw new Error('pairing link code did not decode to JSON') + } + if (!offer || typeof offer !== 'object' || Array.isArray(offer)) { + throw new Error('pairing link code did not decode to an offer object') + } + return offer +} + +async function resolveCell(relay, resumeToken) { + const started = performance.now() + try { + const res = await fetch(`${relay.directorUrl}/v1/resolve`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ v: 1, relayHostId: relay.relayHostId, resumeToken }), + signal: AbortSignal.timeout(RESOLVE_TIMEOUT_MS) + }) + const body = await res.json().catch(() => null) + return { ms: Math.round(performance.now() - started), status: res.status, body } + } catch (err) { + const timedOut = err.name === 'TimeoutError' || err.cause?.name === 'TimeoutError' + return { + ms: Math.round(performance.now() - started), + status: null, + error: timedOut ? `resolve timeout after ${RESOLVE_TIMEOUT_MS} ms` : err.message + } + } +} + +// ---------- shared phases ---------- +async function timedRpc(dial, method, params, timeoutMs = RPC_TIMEOUT_MS) { + const started = performance.now() + const res = await dial + .rpc(method, params, timeoutMs) + .catch((err) => ({ ok: false, error: { code: err.message } })) + const entry = { ms: Math.round(performance.now() - started), ok: Boolean(res.ok) } + if (!res.ok) { + entry.error = res.error?.code + } + return { entry, res } +} + +// What the shipped phone does before publishing 'connected': confirm resume, then a capability +// advisory, serialized. Then the UI gate's status.get, then the session's tabs.list + +// terminal.list for the first worktree, serialized. +async function runConnectedSequence(dial) { + const rpc = {} + const confirmReqId = `confirm-${b64url(nacl.randomBytes(16))}` + const phases = [ + ['confirm', 'pairing.getEndpoints', { resumeConfirmReqId: confirmReqId }], + ['capabilities', CAPABILITY_METHOD, { clientCapabilities: [] }], + ['status.get', 'status.get', undefined], + ['worktree.ps', 'worktree.ps', undefined] + ] + let firstWorktreeId = null + for (const [label, method, params] of phases) { + const { entry, res } = await timedRpc(dial, method, params) + rpc[label] = entry + if (label === 'worktree.ps' && res.ok) { + const list = Array.isArray(res.result) + ? res.result + : (res.result?.worktrees ?? res.result?.items ?? []) + entry.bytes = JSON.stringify(res.result).length + firstWorktreeId = list[0]?.id ?? null + } + } + if (firstWorktreeId) { + for (const method of ['session.tabs.list', 'terminal.list']) { + const { entry } = await timedRpc(dial, method, { worktree: `id:${firstWorktreeId}` }) + rpc[method] = entry + } + } + return { rpc, firstWorktreeId } +} + +function connectedMs(dial, rpc) { + return dial.timings.e2eeAuthenticated + rpc.confirm.ms + rpc.capabilities.ms +} + +async function resumeDial(state) { + return dialRelay({ + cellUrl: state.relay.cellUrl, + relayHostId: state.relay.relayHostId, + credential: state.resumeToken, + expectedKind: 'resume', + deviceToken: state.deviceToken, + desktopPublicKeyB64: state.desktopPublicKeyB64 + }) +} + +async function refreshCell(state, row) { + const resolved = await resolveCell(state.relay, state.resumeToken) + row.resolve = resolved + if (resolved.status !== 200) { + return + } + // The director names the next destination, so vet it the same way a probe origin is vetted: + // the literal check first, then DNS, so a public-looking name that resolves into the operator's + // network is refused before the resume credential is sent anywhere. + const verdict = await vetCellUrl(resolved.body?.cellUrl) + if (!verdict.ok) { + row.resolve = { ...resolved, error: `director named an unusable cell: ${verdict.reason}` } + return + } + state.relay = { + ...state.relay, + cellUrl: resolved.body.cellUrl, + assignmentEpoch: resolved.body.assignmentEpoch + } +} + +export async function vetCellUrl(cellUrl, deps) { + const verdict = classifyPublicHttpsOrigin(cellUrl) + if (!verdict.ok) { + return verdict + } + const resolved = await resolvesToPublicAddress(verdict.origin, deps) + return resolved.ok ? verdict : resolved +} + +function loadState(statePath) { + const state = JSON.parse(readSecretFile(statePath)) + for (const field of ['relayHostId', 'cellUrl', 'directorUrl']) { + if (!state.relay?.[field]) { + throw new Error(`${statePath} has no relay.${field}; re-run pair`) + } + } + for (const [label, value] of [ + ['relay.cellUrl', state.relay.cellUrl], + ['relay.directorUrl', state.relay.directorUrl] + ]) { + const verdict = classifyPublicHttpsOrigin(value) + if (!verdict.ok) { + throw new Error(`${statePath} ${label} ${verdict.reason}`) + } + } + return state +} + +// ---------- commands ---------- +async function pair(pairingUrl, statePath) { + const offer = decodeOffer(pairingUrl) + if (!offer.relay) { + throw new Error('offer has no relay block (desktop relay offline?)') + } + const relay = offer.relay + const verdict = await vetCellUrl(relay.cellUrl) + if (!verdict.ok) { + throw new Error(`offer names an unusable cell: ${verdict.reason}`) + } + const resumeToken = b64url(nacl.randomBytes(32)) + const resumeTokenHash = b64url(sha256(utf8(resumeToken))) + const installReqId = `install-${b64url(nacl.randomBytes(12))}` + console.log(`pair: dialing ${relay.cellUrl} host=${relay.relayHostId}`) + const dial = await dialRelay({ + cellUrl: relay.cellUrl, + relayHostId: relay.relayHostId, + credential: relay.inviteToken, + expectedKind: 'invite', + deviceToken: offer.deviceToken, + desktopPublicKeyB64: offer.publicKeyB64 + }) + console.log('invite dial timings', dial.timings) + const provisionStarted = performance.now() + const provision = await dial.rpc('pairing.provisionRelay', { + reqId: installReqId, + newResumeTokenHash: resumeTokenHash + }) + const provisionMs = Math.round(performance.now() - provisionStarted) + if (!provision.ok) { + throw new Error(`provisionRelay failed: ${JSON.stringify(provision.error)}`) + } + const endpointsStarted = performance.now() + const endpoints = await dial.rpc('pairing.getEndpoints', { installReqId }) + const endpointsMs = Math.round(performance.now() - endpointsStarted) + if (!endpoints.ok || !endpoints.result.relay) { + throw new Error(`getEndpoints failed: ${JSON.stringify(endpoints)}`) + } + console.log(`provisionRelay ${provisionMs} ms, getEndpoints ${endpointsMs} ms`) + dial.close() + const state = { + relay: endpoints.result.relay, + deviceToken: offer.deviceToken, + desktopPublicKeyB64: offer.publicKeyB64, + resumeToken, + resumeCredentialVersion: provision.result.currentVersion, + resumeExpiresAt: provision.result.resumeExpiresAt + } + // The desktop has already burned the provision request, so a failed write loses the credential. + // writeSecretFile creates the parent directory and forces 0600 even on an existing file. + writeSecretFile(statePath, JSON.stringify(state, null, 2)) + console.log(`saved ${statePath} (secret: never commit or share this file)`) +} + +async function run(statePath, runs, opts) { + const state = loadState(statePath) + const rows = [] + for (let index = 0; index < runs; index++) { + const row = { run: index } + if (opts.resolve) { + await refreshCell(state, row) + } + const started = performance.now() + try { + const dial = await resumeDial(state) + row.dial = dial.timings + row.acceptedAs = dial.hello.acceptedAs + const { rpc } = await runConnectedSequence(dial) + row.rpc = rpc + row.totalToConnectedMs = connectedMs(dial, rpc) + row.totalToFirstTerminalListMs = Math.round(performance.now() - started) + dial.close() + } catch (err) { + row.error = err.message + row.stage = err.stage + row.dial = err.timings + } + rows.push(row) + console.log(JSON.stringify(row)) + if (opts.gapMs) { + await new Promise((res) => setTimeout(res, opts.gapMs)) + } + } + const ok = rows.filter((row) => !row.error) + if (!ok.length) { + return + } + const median = (values) => { + const sorted = [...values].sort((a, b) => a - b) + return sorted[Math.floor(sorted.length / 2)] + } + console.log( + `SUMMARY ${JSON.stringify({ + runs: rows.length, + ok: ok.length, + medianMs: { + wsOpen: median(ok.map((row) => row.dial.wsOpen)), + relayHello: median(ok.map((row) => row.dial.relayHello)), + e2eeReady: median(ok.map((row) => row.dial.e2eeReady)), + e2eeAuthenticated: median(ok.map((row) => row.dial.e2eeAuthenticated)), + confirm: median(ok.map((row) => row.rpc.confirm.ms)), + capabilities: median(ok.map((row) => row.rpc.capabilities.ms)), + statusGet: median(ok.map((row) => row.rpc['status.get'].ms)), + toConnected: median(ok.map((row) => row.totalToConnectedMs)), + toTerminalList: median(ok.map((row) => row.totalToFirstTerminalListMs)) + } + })}` + ) +} + +// Simulates a backgrounded phone: connect, go silent for --hold, then find out whether the +// retained socket is still usable and what the fallback resume redial costs. The relay's client +// silence watchdog is ~105 s, so --hold=120000 is the interesting "crossed the watchdog" case. +async function foreground(statePath, opts) { + const state = loadState(statePath) + const row = { mode: 'foreground', holdMs: opts.holdMs } + if (opts.resolve) { + await refreshCell(state, row) + } + const dial = await resumeDial(state) + row.dial = dial.timings + row.acceptedAs = dial.hello.acceptedAs + const { rpc } = await runConnectedSequence(dial) + row.rpc = rpc + row.totalToConnectedMs = connectedMs(dial, rpc) + console.log(`holding socket idle for ${opts.holdMs} ms...`) + await new Promise((res) => setTimeout(res, opts.holdMs)) + row.closedDuringHold = dial.closed + const retained = await timedRpc(dial, 'status.get', undefined) + row.retainedOk = retained.entry.ok + row.retainedAnswerMs = retained.entry.ok ? retained.entry.ms : null + if (!retained.entry.ok) { + row.retainedError = retained.entry.error + } + dial.close() + if (retained.entry.ok && !opts.forceRedial) { + row.redialMs = null + console.log(JSON.stringify(row)) + return + } + if (opts.resolve) { + await refreshCell(state, row) + } + const redialStarted = performance.now() + const second = await resumeDial(state) + const secondSequence = await runConnectedSequence(second) + row.redial = { + dial: second.timings, + rpc: secondSequence.rpc, + totalToConnectedMs: connectedMs(second, secondSequence.rpc) + } + row.redialMs = Math.round(performance.now() - redialStarted) + second.close() + console.log(JSON.stringify(row)) +} + +// ---------- cli ---------- +const USAGE = [ + `every command dials a real desktop over the production relay, so prefix it with ${LIVE_ENV_VAR}=1:`, + ' pair [state.json] [--pairing-url-file=]', + ' reads the orca://pair link from stdin unless --pairing-url-file names a 0600 file, so', + ' the live invite token never enters shell history or the process argument list', + ' run [state.json] [runs] [--resolve] [--gap=ms]', + ' foreground [state.json] [--hold=ms] [--resolve] [--force-redial]' +].join('\n') + +function requireStatePath(value) { + if (value === undefined) { + return DEFAULT_STATE_PATH + } + if (value.startsWith('orca://')) { + refuse( + `the pairing link must not appear in the command line: pipe it on stdin or pass --pairing-url-file=.\n${USAGE}` + ) + } + if (!value.trim()) { + refuse(`state path must not be empty.\n${USAGE}`) + } + return value +} + +async function readStdinText() { + if (process.stdin.isTTY) { + return '' + } + const chunks = [] + for await (const chunk of process.stdin) { + chunks.push(chunk) + } + return Buffer.concat(chunks).toString('utf8') +} + +async function readPairingUrl(options) { + const file = options.get('--pairing-url-file') + const raw = (file ? readSecretFile(file) : await readStdinText()).trim() + if (!raw) { + refuse( + file + ? `${file} is empty; it must hold the orca://pair link.\n${USAGE}` + : `no pairing link on stdin. pipe it in, or pass --pairing-url-file=.\n${USAGE}` + ) + } + return raw +} + +function refuseExtraPositionals(positional, allowed) { + if (positional.length > allowed) { + refuse(`unexpected argument ${JSON.stringify(positional[allowed])}.\n${USAGE}`) + } +} + +async function main(argv) { + const [cmd, ...rest] = argv + const { flags, options, positional } = parseArgs(rest) + if (cmd === 'pair' || cmd === 'run' || cmd === 'foreground') { + requireLiveRun(`${LIVE_ENV_VAR}=1 node relay-phone-connect-bench.mjs ${cmd} ...`) + } + if (cmd === 'pair') { + refuseExtraPositionals(positional, 1) + const statePath = requireStatePath(positional[0]) + await pair(await readPairingUrl(options), statePath) + return + } + if (cmd === 'run') { + refuseExtraPositionals(positional, 2) + await run( + requireStatePath(positional[0]), + requireBoundedInteger(positional[1], 'runs', USAGE, { min: 1, max: MAX_RUNS, fallback: 5 }), + { + resolve: flags.has('--resolve'), + gapMs: requireBoundedInteger(options.get('--gap'), '--gap', USAGE, { + min: 0, + max: MAX_DELAY_MS, + fallback: 0 + }) + } + ) + return + } + if (cmd === 'foreground') { + refuseExtraPositionals(positional, 1) + await foreground(requireStatePath(positional[0]), { + resolve: flags.has('--resolve'), + forceRedial: flags.has('--force-redial'), + holdMs: requireBoundedInteger(options.get('--hold'), '--hold', USAGE, { + min: 0, + max: MAX_DELAY_MS, + fallback: DEFAULT_HOLD_MS + }) + }) + return + } + console.error(USAGE) + process.exitCode = 2 +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + // A bad state file or a refused destination is operator input, not a crash; say what is wrong + // without spilling the credential-bearing stack. + await main(process.argv.slice(2)).catch((err) => { + console.error(err.message) + process.exitCode = 1 + }) +} diff --git a/tests/tools/relay-bench/relay-phone-connect-bench.test.mjs b/tests/tools/relay-bench/relay-phone-connect-bench.test.mjs new file mode 100644 index 00000000000..3075f4591a3 --- /dev/null +++ b/tests/tools/relay-bench/relay-phone-connect-bench.test.mjs @@ -0,0 +1,59 @@ +// Why: decodeOffer used to be `pairingUrl.split('code=')[1]` fed straight to JSON.parse, so a +// missing or malformed pairing link surfaced as a stack trace rather than usage. The link is a +// live credential, so the failure text has to name the problem without echoing the code. +import { describe, expect, it, vi } from 'vitest' +import { decodeOffer, vetCellUrl } from './relay-phone-connect-bench.mjs' + +const encode = (offer) => Buffer.from(JSON.stringify(offer), 'utf8').toString('base64url') + +describe('decodeOffer', () => { + it('decodes a well-formed pairing link', () => { + const offer = { relay: { cellUrl: 'https://cell.example', relayHostId: 'A'.repeat(16) } } + expect(decodeOffer(`orca://pair?code=${encode(offer)}`)).toEqual(offer) + }) + + it('ignores parameters after the code', () => { + const offer = { deviceToken: 'token' } + expect(decodeOffer(`orca://pair?code=${encode(offer)}&v=2`)).toEqual(offer) + }) + + it.each([ + [undefined, /orca:\/\/pair/], + ['', /orca:\/\/pair/], + ['https://example.com/?code=abc', /orca:\/\/pair/], + ['orca://pair', /no code= parameter/], + ['orca://pair?code=', /not base64url/], + ['orca://pair?code=not base64', /not base64url/], + [`orca://pair?code=${Buffer.from('not json').toString('base64url')}`, /did not decode to JSON/], + [`orca://pair?code=${Buffer.from('[1,2]').toString('base64url')}`, /offer object/], + [`orca://pair?code=${Buffer.from('null').toString('base64url')}`, /offer object/] + ])('refuses %j', (value, message) => { + expect(() => decodeOffer(value)).toThrow(message) + }) +}) + +// Why: the cell URL from /v1/resolve carries the resume credential to whatever it names, so it +// gets the same DNS layer as a probe origin, not just the literal-address check. +describe('vetCellUrl', () => { + it('refuses a literal private cell before any lookup', async () => { + const lookup = vi.fn() + const verdict = await vetCellUrl('https://10.0.0.5', { lookup }) + expect(verdict.ok).toBe(false) + expect(lookup).not.toHaveBeenCalled() + }) + + it('refuses a public-looking cell name that resolves into the operator network', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '192.168.1.20', family: 4 }]) + const verdict = await vetCellUrl('https://cell.example', { lookup }) + expect(verdict.ok).toBe(false) + expect(verdict.reason).toContain('192.168.1.20') + }) + + it('returns the normalized origin for a cell that resolves publicly', async () => { + const lookup = vi.fn().mockResolvedValue([{ address: '8.8.8.8', family: 4 }]) + await expect(vetCellUrl('https://Cell.Example/', { lookup })).resolves.toEqual({ + ok: true, + origin: 'https://cell.example' + }) + }) +})