mirror of https://github.com/stablyai/orca.git synced 2026-09-22 08:02:28 +00:00

Files

T

Jinwoo Hong 06780260c0 test(remote-runtime): run an old client and an old server against current code (#12682 )

Mixed versions are the normal state of the remote-server feature: users update clients and servers independently. Until now nothing tested that. Every cross-version claim was made by code reading plus unit tests with hand-written old/new shapes — enough to catch design problems, not enough to catch a real skew regression.

This runs the REAL protocol implementations from two builds against each other in one process: the actual host methods and RPC dispatcher on one side, the actual renderer multiplexer on the other, with a transport that reproduces the production asymmetry — each side decodes with its OWN codec and drops frames whose opcode it does not know. A frame survives only if the RECEIVING build understands it, which is what makes this level sufficient without launching two apps. The old side is a genuine checkout extracted from the release tag; the extracted client was confirmed to lack a symbol that exists only on main.

Journey: subscribe, first snapshot, input reaching the process, live output, hide/reveal snapshot, transport drop, resubscribe, input landing again — across old->new, new->old, and a current/current control. Every step ends on an observed-state barrier; no sleeps. The oracle asserts the recorded step list, the exact 16-frame named sequence, negotiated capabilities, the exact input the host wrote to the PTY, rendered content, and zero decoder-rejected frames. A host method the stub lacks is recorded by name and asserted empty, so a harness gap cannot masquerade as a wire break.

Detection is proven per violation shape, and it attributes each to the correct side: an unnegotiated opcode goes red only where a decoder would reject it, a removed published field goes red only where an old client consumes it, and a legal additive field stays green in all three pairings so the harness will not cry wolf on safe changes.

It also documents the three compatibility rules in docs/reference/remote-wire-compatibility.md, linked from AGENTS.md, since they previously existed only as folklore — notably that "decoders reject unknown opcodes" is true for the desktop decoder but NOT for mobile, which silently drops them.

Deliberately scoped: terminal stream only. The session-tab sync channel is not covered, nor agent-session publications, file/Git RPCs, mobile E2EE framing, or the relay transport. Two version points, so a regression introduced and reverted between them is invisible.

CI selection was verified rather than assumed — `vitest list` confirms 0 matches under the shard's exclude and 4 under the dedicated job — because a lane silently running zero tests is precisely how a host-side defect escaped CI earlier in this series. Closes STA-3469.

2026-08-05 01:31:29 -07:00

4.6 KiB

Raw Blame History

Remote wire compatibility

Orca's remote-server feature pairs a desktop client to a remote Orca runtime, and users update the two independently. Mixed versions are the normal state, not an edge case. This page is the contract for changing anything a paired client and host exchange: the runtime RPC envelope, the terminal binary stream, and the content either side publishes over them.

src/shared/protocol-version.ts says when to bump RUNTIME_PROTOCOL_VERSION. This page covers the changes that do not bump it and are therefore easy to get wrong.

Rule 1 — a new optional JSON field on an existing frame is safe

Every JSON payload is parsed with a decoder that ignores unknown keys (zod .strip() on RPC params, JSON.parse on stream frames). An older peer that has never heard of the field simply does not read it.

Safe:

// host adds a field; older clients ignore it
encodeTerminalStreamJson({ kind, cols, rows, hiddenOutputReason })

The field is safe only for as long as every reader treats it as optional. The moment a newer client requires it, that client is broken against every host that predates the field — which is the same defect as removing a field, just discovered later. If new behavior depends on the field being present, that is Rule 2: negotiate it, or make the reader fall back.

Rule 2 — a new stream opcode is NOT safe; negotiate it

decodeTerminalStreamFrame returns null for an opcode it does not know, and runtime-rpc.ts drops that frame without an error:

const frame = decodeTerminalStreamFrame(bytes)
if (!frame) {
  return // silently dropped — the sender never learns
}

So a new opcode sent to an older peer does not fail loudly. It vanishes, and the feature behind it appears to hang. Input sent under a new opcode is swallowed.

A new opcode must be announced in the subscribe handshake and sent only after the peer confirms it. The existing pattern is SetOutputPaused (opcode 16):

the client advertises support in the Subscribe frame's capabilities;
the host echoes capabilities: { outputPause: 1 } on the subscribed event;
the client sends opcode 16 only after that echo (stream.supportsOutputPause);
the host only acts on opcode 16 when it negotiated it (stream.supportsOutputPause).

Reuse an existing opcode with a new optional payload field (Rule 1) whenever that expresses the change; reach for a new opcode only when framing genuinely differs.

Opcode numbers are permanent. See the Ack = 13 and ClaimViewport = 14 comments in src/shared/terminal-stream-protocol.ts for why a shipped number cannot be reused even if the feature behind it is removed.

Rule 3 — changing what the host publishes breaks old clients with no wire change

The frame shape can be untouched and the skew still real, because clients react to frame content. PR #12641 is the worked example: the host stopped synthesizing a finished agent status, and clients running older code saw different content in an identical frame.

Treat these as wire changes even though nothing in the codec moves:

a field the host stops populating (an old client reading it now sees undefined);
a value whose meaning, units, or nullability changes;
content the host stops synthesizing, trims, or starts deriving from a new source;
a frame the host stops sending, or starts sending, on an existing path.

If old clients cannot interpret the new projection correctly, gate it behind a runtime capability the same way Rule 2 gates an opcode.

Enforcement

tests/e2e/cross-version-wire/cross-version-terminal-wire.unit.test.ts runs the real host RPC methods and the real renderer multiplexer from two builds against each other — current working tree against the newest release tag, in both skew directions — over one scripted terminal journey (subscribe, input, hide/reveal snapshot, drop, reconnect).

Run it with:

pnpm exec vitest run --config config/vitest.config.ts tests/e2e/cross-version-wire/cross-version-terminal-wire.unit.test.ts

It fails when a frame is refused by the receiving build's decoder (Rule 2), when the observed frame sequence changes (Rule 3), or when published snapshot content or negotiated capabilities differ from the contract. Adding an optional field keeps it green (Rule 1); making a client depend on that field turns the new-client/old-host pairing red.

The harness covers the terminal stream only. It does not cover the session-tab sync channel, agent-session publications, file or Git RPCs, mobile/E2EE framing, or the relay transport. A change on those paths still needs its own reasoning against the three rules above.

4.6 KiB Raw Blame History