mirror of
https://github.com/stablyai/orca.git
synced 2026-10-03 16:02:11 +00:00
Brennan BensonandBrennan Benson
11d8673112
* test(cross-version-wire): derive skew expectations from the baseline under test The cross-version wire job pairs current code against whichever release tag is newest, so a hand-written "the old side does not have X" assertion expires by itself: v1.4.192 was the first tag containing the SnapshotStart `terminalOwner` field, and cutting it turned the new-client/old-server pairing red on unrelated pull requests with no code change anywhere. Read what each build publishes from that build. Each host is now paired against a client of its own version to produce a reference, and the skewed pairings are compared against that reference, so the expectation is whatever the release actually shipped. The same class of assertion in the agent-session suite — "the old build advertises no structured capability and registers no structured method" — becomes "each build's advertisement agrees with what it registers", and the "client too old to know this capability" is derived by removing the capability from the baseline's own list. The guard is unchanged in strength: a field the old host still publishes may not be dropped, skew may not change what a host puts on the wire, and a new pairing asserts the oracle still stalls when a peer cannot decode an opcode the other side sends. * test(cross-version-wire): exercise release structured methods * test(cross-version-wire): load the registered method manifest * test(cross-version-wire): assert execution, not registration, on both host gates The release-shaped checkout gate accepted any reply that was not method_not_found, so a registered-but-throwing handler passed it. The capability gate asserted a shared host spy had been called at all, so the second method mapped to that spy could stop reaching the host unnoticed. * test(cross-version): make the release-shaped skew cover the whole agent-session manifest The release-shaped checkout is the only place the "registered means usable" claim is executable today — the baseline release registers none of these methods — and it was exercising one of sixteen. A handler registered and returning an execution error passed the suite. - Declare each method's result in the manifest, so "answered" is the contract rather than "did not say method_not_found". - Give each build a seam to install a host into its own module slot; a release checkout has its own copy, so the working tree's host was never this dispatcher's, and every host-backed method answered structured_agent_session_unsupported — the capability gate's own words. - Run one execution contract over both skews instead of two divergent loops. - Pair the AI Vault never-called spy with a positive control; renaming the runtime method it watches left it green. --------- Co-authored-by: Brennan Benson <brennanbenson@Brennans-MacBook-Pro.local>
39 lines
1.5 KiB
TypeScript
39 lines
1.5 KiB
TypeScript
/**
|
|
* Reading the field shape of a published frame from the frame itself, so a
|
|
* cross-version expectation can be stated against the build that produced it.
|
|
*
|
|
* The baseline this suite pairs against is whichever release tag is newest, and
|
|
* that moves on every cut. An expectation written as a literal list of fields the
|
|
* old side does or does not have therefore expires by itself: the first release
|
|
* containing an already-merged optional field turns the assertion red on whatever
|
|
* pull request happens to be in flight, with no code change anywhere.
|
|
*/
|
|
|
|
export type PublishedFieldSkew = {
|
|
/** Names only the newer side publishes — additive, and safe under Rule 1. */
|
|
added: string[]
|
|
/** Names the older side still publishes and the newer side dropped — a break. */
|
|
removed: string[]
|
|
}
|
|
|
|
/** Sorted union of the keys across one published frame sequence. */
|
|
export function publishedFieldNames(payloads: Record<string, unknown>[]): string[] {
|
|
return [...new Set(payloads.flatMap((payload) => Object.keys(payload)))].sort()
|
|
}
|
|
|
|
/**
|
|
* Which field names the two sides disagree on, by direction. Both sides are read
|
|
* from a real pairing; neither is a list this file knows.
|
|
*/
|
|
export function comparePublishedFields(args: {
|
|
older: string[]
|
|
newer: string[]
|
|
}): PublishedFieldSkew {
|
|
const older = new Set(args.older)
|
|
const newer = new Set(args.newer)
|
|
return {
|
|
added: [...newer].filter((name) => !older.has(name)).sort(),
|
|
removed: [...older].filter((name) => !newer.has(name)).sort()
|
|
}
|
|
}
|