mirror of
https://github.com/stablyai/orca.git
synced 2026-09-29 00:02:56 +00:00
Jinwoo-H
22ef580b0c
#19542 broke `orchestration send --to <handle>` between two terminals in no Run — the FIRST command the orchestration guide teaches — and every gate was green. The CLI tests mock the runtime client, so the runtime's throw was never produced; the RPC harness uses `:memory:` with a bound Run, so the unbound path was never taken; the guide-contract check only matched strings against ORCHESTRATION_COMMAND_SPECS. No gate ran the taught argv through a real binary against a real runtime. This spec does. It boots Electron, spawns the compiled out/cli/index.js, and runs the guide's sequence in the printed order — plain-terminal send, check, reply, restart, run-create, task-create, worker-start, the worker's own check/heartbeat/ask/escalation/worker_done, the coordinator's wait/reply/ack, retain/release, the run:/dispatch:/@group address forms, gates, low-level dispatch --inject, and worker-stop — asserting each receipt's shape rather than a zero exit. Commands run with ORCA_TERMINAL_HANDLE set instead of an added --from, because that is how the guide's argv actually runs. The second half is a drift gate: the spec records every argv it ran and compares it with the commands parsed out of skill-guides/orchestration*.md at test time. A documented command nobody executes fails, and so does a flag the spec executes that the guide never teaches. Excuses are per (verb, flag) with a stated reason, and an excuse that stops matching the guide fails too. The static contract test stays: it runs on every PR in seconds with no build and is the only gate covering the documented commands one local profile cannot reach. Both now read the guide through one parser so they cannot disagree about what is documented.
27 lines
1.3 KiB
JavaScript
27 lines
1.3 KiB
JavaScript
import { describe, expect, it } from 'vitest'
|
|
import { ORCHESTRATION_COMMAND_SPECS } from '../../src/cli/specs/orchestration'
|
|
import { readOrchestrationGuideInvocations } from './orchestration-guide-invocations'
|
|
|
|
/**
|
|
* Kept alongside tests/e2e/orchestration-guide-contract.spec.ts rather than replaced by it:
|
|
* this runs on every PR in seconds with no build, and it is the only gate that covers the
|
|
* documented commands the E2E spec cannot execute (legacy takeover, remote placement). The
|
|
* E2E spec owns the other direction — that the documented sequence still WORKS. Both read
|
|
* the guide through the same parser so the two cannot disagree about what is documented.
|
|
*/
|
|
describe('orchestration guide command contract', () => {
|
|
it('documents only orchestration verbs and flags accepted by the CLI specs', () => {
|
|
const specs = new Map(
|
|
ORCHESTRATION_COMMAND_SPECS.map((spec) => [spec.path[1], new Set(spec.allowedFlags)])
|
|
)
|
|
|
|
for (const invocation of readOrchestrationGuideInvocations()) {
|
|
const allowed = specs.get(invocation.verb)
|
|
expect(allowed, `${invocation.source}: ${invocation.verb}`).toBeDefined()
|
|
for (const flag of invocation.flags) {
|
|
expect(allowed, `${invocation.source}: ${invocation.verb} --${flag}`).toContain(flag)
|
|
}
|
|
}
|
|
})
|
|
})
|