mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 16:02:32 +00:00
Jinwoo Hong
15d0f8aedf
<!-- orca-pr-loc -->
<!-- Programmatic LoC summary. Do not edit by hand; rewritten on every commit. -->
| | Files | Added | Deleted | Net |
| :--- | ---: | ---: | ---: | ---: |
| Test | 6 | $\color{#1a7f37}{\Huge{\mathbf{+}}}$544 | $\color{#cf222e}{\Huge{\mathbf{−}}}$49 | $\color{#1a7f37}{\Huge{\mathbf{+}}}$495 |
| Prod | 36 | $\color{#1a7f37}{\Huge{\mathbf{+}}}$1719 | $\color{#cf222e}{\Huge{\mathbf{−}}}$1703 | $\color{#1a7f37}{\Huge{\mathbf{+}}}$16 |
<!-- /orca-pr-loc -->
## ELI5
Orca ships eight skill guides that agents read before running the CLI. Seven of them (everything except `orchestration`, which #16904 rewrites) were command catalogs that had drifted from the binary. This PR rewrites them so an agent reads the outcome, the done bar, and the safe-failure rule first, loads reference material only at the step that needs it, and never sees a command or flag the installed CLI does not define.
## What changed
- **Seven guides rewritten** to one standard: outcome spine first (Result / Done / Safe failure), conditions instead of case lists, one done bar, one autonomy envelope, references loaded at the point of use via `skills get <topic> --full`, every runnable invocation spelled `ORCA`. `orca-cli` is 424→260 always-loaded lines with three references (browser, automations, publishing); `orca-per-workspace-env` is 794→397 with five (provider-vercel, ssh-host, docker-ssh, windows-scripts, failure-modes).
- **Defects fixed in shipped guides:** `emulator camera` (no such command), iOS `permissions` (backend refuses it), Android pane described as "in development" (shipped in June), `relayGracePeriodSeconds: 0` documented as immediate teardown (it is unbounded), doctor `ok: true` hiding `warn`, an SSH exemplar setting both `jumpHost` and `proxyCommand`, a provisioned-root fetch from `origin`, the Linear unconfirmed-write rule keyed on four verbs when ten emit it. Linear and emulator descriptions dropped embedded commands and angle-bracket placeholders (651→329, 732→404 chars).
- **Generator bundles references.** `skill-guides/<name>/references/*.md` is appended to `--full`; `skills get` help says compact by default, full with references.
- **Stubs single-authored.** The resolver ladder, placeholder rule, and older-binary fallback shared by all eight installable `SKILL.md` files come from one `skill-stubs/_shared/cli-resolution.md` fragment composed by the generator. Projections were byte-identical before the content fixes.
- **Guards:** every `ORCA <cmd>` and flag in every guide and reference resolves against `COMMAND_SPECS` (this found the camera defect); descriptions ≤1024 chars with no angle-bracket tokens; reference routing checked both directions; an always-loaded size ratchet (300 lines) that guides may leave but never join. `orchestration` (440 lines on main) is recorded as an exception until #16904 lands its kernel.
## Relationship to #16904
Split out of #16904 so that PR carries only the orchestration guide. On main, `terminal send` has no `--wait-submit` / `--retry-request` and the orchestration kernel still carries the resolver ladder and worktree-selector rule, so this branch pins `accepted: true` for handoff receipts and leaves the orchestration pins where main has them. The merge in either direction is mechanical: #16904 rebased on this becomes a one-file `orchestration.md` change plus dropping the two exceptions.
## Standard
Compound Engineering's portable skill-authoring guidance (outcome spine, conditions not cases, pinned fragile commands with an ordered hatch, references at point of use). NVIDIA SkillEvaluator Tier 1 (`schema,pii,license,quality,unicode,lint`) was run on every guide; its deterministic checks pass, its template nudges (Instructions/Examples sections, 50–150 char descriptions) do not apply to Orca's stub architecture and were not applied.
## Testing
- `pnpm typecheck:tsc:cli` clean; `check:code-quality:changed` and `check:react-doctor:changed` 0 findings
- `pnpm verify:bundled-skill-guides` and skill-bundle manifest verify clean
- vitest over `config/scripts`, `src/cli/skill-guide-cli-parity.test.ts`, `src/cli/skills.test.ts`, `src/cli/specs/skills.test.ts`, `src/cli/help.test.ts`, `src/main/skills`: 240 files / 2,019 pass
- Live smoke on the built CLI of every `skills get <topic>` and `--full`, every emulator, linear, and vm verb named in the guides, and every projection's resolver, GNOME warning, and bounded fallback (done on the #16904 branch before the split; the guide bodies are identical here except the send-receipt vocabulary noted above)
## Deferred product decisions
Merging `orca-emulator` and `orca-emulator-android` into one skill with a platform branch; collapsing `linear-tickets` to a guide alias; a `skills get --reference <name>` selector so a gate table can load one file; a fresh-agent routing eval before trimming the `orca-cli` (1,015 chars) and `orchestration` descriptions, whose quoted triggers each fixed a routing misroute.
53 lines
2.1 KiB
JavaScript
53 lines
2.1 KiB
JavaScript
import { readdirSync, readFileSync } from 'node:fs'
|
|
import { join, resolve } from 'node:path'
|
|
import { describe, expect, it } from 'vitest'
|
|
import { parse } from 'yaml'
|
|
|
|
const skillsDir = resolve(import.meta.dirname, '../../skills')
|
|
// Why: the Agent Skills spec caps `description` at 1024 chars and conforming installers
|
|
// reject the whole skill (#17935); the frontmatter is what the installer parses, so check it.
|
|
const MAX_DESCRIPTION_LENGTH = 1024
|
|
// Why raw, not backtick-stripped: NVIDIA SkillEvaluator rejects `<tag>` in a description as a
|
|
// schema error, and Cowork's validator parses descriptions as HTML and fails the whole plugin
|
|
// silently (compound-engineering #602). Neither honors backticks, so placeholders belong in the body.
|
|
const ANGLE_BRACKET_TOKEN = /<[A-Za-z][\w.-]*>/u
|
|
|
|
function readDescription(skillName) {
|
|
const skillMarkdown = readFileSync(join(skillsDir, skillName, 'SKILL.md'), 'utf8')
|
|
const frontmatter = /^---\r?\n([\s\S]*?)\r?\n---\r?\n/u.exec(skillMarkdown)?.[1]
|
|
|
|
expect(frontmatter, `${skillName}: missing frontmatter`).toBeDefined()
|
|
|
|
return parse(frontmatter ?? '').description
|
|
}
|
|
|
|
describe('bundled skill descriptions', () => {
|
|
const skillNames = readdirSync(skillsDir, { withFileTypes: true })
|
|
.filter((entry) => entry.isDirectory())
|
|
.map((entry) => entry.name)
|
|
|
|
it('discovers the bundled skills', () => {
|
|
expect(skillNames).toContain('orchestration')
|
|
})
|
|
|
|
it.each(skillNames)('%s keeps description within the Agent Skills spec limit', (name) => {
|
|
const description = readDescription(name)
|
|
|
|
expect(typeof description, `${name}: description must be a string`).toBe('string')
|
|
expect(description.trim().length, `${name}: description is empty`).toBeGreaterThan(0)
|
|
expect(
|
|
description.length,
|
|
`${name}: description is ${description.length} chars`
|
|
).toBeLessThanOrEqual(MAX_DESCRIPTION_LENGTH)
|
|
})
|
|
|
|
it.each(skillNames)('%s keeps angle-bracket placeholders out of its description', (name) => {
|
|
const token = ANGLE_BRACKET_TOKEN.exec(readDescription(name) ?? '')
|
|
|
|
expect(
|
|
token?.[0],
|
|
`${name}: rephrase or move "${token?.[0] ?? ''}" into the skill body`
|
|
).toBeUndefined()
|
|
})
|
|
})
|