Files
orca/config/scripts/verify-cli-bin.test.mjs
T
OrcaWinandOrca Worker 83056e5edb fix(skills): judge official skill updates by ownership, not snapshot matching (#27603)
* fix(skills): judge official skill updates by ownership, not snapshot matching

Orca decided whether an installed official skill was pristine by matching its
bytes against a shipped list of past snapshots, falling back to the skills CLI's
lock hash. Copies installed between releases, CRLF copies, and installs whose lock
recorded a sha256 hash all read "doesn't match the official version" and could not
be updated (#26841), and older content could be labelled newer and never offered.

Now an official skill is Orca's when the skills.sh global lock says it came from
stablyai/orca, and it is current only when its listed files equal the bundled
manifest (line endings and sidecars ignored); anything else is outdated. Install
and update are one pinned `skills add .../tree/v<appVersion>` command, used by the
in-app runner, `orca skills update`, and the Settings rails. The update dialog
preselects Orca-owned rows and lists same-named skills from elsewhere unchecked.

Removes the snapshot registry, release mapping, git-tree identity, lock trust and
convergence logic, and the release-cut provenance step.

* ci(skills): run the pinned-add round trip on a skills CLI that can fetch a commit

* fix(skills): let orca skills update target the shared folder on a bare host

Update became the same pinned add as install, so it inherited install's
refusal when no agent is detected. That broke the bare-host update the old
command guaranteed, and the built-CLI check (update --dry-run, no --agent) on
CI runners. The in-app runner already fell back to universal; both now share
one detectSkillsCliUpdateAgentKeys rule.

Also brings the CLI skills docs off `npx skills update`.

* refactor(skills): derive the update selection from the grouped rows

The selection hook took the live and the retained inventory, re-derived Orca
ownership from installations beside the grouping's own orcaOwned, and kept a
separate display count. Groups already carry both eligibility and ownership,
so the hook reads them once; the button's existing live-scan gate is the only
authorization check. The unselected line is inlined into the headline.

* refactor(skills): drop helpers the ownership design made redundant

- isOwnerManagedSkillScope only negated into the participation check once
  plugin caches stopped participating; fold it in.
- Eligibility re-checked resolvedPath/physicalIdentity, which observation
  already turns into 'inaccessible'.
- The manifest verifier probed access() before a readFile that throws anyway.
- orchestration-install-command.ts only forwarded one constant from the lib
  module its three importers already import.

* i18n(skills): stop naming npx skills update in four skipped-copy strings

The es/ja/ko/zh cantUpdateReason and tipDuplicate strings still named
`npx skills update`, which Orca no longer runs for its skills. Say 'the update
command' instead.

* refactor(skills): inline the single-use pinned source URL helper

orcaSkillsSourceUrl was exported but only called by the builder beside it.

* fix(skills): let an update choice lapse when the install source changes

The selection froze every checked name, Orca's preselection included, the
moment the user ticked anything. A re-check that then attributed a name to
another source kept it checked, replacing it without consent. Store only the
user's explicit choices, keyed by name and lock source, and default to
ownership otherwise.

* fix(skills): label the empty update action 'Update skills', not 'Update 0 skills'

The button stays whenever a row is updatable, so unticking every row (or a
dialog with only non-Orca copies outdated) doesn't reflow the footer. At zero
selected it stays disabled and reads 'Update skills' instead of a zero count.
The label lives beside the dialog's other count copy.

* fix(skills): let keyboard focus open the Not from Orca tooltip

The source of a non-Orca copy is only in the tooltip, and its trigger was a
plain span. Make it focusable like Orca's other tooltip-on-text triggers
(tabIndex 0, cursor-help, the row primitive's focus ring), as in the agreed
mock.

* test(skills): hold the round-trip add equal to the shared install builder

The CI round trip hardcoded its skills add argv, so it could drift from what
Orca runs. A .mjs script can't import the TS builder (extensionless imports),
so the argv moves into an exported roundTripAddArgs and a unit test asserts it
is the builder's unattended add with the tag swapped for the PR commit. The
script runs under the same main guard as the manifest generator.

* i18n(skills): translate the skill row's location count and duplicate reason

es, ko and zh still carried English for SkillUpdateRow.oneLocation and
manyLocations, and es and ko for SkillFreshnessRow.skippedReasonDuplicate.

* fix(skills): pin official skill installs to the build commit and skills CLI

`skills add .../tree/v<version>` only works for stable and RC builds: hourly,
daily and adhoc builds publish elsewhere, so stablyai/orca has no tag for their
version and both install and update failed there. Every build is made from a
commit that exists in stablyai/orca, so the build now stamps `git rev-parse HEAD`
and the one shared builder pins to `tree/<commit>`, which equals the bundled
skills by construction. Main, renderer, web and vitest get it as
ORCA_BUILD_COMMIT (replacing ORCA_APP_VERSION); the tsc-built CLI reads it from
the compiled out/package.json boundary. A missing or malformed commit fails the
build or the command with a clear error instead of building a bad URL.

The builder also pins the skills CLI (`npx skills@1.7.2`): 1.5.x cannot fetch a
commit, and a bare `npx skills` can reuse a stale cache or resolve to another
binary (#18687). The CI round-trip script reuses the same constant.

* ci(skills): run the round trip on the shared pinned skills CLI version

* docs(skills): document the commit-pinned skills add

* refactor(skills): keep the pinned skills CLI version in the install builder

skills-cli-version.ts existed only so the plain-Node round-trip script could
strip-load a .ts file, which needed --disable-warning in CI. The version now
lives beside the builder that uses it; the script keeps a literal, and the
existing round-trip test already asserts its argv, skills@<version> included,
equals the builder's.

* refactor(skills): validate the build commit once, where it is resolved

resolveBuildSourceCommit already refuses anything but a full hash, and both the
define and the CLI's stamped package.json come from it. The builder's second
check could only fire as a module-load throw in the renderer's command
constants. Also drops the resolver's unused cwd parameter.

* test: stamp a fixed build commit instead of running git in the vitest config

Tests only interpolate ORCA_BUILD_COMMIT, so they need not read HEAD (or fail
outside a git checkout) on every run.

* refactor(settings): match only skills add in the Windows npx preflight

Every official skill command is now the builder's skills@<version> add; no
rail emits skills update any more.

* fix(skills): fail an update that leaves a broken skill link behind

Eligibility already skips names with a broken link before the run, so one
afterwards is the run's own damage; report it instead of claiming success.

* ci(release): refuse to tag when the skill manifest is stale

The removed --release step implicitly checked the committed manifest against
skills/. Keep that guarantee with a verify-only run; it writes nothing.

---------

Co-authored-by: Orca Worker <orca-worker@localhost>
2026-10-11 18:30:03 -07:00

127 lines
4.6 KiB
JavaScript

import {
chmodSync,
mkdirSync,
readFileSync,
rmSync,
statSync,
writeFileSync,
mkdtempSync
} from 'node:fs'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { describe, expect, it } from 'vitest'
import { verifyPackageCliBin } from './verify-cli-bin.mjs'
/** Builds a temporary Orca-style project fixture with a compiled CLI entrypoint. */
function makeProjectWithCli(
content,
{ mode = 0o755, rootPackageType, writeOutPackageJson = true } = {}
) {
const projectDir = mkdtempSync(path.join(tmpdir(), 'orca-cli-bin-'))
const cliPath = path.join(projectDir, 'out', 'cli', 'index.js')
const outPackageJsonPath = path.join(projectDir, 'out', 'package.json')
mkdirSync(path.dirname(cliPath), { recursive: true })
writeFileSync(
path.join(projectDir, 'package.json'),
JSON.stringify({ bin: { orca: './out/cli/index.js' }, type: rootPackageType }),
'utf8'
)
if (writeOutPackageJson) {
writeFileSync(outPackageJsonPath, JSON.stringify({ type: 'commonjs' }), 'utf8')
}
writeFileSync(cliPath, content, 'utf8')
if (process.platform !== 'win32') {
chmodSync(cliPath, mode)
}
return { projectDir, cliPath, outPackageJsonPath }
}
describe('verifyPackageCliBin', () => {
it('accepts a non-empty Node entrypoint and can run help through Node', () => {
const { projectDir, cliPath } = makeProjectWithCli(
'#!/usr/bin/env node\nif (process.argv.includes("--help")) process.exit(0)\n'
)
expect(verifyPackageCliBin({ projectDir, runHelp: true })).toMatchObject({
binPath: cliPath
})
})
it('rejects an empty package bin target', () => {
const { projectDir } = makeProjectWithCli('')
expect(() => verifyPackageCliBin({ projectDir })).toThrow('bin.orca target is empty')
})
it('rejects package bin targets without a Node shebang', () => {
const { projectDir } = makeProjectWithCli('console.log("orca")\n')
expect(() => verifyPackageCliBin({ projectDir })).toThrow('Node shebang')
})
it('writes a CommonJS package boundary for the compiled CLI directory', () => {
const { projectDir, outPackageJsonPath } = makeProjectWithCli(
'#!/usr/bin/env node\nexports.ok = true\nif (process.argv.includes("--help")) process.exit(0)\n',
{ rootPackageType: 'module', writeOutPackageJson: false }
)
expect(() => verifyPackageCliBin({ projectDir, runHelp: true })).toThrow(
'compiled CLI package boundary is missing'
)
verifyPackageCliBin({ projectDir, fixPackageJson: true, runHelp: true })
expect(JSON.parse(readFileSync(outPackageJsonPath, 'utf8'))).toEqual({
name: 'orca-compiled-output',
type: 'commonjs',
private: true
})
})
it('stamps the build commit the CLI pins official skill installs to', () => {
const { projectDir, outPackageJsonPath } = makeProjectWithCli(
'#!/usr/bin/env node\nconsole.log("orca")\n',
{ writeOutPackageJson: false }
)
verifyPackageCliBin({ projectDir, fixPackageJson: true, sourceCommit: 'c'.repeat(40) })
expect(JSON.parse(readFileSync(outPackageJsonPath, 'utf8')).commit).toBe('c'.repeat(40))
rmSync(projectDir, { recursive: true, force: true })
})
it('rejects a CLI package boundary that is not CommonJS', () => {
const { projectDir, outPackageJsonPath } = makeProjectWithCli(
'#!/usr/bin/env node\nconsole.log("orca")\n'
)
writeFileSync(outPackageJsonPath, JSON.stringify({ type: 'module' }), 'utf8')
expect(() => verifyPackageCliBin({ projectDir })).toThrow('type=commonjs')
})
it.skipIf(process.platform === 'win32')('can repair the POSIX executable bit', () => {
const { projectDir, cliPath } = makeProjectWithCli(
'#!/usr/bin/env node\nconsole.log("orca")\n',
{ mode: 0o644 }
)
expect(() => verifyPackageCliBin({ projectDir })).toThrow('not executable')
verifyPackageCliBin({ projectDir, fixExecutable: true })
expect(statSync(cliPath).mode & 0o111).not.toBe(0)
expect(readFileSync(cliPath, 'utf8')).toContain('#!/usr/bin/env node')
})
it('repairs both the package boundary and POSIX executable bit together', () => {
const { projectDir, cliPath, outPackageJsonPath } = makeProjectWithCli(
'#!/usr/bin/env node\nconsole.log("orca")\n',
{ mode: 0o644, writeOutPackageJson: false }
)
verifyPackageCliBin({ projectDir, fixExecutable: true, fixPackageJson: true })
expect(JSON.parse(readFileSync(outPackageJsonPath, 'utf8')).type).toBe('commonjs')
if (process.platform !== 'win32') {
expect(statSync(cliPath).mode & 0o111).not.toBe(0)
}
rmSync(projectDir, { recursive: true, force: true })
})
})