Files
orca/config/scripts/generate-skill-bundle-manifest.test.mjs
Brennan Benson 68fca0b076 Add safe skill freshness detection and update rail (#8637)
* Add safe skill freshness detection

* Accept observed copy-mode rail outcomes

* chore(skills): regenerate snapshot artifacts for the merged guide content

The rebase onto main picked up the reviewed guide fixes (#8624), so the
current manifest hashes and a new appended snapshot generation must
match those bytes; the registry keeps all prior snapshots so existing
installs classify as outdated rather than unrecognized.

* fix(skills): canonicalize snapshot file order and guard released history

Historical snapshots kept git ls-tree byte-order while the working-tree
walk and runtime observation use the sorted depth-first order, so any
future multi-file skill would misclassify older installs as unrecognized
and churn spurious registry revisions; all producers now share one
canonical order (no digest changes for today's single-file packages).
Also rejects executable files from shipped skills (Windows observation
cannot see execute bits, which would misclassify pristine Windows
installs) and adds an explicit append-only invariant for released
snapshots so a generation-logic change cannot rewrite them silently.

* fix(skills): throttle focus rescans and correct self-blocked placement copy

Every window focus re-read and re-hashed all installed packages, and the
nudge and panel each forced their own trailing rescan for one event; a
15s cooldown plus a shared invalidation latch keep one bounded scan per
event while install-change events stay immediate. Bundle artifacts are
now loaded once per run instead of re-parsed on every scan. A read-only
or otherwise unsupported outdated placement now explains that it blocks
itself instead of blaming a phantom sibling placement; the supported
topology set moved to shared so eligibility and copy cannot drift.

* feat(skills): move freshness surfacing to a lingering toast and update modal

The Skills page has been unreachable since its toolbox menu entry was
removed (#4535), so surfacing freshness there buried the feature behind
its own nudge. The nudge now lingers until acted on (ignoring it records
nothing; only the explicit close persists dismissal keys) and opens an
update modal hosting the pre-filled editable terminal, an honest
current/blocked summary, and the per-placement rows in a collapsed
Details section. A compact 'Check for skill updates' row in CLI settings
is the manual re-entry point. Skills page restored to main; design-doc
surfacing section records the venue decision.

* fix(skills): mount update dialog inside the link-routing provider and fold freshness into the setup rails

The dialog hosts a live terminal pane that requires the link-routing
preference context; mounted outside the provider it crashed the renderer
the moment an eligible update existed (caught by live QA — unit tests
mock the terminal). It now mounts inside the provider behind its own
recoverable boundary.

The separate 'Check for skill updates' settings button is gone: the
setup rails' own pill now carries freshness (Update available / Up to
date, falling back to Installed for blocked or unrecognized copies and
for non-local runtimes the local-only scan cannot vouch for), and
Re-check refreshes both installation detection and the freshness
inventory. Wired for the CLI, Orchestration, Computer Use, and
Per-Workspace Environments rails.

* fix(skills): use the sleek scrollbar style in the update dialog

* chore(skills): regenerate manifest for merged main (v1.4.142-rc.1)

Main advanced to 1.4.142-rc.1 with a v1.4.141 release, so the embedded
appVersion and release mapping were stale on the PR's merged tree. Only
appVersion and the new release entry change; no snapshot digests move
(released history preserved).

* fix(skills): bound and batch freshness work

* fix(skills): harden freshness integrity checks

* fix(skills): accept observed copy topology outcomes

* chore(skills): regenerate manifest for current main

* fix(skills): preserve update terminal lifecycle

* chore(skills): regenerate manifest for current main

* fix(skills): fail closed on stale freshness scans

* chore(skills): regenerate manifest for current main

* fix(skills): preserve freshness safety under focus churn

* feat(skills): group the update modal by skill with plain-language status

The Update skills modal now lists only skills that will update or that can't
(with why), grouped by skill with their install locations nested underneath —
no more one row per placement.

- Statuses collapse to "Update available" / "Can't update" at the skill level.
- A location's problem is a chip (Duplicate, Unrecognized, Inaccessible, Read
  only, In a repo, External/Broken link, Plugin cache) with a hover tooltip
  that explains what it means for the user and what to do.
- Up-to-date, unrecognized-only, and unreadable-only skills are hidden; a
  current/unrecognized/etc. location only appears when it explains a shown skill.
- Copy is de-jargoned (drops "copy"/"placement"/"snapshot"/"official copy") and
  names the mechanism as the npx skills update command, not "Orca's update".
- Rename the section to "Update details"; drop the unreachable newer-known state.

Renderer-only: derivation is a pure module (groupSkillFreshness) with unit tests;
no IPC or main-process change. Locales updated for all five languages.

* chore(skills): regenerate manifest for current main (v1.4.143-rc.0)

* feat(skills): don't let a duplicate block the update; clearer skipped copy

- Eligibility: a clean standalone duplicate no longer poisons the whole
  name — the canonical copy still updates and the duplicate is flagged;
  a duplicate-only skill stays unoffered.
- Update modal: "Can't update" -> "Skipped" with a reason-specific
  sentence (edited/read-only/in-a-repo/plugin/link); chips describe only
  the location state; footer "Check now" -> "Re-check".
- Settings sidebar nav pills go amber "Update available" when a skill is
  updatable, matching the setup cards.
- Localized new strings across en/es/ja/ko/zh.

* chore(skills): regenerate manifest for merged main (v1.4.144-rc.1)
2026-07-16 14:47:28 -07:00

200 lines
7.2 KiB
JavaScript

import { execFileSync } from 'node:child_process'
import { chmod, mkdir, mkdtemp, rm, symlink, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import {
assertReleasedHistoryPreserved,
classifyFile,
collectPackageFiles,
describeFile,
gitTreeSha,
normalizeText,
packageDigest,
sortManifestFiles
} from './generate-skill-bundle-manifest.mjs'
const temporaryDirectories = []
async function createPackage() {
const directory = await mkdtemp(path.join(tmpdir(), 'orca-skill-manifest-'))
temporaryDirectories.push(directory)
return directory
}
afterEach(async () => {
await Promise.all(
temporaryDirectories.splice(0).map((directory) => rm(directory, { recursive: true }))
)
})
describe('skill bundle manifest generator', () => {
it('folds platform line endings for text identity', () => {
const lf = Buffer.from('first\nsecond\n')
const crlf = Buffer.from('first\r\nsecond\r\n')
expect(classifyFile(lf)).toBe('text')
expect(normalizeText(crlf)).toEqual(lf)
})
it('classifies null-containing and invalid UTF-8 content as binary', () => {
expect(classifyFile(Buffer.from([0, 1, 2]))).toBe('binary')
expect(classifyFile(Buffer.from([0xc3, 0x28]))).toBe('binary')
})
it('uses normalized text identity but exact executable identity', () => {
const skillFile = describeFile('SKILL.md', Buffer.from('line one\r\nline two\r\n'), false)
const executable = describeFile('run.sh', Buffer.from('#!/bin/sh\r\necho ok\r\n'), true)
expect(skillFile.identitySha256).toBe(skillFile.textNormalizedSha256)
expect(skillFile.identitySha256).not.toBe(skillFile.exactSha256)
expect(executable.exactSha256).not.toBe(executable.textNormalizedSha256)
expect(executable.identitySha256).toBe(executable.exactSha256)
expect(packageDigest([skillFile, executable])).toMatch(/^[a-f0-9]{64}$/)
})
it('orders git-history files identically to the filesystem walk', async () => {
const packageRoot = await createPackage()
await mkdir(path.join(packageRoot, 'sub'))
for (const name of ['apple.md', 'sub.md', 'Zebra.md', path.join('sub', 'inner.txt')]) {
await writeFile(path.join(packageRoot, name), `${name}\n`)
}
const walked = await collectPackageFiles(packageRoot)
// Why: git ls-tree emits [Zebra.md, apple.md, sub.md, sub/inner.txt]; index-based
// snapshot matching requires history and observation to share one order.
const gitOrdered = ['Zebra.md', 'apple.md', 'sub.md', 'sub/inner.txt'].map((manifestPath) =>
walked.find((file) => file.path === manifestPath)
)
expect(sortManifestFiles(gitOrdered)).toEqual(walked)
expect(packageDigest(sortManifestFiles(gitOrdered))).toBe(packageDigest(walked))
expect(walked.map((file) => file.path)).toEqual([
'Zebra.md',
'apple.md',
'sub/inner.txt',
'sub.md'
])
})
it('rejects rewrites of released snapshots and allows floating-tail replacement', () => {
const snapshot = (releaseRevision, packageDigest) => ({ releaseRevision, packageDigest })
const artifacts = {
releasedSnapshotCounts: { 'orca-cli': 2 },
snapshotRegistry: {
schemaVersion: 1,
skills: { 'orca-cli': [snapshot(1, 'aaa'), snapshot(2, 'bbb'), snapshot(3, 'ccc')] }
}
}
expect(() =>
assertReleasedHistoryPreserved(
{ schemaVersion: 1, skills: { 'orca-cli': [snapshot(1, 'aaa'), snapshot(2, 'bbb')] } },
artifacts
)
).not.toThrow()
expect(() =>
assertReleasedHistoryPreserved(
{
schemaVersion: 1,
skills: { 'orca-cli': [snapshot(1, 'aaa'), snapshot(2, 'bbb'), snapshot(3, 'stale')] }
},
artifacts
)
).not.toThrow()
expect(() =>
assertReleasedHistoryPreserved(
{
schemaVersion: 1,
skills: { 'orca-cli': [snapshot(1, 'aaa'), snapshot(2, 'rewritten')] }
},
artifacts
)
).toThrow('Released snapshot history changed for orca-cli at revision 2')
expect(() =>
assertReleasedHistoryPreserved(
{
schemaVersion: 1,
skills: {
'orca-cli': [snapshot(1, 'aaa'), { ...snapshot(2, 'bbb'), gitTreeSha: 'rewritten' }]
}
},
artifacts
)
).toThrow('Released snapshot history changed for orca-cli at revision 2')
expect(() =>
assertReleasedHistoryPreserved(
{
schemaVersion: 1,
skills: {
'orca-cli': [snapshot(1, 'aaa'), snapshot(2, 'bbb'), snapshot(3, 'stale')]
}
},
{ ...artifacts, releasedSnapshotCounts: { 'orca-cli': 1 } }
)
).toThrow('Released snapshot history is incomplete for orca-cli')
expect(() => assertReleasedHistoryPreserved(null, artifacts)).not.toThrow()
})
it.runIf(process.platform !== 'win32')(
'rejects executable files in shipped skill packages',
async () => {
const packageRoot = await createPackage()
await writeFile(path.join(packageRoot, 'SKILL.md'), 'skill\n')
await writeFile(path.join(packageRoot, 'run.sh'), '#!/bin/sh\necho ok\n')
await chmod(path.join(packageRoot, 'run.sh'), 0o755)
await expect(collectPackageFiles(packageRoot)).rejects.toThrow(
'Executable file is not allowed in a shipped skill: run.sh'
)
}
)
it.runIf(process.platform === 'linux')('rejects case-colliding paths', async () => {
const packageRoot = await createPackage()
await writeFile(path.join(packageRoot, 'SKILL.md'), 'skill')
await writeFile(path.join(packageRoot, 'Readme.md'), 'one')
await writeFile(path.join(packageRoot, 'README.md'), 'two')
await expect(collectPackageFiles(packageRoot)).rejects.toThrow('Case-colliding skill paths')
})
it.runIf(process.platform !== 'win32')('rejects symlinks inside shipped packages', async () => {
const packageRoot = await createPackage()
await writeFile(path.join(packageRoot, 'SKILL.md'), 'skill')
await symlink('SKILL.md', path.join(packageRoot, 'linked.md'))
await expect(collectPackageFiles(packageRoot)).rejects.toThrow(
'Symlink is not allowed in a shipped skill'
)
})
it('computes the same Git tree identity as Git', async () => {
const packageRoot = path.resolve('skills', 'orca-cli')
const files = await collectPackageFiles(packageRoot)
const expected = execFileSync('git', ['ls-tree', 'HEAD:skills', 'orca-cli'], {
encoding: 'utf8'
})
.trim()
.split(/\s+/)[2]
expect(gitTreeSha(files)).toBe(expected)
})
it('matches Git when a directory and file share a name prefix', async () => {
const packageRoot = await createPackage()
await mkdir(path.join(packageRoot, 'sub'))
await writeFile(path.join(packageRoot, 'sub', 'inner.txt'), 'nested\n')
await writeFile(path.join(packageRoot, 'sub.md'), 'sibling\n')
const files = await collectPackageFiles(packageRoot)
execFileSync('git', ['init', '--quiet'], { cwd: packageRoot })
execFileSync('git', ['add', '-A'], { cwd: packageRoot })
const expected = execFileSync('git', ['write-tree'], {
cwd: packageRoot,
encoding: 'utf8'
}).trim()
expect(gitTreeSha(files)).toBe(expected)
})
})