Files
orca/config/scripts/smoke-packaged-cli.mjs
T
NeilandOrca fdb58695e9 [P1] fix(checks): stop skipped and manual checks reporting as failures (#11700)
* fix(checks): stop skipped and manual checks reporting as failures

Route every check-classification surface through one shared helper so
desktop renderer, desktop main and mobile agree on the same verdict.

- GitLab `manual` jobs and pipelines are neutral again, not action_required/failure
- `skipped` counts as passed everywhere, including mobile
- a neutral check no longer demotes a summary that has passing checks

* fix(checks): move the check-classification parity test into the renderer project

The parity table lived in src/shared but imported a renderer module, and both
config/tsconfig.node.json and config/tsconfig.cli.json are composite projects
that include src/shared without that renderer path, so `pnpm typecheck` failed
with TS6307 on two of its three projects. Only the web project spans both trees.

Co-authored-by: Orca <help@stably.ai>

* fix(checks): stop the Tasks-grid pill contradicting its own verdict

The checks pill's label, tone and icon all read one ProviderCheckSummary, but
getChecksLabel short-circuited on the raw `neutral` counter while the tone and
icon key off `state`. After the classification fix a PR with 19 success + 1
neutral renders an emerald CheckCircle2 pill that reads "1 unresolved", and
mobile's own label (which keys off `state`) reads "19/20 passed" for the same
summary.

Move the label into src/shared/provider-check-summary.ts so desktop and mobile
cannot fork it again, and key it off `state`.

Also covers deriveWorkItemCheckSummary, the desktop-main producer of the summary
that reaches the Tasks grid and the relay-paired mobile client. It was rewritten
here with no test at all; the parity table stands in derivePRCheckStatusFromRollup,
which is a different normalizer. The new main-process test drives getWorkItem with
a real statusCheckRollup fixture, pinning the StatusContext `state` fallback that
would otherwise be deletable with the whole suite still green.

Co-authored-by: Orca <help@stably.ai>

* fix(gitlab): route the pipeline job-array rollup through the shared check classifier

The array path in derivePipelineStatus kept its own copy of the rollup rules, so
manual-only read green and one unrecognized job status demoted a passing pipeline
to neutral — both disagreeing with every other check surface.

Also retry the packaged-CLI smoke temp cleanup on Windows: the copied Orca.exe can
still be locked by AV/indexers after every assertion passed, failing the package job.

Co-authored-by: Orca <help@stably.ai>

* fix(gitlab): stop the skipped pipeline string diverging from the Checks tab

- classifyPipelineString now counts a skipped pipeline as passing, matching
  the per-check classifier; canceled stays neutral and is pinned as an
  explicit, sign-off-pending divergence.
- Pin the production string path (head_pipeline.status) in the parity table
  and note that the job-array branch has no production caller yet.
- Count skipped checks in the Checks panel's passing header so it agrees
  with the checks pill.
- Correct the packaged-CLI smoke retry comment: the EBUSY is the smoke's own
  just-exited Electron process, not AV/indexers.

Co-authored-by: Orca <help@stably.ai>

* fix(checks): finish cross-surface check parity and back out the skipped MR-card flip

Review follow-ups on the check-classification PR.

- PullRequestPage and GitHubItemDialog kept private copies of getCheckCounts /
  getChecksSummaryLabel that still counted only `success` as passing, so a
  2-success/3-skipped PR read "2 passing · 3 skipped" there and "5 passing" in
  the sidebar. Both copies move to pr-check-counts.ts, which routes the passing
  bucket through classifyCheckOutcome; action_required keeps its own amber
  bucket. The summary icon now keys off passing count, so an all-neutral PR
  stops painting a green tick above "0 of N checks passing".
- The sidebar checks header and triage strip still called
  `{status: completed, conclusion: null}` pending, contradicting the grey
  "Unresolved checks" pill. Both now read summarizeProviderChecks and render an
  unresolved chip/strip instead of an amber spinner that can never resolve.
- classifyPipelineString('skipped') is reverted to neutral. That flip painted
  MR cards green for pipelines that never ran, on the only GitLab path with
  production callers, and contradicted the same function's deferral of
  `canceled`. Both tone changes stay deferred, pinned by one test.
- classifyPipelineString('manual') resolves to pending rather than neutral: a
  blocked pipeline is outstanding, and neutral let the worktree card fall
  through to its emerald `open` default while GitLab still refuses the merge.
- TaskPage's checks pill helpers move to task-page-checks-pill.ts so the
  "1 unresolved on a green pill" fix is actually pinned by a test.
- smoke-packaged-cli no longer lets an EBUSY cleanup replace the real failure.

* fix(checks): stop completed unknown checks from spinning

---------

Co-authored-by: Orca <help@stably.ai>
2026-07-31 04:58:15 -07:00

105 lines
3.3 KiB
JavaScript

import { cp, mkdtemp, rm } from 'node:fs/promises'
import { execFile } from 'node:child_process'
import { tmpdir } from 'node:os'
import { basename, join, resolve } from 'node:path'
import { promisify } from 'node:util'
import assert from 'node:assert/strict'
const execFileAsync = promisify(execFile)
function readAppDirArg(argv) {
const explicit = argv.find((arg) => arg.startsWith('--app-dir='))
if (explicit) {
return explicit.slice('--app-dir='.length)
}
if (process.platform === 'darwin') {
return 'dist/mac-arm64/Orca.app'
}
if (process.platform === 'win32') {
return 'dist/win-unpacked'
}
return 'dist/linux-unpacked'
}
function getPackagedCliPath(appDir) {
if (process.platform === 'darwin' || appDir.endsWith('.app')) {
return join(appDir, 'Contents', 'Resources', 'bin', 'orca')
}
if (process.platform === 'win32') {
return join(appDir, 'resources', 'bin', 'orca.exe')
}
return join(appDir, 'resources', 'bin', 'orca-ide')
}
const appDir = resolve(readAppDirArg(process.argv.slice(2)))
const tempRoot = await mkdtemp(join(tmpdir(), 'orca-packaged-cli-smoke-'))
const copiedAppDir = join(tempRoot, basename(appDir))
let smokeFailure = null
try {
await cp(appDir, copiedAppDir, { recursive: true, verbatimSymlinks: true })
const cliPath = getPackagedCliPath(copiedAppDir)
const env = { ...process.env, NODE_PATH: '' }
delete env.ORCA_CLI_CWD
const run = (args) =>
execFileAsync(cliPath, args, {
env,
killSignal: 'SIGKILL',
maxBuffer: 16 * 1024 * 1024,
timeout: 30_000
})
await run(['--help'])
const list = JSON.parse((await run(['skills', 'list', '--json'])).stdout)
assert(list.topics.some((topic) => topic.name === 'orca-cli'))
assert.match((await run(['skills', 'get', 'orca-cli'])).stdout, /name: orca-cli/)
assert.match((await run(['skills', 'get', 'computer-use'])).stdout, /name: computer-use/)
const install = JSON.parse(
(
await run([
'skills',
'install',
'--skill',
'orca-cli',
'--agent',
'codex',
'--dry-run',
'--json'
])
).stdout
)
const update = JSON.parse(
(await run(['skills', 'update', '--skill', 'orca-cli', '--dry-run', '--json'])).stdout
)
assert.equal(install.executed, false)
assert.equal(update.executed, false)
console.log(`[packaged-cli-smoke] help and skills commands passed via ${cliPath}`)
} catch (error) {
smokeFailure = error
}
// Why: on Windows the launcher above spawns the copied Orca.exe (and its crashpad/utility children)
// once per command; those handles can outlive execFile's exit by a few ms, so this cleanup hits
// EBUSY on our own just-exited process after every assertion already passed. Same retry treatment
// as removeHostTree(); a lock that never clears still throws — unless the smoke run itself failed,
// in which case surfacing EBUSY instead of the real assertion would hide the actual regression.
const cleanupFailure = await rm(tempRoot, {
recursive: true,
force: true,
maxRetries: 20,
retryDelay: 250
}).then(
() => null,
(error) => error
)
if (smokeFailure) {
if (cleanupFailure) {
console.warn(`[packaged-cli-smoke] temp cleanup failed: ${cleanupFailure.message}`)
}
throw smokeFailure
}
if (cleanupFailure) {
throw cleanupFailure
}