Files
orca/src/main/text-generation/commit-message-text-generation.test.ts
T
JinjingandOrca ca70be8318 Add {linkedIssue} template variable for commit and PR generation (#10640)
* feat(source-control-ai): add {linkedIssue} recipe variable for commit and PR prompts

Custom commit-message and pull-request recipes can now reference the GitHub
issue linked to the workspace, so a template like "Fixes #{linkedIssue}" lands
the closing trailer without the user retyping the number.

- register `linkedIssue` on the commitMessage and pullRequest actions only,
  with the VARIABLE_INFO entry the chip hover card requires
- substitute unconditionally via `formatLinkedIssueTemplateValue` (empty string
  when nothing resolves) so the token never survives into a prompt; enrich the
  draft context conditionally via `withLinkedIssueDraftContext` so unlinked
  workspaces keep their existing context shape
- attach at the 7 call boundaries (runtime commit x2, runtime PR shared, IPC
  commit x2, IPC PR x2); the pure git gather stays pure
- validate the renderer-supplied worktreeId against the request path and repoId
  before any meta read, comparing SSH paths as raw strings so a Windows host
  cannot rewrite a remote POSIX path
- built-in prompts are unchanged; no GitLab dual-read and no default trailer

* fix(source-control-ai): resolve {linkedIssue} adversarial review findings

Addresses 13 of the 14 findings from the {linkedIssue} code review
(6 minor, 8 nit, 0 critical, 0 major); Issue 5 (GitLab provider naming)
is deferred to design Open Question 3 as product expansion.

Behavior:
- Dialog previews the workspace's real linked issue instead of the
  synthetic 123, in both the chip hover card and the plan preview, so an
  unlinked workspace previews the `Fixes #` it will actually generate.
  Settings dry-runs stay fully synthetic.
- Reject non-positive, fractional and unsafe-integer issue numbers at the
  IPC resolver via a shared isLinkedIssueNumber predicate, so corrupt meta
  never reaches a draft context (previously -7 rendered `Fixes #-7` and
  1e21 rendered `Fixes #1e+21`).
- Fail closed on an empty-string repoId instead of skipping the cross-check.

Structure:
- Split the variable registry into source-control-ai-action-variables.ts
  and re-export it, restoring max-lines headroom with no consumer churn
  and no lint disable.
- Constrain withLinkedIssueDraftContext to contexts declaring linkedIssue.
- Move the misplaced shared imports into their import group.

Docs and tests:
- Document that the IPC id/path validator guards relay/CLI/future callers,
  not the renderer (whose path is id-derived), and rename the three tests
  that read as proof of a protection that cannot fire.
- Add PR-side coverage that was missing: three git:generatePullRequestFields
  handler tests, a built-in PR prompt no-leak guard, and the runtime PR
  unlinked case.
- Replace the coincidental '42' assertion with a fixture-unique sentinel.
- Type the runtime worktree fixture with satisfies, which surfaced and
  fixed pre-existing drift in its git sub-object.
- Add an e2e case covering the preload -> main -> meta -> template chain.

Co-authored-by: Orca <help@stably.ai>

* fix(source-control-ai): resolve {linkedIssue} adversarial re-review findings

Addresses all 8 findings from the {linkedIssue} code re-review
(2 minor, 6 nit, 0 critical, 0 major); none deferred.

Behavior:
- Revert the variableOverrides parameter on planSourceControlTextGeneration.
  Its result is a Save/Generate gate, not a preview, and the recipe it
  validates is saved repo- or globally scoped -- so rendering it against the
  active workspace disabled both buttons with "Command input is empty." for a
  {linkedIssue}-only template on any unlinked workspace, blocking a global
  settings write. Validation is synthetic again; chip previews are unchanged.
- Make the chip hover card additive instead of either/or. A supplied preview
  now appends a "This workspace" sample below the description and Example
  rather than replacing them, so the GitLab-empty and dangling `Fixes #`
  warning survives on the two dialogs where recipes are actually authored.
  basePrompt keeps its preview-only shape, where the preview is the content.

Structure:
- Drop the registry re-export from source-control-ai-actions.ts and move the
  last two consumers onto source-control-ai-action-variables, so one import
  path per symbol keeps a grep of the registry's consumers complete.
- Split the registry/helper suites into source-control-ai-action-variables.test.ts
  so each test file mirrors its module.

Tests:
- Cover the Save/Generate gate at the canRunGeneration level for a bare
  {linkedIssue} recipe on linked and unlinked workspaces, with a negative
  control proving the buttons can still be disabled.
- Cover the chip hover card directly; the dialog tests mock it away.
- Guard the PR mismatched-id test with toHaveLength(1) so it cannot pass
  vacuously on an unrelated early return.
- Add an unlinked-workspace e2e case (saw-issue:empty), which is what
  distinguishes a real resolver from one that always returns a number.
  Spec now runs green: 3 passed.
- Rename the dialog test that claimed a synthetic-fallback assertion it did
  not make, and route its renders through one shared helper.

Docs are worktree-local (.gitignore:84 ignores docs/**): the design doc's
plan-preview and chip-surface claims, the manual QA rows, and both reviews'
statements about pre-existing PR-handler tests are corrected there.

* fix(source-control-ai): make the {linkedIssue} e2e guard and dialog test falsifiable

The e2e unlinked case extracted the echoed issue with `ORCA_E2E_ISSUE=(\d*)`,
which matches zero digits in front of an unexpanded `{linkedIssue}` and reported
it as `empty` — so the case that exists to catch a literal token surviving into
a prompt passed on exactly that regression. Capture the whole line instead: a
literal now arrives as `saw-issue:{linkedIssue}` and fails, verified by dropping
the substitution key for unlinked contexts and watching the case go red.

Also drop the inert `not.toContain('Command input is empty.')` assertion — that
copy is click-driven `generationError` state and this suite renders statically,
so it could never fail; the claim it reached for is carried by the plan test.
Rename two plan tests off the "plan preview" framing the design now rejects.

Local review artifacts (design doc, implementation notes, final review) were
swept to match the tree in the same pass; they are gitignored here.

* Resolve {linkedIssue} from live metadata, not cache

Resolved worktrees are cached for a second, causing commit and PR
generation to use stale linked-issue state. Hosts now implement
getWorktreeLinkedIssue to provide fresh issue metadata by worktree id,
with proper fallback for unlinked workspaces. Updates both commit
message and PR field generation paths; includes integration and e2e
coverage.

* Keep cached linkedIssue when metadata is unavailable

Return undefined from getWorktreeLinkedIssue when live metadata cannot be read
(store not ready), distinguishing it from null (unlinked). The caller now falls
back to the cached worktree value instead of treating unavailable as unlinked.

Also extract the linked-issue echo generator as a shared e2e test helper.

---------

Co-authored-by: Orca <help@stably.ai>
2026-07-25 19:31:44 -07:00

2060 lines
60 KiB
TypeScript

/* eslint-disable max-lines -- Why: local/remote generation, cancellation, and
env propagation share subprocess mocks; splitting would obscure the
cross-path invariants these tests protect. */
import { exec, spawn } from 'node:child_process'
import type * as ChildProcess from 'node:child_process'
import { EventEmitter } from 'node:events'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { getDefaultSettings } from '../../shared/constants'
import { sourceControlAiSettingsFromLegacy } from '../../shared/source-control-ai'
import type { GlobalSettings } from '../../shared/types'
import {
cancelGenerateCommitMessageLocal,
cancelGeneratePullRequestFieldsLocal,
discoverCommitMessageModelsLocal,
discoverCommitMessageModelsRemote,
generateBranchNameFromContext,
generateCommitMessageFromContext,
generatePullRequestFieldsFromContext,
resolveCommitMessageSettings,
trimGeneratedCommitMessage
} from './commit-message-text-generation'
vi.mock('child_process', async (importOriginal) => {
const actual = await importOriginal<typeof ChildProcess>()
return {
...actual,
exec: vi.fn((_command, callback) => {
if (typeof callback === 'function') {
callback(null, '', '')
}
return new actual.ChildProcess()
}),
spawn: vi.fn(actual.spawn)
}
})
const execMock = vi.mocked(exec)
const spawnMock = vi.mocked(spawn)
type MockDiscoveryChild = EventEmitter & {
pid: number
kill: ReturnType<typeof vi.fn>
stdout: EventEmitter
stderr: EventEmitter
stdin: { end: ReturnType<typeof vi.fn> }
}
function createMockDiscoveryChild(): MockDiscoveryChild {
const child = new EventEmitter() as MockDiscoveryChild
child.pid = 123
child.kill = vi.fn()
child.stdout = new EventEmitter()
child.stderr = new EventEmitter()
child.stdin = { end: vi.fn() }
return child
}
function syncSourceControlAiFromLegacy(settings: GlobalSettings): void {
settings.sourceControlAi = sourceControlAiSettingsFromLegacy(settings.commitMessageAi)
}
function withPlatform<T>(platform: NodeJS.Platform, fn: () => T): T {
const original = process.platform
Object.defineProperty(process, 'platform', { configurable: true, value: platform })
try {
return fn()
} finally {
Object.defineProperty(process, 'platform', { configurable: true, value: original })
}
}
function expectChildTerminated(child: { pid: number; kill: ReturnType<typeof vi.fn> }): void {
if (process.platform === 'win32') {
expect(execMock).toHaveBeenCalledWith(`taskkill /pid ${child.pid} /T /F`, expect.any(Function))
expect(child.kill).not.toHaveBeenCalled()
return
}
expect(child.kill).toHaveBeenCalledWith('SIGKILL')
}
beforeEach(() => {
execMock.mockClear()
spawnMock.mockClear()
})
describe('resolveCommitMessageSettings', () => {
it('falls back when a dynamic persisted model was not discovered', () => {
const settings = getDefaultSettings('/tmp')
settings.enableGitHubAttribution = true
settings.commitMessageAi = {
enabled: true,
agentId: 'codex',
selectedModelByAgent: { codex: 'retired-model' },
selectedThinkingByModel: {},
customPrompt: 'Use Conventional Commits.',
customAgentCommand: ''
}
settings.sourceControlAi = undefined
const result = resolveCommitMessageSettings(settings)
expect(result).toEqual({
ok: true,
params: {
agentId: 'codex',
model: 'gpt-5.5',
thinkingLevel: 'low',
customPrompt: 'Use Conventional Commits.',
commandInputTemplate: '{basePrompt}\n\nUse Conventional Commits.'
}
})
})
it('falls back from stale Claude version ids to the CLI alias default', () => {
const settings = getDefaultSettings('/tmp')
settings.commitMessageAi = {
enabled: true,
agentId: 'claude',
selectedModelByAgent: { claude: 'claude-sonnet-4-6' },
selectedThinkingByModel: { sonnet: 'low' },
customPrompt: '',
customAgentCommand: ''
}
syncSourceControlAiFromLegacy(settings)
const result = resolveCommitMessageSettings(settings)
expect(result).toMatchObject({
ok: true,
params: {
agentId: 'claude',
model: 'sonnet',
thinkingLevel: 'low'
}
})
})
it("uses the user's default agent when the AI setting has no explicit agent", () => {
const settings = getDefaultSettings('/tmp')
settings.defaultTuiAgent = 'codex'
const result = resolveCommitMessageSettings(settings)
expect(result).toMatchObject({
ok: true,
params: {
agentId: 'codex',
model: 'gpt-5.5',
thinkingLevel: 'low'
}
})
})
it('preserves dynamic persisted models that were discovered by the CLI', () => {
const settings = getDefaultSettings('/tmp')
settings.commitMessageAi = {
enabled: true,
agentId: 'cursor',
selectedModelByAgent: { cursor: 'gpt-5.2' },
discoveredModelsByAgent: {
cursor: [
{
id: 'gpt-5.2',
label: 'GPT 5.2',
thinkingLevels: [{ id: 'xhigh', label: 'Extra High' }],
defaultThinkingLevel: 'xhigh'
}
]
},
selectedThinkingByModel: { 'gpt-5.2': 'xhigh' },
customPrompt: '',
customAgentCommand: ''
}
syncSourceControlAiFromLegacy(settings)
const result = resolveCommitMessageSettings(settings)
expect(result).toMatchObject({
ok: true,
params: {
agentId: 'cursor',
model: 'gpt-5.2',
thinkingLevel: 'xhigh'
}
})
})
it('uses host-scoped discovered models for SSH worktrees', () => {
const settings = getDefaultSettings('/tmp')
settings.commitMessageAi = {
enabled: true,
agentId: 'cursor',
selectedModelByAgent: { cursor: 'auto' },
selectedModelByAgentByHost: { 'ssh:conn-1': { cursor: 'remote-only' } },
discoveredModelsByAgent: { cursor: [{ id: 'auto', label: 'Auto' }] },
discoveredModelsByAgentByHost: {
'ssh:conn-1': { cursor: [{ id: 'remote-only', label: 'Remote Only' }] }
},
selectedThinkingByModel: {},
customPrompt: '',
customAgentCommand: ''
}
syncSourceControlAiFromLegacy(settings)
const result = resolveCommitMessageSettings(settings, 'ssh:conn-1')
expect(result).toMatchObject({
ok: true,
params: {
agentId: 'cursor',
model: 'remote-only'
}
})
})
it('falls back to the model default thinking level when a persisted level is stale', () => {
const settings = getDefaultSettings('/tmp')
settings.commitMessageAi = {
enabled: true,
agentId: 'codex',
selectedModelByAgent: { codex: 'gpt-5.4-mini' },
selectedThinkingByModel: { 'gpt-5.4-mini': 'turbo' },
customPrompt: '',
customAgentCommand: ''
}
syncSourceControlAiFromLegacy(settings)
const result = resolveCommitMessageSettings(settings)
expect(result).toMatchObject({
ok: true,
params: {
agentId: 'codex',
model: 'gpt-5.4-mini',
thinkingLevel: 'low'
}
})
})
it('passes the per-agent command override into non-interactive planning', () => {
const settings = getDefaultSettings('/tmp')
settings.agentCmdOverrides.codex = 'npx codex'
settings.commitMessageAi = {
enabled: true,
agentId: 'codex',
selectedModelByAgent: { codex: 'gpt-5.4-mini' },
selectedThinkingByModel: {},
customPrompt: '',
customAgentCommand: ''
}
syncSourceControlAiFromLegacy(settings)
const result = resolveCommitMessageSettings(settings)
expect(result).toMatchObject({
ok: true,
params: {
agentId: 'codex',
agentCommandOverride: 'npx codex'
}
})
})
it('falls back when persisted thinking belongs to an undiscovered dynamic model', () => {
const settings = getDefaultSettings('/tmp')
settings.commitMessageAi = {
enabled: true,
agentId: 'cursor',
selectedModelByAgent: { cursor: 'gpt-5.2' },
selectedThinkingByModel: { 'gpt-5.2': 'xhigh' },
customPrompt: '',
customAgentCommand: ''
}
syncSourceControlAiFromLegacy(settings)
const result = resolveCommitMessageSettings(settings)
expect(result).toMatchObject({
ok: true,
params: {
agentId: 'cursor',
model: 'auto'
}
})
})
it('requires a non-empty custom command for custom agents', () => {
const settings = getDefaultSettings('/tmp')
settings.commitMessageAi = {
enabled: true,
agentId: 'custom',
selectedModelByAgent: {},
selectedThinkingByModel: {},
customPrompt: '',
customAgentCommand: ' '
}
syncSourceControlAiFromLegacy(settings)
expect(resolveCommitMessageSettings(settings)).toEqual({
ok: false,
error: 'Custom command is empty. Add one in Settings -> Git -> Source Control AI.'
})
})
})
describe('discoverCommitMessageModelsLocal', () => {
it('returns static catalog models without spawning for static agents', async () => {
const result = await discoverCommitMessageModelsLocal('amp', undefined)
expect(result).toMatchObject({
success: true,
defaultModelId: 'smart'
})
expect(spawnMock).not.toHaveBeenCalled()
})
it('discovers dynamic models through the agent CLI', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = discoverCommitMessageModelsLocal('cursor', undefined)
listeners.get('stdout:data')?.(Buffer.from('auto - Auto\ngpt-5.2 - GPT-5.2\n'))
listeners.get('close')?.(0)
await expect(pending).resolves.toMatchObject({
success: true,
defaultModelId: 'auto',
models: [
{ id: 'auto', label: 'Auto' },
{ id: 'gpt-5.2', label: 'GPT-5.2' }
]
})
expect(spawnMock).toHaveBeenCalledWith(
'cursor-agent',
['--list-models'],
expect.objectContaining({ windowsHide: true })
)
})
it('discovers dynamic models through the configured agent command override', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = discoverCommitMessageModelsLocal('cursor', undefined, 'npx cursor-agent')
listeners.get('stdout:data')?.(Buffer.from('auto - Auto\n'))
listeners.get('close')?.(0)
await expect(pending).resolves.toMatchObject({
success: true,
defaultModelId: 'auto'
})
if (process.platform === 'win32') {
expect(spawnMock).toHaveBeenCalledWith(
expect.stringMatching(/cmd\.exe$/i),
['/d', '/c', expect.stringMatching(/npx\.cmd$/i), 'cursor-agent', '--list-models'],
expect.objectContaining({ windowsHide: true })
)
} else {
expect(spawnMock).toHaveBeenCalledWith(
'npx',
['cursor-agent', '--list-models'],
expect.objectContaining({ windowsHide: true })
)
}
})
it('discovers dynamic models through the selected WSL distro login shell', async () => {
await withPlatform('win32', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = discoverCommitMessageModelsLocal('cursor', undefined, undefined, {
cwd: 'C:\\repo',
wslDistro: 'Ubuntu'
})
listeners.get('stdout:data')?.(Buffer.from('auto - Auto\n'))
listeners.get('close')?.(0)
await expect(pending).resolves.toMatchObject({
success: true,
defaultModelId: 'auto'
})
expect(spawnMock).toHaveBeenCalledWith(
'wsl.exe',
['-d', 'Ubuntu', '--', 'sh', '-lc', expect.any(String)],
expect.objectContaining({
cwd: undefined,
windowsHide: true
})
)
const shellCommand = spawnMock.mock.calls[0]?.[1]?.[5] as string
expect(shellCommand).toContain('getent passwd')
expect(shellCommand).toContain('/mnt/c/repo')
expect(shellCommand).toContain("'cursor-agent'")
expect(shellCommand).toContain('--list-models')
})
})
it('falls back to static models when dynamic discovery returns no parseable models', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = discoverCommitMessageModelsLocal('pi', undefined)
listeners.get('stdout:data')?.(Buffer.from('provider model\n'))
listeners.get('close')?.(0)
await expect(pending).resolves.toMatchObject({
success: true,
defaultModelId: 'github-copilot/gpt-5.4-mini',
models: [{ id: 'github-copilot/gpt-5.4-mini' }]
})
})
it('parses Pi model discovery from stderr when the CLI exits successfully', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = discoverCommitMessageModelsLocal('pi', undefined)
listeners.get('stderr:data')?.(
Buffer.from(
[
'provider model context max-out thinking images',
'github-copilot gpt-5.4-mini 400K 128K yes yes',
'openai-codex gpt-5.5 272K 128K yes yes'
].join('\n')
)
)
listeners.get('close')?.(0)
await expect(pending).resolves.toMatchObject({
success: true,
defaultModelId: 'github-copilot/gpt-5.4-mini',
models: [{ id: 'github-copilot/gpt-5.4-mini' }, { id: 'openai-codex/gpt-5.5' }]
})
})
it('settles and detaches model discovery when timeout kill is ignored', async () => {
vi.useFakeTimers()
const child = createMockDiscoveryChild()
spawnMock.mockReturnValue(child as never)
try {
const pending = discoverCommitMessageModelsLocal('cursor', undefined)
const assertion = expect(pending).resolves.toMatchObject({
success: false,
error: 'Cursor model discovery timed out after 60s.'
})
await vi.advanceTimersByTimeAsync(60_000)
await assertion
expectChildTerminated(child)
expect(child.stdout.listenerCount('data')).toBe(0)
expect(child.stderr.listenerCount('data')).toBe(0)
expect(child.listenerCount('error')).toBe(0)
expect(child.listenerCount('close')).toBe(0)
} finally {
vi.useRealTimers()
}
})
it('settles and detaches model discovery when output exceeds the limit', async () => {
const child = createMockDiscoveryChild()
spawnMock.mockReturnValue(child as never)
const pending = discoverCommitMessageModelsLocal('cursor', undefined)
child.stdout.emit('data', Buffer.alloc(4 * 1024 * 1024 + 1))
await expect(pending).resolves.toMatchObject({
success: false,
error: 'Cursor returned too much model data.'
})
expectChildTerminated(child)
expect(child.stdout.listenerCount('data')).toBe(0)
expect(child.stderr.listenerCount('data')).toBe(0)
expect(child.listenerCount('error')).toBe(0)
expect(child.listenerCount('close')).toBe(0)
})
})
describe('generateCommitMessageFromContext', () => {
it('discovers dynamic models through a remote execution plan', async () => {
const execute = vi.fn(async (plan, cwd, timeoutMs) => {
expect(plan).toEqual({
binary: 'npx',
args: ['cursor-agent', '--list-models'],
stdinPayload: null,
label: 'Cursor'
})
expect(cwd).toBe('/remote/repo')
expect(timeoutMs).toBe(60_000)
return {
stdout: 'auto - Auto\ngpt-5.2 - GPT-5.2\n',
stderr: '',
exitCode: 0,
timedOut: false
}
})
const result = await discoverCommitMessageModelsRemote(
'cursor',
'/remote/repo',
execute,
'npx cursor-agent'
)
expect(result).toMatchObject({
success: true,
defaultModelId: 'auto',
models: [
{ id: 'auto', label: 'Auto' },
{ id: 'gpt-5.2', label: 'GPT-5.2' }
]
})
})
it('reports remote model discovery spawn failures with remote install guidance', async () => {
const result = await discoverCommitMessageModelsRemote('cursor', '/remote/repo', async () => ({
stdout: '',
stderr: '',
exitCode: null,
timedOut: false,
spawnError: 'ENOENT'
}))
expect(result).toEqual({
success: false,
error: 'cursor-agent not found on the remote PATH. Install Cursor there.'
})
})
it('uses a prepared remote execution plan instead of running git on the remote side', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent --message {prompt}'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async (plan, cwd, timeoutMs) => {
expect(cwd).toBe('/repo')
expect(timeoutMs).toBe(60_000)
expect(plan.binary).toBe('agent')
expect(plan.args).toHaveLength(2)
expect(plan.args[0]).toBe('--message')
expect(plan.args[1]).toContain('Staged files:\nM\tREADME.md')
return {
stdout: 'Add README note.\n',
stderr: '',
exitCode: 0,
timedOut: false
}
}
}
)
expect(result).toEqual({
success: true,
message: 'Add README note',
agentLabel: 'agent'
})
})
it('exposes raw CLI failure output only after path sanitization', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: 'You are generating a single git commit message for /secret/repo',
stderr: 'raw failure output with /Users/thebr/My Repo/secret/file.ts',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'agent CLI command failed with code 1: raw failure output with [path]'
})
})
it('formats nonzero exit failures with extracted sanitized CLI details', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: 'ERROR: fatal: C:\\Users\\Brennan Doe\\secret\\file.ts failed',
stderr: '',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'agent CLI command failed with code 1: ERROR: fatal: [path] failed'
})
})
it('redacts UNC paths in CLI failure details', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr: 'ERROR: failed at \\\\server\\share\\Brennan Repo\\secret\\file.ts',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'agent CLI command failed with code 1: ERROR: failed at [path]'
})
})
it('reports an empty result with the stderr excerpt when exit 0 produces no stdout', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr: '\u001b[91m\u001b[1mError: \u001b[0mNo payment method',
exitCode: 0,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'agent returned an empty message. CLI output: Error: No payment method'
})
})
it('surfaces pi auth failure detail end-to-end through the adjusted path sanitizer', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr: [
'No API key found for github-copilot.',
'',
'Use /login to log into a provider via OAuth or API key. See:',
' /private/tmp/pi-exit1-repro/node_modules/@earendil-works/pi-coding-agent/docs/providers.md',
' /private/tmp/pi-exit1-repro/node_modules/@earendil-works/pi-coding-agent/docs/models.md'
].join('\n'),
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error:
'Pi CLI command failed with code 1: No API key found for github-copilot. Use /login to log into a provider via OAuth or API key. See: … [path]'
})
})
it('preserves slash-commands while redacting multi-segment paths in failure detail', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr: 'ERROR: run /login then check /Users/name/repo',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'agent CLI command failed with code 1: ERROR: run /login then check [path]'
})
})
it('redacts a filesystem path embedded in a pi HTTP 401 payload', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr: '401: {"message":"Invalid key loaded from /Users/name/.config/pi/auth.json"}',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'Pi CLI command failed with code 1: 401: {"message":"Invalid key loaded from [path]"}'
})
})
it('redacts a Windows drive path with JSON-escaped backslashes in a payload', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr: '401: {"message":"Invalid key loaded from C:\\\\Users\\\\name\\\\auth.json"}',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'Pi CLI command failed with code 1: 401: {"message":"Invalid key loaded from [path]"}'
})
})
it('keeps a scheme:// remedy URL intact while still redacting paths', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr:
'401: Visit https://console.anthropic.com/settings/keys then check /Users/name/.config/pi/auth.json',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error:
'Pi CLI command failed with code 1: 401: Visit https://console.anthropic.com/settings/keys then check [path]'
})
})
it('redacts key=/path shapes in provider bodies', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr: '401: {"message":"rejected credential_path=/Users/name/.config/pi/auth.json"}',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'Pi CLI command failed with code 1: 401: {"message":"rejected credential_path=[path]"}'
})
})
it.each([
[
'comma-prefixed list path',
'401: {"message":"candidate list a,/Users/name/creds rejected"}',
'Pi CLI command failed with code 1: 401: {"message":"candidate list a,[path] rejected"}'
],
[
'non-drive colon-prefixed path',
'401: {"message":"slot 1:/Users/name/alt failed"}',
'Pi CLI command failed with code 1: 401: {"message":"slot 1:[path] failed"}'
]
])('redacts a %s in provider bodies', async (_shape, stderr, expected) => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr,
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({ success: false, error: expected })
})
it('passes the live colonless 400 gateway payload through the sanitizer unmangled', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr:
'400 {"type":"error","error":{"type":"invalid_request_error","message":"Third-party apps now draw from your extra usage, not your plan limits. Add more at claude.ai/settings/usage and keep going."},"request_id":"req_011CcsZLJ5ZiLLNvpxcxDuU4"}',
exitCode: 1,
timedOut: false
})
}
)
expect(result.success).toBe(false)
if (result.success) {
throw new Error('expected a failure result')
}
expect(result.error.startsWith('Pi CLI command failed with code 1: 400 {"type":"error"')).toBe(
true
)
// The bare domain link survives path redaction so the remedy stays usable.
expect(result.error).toContain('claude.ai/settings/usage')
expect(result.error.length).toBeLessThanOrEqual(300)
})
it('preserves the structured subject and body when formatting the final response', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: 'Update README.\n\n- Explain the generated commit-message flow\n',
stderr: '',
exitCode: 0,
timedOut: false
})
}
)
expect(result).toEqual({
success: true,
message: 'Update README\n\n- Explain the generated commit-message flow',
agentLabel: 'agent'
})
})
it('reports empty remote commit-message output as an empty message', async () => {
let operation = ''
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async (_plan, _cwd, _timeoutMs, requestedOperation) => {
operation = requestedOperation
return {
stdout: ' \n',
stderr: '',
exitCode: 0,
timedOut: false
}
}
}
)
expect(operation).toBe('commit-message')
expect(result).toEqual({
success: false,
error: 'agent returned an empty message.'
})
})
it('sanitizes remote execution transport failures', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => {
throw new Error('relay disconnected while reading /secret/repo')
}
}
)
expect(result).toEqual({
success: false,
error:
'agent could not be reached on the remote PATH. Try again after the SSH connection recovers.'
})
})
it('caps local agent output before buffering unbounded data', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'local',
cwd: '/repo'
}
)
listeners.get('stdout:data')?.(Buffer.alloc(4 * 1024 * 1024 + 1))
listeners.get('close')?.(null)
await expect(pending).resolves.toEqual({
success: false,
error:
'agent CLI command produced too much output. Check the agent CLI configuration and try again.'
})
expectChildTerminated(child)
})
it('passes prepared provider environment to local agent subprocesses', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'orca-test-agent-nope'
},
{
kind: 'local',
cwd: '/repo',
env: { ...process.env, CODEX_HOME: '/managed/codex-home' }
}
)
listeners.get('stdout:data')?.(Buffer.from('Add README note\n'))
listeners.get('close')?.(0)
await expect(pending).resolves.toMatchObject({
success: true,
message: 'Add README note'
})
expect(spawnMock).toHaveBeenCalledWith(
'orca-test-agent-nope',
[],
expect.objectContaining({
env: expect.objectContaining({ CODEX_HOME: '/managed/codex-home' })
})
)
})
it('routes WSL local commit generation through the selected distro login shell', async () => {
await withPlatform('win32', async () => {
process.env.ORCA_HOST_ONLY_SECRET = 'do-not-leak'
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent --mode fast'
},
{
kind: 'local',
cwd: 'C:\\repo',
wslDistro: 'Ubuntu 24.04',
env: { ...process.env, CODEX_HOME: '/home/tester/.codex' }
}
)
listeners.get('stdout:data')?.(Buffer.from('Update README\n'))
listeners.get('close')?.(0)
await expect(pending).resolves.toMatchObject({
success: true,
message: 'Update README'
})
expect(spawnMock).toHaveBeenCalledWith(
'wsl.exe',
['-d', 'Ubuntu 24.04', '--', 'sh', '-lc', expect.any(String)],
expect.objectContaining({
cwd: undefined,
windowsHide: true,
env: expect.objectContaining({ CODEX_HOME: '/home/tester/.codex' })
})
)
const spawnEnv = spawnMock.mock.calls[0]?.[2]?.env as NodeJS.ProcessEnv
expect(spawnEnv.ORCA_HOST_ONLY_SECRET).toBeUndefined()
const shellCommand = spawnMock.mock.calls[0]?.[1]?.[5] as string
expect(shellCommand).toContain('getent passwd')
expect(shellCommand).toContain('exec "\\$_orca_wsl_shell" -ilc')
expect(shellCommand).toContain('/mnt/c/repo')
expect(shellCommand).toContain("'agent'")
expect(shellCommand).toContain('--mode')
expect(shellCommand).toContain('fast')
})
})
it('keeps local commit-message and pull-request cancellation lanes separate', async () => {
const children: {
pid: number
kill: ReturnType<typeof vi.fn>
listeners: Map<string, (value: unknown) => void>
}[] = []
spawnMock.mockImplementation(() => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123 + children.length,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
children.push({ pid: child.pid, kill: child.kill, listeners })
return child as never
})
const commit = generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'local',
cwd: '/repo'
}
)
const pullRequest = generatePullRequestFieldsFromContext(
{
branch: 'feature/pr-fields',
base: 'main',
branchChangedByPreparation: false,
currentTitle: '',
currentBody: '',
currentDraft: false,
commitSummary: '- feat: update README',
changeSummary: 'M\tREADME.md',
patch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'local',
cwd: '/repo'
}
)
cancelGenerateCommitMessageLocal('/repo')
expectChildTerminated(children[0]!)
expect(children[1]?.kill).not.toHaveBeenCalled()
children[0]?.listeners.get('close')?.(null)
const pullRequestStdout = children[1]?.listeners.get('stdout:data')
pullRequestStdout?.(
Buffer.from('{"base":"main","title":"Update README","body":"Details","draft":false}')
)
children[1]?.listeners.get('close')?.(0)
await expect(commit).resolves.toEqual({
success: false,
error: 'Generation canceled.',
canceled: true
})
await expect(pullRequest).resolves.toMatchObject({
success: true,
fields: {
base: 'main',
title: 'Update README',
body: 'Details',
draft: false
}
})
cancelGeneratePullRequestFieldsLocal('/repo')
expect(children[1]?.kill).not.toHaveBeenCalled()
})
it('keeps local pull-request cancellation from stopping commit-message generation', async () => {
const children: {
pid: number
kill: ReturnType<typeof vi.fn>
listeners: Map<string, (value: unknown) => void>
}[] = []
spawnMock.mockImplementation(() => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123 + children.length,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
children.push({ pid: child.pid, kill: child.kill, listeners })
return child as never
})
const commit = generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'local',
cwd: '/repo'
}
)
const pullRequest = generatePullRequestFieldsFromContext(
{
branch: 'feature/pr-fields',
base: 'main',
branchChangedByPreparation: false,
currentTitle: '',
currentBody: '',
currentDraft: false,
commitSummary: '- feat: update README',
changeSummary: 'M\tREADME.md',
patch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'local',
cwd: '/repo'
}
)
cancelGeneratePullRequestFieldsLocal('/repo')
expect(children[0]?.kill).not.toHaveBeenCalled()
expectChildTerminated(children[1]!)
const commitStdout = children[0]?.listeners.get('stdout:data')
commitStdout?.(Buffer.from('Update README\n'))
children[0]?.listeners.get('close')?.(0)
children[1]?.listeners.get('close')?.(null)
await expect(commit).resolves.toEqual({
success: true,
message: 'Update README',
agentLabel: 'agent'
})
await expect(pullRequest).resolves.toEqual({
success: false,
error: 'Generation canceled.',
canceled: true,
branchChangedByPreparation: false
})
})
it('reports empty remote pull-request field output as empty details', async () => {
let operation = ''
const result = await generatePullRequestFieldsFromContext(
{
branch: 'feature/pr-fields',
base: 'main',
branchChangedByPreparation: false,
currentTitle: '',
currentBody: '',
currentDraft: false,
commitSummary: '- feat: update README',
changeSummary: 'M\tREADME.md',
patch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async (_plan, _cwd, _timeoutMs, requestedOperation) => {
operation = requestedOperation
return {
stdout: ' \n',
stderr: '',
exitCode: 0,
timedOut: false
}
}
}
)
expect(operation).toBe('pull-request-fields')
expect(result).toEqual({
success: false,
error: 'agent returned an empty details.',
branchChangedByPreparation: false
})
})
it('reports branch changes when pull request field output cannot be parsed', async () => {
const listeners = new Map<string, (value: unknown) => void>()
spawnMock.mockReturnValue({
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
} as never)
const pullRequest = generatePullRequestFieldsFromContext(
{
branch: 'feature/pr-fields',
base: 'main',
branchChangedByPreparation: true,
currentTitle: '',
currentBody: '',
currentDraft: false,
commitSummary: '- feat: update README',
changeSummary: 'M\tREADME.md',
patch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'local',
cwd: '/repo'
}
)
listeners.get('stdout:data')?.(Buffer.from('not json'))
listeners.get('close')?.(0)
await expect(pullRequest).resolves.toEqual({
success: false,
error: 'Generated pull request details could not be parsed.',
branchChangedByPreparation: true
})
})
it('reports branch changes when pull request generation is canceled', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pullRequest = generatePullRequestFieldsFromContext(
{
branch: 'feature/pr-fields',
base: 'main',
branchChangedByPreparation: true,
currentTitle: '',
currentBody: '',
currentDraft: false,
commitSummary: '- feat: update README',
changeSummary: 'M\tREADME.md',
patch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'local',
cwd: '/repo'
}
)
cancelGeneratePullRequestFieldsLocal('/repo')
listeners.get('close')?.(null)
expectChildTerminated(child)
await expect(pullRequest).resolves.toEqual({
success: false,
error: 'Generation canceled.',
canceled: true,
branchChangedByPreparation: true
})
})
it('settles local commit-message cancellation even when the killed child does not close', async () => {
vi.useFakeTimers()
try {
const listeners = new Map<string, (value: unknown) => void>()
const removeListener = (key: string, callback: (value: unknown) => void): void => {
if (listeners.get(key) === callback) {
listeners.delete(key)
}
}
const child = {
pid: 123,
kill: vi.fn(),
stdout: {
on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)),
off: vi.fn((event, callback) => removeListener(`stdout:${event}`, callback))
},
stderr: {
on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)),
off: vi.fn((event, callback) => removeListener(`stderr:${event}`, callback))
},
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback)),
off: vi.fn((event, callback) => removeListener(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'local',
cwd: '/repo'
}
)
const outcomePromise = pending.then((result) =>
!result.success && result.canceled ? 'canceled' : 'other'
)
cancelGenerateCommitMessageLocal('/repo')
expectChildTerminated(child)
await Promise.resolve()
await Promise.resolve()
await Promise.resolve()
await Promise.resolve()
const outcome = await Promise.race([outcomePromise, Promise.resolve('pending')])
expect(outcome).toBe('canceled')
expect(listeners.has('stdout:data')).toBe(false)
expect(listeners.has('stderr:data')).toBe(false)
expect(listeners.has('error')).toBe(false)
expect(listeners.has('close')).toBe(false)
} finally {
vi.useRealTimers()
}
})
it('routes Windows batch-script agent commands through cmd.exe', async () => {
const originalComSpec = process.env.ComSpec
process.env.ComSpec = 'C:\\Windows\\System32\\cmd.exe'
try {
await withPlatform('win32', async () => {
const listeners = new Map<string, (value: unknown) => void>()
const child = {
pid: 123,
kill: vi.fn(),
stdout: { on: vi.fn((event, callback) => listeners.set(`stdout:${event}`, callback)) },
stderr: { on: vi.fn((event, callback) => listeners.set(`stderr:${event}`, callback)) },
stdin: { end: vi.fn() },
on: vi.fn((event, callback) => listeners.set(event, callback))
}
spawnMock.mockReturnValue(child as never)
const pending = generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'C:/tools/agent.cmd'
},
{
kind: 'local',
cwd: 'C:\\repo'
}
)
listeners.get('stdout:data')?.(Buffer.from('Update README\n'))
listeners.get('close')?.(0)
await expect(pending).resolves.toMatchObject({
success: true,
message: 'Update README'
})
expect(spawnMock).toHaveBeenCalledWith(
'C:\\Windows\\System32\\cmd.exe',
['/d', '/c', 'C:/tools/agent.cmd'],
expect.objectContaining({
cwd: 'C:\\repo',
windowsHide: true
})
)
})
} finally {
if (originalComSpec === undefined) {
delete process.env.ComSpec
} else {
process.env.ComSpec = originalComSpec
}
}
})
it('rejects unsafe argv prompts for Windows batch-script agent commands', async () => {
await withPlatform('win32', async () => {
const result = await generateCommitMessageFromContext(
{
branch: 'main',
stagedSummary: 'M\tREADME.md',
stagedPatch: '+hello & goodbye'
},
{
agentId: 'custom',
model: '',
customAgentCommand: 'C:/tools/agent.cmd {prompt}'
},
{
kind: 'local',
cwd: 'C:\\repo'
}
)
expect(result).toEqual({
success: false,
error:
'C:/tools/agent.cmd cannot be run as a Windows batch command with the prompt in argv. Remove {prompt} so Orca sends the prompt on stdin.'
})
expect(spawnMock).not.toHaveBeenCalled()
})
})
})
describe('generateBranchNameFromContext', () => {
it('sanitizes remote agent output into a short branch slug', async () => {
const result = await generateBranchNameFromContext(
{ firstPrompt: 'Fix login flow' },
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '"Fix/Login Flow now please"\n',
stderr: '',
exitCode: 0,
timedOut: false
})
}
)
expect(result).toEqual({
success: true,
slug: 'fix-login-flow-now',
agentLabel: 'agent'
})
})
it('fails when remote agent output sanitizes to an empty branch slug', async () => {
const result = await generateBranchNameFromContext(
{ firstPrompt: 'Fix login flow' },
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '!!! ___\n',
stderr: '',
exitCode: 0,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'Generated branch name was empty after sanitization.',
failureOutput: { label: 'agent', exitCode: 0, stdout: '!!! ___', stderr: '' }
})
})
it('carries the full CLI output on failures for the local on-demand view', async () => {
const result = await generateBranchNameFromContext(
{ firstPrompt: 'Fix login flow' },
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: 'partial',
stderr: 'No API key found for github-copilot.',
exitCode: 1,
timedOut: false
})
}
)
expect(result.success).toBe(false)
if (result.success) {
throw new Error('expected a failure result')
}
expect(result.failureOutput).toEqual({
label: 'Pi',
exitCode: 1,
stdout: 'partial',
stderr: 'No API key found for github-copilot.'
})
})
it('does not persist stdout-only branch failure detail that may echo the prompt', async () => {
const result = await generateBranchNameFromContext(
{ firstPrompt: 'Customer secret in the first prompt' },
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: 'Customer secret in the first prompt',
stderr: '',
exitCode: 1,
timedOut: false
})
}
)
expect(result).toEqual({
success: false,
error: 'agent CLI command failed with code 1.',
failureOutput: {
label: 'agent',
exitCode: 1,
stdout: 'Customer secret in the first prompt',
stderr: ''
}
})
})
it('describes a signal-terminated generator without a null exit code', async () => {
const result = await generateBranchNameFromContext(
{ firstPrompt: 'Fix login flow' },
{
agentId: 'pi',
model: 'github-copilot/gpt-5.5'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async () => ({
stdout: '',
stderr: 'Process killed by host',
exitCode: null,
timedOut: false
})
}
)
expect(result).toMatchObject({
success: false,
error: 'Pi CLI command was terminated before exiting: Process killed by host'
})
})
it('keeps branch-name guidance first without dropping the output contract', async () => {
let prompt = ''
await generateBranchNameFromContext(
{ firstPrompt: 'Fix login flow' },
{
agentId: 'custom',
model: '',
customAgentCommand: 'agent',
customPrompt: 'Prefer auth terminology.',
commandInputTemplate: 'Prefer auth terminology.\n\n{basePrompt}'
},
{
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async (plan) => {
prompt = plan.stdinPayload ?? ''
return {
stdout: 'fix-login-flow\n',
stderr: '',
exitCode: 0,
timedOut: false
}
}
}
)
expect(prompt.startsWith('Prefer auth terminology.')).toBe(true)
expect(prompt).toContain('Prefer auth terminology.')
expect(prompt).not.toContain('Additional user prompt:')
expect(prompt).toContain('Generate a short git branch name')
expect(prompt).toContain('Output ONLY the branch name on a single line')
})
})
describe('linkedIssue template substitution', () => {
const COMMIT_CONTEXT = {
branch: 'feature/login',
stagedSummary: 'M src/login.ts',
stagedPatch: 'diff --git a/src/login.ts b/src/login.ts'
}
const PULL_REQUEST_CONTEXT = {
branch: 'feature/login',
base: 'main',
branchChangedByPreparation: false,
currentTitle: 'Fix login',
currentBody: '',
currentDraft: false,
commitSummary: 'a1b2c3d Fix login',
changeSummary: 'src/login.ts | 4 ++--',
patch: 'diff --git a/src/login.ts b/src/login.ts'
}
function capturingTarget(capture: (prompt: string) => void): {
kind: 'remote'
cwd: string
missingBinaryLocation: string
execute: (plan: { stdinPayload: string | null }) => Promise<{
stdout: string
stderr: string
exitCode: number
timedOut: boolean
}>
} {
return {
kind: 'remote',
cwd: '/repo',
missingBinaryLocation: 'remote PATH',
execute: async (plan) => {
capture(plan.stdinPayload ?? '')
return {
stdout: '{"base":"main","title":"Fix login","body":"body","draft":false}',
stderr: '',
exitCode: 0,
timedOut: false
}
}
}
}
const templateParams = {
agentId: 'custom' as const,
model: '',
customAgentCommand: 'agent',
commandInputTemplate: '{basePrompt}\n\nFixes #{linkedIssue}'
}
it('substitutes the linked issue into the commit-message prompt', async () => {
let prompt = ''
await generateCommitMessageFromContext(
{ ...COMMIT_CONTEXT, linkedIssue: 42 },
templateParams,
capturingTarget((value) => {
prompt = value
})
)
expect(prompt).toContain('Fixes #42')
expect(prompt).not.toContain('{linkedIssue}')
})
it('renders an empty commit-message issue for null and omitted fields', async () => {
for (const context of [{ ...COMMIT_CONTEXT, linkedIssue: null }, COMMIT_CONTEXT]) {
let prompt = ''
await generateCommitMessageFromContext(
context,
templateParams,
capturingTarget((value) => {
prompt = value
})
)
expect(prompt).toContain('Fixes #')
expect(prompt).not.toContain('{linkedIssue}')
}
})
// Why: a fixture-unique sentinel — a short number like 42 also appears in the
// character counts that truncateDiffForPrompt/limitSection emit, so growing any
// fixture past its limit would fail these guards for reasons unrelated to leakage.
const BUILT_IN_PROMPT_SENTINEL_ISSUE = 987654
const builtInPromptParams = {
agentId: 'custom' as const,
model: '',
customAgentCommand: 'agent'
}
it('leaves the built-in commit prompt free of issue guidance', async () => {
let prompt = ''
await generateCommitMessageFromContext(
{ ...COMMIT_CONTEXT, linkedIssue: BUILT_IN_PROMPT_SENTINEL_ISSUE },
builtInPromptParams,
capturingTarget((value) => {
prompt = value
})
)
expect(prompt).not.toContain(String(BUILT_IN_PROMPT_SENTINEL_ISSUE))
expect(prompt).not.toContain('linkedIssue')
})
it('leaves the built-in pull-request prompt free of issue guidance', async () => {
let prompt = ''
await generatePullRequestFieldsFromContext(
{ ...PULL_REQUEST_CONTEXT, linkedIssue: BUILT_IN_PROMPT_SENTINEL_ISSUE },
builtInPromptParams,
capturingTarget((value) => {
prompt = value
})
)
expect(prompt).not.toContain(String(BUILT_IN_PROMPT_SENTINEL_ISSUE))
expect(prompt).not.toContain('linkedIssue')
})
it('substitutes the linked issue into the pull-request prompt', async () => {
let prompt = ''
await generatePullRequestFieldsFromContext(
{ ...PULL_REQUEST_CONTEXT, linkedIssue: 7 },
templateParams,
capturingTarget((value) => {
prompt = value
})
)
expect(prompt).toContain('Fixes #7')
expect(prompt).not.toContain('{linkedIssue}')
})
it('renders an empty pull-request issue when none resolves', async () => {
let prompt = ''
await generatePullRequestFieldsFromContext(
PULL_REQUEST_CONTEXT,
templateParams,
capturingTarget((value) => {
prompt = value
})
)
expect(prompt).toContain('Fixes #')
expect(prompt).not.toContain('{linkedIssue}')
})
it('leaves a hand-typed linkedIssue literal in branch-name templates', async () => {
let prompt = ''
await generateBranchNameFromContext(
{ firstPrompt: 'Fix login flow' },
{ ...templateParams, commandInputTemplate: '{basePrompt}\n\nIssue {linkedIssue}' },
capturingTarget((value) => {
prompt = value
})
)
expect(prompt).toContain('Issue {linkedIssue}')
})
})
describe('trimGeneratedCommitMessage', () => {
it('removes trailing whitespace from generated messages', () => {
const message = trimGeneratedCommitMessage('Update docs\n\n')
expect(message).toBe('Update docs')
})
})