mirror of
https://github.com/stablyai/orca.git
synced 2026-11-05 19:47:27 +00:00
* fix(native-chat): a child's root exit is reported even during its close, and bookkeeping after it never reads as unproven - Both connections report the root process's exit once, with `expected` set when a close had begun. A close that came back unproven and whose root exits later is finished by the adapter, and its end reaches the host like any other. - A Claude close whose resume-point write fails after the exit was proven, and a Codex close whose terminal row is refused, now end the session and report the failure, instead of keeping a dead child indexed as if its exit were unproven. - A Codex close whose forced tree kill can't prove the descendants gone but saw the root exit reports the descendants and counts the root exit. - Every child exit with an identity, expected or not, is forwarded to the host. * fix(native-chat): the exit ends the child's record; an unfinished stop is the child's own close, which everyone joins - The host keeps no stored "stop still owed" record any more. A stop begins the child's close (`child.close`), which lives on the child and ends with it. A second Stop, the idle reaper, quit, a send and an option/answer/goal/rewind all join that close instead of retrying a separate obligation. - A caller waits on the close only as long as the step deadline; the close itself is never abandoned. A proof that lands after every caller stopped waiting reaches the host as the adapter's report of that exit, which ends the record through the same handler. - Once the exit is proven, draining, settling, the lease release and the adapter's acknowledgement are each attempted and reported on failure; none keeps the child on record. A start, and the handle's close, write a release that failed from this host's proof of that exit, so a failed write never refuses a send. - A start that meets a close still unverifiable is refused with `previousExitUnverifiable`, so the queued message is rejected with a send-again reason; nothing is held and nothing starts beside the old process. - The idle sweep goes back to idle reaping only. - Removes #24333's retry entry points, the wait row and its hold rule, the ask/failure cursors on the stored record, and the stop's own wake. Tests replace the #24333 unproven-stop test: a send joining an unproven close and an in-flight one, a late proof past the caller's bound, a root exiting after its close gave up, a proven exit whose resume-point write and lease release both failed, an unverifiable close rejecting the send and refusing an option change, a surviving descendant, quit and the idle reaper; and Codex's unverifiable, late-exit and joined-close cases. * fix(native-chat): a message refused because the old process's exit is unverifiable says so, and to send again The start failure for a refusal with reason `previousExitUnverifiable` reads "Orca couldn't confirm Claude's previous process ended. Send your message to try again." instead of "Claude couldn't restart." The status-row kind and the refusal reason stay in the shared lists for rows and hosts that still carry them; the catalogs keep one sentence for both. * fix(native-chat): a close's verdict is the root's exit alone, and what follows it is logged - A Claude close resolves as soon as the root's exit is proven: the session ends and its `ended` report goes out then. Saving the resume point runs afterwards and a failure is logged, so a slow or hung write never reads as an unproven exit or keeps a dead child on record. - A root that exits after its close came back unproven finishes that close through the same path as any close, so the session's child work is published as ended (background tasks and subagents no longer stay shown running for a dead agent), and a failure there is logged. - Codex logs a refused final row, and reports a root exit whose forced tree kill could not prove the rest of the tree gone the way Claude does, so the host logs it and blocks nothing. - Both adapters take the host's logger for this bookkeeping. * fix(native-chat): one handler ends every child's exit, and a join waits on the adapter's own close - One exit handler (`structured-agent-session-child-exit`) ends a child's record for an exit expected or not. `expected` only changes what the chat is told: the stop's cause, its end at the stop's ask, the settlement id, and no crash outcome row. The lease release keeps the exit's evidence; the handoff guard, lifecycle barrier, sink release and adapter acknowledgement apply to both. A Claude journal-sink failure ends in the same step as its stop, as Orca's own fault. - Joining a close is asking the adapter, whose close is memoized while it runs and bounded by its own kill escalation; the host keeps no attempt of its own and no 10 s caller bound. An ask after a close came back unproven runs the stop again. - A close's end is stamped where its stop was asked for (a repeated ask moves it), so the closed chat and failed start checks order a message accepted meanwhile after it. - A start refused because the old exit is unverifiable rejects what was queued in the same step. - The end of a close the host asked for no longer waits on the cross-session recovery chain. - The kill no longer waits for the stop event's write; the journal writes rows in order. * fix(native-chat): an exit's lease release lands whatever the length of its reason A crash's reason can carry kilobytes of the provider's stderr, and a lease whose death detail is over 512 characters fails the store's own check. The exit handler cut it, but the release a start or the chat handle's close re-derives did not, so after a crash whose own release failed every message was refused as not resumable until restart. The record's builder now cuts the detail to the record's bound, so no writer can hand it one too long. * fix(claude): a proven close waits at most 2 s for the output it already wrote Once the root's exit is proven, the close still waited for the SDK's output reader to end. Something outside the process tree that holds the output open would keep that close, and every send, Stop and quit joining it, waiting with no bound. The wait is now bounded; past it the close resolves as proven and the open output is logged. * fix(codex): an exit reported inside Orca's close keeps the reason Orca closed it for The connection reports the app-server's exit inside the close that ends it, so that report ended every Codex close and replaced the close's own reason (for example, a provider frame that could not be recorded) with the connection's stderr text in the ended record and the lease's exit evidence. The session now records Orca's close with its reason, and the exit it ends keeps that reason. The test connection reports its exit inside close the way the real one does. * fix(native-chat): quit stops delivery before it drains exit recovery Every exit now wakes delivery, and teardown drained exit recovery before it stopped delivery, so an exit settled in that window could start a fresh agent that teardown then killed. Teardown stops delivery first; queued messages wait for the next launch. * docs(native-chat): the unverifiable-exit refusal no longer names a caller's wait The caller's bounded wait was removed; the comment describes the close as it is now. * fix(native-chat): a stop whose kill did not take is logged, and the next ask kills again When a close's kill leaves the agent's root running, the host now logs it. Tests pin what a later ask does: each connection runs its whole stop again (Codex sends SIGKILL a second time), refuses input meanwhile, and proves the exit once the kill takes. * fix(native-chat): a start refused over the old process says Orca couldn't stop it The host reaches an unverifiable verdict only after its own kill left the agent's root running, on the machine that runs the agent, so the sentence now says that: "Orca couldn't stop {agent}'s previous process." The refusal reason, failure kind and wire shapes are unchanged. The host test also checks the failed kill is logged. * docs(native-chat): an unverifiable close verdict is a root that survived the kill The host's close runs where the agent runs, so lost contact never yields this verdict; the comment no longer says it does. * fix(native-chat): a kill that did not take is reported once, by whoever met it The log added at the close fired beside a Stop's own failure report for the same event. A stop still reports it through its failure; a send or option change refused over it now logs it at the refusal, the only place it is otherwise invisible. * test(native-chat): a second Stop joins a close the first could not prove and retries its kill * fix(native-chat): say a start refused beside an unstopped process plainly The rejection now reads "Couldn't stop {{agent}} from before. Send your message again to try once more." This kind has its own send-again step; every other failure keeps "Send your message to try again." * Move provider process supervision and stream reading out of Codex * Preserve teardown behavior with checked mock types after move * Apply provider launch environment and caller teardown labels * Share managed provider launch, exit observation, and close * Preserve synchronous stdin closure before the grace wait * Keep the stacked Claude adapter within the module size limit * Handle nullable provider stdin and processless fixture identity * Preserve close-time cleanup diagnostics after a late provider exit * Separate provider root and descendant exit observations * Give provider child env one owner and gate Codex contract on the shared reader resolveProviderChildEnv is now the only place that overlays and strips a provider's environment; the spawn spec and the request-scoped Codex session both call it. supervisedPosixLaunch only accepts a launch without env fields, so an override can no longer be silently ignored there. Edits to the shared stream reader or the env rule now run the real-binary Codex contract job. * Give every provider one close result, stderr tail and root-only default - A close reports the root and the descendants with the one verdict vocabulary (live / unverifiable / exited); `tree` is null when the close made no descendant observation, and the fallback teardown's outcome is written into it instead of a side flag a caller could miss. - The managed process drains stderr and keeps the 8 KiB tail, so no provider can forget to drain the pipe. - Root-only providers take the default close policy and completion rule; only Claude overrides them. The one already-exited guard lives in the managed close and is recorded when no earlier close ran. - A failed spawn is never read as an observed root exit. * Report only observed descendant exits from the fallback teardown The fallback teardown answered "accepted", and the close turned that into `tree: 'exited'`, though a Windows tree kill's outcome is never read and an unreadable process table observes nothing. It now returns what it observed: `exited` only when the captured descendants were verified gone (or the kernel reported the group empty), `live` / `unverifiable` as verification found them, and null when it signalled without observing. Codex's process-tree diagnostic fires in exactly the cases it did before. * Give two test fixtures' casts a SAFETY rationale for the changed-code gate * Claim no observation from an empty process group ESRCH from the dedicated-group signal says only that the group is empty: a descendant that left it, or a root that never led it, may still be running. It now reports no observation instead of `exited`. The unreadable-table comment says why that case stays "no observation" for root-only providers. * fix(native-chat): a person's close joining a failed one still binds the turn its child end cuts On main every close of the chat writes its own Stop and settle. Here a later close joins the first and writes no row, and the first's settle closed when its kill failed, so a turn that opened in between and was cut by the next close read as failed. A person's close joining a person's close whose Stop opened a settle now reopens that settle until its attempt is done. Tests: a close whose kill failed still closes its settle; a turn opened between a failed close and the next reads as the person's cancellation (each fails without its half of the fix). * test(native-chat): build the Stop-opened-turn test's identity with the opaque handle The test (#25056) landed before the opaque provider handle (#24991), so main still built the old {kind, threadId} handle. * test(ratchet): require src/main/provider-process now that it has landed * test(native-chat): keep main's opaque-handle import in the Stop-opened-turn test Main's #25706 made the same fix as this branch at a different line; the merge kept both imports.
311 lines
10 KiB
TypeScript
311 lines
10 KiB
TypeScript
import { describe, expect, it, vi } from 'vitest'
|
|
|
|
// Every way Orca reads or kills processes, never settling: a Stop that reaches for one hangs.
|
|
const processWork = vi.hoisted(() => {
|
|
const never = (): Promise<never> => new Promise(() => {})
|
|
return {
|
|
captureDescendantSnapshot: vi.fn(async () => ({
|
|
rootPgid: null,
|
|
descendants: [],
|
|
capturedAtMs: 0
|
|
})),
|
|
terminateDescendantSnapshotAndWait: vi.fn(never),
|
|
terminateDescendantSnapshotWithVerdict: vi.fn(never),
|
|
queryWindowsProcessDescendants: vi.fn(never),
|
|
terminateWindowsProcessTree: vi.fn(never)
|
|
}
|
|
})
|
|
vi.mock('../pty-descendant-termination', async (importOriginal) => ({
|
|
...(await importOriginal<Record<string, unknown>>()),
|
|
captureDescendantSnapshot: processWork.captureDescendantSnapshot
|
|
}))
|
|
vi.mock('../pty-descendant-exit-verification', async (importOriginal) => ({
|
|
...(await importOriginal<Record<string, unknown>>()),
|
|
terminateDescendantSnapshotAndWait: processWork.terminateDescendantSnapshotAndWait,
|
|
terminateDescendantSnapshotWithVerdict: processWork.terminateDescendantSnapshotWithVerdict
|
|
}))
|
|
vi.mock('../providers/windows-foreground-process-rows', async (importOriginal) => ({
|
|
...(await importOriginal<Record<string, unknown>>()),
|
|
queryWindowsProcessDescendants: processWork.queryWindowsProcessDescendants
|
|
}))
|
|
vi.mock('../windows-process-tree-kill', async (importOriginal) => ({
|
|
...(await importOriginal<Record<string, unknown>>()),
|
|
terminateWindowsProcessTree: processWork.terminateWindowsProcessTree
|
|
}))
|
|
import type {
|
|
AgentJournalMessageItem,
|
|
AgentSessionJournalIdentity
|
|
} from '../../shared/agent-session-journal-types'
|
|
import {
|
|
CodexAppServerRequestError,
|
|
type CodexAppServerConnection,
|
|
type CodexAppServerConnectionHandlers,
|
|
type CodexAppServerLaunch,
|
|
type openCodexAppServerConnection
|
|
} from './codex-app-server-connection'
|
|
import { CodexAppServerUnsupportedError } from './codex-app-server-session'
|
|
import {
|
|
CodexStructuredSessionAdapter,
|
|
type CodexStructuredSessionAdapterDeps,
|
|
type CodexStructuredSessionEvent
|
|
} from './codex-structured-session-adapter'
|
|
import { codexProviderHandle } from '../../shared/agent-session-provider-handle-encoding'
|
|
|
|
const THREAD_ID = 'thread-abc'
|
|
const USER_MESSAGE: AgentJournalMessageItem = {
|
|
kind: 'message',
|
|
role: 'user',
|
|
blocks: [{ type: 'text', text: 'ship it' }]
|
|
}
|
|
|
|
type Route = (params: Record<string, unknown> | undefined) => unknown
|
|
type FakeConnection = Omit<CodexAppServerConnection, 'closed'> & {
|
|
closed: boolean
|
|
launch: CodexAppServerLaunch
|
|
handlers: CodexAppServerConnectionHandlers
|
|
calls: { method: string; params?: Record<string, unknown> }[]
|
|
}
|
|
|
|
function identity(): AgentSessionJournalIdentity {
|
|
return {
|
|
sessionId: 'session-1',
|
|
workspaceId: 'ws-1',
|
|
hostId: 'host-1',
|
|
agent: 'codex',
|
|
providerHandle: codexProviderHandle(THREAD_ID)
|
|
}
|
|
}
|
|
|
|
function fakeCodex(): {
|
|
connections: FakeConnection[]
|
|
openConnection: typeof openCodexAppServerConnection
|
|
routes: Record<string, Route>
|
|
} {
|
|
const connections: FakeConnection[] = []
|
|
const routes: Record<string, Route> = {
|
|
'thread/resume': () => ({ thread: { id: THREAD_ID } })
|
|
}
|
|
const openConnection = (async (launch, handlers = {}) => {
|
|
const connection: FakeConnection = {
|
|
launch,
|
|
handlers,
|
|
calls: [],
|
|
pid: 4321,
|
|
closed: false,
|
|
request: async (method, params) => {
|
|
connection.calls.push({ method, params })
|
|
return routes[method]?.(params) ?? {}
|
|
},
|
|
notify: () => {},
|
|
respond: () => {},
|
|
respondWithError: () => {},
|
|
close: async () => {
|
|
connection.closed = true
|
|
return true
|
|
}
|
|
}
|
|
connections.push(connection)
|
|
return connection
|
|
}) as typeof openCodexAppServerConnection
|
|
return { connections, openConnection, routes }
|
|
}
|
|
|
|
async function acquired(
|
|
codex: ReturnType<typeof fakeCodex>,
|
|
events: CodexStructuredSessionEvent[] = [],
|
|
overrides: Partial<Pick<CodexStructuredSessionAdapterDeps, 'now'>> = {}
|
|
): Promise<CodexStructuredSessionAdapter> {
|
|
const adapter = new CodexStructuredSessionAdapter({
|
|
resolveLaunch: async () => ({
|
|
command: 'codex',
|
|
args: ['app-server'],
|
|
cwd: '/work/repo',
|
|
codexHome: null,
|
|
resumeThreadId: THREAD_ID
|
|
}),
|
|
onEvent: (event) => events.push(event),
|
|
openConnection: codex.openConnection,
|
|
readProcessStartTime: async () => 1_700_000_000_000,
|
|
...overrides
|
|
})
|
|
await adapter.acquire({ identity: identity(), fence: 7, spawnToken: 'spawn-9' })
|
|
return adapter
|
|
}
|
|
|
|
function completeTurn(codex: ReturnType<typeof fakeCodex>, turnId = 'turn-1'): void {
|
|
codex.connections[0].handlers.onNotification?.('turn/completed', {
|
|
threadId: THREAD_ID,
|
|
turn: { id: turnId, status: 'interrupted' }
|
|
})
|
|
}
|
|
|
|
describe('CodexStructuredSessionAdapter.cancelTurn', () => {
|
|
it('confirms an interrupt Codex acknowledged', async () => {
|
|
const codex = fakeCodex()
|
|
const adapter = await acquired(codex)
|
|
|
|
await expect(
|
|
adapter.cancelTurn({ sessionId: 'session-1', turnId: 'turn-1', fence: 7 })
|
|
).resolves.toEqual({ cancelled: true, turnId: 'turn-1' })
|
|
expect(codex.connections[0].calls.at(-1)).toEqual({
|
|
method: 'turn/interrupt',
|
|
params: { threadId: THREAD_ID, turnId: 'turn-1' }
|
|
})
|
|
})
|
|
|
|
it('reports not-cancelled when Codex declines or lacks the method', async () => {
|
|
const declined = fakeCodex()
|
|
declined.routes['turn/interrupt'] = () => {
|
|
throw new CodexAppServerRequestError('turn/interrupt', -32602, 'no such turn')
|
|
}
|
|
const absent = fakeCodex()
|
|
absent.routes['turn/interrupt'] = () => {
|
|
throw new CodexAppServerUnsupportedError('no turn/interrupt')
|
|
}
|
|
|
|
await expect(
|
|
(await acquired(declined)).cancelTurn({
|
|
sessionId: 'session-1',
|
|
turnId: 'turn-1',
|
|
fence: 7
|
|
})
|
|
).resolves.toEqual({ cancelled: false, refusal: {} })
|
|
await expect(
|
|
(await acquired(absent)).cancelTurn({
|
|
sessionId: 'session-1',
|
|
turnId: 'turn-1',
|
|
fence: 7
|
|
})
|
|
).resolves.toEqual({ cancelled: false, refusal: {} })
|
|
})
|
|
|
|
it('rethrows an unsettled interrupt so the turn is not shown as cancelled', async () => {
|
|
const codex = fakeCodex()
|
|
codex.routes['turn/interrupt'] = () => {
|
|
throw new Error('codex app-server turn/interrupt exceeded 30000ms')
|
|
}
|
|
|
|
await expect(
|
|
(await acquired(codex)).cancelTurn({
|
|
sessionId: 'session-1',
|
|
turnId: 'turn-1',
|
|
fence: 7
|
|
})
|
|
).rejects.toThrow('exceeded 30000ms')
|
|
})
|
|
|
|
it('accepts an immediate resend after verified interruption', async () => {
|
|
let nextTurn = 0
|
|
const codex = fakeCodex()
|
|
codex.routes['turn/start'] = () => {
|
|
// Codex opens each turn it answers; a send's dispatch waits for that.
|
|
const turnId = `turn-${++nextTurn}`
|
|
codex.connections[0].handlers.onNotification?.('turn/started', {
|
|
threadId: THREAD_ID,
|
|
turn: { id: turnId }
|
|
})
|
|
return { turn: { id: turnId } }
|
|
}
|
|
codex.routes['turn/interrupt'] = () => {
|
|
completeTurn(codex)
|
|
return {}
|
|
}
|
|
const adapter = await acquired(codex)
|
|
|
|
await adapter.dispatch({
|
|
sessionId: 'session-1',
|
|
clientMessageId: 'client-1',
|
|
body: USER_MESSAGE,
|
|
fence: 7
|
|
})
|
|
await adapter.cancelTurn({ sessionId: 'session-1', turnId: 'turn-1', fence: 7 })
|
|
|
|
await expect(
|
|
adapter.dispatch({
|
|
sessionId: 'session-1',
|
|
clientMessageId: 'client-2',
|
|
body: USER_MESSAGE,
|
|
fence: 7
|
|
})
|
|
).resolves.toEqual({
|
|
state: 'admitted'
|
|
})
|
|
})
|
|
|
|
it("publishes the turn's end when the interrupt receipt fails", async () => {
|
|
const events: CodexStructuredSessionEvent[] = []
|
|
const codex = fakeCodex()
|
|
codex.routes['turn/interrupt'] = () => {
|
|
completeTurn(codex)
|
|
throw new Error('interrupt receipt lost')
|
|
}
|
|
const adapter = await acquired(codex, events)
|
|
|
|
await expect(
|
|
adapter.cancelTurn({ sessionId: 'session-1', turnId: 'turn-1', fence: 7 })
|
|
).rejects.toThrow('interrupt receipt lost')
|
|
expect(events).toContainEqual(expect.objectContaining({ method: 'turn/completed' }))
|
|
})
|
|
})
|
|
|
|
// Codex keeps a turn's background terminals alive across an interrupt and kills its one-shot
|
|
// commands itself, so a Stop is the interrupt alone.
|
|
describe('a Codex Stop is the interrupt alone', () => {
|
|
async function stopAfterASend() {
|
|
const events: CodexStructuredSessionEvent[] = []
|
|
const answer = Promise.withResolvers<unknown>()
|
|
const codex = fakeCodex()
|
|
codex.routes['turn/start'] = () => {
|
|
codex.connections[0].handlers.onNotification?.('turn/started', {
|
|
threadId: THREAD_ID,
|
|
turn: { id: 'turn-1' }
|
|
})
|
|
return { turn: { id: 'turn-1' } }
|
|
}
|
|
codex.routes['turn/interrupt'] = () => answer.promise
|
|
const adapter = await acquired(codex, events)
|
|
await adapter.dispatch({
|
|
sessionId: 'session-1',
|
|
clientMessageId: 'client-1',
|
|
body: USER_MESSAGE,
|
|
fence: 7
|
|
})
|
|
const stopped = adapter.cancelTurn({ sessionId: 'session-1', turnId: 'turn-1', fence: 7 })
|
|
await vi.waitFor(() => expect(codex.connections[0].calls.at(-1)?.method).toBe('turn/interrupt'))
|
|
// Codex answers the interrupt, then sends the turn's end; one read can carry both.
|
|
const answerThenEnd = (): void => {
|
|
answer.resolve({})
|
|
completeTurn(codex)
|
|
}
|
|
return { events, stopped, answerThenEnd }
|
|
}
|
|
|
|
it('publishes the interrupted end the moment Codex sends it', async () => {
|
|
const { events, stopped, answerThenEnd } = await stopAfterASend()
|
|
|
|
answerThenEnd()
|
|
|
|
expect(events).toContainEqual(
|
|
expect.objectContaining({
|
|
method: 'turn/completed',
|
|
params: expect.objectContaining({ turn: { id: 'turn-1', status: 'interrupted' } })
|
|
})
|
|
)
|
|
await expect(stopped).resolves.toEqual({ cancelled: true, turnId: 'turn-1' })
|
|
})
|
|
|
|
it('reads and kills no processes, from the send through the Stop', async () => {
|
|
const { stopped, answerThenEnd } = await stopAfterASend()
|
|
|
|
answerThenEnd()
|
|
|
|
for (const work of Object.values(processWork)) {
|
|
expect(work).not.toHaveBeenCalled()
|
|
}
|
|
await expect(stopped).resolves.toEqual({ cancelled: true, turnId: 'turn-1' })
|
|
for (const work of Object.values(processWork)) {
|
|
expect(work).not.toHaveBeenCalled()
|
|
}
|
|
})
|
|
})
|