starred/orca
1
0
Fork 0
mirror of https://github.com/stablyai/orca.git synced 2026-10-01 16:02:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(native-chat): only Codex's turn completion ends a Codex turn (#23682)

Browse Source 
* fix(native-chat): the sink queue keeps a settlement's first batch, as the journal does

The journal applies a lifecycle batch's settlement id once and skips any later
batch with the same id. The deferred sink queue coalesced the same key the other
way: a second batch replaced the first while it was still queued. So which
record survived depended on whether the first had drained yet.

A lifecycle batch now keeps the queued operation with its key, and a later one
is accepted and dropped, which is what the journal does once the first is
written.

* fix(native-chat): only turn/completed ends a Codex turn

Codex follows every turn-ending `error` (willRetry=false) with a failed
`turn/completed` for the same turn, 0-32 ms later. That was captured from the
real app-server on 0.141.0 and 0.158.0 across eight failure scenarios, and it is
how Codex builds a failed turn: it records the error as the turn's last error,
records any pending input, and then derives `failed` from that error when it
completes the turn.

The translator ended the turn twice: once on the error, and again on the
completion, with a guard to make the first end final. Ending on the error threw
away what only the completion carries: Codex's duration, and the completion's
receipt time. It also forgot the turn before Codex recorded the turn's pending
input.

Now the error is only the row the user reads, inside the still-open turn, and
`turn/completed` is the turn's only live end. A process exit between the two is
the existing exit sweep's observed end, recorded as interrupted.

A failed completion is stored as completed with outcome failure, live and on
restore alike. Only `interrupted` maps to the interrupted state.

The first-end-final guard is gone. Codex sends one completion per turn, the only
redelivery Orca has is the retry of a refused frame (which changes nothing), and
the settlement id already keeps the first record in the queue and the journal.

* refactor(codex): delete the unreachable oversized-notification settlement

The translator settled a streamed item when the transport rejected its
notification as oversized. Nothing can produce that frame. The Codex stdio
reader frames with `maxLineBytes: Number.POSITIVE_INFINITY`
(codex-app-server-record-reader.ts), which it has done since the app-server
records were uncapped. With an infinite limit the framer never reports
`line-too-long`: no line, pending suffix or paused queue can exceed it. So the
dispatcher never emits `frame:oversized-notification`, and the arm that settles
it never runs.

The arm, its helper module and its test go. In place of the test, the
connection test now proves the reason: a notification past the old 16 MiB wire
limit arrives whole, and no oversized frame is reported.

* test(codex): replace the captured ids in the turn-endings fixture with synthetic ones

The replay reads ids only to group frames, so the real thread, turn and
response ids from the capture account carry nothing the test needs. The
fixture moves beside the Codex tests that read it.

* test(codex): use a neutral made-up status as the unknown-status example

'cancelled' read as a stop being recorded as a completion.

* test(codex): a restored turn with a status Orca cannot place ends with no verdict

Codex's history carries the same status field as the live completion, so the
restore path is pinned to the same mapping: completed, and no outcome.
This commit is contained in:
Brennan Benson
2026-09-28 20:47:45 -07:00
committed by GitHub
parent a5ce8251e3
commit 94b5a7d256
20 changed files with 699 additions and 412 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/main/codex/__fixtures__/codex-app-server-turn-endings.jsonl
+172
View File
@@ -0,0 +1,172 @@
{"case":"0.141.0-bad-model","t":371,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000001","status":{"type":"active"}}}
{"case":"0.141.0-bad-model","t":371,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000001","turn":{"id":"00000000-0000-7000-8000-000000000002","status":"inProgress"}}}
{"case":"0.141.0-bad-model","t":1745,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000001","turnId":"00000000-0000-7000-8000-000000000002","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000003","clientId":null,"content":[]}}}
{"case":"0.141.0-bad-model","t":1745,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000001","turnId":"00000000-0000-7000-8000-000000000002","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000003","clientId":null,"content":[]}}}
{"case":"0.141.0-bad-model","t":2831,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000001","status":{"type":"systemError"}}}
{"case":"0.141.0-bad-model","t":2831,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000001","turnId":"00000000-0000-7000-8000-000000000002","willRetry":false,"error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-qa-nonexistent' model is not supported when using Codex with a ChatGPT account.\"}}","codexErrorInfo":"other"}}}
{"case":"0.141.0-bad-model","t":2831,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000001","turn":{"id":"00000000-0000-7000-8000-000000000002","status":"failed","error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-qa-nonexistent' model is not supported when using Codex with a ChatGPT account.\"}}","codexErrorInfo":"other"},"durationMs":2488}}}
{"case":"0.141.0-bad-turn-cwd","t":1458,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000004","status":{"type":"active"}}}
{"case":"0.141.0-bad-turn-cwd","t":1458,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000004","turn":{"id":"00000000-0000-7000-8000-000000000005","status":"inProgress"}}}
{"case":"0.141.0-bad-turn-cwd","t":5792,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000004","turnId":"00000000-0000-7000-8000-000000000005","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000006","clientId":null,"content":[]}}}
{"case":"0.141.0-bad-turn-cwd","t":5792,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000004","turnId":"00000000-0000-7000-8000-000000000005","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000006","clientId":null,"content":[]}}}
{"case":"0.141.0-bad-turn-cwd","t":7645,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000004","turnId":"00000000-0000-7000-8000-000000000005","item":{"type":"agentMessage","id":"msg_1","text":""}}}
{"case":"0.141.0-bad-turn-cwd","t":7938,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000004","turnId":"00000000-0000-7000-8000-000000000005","item":{"type":"agentMessage","id":"msg_1","text":"pong"}}}
{"case":"0.141.0-bad-turn-cwd","t":7959,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000004","status":{"type":"idle"}}}
{"case":"0.141.0-bad-turn-cwd","t":7959,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000004","turn":{"id":"00000000-0000-7000-8000-000000000005","status":"completed","durationMs":6532}}}
{"case":"0.141.0-compact-overloaded","t":582,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000007","status":{"type":"active"}}}
{"case":"0.141.0-compact-overloaded","t":582,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000007","turn":{"id":"00000000-0000-7000-8000-000000000008","status":"inProgress"}}}
{"case":"0.141.0-compact-overloaded","t":2940,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000007","turnId":"00000000-0000-7000-8000-000000000008","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000009","clientId":null,"content":[]}}}
{"case":"0.141.0-compact-overloaded","t":2940,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000007","turnId":"00000000-0000-7000-8000-000000000008","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000009","clientId":null,"content":[]}}}
{"case":"0.141.0-compact-overloaded","t":3011,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000007","turnId":"00000000-0000-7000-8000-000000000008","item":{"type":"agentMessage","id":"msg_ok","text":"pong"}}}
{"case":"0.141.0-compact-overloaded","t":3011,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000007","turnId":"00000000-0000-7000-8000-000000000008","item":{"type":"agentMessage","id":"msg_ok","text":"pong"}}}
{"case":"0.141.0-compact-overloaded","t":3017,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000007","status":{"type":"idle"}}}
{"case":"0.141.0-compact-overloaded","t":3017,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000007","turn":{"id":"00000000-0000-7000-8000-000000000008","status":"completed","durationMs":2474}}}
{"case":"0.141.0-compact-overloaded","t":3076,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000007","status":{"type":"active"}}}
{"case":"0.141.0-compact-overloaded","t":3076,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000007","turn":{"id":"00000000-0000-7000-8000-000000000010","status":"inProgress"}}}
{"case":"0.141.0-compact-overloaded","t":3081,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000007","turnId":"00000000-0000-7000-8000-000000000010","item":{"type":"contextCompaction","id":"00000000-0000-7000-8000-000000000011"}}}
{"case":"0.141.0-compact-overloaded","t":3684,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000007","turnId":"00000000-0000-7000-8000-000000000010","willRetry":true,"error":{"message":"Reconnecting... 1/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Selected model is at capacity. Please try a different model."}}}
{"case":"0.141.0-compact-overloaded","t":4508,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000007","turnId":"00000000-0000-7000-8000-000000000010","willRetry":true,"error":{"message":"Reconnecting... 2/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Selected model is at capacity. Please try a different model."}}}
{"case":"0.141.0-compact-overloaded","t":5511,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000007","status":{"type":"systemError"}}}
{"case":"0.141.0-compact-overloaded","t":5511,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000007","turnId":"00000000-0000-7000-8000-000000000010","willRetry":false,"error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"}}}
{"case":"0.141.0-compact-overloaded","t":5511,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000007","turn":{"id":"00000000-0000-7000-8000-000000000010","status":"failed","error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"},"durationMs":2435}}}
{"case":"0.141.0-conn-refused","t":588,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000012","status":{"type":"active"}}}
{"case":"0.141.0-conn-refused","t":588,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000012","turn":{"id":"00000000-0000-7000-8000-000000000013","status":"inProgress"}}}
{"case":"0.141.0-conn-refused","t":700,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000012","turnId":"00000000-0000-7000-8000-000000000013","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000014","clientId":null,"content":[]}}}
{"case":"0.141.0-conn-refused","t":700,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000012","turnId":"00000000-0000-7000-8000-000000000013","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000014","clientId":null,"content":[]}}}
{"case":"0.141.0-conn-refused","t":1354,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000012","turnId":"00000000-0000-7000-8000-000000000013","willRetry":true,"error":{"message":"Reconnecting... 1/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"stream disconnected before completion: error sending request for url (http://127.0.0.1:9/v1/responses)"}}}
{"case":"0.141.0-conn-refused","t":2134,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000012","turnId":"00000000-0000-7000-8000-000000000013","willRetry":true,"error":{"message":"Reconnecting... 2/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"stream disconnected before completion: error sending request for url (http://127.0.0.1:9/v1/responses)"}}}
{"case":"0.141.0-conn-refused","t":3116,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000012","status":{"type":"systemError"}}}
{"case":"0.141.0-conn-refused","t":3116,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000012","turnId":"00000000-0000-7000-8000-000000000013","willRetry":false,"error":{"message":"stream disconnected before completion: error sending request for url (http://127.0.0.1:9/v1/responses)","codexErrorInfo":"other"}}}
{"case":"0.141.0-conn-refused","t":3116,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000012","turn":{"id":"00000000-0000-7000-8000-000000000013","status":"failed","error":{"message":"stream disconnected before completion: error sending request for url (http://127.0.0.1:9/v1/responses)","codexErrorInfo":"other"},"durationMs":2556}}}
{"case":"0.141.0-control","t":487,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000015","status":{"type":"active"}}}
{"case":"0.141.0-control","t":487,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000015","turn":{"id":"00000000-0000-7000-8000-000000000016","status":"inProgress"}}}
{"case":"0.141.0-control","t":2657,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000015","turnId":"00000000-0000-7000-8000-000000000016","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000017","clientId":null,"content":[]}}}
{"case":"0.141.0-control","t":2657,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000015","turnId":"00000000-0000-7000-8000-000000000016","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000017","clientId":null,"content":[]}}}
{"case":"0.141.0-control","t":4651,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000015","turnId":"00000000-0000-7000-8000-000000000016","item":{"type":"agentMessage","id":"msg_2","text":""}}}
{"case":"0.141.0-control","t":4661,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000015","turnId":"00000000-0000-7000-8000-000000000016","item":{"type":"agentMessage","id":"msg_2","text":"pong"}}}
{"case":"0.141.0-control","t":4811,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000015","status":{"type":"idle"}}}
{"case":"0.141.0-control","t":4811,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000015","turn":{"id":"00000000-0000-7000-8000-000000000016","status":"completed","durationMs":4351}}}
{"case":"0.141.0-interrupt","t":670,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000018","status":{"type":"active"}}}
{"case":"0.141.0-interrupt","t":670,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000018","turn":{"id":"00000000-0000-7000-8000-000000000019","status":"inProgress"}}}
{"case":"0.141.0-interrupt","t":2618,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000018","turnId":"00000000-0000-7000-8000-000000000019","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000020","clientId":null,"content":[]}}}
{"case":"0.141.0-interrupt","t":2618,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000018","turnId":"00000000-0000-7000-8000-000000000019","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000020","clientId":null,"content":[]}}}
{"case":"0.141.0-interrupt","t":4153,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000018","turnId":"00000000-0000-7000-8000-000000000019","item":{"type":"agentMessage","id":"msg_3","text":""}}}
{"case":"0.141.0-interrupt","t":5669,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000018","status":{"type":"idle"}}}
{"case":"0.141.0-interrupt","t":5669,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000018","turn":{"id":"00000000-0000-7000-8000-000000000019","status":"interrupted","durationMs":5027}}}
{"case":"0.141.0-model-too-new-a","t":1276,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000021","status":{"type":"active"}}}
{"case":"0.141.0-model-too-new-a","t":1276,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000021","turn":{"id":"00000000-0000-7000-8000-000000000022","status":"inProgress"}}}
{"case":"0.141.0-model-too-new-a","t":6105,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000021","turnId":"00000000-0000-7000-8000-000000000022","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000023","clientId":null,"content":[]}}}
{"case":"0.141.0-model-too-new-a","t":6105,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000021","turnId":"00000000-0000-7000-8000-000000000022","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000023","clientId":null,"content":[]}}}
{"case":"0.141.0-model-too-new-a","t":7424,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000021","status":{"type":"systemError"}}}
{"case":"0.141.0-model-too-new-a","t":7424,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000021","turnId":"00000000-0000-7000-8000-000000000022","willRetry":false,"error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-6-astra' model requires a newer version of Codex. Please upgrade to the latest app or CLI and try again.\"}}","codexErrorInfo":"other"}}}
{"case":"0.141.0-model-too-new-a","t":7424,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000021","turn":{"id":"00000000-0000-7000-8000-000000000022","status":"failed","error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-6-astra' model requires a newer version of Codex. Please upgrade to the latest app or CLI and try again.\"}}","codexErrorInfo":"other"},"durationMs":6195}}}
{"case":"0.141.0-model-too-new-b","t":387,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000024","status":{"type":"active"}}}
{"case":"0.141.0-model-too-new-b","t":387,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000024","turn":{"id":"00000000-0000-7000-8000-000000000025","status":"inProgress"}}}
{"case":"0.141.0-model-too-new-b","t":1162,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000024","turnId":"00000000-0000-7000-8000-000000000025","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000026","clientId":null,"content":[]}}}
{"case":"0.141.0-model-too-new-b","t":1162,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000024","turnId":"00000000-0000-7000-8000-000000000025","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000026","clientId":null,"content":[]}}}
{"case":"0.141.0-model-too-new-b","t":1993,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000024","status":{"type":"systemError"}}}
{"case":"0.141.0-model-too-new-b","t":1993,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000024","turnId":"00000000-0000-7000-8000-000000000025","willRetry":false,"error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-6-astra' model requires a newer version of Codex. Please upgrade to the latest app or CLI and try again.\"}}","codexErrorInfo":"other"}}}
{"case":"0.141.0-model-too-new-b","t":1993,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000024","turn":{"id":"00000000-0000-7000-8000-000000000025","status":"failed","error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-6-astra' model requires a newer version of Codex. Please upgrade to the latest app or CLI and try again.\"}}","codexErrorInfo":"other"},"durationMs":1633}}}
{"case":"0.141.0-overloaded-503","t":407,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000027","status":{"type":"active"}}}
{"case":"0.141.0-overloaded-503","t":407,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000027","turn":{"id":"00000000-0000-7000-8000-000000000028","status":"inProgress"}}}
{"case":"0.141.0-overloaded-503","t":574,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000027","turnId":"00000000-0000-7000-8000-000000000028","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000029","clientId":null,"content":[]}}}
{"case":"0.141.0-overloaded-503","t":574,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000027","turnId":"00000000-0000-7000-8000-000000000028","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000029","clientId":null,"content":[]}}}
{"case":"0.141.0-overloaded-503","t":1296,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000027","status":{"type":"systemError"}}}
{"case":"0.141.0-overloaded-503","t":1296,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000027","turnId":"00000000-0000-7000-8000-000000000028","willRetry":false,"error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"}}}
{"case":"0.141.0-overloaded-503","t":1296,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000027","turn":{"id":"00000000-0000-7000-8000-000000000028","status":"failed","error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"},"durationMs":921}}}
{"case":"0.141.0-overloaded-sse","t":510,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000030","status":{"type":"active"}}}
{"case":"0.141.0-overloaded-sse","t":510,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000030","turn":{"id":"00000000-0000-7000-8000-000000000031","status":"inProgress"}}}
{"case":"0.141.0-overloaded-sse","t":611,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000030","turnId":"00000000-0000-7000-8000-000000000031","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000032","clientId":null,"content":[]}}}
{"case":"0.141.0-overloaded-sse","t":611,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000030","turnId":"00000000-0000-7000-8000-000000000031","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000032","clientId":null,"content":[]}}}
{"case":"0.141.0-overloaded-sse","t":695,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000030","status":{"type":"systemError"}}}
{"case":"0.141.0-overloaded-sse","t":695,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000030","turnId":"00000000-0000-7000-8000-000000000031","willRetry":false,"error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"}}}
{"case":"0.141.0-overloaded-sse","t":695,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000030","turn":{"id":"00000000-0000-7000-8000-000000000031","status":"failed","error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"},"durationMs":213}}}
{"case":"0.141.0-stream-drop","t":510,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000033","status":{"type":"active"}}}
{"case":"0.141.0-stream-drop","t":510,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000033","turn":{"id":"00000000-0000-7000-8000-000000000034","status":"inProgress"}}}
{"case":"0.141.0-stream-drop","t":621,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000033","turnId":"00000000-0000-7000-8000-000000000034","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000035","clientId":null,"content":[]}}}
{"case":"0.141.0-stream-drop","t":621,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000033","turnId":"00000000-0000-7000-8000-000000000034","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000035","clientId":null,"content":[]}}}
{"case":"0.141.0-stream-drop","t":740,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000033","turnId":"00000000-0000-7000-8000-000000000034","willRetry":true,"error":{"message":"Reconnecting... 1/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"stream disconnected before completion: stream closed before response.completed"}}}
{"case":"0.141.0-stream-drop","t":942,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000033","turnId":"00000000-0000-7000-8000-000000000034","willRetry":true,"error":{"message":"Reconnecting... 2/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"stream disconnected before completion: stream closed before response.completed"}}}
{"case":"0.141.0-stream-drop","t":1320,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000033","status":{"type":"systemError"}}}
{"case":"0.141.0-stream-drop","t":1320,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000033","turnId":"00000000-0000-7000-8000-000000000034","willRetry":false,"error":{"message":"stream disconnected before completion: stream closed before response.completed","codexErrorInfo":"other"}}}
{"case":"0.141.0-stream-drop","t":1320,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000033","turn":{"id":"00000000-0000-7000-8000-000000000034","status":"failed","error":{"message":"stream disconnected before completion: stream closed before response.completed","codexErrorInfo":"other"},"durationMs":838}}}
{"case":"0.158.0-bad-model","t":1232,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000036","status":{"type":"active"}}}
{"case":"0.158.0-bad-model","t":1232,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000036","turn":{"id":"00000000-0000-7000-8000-000000000037","status":"inProgress"}}}
{"case":"0.158.0-bad-model","t":7177,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000036","turnId":"00000000-0000-7000-8000-000000000037","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000038","clientId":null,"content":[]}}}
{"case":"0.158.0-bad-model","t":7215,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000036","turnId":"00000000-0000-7000-8000-000000000037","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000038","clientId":null,"content":[]}}}
{"case":"0.158.0-bad-model","t":8236,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000036","status":{"type":"systemError"}}}
{"case":"0.158.0-bad-model","t":8236,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000036","turnId":"00000000-0000-7000-8000-000000000037","willRetry":false,"error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-qa-nonexistent' model is not supported when using Codex with a ChatGPT account.\"}}","codexErrorInfo":"other"}}}
{"case":"0.158.0-bad-model","t":8257,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000036","turn":{"id":"00000000-0000-7000-8000-000000000037","status":"failed","error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'gpt-qa-nonexistent' model is not supported when using Codex with a ChatGPT account.\"}}","codexErrorInfo":"other"},"durationMs":7576}}}
{"case":"0.158.0-bad-turn-cwd","t":4437,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000039","status":{"type":"active"}}}
{"case":"0.158.0-bad-turn-cwd","t":4438,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000039","turn":{"id":"00000000-0000-7000-8000-000000000040","status":"inProgress"}}}
{"case":"0.158.0-bad-turn-cwd","t":7840,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000039","turnId":"00000000-0000-7000-8000-000000000040","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000041","clientId":null,"content":[]}}}
{"case":"0.158.0-bad-turn-cwd","t":7851,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000039","turnId":"00000000-0000-7000-8000-000000000040","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000041","clientId":null,"content":[]}}}
{"case":"0.158.0-bad-turn-cwd","t":9220,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000039","turnId":"00000000-0000-7000-8000-000000000040","item":{"type":"agentMessage","id":"msg_4","text":""}}}
{"case":"0.158.0-bad-turn-cwd","t":9421,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000039","turnId":"00000000-0000-7000-8000-000000000040","item":{"type":"agentMessage","id":"msg_4","text":"pong"}}}
{"case":"0.158.0-bad-turn-cwd","t":9566,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000039","status":{"type":"idle"}}}
{"case":"0.158.0-bad-turn-cwd","t":9566,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000039","turn":{"id":"00000000-0000-7000-8000-000000000040","status":"completed","durationMs":5172}}}
{"case":"0.158.0-compact-overloaded","t":943,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000042","status":{"type":"active"}}}
{"case":"0.158.0-compact-overloaded","t":944,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000042","turn":{"id":"00000000-0000-7000-8000-000000000043","status":"inProgress"}}}
{"case":"0.158.0-compact-overloaded","t":5760,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000042","turnId":"00000000-0000-7000-8000-000000000043","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000044","clientId":null,"content":[]}}}
{"case":"0.158.0-compact-overloaded","t":5765,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000042","turnId":"00000000-0000-7000-8000-000000000043","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000044","clientId":null,"content":[]}}}
{"case":"0.158.0-compact-overloaded","t":5770,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000042","turnId":"00000000-0000-7000-8000-000000000043","item":{"type":"agentMessage","id":"msg_ok","text":"pong"}}}
{"case":"0.158.0-compact-overloaded","t":5771,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000042","turnId":"00000000-0000-7000-8000-000000000043","item":{"type":"agentMessage","id":"msg_ok","text":"pong"}}}
{"case":"0.158.0-compact-overloaded","t":5795,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000042","status":{"type":"idle"}}}
{"case":"0.158.0-compact-overloaded","t":5795,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000042","turn":{"id":"00000000-0000-7000-8000-000000000043","status":"completed","durationMs":4899}}}
{"case":"0.158.0-compact-overloaded","t":5820,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000042","status":{"type":"active"}}}
{"case":"0.158.0-compact-overloaded","t":5820,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000042","turn":{"id":"00000000-0000-7000-8000-000000000045","status":"inProgress"}}}
{"case":"0.158.0-compact-overloaded","t":5821,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000042","turnId":"00000000-0000-7000-8000-000000000045","item":{"type":"contextCompaction","id":"00000000-0000-7000-8000-000000000046"}}}
{"case":"0.158.0-compact-overloaded","t":6421,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000042","turnId":"00000000-0000-7000-8000-000000000045","willRetry":true,"error":{"message":"Reconnecting... 1/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Selected model is at capacity. Please try a different model."}}}
{"case":"0.158.0-compact-overloaded","t":7232,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000042","turnId":"00000000-0000-7000-8000-000000000045","willRetry":true,"error":{"message":"Reconnecting... 2/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Selected model is at capacity. Please try a different model."}}}
{"case":"0.158.0-compact-overloaded","t":8204,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000042","status":{"type":"systemError"}}}
{"case":"0.158.0-compact-overloaded","t":8204,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000042","turnId":"00000000-0000-7000-8000-000000000045","willRetry":false,"error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"}}}
{"case":"0.158.0-compact-overloaded","t":8207,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000042","turn":{"id":"00000000-0000-7000-8000-000000000045","status":"failed","error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"},"durationMs":2388}}}
{"case":"0.158.0-conn-refused","t":1343,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000047","status":{"type":"active"}}}
{"case":"0.158.0-conn-refused","t":1343,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000047","turn":{"id":"00000000-0000-7000-8000-000000000048","status":"inProgress"}}}
{"case":"0.158.0-conn-refused","t":2002,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000049","clientId":null,"content":[]}}}
{"case":"0.158.0-conn-refused","t":2013,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000049","clientId":null,"content":[]}}}
{"case":"0.158.0-conn-refused","t":2616,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","willRetry":true,"error":{"message":"Reconnecting... waiting for network","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Connection failed: error sending request"}}}
{"case":"0.158.0-conn-refused","t":8185,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","willRetry":true,"error":{"message":"Reconnecting... waiting for network","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Connection failed: error sending request"}}}
{"case":"0.158.0-conn-refused","t":18778,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","willRetry":true,"error":{"message":"Reconnecting... waiting for network","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Connection failed: error sending request"}}}
{"case":"0.158.0-conn-refused","t":39406,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","willRetry":true,"error":{"message":"Reconnecting... waiting for network","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Connection failed: error sending request"}}}
{"case":"0.158.0-conn-refused","t":79975,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","willRetry":true,"error":{"message":"Reconnecting... waiting for network","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Connection failed: error sending request"}}}
{"case":"0.158.0-conn-refused","t":140587,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","willRetry":true,"error":{"message":"Reconnecting... waiting for network","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Connection failed: error sending request"}}}
{"case":"0.158.0-conn-refused","t":201225,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000047","turnId":"00000000-0000-7000-8000-000000000048","willRetry":true,"error":{"message":"Reconnecting... waiting for network","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"Connection failed: error sending request"}}}
{"case":"0.158.0-control","t":3620,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000050","status":{"type":"active"}}}
{"case":"0.158.0-control","t":3621,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000050","turn":{"id":"00000000-0000-7000-8000-000000000051","status":"inProgress"}}}
{"case":"0.158.0-control","t":5019,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000050","turnId":"00000000-0000-7000-8000-000000000051","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000052","clientId":null,"content":[]}}}
{"case":"0.158.0-control","t":5254,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000050","turnId":"00000000-0000-7000-8000-000000000051","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000052","clientId":null,"content":[]}}}
{"case":"0.158.0-control","t":7276,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000050","turnId":"00000000-0000-7000-8000-000000000051","item":{"type":"agentMessage","id":"msg_5","text":""}}}
{"case":"0.158.0-control","t":7480,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000050","turnId":"00000000-0000-7000-8000-000000000051","item":{"type":"agentMessage","id":"msg_5","text":"pong"}}}
{"case":"0.158.0-control","t":7659,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000050","status":{"type":"idle"}}}
{"case":"0.158.0-control","t":7659,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000050","turn":{"id":"00000000-0000-7000-8000-000000000051","status":"completed","durationMs":4520}}}
{"case":"0.158.0-interrupt","t":929,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000053","status":{"type":"active"}}}
{"case":"0.158.0-interrupt","t":929,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000053","turn":{"id":"00000000-0000-7000-8000-000000000054","status":"inProgress"}}}
{"case":"0.158.0-interrupt","t":2320,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000053","turnId":"00000000-0000-7000-8000-000000000054","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000055","clientId":null,"content":[]}}}
{"case":"0.158.0-interrupt","t":2325,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000053","turnId":"00000000-0000-7000-8000-000000000054","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000055","clientId":null,"content":[]}}}
{"case":"0.158.0-interrupt","t":4063,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000053","turnId":"00000000-0000-7000-8000-000000000054","item":{"type":"agentMessage","id":"msg_6","text":""}}}
{"case":"0.158.0-interrupt","t":5582,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000053","status":{"type":"idle"}}}
{"case":"0.158.0-interrupt","t":5582,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000053","turn":{"id":"00000000-0000-7000-8000-000000000054","status":"interrupted","durationMs":4711}}}
{"case":"0.158.0-overloaded-503","t":455,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000056","status":{"type":"active"}}}
{"case":"0.158.0-overloaded-503","t":455,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000056","turn":{"id":"00000000-0000-7000-8000-000000000057","status":"inProgress"}}}
{"case":"0.158.0-overloaded-503","t":1114,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000056","turnId":"00000000-0000-7000-8000-000000000057","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000058","clientId":null,"content":[]}}}
{"case":"0.158.0-overloaded-503","t":1131,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000056","turnId":"00000000-0000-7000-8000-000000000057","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000058","clientId":null,"content":[]}}}
{"case":"0.158.0-overloaded-503","t":1722,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000056","status":{"type":"systemError"}}}
{"case":"0.158.0-overloaded-503","t":1722,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000056","turnId":"00000000-0000-7000-8000-000000000057","willRetry":false,"error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"}}}
{"case":"0.158.0-overloaded-503","t":1753,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000056","turn":{"id":"00000000-0000-7000-8000-000000000057","status":"failed","error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"},"durationMs":1302}}}
{"case":"0.158.0-overloaded-sse","t":944,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000059","status":{"type":"active"}}}
{"case":"0.158.0-overloaded-sse","t":944,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000059","turn":{"id":"00000000-0000-7000-8000-000000000060","status":"inProgress"}}}
{"case":"0.158.0-overloaded-sse","t":1196,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000059","turnId":"00000000-0000-7000-8000-000000000060","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000061","clientId":null,"content":[]}}}
{"case":"0.158.0-overloaded-sse","t":1199,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000059","turnId":"00000000-0000-7000-8000-000000000060","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000061","clientId":null,"content":[]}}}
{"case":"0.158.0-overloaded-sse","t":1204,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000059","status":{"type":"systemError"}}}
{"case":"0.158.0-overloaded-sse","t":1204,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000059","turnId":"00000000-0000-7000-8000-000000000060","willRetry":false,"error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"}}}
{"case":"0.158.0-overloaded-sse","t":1206,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000059","turn":{"id":"00000000-0000-7000-8000-000000000060","status":"failed","error":{"message":"Selected model is at capacity. Please try a different model.","codexErrorInfo":"serverOverloaded"},"durationMs":481}}}
{"case":"0.158.0-stream-drop","t":1136,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000062","status":{"type":"active"}}}
{"case":"0.158.0-stream-drop","t":1136,"method":"turn/started","params":{"threadId":"00000000-0000-7000-8000-000000000062","turn":{"id":"00000000-0000-7000-8000-000000000063","status":"inProgress"}}}
{"case":"0.158.0-stream-drop","t":1721,"method":"item/started","params":{"threadId":"00000000-0000-7000-8000-000000000062","turnId":"00000000-0000-7000-8000-000000000063","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000064","clientId":null,"content":[]}}}
{"case":"0.158.0-stream-drop","t":1733,"method":"item/completed","params":{"threadId":"00000000-0000-7000-8000-000000000062","turnId":"00000000-0000-7000-8000-000000000063","item":{"type":"userMessage","id":"00000000-0000-7000-8000-000000000064","clientId":null,"content":[]}}}
{"case":"0.158.0-stream-drop","t":1738,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000062","turnId":"00000000-0000-7000-8000-000000000063","willRetry":true,"error":{"message":"Reconnecting... 1/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"stream disconnected before completion: stream closed before response.completed"}}}
{"case":"0.158.0-stream-drop","t":1959,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000062","turnId":"00000000-0000-7000-8000-000000000063","willRetry":true,"error":{"message":"Reconnecting... 2/2","codexErrorInfo":{"responseStreamDisconnected":{"httpStatusCode":null}},"additionalDetails":"stream disconnected before completion: stream closed before response.completed"}}}
{"case":"0.158.0-stream-drop","t":2514,"method":"thread/status/changed","params":{"threadId":"00000000-0000-7000-8000-000000000062","status":{"type":"systemError"}}}
{"case":"0.158.0-stream-drop","t":2514,"method":"error","params":{"threadId":"00000000-0000-7000-8000-000000000062","turnId":"00000000-0000-7000-8000-000000000063","willRetry":false,"error":{"message":"stream disconnected before completion: stream closed before response.completed","codexErrorInfo":"other"}}}
{"case":"0.158.0-stream-drop","t":2546,"method":"turn/completed","params":{"threadId":"00000000-0000-7000-8000-000000000062","turn":{"id":"00000000-0000-7000-8000-000000000063","status":"failed","error":{"message":"stream disconnected before completion: stream closed before response.completed","codexErrorInfo":"other"},"durationMs":1761}}}
src/main/codex/codex-app-server-connection.test.ts
+30
View File
@@ -583,6 +583,36 @@ describe('openCodexAppServerConnection', () => {
await connection.close()
})
it('delivers a notification beyond the daemon wire limit whole, never as an oversized frame', async () => {
const { child, spawnImpl } = stubChild()
answerInitialize(child)
const frames: string[] = []
const deltas: unknown[] = []
const connection = await openCodexAppServerConnection(
{ command: 'codex', args: ['app-server'] },
{
onUnhandledFrame: (kind) => frames.push(kind),
onNotification: (method, params) => {
if (method === 'item/commandExecution/outputDelta') {
deltas.push(params)
}
}
},
spawnImpl
)
const params = { threadId: 'thread-1', itemId: 'exec-1', delta: '' }
params.delta = 'x'.repeat(16 * 1024 * 1024 + 1)
child.stdout.write(
`${JSON.stringify({ method: 'item/commandExecution/outputDelta', params })}\n`
)
await vi.waitFor(() => expect(deltas).toEqual([params]))
expect(frames).toEqual([])
expect(connection.closed).toBe(false)
await connection.close()
})
it('keeps malformed and non-object JSON non-fatal and processes the next record', async () => {
const { child, spawnImpl } = stubChild()
answerInitialize(child)
src/main/codex/codex-structured-journal-provider-verdicts.ts
-40
View File
@@ -1,40 +0,0 @@
// What Codex's own verdict frames mean beyond the row they print.
//
// Both are decoration to the transcript and load-bearing to the session's state,
// which is why they are read here rather than left to the generic-frame fallback.
import { codexThreadStoppedRunning, readCodexErrorWillRetry } from './codex-structured-thread-facts'
export type CodexProviderVerdict =
/**
* Codex ended this turn with a fault.
*
* The app server emits `error` ONLY for a failure that affects turn status, and
* hardcodes `willRetry: false` there; a stream error it is about to retry carries
* `willRetry: true` and ends nothing. `turn/completed` may never follow, so this
* frame is the turn's only end — without it the running lifecycle row is a latch
* nothing re-derives and the chat reads working for the life of the session.
*/
| 'turn-failed'
/**
* Codex reports the thread is no longer running.
*
* This settles no OPEN turn: the app server clears `running` on every error,
* including the ones it says do not affect turn status, so a turn still open
* here is still running and `turn/completed` is its end. What it does settle is
* a send whose dispatch was never answered — a timed-out dispatch is recorded
* as unverified delivery, reads as work still owed, and nothing else in a live
* session re-derives it.
*/
| 'thread-stopped-running'
| null
export function readCodexProviderVerdict(method: string, params: unknown): CodexProviderVerdict {
if (method === 'error') {
return readCodexErrorWillRetry(params) ? null : 'turn-failed'
}
if (method === 'thread/status/changed') {
return codexThreadStoppedRunning(params) ? 'thread-stopped-running' : null
}
return null
}
src/main/codex/codex-structured-journal-settlement.ts
-76
View File
@@ -174,82 +174,6 @@ export function settleCodexJournalTurn(input: {
return ADMITTED
}
/** Settle streamed items whose terminal notification was rejected as oversized. */
export function settleCodexOversizedNotification(input: {
sessionId: string
threadId: string
method: string
sink: StructuredAgentSessionEventSink
streams: CodexStructuredItemStreams
activeItems: Map<string, CodexActiveJournalItem>
linkageFor: CodexRowLinkage
}): StructuredAgentSessionSinkAdmission {
const itemType = oversizedStreamItemType(input.method)
if (!itemType) {
return ADMITTED
}
const mutations: JournalLifecycleMutationInput[] = []
const activeItemsToForget: { key: string; threadId: string; itemId: string }[] = []
for (const [key, active] of input.activeItems) {
if (active.threadId !== input.threadId || active.item.type !== itemType) {
continue
}
const streamed = input.streams.snapshot(active.threadId, active.item.id)
const translated = streamed
? codexStreamingJournalItem(active.item, streamed.text)
: codexJournalItem(active.item)
const body = interruptedBody(translated.body)
if (body) {
mutations.push(settledRow(input.linkageFor, active, body))
}
activeItemsToForget.push({ key, threadId: active.threadId, itemId: active.item.id })
}
if (mutations.length === 0) {
return ADMITTED
}
const admission = appendCodexLifecycleMutations(
input.sink,
`oversized-notification:${input.sessionId}:${input.threadId}:${input.method}`,
mutations
)
if (!admission.accepted) {
return admission
}
for (const active of activeItemsToForget) {
input.streams.forget(active.threadId, active.itemId)
input.activeItems.delete(active.key)
}
return ADMITTED
}
function oversizedStreamItemType(method: string): CodexThreadItem['type'] | null {
if (method === 'item/agentMessage/delta') {
return 'agentMessage'
}
if (method === 'item/plan/delta') {
return 'plan'
}
if (
method === 'command/exec/outputDelta' ||
method === 'process/outputDelta' ||
method === 'item/commandExecution/outputDelta' ||
method === 'item/commandExecution/terminalInteraction'
) {
return 'commandExecution'
}
if (method === 'item/fileChange/outputDelta' || method === 'item/fileChange/patchUpdated') {
return 'fileChange'
}
if (
method === 'item/reasoning/summaryTextDelta' ||
method === 'item/reasoning/summaryPartAdded' ||
method === 'item/reasoning/textDelta'
) {
return 'reasoning'
}
return null
}
/** A settled item or prompt, naming its producer: the settlement can be the row's first write. */
function settledRow(
linkageFor: CodexRowLinkage,
src/main/codex/codex-structured-journal-translation-frames.ts
-68
View File
@@ -1,68 +0,0 @@
/**
* The translator's provider-frame arms.
*
* Each returns null for a frame it does not own, which is the translator's
* signal to keep looking. Split out so the translator reads as routing rather
* than as the shape checks each arm performs.
*/
import type { CodexStructuredSessionEvent } from './codex-structured-session-adapter'
import type { CodexJournalItems } from './codex-structured-journal-items'
import type { CodexJournalTranslationAdmission } from './codex-structured-journal-contracts'
import { settleCodexOversizedNotification } from './codex-structured-journal-settlement'
import {
readCodexJournalRecord,
readCodexJournalString
} from './codex-structured-journal-translation-values'
type OversizedInput = Parameters<typeof settleCodexOversizedNotification>[0]
/** A notification the transport refused to carry whole: settle whatever it
* opened rather than leaving the item mid-flight. */
export function settleCodexOversizedNotificationFrame(input: {
sessionId: string
threadId: string
kind: string
payload: unknown
sink: OversizedInput['sink']
streams: OversizedInput['streams']
activeItems: OversizedInput['activeItems']
linkageFor: OversizedInput['linkageFor']
}): CodexJournalTranslationAdmission | null {
if (input.kind !== 'frame:oversized-notification') {
return null
}
const method = readCodexJournalString(readCodexJournalRecord(input.payload), 'method')
return method
? settleCodexOversizedNotification({
sessionId: input.sessionId,
threadId: input.threadId,
method,
sink: input.sink,
streams: input.streams,
activeItems: input.activeItems,
linkageFor: input.linkageFor
})
: null
}
export function createCodexOversizedNotificationSettler(
deps: { sink: OversizedInput['sink']; linkageFor: OversizedInput['linkageFor'] },
items: Pick<CodexJournalItems, 'streams' | 'activeItems'>
) {
return settleOversizedNotification
/** Settles the item a notification the transport refused to carry left
* mid-flight; null when the frame is not one. */
function settleOversizedNotification(
event: Extract<CodexStructuredSessionEvent, { type: 'provider-frame' }>
): CodexJournalTranslationAdmission | null {
return settleCodexOversizedNotificationFrame({
...event,
sink: deps.sink,
streams: items.streams,
activeItems: items.activeItems,
linkageFor: deps.linkageFor
})
}
}
src/main/codex/codex-structured-journal-translation-streams.test.ts
-49
View File
@@ -215,55 +215,6 @@ describe('codex journal translation', () => {
).toEqual(['notification:future/notification', 'request:future/request', 'frame:unclassified'])
})
it('terminalizes the active streamed item when an oversized notification is rejected', () => {
const { translator, tap } = translatorWith()
translator.handle(TURN_STARTED)
translator.handle(
notification('item/started', {
item: {
type: 'commandExecution',
id: 'exec-oversized',
command: 'run',
status: 'inProgress'
}
})
)
const admission = translator.handle({
type: 'provider-frame',
sessionId: SESSION_ID,
threadId: THREAD_ID,
kind: 'frame:oversized-notification',
payload: {
reason: 'record-too-large',
observedBytes: 20 * 1024 * 1024,
maxBytes: 16 * 1024 * 1024,
classification: 'notification',
method: 'item/commandExecution/outputDelta'
}
})
expect(admission).toEqual({ accepted: true })
expect(tap.rows).toEqual([
expect.objectContaining({
body: expect.objectContaining({ kind: 'tool-call', state: 'running' })
}),
expect.objectContaining({
body: expect.objectContaining({ kind: 'tool-call', state: 'failed' })
}),
expect.objectContaining({
body: expect.objectContaining({
kind: 'status',
providerFrame: expect.objectContaining({ kind: 'frame:oversized-notification' })
})
})
])
const diagnostic = tap.rows[2]?.body
expect(
diagnostic?.kind === 'status' ? diagnostic.providerFrame?.payload.byteLength : 0
).toBeGreaterThan(0)
expect(JSON.stringify(diagnostic)).toContain('record-too-large')
})
it('admits suppressed diagnostics before settling a completed turn', () => {
const tap = recorder()
let rejectSuppression = true
src/main/codex/codex-structured-journal-translation-turn-boundaries.ts
+8 -50
View File
@@ -140,6 +140,8 @@ export class CodexJournalTurnBoundaries {
return admission
}
/** Codex ends every turn with exactly one `turn/completed`, a failed one after
* its `error` frame included, so this is the only live end. */
complete(event: TurnBoundaryEvent): CodexJournalTranslationAdmission {
const suppressionAdmission = this.deps.flushSuppression()
if (!suppressionAdmission.accepted) {
@@ -154,58 +156,14 @@ export class CodexJournalTurnBoundaries {
// turn boundary is no evidence contact was lost. Only `settleSession` may
// write `unverifiable`.
const status = readCodexTurnStatus(event.params)
return this.end(event, turnId, {
state: codexTurnLifecycleState(status),
outcome: codexTurnOutcome(status),
completedAt: this.receiptTime(event),
durationMs: readCodexTurnDurationMs(event.params)
})
}
/**
* Settles the turn a terminal `error` names.
*
* Codex reports a fault that ended a turn as an `error` notification carrying
* that turn's id, and `turn/completed` may never follow it — the app server
* marks the thread not-running off the error alone. Without this the running
* lifecycle row is a latch nothing re-derives, and the chat reads "Working"
* for the life of the session. A retrying stream error is NOT a turn end and
* never reaches here.
*/
fail(event: TurnBoundaryEvent): CodexJournalTranslationAdmission {
const suppressionAdmission = this.deps.flushSuppression()
if (!suppressionAdmission.accepted) {
return suppressionAdmission
}
const turnId = readCodexTurnId(event.params) ?? this.deps.activeTurns.current(event.threadId)
// An error ends only a turn this host saw open; its `turn/completed` settles any other.
if (!turnId || !this.deps.activeTurns.isActive(event.threadId, turnId)) {
return CODEX_JOURNAL_ADMITTED
}
return this.end(event, turnId, {
state: 'completed',
outcome: 'failure',
completedAt: this.receiptTime(event)
})
}
/**
* A turn's first terminal settlement is final. Codex follows a turn-ending
* `error` with a failed `turn/completed` for the same turn, and by then the
* start and attributed send this row carries are forgotten, so a second end
* could only overwrite the record with less.
*/
private end(
event: TurnBoundaryEvent,
turnId: string,
terminal: TurnTerminal
): CodexJournalTranslationAdmission {
if (this.recentTurns.has(event.threadId, turnId)) {
return CODEX_JOURNAL_ADMITTED
}
const turnLifecycle =
event.threadId === this.deps.primaryThreadId()
? this.settled(event.threadId, turnId, terminal)
? this.settled(event.threadId, turnId, {
state: codexTurnLifecycleState(status),
outcome: codexTurnOutcome(status),
completedAt: this.receiptTime(event),
durationMs: readCodexTurnDurationMs(event.params)
})
: null
const requestOrigin = this.deps.activeTurns.requestOrigin(event.threadId, turnId)
const latestDispatchSequence = this.deps.activeTurns.latestDispatchSequence(
src/main/codex/codex-structured-journal-translation-turn-lifecycle.test.ts
+52 -11
View File
@@ -346,7 +346,7 @@ describe('codex turn lifecycle rows', () => {
const lifecycle = reduced(tap.rows).find((row) => row.key === LIFECYCLE_KEY)
expect(lifecycle?.body).toMatchObject({
kind: 'turn',
state: 'interrupted',
state: 'completed',
outcome: 'failure',
requestedAt: 900
})
@@ -381,17 +381,17 @@ describe('codex turn lifecycle rows', () => {
})
// `TurnStatus` in the app-server protocol is `completed | interrupted | failed |
// inProgress`, and every one of those collapses to the same terminal lifecycle
// arm. `outcome` is what keeps a Codex failure distinguishable from a stop, and
// a status this build cannot place stays unknown rather than borrowing one.
// inProgress`. Only `interrupted` is a stop; every other end completed the
// turn, and `outcome` says how. A status this build cannot place stays unknown
// rather than borrowing a verdict.
it.each([
['interrupted', 'cancellation'],
['failed', 'failure'],
['cancelled', undefined],
['inProgress', undefined]
['interrupted', 'interrupted', 'cancellation'],
['failed', 'completed', 'failure'],
['someFutureStatus', 'completed', undefined],
['inProgress', 'completed', undefined]
] as const)(
'maps a %s turn status to an interrupted lifecycle with outcome %s',
(status, outcome) => {
'maps a %s turn status to a %s lifecycle with outcome %s',
(status, state, outcome) => {
const tap = recorder()
const translator = translatorFor(tap)
@@ -405,7 +405,7 @@ describe('codex turn lifecycle rows', () => {
body: {
kind: 'turn',
turnId: TURN_ID,
state: 'interrupted',
state,
...(outcome ? { outcome } : {}),
userItemId: USER_ITEM_ID,
startedAt: 1_000,
@@ -512,6 +512,21 @@ describe('codex turn lifecycle rows', () => {
completedAt: 1_700_000_101,
items: []
},
{
id: 'turn-failed',
status: 'failed',
startedAt: 1_700_000_150,
completedAt: 1_700_000_152,
durationMs: 2_400,
items: []
},
{
id: 'turn-unplaced',
status: 'someFutureStatus',
startedAt: 1_700_000_170,
completedAt: 1_700_000_171,
items: []
},
{ id: 'turn-open', status: 'inProgress', startedAt: 1_700_000_200, items: [] },
{ id: 'turn-untimed', status: 'completed', items: [] }
]
@@ -544,6 +559,32 @@ describe('codex turn lifecycle rows', () => {
startedAt: 1_700_000_100_000,
completedAt: 1_700_000_101_000
}
},
{
// The same shape a live failed completion writes.
key: 'legacy:codex:session-1:turn-lifecycle%3Aturn-failed',
body: {
kind: 'turn',
turnId: 'turn-failed',
state: 'completed',
outcome: 'failure',
userItemId: 'codex:thread-abc:turn-failed:0',
startedAt: 1_700_000_150_000,
completedAt: 1_700_000_152_000,
durationMs: 2_400
}
},
{
// Ended, but not a status this build can place: no verdict, never a clean finish.
key: 'legacy:codex:session-1:turn-lifecycle%3Aturn-unplaced',
body: {
kind: 'turn',
turnId: 'turn-unplaced',
state: 'completed',
userItemId: 'codex:thread-abc:turn-unplaced:0',
startedAt: 1_700_000_170_000,
completedAt: 1_700_000_171_000
}
}
])
expect(tap.tombstones).toEqual([])
src/main/codex/codex-structured-journal-translation-turn-settles-once.test.ts
+114 -25
View File
@@ -21,13 +21,28 @@ type Row = { key: string; body: AgentJournalItemBody }
function recorder() {
const rows: Row[] = []
let refuseTerminal = 0
const sink: StructuredAgentSessionEventSink = {
appendItem: (identity: AgentJournalItemIdentity, body) =>
rows.push({ key: agentJournalItemKey(identity), body }),
tryAppendItem: (identity: AgentJournalItemIdentity, body) => {
if (body.kind === 'turn' && body.state !== 'running' && refuseTerminal > 0) {
refuseTerminal -= 1
return { accepted: false, reason: 'backpressure' }
}
rows.push({ key: agentJournalItemKey(identity), body })
return { accepted: true }
},
appendTombstone: () => {},
publish: () => {}
}
return { sink, rows }
return {
sink,
rows,
refuseNextTerminalWrite: () => {
refuseTerminal += 1
}
}
}
function notification(method: string, params: unknown, observedAt: number) {
@@ -50,11 +65,15 @@ function translator(tap: ReturnType<typeof recorder>) {
})
}
/** Every terminal write for a turn, in order: what a reader could observe. */
/** Every lifecycle write for a turn, in order: what a reader could observe. */
function turnWrites(rows: readonly Row[], turnId: string) {
return rows.flatMap((row) =>
row.body.kind === 'turn' && row.body.turnId === turnId ? [row.body] : []
)
}
function terminalWrites(rows: readonly Row[], turnId: string) {
return rows
.map((row) => row.body)
.filter((body) => body.kind === 'turn' && body.turnId === turnId && body.state !== 'running')
return turnWrites(rows, turnId).filter((body) => body.state !== 'running')
}
/** The body the journal reducer keeps for a turn's lifecycle row. */
@@ -109,8 +128,8 @@ function runFailedTurn(handle: (event: CodexStructuredSessionEvent) => unknown)
)
}
describe('a Codex turn settles once', () => {
it('keeps the failure the error settled when the failed completion follows it', () => {
describe('a Codex turn ends once, on its completion', () => {
it('records the failed completion Codex sends after the error, with the start this host saw', () => {
const tap = recorder()
const codex = translator(tap)
@@ -125,35 +144,105 @@ describe('a Codex turn settles once', () => {
userItemId: agentJournalSubmissionKey(CLIENT_MESSAGE_ID),
startedAt: 1_000,
requestedAt: 900,
completedAt: 2_000
completedAt: 2_100,
durationMs: 1_100
})
expect(
tap.rows.filter((row) => row.body.kind === 'status' && row.body.tone === 'error')
).toHaveLength(1)
})
it('ignores a duplicate completion for a turn it already settled', () => {
it('records an exit between the error and the completion as interrupted, with no verdict', () => {
// Orca records only an end it observed. The exit is that end; the failure
// Codex would have named arrives only in the completion, which never came.
const tap = recorder()
const codex = translator(tap)
const events: CodexStructuredSessionEvent[] = []
runFailedTurn((event) => events.push(event))
for (const event of events.slice(0, -1)) {
codex.handle(event)
}
codex.handle({
type: 'ended',
sessionId: SESSION_ID,
reason: 'lost child',
cause: 'unexpected-exit',
fence: 1,
acquisitionGeneration: 'generation-1',
observedAt: 2_050
})
expect(terminalWrites(tap.rows, TURN_ID)).toEqual([
{
kind: 'turn',
turnId: TURN_ID,
state: 'interrupted',
userItemId: agentJournalSubmissionKey(CLIENT_MESSAGE_ID),
startedAt: 1_000,
requestedAt: 900,
completedAt: 2_050
}
])
expect(
tap.rows.filter((row) => row.body.kind === 'status' && row.body.tone === 'error')
).toHaveLength(1)
})
it('revises the still-open turn with a send Codex echoes between the error and the completion', () => {
// Codex records a failed turn's pending input after its `error` frame and
// before `turn/completed`, so the echo belongs to a turn that is still open.
const tap = recorder()
const codex = translator(tap)
const events: CodexStructuredSessionEvent[] = []
runFailedTurn((event) => events.push(event))
const [started, echo, reply, error, completion] = events
for (const event of [started, reply, error, echo, completion]) {
if (event) {
codex.handle(event)
}
}
expect(turnWrites(tap.rows, TURN_ID).map((body) => body.state)).toEqual([
'running',
'running',
'completed'
])
expect(settledRecord(tap.rows, TURN_ID)).toMatchObject({
outcome: 'failure',
userItemId: agentJournalSubmissionKey(CLIENT_MESSAGE_ID),
requestedAt: 900,
startedAt: 1_000,
durationMs: 1_100
})
})
it('settles once when the sink refuses the completion and its retry lands', () => {
// A refused frame is Orca's only redelivery of a completion, and a refused
// end changes nothing, so the retry settles the turn exactly once.
const tap = recorder()
const codex = translator(tap)
const completion = notification(
'turn/completed',
{ turn: { id: TURN_ID, status: 'completed', durationMs: 900 } },
2_000
)
codex.handle(notification('turn/started', { turn: { id: TURN_ID } }, 1_000))
codex.handle(
notification(
'turn/completed',
{ turn: { id: TURN_ID, status: 'completed', durationMs: 900 } },
2_000
)
)
codex.handle(notification('turn/completed', { turn: { id: TURN_ID, status: 'failed' } }, 3_000))
tap.refuseNextTerminalWrite()
expect(codex.handle(completion)).toEqual({ accepted: false, reason: 'backpressure' })
expect(codex.handle(completion)).toEqual({ accepted: true })
expect(terminalWrites(tap.rows, TURN_ID)).toHaveLength(1)
expect(settledRecord(tap.rows, TURN_ID)).toMatchObject({
state: 'completed',
outcome: 'success',
startedAt: 1_000,
completedAt: 2_000,
durationMs: 900
})
expect(terminalWrites(tap.rows, TURN_ID)).toEqual([
expect.objectContaining({
state: 'completed',
outcome: 'success',
startedAt: 1_000,
completedAt: 2_000,
durationMs: 900
})
])
})
it('settles an ordinary turn exactly as before', () => {
src/main/codex/codex-structured-journal-translation-turn-state.ts
+1 -12
View File
@@ -57,12 +57,6 @@ export class CodexJournalActiveTurns {
return [...(this.byThread.get(threadId) ?? [])].at(-1) ?? null
}
/** Whether this turn is still open here. A terminal row already written carries
* the turn's start and duration, so a later end must not overwrite it. */
isActive(threadId: string, turnId: string): boolean {
return this.byThread.get(threadId)?.has(turnId) === true
}
startedAt(threadId: string, turnId: string): number | undefined {
return this.startedAtByTurn.get(this.turnKey(threadId, turnId))
}
@@ -167,8 +161,7 @@ type RecentTurn = {
bytes: number
}
/** Bounded terminal lifecycle window: exact echoes that arrive after completion
* revise it, and a later end for a turn in it is not a second settlement. */
/** Bounded terminal lifecycle window: exact echoes that arrive after completion revise it. */
export class CodexJournalRecentTurns {
private readonly turns = new Map<string, RecentTurn>()
private retainedBytes = 0
@@ -232,10 +225,6 @@ export class CodexJournalRecentTurns {
}
}
has(threadId: string, turnId: string): boolean {
return this.turns.has(this.turnKey(threadId, turnId))
}
requestOriginRevision(
threadId: string,
turnId: string,
src/main/codex/codex-structured-journal-translation-turns.ts
+3 -3
View File
@@ -43,12 +43,12 @@ export function codexTurnLifecycleBody(
return agentJournalTurnBody(turnLifecycle)
}
/** Maps a `turn/completed` status; a missing one is a clean finish. A terminal
* `error` also ends a turn and names its own outcome rather than coming here. */
/** Maps a `turn/completed` status, live or restored. Only `interrupted` is a stop;
* a failed turn completed, and `codexTurnOutcome` says it failed. */
export function codexTurnLifecycleState(
status: string | null
): Extract<AgentJournalTurnLifecycleState, 'completed' | 'interrupted'> {
return status === null || status === 'completed' ? 'completed' : 'interrupted'
return status === 'interrupted' ? 'interrupted' : 'completed'
}
/**
src/main/codex/codex-structured-journal-translation-writers.ts
+1 -3
View File
@@ -8,7 +8,6 @@ import { CodexJournalGenericFrames } from './codex-structured-journal-generic-fr
import { CodexJournalGoals } from './codex-structured-journal-goals'
import { CodexJournalItems } from './codex-structured-journal-items'
import { CodexJournalPrompts } from './codex-structured-journal-prompts'
import { createCodexOversizedNotificationSettler } from './codex-structured-journal-translation-frames'
import { CodexJournalActiveTurns } from './codex-structured-journal-translation-turn-state'
import { CodexSubagentRoster } from './codex-subagent-roster'
@@ -39,7 +38,6 @@ export function createCodexJournalTranslatorWriters(deps: CodexJournalTranslator
producerDeps,
(threadId, itemId) => items.detailFor(threadId, itemId),
activeTurn
),
settleOversizedNotification: createCodexOversizedNotificationSettler(producerDeps, items)
)
}
}
src/main/codex/codex-structured-journal-translation.test.ts
+22 -31
View File
@@ -287,7 +287,7 @@ describe('codex journal translation', () => {
).toBe('idle')
})
describe('a terminal error ends the turn it names', () => {
describe('a turn-ending error is a row on the turn it names, and the completion ends it', () => {
function statusOf(rows: readonly Row[]) {
return projectStructuredAgentSessionStatus(
reduced(rows).map((row, sequence) => ({
@@ -308,7 +308,11 @@ describe('codex journal translation', () => {
})
}
it('settles the turn and keeps the provider sentence as its own row', () => {
const FAILED_COMPLETION = notification('turn/completed', {
turn: { id: TURN_ID, status: 'failed', durationMs: 2_400 }
})
it('keeps the turn working through the error, and the failed completion settles it', () => {
const tap = recorder()
const translator = createCodexJournalTranslator({
sink: tap.sink,
@@ -316,17 +320,21 @@ describe('codex journal translation', () => {
})
translator.handle(TURN_STARTED)
expect(statusOf(tap.rows)).toBe('working')
translator.handle(errorNotification({ turnId: TURN_ID, willRetry: false }))
expect(statusOf(tap.rows)).toBe('working')
expect(tap.rows.filter((row) => row.body.kind === 'turn')).toHaveLength(1)
translator.handle(FAILED_COMPLETION)
expect(statusOf(tap.rows)).toBe('idle')
expect(reduced(tap.rows).map((row) => row.body)).toContainEqual(
expect.objectContaining({
kind: 'turn',
turnId: TURN_ID,
state: 'completed',
outcome: 'failure'
outcome: 'failure',
durationMs: 2_400
})
)
// The message the user reads is still a row of its own.
@@ -353,7 +361,7 @@ describe('codex journal translation', () => {
expect(tap.rows.filter((row) => row.body.kind === 'turn')).toHaveLength(1)
})
it('does not overwrite the terminal row of a turn that already completed', () => {
it('writes only its row for an error after the turn completed', () => {
const tap = recorder()
const translator = createCodexJournalTranslator({
sink: tap.sink,
@@ -361,21 +369,16 @@ describe('codex journal translation', () => {
})
translator.handle(TURN_STARTED)
translator.handle(
notification('turn/completed', {
turn: { id: TURN_ID, status: 'failed', durationMs: 4_000 }
})
)
translator.handle(FAILED_COMPLETION)
const settled = reduced(tap.rows).find((row) => row.body.kind === 'turn')?.body
translator.handle(errorNotification({ turnId: TURN_ID, willRetry: false }))
// The completion carries the duration a late error could not reconstruct.
expect(reduced(tap.rows).find((row) => row.body.kind === 'turn')?.body).toEqual(settled)
expect(statusOf(tap.rows)).toBe('idle')
})
it('settles the running turn when the error names no turn', () => {
it('settles nothing when the error names no turn', () => {
const tap = recorder()
const translator = createCodexJournalTranslator({
sink: tap.sink,
@@ -385,21 +388,8 @@ describe('codex journal translation', () => {
translator.handle(TURN_STARTED)
translator.handle(errorNotification({ willRetry: false }))
expect(statusOf(tap.rows)).toBe('idle')
})
it('does not reopen a running row when the completion arrives after the error', () => {
const tap = recorder()
const translator = createCodexJournalTranslator({
sink: tap.sink,
primaryThreadId: () => THREAD_ID
})
translator.handle(TURN_STARTED)
translator.handle(errorNotification({ turnId: TURN_ID, willRetry: false }))
translator.handle(notification('turn/completed', { turn: { id: TURN_ID, status: 'failed' } }))
expect(statusOf(tap.rows)).toBe('idle')
expect(statusOf(tap.rows)).toBe('working')
expect(tap.rows.filter((row) => row.body.kind === 'turn')).toHaveLength(1)
})
})
@@ -446,14 +436,12 @@ describe('codex journal translation', () => {
expect(stopped).toEqual([])
})
it('releases after systemError arrives before the terminal error notification', () => {
it('releases after the failed completion that follows systemError and the error', () => {
const stopped: string[] = []
const { translator } = translatorReporting(stopped)
translator.handle(TURN_STARTED)
translator.handle(statusChanged('systemError'))
expect(stopped).toEqual([])
translator.handle(
notification('error', {
turnId: TURN_ID,
@@ -461,6 +449,9 @@ describe('codex journal translation', () => {
error: { message: 'fatal' }
})
)
expect(stopped).toEqual([])
translator.handle(notification('turn/completed', { turn: { id: TURN_ID, status: 'failed' } }))
expect(stopped).toEqual([THREAD_ID])
})
src/main/codex/codex-structured-journal-translation.ts
+10 -28
View File
@@ -11,9 +11,8 @@ import { restoreCodexJournalThread } from './codex-structured-journal-translatio
import { CodexJournalTurnBoundaries } from './codex-structured-journal-translation-turn-boundaries'
import { createCodexJournalTranslatorWriters } from './codex-structured-journal-translation-writers'
import { publishCodexTurnLifecycle } from './codex-structured-journal-translation-turns'
import { readCodexProviderVerdict } from './codex-structured-journal-provider-verdicts'
import { createCodexThreadItemRouter } from './codex-structured-journal-thread-item-routing'
import { readCodexTurnId } from './codex-structured-thread-facts'
import { codexThreadStoppedRunning, readCodexTurnId } from './codex-structured-thread-facts'
import type { CodexStructuredSessionEvent } from './codex-structured-session-adapter'
export type {
@@ -36,17 +35,8 @@ export {
export function createCodexJournalTranslator(
deps: CodexJournalTranslatorDeps
): CodexJournalTranslator {
const {
activeTurns,
subagents,
linkageFor,
genericFrames,
items,
compactions,
goals,
prompts,
settleOversizedNotification
} = createCodexJournalTranslatorWriters(deps)
const { activeTurns, subagents, linkageFor, genericFrames, items, compactions, goals, prompts } =
createCodexJournalTranslatorWriters(deps)
const flushStreams = (): CodexJournalTranslationAdmission =>
items.streams.flush() ? CODEX_JOURNAL_ADMITTED : { accepted: false, reason: 'backpressure' }
let readActivity = createCodexProviderActivityReader()
@@ -205,10 +195,6 @@ export function createCodexJournalTranslator(
)
}
if (event.type === 'provider-frame') {
const settlement = settleOversizedNotification(event)
if (settlement && !settlement.accepted) {
return settlement
}
return genericFrames.appendUnhandled(event.kind, event.payload, event.threadId)
}
if (event.method === 'turn/started' || event.method === 'turn/completed') {
@@ -249,28 +235,24 @@ export function createCodexJournalTranslator(
return publishActivity(event, routed)
}
}
const verdict = readCodexProviderVerdict(event.method, event.params)
// A thread that stopped running settles no open turn: Codex clears `running`
// on every error, and an open turn ends on its `turn/completed`. It releases
// a send whose dispatch was never answered, which nothing else re-derives live.
if (
verdict === 'thread-stopped-running' &&
event.method === 'thread/status/changed' &&
codexThreadStoppedRunning(event.params) &&
event.threadId === (deps.primaryThreadId?.() ?? null)
) {
primaryThreadStoppedRunning = true
reportPrimaryThreadStoppedRunning()
}
// The row carries the provider's sentence and is written first, so it lands
// inside the turn this same frame is about to end.
// A turn-ending `error` is a row inside the turn it names; the failed
// `turn/completed` Codex sends after it is that turn's end.
const unhandled = genericFrames.appendUnhandled(
`notification:${event.method}`,
event.params,
event.threadId
)
if (unhandled.accepted && verdict === 'turn-failed') {
const failed = turnBoundaries.fail(event)
if (!failed.accepted) {
return failed
}
reportPrimaryThreadStoppedRunning()
}
return publishActivity(event, unhandled)
},
cancelPrompt: (journalItemId) => prompts.cancel(journalItemId),
src/main/codex/codex-structured-journal-turn-endings-replay.test.ts
+206
View File
@@ -0,0 +1,206 @@
// Real `codex app-server` frame orders, replayed through the translator.
//
// The fixture is trimmed from captures of the real binary on 0.141.0 (the oldest
// version Orca exercises) and 0.158.0, with only the upstream provider faked:
// statuses, `willRetry`, the provider's sentence and `durationMs`, at the host
// receipt time `t` each frame arrived. Ids are replaced with synthetic ones.
import { readFileSync } from 'node:fs'
import { join } from 'node:path'
import { describe, expect, it } from 'vitest'
import type {
AgentJournalItemBody,
AgentJournalItemIdentity
} from '../../shared/agent-session-journal-types'
import { agentJournalItemKey } from '../../shared/agent-session-journal-item-key'
import type { StructuredAgentSessionEventSink } from '../native-chat/agent-session-wire/structured-agent-session-event-sink'
import { createCodexJournalTranslator } from './codex-structured-journal-translation'
import {
readCodexThreadId,
readCodexTurnDurationMs,
readCodexTurnId,
readCodexTurnStatus
} from './codex-structured-thread-facts'
const SESSION_ID = 'session-1'
type CapturedFrame = { case: string; t: number; method: string; params: Record<string, unknown> }
type Row = { key: string; body: AgentJournalItemBody }
const FRAMES: CapturedFrame[] = readFileSync(
join(__dirname, '__fixtures__', 'codex-app-server-turn-endings.jsonl'),
'utf8'
)
.split('\n')
.filter((line) => line.length > 0)
.map((line) => JSON.parse(line))
function framesOf(name: string): CapturedFrame[] {
const frames = FRAMES.filter((frame) => frame.case === name)
expect(frames.length).toBeGreaterThan(0)
return frames
}
/** Replays a case's frames, or only its first `count`. */
function replay(name: string, count = Number.POSITIVE_INFINITY) {
const frames = framesOf(name).slice(0, count)
const threadId = readCodexThreadId(frames[0]?.params) ?? ''
const rows: Row[] = []
const sink: StructuredAgentSessionEventSink = {
appendItem: (identity: AgentJournalItemIdentity, body) =>
rows.push({ key: agentJournalItemKey(identity), body }),
appendTombstone: () => {},
publish: () => {}
}
const translator = createCodexJournalTranslator({
sink,
sessionId: SESSION_ID,
primaryThreadId: () => threadId
})
for (const frame of frames) {
expect(
translator.handle({
type: 'notification',
sessionId: SESSION_ID,
threadId,
method: frame.method,
params: frame.params,
observedAt: frame.t
})
).toEqual({ accepted: true })
}
return { frames, rows, translator }
}
function turnWrites(rows: readonly Row[], turnId: string) {
return rows
.map((row) => row.body)
.filter((body) => body.kind === 'turn' && body.turnId === turnId)
}
function terminalWrites(rows: readonly Row[], turnId: string) {
return turnWrites(rows, turnId).filter((body) => body.kind === 'turn' && body.state !== 'running')
}
function receipt(frames: readonly CapturedFrame[], method: string, turnId: string): number {
const frame = frames.find(
(entry) => entry.method === method && readCodexTurnId(entry.params) === turnId
)
if (!frame) {
throw new Error(`no ${method} for ${turnId}`)
}
return frame.t
}
function completion(frames: readonly CapturedFrame[], turnId: string) {
const frame = frames.find(
(entry) => entry.method === 'turn/completed' && readCodexTurnId(entry.params) === turnId
)
return frame
? {
t: frame.t,
status: readCodexTurnStatus(frame.params),
durationMs: readCodexTurnDurationMs(frame.params)
}
: null
}
const VERDICT: Record<string, { state: string; outcome: string }> = {
completed: { state: 'completed', outcome: 'success' },
failed: { state: 'completed', outcome: 'failure' },
interrupted: { state: 'interrupted', outcome: 'cancellation' }
}
const ENDED_CASES = [...new Set(FRAMES.map((frame) => frame.case))].filter(
(name) => name !== '0.158.0-conn-refused'
)
describe('captured Codex turns end once, on their own completion', () => {
it.each(ENDED_CASES)('%s', (name) => {
const { frames, rows } = replay(name)
const turnIds = frames
.filter((frame) => frame.method === 'turn/started')
.map((frame) => readCodexTurnId(frame.params) ?? '')
expect(turnIds.length).toBeGreaterThan(0)
for (const turnId of turnIds) {
const end = completion(frames, turnId)
expect(end).not.toBeNull()
if (!end) {
continue
}
const verdict = VERDICT[end.status ?? '']
expect(verdict).toBeDefined()
// One terminal write per turn, and it is Codex's own completion: its time,
// its duration and its verdict, with the start this host saw.
expect(terminalWrites(rows, turnId)).toEqual([
expect.objectContaining({
state: verdict?.state,
outcome: verdict?.outcome,
startedAt: receipt(frames, 'turn/started', turnId),
completedAt: end.t,
durationMs: end.durationMs
})
])
}
})
it('keeps every failed turn working through its error until the completion', () => {
for (const name of ENDED_CASES) {
const frames = framesOf(name)
const failedTurns = frames
.filter((frame) => frame.method === 'error' && frame.params.willRetry === false)
.map((frame) => readCodexTurnId(frame.params) ?? '')
for (const turnId of failedTurns) {
const errorAt = frames.findIndex(
(frame) =>
frame.method === 'error' &&
frame.params.willRetry === false &&
readCodexTurnId(frame.params) === turnId
)
// Through the error, the turn is open; the completion right after it ends it.
expect(terminalWrites(replay(name, errorAt + 1).rows, turnId)).toEqual([])
expect(terminalWrites(replay(name).rows, turnId)).toHaveLength(1)
}
}
})
it('keeps a turn Codex is still retrying on 0.158.0 running, and the exit sweep ends it', () => {
const { frames, rows, translator } = replay('0.158.0-conn-refused')
const turnId =
readCodexTurnId(frames.find((frame) => frame.method === 'turn/started')?.params) ?? ''
const retries = frames.filter((frame) => frame.method === 'error')
// Seven "Reconnecting... waiting for network" frames over four minutes, all
// willRetry, and no completion: the turn is genuinely still open.
expect(retries).toHaveLength(7)
expect(retries.every((frame) => frame.params.willRetry === true)).toBe(true)
expect(turnWrites(rows, turnId)).toEqual([
expect.objectContaining({
state: 'running',
startedAt: receipt(frames, 'turn/started', turnId)
})
])
translator.handle({
type: 'ended',
sessionId: SESSION_ID,
reason: 'lost child',
cause: 'unexpected-exit',
fence: 1,
acquisitionGeneration: 'generation-1',
observedAt: 250_000
})
expect(terminalWrites(rows, turnId)).toEqual([
{
kind: 'turn',
turnId,
state: 'interrupted',
userItemId: `codex:${readCodexThreadId(frames[0]?.params)}:${turnId}:0`,
startedAt: receipt(frames, 'turn/started', turnId),
completedAt: 250_000
}
])
})
})
src/main/codex/codex-structured-journal-turn-settles-once-journal.test.ts
+47 -9
View File
@@ -73,8 +73,14 @@ async function session() {
}
}
function turnRecord(rows: Awaited<ReturnType<Awaited<ReturnType<typeof session>>['items']>>) {
return rows
.map((item) => readAgentJournalTurn(item.body))
.findLast((record) => record?.turnId === TURN)
}
describe('a failed Codex turn in the journal', () => {
it('keeps the failure and its duration when the failed completion lands while the error is still queued', async () => {
it('keeps the failed completion, its duration and the start, when it lands while the error is still queued', async () => {
const { on, drained, items } = await session()
on('turn/started', { turn: { id: TURN } }, 1_000)
on(
@@ -85,7 +91,7 @@ describe('a failed Codex turn in the journal', () => {
await drained()
// Codex writes both frames back to back; the error's status row is still being
// written when the failed completion arrives, so the error's settlement is queued.
// written when the failed completion arrives.
on(
'error',
{ turnId: TURN, willRetry: false, error: { message: 'stream disconnected' } },
@@ -94,18 +100,50 @@ describe('a failed Codex turn in the journal', () => {
on('turn/completed', { turn: { id: TURN, status: 'failed', durationMs: 1_900 } }, 3_100)
const rows = await items()
const turn = rows
.map((item) => readAgentJournalTurn(item.body))
.findLast((record) => record?.turnId === TURN)
expect(turn).toMatchObject({
expect(turnRecord(rows)).toMatchObject({
state: 'completed',
outcome: 'failure',
startedAt: 1_000,
completedAt: 3_000
completedAt: 3_100,
durationMs: 1_900
})
// The duration "Worked for" shows under the turn's message.
// "Worked for" under the turn's message reads Codex's own duration.
expect(
completedStructuredAgentTurnSeconds(selectStructuredAgentRunningTurnTiming(rows, TURN))
).toBe(2)
).toBe(1)
})
// Codex ends a turn once, so a second completion is not expected. If one came,
// the settlement id keeps the first record, whether it is still queued or written.
it.each([
['still queued', false],
['already written', true]
] as const)(
'keeps the first completion when a second one arrives while the first is %s',
async (_label, drainFirst) => {
const { on, drained, items } = await session()
on('turn/started', { turn: { id: TURN } }, 1_000)
await drained()
// The reply's row is being written, so the first completion waits behind it.
on(
'item/completed',
{ turnId: TURN, item: { type: 'agentMessage', id: 'agent-1', text: 'Done' } },
1_900
)
on('turn/completed', { turn: { id: TURN, status: 'completed', durationMs: 900 } }, 2_000)
if (drainFirst) {
await drained()
}
on('turn/completed', { turn: { id: TURN, status: 'failed' } }, 3_000)
expect(turnRecord(await items())).toMatchObject({
state: 'completed',
outcome: 'success',
startedAt: 1_000,
completedAt: 2_000,
durationMs: 900
})
}
)
})
src/main/codex/codex-structured-thread-facts.ts
-6
View File
@@ -62,12 +62,6 @@ export function readCodexTurnDurationMs(payload: unknown): number | null {
return typeof value === 'number' && Number.isFinite(value) && value >= 0 ? value : null
}
/** `error` carries `willRetry`: Codex sets it on a stream error it is about to
* retry, and omits it (false) on one that ended the turn the frame names. */
export function readCodexErrorWillRetry(payload: unknown): boolean {
return record(payload)?.willRetry === true
}
/** `thread/status/changed` carries a TAGGED status (`{status:{type}}`), never a
* bare string. `idle` and `systemError` are the two arms that mean the thread
* is not running; `active` and `notLoaded` are not. */
src/main/native-chat/agent-session-wire/structured-agent-session-event-sink-queue.ts
+6 -1
View File
@@ -15,6 +15,8 @@ export type StructuredAgentSessionSinkOperation = {
lifecycleBytes?: number
lifecycle?: boolean
coalescingKey?: string
/** The queued operation with this key wins, as the journal keeps a settlement's first batch. */
keepsFirst?: boolean
run: (target: StructuredAgentSessionEventTarget) => Promise<unknown> | void
}
@@ -117,10 +119,13 @@ export class StructuredAgentSessionSinkQueue {
if (this.failure !== null) {
return { accepted: false, reason: 'failed' }
}
const sequence = ++this.acceptedSequence
const key = options.coalescingKey ?? operation.coalescingKey
const replaceAt = key ? this.queue.findIndex((queued) => queued.coalescingKey === key) : -1
const replaced = replaceAt >= 0 ? this.queue[replaceAt] : undefined
if (replaced && operation.keepsFirst) {
return { accepted: true }
}
const sequence = ++this.acceptedSequence
const lifecycle = operation.lifecycle ?? options.lifecycle === true
const lifecycleBytes = lifecycle ? (operation.lifecycleBytes ?? operation.bytes) : 0
const nextBytes = this.queuedBytes - (replaced?.bytes ?? 0) + operation.bytes
src/main/native-chat/agent-session-wire/structured-agent-session-event-sink.test.ts
+26
View File
@@ -384,6 +384,32 @@ describe('deferred structured agent-session event sink', () => {
expect(log).toEqual([{ call: 'appendItem', fence: 6, ordinal: 1 }])
})
it('keeps the first queued lifecycle batch for a settlement, as the journal does', async () => {
// The journal applies a settlement id once and skips any later batch with it,
// so the queue must not let a later batch replace one it has not run yet.
const log: Recorded[] = []
const deferred = createDeferredStructuredAgentSessionEventSink()
const batch = (ordinal: number) => [
{ kind: 'item' as const, identity: identity(ordinal), body: BODY }
]
deferred.sink.appendLifecycleBatch?.('turn-completed:turn-1', batch(0))
expect(deferred.sink.tryAppendLifecycleBatch?.('turn-completed:turn-1', batch(1))).toEqual({
accepted: true
})
expect(deferred.state().queuedOperations).toBe(1)
const bound = target(6, log)
deferred.bind(bound)
await deferred.drained()
expect(
vi
.mocked(bound.journal.appendLifecycleBatch)
.mock.calls.map(([input]) => input.mutations.map((mutation) => mutation.identity))
).toEqual([[identity(0)]])
})
it('keeps a replacement checkpoint after distinct intervening operations', async () => {
const log: Recorded[] = []
const deferred = createDeferredStructuredAgentSessionEventSink()
src/main/native-chat/agent-session-wire/structured-agent-session-event-sink.ts
+1
View File
@@ -205,6 +205,7 @@ export function createDeferredStructuredAgentSessionEventSink(
{
bytes: Buffer.byteLength(JSON.stringify({ settlementId, mutations }), 'utf8') + 512,
coalescingKey: `lifecycle:${settlementId}`,
keepsFirst: true,
run: (bound) =>
bound.journal.appendLifecycleBatch({
settlementId,