Commit Graph
9 Commits
Author SHA1 Message Date
OrcaWin cd05f2ff93 Implement robust orchestration primitives and connected-server workers (#9925) 2026-07-27 12:31:37 -07:00
Jinjing 52388c1ab7 Fix worker done pane identity (#8473)
* fix(orchestration): bind lifecycle sends to dispatched pane

* fix(orchestration): bind injected worker messaging

* Fix worker_done pane identity gating and rename SSH linear CLI files

- Lifecycle reconciliation now returns an explicit `rejected` action
  (instead of `ignored`) for persisted sender_not_assignee markers, and
  only treats a payload's `_orcaLifecycleRejection` as trusted state
  when it exactly matches that reserved shape, preventing caller-
  supplied markers from spoofing a rejected send as success.
- SSH legacy fallback CLI now fails closed with
  `no_active_sender_terminal` when a worker_done/heartbeat send has no
  resolvable sender identity, rejects mixed raw/structured payload
  flags, and reconstructs structured lifecycle payload fields
  (task/dispatch id, files-modified, report-path, phase) instead of
  dropping them — extracted into ssh-remote-orchestration-send.ts.
- Renamed ssh-remote-linear-argument-error.ts to
  ssh-remote-cli-argument-error.ts since RemoteCliArgumentError is now
  shared beyond the Linear CLI.

* Add re-read logging for already-converted lifecycle rejections

Send-path reconcile converts worker_done/heartbeat rejections with a
no-op logger, so the coordinator's later re-read is the only chance
to surface the rejection message. Log it there instead of dropping it.
2026-07-12 22:45:33 -07:00
JinjingandOrca 26934b11bf fix(orchestration): complete tasks on worker_done + coordinator UX fixes (#8030)
* fix(orchestration): complete worker tasks and improve coordinator UX

* Fix orchestration lifecycle sender resolution and peek/check compat hand

- Lifecycle sends (worker_done/heartbeat) now use ORCA_TERMINAL_HANDLE
  verbatim, skipping the liveness probe and pane remint that could
  block delivery during restarts or mismatch stale-runtime assignee
  handles.
- --peek now round-trips as {peek:true, unread:false} so older runtimes
  that strip unknown params degrade to non-destructive "all" instead of
  mark-read, with client-side filtering to restore peek semantics and a
  clear error when --peek --wait can't be honored.
- Reject combined read-mode flags (--unread/--peek/--all) before calling
  the runtime.
- Distinguish suppressed (already-consumed) lifecycle messages from
  ignored ones so send doesn't wake --wait waiters for stale heartbeats.
- Fix task summary truncation to avoid splitting UTF-16 surrogate pairs
  and to not misreport whitespace normalization as truncation.

* Add shared helper to abbreviate orchestration task specs for brief listi

- Normalizes whitespace and caps spec length at 160 chars, flagging
  truncation separately from whitespace-only changes
- Truncates on UTF-16 code point boundaries to avoid splitting
  surrogate pairs and emitting malformed strings

* Add pane-key identity to worker_done/heartbeat reconciliation and server

- Records the sender's pane key on messages and dispatch contexts so
  worker_done/heartbeat ownership can be verified by the remint-stable
  pane leaf instead of the terminal handle, which is reissued across
  restarts.
- Rejects lifecycle messages from a genuinely foreign pane while still
  tolerating handle remints, tab break-outs, and older CLIs that lack
  pane identity.
- Moves task-spec abbreviation server-side (orchestration.taskList
  --brief) so full specs no longer cross SSH/relay transports, with a
  client-side fallback for older runtimes; consolidates the shared
  abbreviation helper under src/shared.
- Adds a stderr warning when a pre-peek runtime's --peek response hits
  the 100-row cap, since older unread messages may be missing.

* Isolate ORCA_PANE_KEY in CLI test beforeEach to fix leaked senderPaneKey

Co-authored-by: Orca <help@stably.ai>

* Fix pane-key remint bypassing dispatch mutual-exclusion lock

- Dispatch locking only matched on assignee_handle, so a reminted
  terminal handle (tab break-out) could open a second concurrent
  dispatch on the same pane.
- Add leaf-UUID-based pane key comparison (parsePaneKey) as a
  secondary lock, falling back to exact handle match for legacy
  rows without pane keys.

* Update orchestration skill docs for lifecycle authority and CLI flag add

- Clarify that dispatch lifecycle is tied to taskId+dispatchId verified against
  the dispatched pane, not the terminal handle, since handles can be reminted
  after restart
- Document new `check --peek`/`--all` and `task-list --brief` flags, with
  fallback guidance for older CLIs that reject them
- Note that a valid worker_done auto-completes the task/dispatch, so workers
  shouldn't also call task-update manually

---------

Co-authored-by: Orca <help@stably.ai>
2026-07-12 02:15:11 -07:00
Jinjing 1419193bea Add explicit task title and display name to orchestration tasks (#5901)
Introduce optional `task-title` and `display-name` parameters for orchestration tasks, persisting them in the database and propagating them through the RPC and Orca runtime.

This allows the CLI, dashboard, activity page, and sidebar to display concise, user-friendly labels for dispatched worker agents instead of verbose, raw system preambles.
2026-06-20 03:29:21 -07:00
Brennan Benson 2acf8549f3 Prevent worker completion reports from reaching unrelated agents (#5333) 2026-06-13 21:28:12 -07:00
Jinwoo Hong 62600ef808 Fix orchestration payloads on Windows (#4608) 2026-06-03 21:50:16 -04:00
Trevin Chow f3b452fff5 Clarify orchestration reset scope flags
Clarify the CLI reset usage string and add focused handler coverage for reset flag mapping.
2026-05-28 16:40:06 -04:00
Jinwoo HongandOrca ea8ea08116 feat(orchestration): bundled improvements — check-wait, stale-base, preamble+ask, QoL (#1403)
* feat(orchestration): transport keepalive + delivered_at split for check --wait

Implements the four §3 fixes from the check-wait design doc:

- §3.1 Transport keepalive: long-poll RPCs (orchestration.check --wait) emit
  `{"_keepalive":true}` frames every 10s so neither server nor client tears
  the socket down on idle. A `longPoll` admission counter capped at 16 fails
  fast with `runtime_busy` when saturated; an AbortController wired through
  the RPC dispatcher cancels the inner waiter the moment the socket closes.
- §3.2 delivered_at split: push-on-idle now stamps `delivered_at` instead of
  flipping `read`, so the check caller remains the sole consumer of its
  queue. Adds a synchronous idempotent schema migration that hard-fails on
  error.
- §3.3 inbox/check parity: `orchestration inbox --terminal <handle>` and
  `orchestration check --all` agree on the same rows (sequence DESC, no
  mark-read). `check --unread=false` kept for one release as a compat shim.
- §3.4 CLI heartbeat: `orca orchestration check --wait` emits JSON heartbeat
  lines to stderr every 15s so Claude Code's Bash tool sees continuous
  output and doesn't auto-background the subprocess.

Tests: extends runtime-rpc, orca-runtime, envelope-schema, orchestration
method, and formatter suites; adds a subprocess test that spawns the built
CLI and verifies stderr line-flushing, heartbeat cadence, and stdout
cleanliness end-to-end.

Co-authored-by: Orca <help@stably.ai>

* feat(orchestration): preamble rules + heartbeat schema

- Preamble (#7, #15, #9): worker_done body ("3-sentence summary" + reportPath),
  BEHAVIOR RULE #1 forbidding AskUserQuestion, heartbeat every 5 minutes with
  taskId+dispatchId payload, AFTER YOU SEND grace window.
- Schema v2 migration: adds 'heartbeat' to messages.type CHECK, adds
  dispatch_contexts.last_heartbeat_at, gated by user_version PRAGMA with
  transactional rebuild + explicit CREATE INDEX to avoid silent perf regress.
- DB helpers: recordHeartbeat (dispatched-only), getStaleDispatches,
  getThreadMessagesFor (thread+handle scoped for ask).

Co-authored-by: Orca <help@stably.ai>

* feat(orchestration): coordinator heartbeat + stale detector

Handle incoming 'heartbeat' messages by calling recordHeartbeat keyed on
payload.dispatchId (strict — log-and-skip if missing, no taskId fallback so
a straggler heartbeat from a previously-failed dispatch cannot mask a hung
retry per §5.3.4). On every tick after the 10-minute threshold, emit one
log per stale dispatched row — no auto-fail.

Also threads dispatchId through buildDispatchPreamble so workers can
attribute their heartbeats back to the correct dispatch context.

Co-authored-by: Orca <help@stably.ai>

* feat(orchestration): orca orchestration ask verb

Adds a CLI verb that sends a decision_gate message and blocks on the
coordinator's reply, scoped to the outbound message's thread. Group
addresses (@all, @idle, …) are rejected — fan-out questions must use
send --type decision_gate explicitly.

--json emits bare single-line JSON (bypassing printResult) so workers can
pipe `orca orchestration ask … --json | jq -r .answer` without unwrapping
an RPC envelope; human mode prints just the answer. On timeout the verb
exits 1 and returns {answer: null, timedOut: true}.

This is the CLI surface BEHAVIOR RULE #1 in the dispatch preamble points
workers at instead of AskUserQuestion.

Co-authored-by: Orca <help@stably.ai>

* feat(orchestration): QoL bundle — preamble visibility, status enum, dispatch cross-ref, inbox --full

Addresses four items from ORCHESTRATOR_FEEDBACK:

- #5 preamble visibility: `dispatch-show --preamble` regenerates the preamble
  text for a task; `dispatch --inject --dry-run` previews without mutating
  state; `dispatch --return-preamble` echoes the injected preamble in the JSON
  response so coordinators can audit what a worker received.
- #6 status enum validation: CLI rejects unknown `task-update --status` values
  with `invalid status '<x>', expected one of: pending, ready, dispatched,
  completed, failed, blocked` before the RPC's generic Zod message. Valid
  statuses are listed under Notes in `task-update --help`.
- #13 task-list dispatch cross-ref: `task-list --json` now includes
  `assignee_handle` and `dispatch_id` for tasks in status=dispatched via a
  read-only LEFT JOIN on dispatch_contexts. Non-dispatched rows keep their
  legacy shape so existing consumers are unaffected.
- #14 inbox body visibility: `inbox --full` prints body + payload verbatim;
  default output is unchanged (id/from/to/subject only).

No DB migrations; join-only change on dispatch_contexts so the sibling
preamble PR's `last_heartbeat_at` column addition will not conflict.

Co-authored-by: Orca <help@stably.ai>

* fix(worktree): prevent stale-base worktree creation and dispatch

Addresses feedback #16 per DESIGN_DOC_STALE_BASE_FIX.md §0. Four v1
components coordinated by a single shared fetch cache on the runtime:

1. Concurrent-fetch-with-gate in UI create path: `createLocalWorktree`
   fires `git fetch` BEFORE the suffix loop / PR probe / path
   resolution, then awaits right before `addWorktree` so the new branch
   always spawns from a fresh remote tip. Renderer sees a two-phase
   spinner via the new `createWorktree:progress` IPC event. The cache
   is a `Map<repoPath::remote, Promise<void>>` + 30s success-only
   timestamp on `OrcaRuntimeService` (§7.1 — shared with dispatch).
2. Dispatch pre-flight drift guard in `Coordinator.dispatchTask`:
   probes `rev-list --left-right --count` against the target worktree
   and silently returns (preserves `ready`, no circuit-breaker burn)
   when `behind > 20` unless the task spec carries
   `allow-stale-base: true`. Parsing strips the flag so it never leaks
   into the worker's `--- TASK ---` block.
3. Preamble drift section: populated only when dispatch detected drift.
   Workers see `--- BASE DRIFT ---` with the N-most-recent subjects
   they don't have, so they can pull them in before running.
4. §3.3 Lifecycle: `.finally()` evicts Map entries on BOTH success and
   rejection; timestamp is written ONLY on success. Prevents a single
   DNS hiccup from wedging every future create on the repo until
   restart, and keeps the freshness window honest.

Defers the DB `allow_stale_base` column (§0.2) and the create-time
warn toast; both can layer in later without migration.

Tests: 35 new/updated unit tests covering drift preamble, dispatch
refusal, spec-text flag parsing, fetch Map eviction after rejection,
freshness-window short-circuit, and concurrent-caller serialization.

Co-authored-by: Orca <help@stably.ai>

* test(orchestration): seed v2 DB in migration hard-fail test

After consolidating the schema bump, fresh DBs are initialized directly at
v3 via createTables(), so the v2→v3 ALTER TABLE is skipped on new installs
and the prior test's stub never fired. Seed a v2-shape file on disk so the
guarded ALTER actually runs and the "simulated migration failure" stub
propagates as intended.

Co-authored-by: Orca <help@stably.ai>

---------

Co-authored-by: Orca <help@stably.ai>
2026-05-04 12:10:53 -07:00
Jinwoo HongandOrca c9391e203f feat(orchestration): add inter-agent orchestration system (#1188)
Co-authored-by: Orca <help@stably.ai>
2026-04-28 12:21:31 -07:00