Commit Graph
15 Commits
Author SHA1 Message Date
Neil c99e71b315 refactor: group renderer runtime resource owners (#39571) 2026-11-07 20:52:26 -08:00
Neil 4d33cd22cc refactor: group native chat session reads and delivery by owner (#39526) 2026-11-07 10:02:15 -08:00
Neil 1c950baf33 refactor(main): group session lifecycle and journal modules by owner (#38638)
* refactor: group structured session modules by owner

* test: retire status history work-count oracle suite

* refactor(main): organize journal storage and projection owners

* Consolidate restart recovery fixture imports

* Consolidate concrete session option model imports
2026-11-06 06:00:08 -08:00
Neil 9b39188021 refactor: organize session option and conversation owners (#37289) 2026-11-01 21:33:42 -08:00
Neil 89f9ce08de refactor(shared): organize session contracts and projections (#36979)
* refactor(shared): organize session contracts and projections

* chore: tighten dropped-path flaky pattern allowance

* docs: preserve current agent instructions with valid formatting
2026-11-01 03:14:46 -08:00
Neil 57454e6d7e refactor(agent-session): group lifecycle records and direct owners (#36909) 2026-11-01 01:24:37 -08:00
Neil 7d76028a5c refactor: group agent session journal and request modules (#36882) 2026-10-31 22:33:23 -07:00
Brennan Benson a238091bc0 Warn before closing an active native chat (#34236)
* fix(chat): confirm before closing active native chats

* Repair native chat close settings and consent coverage

* Cover native close consent in workspace and floating tests

* Require consent before closing active native chats

* Fix close-warning typed values in deferred actions and tests

* Preserve admission ownership and reconcile relevant send evidence

* Separate chat read lifetime from send settlement and activation effects

* Fix required read activation typing
2026-10-27 12:32:53 -07:00
Brennan Benson 9ae33a96f9 fix(native-chat): keep damaged chats readable without granting agent actions (#31931)
* fix(native-chat): validate saved next-fence floors before use

* WIP: separate saved conversation observations from runtime proof

* Consolidate duplicate imports for WIP compiler checkpoint

* Preserve saved conversation observations with strict runtime authority

* Consolidate session admission and catalog imports

* fix: align conversation observation contracts and fixtures

* test: update conversation observation and writer fixtures

* test: use checked host fixtures for structured workers

* fix(native-chat): bound conversation projection work per request

* test(native-chat): consolidate scale fixture imports

* test(native-chat): use checked scale fixture types

* test(native-chat): bound unreadable inventory traversal

* Test native CLI admission and durable reference reads

* test: bound cold conversation name subscription work

* test: distinguish native receipt probes from terminal identity

* test: retain exact mutation refusal recovery identity

* fix: explain agent conversation preparation refusals

* test: adapt remaining conversation readers and document CLI timeout

* fix: register preparation refusal English message

* test: align conversation fixtures with checked read APIs

* test(native-chat): use checked conversation in Codex fixture
2026-10-26 13:16:41 -07:00
OrcaWinandOrcaWin fdcf887bf0 chore(lint): fix suppressed lint violations instead of ignoring them (#33294)
* chore(lint): remove disable directives that suppress nothing

Each was reported unused by the oxlint config that owns its rule (root,
casting, react-doctor, anti-slop, native-plugins), including 17 file-level
react-doctor blanket disables whose rule no longer fires. An unused
react-hooks disable was masking a react/purity finding in
useAgentBucketCounts (compiler rules skip functions with hook
suppressions); suppress that one explicitly where the code explains it.

* test(tab-bar): use typed tab/file fixtures instead of @ts-expect-error partials

* chore(lint): fix misc suppressed lint violations instead of disabling

- consistent-type-imports: the "vi.importActual requires inline import()"
  rationale was wrong; use typed vi.mock(import(...), importOriginal) or a
  type-only namespace import.
- no-control-regex: source-control commit/push failure summaries, shell PATH
  hydration, and ipynb ANSI parsing reuse the shared ansi-escape-sequences
  stripper and control-character pattern instead of private copies.
- no-explicit-any / no-useless-spread / no-this-alias: typed `this` in
  bridge test mocks, drop blanket file-level disables, drop a this-alias.

* refactor(main): replace avoidable type assertions with checked narrowing

17 production cast suppressions in main/wsl-guest removed: untrusted IPC
and RPC payloads (Jira issue update, WSL guest plugin install, handshake
type) now narrow through isRecord/type guards and build typed results;
redundant casts dropped; hand-rolled object checks and errno casts use the
existing isRecord / hasErrorCode helpers.

* refactor(renderer,mobile): replace avoidable type assertions with checked types

13 production cast suppressions removed: RPC failures read through the
`ok` discriminant instead of casting to RpcFailure; react-native-web host
refs narrowed with instanceof HTMLElement; redundant reply/diff casts
become checked annotations; the TUI agent param uses z.custom(isTuiAgent)
like its sibling schemas; the agent-status slice cast that hid missing
actions is gone so the creator's return type checks the whole slice.

* chore(lint): drop dead tiptap wrapper; scope anti-slop mocking rule to test support

- safe-react-node-view-renderer patched handleSelectionUpdate, which
  @tiptap/react 3.31.3 no longer has, so it always returned early. Use the
  stock ReactNodeViewRenderer.
- The anti-slop test override only matched *.test.{ts,tsx}, forcing
  file-level no-module-mocking disables in test harnesses and fixtures.
  Match *-test-harness, *-test-fixture(s), .test-fixture and *.test.mjs,
  then drop those disables and the now-redundant per-file exemptions.

* test: replace repeated cast suppressions with typed fixtures (main/shared/e2e)

Net -119 cast directives. Typed fixtures replace recurring casts:
unusedSshConnection/unusedOrcadHostTransport placeholders, a typed
readPersistedState harness read (exposed a test treating optional
httpProxyUrl as always set), real Response objects in fetch spies, an
updater main-window fixture, a CDP webContents adapter, one shared
releaseModuleExport for cross-version tests, and requireOverlayDir in
place of `as string` on possibly-undefined overlay paths.

* fix(runtime): validate loaded device rows and host scope instead of casting

Two cast suppressions claimed checks that did not happen:
- device-registry-file-read: "normalizeLoadedDeviceRow re-validates every
  field" was false; deviceId, token, name and timestamps were never
  checked. Rows without an identity are now dropped and the file is set
  aside through the existing corrupt-file path (registry reads as
  doubtful, so no revocation verdicts); a non-string token is stripped
  so the row stays listed but cannot authenticate.
- client-ui: "only optionality differs" was false; any string reached
  persisted workspaceHostScope. Narrow it with isWorkspaceHostScope and
  ignore unknown scopes (wire schema unchanged for mixed versions). The
  remaining cast now states what it actually covers.

* test(runtime,ipc): drop redundant cast suppressions; typed runtime store fixture

-98 cast directives across runtime/ipc tests: 61 casts that typecheck
without the assertion; a typed vi.fn for scoped invalidation; a
partialRuntimeStore fixture in place of `store as never` for
OrcaRuntimeService; typed worker terminal/authority builders in the
incarnation-fallback suite (the old stub lacked runtimeId, ptyId,
worktreeId and launchTokenHash) plus removal of stacked duplicate
directives there.

* chore(lint): typed vi.mock imports and explicit listener snapshot in shared-bucket tests

Remaining files from the misc-rule pass that overlapped other buckets:
codex-accounts mocks use the typed vi.mock(import(...)) form instead of
a consistent-type-imports disable; the daemon late-exit fixture copies
its listener set with Array.from instead of a no-useless-spread disable.

* refactor(github,mobile): narrow routing settings type; drop remaining avoidable casts

- work-item-routing cast a one-field object to AppState['settings'] under
  a claim that readers only consume activeRuntimeEnvironmentId. Return the
  existing RuntimeFocusSettings pick instead and narrow the cache-key
  chain's parameters, so the compiler enforces that claim.
- mobile file preview reads refusals through the RpcResponse `ok`
  discriminant rather than a cast.
- terminal document host-notify accepts `unknown` errors, removing the
  message-bridge cast.
- native chat readability drops a redundant repo-list cast.

* test(renderer): replace repeated cast suppressions with typed fixtures

-140 cast directives (and ~150 unsuppressed casts of the same shapes):
- mockPanePtyArgs owns the connectPanePty fixture casts, so a wrong
  argument order or fixture is now a type error; drop binding casts for
  members PanePtyBinding already declares.
- Seed AppState tab rows with the typed makeTerminalTab; this exposed a
  split direction 'row' that is not a TerminalPaneSplitDirection.
- Partial GlobalSettings use getDefaultSettings overrides; Jira-only
  stores use createTestStore; runtime environment rows/statuses come from
  a typed paired-runtime-environment fixture.

* chore(lint): drop no-module-mocking disable now covered by the test-fixture glob

* test(e2e): fix all 221 grandfathered e2e type errors; baseline to zero

The e2e typecheck baseline grandfathered 221 errors in 119 files. About a
third were tests drifted from the current API, several of which would
fail or mislead at runtime:
- terminal-shortcuts called a nonexistent xterm buffer method;
- the daemon-lifecycle fake lacked terminateOwnedTree, hidden by importing
  its type from a module that does not export it;
- arguments in the wrong position (listProcesses(hostId, opts), a dropped
  pty.write inputKind, an extra toWebTerminalSurfaceTabId argument);
- removed enum values ('drawer', 'idle', 'editor',
  'session-insecure-origin') and a launch_source the telemetry schema
  rejects;
- missing required fields (turnScope, childWorkOpen, providerIdentity,
  reviewLookupOutcome, runtimeServiceAccount behind an `as RelayConfig`);
- dead reads (tab.customTitle, bucket.targets, changedPixels).
Also lifts variables that narrowed to `never` after poll-callback
assignment, and adds a typed xterm render-dimensions accessor. Includes
the cross-version suites' shared releaseModuleExport adoption.

* chore(typecheck): gate the now-clean e2e typecheck in local lint

The e2e baseline is empty, so the ratchet now rejects any tests/ type
error. CI already runs it via run-ratchets; add it to pnpm lint's
explicit ratchet list and replace the stale 'not yet clean' note.

* test(mobile): typecheck every mobile test; baseline 105 files -> 0

mobile/tests-typecheck-baseline.txt grandfathered 105 test files (531
errors) whose types CI never checked. Contract changes had landed without
these tests following, and type pins in them silently checked nothing.

- Root cause: salvagedOptional (src/shared/zod-salvage.ts) cast away
  zod's optional marker, so every reply type built on it claimed
  salvaged fields were always present. Return a real ZodOptional;
  runtime is unchanged (the outer optional short-circuits undefined).
- Production now takes the port it uses: RpcOperationSender instead of
  RpcClient for send-only callers, a narrow fetch signature, and only the
  request fields push-receive reads.
- findAllByHostType/findByHostType replace `'Text' as never` lookups;
  stubRpcClient gives tests a typed RpcClient.
- Real test bugs: settings-host-client-lifecycle "relay" hosts were
  direct hosts (relayHostId no longer exists); two toEqualTypeOf pins
  could never hold; a recovery hook ran with an undefined scope key;
  a create-PR test cast to a never-imported type; "no URL" assertions
  passed on null; plus stale required props, args and enum values.

* test(e2e): parse workerStart result instead of casting it

* test(runtime): typed partial runtime stores with production-shaped fixture data

Route every partial store passed to OrcaRuntimeService through
partialRuntimeStore with typed row builders, removing 60 cast directives
that hid fixture data production never produces:
- paired-client-navigation-test-harness keyed setWorktreeMeta writes by
  `[object Object]` (it took an id where the store takes a target), so
  reads never saw them;
- invalid worktree lineage origin/capture and incomplete workspace lineage;
- getWorkspaceSession returning null where the store returns the default
  session; getGitHubCache returning undefined;
- orcad migration states missing migrationId/manifest/receipt, with a
  toEqual pinned to that incomplete shape;
- partial repos, metadata, folders, project groups and notification rows.

* refactor(renderer): resolve suppressed React hook lint instead of disabling it

109 of 187 react-hooks/react-doctor suppressions fixed:
- ~95 exhaustive-deps disables only hid stable refs and setters passed
  through controller objects; list them (identity never changes, so
  nothing re-runs), plus a few real stale closures (tabId/worktreeId).
- Effects keyed on one value that read the latest props now use
  useEffectEvent (WebConnect, skill freshness, Bitbucket dialog seeding,
  Landing preflight, automation catalog, structured-session recovery,
  file explorer).
- The conditional layout-owner hook is selected once at module load from
  the build constant instead of a rules-of-hooks disable that also hid
  the whole controller from compiler and react-doctor rules.
- A hook disable was masking a react/purity Date.now() in the palette's
  recent-tabs memo; sample it when the snapshot is captured.
- Three identical debounced-search effects share
  useTaskSearchDebouncedApply; layout snapshot merge helpers move out to
  stay under max-lines.
Remaining suppressions are mount-once resources, invalidation epochs,
deliberately keyed fields, and test harnesses that mock the dispatcher.

* chore(ratchet): exempt test harnesses and fixtures from the focus-read ratchet

The ratchet skipped *.test-fixture.ts but scanned *-test-harness,
*-test-fixture(s) and *.test-support files. Those seed settings with
getDefaultSettings (which the ratchet discovers as a focus reader) and
never route host work, so 17 baseline entries were false positives, and
replacing a test's `settings: {...} as never` with typed defaults tripped
it. Match the repo's test-support names and prune (which also drops one
stale entry), and rename the frame-ordering harness to *-test-harness like
its siblings. A blanket "-harness" match would also exempt
e2e-terminal-pty-harness, which production imports, so the names stay
explicit.

* test: merge duplicate shared/constants imports

* revert(file-explorer): restore original tree load effects pending SSH e2e investigation

* test(runtime): drop RuntimeStore cast now that the fixture is typed

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
2026-10-24 19:37:19 -07:00
Brennan Benson dd3bbf680e Catch up desktop chats from the newest page (#29125)
* fix(native-chat): retire acknowledged send echoes after catch-up

* test: follow phone lint rules in reconnect regression

* test: format reconnect message expectation

* Keep current chat requests independent of loaded history

* Read current chat requests in bounded cursor pieces

* Bound desktop catch-up after independent request support

* Fix current request subscription static checks

* Match generated request schema import ordering

* fix(native-chat): correct current request type boundaries

* Accept readonly client capability fixtures

* Exercise current request reads in wire compatibility inventory

* Align current request fixtures with provider commit and RPC surface

* test(native-chat): include current request browser capability

* Keep the mobile chat session hook within its line budget

* Format mobile current request readers

* fix(native-chat): reopen once for a current-request version whose read keeps failing

A deterministic read failure reopened the subscription, whose first frame re-read the same version and reset the backoff, so the pane resubscribed every 750 ms forever.

* test(native-chat): type the current-request snapshot fixture

* test(native-chat): wait for the hydrated open before driving the subscription
2026-10-16 17:46:54 -07:00
Brennan Benson e817b0e237 refactor(native-chat): keep the provider resume handle opaque to shared code (#24991)
* refactor(native-chat): keep the provider resume handle opaque to shared code

Shared structured-chat code parsed each provider's resume handle: Claude's
session id and branch leaf, Codex's thread id, through a 'claude' | 'codex'
union every new agent had to widen. The in-memory handle is now
{ transport, agent, nativeId, providerData? }: shared readers use nativeId,
lease and handle-chain checks compare transport and agent, and only the
Claude adapter reads its leaf (providerData).

Stored and wire forms are unchanged for Claude and Codex. One encoding
module writes their typed shapes and decodes both those and the neutral
shape a new transport uses, which an older build refuses as unreadable
rather than reading as Codex. Key and root strings, which fork seeds,
superseded creations and resume offers persist, stay byte-identical.

The journal's own handle type becomes the journal-row and attach-wire
encoding of the same handle, and the journal identity carries the
neutral handle (null before the provider proves one).

No user-visible change.

* fix(native-chat): derive journal-row provider handles from the journal identity

The journal row converter now takes the identity every caller already holds,
so a row's handle has one obvious constructor. Tests that wrote the in-memory
handle straight into journal rows now build it through that converter, and the
processless Claude fixture names a not-yet-proved handle as null.

* fix(native-chat): refuse a stored provider handle written in both forms

A typed Claude or Codex handle that also carries the neutral form's
transport, agent, native id or provider data named two identities; it
was read as Claude or Codex and the next write dropped the other one.
Such a row now stays unreadable and is set aside untouched.

* test(native-chat): use opaque handle in queued rejection fixture

* test(native-chat): share one Codex journal identity in the integration suite

Main grew the suite to the 800-line limit; the opaque-handle import pushed it
over. The two tests built the same identity inline.

* refactor(agent-session): name the handle's adapter state resumeCursor

Rename the neutral provider handle's providerData to resumeCursor before any
row persists the neutral form: it is an adapter-owned resume position (Claude's
transcript leaf), never identity. Claude/Codex stored and wire bytes are
unchanged; their typed shapes never carried the field.

State the stored-form contract (a handle's field set is closed; later per-link
data goes on the chain link, which every build preserves) and pin it with a
record round-trip test. Document that transport records the id space the
native id was minted in, which can differ from the agent's current transport.
2026-10-05 14:48:12 -07:00
Brennan Benson c6cfcc034e refactor(native-chat): structured chat failures always reach the diagnostics log (#24312)
* refactor(native-chat): give the structured chat host one required logger

The structured chat runtime took an optional onError callback that the
desktop never passed, so a late dispatch settlement, an unanswered-dispatch
release, a journal event-sink write and a provider lifecycle delivery that
failed were dropped with no trace. Other host failures went to scattered
console.warn calls, which reach nothing in a packaged desktop build.

The runtime and host now take one required logger (warn/error with a scope
and fields). The production logger writes each entry as a failed span to
<userData>/logs/main.trace.ndjson, which the diagnostic bundle collects, and
to the console (stderr under a supervised headless host). The runtime and the
host wrap it so a logger that throws never fails what it reports, and the
install refuses without one. Sites that deliberately kept a recovery-capsule
error out of the log still log no error object.

* refactor(native-chat): hand the chat host's collaborators the logger, and give orcad its trace file

The delivery loop, idle sweep, queued-message drain, lease renewer, event
sink, conversation map and provider start/exit settlement each took an
internal error callback that the host mapped onto the logger. They now take
the logger itself and log under their own scope. The event sink keeps one
onFailed hook, which decides whether to stop the provider, not whether to
report. The dead-generation settlement returns its failure so each caller
logs it under its own scope.

orcad now installs the desktop's local trace sink under its own data root, so
a headless host's chat failures reach <data-root>/logs/main.trace.ndjson as
well as stderr.

Also passes the logger in the test fixtures the first commit missed, which
tc:node caught.

* fix(native-chat): keep repeated chat failures from flooding the trace file, and record their causes

- The production structured-chat logger writes a repeated failure (same level, scope, session,
  message and error text) once per 5 minutes, carrying how many repeats it swallowed; the
  tracked set is capped at 256.
- Trace entries now carry the error's code (and SQLite errcode) and up to three causes by name and
  message.
- A chat read whose conversation will not open is logged through the host's logger
  (open-for-read), and so are the runtime's chat-tab bookkeeping failures that already hold the
  host.
- orcad writes its own orcad.trace.ndjson, closes it after every quit handler, and flushes it on
  process exit; a trace file that cannot be opened leaves tracing off instead of stopping the app
  or orcad.
- Tests: the desktop wiring test proves the logger reaches the trace sink, and the privacy tests
  read every level the logger received.

* fix(native-chat): log a created chat's tab-publication and launch-prompt failures through the host's logger

* fix(native-chat): key a repeated chat failure on everything its entry writes

The repeat suppression keyed on the message and the error's text, so two refusals with the same
code but different causes, a plain error and a refusal of one code, or two object-valued errors
shared a key and the second was swallowed for five minutes. The key is now the entry's whole
written content (fields, code, errcode, refusal reason, cause chain, a stable rendering of a
non-error value) plus the error's name and message; a refusal's reason is also written.

* test(native-chat): pin that an error's name keeps two repeated failures apart

* test(native-chat): build the refusal in the repeat-key test as the wire does

* fix(native-chat): read an error's code and a refusal's reason by narrowing, not Reflect.get
2026-10-01 11:21:50 -07:00
Brennan Benson 59b746ff3c feat(native-chat): one structured-chat journal database per host, owned by one process (#23613)
* feat(native-chat): one structured-chat journal database per host, owned by one process

Every structured chat on a state directory now lives in one SQLite file,
agent-session-journal.db, opened once by the process holding
agent-session-journal.owner: an empty SQLite file whose held BEGIN EXCLUSIVE is a
kernel byte-range lock, refused while another process holds it and released when
the holder dies.

- Stores own no connection: the per-chat handle, its close contract and the
  close-retry registry are gone; closing a conversation drains its writes, and the
  one connection closes last at teardown.
- The owner lock is taken at runtime start, before orca-runtime.json is written;
  a process that does not own the chats is not published and refuses every
  structured request with journalUnavailable and words that say what to do. It
  retries the lock with backoff and runs the full install once it holds it.
- A journal that will not open fails the host install: every chat says "Unable
  to load this chat." (journalCorrupt), and nothing is renamed, deleted or
  rebuilt. A newer build's database is refused and left byte-identical.
- An append is one INSERT. The listing status is a column, written after the
  rows it describes and keyed by (epoch, sequence).
- A per-chat journal from an earlier build is copied in verbatim (epoch UUID and
  every sequence) on that chat's first open, and its directory is retired only
  after the copy commits.
- auto_vacuum = INCREMENTAL, with freed pages handed back in bounded steps after
  every delete.

* perf(native-chat): key journal rows by block so one chat's rows sit together

Each chat's live epoch owns a block of row ids, block * 2^32 + seq, so a chat's
rows share leaf pages with nobody else's, a replay is one range scan, and
replacing or rewinding a chat deletes one contiguous range. Measured on the
largest real chat (61 MB) beside 19 interleaved peers: 39 ms and 7.5 MB of WAL,
against 214 ms and 102 MB for a (session_id, epoch, seq) key.

- Ids are computed in Number arithmetic, never bitwise. A sequence is refused
  outside [1, 2^32) and a block at 2^21, which keeps every id below 2^53.
- A replace, roll or import allocates a fresh block, moves the chat's pointer,
  and deletes the old block in the same transaction, so no orphan block exists.
- The listing status write moves into its own writer beside the column.

* feat(native-chat): copy a chat's per-chat journal again when an older Orca wrote it after a downgrade

A per-chat journal.db that reappears after its chat was copied in is the newer
history: an older build, run after a downgrade, attached the chat and wrote it.

- journal_imports records the (epoch, tip) each chat was copied from, in the
  copy's own transaction. A file already copied is never copied again, across
  any number of restarts after a failed rename; a file that differs always is.
- Newest writer wins, per chat, with a row saying the chat was continued in an
  older version of Orca. When both builds wrote past the recorded tip under one
  epoch, the copy takes a fresh epoch, so readers reset instead of skipping rows.
- Each copied directory retires to its own .imported-<epoch8>-<ms> name, so a
  second downgrade and re-upgrade never collides with the first.

* test(native-chat): fixture deps match the host journal database shape

Attach-flow and reconcile-attach fixtures stop passing a journal database those inputs do not take, and host and restore fixtures pass the one they now require instead of the removed journal root.

* test(native-chat): state why the runtime-state fixtures' existing casts are safe

* fix(native-chat): start up normally when this process cannot open the chat journal

A process refused the chat journal, because another Orca owns it or because its own journal will not open, failed startup restoration: the window booted in degraded no-save mode and a paired phone could not list any tabs. Startup restoration now treats the refusal structured requests are getting as having no structured host; terminals, tabs and saving go on, structured requests are still refused by the gate, and the install is retried on the next one. Any other install error fails startup as before.

* test(native-chat): name the owner-lock sweep test after the two sweeps it runs

* fix(native-chat): session history and terminal resume work while chats are refused

Session history (listing and preparing a resume) and a terminal typing a resume command only check whether a structured chat owns a provider session. In a process refused the chat journal they failed outright. They now take the refusal chats are getting as having no structured host, the same treatment startup restoration gets, through one shared helper; any other install failure still fails them. Chat requests keep the gate's refusal.

* test(native-chat): the first-work rename's fake journal saves the listing status

* fix(native-chat): open a chat whose per-chat journal file never got its schema

A crash between creating a chat's journal.db and creating its tables left an empty or schema-less file. Each chat used to open that file as an empty chat; the importer instead refused the open as "try again" forever. A file with no journal_sessions table is now read as never written, the same as one with no rows. A file that is not a database, or whose read fails, is still refused.

* fix(native-chat): let the event loop run between chats during startup restore

Opening a chat's journal is synchronous SQLite now that no per-chat directory
is created first, so the restore of every visible chat ran as one main-thread
task. Each chat now waits for a macrotask before it opens.

* fix(native-chat): import a per-chat journal in bounded batches

The one-time copy of an earlier build's per-chat journal ran as one
transaction, which blocked the main thread for 650 ms on the largest chat.
Rows now copy 512 at a time, each batch its own transaction, yielding to the
event loop between batches. The rows go into a block journal_import_blocks
reserves, which no reader follows and no other chat is allocated; the last
batch publishes the chat's pointer, repair marker and import marker together
and releases the reservation. A copy that stops midway leaves only that
block, which the next open clears and copies again. Two opens of one chat
import one after the other.

* fix(native-chat): refuse chats when the owner lock file cannot be opened

A lock file that is not a database, or cannot be opened, made the claim throw
before any refusal was recorded, so startup restoration failed on every
launch. The claim now sits in the same try as the database open and records
the same typed refusal.

* fix(native-chat): finish reclaiming pages a delete frees during a running pass

A reclaim pass ended as soon as the freelist stopped shrinking between steps,
so a second delete that freed more than one step's worth mid-pass ended it
early and left those pages on the freelist. A pass now ends only when a step
itself frees nothing, or the freelist is empty.

* test(native-chat): desktop session history is served while chats are refused

* fix(native-chat): a send to a chat holding a newer Orca's rows says to update

A chat opened read-only because a newer Orca wrote rows to it answered a send
with the generic write failure. It now refuses the way a database a newer Orca
wrote does, with the same reason and words.

* fix(native-chat): keep chat tabs while this process cannot list its chats

A process whose chats another Orca owns, or whose chat journal will not
open, has no structured host. Its session-tabs inventory still answered,
with no chat rows, and the renderer read that as "every chat was closed":
it removed the restored chat tabs and the next session save persisted
their placement away.

The inventory now says `agentSessionsUnverifiable` when the last tab
restore ran with chats on disk but no host to list them. The flag is set
and cleared at the per-client projection point beside the client-hosted
page hold, and the restore is memoised only once a host answered, so a
later lock takeover or journal open republishes the chats and clears it.
The renderer keeps agent-session tabs, and keeps cancellation tombstones,
against an inventory that does not affirm its chat set.

* fix(native-chat): say chats are open in another Orca, with this process's way past it

A process refused because another Orca owns the profile's chats sent the
generic `journalUnavailable` reason, so current desktop and phone surfaces
said "couldn't open this chat's history right now. Try again." — a step
that never helps while the other Orca runs.

The refusal now names its own reason, `journalOwnedElsewhere`, with the
refused process's kind (dev desktop, packaged, orcad) as a fact. Each kind
gets its own step: quit the other Orca, or give this one its own profile
(ORCA_DEV_USER_DATA_PATH) or data folder (ORCA_USER_DATA). The sentences
are added to the shared notice copy, the desktop catalogs in all six
locales, and the boot catalog.

A client that predates the reason reads it as none and keeps the code's
words; an unknown kind reads as the packaged app's step. The `message`
released clients print is unchanged. Which requests refuse does not change.

* fix(native-chat): restore chats on taking ownership, without a list to ask

A refused startup kept its hostless result, so after the owner quit this
process never installed a host, never reconciled restart leases, and kept
telling clients it could not list its chats until a desktop chat request.
Taking the lock now reruns startup restoration once and pushes the chats.

* test(native-chat): a navigation reply says chats are unverifiable while refused

* fix(native-chat): retry a refused owner lock at most every 5 seconds

The lock frees as its holder exits, but a refused process only learns that
on its next retry, and the 30 s cap left a second Orca refusing chats for up
to half a minute after the owner quit. One retry is an open and BEGIN
EXCLUSIVE on an empty file.

* fix(native-chat): install before deciding whether a takeover must republish chats

A list that landed on the refused startup after the lock was taken finished
after the takeover had already checked, so nothing republished. The takeover
now installs first, waits for any restore in flight, and restores only then;
the restore that clears "cannot tell" pushes the frames itself, so a list
that heals the inventory first reaches subscribers too.

* fix(native-chat): no takeover lands a host after the runtime stop

Quitting cancelled a refused claim's retry only at its end, so a retry firing
during the stop's awaits took the lock and installed a host the stop never
tore down, and the lock was then released under an open journal. The stop
now cancels the retry first, keeping the refusal, and repeats its teardown
while an install that began during it (a takeover already under way) is
pending, so no journal connection outlives the lock.

* fix(native-chat): show a thrown refusal in its own words, not its code

A refusal the host throws reaches the client as an RPC error whose message is
the bare code; its reason and facts ride only in the error's data, which no
client read. The chat pane's status line therefore printed
agent_session_journal_unreadable, a send took the bare "not sent" path, and
other writes said the outcome was unconfirmed.

One shared reader, agentSessionThrownRefusal, now reads the refusal from the
error data. A failed history read shows the refusal's read-history words, a
send keeps the refusal behind its Retry exactly as a returned refusal does, and
the other writes (desktop and phone) name the refusal instead of doubting the
outcome. The phone's read failure goes through the same reader.

* fix(native-chat): log a failed journal open once per distinct failure

Every chat request retries a journal open that failed, which is intended, but
each retry also logged the failure with its full stack: a junk database file
logged the same "file is not a database" error 189 times in a minute. The open
now logs a failure only when its code and message differ from the last one
logged, and forgets it once an open succeeds. The retry is unchanged.

The open moves to its own module beside the runtime, which had no room left.

* fix(native-chat): restore lists a chat from its per-chat file and copies it on first use

Startup restore opened every restored chat, and that open copied the chat's
per-chat file into the host database, so the first boot after an upgrade paid
the whole one-time copy before the chat list appeared.

A restore open now reads a chat that is still in its per-chat file straight
from that file, read-only, with the importer's own reader, and closes the file
before moving on. That read drives the listing, the status row and the
restart offer, as it did when every chat had its own file. The copy becomes
owed work on the chat's write queue: it runs before the chat's first write,
and a reader that reaches the chat awaits it. A chat the host already holds,
or that was copied before, still opens through the import and its reimport
rules, and so does a file whose read needs a repair written.

* fix(native-chat): no host stays registered after a stop an install spanned

Each teardown pass clears the registered host before it awaits an install in
flight, and that install registers its host when it finishes. The pass then
tore the host down but left it registered, so a request after the stop was
served by a host whose journal was closed. The stop now clears the slot once
its passes are done.

* fix(native-chat): checkpoint the journal with a full flush on macOS

synchronous = FULL fsyncs each commit, but macOS fsync leaves the drive cache
unflushed, so FULL alone does not survive a power loss there. With
checkpoint_fullfsync, each checkpoint uses F_FULLFSYNC; elsewhere it is a no-op.
The comment that said FULL alone was enough is corrected.

* fix(native-chat): delete a per-chat journal once its copy verifies

An imported chat's per-chat file was kept under an `.imported-*` name, which
doubled the disk its history takes. The copy now reads back from the host
database before it is published: its items, submissions, epoch and tip must
match the file's. Only then does one transaction publish the chat with its
import marker, and the file and its WAL files are deleted, the directory too
when nothing else is in it (a pre-SQLite transcript there is kept).

A copy that does not match is never published: the file stays, the chat is
refused as unreadable ("Unable to load this chat."), and the mismatch is logged
once. A file left behind by a failed delete or a crash matches the marker, so
the next open deletes it rather than copying it again; a file an older build
wrote after a downgrade still differs, and is still copied again.

* fix(native-chat): verify an imported chat a batch at a time

The check that a copied chat reads back as its per-chat file folded both whole,
each in one synchronous task: over half a second on the largest chat. Both
reads now go a batch at a time between turns of the event loop, like the copy
itself, and count rows as well, so a copy that lost a row with no item in it
is caught too.

* fix(native-chat): restore reads a chat's per-chat file a batch at a time

Restore folded a chat still in its per-chat file in one task, so the largest
chat's file held the main thread for about half a second at startup. The fold
now takes the file a batch of rows per turn of the event loop, into the same
fold a replay uses, and nothing reads it before it is done. The file is still
closed before restore moves on.

* fix(native-chat): end a per-chat copy on a turn of its own

A chat's first open ran the copy's last steps (the verified publish and the
per-chat file delete) and the replay of what was copied in one task. The copy
now yields before it returns, so the replay, which every open runs, is a task
of its own.

* fix(native-chat): commit a per-chat copy's batches without an fsync each

Each 512-row batch of a chat's first-use copy committed under synchronous =
FULL, so a large chat paid one fsync per batch, about a quarter of its first
open. The batches now commit under NORMAL, set and restored in the batch's own
task so no other chat's commit runs under it. The publish that makes the copy
visible still commits under FULL, and under WAL that sync makes every earlier
batch durable with it. A crash before it leaves only the unpublished block,
which the next open clears and copies again.

* fix(native-chat): roll back a chat journal transaction whose COMMIT fails

The shared connection's transaction rolled back only when its body threw. A
COMMIT that failed left the transaction open, so every later write, for any
chat, failed with "cannot start a transaction within a transaction", and reads
saw rows that never committed. Under the unsynced copy the failure also tried
to restore the sync level inside the open transaction, which SQLite refuses,
so the caller got that error instead of the COMMIT's.

One transaction helper now covers the body and the COMMIT, rolls back whatever
transaction survives, and rethrows the original error. Schema creation uses it
too. If that ROLLBACK fails as well, the connection is marked stranded: each
later use retries the ROLLBACK, and until one goes through every chat gets the
same "history unavailable, try again" refusal a journal that will not open
gives. The rollback that frees it also restores the FULL sync level.

* fix(native-chat): keep the chat journal connection until its close succeeds

Closing the journal dropped its connection handle before closing it. A close
that failed left the database reporting itself closed with the connection still
open, so the stop that retried the teardown found nothing to close and released
the owner lock over a live connection.

The handle is now dropped only once the close succeeds. A failed close keeps
the runtime pending and the lock held, and the next stop closes that same
connection before it releases the lock.

* fix(native-chat): publish the runtime only once it holds the chat journal lock

When this process could not open the owner lock file at all (a permission
error, or a file that is not a database), the runtime counted that as owning
the chats and wrote orca-runtime.json. That overwrote the real owner's entry,
so the CLI was sent to a process that cannot serve its chats.

A claim that throws is now refused like one another process holds: the runtime
starts but does not publish, the claim's existing retry keeps asking for the
lock, and discovery publishes once the retry takes it. Chats still get the
refusal for the failure itself, and startup restoration reruns on the takeover
the same way it does after another owner quits. A sole process whose lock file
never opens is not found by the CLI until it does.

* fix(native-chat): keep a chat's history when an older build started it over

The first copy deletes a chat's per-chat file, so an older build run after a
downgrade finds no file and starts the chat from nothing. On the re-upgrade that
fresh file was copied in as the newer history, replacing everything the shared
database held for the chat, and then deleted.

A file whose epoch is not the one last copied and that opens with
`session_created` is now kept: neither copied nor deleted, and the chat keeps
the history it has. A file that carried the copied epoch on is still copied
again, as before.

* test(native-chat): pin which chats startup restore copies

Restore copies a chat still in its per-chat file only when restore itself has
to write to it: settling what the last run left open, here a running tool call
or a send handed over and never answered. Every other restored chat stays in
its file until its first use.

* test(native-chat): pin the copy wait on a read that opens a chat restore opened

A read queued behind restore's open of the same chat reaches the conversation
through its own open rather than the listing. It must still wait for the
owed copy, or it reads the chat before its history is in the one database.

* fix(native-chat): record a set-aside per-chat file so no later open reads it

Setting aside a file an older build started over is decided once and kept in
the new `journal_set_aside` table (schema 2, additive), with the file's epoch
and tip as they were. Every later open of the chat skips the file without
opening it, across restarts and after the older build writes more to it:
anything written there grows from that build's own start, never from this
build's history.

The best-effort delete moves beside the per-chat file reader.

* fix(native-chat): set aside any per-chat file at an epoch this build never copied

A chat's per-chat file is deleted once its copy verifies, so a file that
reappears at another epoch was never this build's history, whatever its first
row says: an older build started the chat over, possibly rewinding it after
(`handle_forked`), or rolled the epoch of a file whose delete had failed.
Copying any of them would replace everything the chat holds, so each is set
aside. Only a file still at the copied epoch is copied again (it grew) or
deleted (it did not). The first-row check is gone.

* fix(native-chat): copy a reappearing per-chat file again only while this build has not written past the copy

A per-chat file that an older build carried on under the copied epoch was
copied again even when this build had also written to the chat since the
copy, or had rolled its epoch. The second copy replaced the chat's block,
so what was sent in this build after the copy was gone for good.

Now the file is copied again only when the chat still stands exactly as it
was copied: the same epoch and tip the import marker recorded. Otherwise it
is set aside like any other file that is not this build's history, left on
disk untouched and recorded so no later open reads it. A second copy
therefore never replaces rows this build wrote, keeps the file's own epoch,
and the fresh-epoch rewrite goes away. The row it adds now says the history
includes what the older version recorded, not that anything was replaced.

* test(native-chat): pin that a chat founded here keeps its history, and the v1 schema upgrade

A chat this build founded has a pointer and no import marker, so a per-chat
file an older build later starts for it is set aside. Nothing pinned that
half of the rule: letting such a chat be copied again replaced its history
and every test still passed. A second test pins that a database written at
schema version 1 upgrades in place, gaining the set-aside table and keeping
its import markers.

* test(native-chat): drop a lost copied row by patching the source, not wrapping it

* chore(mobile): restore the mobile lockfile to main's

* fix(native-chat): pass a classified journal refusal through a send or Stop unchanged

* fix(native-chat): refuse a read whose owed copy fails as a failed open does

* test(native-chat): measure only the replace's WAL in the block-key case

Opening the chats starts a free-page pass that waits one event-loop turn,
and the seed never yields one, so that pass was still pending when the
replace committed. It woke during the async stat and reclaimed the pages
the replace freed, adding ~500 KB of WAL whenever the stat lost the race
(Linux CI). Drain that pass before measuring and stub the replace's own.

* test(native-chat): the RPC fixture's status journal can save its listing status

The status feed now hands every projection to the journal, which decides whether it is worth saving.

* test(native-chat): state why the RPC fixture's status journal cast is safe

* fix(native-chat): refuse a per-chat copy whose rows differ from the file, not only its counts

* fix(bench): build the replay benchmark's baseline arm from the base tree and release its handles on failure

* fix(native-chat): retry a failed listing status save on the next read of a cached status

* refactor(native-chat): drop the chat journal owner lock; the process instance lock already guards the profile

The journal carried its own exclusive lock, with a retry loop, an in-process
takeover, lock-gated runtime discovery and a "chats are open in another Orca"
refusal. Every shipped process kind (packaged desktop, serve mode, orcad)
already refuses a second instance on one profile before the journal opens, so
the lock only ever mattered for dev desktops, which the next commit covers at
the process level instead.

The host now opens its one journal connection at install with no lock. What a
sole process whose journal will not open needs stays: the install refusal
recorded for the gate, the no-host startup path, and the unverifiable chat
inventory, now in structured-agent-session-host-refusal.ts. The unreleased
journalOwnedElsewhere reason, its processKind fact and their copy are removed.

* fix(startup): dev desktops take the single-instance lock, and a second one says why it quit

Dev skipped Electron's single-instance lock so parallel `pnpm dev` runs from
several worktrees would not quit silently, but two dev processes on the
default orca-dev profile then write the same stores at once. Dev now takes
the lock like packaged builds: a second launch on the same profile focuses
the first window and exits with code 3, printing one stderr line that names
the taken profile and how to run another copy (ORCA_DEV_USER_DATA_PATH).

Serve mode, the macOS diagnostic bypass and the E2E harness are unchanged:
an E2E launch still skips the lock unless it sets
ORCA_E2E_ENFORCE_SINGLE_INSTANCE_LOCK=1.

* refactor(native-chat): key journal rows by chat, epoch and sequence

Rows in the host's journal database are now addressed by the chat's own
identity, with `(session_id, epoch, seq)` as the primary key, the same
shape each per-chat file already used. The block-keyed layout goes with
everything built on it: the block column and its allocator, the 2^21
block ceiling, and the import's reserved block table.

A first-use copy writes its rows under the file's epoch, which the chat's
pointer does not name until the verified copy publishes it, so no reader
sees a half-copied chat. A try that stopped midway leaves only rows no
pointer names, and the next try deletes them before it copies again.
Replace, rollover and repair delete by (chat, epoch).

This build's history always wins: once a chat was copied or founded here,
any per-chat file that reappears is set aside, and the same-epoch copy
again after a downgrade is removed.

The bounded free-page reclaim after every delete is dropped;
`auto_vacuum = INCREMENTAL` stays at file creation, so a later periodic
reclaim can still be added. Session search keeps its own step.

The schema moves to version 3. Versions 1 and 2 were written only by
unreleased builds of this change and are refused as found, not migrated.

* fix(native-chat): open a chat journal a newer Orca wrote read-only instead of refusing it

After a downgrade, the host's journal database carries a newer user_version. It was refused
outright, so every chat's history disappeared. It now opens on a read-only connection, as the
per-chat journals did: each chat shows what this build can read, from the database or a per-chat
file never copied in, and every write is refused with "Chats were saved by a newer Orca. Update
Orca to keep using them." Nothing is written, copied, repaired or founded, and the file stays
byte-identical. A table the newer schema changed reads as the same read-only refusal, not damage.

* refactor(native-chat): leave the saved listing status to the change that reads it

Nothing in this change reads the per-chat listing status column: it was a stored copy of a fact
the status feed derives, written after every turn end and cleared on every epoch change. The
status_json / status_seq columns, their writer, the saved-status type, the status feed's save and
its retry on a cached projection all go, with their tests. The change that lists chats from a
saved status adds the column back beside its reader.

* fix(native-chat): a chat saved by a newer Orca says to update Orca, not to try again

When a newer Orca wrote the chat journal, this build opens it read-only. A send or a Stop was
refused with the reason `journalUnavailable`, so today's desktop and phone clients chose the
words for an open that can clear: "Orca couldn't open this chat's history right now. Try again."
Retrying never cleared it; only updating Orca does.

The refusal now names its own reason, `journalWrittenByNewerOrca`, whose words are "Chats were
saved by a newer Orca. Update Orca to keep using them." A read refused the same way names it
too. An older client does not know the reason, drops it, and falls back to the code's words
("Orca couldn't read this chat's saved history."), and released clients still print the message.

* fix(native-chat): a chat journal from an unreleased build reads as unusable, not as retryable

A chat journal database stamped with schema 1 or 2 was written only by unreleased development
builds of this change. Opening it threw a plain error, which every chat reported as "Orca couldn't
open this chat's history right now. Try again." Retrying never cleared it.

It now throws a named error that is classified as unusable, so every chat says "Unable to load
this chat." The one log line names the file, says an unreleased development build wrote it, and
says to move it aside. Nothing migrates or renames it.

* docs(native-chat): drop the second-Orca-owns-the-chats case from three comments

The chat-only owner lock is gone, so only a chat journal that will not open leaves a runtime
unable to list its chats.

* docs(native-chat): correct three chat-journal comments the redesign left behind

A per-chat file left without its WAL is set aside, not copied again; nothing runs an incremental
vacuum yet, so the auto_vacuum mode is kept for a later pass; and the idle sweep drops a chat's
in-memory fold, since a chat holds no journal connection.

* refactor(native-chat): stop exporting chat-journal names nothing imports

Each is used only inside its own module now; the teardown's export served a deleted test.

* test(native-chat): name the version-0 test for what it covers, and check every journal table

The test named 'migrates an older user_version forward' covers only a version-0 file that already
has its tables; versions 1 and 2 are refused. The table test now also checks journal_imports and
journal_set_aside.

* fix(startup): a second dev launch's exit line no longer claims it focused a window

The running dev instance may be a background launch or a server, which show no window. The line
now says only that this launch passed its request to that instance.

* fix(native-chat): a failed structured-chat install closes the journal connection it opened

The install opened the chat journal database and closed it only if the record store then failed
to open. A later failure, such as the model catalog wiring or the host constructor, left the
connection open, and the next install opened a second one in the same process. Every failure
after the open now closes it.
2026-09-29 16:42:29 -07:00
Brennan Benson a4c11f1889 fix(native-chat): stop a bounded tail read from moving the chat cursor past unapplied rows (#20581)
* fix(native-chat): stop a bounded tail read from moving the chat cursor past unapplied rows

A structured chat pane could latch "Working for N" forever after the agent had
finished, showing the send arrow rather than Stop, while the sidebar and
`worktree ps` correctly read idle.

The client replica has one position (`state.cursor`) and one body. Two
operations keep those consistent: replace (both from one host snapshot) and
append (rows contiguous with the cursor). The `tail-page` branch was a third
thing: it took the cursor from the journal head, the items from a bounded page
(200 items, byte-capped), then merged retained client submissions over the
page's. Under continuous journal writes the client is always slightly behind,
so the branch ran on every window focus and on every pane re-activation. When
more than a page of rows had landed since a send, that send's user item fell
off the page, its submission was not carried, the retained `pending` survived,
and the cursor jumped past the dispatch-acceptance row. Nothing re-sends it: a
batch carries only touched items and that submission is never touched again.

Delete the third operation rather than guard it. A live subscription is now the
only thing that moves the cursor, and `subscribe({ cursor })` already replays
exactly the missed rows.

- remove the window `focus` listener and the owner/transport `refresh` contract
- skip warm hydration: a retained owner subscribes at its applied cursor
- cold hydration keeps its history read, applied as the existing `snapshot`
  (replace) event rather than `tail-page`
- delete the `tail-page` action and its reducer branch
- delete `resumeCursor` and `shouldAdvanceStructuredResumeCursor`; two cursors
  with two advancement rules were how position and body drifted apart

`older-page`/`loadOlder`, the unattached-refusal grace, generation guards and
the coalescer are unchanged. No host, wire or schema change.

Also fixes a second cost of the same branch: focus during a busy turn discarded
paged-in older items, shrinking the transcript to one bounded page mid-turn.

* fix(native-chat): preserve unavailable mixed-version session fences
2026-09-14 10:28:16 -07:00