mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 00:02:31 +00:00
2b19f21adab5907697ef76ce429bafcd2cfea9ec
10708
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 github-actions[bot]
|
2b19f21ada | release: v1.4.201 v1.4.201 | ||
 Merge Sim
|
17c9f8291d |
ci: sync .github/workflows with main for the 1.4.201 tag push
Why: the pick branch is based on the 2026-09-11 cut and main has since moved
164 commits, diverging 15 workflow files. GitHub's create-or-update-workflow
check on the tag push timed out and failed closed twice
("workflows scope may be required"), blocking release-cut.
CI definitions only. No product code, no lockfile, no dependency change --
the shipped tree is byte-identical to
|
||
 Brennan Benson
|
92b43c6fa3 |
fix(workspaces): complete a worktree create when a post-create step throws (#20175)
* fix(workspaces): complete a worktree create when a post-create step throws
executeWorktreeCreation's try/catch ends once createWorktree resolves, and all
three callers fire it with a bare void and no .catch. completeWorktreeCreation
is the only thing that removes the pending creation, so a throw in that tail
left pendingWorktreeCreations and activePendingCreationId set: the creation
surface stayed up, workspaceChromeActive went false, and the finished workspace
rendered no tab chrome while its panes mounted invisibly behind the panel. It
only cleared when the user switched workspaces, because setActiveWorktree nulls
the pointer. Silently -- no toast, no error state.
activateAndRevealWorktree, ensureWorktreeHasInitialTerminal and
ensureWebRuntimeWorktreeTerminalAfterWake are all synchronous with no internal
guard; launchStructuredWorktreeSession guards only its awaited launch, and that
catch's comment already names this stranding hazard.
The worktree exists past that point, so each follow-up step is now guarded
individually and falls back to the values the skip paths already used; control
flow always reaches completion. The structured-launch cancelled/visibility
returns keep their semantics, and a throw there is treated as a failed launch,
matching what that module already returns for 'failed'. A .catch backstop on
the three call sites turns anything that still escapes -- including
prepareRequestForCreate, whose VM await has try/finally with no catch -- into a
visible error state plus toast.
Ablated: with the guards removed the new suite fails 4 of 5, the survivor being
the no-throw control.
* fix(workspaces): recover terminal after partial activation
* fix(workspaces): preserve stamped launch tab on recovery
* test(workspaces): cover recovered agent tab delivery
* test(workspaces): name recovered agent delivery coverage
---------
Co-authored-by: Merge Sim <sim@local>
(cherry picked from commit
|
||
|
Brennan Benson
|
fff04a8d88 |
fix(terminal): re-run deferred tab admission when a plan installs mid-activation (#20176)
planColdActivationTabDeferral can install an empty allowed set, deferring every
tab so the pane filter renders none. The drain that undoes that,
useActivationDeferredTabAdmission, depends only on backgroundMountRevision and
renderedActiveWorktreeId while reading the deferred set from a mutable ref, and
the install bumps neither: the only revision producers are the drain itself and
the background-mount event path, which the activation plan never reaches.
So this pass strands the workspace with zero panes: the worktree is already
rendered-active while the startup gate is closed, which resets
lastActivationWorktreeIdRef, then the gate opens on the same worktree and
installs a plan. Nothing re-runs the drain until the user switches workspaces
and back. The hook's own comment anticipates this launch shape and relies on
re-reading on growth, but that re-read only happens on a dep change.
applyTerminalColdActivation now returns activationDeferralPlanRevision, backed
by a ref in the parking foundation and incremented only when the plan actually
installs, which the admission effect takes as a dep. A ref rather than state
because the pass runs during render, where a setState would be a render-phase
update.
The 4-tab admission cap is deliberately untouched: it reproduces the warm set
an eager activation used to mount, so steady-state pane and WebGL-context
population is unchanged.
Ablated: with the change reverted the new suite's drain case fails on the
deferred set surviving the timers; the precondition and the away-and-back
control pass either way.
Co-authored-by: Merge Sim <sim@local>
(cherry picked from commit
|
||
|
Brennan Benson
|
ca46cb1f5a |
fix(browser): decode Chromium SameSite storage values (135 cookies silently lost per profile) (#20076)
* fix(browser): decode Chromium SameSite storage values
* fix(browser): document Firefox's real SameSite domain and pin its default-arm inputs
Third-reviewer finding: the replacement comment reproduced the failure mode this
PR exists to kill. It said "Firefox's moz_cookies uses the same 0/1/2 values",
which is true of the overlap and silently wrong about the rest of the domain.
Firefox writes 256 (nsICookie SAMESITE_UNSET) for every cookie with no SameSite
attribute -- the most common shape in a modern profile -- NULL on pre-v10 rows,
and 0 for explicit None OR a legacy unset row the schema-15 migration left
behind, which are not distinguishable in the column. All of those must land on
unspecified, so the default arm is load-bearing for Firefox rather than
incidental. A later reader making the switch exhaustive against the old comment
would have regressed Firefox silently.
Also pins the inputs that were riding the default untested: 256, and the
non-integer arrivals (null, undefined, NaN) that the `?? -1` scan fallback and
pre-v10 Firefox rows produce. Decoder behaviour is unchanged; 11/11 pass.
---------
Co-authored-by: Merge Sim <sim@local>
(cherry picked from commit
|
||
 Neil
|
5c38ef9a6d |
fix(deps): take Electron 43.7.0 for the glibc environ use-after-free (#20089)
(cherry picked from commit
|
||
 Jinwoo Hong
|
729491597f | feat(desktop): measure relay regions and reconnect after idle cutover (#20106) | ||
|
Jinwoo Hong
|
1d7bb47a11 | test(relay): harden regional rehome race coverage (#20136) | ||
|
Jinwoo Hong
|
cd9aa43a2c |
feat(relay): correct regional placement only when the source is idle (#20105)
* feat(relay): correct regional placement only at an idle source * test(relay): lock source activity capacity semantics |
||
|
Brennan BensonandMerge Sim
|
9a56797486 |
fix(mobile): surface host create warnings and terminal-create errors (#20125)
* fix(mobile): surface host create warnings and terminal-create errors
A workspace created from the phone could land on "No tabs in this session"
with a bare red "Failed to create terminal" and no way to tell why. Two
independent drops hid the host's own explanation:
- createWorktreeWithNameRetry returned only {worktreeId, name}, discarding
worktree.create's `warning`, and hostNewWorktreeSessionRoute built the
session route with only `name` + `created=1`. The session screen has always
had the banner (MobileSessionContentRow + createWarningState) -- only the
tasks create path ever fed it, so the New Workspace path could never report
a startup terminal that failed to spawn.
- handleCreateTerminal collapsed every failure to the literal
'Failed to create terminal', throwing away response.error.message.
Both now propagate, so the daemon's pty-allocation hint ("Your system cannot
allocate any more pty devices.") reaches the phone instead of dying in the
main process. Behaviour is otherwise unchanged: a blank warning is still
omitted from the route, and a host that gives no reason still reads
'Failed to create terminal'.
* test(mobile): refresh route parity baselines
---------
Co-authored-by: Merge Sim <sim@local>
|
||
|
Jinwoo Hong
|
ec9c3e0550 |
Fix remote hosted review browser routing (#20030)
* Fix remote hosted review browser routing * Fix remote review modifier hint * Address review feedback and fix routing test types * Avoid assuming active runtime owns workspace links * Align runtime routing regression expectation * Respect explicit local link ownership |
||
|
Brennan BensonandMerge Sim
|
fb19c969a6 |
feat(native-chat): show when a Codex goal is set, changed, or cleared (#19923)
* feat(native-chat): show when a Codex goal is set, changed, or cleared Codex never emits the model's `create_goal` call as an item, so `thread/goal/updated` is the only truthful evidence that a goal exists. Both goal notifications were classified `status-chrome`, which meant no typed handler read them and no row was written -- the only thing reaching the reader was the model's own prose. That prose can be wrong: in a session where no goal was ever created the model still wrote "Goal created: ...". Classify both frames as timeline-substantive and give them a sentence, so the reader can tell a goal that exists from one the model merely claimed. Status is translated rather than echoed, and an unrecognised future status still reads as "Goal updated: <objective>" instead of the bare opcode. Codex re-sends the goal as its token and time counters climb, so rows are deduped on what a reader would notice -- objective, status and budget. A live session sent the same goal three times in one turn with only accounting moving. * fix(native-chat): write the goal signature separator as an escape, not a raw NUL A literal NUL byte in the source made git classify the file as binary, which hid its diff from review. The string built at runtime is unchanged. * fix(native-chat): make Codex goal rows retry-safe * fix(native-chat): preserve Codex goal identity on resume * fix(codex): ignore empty goal clear snapshots * fix(codex): preserve goal lifecycle across rewinds --------- Co-authored-by: Merge Sim <sim@local> |
||
|
Jinwoo Hong
|
08a24efaba | fix(push): preserve distinct Android alerts while offline (#20066) | ||
|
github-actions[bot]
|
9aa0f7e77d | Update README downloads badge | ||
|
Neil
|
a0799d8f1c |
fix(terminal): move the recovery ledger onto the tab row and gate it on observed outcome (#20025)
* fix(terminal): move the recovery ledger onto the tab row and gate it on outcome The recovery budget lived in module-level Maps keyed by tabId. Anything keyed outside the row needs a release path, and that release fired on every remount-driven pane disposal, so each remount erased the budget it had just consumed (crash b5cfc6ca). Put the ledger on TerminalTab and write it in the same set() as the generation bump: reading the budget is now reading the tab, so releasing it independently has no expression. Counting was also the wrong control. Every remount mounts a pane that captures a FRESH recovery epoch, so the epoch check can never refuse its request — recovery re-requested the exact action that had just failed with no evidence anything changed. Gate on an observed outcome instead, reusing the direct-SSH pane retry vocabulary (success | failed | timed-out | superseded) and its settle call sites: an unsettled attempt blocks the next one, and a settled failure refuses the same reason until a new trigger arrives (generation move, or the user's Retry). The 3-per-5min cap stays as a breadcrumb-emitting backstop, not the control. viewMode now also lands on the row from the local toggles, mirroring how pin already does it, so the chat-ownership guard reads one index instead of OR-ing two. * fix(terminal): persist the row's viewMode and keep both chat-ownership reads The narrowed chat-ownership guard read a field the session schema strips: terminalTabSchema never declared viewMode, so the terminal row lost it on every load while the unified tab kept it. After a restart the row read undefined and recovery would remount a chat-owned tab's hidden surface — the race #19745's guard exists to prevent. Declare viewMode on terminalTabSchema so the row is durable, and keep the disjunction rather than replacing it. The schema cannot retroactively add the field to sessions already on disk, so the first load after upgrade still has it only on the unified tab; and for a safety check over two partly-redundant sources, a hole in either index should err toward declining a heal. Also cover three structural guards that no test was holding: both remote ledger-carry paths (terminal-build, remote-workspace-session-merge) and the only success settle in the state machine, including its placement past the failure branches. * fix(terminal): settle a fresh spawn's outcome and prove the ownership guard across a reload spawn-left-pane-unbound was the one recovery reason with no success settle: its remount heals by spawning, not reattaching, so it reached none of the reattach settle points and left the attempt 'pending' for the full 31s bound. A fresh spawn that binds a PTY now reports it, the dual of the unbound settle that already reported failure. Two tests outside src/ still called remountTerminalTabForRecovery by its old boolean contract and broke CI; both are updated to the admission result. Also strips the client-local recovery ledger at the remote-workspace projection boundary, in the type as well as the destructure, so a future producer cannot put another machine's Date.now() on the wire. * fix(terminal): resolve the pane's tab row once for both epochs after the main merge #20034 replaced connect-pane-pty's inline tab resolution with findTerminalTabForPane, and this branch had rewritten the line below it to read the recovery epoch off the row that block used to bind. The merge was textually clean and semantically broken: `terminalTab` no longer existed, so typecheck failed and every test that connects a pane threw ReferenceError. Resolve the row once through the new helper and feed both epochs from it, which keeps #20034's refactor and this branch's reason for reading the row here — a second lookup would put another tabsByWorktree scan on the connect path. captureTabRecoveryGeneration is narrowed to the one field it reads so the helper's record type can carry it. |
||
|
Neil
|
94c2f96ea4 |
perf(daemon): stop scanning every cell for OSC links that cannot exist (#20077)
collectHeadlessOscLinkRanges walks every cell of every row on each snapshot, and called xterm's getCell without the reuse argument its own docs recommend, so a link-free scrollback paid a CellData allocation per cell for a guaranteed empty result. Skip the scan when xterm holds no OSC 8 registration, and reuse one cell when it does. Measured over a 5000-row link-free buffer at 200 cols, same harness back to back, median of 25: 43.85ms -> 0.00ms. This is our bug, not xterm's: xterm already reuses cells in its own serializer and documents the getCell(x, cell) overload for exactly this. |
||
 
|
20c56249d5 |
fix(terminal): keep a deliberately slept workspace cold until it is woken (#20075)
* fix(terminal): keep a deliberately slept workspace cold until it is woken Sleeping a workspace kills its PTYs but keeps its panes mounted and keeps each tab's session id as a wake hint. Any later remount of those panes (recovery, parking, portals) reattached that dead id, and the daemon's create-or-attach spawned a fresh shell, so slept workspaces revived on their own (#10205). The existing sleep-intent marker now outlives teardown and gates the deferred connect itself, so both the reattach and fresh-spawn arms stay cold. It is released by activating the workspace, by any PTY binding to one of its tabs (CLI, automation, client wake), and by purge. A queued startup still connects. Reproduces the community root cause from gatsby74 in #13343; the regression e2e remounts a slept hidden pane and fails on main. Co-authored-by: gatsby74 <gatsby74@users.noreply.github.com> Co-authored-by: mmarabel <mmarabel@users.noreply.github.com> * fix(terminal): let a slept pane wait for its wake instead of latching cold A pane whose connect ran while its workspace was slept used to mark itself connected and stop; nothing re-armed it, so a wake that produced a live PTY before the user clicked (CLI create, background agent resume, split panes) left panes stranded. The connect now waits on the sleep marker and resumes when the marker clears, and a torn-down pane drops its listener. Tabs created with a live PTY clear the marker too, the sleep flow marks each workspace only when its own teardown starts, and purge forgets the marker without waking anything. * fix(terminal): wake a waiting pane once, in its remounted generation Activation clears the sleep marker after the set() that bumps dead tabs' generations, and the waiting pane only resumes its connect when its tab generation is still current. Otherwise the stale pane and its remounted successor both reattached the same session id on a deliberate wake. * fix(terminal): resolve the waiting pane's tab by either id and re-arm after wake The wake listener looked the tab up by the pane's render id, which can be a unified id whose terminal tab lives under entityId, so the generation check declined forever for those panes. Mount, fresh spawn, and the wake listener now share one live resolver. The wait flag resets when the listener fires so a second sleep can hold the pane again, listener dispatch is guarded, folder activation clears after its own set(), and the sleep flow re-asserts the marker after each teardown while releasing a workspace the user activated meanwhile. * fix(terminal): ignore PTY binds that land inside the sleep teardown window A spawn resolving while shutdown was still awaiting the host bound a PTY and cleared the marker, waking every waiting pane mid-sleep; re-marking afterwards could not un-connect them. The sleep flow now scopes each teardown so binds in that window are not wakes. The e2e asserts a deliberate wake yields exactly one PTY, and the dispose test proves the listener is gone. --------- Co-authored-by: Jinwoo-H <jinwoo0825@gmail.com> Co-authored-by: mmarabel <mmarabel@users.noreply.github.com> |
||
|
Neil
|
22d12388a5 | fix(pi): load extension providers for source control generation (#20070) | ||
 Brennan BensonandNeil
|
78e985cd99 |
fix(pi): claim the status pane when the inherited owner PID is dead (STA-5245) (#16631)
* fix(pi): claim the status pane when the inherited owner PID is dead (STA-5245) The managed pi/omp/prime-agent status extension suppressed itself whenever ORCA_PI_STATUS_OWNED held a PID other than its own, with no check that the owner still existed. A restart leaves the previous owner's PID in the inherited env, so every later load returned early and the pane stopped reporting status permanently. Probe the owner before suppressing. Only ESRCH proves it is gone; any other probe result keeps suppression so a live foreign owner still cannot double-report. This mirrors the tri-state in main/agent-hooks/managed-hook-owner-identity.ts, which the extension cannot import because it loads inside the pi/omp runtime with no Orca deps. Also extracts the generated-source test harness into its own module so the suite stays under the max-lines limit. * fix(pi): validate inherited status owner pid markers --------- Co-authored-by: Neil <neil@stably.ai> |
||
|
Neil
|
6bb2b0c6d7 |
test(runtime): capture real Antigravity transcripts — the detector is inverted on live output (#19983)
* test(runtime): capture real agent PTY transcripts before rewriting Antigravity readiness
Antigravity readiness has been written five times against a five-line screen
typed from memory. There is no Antigravity transcript in this repository, so
every attempt was a guess tested against another guess. This adds the recorder,
the protocol and the fixture-driven suite so the sixth attempt can be written
against evidence, and changes no detector logic.
- config/scripts/capture-agent-pty-transcript.mjs records a live agent session
through a real PTY, escapes and wrapping intact. Ctrl-] is consumed by the
recorder and never forwarded, which is the only way to end a capture while a
dialog still owns the screen.
- config/scripts/pty-transcript-secret-scan.mjs finds account identifiers and
credentials, redacts them with same-length placeholders so wrapping survives,
and recognises its own placeholders so a scrubbed file verifies clean.
- src/main/runtime/antigravity-readiness-transcripts.test.ts asserts a verdict
per transcript and skips by name until the transcripts land, with a
doc-coverage ratchet and a guard that a fixture contains escape bytes.
The escape-byte guard exists because the three cursor-agent fixtures carry a
comment claiming they were captured verbatim through Orca, yet contain zero ESC
bytes and zero carriage returns. That comment is corrected here to say what
those files are; the fixtures and the rules built on them are untouched.
* test(runtime): capture real Antigravity transcripts, and pin what they prove
`agy` 1.1.25 turned out to be installed, so the transcripts this scaffold was
built for now exist. Six are recorded from live sessions and committed; the
rest are named as skipped, because reaching them would mean signing the
operator out or deleting their config.
The captures invert the story. On real output the shipped detector refuses a
genuinely ready screen and accepts a live `/model` picker:
- Antigravity paints a block-glyph logo down the left, so the model row never
starts a line. `startsWith('gemini', trimmedStart)` cannot match a real ready
screen, on any account or model. Stripping the logo flips the same screen to
ready, which means a decorative glyph decides readiness today.
- The `/model` picker prints `Gemini 3.x Flash` one per line, at line start, and
a bare `>` composer sits earlier in the tail. Both halves of the rule are
satisfied while a dialog owns the screen.
- For an API-key user the identity row reads `Gemini API key` — no `@`, no
domain — and `AGY_CLI_HIDE_ACCOUNT_INFO=1` removes the row entirely. The
account-row requirement of attempts 4 and 5 can never pass for those users.
- The banner is printed once and never reprinted after a dialog is dismissed, so
`headerIndex` cannot be the ordering anchor.
Four suite cases are pinned as KNOWN DEFECT: they assert what the detector does
so CI stays honest instead of permanently red, and flip to failing the moment
someone fixes it. No detector logic changed.
The recorder gains `--send "<ms>:<text>"` because a dialog capture has to be
driven and an unattended run has no TTY, and the scrub scanner gains a UUID rule
because agy prints a resumable conversation id on exit.
* test(runtime): capture agy mid-turn, and make the scan file reviewable
Answers the busy-frame question a P1 review raised against attempt six, with
two new captures from a live turn.
At the frame level the review is right: a busy frame parks the caret with the
same bytes as an idle one, `CR ESC[2A ESC[2C`, and the only differing row —
`esc to cancel` versus `? for shortcuts` — is erased by that park.
At the retained-tail level it does not reproduce. Each spinner tick is its own
repaint with its own `CR ESC[2A`, two rows higher than the frame's, which
splices the composer away: a live turn's tail ends on `⣟ Generating...`, with
no bare caret to match. A constructed input that keeps the park and edits only
the status text is not faithful, because a live turn has a spinner row
repainting below the composer.
The residual is the gap between a frame park and the next tick, where the tail
does end on the bare caret. Quiescence-gated paths are safe there because ticks
keep arriving; text-only paths are not, and for those the capture supports one
clause: a braille glyph on the last visible line means working. That predicate
already exists here for cursor-agent and should be reused, scoped to the last
line — a first-run transcript prints `⠾ Signing in...` during startup.
Also in this commit, from the same review:
- pty-transcript-secret-scan.mjs held raw 0x00-0x1f bytes in a character class,
so the one file gating real PTY data into history was binary to git and
unreviewable in a diff. It now tests codepoints, which the formatter cannot
fold back into control bytes.
- Pin `src/main/runtime/__fixtures__/*.txt` as -text. A Windows checkout would
otherwise normalise line endings and rewrite the CR bytes that make these
files evidence.
The recorder now stops appending at the stop moment rather than through
shutdown: an agent repaints an idle frame on its way out, which was overwriting
the mid-turn state the capture existed to record.
* test(tooling): allowlist the transcript scan test in the batch-shim ratchet
pty-transcript-secret-scan.test.mjs asserts that the capture recorder routes
an 'agy.cmd' shim through cmd.exe, so the shim literal it names is the
assertion, not a spawn. Fits the existing assert-on-shim-files category.
|
||
|
Neil
|
2167cd2994 |
test(monaco): drive the real Monarch tokenizer instead of walking rule tables (#19981)
* test(monaco): drive the real Monarch tokenizer instead of walking rule tables
* fix(monaco): stop a truncated JSONL record poisoning every record after it
`@string` was pushed unconditionally, and Monarch state survives the line
break, so one truncated record left every later record inside the string state
— the whole rest of the file rendered as a single string. A truncated record is
a normal way for a .jsonl log to end.
Gate the push on a lookahead proving the closing quote is on this line, and
consume an unterminated remainder as `string.invalid` without pushing anything.
Measured against the real tokenizer over realistic JSONL. Two alternatives were
rejected: collapsing the string into one regex fixes the poisoning but loses
every `string.escape` token on well-formed records, and `includeLF` does not
fix the primary case at all (the `[^"\\]+` content rule swallows the newline
before an EOL rule can match). This candidate's token stream is byte-identical
to the previous grammar on every well-formed record — the existing inline
snapshot did not move — and is at or faster than it on 20k pathological lines.
* test(monaco): pin monaco's own mdx grammar as the embed-recursion regression
Upstream's shipped mdx grammar enters a `js` embed on every `{` and pops
on `}` with no budget, so it reproduces the unbounded embed-entry
recursion exactly — evidence the shape is monaco's, not something Orca's
grammars invented, and a tripwire for a monaco upgrade that changes it.
The same file proves an Orca grammar stays inside the budget under the
identical line. Measured here: 500 interpolations -> 500 frames, no error;
3000 chars -> RangeError at 945 frames. The ceiling is runtime-dependent,
so the test asserts the failure, not the number.
Adds a regression for astro's `^`-anchored frontmatter pop rule
(monaco-editor#1127) so an indented or trailing `---` cannot close the
fence early, de-duplicates the line-cap constant onto the budget module's
`MAX_TOKENIZATION_LINE_LENGTH`, and renames the recursion suite: embeds
cannot nest, so "embedded recursion depth" described the wrong thing.
|
||
|
Neil
|
5fa62feda7 |
perf(terminal): mount only the visible pane on a worktree switch (#20034)
* perf(terminal): mount only the visible pane on a worktree switch Activating a worktree mounted a TerminalPane for every tab it holds, not just the one on screen. Cold-activation deferral existed for this but engaged only past four deferrable hidden tabs, which exempted the 2-5 tab worktrees that make up almost every real switch. Deferral now engages for any deferrable hidden tab, and the siblings it skips are admitted one per idle frame after the reveal, capped at the population the old threshold would have mounted eagerly. Steady-state pane, WebGL-context and heap population are therefore unchanged; only the frame the mounts land on moved. * fix(terminal): judge admission eligibility on the largest deferred set seen Review found the launch worktree never warms up: it is restored active before hydration opens the startup gate, so admission read an empty deferred set, cached ineligible, and never recomputed once the real plan landed. Judge on the high-water mark instead - an over-cap worktree still stays ineligible as its set drains, but a later plan is seen. Also from review: the e2e WebGL counter read getPanes(), which returns a public projection with no webglAddon field, so it was always 0; read getRenderingDiagnostics() instead. Filler worktrees now clean up on failure (testRepoPath is worker-scoped), and the restore metric is named for what it measures rather than implying a pixel assertion. * test(e2e): wait for the reveal to restore, and scope the latency budget off CI CI failed with 'revealed terminal never restored its content': the harness sampled a fixed 4s window, which a shared runner can outlast, so a slow restore was recorded as no restore. Poll for the restore instead. Also stop asserting a latency budget on CI. Shared runners cannot hold a threshold; the structural invariants (one pane mounted by the switch, warm set restored) are exact and stay asserted everywhere. |
||
|
Brennan BensonandMerge Sim
|
fab78c7669 |
fix(native-chat): show one live-turn indicator, and make Thinking mean reasoning (#19977)
* native-chat: render one indicator row for the live desktop turn The turn-timing row and the spinner+activity line were two rows saying "Working" at once. A settled turn keeps its own row; the live turn now has only the spinner row, labelled provider activity -> Thinking -> Working for N through the shared resolver. Reasoning is the turn's content, so it no longer becomes the activity label, and "Thinking" now means the turn is reasoning right now rather than that it has produced no output yet. * mobile: give the live turn row a spinner and the shared indicator label Mobile's per-turn row is already the only live indicator on the structured lane, but it pulsed a bare word and never showed what the provider said it was doing. It now renders a spinner beside the same resolved label desktop uses, and reads reasoning from the journal instead of inferring it from missing output. The bridge lane's four prompt/interrupt write seams move to one module so the controller stays under its line cap. * codex: mark streamed reasoning as reasoning too, and pin the provider markers The settled reasoning item carried the marker but the streaming one did not, so a live Codex turn - the only time the indicator is on screen - never read as reasoning. Both paths now stamp it; a plan document keeps its own presentation and must never read as reasoning. * fix(native-chat): tighten live turn reasoning state --------- Co-authored-by: Merge Sim <sim@local> |
||
 Jinwoo HongandOmar Shahine
|
3b82d8de64 |
fix(runtime): let connections own host status recovery (#20003)
* fix(runtime): let connections own host status recovery Verify runtime status after authenticated connection recovery and publish ordered snapshots to desktop and browser viewers. Consolidate failed-status retries in the connection owner and remove renderer retry/diagnostics merging. Adapt sidebar host-state derivation and regression coverage from Omar Shahine's original fix in https://github.com/stablyai/orca/pull/19163. Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com> * fix(runtime): show blocked hosts honestly and remove obsolete status options * fix(runtime): preserve timeout guidance and update IPC test fixtures * fix(runtime): preserve status evidence and address review gaps * test(sidebar): assert workspace host icons dimming and recovery tooltips * fix(palette): require available hosts before adding implicit badges * fix: retain disconnected host snapshots for new renderers --------- Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com> |
||
|
Brennan BensonandMerge Sim
|
b6e4457552 |
fix(worktrees): close an idle structured chat on delete instead of refusing (#19762)
* fix(worktrees): close an idle structured chat on delete instead of refusing `worktree rm` refused whenever any structured chat session was attached to the workspace, so an idle Codex/Claude chat that had already answered was harder to delete than a terminal actively running the same agent. The PTY sweep stops every terminal it owns and refuses only for the ones whose exit it could not verify. The structured sweep refused on `live` alone and never attempted the close, which ran only under force. `live` is lease state — a provider child is attached — not work in flight, so it was never the right proxy for "you would lose something". Close first, refuse only on what did not settle. The refusal now means the same thing the unverified-PTY one does, so the toast takes that wording. * fix(worktrees): fence, bound and word the structured-session sweep Review follow-ups on the close-first structured sweep. The close-first direction is unchanged; four things it got wrong are not. Host fence. `listLiveStructuredSessionsForWorktree` matched on `location.workspaceId` alone, and a `repoId::path` id names a DIFFERENT workspace on every host (STA-4343). Once the sweep started closing rather than refusing, deleting a local workspace could close a live chat on an SSH or paired-runtime copy of the same id. It now takes the same two host fields the PTY sweeps already fence on, compared against the session's own `location.executionHostId`; neither field set means this machine. Shared budget. The close ran to completion before the first PTY sweep was constructed, and it is serial with a provider round trip per session — so a slow one spent the whole budget and the sweeps then rejected with a timeout for a stop they never attempted. It is now issued first but joined before the verdict, so the agent plane is still asked ahead of the terminal plane while the two share the clock. Timeout wording. The close raced the deadline fail-closed, and that sentinel carries the PTY timeout prefix, which the classifier reads first — so a wedged session close refused in terminal wording and refused identically again under the Force Delete meant to clear it (#11960). A close that ran out of time is now a session the removal could not confirm closed, which is what the refusal already words. Tracked, so a forced removal still waits out the abandoned-sweep grace before deleting files. Verdict fidelity. `closeStructuredAgentSessionChild` re-observes after the close, and that verdict was being discarded — so a session Orca watched stay attached and one it merely could not reach produced the same message, while the toast asserted "could not confirm" for both. `removal.ts` documents flattening those two as the thing not to do. The unclosed sessions now carry their post-close status, the detail uses the shared `still live:` marker, and the toast branches on it like the PTY pair above it. Also: the close takes the enumerated list instead of re-deriving it, so it no longer runs every liveness observation twice or names a session it never touched; and both teardown log lines count structured closes, since closing a chat is now an ordinary outcome of this verb. * fix(worktrees): keep a proven-exited session from refusing removal The structured sweep re-observes after a close that reported `stopped: false`, but folded a proven `exited` into `unverifiable` — so a close that threw past its own observation, or one whose death evidence landed a beat later, refused a delete over a child that is demonstrably gone. That is the defect this sweep exists to remove, and the PTY gate it mirrors never refuses on a proven exit. Take the proof, and run the tab retirement the close skipped when it gave up: a chat tab left behind re-attaches a released session pointing at a workspace that is about to be deleted. * fix(worktrees): name every unclosed structured session, not just the live ones The refusal named only the proven-live subset when any session was live, so a sweep that left one attached and two unconfirmed told the user "1 agent session (claude)" while three were about to be discarded — and dropped the providers of the ones it hid. The PTY sibling may drop everything outside its live list because a fresh inventory PROVED those exited; nothing proves that here, so both groups are counted. The `still live:` marker still leads, so the delete toast keeps showing the stronger warning. Also carries the structured close count through the forced-removal early return: that path skips the per-PTY verdict, not the sweep that already ended a user's chats, so the removal log claimed `structured=0` for chats it had just closed. * fix(worktrees): stop the forced-removal warn asserting a verdict it does not have The structured sweep splits its post-close verdict in two on purpose: "we watched it stay attached" and "we could not confirm it closed" are different things to waive, and `removal.ts` keeps a marker and a matcher together so the delete toast can tell them apart. The force-path warn then appended "still attached" to whichever verdict it got, so a removal forced over a close that merely ran out of time logged that Orca had seen the session running. That line is the only record a forced removal leaves of a child left pointing at a deleted `cwd`, so it is the one place the two must not be flattened. Carry the verdict verbatim, the way the unstopped-PTY warn above already does. * fix(worktrees): report the closes that landed when the sweep budget expires The structured close loop is serial, so the shared sweep budget can expire part-way through it. The timeout fallback was assembled by the caller and could only name the whole list: sessions this removal had already closed were reported as unclosed, named in the refusal the user reads, and logged as `structured=0`. The loop now records progress into a structure the timeout path reads, so both the refusal and the count say only what was observed. A session with no recorded outcome reports `unverifiable` — the same verdict as an attempted close that stayed unproven, because "never asked" and "asked, unconfirmed" are both exactly "not observed exited", and neither may claim `live`. The loop also checks the deadline before each close, so one slow provider round trip no longer starves every session behind it. It stops ISSUING closes; an in-flight one is left to finish, since nothing here can cancel a round trip. The structured host fence now reuses the PTY fence's own type instead of a look-alike that read `undefined` as local while the other read it as match-all, with both claiming the same precedence. `null` means this machine on both sides; ABSENT stays narrowed to local here, documented and pinned, because a single-host-id comparison cannot express match-all. Also pins a tradeoff that was accepted rather than wanted: the PTY sweeps run concurrently with the structured close, so a removal that refuses over a stuck session has already killed that workspace's terminals. * fix(worktrees): put the chat tab back when a structured close does not land `closeStructuredAgentSessionChild` hides the session's chat tab before it issues the close, so every failure past that point left a refused delete having still taken the tab out of the durable restore index. The conversation survived under `userData`, but nothing brought the tab back at the next launch. Both failure shapes now roll the hide back: `host.close` throwing, and the post-close observation coming back not-`exited`. The restore is gated on the visibility read taken BEFORE the hide, so it never publishes a tab for a session that was already hidden, and on a fresh observation, so it never resurrects one for a child a throwing close still took with it — which is what the worktree sweep reads when it counts such a session closed. It cannot throw out of the function, so the caller's original reason is still what the user is asked to act on. * fix(worktrees): keep the chat-tab rollback out of removals that delete the workspace The rollback added for a refused close ran on every unproven close, including the two shapes of removal that cannot refuse. Force Delete warns and deletes the checkout; a folder-workspace removal never refuses at all. Putting the tab back on those paths leaves a durable restore-index entry for a workspace that is then gone, and the chat republishes at the next launch pointing at it — the outcome this sweep exists to remove. The close now takes `restoreTabOnUnprovenClose`, on by default so `worker-stop` and `worker-release` keep the rollback, and the teardown sweep passes it only when the removal can still refuse. Second hole, same chain: `host.close` can return before the child's exit is recorded, so the close's own observation reads unverifiable and restores the tab, while the sweep's re-read one store write later proves the exit and counts the session closed. The two observations straddle that write and disagree. The sweep now re-drops the tab reference when it takes that proof, and the comment claiming the re-observation alone covers this is corrected. * fix(native-chat): stop a closing chat reading as a conversation that would not load Deleting a workspace now closes the structured chats inside it, and the chat pane outlives that close by a few frames. Every read it makes in that window — `agentSession.history` on refresh, `agentSession.subscribe` on reconnect — resolves through the host's `requireSession`, which refuses with `agent_session_ownership_unknown` for a session it no longer holds. The pane turned that into its terminal error surface, so an ordinary delete flashed `Could not load conversation` over the transcript before the tab retired. That code, raised by a READ, never means the transcript could not be read. It means this host has no session object by that id: one it has just closed, or one it has not attached yet, since the surface's hold is what attaches a session at all. Both windows end on their own. The genuinely latched lease — Orca cannot prove the previous owner exited — reaches the client through the acquisition path instead, so narrowing on the code costs a read no real diagnosis. So the read transport classifies before it reports: an unattached refusal stays on the reconnect loop it is already the subject of, and the pane keeps the transcript it has. It is a window, not a mute. A read still refusing that way past the grace is no longer transitional, and the pane is owed the failure rather than a spinner that never resolves. Every other failure still surfaces immediately, unchanged. The refusal code now has one definition, shared by the host that raises it and the client that narrows on it, so the two cannot drift into a red error nobody meant. Deliberately NOT changed: the order of teardown. The tab is retired after the close proves, not before it, because a close that does not settle has to put the user's chat tab back — the rollback this PR already establishes. Retiring the pane first would unmount it ahead of a close that may be refused, so the pane instead treats a session that has gone as a neutral terminal state. Mobile's structured chat reaches the same reducer but has no reconnect loop, and its hold refusal is what carries the diagnosis there, so the grace does not transfer; it keeps reporting as before. --------- Co-authored-by: Merge Sim <sim@local> |
||
 
|
b3e0a33fa4 |
fix(runtime): agent-neutral wait-blocked reasons (#19749)
* fix(runtime): agent-neutral wait-blocked reasons and non-Gemini Antigravity readiness Reported by a user via the in-app help menu (report "not captured", 1.4.198). The trust/interactive/update/cwd prompt matchers are agent-agnostic - they match on dialog wording and never inspect the pane's agent - yet emitted hardcoded codex-* reasons. Those reached users verbatim in worker receipts (local-worker-start, federation), two automation surfaces, and raw CLI output, so an Antigravity user was told they had a Codex problem. findAntigravityReadyPromptIndex also required the model line to start with the literal "gemini". Antigravity CLI is not Gemini-only, so a non-Gemini session never registered as ready, stale trust text was never superseded, and the pane stayed blocked - which is why dispatch --inject answered agent_prompt_blocked. Add agent-neutral reasons additively (codex-* members kept on the wire per docs/reference/remote-wire-compatibility.md, with a legacy alias for older hosts) and decide Antigravity readiness structurally: header, then model/account rows, then the prompt caret. codex-model-migration-prompt and codex-hooks-review-prompt stay Codex-named - both key on Codex's own wording. * fix(runtime): finish the agent-neutral rename, revert the Antigravity readiness rewrite Review follow-up on this branch. Splits the two halves of the original commit: the reason rename lands, the Antigravity readiness detector goes back to merge-base until someone captures a real transcript. Rename half: - 'hooks need review' + 'press enter to confirm' inspects no agent, so it now publishes agent-hooks-review-prompt. That was the last agent-agnostic codex-* emission left, and it is the one the original report was about: a Claude Code user hitting a hooks dialog still read "codex-hooks-review-prompt". - The legacy alias is applied at all three surfaces that render a raw reason, not just the CLI. describeTerminalWaitBlockedReason() is the single formatter; the worker and federation "Agent startup blocked:" receipts use it too. Kept one-directional: nothing consumes agent-* -> codex-*, since an old client renders with its own shipped code. - Restores the compat note deleted at the permission-choices site. The Rule 1 citation is correct - remote-wire-compatibility.md names this enum by name. Antigravity half, reverted: findAntigravityReadyPromptIndex goes back to merge-base (header + a 'gemini' model line + a lone '>' caret) and antigravity-ready-prompt-index.ts is removed. Executing both builds against constructed tails, the rewrite read a live startup dialog as ready. Adding the account row from this repo's own ready-screen fixture to five silent startup dialogs (sign-in, model picker, theme picker, privacy notice, update banner) flipped all five from unready to ready; so did any narration line containing an email address, with no account row at all. Readiness is what gates typing the task prompt into the pane, so that path types a task prompt into a live authentication dialog. Merge-base returns unready for all ten. The rewrite also did not reliably fix the wedge it targeted: with no account row and a non-Gemini model - a personal or API-key user - it still returns unready. No real Antigravity transcript exists in this repo. The cursor-agent rules are derived from captures under src/main/runtime/__fixtures__; Antigravity has no equivalent, and every attempt so far has been tuned against a hand-written 5-line fixture. A false negative (the agent waits) is safer than a false positive (we type into an auth dialog), so this ships the known behaviour. Reverting restores a pre-existing gap, not a regression: a non-Gemini Antigravity session wedges on merge-base too. Closing it needs a captured ready screen and a captured dismissed-dialog screen, for a personal/API-key account as well as a Business one. Tests: - Ten ratchet fixtures pin the shapes any replacement detector must refuse - the five silent dialogs with an account row, and each with a narrated email. All ten fail against the reverted rewrite. - Vacuous tests rewritten so they fail without the code they cover: the CLI alias tests asserted only the absence of a suffix, and the worker receipt test asserted the raw token. Tests that are characterization rather than a guard now say so on the line above. --------- Co-authored-by: m4air <m4air@m4airs-MacBook-Air.local> Co-authored-by: Neil <neil@stably.ai> |
||
|
|
26db907895 |
fix(monaco): bound embedded-language recursion in svelte/astro/vue grammars (#19748)
* fix(monaco): bound embedded-language recursion in svelte/astro/vue grammars
Monarch's _nestedTokenize and _myTokenize tail-call each other on every
mid-line embed entry, and V8 has no TCO, so JS stack depth grew one level
per <script>/<style>/<!-- -->/{expr} on a line - bounded only by line
length. One 17,000-char line of '<script></script>' overflows svelte at
depth 997; 19,603 chars of '<!---->' overflows astro at 1748. Both are
under Monaco's own 20,000 maxTokenizationLineLength, so it was no defence.
The same recursion rescans the line remainder per level (quadratic),
matching the 38s synchronous stall before report 25d10fa1's
STATUS_STACK_OVERFLOW on a flat, healthy heap.
Add a shared embed-entry budget: enter an embed only while <=512 chars
remain. Each entry consumes a character, so depth is bounded by
construction; over-budget remainders continue on parallel non-embedded
states that keep tag-level colouring.
Also fix the zero-width nextEmbedded rules that dropped their embed
(token must be '@rematch'), the source of the "cannot pop embedded
language if not inside one" breadcrumbs, and rework vue's expression exit.
Fixing that drop without the budget would have armed the overflow in vue.
* fix(monaco): re-embed script and style bodies after an over-budget opening tag
`scriptBodyPlain` / `styleBodyPlain` were the only over-budget mirror states
without a re-entry rule, and they dropped `$S2` as well. A `<script>` or
`<style>` opening tag carrying more than 512 trailing characters therefore left
the whole block unhighlighted until its closing tag, however short the following
lines were. Carry the language through and re-enter the embed as soon as the
rest of the line fits, matching the markup and expression mirror states.
Also extends the recursion ramp so the densest embed shape (`{a}` / `{{a}}`) is
driven at Monaco's line cap (19_800 / 19_528 chars) instead of stopping at
7_500, and renames the inverted private `restOfLineTooLong` constant.
The budget stays at 512: an A/B of the real tokenizer at 512 vs 256 over
realistic SFCs differs on 8 lines, all of them 256 losing the html or
typescript embed on ordinary shapes such as a ~430-character Tailwind class
attribute.
* fix(monaco): pin the tokenization line cap and correct the budget's claims
Sets `maxTokenizationLineLength` explicitly instead of inheriting it. It is
an `IGlobalEditorOptions` value, so the one file-editor site pins it for
diff and Peek surfaces too. Defense-in-depth only — the comment says
plainly that it does NOT guard the embed recursion, which overflowed at
~17_000 chars, under this cap.
Corrects two overstatements in the budget module's own comments: Monarch
refuses to nest embeds, so the counts are sequential enter/exit
transitions (stack frames), not nesting depth; and the `RangeError` is
caught per line by Monaco's `safeTokenize`, so what is demonstrated is a
line that silently loses highlighting, not a dead renderer.
Notes monaco-editor#1127 at astro's `^`-anchored frontmatter pop rule: the
two-caret symptom is fixed in 0.55.1, but `^` in a pop rule is still
measured from where the embed was entered, not from line start.
---------
Co-authored-by: m4air <m4air@m4airs-MacBook-Air.local>
Co-authored-by: Neil <neil@stably.ai>
|
||
|
|
7d367b2aa4 |
fix(terminal): stop the recovery budget erasing itself during a remount (#19745)
* fix(terminal): stop the recovery budget erasing itself during a remount A recovery remount disposes the pane's xterm, and the disposal handler released the tab's recovery budget whenever getTab said the tab was gone. getTab reads unifiedTabsByWorktree while remountTerminalTabForRecovery reads and mutates tabsByWorktree; on the direct-SSH path the two indices diverge, so every successful remount deleted the timestamp it had just written. The cap never engaged and the pane remounted at render speed. Report b5cfc6ca (1.4.198, Windows): 8878 remounts across 8 tabs in 122s, all reason=reattach-unverifiable, against a cap of 3 per tab per 5 min, ending in a Skia bitmap allocation abort. Gate the release on the same index that governs remounting, via a shared locateTerminalTabForRecovery so the two cannot drift apart again. * refactor(terminal): resolve a recovery tab through one tabsByWorktree scan Collapse the recovery lookup onto a single primitive, locateTerminalTab, and express the already-exported isTerminalTabPresent in terms of it. The budget release now calls isTerminalTabPresent directly, so the store drops the hasTerminalTabForRecovery action added alongside it. The native-chat ownership guard also consults tabsByWorktree: the unified tab index owns viewMode, but it can transiently drop a row the remount index still holds, and that hole used to read as "not chat-owned" and remount a chat-owned tab. Drops the storm control case that passed with and without the fix, and pins the surviving drift case to the exact remount count the cooldown produces. --------- Co-authored-by: m4air <m4air@m4airs-MacBook-Air.local> Co-authored-by: Neil <neil@stably.ai> |
||
|
Neil
|
a93e0aea80 |
fix(pi): retire stale spinner and prompt timers across reloads (#20035)
* fix(pi): retire stale titlebar extension timers * test(pi): cover prompt timers during generation replacement |
||
|
Jinwoo Hong
|
e187c82678 | Revert mobile push rollout pending delivery investigation (#20040) | ||
|
NeilandMerge Sim
|
1113df6b50 |
fix(daemon): pin owner-only modes on persisted terminal history (#19954)
* fix(daemon): pin owner-only modes on persisted terminal history Terminal history persists verbatim screen and scrollback to disk with no file mode pinned, so every directory and file landed at whatever umask applied — world-readable under a default umask. checkpoint.json holds snapshotAnsi + scrollbackAnsi, retention keeps 10,000 sessions, and any other local user could read all of it. Pin 0o700 on the history directories and 0o600 on checkpoint.json, output.log, meta.json, the recovery-protection marker, and the checkpoint tmp file that is renamed into place. `mode` only applies at creation, so existing trees also need a chmod: a bounded, marker-guarded sweep of the base dir repairs the backlog once, and each session tree is tightened as its writer attaches. Windows ignores POSIX mode bits and can reject chmod outright, so every chmod is skipped on win32 and swallowed elsewhere; history writing and daemon startup never depend on it. WSL and remote SSH Linux hosts run the same path and do get the modes. While here, hoist the duplicated session-file name list into terminal-history-session-files so the stale-file reset and the permission tightening share one definition. * fix(daemon): keep the history permission sweep off frozen sessions and off startup Three follow-ups to the owner-only history modes, all consequences of running the backlog sweep on the daemon-init critical path. The sweep chmods every session directory, and chmod moves `mode` and `ctimeMs` -- the two fields `fingerprintTerminalHistorySession` hashes. A sweep landing inside a recovery-freeze window therefore failed the re-check in `registerWriter`/`openSession` with `terminal_history_recovery_generation_changed`, which disables the session's writer. `onWriteError` has no production caller, so that pane silently persisted no scrollback for the rest of the run. The sweep now skips any session tree under recovery protection: an open freeze, tracked process-wide because the freeze and the sweep live in different modules over one base path, or the on-disk `.unreadable-recovery` marker left by a failed quarantine. Nothing is left loose -- a live session's own writer tightens its tree when it attaches. The fingerprint is unchanged; mode and ctime stay in it, because this was a scheduling problem, not a corruption check that was too strict. `getDaemonHistoryDir()` runs more than once per startup -- directly, and again through the `historyPath = getHistoryDir()` default in `createLegacyDaemonAdapters` -- and the scheduler had no guard, so two full sweeps ran concurrently over one tree. The scheduler now arms at most one sweep per base path per process. The set is never cleared, so a sweep that throws cannot wedge a retry loop; the on-disk marker is what carries the decision across launches. The sweep also ran inline on the daemon-init path, measured at 1,581 ms over 8,769 files. It now defers 10s, the same wait the sibling history GC takes before walking this very directory and for the same stated reason. The inline cost of the accessor drops to ~0.07 ms. Deferral does not carry the first fix: a freeze can still be open at 10s, so the skip stands on its own. `HistoryManager`'s freeze bookkeeping moves to `TerminalHistoryRecoveryFreezes` so the in-memory freeze and its sweep hold cannot drift apart across the many release paths, and so the file stays under its line cap. Co-authored-by: Merge Sim <sim@local> --------- Co-authored-by: Merge Sim <sim@local> |
||
|
Jinwoo Hong
|
2ecde717b4 | refactor(rpc): make defineMethod preserve the method name and handler result (#20016) | ||
|
Jinwoo Hong
|
1fedda14ca |
feat(ai-vault-search): session search indexer library owning backfill, reconcile and retention (#19751)
* feat(ai-vault-search): expose the seams a scheduler needs to stay honest
Three questions a lifecycle owner has to be able to ask, none of which had an
answer: how many unfinished writes this open tombstoned, what the index
believes it holds, and whether the session list's cursor already covers a file.
The last one is not a nicety. The reader only opens a transcript the list still
needs, so any file the list scanned before the index existed would be reused
from cache and never reach the index at all. Naming the rule where it lives
keeps one spelling of it instead of two.
* feat(ai-vault-search): the bounds an unasked background index has to respect
An injected clock, a retention window, a per-cycle allowance in files and
bytes, a bounded re-read queue, and the load-aware pacing from the original
branch.
The allowance deliberately does not carry unspent room forward: accumulating it
would let a long idle stretch buy one unbounded cycle, which is the stall the
budget exists to prevent. Work that does not fit is queued, not dropped, and a
transcript larger than a whole cycle's bytes is admitted alone rather than
being refused forever.
* feat(ai-vault-search): read a candidate set into the index, and say what happened
One pass over a candidate list, plus the four things a caller has to be told
about afterwards: what was discovered, which roots could not be read, which
sources are provably gone, and where the whole run stands.
Two absence rules are load-bearing. The file walker swallows a readdir failure
and returns, so an unreadable root and an uninstalled agent both arrive as "no
files"; only a root that yielded nothing is re-probed, and only ENOENT counts
as absent. A source is retired on the same evidence and no weaker: an EACCES,
an EIO or a stalled distro keeps its rows.
Progress lives in the index's own files table, so a pass skips what it already
covers and an interrupted run resumes instead of starting over.
* feat(ai-vault-search): a whole-machine sweep and a recency-window cycle
The sweep enumerates every root without a limit and is the only pass that can
retire a source deleted while nothing was running. The cycle re-stats the
newest N per agent, which is the sidebar's own rule rather than a second one,
folds in the store's stale set and anything a caller invalidated, and reads
what changed inside the cycle's allowance.
A replaced file gets a whole re-read in the cycle that notices it. Waiting for
the index to decline an append and mark itself stale would cost a second cycle
and, because the reader resumes from the session list's cursor, would decline
again every cycle after that.
* feat(ai-vault-search): SessionSearchIndexer, with its freshness claim tested
The object that owns freshness for the index: a full sweep on start, a timer
that reconciles the recent window, retention that purges when it narrows and
re-sweeps when it widens, and a pause that stops the writers rather than only
the timer.
A library, not a service. No Electron, no app lifecycle, no settings read, no
IPC, and nothing constructs it. The clock is injected because a guarantee
stated in wall time is a claim until a test can advance the clock and watch it
hold: a transcript in the recent window that grows, is rename-replaced, or is
deleted is reflected within one interval, each with its own test.
* test(ai-vault-search): index a conversation held in Orca's own chat
Reviewer F4 and the plan's fourth open decision. A conversation held in the
panel writes the same file in the same place as one held in the terminal, so
it must be searchable through the same path with nothing else running. The
test constructs the indexer over an isolated root, writes and then appends
native-chat-shaped rows, advances the clock one interval, and reads the rows
back through the published views.
* fix(ai-vault-search): ask the store before reading its cursor
The pass read the index's own file row to decide how to read a candidate, and
only then asked whether the store would accept it. A store that is paused or
already closed answers no to everything, so those cursor reads were against a
handle it had given up.
* feat(ai-vault-search): take the reader's whole-read seam and PR 2's pause rules
`requestWholeTranscriptRead` replaces the local invalidation: the reader owns
the resume point, so asking it is the honest way to say the index needs a span
the session list has already moved past.
Two callers, and the second is the one no decline can reach. A forced path is
one the store handed back from `takeStale` or a caller invalidated, and it has
to arrive as a `replace` or the consumer declines the same append forever. But
when the list's cursor already sits at a file's current stat the parse opens
nothing at all, so no consumer is asked and there is nothing to record. That is
every transcript on the machine the first time the index is switched on inside
a running app, so it gets a test on both the sweep and the cycle path.
Pausing now keeps the store's re-read set, so `filesPending` and
`droppedPending` add both bounded queues together: a caller cannot act on one
of them alone. The schema creates the index directory, so the indexer no longer
does.
* refactor(ai-vault-search): one ceiling for both re-read queues
The indexer's scheduled-work queue carried a bound of its own beside the
store's STALE_PATH_LIMIT, so `droppedPending` summed two numbers that meant two
different things. It now shares the store's ceiling, set by the indexer, which
is the only thing holding both queues.
They stay separate queues. The store records that the index has a hole in a
file; this records work scheduled and not yet done. Merging them by pushing
budget leftovers through `markStale` would turn every deferred append into a
whole re-read.
* fix(ai-vault-search): stop the indexer from going quiet and calling it current
Seven review findings, all in the seam between "stopped working" and "finished".
One commit because they meet in the same three files.
A pause part way through a backfill abandoned it. The flag was cleared on entry,
the abort was swallowed as a normal stop, and resume only re-swept when the pause
outlasted an interval. A sweep is now due until one completes, and work drained
out of a queue and then not read goes back: a stale or invalidated path is a hole
in the index, not finished work.
The retention cutoff was set once at construction while purges took a fresh one,
so with a one-day window a sweep three days later deleted a row the very next
accept check re-indexed. It is refreshed from the clock before any accept
decision in a pass.
An invalidated path the index had never held was dropped unread, because it was
resolved only through rows the index already had. It now resolves the agent from
the same root table discovery reads, and what still cannot be resolved stays
queued and counted rather than vanishing.
Status told four small lies: a file count that tallied attempts and grew past the
total, `current` while work was queued or before any sweep had finished, `current`
after close, and a declined read counted as indexed because the parse returned
without throwing. It now counts what the store holds, requires all three
conditions for `current`, has a `closed` phase, and counts a file only when the
index's own cursor moved.
The cursor drop ran outside the per-path parse lane, so an overlapping sidebar
parse could store its entry in between and turn a forced whole read back into a
cache reuse. The decision moves inside the lane as a read requirement, which is
the only place that is atomic against it.
A sweep retired every indexed file it did not discover. An unmounted volume
ENOENTs its whole tree at once, so that deleted a user's searchable history for a
detached drive. A root that cannot be read, or that lists nothing where it listed
transcripts before, is degraded, and a degraded root's files are never retired
however loudly the filesystem says they are gone.
The pass loop moves to `session-search-work-loop.ts`: one task at a time, one
pending tick, one abort. It is the part with no opinion about transcripts, and
the indexer was over the line limit with it inline.
* fix(ai-vault-search): count files owed a read once, and pick the ceiling on purpose
`filesPending` summed the store's re-read set and the indexer's queue, and a
path sits in both the moment a read is declined during a pause and a caller
then invalidates the same file. One transcript read as two, with nothing to
distinguish that from two transcripts. It is a union by path now. The drop
counts stay a sum, because a drop is an event rather than a membership and
nothing retains the paths to deduplicate afterwards.
The previous commit raised this queue's cap from 2,000 to the store's 20,000 as
a side effect of trying to make one number out of two, which the double-count
shows it never was. Both queues retain a candidate per entry, so that quietly
doubled the worst-case memory of a background feature. Back to 2,000, with the
reason written down: the store's set is filled by the reader at machine speed
during a pause and needs headroom proportional to the transcripts on the disk,
while this one is filled by a cycle's budget rollover, bounded by one recent
window at a few hundred, and by `invalidate()`, where 2,000 outstanding requests
is already a malfunctioning caller.
* fix(ai-vault-search): apply the round-1 guards on the cycle path too
Two of the round-1 fixes were written on the sweep and the cycle walked around
them twenty seconds later.
The degraded-root fence now lives inside the retirement function itself rather
than at one call site, so both passes get it from one place and the cycle
cannot delete what the sweep just protected. The cycle also reads the sweep's
root counts, so it can see a tree that went to zero at all; it does not write
them back, because a recent-window discovery is not a census.
The N-to-zero alarm was single-shot: the degraded sweep's own zero became the
baseline, so the next sweep compared zero with zero and retired the tree it had
just spared. A root keeps its last healthy count until one lists it non-empty
again.
Taking a file no longer means the cursor moved. A forced whole re-read of an
unchanged file writes an identical cursor, so an invalidated file that turned
out not to have changed was never settled: owed forever, re-read whole every
interval, status pinned. It means the index now covers the file at this stat,
which is the same question the skip at the top of the pass asks, and it is the
same function.
An aborted cycle handed the store's re-read set to a queue a tenth its size,
which silently discarded the difference. What came from the store goes back to
the store, under its own bound and its own retention rule.
Whether a sweep finished is now an argument rather than a call site's position,
so an aborted one cannot latch `current` by being reported a line too early.
* fix(ai-vault-search): fence real directories, and let the alarm release
The degraded-root fence did nothing at all for OpenClaw, the one agent whose
roots are alternates for a single install. Discovery reports those as one
discovery whose rootDir is every path joined by the platform's path delimiter,
and that string is not a directory: the probe readdir'd it and got ENOENT, the
containment check never matched a file under it, and a scan issue recorded
against a real root never compared equal to it. So the agent most likely to
live on a mounted volume was the one an unmount deleted, and the degraded root
it reported was not a path anyone could act on.
Health now runs on the constituent directories, taken from the same source
table discovery reads rather than by splitting the joined string back apart,
which would be its own bug: a directory may legally contain the delimiter.
Files are attributed to the root they actually live under, so one alternate can
be unreadable while the other keeps indexing and retiring normally.
The N-to-zero alarm also never released. Carrying only counts above zero meant
a root the user legitimately emptied stayed degraded for the life of the
process, its rows never retired and the phase pinned. The rule is now explicit:
a root that cannot be listed keeps its last healthy count and stays degraded
indefinitely, while one that lists successfully and empty on two consecutive
full sweeps is believed. One sweep is not enough, because that is also what a
freshly unmounted volume looks like, and only a sweep counts: a recent-window
cycle can see a root at zero but is not a census.
The allowance still charges a forced read at the size discovery saw. That is an
under-count when a file grows mid-cycle, and it is deliberate: the budget paces
a cycle rather than accounting for it, and the error is bounded by what one
cycle's writers appended.
* fix(ai-vault-search): prove a root once held transcripts from the index, not memory
The fence was inert on the first sweep of every process. The evidence that a
root had ever held anything lived only in memory, so after a restart it was
empty, and a missing directory is what a detached volume and an agent that was
never installed both look like. Index a transcript, close, detach the volume,
open the same database: every row retired on that one sweep, with no degraded
root reported. The evidence now comes from the store, which is the thing that
actually outlives the process, through a range scan on the path key. A root
that is missing while the index holds files under it is degraded; only one the
index holds nothing under is absent.
Consecutive also has to mean consecutive. An unreadable sweep left the tally
alone rather than breaking it, so empty, unreadable, empty added up to a
deletion nobody performed. Anything that is not a successful empty listing now
resets the run.
Rows under no configured root were immortal: nothing refreshed them, nothing
retired them, nothing reported them, and searches still returned them. That
happens when a profile moves or a root is reconfigured. One rule, written at
the function: such a file is retired exactly like any other if its path answers
ENOENT, because that is proof, and otherwise the rows stay and `orphanedFiles`
reports them. An index holding content the current configuration cannot reach
is a configuration problem to surface, not a licence to delete history.
An aborted sweep no longer publishes findings it never gathered. It stops
probing on abort, so its empty degraded list would have cleared a live alarm,
and its partial view must not count toward emptying a root either. It now skips
the health pass entirely and carries the previous state forward.
* fix(ai-vault-search): an empty mountpoint is not an emptied root either
Round 4 moved the missing-root branch onto the store and left the other one on
memory. An unmount on Linux, WSL or sshfs does not remove the mountpoint: it
leaves it present and empty, so a detached volume takes the listable-but-empty
branch, and that branch armed its two-sweep grace from a count that is zero on
the first sweep of every process. Index three transcripts, close, detach: all
three retired on that one sweep, with no alarm and a phase of `current`. Both
branches now ask the store, which is the only thing that outlives the process.
Nothing re-armed a sweep when a root came back. A root absent at start is
correctly ignored, but after it returns a cycle only reads the newest N per
agent, so one file was indexed and the rest stayed unreachable for the life of
the process. A cycle that sees a root listing again where a pass judged it
absent or degraded now asks for a sweep. Only after one sweep has completed:
before that, a root with no recorded count has simply never been censused, and
treating that as a recovery would turn every early cycle into a full sweep.
`hasIndexedFilesUnder` was already bounded at a path segment; nothing pinned
it, which is why the bare-prefix mutation lived. It has a test now, on both
separators, including that a root is not held under itself.
* refactor(ai-vault-search): prove a deletion by walking to the root, not by remembering
Retirement had grown a root-health state machine: a per-root healthy count, a
two-consecutive-empty-sweeps tally, a census flag, a store query for whether the
index had ever held files under a root, and a fence every call site had to
remember to apply. Four review rounds found the same bug in four shapes, because
each shape was a new way for the machine to conclude "empty" from something that
was not.
The rule is structural now. A row retires only when a directory between the file
and its configured root lists successfully and the next component toward the
file is absent from that listing; a directory that ENOENTs is walked up, and any
other failure is unverifiable at once. The walk stops at the configured root, so
everything above it -- a home on an unmounted volume, a detached drive, a
dropped SSH mount -- is out of scope by construction rather than by memory, and
the rule reads the same on the first pass of a process as on the thousandth. The
invariants are written at the top of the module and each is a test.
One bit per root survives: a root that held transcripts on the previous pass and
holds none on this one gets a pass of grace, so a directory swapped out for a
moment cannot retire a tree. What that does not cover is stated in the module
and pinned by two tests.
degradedRoots becomes a per-pass signal with no memory: roots discovery recorded
an issue against, roots the walk could not read through, and roots that yielded
nothing and refuse to list at all.
* fix(ai-vault-search): close the loop, budget the backfill, and let a pause mean it
Five lifecycle defects, all of them cases where a call did more or less than it
says.
close() disarmed the timer and aborted the task in flight but left the queue
running, so a clear() queued a moment earlier would go on to delete the
database, open a new one and register a consumer against it, behind an indexer
whose caller had finished with it. Closing the work loop makes every queued task
a no-op.
The backfill was the one pass that read transcript bytes without a budget: a
first run over a large disk owned the process until it finished. It now spends
an allowance of its own and hands back the rest of its plan, which the passes
that follow drain without re-discovering. The allowance is separate from the
cycle's and much larger, because a first run has a backlog and steady state does
not: 128 MB a pass drains 20 GB in about 53 minutes where the cycle budget would
take about 14 hours.
A pause now stops purges and compaction too: narrowing the history window while
paused records that a purge is owed and runs it on the first pass allowed to
write. resume() no longer sweeps on its own, however long the pause was; every
read declined while paused is already in the store's re-read set, which the next
cycle drains. start() while paused arms on resume instead of queueing a pass
that returns immediately and resolves as though one had run.
A root that recovers still buys a full sweep, but at most one per recovery: it
has to be listed healthy on the pass after the one that re-armed before it can
buy another, so a root flapping every interval costs one sweep rather than one a
flap.
Smaller: clear() resets the swept flag, so an emptied index is not reported as
current before the sweep that refills it; a sweep watches only what it could not
settle rather than every path it discovered; and the progress pair is measured
against one population, so a ratio cannot exceed 100 percent.
The round-6 lifecycle matrix lives in the repository now: 11 operations against
3 unreachable-root shapes against both ways discovery reports a root, 66 cells
on four invariants.
* test(ai-vault-search): drop the tests the old retirement rule owned
Two of them asserted the same behaviour as the emptied-root tests that replaced
them, and both were named for a rule that no longer exists: retirement waiting
for a second sweep to agree, and an unmounted volume being recognised by its
root listing empty. Comments that pointed at review rounds rather than at the
behaviour go with them, and the merged-root test is named for what it covers now
that there is no root-health module for it to be about.
* fix(ai-vault-search): stop a sweep from erasing the request that arrived during it
Four round-8 findings, all in round-7 code.
A full-sweep request raised while a sweep was running was erased by the sweep it
arrived during. The flag stayed set across the await and was cleared on the way
out, so widening the history window or calling reconcile({ full: true }) part
way through a backfill left status reading `current` with the widened-in
transcripts never read. The pass takes the flag on entry now; an unfinished
sweep is what puts it back.
clear() followed by close() left the database on disk. The removal was queued on
the work loop, close() makes queued tasks no-ops, and clear()'s promise resolved
anyway -- a privacy action that reports success without doing anything. The
store, its consumer registration and its file on disk now have one owner and one
lifetime, and closing performs a removal that is still owed.
The backfill drain bounded bytes but not wall time. The pacer backs off 15
seconds a batch on a loaded host, so a pass could hold the loop for a quarter of
an hour without going near its byte budget, and since the drain runs inside the
reconcile cycle that is the recent-N-per-interval promise gone. Every read pass
now stops at one interval and hands the rest back.
A row whose path names an entry inside a container rather than a file of its own
was proven only against the container, so an entry deleted inside it could never
be retired. Such a row is now proven by the container's own enumeration, under
the same bar a directory listing has to meet: exhaustive, successful, and not
empty. Nothing in this PR can hold such a row yet -- the index pass refuses a
source whose messages the channel cannot reach, which is every OpenCode SQLite
session -- so this is the guard for the day that changes, and a test pins the
precondition.
Also: status() reports `idle` before start() rather than describing work no
timer was going to do, and reconcile() before start() is refused rather than
writing the index once and leaving it to go stale.
* test(ai-vault-search): date the widened-in transcript on the clock retention reads
The transcript meant to sit outside a 30-day window was dated against wall time
while the window is measured against the test clock, which runs a year behind
it, so the file was inside the window and the test proved nothing: it passed
with the fix reverted.
* fix(ai-vault-search): meet the rewritten store where PR 2 left it
The rebase onto the one-transaction-per-file store: re-add the cursor
predicate PR 2 dropped, read `sessions`/`messages` now the visibility views
are gone, and delete `recoveredRows` -- there is no tombstone table left for
a crashed writer to leave rows in, so the counter could only ever read zero.
* refactor(ai-vault-search): make the indexer immutable, with one queue and one bound
A configuration change is now "close it, construct a new one", so the object
has one store, one registration and one lifetime. `pause`, `resume`, `clear`,
`setHistoryDays` and `invalidate` are gone, and with them every flag that only
existed to keep a second lifetime in step: `paused`, `pausedAt`, `purgeDue`,
the deferred-purge path, the resume-sweep rules and the start-while-paused
case. Throwing the index away is close, `removeSessionSearchDatabase`, and a
new instance; widening retention is a new instance whose opening sweep admits
the older files, and narrowing is the purge that opens every full sweep.
One queue, not three. The indexer's pending queue and the sweep remainder are
deleted; the store's re-read set is the one bounded queue, already the place a
declined read lands, and `filesPending` is its size rather than a union across
queues that could count one transcript twice.
One pacer, not three. The files-and-bytes allowance and the load-average
back-off are deleted; a pass reads until its wall-clock deadline and hands the
rest back. A pass that runs out of time defers only files it would actually
have read, so a truncated pass cannot buy a whole re-read for a file the index
already covers.
The sweep cadence subsumes root recovery: a full sweep runs on start and every
`fullSweepEveryCycles` after it, so a root that comes back is picked up by the
next one instead of by a flap-bounded re-arm rule.
* test(ai-vault-search): prove close disarms the timer, not only the store
Removing `loop.close()` from `close()` failed no test: the unregister already
stopped a later scan reaching the index, so the surviving timer and the task
queued behind it were invisible. The close test now asserts the timer is gone
and that advancing the clock past it reports nothing, which is what a pass
running against a shut store would have done.
* refactor(ai-vault-search): drop the options and fields nothing reads
`retirementChecksPerCycle` had no caller in the stack, so the reconciler keeps
its own constant; the error reporter is only needed while the store and the
loop are being built, so it stops being a field.
* refactor(ai-vault-search): let indexedSources walk the table it is asked for
The per-path arm existed for `invalidate()`, which had to resolve a path the
index might never have held. Only the sweep reads this now, and it reads all
of it.
* fix(ai-vault-search): never call a half-written file current
PR 2 now reports a file a chunked read left half written with a null cursor
under the whole file's mtime and size, so the freshness check has to start at
`requiresWholeRead`: comparing only the stat calls a prefix current and leaves
it in the index for good. Two consequences, one test each. The skip check no
longer skips such a file, and the took-it check no longer reports it indexed,
so it stays owed until a read finishes it. The pass reads it whole rather than
appending, which repairs it in one pass instead of waiting for the consumer to
decline an append it was never going to take.
* fix(ai-vault-search): four ways a pass reached the wrong conclusion
Round 10, each reproduced on 6a1bcfdf49 first and each repro kept as a test.
A sweep watched only what it could not settle, which is nothing on a healthy
machine, so the cycle after a sweep had no deletion candidates and the cycle
after that no longer remembered the file. A transcript deleted in that interval
survived until the next periodic sweep, five minutes later. The sweep now seeds
the watch set with its own recency window, taken from its own discoveries
through the same class discovery selects with, so there is no second spelling
of the rule and no second walk of the trees.
A transcript the reader cannot open was recorded stale by the consumer on every
attempt and re-read every cycle for ever. Three failures at one unchanged stat
now hold a file out until that stat moves, which is the only thing that can
mean it changed. `failures`, a tally of attempts that climbed without bound,
becomes `unreadableFiles`, a gauge of files being held; a non-zero value is
degradation, because waiting will not close that gap.
`close()` part way through a pass left the pass reading a shut handle and
reported three database errors to the owner who asked for the close, and
`status()` afterwards opened it again to answer zero files. The pass stops at
the cancellation the close raises, and a closed indexer reports what it last
knew. `indexedSources` throws rather than answering with an empty list: its
caller is a sweep deciding what nothing rediscovered, and an empty answer is
the one conclusion an unreadable handle must not reach. A sweep that threw puts
its own flag back, so something is still armed to try again.
`droppedPending` was a lifetime tally under a doc that promised it meant the
queue was incomplete until the next sweep. A completed sweep now clears it, and
`bytesIndexed` resets per pass rather than per sweep, so it stops sawtoothing
every fifteen cycles.
Two indexers on one database both registered with the reader and wrote every
transcript twice. The second construction throws.
* test(ai-vault-search): prove the three conclusions a broken pass must not reach
Three round-10 mutations survived the first pass of tests, all of them the same
shape: a pass that failed still reached a verdict, and nothing checked.
The drop reset moves into the sweep itself, where a test holding the store can
overflow the queue and watch a completed sweep clear it; from the indexer the
call was unreachable without twenty thousand files.
A sweep whose held-file read throws now has a test that the failure travels
rather than being folded into an empty list, and a sweep that threw part way
has one that the flag saying a sweep is owed comes back.
* feat(ai-vault-search): put what a file still owes on the file's own row
Three additive columns on `files`, and the consumer writes them. `state` is
'current', 'due' or 'failed'; `fail_count` and `failed_mtime_ms` are what stop
an unreadable transcript being retried on every pass for ever. Schema version 4,
so a stale index rebuilds.
Every refusal now leaves its record on the row rather than in a map beside it. A
declined append is 'due': the index is behind on a span no append reaches, so
the next pass reads the file whole. A read that started and did not commit is
'failed', counted, and stamped with the stat it failed at, because a transcript
the reader cannot open fails identically every time and only a change to that
stat can mean the file itself changed. A path the file table does not name needs
no record at all: the next pass reads it because the index holds nothing for it.
Deleted with the in-memory set they served: `markStale`, `takeStale`,
`pendingFileCount`, `droppedPendingFileCount`, `forgetDroppedPending`,
`setAcceptingWrites`, `acceptsCandidate`, `STALE_PATH_LIMIT`. Added: `files()`,
`setFileState()`, `stateCounts()`, `retentionCutoff`. The retention gate moves
into `beginWrite`, because the consumer observes every read the session list
makes and not only the ones the index asked for.
The indexer rewrite that consumes this surface is the commit after; this one is
kept to the store, the schema and the consumer so PR 2's own final commits can
rebase over it.
* fix(ai-vault-search): count a failure for a file the index never held
The common unreadable transcript is one no read ever got through: a file behind
the wrong mode bits fails on its first attempt, so there is no row to count the
failure on and it would be read again on every pass for the life of the process.
The failure now inserts its own row, holding a zero cursor and no session, which
is what "the index holds nothing for this file" already looked like.
* refactor(ai-vault-search): make the store the indexer's only memory
Design v3. Every question a pass asks between passes is a row in `files`: what
is owed a read, what has failed and how often, what the index holds and
therefore what may have been deleted, what to report. Two things outlive a pass
and are not rows -- the timer, and one bit per root for the retirement walk's
grace -- and both are named in the class doc.
One loop, four steps. Discover: the only filesystem walk, every root on a sweep
and the newest N per agent on a cycle. Decide: the candidate's stat against its
row, as one pure function with its own test. Retire: the rows discovery did not
return, inside the scope it covered, through the unchanged walk. Report: a
`GROUP BY state` over the same rows.
`session-search-backfill.ts` and `session-search-reconciler.ts` become one
`session-search-pass.ts`, because the two differed only in discovery scope.
Deleted with them: the watch set redefined three times, the hold-out map, the
`sweptClean` latch, `session-search-indexing-status.ts` and every counter with a
rule about when to reset, `session-search-unreadable-files.ts`, and PR 3's
addition to `session-search-file-cursor.ts`, which the decide step replaced.
Two things the design did not anticipate, both found by its own tests. A cycle
lists the newest N per agent, so a backlog outside that window is invisible to
it and a first run would have crawled: a pass that runs out of time now asks for
a sweep, which is self-limiting because the first pass that finishes its reads
hands the interval back. And a cycle's retirement candidates are the newest
rows under the roots it listed, capped, so a deletion inside the recency window
is proven on the next cycle whenever it happened rather than waiting for a
sweep.
* test(ai-vault-search): make the retirement cap test prove its ordering
Removing the newest-first sort from a cycle's retirement scope failed nothing:
the fixture wrote its transcripts oldest first and the sweep indexed them
newest first, so the table's own row order already put the oldest last and an
unsorted slice happened to reach the same answer. The oldest file is now
indexed on its own first, which makes it the earliest row as well as the oldest
file, and the two orders disagree.
* fix(ai-vault-search): take schema version 5 for the three files columns
PR 2's final pass took version 4 when it dropped conversation_fts, and the
rebase merged both bumps into one number. The columns take 5. Two smaller
things the same rebase left behind: the stub store in the identity test still
declared the two methods the consumer no longer calls, and STALE_PATH_LIMIT
outlived the set it bounded.
This sits one commit above the columns rather than inside them, because the
rebase onto
|
||
|
Jinwoo Hong
|
f31bfa8fb7 |
Revert "feat(ai-vault-search): session search query engine over the index (#19750)" (#20023)
This reverts commit
|
||
 Jinjing
|
37603a6cf3 |
refactor(shared): derive WellKnownAgentType from TuiAgent (#19645)
The hand-written 22-member WellKnownAgentType union was a stale copy of the
launchable-agent list, 15 members behind TuiAgent: aug, autohand,
claude-agent-teams, cline, codebuff, continue, crush, goose, kilo, kimi, kiro,
mistral-vibe, openclaw, qwen-code, rovo. All 21 non-'unknown' members it did
carry were already TuiAgent members, so the union is now derived
(`TuiAgent | 'unknown'`) and cannot drift again.
'unknown' stays outside TuiAgent: it is the "no agent identified yet" sentinel,
not a launchable agent.
AgentType is structurally unchanged. `(string & {})` absorbs the union, so
AgentType was and remains `string` — this is a documentation/staleness fix with
no behaviour change and zero consumers affected (WellKnownAgentType had none
repo-wide beyond the AgentType alias itself).
tui-agent.ts is a pure type union with no imports, so this adds no cycle.
Adds a type-level coverage test that fails to compile if anyone reverts to a
hand-written list.
|
||
|
Jinwoo Hong
|
cf7408a37f |
feat(ai-vault-search): session search query engine over the index (#19750)
* feat(ai-vault-search): give the index a generation a page cursor can be fenced by
A search page is a slice of one ranked list, so a cursor only means anything
against the snapshot that produced it. The store now keeps a monotone
generation in `meta`, bumped by every mutation that can change which rows a
read returns, and starts a new one on open so a write whose bump never landed
cannot leave a cursor pointing at content that is already gone.
Schema version 2 adds the two tables the query layer needs: `messages_vocab`
(fts5vocab over messages_fts, the typo repair's whole dictionary) and
`search_log`. Both are pure additions and the index is a cache, so a version-1
file is dropped and rebuilt exactly as any other mismatch is.
* feat(ai-vault-search): plan a query the way the index tokenized it
The planner unfolds the tokenizer contract instead of asking SQLite: same
boundaries as `unicode61 tokenchars '_.-/+'`, pinned against real fts5vocab
output so a query can be planned without a round trip. It decides the route
ladder's first rung (literal shape), strips stop words from prose but never
from a literal, and fans an identifier out into its pieces for the OR fallback.
Typo repair uses the index's own vocabulary as its dictionary, so it can never
suggest a term the index does not hold. Its exact-match probe now joins
`visible_messages`: `messages_vocab` is a view over the FTS b-tree and still
lists staged and tombstoned terms, and the PR 2 read ratchet is right to
demand the join.
`splitAiVaultSearchQuery` is the index's reading of repo: / path:. It keeps
operator case, which the panel folds and cwd_key must not; a census test pins
the two parsers to the same answer about what is an operator until PR 7 moves
the panel onto this one.
* feat(ai-vault-search): narrow a search the way the sidebar keys a folder
Every caller-supplied narrowing in one place, so retrieval, the operator-only
page and the session load cannot drift apart: agents, an updated-at floor, the
retention cutoff, scope paths, and the repo: / path: operators.
Scope keying goes through PR 2's `cwdKey`, which is the sidebar's
`folderGroupKey` without its prefix, rather than the original branch's second
spelling. That drops the branch's WSL distro qualification, which PR 2 removed
on purpose, and it makes the filesystem root a key that already ends in a
separator, so the child-prefix range is built from the key rather than by
appending one; `//` sorts below every real child and would scope the root to
nothing.
Engine types land here too, under src/main and not src/shared: nothing in this
PR is a wire type, and PR 5 lifts what a caller may receive.
* feat(ai-vault-search): rank, page and answer a session search
`SessionSearchEngine.search()` over the PR 2 store: route ladder (phrase, AND,
typo repair, OR), BM25 weights per corpus, one hit per session, fork folding,
and a page.
- `scope` picks the corpus and the engine never second-guesses it.
`conversation` is user and assistant turns; `all` adds tool output and the
identifier shadow column. Switching corpus while typing is PR 7's policy; an
engine that widened on a miss would make a result impossible to reproduce
from its own request.
- Pagination is an offset into one ranked list, fenced by the index generation
and by a hash of everything that changes the ranking. A cursor from another
generation or another query is refused with a typed error rather than
silently re-run. Ranking breaks every tie by session id, because a cursor
indexes into that order and retrieval does not promise one.
- Snippets and source presence are paid for by the page, not the list. A
snippet past the per-hit ceiling is cut on a code point, never between `[[`
and its `]]`, and flagged on the hit.
- Source presence is read from the `files` table. No stat on the query path,
and no `missing`: only a proven deletion may claim one, and this read cannot
prove it.
- `SESSION_SEARCH_CANDIDATE_LIMIT_DEFAULT` is an option, not a constant, and
the result says when the limit was what cut the answer.
- The engine is the only caller of `store.warm()`, on first search.
Library only: no IPC, no settings, no Electron, nothing constructs it in
production.
* perf(ai-vault-search): measure what a query costs and what the candidate limit buys
Two corpora, because they answer different questions. The 10.5 MB / 40-session
corpus is what the scope split costs a reader: conversation is about 1.6x
faster at p50 and 3.4x at p95 than the full corpus, which is the argument for
the second FTS table being the one a keystroke can afford.
The candidate limit needs more sessions than the limit before it costs
anything, so it is swept over 2,500 one-turn transcripts with every session
matching. Limits are interleaved sample by sample: run back to back, the first
configuration pays for every page the OS cache had not seen and the ordering
alone moved p95 further than the limit did.
The doc says plainly what these numbers do not cover. They are cost, not
relevance; the MRR figures quoted beside the BM25 weights and the identifier
shadow column come from a shoot-out over real transcripts and cannot be
reproduced from this repository.
* refactor(ai-vault-search): key a page once per search, and report reachable pages honestly
The page key was hashed twice per search, once to decode the incoming cursor
and once to mint the outgoing one. The benchmark's reachable-page figure was
clamped against a constant that could never bind; it is the limit over the page
size and nothing else.
* test(ai-vault-search): pin the two engine seams nothing was holding
The warm wiring and the query-length cap were both written and neither was
observable. A spy pins that a search is what warms the store, and the cap is
pinned through the one input the planner's own term limit does not already
bound: a single enormous token, where the cut is what decides whether the term
matches the indexed one at all.
* fix(ai-vault-search): fence the generation against writers this process cannot see
The generation was cached in memory and moved only on this store's own writes,
and the bump itself was a read-then-write outside any transaction. Two handles
on one file is the normal case once PR 3 lands: the indexer writes in the
scanner child while an engine reads elsewhere. A reader would see that writer's
deletions while its own generation stood still, honour a stale cursor, and skip
a session; two writers could mint one generation for two snapshots.
`generation` now reads `meta` on every call, and the bump is a single
`ON CONFLICT DO UPDATE ... + 1` inside the transaction that makes the change.
That closes the crash hole the bump-on-open existed for, so opening a store no
longer invalidates anyone's cursor.
Only a change to what a read returns counts. Retiring a path the index never
held hides nothing, and the row deletes that drain a tombstone take away rows
that were already invisible: bumping there would refuse a cursor every 256 rows
and leave pagination unusable for as long as indexing ran.
Also drops redaction from the query log, following PR 2's decision to store
transcript content as written; the module it called no longer exists.
* fix(ai-vault-search): answer from a version-1 index, and keep a repaired literal whole
Two smaller findings.
An engine can be handed a connection to a version-1 file that another handle is
still answering from, and this PR is what first makes that reachable. It now
probes once for the two tables version 2 added and names what it cannot serve
on every result, instead of throwing at the first query that reaches for a
vocabulary that is not there. The route ladder simply skips its repair rung.
Which process may unlink and rebuild the index is PR 3b's decision and is not
solved here.
Typo repair re-planned the corrected query from scratch, and a corrected
spelling can read as prose even when what was typed was a literal:
`parseJsonn(the, data)` has the punctuation, `parsejson the data` does not, so
the re-plan dropped `the` as a stop word. The repaired query searched for less
than was asked and `repairedTerms` reported a body nobody typed. The re-plan is
now told what the original decided, because a repair changes spellings, not the
query's character.
`repairedTerms` is documented as the whole body the repaired plan ran, with the
index's own case-folded spelling for the terms it corrected.
* fix(ai-vault-search): make repo: and path: mean one thing in the list and the index
The engine said `repo:` and `path:` a second time, in SQL, and SQL cannot say
them. LIKE folds ASCII and nothing else, so `path:CAFÉ` missed `café`; the
engine searched `cwd_key` while the panel searches the working directory and
the transcript path, so `path:jsonl` matched every session in the panel and
none in the index; and the engine compared one path segment where the panel
compares the last two, so `repo:orca/session-search` missed. All four
reproduce in both directions.
So there is one definition now, not two that resemble each other.
`matchesAiVaultQueryOperators` moves into the shared filter module beside the
panel that already owned the semantics, the panel calls it, and the engine
applies it over the rows it retrieved. SQL keeps only what it can express
exactly: the `cwd_key` prefix range for `scopePaths`.
`parseVaultQuery` now parses through `splitAiVaultSearchQuery`, so one parser
decides what an operator is. Its existing tests pass unchanged; four degenerate
shapes do answer differently and are pinned as decisions rather than left to be
discovered.
Two consequences worth stating. The operators are conjunctive now, because that
is what the panel has always done, where the engine had been ORing within a
key. And the operator-only page walks newest sessions in bounded pages applying
the predicate, rather than taking one cut of the newest N and filtering it,
which would have answered `repo:x` with nothing on a busy index.
* docs(ai-vault-search): re-measure the query benchmark after the operator change
repo: and path: moved out of SQL, so the operator-only row measures something
different now and the note that it is a range seek was wrong. The rest of the
table is re-measured on an idle machine: the previous run's p95 column was
mostly contention, which is why conversation looked 3.4x faster at p95 rather
than the 1.7x it actually is.
Adds what this PR does not settle: which process may unlink and rebuild the
index is PR 3b's, and PR 4 is only the first thing that makes reading it
reachable.
* fix(ai-vault-search): stop reporting a search that gave up as a search that finished
The operator-only walk stops at a scan ceiling as well as at a full candidate
set, and only the first of those reached the result. A query whose one match
sat past the ceiling came back with no hits and truncated.candidates false,
which is the engine claiming there is nothing to find when what happened is
that it stopped looking. Retrieval now says why it stopped, because it is the
only layer that knows, and the count it used to return could not distinguish
the two cases.
The cursor fence stays as it is: any published read moves the generation, so an
outstanding cursor is refused, and that is what F11 asked for. What was wrong
was the claim next to the row-delete skip that pagination stays usable through
indexing. It does not, and the engine now says so. The rejection carries the
generation the cursor was minted in and the one the index is at, so a caller
can tell a moved index from a bad cursor and re-issue page one without showing
anyone an error.
The capability probe was nearly dead code, since every store opens through a
function that rebuilds a stale file. It is not dead, because two handles can be
open on one file, so the claim is corrected rather than the probe deleted. It
now runs per search: a verdict cached in the constructor is wrong in both
directions once another handle rebuilds the index.
Also says why the row-delete loop may skip the bump: those messages keep
batch_id NULL and stay in visible_messages, so what makes them unreachable is
their session's tombstone, and the read ratchet is what keeps every reader
joining the view that applies it.
* fix(ai-vault-search): restore the panel's reading of a quote that does not end a word
Unifying the two parsers changed panel behaviour on nine of twenty probed
shapes, not the three previously pinned. Six of the nine were regressions, all
from one rule: the shared parser refused a quoted span whose closing quote was
not followed by a space, so `"a b"c` and `repo:"a"b` became single terms
carrying their own quote characters, which match nothing.
The rule was justified as what stops the apostrophes in `it's a repo:orca
thing's` from swallowing the operator between them. It is not: a span only ever
opens at a token start, and the quote in `it's` is not at one. Dropping the
rule restores all six shapes to what the panel has always done and leaves that
protection intact.
Three changes remain and are kept because the old answer was worse in each: an
operator with an empty quoted value is dropped rather than filtering on `""`
and silently emptying the list, and a bare pair of quotes reads as an empty
term rather than as the two characters. Each is pinned with a test that says
which behaviour it is and why.
* fix(ai-vault-search): trim operator values, and report a query the engine had to cut
Three lows.
`repo:" "` survived as a term and matched no label, silently emptying the
list, which is the exact defect the empty-value drop exists to prevent wearing
different clothes. Operator values are trimmed, and a whitespace-only one drops
like an empty one. The substring matcher's copy of a free-text term is trimmed
too, so `" "` reads as the empty term already does; the span kept for FTS is
still the query verbatim.
Two caps upstream of retrieval fired silently: the planner searches at most 48
terms, and the engine cuts the query at 512 characters. A 56-term query whose
only match was the 56th came back with no hits and nothing truncated, which
claims there is nothing to find. `truncated.query` now says when either fired,
alongside the candidate and snippet flags that already did.
Every cursor refusal now carries the generation the index is at, which the
engine knows before it looks at the cursor, and the generation the cursor
claimed wherever that survived parsing. The doc says exactly when each is
present instead of leaving absence unexplained.
* refactor(ai-vault-search): read the tables the simplified index writes, and own the fence
PR 2 deleted the visibility views, the staging tables and the store's
generation, so this reads `sessions` and `messages` directly and carries the
three schema objects only a query needs — the vocabulary, the query log, and
the triggers that move the generation — as its own extension over the store's
schema.
The fence is now three triggers on `files`, because every transaction the store
opens that can change what a search returns writes that table and nothing else
does; retention's orphan drain is the one write path that touches neither, and
it is the one that must not bump. The triggers live in the file, so a writer in
another process moves the generation without knowing a reader exists.
A message row can now outlive its session row until the drain reaches it, so
the snippet read joins `sessions` and the typo repair asks for a live posting
instead of trusting the vocabulary's document count.
The engine takes a connection rather than a store: PR 2's store keeps its
connection private, and which process may open or rebuild the index file is
PR 3b's decision, not a query engine's.
* perf(ai-vault-search): price the second FTS table, and re-measure without warmup
Open decision 3. A column filter over `messages_fts` returns the identical
rowid set as `conversation_fts` — checked here per query rather than assumed —
so the table exists for latency alone. On a 105 MB corpus at both ends of the
tool-output band, the column-filtered form costs 1.16-1.42x at p95, against a
bar of 2x, so the recommendation is to delete it.
The shoot-out writes its own corpus because the answer turns on the one
property the shared generator fixes: how much of a transcript is tool output.
Half the tokens in that output are words the conversation also uses, which is
deliberately generous to the table under question.
The doc records the number that argues the other way. PR 2 priced the table at
about a quarter of the index on a corpus whose tool output is 56% of its
message text; on a tool-heavy one it is 6.7-11%, because `messages_fts` grows
with the tool text and the second table does not.
Page warmup is not re-added. The measurement behind it was on a 4 GB index,
removing it moves this corpus by less than the run-to-run spread, and a
cancellable background pass needs a lifecycle a query library does not have.
* test(ai-vault-search): pin that an append moves the generation a cursor is fenced by
* refactor(ai-vault-search): answer the conversation scope with a column filter
PR 2 deleted `conversation_fts` on the strength of this PR's shoot-out, so the
scope is a column filter over the one FTS table now. `ftsTableFor` is gone; a
scope is a pair of `scopedExpression` and `scopedWeights`, and the table name
no longer travels through the engine, the snippet builder or a hit.
The filter is parenthesised, and that is the whole of it: `{cols}: (a AND b)`
binds both terms, while `{cols}: a AND b` binds only the first and searches
tool output for the rest. A test drives an AND whose second term lives only in
tool output through both scopes.
The snippet keeps one guard, not two. Its column list and its expression were
each hiding the other's mistakes — a tool-only row was unreachable through
either — so the list is the same four columns for every scope and the scoped
expression is what makes a conversation snippet impossible to draw out of tool
output. Dropping it now leaks that row, which a test catches.
One behaviour the deleted table did not have, pinned rather than wished away:
bm25 normalises by the whole row's length and has no per-column length, so two
rows with identical prose score differently when one also holds tool output.
The rowid set is unchanged; the order within it can move.
Re-measured on the shipping schema. The conversation scope is 1.2-1.4x faster
than `all` at every rung, and the index is 57 MB rather than about 150 MB at
93% tool output, because a tool row is now capped at 3,072 characters.
* fix(ai-vault-search): repair a spelling inside the scope that will answer it
Typo repair read `messages_vocab` and probed `messages_fts` with no column
filter, so tool output decided whether a conversation-scoped query was
repaired, in both directions. A tool row carrying the misspelling made the
query look correctly spelled and suppressed the repair; a tool row carrying a
rare word became the suggestion, naming in `repairedTerms` a string from a
column the scope will never show. Both reproduced against a control index that
differs by exactly that one row.
The vocabulary proposes and a scoped count disposes. fts5vocab is per table and
cannot be column-filtered, so every decision that reaches the plan — already
spelled right, eligible, and which of two equally close candidates wins — now
comes from a `messages_fts MATCH` under the same filter retrieval uses, joined
to `sessions`.
That also takes the vocabulary's `doc` out of the ranking, which is the half of
the drain defect that belongs here: `doc` counts rows whose session a purge has
already cut loose, so reclaiming them changed which word a query was repaired
to. Candidates are ordered by term now, because the ordering decides which of
them survive the scan limit, and ties on similarity go to the more common word
counted live rather than to the vocabulary's number.
The cost is one bounded count per candidate examined, at most eight per prefix,
and only for a term the scope has no posting for at all.
* fix(ai-vault-search): fence the rows a purge reclaims after it cuts a session loose
Retention's second half deletes from `messages` alone and touched neither
`files` nor `sessions`, so it moved no generation. The argument was that those
rows answer nothing, which was true of retrieval and not of the engine: the
typo repair's dictionary is a view over the FTS b-tree and listed them, so a
drain running between two pages swapped the repair under a cursor that was
still honoured, and a search that had answered stopped answering.
The commit before this one fixes that at its source by counting live rows. It
does not make the drain provably inert — the vocabulary still decides which
candidates survive its scan limit, and reclaiming a term's last row moves where
that limit cuts — so the fence is what covers the rest.
A fourth trigger, on `messages`, with a `WHEN` clause that is the whole reason
it is affordable: a replace and a `removeFile` delete a session's rows while
its `sessions` row still stands, so neither fires, and both already bump
through `files`. Only the drain deletes a row whose session is gone.
The price is named rather than avoided: a cursor outstanding while a purge runs
is now refused once per batch, which `SessionSearchCursorError` reports as
`stale-generation` so a caller re-issues page one. The test that pinned the old
contract is replaced by one for the new one, and by one proving a replace still
does not fire it.
* fix(ai-vault-search): tell a highlight from a transcript that contains brackets
The snippet builder asked each of a row's four columns for a marked snippet and
showed the first whose text contained `[[`. Transcripts contain `[[`: a bash
`if [[ -f … ]]`, numpy's `[[1, 2], [3, 4]]`. A row matching only in tool output
was shown its user turn instead, with nothing highlighted in it, and the
any-column fallback an identifier-only match depends on was unreachable behind
the same collision.
Whether a column matched is now the difference between two renderings of the
same text: `snippet(…, MARK, MARK, …)` beside `snippet(…, '', '', …)`. Content
cannot forge a difference between those two, because it is the same content
either way.
The marks FTS5 inserts are private-use code points, rewritten to the public
`[[` and `]]` once, at the end. That is for the other decision that has to tell
a mark from content: the truncation refuses to cut between an open mark and its
close, and a transcript's own bracket used to move that cut.
* fix(ai-vault-search): cut a query on a code point and bind ids in batches
Two small ones from the review's not-routed list.
`query.slice(0, 512)` can land between the halves of a surrogate pair, leaving
a lone half that matches nothing and that a caller cannot echo back. The reader
already has `sliceAtCodeUnitLimit` for exactly this.
`loadSessions` bound one parameter per candidate id in a single statement. The
list is as long as the candidate limit, the tuning doc invites a host to raise
that limit, and SQLite's `SQLITE_MAX_VARIABLE_NUMBER` is 999 on builds older
than 3.32 — so one settings change away from `too many SQL variables`. Read in
batches of 500, leaving room for the filter's own bound values.
* docs(ai-vault-search): price the repair rung, and record what is left open
Typo repair is the one rung whose cost tracks the vocabulary rather than the
result, and it only runs for a term the scope has no posting for. Measured over
1.6 M distinct terms: 10 ms for one unknown term, 387 ms for a 480-character
query of thirty-nine of them.
The scoped-count fix made that cheaper rather than dearer, from 737 ms, because
ordering the vocabulary scan by term drops the sort `doc DESC` needed and the
counts it adds are at most eight bounded probes per prefix. A cap on unknown
terms per query is a follow-up in the split plan, with the five other items the
final review raised and did not route.
* test(ai-vault-search): make each snippet mark mechanism answer for itself
Two mechanisms landed together and hid each other: choosing a column by
comparing a marked rendering against an unmarked one, and marking with
private-use code points instead of `[[`. Either alone fixed the bracket repro,
so neither had a mutation against it — the same masking the snippet's two
column guards had a round ago.
They do different jobs, so both stay and each gets the test that needs it. A
transcript holding a private-use code point of its own is what the comparison
is for; agent output carries Nerd Font glyphs from that block. A snippet past
the character ceiling with a bracket after its last real mark is what the
private-use marks are for, because the truncation has to find that mark by
searching the text.
The two one-line fixes get honest framing rather than a mutation neither can
have. A lone surrogate is not a token character, so the planner drops it either
way and the safe cut is hygiene. And no SQLite this stack runs refuses 1,100
bound ids — 32,766 has been the floor since 3.32 — so the batch is about
owning the ceiling here rather than rescuing a reachable failure.
* refactor(ai-vault-search): keep the scope's expression with the other expressions
Making the typo repair ask its questions in the search's own scope put an
import from retrieval into it, and retrieval already owns the repair — a cycle
the native audit catches. `scopedExpression` belongs beside `phraseExpression`,
`andExpression` and `orExpression` anyway: it builds a MATCH expression, and
two callers now need it. The bm25 weights stay in retrieval, where the SQL that
uses them is.
* docs(ai-vault-search): say which delete paths fire the orphan-reclaim trigger after a replace cuts loose
|
||
|
8acce092ef |
Hide desktop theme imports from paired web clients (#20015)
* Hide Ghostty import from paired web clients - Ghostty import now respects showDesktopOnlySettings, matching Warp behavior - Consolidates desktop-only theme imports under a single showDesktopThemeImports flag - Adds showGhosttyImport option to control visibility across settings UI and search * Hide desktop theme imports from web clients Consolidate Warp and Ghostty import visibility behind a single `showDesktopThemeImports` flag. These theme import flows are desktop-only and should not appear on paired web clients. --------- Co-authored-by: m4air <m4air@m4airs-MacBook-Air.local> Co-authored-by: m4air <m4air@Mac.localdomain> |
||
|
Brennan BensonandMerge Sim
|
2ffac471bd |
fix(workspaces): seed shells only for blank selection (#19940)
* fix(workspaces): seed shells only for blank selection * fix(workspaces): create runtime-owned launch surfaces * fix(workspaces): report runtime surface failures --------- Co-authored-by: Merge Sim <sim@local> |
||
|
Jinwoo Hong
|
d33354cfd2 |
feat(mobile): receive native push notifications from paired desktops (#19951)
* feat(mobile): deliver native push notifications from paired desktops * fix(mobile): retry push capability probes * fix(mobile): cancel retired push capability probes * fix(mobile): ignore stale push reconciliations * fix(mobile): type capability probe at its boundary * fix(notifications): route mobile push taps to the originating pane * Require explicit mobile push-service consent on upgrade |
||
|
Brennan BensonandMerge Sim
|
1798786d4e |
perf(native-chat): mount only the transcript rows near the viewport (#19869)
* refactor(native-chat): share one row-content derivation between row and list Windowing needs the list and the row to agree on which messages draw nothing: a row the list counts but the row declines to render would reserve estimated height for an empty slot. Extracts the block derivation out of NativeChatMessageRow into a module cached on the block array, so a streaming turn pays for it once per revision rather than once per consumer. * refactor(native-chat): keep an opened tool run open past its row's lifetime A tool run, tool line or diff card the reader opened is state they created, but it lives in the component's own `useState`. That is fine while every row is mounted forever. It stops being fine the moment rows can be unmounted: the run silently re-collapses behind the reader's back. Rows now read their disclosure from a transcript-level map when one is provided and fall back to their own state when they are rendered standalone. The controls that re-sync a run — the toolbar's expand-all, a turn's disclosure, a diff reveal — are folded into the key the choice is remembered under, so a control flip reads as "nothing recorded yet" and the new default stands without a mid-render write to a map an ancestor owns. `ToolLine` moves to its own file; the run was over the line cap with it. * perf(native-chat): mount only the transcript rows near the viewport A settled transcript mounts every row it has ever loaded, so the cost of opening a conversation grows with its length even though only a screenful is legible. Rows near the viewport are now the only ones in the document; the rest are reserved as estimated height and measured when they arrive. Four things had to change for that to be safe: - `zoom` moves from the transcript column onto the scroll container. Item measurements are in the zoomed content's pixels while `scrollTop` is not, so with the two split across the boundary the window's arithmetic was off by exactly the font scale — correct at the top of a transcript and blank deep inside it. The column's padding moves to a new inner element to keep the layout it had. This does mean the scrollbar itself zooms with the text. - The three siblings that made up a row — the message, the turn status, the turn's diff rollup — move into one wrapper that carries the spacing they used to take from the column. The spacing between rows is the window's `gap`, never the height estimate, which would otherwise be counted twice. - Messages that draw nothing no longer take a slot. Counted but undrawn, each one would reserve estimated height for a row that never appears. - Paging in older history is driven by scroll events alone. Every row that resolves its real height moves the content and re-fires the size observers, so the old "am I near the top?" test would have asked for another page once per measurement. It now also requires the view to have moved upwards and requires new items since the last request. Anchoring is the virtualizer's: `anchorTo: 'end'` re-resolves the row at the current offset across a count change, which replaces the hand-rolled prepend anchor, and `followOnAppend` keeps a reader at the bottom pinned there. The document-level bottom pin stays, because the typing indicator, the activity line and the column's end padding all live past the last row. Revealing a diff from a turn rollup can target a row that isn't mounted, so that row is pinned into the window and the card still reports its own position — a turn that touched four files lands on the one that was asked for. * fix(native-chat): let a pinned row reach the mounted window Two faults the windowing tests turned up, plus the handles they needed. The virtualizer memoizes its mounted index list on the range extractor's identity. Holding that identity stable — which is right for the measurement memo, and was the reason it was written that way — meant a row pinned after the fact was never picked up: revealing a diff in a row the window had left behind pointed at a row that stayed unmounted. The extractor now changes identity with the pinned set, which is not a dependency of the measurement memo, so nothing expensive is rebuilt. The offset a row sits at is read off the `offsetParent` chain, with a rect-based fallback for the case where there is none. Using that fallback for the window's own scroll margin was wrong in kind: with no layout to measure, it returns the scroll position itself, so the margin tracked the offset and the window sat at the top of the transcript wherever the reader scrolled. The margin now takes the offset chain or nothing; the fallback stays where it belongs, on the reveal. The scroll root and the window's spacer are named, so measurement can find the scroll root without depending on which utility class makes it scroll, and so a test can tell a window from a whole transcript. * test(native-chat): cover the windowed transcript, and prove the window engaged The integration harness stubs `offsetHeight` — on the scroll root and on every row — because that is what the virtualizer measures with, and a DOM without layout answers zero to all of it. Rows report the height their own estimate predicted, which keeps the reserved totals exact no matter which rows have been mounted long enough to be measured. Every case reads the window through one helper that refuses to pass when there is no window. Without that, raising the usability gate would send all of them down the whole-transcript path, where "fewer rows mounted than messages" is false but every other assertion still holds — and they would go on reporting green while covering nothing. Reserved height is asserted as an exact total rather than "greater than zero", which a degenerate empty window also satisfies, and the mounted range is asserted to bracket the offset rather than merely to be smaller than the transcript. Covered: the window mounts a subset and moves with the reader; the newest row and a reveal's target stay mounted from outside it; an opened tool run is still open when its row comes back; a message that draws nothing takes no slot; and the scroll root with no usable height still renders every row as a direct child of the transcript column. What the environment cannot show is stated where it matters rather than faked: its ResizeObserver never fires and a scroll assignment emits no event, so measurement settling, the bottom pin under a streaming turn, prepend anchoring and smooth scrolling are covered as pure decisions — height estimation, the pinned set, range extraction, and whether a position should page in older history — and left to a real renderer as behaviour. * docs(native-chat): say that one offset path does read rects * test(native-chat): pin the window against a row that grows in place Whole-message appends were covered; a row being replaced by a taller version of itself — what a streaming reply is — was not. The existing windowing harness gains two things it needs to see that: a scroll root with a real document (a height, a viewport, and a scrollTop that clamps), and a resize observer that delivers when a target's height actually changed, since happy-dom's never fires and nothing re-measures without it. Frame by frame, while one row grows from 24px to 6358px: the view stays 0px from the bottom, the row stays mounted, and the reserved total tracks the measurement rather than the estimate. A reader who scrolls up mid growth keeps the exact offset they chose for the rest of it. * test(native-chat): guard history prepend anchoring * test(native-chat): strengthen prepend anchor contract * fix(native-chat): preserve provider tool call identity * fix(native-chat): harden transcript windowing lifecycle * test(native-chat): install virtualizer viewport for turn timing * fix(native-chat): reject blank tool call identities --------- Co-authored-by: Merge Sim <sim@local> |
||
|
OrcaWinandm4air
|
6c1580aeba |
i18n: Make file reveal labels translatable (#20010)
Convert hardcoded "Reveal in Finder", "Reveal in File Explorer", and "Open Containing Folder" labels to use i18n.translate() in three menu components. Add corresponding English locale entries so these platform-specific labels are now part of the translation system instead of untranslated strings. Co-authored-by: m4air <m4air@Mac.localdomain> |
||
|
Neil
|
3b99a59ea7 |
perf(terminal): stop spending reveal-restore frames on panes that replay nothing (#19972)
The hidden-output restore queue drains one entry per 16 ms frame. An entry whose pane has since gone hidden, been disposed, or had its restore superseded hits a guard and returns without replaying anything — but it still consumed the frame, pushing the next on-screen pane back 16 ms per dead entry. The scheduled callback now reports whether it started a replay, and the drain walks past entries that report false within the same tick. Order is unchanged (strict FIFO) and the one-real-replay-per-frame pacing is unchanged; only the no-op entries stop costing a frame. |
||
|
Brennan BensonandMerge Sim
|
9b83f976f9 |
feat(native-chat): describe slash commands from the provider's own report (#19928)
* feat(native-chat): describe slash commands from the provider's own report The Claude session reports a description and argument hint for every command it can run, but the catalog kept only the name, so the `/` picker described the handful of commands our curated map covers and left the rest — `/goal` included — with a blank row. Carry `description`/`argumentHint` through the catalog and the session wire (both optional, so mixed-version hosts are unaffected), and let a reported description win over the curated one, which stays as the fallback for the name-only report shape. The curated maps are untouched, so structured dispatch still claims exactly the commands it claimed before. * feat(native-chat): show the reported argument hint in the slash picker `argumentHint` was carried to the renderer but nothing read it. Show it beside the command token — `/goal <objective>` over the description — so a row says how the command is invoked, not just what it does. It sits at the row's existing 11px muted tier, subordinate to the description, and truncates in a min-width-0 flex row; the picker also caps the hint at 80 characters, so a provider cannot swamp the row. * fix(native-chat): normalize slash command descriptors consistently --------- Co-authored-by: Merge Sim <sim@local> |
||
|
Jinwoo Hong
|
c84007c541 | feat(rpc): generate a shared params catalog from the host registry, gated on parse parity (#19961) | ||
|
Neil
|
cdf41df37c |
perf(terminal): stop rebuilding per-workspace collections on a worktree switch (#19975)
Two allocations scale with the workspace count and are rebuilt on inputs that cannot change their result. `workspaceSurfaces` took `renderedActiveWorktreeId` as a memo dep, but `projectWorkspaceSurfaces` reads that id only behind a truthy `activeWorkspaceResolvedHostId` (the folder-collision tie-break), which is null unless the active workspace is itself a folder workspace. Every git-worktree switch therefore re-projected every surface and re-derived the id array to reach an identical answer. Gate the id on the host so the memo holds. The parked-watcher sync built a fresh empty `Set` for every workspace surface, even though only a mounted workspace can park a tab and the sync only reads the set. Share one empty instance for the rest. Both keep every effect firing on exactly the inputs it fired on before. |
||
|
Brennan BensonandMerge Sim
|
ecd7b19ad4 |
fix(native-chat): pass agent-implemented slash commands through to the agent (#19929)
* fix(native-chat): pass agent-implemented slash commands through to the agent Claim what the host implements; pass through what the agent implements. Claude's harness expands a slash command out of the message text, so the host claimed catalog commands it had no way to run and answered "/init is not available in chat sessions" for commands Claude does run. Codex's app-server has no slash parser at all, so its catalog stays claimed — except /goal, which the model carries out through its own goal tools. * fix(native-chat): offer the agent-run commands in the structured picker Codex reports no command catalog, so its structured `/` menu is the host fallback -- which listed only the host's own commands and hid `/goal`, the one command the model itself acts on. The picker now appends the profile's text-driven commands, described from the curated catalog, so a command that passes through is discoverable and not merely typable. The menu invariant holds either way: a pick is answered by the host or run by the agent, never refused with "not available in chat sessions". * fix mobile structured command reconciliation * fix(mobile): keep native chat controller within lint budget * fix mobile controller lint budget --------- Co-authored-by: Merge Sim <sim@local> |
||
|
Jinwoo Hong
|
2c984bcdaa |
feat(ai-vault-search): session search index as a transcript reader consumer (#19687)
* feat(ai-vault-search): add the session search index schema and row modules
The FTS5 index that PR 2 folds the transcript reader's message stream into:
two FTS tables behind visibility views, the identifier shadow column, the
resumable fork digest, redaction at the row-insert choke point, and the
bounded compaction, warm-up and retention lanes.
Schema version starts at 1: this branch drops the query log and the fts5vocab
table, which the query engine reintroduces with its own bump.
* feat(ai-vault-search): stage index writes and publish them atomically
The writer stages a read's rows against an unpublished session row and a batch
id, then flips the whole read visible in one transaction: both FTS tables are
written together per row, and publish nulls the batch pointer before dropping
the batch so a recycled rowid can never name a later in-flight write.
Buffering replaces the branch's streamed producer. The transcript reader pushes
messages synchronously, so a staged write cannot make it wait; rows are held to
128 of them or 256 KB and then flushed in one transaction, which bounds both the
retained bytes and one stall.
The store owns the database and the set of files the index is behind on. It
carries no query, coverage or scheduling: draining that set is the indexer's.
* feat(ai-vault-search): register the index as a transcript reader consumer
The index keeps its own per-file cursor in the `files` table and never consults
the parse cache; the branch's AsyncLocalStorage capture scope is gone.
Three refusals, each leaving the cursor where it was and recording the file for
a later whole re-read: an append whose predecessor offset is not this index's
own cursor (or whose dev/ino changed) is declined in `beginRead`, a staging
failure ends the read's rows without throwing back at the reader, and an
`incomplete` outcome never publishes. A null session drops the file's rows and
still advances the cursor, because the file was read through.
Nothing in production registers it: the indexer decides when the index is live.
* test(ai-vault-search): measure the index's disk and latency cost
The write benchmark indexes a synthetic corpus through the real reader and
reports rows/s, per-table bytes per transcript MB, write amplification and
rebuild time; the retention benchmark compares the batched purge against a
whole-file delete. The corpus generator ships with them, so the numbers are
reproducible on any host and no real transcript is ever read.
* fix(ai-vault-search): decline a source the message channel cannot reach
An OpenCode SQLite candidate decodes inside a worker, so the reader reports
every read of it as incomplete. Staging a batch first meant one staging session
and one tombstone per scan of every such session, forever. Declining in
`beginRead` also keeps it out of the re-read set, because no re-read helps: the
index cannot cover that source until its capture path lands.
* fix(ai-vault-search): redact a message before it is cut into rows
A credential is a shape, and a shape split across two chunks matches neither
half: an 8000-char boundary landing inside a PEM block or a JWT indexed the key
material in full, identifier shadow terms included. Redaction moves ahead of
chunking, and the row type is branded so only `searchMessageRows` can mint a row
an insert site accepts.
`upsertFile` stops erasing dev/ino when a candidate carries none. A host that
cannot stat identity (SSH, WSL, a degraded Windows stat) was overwriting a
proven identity with NULL, which made the next rename-replace of that path
undetectable.
* test(ai-vault-search): pin each index guard on its own
Three guards were each shadowed by another, so mutating any one of them left
every test green. Driving the store directly separates the writer's own
predecessor-offset check from the consumer's cursor check, and a stubbed store
proves `beginRead` refuses before the writer is ever asked.
`publishable` gets the case it never had: a second read of the same path
publishes first, and the stale stage is tombstoned rather than resurrected. It
covers the overlap the parse file lane normally prevents, so the invalidation
slot being per-path is not load-bearing on its own.
Two ratchets: every provider fingerprint needs a fixture that reaches past
`SECRET_ANCHOR`, and every FTS read in the index modules must subtract staged
rows through a visibility view.
* fix(ai-vault-search): rebuild an index that cannot be trusted, and index retention
Opening self-heals. A file too torn to open, or one whose recovery fails, is
unlinked with its sidecars and reopened once; a second failure throws. Before
this a torn index broke open permanently, even though the index is a cache over
the transcripts and throwing it away costs only a re-scan.
A `meta` table with no readable version row is now `stale`, not `fresh`. Seeding
the current version over it would have kept whatever rows the old schema left.
`fresh` means no `meta` table at all, and no longer triggers a rebuild: the
first open of a new profile was doing create-remove-create.
Recovery moves inside open, because retiring a batch that outlived its writer is
part of opening the index rather than of using it. Its append case gets the test
it never had: 200 rows staged onto a live session and abandoned leave the
published generation intact and nothing else.
Also an index on files(mtime_ms), so the retention pass seeks the expiring end
of the list instead of scanning and sorting it, a pragma read-back test, and a
sessions.file_path comment that says what is actually true of OpenCode.
* fix(ai-vault-search): key a session's cwd the way the sidebar already does
`cwd_key` had its own normalizer, which qualified a WSL cwd with its distro:
`/home/me/repo` was stored as `//wsl/ubuntu/home/me/repo` while the sidebar's
`folderGroupKey` stored `/home/me/repo`. Any later join between an indexed hit
and a sidebar group would have returned nothing for exactly the WSL users the
qualification was meant to help. The key is now the shared normalizer verbatim,
pinned against `folderGroupKey` for POSIX, Windows drive, /mnt/c, both WSL UNC
aliases, a Linux path, and the root.
The collision it guarded against is real: two distros both spell
`/home/me/repo`. It is also a collision every SSH host has, neither key
qualifies for SSH, and the honest fix is a column naming the execution host
rather than a path key that only some hosts spell differently.
`/` now keys as `/` instead of the empty string. An empty key cannot be told
apart from a session with no cwd, and the scope filter builds its child prefix
as `key + '/'`, which would have been `//`.
* test(ai-vault-search): widen the FTS visibility ratchet past whole literals
Concatenating or interpolating a table name hid it from the per-literal scan,
so `'SELECT rowid FROM ' + table` passed. The unit is now the file, the scan
covers src/main and src/relay rather than one directory, and both bypass shapes
are checker unit tests. Exemptions carry a reason and the census fails on one
that has stopped being needed, which is how the schema module lost its.
* fix(ai-vault-search): keep a failed rebuild's real cause, and read per statement
Closing the stale handle before the unlink left `db` dangling: if the unlink or
the reopen threw, the failure path closed it a second time and node:sqlite's
ERR_INVALID_STATE replaced the real cause. Nothing classifies that as worth
retrying, so an index a virus scanner or a second Orca was holding would never
rebuild. The handle is nulled while none is open.
The visibility ratchet moves from the file to the statement. A file is far too
coarse for what it exists to guard: the query module will name both views
somewhere, and that whitelisted every raw read in it. Statements come from
literals and their concatenation chains, split on `;`, and a table name
assembled at runtime counts as a read in any file that names an FTS table. The
roots now cover cli, shared and preload as well.
Half a recorded identity is now stated to be no identity. `remote-session-file-stat`
spreads dev and ino independently and the COALESCE preserves whichever half a
host could prove, so one number cannot tell a rename-replace from a same-file
re-read; comparing it would decline healthy resumes on a coincidence.
`discard` clearing the staging slot gets a test: without it the map grew one
entry per path for the store's life.
* fix(ai-vault-search): give a behind consumer a way to get the read it needs
`markStale` promised a whole re-read that nothing could deliver. The reader
picks append or replace from the session list's resume point, so with an empty
index and a warm parse cache every read arrives as `append`, the consumer
declines every one, and nothing is ever indexed. That is the state on first
enablement inside a running app.
`requestWholeTranscriptRead` in the reader drops that path's resume point, which
is the one lever that changes the next read's mode, and it lives with the cache
that owns it rather than with the consumer that wants it. `takeStale` documents
that its paths need it before a scan is re-dispatched.
Pausing no longer empties the re-read set. `acceptsCandidate` answers whether a
write may start now and was being used for both jobs, so reconfiguring retention
while paused dropped every entry and a declined read during a pause was never
recorded at all. Retention alone prunes; a paused decline is recorded, bounded,
with the oldest dropped and the drops counted, because a set that silently
forgets is worse than one that says it is incomplete.
A read that decodes no session now tombstones its staged rows before the store
schedules cleanup. The cleanup lane reads the tombstone table the moment it is
scheduled, so the old order left that batch on disk until some later write, and
for the last read before a shutdown that is never.
* fix(ai-vault-search): create the directory the index lives in
Nothing made `<userData>/ai-vault-search/`, and SQLite's failure for a missing
parent is `unable to open database file`, which is correctly not classified as
corruption — so the retry never fired and the feature stranded on any profile
that had never held an index.
* refactor(ai-vault-search): store transcript content as written
Decision: the index does not redact. A secret in a transcript is already
plaintext under the user's home directory and is treated as compromised, so the
index is a second copy of content the user already holds, not a new exposure.
The reference agent-session-search products do not redact either. What a snippet
may carry once it leaves this machine is a transport policy and belongs where
the wire is, not in the write path.
Removes the redaction module and its census, the branded row type that existed
to prove redaction had run before an insert, and the two chunk-boundary tests.
`insertSearchMessage` takes a plain chunk again and identifier shadow terms come
off the raw text. `src/main/observability/redactor.ts` goes back to what main
has, so this PR no longer touches it at all.
Chunking and the fork digest are unchanged; the digest always folded raw message
text, ahead of chunking, so its tests hold as written.
Cost: redaction was 14 ms per 10.5 MB of transcript, about 3% of a rebuild and
below the benchmark's run-to-run spread, so the write numbers are unchanged at
roughly 22-25k rows/s and 22-27 MB/s.
* fix(ai-vault-search): hide a tombstoned session's messages, not only its row
`visible_sessions` already subtracted session-keyed tombstones; the message
half filtered on the batch pointer alone. A published row outlives its session
row until the cleanup lane reaches it, so between a `replace` publish and that
drain both generations answered, and a removed file kept answering after its
session was gone.
Both views now subtract the same set, over a partial index so neither read
scans the tombstone table.
* fix(ai-vault-search): drain the rows a read that never published staged
Only `writePublished` and `removeFile` scheduled the cleanup lane. A read that
staged rows and then declined to publish them tombstoned its batch and told
nobody, so 256 rows of a 300-message incomplete read sat in `messages` and both
FTS tables until some unrelated write happened to schedule a pass. Open-time
recovery had the same hole: it wrote the tombstones and drained none.
The abandoned branch now schedules the drain the published branch already got,
and opening schedules one pass for what recovery just tombstoned. Recovery no
longer adds a batch tombstone when the session-keyed one already covers those
rows, which it did once more on every reopen.
* fix(ai-vault-search): continue the cursor of a file that decoded no session
A read that decodes no session — an excluded Codex worker transcript — advances
the cursor and leaves `files.session_row_id` null. Every later append was then
declined, so a file that only ever grows would be re-read whole on every pass
for the rest of its life.
An append now only has to continue this index's own cursor; it creates the
session row when there is none to resume. That also collapses the `append` flag,
which meant both "resume this session" and "do not own it", into the resumed row
id itself.
* refactor(ai-vault-search): drop the index path module nothing calls
No caller in this PR or the two that follow it: the composition root that would
capture the userData path is PR 3b's.
* refactor(ai-vault-search): read the schema version as stale or not
Only the stale answer was ever acted on; the fresh/current split named two ways
of being fine.
* fix(ai-vault-search): read the resumed session id off an optional row
* refactor(ai-vault-search): leave page warmup and the freshness check to PR 4
`warm()` and `session-search-page-warmup.ts` have their first caller in the
query engine, which is what knows which pages are worth warming; the module
itself went with the previous commit. `isSessionSearchFileCurrent` has its first
caller in the reconciler. `session-search-file-cursor.ts` stays, because
`fileIdentity` and both its types are load-bearing here.
* refactor(ai-vault-search): stop pruning the re-read set on a retention change
The prune walked the whole set to drop what the next `beginRead` would refuse
anyway: `acceptsCandidate` applies the window when the re-read is dispatched,
and `markStale` applies it again before recording anything. The bound stays.
* fix(ai-vault-search): keep a batch tombstone from outliving its batch row
Draining a session-keyed tombstone deletes the session's batch rows, but left
any batch-keyed tombstone naming them in place. `search_write_batches` empties
on every publish, so ids restart low and the next read takes the freed rowid;
the stale tombstone then deleted that live batch's staged rows and the session
published missing messages, with no re-read to fill the gap.
The session drain now clears those tombstones in the same transaction.
* refactor(ai-vault-search): commit a file's rows and its cursor in one transaction
The staged-publish model is replaced by one SQLite transaction per file in WAL
mode. A read buffers its decoded rows and writes them, its session and its
cursor together; a reader on another handle sees the last committed state, which
is the "never a torn session" guarantee the staging machinery was built to
provide. A crash rolls the whole file back and it is re-read.
Gone with it: `search_write_batches`, `search_pending_deletes`,
`messages.batch_id`, `sessions.index_ready`, both `visible_*` views, the
open-time recovery pass, the cleanup lane and `settled()`, and the ratchet test
that made every query site read through a view. Rounds 2 through 6 were all
seams between those pieces.
A file whose rows exceed `SESSION_SEARCH_COMMIT_CHARS` is cut into chunks. The
reader only hands out a byte offset when a read finishes, so a chunk records one
no append can continue from: its rows answer searches as a coherent prefix of
the session, and the next whole read replaces them.
Retention keeps no record of unfinished work. It cuts a session loose from its
file in one small transaction, which is what stops it answering, then reclaims
its rows in bounded batches and hands the freed pages back as it goes. Rows
whose session row is gone are the record of what an interrupted purge left.
Deleted for want of a caller in PR 2, 3 or 4: the WAL budget (a buffered write
has no staging window to grow one across), the compaction module (folded into
the drain), the `sessions_content_hash` index (fork folding runs in JS over rows
PR 4 already holds), `lastWriteAt`, `failures`, `SessionSearchStoreOptions`,
`openStageCount` and `chunkMessageText`.
* test(ai-vault-search): price a per-file commit and the ceiling that bounds it
The write benchmark now times every transaction, because with one per file that
is the whole stall a file costs. A second phase indexes a single synthetic
100 MB transcript, which is what the commit ceiling is chosen against.
* fix(ai-vault-search): stop a fenced write reopening a transaction per message
A chunk write that finds the file record moved under it — a `removeFile`, or an
overlapping read of the same path — can never land anything afterwards. It kept
buffering, so every remaining message re-entered `BEGIN IMMEDIATE` only to roll
back, once per message for the rest of a file that may be a hundred megabytes.
It now stops at the first refusal and drops what it holds.
* fix(ai-vault-search): never hand a live session the id of a purged one
`sessions.id` was a plain rowid alias, so SQLite reissued it as max+1. That id
names rows in `messages` for far longer than the row itself lives: retention
cuts a session loose in one transaction and reclaims its messages over many, and
a session created inside that window was handed a freed id and adopted whatever
of the purged conversation the drain had not reached. The drain then skipped
those rows for good, because their session row exists again, and a search
answered for a purged transcript under a live session's name.
Reachable with no crash at all: an append of a growing transcript the parser
decoded no session from creates its session row mid-purge. AUTOINCREMENT is the
class fix; the schema version goes to 3 so a file written by an earlier commit
of this branch rebuilds rather than keeping a plain-rowid table under a
`CREATE TABLE IF NOT EXISTS`. `messages.id` was checked and is not exposed to
the same window: a row and its two FTS entries always go in one transaction.
Two smaller ones in the same pass. `removeFile` did not fence a read of a path
this index had never written: `current()` compared a cursor, and on an unknown
path the absent row and the absent expectation are both undefined, so the write
recreated the source after its owner had proven it gone. The writer now counts
removals per path and a write compares that count, which is a positive fence
rather than an inferred one. And `drainOrphanedMessages`,
`CONTENT_HASH_MESSAGE_LIMIT` and `CONTENT_HASH_MIN_MESSAGES` are no longer
exported: nothing outside their own modules reads them.
* fix(ai-vault-search): report a half-written file as held, not as unknown
`indexedFile` returned null for a file a chunked read left partway through, the
same answer it gives for a path the index has never seen. A caller asking "do
you hold this file" therefore read a half-written one as new, asked for whatever
read the parse cache offered, and the reader picked append — which the consumer
then declined. Only the stale set healed it, a cycle later.
It now reports the record with a null `byteOffset`, and
`requiresWholeRead(indexed)` names that state. Null rather than an added flag
because the mtime and size on that record are the file's real ones: a freshness
check comparing only those would call a half-written file current, and a boolean
is easy to not read, while every site that does arithmetic on the offset has to
say what null means at compile time.
The test also found the writer accepting an append that passed the partial
sentinel back as its predecessor offset. Nothing in the reader produces a
negative offset, but the sentinel is not a byte offset and no caller should be
able to continue it; `beginWrite` now refuses it outright.
* fix(ai-vault-search): cut a chunk at whitespace, never mid-token
The 8,000-char chunker backed up only to a newline, and only when one sat in
the second half of the window. A wrapped paragraph, a CJK transcript or a
minified log has no newline there, so the cut landed inside whatever word
straddled the target: the user's term was filed as two halves and matched
neither. It now backs up to the last Unicode whitespace in the second half,
and falls through to the target when there is none, because 4,000 characters
without a space is not a word.
A phrase that straddles a chunk boundary is still not matched. Chunks are
separate FTS rows and FTS5 cannot span them; that is stated at the chunker.
* feat(ai-vault-search): cap an indexed tool row at 3,072 characters
Tool output is 80-97 % of a transcript's bytes and a single message may be a
quarter of a megabyte, so without a cap the index, the buffer a read holds and
the transaction it commits are all sized by how much a tool printed rather
than by how much is worth searching. A `tool` message now becomes one row of
at most 3,072 characters, kept from the head, where the command and the first
lines of its output are. User and assistant text is never capped.
Measured on the write benchmark, 40 sessions with tool output at 95 % of
message text (the real band), per transcript MB: index 1,334,126 -> 353,973
bytes, write amplification 1.27x -> 0.34x, rows 18,804 -> 9,600, rebuild
1,676 -> 379 ms, largest transaction 49.7 -> 12.1 ms. On the default corpus,
whose tool results are 1.4 KB and so under the cap, nothing changes at all:
2,167,887 bytes per transcript MB on both arms.
The corpus generator takes `toolResultWords` and the benchmark reads
ORCA_SEARCH_BENCH_TOOL_WORDS so both arms are the same harness.
* fix(ai-vault-search): check the commit ceiling per row, not per message
The buffered-chars check ran after a whole message had been folded into rows,
so a single message could carry a transaction as far past the ceiling as it
was large. A conversation turn is one message and can be megabytes; the
ceiling exists to bound how long one commit holds the process and how large a
WAL it produces, and neither bound survived a message that overshot it.
* style(ai-vault-search): undo a stray reformat of untouched assertions
Three assertions in the file-write test were expanded by a formatter run that
was not the repo's, and the expansion survived because a multi-line object
literal is preserved once it exists. Restored to what they were.
* feat(ai-vault-search): write a session's identity with its first commit
A chunked read created its session row with an empty session id, an empty
title, an empty resume command and null cwd and timestamps, and only filled
them in on the final commit. Those chunks answer searches the moment they
land, so until the read ended every hit they produced named a session nothing
could identify — and a crash between chunks left it that way for good, since
the re-read that heals it is the same read starting over.
The reader already knows the id, cwd and timestamps from a transcript's
opening lines; it just had nowhere to put them. `TranscriptReadStart` now
carries an optional `identity()` the consumer calls during the read, backed by
an optional `identity()` on `ResumableSessionParseState`: one line in the
shared accumulator fold (which covers cursor, copilot, droid, gemini,
antigravity and the graph parsers) and one each in the Claude and Codex folds,
which keep their own state. A chunk commit writes what it returns; the final
commit overwrites it from the decoded session, so the mid-read title stays
provisional.
Two `cwd` guards in the Codex fold collapse into the `?? ` form the `branch`
line beside one of them already used. `extractString` never returns an empty
string, so it is the same assignment in one line instead of four, and it is
what keeps the file under the 300-line cap with the identity accessor added.
* refactor(ai-vault-search): drop conversation_fts for a column filter
The second FTS table existed for query latency alone: a column-filtered
`messages_fts MATCH '{user_text assistant_text}: q'` returns the identical
rowid set, which PR 2's round 5 review proved and PR 4's benchmark re-checked
per query. PR 4 then measured the filter at 1.16-1.36x the p95 of the second
table on a 105 MB corpus across two points in the tool-output band, under the
2x bar the decision was set at, and closes the stack's open decision 3.
It cost a quarter of the index on a corpus whose tool output is half the
message text. With the tool-row cap it is a smaller saving than the decision
was framed around, but it is still a table, a write per conversational row and
a delete per reclaimed one.
Schema version 4: `CREATE TABLE IF NOT EXISTS` is a no-op over an existing
index, so only the bump makes a file that still carries the table go.
* feat(ai-vault-search): expose the index handle a composed reader queries
PR 4's engine reads the index this store owns. One getter lets it compose over
the store's handle instead of opening a second connection to the same file,
and it carries the two rules this PR measured: never hold a read transaction
across an await, and no `.iterate()` outliving its statement. Either pins a
read snapshot, and a checkpoint cannot pass one, so the WAL grows without
bound for as long as it is held (10 MB to 266 MB on the write benchmark).
* fix(ai-vault-search): cut a chunk at punctuation, not only whitespace
The 8,000-character chunker backed up to the last whitespace in the second
half of the window and fell through to the target when it found none. A
minified tool result has none to find: valid minified JSON runs past 8,000
characters without a space, so the cut landed inside whatever word straddled
the target, and a search for `pericardium` matched neither `perica` nor
`rdium`. That is the shape most likely to be chunked in the first place,
because tool output is 80-97 % of a transcript's bytes.
The backoff now takes any character that cannot be inside a token — anything
outside a letter, digit or underscore — and still falls through to the target
when the window holds none, because 4,000 characters without one is not a
word. Surrogates are excluded from the class so a cut never lands between the
halves of an astral character.
A few of the tokenizer's own `tokenchars` (`. - / +`) are cut on even though
unicode61 keeps them inside a token. That costs the joined form of a path, and
only in a window with no whitespace anywhere, which is a far smaller loss than
the torn word this exists to prevent.
* fix(ai-vault-search): never chunk a read that cannot name its session
`updateProvisionalSession` is fed by the resumable readers alone. Claude and
Codex carry an `identity()` off their fold; `readWholeTranscript` supplies
none, because a format rewritten in place has no resumable state to ask. So a
Grok, Cursor, Gemini or OpenCode file large enough to pass the commit ceiling
published its chunks under a session with an empty id, an empty title and a
null cwd — rows that answer searches at once and that an interrupted read
leaves behind for good, since the re-read that heals them is the same read
starting over.
`add` now commits a chunk only while the read can name what it is writing. A
read with no identity, or one whose parser has decoded no id yet, keeps
buffering and commits whole at `finish`. The whole-file formats are the ones
with nothing to give and they are small — the largest on this machine is 5 MB
— so buffering one to the end costs nothing, and chunking stays reserved for
the readers that can say which session a prefix belongs to.
`updateProvisionalSession` takes a non-null identity now, so the invariant is
the type rather than a guard that silently wrote nothing.
* perf(ai-vault-search): cut a replaced generation loose instead of deleting it
The first transaction of a replace deleted every row of the old session before
inserting its own bounded chunk, so the transaction was sized by the history
being replaced rather than by the rows being written. On the synthetic 100 MB
transcript the longest replace transaction was 1,255-1,283 ms against
668-672 ms for the same file read fresh, and the gap grows with the session.
A replace now mints a new session row, points the `files` row at it and
deletes the one old `sessions` row, all in the same transaction. Every
retrieval joins `sessions`, so the old generation stops answering the moment
that commits — the same thing retention's first transaction does — and its
messages are reclaimed afterwards by `drainOrphanedMessages`, the bounded
batch loop that already exists for exactly this set. `sessions.id` is
AUTOINCREMENT, so the freed id is never handed to another session while those
rows still name it.
The longest replace transaction is now 746-848 ms, the same band as a fresh
read. The trade is stated plainly: the pass does more total work, 3.7 s
against 3.1 s, because the reclaim is 360 bounded transactions with an
incremental_vacuum step each instead of one large delete. It yields between
them, so none of it holds the process, and returning the pages per batch also
takes the index from 173.6 MB to 162.0 MB.
The store schedules the drain off the committing stack: an async function runs
synchronously to its first `await`, so calling it inline would put the first
batch back inside the transaction's own call. One drain at a time, with a
repeat flag for a replace that commits while one is running.
* fix(ai-vault-search): preserve search tokens and index Codex tools
* fix(ai-vault-search): match SQLite marks and Codex file-change records
* fix(ai-vault-search): preserve stat pairs and validate benchmark results
|
||
|
Jinwoo Hong
|
47b6c756f0 |
fix: prompt unexpectedly signed-out Cloud users once per version (#19966)
* fix: prompt unexpectedly signed-out Cloud users once per version * fix: align sign-in card English catalog with runtime defaults * fix: stack notification cards by their rendered height * fix: wait for fresh auth before showing signout card * fix: require verified auth before signout recovery * fix: retry transient auth readiness failures |
||
|
Jinwoo Hong
|
58ff95becb | refactor(mobile): name the RPC acceptance policies call sites hand-rolled (#19960) | ||
|
Jinjing
|
54216a7868 |
feat(cmd-j): compact palette location layout (#19939)
* feat(cmd-j): compact palette location layout * chore(i18n): sync palette runtime fallbacks * fix(cmd-j): preserve agent snippet context * fix(cmd-j): elide all file-backed tab paths |