mirror of
https://github.com/stablyai/orca.git
synced 2026-11-14 18:07:45 +00:00
de3da487fa6bbfde6f9e4b53fba7672fada5322c
118
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Neil
|
3a0e9a27d0 |
refactor: organize backend runtime by defining owner (#39625)
* refactor(main): organize runtime implementation owners * test: retire wall-clock CLI title recovery case * refactor: route new folder mutation to runtime store owner * test: import folder mutation store contract from its owner |
||
|
Neil
|
9bd6272fdf |
refactor(tooling): organize build, CI, and test owners (#38130)
* refactor(tooling): organize test runtime and maintenance owners * test(tooling): preserve relocated runtime fixture dependency edges * refactor(tooling): preserve xterm routing owner checks * ci(tooling): include routing owner in sparse checkout * test(tooling): preserve copied workflow dependency closures * fix(tooling): normalize hashed JSON assets in release parity * fix(tooling): distinguish asset aliases from literal references * fix(tooling): frame normalized asset reference payloads * fix(tooling): normalize decoded JSON asset references * refactor(tooling): organize build, CI and release owners * fix(tooling): align moved generator and workflow path contracts * fix(tooling): update generated plugin bundle owner banners * test(tooling): create relocated mobile verifier fixture directories * test(tooling): copy headless detector dependency closure * test(tooling): align native batch policy fixtures with owner path * fix(tooling): link moved reliability command to historical evidence * fix(tooling): preserve nested sandbox owners and canonical guide source * fix(tooling): target release version owners in checkout fixture |
||
|
Neil
|
7df246b9d1 | feat(plugins): components (TSX) plugin panels (#36814) | ||
  OrcaWinandOrcaWin
|
d1f3532bb1 |
docs: restore AGENTS.md reference docs under docs/agents-md-refs (#34322)
#34008 removed docs still linked from AGENTS.md (test anti-patterns, cross-platform support, git workflow, component fixtures) plus the macOS daemon TCC doc that cross-platform-support links to. Restore them in a dedicated folder whose name marks them as AGENTS.md inputs. Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
|
OrcaWinandOrcaWin
|
2a28fc7d4e |
[comment audit] Finish launcher and configuration comment review (#33513)
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
|
Neil
|
a30d4f4f28 |
feat(devex): component fixtures with headless screenshots for fast UI checks (#33406)
* feat(devex): component fixtures with headless screenshots for fast UI checks Colocated *.fixture.tsx files export named fixtures (render + optional store seed, setup, interact, width). A dev-only Vite entry (vite.fixtures.config.ts, extending vite.web.config.ts) renders one fixture in isolation with the real I18nProvider, TooltipProvider, theme class, default-seeded store, and the web client's preload bridge (installWebPreloadApi) as window.api. `pnpm fixtures:shot <id|glob> [--theme dark|light|both]` starts or reuses a detached server (Vite + one headless Chromium) and writes PNGs to .cache/fixtures/. A warm page re-renders in place (~100ms/shot) and reloads only after source changes. A stdio MCP server (registered in .mcp.json) exposes list_fixtures and screenshot_fixture. Fixtures load only through import.meta.glob in the fixture entry, which is not a production build input; oxlint rejects static imports of *.fixture. * Keep fixture config and agent config inside reviewed root entries * Baseline the fixture page's default-settings seed in the focus-read ratchet * docs(agents): shorten fixtures pointer * Allowlist the dev-only fixture explorer's labels in localization coverage |
||
|
Neil
|
d08774d336 |
feat(devex): pnpm check:changed as the single pre-handoff gate (#33432)
* feat(devex): add pnpm check:changed as the single pre-handoff gate Runs format, oxlint, the changed-code gate, affected typecheck projects, affected unit tests, and the ratchets/verifiers whose inputs changed, in parallel against the origin/main merge base plus local edits. Printing is limited to failures, and a pass is cached by a hash of the diff, untracked files, lockfile, and config so an unchanged rerun exits immediately. * fix(devex): keep the check:changed pass valid across staging and commits * docs(agents): slim Useful Commands to the commands agents need |
||
|
Neil
|
e39330e15d |
docs: slim STYLEGUIDE.md into a core guide plus topic deep dives (#33402)
Keep the rules most UI changes need in docs/STYLEGUIDE.md (32.9KB -> 10.1KB) and move per-component specs, native chat scale, UX rationale, and the screen review rubric into docs/styleguide/. Values now point at main.css instead of being restated. Lint messages and comments that cited moved sections now point at the new files. |
||
|
OrcaWinandOrcaWin
|
1dbdc14bc0 |
docs: remove checked-in audit artifacts (#32903)
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
 NeilandBrennan Benson
|
d9dc87b2fb |
Use incremental tsrs locally and update Bun to 1.4.4 (#32276)
* Use incremental tsrs for local typechecks * Update pinned Bun to 1.4.4 * fix(deps): bump dompurify to 3.4.17 and docs next to 16.4.1 for security advisories dompurify 3.4.17 patches GHSA-cg2g-xh64-v4q7, GHSA-mrv2-3pqr-j5jr and GHSA-c6gr-qvv4-5v3f in the root and mobile lockfiles. next 16.4.1 patches GHSA-2464-fhr4-w9xw, GHSA-673q-mjm6-r5xr and GHSA-p3m5-59mp-jv5r in the docs site. * Prepare mobile bundle for smoke benchmarks * Give native compiler benchmarks a larger ARM runner --------- Co-authored-by: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> |
||
|
Neil
|
88aa3f27ce | docs: track sequential module organization cleanup (#31670) | ||
|
Neil
|
ceb2a6c6d9 |
docs: add test anti-pattern reference (#31277)
* docs: add test anti-pattern reference * docs: note real-time test waits * docs: qualify redundant test guidance * docs: fix agent guide formatting |
||
|
Neil
|
ec44a502d5 | docs: clarify first-class platform, remote, and mobile priorities (#31265) | ||
|
Neil
|
bc6b2dd84c | docs: load Git guidance on demand [skip ci] | ||
|
OrcaWinandOrcaWin
|
b3dfd54827 |
perf(build): skip unnecessary work and reclaim Vite caches (#29479)
* perf(build): remove unnecessary development and config work * feat(dev): reclaim inactive worktree Vite caches --------- Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
|
Neil
|
f152101010 |
ci: reuse prepared Windows security addon in PR packages (#29352)
* ci: reuse prepared Windows security addon in PR packages * ci: prepare bounded Windows security reuse qualification * ci: ignore owned Windows benchmark receipts during source checks * test: inject Windows security qualification dependencies * test(mobile): register chat view mocks in test roots |
||
 OrcaWinandOrca Worker
|
4559c250b3 |
fix(windows): restrict secure files in-process instead of spawning icacls (#28135)
* fix(windows): restrict secure files in-process instead of spawning icacls Every secure write on Windows spawned icacls.exe five times on the calling thread (the Electron main thread for most stores): verify, /reset, /inheritance:r /grant:r, verify, verify. Process creation under endpoint security costs ~0.6 s each, so each write froze the UI for seconds (#26027). The /reset + /inheritance:r pair is also what behavioural EDR scores as ransomware tradecraft (#25583). Add native/windows-security (@orca/windows-security): a small N-API addon over advapi32 that reads and writes DACLs in-process. It exposes the token user SID, a DACL read as SDDL, a DACL write (sync and on the thread pool), a create-with-DACL file write, and an additive grant. Policy and verification stay in TypeScript. - writeSecureFile creates each temp file already carrying the protected user/SYSTEM/Administrators DACL (CreateFileW + SECURITY_ATTRIBUTES), reads it back once, and renames it. No ACL is modified on the write path and no process starts. Before: 5-7 spawns, 94-138 ms per write. After: 0 spawns, ~0.5 ms. - A secure directory is verified once per process. It is rewritten only when verification fails, on the thread pool, because the write walks every descendant (~2 s on a 29k-entry userData). - The verifier follows OpenSSH's rule: only the user, SYSTEM and Administrators may be granted anything, and the path must be protected (or, for a file, inherit from a secure parent). DACLs written by the icacls-era hardening pass unchanged, so there is no migration. - If the addon cannot load, the write still happens and reports unrestricted, logged once. There is no icacls or PowerShell fallback. - grantDirAcl, the startup userData grant, and the install-dir probe and package repair use the same addon, so whoami.exe and icacls.exe are no longer spawned anywhere at runtime. - orcad's Windows slots ship the addon beside their bundles (built per arch, PE-machine checked). Its data-root privacy check fails closed, so without the addon a Windows orcad would not start. Fixes #26027 Fixes #25583 * refactor(secure-file): harden existing Windows files through the sync lane The read path called restrictWindowsPathSync directly and recorded success itself; the existing sync lane already does both. One call site fewer. * refactor(windows-security): one load, one rule, no fallthrough (review #28135) - The descriptor parser is anchored, so a conditional ACE (its body holds parentheses) or anything else it cannot read fails closed instead of verifying the ACEs before it. - findSecureDaclViolation is one flat rule: protected, allow rules only for the user, SYSTEM and Administrators (OpenSSH's ACE rule), and full control for the user. The inherits-from-a-secure-parent exception is gone: it accepted exactly what loosening the parent reaches. - The addon and the user SID load once, behind one wrapper that adds the extended-length prefix to every path and runs async DACL writes to one path in order, so a startup grant cannot land after a re-lock and restore the broader DACL. - With the addon loaded, a failed born-restricted create throws with the matching Node error code instead of falling back to a plain write. - A grant leaves a NULL DACL untouched instead of turning 'everyone' into 'only this SID'. WriteFile runs once and fails on a short write. - The artifact create-intent store writes through writeSecureFile and drops its own temp writer, directory LRU and synchronous directory re-lock. - Test seams the global fake makes redundant are gone. * fix(packaging): fail a Windows build whose workspace addon binary is missing or foreign-arch The @orca/windows-* workspace links exist on every host, so their presence proved nothing: a missing or wrong-architecture .node only surfaced at runtime, as unrestricted secure files or an orcad that refuses to start. beforePack now reads each addon's PE machine against the target arch. * refactor(windows-security): prune what the native ACL left dead (review #28135) - Sync-spawn ratchet: hardenSecurePath and the sync-flag exemption it alone used are gone, and the advice and baseline header no longer say secure writers run icacls. - The install-dir probe reads ACEs through parseSddlDacl, so an ACE it cannot parse fails the whole reading closed instead of being skipped; AC is a resolved alias there now. - The packaging check reads the addon binary through node_modules, the copy that ships. - No winCode-to-errno table: nothing reads .code off a failed secure write. - The load memo is not mutated; secure-path keeps its own log-once flag. - grantInstallDirPackageAcl is a plain function, and secure-file restricts a Windows directory and a POSIX path directly instead of through one two-platform helper. * fix(windows-security): resolve LA in real-ACL tests; tear a file whose close fails - The GitHub Windows runner is the built-in Administrator (RID 500), whose SID reads back as LA. Production already resolves it through localDomainSidOf; the win32 test helpers parsed without it. One shared readDaclOf resolves aliases for every real-ACL suite, and the verifier's unit test covers an LA-aliased directory and LA as a stranger. - createFileWithDacl treats a failed CloseHandle like a failed write (lost data on a network volume) and deletes the torn file. - Drop a duplicate test reset and a typo. * fix(windows-security): load only from inside the app; read DACLs as structure (review #28135) - Loader: orcad's staged binary first, then the workspace package only if its real path is inside the app (process.resourcesPath, or this checkout). A bare specifier otherwise walks every parent up to C:\node_modules, which any local user may create, and orcad holds credentials. - The addon reads a DACL as { protected, aces: [{ type, flags, mask, sid }] } via GetAce, with a SID only for plain allow/deny ACEs and null for a NULL DACL. findSecureDaclViolation and the install-dir probe check that structure, so conditional, callback and object ACEs fail closed. The SDDL parser, its alias table, LA/localDomainSid resolution and getDaclSddl are gone; SDDL is only the write format. - Install-dir tree pass: files flagless as before, but a directory only when its own DACL is protected (inheritably). Re-granting inheriting directories re-propagated each subtree (3.2 s for 1,093 directories). The root is no longer granted, or counted, twice. The two passes are plain functions behind a { root, tree } seam. - One namespacing (path.win32.toNamespacedPath) at the addon boundary, also the per-path ordering key; the comment states the real sync-grant invariant. - restrictWindowsDirectory(dir): Promise<boolean> replaces the callback lane; the reporter is guarded once, so the 'settle' stage is gone. - Sync-spawn ratchet: the secure writers no longer spawn or block, so they leave the gate (109 -> 61 uses). - Errors name the Win32 call and code only; a failed GetSecurityDescriptorControl fails the write instead of defaulting to unprotected. * test(secure-file): assert the durable POSIX write's outcome, not the old chmod order writeSecureFile now chmods the temp file to 0o600 before fsyncing it, so the fsync also covers the mode change and the file is owner-writable when opened. Assert what matters under umask 0o200: the write succeeds, the temp file is fsynced read-only, and the published file is exactly 0o600. * fix(windows-security): namespace paths with the host's toNamespacedPath path.win32.toNamespacedPath rewrote POSIX temp paths (/tmp/x -> \tmp\x) in the Linux runs of the Windows-logic unit tests, so the fake addon wrote to the wrong place. The addon only runs on Windows, where the host function is the win32 one; elsewhere it is the identity. The long-path assertion runs on Windows only. * test(windows-security): assert the install-dir long-path prefix on Windows only The prefix comes from the host's toNamespacedPath, which is the identity off Windows; the Linux run of this suite expected the win32 escape. * fix(windows-user-data-acl): grant the root even when its children cannot be listed (review #28135) A readdir failure on userData aborted the whole grant, so the root grant that could repair a DACL refusing the listing was never attempted. The listing failure is now recorded, the root is still granted, and the marker stays withheld so the next launch retries. * test(windows-security): wait for relock propagation, accept Entra ID SIDs (review #28135) - The large-relock test waited only for the parent to turn protected, which happens before propagation reaches the children, then read a child. It now waits on the parent and both ends of the child range. - The token-user test rejected S-1-12-1 (Entra ID) accounts, failing the suite on Entra-joined dev machines. --------- Co-authored-by: Orca Worker <orca-worker@localhost> |
||
|
OrcaWinandOrca Worker
|
7909732283 |
perf(claude): run the Windows Claude hook as a signed native claude-hook.exe (#27719)
* perf(claude): run the Windows Claude hook as a native orca-hook.exe
Each Claude hook event on Windows started cmd.exe (plus the user's cmd
AutoRun) and then curl.exe on top of the agent's own shell. A small signed
Rust binary now does the same synchronous POST in one process, under Git
Bash, PowerShell and direct CreateProcess alike.
- native/windows-agent-hook: std-only crate, same contract as the .cmd hook
(prints {}, env guards before stdin, endpoint re-read, 0.5s/1.5s caps,
waits for the listener reply, always exit 0). Its installed file name
selects the agent profile, so the registration stays a bare path.
- Packaged as resources/bin/orca-hook.exe (SignPath signs it with the other
inner PEs); afterPack fails on a missing or wrong-machine binary.
- Claude install copies it verbatim to ~/.orca/agent-hooks/claude-hook.exe
(temp + rename, running copy moved aside) and registers it; it falls back
to the .cmd hook when the bundle is missing, unsigned or cannot be copied.
Only Claude opts in, via windowsNativeHookExecutable.
- The managed-command matcher sweeps .exe entries alongside .cmd/.ps1/.sh.
- The CLI launcher and hook builds share windows-rust-executable-build.mjs,
which now also checks the PE machine.
Refs #20913
* fix(build): publish native Windows binaries atomically so parallel builds cannot collide
The Windows CI lane runs every orca-hook.exe suite in one vitest
invocation, each building through build-windows-agent-hook.mjs. The build
copied straight onto native/windows-agent-hook/.build/orca-hook.exe, and
CopyFile opens its destination unshared, so a second build (or a scanner
still holding the fresh image) failed with EBUSY.
- windows-rust-executable-build.mjs skips the publish when the output
already has the built bytes, otherwise stages a per-process temp file and
renames it over the output, retrying briefly while another handle holds it.
- The test fixture hands each suite a private copy of the build output, so
no suite reads or starts the shared file while another build replaces it.
Refs #20913
* refactor(claude): simplify the native Windows hook to one Claude binary
Review follow-up on the native hook (#20913): the same behaviour with less
machinery.
- The binary is built, shipped and installed as claude-hook.exe, so the
installed name matches its PE OriginalFilename. It is Claude-only, so the
file-name profile table and the per-agent opt-in flag are gone; the
registration is gated by the existing Claude-only usesWindowsEntry check.
- Install and refresh reuse writeManagedScript / refreshManagedScriptIfPresent
(now accepting bytes): temp file, rename, skip when identical. A copy whose
hook is still running cannot be replaced; install then registers the .cmd
hook and refresh keeps the old build until a later start, instead of the
rename-aside and *.retired sweep.
- The install-time Authenticode-presence check is dropped. Release CI now
requires resources\bin\claude-hook.exe to be signed (release-cut and the
signing rehearsal), next to resources\bin\orca.exe.
- getStatus counts either Windows form through findManagedEventHookForms in
hook-settings.ts, replacing managed-event-hook-status.ts.
- Rust: plain Claude constants, one endpoint-file lookup (last assignment
wins, as under `call`), one request write and a deadline-bounded drain.
Guard order and form encoding are unchanged.
- Build: PR CI builds the hook once before the Windows tests, so the suites
only read it; the launcher publish goes back to copyFileSync and the retry
loop and its test are gone. The PE-machine check lives only in afterPack.
Cargo.lock joins both crates' build fingerprints, and the launcher's alias
exports are removed.
Refs #20913
* test(ci): run the managed hook stdin-lifecycle suite in the Windows lane
Its claude-hook.exe case (and the other Windows-only cases) are gated per
test, so the lane-registration guard did not require the file and it ran
only on Linux, where those cases skip. Register it in both Windows lists.
Refs #20913
* feat(agent-hooks): ship the native hook as orca-agent-hook.exe <agent>
The native Windows hook (#20913) becomes one general binary that stays
Claude-only in behaviour.
- Named orca-agent-hook.exe everywhere: build output, PE OriginalFilename
and InternalName, resources\bin, the afterPack check, the installed copy
in ~/.orca/agent-hooks, and the release signed-target lists.
- It takes the agent as its first argument and Claude registers
`<path> claude`. The CLAUDE_JOB_DIR exit and the Grok/Devin drain belong
to that agent; an unknown or missing agent answers {} without reading
stdin. The managed-command matcher owns `orca-agent-hook.exe claude` and
still owns the old claude-hook.exe entry, so installs migrate; install
deletes a leftover claude-hook.exe when nothing runs it.
- A copy still running from the previous build no longer forces the .cmd
fallback. Identical bytes are not rewritten; otherwise the temp file is
renamed over the target, and when Windows refuses because the image is
mapped, the running copy is renamed to orca-agent-hook.exe.old first.
Install deletes a stale .old; only when that copy is also still running
does the install fall back to .cmd. writeManagedScript and
refreshManagedScriptIfPresent take the aside path, sharing one helper.
Refs #20913
* fix(agent-hooks): roll back a failed aside rename and keep an installed hook exe
Design-review follow-up on the native hook (#20913).
- If the running copy was moved to orca-agent-hook.exe.old but the new
file then could not take its name, the copy is renamed back before the
error is rethrown, so settings.json never points at a missing exe. The
rename helper now owns the `.old` name, next to the comment that
explains it; a mocked-fs unit test covers both the sync and async paths.
- A failed write, or a build with no bundled exe, keeps registering a copy
that is already installed (every build knows `claude`), so a dev checkout
and an installed build no longer flip settings.json between forms.
- The claude-hook.exe migration is gone: no release ever shipped that
name, so the legacy sweep, the `.exe` matcher needle and their test are
removed.
Refs #20913
* fix(agent-hooks): derive the native hook's agent with win32 path rules
getWindowsNativeHookAgent took the host's basename of the managed script
path, so on a POSIX host (Linux CI) a Windows path was not split and the
registered command became `orca-agent-hook.exe C:\...\claude`. It now takes
the path and applies path.win32.basename itself. The install destination
and the `.old` aside stay on host paths: they name files the host opens.
Refs #20913
---------
Co-authored-by: Orca Worker <orca-worker@localhost>
|
||
|
Neil
|
ac156c0302 |
perf(ci): overlap Bun typechecking and skip redundant alias scans (#27382)
* perf(ci): skip unnecessary alias scans and compare preflight compilers * perf(ci): overlap measured Bun typechecking with static checks * perf(ci): use measured Bun lint runtime and strengthen compiler parity checks |
||
|
Neil
|
16cd1e0c75 |
docs: remove stale internal reference documentation (#26328)
* docs: remove stale internal reference documentation * test: avoid pooled relay hook sockets across fake clock advances * test: remove checks for deleted headless server documentation |
||
  Neilandczzczz
|
4c2cb3cf12 |
fix(jcode): quote the managed hook path, drop tools from patch prompts
Three fixes, all on paths this PR could not exercise locally. The managed hook command was stored as a bare path. jcode tokenizes that string shell-style before exec'ing it directly (parse_hook_command, crates/jcode-terminal-launch/src/lib.rs): unquoted whitespace splits, and every unquoted backslash is consumed as an escape. So on Windows `C:\Users\me\.orca\agent-hooks\jcode-hook.cmd` reached exec as `C:Usersme.orcaagent-hooksjcode-hook.cmd` and no hook fired at all, and a POSIX home with a space split into two arguments. Store the path single-quoted (verbatim, backslashes included), falling back to double quotes for a path containing a single quote. Existing bare entries are already repointed by the stale-key path, and getStatus accepts both forms so the repair is not reported as a user-owned hook. The quoting helper was previously dead code that only tests called; the three production sites now use it. isJcodeManagedCommand also normalizes separators, since a `/`-only needle never matched a Windows entry. Commit-message generation feeds a staged patch to `jcode run` as the prompt — attacker-influenced text — while jcode's default profile exposes shell, read, write, and MCP. Pass `--tool-profile none`, which resolves to an empty allowed-tool set in jcode's config (base_allowed_tools), matching the read-only posture claude (plan) and codex (read-only) already take. docs/reference/jcode-hook-events.md was never actually in this PR: the repo ignores docs/** and tracks reference docs by allow-list only, so the captured-payload evidence four source comments point at was silently dropped. Allow-list it. Co-authored-by: czzczz <chanzrz_zbf@foxmail.com> |
||
 Jinwoo Hong
|
daf63e659c |
fix(runtime): read Antigravity, Cline and Prime Agent readiness from the live screen (#24222)
* fix(runtime): decide Antigravity readiness from the live screen agy paints its composer with cursor addressing, so the line-folded wait text misses the 1.2.14 accept-edits and plan composers and an ended turn, while the grid keeps the bare `>` caret painted mid-turn and behind the /model picker. Read the screen's bottom rows instead: rule, caret, rule, `? for shortcuts`. A clocked pane is held to quiescence (tier 1b) because the submit repaint reads ready for a moment; a clockless restored pane settles from the screen alone. When a trustworthy screen exists it decides, so the name-only title lane no longer settles an open picker. Retires the visible-read probe's Antigravity branch: the probe now runs the shared screen rule for any screen-ruled agent without an output clock, and keeps its generic empty-pane read for everyone else. Adds twelve agy 1.2.14 recordings and a replay suite shared by screen-ruled agents. STA-8741. * fix(runtime): decide Cline readiness from the live screen Cline paints its composer box with cursor addressing on the alternate screen, so no text rule saw it and worker-start timed out at agent_readiness (#23268). Read the box off the grid: rule, an empty composer with one of the captured placeholders, rule, the Plan/Act row and the auto-approve row, with no braille spinner above it. A streaming reply repaints the same box once its spinner has scrolled away, so Cline is tier 1b only: a clocked pane waits for quiet and a clockless one never settles from the screen. The screen now decides for a Cline pane, which shuts the quiet-process lane that would have settled its unworded tool-approval prompt and the Cline Desktop promo. readLiveTerminalScreenLines now returns raw rows: the read projection blanks a composer it takes for a draft, and it takes Cline's placeholder for one, so a typed draft and an empty composer looked the same. Adds nine cline 3.0.66 recordings (macOS) and the 3.0.65 Windows capture from #23269. STA-8741. * fix(runtime): decide Prime Agent readiness from the live screen Prime redraws its composer on the alternate screen, so the text tail never showed a settled prompt and tui-idle timed out (#22153). Read the grid instead: a bare `>` directly over the `<- manage` footer, with no braille status row (`Writing - 6s`) above it. The footer and caret alone stay painted for a whole turn. Replayed chunk by chunk, Prime erases that status row before redrawing it, and on first launch paints the idle composer just before the trace-sharing question covers it. Both keep repainting, so a clocked pane is held to quiescence (tier 1b); a clockless restored pane settles from the screen alone. Adds nine prime-agent 0.9.8 recordings (isolated HOME, OpenRouter) and the two 0.9.5 captures from #22154. STA-8741. * refactor(runtime): drop Cline-only readiness branches Cline now follows the same pattern as Antigravity and Prime: a screen rule plus table entries. - Drop MID_TURN_COMPOSER_AGENTS. onPtyData stamps lastOutputAt on every chunk, so a re-attached streaming pane has an output clock from its first byte; the exception only guarded a pane that printed nothing since attach. A clockless Cline pane now settles from its screen like the other two. - Drop the 'ready-body' rest-signal entries for all three agents. The rest signal is read only by quietForegroundLane, and a readable screen already shuts that lane and the title lane (isReadinessDecidedByScreen), so the entries only removed the quiet-process fallback for a pane with no trustworthy grid. The census now checks that screen-shut instead. - Drop the Cline rule's auto-approve row check; no recorded verdict depends on it. Kept: raw rows from readLiveTerminalScreenLines. Every frame of every codex-* and qoder-* capture at 120x40, 80x24 and 100x32 gives the same isKnownReadyPromptBody (with and without a clock) and isQuietReadyScreenBody verdict through both readers. Serializer known-failures for the new captures are pre-existing serializer behaviour, not this branch: row-0 cells restore with a true-colour background where the source has the default (the DSH class), and Prime's cursor restores at column 119 instead of the pending wrap at 120 (the qoder class). STA-8741. * fix(runtime): trust a screen rule only on the PTY's own grid Review findings on the screen-ruled readiness (STA-8741): - A grid out of step with the PTY garbles cursor-addressed chrome, and a model resize does not make the TUI repaint. readLiveTerminalScreenLines now returns null unless the emulator's grid matches the PTY's reported size and was never reflowed without a repaint (a re-attach that learned the real size late), so the pre-existing lanes decide there instead of timing out. - The visible-read probe reads the draft-blanking projection, which turns Cline's `❯ Ask anything...` into a bare `❯`. It now restores the blanked composer row before the rule reads it; `terminal read --screen` output is unchanged. - The quiet lane no longer ORs the text rules over a trustworthy screen that refused; without one, tier 1 already ran them. No recorded verdict changes. Tests: ready recordings on a mismatched and on a reflowed grid settle through the old lanes; the restored-pane probe runs every ready recording through the real projection; the rest-signal census checks the lane verdict with and without a screen. * test(runtime): trim STA-8741 recordings to the screens they prove * refactor(runtime): one screen verdict for every screen-ruled lane readScreenRuledReady, readScreenRuledQuietReady and isReadinessDecidedByScreen each re-derived the same thing: the agent's rule applied to a trustworthy live screen. They collapse into readScreenRuledVerdict (true / false / null), which tier 1, tier 1b and the lane gate read. This also makes a refusal final in tier 1: a clockless pane whose trustworthy screen refused fell through to the text rules, so retained ready text could settle over an open picker (Greptile review). The quiet tier already refused there; now both do. The tier-1b agent set derives the screen-ruled agents from the rule table instead of listing them again, and the lane test that repeated the census case is dropped. * refactor(runtime): let the visible-read probe read its own output clock The probe's clock was captured at start and threaded through the wait dependencies as a one-off parameter. The probe now reads it from the live record when its screen read returns, which is also the fresher answer. * fix(runtime): trust a reflowed grid again once a PTY resize repaints it The reattach-reflow flag was never cleared, so a pane stayed on the old lanes for the rest of its life even after a real resize made the TUI repaint (Greptile review). The record now keeps the reflowed grid, and a PTY resize off that grid clears it; an echo of the same size sends no SIGWINCH and keeps it. Tests: the reflow case in every screen-ruled suite now includes a same-size echo, and an Antigravity recording only the screen reads ready settles after a resize and repaint. * refactor(runtime): keep screen-rule trust and raw rows to screen-ruled agents Two shared changes reached agents this PR does not target: the live screen reader returned raw rows, and it refused a grid that did not match the PTY. Both now live in readScreenRuledLines, which only the screen-ruled agents read (screenReader picks it from the rule table); readLiveTerminalScreenLines is main's again. The probe keeps main's Antigravity-banner trigger, so a Codex or unknown pane is probed exactly as before. Proof: the non-screen-ruled suites give identical pass sets on this branch and its base (1,781 tests), and replaying every other recording frame by frame through the readiness and blocked verdicts, for its agent and for an unknown pane, gives identical results (93 pairs). A new test keeps a Codex pane reading its screen when the PTY reports another grid; it fails if the trust check moves back into the shared reader. |
||
|
Neil
|
d2dbe2c385 |
fix(windows): replace the managed CLI launcher with a native one (#24094)
* docs(security): add the antivirus clearance path for future releases Every AV false positive here has been handled one vendor and one shipped version at a time. Document the programs that clear future releases instead -- signer and product enrollment rather than per-build sample submission -- and add a script that reports an RC's current detection state by hash, so a verdict is found before users meet it in an issue report. Hash lookup only by default; --upload transmits the artifact and stays manual. * fix(windows): replace the managed CLI launcher with a native one resources\bin\orca.exe was a csc-compiled MSIL assembly: a small, freshly compiled .NET image in a user-writable directory that mutates environment variables and proxies a child process. That is the shape .NET dropper heuristics are trained on, and every verdict against it named the family -- MSILHeracles from two vendors, Wacatac!ml from a third. Signing the file does not change its shape, so signing never cleared it. Rebuild it in Rust. Same resolution, same environment contract, same argv passthrough that keeps newline-bearing orchestration bodies intact (#8374), and the child still inherits our environment block rather than an explicit map, so a block carrying both PATH and Path survives (#12046). The PE now carries publisher, version, icon and an asInvoker manifest from build.rs. Refs #23383 * ci(windows): install the Rust toolchain before building the CLI launcher The hosted runners happen to ship cargo, but a real Windows dev box does not -- verified on our own Windows QA host, where cargo and rustc were both absent. Relying on the image means a future image change fails deep inside electron-builder's native hook instead of at an obvious step. |
||
|
Neil
|
7ea01279cd |
feat(search): bundle ripgrep for local, WSL, and SSH search (#22396)
* feat(search): bundle ripgrep for local, WSL, and SSH search Ship @vscode/ripgrep-universal's prebuilt rg for all six relay platforms in every desktop artifact. Local and WSL searches spawn the bundled binary and drop the git ls-files / git grep fallbacks; SSH deploys upload the remote's binary once per ripgrep version and the relay prefers it over PATH rg. * fix(search): address bundled ripgrep review findings - Key the SSH ripgrep cache on the binary's content hash; a package bump is the only update step - glibc verifier: read arch tokens below the slice root and accept static ELFs (arm64 release blocker) - Ship ripgrep/PCRE2/musl license notices; bundle rg with orcad - Packaged builds never spawn a bare rg; report fd pressure as transient - SSH: install rg before sweep/GC, size-validate installs, back off instead of disabling on launch failure - Scope Dependabot to @vscode/ripgrep-universal; revert unrelated lockfile churn * chore(search): drop bundled-ripgrep reference doc; assert full packaging layout parity * refactor(search): one entry point for spawning the bundled ripgrep Local Quick Open, Quick Open path search, the Explorer name filter, and runtime text search each repeated the same three steps: resolve the bundled command, spread in the WSL distro, spread in the WSL shell expression. Fold that into spawnBundledRipgrep so one place owns the rule that a bare 'rg' must never reach spawn, and simplify the resolver's command/packaged checks. Restore the AGENTS.md ripgrep rule dropped alongside its reference doc in |
||
|
Neil
|
0afc66ebd3 |
fix(opencode2): only treat the question tool's form as a pane blocker (#22399)
* fix(opencode2): only treat the question tool's form as a pane blocker OpenCode 2 has one form primitive and several producers, and Orca's setup bridge mapped every `form.created` to `question.asked` — the un-evictable "the pane owner must answer this" blocker. Against opencode v2.0.12 only `metadata.kind === "question"` is the agent's question tool; `websearch.provider` is a provider picker and `mcp-elicitation` is an MCP server prompt raised on sessionID "global", which is not a session and so can never be retired by that session going idle. Admit only the question kind, remember the admitted form ids, and drop `form.replied`/`form.cancelled` for forms that were never admitted so an ignored form's resolution cannot retire a live blocker. Evidence (live v2.0.12 capture, real TUI in a PTY against `opencode serve`) in docs/bug-reproductions/opencode2-form-created-kinds. That capture also shows the reported Subagents/Shell/Terminals dock and the agent picker emit no server event at all, so they were never the `form.created` source. Refs #22371 * refactor(opencode2): drop the unreachable form-resolution guard Review was right that the admitted-form-id set defended against nothing. `clearAttentionForResolution` builds the exact key [factoryID, "AskUserQuestion", form.id, sourceSessionID] and returns null on a miss, with no session-wide fallback, and form ids are unique — so a resolution for a form Orca ignored already matches no live blocker. The guard's comment claimed a collision the key structure rules out, which is worse than no comment. Removes the set, its FIFO eviction helper, and the claim; the kind check on form.created is the whole fix. The end-to-end test stays: it pins the behavior that an MCP form raised and cancelled leaves a live question blocker standing, which is the property worth holding regardless of how it is achieved. |
||
|
921882619e |
fix: retire closed editor models from the app shell (#21178)
* fix: retire closed editor models from the app shell * test(editor): use checked Monaco attachment calls * Preserve bounded editor view caches when retiring closed models * docs(editor): describe batched model retirement * fix(editor): preserve cleanup work across registry replacement * fix(editor): build editor model URIs with the file scheme Monaco keys its model registry by `uri.toString()`, and both `@monaco-editor/react` (via the `path` prop) and the closed-tab disposal path built that key with `Uri.parse`. On Windows a raw path such as `C:\repo\a.ts` parses as scheme `c`, which fails the scheme gate in `modelService._schemaShouldMaintainUndoRedoElements`, so closed-file undo history was dropped for every file at any size — not only the large files the tradeoff note covers. Add `toEditorModelUri`, the one filesystem-path -> model-key function, built on `Uri.file` so the result always carries the `file:` scheme and re-parses to itself. Route model creation, disposal lookup and the still-open ownership comparison through it so all three agree; a divergence there would dispose a model an open editor is still editing. --------- Co-authored-by: m4air <m4air@Mac.localdomain> Co-authored-by: Neil <neil@stably.ai> |
||
|
|
89acf1e1fa |
fix(plugins): release diagnostic logs after successful uninstall (#21185)
* fix(plugins): retire log owners after successful uninstall * fix: address memory PR review regressions and withdraw false positives * fix(plugins): fence stale activation after uninstall * chore: allow durable plugin uninstall audit evidence --------- Co-authored-by: m4air <m4air@Mac.localdomain> Co-authored-by: Neil <neil@stably.ai> |
||
|
OrcaWin
|
7063c2cbdd |
fix: read crash diagnostics without loading whole dumps into memory
Read crash diagnostics incrementally to avoid whole-dump memory spikes. |
||
|
Neil
|
ea01cd0ccd |
fix(windows): reject a node-pty addon that predates the MSYS breakaway denial (#20047)
* docs(windows): record the measured MSYS job-breakaway mechanism The per-PTY job already denies JOB_OBJECT_LIMIT_BREAKAWAY_OK for Cygwin/MSYS shells (#19068), but nothing records why, and a conpty.node built before that commit fails windows-msys-job.win32.test.ts in a way that reads as a source defect. Measured on a real Windows 11 host: both the plain and the exec- replacement Git Bash shapes leak, the escape is the MSYS runtime's own spawn/exec (fork keeps membership), and a single-variable A/B on usesCygwinRuntime flips the result 0/2 -> 4/4. Also names the gap the failure hid behind: node-pty-job-ownership.cjs asserts symbol presence, which cannot distinguish patch revisions. * fix(windows): reject a node-pty addon that predates the MSYS breakaway denial The native-runtime gate asserted only that terminateJob, listJobProcessIds and assignCurrentProcessToJob were exported. All three predate the Cygwin/MSYS breakaway denial, so an addon built before it passes every gate, isPtyJobOwnershipAvailable() returns true, and windows-pty-job.win32.test.ts passes 6/6 -- while every Git Bash child is created outside its pane's job and survives terminatePtyJob. Read the resolved .node and require the wide msys-2.0.dll literal that usesCygwinRuntime holds, the way stagedRelayAddonIsUnpatched() already tells a patched windows-process-tree addon from a published one. An addon the caller cannot name is refused rather than skipped: a gate that cannot see its subject is not a gate. Verified against real binaries on a Windows 11 host: the shared checkout's pre-#19068 build errors, a build from current patched source passes, a missing path errors. Also closes the cross-host packaging skip. The export half has to load the addon so it cannot run when the packaging host is not the target, which is how a Windows release built elsewhere could ship this. The marker is a file read and needs neither; an unrecognised layout warns rather than fails a release that was packaging fine. * fix(windows): check the MSYS breakaway denial on the rebuild path too The Electron probe carried the marker check, but it lives inside probeElectronNativeModules, which returns early whenever the Electron package binary is unusable. Covered by another path is not this path checks -- and the defect this whole change closes was a gate that looked like it checked. Reading the binary needs neither a loadable Electron nor an executable target arch, so assert it after the rebuild, beside the windows-process-tree assertion that exists for the same reason: this is the addon copied into the packaged app. Absent warns (a cross-platform rebuild need not leave a win32 addon on this disk); present and unmarked is fatal. The fixtures now write a real addon file, because the gate reads the binary it was told about rather than trusting the exports. Verified against the two real binaries measured on the Windows host: the pre-#19068 build fails this path, the build from current patched source passes. * fix(windows): check the marker on every ConPTY path the packaged app can load The packaged marker check read one hard-coded path, `build/Release/conpty.node`, and warned when it was absent. `loadNativeModule` tries `build/Release`, then `build/Debug`, then `prebuilds/win32-<arch>`, swallowing each failure, and `prunePackagedNodePty` drops the published prebuild only when a same-arch `build/Release` exists to replace it. So the two packages the check was added for were the two it could not see: - cross-host: no host but Windows can build conpty.node, so there is no `build/Release` and the prebuild is what ships. The check warned and returned. - cross-arch: `build/Release` is the packaging host's own arch, patched and marked, so the check printed OK -- while the target app cannot load it and falls through to the unmarked prebuild underneath. Measured, not assumed: both published Windows prebuilds in the node-pty tarball contain neither `msys-2.0.dll` nor `cygwin1.dll` in any encoding. They are the binary that leaks every MSYS pane child out of its job. It now sweeps every candidate present for the *target* arch and refuses a package with no candidate at all, which is a package with no ConPTY backend rather than a layout to shrug at. It runs for every Windows slice instead of only the branch the export check skips, so deleting the export check cannot silently take it too. A stale source build keeps the rebuild advice; the prebuild gets the advice that actually works, which is to package the slice on a Windows host of that arch. Also: the marker constant was re-typed in four places and was tied to the C++ literal that produces it by nothing at all, so editing the patch would have left a gate that fails every correctly rebuilt addon and tells the developer to do the one thing that cannot help. The fixtures now take the constant from the gate, and a test asserts the patch still adds `L"msys-2.0.dll"` to conpty.cc. And the rebuild path treated a missing addon as a warning even on the host that will run the install, where node-pty would fall through to that same prebuild. The verdict is now a value, so it is tested without a platform gate. * fix(windows): resolve the packaged ConPTY the way its loader does Sweeping every candidate and demanding the marker on all of them was wrong in the one case it was meant to make safe. `beforeBuild` runs `rebuild-native-deps.mjs --platform=win32 --arch=<target>`, so a cross-arch slice normally does get a patched `build/Release` for the target; `prunePackagedNodePty` keeps the prebuild anyway because its guard is `electronArch === process.arch` rather than the arch of the binary. That package is correct and its leftover prebuild is never reached, and the sweep failed it -- telling whoever ran it to package on a Windows arm64 host, which is both the wrong remedy and one no runner here can offer. Presence cannot separate that package from the one whose cross-arch rebuild quietly emitted the host's architecture, because the only difference is the arch of `build/Release`. So the gate now resolves the addon the way `loadNativeModule` does -- first candidate whose PE `IMAGE_FILE_HEADER.Machine` matches the target, walking root-then-lib for each layout in node-pty's own order -- and checks the marker on the one that will actually run. A package with no candidate, or none of the target's architecture, is refused: it has no ConPTY backend either way, and the second is exactly what a silently host-arch cross-build looks like. The PE machine reader already existed, privately, in the relay addon builder that needed the same "a cross-build cannot silently emit host arch" guarantee. It is now shared rather than copied. Two seams were unreachable from anything but Windows, so nothing tested them: - the afterPack hook's win32 block was an inline if/else that only a source-text assertion could inspect, and that assertion could not tell the difference between the check running and the check being wrapped in `try {} catch {}`. It is now `verifyPackagedWindowsNodePty`, and "the marker check runs even where the export check cannot" is four spied assertions instead of a string match. - the rebuild path's verdict read `process` directly, so the branch that fires only on the host being rebuilt for was dead on every other host. It now takes the host as arguments, and the fs checks, the warning and the failure are all exercised from macOS. Fixtures write a real PE header rather than `MZ fake addon`, since the gate now reads one. The machine table is pinned to the documented IMAGE_FILE_MACHINE values, because every fixture builds its header from that table and a table wrong in both entries would otherwise agree with itself. * fix(windows): say why the packaged ConPTY fell back, not just that it did The previous commit resolved the addon by architecture but still had one message for every way the resolution could land on the published prebuild. Those ways want opposite remedies, and the one it printed was the remedy the commit before it had just called wrong: - no source build in the package at all — the slice has to be built somewhere that can build node-pty for the target arch. - a source build that is there but is the packaging host's architecture, because the cross-arch rebuild did not honour `--arch` — re-running that rebuild is the fix, and "package on a Windows arm64 host" is neither necessary nor possible. The second is the common one, since node-pty publishes a prebuild for both Windows arches and prune keeps the target's on every cross-arch package. So the old text fired mostly on the case it described least. It now reports which source builds were skipped and the machine field each carried, and names the rebuild command. "Nothing the target can load" had the same problem in reverse: a zero-length or truncated `conpty.node` got a cross-architecture diagnosis. Every candidate is now named with what was actually read, including "not a PE image". The rebuild path asserts the architecture too. A rebuild that ignored `--arch` was otherwise only visible at packaging, two steps from the command that fixes it. Arches with no known machine value are left unjudged rather than guessed at. Two things the extraction broke or nearly broke, both found by mutation: - the shared PE reader answers `null` where the relay builder's private copy returned a number, which would have turned its "node-gyp ignored --arch" error into a `TypeError`. Both callers now go through `describePeMachine`. - the rebuild fixtures stage a script's co-located modules by walking its imports, and the walker only understood `from '...'` — so the gate's new `require('./windows-pe-machine.cjs')` was left behind and every subprocess test failed with a resolution error, which is the exact failure its own comment warns about. It now follows `require` and bare side-effect `import` as well, and has tests; the fixture stages the gate by walking it rather than by naming one file. Fixtures write real PE headers through one shared builder instead of three hand-rolled ones. * fix(windows): run the node-pty addon gates on the Windows job that can `rebuild-native-deps-node-pty.test.mjs` carries four `skipIf(platform !== 'win32')` tests. The full suite runs on ubuntu, and the Windows PR job runs an explicit file list that never named this file -- so those tests were skipped on Linux and never reached anywhere else. Three of them predate this branch. The Windows job is added the four node-pty addon suites plus the module-walker one; the comment above that list already says why it is the right place, which is that the addon assertions only hold once natives have been rebuilt. Running the path-joining suites there also covers the separator this gate's candidate list is built from. The rest is round-three review: - the rebuild-time arch assertion told a reader "node-gyp did not honour --arch" about a file that was not a PE image at all, which is a truncated or quarantined artifact and a different command to run. The two now read differently, and neither claims the other's cause. Same fix the packaged gate had one commit ago, in the place that had not had it yet. - the missing-addon error said node-pty "would load" a prebuild without checking it is there. It says "fall through to" now, which is true either way. - `isLoadableByArch` had no caller left once the packaged gate started needing the raw machine field for its message. Removed rather than kept warm. - each candidate's header is read once instead of up to three times. - the module walker's comment claimed every shape that reaches a co-located module; it does not follow `projectRequire`/`requireLocal`, and it must not -- those specifiers resolve against the project root, so following one stages the wrong path and the copy fails. Proven by trying: widening the pattern to require-shaped names broke nine tests on `projectRequire('./config/scripts/...')`. The comment now says what it follows and why it stops there. - a new test resolved a file URL with `.pathname`, which keeps the drive-letter slash on Windows -- the very job this commit adds it to. * docs(windows): put the superseded export-only gate in the past tense It describes what used to pass a broken addon, so present tense reads as a description of the gate the same document then explains replacing it. * fix(windows): repair what running the node-pty suites on Windows exposed Putting these files on the Windows job turned four assertions red on the first run. Three of them were in tests that carried `skipIf(platform !== 'win32')` and had therefore never executed anywhere, on any branch. - `writeFakeElectronRebuild` emitted the `windows-process-tree` addon a real rebuild leaves but never node-pty's, so every Windows test of the rebuild path ran against a tree no real rebuild can produce: node-pty "rebuilt" with nothing in `build/Release`. The new same-host check reads that state correctly and said so. The fake rebuild now writes `build/Release/conpty.node` when it was asked to rebuild node-pty for win32, with the marker and the target machine. - `mkTempProject` never staged `windows-process-tree-creation-time.cjs`. The rebuild script reaches it through `projectRequire`, which resolves against the project root, so the module walker cannot follow it and must not try. Staged by name, with a comment saying which of the two it is. Without it the windows-process-tree probe failed to load its own checker and the module joined `modulesToRebuild`, which is the second and third red assertion. - the two `nodePtyAddonPath` cases compared against a literal POSIX string. `resolve` returns a drive letter and backslashes on Windows, so they could only ever pass off it. Built from segments now, which still pins the `..` traversal that is the point of the test. Verified on macOS: ensure-native-runtime-job-ownership, verify-packaged-node-pty-job-ownership, windows-pe-machine, script-module-dependencies, rebuild-native-deps-node-pty, rebuild-native-deps, rebuild-native-deps-windows-process-tree, ensure-native-runtime -- 109 passed, 6 skipped. The 6 are the Windows-gated rebuild tests, which is the job this change is aimed at; Windows CI is the arbiter. * fix(windows): give the packaged fallback a third verdict, for a file that is no image The packaged gate had two remedies for landing on the published prebuild and picked between them on `!prebuilt`, which puts a truncated, empty or quarantined `build/Release/conpty.node` in the cross-arch bucket: "the source build beside it is the wrong architecture ... re-run with --arch". It is not the wrong architecture, it is not an architecture, and `--arch` is not the command. The rebuild-path gate was split for exactly this a commit ago; this is the same split in the place that had not had it. Also from review of the settled state: - the stale-source-build branch ended in a call that happened to throw, so a reader could not see it was terminal and the file was read twice to get there. The verdict is now an Error the caller throws, built once from the read it already did, and shared with `assertCygwinBreakawayDenied` rather than copied. - four injection seams had no consumer in production or in tests (`deniesBreakaway`, `peMachine`, and `exists`/`peMachine` on the rebuild verdict). An unused seam is a way for the tested path and the real one to drift apart; the tests drive both with real files. Removed. - the loader table existed in a docblock and in the reference doc, already disagreeing about row four. The docblock cites the doc now. - `peImage` stamped machine `0x0000` for an arch it had no value for, because `writeUInt16LE(undefined)` coerces to zero. A fixture that quietly invents the field the gates read is the same species of silent lie the gates exist to catch; it throws, and a test holds it to that. - a test named for refusing an unreadable candidate asserted only that something threw. Renamed to what it proves. * fix(windows): make the rebuild fixtures represent a tree that can exist Second round of what running these suites on Windows exposed. The module the walker could not stage is now staged, so the probe reached its own checker and the real reasons surfaced: - `writeFakeWindowsProcessTree` exported `{}`. The creation-time gate reads `supportedProcessDataFlags` off the addon and calls its absence "the tarball prebuilt, not a build of the patched source" — correctly. The fixture predates that gate and, being Windows-only, never met it. The healthy fake now reports the flag, taken from the gate's own constant. Two tests were failing on this, the second only because the module then joined `modulesToRebuild`. - `rebuilds a loadable ConPTY native that lacks Orca job ownership` asked for a node-pty rebuild in a tree where node-pty had none of the payload its package ships. It gets `writeFakeNodePtyConptyPayload` like its two siblings. I also tried making the fake rebuild emit `build/Release/conpty.node` the way a real one does, and backed it out: `restoreNodePtyWindowsConptyRuntime` keys off that file and then reads `third_party/conpty`, so emitting it in a tree without the package payload turns one honest gap into an ENOENT two steps away. The payload fixture is where "node-pty has its addon" belongs. macOS: ensure-native-runtime-job-ownership, verify-packaged-node-pty-job-ownership, windows-pe-machine, script-module-dependencies, rebuild-native-deps-node-pty, rebuild-native-deps, rebuild-native-deps-windows-process-tree, ensure-native-runtime — 112 passed, 6 skipped. The 6 are the Windows-gated rebuild tests; Windows CI is the arbiter and is why they are on that job now. * fix(windows): register the node-pty addon suites in the scope list too Putting the five suites in the Windows lane's vitest argv gets them run once the job starts; `WINDOWS_PACKAGE_TESTS` in `pr-code-change-scope.mjs` is what decides whether the job starts at all. Only the argv was updated, so a PR touching just `rebuild-native-deps-node-pty.test.mjs` would not have started the Windows job, and its four Windows-only cases — including the same-host-absent one added here — would have run on no machine for that PR. Exactly the shape of gap this branch is about. Both lists now name all five, and `windows-pe-machine`, `windows-pe-image-fixture` and `script-module-dependencies` join `NATIVE_RUNTIME_PREFIXES` so a change to the modules themselves starts it too. `win32-test-lane-registration.test.mjs` exists to catch precisely this and did not, because its matcher only recognises suite-level gates (`describe.runIf` / `describe.skipIf`) and a `.win32.` filename. These tests gate per `it`. Widening it is not this branch's change to make: about thirty files across the repo carry per-`it` Windows gates and are unregistered, so the ratchet would move far beyond node-pty. Flagged rather than done. Message repairs from the same review: - the non-PE arm of the rebuild-time arch error read "... is not a PE image, so nothing can load it, so node-pty would fall back ...". The shared consequence clause already opens with ", so". - the no-source-build packaging error ended "Package this Windows slice on such a host", which is wrong advice for the case where the host IS such a host and the rebuild simply left nothing — reachable when the artifact is removed before prune runs. It now names both readings and points at the beforeBuild output. - the relay-addon builder blamed `--arch` for a build output that is not a PE at all, the same guess the node-pty gate was taught to stop making. - the patch-drift assertion was a bare `toBe(true)`, so a real drift read as "expected false to be true". It now names the two things that can have drifted and what happens until they agree. |
||
|
Neil
|
13ba649c22 |
fix(terminal): let a runtime-created Windows terminal BE the requested shell (#20825)
* fix(terminal): let a runtime-created Windows terminal BE the requested shell
`orca terminal create --environment <windows-host> --command 'cmd.exe'` never
created a cmd terminal. `--command` is text the provider TYPES into whatever
shell it spawned, so the PTY stayed the host's default shell with cmd running
inside it. Captured on `awin`, whose default is Git Bash:
$ orca terminal create --environment awin --command 'cmd.exe' --json
$ orca terminal send --environment awin --terminal term_10656cf7... \
--text exit --enter
$ orca terminal read --environment awin --terminal term_10656cf7... --screen
neil@awin MINGW64 ~/orca/orca ((30f820708f...))
$ cmd.exe
Microsoft Windows [Version 10.0.26200.9445]
C:\Users\neil\orca\orca>exit
neil@awin MINGW64 ~/orca/orca ((30f820708f...))
$
The handle is alive the whole time and `terminal list` shows one healthy
terminal, because the PTY never changed — so the only symptom is that the
caller's terminal is now a shell it never asked for, and every later `send` is
quoted for the wrong one. On `win-lowspec` (default pwsh) the same create lands
cmd inside PowerShell.
Root cause
----------
There are two spawn preflights and they are twins:
- `src/main/ipc/pty/ipc/spawn-preflight.ts` — renderer/IPC spawns, i.e. a
terminal tab opened in the app.
- `src/main/ipc/pty/runtime/spawn-preflight.ts` — runtime spawns: the CLI's
`terminal.create`, headless `orca serve`, and every paired remote
environment.
Only the IPC twin read the caller's requested shell. The runtime twin passed a
literal `requestedShellOverride: undefined`, so a runtime-created terminal on
Windows could only ever be the host default. Everything downstream of that
point — `spawn-options`, the daemon, `resolvePtyShellOverride` in the relay,
`local-pty-launch-plan` — already honoured `shellOverride`; nothing upstream
could supply one.
Change
------
- Thread `shellOverride` through the runtime lane: `RuntimePtySpawnArgs` ->
runtime `spawn-preflight` -> `RuntimePtyController.spawn` ->
`TerminalCreateOptions` -> the `terminal.create` RPC's new `shell` param ->
`orca terminal create --shell`.
- Thread it through the renderer-backed lane too (`createDesktopTerminal` ->
`terminal:requestTabCreate` -> `store.createTab`), so `--shell --focus` is not
silently dropped on a local Windows app.
- An agent launch quotes its startup command for the shell it will actually run
in, so a requested shell now owns the startup-shell family instead of the
global `terminalWindowsShell` setting.
- Lift the relay's `ALLOWED_WINDOWS_SHELL_OVERRIDES` into
`isSupportedWindowsShellOverride` in `src/shared/windows-terminal-shell.ts`
(membership unchanged) so the CLI, the zod param schema, and the relay refuse
the same names. `--shell` therefore cannot carry a path or a command line into
`pty.spawn`; only allowlisted bare shell names pass.
- Gate on `TERMINAL_CREATE_SHELL_SELECTION_RUNTIME_CAPABILITY`. An older host
strips the unknown `shell` param and answers with a healthy terminal running
its default shell — a reply indistinguishable from success — so the CLI
refuses before creating anything rather than creating the wrong shell quietly.
`--shell` stays Windows-only; macOS and Linux hosts spawn the login shell and
the relay drops the value off win32 rather than honouring it half-way. A WSL
project runtime still outranks it, unchanged.
Tests
-----
- `pty-spawn-shell-override-parity.test.ts` pins both preflights against the
exact drift that caused this (verified failing with the fix reverted).
- `createTerminal` passes `shellOverride` to `ptyController.spawn` with no
startup command.
- CLI: sends `shell`, refuses a shell the host cannot spawn, and refuses a host
without the capability — in both refusals without making the round trip.
- Allowlist and `terminal.create` schema accept/refuse cases, including paths
and appended arguments.
* fix(terminal): refuse a requested shell the execution host cannot apply
The first commit made `--shell` reach the spawn, but only a LOCAL win32
execution host applies it: `spawn-options` gates the override on
`process.platform === 'win32' && !args.connectionId`. So `--shell cmd.exe`
against an SSH-routed worktree, or against a macOS/Linux host, still returned a
healthy terminal running that host's default shell — the same
indistinguishable-from-success reply the capability gate exists to prevent, one
layer down.
Refuse instead, before anything spawns. The check sits at the top of
`resolveAgentTerminalCreateOptions`, which every create lane funnels through, so
neither lane has to remember it; the desktop lane additionally refuses a
worktree-less create, which has no execution host to resolve a shell on.
An SSH host's platform and installed shells are not visible to this runtime, and
a POSIX host has no Windows shell to pick. Neither can honour the request, and
saying so is the whole point of the flag.
Docs and the CLI spec now say "refused", not "ignored".
* fix(terminal): refuse a shell that contradicts the project execution runtime
`resolveLocalWindowsTerminalRuntimeOptions` does not merely rank the project's
execution runtime above a per-terminal pick -- it REWRITES the pick, in both
directions, and says nothing:
- a WSL project forces `wsl.exe`, discarding `--shell cmd.exe`;
- a Windows-host project discards a WSL name and falls back to `COMSPEC`
(`getHostShellForProjectRuntime`), so `--shell wsl.exe` spawns cmd. That is
the common case, not an edge: `resolveProjectExecutionRuntime` resolves
`windows-host` for every project that is not WSL, while a repo belonging to no
project honours `wsl.exe` -- so the same flag behaved differently depending on
whether the repo was in a project.
Either rewrite returns a healthy terminal running a shell the caller did not ask
for, which is the failure `--shell` exists to remove.
It also split an agent launch's quoting from the shell that receives it. The
previous commit made the startup-shell family follow the REQUESTED shell, so
`--shell wsl.exe --command codex` on a Windows-host project typed POSIX-quoted
launch args into cmd. Refusing the contradiction removes that case rather than
papering over it.
Refuse instead, alongside the SSH and non-Windows refusals, from the same
`resolveAgentTerminalCreateOptions` seam every create lane funnels through.
Also from review:
- the allowlist test looped the list against itself; spell the members out.
- the runtime spec case claimed to prove the pty's shell when it asserts the
controller received the field; name it for what it checks.
Reported by an adversarial review of the branch.
* fix(terminal): canonicalize --shell and refuse a WSL-path rewrite
Review of the --shell create path turned up two ways the terminal could
still end up being a shell the caller never asked for -- the exact failure
--shell exists to remove.
Bare and mixed-case spellings passed the allowlist but reached consumers
that exact-match the canonical name: resolveWindowsShellStartupFamily
classified `cmd` as the PowerShell family, resolveWindowsShellLaunchArgs
fell through to empty shellArgs (no `chcp 65001`, no OSC 133 bootstrap that
Windows foreground status depends on), and resolveWindowsGitBashShellPath
compares case-sensitively so `Git-Bash` spawned a literal `Git-Bash`.
The allowlist is now one canonical-name map and terminal.create canonicalizes
on parse, so the spawn path only ever sees `.exe` spellings. `pwsh` and
`powershell` stay distinct binaries.
A `\\wsl$\<distro>\...` cwd made the providers force wsl.exe regardless of
the request, and terminalShellOverrideRefusal only inspected the project
runtime -- undefined for a folder workspace with no project. Refuse on the
resolved cwd and the workspace path, judging what the PTY actually gets.
Also: the capability gate reported an unreachable host as too old rather
than unavailable; the SSH CLI shim dropped capabilities from status, so
--shell there blamed the host version instead of naming SSH; and --shell
had no help entry, rendering bare in `orca terminal create --help`. Adding
that entry crossed help.ts's max-lines cap, so the flag table moved to
flag-help-text.ts rather than suppressing the rule.
Adds a behavioural test for the runtime preflight (the one-line fix was
pinned only by a source-text scan), plus coverage for the startup-command
quoting family, the no-workspace refusal, and the WSL-path refusal.
* fix(build): keep tests out of the RPC params catalog bundle
The catalog walk under methods/ already skips *.test.ts, but the contract
directory glob took every .ts. terminal-create-shell-param.test.ts is the
first test to live there, so the bundle pulled vitest into a CJS build and
the generator threw on require(). Same exclusion, same reason.
|
||
|
Neil
|
11180fa532 |
chore(lint): add anti-slop oxlint plugin (pinned, all rules off) (#20726)
* chore(lint): add anti-slop oxlint plugin (all rules off) Vendors dmmulroy/anti-slop (MIT) plus no-call-only-assertions and no-pass-through-type-alias from maharshi365/deslop (MIT). Every rule starts "off"; each follow-up PR fixes one rule's violations and flips it to "error". * fix(lint): actually exclude the vendored plugin from the anti-slop audit oxlint does not honour ignorePatterns supplied via --config, so the config/oxlint-plugins/anti-slop/** entry never matched and the vendored rule source was being linted as first-party code (505 violations). Move the exclusion to the --ignore-pattern CLI flag in audit:anti-slop, which does work, and drop the entry that gave a false sense of coverage. Keeping vendored source unlinted matters because anti-slop is updated by three-way merge against the upstream snapshot; reformatting it locally would conflict on every update. * chore(lint): pin anti-slop instead of vendoring it; drop deslop Replaces the ~5k vendored lines with a git-pinned devDependency: oxlint-plugin-anti-slop: github:dmmulroy/anti-slop#c44ef22 anti-slop ships raw .ts with no build step, and Node refuses to type-strip anything under node_modules (ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING), so oxlint cannot load it from there -- which is why upstream says to vendor it. A postinstall step copies the pinned package's source to .anti-slop-plugin/ (gitignored), which Node will type-strip because it sits outside node_modules. Upgrading is now a SHA bump rather than a re-vendor and three-way merge. Verified byte-identical rule output to the vendored copy across all 16 rules that fire. Drops maharshi365/deslop and its two rules (no-call-only-assertions, no-pass-through-type-alias). It is not on npm either, so it would need a second git pin and copy step, and it is a 5-star single-maintainer repo that is itself a re-namespaced copy of anti-slop. One upstream is enough. * ci(lint): run audit:anti-slop in PR CI config/scripts/pr-workflow-lint-parity.test.mjs requires every step in `pnpm lint` to have a matching step in .github/workflows/pr.yml; adding audit:anti-slop to lint without the workflow step failed that ratchet. Also makes audit:anti-slop sync the plugin itself before linting. The generated .anti-slop-plugin/ directory is gitignored and otherwise only created by postinstall, so a cached install that skips postinstall would leave oxlint unable to load the plugin. |
||
|
Jinwoo Hong
|
1f7655f3e3 |
feat(ai-vault-search): public session search contract and transports (#20277)
* feat(ai-vault-search): define public contract and service seam * feat(ai-vault-search): add IPC runtime relay and web transports * fix(ai-vault-search): register search IPC at the core handler site ai-vault.ts was two lines over the 300-line max-lines limit; the search handlers belong with the other register*Handlers calls anyway. * fix(ai-vault-search): withhold degraded-root paths from relay status Status carried local filesystem paths over the relay while hits redact theirs. redactStatusForTransport applies the same policy at the same boundary: relay callers keep each root's reason and the array length as the count, so the type only makes root optional. * fix(ai-vault-search): close diagnostic path leak and remove test casts * feat(ai-vault-search): carry an execution host id and per-host outcomes on hits * feat(ai-vault-search): route desktop search by execution host scope, including runtimes * feat(preload): accept an execution host scope on session search * feat(web): answer only for the paired runtime on session search * docs(ai-vault-search): describe execution-host routing and the all-hosts merge * test(ai-vault-search): cover every host scope, the all-hosts merge and wire compat * fix(ai-vault-search): resume every host mid-page so a merged page never drops a hit * fix(ai-vault-search): decode the merged cursor with a schema instead of casts CI's type-aware audit refuses type assertions; a zod record validates the per-host entries and yields the typed map without one. * refactor(ai-vault-search): defer cross-host merged search |
||
|
Neil
|
e86cba888b |
build: reduce native dependency installs to the host platform (#20420)
* Reduce native dependency installs to the host platform * Remove install policy documentation * Guard cross-arch packaging and scope release installs to the runner electron-builder only logs a warning for a missing extraResources source, so a host-only install silently shipped a foreign-arch slice without its natives — `pnpm build:mac` on Apple Silicon produced an x64 DMG with no sherpa-onnx-darwin-x64 and no @parcel/watcher-darwin-x64. The previous beforePack hook covered only win32. - Add assertPackagedNativeVariantsInstalled, an arch-aware check over the target's sherpa-onnx, @parcel/watcher, and (on Windows) node-gyp addons. beforePack now runs it for every platform, with remedies split: another architecture comes from install:release, the os:win32 addons need a Windows host. - Drop --os from the release installs. Every packaging job already runs on a runner whose OS matches its target, so only the macOS lanes need extra breadth, and only on CPU for their x64+arm64 config. Windows and Linux packaging return to a plain host-only install. - Add --frozen-lockfile to install:release so a bare run cannot rewrite the lockfile. - Restore the install policy reference doc and the CONTRIBUTING note, plus the rationale comments dropped from the runtime contract test. - Gate the packaging-closure assertions on whether the Windows addons are installed rather than on the host OS, so a cross-arch install exercises them off Windows too. - Make the workflow contract test read `run:` steps as well as retry-action commands, and enforce host-only scoping on the non-macOS packaging lanes. - Remove the unreferenced install measurement script; its numbers live in the policy doc. * Track the install policy doc and index it from AGENTS.md docs/** is ignored behind a per-file allow-list, so the new reference doc was only committed via git add -f and future edits would be skipped. Add it to the allow-list and give it an AGENTS.md entry like every other tracked reference doc, so the host-only install rule is discoverable before someone packages a second architecture. * Route Windows-lane removals through the retrying helper Adding these four specs to the PR Windows lane pulled them into the windows-lane-tree-removal-boundary ratchet, which failed on 20 raw recursive removals. On Windows a bare rmSync races a handle the OS has not released, throwing EPERM after the assertions already passed and reporting a green test as a lane failure. * Adapt the packaging guard to the vendored Windows registry addon main vendored windows-native-registry as the workspace package @orca/windows-registry (#20438). A workspace link resolves on every host, so including it in the installed-Windows-addons checks proved nothing. @vscode/windows-process-tree is the only os: win32 npm addon left, so it alone decides whether the win32 resource plan resolves. |
||
|
Neil
|
5127d1eb3b |
refactor(windows): vendor the registry addon as @orca/windows-registry (#20438)
* refactor(windows): vendor the registry addon as @orca/windows-registry windows-native-registry@3.2.2 was last published in 2023 by a single maintainer. Orca called two of its exports, both read-only, so the whole dependency is replaced by a local N-API addon under native/. The vendored addon is read-only by construction: setValue, createKey and deleteKey are gone, so RegDeleteTreeW no longer ships in the app. Two upstream defects are also fixed rather than carried over — the name/data scratch buffers were file-scope statics that concurrent reads would scribble over, and createKey/deleteKey called .c_str() on a temporary. Build wiring keeps the existing shape: still an optionalDependency gated to win32, still excluded from pnpm's allowBuilds so only Orca's own Windows rebuild runs node-gyp for it, still copied into the packaged resources. The CI native caches now key on the vendored sources so an addon.cc edit cannot restore a stale .node. * test(windows): check the vendored registry addon against reg.exe The addon is vendored source, so no upstream release proves it still decodes values the way Orca's PATH readers expect. reg.exe is the only independent oracle on the box. * ci(windows): register the registry addon test on the Windows runner A Windows-gated file self-skips on ubuntu, so without both registrations it reports success while running on no machine at all. * fix(build): link the registry addon as a workspace package, not file: As a `file:` dependency pnpm re-resolved and re-linked the package on every install, including `--frozen-lockfile` (measured: "added 1" on a repeat no-op install). That virtual-store churn ran concurrently with node-gyp reading the same tree and cost @vscode/windows-process-tree its binding.gyp mid-rebuild, failing package (windows) whenever the native cache hit and only that module needed building. The linux packaging job hit the same race from the other side, as a pnpm staging move failure. A workspace link resolves once and leaves the store alone; repeat installs are now 55ms no-ops. native/windows-registry is listed explicitly so `packages:` still does not auto-discover mobile/. * fix(build): stop tracking node-gyp output for the vendored addon The build/ tree is generated per host and ABI; the committed copy was macOS-specific gyp scaffolding from a local build and would have shipped stale Makefiles to every checkout. * chore: ignore the vendored addon's node-gyp bin output too node-gyp also emits bin/<platform>-<abi>/ beside build/; both are per-host generated output that must never be committed. |
||
|
Jinwoo Hong
|
fa3d29fa11 |
feat(ai-vault-search): session search query engine over the index (#20029)
* feat(ai-vault-search): give the index a generation a page cursor can be fenced by
A search page is a slice of one ranked list, so a cursor only means anything
against the snapshot that produced it. The store now keeps a monotone
generation in `meta`, bumped by every mutation that can change which rows a
read returns, and starts a new one on open so a write whose bump never landed
cannot leave a cursor pointing at content that is already gone.
Schema version 2 adds the two tables the query layer needs: `messages_vocab`
(fts5vocab over messages_fts, the typo repair's whole dictionary) and
`search_log`. Both are pure additions and the index is a cache, so a version-1
file is dropped and rebuilt exactly as any other mismatch is.
* feat(ai-vault-search): plan a query the way the index tokenized it
The planner unfolds the tokenizer contract instead of asking SQLite: same
boundaries as `unicode61 tokenchars '_.-/+'`, pinned against real fts5vocab
output so a query can be planned without a round trip. It decides the route
ladder's first rung (literal shape), strips stop words from prose but never
from a literal, and fans an identifier out into its pieces for the OR fallback.
Typo repair uses the index's own vocabulary as its dictionary, so it can never
suggest a term the index does not hold. Its exact-match probe now joins
`visible_messages`: `messages_vocab` is a view over the FTS b-tree and still
lists staged and tombstoned terms, and the PR 2 read ratchet is right to
demand the join.
`splitAiVaultSearchQuery` is the index's reading of repo: / path:. It keeps
operator case, which the panel folds and cwd_key must not; a census test pins
the two parsers to the same answer about what is an operator until PR 7 moves
the panel onto this one.
* feat(ai-vault-search): narrow a search the way the sidebar keys a folder
Every caller-supplied narrowing in one place, so retrieval, the operator-only
page and the session load cannot drift apart: agents, an updated-at floor, the
retention cutoff, scope paths, and the repo: / path: operators.
Scope keying goes through PR 2's `cwdKey`, which is the sidebar's
`folderGroupKey` without its prefix, rather than the original branch's second
spelling. That drops the branch's WSL distro qualification, which PR 2 removed
on purpose, and it makes the filesystem root a key that already ends in a
separator, so the child-prefix range is built from the key rather than by
appending one; `//` sorts below every real child and would scope the root to
nothing.
Engine types land here too, under src/main and not src/shared: nothing in this
PR is a wire type, and PR 5 lifts what a caller may receive.
* feat(ai-vault-search): rank, page and answer a session search
`SessionSearchEngine.search()` over the PR 2 store: route ladder (phrase, AND,
typo repair, OR), BM25 weights per corpus, one hit per session, fork folding,
and a page.
- `scope` picks the corpus and the engine never second-guesses it.
`conversation` is user and assistant turns; `all` adds tool output and the
identifier shadow column. Switching corpus while typing is PR 7's policy; an
engine that widened on a miss would make a result impossible to reproduce
from its own request.
- Pagination is an offset into one ranked list, fenced by the index generation
and by a hash of everything that changes the ranking. A cursor from another
generation or another query is refused with a typed error rather than
silently re-run. Ranking breaks every tie by session id, because a cursor
indexes into that order and retrieval does not promise one.
- Snippets and source presence are paid for by the page, not the list. A
snippet past the per-hit ceiling is cut on a code point, never between `[[`
and its `]]`, and flagged on the hit.
- Source presence is read from the `files` table. No stat on the query path,
and no `missing`: only a proven deletion may claim one, and this read cannot
prove it.
- `SESSION_SEARCH_CANDIDATE_LIMIT_DEFAULT` is an option, not a constant, and
the result says when the limit was what cut the answer.
- The engine is the only caller of `store.warm()`, on first search.
Library only: no IPC, no settings, no Electron, nothing constructs it in
production.
* perf(ai-vault-search): measure what a query costs and what the candidate limit buys
Two corpora, because they answer different questions. The 10.5 MB / 40-session
corpus is what the scope split costs a reader: conversation is about 1.6x
faster at p50 and 3.4x at p95 than the full corpus, which is the argument for
the second FTS table being the one a keystroke can afford.
The candidate limit needs more sessions than the limit before it costs
anything, so it is swept over 2,500 one-turn transcripts with every session
matching. Limits are interleaved sample by sample: run back to back, the first
configuration pays for every page the OS cache had not seen and the ordering
alone moved p95 further than the limit did.
The doc says plainly what these numbers do not cover. They are cost, not
relevance; the MRR figures quoted beside the BM25 weights and the identifier
shadow column come from a shoot-out over real transcripts and cannot be
reproduced from this repository.
* refactor(ai-vault-search): key a page once per search, and report reachable pages honestly
The page key was hashed twice per search, once to decode the incoming cursor
and once to mint the outgoing one. The benchmark's reachable-page figure was
clamped against a constant that could never bind; it is the limit over the page
size and nothing else.
* test(ai-vault-search): pin the two engine seams nothing was holding
The warm wiring and the query-length cap were both written and neither was
observable. A spy pins that a search is what warms the store, and the cap is
pinned through the one input the planner's own term limit does not already
bound: a single enormous token, where the cut is what decides whether the term
matches the indexed one at all.
* fix(ai-vault-search): fence the generation against writers this process cannot see
The generation was cached in memory and moved only on this store's own writes,
and the bump itself was a read-then-write outside any transaction. Two handles
on one file is the normal case once PR 3 lands: the indexer writes in the
scanner child while an engine reads elsewhere. A reader would see that writer's
deletions while its own generation stood still, honour a stale cursor, and skip
a session; two writers could mint one generation for two snapshots.
`generation` now reads `meta` on every call, and the bump is a single
`ON CONFLICT DO UPDATE ... + 1` inside the transaction that makes the change.
That closes the crash hole the bump-on-open existed for, so opening a store no
longer invalidates anyone's cursor.
Only a change to what a read returns counts. Retiring a path the index never
held hides nothing, and the row deletes that drain a tombstone take away rows
that were already invisible: bumping there would refuse a cursor every 256 rows
and leave pagination unusable for as long as indexing ran.
Also drops redaction from the query log, following PR 2's decision to store
transcript content as written; the module it called no longer exists.
* fix(ai-vault-search): answer from a version-1 index, and keep a repaired literal whole
Two smaller findings.
An engine can be handed a connection to a version-1 file that another handle is
still answering from, and this PR is what first makes that reachable. It now
probes once for the two tables version 2 added and names what it cannot serve
on every result, instead of throwing at the first query that reaches for a
vocabulary that is not there. The route ladder simply skips its repair rung.
Which process may unlink and rebuild the index is PR 3b's decision and is not
solved here.
Typo repair re-planned the corrected query from scratch, and a corrected
spelling can read as prose even when what was typed was a literal:
`parseJsonn(the, data)` has the punctuation, `parsejson the data` does not, so
the re-plan dropped `the` as a stop word. The repaired query searched for less
than was asked and `repairedTerms` reported a body nobody typed. The re-plan is
now told what the original decided, because a repair changes spellings, not the
query's character.
`repairedTerms` is documented as the whole body the repaired plan ran, with the
index's own case-folded spelling for the terms it corrected.
* fix(ai-vault-search): make repo: and path: mean one thing in the list and the index
The engine said `repo:` and `path:` a second time, in SQL, and SQL cannot say
them. LIKE folds ASCII and nothing else, so `path:CAFÉ` missed `café`; the
engine searched `cwd_key` while the panel searches the working directory and
the transcript path, so `path:jsonl` matched every session in the panel and
none in the index; and the engine compared one path segment where the panel
compares the last two, so `repo:orca/session-search` missed. All four
reproduce in both directions.
So there is one definition now, not two that resemble each other.
`matchesAiVaultQueryOperators` moves into the shared filter module beside the
panel that already owned the semantics, the panel calls it, and the engine
applies it over the rows it retrieved. SQL keeps only what it can express
exactly: the `cwd_key` prefix range for `scopePaths`.
`parseVaultQuery` now parses through `splitAiVaultSearchQuery`, so one parser
decides what an operator is. Its existing tests pass unchanged; four degenerate
shapes do answer differently and are pinned as decisions rather than left to be
discovered.
Two consequences worth stating. The operators are conjunctive now, because that
is what the panel has always done, where the engine had been ORing within a
key. And the operator-only page walks newest sessions in bounded pages applying
the predicate, rather than taking one cut of the newest N and filtering it,
which would have answered `repo:x` with nothing on a busy index.
* docs(ai-vault-search): re-measure the query benchmark after the operator change
repo: and path: moved out of SQL, so the operator-only row measures something
different now and the note that it is a range seek was wrong. The rest of the
table is re-measured on an idle machine: the previous run's p95 column was
mostly contention, which is why conversation looked 3.4x faster at p95 rather
than the 1.7x it actually is.
Adds what this PR does not settle: which process may unlink and rebuild the
index is PR 3b's, and PR 4 is only the first thing that makes reading it
reachable.
* fix(ai-vault-search): stop reporting a search that gave up as a search that finished
The operator-only walk stops at a scan ceiling as well as at a full candidate
set, and only the first of those reached the result. A query whose one match
sat past the ceiling came back with no hits and truncated.candidates false,
which is the engine claiming there is nothing to find when what happened is
that it stopped looking. Retrieval now says why it stopped, because it is the
only layer that knows, and the count it used to return could not distinguish
the two cases.
The cursor fence stays as it is: any published read moves the generation, so an
outstanding cursor is refused, and that is what F11 asked for. What was wrong
was the claim next to the row-delete skip that pagination stays usable through
indexing. It does not, and the engine now says so. The rejection carries the
generation the cursor was minted in and the one the index is at, so a caller
can tell a moved index from a bad cursor and re-issue page one without showing
anyone an error.
The capability probe was nearly dead code, since every store opens through a
function that rebuilds a stale file. It is not dead, because two handles can be
open on one file, so the claim is corrected rather than the probe deleted. It
now runs per search: a verdict cached in the constructor is wrong in both
directions once another handle rebuilds the index.
Also says why the row-delete loop may skip the bump: those messages keep
batch_id NULL and stay in visible_messages, so what makes them unreachable is
their session's tombstone, and the read ratchet is what keeps every reader
joining the view that applies it.
* fix(ai-vault-search): restore the panel's reading of a quote that does not end a word
Unifying the two parsers changed panel behaviour on nine of twenty probed
shapes, not the three previously pinned. Six of the nine were regressions, all
from one rule: the shared parser refused a quoted span whose closing quote was
not followed by a space, so `"a b"c` and `repo:"a"b` became single terms
carrying their own quote characters, which match nothing.
The rule was justified as what stops the apostrophes in `it's a repo:orca
thing's` from swallowing the operator between them. It is not: a span only ever
opens at a token start, and the quote in `it's` is not at one. Dropping the
rule restores all six shapes to what the panel has always done and leaves that
protection intact.
Three changes remain and are kept because the old answer was worse in each: an
operator with an empty quoted value is dropped rather than filtering on `""`
and silently emptying the list, and a bare pair of quotes reads as an empty
term rather than as the two characters. Each is pinned with a test that says
which behaviour it is and why.
* fix(ai-vault-search): trim operator values, and report a query the engine had to cut
Three lows.
`repo:" "` survived as a term and matched no label, silently emptying the
list, which is the exact defect the empty-value drop exists to prevent wearing
different clothes. Operator values are trimmed, and a whitespace-only one drops
like an empty one. The substring matcher's copy of a free-text term is trimmed
too, so `" "` reads as the empty term already does; the span kept for FTS is
still the query verbatim.
Two caps upstream of retrieval fired silently: the planner searches at most 48
terms, and the engine cuts the query at 512 characters. A 56-term query whose
only match was the 56th came back with no hits and nothing truncated, which
claims there is nothing to find. `truncated.query` now says when either fired,
alongside the candidate and snippet flags that already did.
Every cursor refusal now carries the generation the index is at, which the
engine knows before it looks at the cursor, and the generation the cursor
claimed wherever that survived parsing. The doc says exactly when each is
present instead of leaving absence unexplained.
* refactor(ai-vault-search): read the tables the simplified index writes, and own the fence
PR 2 deleted the visibility views, the staging tables and the store's
generation, so this reads `sessions` and `messages` directly and carries the
three schema objects only a query needs — the vocabulary, the query log, and
the triggers that move the generation — as its own extension over the store's
schema.
The fence is now three triggers on `files`, because every transaction the store
opens that can change what a search returns writes that table and nothing else
does; retention's orphan drain is the one write path that touches neither, and
it is the one that must not bump. The triggers live in the file, so a writer in
another process moves the generation without knowing a reader exists.
A message row can now outlive its session row until the drain reaches it, so
the snippet read joins `sessions` and the typo repair asks for a live posting
instead of trusting the vocabulary's document count.
The engine takes a connection rather than a store: PR 2's store keeps its
connection private, and which process may open or rebuild the index file is
PR 3b's decision, not a query engine's.
* perf(ai-vault-search): price the second FTS table, and re-measure without warmup
Open decision 3. A column filter over `messages_fts` returns the identical
rowid set as `conversation_fts` — checked here per query rather than assumed —
so the table exists for latency alone. On a 105 MB corpus at both ends of the
tool-output band, the column-filtered form costs 1.16-1.42x at p95, against a
bar of 2x, so the recommendation is to delete it.
The shoot-out writes its own corpus because the answer turns on the one
property the shared generator fixes: how much of a transcript is tool output.
Half the tokens in that output are words the conversation also uses, which is
deliberately generous to the table under question.
The doc records the number that argues the other way. PR 2 priced the table at
about a quarter of the index on a corpus whose tool output is 56% of its
message text; on a tool-heavy one it is 6.7-11%, because `messages_fts` grows
with the tool text and the second table does not.
Page warmup is not re-added. The measurement behind it was on a 4 GB index,
removing it moves this corpus by less than the run-to-run spread, and a
cancellable background pass needs a lifecycle a query library does not have.
* test(ai-vault-search): pin that an append moves the generation a cursor is fenced by
* refactor(ai-vault-search): answer the conversation scope with a column filter
PR 2 deleted `conversation_fts` on the strength of this PR's shoot-out, so the
scope is a column filter over the one FTS table now. `ftsTableFor` is gone; a
scope is a pair of `scopedExpression` and `scopedWeights`, and the table name
no longer travels through the engine, the snippet builder or a hit.
The filter is parenthesised, and that is the whole of it: `{cols}: (a AND b)`
binds both terms, while `{cols}: a AND b` binds only the first and searches
tool output for the rest. A test drives an AND whose second term lives only in
tool output through both scopes.
The snippet keeps one guard, not two. Its column list and its expression were
each hiding the other's mistakes — a tool-only row was unreachable through
either — so the list is the same four columns for every scope and the scoped
expression is what makes a conversation snippet impossible to draw out of tool
output. Dropping it now leaks that row, which a test catches.
One behaviour the deleted table did not have, pinned rather than wished away:
bm25 normalises by the whole row's length and has no per-column length, so two
rows with identical prose score differently when one also holds tool output.
The rowid set is unchanged; the order within it can move.
Re-measured on the shipping schema. The conversation scope is 1.2-1.4x faster
than `all` at every rung, and the index is 57 MB rather than about 150 MB at
93% tool output, because a tool row is now capped at 3,072 characters.
* fix(ai-vault-search): repair a spelling inside the scope that will answer it
Typo repair read `messages_vocab` and probed `messages_fts` with no column
filter, so tool output decided whether a conversation-scoped query was
repaired, in both directions. A tool row carrying the misspelling made the
query look correctly spelled and suppressed the repair; a tool row carrying a
rare word became the suggestion, naming in `repairedTerms` a string from a
column the scope will never show. Both reproduced against a control index that
differs by exactly that one row.
The vocabulary proposes and a scoped count disposes. fts5vocab is per table and
cannot be column-filtered, so every decision that reaches the plan — already
spelled right, eligible, and which of two equally close candidates wins — now
comes from a `messages_fts MATCH` under the same filter retrieval uses, joined
to `sessions`.
That also takes the vocabulary's `doc` out of the ranking, which is the half of
the drain defect that belongs here: `doc` counts rows whose session a purge has
already cut loose, so reclaiming them changed which word a query was repaired
to. Candidates are ordered by term now, because the ordering decides which of
them survive the scan limit, and ties on similarity go to the more common word
counted live rather than to the vocabulary's number.
The cost is one bounded count per candidate examined, at most eight per prefix,
and only for a term the scope has no posting for at all.
* fix(ai-vault-search): fence the rows a purge reclaims after it cuts a session loose
Retention's second half deletes from `messages` alone and touched neither
`files` nor `sessions`, so it moved no generation. The argument was that those
rows answer nothing, which was true of retrieval and not of the engine: the
typo repair's dictionary is a view over the FTS b-tree and listed them, so a
drain running between two pages swapped the repair under a cursor that was
still honoured, and a search that had answered stopped answering.
The commit before this one fixes that at its source by counting live rows. It
does not make the drain provably inert — the vocabulary still decides which
candidates survive its scan limit, and reclaiming a term's last row moves where
that limit cuts — so the fence is what covers the rest.
A fourth trigger, on `messages`, with a `WHEN` clause that is the whole reason
it is affordable: a replace and a `removeFile` delete a session's rows while
its `sessions` row still stands, so neither fires, and both already bump
through `files`. Only the drain deletes a row whose session is gone.
The price is named rather than avoided: a cursor outstanding while a purge runs
is now refused once per batch, which `SessionSearchCursorError` reports as
`stale-generation` so a caller re-issues page one. The test that pinned the old
contract is replaced by one for the new one, and by one proving a replace still
does not fire it.
* fix(ai-vault-search): tell a highlight from a transcript that contains brackets
The snippet builder asked each of a row's four columns for a marked snippet and
showed the first whose text contained `[[`. Transcripts contain `[[`: a bash
`if [[ -f … ]]`, numpy's `[[1, 2], [3, 4]]`. A row matching only in tool output
was shown its user turn instead, with nothing highlighted in it, and the
any-column fallback an identifier-only match depends on was unreachable behind
the same collision.
Whether a column matched is now the difference between two renderings of the
same text: `snippet(…, MARK, MARK, …)` beside `snippet(…, '', '', …)`. Content
cannot forge a difference between those two, because it is the same content
either way.
The marks FTS5 inserts are private-use code points, rewritten to the public
`[[` and `]]` once, at the end. That is for the other decision that has to tell
a mark from content: the truncation refuses to cut between an open mark and its
close, and a transcript's own bracket used to move that cut.
* fix(ai-vault-search): cut a query on a code point and bind ids in batches
Two small ones from the review's not-routed list.
`query.slice(0, 512)` can land between the halves of a surrogate pair, leaving
a lone half that matches nothing and that a caller cannot echo back. The reader
already has `sliceAtCodeUnitLimit` for exactly this.
`loadSessions` bound one parameter per candidate id in a single statement. The
list is as long as the candidate limit, the tuning doc invites a host to raise
that limit, and SQLite's `SQLITE_MAX_VARIABLE_NUMBER` is 999 on builds older
than 3.32 — so one settings change away from `too many SQL variables`. Read in
batches of 500, leaving room for the filter's own bound values.
* docs(ai-vault-search): price the repair rung, and record what is left open
Typo repair is the one rung whose cost tracks the vocabulary rather than the
result, and it only runs for a term the scope has no posting for. Measured over
1.6 M distinct terms: 10 ms for one unknown term, 387 ms for a 480-character
query of thirty-nine of them.
The scoped-count fix made that cheaper rather than dearer, from 737 ms, because
ordering the vocabulary scan by term drops the sort `doc DESC` needed and the
counts it adds are at most eight bounded probes per prefix. A cap on unknown
terms per query is a follow-up in the split plan, with the five other items the
final review raised and did not route.
* test(ai-vault-search): make each snippet mark mechanism answer for itself
Two mechanisms landed together and hid each other: choosing a column by
comparing a marked rendering against an unmarked one, and marking with
private-use code points instead of `[[`. Either alone fixed the bracket repro,
so neither had a mutation against it — the same masking the snippet's two
column guards had a round ago.
They do different jobs, so both stay and each gets the test that needs it. A
transcript holding a private-use code point of its own is what the comparison
is for; agent output carries Nerd Font glyphs from that block. A snippet past
the character ceiling with a bracket after its last real mark is what the
private-use marks are for, because the truncation has to find that mark by
searching the text.
The two one-line fixes get honest framing rather than a mutation neither can
have. A lone surrogate is not a token character, so the planner drops it either
way and the safe cut is hygiene. And no SQLite this stack runs refuses 1,100
bound ids — 32,766 has been the floor since 3.32 — so the batch is about
owning the ceiling here rather than rescuing a reachable failure.
* refactor(ai-vault-search): keep the scope's expression with the other expressions
Making the typo repair ask its questions in the search's own scope put an
import from retrieval into it, and retrieval already owns the repair — a cycle
the native audit catches. `scopedExpression` belongs beside `phraseExpression`,
`andExpression` and `orExpression` anyway: it builds a MATCH expression, and
two callers now need it. The bm25 weights stay in retrieval, where the SQL that
uses them is.
* docs(ai-vault-search): say which delete paths fire the orphan-reclaim trigger after a replace cuts loose
* test(ai-vault-search): make the trigger-restore test exercise the trigger it drops
* fix(ai-vault-search): keep a scope nothing could key from widening the search
* fix(ai-vault-search): read a fractional or negative cursor generation as malformed
* fix(ai-vault-search): highlight only the marks FTS5 inserted, not the text's own
* test(ai-vault-search): compare the whole query, so a quoted operator value survives
* fix(ai-vault-search): filter routes and fence page reads; simplify query engine
* refactor(ai-vault-search): narrow retrieval API and clarify page rejection
|
||
|
Neil
|
6bb2b0c6d7 |
test(runtime): capture real Antigravity transcripts — the detector is inverted on live output (#19983)
* test(runtime): capture real agent PTY transcripts before rewriting Antigravity readiness
Antigravity readiness has been written five times against a five-line screen
typed from memory. There is no Antigravity transcript in this repository, so
every attempt was a guess tested against another guess. This adds the recorder,
the protocol and the fixture-driven suite so the sixth attempt can be written
against evidence, and changes no detector logic.
- config/scripts/capture-agent-pty-transcript.mjs records a live agent session
through a real PTY, escapes and wrapping intact. Ctrl-] is consumed by the
recorder and never forwarded, which is the only way to end a capture while a
dialog still owns the screen.
- config/scripts/pty-transcript-secret-scan.mjs finds account identifiers and
credentials, redacts them with same-length placeholders so wrapping survives,
and recognises its own placeholders so a scrubbed file verifies clean.
- src/main/runtime/antigravity-readiness-transcripts.test.ts asserts a verdict
per transcript and skips by name until the transcripts land, with a
doc-coverage ratchet and a guard that a fixture contains escape bytes.
The escape-byte guard exists because the three cursor-agent fixtures carry a
comment claiming they were captured verbatim through Orca, yet contain zero ESC
bytes and zero carriage returns. That comment is corrected here to say what
those files are; the fixtures and the rules built on them are untouched.
* test(runtime): capture real Antigravity transcripts, and pin what they prove
`agy` 1.1.25 turned out to be installed, so the transcripts this scaffold was
built for now exist. Six are recorded from live sessions and committed; the
rest are named as skipped, because reaching them would mean signing the
operator out or deleting their config.
The captures invert the story. On real output the shipped detector refuses a
genuinely ready screen and accepts a live `/model` picker:
- Antigravity paints a block-glyph logo down the left, so the model row never
starts a line. `startsWith('gemini', trimmedStart)` cannot match a real ready
screen, on any account or model. Stripping the logo flips the same screen to
ready, which means a decorative glyph decides readiness today.
- The `/model` picker prints `Gemini 3.x Flash` one per line, at line start, and
a bare `>` composer sits earlier in the tail. Both halves of the rule are
satisfied while a dialog owns the screen.
- For an API-key user the identity row reads `Gemini API key` — no `@`, no
domain — and `AGY_CLI_HIDE_ACCOUNT_INFO=1` removes the row entirely. The
account-row requirement of attempts 4 and 5 can never pass for those users.
- The banner is printed once and never reprinted after a dialog is dismissed, so
`headerIndex` cannot be the ordering anchor.
Four suite cases are pinned as KNOWN DEFECT: they assert what the detector does
so CI stays honest instead of permanently red, and flip to failing the moment
someone fixes it. No detector logic changed.
The recorder gains `--send "<ms>:<text>"` because a dialog capture has to be
driven and an unattended run has no TTY, and the scrub scanner gains a UUID rule
because agy prints a resumable conversation id on exit.
* test(runtime): capture agy mid-turn, and make the scan file reviewable
Answers the busy-frame question a P1 review raised against attempt six, with
two new captures from a live turn.
At the frame level the review is right: a busy frame parks the caret with the
same bytes as an idle one, `CR ESC[2A ESC[2C`, and the only differing row —
`esc to cancel` versus `? for shortcuts` — is erased by that park.
At the retained-tail level it does not reproduce. Each spinner tick is its own
repaint with its own `CR ESC[2A`, two rows higher than the frame's, which
splices the composer away: a live turn's tail ends on `⣟ Generating...`, with
no bare caret to match. A constructed input that keeps the park and edits only
the status text is not faithful, because a live turn has a spinner row
repainting below the composer.
The residual is the gap between a frame park and the next tick, where the tail
does end on the bare caret. Quiescence-gated paths are safe there because ticks
keep arriving; text-only paths are not, and for those the capture supports one
clause: a braille glyph on the last visible line means working. That predicate
already exists here for cursor-agent and should be reused, scoped to the last
line — a first-run transcript prints `⠾ Signing in...` during startup.
Also in this commit, from the same review:
- pty-transcript-secret-scan.mjs held raw 0x00-0x1f bytes in a character class,
so the one file gating real PTY data into history was binary to git and
unreviewable in a diff. It now tests codepoints, which the formatter cannot
fold back into control bytes.
- Pin `src/main/runtime/__fixtures__/*.txt` as -text. A Windows checkout would
otherwise normalise line endings and rewrite the CR bytes that make these
files evidence.
The recorder now stops appending at the stop moment rather than through
shutdown: an agent repaints an idle frame on its way out, which was overwriting
the mid-turn state the capture existed to record.
* test(tooling): allowlist the transcript scan test in the batch-shim ratchet
pty-transcript-secret-scan.test.mjs asserts that the capture recorder routes
an 'agy.cmd' shim through cmd.exe, so the shim literal it names is the
assertion, not a spawn. Fits the existing assert-on-shim-files category.
|
||
|
Jinwoo Hong
|
f31bfa8fb7 |
Revert "feat(ai-vault-search): session search query engine over the index (#19750)" (#20023)
This reverts commit
|
||
|
Jinwoo Hong
|
cf7408a37f |
feat(ai-vault-search): session search query engine over the index (#19750)
* feat(ai-vault-search): give the index a generation a page cursor can be fenced by
A search page is a slice of one ranked list, so a cursor only means anything
against the snapshot that produced it. The store now keeps a monotone
generation in `meta`, bumped by every mutation that can change which rows a
read returns, and starts a new one on open so a write whose bump never landed
cannot leave a cursor pointing at content that is already gone.
Schema version 2 adds the two tables the query layer needs: `messages_vocab`
(fts5vocab over messages_fts, the typo repair's whole dictionary) and
`search_log`. Both are pure additions and the index is a cache, so a version-1
file is dropped and rebuilt exactly as any other mismatch is.
* feat(ai-vault-search): plan a query the way the index tokenized it
The planner unfolds the tokenizer contract instead of asking SQLite: same
boundaries as `unicode61 tokenchars '_.-/+'`, pinned against real fts5vocab
output so a query can be planned without a round trip. It decides the route
ladder's first rung (literal shape), strips stop words from prose but never
from a literal, and fans an identifier out into its pieces for the OR fallback.
Typo repair uses the index's own vocabulary as its dictionary, so it can never
suggest a term the index does not hold. Its exact-match probe now joins
`visible_messages`: `messages_vocab` is a view over the FTS b-tree and still
lists staged and tombstoned terms, and the PR 2 read ratchet is right to
demand the join.
`splitAiVaultSearchQuery` is the index's reading of repo: / path:. It keeps
operator case, which the panel folds and cwd_key must not; a census test pins
the two parsers to the same answer about what is an operator until PR 7 moves
the panel onto this one.
* feat(ai-vault-search): narrow a search the way the sidebar keys a folder
Every caller-supplied narrowing in one place, so retrieval, the operator-only
page and the session load cannot drift apart: agents, an updated-at floor, the
retention cutoff, scope paths, and the repo: / path: operators.
Scope keying goes through PR 2's `cwdKey`, which is the sidebar's
`folderGroupKey` without its prefix, rather than the original branch's second
spelling. That drops the branch's WSL distro qualification, which PR 2 removed
on purpose, and it makes the filesystem root a key that already ends in a
separator, so the child-prefix range is built from the key rather than by
appending one; `//` sorts below every real child and would scope the root to
nothing.
Engine types land here too, under src/main and not src/shared: nothing in this
PR is a wire type, and PR 5 lifts what a caller may receive.
* feat(ai-vault-search): rank, page and answer a session search
`SessionSearchEngine.search()` over the PR 2 store: route ladder (phrase, AND,
typo repair, OR), BM25 weights per corpus, one hit per session, fork folding,
and a page.
- `scope` picks the corpus and the engine never second-guesses it.
`conversation` is user and assistant turns; `all` adds tool output and the
identifier shadow column. Switching corpus while typing is PR 7's policy; an
engine that widened on a miss would make a result impossible to reproduce
from its own request.
- Pagination is an offset into one ranked list, fenced by the index generation
and by a hash of everything that changes the ranking. A cursor from another
generation or another query is refused with a typed error rather than
silently re-run. Ranking breaks every tie by session id, because a cursor
indexes into that order and retrieval does not promise one.
- Snippets and source presence are paid for by the page, not the list. A
snippet past the per-hit ceiling is cut on a code point, never between `[[`
and its `]]`, and flagged on the hit.
- Source presence is read from the `files` table. No stat on the query path,
and no `missing`: only a proven deletion may claim one, and this read cannot
prove it.
- `SESSION_SEARCH_CANDIDATE_LIMIT_DEFAULT` is an option, not a constant, and
the result says when the limit was what cut the answer.
- The engine is the only caller of `store.warm()`, on first search.
Library only: no IPC, no settings, no Electron, nothing constructs it in
production.
* perf(ai-vault-search): measure what a query costs and what the candidate limit buys
Two corpora, because they answer different questions. The 10.5 MB / 40-session
corpus is what the scope split costs a reader: conversation is about 1.6x
faster at p50 and 3.4x at p95 than the full corpus, which is the argument for
the second FTS table being the one a keystroke can afford.
The candidate limit needs more sessions than the limit before it costs
anything, so it is swept over 2,500 one-turn transcripts with every session
matching. Limits are interleaved sample by sample: run back to back, the first
configuration pays for every page the OS cache had not seen and the ordering
alone moved p95 further than the limit did.
The doc says plainly what these numbers do not cover. They are cost, not
relevance; the MRR figures quoted beside the BM25 weights and the identifier
shadow column come from a shoot-out over real transcripts and cannot be
reproduced from this repository.
* refactor(ai-vault-search): key a page once per search, and report reachable pages honestly
The page key was hashed twice per search, once to decode the incoming cursor
and once to mint the outgoing one. The benchmark's reachable-page figure was
clamped against a constant that could never bind; it is the limit over the page
size and nothing else.
* test(ai-vault-search): pin the two engine seams nothing was holding
The warm wiring and the query-length cap were both written and neither was
observable. A spy pins that a search is what warms the store, and the cap is
pinned through the one input the planner's own term limit does not already
bound: a single enormous token, where the cut is what decides whether the term
matches the indexed one at all.
* fix(ai-vault-search): fence the generation against writers this process cannot see
The generation was cached in memory and moved only on this store's own writes,
and the bump itself was a read-then-write outside any transaction. Two handles
on one file is the normal case once PR 3 lands: the indexer writes in the
scanner child while an engine reads elsewhere. A reader would see that writer's
deletions while its own generation stood still, honour a stale cursor, and skip
a session; two writers could mint one generation for two snapshots.
`generation` now reads `meta` on every call, and the bump is a single
`ON CONFLICT DO UPDATE ... + 1` inside the transaction that makes the change.
That closes the crash hole the bump-on-open existed for, so opening a store no
longer invalidates anyone's cursor.
Only a change to what a read returns counts. Retiring a path the index never
held hides nothing, and the row deletes that drain a tombstone take away rows
that were already invisible: bumping there would refuse a cursor every 256 rows
and leave pagination unusable for as long as indexing ran.
Also drops redaction from the query log, following PR 2's decision to store
transcript content as written; the module it called no longer exists.
* fix(ai-vault-search): answer from a version-1 index, and keep a repaired literal whole
Two smaller findings.
An engine can be handed a connection to a version-1 file that another handle is
still answering from, and this PR is what first makes that reachable. It now
probes once for the two tables version 2 added and names what it cannot serve
on every result, instead of throwing at the first query that reaches for a
vocabulary that is not there. The route ladder simply skips its repair rung.
Which process may unlink and rebuild the index is PR 3b's decision and is not
solved here.
Typo repair re-planned the corrected query from scratch, and a corrected
spelling can read as prose even when what was typed was a literal:
`parseJsonn(the, data)` has the punctuation, `parsejson the data` does not, so
the re-plan dropped `the` as a stop word. The repaired query searched for less
than was asked and `repairedTerms` reported a body nobody typed. The re-plan is
now told what the original decided, because a repair changes spellings, not the
query's character.
`repairedTerms` is documented as the whole body the repaired plan ran, with the
index's own case-folded spelling for the terms it corrected.
* fix(ai-vault-search): make repo: and path: mean one thing in the list and the index
The engine said `repo:` and `path:` a second time, in SQL, and SQL cannot say
them. LIKE folds ASCII and nothing else, so `path:CAFÉ` missed `café`; the
engine searched `cwd_key` while the panel searches the working directory and
the transcript path, so `path:jsonl` matched every session in the panel and
none in the index; and the engine compared one path segment where the panel
compares the last two, so `repo:orca/session-search` missed. All four
reproduce in both directions.
So there is one definition now, not two that resemble each other.
`matchesAiVaultQueryOperators` moves into the shared filter module beside the
panel that already owned the semantics, the panel calls it, and the engine
applies it over the rows it retrieved. SQL keeps only what it can express
exactly: the `cwd_key` prefix range for `scopePaths`.
`parseVaultQuery` now parses through `splitAiVaultSearchQuery`, so one parser
decides what an operator is. Its existing tests pass unchanged; four degenerate
shapes do answer differently and are pinned as decisions rather than left to be
discovered.
Two consequences worth stating. The operators are conjunctive now, because that
is what the panel has always done, where the engine had been ORing within a
key. And the operator-only page walks newest sessions in bounded pages applying
the predicate, rather than taking one cut of the newest N and filtering it,
which would have answered `repo:x` with nothing on a busy index.
* docs(ai-vault-search): re-measure the query benchmark after the operator change
repo: and path: moved out of SQL, so the operator-only row measures something
different now and the note that it is a range seek was wrong. The rest of the
table is re-measured on an idle machine: the previous run's p95 column was
mostly contention, which is why conversation looked 3.4x faster at p95 rather
than the 1.7x it actually is.
Adds what this PR does not settle: which process may unlink and rebuild the
index is PR 3b's, and PR 4 is only the first thing that makes reading it
reachable.
* fix(ai-vault-search): stop reporting a search that gave up as a search that finished
The operator-only walk stops at a scan ceiling as well as at a full candidate
set, and only the first of those reached the result. A query whose one match
sat past the ceiling came back with no hits and truncated.candidates false,
which is the engine claiming there is nothing to find when what happened is
that it stopped looking. Retrieval now says why it stopped, because it is the
only layer that knows, and the count it used to return could not distinguish
the two cases.
The cursor fence stays as it is: any published read moves the generation, so an
outstanding cursor is refused, and that is what F11 asked for. What was wrong
was the claim next to the row-delete skip that pagination stays usable through
indexing. It does not, and the engine now says so. The rejection carries the
generation the cursor was minted in and the one the index is at, so a caller
can tell a moved index from a bad cursor and re-issue page one without showing
anyone an error.
The capability probe was nearly dead code, since every store opens through a
function that rebuilds a stale file. It is not dead, because two handles can be
open on one file, so the claim is corrected rather than the probe deleted. It
now runs per search: a verdict cached in the constructor is wrong in both
directions once another handle rebuilds the index.
Also says why the row-delete loop may skip the bump: those messages keep
batch_id NULL and stay in visible_messages, so what makes them unreachable is
their session's tombstone, and the read ratchet is what keeps every reader
joining the view that applies it.
* fix(ai-vault-search): restore the panel's reading of a quote that does not end a word
Unifying the two parsers changed panel behaviour on nine of twenty probed
shapes, not the three previously pinned. Six of the nine were regressions, all
from one rule: the shared parser refused a quoted span whose closing quote was
not followed by a space, so `"a b"c` and `repo:"a"b` became single terms
carrying their own quote characters, which match nothing.
The rule was justified as what stops the apostrophes in `it's a repo:orca
thing's` from swallowing the operator between them. It is not: a span only ever
opens at a token start, and the quote in `it's` is not at one. Dropping the
rule restores all six shapes to what the panel has always done and leaves that
protection intact.
Three changes remain and are kept because the old answer was worse in each: an
operator with an empty quoted value is dropped rather than filtering on `""`
and silently emptying the list, and a bare pair of quotes reads as an empty
term rather than as the two characters. Each is pinned with a test that says
which behaviour it is and why.
* fix(ai-vault-search): trim operator values, and report a query the engine had to cut
Three lows.
`repo:" "` survived as a term and matched no label, silently emptying the
list, which is the exact defect the empty-value drop exists to prevent wearing
different clothes. Operator values are trimmed, and a whitespace-only one drops
like an empty one. The substring matcher's copy of a free-text term is trimmed
too, so `" "` reads as the empty term already does; the span kept for FTS is
still the query verbatim.
Two caps upstream of retrieval fired silently: the planner searches at most 48
terms, and the engine cuts the query at 512 characters. A 56-term query whose
only match was the 56th came back with no hits and nothing truncated, which
claims there is nothing to find. `truncated.query` now says when either fired,
alongside the candidate and snippet flags that already did.
Every cursor refusal now carries the generation the index is at, which the
engine knows before it looks at the cursor, and the generation the cursor
claimed wherever that survived parsing. The doc says exactly when each is
present instead of leaving absence unexplained.
* refactor(ai-vault-search): read the tables the simplified index writes, and own the fence
PR 2 deleted the visibility views, the staging tables and the store's
generation, so this reads `sessions` and `messages` directly and carries the
three schema objects only a query needs — the vocabulary, the query log, and
the triggers that move the generation — as its own extension over the store's
schema.
The fence is now three triggers on `files`, because every transaction the store
opens that can change what a search returns writes that table and nothing else
does; retention's orphan drain is the one write path that touches neither, and
it is the one that must not bump. The triggers live in the file, so a writer in
another process moves the generation without knowing a reader exists.
A message row can now outlive its session row until the drain reaches it, so
the snippet read joins `sessions` and the typo repair asks for a live posting
instead of trusting the vocabulary's document count.
The engine takes a connection rather than a store: PR 2's store keeps its
connection private, and which process may open or rebuild the index file is
PR 3b's decision, not a query engine's.
* perf(ai-vault-search): price the second FTS table, and re-measure without warmup
Open decision 3. A column filter over `messages_fts` returns the identical
rowid set as `conversation_fts` — checked here per query rather than assumed —
so the table exists for latency alone. On a 105 MB corpus at both ends of the
tool-output band, the column-filtered form costs 1.16-1.42x at p95, against a
bar of 2x, so the recommendation is to delete it.
The shoot-out writes its own corpus because the answer turns on the one
property the shared generator fixes: how much of a transcript is tool output.
Half the tokens in that output are words the conversation also uses, which is
deliberately generous to the table under question.
The doc records the number that argues the other way. PR 2 priced the table at
about a quarter of the index on a corpus whose tool output is 56% of its
message text; on a tool-heavy one it is 6.7-11%, because `messages_fts` grows
with the tool text and the second table does not.
Page warmup is not re-added. The measurement behind it was on a 4 GB index,
removing it moves this corpus by less than the run-to-run spread, and a
cancellable background pass needs a lifecycle a query library does not have.
* test(ai-vault-search): pin that an append moves the generation a cursor is fenced by
* refactor(ai-vault-search): answer the conversation scope with a column filter
PR 2 deleted `conversation_fts` on the strength of this PR's shoot-out, so the
scope is a column filter over the one FTS table now. `ftsTableFor` is gone; a
scope is a pair of `scopedExpression` and `scopedWeights`, and the table name
no longer travels through the engine, the snippet builder or a hit.
The filter is parenthesised, and that is the whole of it: `{cols}: (a AND b)`
binds both terms, while `{cols}: a AND b` binds only the first and searches
tool output for the rest. A test drives an AND whose second term lives only in
tool output through both scopes.
The snippet keeps one guard, not two. Its column list and its expression were
each hiding the other's mistakes — a tool-only row was unreachable through
either — so the list is the same four columns for every scope and the scoped
expression is what makes a conversation snippet impossible to draw out of tool
output. Dropping it now leaks that row, which a test catches.
One behaviour the deleted table did not have, pinned rather than wished away:
bm25 normalises by the whole row's length and has no per-column length, so two
rows with identical prose score differently when one also holds tool output.
The rowid set is unchanged; the order within it can move.
Re-measured on the shipping schema. The conversation scope is 1.2-1.4x faster
than `all` at every rung, and the index is 57 MB rather than about 150 MB at
93% tool output, because a tool row is now capped at 3,072 characters.
* fix(ai-vault-search): repair a spelling inside the scope that will answer it
Typo repair read `messages_vocab` and probed `messages_fts` with no column
filter, so tool output decided whether a conversation-scoped query was
repaired, in both directions. A tool row carrying the misspelling made the
query look correctly spelled and suppressed the repair; a tool row carrying a
rare word became the suggestion, naming in `repairedTerms` a string from a
column the scope will never show. Both reproduced against a control index that
differs by exactly that one row.
The vocabulary proposes and a scoped count disposes. fts5vocab is per table and
cannot be column-filtered, so every decision that reaches the plan — already
spelled right, eligible, and which of two equally close candidates wins — now
comes from a `messages_fts MATCH` under the same filter retrieval uses, joined
to `sessions`.
That also takes the vocabulary's `doc` out of the ranking, which is the half of
the drain defect that belongs here: `doc` counts rows whose session a purge has
already cut loose, so reclaiming them changed which word a query was repaired
to. Candidates are ordered by term now, because the ordering decides which of
them survive the scan limit, and ties on similarity go to the more common word
counted live rather than to the vocabulary's number.
The cost is one bounded count per candidate examined, at most eight per prefix,
and only for a term the scope has no posting for at all.
* fix(ai-vault-search): fence the rows a purge reclaims after it cuts a session loose
Retention's second half deletes from `messages` alone and touched neither
`files` nor `sessions`, so it moved no generation. The argument was that those
rows answer nothing, which was true of retrieval and not of the engine: the
typo repair's dictionary is a view over the FTS b-tree and listed them, so a
drain running between two pages swapped the repair under a cursor that was
still honoured, and a search that had answered stopped answering.
The commit before this one fixes that at its source by counting live rows. It
does not make the drain provably inert — the vocabulary still decides which
candidates survive its scan limit, and reclaiming a term's last row moves where
that limit cuts — so the fence is what covers the rest.
A fourth trigger, on `messages`, with a `WHEN` clause that is the whole reason
it is affordable: a replace and a `removeFile` delete a session's rows while
its `sessions` row still stands, so neither fires, and both already bump
through `files`. Only the drain deletes a row whose session is gone.
The price is named rather than avoided: a cursor outstanding while a purge runs
is now refused once per batch, which `SessionSearchCursorError` reports as
`stale-generation` so a caller re-issues page one. The test that pinned the old
contract is replaced by one for the new one, and by one proving a replace still
does not fire it.
* fix(ai-vault-search): tell a highlight from a transcript that contains brackets
The snippet builder asked each of a row's four columns for a marked snippet and
showed the first whose text contained `[[`. Transcripts contain `[[`: a bash
`if [[ -f … ]]`, numpy's `[[1, 2], [3, 4]]`. A row matching only in tool output
was shown its user turn instead, with nothing highlighted in it, and the
any-column fallback an identifier-only match depends on was unreachable behind
the same collision.
Whether a column matched is now the difference between two renderings of the
same text: `snippet(…, MARK, MARK, …)` beside `snippet(…, '', '', …)`. Content
cannot forge a difference between those two, because it is the same content
either way.
The marks FTS5 inserts are private-use code points, rewritten to the public
`[[` and `]]` once, at the end. That is for the other decision that has to tell
a mark from content: the truncation refuses to cut between an open mark and its
close, and a transcript's own bracket used to move that cut.
* fix(ai-vault-search): cut a query on a code point and bind ids in batches
Two small ones from the review's not-routed list.
`query.slice(0, 512)` can land between the halves of a surrogate pair, leaving
a lone half that matches nothing and that a caller cannot echo back. The reader
already has `sliceAtCodeUnitLimit` for exactly this.
`loadSessions` bound one parameter per candidate id in a single statement. The
list is as long as the candidate limit, the tuning doc invites a host to raise
that limit, and SQLite's `SQLITE_MAX_VARIABLE_NUMBER` is 999 on builds older
than 3.32 — so one settings change away from `too many SQL variables`. Read in
batches of 500, leaving room for the filter's own bound values.
* docs(ai-vault-search): price the repair rung, and record what is left open
Typo repair is the one rung whose cost tracks the vocabulary rather than the
result, and it only runs for a term the scope has no posting for. Measured over
1.6 M distinct terms: 10 ms for one unknown term, 387 ms for a 480-character
query of thirty-nine of them.
The scoped-count fix made that cheaper rather than dearer, from 737 ms, because
ordering the vocabulary scan by term drops the sort `doc DESC` needed and the
counts it adds are at most eight bounded probes per prefix. A cap on unknown
terms per query is a follow-up in the split plan, with the five other items the
final review raised and did not route.
* test(ai-vault-search): make each snippet mark mechanism answer for itself
Two mechanisms landed together and hid each other: choosing a column by
comparing a marked rendering against an unmarked one, and marking with
private-use code points instead of `[[`. Either alone fixed the bracket repro,
so neither had a mutation against it — the same masking the snippet's two
column guards had a round ago.
They do different jobs, so both stay and each gets the test that needs it. A
transcript holding a private-use code point of its own is what the comparison
is for; agent output carries Nerd Font glyphs from that block. A snippet past
the character ceiling with a bracket after its last real mark is what the
private-use marks are for, because the truncation has to find that mark by
searching the text.
The two one-line fixes get honest framing rather than a mutation neither can
have. A lone surrogate is not a token character, so the planner drops it either
way and the safe cut is hygiene. And no SQLite this stack runs refuses 1,100
bound ids — 32,766 has been the floor since 3.32 — so the batch is about
owning the ceiling here rather than rescuing a reachable failure.
* refactor(ai-vault-search): keep the scope's expression with the other expressions
Making the typo repair ask its questions in the search's own scope put an
import from retrieval into it, and retrieval already owns the repair — a cycle
the native audit catches. `scopedExpression` belongs beside `phraseExpression`,
`andExpression` and `orExpression` anyway: it builds a MATCH expression, and
two callers now need it. The bm25 weights stay in retrieval, where the SQL that
uses them is.
* docs(ai-vault-search): say which delete paths fire the orphan-reclaim trigger after a replace cuts loose
|
||
 Brennan BensonandMerge Sim
|
ebb1acfa37 |
refactor(agent-status): publish structured sessions into the hook server store (#19683)
* refactor(agent-status): publish structured sessions into the hook server store Structured (native chat) sessions have no PTY and no hook script, so their status never reached the hook server's store; #19217 gave `worktree ps` its own adapter over the structured feed instead. The feed now writes every projection into that store through a status sink the runtime wires, drops the row when the host closes the session, and `worktree ps` reads the one snapshot like every other agent. Rows carry a `structuredHost` marker and the journal clock; they are never persisted to last-status.json, and the main process does not forward them to the renderer yet, whose feed bridge still owns them until it is retired. Design and the two follow-ups: docs/reference/agent-status-store.md. * chore: drop stray @pnpm/exe lockfile entry An unrelated local pnpm run added @pnpm/exe as a packageManagerDependency with no package.json change, so CI's --frozen-lockfile install failed before any job ran. * docs(agent-status): describe the step that actually landed The design record claimed PR 1 deletes RuntimeAgentRowStore, drops the retained-versus-hook reconciliation, stamps terminalHandle on OSC rows, and tags rows with a source field of 'structured-host'. None of that is true of the shipped code: the retained store and its reconciliation are still in place, and the row field is structuredHost: 'held' | 'owned'. AGENTS.md points every future contributor here before they touch agent status, so split the roadmap into the 1a that landed and the 1b that has not, and name the fields the code actually writes. * fix(agent-status): pair session removal with the status-row forget A session dropped from the host's map without an explicit forget left its row in the store forever: `structuredHostOwned` bypasses the staleness check, so a failed re-attach (the Claude rewind path reaches one) stranded a permanently working agent in `worktree ps` and on mobile with no UI able to clear it. Deletion and forget are now one operation both callers route through. * fix(agent-status): give orcad the store worktree ps reads from `orcad` constructed its runtime with neither `getAgentStatusSnapshot` nor `structuredAgentStatusSink`, so once `worktree ps` sourced rows only from that snapshot the headless host published nowhere and listed nothing. The hook server's store is a module singleton whose import tree never reaches Electron, and its file paths come from `start()`, which orcad never calls. * fix(agent-status): drop a structured row without a renderer clear `dropStructuredStatus` went through `clearPaneState`, which fans a pane clear out to the renderer for a pane key the renderer's own feed bridge still writes - so 'exactly one writer per pane key' held for writes and not for deletes. `dropStatusEntry` routes through the status-drop tap instead, and skips the resume-identity remnant: a structured session has no pane to resume into, and every null-status publish would otherwise re-mint one. * test(agent-status): pin both half-migration structured-row filters Neither the `agentStatus:getSnapshot` filter nor the main-window listener's had a single assertion, so deleting either — the first step of PR 2 — was green everywhere. Also covers the perf skip and the drop's lack of a renderer clear. * docs(agent-status): correct three statements this PR made false The sink JSDoc claimed only tests construct a host without one; `orcad` did. The doc argued a structured row needs no tab mirror 'because headless serve has no renderer', reasoning about exactly the topology the wiring had not reached. The deleted runtime adapter's warning that the pane key must be the DERIVED one - never a bearer handle or minted worker key - was lost with it. * test(agent-status): declare orcad in the hook-row producer census Wiring the hook store into the orcad runtime added a production site that hands hook rows to a consumer, which the census ratchet pins deliberately. --------- Co-authored-by: Merge Sim <sim@local> |
||
|
Jinwoo Hong
|
d53cbed43f |
revert: hold mobile push feature for user testing (#19203)
Reverts
|
||
|
Jinwoo Hong
|
3160b54c69 |
feat: real background push notifications for the mobile app (#8129) (#18554)
* feat(cloud): add the mobile push gateway and its contract package (#8129) A small open-source service that holds the APNs key and FCM credentials and sends background push to paired phones on the desktop's behalf. Hosts authenticate with a box challenge and HMAC proof on their pairing key, the same shape the relay uses, so signed-in and accountless desktops share one path. Tokens are stored; alert text is held only for the coalescing window. The contract doc in docs/reference is the source of truth for every wire shape. The interop test runs the real desktop answerer against a real gateway-issued challenge so transcript drift fails in CI. * feat(push): register phones and send background push from the desktop (#8129) Adds the notifications.remote-push.v1 capability, the registerPush and unregisterPush RPCs on the mobile allowlist, a gateway client with a cached session and 401 re-auth, a durable unregister outbox, and a dispatcher that offers every mobile notification to the gateway after the socket fan-out. The dispatcher is fire-and-forget with one retry and drops registrations the gateway reports dead. Puts agentState on the mobile frame and fixes the #4375 wording so a working agent is never announced as finished. The relay host-proof code moves onto a shared envelope module with no behaviour change. * feat(mobile): background push registration, receive, and settings (#8129) Fetches the native APNs or FCM token, registers it with every paired host that advertises the capability, and re-registers on token change. Foreground pushes are suppressed inside handleNotification against the same seen set the socket path uses, so nothing shows twice. Taps route by host fingerprint. One Background notifications switch, off by default, with the disclaimer and needs-input / finished sub-switches; hidden until a paired desktop is new enough. Adds google-services.json and the expo-notifications plugin. * chore(cloud): Terraform and deploy workflow for the push gateway (#8129) Declares the Cloud Run service, runtime account, secrets, and orca_push database behind push_gateway_enabled, true only in production. The deploy workflow is gated like the relay's, deploys with no traffic, probes /ready and a validate-only FCM send, then shifts traffic. It runs as the shared production deploy account because the Cloud SQL rollout lease grant is foundation-owned; its extra authority is three bindings on the push service. docs/push-gateway.md carries the import commands for the resources created by hand and the APNs key rotation procedure. * docs: describe background notifications on the phone (#8129) * docs: check in the mobile push contract (#8129) Seven committed files cite it as the source of truth for every wire shape; docs/reference is allowlisted per file, so add the entry. * test(push): replay one checked-in host-proof vector on both sides (#8129) Cloud Verify installs only the cloud workspace, so the gateway suite cannot import the desktop answerer. Replace the cross-workspace import with a fixed challenge vector generated from the contract package; the gateway fixture and the desktop answerer each replay it and must produce the same HMAC. A transcript drift on either side now fails in that side's own suite. * fix(cloud): open the push gateway with invoker_iam_disabled, not an allUsers binding (#8129) The production domain-restricted-sharing policy rejects an allUsers run.invoker member, which the runbook anticipated. Opt the service out of invoker IAM the way the relay director already does; the host proof is the authentication either way. * docs(cloud): the push.onorca.dev record exists and is hand-managed (#8129) * fix(push): close review findings in the gateway (#8129) - Quota reservation takes a per-host advisory lock; READ COMMITTED admitted a whole burst past the cap (80/80 without, 60/80 with, against Postgres 16). - Challenge issuance no longer writes push_hosts; the row lands on proof verification. Stale hosts prune after 30 days. Per-IP token bucket on the two unauthenticated routes. - Streaming body limit via hono bodyLimit; a chunked body bypassed the Content-Length check. - registrationIds deduped in the schema; per-host device cap of 64; list bounded to its schema. - Gateway-side challenge TTL is the specified 10 s, not 40 s. - APNs stream settles on close as well as end/error. * fix(push): close review findings in the desktop client (#8129) - A gateway registration the registry cannot persist is enqueued for delete instead of leaking a live token. - Unregister outbox re-reads pending per pass, honours enqueues during a drain, and retries with backoff instead of waiting for the next launch. - Dispatcher batches registrations by 20 rather than starving the rest. - 401 compare-and-clear; a 401 after re-auth is unreachable; refused handshakes and 429s are cached briefly instead of re-handshaking per event. - Service is stopped on quit. * fix(mobile): close review findings in push registration and receive (#8129) - Consent generation guards a register that finishes after the switch went off; the host is re-queued for unregister instead of recorded live. - Foreground pushes seed the watermark before adopting the epoch, so a push on a never-connected session cannot wipe a valid watermark. - aps-environment follows the build via app.config.js; the iOS release workflow sets it to production. A bare plugin entry wrote development. - Pushes the OS showed while closed are marked seen before catch-up replay. - Token null result is not cached; failed capability probes are retried and never block an unregister; coalesced summaries are shown but not marked. - Unresolvable fingerprint routes nowhere and is suppressed in foreground. - Android channel ensured at boot; capability hook diffs clients by identity. * fix(cloud): harden the push deploy workflow and size the gateway to the budget (#8129) - Roll traffic back on a failed post-shift check; delete a candidate that never took traffic; retry the origin probe and the FCM probe. - Assert Terraform-owned scaling instead of mutating it from the workflow. - Build before taking the Cloud SQL rollout lease. - Declare the database pool in Terraform (2 per instance, max 2 instances) and add the gateway to the connection budget; the previous default put the shared instance 65 connections over its ceiling. - State plainly that the shared deploy identity's relay authority is inherited. * fix(push): read the runtime from shared state at push startup (#8129) Threading the runtime through launchDesktopMode put the launch module one line over the 300-line lint budget after the rebase. * fix(push): key the unauthenticated rate limit on the hop Cloud Run wrote (#8129) Cloud Run appends the connecting peer to x-forwarded-for; the limiter read the left-most value, which the caller controls, so a forged first hop earned a fresh bucket per request. * fix(push): close the final security review findings in the gateway and infra (#8129) - app.onError logs only the error name and answers a bare 500; hono's default handler printed the whole error, and a pg error carries the row in detail - a second per-IP bucket (240/min) runs ahead of the bearer lookup on every authenticated route, so forged bearers cannot spend the two-connection pool - one live session per host: minting deletes the host's earlier row - device-less hosts are pruned after 1 h, not 30 d; any keypair mints one free - notificationId is printable ASCII, since it becomes the APNs collapse header - the impersonated FCM probe token is masked in the workflow log - prevent_destroy on the Apple secrets and the orca_push database * fix(push): close the final security review findings in the desktop client (#8129) - fetch never follows a redirect: a 307 would replay the host proof and the phone's token to whatever origin the redirect named - registerPush params are strict and the paired identity is spread last - a per-device bucket (10/min) bounds a phone looping registerPush, which costs a gateway write and a synchronous registry write each time * fix(mobile): close the final security review findings in push receive (#8129) - a push with no epoch can no longer claim a seq-derived dedup key, in the foreground or from the tray; a forged seq:N could otherwise swallow the real bell at that seq - a provider-delivered push with no host catalog, or no fingerprint at all, stays unrouted instead of falling back to the hostId its raw data carries * docs(push): record the ip buckets, session and host retention, and the token-ownership limit (#8129) * fix(push): apply the schema on an untimed pool and retry statement-timeout aborts (#8129) Ports the relay's #18722 pattern to the gateway: DDL runs on a one-connection pool with statement_timeout 0 that is closed before the serving pool opens, and SQLSTATE 57014 joins the bounded transaction retry path. * fix: harden mobile push delivery and deployment recovery * feat: align mobile notification preferences with desktop delivery * fix: accept variable-length APNs device tokens * fix: deduplicate native APNs and background socket notifications |
||
|
OrcaWinandOrca Worker
|
c252d855ac |
fix(windows): resolve npm/pnpm .cmd shims past cmd.exe (#17869)
* fix(windows): resolve npm/pnpm .cmd shims past cmd.exe
A `.cmd` target forces every spawn through `cmd.exe /c` with each argument
caret-escaped, and Microsoft Defender for Endpoint scores a long `cmd.exe /c`
line carrying caret-escaped natural language as obfuscation. `codex.cmd` is
named in the spawn cluster of the MDE incident this addresses.
npm's `cmd-shim` and pnpm's `@zkochan/cmd-shim` generate files whose whole body
is "find node, run this script". Read one, and the spawn can go straight to
`node.exe <script> <args>` — no cmd.exe, no caret escaping. Anything the parser
does not recognise exactly, or whose target cannot be confirmed on disk, keeps
the existing cmd.exe path.
Incidentally fixes a real bug: cmd ends its command at a raw CR/LF whatever the
quote state, so a multi-line agent prompt through a `.cmd` shim had to be
rejected. Resolved shims have no such limit.
* fix(windows): refuse drive-relative shim paths and run the win32 tests in CI
Two blocking findings from review.
A drive-relative path defeated the absolute-path guard:
`win32.isAbsolute('D:evil.js')` is false, but `win32.resolve` reads the drive
letter and lands on `D:\evil.js`, outside the shim directory. cmd would have
built `C:\shim\D:evil.js` and failed; we would have executed the wrong file.
Adding `:` to the unsafe-character set closes it, and the alternate-data-stream
spelling `a.js:zone` with it. It costs no coverage: 84 of the 91 real shims on
this box still resolve, the same seven fall back.
Neither `windows-cmd-shim-resolution.test.ts` nor its `.win32` sibling was in
the Windows package job's file list, so the whole filesystem/resolution half and
the real-spawn equivalence suite ran nowhere. Both are now in
`WINDOWS_PACKAGE_TESTS` and in the pr.yml step.
Also from review: clear `windowsVerbatimArguments` explicitly on the resolved
branch rather than inheriting it, since there is no caller-built command line
there; document the kill switch and the PTY/hook-wrapper scope limits in
docs/reference; and cover drive-relative, BOM, line-ending, casing and `%*`
tampering in the platform-independent half of the tests.
* docs(windows): justify the shim-path colon guard from the filesystem rule
The guard was argued empirically ("none of the 91 shims on this box has one"),
which invites a future reader to relax it for a shim we have not seen. Windows
reserves `:` within a path segment, so a relative path cannot carry one at all:
the only spellings that can are drive-qualified, an alternate data stream, or a
`\?\` device path, and the last is already refused as absolute. That makes a
false refusal impossible rather than unobserved.
* refactor(child-process): move resolveSpawn into its own module
The merge with main pushed run-process.ts one line past the 300-line cap:
both sides grew it. The spawn-argv decision is already a pure, separately
tested unit, so it moves out rather than the cap moving up. run-process.ts
re-exports it, so no caller changes.
* perf(child-process): cache the shim interpreter lookup
The parse cache spared the shim read but not the PATH walk, so a second
resolution of the same .cmd did 0 reads and one statSync per PATH entry --
30 on a 30-entry PATH, synchronous on resolveSpawn, where one dead network
mount blocks the calling thread on every spawn.
Keyed by shim directory AND PATH, since the shim's own rule is
%~dp0\node.exe first then PATH, and a PATH edit between spawns must miss.
Corrects the stat comment, which accounted only for the shim itself.
* fix(child-process): revalidate a cached shim interpreter before using it
The node cache was held for process life and never rechecked, so a cached
node.exe that was later uninstalled -- or dropped from PATH by a version
manager -- was still handed to resolveSpawn, failing the spawn with ENOENT.
An uncached process in the same state returns null and falls back to
cmd.exe successfully, so the cache was strictly worse than no cache.
One statSync on a non-null hit, not one per PATH entry, so the walk this
cache exists to skip is still skipped. The stale-null direction stays
uncorrected on purpose: it only keeps the working cmd.exe fallback. Both
directions are now stated in the comment, along with the known miss for
callers that vary PATH per spawn.
* fix(child-process): honour PATHEXT when resolving the shim interpreter
The doc claimed a node.com/.bat/.cmd on PATH returned null and fell back to
cmd.exe. The scan actually skipped those entries and kept looking for a
node.exe, so PATH=C:\A;C:\B with C:\A\node.com and C:\B\node.exe resolved to
B's node.exe while the shim runs A's node.com -- a different binary, chosen
silently, on the one axis this module must not get wrong.
The scan now follows cmd's rule: first PATH directory holding any PATHEXT
spelling wins, PATHEXT order decides within it, and only an .exe winner is
returned. Anything else gives up and keeps the cmd.exe path, which restores
the strict-subset-of-cmd property everywhere except the documented cwd case.
PATHEXT is read from the child's env and joined into the cache key, since it
now changes the answer. Costs one stat per PATHEXT entry per node-less
directory, paid once per process behind the cache.
---------
Co-authored-by: Orca Worker <orca-worker@localhost>
|
||
|
OrcaWinandOrca Worker
|
fba90e017c |
fix(windows): copy the daemon host exe verbatim instead of renaming it (MDE T1036) (#17865)
* docs(windows): document the EDR signal surface Six Microsoft Defender for Endpoint incidents fired against Orca 1.4.192 in eight days on one enterprise Windows 11 / Intune tenant. All six were behavioural process-tree scoring, not signature hits; two escalated to multi-stage incidents mapped to ATT&CK Execution and Collection. Add a reference doc mapping each attack-technique-shaped behaviour to the code that produces it and to why it exists: the renamed daemon image (T1036), the per-process PEB read, encoded policy-bypassed PowerShell (T1049), caret-escaped cmd.exe lines, and computer-use screen capture plus runtime-compiled MSIL (T1113). Records that signing is not the gate -- reputation is signer plus hash-keyed prevalence -- and carries the two evidence gaps the report noted. Adds an engineer checklist, deployment guidance for admins (AV path exclusions do not suppress EDR behavioural alerts; an MDE alert suppression rule does), and an explicit pre-deployment warning about computer use. * docs(windows): correct the PowerShell flag inventory and admin paths Review corrections to the EDR posture doc. The "encoded, policy-bypassing PowerShell" list conflated three different shapes and was incomplete. Split it into the three tiers an EDR actually scores differently -- bypass plus encoding, encoding alone, and bypass alone -- and add the sites it missed, including windows-mobile-firewall.ts, which encodes a script and launches it elevated through Start-Process -Verb RunAs. system-fonts.ts (-Command) and desktop-script-provider-bridge.ts (-File) were listed as encoded and are not. Notes that a raw grep under-reports, because the hook sites reach -EncodedCommand through wrapWindowsPowerShellEncodedCommand. Attribute the in-payload Set-ExecutionPolicy move to #16576 rather than to #16003's measurement, which keyed on -WindowStyle Hidden + -EncodedCommand, and record that the launcher's own tradeoff is unverified on a real box. Admin guidance was missing two ways a suppression rule pinned to one full path misses real activity: the .staging-<hex> sibling that exists mid-update, which is when the update-cluster incidents fire, and the userData fallback when LOCALAPPDATA is unset. Also: state the measurement conditions on the process-table timings, note that Hermes has surface even though we have no telemetry for it, note that the uninstaller names are electron-builder-generated and in no repo file, drop a volatile line count, and mark the per-operation computer-use shape as being addressed by an unmerged change. Drops the duplicated AGENTS.md section, keeping the indexed bullet. * docs(windows): reconcile the EDR posture doc with the shipped remediation Three claims in this doc became false once the rest of the Windows EDR set landed, and two told engineers the opposite of what the release does. The process-table section still described one shared snapshot taken with `Memory | CommandLine | CreationTime`, argued that splitting the cache per field set "would restore exactly the fan-out it exists to prevent", and concluded the shape was unfixable because "the information is only in the PEB". The split shipped (identity opens no handle at all), `Memory` is retired, and the command line now comes from the kernel through `ProcessCommandLineInformation` -- `ReadProcessMemory` is absent from the compiled addon and a ratchet asserts it against the import table. An engineer reading the old text would have concluded both fixes were dead ends. The PowerShell site inventories were stale in three of four lists: the port scan went native, every `-ExecutionPolicy Bypass` + `-EncodedCommand` pair was dropped as a measured no-op, and of the unencoded-bypass list only `wsl-cli-scripts.ts` survives. Regenerated against the merged tree, including the sites that reach the flag through `wrapWindowsPowerShellEncodedCommand` and never spell it, which a raw `rg` misses. Incident-evidence sections are left alone: they record what the tenant observed on 1.4.192, not what the code does now. * fix(windows): copy the daemon host exe verbatim instead of renaming it Microsoft Defender for Endpoint flagged `orca-terminal-daemon.exe` as MITRE T1036 (Masquerading): Orca copied its own `Orca.exe` into %LOCALAPPDATA% under a different name, specifically so the NSIS updater's `taskkill /IM Orca.exe` could not match, then ran it detached. Because that process is what every other flagged action was attributed to, the name mismatch acted as a reputation multiplier on unrelated findings. The rename was never what made the daemon survive. In app-builder-lib 26.15.3 the installer's FIND_PROCESS/KILL_PROCESS select processes whose image path is under $INSTDIR; `taskkill /IM` is only the fallback for hosts where PowerShell is missing or blocked. Survival is a property of the path, and %LOCALAPPDATA%\Orca\daemon-host is outside $INSTDIR whatever the file is called. Derive the host exe name from process.execPath so the copy is byte-for-byte, name included — it keeps its Authenticode signature and carries no renamed-image signal. On the no-PowerShell fallback the daemon is now killed with the app and terminals cold-restore, which is the documented pre-relocation outcome the update harness already asserts, not a regression. The uninstall macro no longer needs a distinct name to find the daemon; it kills the app's own image name (plus the legacy name, for hosts left by older builds). Adds docs/reference/windows-daemon-host-relocation.md with the survival contract, the rejected alternatives and their measured costs, and the invariants to keep. * fix(windows): apply daemon-host relocation review corrections Scope the uninstall taskkill to the current user with `/FI "USERNAME eq %USERNAME%"` via cmd.exe, matching upstream's per-user KILL_PROCESS — without it an elevated machine-wide uninstall reaches another logged-on user's session, so the "no collateral" claim in the comment was overstated. Comment the rmSync-before-publish: Windows refuses to delete a running image, so a live daemon already hosted in this version's dir (same-version reinstall, or a dev channel reusing a version) throws and materialization fails open. Doc corrections: - The fallback selector is the full per-user `taskkill /F /IM "<app>.exe" /FI "PID ne $pid" /FI "USERNAME eq %USERNAME%"`, not a bare `taskkill /IM`. - The probe reads `Get-ExecutionPolicy -Scope Process`, not the effective policy, and GPO writes MachinePolicy/UserPolicy — so GPO-managed hosts take the primary path-scoped branch. Narrow the fallback triggers accordingly. - Drop the Authenticode sentence: the old name was equally byte-identical and equally signed, so a filename has no bearing on signature validity. - Name the new update-abort path: the daemon now matches FIND_PROCESS, so on the fallback branch an unkillable host reaches the retry loop's MessageBox /SD IDCANCEL and Quits, aborting a silent update. - Correct the customCheckAppRunning rejection. It is ~6 lines, not a rewrite; it is wrong because forcing the PowerShell branch where PowerShell is absent makes FIND/KILL silently no-op and leaves the real app running with files in use. - Bound the win honestly: OriginalFilename is empty on the shipped binary, so the strongest T1036 indicator never fired, and the residual copy-and-run-detached shape still maps to T1036.005. Reconcile docs/reference/windows-edr-posture.md, which documents the rename as a live finding and would otherwise contradict this change. Content-only edit: markdown under docs/reference/ is not oxfmt-formatted as a matter of practice and nothing in CI gates it, so the file is left consistent with its neighbours. * fix(windows): expand USERNAME in NSIS instead of spawning cmd.exe The uninstall macro routed both taskkills through `"$SYSDIR\cmd.exe" /C` purely so `%USERNAME%` would expand — two extra interpreter spawns on the uninstall path, in a change whose whole point is not adding scored behaviour, and the exact `cmd.exe /c` shape the new AGENTS.md EDR bullet warns about. NSIS reads the variable itself with ReadEnvStr, so the spawns buy nothing. Verified on Windows 11 that the generated command line does what the filter is there for: a copy of cmd.exe running as orca-nonexistent-probe.exe (pid 34244) was terminated by `taskkill /F /IM "orca-nonexistent-probe.exe" /FI "USERNAME eq <user>"` — SUCCESS, exit 0, process gone. Guarded on an empty USERNAME because the degenerate case is silent: taskkill rejects an empty filter value outright ("The search filter cannot be recognized") and kills nothing, which would leave exactly the orphaned daemon this macro exists to reap. `*` is rejected as a filter value too, so there is no branchless spelling. With no USERNAME to scope by it kills unfiltered, as the macro did before the filter was added. Stack stays balanced: three pushes, two nsExec pops, three restores. Also strike the last stale row in windows-edr-posture.md's remediation table. "Copying our own image under a different name" read as outstanding work; it is done by this change, so the row now points at the relocation doc. Same class of staleness as the section reconciled in the previous commit, and git would not have flagged it either. * fix(windows): port the daemon-host uninstall sweep into the live NSIS include The uninstall macro this branch rewrote lived in config/nsis/daemon-host-uninstall.nsh, which main no longer includes: #17906 consolidated every Windows installer hook into config/nsis/orca-installer-hooks.nsh because electron-builder accepts exactly one `nsis.include`. Merged as-is, the rewritten macro would have been dead code while the shipped uninstaller kept running main's stale sweep — `taskkill /F /IM orca-terminal-daemon.exe`, which matches nothing now that the relocated host is a verbatim Orca.exe copy. The RMDir that follows then cannot delete the running image, so a live orphaned daemon and its ~224 MB tree would survive every uninstall. Ported into the live include: the ${APP_EXECUTABLE_FILENAME} kill, the USERNAME filter that keeps an elevated machine-wide uninstall out of another logged-on user's session, and the register save/restore around both. The legacy orca-terminal-daemon.exe kill stays so hosts left by older builds are still reaped. The ratchet that was meant to catch exactly this pinned only the legacy image name, which main's stale macro already satisfied, so it passed both ways. It now asserts the app-exe kill and the USERNAME filter, against comment-stripped script — the prose above the macro names both image names, so a toContain over the raw file proves nothing. --------- Co-authored-by: Orca Worker <orca-worker@localhost> |
||
 foXaCe
|
49d6d35b16 |
feat(i18n): add French UI locale
foXaCe <290678+foXaCe@users.noreply.github.com> |
||
|
OrcaWinandOrca Worker
|
2c4989ea94 |
docs(windows): document the EDR signal surface (#17856)
* docs(windows): document the EDR signal surface Six Microsoft Defender for Endpoint incidents fired against Orca 1.4.192 in eight days on one enterprise Windows 11 / Intune tenant. All six were behavioural process-tree scoring, not signature hits; two escalated to multi-stage incidents mapped to ATT&CK Execution and Collection. Add a reference doc mapping each attack-technique-shaped behaviour to the code that produces it and to why it exists: the renamed daemon image (T1036), the per-process PEB read, encoded policy-bypassed PowerShell (T1049), caret-escaped cmd.exe lines, and computer-use screen capture plus runtime-compiled MSIL (T1113). Records that signing is not the gate -- reputation is signer plus hash-keyed prevalence -- and carries the two evidence gaps the report noted. Adds an engineer checklist, deployment guidance for admins (AV path exclusions do not suppress EDR behavioural alerts; an MDE alert suppression rule does), and an explicit pre-deployment warning about computer use. * docs(windows): correct the PowerShell flag inventory and admin paths Review corrections to the EDR posture doc. The "encoded, policy-bypassing PowerShell" list conflated three different shapes and was incomplete. Split it into the three tiers an EDR actually scores differently -- bypass plus encoding, encoding alone, and bypass alone -- and add the sites it missed, including windows-mobile-firewall.ts, which encodes a script and launches it elevated through Start-Process -Verb RunAs. system-fonts.ts (-Command) and desktop-script-provider-bridge.ts (-File) were listed as encoded and are not. Notes that a raw grep under-reports, because the hook sites reach -EncodedCommand through wrapWindowsPowerShellEncodedCommand. Attribute the in-payload Set-ExecutionPolicy move to #16576 rather than to #16003's measurement, which keyed on -WindowStyle Hidden + -EncodedCommand, and record that the launcher's own tradeoff is unverified on a real box. Admin guidance was missing two ways a suppression rule pinned to one full path misses real activity: the .staging-<hex> sibling that exists mid-update, which is when the update-cluster incidents fire, and the userData fallback when LOCALAPPDATA is unset. Also: state the measurement conditions on the process-table timings, note that Hermes has surface even though we have no telemetry for it, note that the uninstaller names are electron-builder-generated and in no repo file, drop a volatile line count, and mark the per-operation computer-use shape as being addressed by an unmerged change. Drops the duplicated AGENTS.md section, keeping the indexed bullet. * docs(windows): reconcile the EDR posture doc with the shipped remediation Three claims in this doc became false once the rest of the Windows EDR set landed, and two told engineers the opposite of what the release does. The process-table section still described one shared snapshot taken with `Memory | CommandLine | CreationTime`, argued that splitting the cache per field set "would restore exactly the fan-out it exists to prevent", and concluded the shape was unfixable because "the information is only in the PEB". The split shipped (identity opens no handle at all), `Memory` is retired, and the command line now comes from the kernel through `ProcessCommandLineInformation` -- `ReadProcessMemory` is absent from the compiled addon and a ratchet asserts it against the import table. An engineer reading the old text would have concluded both fixes were dead ends. The PowerShell site inventories were stale in three of four lists: the port scan went native, every `-ExecutionPolicy Bypass` + `-EncodedCommand` pair was dropped as a measured no-op, and of the unencoded-bypass list only `wsl-cli-scripts.ts` survives. Regenerated against the merged tree, including the sites that reach the flag through `wrapWindowsPowerShellEncodedCommand` and never spell it, which a raw `rg` misses. Incident-evidence sections are left alone: they record what the tenant observed on 1.4.192, not what the code does now. --------- Co-authored-by: Orca Worker <orca-worker@localhost> |
||
|
Neil
|
6aba202d5e |
feat(docs): publish OSS docs with stable releases
Publish the standalone docs site under docs/site and deploy it on stable desktop releases. |
||
|
Neil
|
99b59064fd |
docs(ime): codify desktop composition regression checks (#17172)
* docs(ime): codify desktop composition regression checks * docs(ime): define remote verification verdicts * docs(ime): narrow placeholder masking invariant * docs(ime): require final-cell caret containment * docs(ime): cover async attachment settlement * docs(ime): correct semantic placeholder contract * docs(ime): record bounded ownership contracts |
||
|
Neil
|
58a52a8ce3 |
test(wsl): guard probes that report failure as a negative answer (#17352)
* test(wsl): guard probes that report failure as a negative answer A WSL probe that cannot reach its distro returns the same value as one that asked and got "no". Downstream nothing can tell them apart, so a distro that was busy for a second reports no git, or no agent sessions, until relaunch — sticky, silent, and identical to the real thing. That has shipped three times: preflight CLI probes, the glab auth fallback (#8941), and listRunningWslDistrosAsync failing closed with no last-known-good while polled every 2s (PR #17072). Scan the WSL and preflight probe modules for the shape and hold the current set in an allowlist that only shrinks. Scoped deliberately: the same shape appears ~850 times across src/ and is usually correct, because for most callers a failure really does mean absent. It is only dangerous where the answer describes a distro. The guard cannot see the dangerous part — whether the value is later cached or gates discovery is dataflow, not syntax. It stops a new swallow site appearing here without someone saying why it is safe to pin, which is the review that was missing all three times. * test(wsl): make ratchet failures actionable A red build must say what to do. Name the offending files, say the allowlist is where a safe case goes, and — for a stale entry — say the change is fine and the list just needs to shrink. * docs(wsl): track the probe failure-semantics reference docs/** is gitignored with an explicit allowlist, so the reference the ratchet points contributors to was silently left out of the branch. A guard whose error message cites a doc that is not in the repo is worse than no doc. * test(wsl): catch a swallow whose reason trails the return The guard only tolerated comments before `return`, so `return false // ...` slipped past — including the exact snippet the doc and the test's own docstring use as the canonical example. The doc asks authors to write down why a swallow is safe, and the natural place for that sentence is trailing the return, so following the guidance defeated the guard. Verified against both shapes: trailing comment and comment on the line after. |
||
 Neilandinnocarpe
|
f400f8fd5f |
fix(macos): opt out of press-and-hold so held keys repeat (#14746) (#15589)
* fix(macos): opt out of press-and-hold so held keys repeat (#14746)
macOS routes press-and-hold to the accent picker unless an app sets
ApplePressAndHoldEnabled=false for its own bundle, so holding j in vim
inserted one character instead of repeating. Orca never set it.
Written at most once, and never over an explicit value: `defaults read`
is domain-scoped and exits 1 when the key is absent, which is the only
way to tell "unset" from a deliberate false — Electron's
systemPreferences.getUserDefault reports false for both. A recorded
decision in userData keeps a later launch from re-clobbering a user who
deletes the key to get the accent picker back.
* docs(macos): record the revert hazard and CI's macOS test gap
Two things a reader of this module cannot otherwise know.
A revert leaves the key written in every user's domain forever. AppKit reads
the plist, not this file, so removing the code alone keeps press-and-hold
disabled for everyone who ran an affected build. The sibling period-substitution
module carries the same warning because that fix was already lost once this way.
And the real-binary test file that pins the defaults(1) exit-code semantics this
design rests on never runs in CI: the e2e workflow and both unit-test jobs are
ubuntu and windows, and the only macOS runners in the repo are build and
packaging jobs that run no tests. Those six tests plus the real-bundle e2e case
pass on a developer Mac and execute zero times in a green PR, so the comment
should not imply enforcement that is not there.
Refs #14746
* feat(macos): let users turn the accent menu back on (#14746)
Orca disables press-and-hold for its own preferences domain so held keys
repeat. That is the right default, but the way back was a `defaults write`
buried in a source comment: nothing in docs/ or the README mentioned it, and
the preference is per-application, so it silently takes the accent picker
away from the Markdown editor and every other text field too.
Terminal -> Advanced now carries a "Character Accent Menu" switch, macOS and
desktop only. A web client cannot write a macOS preference for the machine the
user is looking at, so the control and its search-index entry are both gated on
that, not on the client's platform alone.
Precedence, which is the part that is easy to get wrong: the setting is
`undefined` until the user touches it, which is what keeps a hand-run `defaults
write` in charge for everyone who never opens the toggle. Once used, Orca owns
the key and writes exactly what the switch asks for -- `ApplePressAndHoldEnabled`
*is* the accent-menu switch, so it maps straight through with no inversion. The
choice is compared against `appliedSetting` in the existing decision record
rather than against the domain, so a `defaults write` made *after* using the
toggle is still the newer choice and survives the next launch. Re-asserting the
value every launch would have reintroduced the clobbering the record exists to
prevent.
The write lands for the next launch, since AppKit reads the preference as the
process starts, so the toggle shows the same restart banner the window-blur
setting uses. That banner is now a shared component, keeping its original
translation keys.
docs/reference/macos-press-and-hold.md records the precedence rules, the
`defaults read` rationale, the revert hazard, and the fact that none of this
executes in CI: every macOS job builds or packages and runs no tests, so the
real-binary and e2e coverage here passes only on a developer Mac.
* docs(macos): stop asserting when AppKit re-reads the press-and-hold key
Five places stated "AppKit reads the preference as the process starts" as
fact. That is the reason given for requiring a relaunch, and it is not
something this change ever measured.
Evidence points the other way: terminal emulators that register this key
after their process has started get key repeat in that same launch, which a
read-once-at-startup model cannot explain.
The relaunch requirement itself still looks right, but for a different and
verifiable reason: the write goes out through a separate `defaults` process,
so this app's own cached copy need not observe it. That is what the comments
now say, with the AppKit question left open rather than answered.
Refs #14746
* docs(macos): correct the startup comment's launch-timing claim
The comment said this call site is "the last point that can still matter for
this launch", which contradicts the rest of the module: the write is assumed
to land for the next launch because it goes out through a separate `defaults`
process. Reported on the PR by @innocarpe, who also supplied the replacement
wording.
Co-authored-by: innocarpe <innocarpe@users.noreply.github.com>
* refactor(macos): probe press-and-hold through the shared spawn chokepoint
`src/shared/child-process/child-process-import-boundary.test.ts` forbids a
direct `node:child_process` import outside its allowlist, and the allowlist only
shrinks — so this module moves to `runProcessSync`, which exists for callers
that genuinely cannot await. This one runs before `app.whenReady()`.
`runProcessSync` returns a non-zero exit instead of throwing it, so the
three-way read decision is re-expressed against `ProcessResult`: exit 0 is an
explicit value, exit 1 is a missing key, and a timeout, a signal kill, any other
exit, or a child that never started all stay 'unknown'. The throw path is now
inside `interpretDefaultsRead` so a spawn failure is reachable from a test
rather than hidden in an untested catch, and the write checks the exit code —
a refused `defaults write` no longer looks like success.
Both boundary-test failures were the same import: with it gone the offender
count returns to 155, so no ratchet baseline is bumped.
* Revert "feat(macos): let users turn the accent menu back on (#14746)"
This reverts commit
|
||
|
Neil
|
5631aa00dd |
feat(orcad): items 2–7 — degradation, natives, daemon, ops, deploy (#16398)
* fix(ports): stop joining an undefined resourcesPath on a non-Electron host `resolveWorkerEntryPath` branched on `isPackaged` alone and joined `process.resourcesPath`. orcad reports `isPackaged` true — correctly, it is a production build, and ~15 consumers read it that way to gate HTTPS-only skill downloads and the real CLI name — but `process.resourcesPath` is Electron-only and `undefined` under plain Node. So the packaged branch threw `TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string` where a clean "worker unavailable" was the honest outcome. The type said `resourcesPath: string`, which is how it went unnoticed; it is now `string | undefined`, so the compiler carries the fact. A host with no Electron resources tree has no asar to look in, so it falls back to the module directory and lets the caller report a missing worker. Found by the item 1 agent while auditing the same `isPackaged` defect class in the watcher. Verified in both directions: reverting the guard reproduces the TypeError. * feat(orcad): prove node-pty loads before anything requires it Of the two ways node-pty fails, only one is catchable. A missing module throws MODULE_NOT_FOUND. A module built against the wrong libc or Node ABI is refused by the dynamic loader, and in the worst case takes the process down before any handler exists — that is #9902, which crashed the desktop app on Ubuntu 20.04 before a window appeared. There was no libc or ABI precondition anywhere in the tree. So orcad now proves the load in a CHILD process, from main.ts, before anything requires node-pty. Whatever the child does — throw, abort, die on a signal — is data rather than our own death, and the operator gets a sentence naming the host's libc, Node ABI and prebuild slot plus the command to run. Proven-unloadable exits 78 (EX_CONFIG), so a supervisor does not restart an unequippable host forever. A probe that never answered is unverifiable, not blocked: refusing to boot on an inconclusive signal would take down hosts that work. The child dlopens the file node-pty would have chosen, before requiring the package. node-pty's loader walks several directories and rethrows only the LAST error, so a refused binary reads as "Cannot find module ./prebuilds/..." — which sends the operator to install a module that is already there. It also reports through stdout: node echoes the whole -e source above a stack trace, and matching tokens against stderr made the probe's own source text answer for the verdict. Verdicts reach clients as a terminal_unavailable degradation alongside the existing browser_unavailable one, through the same cause-registry shape. degradations[].code is now an open vocabulary; clients already render only `message`. Prebuilds are compiled from PATCHED sources — the patch IS the glibc-floor fix, so an upstream tarball reproduces #9902 — into linux-{x64,arm64}-{glibc,musl} and darwin-{x64,arm64} slots. libc is in the slot name because node-pty's loader falls back to prebuilds/<platform>-<arch> and cannot tell glibc from musl. orcad installs the matching slot at boot, so a host with no compiler serves terminals. The relay's five pure toolchain-diagnosis functions moved to a transport-free module so the Node bundle can reuse them without dragging ssh2 in behind them; the relay keeps its API by re-export. macOS gets `xcode-select --install` rather than the cross-distro apt/dnf/pacman/apk menu, every line of which is wrong there. * test(orcad): pin the node-pty precondition to ground truth, not a prepared host CI's test shard runs `vitest` directly, so `ensure-native-runtime --runtime=node` never prepares node-pty for the Node ABI — `degraded` is the correct verdict there, and asserting 'ok' encoded an environment the shard does not have. Asserting whatever it returned would be vacuous, so the expectation is now derived from an independent require() of node-pty. Verified it still bites: forcing the precondition to always report 'ok' fails the suite. * feat(orcad): run the terminal daemon, and the ops contract around it orcad declared `canRecoverPersistentLocalPtys: () => false` because it did not run the terminal daemon, so every restart, update and rollback SIGKILLed every running terminal — on the host whose selling point is that work survives the client going away. That is the one property `ssh-execution-boundary.md` recommends the peer model for. Item 4 — the daemon: - Port the launch path off electron: `daemon-init.ts`, `daemon-host-relocation.ts` and `observability/logs-directory.ts` now read the `AppEnvironment` port. Relocation additionally asks whether the app root is an asar archive rather than whether the build is packaged, so a Node host answering `isPackaged() === true` no longer walks into an Electron-only NSIS-escape path (same precedent as `parcel-watcher-entry-path.ts`). - `build-orcad.mjs` emits `daemon-entry.js` beside `orcad.js`, scans the forked children's metafiles for electron/node:sqlite, and load-checks the child under plain Node. - orcad spawns and adopts the daemon; shutdown disconnects and never kills it. `canRecoverPersistentLocalPtys` now reads the live provider and is false under degraded routing, where fresh terminals would die with the process. Item 3 — the ops contract (docs/reference/orcad-operations.md): - Bind policy: `--bind`, default loopback, pinned so neither `orca serve`'s wide default nor the connected-device widen can override it, and so a paired client cannot rebind the listener from outside. - Instance lock on the data root before profile load, scoped to the runtime role so it never refuses a restart that a live daemon makes worthwhile. - Supervision: exit codes a supervisor can act on (78 = do not retry), second-signal escalation, a shutdown deadline, and crash-loop containment on daemon respawn. - Health in the readiness payload: build hash, Node ABI, and a PTY self-test that spans both processes — the daemon spawns a real PTY in its own process and the verdict crosses its socket. Both bundle load-checks now assert on exit codes: these bundles are minified onto one line, so Node's uncaught-exception report echoes every string literal in the bundle and the previous message match passed against a bundle that never loaded. * feat(orcad): deploy, activate and roll back a versioned orcad install Plan items 6 and 7 from docs/design/shipping-orcad.html. Install reuses the relay's transaction verbatim — per-version lock, staged SFTP write, .install-complete sentinel, stale-lock recovery — under a parameterized namespace, so orcad-<v>/ sits beside relay-<v>/ permanently (§06). Parameterizing GC is the trap that creates: each model now collects only its own directories, enforced twice (prefix-scoped remote listing plus a local ownership re-check), and a client picks its model from how the host is registered, never from what it finds on disk. Activation is separate from installation, because a versioned directory selects nothing. A candidate is launched, publishes orca_server_ready, and only becomes active if its cross-process health payload passes: right build hash, listening, daemon live, PTY self-test green. A rejected candidate is stopped and the incumbent restarted, so a careful deploy cannot cause the outage it was being careful about. Update and rollback are shaped by the daemon. An update restarts orcad, the daemon outlives it, and the surviving daemon was forked from the outgoing bundle — so live terminals defer the update rather than proceed, and GC pins the active version, the rollback target and the live daemon's bundle. Orca's persisted state carries no schema version, so rollback restores a pre-activation snapshot rather than trusting backward-readability; the point past which it is unsafe is the first terminal created after activation, which the snapshot cannot describe and the surviving daemon still owns. Running the generated shell for real found two bugs the text assertions missed: tar members re-quoted inside a shell variable captured nothing, and kill -0 reports a zombie as alive. * test(orcad): assert the precondition is self-consistent, not environment-shaped The real-host case cannot predict a status: CI's shard runs vitest directly, so node-pty is never built for the Node ABI and 'degraded' is correct there, while a prepared checkout gives 'ok'. The previous attempt used require('node-pty') as ground truth, which resolves the JS wrapper while the native binding loads lazily — it proved strictly less than the precondition checks, and failed CI for exactly that reason. What is invariant on a host with node-pty installed: never 'blocked', and never a degraded verdict carrying an unestablished reason. The injected-input tests keep the logic coverage. * fix(orcad): drop an eslint-disable the rule no longer needs * test(orcad): separate slot placement from the load verdict Both remaining CI failures were the same shape: tests reaching into node_modules for a pty.node that only exists after `ensure-native-runtime --runtime=node`, which CI's shard never runs because it invokes vitest directly. Slot *placement* is the logic worth checking on every host, so it now uses a synthetic payload and asserts the verdict stays honest about not loading. The three assertions that genuinely need a Node-ABI binding are gated on it existing. Verified: breaking slot installation fails both placement tests; with the real pty.node hidden the file is 17 passed / 3 skipped instead of ENOENT. * test(orcad): gate the load-dependent cases on a real load, not on the file existing CI ships a pty.node built for Electron's ABI, so existsSync was true while require still failed — the gate ran exactly the tests that host can never satisfy. It now probes the binding in a child process, so a bad one cannot take the runner down. The self-consistency assertion also allowed too little: 'blocked' is the honest verdict for a corrupt binding, alongside 'ok' on a prepared host and 'degraded' on an unprepared one. What stays invariant is that anything other than 'ok' names an established cause, so a terminal is never declined for a reason nobody worked out. Verified against all three host states: prepared (19 passed), unprepared, and a corrupt binding (17 passed / 3 skipped, no failures). * test(orcad): gate on the whole premise — binding AND spawn-helper CI has a loadable pty.node but no spawn-helper, and a slot without the helper is legitimately 'degraded'. So the previous gate let a test run whose premise ('a complete slot yields ok') that host cannot satisfy. Verified in both states: with the helper present 19 pass; with it removed the load-dependent cases skip (17 passed / 3 skipped) instead of failing. * fix(orcad): preserve degradation types after rebase |