mirror of
https://github.com/stablyai/orca.git
synced 2026-10-31 09:07:40 +00:00
Brennan Benson
027eb0ec83
* Persist session parse results beyond the memory cache limit * Expire Session History parse rows by idle time instead of a count cap The parse cache dropped its oldest row once it held 4,096, so a scan over more transcripts evicted its own upcoming hits and re-parsed every file on every scan. Rows now stay until no reader has used them for 24 hours, swept after each completed History scan and each search-index pass. Only the live parser state keeps a 4,096 count bound; dropping it costs one whole re-read of a file that changed, never a re-parse of one that did not. Removes the overflow directory and the orcad persistence wiring from the earlier revision of this change. * Expire only rows History lists, and keep the newest transcripts under bounds The search pass ran the idle sweep every 20 s but never re-stores History's unchanged rows, so with search on a day away from History dropped them all and reopening it re-parsed everything. Only a completed History scan sweeps now, and it expires only rows History listed (on a monotonic clock). Rows only the search index or a title lookup created keep main's 4,096 count bound, so an index build can't grow the scanner process without limit. Scans store newest transcripts first, so every store-order bound dropped the newest, most active files: the saved snapshot kept the oldest ~11k rows and the resume bound stripped the newest files' parser state. The snapshot is now ordered by transcript mtime, and both count bounds drop the tenth of their rows with the oldest transcripts. * Drop the 24 h idle sweep; History rows live for the process The sweep let whichever listing ran last decide when a row died, and listings differ in scope: a smaller one (the phone's Agent History, a lower desktop limit) expired rows only a larger one used, so the next Unlimited scan re-parsed them. Rows History lists now live as long as the scanner process; the snapshot's newest-transcripts cap bounds what survives a reload, and rows other readers made keep the 4,096 bound. Also clears the resume bookkeeping for a row the unlisted bound drops in the same store. * Skip resume bookkeeping for a row the unlisted bound just dropped Running the resume trim first let a row the unlisted trim then deleted push the resume set over its bound and cost 409 live resume points. * perf(ai-vault): store the Session History parse cache one row per transcript The parse cache was persisted as one JSON file, rewritten whole after any change and capped at 64 MB / 1M tokens (about 11k rows). Past that cap, a restart re-read the oldest unchanged transcripts, and an agent writing while History was open rewrote 10-27 MB every few seconds. The scanner child now keeps the rows in their own SQLite file. A miss in the in-memory map reads the row by path; changes are written per row, debounced and chunked; deletes run immediately. A used_at column, refreshed at most daily, expires rows nothing has used for 30 days, so rows for transcripts deleted outside Orca have a way to die. The old JSON file is imported once and removed. Electron main only records the path for the child. The rebuild-on-corruption open moves out of the search index into a shared versioned cache database helper used by both. * fix(ai-vault): import the old parse cache at open and expire rows only for missing transcripts Review round 1 on the per-row parse cache store: - The old JSON was imported only by a History scan or search pass, and only into an empty database, but deleted either way. Tab-title sync writes a row first, so the import was skipped and the first History open after updating re-read everything. The import now runs when the database opens, before any lookup, write or delete can reach it, keeping rows already present. - A 30-day last-use expiry also dropped rows for transcripts that still exist, so opening History less than monthly re-read them. Rows unparsed for 30 days are now checked on disk and dropped only when the transcript is gone, then the freed pages are reclaimed. The daily last-use rewrite goes away. - 8 KB pages (a ~2.3 KB row filled a 4 KB page), busy_timeout before the WAL switch, and lookups served from rows still waiting to be written. - Revert accidental pnpm-lock.yaml churn. * chore: revert accidental pnpm-lock.yaml churn * fix(ai-vault): never stat WSL transcripts from the parse cache recheck A stat on a \\wsl.localhost path outside the scanner's running-distro filter and gated WSL lane could boot a stopped distro or hang on a stuck one. WSL rows now skip the existence check and age out after 180 days unparsed. SQLite-backed session keys use the shared OpenCode path splitter. * fix(ai-vault): register the parse cache database with the WSL mover census Each schema keeps its own meta table again, so the shared open helper creates none and the search schema's SQL is unchanged. The parse cache's tables are recorded as not moved: a cache rebuilt from transcripts. * fix(ai-vault): a forced whole read never reuses a row a failed database delete left behind
123 lines
7.2 KiB
JavaScript
123 lines
7.2 KiB
JavaScript
import { SQLITE_RUNTIME_INCLUDE } from './vitest-sqlite-runtime-files.mjs'
|
|
|
|
// SQLite publication, native bindings, worker IPC, socket liveness, and V8 retention use Node.
|
|
export const NODE_RUNTIME_INCLUDE = [
|
|
'config/scripts/vitest-sqlite-runtime-boundary.test.ts',
|
|
'config/scripts/cloud-security-history-workflow.test.mjs',
|
|
'config/scripts/skill-recipe-shell.test.mjs',
|
|
'src/main/ssh/ssh-relay-endpoint-incumbent-shell.integration.test.ts',
|
|
'src/main/ssh/ssh-remote-commands.test.ts',
|
|
'src/packages/process-host/src/run-process.test.ts',
|
|
'tests/e2e/cursor-quota-transport.unit.test.ts',
|
|
'tests/e2e/helpers/electron-process-shutdown.unit.test.ts',
|
|
'src/main/ai-vault-search/session-search-index-writer.test.ts',
|
|
'src/main/ai-vault/session-scanner-unlimited-dedup.test.ts',
|
|
// The parse cache persists through node:sqlite.
|
|
'src/main/ai-vault/session-parse-cache-antigravity-upgrade.test.ts',
|
|
'src/main/ai-vault/session-parse-cache-devin-upgrade.test.ts',
|
|
'src/main/ai-vault/session-parse-cache-large-scan.test.ts',
|
|
'src/main/ai-vault/session-parse-cache-metadata-checkpoint.test.ts',
|
|
'src/main/ai-vault/session-parse-cache-persistence.test.ts',
|
|
// Electron main bridges streams with Node's implementation.
|
|
'src/main/host/electron-http-client.test.ts',
|
|
'src/main/daemon/daemon-init-provider-installation.test.ts',
|
|
'src/main/orcad/orcad-daemon-teardown-linux.test.ts',
|
|
'src/renderer/src/components/terminal-pane/pty-input-write-queue.test.ts',
|
|
'src/renderer/src/lib/react-commit-cascade-observer.test.ts',
|
|
'src/renderer/src/lib/react-commit-cascade-telemetry.test.ts',
|
|
...SQLITE_RUNTIME_INCLUDE,
|
|
'src/cli/**/*.test.ts',
|
|
'src/main/persistence/**/*.test.ts',
|
|
'src/main/persistence*.test.ts',
|
|
'src/main/sqlite/**/*.test.ts',
|
|
'src/main/orca-profiles/**/*.test.ts',
|
|
'src/main/managed-data-accounts/**/*.test.ts',
|
|
'src/main/quit-path-durable-write-blocking.test.ts',
|
|
'src/main/automations/automation-worker-durability.test.ts',
|
|
'src/**/*.node-pty.test.ts',
|
|
'src/**/node-pty-*.test.ts',
|
|
'src/**/*.live-shell.test.ts',
|
|
'src/**/*real-pty*.test.ts',
|
|
'src/**/*-native.test.ts',
|
|
'src/**/*retention*.test.ts',
|
|
'src/**/*lifetime*.test.ts',
|
|
'src/**/*memory*.test.ts',
|
|
'src/**/*allocation*.test.ts',
|
|
'src/**/*expiry*.test.ts',
|
|
'src/renderer/src/lib/flatten-retained-slice.test.ts',
|
|
'src/shared/own-retained-string.test.ts',
|
|
// Times V8 string flattening, which is what Electron main runs.
|
|
'src/main/runtime/terminal-tail-redraw-wall-time.test.ts',
|
|
'src/main/agent-hooks/server-transport-interference.test.ts',
|
|
'src/main/plugins/plugin-worker-supervision.integration.test.ts',
|
|
// Reconciliation loads the real native Parcel watcher; Bun can abort on Linux ARM.
|
|
'src/main/plugins/plugin-service-reconciliation.test.ts',
|
|
'src/main/usage/usage-scan-worker-event-loop.test.ts',
|
|
// Asserts Node's spawn-failure close signal and that a dead worker closes its children's pipes.
|
|
'src/main/worker-thread-child-process/worker-thread-child-process.test.ts',
|
|
'src/main/computer/macos-native-provider-client.test.ts',
|
|
'src/main/ai-vault-search/session-search-schema.test.ts',
|
|
'src/main/ai-vault-search/session-search-fts5-contract.test.ts',
|
|
'src/main/foreign-sqlite-readers/**/*.test.ts',
|
|
'src/main/browser/*cookie*.test.ts',
|
|
// Playwright's CDP transport needs Node's WebSocket implementation.
|
|
'src/main/browser/browser-popup-compatibility.electron.test.ts',
|
|
'src/main/browser/browser-popup-native-compatibility.electron.test.ts',
|
|
'src/main/browser/browser-popup-background-compatibility.electron.test.ts',
|
|
'src/main/ai-vault/session-scanner-devin-db.test.ts',
|
|
'src/shared/workspace-session-salvage.test.ts',
|
|
'src/main/git/repo-detection.test.ts',
|
|
'src/renderer/src/components/browser-pane/annotate/browser-grab-toast-owner-cleanup.test.tsx',
|
|
'src/main/window/native-chat-paste-files.test.ts',
|
|
'src/main/computer/desktop-script-serve-channel.test.ts',
|
|
'src/main/providers/local-pty-child-process-verdict.test.ts',
|
|
'src/main/providers/working-directory-validation.test.ts',
|
|
'src/main/pty/posix-pty-process-groups.integration.test.ts',
|
|
'src/shared/ephemeral-vm-recipe-output-capture.test.ts',
|
|
'src/main/gitlab/gitlab-known-host-retirement.test.ts',
|
|
'src/main/window/main-window-updater-deferred-setup.test.ts',
|
|
'src/renderer/src/lib/pane-manager/pane-dom-focus-class-sync-lifecycle.test.ts',
|
|
'src/renderer/src/components/terminal-pane/terminal-pane-host-state-memo.test.ts',
|
|
'src/renderer/src/components/automations/use-automation-runs-dashboard-payload.test.tsx',
|
|
'src/renderer/src/components/right-sidebar/parent-pr-checks-projection-selector.test.ts',
|
|
'src/renderer/src/store/github/cache-persistence.test.ts',
|
|
'src/main/native-chat/agent-session-journal/**/*.test.ts',
|
|
'src/main/native-chat/agent-session-wire/structured-agent-session-status-feed.test.ts',
|
|
'src/main/native-chat/agent-session-wire/structured-agent-session-status-feed-conversation-name.test.ts',
|
|
'src/main/native-chat/agent-session-wire/structured-agent-session-status-first-input-identity.test.ts',
|
|
'src/main/native-chat/agent-session-wire/structured-agent-session-outgrown-child.test.ts',
|
|
'src/main/native-chat/agent-session-wire/structured-agent-session-permission-mode.test.ts',
|
|
'src/main/native-chat/structured-chat-naming-command-first.test.ts',
|
|
'src/main/native-chat/structured-chat-naming-status-hook.test.ts',
|
|
'src/main/runtime/agent-session-conversation-name-store.test.ts',
|
|
'src/main/runtime/structured-session-mail-redrive-wiring.test.ts',
|
|
'src/main/runtime/rpc/ws-transport.test.ts',
|
|
'src/main/runtime/rpc/dispatcher-transport-parity.test.ts',
|
|
'src/main/runtime/rpc/methods/structured-agent-session-visual.test.ts',
|
|
// Bun resolves 'ws' to its built-in client, which bypasses the patched package.
|
|
'src/main/runtime/rpc/ws-empty-payload-frame-write.test.ts',
|
|
'src/shared/remote-runtime-client.test.ts',
|
|
'src/shared/remote-runtime-connect-bound.test.ts',
|
|
// Bun's built-in ws client ignores maxPayload; the product opener only runs on Node.
|
|
'src/shared/remote-runtime-request-websocket.test.ts',
|
|
'src/shared/remote-runtime-subscription-connect-bound.test.ts',
|
|
'src/main/codex/codex-shared-server-probe.test.ts',
|
|
'src/main/opencode/opencode-launch-model-context-supervision.test.ts',
|
|
'src/main/hermes/hermes-config-remote-adapter.test.ts',
|
|
'src/main/ipc/orca-profiles-switch-persistence.test.ts',
|
|
'src/main/ipc/settings-delayed-response.test.ts',
|
|
'src/main/runtime/structured-agent-session-startup-tab-restore.test.ts',
|
|
'src/main/runtime/structured-agent-session-windowless-layout-host.test.ts',
|
|
'src/shared/remote-runtime-shared-control-connection.test.ts',
|
|
'src/main/runtime/relay/relay-control-client.test.ts',
|
|
'src/main/ipc/filesystem-watcher-ignore-real.test.ts',
|
|
'src/main/ipc/runtime-environment-preference.test.ts',
|
|
'src/main/ipc/runtime-environment-transport-routing-tailscale-hint.test.ts',
|
|
'src/renderer/src/components/emulator-pane/emulator-device-frame-visibility.test.tsx',
|
|
'src/main/claude/claude-agent-sdk-contract-pins.test.ts',
|
|
'config/scripts/profile-state-worker-smoke.test.mjs',
|
|
'config/scripts/process-host-dev-rebuild.test.ts',
|
|
'config/scripts/vitest-node-runtime-boundary.test.ts',
|
|
'config/scripts/vitest-node-runtime-pool.test.ts'
|
|
]
|