perf(sqlite): refresh the legacy profile copy hourly and cap the profile WAL file (#40136)

* perf(sqlite): refresh the legacy profile copy hourly and recover crashed row caches

- The legacy whole-document refresh rewrote every changed split domain each 5 minutes
  (~4-8 MB per cycle, ~200 MB/hour of WAL plus checkpoint on an active profile). Writer
  start and graceful close already bring it current, so refresh at most hourly in between.
- Cap the profile-state WAL at 8 MB; without journal_size_limit it stayed at the size of
  the largest transaction ever written.
- Row caches: a read-only startup read cannot roll back a hot journal (or recover a WAL
  without -shm), so retry once on a writable connection instead of disabling the store
  on every launch.
- Row caches: drop the saved baseline when a commit throws, since COMMIT may report an
  error after the rows landed and our own commit does not move data_version.

* test(sqlite): accept Windows exit status for the crashed-writer fixture; document recovery cost

* docs(sqlite): state the crash-recovery cost without overstating its bound

* fix(sqlite): rebuild a row cache a crash left needing recovery instead of replaying it on main

* revert(sqlite): keep crash-recovery failures non-destructive; drop the rebuild fallback
This commit is contained in:
Neil
2026-11-08 21:42:55 -08:00
committed by GitHub
parent 39902f398b
commit 345ffb56d2
4 changed files with 14 additions and 8 deletions
@@ -213,6 +213,8 @@ function configureProfileStatePragmas(db: Database.Database): void {
// User state needs FULL durability; macOS checkpoints also need F_FULLFSYNC to flush drive caches.
db.pragma('synchronous = FULL')
db.pragma('checkpoint_fullfsync = ON')
// Without a limit the WAL keeps the size of the largest transaction ever written.
db.pragma('journal_size_limit = 8388608')
}
function migrateProfileStateSchema(
@@ -8,9 +8,10 @@ import { WORKSPACE_SESSION_DOMAINS } from '../workspace-session-domains'
* this layer still sees one document per domain.
*
* Older builds ignore these tables and read the domain's legacy `profile_state_documents`
* row. That row is refreshed from the parts when the writer starts, at most every five minutes while it runs, and at graceful close; when an
* older build rewrites it, its revision no longer matches `legacy_revision` and the legacy
* row becomes authoritative again until the next write re-splits it.
* row. That row is refreshed from the parts when the writer starts, at most hourly while it
* runs, and at graceful close; when an older build rewrites it, its revision no longer matches
* `legacy_revision` and the legacy row becomes authoritative again until the next write
* re-splits it.
*/
export {
PROFILE_STATE_DOCUMENT_PARTS_META_TABLE,
@@ -12,12 +12,12 @@ import {
import { withProfileStateWriteTransaction } from './profile-state-write-transaction'
/**
* Older builds read the legacy rows. Starting the writer brings them current (covering an
* earlier crash); while it runs they lag the parts by at most this long, even when writes
* stop, which bounds what a downgrade straight after a crash can miss. Each refresh rewrites
* only stale rows.
* Older builds read the legacy rows. Starting the writer and graceful close bring them current;
* while it runs they lag the parts by at most this long, which bounds what a downgrade straight
* after a crash can miss. A refresh rewrites each stale domain's whole document (several MB for
* sessions), so a short interval would dominate profile-state disk writes.
*/
const LEGACY_DOCUMENT_REFRESH_INTERVAL_MS = 5 * 60 * 1000
const LEGACY_DOCUMENT_REFRESH_INTERVAL_MS = 60 * 60 * 1000
type DatabaseSource = () => Database.Database | undefined
@@ -228,6 +228,9 @@ export class KeyedJsonRowCache {
this.dataVersion = version
} catch (error) {
rollback(db)
// COMMIT can report an error after the rows landed; our own commit leaves data_version
// unchanged, so drop the baseline and re-read it before the next diff.
this.dataVersion = undefined
throw error
}
}