mirror of
https://github.com/stablyai/orca.git
synced 2026-09-30 16:02:56 +00:00
* Cap automationRuns retention to stop unbounded state file growth `automationRuns` was the only unbounded durable collection in `orca-data.json`, and the whole blob is re-serialized and rewritten on every save. On a machine running four `* * * * *` automations it had grown to 11,184 rows / 21 MB of a 28.5 MB file — all of them `skipped_precheck` no-ops — so each synchronous `flush()` blocked the Electron main thread for 190-210 ms and macOS filed 24 `disk writes` diagnostic reports against Orca (12-30 MB/s sustained, 549 GB/session). Prune to the newest 100 runs per automation, on load and on append. This mirrors the retention that `pruneLocalTerminalScrollbackBuffers` and `pruneWorkspaceSessionBrowserHistory` already apply to their collections; `automationRuns` was simply missed. The load-path prune marks state dirty, so an oversized file heals on first load. Without that flag the shrink lives only in memory: the sole load-time save trigger is `normalized.changed || loadNeedsSave || adaptedProjectGroups`, and `normalized` covers pane identity only. A user who took the documented workaround (`automations edit --disabled`) fires no runs, so nothing would ever rewrite the file. Measured against the affected 28.5 MB file: flush() 190 ms -> 9 ms, bytes written per save 28.5 MB -> 1.3 MB. Pruning breaks the old `runNumber` derivation, which counted retained runs, so every run after the cap would have been titled "run 101". Carry the ordinal on the run itself and derive the next number from the highest survivor. Legacy rows are numbered from the highest number their automation already carries, not from their append position: a downgrade to a pre-`runNumber` build appends unnumbered runs after pruned survivors numbered 101+, and a position would reissue one of those, giving two runs the same title. Fixes #8118 * Never evict in-flight automation runs from retention A dispatched run's completion can land hours later (renderer round-trip or headless completion promise); pruning it makes updateAutomationRun throw 'Automation run not found.'. Only final-status runs are evictable now, with the final-status predicate shared between retention and the service. Co-authored-by: Orca <help@stably.ai> * Skip the usage write when retention evicted the run mid-collection markDispatchResult finalizes a run, awaits usage collection, then writes usage by id. The run is final during that await, so a concurrent create-time prune can evict it and the write threw 'Automation run not found.' — in the headless path that cascaded into an unhandled rejection. Co-authored-by: Orca <help@stably.ai> * Pin backfill-before-prune ordering with true legacy fixture rows The heal-on-load fixture rows carried runNumber, so swapping backfill and prune passed every test while renumbering real legacy survivors 1..100 and re-minting colliding titles. Seed unnumbered rows and assert healed ordinals. Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Neil <4138956+nwparker@users.noreply.github.com> Co-authored-by: Orca <help@stably.ai>
66 lines
2.9 KiB
TypeScript
66 lines
2.9 KiB
TypeScript
import { isFinalAutomationRunStatus, type AutomationRun } from './automations-types'
|
|
|
|
export const MAX_AUTOMATION_RUNS_PER_AUTOMATION = 100
|
|
|
|
// Why: the whole state blob is re-serialized on every save, so an unbounded
|
|
// automationRuns made each flush() permanently slower (28.5 MB → 210 ms blocked).
|
|
export function pruneAutomationRuns(
|
|
runs: readonly AutomationRun[],
|
|
maxPerAutomation: number = MAX_AUTOMATION_RUNS_PER_AUTOMATION
|
|
): AutomationRun[] {
|
|
const kept = new Set<string>()
|
|
// Why: a dispatched run's completion can land hours later, and
|
|
// updateAutomationRun throws if its row is gone — only final runs are evictable.
|
|
const finalRuns = runs.filter((run) => isFinalAutomationRunStatus(run.status))
|
|
for (const automationRuns of Map.groupBy(finalRuns, (run) => run.automationId).values()) {
|
|
// Why: `createdAt` is the append time; `scheduledFor` breaks ties so runs
|
|
// minted in the same millisecond drop in a stable, reproducible order.
|
|
automationRuns.sort((a, b) => b.createdAt - a.createdAt || b.scheduledFor - a.scheduledFor)
|
|
// Why: clamp — a negative `slice` end drops from the tail instead of keeping nothing.
|
|
for (const run of automationRuns.slice(0, Math.max(0, maxPerAutomation))) {
|
|
kept.add(run.id)
|
|
}
|
|
}
|
|
|
|
// Survivors keep their original append order — callers index by position.
|
|
return runs.filter((run) => kept.has(run.id) || !isFinalAutomationRunStatus(run.status))
|
|
}
|
|
|
|
/**
|
|
* Stamp `runNumber` onto legacy runs that predate the field. Must run before
|
|
* {@link pruneAutomationRuns} so the surviving runs carry their true numbers and
|
|
* numbering never restarts.
|
|
*
|
|
* Numbers continue from the highest number the automation already carries, not from
|
|
* the run's append position: a downgrade to a pre-`runNumber` build appends unnumbered
|
|
* runs after pruned survivors numbered 101+, and a position would reissue one of those.
|
|
*/
|
|
export function backfillAutomationRunNumbers(runs: readonly AutomationRun[]): AutomationRun[] {
|
|
const highestPerAutomation = new Map<string, number>()
|
|
for (const run of runs) {
|
|
if (run.runNumber !== undefined) {
|
|
const highest = highestPerAutomation.get(run.automationId) ?? 0
|
|
highestPerAutomation.set(run.automationId, Math.max(highest, run.runNumber))
|
|
}
|
|
}
|
|
return runs.map((run) => {
|
|
if (run.runNumber !== undefined) {
|
|
return run
|
|
}
|
|
const runNumber = (highestPerAutomation.get(run.automationId) ?? 0) + 1
|
|
highestPerAutomation.set(run.automationId, runNumber)
|
|
return { ...run, runNumber }
|
|
})
|
|
}
|
|
|
|
/** Next run number for one automation, given only the runs still retained. */
|
|
export function nextAutomationRunNumber(runsForAutomation: readonly AutomationRun[]): number {
|
|
// Why: seed with the count so legacy runs that predate `runNumber` still advance.
|
|
return (
|
|
runsForAutomation.reduce(
|
|
(n, run) => Math.max(n, run.runNumber ?? 0),
|
|
runsForAutomation.length
|
|
) + 1
|
|
)
|
|
}
|