mirror of
https://github.com/stablyai/orca.git
synced 2026-11-13 17:37:40 +00:00
Neil
9bd6272fdf
* refactor(tooling): organize test runtime and maintenance owners * test(tooling): preserve relocated runtime fixture dependency edges * refactor(tooling): preserve xterm routing owner checks * ci(tooling): include routing owner in sparse checkout * test(tooling): preserve copied workflow dependency closures * fix(tooling): normalize hashed JSON assets in release parity * fix(tooling): distinguish asset aliases from literal references * fix(tooling): frame normalized asset reference payloads * fix(tooling): normalize decoded JSON asset references * refactor(tooling): organize build, CI and release owners * fix(tooling): align moved generator and workflow path contracts * fix(tooling): update generated plugin bundle owner banners * test(tooling): create relocated mobile verifier fixture directories * test(tooling): copy headless detector dependency closure * test(tooling): align native batch policy fixtures with owner path * fix(tooling): link moved reliability command to historical evidence * fix(tooling): preserve nested sandbox owners and canonical guide source * fix(tooling): target release version owners in checkout fixture
112 lines
6.8 KiB
Markdown
112 lines
6.8 KiB
Markdown
# Shared historical source for cross-version tests (P2)
|
|
|
|
Cross-version tests extract historical releases under each worktree's
|
|
`tests/e2e/.cross-version-checkouts`. Previously, branches testing the same release each
|
|
retained a full copy. Local tests now prepare one read-only source seed and share its file
|
|
data with each worktree's independent import directory.
|
|
|
|
## How it works
|
|
|
|
Seeds live under `<git-common-dir>/orca-cache/release-source/format-8/<commit>`.
|
|
The resolved commit and extraction format identify the source, including its rewritten imports.
|
|
The common Git directory lets sibling worktrees reuse the same seed without scanning them.
|
|
Existing local checkouts remain in place; the new format creates fresh import directories.
|
|
|
|
Missing-package stand-ins depend on the current branch's dependencies, so each worktree
|
|
creates those locally. Mutable copies of current-branch modules also remain local.
|
|
The existing lock, staging directory, and rename transaction publishes complete seeds and
|
|
import directories. Invalid stamps, missing protocol source, or invalid package metadata
|
|
cause rebuilding or ordinary local extraction.
|
|
|
|
macOS uses APFS clones. Linux tries reflinks, then read-only hardlinks. Copies preserve the
|
|
seed's read-only permissions; these historical files are never edited by the harness.
|
|
Windows uses independent copies because removing read-only hardlinks can change permissions
|
|
on surviving links. Unsupported sharing falls back to a private copy. If the optional shared
|
|
cache fails, the harness extracts locally and continues the test.
|
|
|
|
The current format retains at most eight seeds, ordered by recent use. A shared lock covers
|
|
publication, copying, and eviction, so cleanup cannot delete a source while another test
|
|
copies it. Copying and permission changes are asynchronous so the lock heartbeat keeps running.
|
|
Interrupted staging directories are removed on the next use. Retention only touches the
|
|
current format; older implementations may still be using other formats. Eviction leaves
|
|
already-created import directories usable.
|
|
|
|
## Controls and scope
|
|
|
|
Sharing is enabled for ordinary local cross-version tests. Set
|
|
`ORCA_DISABLE_SHARED_RELEASE_CACHE=1` to keep ordinary local extraction. CI and explicit
|
|
`cacheRoot` options remain isolated unless a caller explicitly supplies `sharedCacheRoot`.
|
|
|
|
This only changes test tooling. It adds no development-server task or background worktree
|
|
scan. Sharing does not migrate or delete existing local caches. Tests running on a remote host use that
|
|
host's repository and filesystem; no client filesystem or remote protocol is involved.
|
|
Folder contexts without a Git repository cannot provide a historical release checkout.
|
|
|
|
## Cleaning up older historical caches
|
|
|
|
Preview historical-cache cleanup across the registered local worktrees and shared source cache:
|
|
|
|
```sh
|
|
pnpm reclaim-space
|
|
```
|
|
|
|
The default selects stamped release checkouts older than 30 days, including previous extraction
|
|
formats. Answer `Y` after the preview to apply it, or run the same selection directly:
|
|
|
|
```sh
|
|
pnpm reclaim-space --apply
|
|
```
|
|
|
|
Use `--release-cache-min-age-days 7` for a shorter retention window, or `--release-cache-min-age-days 0` to select all
|
|
recognised completed release checkouts. `--repo DIRECTORY` selects another local repository.
|
|
Each run rescans and rechecks eligibility; it does not apply a saved preview.
|
|
|
|
Shared seeds are always eligible because the format lease blocks deletion while tests copy them.
|
|
Older harnesses do not hold a lease while importing a local checkout, so local checkouts are
|
|
selected only from worktrees idle for 7 days (no process references the worktree path, HEAD has
|
|
not moved, and no changed or untracked Git path was modified); active worktrees are reported as
|
|
retained. After stopping cross-version tests in all worktrees, add `--tests-stopped`
|
|
to include active worktrees too; it acknowledges stopped consumers but does not detect them.
|
|
Locks prevent deletion during publication or shared-source copying. Local checkout age uses
|
|
directory modification time, which ordinary reuse does not refresh; shared seed timestamps reflect
|
|
recent copying.
|
|
|
|
The script checks matching commit/format stamps, skips symlinked cache paths and future formats,
|
|
and leaves mutable `working-tree` copies, source, dependencies, and worktree registrations alone.
|
|
It reports logical bytes rather than promised disk savings: another clone may retain the same
|
|
blocks. Deleted historical source rebuilds on the next test run. Run the command on the execution
|
|
host for SSH repositories. The same command also performs P1's Electron maintenance; automatic
|
|
dev cleanup includes historical test caches under the idle-worktree rule, without `--tests-stopped`.
|
|
|
|
## Measurement and validation
|
|
|
|
A disposable macOS/APFS run against `v1.4.190` compared ordinary extraction with two separate
|
|
local caches using one shared seed. Each import tree contained 8,175 files. APFS private file
|
|
allocation was measured after all three views existed, without following symlinks.
|
|
|
|
| Method | Elapsed time | Private file data per import directory |
|
|
| -------------------------------- | ------------ | -------------------------------------- |
|
|
| Ordinary extraction | 4.64 s | 68.79 MiB |
|
|
| Shared source, creating the seed | 22.06 s | 44 KiB |
|
|
| Shared source, reusing the seed | 1.33 s | 44 KiB |
|
|
|
|
Shared blocks still occupy approximately one source copy. Private bytes exclude those blocks
|
|
and filesystem metadata, so they are not the total footprint. The cold seed costs extra work;
|
|
one timing sample on a busy machine does not establish a speed guarantee. Deleting the seed
|
|
left the other import directory readable. No existing worktrees were cleaned up.
|
|
|
|
Tests cover concurrent publication, lock heartbeats and eviction, independent branch stand-ins,
|
|
corrupt seeds, fallback, the opt-out, and preserved read-only files. Run them with:
|
|
|
|
```sh
|
|
ORCA_BACKGROUND_LAUNCH=1 pnpm test tests/e2e/cross-version-wire/release-source-cache.unit.test.ts tests/e2e/cross-version-wire/release-source-retention.unit.test.ts tests/e2e/cross-version-wire/release-checkout.unit.test.ts tests/e2e/cross-version-wire/release-checkout-tree.unit.test.ts tests/e2e/cross-version-wire/release-missing-packages.unit.test.ts config/scripts/artifact-storage/reflink-copy/space-sharing-copy.test.ts
|
|
```
|
|
|
|
## P3 remains an experiment
|
|
|
|
P3 shares the tracked source of new Orca worktrees. It requires independent writable files,
|
|
Git indexes, and matching checkout settings, and has more compatibility risk than P2's
|
|
immutable test source. The APFS experiment demonstrated substantial storage savings but
|
|
variable creation time and stays out of this change. Its code, measurements, and design notes
|
|
are preserved on `nwparker/tracked-source-sharing-experiment`; [the draft PR is closed](https://github.com/stablyai/orca/pull/27618).
|