starred/orca
1
0
Fork 0
mirror of https://github.com/stablyai/orca.git synced 2026-11-15 18:37:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(cloud): premerge runs at most 2 at once per machine; extra runs queue and log the wait

Browse Source 
The shared Mac hit load 60 from parallel premerges and vitest runs, starving the e2e harness. In the merged copy
every branch re-runs (#447), after the merge and before install/typecheck/lint/tests, the script now takes one of
PREMERGE_MAX_CONCURRENT (default 2) flock'd slot files under PREMERGE_LOCK_DIR (default /tmp/vdrv), held by a python
parent for the run (a killed run frees its slot). Queued runs print 'premerge: queued for one of 2 slots …, waited Ns'
every 30 s and 'got slot i/2 after Ns'. Tested: 3 concurrent runs with a limit of 2, the third queued 10 s.
This commit is contained in:
Neil
2026-11-11 11:42:24 -08:00
parent 3df8851c3b
commit 6f8a4e9183
1 changed files with 32 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
config/scripts/cloud-integration/premerge-check.sh
+32
View File
@@ -24,6 +24,38 @@ if [[ -z "${PREMERGE_IN_WORKTREE:-}" ]]; then
fi
dir=$PREMERGE_IN_WORKTREE
branch=${PREMERGE_BRANCH:-unknown}
# At most PREMERGE_MAX_CONCURRENT (default 2) premerges run their heavy part (install, typecheck, lint, tests) at once
# on this machine; the rest queue here and log while they wait (one shared Mac, load 60 starved the e2e harness).
# Slots are flock'd files under PREMERGE_LOCK_DIR (default /tmp/vdrv, else $TMPDIR), held by a python parent until the
# run ends, so a killed run frees its slot. This sits after the merge and before anything heavy, in the merged copy
# every branch re-runs (#447), so all branches pick it up. Set PREMERGE_SLOT=none to bypass (for tests only).
if [[ -z "${PREMERGE_SLOT:-}" ]]; then
lock_dir=${PREMERGE_LOCK_DIR:-$([[ -d /tmp/vdrv ]] && echo /tmp/vdrv || echo "${TMPDIR:-/tmp}")}
exec /usr/bin/python3 -I -c '
import fcntl, os, subprocess, sys, time
lock_dir, slots, script, branch = sys.argv[1], max(1, int(sys.argv[2])), sys.argv[3], sys.argv[4]
start, last, held = time.time(), -1.0, None
while held is None:
for i in range(1, slots + 1):
f = open(os.path.join(lock_dir, f"premerge-slot-{i}.lock"), "a")
try:
fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
held = (i, f)
break
except BlockingIOError:
f.close()
if held is None:
waited = time.time() - start
if last < 0 or waited - last >= 30:
print(f"premerge: queued for one of {slots} slots ({branch}), waited {int(waited)}s", flush=True)
last = waited
time.sleep(5)
waited = int(time.time() - start)
print(f"premerge: got slot {held[0]}/{slots} after {waited}s", flush=True)
env = dict(os.environ, PREMERGE_SLOT=str(held[0]))
sys.exit(subprocess.call(["bash", script], env=env))
' "$lock_dir" "${PREMERGE_MAX_CONCURRENT:-2}" "$dir/config/scripts/cloud-integration/premerge-check.sh" "$branch"
fi
trap 'git -C "$dir" worktree remove --force "$dir" >/dev/null 2>&1 || true' EXIT
cd "$dir"
pnpm install --frozen-lockfile >/dev/null