mirror of
https://github.com/stablyai/orca.git
synced 2026-11-17 11:24:37 +00:00
Merge pull request #172 from nwparker/cloud/w9-eval
chore(cloud): desktop acceptance timestamps app logs; asleep delete verified at 1.5 s after #171
This commit is contained in:
@@ -176,8 +176,17 @@ async function launchApp(userData, label) {
|
||||
})
|
||||
child.stdout.removeAllListeners('data')
|
||||
child.stderr.removeAllListeners('data')
|
||||
child.stdout.pipe(out)
|
||||
child.stderr.pipe(out)
|
||||
// From here on, every app log line carries a wall-clock stamp so it can be lined up with steps.
|
||||
const stamp = (stream) => {
|
||||
let rest = ''
|
||||
stream.on('data', (d) => {
|
||||
const lines = (rest + d.toString()).split('\n')
|
||||
rest = lines.pop() ?? ''
|
||||
for (const l of lines) out.write(`${new Date().toISOString()} ${l}\n`)
|
||||
})
|
||||
}
|
||||
stamp(child.stdout)
|
||||
stamp(child.stderr)
|
||||
log('instance', instance.cdpUrl, instance.rendererUrl, 'runner pid', child.pid)
|
||||
const deadline = Date.now() + 300_000
|
||||
let seen = null
|
||||
@@ -603,6 +612,7 @@ async function uiDelete(step, name, label, timeoutMs = 90_000, watchAfterMs = 0)
|
||||
const dialog = (await d.innerText()).replace(/\s+/g, ' ')
|
||||
await shot(step, `${label}-confirm`)
|
||||
const t0 = Date.now()
|
||||
const clickedAt = new Date(t0).toISOString()
|
||||
await d.getByRole('button', { name: /^Delete( Workspace)?$/ }).click()
|
||||
let outcome = null
|
||||
while (Date.now() - t0 < timeoutMs) {
|
||||
@@ -621,6 +631,8 @@ async function uiDelete(step, name, label, timeoutMs = 90_000, watchAfterMs = 0)
|
||||
}
|
||||
outcome ??= { kind: 'timeout' }
|
||||
outcome.ms = Date.now() - t0
|
||||
outcome.clickedAt = clickedAt
|
||||
outcome.doneAt = new Date().toISOString()
|
||||
if (outcome.kind === 'removed' && watchAfterMs) {
|
||||
// Teardown can fail after the row is gone; look for the error wherever the app puts it.
|
||||
const t1 = Date.now()
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"<b>Wake delivery (2026-11-10, target p95 < 6 s):</b> on golden snap_aQmW (includes #141, #151, #162), one persistent connection per wake, n=22: total p50 5.13 s, p95 6.49 s, p99 6.72 s. Just over target at p95. Phases (p50 / p95): Vercel resume to orcad spawn 2.98 / 4.27 s; orcad ready 1.36 / 1.68 s; first terminal list incl. handshake 0.43 / 0.45 s; terminal create 0.14 / 0.17 s; echo 0.14 / 0.16 s. Before: 6.8 / 11.0 s (#141 only, reconnect per call) and 7.7 / 9.5 s (pre-#141). #151/#162 cut create+echo from 1.8 s to 0.29 s. What is left is Vercel resume (about 58% of p50) and orcad ready.",
|
||||
"<b>Rows closed 2026-11-10:</b> real 15-min idle sleep: an idle VM with no client stopped by itself after 19.2 min. Agent sign-in persistence: Claude answers, onboarding is pre-seeded, and the auth homes are kept fresh, after a wake and after a rollover. Auto-resume after Restart on m4air: one <code>continue</code> 29.4 s after the click, turn finished, no double continue. Wake on message: a message to a sleeping workspace was delivered in 6.1 s. Scheduled routine: fired by itself and succeeded on a fresh VM, which was then destroyed. File search: ripgrep over the pairing found both needles in 5,002 files, p50 88 ms. Usage metering: control-plane outbox, then billing hours, then the Stripe meter. Overage cap blocks new starts at $5. Dunning end to end: grace expiry (past_due_since backdated 97 h) leads the control-plane cron to suspend the running VM in 123 s; paying lets the account start again and the VM resumes. Desktop keystroke n=50: p50/p95/p99 84/94/144 ms; the 407 ms outlier did not recur. Web client is post-MVP.",
|
||||
"<b>Control-plane billing gate (closed via core #166):</b> a <code>dev:billing-test-*</code> account goes through the real billing Worker. No plan → 403 no_subscription; active plan → 200; 4th concurrent VM on 1 seat → 403; included hours used up → 403; failed renewal → 403 payment_past_due; grace expired → the running VM was suspended by the cron in 71 s and resume → 403; paid → resume 200. Staging team checks pass again (teams are deleted by the shared cleanup script).",
|
||||
"<b>CLI create (closed via W5 #167):</b> <code>orca worktree create --on cloud --repo …</code> with an API key, headless: running in 3.7 s; <code>orca cloud workspace destroy</code> works; signed out gives cloud_not_signed_in. <b>Move, Sleep, then Delete:</b> no wake, destroyed, Vercel 404, no misleading toasts. The row took 31 s to disappear, against about 2 s for other asleep deletes (sent to core-desktop as a perf note).",
|
||||
"<b>CLI create (closed via W5 #167):</b> <code>orca worktree create --on cloud --repo …</code> with an API key, headless: running in 3.7 s; <code>orca cloud workspace destroy</code> works; signed out gives cloud_not_signed_in. <b>Move, Sleep, then Delete:</b> no wake, destroyed, Vercel 404, no misleading toasts. Row removal went from 31 s to 1.5 s / 1.7 s after core #171, which skips the hooks check for asleep sandboxes.",
|
||||
"<b>orca.yaml setup in the VM (closed via core #168):</b> an API-created workspace on test-repo branch <code>w9-setup-script</code> ran scripts.setup inside the VM exactly once, including after sleep/wake. The environment-build skip path is unit-tested by core; the desktop path runs after the trust prompt.",
|
||||
"<b>W1-1 (fixed, #118/#123):</b> builds used to advance only while a client streamed <code>/events</code>; GET polling sat in <code>running</code> for 25 min. With the Durable Object runner deployed, builds finish when the client only polls GET: build 1 of a fresh environment in 17 s, build 2 in 18 s. An earlier 250 s first build had started before #123 went live. The harness now polls by default; <code>ORCA_ACCEPTANCE_BUILD_EVENTS=1</code> follows the events stream.",
|
||||
"<b>D-1 (bug, core desktop, high): — fixed in #127 (bab8369a229); main-process paths verified headlessly by core against the dev control plane; desktop UI re-run pending (no-windows rule):</b> starting an Orca Cloud create before signing in, then signing in from the card, created a sandbox owned by the fallback dev user (<code>desktop-dogfood</code>), not the signed-in user. Keep-alive, Copy SSH Command and Delete then got <code>not_found</code>. Likely cause: <code>getHostedSandboxConfig()</code> caches the token for the app process and does not reset it on sign-in (<code>src/main/hosted-sandbox/hosted-sandbox-config.ts</code>).",
|
||||
|
||||
Reference in New Issue
Block a user