mirror of
https://github.com/stablyai/orca.git
synced 2026-10-01 16:02:03 +00:00
e899809ff88a29af6b00a9dff38d54f1df1f120f
46
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Neil
|
9179b93ebf |
ci: reduce repeated runner work and validate affected-test selection (#23540)
* ci: stage heavy checks and measure affected-test selection * fix(ci): exercise the real Git boundary in unit selection planning * Harden review cancellation and CI demand reporting |
||
  OrcaWinandm4air
|
27b823f934 |
ci: compile the E2E CLI once for all consumers (#23384)
* ci: share compiled CLI output across E2E consumers * ci: preserve CLI setup and old-ref fallback for shared artifacts * docs: record shared E2E CLI benchmark evidence * docs: include final CLI reuse timing range --------- Co-authored-by: m4air <m4air@m4airs-Air.localdomain> |
||
|
OrcaWinandm4air
|
c15f082031 |
ci: build independent Electron targets together for E2E (#23378)
* ci: reuse parallel Electron targets for E2E builds and guard cache action setup * test: recognize the top-level cache repository preload * docs: record E2E build timings and exact output parity --------- Co-authored-by: m4air <m4air@m4airs-Air.localdomain> |
||
|
OrcaWinandm4air
|
7a4f83336b |
ci: shard SSH Docker coverage and warm Windows native caches (#23286)
Co-authored-by: m4air <m4air@m4airs-Air.localdomain> |
||
|
Neil
|
403b62a8d8 |
ci: balance existing unit and E2E shards using recorded timings (#20367)
* ci: balance existing unit and E2E shards using recorded timings * ci: fix timing refresh units and deferred-menu test traversal * ci: preserve isolated E2E window launch policy * ci: keep diagnostic artifact outages from failing tests |
||
  Brennan BensonandMerge Sim
|
76c8e91d4a |
fix(e2e): run worktree first-paint probe on a mapped window (#20197)
Co-authored-by: Merge Sim <sim@local> |
||
|
Neil
|
e182930670 |
test: cover input in five simultaneously flooding SSH panes (#19071)
* test: cover keyboard input in five simultaneously flooding SSH panes * test: capture pane focus and buffers on flood input failure * test: capture pane focus and buffers on flood input failure * test: capture pane focus and buffers on flood input failure * test: record replay input loss and application fix dependency * test: record merged replay-input fix in the five-pane flood gate |
||
|
Neil
|
4d9e963ffd |
test: enable localhost SSH terminal and hook journey in CI (#19097)
* test: run localhost SSH terminal and hooks in CI * test: isolate localhost SSH session fixtures across repetitions * test: route remote agent hook source changes to localhost journey * test: record localhost SSH reliability evidence and remaining gaps * test: route the real SSH session hook authority |
||
|
Neil
|
f5960cec00 |
test: enable Docker SSH browser network route coverage in CI (#19095)
* test: enable Docker SSH browser network route journeys in CI * test: register Docker browser job in token permissions contract * test: declare SSH client dependency and narrow browser fixture routing |
||
|
Neil
|
3631f886a7 |
test: enable direct and client-hosted SSH browser coverage (#19090)
* test: enable direct and client-hosted SSH browser coverage * test: record twelve passing SSH browser journey repetitions * test: distinguish SSH journey evidence from unit runtime budget |
||
|
Neil
|
1d2e00819f |
test: restore SSH bulk-open freeze coverage in headed CI (#19081)
* test: restore SSH bulk-open freeze coverage in headed CI * test: record ten passing headed SSH freeze repetitions * test: record ten passing headed SSH freeze repetitions * test: route changed SSH freeze spec only to its dedicated lane |
||
|
Neil
|
eebedf206f | fix(tests): provide a window manager for Linux Electron CI (#19007) | ||
|
Neil
|
fd10758eae | ci: expose existing E2E spec selection for manual dispatch (#18987) | ||
|
Neil
|
7b467bd0a6 |
ci: gate PRs on a real input method, and prove the lane engaged one (#17365)
* ci: gate PRs on a real input method, and prove the lane engaged one No job on the PR gate has ever run a real input method. pr.yml and e2e.yml are ubuntu-latest with CDP `Input.imeSetComposition`, which is a synthetic composition; the only job that drives ibus-hangul through xdotool is terminal-ime-e2e.yml, and it is schedule + dispatch only. A PR could turn the real-IME path red and merge green. Route IME source to that lane from pr.yml through the existing pr-e2e-source-routing mechanism, so it runs on IME-touching PRs and nothing else. The lane stays out of verify.needs — advisory, like `e2e` — because its reliability is known only from nightly main runs. Deliberately no continue-on-error: that reports green and hides the signal. The harness fails open in ways that all look like success: Playwright reports a skipped test as a pass, so an unset ORCA_E2E_NATIVE_IBUS_HANGUL, a renamed test, or a session with no engine all exit 0 having exercised nothing. The specs now append an engagement receipt only after observing real composition events, and the runner requires one per expected test before the lane may report success. Also drop the native spec from changed-e2e: it was already routed there by its own filename, where it self-skips for want of an ibus session and reported that skip as coverage. * ci: let the real-IME step report even when the synthetic step failed |
||
|
Neil
|
63ff0a515d |
Prime native cache before E2E fanout (#17280)
* Prime E2E native cache before fanout * Update E2E permission contract |
||
|
Neil
|
51ed7d4f67 |
Pin pnpm and rebalance scheduled E2E (#17133)
* Pin pnpm and rebalance scheduled E2E * Give scheduled E2E failure headroom |
||
|
Brennan Benson
|
6bef6d2727 |
fix(ci): stop the Linux Electron probe step from starving its own probes (#17071)
* fix(ci): stop the Linux Electron probe step from starving its own probes The package job's "Test Linux Electron lifecycle boundary" step ran five Electron probe files under Vitest's default file parallelism, so four full Electron stacks competed for a 4-vCPU runner. Each probe carries its own in-process deadline (20s for WebRTC, 25s for H3), and every observed failure was one of those deadlines expiring: exit code 2, "no result", with every sibling file in the same run slower than its own green maximum. Run the step with --no-file-parallelism so each probe owns the runner, and stop each probe nesting a private `xvfb-run --auto-servernum` X server inside the step's own xvfb-run: reuse an inherited DISPLAY, and only own one when there is none (shards, local dev), which leaves those lanes unchanged. Also set each Docker-SSH E2E step's Playwright output aside before the next step starts, because Playwright empties test-results/ on every run and only the last lane's traces survived to the artifact. * fix(ci): route the persisted-worker probe through the same display resolver |
||
|
Neil
|
971d987c4b |
ci(e2e): trigger the Docker-SSH lane from SSH source and claim every gated spec (#16746)
The Docker-SSH e2e lane only ran when a PR's changed specs happened to include
`ssh-startup-exec-readiness.spec.ts` or `paired-startup-exec-readiness.spec.ts`.
Editing SSH source itself did not trigger it, and pruning either spec from a
route's list would have silently retired the whole lane. Meanwhile the sharded
lanes set no `ORCA_E2E_SSH_DOCKER`, so every Docker-gated spec skipped itself
while the shard still reported green -- the exact silent-skip shape
`docs/reference/ssh-reconnect-source-recovery.md` blames for four regressions
that reached users.
Separately, the modules that actually own direct-SSH workspace and tab restore
carry no "ssh" in their names, so the `ssh-terminal-source` route never reached
them. Measured on the real script before this change:
printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \
src/main/ipc/remote-workspace-snapshot-normalization.ts \
src/renderer/src/lib/worktree-initial-terminal-seeding.ts \
src/shared/remote-workspace-session-projection.ts \
| node config/scripts/pr-e2e-source-routing.mjs
=> []
Three changes, all pinned by the executable gate contract:
- `hasSshSourceChange` derives an `ssh_source_changed` signal from the SSH
routes themselves, plumbed pr.yml -> e2e.yml, so the lane triggers on source
rather than on a spec name surviving in a list. One list, so the two cannot
drift.
- A sibling `ssh-workspace-session-restore` route names the restore seams
(`remote-workspace-*`, `worktree-initial-terminal-seeding`,
`worktree-default-terminal-tabs`, `initial-terminal`) and routes them to the
two restore specs -- a sibling rather than more paths on `ssh-terminal-source`
so a tab-tombstone edit does not run the whole SSH terminal list.
- A new `test:e2e:ssh-docker` runner claims the remaining Docker-gated specs on
the one VM that sets the flag, and the contract now fails by name when any
Docker-gated spec is claimed by no runner. `ssh-docker-relay-perf` and
`ssh-codex-display-artifacts-repro` are recorded exemptions (wall-clock
budgets; needs a real remote codex binary) and the contract asserts each
exemption still corresponds to a real gated spec, so a stale one cannot
quietly excuse a gap. Lane timeout raised 35 -> 60 minutes for the added
serial specs.
The lane's first act was to surface four latent bugs in a spec that had been
silently skipping. `ssh-docker-bulk-open-freeze-repro.spec.ts` is four call sites
out of date against `tests/e2e/helpers/terminal.ts`: `startDockerSshRelayTarget()`
is called with no argument though the helper dereferences `testInfo.workerIndex`
(a 100% failure, not a flake), `execInTerminal` gained a `ptyId` parameter, and
`splitActiveTerminalPane` gained a direction. It was invisible because it ran
nowhere and `typecheck:e2e` is red on main with 240 pre-existing errors, so four
more could not be seen.
The `testInfo` bug is fixed here -- correct on its own, and it removes one real
error from `typecheck:e2e` (240 -> 239). The other three are not, because they
are not argument plumbing: repairing them requires choosing which ptyId to
capture and which split direction to use, and both change what the repro
measures.
The spec is therefore added to the exemption list rather than repaired, for two
independent reasons recorded in the runner: it is a perf oracle, not a
correctness one (`SOFT_FREEZE_LAG_MS=2500` / `HARD_FREEZE_LAG_MS=5000` measured
under a deliberate 5-pane flood on a 420s budget -- the same rule already applied
to `ssh-docker-relay-perf.spec.ts`), and it is known-rotted. Repair is tracked in
stablyai/orca#16764. Applying an existing written rule to a sibling that plainly
meets it is consistency; inventing a new exemption to dodge a red would not be.
Three hardening fixes to the contract itself:
- Runner text is comment-stripped before the claimed-by-a-lane scan. A substring
scan over raw text lets a spec merely *discussed* in a runner comment count as
claimed -- the silent skip this assertion exists to catch, re-entering through
the documentation. Not live today only because the existing comments write the
spec names without their `tests/e2e/` prefix.
- An exempt spec must not be invoked by any runner. `unreachableSpecs`
short-circuits the unclaimed check, so a spec could be documented as exempt
while a runner still ran it -- an exemption that reads as coverage removal but
changes nothing, leaving the lane red for a reason the file says it excluded.
This is not hypothetical: adding the bulk-open exemption without removing it
from the runner's spec list produced exactly that state, and this assertion is
what caught it.
- The Docker-gate detector is now `/ORCA_E2E_SSH_DOCKER\s*[!=]==\s*['"]1['"]/`
rather than one fixed string, so a double-quoted or `!==` spelling can no
longer escape the contract.
`ssh-restart-tab-accumulation.spec.ts` is a new three-cycle restart fence
asserting tab-id set identity, not just the active pane's reclaimed ptyId as
`ssh-cold-activation-restore.spec.ts:241` did. It passes today; it was validated
by a negative control that injected one tab after cycle 1 and correctly failed.
|
||
 Jinjing
|
9e0704b9d4 | fix(ci): prepare node runtime for e2e build | ||
|
Neil
|
350423b7cb |
Speed up PR CI with path skips, native caches, and fewer shards (#16863)
* Speed up PR CI with per-job path skips and native caches Skip git-compat, xterm, packaging, and shell jobs when their inputs are unchanged, reuse the composite install action (including Windows node-pty cache), skip compiling the Windows CLI launcher on a cache hit, and cut the test matrix from 16x2 to 8x2 shards without dropping coverage. * Widen PR job skip prefixes for orcad browser and live shells Chrome session/tab modules and zsh/fish wrapper templates are inputs to required jobs the classifier previously skipped. Include that implementation graph so those jobs still run when the files they load change. * Fix CI cache safety and required gates * Build scriptless Windows addons explicitly * Preserve node-pty Windows support prebuild * Remove duplicated Windows launcher unit lane |
||
 Jinwoo Hong
|
0a613d5fed | test(e2e): stabilize paired Quick Open large-tree coverage (#16058) | ||
|
Jinwoo Hong
|
a77a2f93f7 | fix(remote): search Quick Open paths on the host (#15158) | ||
|
Brennan Benson
|
53bd956ef6 |
ci(e2e): keep the relay version markers in the e2e build artifact (#15303)
build-relay.mjs writes each relay's marker as out/relay/<platform>/.version, and upload-artifact excludes dotfiles unless include-hidden-files is set. The markers were therefore stripped from e2e-build-out, so every consumer that actually starts a relay failed with: Orca's local relay build is missing its version marker at out/relay/linux-x64/.version This stayed latent because the sharded e2e lane never sets ORCA_E2E_SSH_DOCKER, so its SSH specs skip instead of touching the relay. The changed-specs lane does set it whenever a changed spec needs Docker SSH, which is why the failure only appears on PRs that touch SSH-adjacent code. |
||
|
Neil
|
442b46f020 |
ci(e2e): install a CJK font on the e2e runners (#15259)
The runners have no font covering Hangul, Han or Kana, so any spec that asserts how CJK text renders is measuring tofu rather than the glyph. That is not hypothetical. An end-of-row preedit spec added for #12729 measured the composition overlay at 1.02 cells against an 8.43px grid and failed its "wider than one cell" assertion. The overlay is shrink-to-fit with no width of its own, so it tracks the glyph's advance rather than the two cells the grid reserves for a wide character. With no Korean font that advance is one cell, and the assertion cannot distinguish a real result from a missing font - which is exactly the question that spec exists to answer. fonts-noto-cjk covers all three scripts and is added to both jobs that execute specs, the sharded suite and the changed-spec job, plus the ssh docker lane so the three stay consistent. This does not make any spec pass on its own. It makes the CJK ones mean something. |
||
|
Jinjing
|
7a695c70f1 |
test(e2e): harden triaged CI failures (#14656)
* test(e2e): harden triaged failures
* test(e2e): ship relay bundle to reusable shards
* test(e2e): tolerate expected IPC closures in daemon shutdown
A normal client exit can close the IPC channel before the finish ack
lands. Distinguish this from real failures by checking error codes,
only throwing if forced cleanup occurred or the error is not an IPC
closure.
* rm doc
* test(e2e): return termination status from legacy close handler
- terminateLegacyCloseClient now returns a discriminated union indicating
whether the process had already exited ('already-exited') or termination
was actually attempted ('termination-attempted')
- Allows finishLegacyCloseClient to only set forcedCleanup when termination
was genuinely needed, not when the process exited cleanly on its own
* test(e2e): fix dispatch contract and voice mic locator
Point the release E2E contract at the renamed build step, and assert the
relabeled microphone through the Voice pane combobox even when Radix
leaves the listbox open.
* test(e2e): add contract test for relay artifact dispatch
Validate that the relay artifact built in CI is properly uploaded,
downloaded, and passed via ORCA_RELAY_PATH to E2E test runs.
* Distinguish between terminated and already-exited processes
Detect when processes have already exited instead of always reporting
termination success. Return booleans from cleanup functions to indicate
whether they actually signalled a process, catch tree-capture failures
when the root process exits before recording completes, and use these
signals to return accurate exit status from termination handlers.
* test(e2e): stabilize file creation and voice microphone tests
Use stable locators (aria-autocomplete, named triggers) and add retry
logic to handle file scans and device events that can interfere with
listbox state. Increase timeouts to allow async operations to complete.
* Add retry logic for transient GitHub API errors in PR body updates
GitHub API occasionally returns transient 5xx errors. Retry up to 3 times
with exponential backoff (1s, 2s, 4s) to improve reliability during
temporary service disruptions. Export updatePullRequest and add sleepImpl
parameter for test injection.
* Add tab search result retention during typing
Keep search results on screen while the deferred query catches up with
the live query. Re-validates results against the current input without
dropping rows prematurely, ensuring the user can select from what they see.
* Add proper types to tab search mock
Replace `unknown` with concrete types (`OpenTabSearchResult`,
`OpenTabSearchEntries`, `SearchableWorkspaceTab`) and use type guards
for discriminated unions to improve test type safety.
|
||
|
Neil
|
13b10e0b54 |
ci: cut PR wall clock by caching what CI recomputes every run (#15211)
None of these change what CI checks — they remove work the runners repeated on every PR. - install-node-dependencies installed with --no-frozen-lockfile, so every job re-resolved the graph against the registry to recompute what the lockfile already pins. Measured at ~62 MB of packument metadata per job; the pnpm store cache does not cover the metadata cache, so this was paid ~39 times per run. The `git diff` guard that made the re-resolution redundant stays. - --ignore-scripts leaves node-pty with no build/Release, so ensure-native-runtime node-gyp-compiled it in every job asking for a runtime. Cache the build under an ABI-bound key (runtime, resolved Node version, node-pty patch) with no restore-keys, since a partial match is exactly the mismatched build that would be recompiled anyway. - The four fetch-depth: 0 checkouts pulled full history including every historical blob (blobs are ~89% of this repo's pack). They only need the commit graph for a merge-base diff, so fetch them blobless. Measured 30-43s each today versus 8s for the shallow checkouts. One of them, e2e-paths, gates the entire E2E chain. - E2E jobs ordered setup-node before pnpm, which meant setup-node could not find the store and no E2E job cached dependencies at all. Reorder and cache; this sits on the critical path in both the build job and each shard. - git_compatibility rebuilt Git 2.25.5 from a pinned tarball on every PR. Cache the build; the sha256 assertion still guards the miss path. - typecheck ran three independent tsc passes back to back and discarded the .tsbuildinfo each project already emits. Run them concurrently and cache the incremental state. - package (windows) built the electron-vite targets serially via build:release. Use a :parallel variant that overlaps them, matching what the Linux package job already packages and smoke-tests from. Contract tests cover each new cache's ordering and key so none of them can silently start serving a stale or ABI-mismatched artifact. |
||
|
Neil
|
24e662adc1 |
feat(ssh): verify host keys, and restore panes correctly across a reconnect (#14844)
* docs(ssh): design for real host key verification (STA-4319)
Today's ssh2 verifier records a fingerprint and returns true — every host key is
accepted, with no known_hosts consult and no change detection anywhere in
src/main/ssh/. Scope is per-connection, so exec, SFTP, port forwarding, the
watcher and relay deploy all ride that one unverified handshake, and the
ProxyJump path puts the final hop — the topology most likely to cross untrusted
network — on ssh2 specifically.
Decisions worth calling out:
- Read the user's known_hosts as a trust source but NEVER write to it. That file
is shared with every other SSH tool on the machine; appending means line
endings, permissions, concurrent writers and a corruption blast radius well
beyond us. Accepted keys go to our own per-target store. Reading theirs is also
the entire migration story: most developers already have their hosts there.
- Mismatch is scoped to the SAME key type. A host with only an RSA entry that
presents ed25519 is unknown, not changed. ssh2 negotiates ed25519 first, so
without this we would fire a change-of-key alarm at nearly every existing user
on their first upgraded connect — training them to dismiss the one warning that
is supposed to mean something. Flagged in review as the decision I am least
sure of; a downgrade-vector argument against it is being tested.
- Changed key hard-fails with no override button; recovery is a separate explicit
action, offered only when OUR store is what disagreed, because forgetting our
record cannot unblock a known_hosts conflict.
- Background reconnects deny rather than prompt. A dialog the user cannot place
in context only teaches click-through.
Two traps are documented because either would make the fix silently do nothing:
an async verifier returns a Promise, which ssh2 reads as truthy and accepts
immediately; and the existing test mock invokes hostVerifier with one argument
and ignores the return, so it would pass against a verifier that never decides.
Design only — no behaviour change. The doc is added to the tracked-reference
allowlist in .gitignore alongside the other docs/reference entries.
* docs(ssh): revise the host key design after security and migration review
Three things the reviews changed, kept visible rather than quietly edited out.
THREAT MODEL WAS WRONG IN THREE PLACES. Jump hosts are not the worst case — they
are already safe: shouldUseSystemSshTransport branches on exactly the inputs
resolveEffectiveProxy does, and attemptConnect returns after the system probe, so
ProxyJump goes through OpenSSH and is verified. Agent forwarding was overstated
(gated on the user's ForwardAgent). Credential theft was understated: any auth
error counts as agent fallback, so a MITM walks the user to the password AND
private-key passphrase prompts, and cachedPassword replays without prompting. The
relay claim was backwards — the attacker owns their own machine; the real impact
is the return direction, where they become the host our workspace trusts.
TYPE SCOPING IS A DOWNGRADE VECTOR WITHOUT ALGORITHM ORDERING. This was the
decision I flagged as least certain and asked to have argued both ways. OpenSSH
is safe only because order_hostkeyalgs() puts known types first and RFC 4253
gives the client's order priority. ssh2 negotiates ed25519 first regardless, so
an attacker who cannot forge the RSA key on file just presents ed25519 and gets a
friendly first-contact prompt instead of a hard failure. Keep scoping, but set
algorithms.serverHostKey to lead with the types on file — and add a sixth
outcome for 'unknown type, known host', which must never read as first contact.
SHIP THE DEFENCE BEFORE THE DIALOG. Startup restore fires eager connects for all
targets in parallel with a 15s timeout while a prompt would live 120s; ephemeral
VM targets present a new key every launch; paired-web connects run on the host
desktop, so the dialog opens on someone else's screen. Phase 1 is therefore no
modal at all: consult known_hosts and our store, match connects, unknown persists
with accept-new semantics, mismatch and revoked hard-fail. That is the whole MITM
defence with none of the migration risk.
Also folded in, verified live against OpenSSH 10.2p1: the without-port fallback
(bracketed lookup first, then bare, where the second pass can only yield match or
unknown — otherwise a bare line plus a non-default port produces a spurious
prompt); hashed entries hash the candidate form; multiple files union; a
cert-authority line does not match a plain key. IPv6 and bracket parsing moved
INTO scope — that is a parser requirement, not a scope call, and getting it wrong
produces the prompt-training harm the design exists to avoid.
* feat(ssh): parse and match OpenSSH known_hosts
The matcher half of STA-4319. No behaviour change yet — nothing calls this.
Hand-rolled because no maintained JS implementation exists, and written against
behaviour observed from OpenSSH 10.2p1 rather than inferred from the man page.
Three of those behaviours a reasonable reading gets wrong:
- A non-default port is TWO ordered lookups, not one candidate set: '[host]:port'
first, then bare host ('checking without port identifier' in ssh -v). The
fallback pass can only yield match or unknown — OpenSSH downgrades a wrong key
there rather than reporting a change. Collapse them and anyone holding a bare
line who connects off-port gets a spurious first-contact result; treat the
fallback as authoritative and they get a false change-of-key alarm.
- Revocation resolves in its own pass so the verdict cannot depend on line order.
Verified both orderings.
- A cert-authority line never matches a plain host key; it only validates
certificates. A normal line alongside it still decides.
Mismatch is scoped to the same key type, and a host known by a DIFFERENT type
returns unknown-type-known-host rather than plain unknown — an attacker who
cannot forge the key on file must not get a friendly first-contact result by
presenting another type. That outcome is only half the defence; the other half
(leading serverHostKey with known types) lands with the wiring.
47 tests from vectors executed against real sshd, including ssh-keygen -H hashed
entries. Each of six mutations reddens it: collapsing the passes, letting the
fallback report mismatch, dropping type scoping, resolving revocation in line
order, honouring an unrecognised marker, and skipping the blob/type agreement
check.
* feat(ssh): decide what to do with a presented host key
The policy half of STA-4319, kept separate from the ssh2 wiring so it is testable
without a handshake and injected rather than importing its sources, so a test
states its own trust state instead of writing files.
Phase 1 ships no dialog — a test asserts the decision is never 'prompt'. Startup
restore opens every previously-active target at once, ephemeral VM targets would
ask every launch, and paired-web connects run on the host desktop where the
dialog would appear on someone else's screen.
Ordering that matters: revocation outranks everything including
StrictHostKeyChecking=no, because a revoked key is a statement that this key is
known-bad rather than merely unrecognised. known_hosts is named before our own
store on a change, because its remedy (ssh-keygen -R) is the one that also
unblocks ssh and git — pointing at a remedy that cannot work is worse than none.
Two carve-outs with reasons: an ephemeral runtime target accepts WITHOUT
recording, since a fresh VM presents a new key every launch and a stored record
would accumulate per launch and eventually read as a spurious change; and when
ssh -G ran on the HOME-divergent path that suppresses /etc/ssh/ssh_config, an
unknown host is denied, because a site-wide policy may forbid it and being laxer
than ssh is the one outcome that is never acceptable.
Rejection text deliberately avoids 'authentication failed' and 'permission
denied': the reconnect ladder classifies on those substrings, so a denial phrased
that way is retried forever against a decision that will never change. Pinned by
a test.
* feat(ssh): build the host key verifier and the algorithm order that makes it safe
Still not wired into the handshake — that lands next. This is the piece that
turns a decision into an ssh2 callback, plus the half of the design that is easy
to forget because it lives in a different config field.
The verifier MUST be a plain function returning undefined. ssh2 does
'const ret = verifier(key, verify); if (ret !== undefined) verify(ret)', so an
async function returns a Promise — neither undefined nor falsy — and ssh2 accepts
the key immediately while ignoring whatever the callback later decides. Making
this async would silently restore exactly the accept-everything behaviour the
module exists to remove, so a test asserts the return value is undefined.
orderServerHostKeyAlgorithms is what makes type-scoped matching safe rather than
a downgrade. RFC 4253 gives the client's algorithm order priority, so leading
with the types we already hold for a host denies a server the choice of
presenting some other type to convert a hard failure into first contact. Without
it, an attacker who cannot forge the key on file just offers a different
algorithm. Revoked entries never contribute to that order.
Also fails closed on two paths that would otherwise hang or over-trust: a key
whose own length-prefixed header cannot be read is refused rather than reasoned
about, and a throw from any dependency denies, because ssh2 may not catch an
exception raised inside the verifier and the handshake would hang instead of
failing.
18 tests. Includes the two negative cases that matter — first-contact keys are
recorded, but keys we already know, rejected keys, ephemeral runtime targets and
a lax StrictHostKeyChecking are not.
* fix(ssh): promote every RSA signature algorithm for a known ssh-rsa key
A known_hosts entry names the KEY type, which is not the negotiated ALGORITHM
name. One ssh-rsa key is offered as rsa-sha2-512, rsa-sha2-256 or ssh-rsa
depending on the signature algorithm, so matching the literal name only would
leave a host we know by RSA ordered behind ed25519 — precisely the ordering this
function exists to prevent, and precisely the population (RSA-era known_hosts
entries) it was written for.
Verified from ssh2's own negotiation while wiring this: kex.js iterates the
CLIENT list and takes the first entry the server also offers, so client order
does decide, as RFC 4253 says. ssh2's default order leads with ed25519 and places
the RSA algorithms fifth through seventh.
* fix(ssh): verify host keys instead of accepting every one (STA-4319)
The actual fix. ssh-connection's verifier recorded a fingerprint and returned
true, so every ssh2 connection accepted every host key — no known_hosts consult,
no change detection. It now consults the user's known_hosts plus our own store
and refuses a changed, revoked or unverifiable key.
Phase 1 by design: no dialog. Unknown hosts are accepted and recorded
(accept-new semantics), because startup restore opens every previously-active
target at once, ephemeral VM targets present a new key each launch, and
paired-web connects run on the host desktop where a prompt would appear on
someone else's screen. The MITM defence lands now; the prompt is Phase 2.
Also sets algorithms.serverHostKey to lead with the types already known for the
host. Without it the type-scoped matching is a downgrade — an attacker who cannot
forge the key on file just presents another type and turns a hard failure into
first contact. Verified from ssh2's kex.js that the client list decides.
Denial replaces ssh2's generic handshake error with the specific reason, because
the reconnect ladder cannot distinguish a generic failure from a transient fault
and would retry forever against a decision that will never change.
An unreadable trust store degrades to known_hosts only rather than failing the
connect: a changed key is still refused, and a host trusted only by us falls back
to first contact and is re-recorded, reaching the same decision.
The ssh2 mock now uses the callback form and aborts the handshake on denial. As
written it called hostVerifier(key) with one argument and ignored the result, so
it would have passed against a verifier that never decides — flagged in the
design as a mock that had to change, not a test to quietly rewrite. Two new tests
pin the wiring rather than the module: an unidentifiable blob is refused, and a
well-formed key is accepted.
Note for review: commit
|
||
|
Jinwoo Hong
|
cd6114ab7e | fix(browser): acknowledge paired tab before navigation (#14402) | ||
|
Jinwoo Hong
|
77b37d85e2 |
feat(vm): create workspaces from provisioned SSH roots (#14359)
* feat(vm): use recipe-provisioned SSH roots * fix(vm): preserve ordinary create failure timing * test(vm): prepare provisioned root SSH fixture * ci(vm): enable SSH setup for provisioned root E2E |
||
|
Jinwoo Hong
|
9a10561258 | fix(terminal): retain SSH startup delivery through reconnect (#14161) | ||
|
Jinwoo Hong
|
de729d6067 | Fix paired web creation failure handling (STA-4024, STA-4025, STA-4063) (#14100) | ||
|
Neil
|
676964b099 | ci: run only changed e2e specs on pull requests (#11834) | ||
|
Brennan Benson
|
e467b3ff7b |
fix(remote): stabilize shared control and terminal parking (#11656)
* fix(remote): stabilize shared control and terminal parking * fix(remote): harden parking review edge cases * fix(terminal): restore parked local floating buffer * fix(ci): drop superseded paired parking evidence * fix(terminal): preserve floating park watchers * fix(ci): include web client in paired e2e artifact * fix(ci): reuse renderer build for paired e2e |
||
 JinjingandOrca
|
a7c8b8e071 |
fix(terminal): bound SSH & remote hidden-worktree terminal retention (C1) (#10625)
* fix(terminal): park SSH worktrees like local ones (C1 retention, slice A) SSH ptys were blanket-excluded from hidden-view parking, so a hidden SSH worktree retained every pane forever (C1: renderer heap climbs to the V8 ceiling). SSH bytes transit local main — fact-mode watchers already cover them, and main keeps a headless model served over pty:getMainBufferSnapshot that the SSH reattach path never consulted. - isParkRestorableTerminalPty: snapshot-backed OR (SSH + policy); threaded through both park verdicts, both selectors, watcher coverage, and the watcher start guard. Remote-runtime/fail-open/foreign/null unchanged. - Parked-SSH reveal paints from main's headless model (dimension-matched, ~5k rows) and degrades to the relay 100KiB replay unless the snapshot is a non-empty source==='headless' payload — never a blank/stale paint. - Kill switch: settings.terminalSshViewParking (default on). DESIGN.md records the approved plan and the H1 magnitude non-claim. Co-authored-by: Orca <help@stably.ai> * fix(terminal): bound hidden-worktree retention with a force-park budget (C1, slice B) Un-parkable worktrees (remote-runtime ptys, uncoverable tabs, SSH with the slice-A switch off) had unlimited retention: the parking cap/TTL only ever saw eligibility-passing worktrees, so one bad tab pinned a whole worktree's panes forever. Retention is now memory-bounded, not eligibility-bounded. - terminal-hidden-worktree-retention.ts: retention budget (12 hidden / 45min TTL, sized from the measured 2.5-19MB per-pane V8 cost, DESIGN.md §2) over hidden worktrees ordinary parking can never evict; reuses the hot-retain ranking so last-active exemption, deterministic ties, and deadline-driven rechecks hold. Fail-open/foreign-pty tabs are eviction-exempt (a remount would fresh-spawn and orphan the live shell). - Terminal.tsx: force-parked ids join the parked set AFTER the coverage veto (darkness for uncoverable tabs is the accepted cost); buffers captured via the sleep-flow registry before the unmount render; retention TTL added to the recheck deadlines for budget candidates only. - Verdict stays out of its own effect deps; policy test asserts idempotence and time-monotone membership (flip-loop dwell regression). - Kill switch: settings.terminalHiddenWorktreeRetentionBudget (default on). Co-authored-by: Orca <help@stably.ai> * fix(terminal): demote hidden scrollback for eviction-exempt worktrees (C1, slice C) The retention budget (slice B) must exempt worktrees holding fail-open or foreign-worktree ptys — a remount would fresh-spawn and orphan the live shell — which would leave that class unbounded again. Instead, past the same 45min retention TTL their hidden panes drop to the minimum scrollback tier (measured: ~19MB -> ~1.3MB V8 heap per 50k-row pane; trimmed history is gone by design, reveal restores the configured cap for future output). - terminal-hidden-scrollback-demotion.ts: module-state verdict registry (parked-watcher pattern) with content-equality notify damping; applied in the existing scrollback-rows effect in use-terminal-pane-lifecycle. - selectScrollbackDemotedTerminalWorktrees: pure, TTL-gated, time-monotone. - Retention TTL wakeups now also cover exempt worktrees so demotion fires. - Kill switch: settings.terminalHiddenScrollbackDemotion (default on). Co-authored-by: Orca <help@stably.ai> * fix(terminal): paint the SSH model snapshot inline, not via nested coordinator (C1 slice A fix) applyMainBufferSnapshot runs its own structuralReplayCoordinator.run; calling it from applyReattachPayload (already inside the coordinator when a relay replay exists) deadlocks on the coordinator's tail chain. The model paint now mirrors the daemon-snapshot branch inline (folded scrollback + rehydrate + screen, dimension-matched, escape tail last) and arms the restored-snapshot seq baseline so deferred/live chunks the snapshot covers dedupe instead of double-painting. Also falls through (no early return) so reattachPayloadApplied still latches. Adds the folder-workspace id parity unit case. Co-authored-by: Orca <help@stably.ai> * test(terminal): SSH park+reveal e2e round-trip + as-built design notes (C1) Docker-gated (ORCA_E2E_SSH_DOCKER=1) spec: SSH tab parks behind a decoy and reveal restores marker content at multi-viewport scrollback depth. DESIGN.md records the as-built deltas (inline paint, force-park shape, last-active floor) and the residuals so follow-ups aren't lost. Co-authored-by: Orca <help@stably.ai> * fix(terminal): paint SSH reveal from main's model even when the relay replay is empty (C1 review #1) A relay restart empties the replay buffer; the reveal previously painted nothing even when main's headless model held the session. The reattach now prefetches the model snapshot when no structural replay exists (SSH-shaped ptys only) and paints it inside the coordinator; emptiness is judged on the composed payload (scrollbackAnsi + data + pendingEscapeTailAnsi) so an alt-screen snapshot with an empty screen frame still paints. Co-authored-by: Orca <help@stably.ai> * fix(terminal): decouple scrollback demotion (slice C) from the retention-budget switch (C1 review #2) Per the approved contract each slice reverts behind its own switch: slice C now requires only the master terminalHiddenViewParking plus its own terminalHiddenScrollbackDemotion flag. The TTL wakeup timer fires for demotion candidates even with the budget switch off. No DEFAULT_SETTINGS entries exist for sibling flags (defaults are the '!== false' optional pattern), so no explicit defaults are added. Co-authored-by: Orca <help@stably.ai> * fix(terminal): scope eviction exemption to the tab, not the worktree (C1 review #3) One eviction-exempt tab (fail-open/foreign pty) previously vetoed force-park for its whole worktree, pinning co-located remote-runtime tabs forever. The worktree now force-parks while exempt tabs keep their mounted panes via a per-tab exclusion mirroring the Activity-portal pattern (legacy watcher sync, legacy render, and the overlay cold-parking hook). Ordinary parking is untouched — a worktree with an exempt tab still cannot ordinary-park. Slice C now also demotes exempt tabs' panes as soon as their worktree force-parks under the count budget (they are the only panes left mounted). Co-authored-by: Orca <help@stably.ai> * fix(terminal): demote un-parkable worktrees the force-park lever spared (C1 review #4) The last-active exemption means a single hidden un-parkable worktree never force-parks — and slice C previously only targeted exempt-tab worktrees, so its panes held full scrollback forever. Demotion now also covers un-parkable non-exempt worktrees past the retention TTL that are absent from the force-parked set (last-active spared, or slice B switched off). Membership stays time-monotone for fixed inputs; covered by new idempotence/monotone selector tests. Co-authored-by: Orca <help@stably.ai> * fix(terminal): keep the hidden clock running through transient background-measure windows (C1 review #5) Whole-worktree background mounts (browser-automation bootstrap lease, mobile mounts, agent wakes) open a ~3s self-clearing measure window that previously deleted hiddenSince — every remount restarted the 30s hysteresis and the 45min retention TTL, so a periodically re-mounted force-parked worktree never re-parked. The measure window still pauses parking/eviction verdicts (all selectors skip measuring candidates); only the clock survives, so the prior verdict resumes as soon as the window closes. Visible and portal-holding worktrees still reset the clock. Co-authored-by: Orca <help@stably.ai> * test(terminal): make the SSH park+reveal depth assertion prove the model paint (C1 review #6a) Pad the session with ~180KB of output after the numbered markers so the earliest marker falls outside the relay's 100KiB rolling replay buffer while staying inside main's ~5k-row headless model; asserting marker_1 after reveal now proves the headless-model paint rather than passing under the relay fallback. Co-authored-by: Orca <help@stably.ai> * docs(terminal): rewrite DESIGN.md as the single as-built C1 contract (review #7) One contract matching the code: status IMPLEMENTED around force-park (not the unmount proposal), real kill-switch names with coupling + revert matrices, the true retention-floor formula with measured per-pane and demotion numbers, an explicit when-OOM-is-still-possible paragraph naming the H2 pendingSideEffects residual, the applyMainBufferSnapshot deadlock constraint inside the slice-A section, stable-signal phrasing instead of a capability latch, fail-open AND foreign-worktree exemption class, verified cites, and a planned/landed/follow-up test matrix. Co-authored-by: Orca <help@stably.ai> * fix(terminal): resolve the eviction exemption per pane, not per tab (C1 review #8) isEvictionExemptTerminalTab read only tab.ptyId — the FIRST leaf's pty — while the coverage veto that makes a worktree a retention candidate walks every pane. A split tab whose second leaf held an unrestorable pty therefore failed coverage (→ force-park target) yet looked exempt-free, so force-park unmounted it and orphaned the live shell. The exemption now resolves panes through the same resolveParkedTerminalPaneCandidates, keeping tab.ptyId in the union for the no-layout/no-capture case. Also from the same review round: - force-park's capture passes includeLocalBuffers:false like every other shutdownBufferCaptures caller; it was serializing up to 512KB/pane of scrollback into the store inside a fix meant to bound renderer heap. - Terminal.tsx unmount resets the scrollback-demotion registry — module state with no reset path, read by a pane effect that runs before the host effect that would clear it, so a stale verdict trimmed restore replays. - memoize watcher coverage per tab within the parking pass; the retention candidates re-asked it for every mounted worktree, not just the parked few. * docs(terminal): drop DESIGN.md — the as-built C1 contract moves to the PR body Co-authored-by: Orca <help@stably.ai> * fix(terminal): cap the deferred PTY side-effect queue (C1 residual H2) pendingSideEffects grew without bound under background timer throttling (~64 drained/s vs hundreds queued/s overnight). Cap at 512 entries with oldest-first eviction: titles drop (last-wins), a pending bell latches onto the next survivor, agent-status payloads collapse onto the survivor keeping the newest 16 (last-wins store state, KB-scale strings). Co-authored-by: Orca <help@stably.ai> * fix(terminal): carry command-lifecycle facts through parked watchers (C1 follow-up) Parked fact-mode watchers omitted onCommandFinished/onCommandCode*, so OSC 133;D and Command Code scrape signals went dark while parked. New parked-terminal-command-status.ts ports the store-level subset: git-UI nudge on every command finish, same-turn status-row drop for SSH PTYs (exact mounted-path parity — the foreground tracker refuses SSH ids), and the Command Code working seed / 1500ms done settle. Byte mode scans the same shared parsers for authority-off parity. Local-PTY status drops stay with the mounted pane: they need pty-connection's process-confirm ladder to tell a leaked nested-shell 133;D from a real agent exit. Co-authored-by: Orca <help@stably.ai> * test(terminal): retention-budget force-park e2e with a retentionLimit override (C1 6b) ORCA_E2E_TERMINAL_RETENTION_LIMIT flows preload → e2e-config → getTerminalParkingPolicyOverrides (exposeStore-gated, positive-integer only) so a spec can shrink the force-park budget to 1. The Docker-gated spec opens two remote worktrees on one relay target (second pre-seeded remote repo), disables terminalSshViewParking to make both un-parkable, hides both behind the local context, and proves the older one force-parks while the last-active exemption spares the newest; re-activating the evicted worktree restores the marker tail via relay replay. Co-authored-by: Orca <help@stably.ai> * test(terminal): retention-budget e2e via same-repo remote worktrees (passes docker lane) The first draft added a second remote repo mid-session, whose pane pty spawn misroutes to the local daemon with the remote cwd (pre-existing multi-repo issue, reproducible without any retention override — a seeded local repo plus one remote repo shows the same misroute). The spec now budgets across three worktrees of the ONE connected repo, created through the product createWorktree path (an external git-worktree-add only lands as a detected worktree needing adoption) and polled through the relay's transient post-connect reconnect window. Verified green on the local Docker lane in 20.8s. Co-authored-by: Orca <help@stably.ai> * fix(terminal): prevent remount thrashing during post-measure cool-down ( Implements the C1 retention contract: preserve worktree `hiddenSinceMs` through a background-measure window (so TTL/ranking stay honest), but re-park waits for a full `coldParkDelayMs` cool-down after the measure ends. Without the cool-down, every ~3s measure lease on a past-deadline worktree thrashes remount/reattach. Core changes: - Terminal.tsx: add measure clock (measuringTerminalWorktreeIdsRef) and post-measure cool-down tracking (terminalWorktreeParkCooldownUntilRef); gate parking candidates until cool-down expires. - Extract snapshot replay choreography to shared terminal-snapshot-replay-paint.ts (used by SSH reattach + daemon restore paths). - Add SSH model snapshot timeout (750ms) with fallback to relay replay. - Move cold-park recheck deadline logic to terminal-cold-park-recheck-deadlines.ts; add cool-down deadline to scheduling. - useTerminalTabColdParking: implement matching measure-clock contract with per-tab cool-down gate to keep tab deadlines synced with worktree retention clock. - Add resolveTerminalMountScrollbackRows() to demote new xterms under demoted worktrees (pane births during demotion must take the demoted tier at create). - Add kill switches: terminalSshViewParking, terminalHiddenWorktreeRetentionBudget, terminalHiddenScrollbackDemotion. * fix(terminal): detect Command Code completion in parked mid-turn panes Seed the byte watcher with in-flight turn state from agent status: the watcher is recreated per park cycle with no startup command to arm it, and the banner scrolled away before parking. Also memoize eviction-exempt checks and use SSH PTY ID builder in tests. * fix(terminal): flush pending command-code settles on reveal remount When a parked pane reveals mid-Command Code turn, the new detector cannot re-observe the already-passed idle composer. Cancelling the settle leaves the row stranded at 'working', so dispose now flushes the pending settle instead. Extract readInFlightCommandCodeTurn to shared space and seed detectors with in-flight turns so remounts complete mid-flight commands. Also memoize SSH model probes to prevent double timeouts on reattach. * fix(terminal): remove scrollback demotion (C1 slice C) The scrollback demotion feature for eviction-exempt hidden worktrees is no longer needed. Retention budget limits are now sufficient without this additional bound. Remove the terminal-hidden-scrollback-demotion module, the selectScrollbackDemotedTerminalWorktrees function, and related per-pane demotion logic. * test(terminal): assert bounded probe during stalled reveal Add assertion to verify that a stalled reveal operation makes exactly one `getMainBufferSnapshot` call, ensuring retry logic doesn't introduce redundant probes that would extend the timeout window before relay fallback. * fix(terminal): implement C1 retention budget for hidden parked worktrees Addresses OOM regressions in hidden parked terminals by force-evicting worktrees past a retention budget: at most 12 mounted while hidden, none past 45 minutes (absolute, not exempted by last-active). Eviction is least-recently-hidden-first. Exempt tabs (unrestorable local PTYs) keep their panes to avoid orphaning shells; worktrees are force-parked even if they contain exempts, and their buffers released elsewhere. SSH/remote worktrees serialize buffers pre-eviction for reveal; local worktrees keep daemon snapshots. Command Code's done-settle window is transferred across park/reveal boundaries so the row cannot strand at 'working'. Model probe on SSH reattach is scoped to park-reveal only, not ordinary reconnects. Includes new E2E suite proving the budget actually releases memory. * memoize eviction-exempt terminal tabs to avoid redundant store reads Each tab's exemption check re-reads the store and walks the layout tree. Introduce selectEvictionExemptTerminalTabIds() to resolve all exempt tabs for a worktree in a single pass, then memoize the result in Terminal.tsx and useTerminalTabColdParking. This prevents O(n) store reads when checking exemptions across multiple tabs and ensures the set remains stable across unrelated re-renders. * refactor: reformat hidden-worktree retention comments Reflow to 80-character lines and remove internal ticket references (C1, C1 slice C). * fix(lint): split overlay slot and eviction-exempt tabs under max-lines Static analysis failed because TerminalPaneOverlayLayer (401) and terminal-parked-tab-watchers (304) exceeded oxlint max-lines. Extract the slot component and eviction-exempt helpers into dedicated modules. * test(terminal): stabilize retention budget e2e control arm Stage un-parkable remote pty ids only after both worktrees are hidden, and keep re-staging during the control-arm poll so a late updateTabPtyId cannot flip the decoy back to park-restorable and ordinary-park it before budget engages. * test(terminal): pin retention e2e decoy to a mounted pane snapshot Use the active pane-identity snapshot for the decoy tab instead of all worktree tabs, and re-assert un-parkable ids after the control-arm hold so a deferred/empty tab id cannot fail the budget-off mounted-count check. * fix: memoize terminal eviction exemptions on layout leaf PTYs Splits add leaf panes to the layout store without changing the tabs array. A memo keyed only on tabs misses this change, leaving new panes unexempted for unmount. Include layout leaf PTYs in the exemption memo key so it recalculates when splits occur or PTYs are re-minted. --------- Co-authored-by: Orca <help@stably.ai> |
||
 JinjingandOrcaWin
|
a40183389b |
feat: bound direct SSH reconnect fan-out and recovery (#11003)
* docs: design for direct SSH reconnect fan-out Capture the implementation-ready plan for host-qualified, epoch-fenced SSH reconnect recovery after two rounds of multi-model LLM counsel review. * docs: reconcile SSH reconnect fan-out design * docs: close reconnect design consistency gaps * feat: implement bounded direct SSH reconnect recovery * fix: bound direct SSH retry settlement * fix: harden direct SSH reconnect authority * fix: preserve split SSH retry ownership * fix: preserve SSH split continuation authority * docs: record final SSH reconnect validation * fix: preserve SSH authority through retained and detached state * fix: retain SSH authority across delayed split mounts * fix: close SSH authority recovery gaps * fix: fence stale SSH transport replacement * fix: serialize SSH target teardown * fix: settle SSH teardown failures before reconnect * fix: retire failed SSH reset sessions * test: reconcile current main E2E contracts * fix: close direct SSH reconnect review gaps * fix: fence stale SSH reconnect side effects * fix: close final SSH reconnect lifecycle gaps * test: stabilize current-main reliability gates * test: prove plugin navigation containment * test: make plugin navigation oracle authoritative * test: make plugin navigation oracle deterministic * ci: allow sharded e2e suite to finish * test: wait for runtime pane publication * test: classify pane readiness by error code * test: select close persistence terminal by tab identity * docs: mark reconnect implementation validated --------- Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com> |
||
|
1fa9ffb5ea |
ci(pr): run E2E when a PR touches tests/e2e paths (advisory) (#11131)
* ci(pr): run E2E when a PR touches tests/e2e paths Regression specs under tests/e2e never ran on PR CI — only schedule and release called e2e.yml — so a red regression test could merge green. Path-filter and workflow_call the E2E suite when E2E-relevant files change. Use merge-base diffs so base-branch drift does not false-trigger E2E, fail the detector when git diff cannot compute the PR range, and pin least-privilege contents:read on both the detector and reusable E2E workflow. Closes #10518 Co-authored-by: Wooseong Kim <innocarpe@gmail.com> Co-authored-by: Orca <help@stably.ai> * ci(pr): make the E2E path gate actually block, and match the real config path Two fixes to the new path-filtered E2E job. The gate did not gate. pr.yml's `verify` job is the required check, and it enumerates its dependencies explicitly — `e2e` was in neither `needs` nor the result list, so a failing shard left `verify` green. That reproduces the exact hole this job exists to close: a red spec merges green, just with a red box further down the page. Add `e2e` to both. Because the job is path-filtered, `skipped` is the normal result on a PR that touches no E2E files and has to keep passing. That allowance is checked after the strict loop rather than inside it, so it can never leak to the six jobs that are always required. The `playwright.` pattern matched nothing. The config is tests/playwright.config.ts — beside tests/e2e/, not inside it — so no tracked file starts with `playwright.` and editing the runner config would silently skip E2E. Anchor it at `tests/playwright.`. Adds a contract test alongside the existing release-e2e one. Verified it fails when either fix is reverted, and simulated the gate across success/skipped/failure/cancelled plus the skip-must-not-mask-a-real-failure case. * test(ci): close two gaps in the E2E gate contract CodeRabbit was right on both counts — verified by reverting each and watching the contract stay green. The path filter was unasserted, so `e2e` could lose its `if:` and run on every PR — the cost the filter exists to avoid — without failing anything. The strict-loop check hardcoded four of the six required jobs, so dropping GIT_COMPATIBILITY or SHELL_CONTRACTS left them unenforced while the contract passed. Derive the list from verify.needs instead, so a newly added required job that misses the loop fails here rather than silently going unchecked. * ci(pr): land the E2E path gate advisory instead of blocking The E2E suite is currently failing every scheduled run on main — 22 of the last 22 — so making verify depend on it would block any PR touching tests/e2e/**, including the PRs that fix the suite. This PR's own run reproduced that: 3 of 12 shards failed on specs unrelated to it (agent-session resume, Jira linking, plugin containment, terminal artifacts). So the job runs and reports on E2E-path PRs but is left out of verify.needs for now. The detector, the tests/playwright. path fix, and the contract tests are unaffected — those stand on their own and were the substance of the review. Flipping to blocking is a three-line change once the suite is green; the exact wiring, including why the skipped allowance must sit outside the strict loop, is recorded on verify's Require-successful-checks step. The contract test pins the advisory choice so it reads as deliberate rather than as the unwired-gate bug it originally caught, and still fails if the path filter, the strict-loop coverage, or the config path regress. --------- Co-authored-by: Wooseong Kim <innocarpe@gmail.com> Co-authored-by: Orca <help@stably.ai> |
||
|
Neil
|
c6076a507c |
ci(release): detach non-blocking E2E (#11031)
* ci(release): detach non-blocking E2E * test(release): pin E2E dispatch retries |
||
|
Jinjing
|
b69c6043e3 |
Ssh watcher isolation e2e (#8494)
* Add a Docker SSH watcher-isolation E2E test to verify remote relay-watch - Covers two scenarios: crashed watcher children are respawned under the same relay without dropping the terminal PTY or file-explorer view, and a missing deployed relay-watcher.js artifact is repaired on reconnect - Extracts shared connect/disconnect/reconnect logic out of the perf spec into docker-ssh-relay-connection.ts, and adds docker-ssh-relay-processes.ts for inspecting/signaling remote relay and watcher PIDs - Wires the new spec into a dedicated CI job and pnpm script * Fix Windows and Linux-only issues in Docker SSH watcher-isolation E2E ha - node-gyp override only applies on Linux runners now, since the CI job moved to ubuntu-latest but shares the workflow with non-Linux jobs - spawn the e2e runner scripts through a shell on win32 to satisfy Node's CVE-2024-27980 restriction on unshelled .cmd spawns - harden relay process row parsing against empty pid/ppid fields so a vanished /proc entry fails loudly instead of coercing to pid 0 - dedupe the reconnect helpers and export shellQuote for reuse across the docker-ssh-relay test helpers |
||
 Brennan BensonandNeil
|
d9d870d93f |
Stabilize sharded e2e CI concurrency (#6312)
Co-authored-by: Neil <neil@stably.ai> |
||
|
Neil
|
e53300a4be | ci: upgrade GitHub Actions to Node 24 (#2978) | ||
|
Jinjing
|
85ad942954 | ci: schedule e2e tests (#2098) | ||
|
Neil
|
6eae8fa540 | ci: shard electron e2e workflow (#2070) | ||
|
NeilandOrca
|
4318f3bfa7 |
chore: reduce root-directory clutter (#1275)
Co-authored-by: Orca <help@stably.ai> |
||
|
Brennan BensonandOrca
|
3a138a2f07 |
ci: stabilize pnpm native addon install in PR checks (#733)
Co-authored-by: Orca <help@stably.ai> |
||
|
NeilandOrca
|
eeffbf0a73 |
ci: cache pnpm store on release + centralize node version in .nvmrc (#1106)
Co-authored-by: Orca <help@stably.ai> |
||
|
Brennan Benson
|
660b5f4149 | ci: run E2E tests on every PR, surface on release (non-blocking) (#842) |