mirror of
https://github.com/stablyai/orca.git
synced 2026-10-22 00:03:47 +00:00
docs(e2e): classify failed-run findings
This commit is contained in:
@@ -2,32 +2,40 @@
|
||||
|
||||
Run: [E2E attempt 1](https://github.com/stablyai/orca/actions/runs/31749134781/attempts/1), scheduled on `main` at `e06cdbb4ee0442599eaa7ecef759b8a600bb2cf5` (2026-08-13). Three of ten shards failed; each had one failing test. The other seven shards and Docker SSH isolation passed.
|
||||
|
||||
## Classification
|
||||
|
||||
| Failure | Classification | Product impact | Resolution in this PR |
|
||||
| --- | --- | --- | --- |
|
||||
| Terminal link tooltip | **Test issue** | No product regression confirmed. The test used an untrusted synthetic pointer event. | Use real Playwright pointer movement and assert the visible tooltip. |
|
||||
| Paired terminal link to Orca Browser | **Historical product issue** | The failing SHA had a paired browser-tab registration race. The upstream product fix is already present on this branch and the focused test now passes. | Validate the existing upstream fix; no duplicate product change. |
|
||||
| Terminal-output scheduler | **Test issue** | No product regression. A valid high-priority drain permits up to 8 writes, while the test always expected 2. | Record drain priority and assert the corresponding cap. |
|
||||
|
||||
## `e2e 3-of-10`: terminal link tooltip
|
||||
|
||||
- **Test:** `tests/e2e/issue-12656-terminal-link-tooltip.spec.ts:121`, `clears hover state without permanently shrinking the terminal`.
|
||||
- **Evidence:** after 10 seconds repeatedly moving to the printed URL, the expected `{ display: '', currentLinkText: '<url>' }` remained `{ display: 'none', currentLinkText: null }` at line 154. The terminal geometry assertion immediately before the hover passed.
|
||||
- **Likely cause:** the xterm linkifier did not register the synthetic `mousemove` target. This looks like a hover hit-testing/settling race, not the prior permanent-gap regression: the terminal was already its full height. The test calculates coordinates from `screen.getBoundingClientRect()` and dispatches an untrusted `MouseEvent`, which is fragile across renderer fit/reflow timing.
|
||||
- **Affected area:** `tests/e2e/issue-12656-terminal-link-tooltip.spec.ts`; investigate `src/renderer/src/components/terminal-pane/terminal-url-link-hit-testing.ts` and terminal tooltip geometry CSS.
|
||||
- **Ownership clue:** introduced/last materially changed by Jinwoo Hong (`Jinwoo-H`) in [#13075](https://github.com/stablyai/orca/pull/13075), commit `8c3e9535c7`.
|
||||
- **Suggested fix:** reproduce with a real Playwright pointer move and wait for post-fit dimensions before testing; if it still fails, correct xterm coordinate/hit-testing after reflow. Keep an assertion that hover activates and that blur restores full terminal geometry.
|
||||
- **Classification:** **test issue**, not a confirmed product bug.
|
||||
- **Cause:** the test calculated a coordinate and dispatched an untrusted `MouseEvent`; xterm did not reliably treat it as a real hover after fit/reflow. The terminal geometry assertion immediately before the hover passed.
|
||||
- **Resolution:** use real Playwright pointer movement at xterm CSS-cell coordinates, assert the user-visible tooltip text, and keep the blur/geometry assertion.
|
||||
|
||||
## `e2e 4-of-10`: paired terminal link to Orca Browser
|
||||
|
||||
- **Test:** `tests/e2e/paired-remote-terminal-browser-link.spec.ts:126`, `opens a paired-runtime terminal link on its owning host`.
|
||||
- **Evidence:** after choosing **Orca Browser**, the test waited 60 seconds for a single host-owned browser tab. It found no matching client workspace/page/remote handle/host tab identity (`Received: null`) at line 200.
|
||||
- **Likely cause:** a paired-runtime browser-tab registration race. The terminal link action is routed to the host, but the paired client has not yet materialized the returned page identity/handle. The failure is not an external-server failure: the app completed the action but never reached the expected paired browser-tab state.
|
||||
- **Classification:** **historical product issue** on the failing SHA.
|
||||
- **Cause:** a paired-runtime browser-tab registration race. The terminal link action reached the host, but the paired client did not materialize the returned page identity/handle.
|
||||
- **Affected area:** `src/renderer/src/lib/workspace-browser-tab-open.ts`, `src/renderer/src/runtime/web-runtime-session.ts`, and `tests/e2e/paired-remote-terminal-browser-link.spec.ts`.
|
||||
- **Ownership clue:** regression path was introduced by Jinwoo Hong (`Jinwoo-H`) in [#14117](https://github.com/stablyai/orca/pull/14117), commit `e8350f1990`.
|
||||
- **Known follow-up:** after this run, [#14363](https://github.com/stablyai/orca/pull/14363) (`232e398c35`, refined by `2f85dda183`, author `OrcaWin`/`m4air`) added a registration-race guard in this exact area. Validate this shard on a commit containing that PR before making a second fix.
|
||||
- **Resolution:** [#14363](https://github.com/stablyai/orca/pull/14363) (`232e398c35`, refined by `2f85dda183`) already adds a registration-race guard and is present on this branch. The focused E2E now passes, so this PR adds no duplicate product code.
|
||||
|
||||
## `e2e 8-of-10`: background terminal-output scheduler
|
||||
|
||||
- **Test:** `tests/e2e/terminal-output-scheduler.spec.ts:179`, `background tab output bursts use the shared drain while the active tab renders`.
|
||||
- **Evidence:** line 282 expected the largest recorded drain to be at most 2 writes, but received 4. The same shard passed the foreground-marker and background-retention checks, then reported `53 passed`, `2 skipped`, `1 failed`.
|
||||
- **Likely cause:** the test asserts the normal `MAX_WRITES_PER_DRAIN = 2` for every drain, while the scheduler intentionally uses `HIGH_PRIORITY_MAX_WRITES_PER_DRAIN = 8` whenever a high-priority/large backlog is present. A 4-write drain is therefore consistent with the implementation's high-priority path; determine whether the burst is incorrectly promoted or the test needs to observe and exempt high-priority drains.
|
||||
- **Classification:** **test issue**, not a product bug.
|
||||
- **Cause:** the test asserted the normal `MAX_WRITES_PER_DRAIN = 2` for every drain, while the scheduler intentionally uses `HIGH_PRIORITY_MAX_WRITES_PER_DRAIN = 8` for high-priority backlog. A 4-write drain is valid.
|
||||
- **Affected area:** `src/renderer/src/lib/pane-manager/pane-terminal-output-scheduler.ts:1039-1041`, its unit tests, and `tests/e2e/terminal-output-scheduler.spec.ts`.
|
||||
- **Ownership clue:** scheduler scenario originated with Brennan Benson (`brennanb2025`) in [#1699](https://github.com/stablyai/orca/pull/1699), commit `daaea6acaa`; the high-priority branch was added by Neil in `e896dd3eb7`.
|
||||
- **Suggested fix:** expose whether each debug drain used high priority. Assert `<= 2` only for normal drains; separately assert the high-priority cap and preserve the existing foreground-progress check. If this burst should never be high priority, trace `entry.highPriority`/queued character transitions and fix the promotion instead.
|
||||
- **Resolution:** expose whether each debug drain used high priority. Assert `<= 2` for normal drains and `<= 8` for high-priority drains, while preserving the foreground-progress check.
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
Reference in New Issue
Block a user