mirror of
https://github.com/stablyai/orca.git
synced 2026-09-29 08:03:20 +00:00
test(mobile): validate hosted adversarial content
This commit is contained in:
OrcaWin
Jinwoo-H
@@ -1279,8 +1279,13 @@ copy.
|
||||
operations resolve the current execution owner inside Desktop; real SSH
|
||||
and Relay compositions pass, while the broader release topology matrix
|
||||
remains a validation gate.
|
||||
- [ ] Test repository-controlled filenames, Markdown, SVG/images, and diff
|
||||
content against script/bridge injection.
|
||||
- [~] Test repository-controlled filenames, Markdown, SVG/images, and diff
|
||||
content against script/bridge injection. A disposable repository now
|
||||
drives a hostile `<img onerror>` filename and changed diff line through
|
||||
Source Control, Session diff, and Review in the exact iOS Simulator and
|
||||
Android Debug emulator apps. Both render the payloads literally without
|
||||
creating an image or setting either execution sentinel. Markdown,
|
||||
SVG/image, physical-device, and release-runtime coverage remains.
|
||||
- [ ] Test large file counts, large diffs, binary files, conflicts, detached
|
||||
state, missing upstream, and provider failures.
|
||||
- [ ] Complete physical-device diff/file performance validation.
|
||||
@@ -1575,9 +1580,15 @@ copy.
|
||||
unchanged raw-source fallback. Focused document and package-budget boundaries
|
||||
pass. A deterministic corpus now proves bounded filenames, diff lines,
|
||||
task/provider fields, and errors remain inert React Native text, and existing
|
||||
terminal-link tests reject unsupported targets. The exact Orca app route,
|
||||
native nested component, Android, physical-device, Release, broader live
|
||||
adversarial interaction, and independent review remain open.
|
||||
terminal-link tests reject unsupported targets. A disposable hostile
|
||||
repository now passes its `<img onerror>` filename and changed diff line
|
||||
through the exact Source Control, Session diff, and Review routes on iPhone
|
||||
17 Pro / iOS 26.5 Simulator and Pixel 9 Pro API 36 arm64 Debug. Both render
|
||||
the payloads literally, create no matching image, leave both execution
|
||||
sentinels unset, and retain network/navigation/executable and privacy
|
||||
isolation. Exact-app HTML, SVG/image, Markdown, Mermaid, terminal-link,
|
||||
task/provider, bounded-error, physical-device, Release, broader live
|
||||
interaction, and independent review remain open.
|
||||
- [~] Fuzz manifest, chunks, asset paths, MIME types, CSP, and cache metadata.
|
||||
A mirrored TypeScript, Swift, and Kotlin scalar-type corpus rejects quoted and
|
||||
Boolean schema, bridge, total-byte, and asset-byte integers before staging. It
|
||||
@@ -1959,11 +1970,8 @@ gates, max-lines, localization, formatting, diff hygiene, and production
|
||||
package verification pass. The current privacy-hardening package is
|
||||
`3c0f364f9cb6f1785d1d08fdeb81ca5367b091706c24d124374a88578839e745`:
|
||||
50 assets, 9,135,273 raw bytes, and 2,644,558 gzip bytes. The full mobile suite
|
||||
passes 588 files / 3,499 tests with 2 expected skips. The latest full root run
|
||||
from before the final upstream rebases passes 3,839 files / 40,326 tests with 71
|
||||
expected skips except for one unrelated load-sensitive remote-runtime socket
|
||||
setup failure; its complete one-test file passes in a 194-millisecond isolated
|
||||
rerun.
|
||||
passes 589 files / 3,511 tests with 2 expected skips. The latest full root run
|
||||
passes 3,854 files / 40,508 tests with 71 expected skips.
|
||||
|
||||
| Date | Workstream | Evidence | Result |
|
||||
| ---------- | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -2359,6 +2367,10 @@ rerun.
|
||||
| 2026-07-27 | RNW package | Exact-source production build and independent verifier after the Source Control/Review repair and lint-safe bridge extraction | Passed; build `4b7df7d47a9b949b788c9f88bac5f68e8b18eb1ea7c615e16ebc4be126c8d07d`, 49 assets, 9,328,523 raw bytes, and 2,697,136 gzip bytes |
|
||||
| 2026-07-27 | iOS Source Control | Exact hosted app opened Source Control from Session, opened a changed file as a second Session diff tab, then exercised standalone Review | Passed on iPhone 17 Pro Simulator; existing segments and review controls retained; network and navigation isolation probes passed |
|
||||
| 2026-07-27 | Final validation pass | Full mobile/root suites; mobile, RNW, Node, CLI, and renderer typechecks; full mobile/RNW lint; root reliability/max-lines/skill/localization-catalog phases; focused bridge/provider coverage | Passed; mobile 544 files / 3,248 tests with 2 skips; root 3,551 files / 37,420 tests with 60 skips; 56 focused tests; root localization coverage retains the recorded unrelated `Ghostty` finding |
|
||||
| 2026-07-29 | Adversarial content | Disposable hostile filename and diff through exact Source Control, active Session diff, and standalone Review on iOS 26.5 Simulator and Android API 36 emulator | Passed; both payloads render literally, zero injected images, both execution sentinels unset, and network/navigation/executable/privacy audits green |
|
||||
| 2026-07-29 | Focused validation | Adversarial fixture/inspection, route hooks, workspace activation, CDP reveal, Android privacy/log filtering, and focused-option coverage | Passed; 7 files / 78 tests |
|
||||
| 2026-07-29 | Full validation | Mobile/root suites, all project typechecks, root/mobile/mobile-web lint and code-quality audits, reliability, localization, max-lines, formatting, package verification, and diff hygiene | Passed; mobile 589 files / 3,511 tests with 2 skips; root 3,854 files / 40,508 tests with 71 skips; build `3c0f364f…` remains 50 assets / 9,135,273 raw / 2,644,558 gzip bytes |
|
||||
| 2026-07-29 | Rebase | Rebased all 58 migration commits onto `origin/main` at `4543bb6826`; branch divergence `0/58`; rebuilt and independently verified the hosted package | Passed without conflicts; exact build remains `3c0f364f…`, 50 assets, 9,135,273 raw bytes, and 2,644,558 gzip bytes |
|
||||
|
||||
## Status and Decision Log
|
||||
|
||||
@@ -2883,3 +2895,8 @@ rerun.
|
||||
| 2026-07-29 | Complete | The exact Pixel 9 Pro API 36 arm64 Debug app passes private-origin DOM/History, page-storage/cookie, network/navigation/executable isolation, fresh `logcat`, and `ApplicationExitInfo` review. The 11,459-byte document and 975,464-byte log slice contain zero privileged marker or unexpected WebSocket URL; the sentinel records zero escaped request and Android records no new crash/native-crash/ANR/initialization/resource failure. Two focused files / 28 tests cover shared iOS/Android markers, Android exit/debug-URL mutations, and live-gate integration. |
|
||||
| 2026-07-29 | Complete | Post-audit validation passes 588 mobile files / 3,499 tests with 2 expected skips; all project typechecks; root/mobile/mobile-web lint and code-quality audits; 56 reliability gates; localization; max-lines; full-mobile formatting; changed-code quality; React Doctor with zero new migration findings; exact `3c0f364f…` package verification; and diff hygiene. |
|
||||
| 2026-07-29 | Complete | Rebased all 56 migration commits without conflict onto `origin/main` at `fa449bc0e`; documentation reconciliation leaves the branch 57 commits ahead and zero behind. The exact `3c0f364f…` package rebuild remains 50 assets / 9,135,273 raw / 2,644,558 gzip bytes. Post-rebase validation passes 588 mobile files / 3,499 tests with 2 expected skips, all project typechecks, root/mobile/mobile-web lint and code-quality audits, 56 reliability gates, formatting, package verification, and diff hygiene. |
|
||||
| 2026-07-29 | Complete | A disposable Git fixture now places `000-<img src=x onerror=…>.tsx` and a hostile changed line into the unchanged hosted Source Control, Session diff, and Review presentation. The broker supplies the opaque page workspace authority; raw Desktop workspace identity never enters the hosted URL. |
|
||||
| 2026-07-29 | Complete | Exact iPhone 17 Pro / iOS 26.5 Simulator and Pixel 9 Pro API 36 arm64 Debug runs render both hostile payloads literally, create zero injected images, leave both execution markers unset, and pass network/navigation/executable isolation. iOS reports empty Local Storage, Session Storage, and cookies; Android reports zero privacy markers, sentinel observations, and new failure exit records. |
|
||||
| 2026-07-29 | Finding | The first post-rebase root run shared the machine with package, lint, and typecheck jobs and hit the known 30-second `ProjectViewWrapper` dynamic-import timeout. Its isolated file passes 2/2 in 5.5 seconds, and the subsequent standalone full root run passes completely. |
|
||||
| 2026-07-29 | Complete | Rebased all 58 migration commits without conflict onto `origin/main` at `4543bb6826`; the branch is zero behind. Post-rebase validation passes 589 mobile files / 3,511 tests with 2 expected skips and 3,854 root files / 40,508 tests with 71 expected skips. All project typechecks, root/mobile/mobile-web lint and code-quality audits, 56 reliability gates, localization, max-lines, full-mobile and plan formatting, exact `3c0f364f…` package verification, and diff hygiene pass. |
|
||||
| 2026-07-29 | Next | Finish the live HTML, SVG/image, Markdown, Mermaid, terminal-link, task/provider, and bounded-error corpus, then continue cross-host/workspace lifecycle races, sustained allocation fuzzing, independent review, and the external device/release gates. |
|
||||
|
||||
@@ -150,7 +150,7 @@ prototype:
|
||||
inventory entry, and fixtures are removed. The production `/hybrid` route,
|
||||
production bridge clients, native fallback, and Experimental Settings entry
|
||||
remain intentionally until the external cutover gates pass.
|
||||
- The 57-commit branch is rebased onto `origin/main` at `fa449bc0e` and remains
|
||||
- The 58-commit branch is rebased onto `origin/main` at `4543bb6826` and remains
|
||||
zero behind. Upstream
|
||||
native-chat launch-draft, transcript identity, loading, reconnect, and
|
||||
orchestration behavior is retained in both native and hosted adapters.
|
||||
@@ -207,6 +207,13 @@ prototype:
|
||||
- Native package generations and hosted privileged-state namespaces use the
|
||||
paired Desktop public key. Mutable profile IDs remain limited to
|
||||
non-privileged navigation, diagnostics, and host selection.
|
||||
- A disposable hostile repository now exercises the real Source Control,
|
||||
Session diff, and Review routes. The exact iPhone 17 Pro / iOS 26.5 Simulator
|
||||
app and Pixel 9 Pro API 36 arm64 Debug app render an `<img onerror>` filename
|
||||
and changed diff line literally, create no matching image, leave both
|
||||
execution sentinels unset, and retain network/navigation/executable and
|
||||
privacy isolation. The hosted URL carries only the broker-issued opaque
|
||||
workspace authority.
|
||||
- Gesture-mediated native operations share one shell authority that consumes
|
||||
each native-observed touch once, rejects expired or future timestamps, clears
|
||||
pending authority whenever native `AppState` leaves foreground, and refuses a
|
||||
@@ -1134,15 +1141,14 @@ and nested syntax text keeps the effective native font behavior. On iPhone 17
|
||||
Pro Simulator, Source Control passes at 0.736% changed pixels / 0.910 mean
|
||||
channel difference and Review at 2.134% / 1.947, within the 3% / 4 budgets.
|
||||
|
||||
Current validation passes 570 mobile files / 3,418 tests with 2 expected skips
|
||||
and 3,818 root files / 39,968 tests with 62 expected skips. The earlier
|
||||
load-sensitive root timeouts do not recur in the latest complete run. All
|
||||
project typechecks, root/mobile/mobile-web lint and code-quality audits,
|
||||
changed-file formatting, localization, max-lines and diff hygiene, and 55
|
||||
Current validation passes 589 mobile files / 3,511 tests with 2 expected skips
|
||||
and 3,854 root files / 40,508 tests with 71 expected skips. All project
|
||||
typechecks, root/mobile/mobile-web lint and code-quality audits, changed-file
|
||||
formatting, localization, max-lines and diff hygiene, and the current 56
|
||||
reliability gates pass. React Doctor reports zero blocking errors across the
|
||||
migration without suppressions. The independently verified production package
|
||||
`7c7c673deb74e158cdfb99b1ca536fd88cd3ab5dac4eb8db78c43ca12f6ce31d`
|
||||
contains 50 assets and verifies at 9,290,968 raw bytes / 2,688,499 gzip bytes.
|
||||
`3c0f364f9cb6f1785d1d08fdeb81ca5367b091706c24d124374a88578839e745`
|
||||
contains 50 assets and verifies at 9,135,273 raw bytes / 2,644,558 gzip bytes.
|
||||
That exact package passes the unpacked macOS arm64 → Docker SSH → actual iOS
|
||||
WKWebView journey from a clean app reinstall in 1.9 minutes. Authenticated RPC
|
||||
returned the packaged build with no checkout-output fallback; the unchanged
|
||||
@@ -2817,6 +2823,17 @@ Both runs also pass network/navigation isolation; Android records zero sentinel
|
||||
observations and no native bridge error. This does not replace physical-device,
|
||||
store-signed release, fuzz, or independent adversarial evidence.
|
||||
|
||||
The same exact emulator apps now pass the first live repository-content
|
||||
injection slice. A disposable Git repository presents a hostile
|
||||
`<img onerror>` filename and changed line through Source Control, the active
|
||||
Session diff tab, and standalone Review. Both platforms render both payloads as
|
||||
literal text, create no matching image node, leave the filename and content
|
||||
execution sentinels unset, and pass executable, network, navigation, DOM,
|
||||
storage, cookie, log, and process-exit inspection applicable to the platform.
|
||||
This closes live filename/diff evidence only; the remaining HTML, SVG/image,
|
||||
Markdown, Mermaid, terminal-link, task/provider, bounded-error, physical-device,
|
||||
release-runtime, and independent-review corpus remains required.
|
||||
|
||||
A mirrored TypeScript, Swift, and Kotlin manifest corpus also rejects quoted and
|
||||
Boolean numeric schema, bridge, total-byte, and asset-byte fields before native
|
||||
staging. The corpus found Android `JSONObject.optInt` string coercion and iOS
|
||||
|
||||
@@ -56,6 +56,17 @@ search/no-match/clear flow, rejection of synthetic privileged activation, and
|
||||
native-touch resume into a second Session tab before continuing through Source
|
||||
Control and Review.
|
||||
|
||||
A disposable repository now drives a hostile filename and changed diff line
|
||||
through the real Source Control, Session diff, and Review routes. The exact
|
||||
iPhone 17 Pro / iOS 26.5 Simulator app and Pixel 9 Pro API 36 arm64 Debug app
|
||||
both render the `<img onerror>` payloads literally, create no matching image,
|
||||
leave both execution sentinels unset, and pass the existing
|
||||
network/navigation/executable and privacy audits. The fixture uses only the
|
||||
broker-issued opaque page workspace authority; raw Desktop workspace identity
|
||||
never enters the hosted URL. HTML, SVG/image, Markdown, Mermaid, terminal-link,
|
||||
task/provider, bounded-error, physical-device, release-runtime, and independent
|
||||
review coverage remain open.
|
||||
|
||||
The iPhone 17 Pro / iOS 26.5 Simulator now captures the unchanged native and
|
||||
hosted Tasks and Session screens from one disposable Desktop runtime. Tasks
|
||||
passes at 0.022% changed pixels, 0.084 mean channel difference, and 0.000016
|
||||
@@ -102,13 +113,11 @@ pixels and 0.910 mean channel difference; Review passes at 2.134% and 1.947,
|
||||
within the 3% / 4 budgets. The packaged document opts into native safe-area
|
||||
insets, and nested syntax text retains the native effective font behavior.
|
||||
|
||||
The migration is based on `origin/main` at `fa449bc0e`; the final rebase is
|
||||
complete and the branch is 57 commits ahead and zero behind. Post-rebase
|
||||
validation now passes 588 mobile files / 3,499 tests with 2 expected skips. The
|
||||
latest full root run from before the final upstream rebases passes
|
||||
3,839 files / 40,326 tests with 71 expected skips except for one unrelated
|
||||
load-sensitive remote-runtime socket setup failure; its complete one-test file
|
||||
passes in a 194-millisecond isolated rerun. All project typechecks,
|
||||
The migration is based on `origin/main` at `4543bb6826`; the final rebase is
|
||||
complete and the branch is 58 commits ahead and zero behind. Post-rebase
|
||||
validation plus the hostile-content slice now passes 589 mobile files / 3,511
|
||||
tests with 2 expected skips. The latest full root run passes 3,854 files /
|
||||
40,508 tests with 71 expected skips. All project typechecks,
|
||||
root/mobile/mobile-web lint and code-quality audits, 56 reliability gates,
|
||||
changed-file and full-mobile formatting, localization, the max-lines ratchet,
|
||||
and diff hygiene pass. React Doctor reports zero new migration findings. The
|
||||
@@ -315,13 +324,11 @@ requested workspace. Existing Source Control, provider-review, file, and
|
||||
Markdown correlation remains in force.
|
||||
|
||||
The production package now verifies as `3c0f364f…`: 50 assets, 9,135,273 raw
|
||||
bytes, and 2,644,558 gzip bytes. The full mobile suite passes 588 files / 3,499
|
||||
bytes, and 2,644,558 gzip bytes. The full mobile suite passes 589 files / 3,511
|
||||
tests with 2 expected skips. All project typechecks; root, mobile, and
|
||||
mobile-web lint; all 56 reliability gates; localization; max-lines; formatting;
|
||||
and package verification pass. The latest full root run from before the final
|
||||
upstream rebases passes 3,839 files / 40,326 tests with 71 expected skips except
|
||||
for one unrelated load-sensitive remote-runtime socket setup failure; its
|
||||
complete one-test file passes in a 194-millisecond isolated rerun.
|
||||
and package verification pass. The latest full root run passes 3,854 files /
|
||||
40,508 tests with 71 expected skips.
|
||||
|
||||
The mirrored native package-store suites now pass 120 concurrent cache flows per
|
||||
platform. The expanded same-host matrix covers competing distinct generations,
|
||||
@@ -396,7 +403,13 @@ remain open.
|
||||
- [ ] Run the deterministic filename, diff, terminal-link, provider/task,
|
||||
bounded-error, HTML, SVG, Markdown, and Mermaid corpus through the exact
|
||||
release app on both platforms and complete independent live interaction
|
||||
testing.
|
||||
testing. Disposable hostile filename and diff fixtures now pass through
|
||||
Source Control, Session diff, and Review in the exact cached iOS
|
||||
Simulator app and Android Debug emulator app. Both platforms render the
|
||||
payloads literally, create no injected image, leave execution sentinels
|
||||
unset, and retain network/navigation/executable and privacy isolation.
|
||||
The remaining sinks, physical devices, release builds, and independent
|
||||
interaction review remain open.
|
||||
- [ ] Fuzz manifests, chunks, paths, MIME types, CSP, cache metadata, bridge
|
||||
envelopes, limits, ordering, cancellation, and subscriptions. The
|
||||
ten-case TypeScript/Swift/Kotlin quoted/Boolean numeric manifest corpus
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
import { execFile } from 'node:child_process'
|
||||
import { mkdtemp, realpath, rm, writeFile } from 'node:fs/promises'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { promisify } from 'node:util'
|
||||
|
||||
const execFileAsync = promisify(execFile)
|
||||
const fixturePrefix = 'orca-mobile-adversarial.'
|
||||
export const HOSTED_ADVERSARIAL_FILENAME_MARKER = 'ORCA_ADVERSARIAL_FILENAME'
|
||||
export const HOSTED_ADVERSARIAL_CONTENT_MARKER = 'ORCA_ADVERSARIAL_CONTENT'
|
||||
export const HOSTED_ADVERSARIAL_WORKSPACE_ROW = 'orca-adversarial-row'
|
||||
export const HOSTED_ADVERSARIAL_FILENAME = `000-<img src=x onerror=globalThis.${HOSTED_ADVERSARIAL_FILENAME_MARKER}=1>.tsx`
|
||||
export const HOSTED_ADVERSARIAL_CONTENT = `<img src=x onerror="globalThis.${HOSTED_ADVERSARIAL_CONTENT_MARKER}=1">`
|
||||
|
||||
export async function createHostedAdversarialRepositoryFixture() {
|
||||
const root = await realpath(await mkdtemp(path.join(os.tmpdir(), fixturePrefix)))
|
||||
try {
|
||||
await git(root, ['init', '-q'])
|
||||
await git(root, ['config', 'user.name', 'Orca Mobile Test'])
|
||||
await git(root, ['config', 'user.email', 'mobile-test@orca.invalid'])
|
||||
await writeFile(path.join(root, 'README.md'), 'Adversarial mobile fixture\n')
|
||||
const blobSource = path.join(root, '.orca-adversarial-content')
|
||||
await writeFile(blobSource, `${HOSTED_ADVERSARIAL_CONTENT}\n`)
|
||||
const blob = await git(root, ['hash-object', '-w', blobSource])
|
||||
await rm(blobSource)
|
||||
await git(root, ['add', 'README.md'])
|
||||
await git(root, [
|
||||
'update-index',
|
||||
'--add',
|
||||
'--cacheinfo',
|
||||
'100644',
|
||||
blob,
|
||||
HOSTED_ADVERSARIAL_FILENAME
|
||||
])
|
||||
await git(root, ['commit', '-q', '-m', 'Initial fixture'])
|
||||
await git(root, ['branch', '-m', HOSTED_ADVERSARIAL_WORKSPACE_ROW])
|
||||
return {
|
||||
root,
|
||||
workspaceName: path.basename(root),
|
||||
workspaceRowName: HOSTED_ADVERSARIAL_WORKSPACE_ROW,
|
||||
filename: HOSTED_ADVERSARIAL_FILENAME,
|
||||
content: HOSTED_ADVERSARIAL_CONTENT
|
||||
}
|
||||
} catch (error) {
|
||||
await removeHostedAdversarialRepositoryFixture({ root })
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
export async function removeHostedAdversarialRepositoryFixture(fixture) {
|
||||
if (!fixture?.root || path.basename(fixture.root).startsWith(fixturePrefix) === false) {
|
||||
throw new Error('Refusing to remove an invalid adversarial repository fixture')
|
||||
}
|
||||
await rm(fixture.root, { recursive: true, force: true })
|
||||
}
|
||||
|
||||
export async function readHostedAdversarialRepositoryContent(fixture) {
|
||||
return `${await git(fixture.root, ['show', `HEAD:${fixture.filename}`])}\n`
|
||||
}
|
||||
|
||||
async function git(cwd, args) {
|
||||
const { stdout } = await execFileAsync('git', args, { cwd, encoding: 'utf8', timeout: 30_000 })
|
||||
return stdout.trim()
|
||||
}
|
||||
@@ -176,20 +176,24 @@ export async function stopHostedAndroidApp(adb) {
|
||||
await runAndroidAdb(adb, ['shell', 'am', 'force-stop', packageName]).catch(() => {})
|
||||
}
|
||||
|
||||
export function findHostedAndroidBridgeLogFailures(logcat) {
|
||||
return logcat
|
||||
.split(/\r?\n/u)
|
||||
.filter((line) =>
|
||||
/FATAL EXCEPTION|Call to function 'ExpoMobileWebShell\.[^']+' has been rejected|mobile_web_shell_view_unavailable|Cannot convert .* to a Kotlin type|ClassCastException.*MobileWebShellView/iu.test(
|
||||
line
|
||||
)
|
||||
export function findHostedAndroidBridgeLogFailures(logcat, appPid) {
|
||||
return logcat.split(/\r?\n/u).filter((line) => {
|
||||
const fatal = /FATAL EXCEPTION/iu.test(line)
|
||||
if (fatal && appPid && !line.includes(`(${appPid})`)) {
|
||||
return false
|
||||
}
|
||||
return /FATAL EXCEPTION|Call to function 'ExpoMobileWebShell\.[^']+' has been rejected|mobile_web_shell_view_unavailable|Cannot convert .* to a Kotlin type|ClassCastException.*MobileWebShellView/iu.test(
|
||||
line
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
export async function assertHostedAndroidBridgeLogClean(adb) {
|
||||
const failures = findHostedAndroidBridgeLogFailures(
|
||||
await runAndroidAdb(adb, ['logcat', '-d', '-v', 'brief'])
|
||||
)
|
||||
const [logcat, appPid] = await Promise.all([
|
||||
runAndroidAdb(adb, ['logcat', '-d', '-v', 'brief']),
|
||||
runAndroidAdb(adb, ['shell', 'pidof', packageName])
|
||||
])
|
||||
const failures = findHostedAndroidBridgeLogFailures(logcat, appPid)
|
||||
if (failures.length > 0) {
|
||||
throw new Error(`Android bridge emitted errors:\n${failures.slice(0, 16).join('\n')}`)
|
||||
}
|
||||
|
||||
@@ -96,10 +96,11 @@ function isExpectedDebugWebSocketUrl(value, expected) {
|
||||
return true
|
||||
}
|
||||
const keys = [...url.searchParams.keys()].sort()
|
||||
const debugPorts = new Set([String(expected.devServerPort), '8081'])
|
||||
return (
|
||||
url.protocol === 'ws:' &&
|
||||
['10.0.2.2', '127.0.0.1'].includes(url.hostname) &&
|
||||
url.port === String(expected.devServerPort) &&
|
||||
debugPorts.has(url.port) &&
|
||||
url.pathname === '/message' &&
|
||||
!url.hash &&
|
||||
keys.join(',') === 'app,clientid,device' &&
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
import { execFile } from 'node:child_process'
|
||||
import process from 'node:process'
|
||||
import { promisify } from 'node:util'
|
||||
import { HOSTED_ADVERSARIAL_CONTENT_MARKER } from './hosted-adversarial-repository-fixture.mjs'
|
||||
import {
|
||||
captureHostedWebViewAdversarialObservation,
|
||||
hostedWebViewAdversarialContentObservations
|
||||
} from './hosted-webview-adversarial-content.mjs'
|
||||
import { readHostedWebViewTextPoint } from './hosted-webview-cdp-session.mjs'
|
||||
import { tapHostedIosPoint } from './hosted-ios-emulator-accessibility.mjs'
|
||||
import { registerWorktreeForPairingRuntime } from './start-emulator-pairing-runtime.mjs'
|
||||
|
||||
const execFileAsync = promisify(execFile)
|
||||
|
||||
export async function registerHostedIosAdversarialRepository(
|
||||
{ fixture, orcaCli, pairingRuntimeUserDataPath },
|
||||
runCli = execFileAsync
|
||||
) {
|
||||
const env = {
|
||||
...process.env,
|
||||
ORCA_DEV_USER_DATA_PATH: pairingRuntimeUserDataPath,
|
||||
ORCA_USER_DATA_PATH: pairingRuntimeUserDataPath
|
||||
}
|
||||
await registerWorktreeForPairingRuntime({ env }, fixture.root, {
|
||||
logStep: () => {},
|
||||
logSuccess: () => {},
|
||||
orca: async (args, options) => {
|
||||
const result = await runCli(orcaCli, args, {
|
||||
cwd: options.cwd,
|
||||
encoding: 'utf8',
|
||||
env: options.env,
|
||||
timeout: options.timeout
|
||||
})
|
||||
return {
|
||||
stdout: String(result.stdout).trim(),
|
||||
stderr: String(result.stderr).trim()
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export function createHostedIosAdversarialContentInspector({ emulator, fixture, timeoutMs }) {
|
||||
const observations = []
|
||||
return {
|
||||
async inspect({ document, phase }) {
|
||||
if (phase === 'sessionDiff') {
|
||||
const point = await readHostedWebViewTextPoint(document, fixture.filename)
|
||||
await tapHostedIosPoint(emulator, point)
|
||||
await delay(250)
|
||||
}
|
||||
observations.push(
|
||||
await captureHostedWebViewAdversarialObservation({
|
||||
document,
|
||||
expectedMarker: phase === 'sessionDiff' ? HOSTED_ADVERSARIAL_CONTENT_MARKER : undefined,
|
||||
timeoutMs: Math.min(timeoutMs, 15_000)
|
||||
})
|
||||
)
|
||||
},
|
||||
evidence() {
|
||||
return hostedWebViewAdversarialContentObservations(observations)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function delay(ms) {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms))
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
import { spawn } from 'node:child_process'
|
||||
import path from 'node:path'
|
||||
import process from 'node:process'
|
||||
|
||||
export function startHostedIosMobileLauncher({
|
||||
deviceUdid,
|
||||
emulatorControlUserDataPath,
|
||||
orcaCli,
|
||||
runtimeDirectory,
|
||||
worktree
|
||||
}) {
|
||||
return spawn(
|
||||
process.execPath,
|
||||
[
|
||||
path.join(worktree, 'mobile', 'scripts', 'start-emulator.mjs'),
|
||||
'--worktree',
|
||||
worktree,
|
||||
'--device',
|
||||
deviceUdid,
|
||||
'--wait-for-ready'
|
||||
],
|
||||
{
|
||||
cwd: worktree,
|
||||
env: {
|
||||
...process.env,
|
||||
ORCA_CLI: orcaCli,
|
||||
ORCA_E2E_MOBILE_AUTO_SELECT_PAIRED_HOST: '1',
|
||||
ORCA_E2E_MOBILE_AGENT_HISTORY_FIXTURE: '1',
|
||||
ORCA_E2E_MOBILE_RUN_DIRECTORY: path.join(runtimeDirectory, 'paired-host'),
|
||||
ORCA_E2E_MOBILE_RESTART_HOLD_MS: '2000',
|
||||
ORCA_E2E_MOBILE_EMULATOR_CONTROL_USER_DATA_PATH: emulatorControlUserDataPath
|
||||
},
|
||||
stdio: ['ignore', 'pipe', 'pipe']
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
export function waitForHostedIosMobileLauncher(child, timeoutMs) {
|
||||
return new Promise((resolve, reject) => {
|
||||
let outputTail = ''
|
||||
let settled = false
|
||||
const timer = setTimeout(() => {
|
||||
finish(new Error(`Mobile launcher timed out.\n${outputTail}`))
|
||||
}, timeoutMs)
|
||||
const consume = (chunk, target) => {
|
||||
const text = String(chunk)
|
||||
target.write(text)
|
||||
outputTail = (outputTail + text).slice(-32 * 1024)
|
||||
if (outputTail.includes('Setup complete!')) {
|
||||
finish()
|
||||
}
|
||||
}
|
||||
const finish = (error) => {
|
||||
if (settled) {
|
||||
return
|
||||
}
|
||||
settled = true
|
||||
clearTimeout(timer)
|
||||
child.off('exit', handleExit)
|
||||
if (error) {
|
||||
reject(error)
|
||||
} else {
|
||||
resolve()
|
||||
}
|
||||
}
|
||||
const handleExit = (code) => {
|
||||
finish(new Error(`Mobile launcher exited with code ${code}.\n${outputTail}`))
|
||||
}
|
||||
child.stdout.on('data', (chunk) => consume(chunk, process.stdout))
|
||||
child.stderr.on('data', (chunk) => consume(chunk, process.stderr))
|
||||
child.once('error', finish)
|
||||
child.once('exit', handleExit)
|
||||
})
|
||||
}
|
||||
@@ -22,6 +22,7 @@ export async function verifyHostedSourceControlReviewJourney({
|
||||
sessionDocument,
|
||||
timeoutMs,
|
||||
expectedSessionDiffText = '2 tabs',
|
||||
inspectChangedContent,
|
||||
tapPoint = tapHostedJourneyPoint
|
||||
}) {
|
||||
const sourceControl = await journeyStep('wait for Source Control route', () =>
|
||||
@@ -48,6 +49,11 @@ export async function verifyHostedSourceControlReviewJourney({
|
||||
if (!changedFileLabel) {
|
||||
throw new Error('Source Control has no changed file available for Review.')
|
||||
}
|
||||
if (inspectChangedContent) {
|
||||
await journeyStep('inspect Source Control content', () =>
|
||||
inspectChangedContent({ phase: 'sourceControl', document: sourceControl })
|
||||
)
|
||||
}
|
||||
if (nativeBaselines) {
|
||||
sourceState = await journeyStep('wait for stable Source Control parity state', () =>
|
||||
waitForSourceControlParityState(sourceControl, timeoutMs, sourceState)
|
||||
@@ -77,6 +83,11 @@ export async function verifyHostedSourceControlReviewJourney({
|
||||
timeoutMs
|
||||
})
|
||||
)
|
||||
if (inspectChangedContent) {
|
||||
await journeyStep('inspect Session diff content', () =>
|
||||
inspectChangedContent({ phase: 'sessionDiff', document: sessionDiff })
|
||||
)
|
||||
}
|
||||
await journeyStep('open standalone Review route', () =>
|
||||
navigateHostedWebViewRoute(sessionDiff, standaloneReviewRoute(sourceState.href))
|
||||
)
|
||||
@@ -94,6 +105,11 @@ export async function verifyHostedSourceControlReviewJourney({
|
||||
throw new Error(`Review is missing ${label}.`)
|
||||
}
|
||||
}
|
||||
if (inspectChangedContent) {
|
||||
await journeyStep('inspect Review content', () =>
|
||||
inspectChangedContent({ phase: 'review', document: review })
|
||||
)
|
||||
}
|
||||
const hostedReview = nativeBaselines
|
||||
? await journeyStep('capture Review parity', () =>
|
||||
captureHostedSourceControlReviewScreen({
|
||||
@@ -150,7 +166,7 @@ async function openSourceControlRoute({
|
||||
for (let attempt = 0; attempt < 3; attempt += 1) {
|
||||
try {
|
||||
const point = await readHostedWebViewControlPoint(sessionDocument, 'Open source control')
|
||||
await tapPoint(emulator, point, 'Open source control', attempt)
|
||||
await tapPoint(emulator, point, 'Open source control', attempt, sessionDocument)
|
||||
return await waitForVisibleHostedWebView({
|
||||
discoveryUrl,
|
||||
expectedText: 'Source Control',
|
||||
@@ -177,7 +193,7 @@ async function openSessionDiffRoute({
|
||||
for (let attempt = 0; attempt < 3; attempt += 1) {
|
||||
try {
|
||||
const point = await readHostedWebViewControlPoint(sourceControl, label)
|
||||
await tapPoint(emulator, point, label, attempt)
|
||||
await tapPoint(emulator, point, label, attempt, sourceControl)
|
||||
} catch (error) {
|
||||
lastError = error
|
||||
continue
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
import { WebSocket } from 'ws'
|
||||
import {
|
||||
HOSTED_ADVERSARIAL_CONTENT,
|
||||
HOSTED_ADVERSARIAL_CONTENT_MARKER,
|
||||
HOSTED_ADVERSARIAL_FILENAME,
|
||||
HOSTED_ADVERSARIAL_FILENAME_MARKER
|
||||
} from './hosted-adversarial-repository-fixture.mjs'
|
||||
import {
|
||||
evaluateHostedDocumentWithRetry,
|
||||
readHostedWebViewState
|
||||
} from './hosted-webview-cdp-session.mjs'
|
||||
|
||||
const executionExpression = `JSON.stringify({
|
||||
filenameExecuted: Boolean(globalThis.${HOSTED_ADVERSARIAL_FILENAME_MARKER}),
|
||||
contentExecuted: Boolean(globalThis.${HOSTED_ADVERSARIAL_CONTENT_MARKER}),
|
||||
injectedImageCount: [...document.images].filter((image) => image.getAttribute('src') === 'x').length
|
||||
})`
|
||||
|
||||
export async function verifyHostedWebViewAdversarialContent({
|
||||
document,
|
||||
documents,
|
||||
WebSocketCtor = WebSocket
|
||||
}) {
|
||||
const targets = documents ?? (document ? [document] : [])
|
||||
if (targets.length === 0) {
|
||||
throw new Error('Hosted adversarial content requires a document')
|
||||
}
|
||||
const observations = await Promise.all(
|
||||
targets.map((target) => readHostedWebViewAdversarialContent(target, WebSocketCtor))
|
||||
)
|
||||
return hostedWebViewAdversarialContentObservations(observations)
|
||||
}
|
||||
|
||||
export async function readHostedWebViewAdversarialContent(document, WebSocketCtor = WebSocket) {
|
||||
const [state, executionValue] = await Promise.all([
|
||||
readHostedWebViewState(document, WebSocketCtor),
|
||||
evaluateHostedDocumentWithRetry(document, executionExpression, WebSocketCtor)
|
||||
])
|
||||
return { state, execution: JSON.parse(executionValue) }
|
||||
}
|
||||
|
||||
export async function captureHostedWebViewAdversarialObservation({
|
||||
document,
|
||||
expectedMarker,
|
||||
timeoutMs,
|
||||
WebSocketCtor = WebSocket
|
||||
}) {
|
||||
const deadline = Date.now() + timeoutMs
|
||||
let observation
|
||||
do {
|
||||
observation = await readHostedWebViewAdversarialContent(document, WebSocketCtor)
|
||||
const text = `${observation.state.bodyText}\n${observation.state.labels.join('\n')}`
|
||||
if (!expectedMarker || text.includes(expectedMarker)) {
|
||||
return observation
|
||||
}
|
||||
await delay(250)
|
||||
} while (Date.now() < deadline)
|
||||
throw new Error(`Hosted adversarial marker was not rendered: ${expectedMarker}`)
|
||||
}
|
||||
|
||||
export function hostedWebViewAdversarialContentObservations(observations) {
|
||||
for (const { execution } of observations) {
|
||||
if (
|
||||
execution?.filenameExecuted !== false ||
|
||||
execution?.contentExecuted !== false ||
|
||||
execution?.injectedImageCount !== 0
|
||||
) {
|
||||
throw new Error(`Hosted adversarial content executed: ${JSON.stringify(execution)}`)
|
||||
}
|
||||
}
|
||||
return hostedWebViewAdversarialContentEvidence({
|
||||
text: observations
|
||||
.map(({ state }) => `${state.bodyText}\n${state.labels.join('\n')}`)
|
||||
.join('\n'),
|
||||
execution: {
|
||||
filenameExecuted: false,
|
||||
contentExecuted: false,
|
||||
injectedImageCount: 0
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export function hostedWebViewAdversarialContentEvidence({ text, execution }) {
|
||||
const filenameRendered = text.includes(HOSTED_ADVERSARIAL_FILENAME)
|
||||
const diffRendered = text.includes(HOSTED_ADVERSARIAL_CONTENT)
|
||||
if (!filenameRendered || !diffRendered) {
|
||||
throw new Error(
|
||||
`Hosted adversarial content was not rendered: filename=${filenameRendered} diff=${diffRendered}`
|
||||
)
|
||||
}
|
||||
if (
|
||||
execution?.filenameExecuted !== false ||
|
||||
execution?.contentExecuted !== false ||
|
||||
execution?.injectedImageCount !== 0
|
||||
) {
|
||||
throw new Error(`Hosted adversarial content executed: ${JSON.stringify(execution)}`)
|
||||
}
|
||||
return {
|
||||
filenameRenderedAsText: true,
|
||||
diffRenderedAsText: true,
|
||||
injectedImageCount: 0,
|
||||
scriptMarkersExecuted: false
|
||||
}
|
||||
}
|
||||
|
||||
function delay(ms) {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms))
|
||||
}
|
||||
@@ -236,7 +236,7 @@ export async function readHostedWebViewTextPoint(
|
||||
rect.left < innerWidth;
|
||||
};
|
||||
const matches = Array.from(document.querySelectorAll('body *')).filter((candidate) =>
|
||||
candidate.children.length === 0 && isVisible(candidate) && (
|
||||
candidate.children.length === 0 && (${options.reveal === true} || isVisible(candidate)) && (
|
||||
${
|
||||
options.ignoreCase === true
|
||||
? "String(candidate.textContent ?? '').trim().toLocaleLowerCase()"
|
||||
@@ -246,6 +246,9 @@ export async function readHostedWebViewTextPoint(
|
||||
);
|
||||
const element = matches[${options.occurrence ?? 0}];
|
||||
if (!element) return '';
|
||||
if (${options.reveal === true}) {
|
||||
element.scrollIntoView({ block: 'nearest', inline: 'nearest' });
|
||||
}
|
||||
const rect = element.getBoundingClientRect();
|
||||
const screenWidth = Number(screen.width);
|
||||
const screenHeight = Number(screen.height);
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
import process from 'node:process'
|
||||
|
||||
const usage =
|
||||
'Usage: node scripts/run-hosted-webview-simulator-e2e.mjs [--device <name|udid>] [--timeout-ms <ms>] [--expected-build <sha256>] [--accounts-only] [--security-only] [--isolation-only] [--clipboard-image-only] [--photos-revocation-only] [--files-preview-only] [--native-settings-only] [--source-control-only] [--skip-native-build] [--reuse-native-install]'
|
||||
'Usage: node scripts/run-hosted-webview-simulator-e2e.mjs [--device <name|udid>] [--timeout-ms <ms>] [--expected-build <sha256>] [--accounts-only] [--security-only] [--isolation-only] [--clipboard-image-only] [--photos-revocation-only] [--files-preview-only] [--native-settings-only] [--source-control-only] [--adversarial-content] [--skip-native-build] [--reuse-native-install]'
|
||||
|
||||
export function parseHostedWebViewSimulatorE2eOptions(args) {
|
||||
const parsed = {
|
||||
accountsOnly: false,
|
||||
adversarialContent: false,
|
||||
clipboardImageOnly: false,
|
||||
device: 'iPhone 17 Pro',
|
||||
expectedBuild: undefined,
|
||||
@@ -44,6 +45,8 @@ export function parseHostedWebViewSimulatorE2eOptions(args) {
|
||||
parsed.nativeSettingsOnly = true
|
||||
} else if (args[index] === '--source-control-only') {
|
||||
parsed.sourceControlOnly = true
|
||||
} else if (args[index] === '--adversarial-content') {
|
||||
parsed.adversarialContent = true
|
||||
} else if (args[index] === '--skip-native-build') {
|
||||
parsed.skipNativeBuild = true
|
||||
} else if (args[index] === '--reuse-native-install') {
|
||||
@@ -67,6 +70,7 @@ export function parseHostedWebViewSimulatorE2eOptions(args) {
|
||||
if (
|
||||
[
|
||||
parsed.accountsOnly,
|
||||
parsed.adversarialContent,
|
||||
parsed.clipboardImageOnly,
|
||||
parsed.isolationOnly,
|
||||
parsed.securityOnly,
|
||||
@@ -80,5 +84,6 @@ export function parseHostedWebViewSimulatorE2eOptions(args) {
|
||||
}
|
||||
parsed.securityOnly ||=
|
||||
parsed.clipboardImageOnly || parsed.isolationOnly || parsed.photosRevocationOnly
|
||||
parsed.sourceControlOnly ||= parsed.adversarialContent
|
||||
return parsed
|
||||
}
|
||||
|
||||
@@ -19,6 +19,13 @@ export async function activateHostedWorkspaceRow(
|
||||
activeDocument = await resolveDocument()
|
||||
continue
|
||||
}
|
||||
if (isMissingControl(error) && resolveDocument) {
|
||||
const resolvedDocument = await resolveDocument()
|
||||
if (resolvedDocument.href !== activeDocument.href) {
|
||||
return
|
||||
}
|
||||
activeDocument = resolvedDocument
|
||||
}
|
||||
if (!isMissingControl(error)) {
|
||||
throw error
|
||||
}
|
||||
@@ -33,7 +40,8 @@ async function activateHostedWorkspaceRowOnce(document, workspaceName, activateC
|
||||
try {
|
||||
await activateControl(document, {
|
||||
kind: 'label',
|
||||
value: `Open ${workspaceName}`
|
||||
value: `Open ${workspaceName}`,
|
||||
reveal: true
|
||||
})
|
||||
return
|
||||
} catch (error) {
|
||||
@@ -46,7 +54,8 @@ async function activateHostedWorkspaceRowOnce(document, workspaceName, activateC
|
||||
kind: 'text',
|
||||
value: workspaceName,
|
||||
ignoreCase: true,
|
||||
occurrence: 1
|
||||
occurrence: 1,
|
||||
reveal: true
|
||||
})
|
||||
} catch (error) {
|
||||
if (!isMissingControl(error)) {
|
||||
@@ -55,7 +64,8 @@ async function activateHostedWorkspaceRowOnce(document, workspaceName, activateC
|
||||
await activateControl(document, {
|
||||
kind: 'text',
|
||||
value: workspaceName,
|
||||
ignoreCase: true
|
||||
ignoreCase: true,
|
||||
reveal: true
|
||||
})
|
||||
await delay(250)
|
||||
try {
|
||||
@@ -63,7 +73,8 @@ async function activateHostedWorkspaceRowOnce(document, workspaceName, activateC
|
||||
kind: 'text',
|
||||
value: workspaceName,
|
||||
ignoreCase: true,
|
||||
occurrence: 1
|
||||
occurrence: 1,
|
||||
reveal: true
|
||||
})
|
||||
} catch (fallbackError) {
|
||||
if (isStaleDocument(fallbackError)) {
|
||||
|
||||
@@ -6,6 +6,11 @@ import process from 'node:process'
|
||||
import { promisify } from 'node:util'
|
||||
import { resolveEmulatorOrcaCli } from './emulator-orca-cli-selection.mjs'
|
||||
import { verifyHostedAndroidAgentHistoryJourney } from './hosted-android-agent-history-journey.mjs'
|
||||
import {
|
||||
createHostedAdversarialRepositoryFixture,
|
||||
HOSTED_ADVERSARIAL_CONTENT_MARKER,
|
||||
removeHostedAdversarialRepositoryFixture
|
||||
} from './hosted-adversarial-repository-fixture.mjs'
|
||||
import {
|
||||
readHostedAndroidExitInfo,
|
||||
verifyHostedAndroidPrivacyAudit
|
||||
@@ -38,6 +43,10 @@ import {
|
||||
} from './hosted-webview-cdp-session.mjs'
|
||||
import { verifyHostedWebViewExecutableIsolation } from './hosted-webview-executable-isolation.mjs'
|
||||
import { verifyHostedWebViewPrivacyIsolation } from './hosted-webview-privacy-isolation.mjs'
|
||||
import {
|
||||
captureHostedWebViewAdversarialObservation,
|
||||
hostedWebViewAdversarialContentObservations
|
||||
} from './hosted-webview-adversarial-content.mjs'
|
||||
import { resolveHostedWebViewRuntimeDirectory } from './hosted-webview-runtime-directory.mjs'
|
||||
import { activateHostedWorkspaceRow } from './hosted-webview-workspace-activation.mjs'
|
||||
import { verifyHostedSourceControlReviewJourney } from './hosted-ios-source-control-review-journey.mjs'
|
||||
@@ -72,6 +81,7 @@ async function main() {
|
||||
let probe
|
||||
let inspector
|
||||
let exitInfoBaseline
|
||||
let adversarialFixture
|
||||
const reversePorts = new Set()
|
||||
try {
|
||||
await stage('Android emulator', () => runAndroidAdb(adb, ['get-state']))
|
||||
@@ -82,6 +92,12 @@ async function main() {
|
||||
if (!options.skipNativeBuild) {
|
||||
await stage('Android debug app build', () => buildHostedAndroidDebugApp({ adb, androidDir }))
|
||||
}
|
||||
if (options.adversarialContent) {
|
||||
adversarialFixture = await stage(
|
||||
'adversarial repository fixture',
|
||||
createHostedAdversarialRepositoryFixture
|
||||
)
|
||||
}
|
||||
probe = await stage('network isolation sentinel', startHostedWebViewSecurityProbe)
|
||||
runtime = await stage('temporary paired desktop runtime', () =>
|
||||
startHeadlessPairingRuntime({
|
||||
@@ -94,9 +110,12 @@ async function main() {
|
||||
logSuccess: () => {}
|
||||
})
|
||||
)
|
||||
runtime.env.ORCA_E2E_MOBILE_AGENT_HISTORY_FIXTURE = '1'
|
||||
if (!options.adversarialContent) {
|
||||
runtime.env.ORCA_E2E_MOBILE_AGENT_HISTORY_FIXTURE = '1'
|
||||
}
|
||||
const testWorkspace = adversarialFixture?.root ?? worktree
|
||||
await stage('test workspace registration', () =>
|
||||
registerWorktreeForPairingRuntime(runtime, worktree, {
|
||||
registerWorktreeForPairingRuntime(runtime, testWorkspace, {
|
||||
orca: runOrca,
|
||||
logStep: () => {},
|
||||
logSuccess: () => {}
|
||||
@@ -134,7 +153,8 @@ async function main() {
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
)
|
||||
const expectedWorkspace = path.basename(worktree)
|
||||
const expectedWorkspace = path.basename(testWorkspace)
|
||||
const workspaceRowName = adversarialFixture?.workspaceRowName ?? expectedWorkspace
|
||||
const workspaceDocument = await stage('hosted workspace data', () =>
|
||||
waitForVisibleHostedWebView({
|
||||
discoveryUrl,
|
||||
@@ -142,23 +162,20 @@ async function main() {
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
)
|
||||
let privacyIsolation = null
|
||||
if (options.securityOnly) {
|
||||
privacyIsolation = await stage('workspace privacy isolation probe', () =>
|
||||
verifyHostedWebViewPrivacyIsolation({ document: workspaceDocument })
|
||||
)
|
||||
}
|
||||
const privacyIsolation = await stage('workspace privacy isolation probe', () =>
|
||||
verifyHostedWebViewPrivacyIsolation({ document: workspaceDocument })
|
||||
)
|
||||
await stage('workspace activation', async () => {
|
||||
try {
|
||||
await activateHostedWorkspaceRow(
|
||||
workspaceDocument,
|
||||
expectedWorkspace,
|
||||
workspaceRowName,
|
||||
(document, target) => activateAndroidWorkspaceControl(emulator, document, target),
|
||||
Math.min(options.timeoutMs, 15_000),
|
||||
() =>
|
||||
waitForVisibleHostedWebView({
|
||||
discoveryUrl,
|
||||
expectedText: 'Orca Desktop',
|
||||
expectedText: workspaceRowName,
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
)
|
||||
@@ -194,29 +211,69 @@ async function main() {
|
||||
})
|
||||
)
|
||||
let agentHistory = null
|
||||
let adversarialContent = null
|
||||
const adversarialObservations = []
|
||||
let sourceControlReview = null
|
||||
let isolationDocument = sessionDocument
|
||||
if (!options.securityOnly) {
|
||||
const agentHistoryResult = await stage('Agent History journey', () =>
|
||||
verifyHostedAndroidAgentHistoryJourney({
|
||||
discoveryUrl,
|
||||
emulator,
|
||||
sessionDocument,
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
)
|
||||
const { returnedSessionDocument, ...evidence } = agentHistoryResult
|
||||
agentHistory = evidence
|
||||
let sourceSessionDocument = sessionDocument
|
||||
if (!options.adversarialContent) {
|
||||
const agentHistoryResult = await stage('Agent History journey', () =>
|
||||
verifyHostedAndroidAgentHistoryJourney({
|
||||
discoveryUrl,
|
||||
emulator,
|
||||
sessionDocument,
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
)
|
||||
const { returnedSessionDocument, ...evidence } = agentHistoryResult
|
||||
agentHistory = evidence
|
||||
sourceSessionDocument = returnedSessionDocument
|
||||
}
|
||||
sourceControlReview = await stage('Source Control and Review journey', () =>
|
||||
verifyHostedSourceControlReviewJourney({
|
||||
discoveryUrl,
|
||||
emulator,
|
||||
sessionDocument: returnedSessionDocument,
|
||||
expectedSessionDiffText: '3 tabs',
|
||||
sessionDocument: sourceSessionDocument,
|
||||
expectedSessionDiffText: options.adversarialContent ? '2 tabs' : '3 tabs',
|
||||
inspectChangedContent: options.adversarialContent
|
||||
? async ({ document, phase }) => {
|
||||
if (phase === 'sessionDiff') {
|
||||
await activateAndroidAdversarialDiffTab(
|
||||
emulator,
|
||||
document,
|
||||
adversarialFixture.filename
|
||||
)
|
||||
}
|
||||
adversarialObservations.push(
|
||||
await captureHostedWebViewAdversarialObservation({
|
||||
document,
|
||||
expectedMarker:
|
||||
phase === 'sessionDiff' ? HOSTED_ADVERSARIAL_CONTENT_MARKER : undefined,
|
||||
timeoutMs: Math.min(options.timeoutMs, 15_000)
|
||||
})
|
||||
)
|
||||
}
|
||||
: undefined,
|
||||
timeoutMs: options.timeoutMs,
|
||||
tapPoint: tapHostedAndroidJourneyControl
|
||||
})
|
||||
)
|
||||
if (options.adversarialContent) {
|
||||
adversarialContent = await stage('adversarial filename and diff presentation', () => {
|
||||
try {
|
||||
return hostedWebViewAdversarialContentObservations(adversarialObservations)
|
||||
} catch (error) {
|
||||
const states = adversarialObservations.map(({ state }) => ({
|
||||
bodyText: state.bodyText.slice(0, 1024),
|
||||
labels: state.labels.slice(0, 16)
|
||||
}))
|
||||
throw new Error(
|
||||
`${error instanceof Error ? error.message : String(error)}. States ${JSON.stringify(states)}`
|
||||
)
|
||||
}
|
||||
})
|
||||
}
|
||||
isolationDocument = await waitForVisibleHostedWebView({
|
||||
discoveryUrl,
|
||||
expectedText: 'reviewed',
|
||||
@@ -243,9 +300,6 @@ async function main() {
|
||||
probeId: probe.token
|
||||
})
|
||||
)
|
||||
privacyIsolation ??= await stage('privacy isolation probe', () =>
|
||||
verifyHostedWebViewPrivacyIsolation({ document: isolationDocument })
|
||||
)
|
||||
await delay(500)
|
||||
if (probe.observations.length > 0) {
|
||||
throw new Error(
|
||||
@@ -269,6 +323,7 @@ async function main() {
|
||||
pid: inspector.pid,
|
||||
workspace: expectedWorkspace,
|
||||
agentHistory,
|
||||
adversarialContent,
|
||||
sourceControlReview,
|
||||
networkIsolation,
|
||||
navigationIsolation,
|
||||
@@ -292,6 +347,9 @@ async function main() {
|
||||
await metro?.stop()
|
||||
await runtime?.stop({ shutdownDaemon: true })
|
||||
await probe?.stop()
|
||||
if (adversarialFixture) {
|
||||
await removeHostedAdversarialRepositoryFixture(adversarialFixture)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -335,6 +393,14 @@ async function activateAndroidWorkspaceControl(emulator, document, target) {
|
||||
if (target.kind !== 'text') {
|
||||
return activateHostedWebViewControl(document, target)
|
||||
}
|
||||
if (target.reveal) {
|
||||
await readHostedWebViewTextPoint(document, target.value, undefined, {
|
||||
ignoreCase: target.ignoreCase,
|
||||
occurrence: target.occurrence,
|
||||
reveal: true
|
||||
})
|
||||
await delay(250)
|
||||
}
|
||||
const point = await readHostedWebViewTextPoint(document, target.value, undefined, {
|
||||
ignoreCase: target.ignoreCase,
|
||||
occurrence: target.occurrence
|
||||
@@ -342,17 +408,26 @@ async function activateAndroidWorkspaceControl(emulator, document, target) {
|
||||
await tapHostedAndroidPoint(emulator, point)
|
||||
}
|
||||
|
||||
async function tapHostedAndroidJourneyControl(emulator, point, label) {
|
||||
if (label) {
|
||||
async function tapHostedAndroidJourneyControl(emulator, point, label, attempt = 0, document) {
|
||||
if (label && attempt === 0) {
|
||||
try {
|
||||
return await tapHostedAndroidAccessibilityControl(emulator, label, 5_000)
|
||||
} catch {
|
||||
// Chromium may omit a WebView descendant during an accessibility-tree refresh.
|
||||
}
|
||||
}
|
||||
if (label && attempt > 0 && document) {
|
||||
return activateHostedWebViewControl(document, { kind: 'label', value: label, reveal: true })
|
||||
}
|
||||
return tapHostedAndroidPoint(emulator, point)
|
||||
}
|
||||
|
||||
async function activateAndroidAdversarialDiffTab(emulator, document, filename) {
|
||||
const point = await readHostedWebViewTextPoint(document, filename)
|
||||
await tapHostedAndroidPoint(emulator, point)
|
||||
await delay(250)
|
||||
}
|
||||
|
||||
async function proveSentinelReachability(command, probe) {
|
||||
await runAndroidAdb(command, ['shell', 'nc', '-z', '-w', '5', '127.0.0.1', String(probe.port)])
|
||||
if (!probe.observations.includes('tcp:connection')) {
|
||||
@@ -388,6 +463,7 @@ function parseOptions(args) {
|
||||
const result = {
|
||||
adb: null,
|
||||
apk: defaultApk,
|
||||
adversarialContent: false,
|
||||
securityOnly: false,
|
||||
skipNativeBuild: false,
|
||||
timeoutMs: 90_000
|
||||
@@ -400,6 +476,8 @@ function parseOptions(args) {
|
||||
result.adb = requireValue(args, ++index, option)
|
||||
} else if (option === '--apk') {
|
||||
result.apk = path.resolve(requireValue(args, ++index, option))
|
||||
} else if (option === '--adversarial-content') {
|
||||
result.adversarialContent = true
|
||||
} else if (option === '--skip-native-build') {
|
||||
result.skipNativeBuild = true
|
||||
} else if (option === '--security-only') {
|
||||
@@ -413,6 +491,9 @@ function parseOptions(args) {
|
||||
if (!Number.isInteger(result.timeoutMs) || result.timeoutMs < 1_000) {
|
||||
throw new Error('--timeout-ms must be an integer of at least 1000')
|
||||
}
|
||||
if (result.adversarialContent && result.securityOnly) {
|
||||
throw new Error('--adversarial-content and --security-only are mutually exclusive')
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { execFile, spawn } from 'node:child_process'
|
||||
import { execFile } from 'node:child_process'
|
||||
import { mkdirSync } from 'node:fs'
|
||||
import net from 'node:net'
|
||||
import path from 'node:path'
|
||||
@@ -8,6 +8,10 @@ import process from 'node:process'
|
||||
import { promisify } from 'node:util'
|
||||
import { startCdpServer } from 'inspect-webkit'
|
||||
import { resolveEmulatorOrcaCli } from './emulator-orca-cli-selection.mjs'
|
||||
import {
|
||||
createHostedAdversarialRepositoryFixture,
|
||||
removeHostedAdversarialRepositoryFixture
|
||||
} from './hosted-adversarial-repository-fixture.mjs'
|
||||
import { stopHostedChildProcess } from './hosted-child-process-shutdown.mjs'
|
||||
import { parseHostedWebViewSimulatorE2eOptions } from './hosted-webview-simulator-e2e-options.mjs'
|
||||
import {
|
||||
@@ -24,6 +28,10 @@ import {
|
||||
import { verifyHostedWebViewExecutableIsolation } from './hosted-webview-executable-isolation.mjs'
|
||||
import { verifyHostedWebViewPrivacyIsolation } from './hosted-webview-privacy-isolation.mjs'
|
||||
import { captureNativeAgentHistoryBaseline } from './hosted-ios-agent-history-parity.mjs'
|
||||
import {
|
||||
createHostedIosAdversarialContentInspector,
|
||||
registerHostedIosAdversarialRepository
|
||||
} from './hosted-ios-adversarial-content.mjs'
|
||||
import {
|
||||
captureHostedCoreRouteParity,
|
||||
captureNativeCoreRouteBaselines
|
||||
@@ -38,6 +46,10 @@ import {
|
||||
} from './hosted-ios-workspace-parity.mjs'
|
||||
import { verifyHostedAgentHistoryJourney } from './hosted-ios-agent-history-journey.mjs'
|
||||
import { openHostedIosHybridRoute } from './hosted-ios-hybrid-route-handoff.mjs'
|
||||
import {
|
||||
startHostedIosMobileLauncher,
|
||||
waitForHostedIosMobileLauncher
|
||||
} from './hosted-ios-mobile-launcher.mjs'
|
||||
import { verifyHostedNativeTerminalSettingsHandoff } from './hosted-ios-native-settings-handoff.mjs'
|
||||
import { verifyHostedSourceControlReviewJourney } from './hosted-ios-source-control-review-journey.mjs'
|
||||
import { captureNativeSourceControlReviewBaselines } from './hosted-ios-source-control-review-parity.mjs'
|
||||
@@ -56,7 +68,6 @@ import { hostedIosSimulatorAppPreparation } from './hosted-ios-simulator-app-pre
|
||||
|
||||
const execFileAsync = promisify(execFile)
|
||||
const worktree = path.resolve(import.meta.dirname, '../..')
|
||||
const launcherPath = path.join(worktree, 'mobile', 'scripts', 'start-emulator.mjs')
|
||||
const options = parseHostedWebViewSimulatorE2eOptions(process.argv.slice(2))
|
||||
const runtimeDirectory = resolveHostedWebViewRuntimeDirectory({
|
||||
worktree,
|
||||
@@ -81,7 +92,12 @@ async function main() {
|
||||
let networkProbe = null
|
||||
let emulatorController = null
|
||||
let nativeAppPath = null
|
||||
let adversarialFixture = null
|
||||
let adversarialInspector = null
|
||||
try {
|
||||
if (options.adversarialContent) {
|
||||
adversarialFixture = await createHostedAdversarialRepositoryFixture()
|
||||
}
|
||||
networkProbe = await startHostedIosWebViewSecurityProbe()
|
||||
await bootSimulator(deviceUdid)
|
||||
emulatorController = await startHostedIosEmulatorController({
|
||||
@@ -97,8 +113,14 @@ async function main() {
|
||||
resetHostedIosPhotosPermission(deviceUdid)
|
||||
)
|
||||
}
|
||||
launcher = startMobileLauncher(deviceUdid, emulatorController.userData)
|
||||
await waitForLauncher(launcher, options.timeoutMs)
|
||||
launcher = startHostedIosMobileLauncher({
|
||||
deviceUdid,
|
||||
emulatorControlUserDataPath: emulatorController.userData,
|
||||
orcaCli: orcaSelection.command,
|
||||
runtimeDirectory,
|
||||
worktree
|
||||
})
|
||||
await waitForHostedIosMobileLauncher(launcher, options.timeoutMs)
|
||||
const emulator = {
|
||||
deviceUdid,
|
||||
orcaCli: orcaSelection.command,
|
||||
@@ -109,6 +131,20 @@ async function main() {
|
||||
const nativeOnboarding = await evidenceStep('native onboarding', () =>
|
||||
completeHostedIosNativeOnboarding(emulator, expectedWorkspace, options.timeoutMs)
|
||||
)
|
||||
if (adversarialFixture) {
|
||||
await evidenceStep('adversarial repository registration', () =>
|
||||
registerHostedIosAdversarialRepository({
|
||||
fixture: adversarialFixture,
|
||||
orcaCli: orcaSelection.command,
|
||||
pairingRuntimeUserDataPath: path.join(runtimeDirectory, 'paired-host', 'userData')
|
||||
})
|
||||
)
|
||||
adversarialInspector = createHostedIosAdversarialContentInspector({
|
||||
emulator,
|
||||
fixture: adversarialFixture,
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
}
|
||||
const nativeWorkspace =
|
||||
options.securityOnly ||
|
||||
options.filesPreviewOnly ||
|
||||
@@ -140,6 +176,7 @@ async function main() {
|
||||
)
|
||||
const nativeCoreRoutes =
|
||||
options.accountsOnly ||
|
||||
options.adversarialContent ||
|
||||
options.securityOnly ||
|
||||
options.filesPreviewOnly ||
|
||||
options.nativeSettingsOnly ||
|
||||
@@ -171,6 +208,7 @@ async function main() {
|
||||
)
|
||||
const nativeSourceControlReview =
|
||||
options.accountsOnly ||
|
||||
options.adversarialContent ||
|
||||
options.securityOnly ||
|
||||
options.filesPreviewOnly ||
|
||||
options.nativeSettingsOnly
|
||||
@@ -211,6 +249,11 @@ async function main() {
|
||||
expectedText: 'Orca Desktop',
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
const workspacePrivacyIsolation = options.adversarialContent
|
||||
? await evidenceStep('workspace privacy isolation probe', () =>
|
||||
verifyHostedWebViewPrivacyIsolation({ document: workspaceDocument })
|
||||
)
|
||||
: null
|
||||
const securityJourney = {
|
||||
deviceUdid,
|
||||
discoveryUrl,
|
||||
@@ -339,13 +382,13 @@ async function main() {
|
||||
if (options.sourceControlOnly) {
|
||||
await activateHostedWorkspaceRow(
|
||||
workspaceDocument,
|
||||
expectedWorkspace,
|
||||
adversarialFixture?.workspaceRowName ?? expectedWorkspace,
|
||||
activateHostedWebViewControl,
|
||||
options.timeoutMs,
|
||||
() =>
|
||||
waitForVisibleHostedWebView({
|
||||
discoveryUrl: `http://127.0.0.1:${inspectorPort}`,
|
||||
expectedText: 'Orca Desktop',
|
||||
expectedText: adversarialFixture?.workspaceRowName ?? 'Orca Desktop',
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
)
|
||||
@@ -363,11 +406,17 @@ async function main() {
|
||||
emulator,
|
||||
expectedSessionDiffText: options.sourceControlOnly ? '2 tabs' : '3 tabs',
|
||||
nativeBaselines: nativeSourceControlReview,
|
||||
inspectChangedContent: adversarialInspector?.inspect,
|
||||
runtimeDirectory,
|
||||
sessionDocument,
|
||||
timeoutMs: options.timeoutMs
|
||||
})
|
||||
})
|
||||
const adversarialContent = adversarialInspector
|
||||
? await evidenceStep('adversarial filename and diff presentation', () =>
|
||||
adversarialInspector.evidence()
|
||||
)
|
||||
: null
|
||||
const securityDocument =
|
||||
options.securityOnly && terminalDeviceInput
|
||||
? (terminalDeviceInput.terminalClipboardImagePaste?.sessionDocument ??
|
||||
@@ -399,9 +448,11 @@ async function main() {
|
||||
probeId: networkProbe.token
|
||||
})
|
||||
)
|
||||
const privacyIsolation = await evidenceStep('privacy isolation probe', () =>
|
||||
verifyHostedWebViewPrivacyIsolation({ document: securityDocument })
|
||||
)
|
||||
const privacyIsolation =
|
||||
workspacePrivacyIsolation ??
|
||||
(await evidenceStep('privacy isolation probe', () =>
|
||||
verifyHostedWebViewPrivacyIsolation({ document: securityDocument })
|
||||
))
|
||||
await delay(500)
|
||||
if (networkProbe.observations.length > 0) {
|
||||
throw new Error(
|
||||
@@ -436,7 +487,8 @@ async function main() {
|
||||
agentHistory: historyEvidence,
|
||||
coreRouteParity: hostedCoreRoutes?.evidence ?? null,
|
||||
filesPreviewParity: hostedFilesPreview?.evidence ?? null,
|
||||
sourceControlReview
|
||||
sourceControlReview,
|
||||
adversarialContent
|
||||
},
|
||||
null,
|
||||
2
|
||||
@@ -445,6 +497,9 @@ async function main() {
|
||||
} finally {
|
||||
inspector?.stop()
|
||||
await stopHostedChildProcess(launcher)
|
||||
if (adversarialFixture) {
|
||||
await removeHostedAdversarialRepositoryFixture(adversarialFixture)
|
||||
}
|
||||
await emulatorController?.stop()
|
||||
await clearHostedIosWebViewSecurityProbe(deviceUdid)
|
||||
await networkProbe?.stop()
|
||||
@@ -516,64 +571,6 @@ async function bootSimulator(deviceUdid) {
|
||||
await execFileAsync('xcrun', ['simctl', 'bootstatus', deviceUdid, '-b'])
|
||||
}
|
||||
|
||||
function startMobileLauncher(deviceUdid, emulatorControlUserDataPath) {
|
||||
return spawn(
|
||||
process.execPath,
|
||||
[launcherPath, '--worktree', worktree, '--device', deviceUdid, '--wait-for-ready'],
|
||||
{
|
||||
cwd: worktree,
|
||||
env: {
|
||||
...process.env,
|
||||
ORCA_CLI: orcaSelection.command,
|
||||
ORCA_E2E_MOBILE_AUTO_SELECT_PAIRED_HOST: '1',
|
||||
ORCA_E2E_MOBILE_AGENT_HISTORY_FIXTURE: '1',
|
||||
ORCA_E2E_MOBILE_RUN_DIRECTORY: path.join(runtimeDirectory, 'paired-host'),
|
||||
ORCA_E2E_MOBILE_RESTART_HOLD_MS: '2000',
|
||||
ORCA_E2E_MOBILE_EMULATOR_CONTROL_USER_DATA_PATH: emulatorControlUserDataPath
|
||||
},
|
||||
stdio: ['ignore', 'pipe', 'pipe']
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
function waitForLauncher(child, timeoutMs) {
|
||||
return new Promise((resolve, reject) => {
|
||||
let outputTail = ''
|
||||
let settled = false
|
||||
const timer = setTimeout(() => {
|
||||
finish(new Error(`Mobile launcher timed out.\n${outputTail}`))
|
||||
}, timeoutMs)
|
||||
const consume = (chunk, target) => {
|
||||
const text = String(chunk)
|
||||
target.write(text)
|
||||
outputTail = (outputTail + text).slice(-32 * 1024)
|
||||
if (outputTail.includes('Setup complete!')) {
|
||||
finish()
|
||||
}
|
||||
}
|
||||
const finish = (error) => {
|
||||
if (settled) {
|
||||
return
|
||||
}
|
||||
settled = true
|
||||
clearTimeout(timer)
|
||||
child.off('exit', handleExit)
|
||||
if (error) {
|
||||
reject(error)
|
||||
} else {
|
||||
resolve()
|
||||
}
|
||||
}
|
||||
const handleExit = (code) => {
|
||||
finish(new Error(`Mobile launcher exited with code ${code}.\n${outputTail}`))
|
||||
}
|
||||
child.stdout.on('data', (chunk) => consume(chunk, process.stdout))
|
||||
child.stderr.on('data', (chunk) => consume(chunk, process.stderr))
|
||||
child.once('error', finish)
|
||||
child.once('exit', handleExit)
|
||||
})
|
||||
}
|
||||
|
||||
function findAvailableLoopbackPort() {
|
||||
return new Promise((resolve, reject) => {
|
||||
const server = net.createServer()
|
||||
|
||||
@@ -22,4 +22,15 @@ describe('hosted Android emulator session', () => {
|
||||
`)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('ignores automation-process fatals while retaining app-process fatals', () => {
|
||||
const logcat = `
|
||||
E/AndroidRuntime(20196): FATAL EXCEPTION: UiAutomation
|
||||
E/AndroidRuntime(20200): FATAL EXCEPTION: main
|
||||
`
|
||||
|
||||
expect(findHostedAndroidBridgeLogFailures(logcat, '20200')).toEqual([
|
||||
' E/AndroidRuntime(20200): FATAL EXCEPTION: main'
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -50,18 +50,20 @@ describe('hosted Android privacy audit', () => {
|
||||
[
|
||||
'ws://10.0.2.2:8081/message?device=Pixel%209%20-%2016%20-%20API%2036&app=com.stably.orca.mobile&clientid=BridgelessDevSupportManager',
|
||||
'ws://10.0.2.2:8081/message?device=Pixel%209%20-%2016%20-%20API%2036&app=com.stably.orca.mobile&clientid=DevLauncherBridgelessDevSupportManager',
|
||||
'ws://10.0.2.2:57999/message?device=Pixel%209&app=com.stably.orca.mobile&clientid=BridgelessDevSupportManager',
|
||||
'ws://127.0.0.1:53250/socket-probe'
|
||||
].join('\n'),
|
||||
{ devServerPort: 8081, probePort: 53250 }
|
||||
{ devServerPort: 57999, probePort: 53250 }
|
||||
)
|
||||
|
||||
expect(evidence.expectedDebugWebSocketUrls).toBe(3)
|
||||
expect(evidence.expectedDebugWebSocketUrls).toBe(4)
|
||||
expect(evidence.counts.webSocketUrl).toBe(0)
|
||||
})
|
||||
|
||||
it.each([
|
||||
'ws://10.0.2.2:8081/message?device=Pixel&app=com.stably.orca.mobile&clientid=BridgelessDevSupportManager&token=secret',
|
||||
'ws://10.0.2.2:8081/message?device=Pixel&app=other&clientid=BridgelessDevSupportManager',
|
||||
'ws://10.0.2.2:8082/message?device=Pixel&app=com.stably.orca.mobile&clientid=BridgelessDevSupportManager',
|
||||
'wss://127.0.0.1:53250/socket-probe',
|
||||
'ws://127.0.0.1:53250/socket-probe?token=secret'
|
||||
])('rejects mutated debug WebSocket %s', (url) => {
|
||||
|
||||
@@ -122,6 +122,7 @@ describe('hosted iOS Source Control and Review journey', () => {
|
||||
|
||||
it('accepts a platform-specific native tap implementation', async () => {
|
||||
const tapPoint = vi.fn().mockResolvedValue(undefined)
|
||||
const inspectChangedContent = vi.fn().mockResolvedValue(undefined)
|
||||
|
||||
await verifyHostedSourceControlReviewJourney({
|
||||
discoveryUrl: 'http://127.0.0.1:9222',
|
||||
@@ -130,6 +131,7 @@ describe('hosted iOS Source Control and Review journey', () => {
|
||||
href: 'orca-mobile-web://build/h/host/session/workspace'
|
||||
},
|
||||
expectedSessionDiffText: '3 tabs',
|
||||
inspectChangedContent,
|
||||
timeoutMs: 30_000,
|
||||
tapPoint
|
||||
})
|
||||
@@ -139,6 +141,11 @@ describe('hosted iOS Source Control and Review journey', () => {
|
||||
'Open source control',
|
||||
'Open changed file mobile/app/index.tsx'
|
||||
])
|
||||
expect(inspectChangedContent.mock.calls.map((call) => call[0].phase)).toEqual([
|
||||
'sourceControl',
|
||||
'sessionDiff',
|
||||
'review'
|
||||
])
|
||||
expect(mocks.waitForDocument).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
expect.objectContaining({ expectedText: '3 tabs' })
|
||||
|
||||
@@ -0,0 +1,217 @@
|
||||
import { execFile } from 'node:child_process'
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { promisify } from 'node:util'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import {
|
||||
createHostedAdversarialRepositoryFixture,
|
||||
HOSTED_ADVERSARIAL_CONTENT,
|
||||
HOSTED_ADVERSARIAL_CONTENT_MARKER,
|
||||
HOSTED_ADVERSARIAL_FILENAME,
|
||||
HOSTED_ADVERSARIAL_FILENAME_MARKER,
|
||||
HOSTED_ADVERSARIAL_WORKSPACE_ROW,
|
||||
readHostedAdversarialRepositoryContent,
|
||||
removeHostedAdversarialRepositoryFixture
|
||||
} from '../../scripts/hosted-adversarial-repository-fixture.mjs'
|
||||
import {
|
||||
hostedWebViewAdversarialContentEvidence,
|
||||
hostedWebViewAdversarialContentObservations,
|
||||
verifyHostedWebViewAdversarialContent
|
||||
} from '../../scripts/hosted-webview-adversarial-content.mjs'
|
||||
|
||||
const execFileAsync = promisify(execFile)
|
||||
const fixtures: Array<Awaited<ReturnType<typeof createHostedAdversarialRepositoryFixture>>> = []
|
||||
const androidHarnessSource = readFileSync(
|
||||
new URL('../../scripts/run-hosted-android-source-control-review-e2e.mjs', import.meta.url),
|
||||
'utf8'
|
||||
)
|
||||
const androidHarnessPath = fileURLToPath(
|
||||
new URL('../../scripts/run-hosted-android-source-control-review-e2e.mjs', import.meta.url)
|
||||
)
|
||||
const iosHarnessSource = readFileSync(
|
||||
new URL('../../scripts/run-hosted-webview-simulator-e2e.mjs', import.meta.url),
|
||||
'utf8'
|
||||
)
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(fixtures.splice(0).map(removeHostedAdversarialRepositoryFixture))
|
||||
})
|
||||
|
||||
describe('hosted WebView adversarial content', () => {
|
||||
it('creates one disposable hostile filename and diff', async () => {
|
||||
const fixture = await createHostedAdversarialRepositoryFixture()
|
||||
fixtures.push(fixture)
|
||||
|
||||
const status = await execFileAsync('git', ['status', '--short'], {
|
||||
cwd: fixture.root,
|
||||
encoding: 'utf8'
|
||||
})
|
||||
expect(status.stdout).toContain(HOSTED_ADVERSARIAL_FILENAME_MARKER)
|
||||
const diff = await execFileAsync('git', ['diff', '--'], {
|
||||
cwd: fixture.root,
|
||||
encoding: 'utf8'
|
||||
})
|
||||
expect(diff.stdout).toContain(HOSTED_ADVERSARIAL_CONTENT_MARKER)
|
||||
expect(await readHostedAdversarialRepositoryContent(fixture)).toBe(
|
||||
`${HOSTED_ADVERSARIAL_CONTENT}\n`
|
||||
)
|
||||
const branch = await execFileAsync('git', ['symbolic-ref', '--short', 'HEAD'], {
|
||||
cwd: fixture.root,
|
||||
encoding: 'utf8'
|
||||
})
|
||||
expect(branch.stdout.trim()).toBe(HOSTED_ADVERSARIAL_WORKSPACE_ROW)
|
||||
expect(fixture.workspaceRowName).toBe(HOSTED_ADVERSARIAL_WORKSPACE_ROW)
|
||||
})
|
||||
|
||||
it('accepts literal markers without created elements or execution', () => {
|
||||
expect(
|
||||
hostedWebViewAdversarialContentEvidence({
|
||||
text: `${HOSTED_ADVERSARIAL_FILENAME}\n${HOSTED_ADVERSARIAL_CONTENT}`,
|
||||
execution: {
|
||||
filenameExecuted: false,
|
||||
contentExecuted: false,
|
||||
injectedImageCount: 0
|
||||
}
|
||||
})
|
||||
).toEqual({
|
||||
filenameRenderedAsText: true,
|
||||
diffRenderedAsText: true,
|
||||
injectedImageCount: 0,
|
||||
scriptMarkersExecuted: false
|
||||
})
|
||||
})
|
||||
|
||||
it('aggregates literal markers across route documents', async () => {
|
||||
const values = [
|
||||
JSON.stringify({
|
||||
href: 'https://orca-mobile-web.invalid/source-control',
|
||||
bodyText: HOSTED_ADVERSARIAL_FILENAME,
|
||||
labels: [],
|
||||
placeholders: []
|
||||
}),
|
||||
JSON.stringify({
|
||||
filenameExecuted: false,
|
||||
contentExecuted: false,
|
||||
injectedImageCount: 0
|
||||
}),
|
||||
JSON.stringify({
|
||||
href: 'https://orca-mobile-web.invalid/session',
|
||||
bodyText: HOSTED_ADVERSARIAL_CONTENT,
|
||||
labels: [],
|
||||
placeholders: []
|
||||
}),
|
||||
JSON.stringify({
|
||||
filenameExecuted: false,
|
||||
contentExecuted: false,
|
||||
injectedImageCount: 0
|
||||
})
|
||||
]
|
||||
class FakeWebSocket {
|
||||
private message: ((data: Buffer) => void) | undefined
|
||||
|
||||
once(event: string, listener: () => void) {
|
||||
if (event === 'open') {
|
||||
queueMicrotask(listener)
|
||||
}
|
||||
}
|
||||
|
||||
on(event: string, listener: (data: Buffer) => void) {
|
||||
if (event === 'message') {
|
||||
this.message = listener
|
||||
}
|
||||
}
|
||||
|
||||
send(payload: string) {
|
||||
const id = JSON.parse(payload).id
|
||||
const value = values.shift()
|
||||
queueMicrotask(() => {
|
||||
this.message?.(
|
||||
Buffer.from(JSON.stringify({ id, result: { result: { value: value ?? '' } } }))
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
close() {}
|
||||
}
|
||||
|
||||
await expect(
|
||||
verifyHostedWebViewAdversarialContent({
|
||||
documents: [
|
||||
{ webSocketDebuggerUrl: 'ws://127.0.0.1/devtools/page/source' },
|
||||
{ webSocketDebuggerUrl: 'ws://127.0.0.1/devtools/page/session' }
|
||||
],
|
||||
WebSocketCtor: FakeWebSocket as never
|
||||
})
|
||||
).resolves.toMatchObject({
|
||||
filenameRenderedAsText: true,
|
||||
diffRenderedAsText: true,
|
||||
scriptMarkersExecuted: false
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects missing literal markers and executed content', () => {
|
||||
expect(() =>
|
||||
hostedWebViewAdversarialContentEvidence({
|
||||
text: HOSTED_ADVERSARIAL_FILENAME,
|
||||
execution: {
|
||||
filenameExecuted: false,
|
||||
contentExecuted: false,
|
||||
injectedImageCount: 0
|
||||
}
|
||||
})
|
||||
).toThrow('was not rendered')
|
||||
expect(() =>
|
||||
hostedWebViewAdversarialContentEvidence({
|
||||
text: `${HOSTED_ADVERSARIAL_FILENAME}\n${HOSTED_ADVERSARIAL_CONTENT}`,
|
||||
execution: {
|
||||
filenameExecuted: false,
|
||||
contentExecuted: true,
|
||||
injectedImageCount: 1
|
||||
}
|
||||
})
|
||||
).toThrow('content executed')
|
||||
})
|
||||
|
||||
it('rejects malformed execution observations without coercion', () => {
|
||||
expect(() =>
|
||||
hostedWebViewAdversarialContentObservations([
|
||||
{
|
||||
state: {
|
||||
bodyText: `${HOSTED_ADVERSARIAL_FILENAME}\n${HOSTED_ADVERSARIAL_CONTENT}`,
|
||||
labels: []
|
||||
},
|
||||
execution: {
|
||||
filenameExecuted: 'false',
|
||||
contentExecuted: false,
|
||||
injectedImageCount: 0
|
||||
}
|
||||
}
|
||||
])
|
||||
).toThrow('content executed')
|
||||
})
|
||||
|
||||
it('keeps the disposable corpus wired into the exact Android route gate', () => {
|
||||
expect(androidHarnessSource).toContain('createHostedAdversarialRepositoryFixture')
|
||||
expect(androidHarnessSource).toContain('captureHostedWebViewAdversarialObservation')
|
||||
expect(androidHarnessSource).toContain('hostedWebViewAdversarialContentObservations')
|
||||
expect(androidHarnessSource).toContain('removeHostedAdversarialRepositoryFixture')
|
||||
})
|
||||
|
||||
it('rejects conflicting Android security journeys before device access', async () => {
|
||||
await expect(
|
||||
execFileAsync(process.execPath, [
|
||||
androidHarnessPath,
|
||||
'--adversarial-content',
|
||||
'--security-only'
|
||||
])
|
||||
).rejects.toMatchObject({
|
||||
stderr: expect.stringContaining('mutually exclusive')
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps the same disposable corpus wired into the exact iOS route gate', () => {
|
||||
expect(iosHarnessSource).toContain('registerHostedIosAdversarialRepository')
|
||||
expect(iosHarnessSource).toContain('createHostedIosAdversarialContentInspector')
|
||||
expect(iosHarnessSource).toContain('removeHostedAdversarialRepositoryFixture')
|
||||
})
|
||||
})
|
||||
@@ -296,13 +296,16 @@ describe('hosted WebView CDP target selection', () => {
|
||||
{ webSocketDebuggerUrl: 'ws://127.0.0.1/devtools/page/current' },
|
||||
'Agent Session History',
|
||||
fakeCdpConstructor(socket),
|
||||
{ ignoreCase: true, occurrence: 1 }
|
||||
{ ignoreCase: true, occurrence: 1, reveal: true }
|
||||
)
|
||||
).resolves.toEqual({ x: 0.25, y: 0.125 })
|
||||
expect(socket.evaluations[0]?.params.expression).toContain('getBoundingClientRect')
|
||||
expect(socket.evaluations[0]?.params.expression).toContain("style.visibility !== 'hidden'")
|
||||
expect(socket.evaluations[0]?.params.expression).toContain('toLocaleLowerCase')
|
||||
expect(socket.evaluations[0]?.params.expression).toContain('matches[1]')
|
||||
expect(socket.evaluations[0]?.params.expression).toContain(
|
||||
"scrollIntoView({ block: 'nearest', inline: 'nearest' })"
|
||||
)
|
||||
})
|
||||
|
||||
it('rejects incomplete adversarial navigation evidence', async () => {
|
||||
|
||||
@@ -5,6 +5,7 @@ describe('hosted WebView simulator E2E options', () => {
|
||||
it('retains bounded defaults', () => {
|
||||
expect(parseHostedWebViewSimulatorE2eOptions([])).toEqual({
|
||||
accountsOnly: false,
|
||||
adversarialContent: false,
|
||||
clipboardImageOnly: false,
|
||||
device: 'iPhone 17 Pro',
|
||||
expectedBuild: undefined,
|
||||
@@ -80,6 +81,13 @@ describe('hosted WebView simulator E2E options', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('maps adversarial content onto the focused Source Control journey', () => {
|
||||
expect(parseHostedWebViewSimulatorE2eOptions(['--adversarial-content'])).toMatchObject({
|
||||
adversarialContent: true,
|
||||
sourceControlOnly: true
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects mutually exclusive focused journeys', () => {
|
||||
expect(() =>
|
||||
parseHostedWebViewSimulatorE2eOptions(['--accounts-only', '--source-control-only'])
|
||||
|
||||
@@ -11,11 +11,12 @@ describe('hosted WebView workspace activation', () => {
|
||||
expect(activate).toHaveBeenCalledOnce()
|
||||
expect(activate).toHaveBeenCalledWith(document, {
|
||||
kind: 'label',
|
||||
value: 'Open mobile-rearch'
|
||||
value: 'Open mobile-rearch',
|
||||
reveal: true
|
||||
})
|
||||
})
|
||||
|
||||
it('retains grouped-list text expansion for older row markup', async () => {
|
||||
it('reveals a grouped text row when accessible row labels are absent', async () => {
|
||||
const missing = new Error('Hosted WebView control was not found: mobile-rearch')
|
||||
const activate = vi
|
||||
.fn()
|
||||
@@ -33,7 +34,8 @@ describe('hosted WebView workspace activation', () => {
|
||||
kind: 'text',
|
||||
value: 'mobile-rearch',
|
||||
ignoreCase: true,
|
||||
occurrence: 1
|
||||
occurrence: 1,
|
||||
reveal: true
|
||||
}
|
||||
)
|
||||
expect(activate).toHaveBeenNthCalledWith(
|
||||
@@ -42,7 +44,8 @@ describe('hosted WebView workspace activation', () => {
|
||||
{
|
||||
kind: 'text',
|
||||
value: 'mobile-rearch',
|
||||
ignoreCase: true
|
||||
ignoreCase: true,
|
||||
reveal: true
|
||||
}
|
||||
)
|
||||
expect(activate).toHaveBeenNthCalledWith(
|
||||
@@ -52,7 +55,8 @@ describe('hosted WebView workspace activation', () => {
|
||||
kind: 'text',
|
||||
value: 'mobile-rearch',
|
||||
ignoreCase: true,
|
||||
occurrence: 1
|
||||
occurrence: 1,
|
||||
reveal: true
|
||||
}
|
||||
)
|
||||
})
|
||||
@@ -74,7 +78,8 @@ describe('hosted WebView workspace activation', () => {
|
||||
expect(resolveDocument).toHaveBeenCalledOnce()
|
||||
expect(activate).toHaveBeenLastCalledWith(replacement, {
|
||||
kind: 'label',
|
||||
value: 'Open mobile-rearch'
|
||||
value: 'Open mobile-rearch',
|
||||
reveal: true
|
||||
})
|
||||
})
|
||||
|
||||
@@ -95,7 +100,8 @@ describe('hosted WebView workspace activation', () => {
|
||||
expect(resolveDocument).toHaveBeenCalledOnce()
|
||||
expect(activate).toHaveBeenLastCalledWith(replacement, {
|
||||
kind: 'label',
|
||||
value: 'Open mobile-rearch'
|
||||
value: 'Open mobile-rearch',
|
||||
reveal: true
|
||||
})
|
||||
})
|
||||
|
||||
@@ -113,10 +119,10 @@ describe('hosted WebView workspace activation', () => {
|
||||
await activateHostedWorkspaceRow({}, 'mobile-rearch', activate, 1_000)
|
||||
|
||||
expect(activate).toHaveBeenCalledTimes(6)
|
||||
expect(activate).toHaveBeenLastCalledWith({}, expect.objectContaining({ occurrence: 1 }))
|
||||
expect(activate).toHaveBeenLastCalledWith({}, expect.objectContaining({ reveal: true }))
|
||||
})
|
||||
|
||||
it('accepts navigation after activating the only text row', async () => {
|
||||
it('accepts activation of the only matching text row', async () => {
|
||||
const missing = new Error('Hosted WebView control was not found: mobile-rearch')
|
||||
const stale = new Error('WebKit CDP connection closed')
|
||||
const activate = vi
|
||||
@@ -130,4 +136,27 @@ describe('hosted WebView workspace activation', () => {
|
||||
|
||||
expect(activate).toHaveBeenCalledTimes(4)
|
||||
})
|
||||
|
||||
it('accepts an SPA route transition after the activated row disappears', async () => {
|
||||
const missing = new Error('Hosted WebView control was not found: mobile-rearch')
|
||||
const activate = vi
|
||||
.fn()
|
||||
.mockRejectedValueOnce(missing)
|
||||
.mockRejectedValueOnce(missing)
|
||||
.mockResolvedValueOnce(undefined)
|
||||
.mockRejectedValueOnce(missing)
|
||||
const resolveDocument = vi.fn().mockResolvedValue({
|
||||
href: 'https://orca-mobile-web.invalid/h/host/session/workspace'
|
||||
})
|
||||
|
||||
await activateHostedWorkspaceRow(
|
||||
{ href: 'https://orca-mobile-web.invalid/h/host' },
|
||||
'mobile-rearch',
|
||||
activate,
|
||||
1_000,
|
||||
resolveDocument
|
||||
)
|
||||
|
||||
expect(resolveDocument).toHaveBeenCalledOnce()
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user