mirror of
https://github.com/stablyai/orca.git
synced 2026-09-26 16:02:43 +00:00
* fix(mobile): keep the streamed browser pane flipping on slow phones, and stop double taps
Frame pacing. The pane decodes each frame on a hidden layer and flips to it on
onLoad. While one frame decoded, every newer frame re-pointed that same hidden
layer, which cancels the in-flight load. On a phone that decodes a frame
slower than frames arrive (~10/s during page loads, menus, spinners), onLoad
never fired for any of them and the pane sat on an old frame until the page
went still. A decoding layer is now never re-pointed: only the newest frame is
held, and it takes the layer once the decode settles. A 1.5s watchdog frees a
layer whose decode never reports, and a frame the hidden layer already holds
(a blinking caret alternating two frames) flips at once, since an unchanged
source reloads nothing.
Double taps. When browser.mouseClick failed, the pane replayed the tap as
move/down/up. On a timeout the click is still queued on the host, so the
replay landed a second tap on whatever the first one opened. The replay now
runs only when the click definitely did not reach the host.
* test(mobile): re-record the corpus without the timed-out tap replay
The corpus certified the move/down/up replay after a transport-rejected
browser.mouseClick, which the commit before removes. Scoped like #22179:
baseline bumped by editing that one line, then --record.
788 files. Every changed line classified:
- `baseline`: 787 files (786 goldens + pilot-scenarios.json), nothing else.
- matrix-browser.pointer-click-browser.mouseclick-1.json: the
transport-rejection and transport-rejection-no-message partitions of
browser-pointer-click-fallback now send only browser.mouseClick#1. The
refused partitions still replay, unchanged.
* test(mobile): move the session closure pin past #22452's main-agent-status modules
#22452 changed only src/shared, so its CI never ran the page-closure suite; main
now measures 4220 modules (1034 local) against a pin of 4218. This branch adds
nothing to the closure: its own count matches main's.
* docs(mobile): say a re-pointed decoding layer loses its onLoad, as measured on Android
* refactor(mobile): give the streamed browser pane's double buffer one owner
The frame pacing, decode watchdog, layer flip and reset were spread over three
hooks and a helper module, wired back through the stream hook and the pane.
They now live in one plain pacer (browser-frame-pacer.ts) with one timer, and
the pane binds each layer's View/Image straight to it.
Behaviour fixed on the way, each with a failing test first:
- A slow last frame with nothing newer queued was abandoned by the watchdog and
never shown. The decode deadline now only applies when a newer frame waits.
- Any pane re-render re-pointed both layers at the newest frame behind the
pacer's back, so a blinking caret froze. The Image source prop is now only
the mount-time frame; every later source write is the pacer's.
- A frame that failed to decode left its layer marked as holding it, so an
identical frame flipped to an undecoded layer. Giving up on a decode now
clears the layer's source.
- A native onLoad for a source the layer has since moved off could flip early.
The flip now checks nativeEvent.source.uri, which Android and iOS Fabric both
report as the raw source string; RN Web's own load event has none, so the web
flips only through its decode probe.
The session closure pin drops by the two modules this removes.
* refactor(mobile): send the tap's mouseClick directly instead of through a flag
The delivery-unknown check was a mutable flag set inside the request callback.
The click now calls browser.mouseClick itself in a try/catch: a delivered click
returns, a delivery-unknown failure returns without replaying, and a refusal or
null result still replays as move/down/up. Same wire traffic; the corpus
certifies it unchanged.
* fix(mobile): never cut a streamed frame's decode short
The 1.5 s decode watchdog abandoned a slow decode whenever a newer frame was
queued and re-pointed its layer. On an Android emulator under load that is the
original freeze again: noise frames decode in 2-10 s, every abandon starts a
decode the next abandon cuts, Fresco reports the superseded loads (30 stale
onLoads in one run) and the pane showed 11 of 41 applied frames. Without it,
the same run flips every applied frame (16/16, no stale load), and a slow last
frame is shown in every cycle.
Nothing else needs it: with the layer never re-pointed mid-decode, native
answers every load with onLoad or onError, and the web probe's decode()
always settles. The pacer keeps one timer, for the interval.
* fix(mobile): track what each frame layer's Image holds, so no write goes unanswered
The pacer cleared a layer's source when it gave up on a decode (a reset
mid-decode, or a failed decode) while the native Image still held it. The
next identical frame was then written again, which is a native no-op on
Android (ReactImageView.setSource returns on equal sources) and iOS
(ImageShadowNode skips equal requests): no onLoad, no onError, and the pane
stayed frozen until the stream restarted. Returning from the background to
an unchanged page is enough to trigger it.
Each layer now records the source its Image holds and whether that source
has answered (loading, ready, failed). A layer is written only when it is
not loading and only with a different source, so every write gets exactly
one answer. A reset no longer abandons anything; the load under way still
answers for its layer. A frame the hidden layer already holds flips at once
if it decoded and is skipped if it failed.
The pane test's native model now treats a same-source write as a no-op and
answers each change once with the layer's current source; both new cases
(reset mid-decode then the same frame, failed decode then the same frame)
freeze on the previous head.
Also, per review: the pacer no longer touches busy or metadata. The stream
hook creates it and receives each frame as it goes on screen, so metadata
(and with it touch mapping) now follows the visible frame rather than one
still decoding.
* test(mobile): hold the pane test's AppState listener without a type assertion
* fix(mobile): never replay a tap the host answered
A fulfilled browser.mouseClick ran on the host, but a null result still
replayed it as move/down/up. The native bridge always answers { clicked },
while the external-Chromium provider returns agent-browser's `data` as is,
which can be null, so a right-click there was a double tap. Only a refusal
that is not delivery-unknown now replays.
* test(mobile): re-record the corpus for the answered-tap rule and rename its checkpoint
The commit before stops replaying a tap the host answered with a null
result. The seed's checkpoint was named clicked-by-fallback, which several
partitions no longer do, so it is renamed tap-settled in the same record.
Baseline bumped to 2c2b84ce9e by editing that line, then --record.
788 files, 841 changed lines each side. Every one classified:
- `baseline`: 787 goldens + pilot-scenarios.json.
- the checkpoint rename: pilot-scenarios.json (1), its id in
browser-pointer-click-fallback.json (1) and the eleven partition ids in
each of the four matrix-browser.pointer-click-*-1.json (44), plus
`scenarioSha256` in those five goldens.
- behaviour, one checkpoint: the result-null partition in
matrix-browser.pointer-click-browser.mouseclick-1.json now sends only
browser.mouseClick#1 (sender and payloads drop move/down/up). The refused,
method-not-found and result-absent partitions still replay, unchanged.
* fix(mobile): cover the frame layer remount, and trim the pacer's edges
Nothing tested the remount path: a fresh Image loads the source it mounted
with, so the pacer re-arms that load and puts back what the layer held.
The pane test's native model now mounts each host fresh (an Image loads its
mount source), and a new case remounts both Images mid-decode through a
zero-size layout: the visible layer keeps its frame and the stream goes on.
Deleting either the re-arm or the put-back fails it.
Also per review: drop the dead mountedUri guard in drain, return early from
attachImage when nothing is mounted, say that replace writes over a loading
layer, unexport the unused pacer types, cover the modifier bail in the tap's
comment, and move the frameUri state into the stream hook, which now returns
the source the layers mount with.
606 lines
26 KiB
JavaScript
606 lines
26 KiB
JavaScript
/**
|
|
* The browser pane, mounted in a real page and painted with a real frame.
|
|
*
|
|
* Every other C6 check reads one half: the shell suites drive `BridgeHostSubscriptions` with no
|
|
* page, the page suites drive the hooks with no shell, and the parity pin certifies the input
|
|
* path from a recording. This is the only place the whole frame path runs — the encoder's base64
|
|
* crossing the bridge, the page's decoder rebuilding the frame, the `.web.ts` layer writes
|
|
* painting it, and the decode-then-flip that native gets for free from `Image.onLoad`.
|
|
*
|
|
* C6 ruling 4: no route is added for it. `bundleMobileWebApp` already takes an `appDir`, so the
|
|
* check builds a one-route tree of its own, mounts the pane in it, and nothing under `mobile/app`
|
|
* moves or is registered.
|
|
*
|
|
* The frames are JPEGs the page encodes from a noise canvas, for the reason the budget uses noise:
|
|
* it is the image JPEG compresses least, so the over-cap case is over the cap for the reason a
|
|
* real page would be rather than because the check inflated one.
|
|
*/
|
|
import { mkdtemp, mkdir, rm, writeFile } from 'node:fs/promises'
|
|
import { join } from 'node:path'
|
|
import { fileURLToPath } from 'node:url'
|
|
import { afterAll, beforeAll, describe, expect, it } from 'vitest'
|
|
import { chromium } from 'playwright-core'
|
|
import { buildMobileWebAppBundle } from './build-mobile-web-app-bundle.mjs'
|
|
import { MOBILE_WEB_APP_ROUTE_ROOT } from './mobile-web-app-route-manifest.mjs'
|
|
import { mobileWebAppDependenciesPresent } from './mobile-web-app-bundle-dependencies.mjs'
|
|
import {
|
|
createBundleServer,
|
|
installShellDouble,
|
|
readBridgeFaultGrant,
|
|
readBridgeProtocolVersion,
|
|
readBridgeWindowCaps,
|
|
readBrowserFrameQuality,
|
|
readShellCsp
|
|
} from './mobile-web-app-render-harness.mjs'
|
|
|
|
const mobileDir = fileURLToPath(new URL('../../mobile', import.meta.url))
|
|
|
|
const HOST_ID = 'render-check-host'
|
|
const ROUTE = { pathname: '/h' }
|
|
const SHELL_HOST = {
|
|
id: HOST_ID,
|
|
name: 'Render Check Host',
|
|
endpoint: 'ws://render-check',
|
|
lastConnected: 1
|
|
}
|
|
const VIEWPORT = { width: 390, height: 844 }
|
|
/** The grant C6.1 named for the binary lane, and the one the negative case withholds. */
|
|
const BINARY_GRANT = 'screencastBinary'
|
|
const SCREENCAST = 'browser.screencast'
|
|
|
|
/** The frame's source viewport, which is what a tap is mapped back into. */
|
|
const SOURCE = { deviceWidth: 390, deviceHeight: 712 }
|
|
|
|
/**
|
|
* The frame the pane asks this viewport for, read off its own subscribe: `maxWidth` 390 by
|
|
* `maxHeight` 698 in web view mode.
|
|
*
|
|
* Not the phone's mobile-mode frame, which is 780x1424 and, as noise, encodes to 811,168 base64
|
|
* characters — 124% of the cap, which is the measurement the area budget exists for. That one is
|
|
* the over-cap case below rather than the frame that paints.
|
|
*/
|
|
const FRAME = { width: 390, height: 698 }
|
|
/**
|
|
* The page scale Chromium reports for a page with no `<meta name="viewport">`.
|
|
*
|
|
* Measured on Chromium 1217, 2026-09-20 against the C6.6 `dialog.html` fixture: under a mobile
|
|
* emulation the page lays out at Chromium's 980 px default and is scaled into the device width, so
|
|
* `deviceWidth` stays the emulated width and `pageScaleFactor` carries the ratio. The browser's
|
|
* input commands take page CSS pixels, so a tap sent in the frame's device space lands at that
|
|
* fraction of the aim — 41% on the phone, which is how the proof found it. The case above is the
|
|
* control a page with a viewport meta produces, where the scale is one and the mapping is exact.
|
|
*/
|
|
const NO_VIEWPORT_META_PAGE_SCALE = SOURCE.deviceWidth / 980
|
|
/** Noise at the largest layout the clamps admit, measured at 3,761,580 characters: 574% of the cap. */
|
|
const OVER_CAP_FRAME = { width: 2400, height: 2160 }
|
|
|
|
/**
|
|
* The scratch route: the pane and nothing else.
|
|
*
|
|
* The imports are relative and of a fixed depth, so this source carries no path from the machine
|
|
* that generated it. `screencastSupported` is the desktop's answer, which this route stands in for
|
|
* because the capability probe is the session screen's to make and C7's to prove.
|
|
*/
|
|
const ROUTE_SOURCE = `
|
|
import { useHostClient } from '../../../src/transport/client-context'
|
|
import { MobileBrowserPane } from '../../../src/browser/MobileBrowserPane'
|
|
|
|
const TAB = {
|
|
type: 'browser',
|
|
id: 'render-check-tab',
|
|
title: 'Render check',
|
|
browserWorkspaceId: 'render-check-workspace',
|
|
browserPageId: 'render-check-page',
|
|
url: 'https://example.test/',
|
|
loading: false,
|
|
canGoBack: false,
|
|
canGoForward: false,
|
|
isActive: true
|
|
}
|
|
|
|
export default function BrowserPaneRenderCheckRoute() {
|
|
const { client } = useHostClient('${HOST_ID}')
|
|
return (
|
|
<MobileBrowserPane
|
|
client={client}
|
|
worktreeId="render-check-worktree"
|
|
tab={TAB}
|
|
screencastSupported={true}
|
|
keyboardLift={0}
|
|
bottomInset={0}
|
|
onToast={() => {}}
|
|
/>
|
|
)
|
|
}
|
|
`
|
|
|
|
const bundles = mobileWebAppDependenciesPresent()
|
|
const describePane = bundles ? describe : describe.skip
|
|
|
|
let scratch = null
|
|
let server = null
|
|
let origin = null
|
|
let browser = null
|
|
let cspHeader = null
|
|
let bridgeVersion = null
|
|
let faultGrant = null
|
|
let windowCaps = null
|
|
|
|
beforeAll(async () => {
|
|
if (!bundles) {
|
|
return
|
|
}
|
|
cspHeader = await readShellCsp()
|
|
bridgeVersion = await readBridgeProtocolVersion()
|
|
faultGrant = await readBridgeFaultGrant()
|
|
windowCaps = await readBridgeWindowCaps()
|
|
// Inside mobile/ rather than the system temp dir: the route resolves `react-native` and the
|
|
// pane's own modules, and esbuild resolves a bare specifier from the importer upward.
|
|
await mkdir(join(mobileDir, '.tmp'), { recursive: true })
|
|
scratch = await mkdtemp(join(mobileDir, '.tmp', 'browser-pane-render-'))
|
|
const routeDir = join(scratch, MOBILE_WEB_APP_ROUTE_ROOT)
|
|
await mkdir(routeDir, { recursive: true })
|
|
await writeFile(join(routeDir, 'index.tsx'), ROUTE_SOURCE)
|
|
const { outDir } = await buildMobileWebAppBundle({
|
|
appDir: scratch,
|
|
outDir: join(scratch, 'bundle'),
|
|
pageRoutes: [{ pathname: ROUTE.pathname, grants: [BINARY_GRANT] }]
|
|
})
|
|
const served = await createBundleServer({ outDir, cspHeader })
|
|
server = served.server
|
|
origin = served.origin
|
|
const executablePath = process.env.ORCA_MOBILE_WEB_RENDER_BROWSER
|
|
browser = await chromium.launch({
|
|
headless: true,
|
|
...(executablePath ? { executablePath } : {})
|
|
})
|
|
}, 300_000)
|
|
|
|
afterAll(async () => {
|
|
await browser?.close()
|
|
server?.close()
|
|
if (scratch) {
|
|
// This run's directory only. `mobile/.tmp` is a shared ignored root and another suite may be
|
|
// holding one of its own.
|
|
await rm(scratch, { recursive: true, force: true })
|
|
}
|
|
})
|
|
|
|
/** One page with the shell double installed, its console and its requests watched. */
|
|
async function openPane({ grants }) {
|
|
const context = await browser.newContext({ viewport: VIEWPORT })
|
|
const page = await context.newPage()
|
|
const consoleErrors = []
|
|
const foreignRequests = []
|
|
page.on('console', (message) => {
|
|
if (message.type() === 'error') {
|
|
consoleErrors.push(message.text())
|
|
}
|
|
})
|
|
page.on('pageerror', (error) => consoleErrors.push(error.message))
|
|
page.on('request', (request) => {
|
|
if (!request.url().startsWith(origin) && !request.url().startsWith('data:')) {
|
|
foreignRequests.push(request.url())
|
|
}
|
|
})
|
|
await page.addInitScript(installShellDouble, {
|
|
version: bridgeVersion,
|
|
sessionId: 'render-check-session',
|
|
buildId: 'render-check-build-id',
|
|
route: ROUTE,
|
|
host: SHELL_HOST,
|
|
storage: {},
|
|
faultGrant,
|
|
grants,
|
|
pageRoutes: [ROUTE.pathname],
|
|
replies: { 'browser.mouseClick': { ok: true } },
|
|
streams: [SCREENCAST],
|
|
windowCaps
|
|
})
|
|
// The page reports a CSP violation as a document event; the header is the shell's own.
|
|
await page.addInitScript(() => {
|
|
globalThis.__orcaRenderCheckCsp = []
|
|
document.addEventListener('securitypolicyviolation', (event) => {
|
|
globalThis.__orcaRenderCheckCsp.push({
|
|
directive: event.violatedDirective,
|
|
blockedUri: event.blockedURI
|
|
})
|
|
})
|
|
})
|
|
await page.goto(`${origin}${ROUTE.pathname}`, { waitUntil: 'domcontentloaded' })
|
|
await page.waitForFunction(() => document.querySelector('#root')?.childElementCount > 0)
|
|
return {
|
|
page,
|
|
context,
|
|
consoleErrors,
|
|
foreignRequests,
|
|
csp: () => page.evaluate(() => globalThis.__orcaRenderCheckCsp)
|
|
}
|
|
}
|
|
|
|
/** A JPEG of deterministic noise, encoded in the page, returned as the base64 the bridge carries. */
|
|
async function encodeNoiseJpeg(page, { width, height, seed }) {
|
|
const quality = await readBrowserFrameQuality()
|
|
return page.evaluate(
|
|
({ width, height, seed, quality }) => {
|
|
const canvas = document.createElement('canvas')
|
|
canvas.width = width
|
|
canvas.height = height
|
|
const context = canvas.getContext('2d')
|
|
const image = context.createImageData(width, height)
|
|
let state = seed >>> 0
|
|
for (let index = 0; index < image.data.length; index += 4) {
|
|
state = (Math.imul(state, 1_664_525) + 1_013_904_223) >>> 0
|
|
image.data[index] = (state >>> 24) & 0xff
|
|
image.data[index + 1] = (state >>> 16) & 0xff
|
|
image.data[index + 2] = (state >>> 8) & 0xff
|
|
image.data[index + 3] = 255
|
|
}
|
|
context.putImageData(image, 0, 0)
|
|
return canvas.toDataURL('image/jpeg', quality).split(',')[1]
|
|
},
|
|
{ width, height, seed, quality }
|
|
)
|
|
}
|
|
|
|
/** Hand the page one frame, and say what the double did with it. */
|
|
function emitFrame(page, { b64, frameSeq, width, height, pageScaleFactor = 1 }) {
|
|
return page.evaluate(
|
|
({ b64, frameSeq, width, height, pageScaleFactor, source }) => {
|
|
const subscription = globalThis.__orcaRenderCheckSubscribes.at(-1)
|
|
if (!subscription) {
|
|
return 'no-subscription'
|
|
}
|
|
return globalThis.__orcaRenderCheckEmitBinary(subscription.id, {
|
|
b64,
|
|
format: 'jpeg',
|
|
frameSeq,
|
|
metadata: {
|
|
offsetTop: 0,
|
|
pageScaleFactor,
|
|
deviceWidth: source.deviceWidth,
|
|
deviceHeight: source.deviceHeight,
|
|
imageWidth: width,
|
|
imageHeight: height,
|
|
scrollOffsetX: 0,
|
|
scrollOffsetY: 0,
|
|
timestamp: 1_758_326_400.123456
|
|
}
|
|
})
|
|
},
|
|
{ b64, frameSeq, width, height, pageScaleFactor, source: SOURCE }
|
|
)
|
|
}
|
|
|
|
/**
|
|
* The frame on screen, read off the DOM the way RN Web paints it.
|
|
*
|
|
* Selected by the inline `background-image` rather than by a testID, because that write is the
|
|
* thing under test: `browser-frame-layer-paint.web.ts` puts the data URI on the element RN Web
|
|
* gives `<Image>` a background on, and a handle added for this check could be on an element the
|
|
* paint never touches.
|
|
*/
|
|
function readPaintedLayers(page) {
|
|
return page.evaluate(() => {
|
|
const painted = [...document.querySelectorAll('*')].filter((element) =>
|
|
element.style?.backgroundImage?.startsWith('url("data:image/jpeg')
|
|
)
|
|
return painted.map((element) => {
|
|
// The layer whose opacity the flip writes is the `<View>` above the `<Image>` surface.
|
|
let layer = element.parentElement
|
|
while (layer && layer.style.opacity === '') {
|
|
layer = layer.parentElement
|
|
}
|
|
return {
|
|
uri: element.style.backgroundImage.length,
|
|
digest: element.style.backgroundImage.slice(-24),
|
|
opacity: layer?.style.opacity ?? null
|
|
}
|
|
})
|
|
})
|
|
}
|
|
|
|
const waitForPaint = (page, count) =>
|
|
page.waitForFunction(
|
|
(expected) =>
|
|
[...document.querySelectorAll('*')].filter((element) =>
|
|
element.style?.backgroundImage?.startsWith('url("data:image/jpeg')
|
|
).length >= expected,
|
|
count,
|
|
{ timeout: 15_000 }
|
|
)
|
|
|
|
/** Which of the pane's two layers is on screen, by its position among them. */
|
|
async function visibleLayerIndex(page) {
|
|
const layers = await readPaintedLayers(page)
|
|
return layers.findIndex((layer) => layer.opacity === '1')
|
|
}
|
|
|
|
/**
|
|
* Waits for the page's own applied-frame signal: the double buffer's flip.
|
|
*
|
|
* The pacer writes the next frame's URI onto the hidden layer as soon as the frame lands and
|
|
* only flips the opacity once the decode resolves, so "some painted layer carries a new digest" is
|
|
* true before the frame is on screen. Measured here on 2026-09-20: the write landed at 80.7 ms
|
|
* after the emit and the flip at 85.7 ms, a 5 ms window in which a wait on the digest returns and
|
|
* the visible layer is still the previous frame. That is what made this file fail once in CI with
|
|
* the second frame's digest equal to the first's and no console errors.
|
|
*
|
|
* The flip is one opacity write, at the pacer's `flip`, and it is the behaviour under test
|
|
* rather than a proxy for it, so waiting on it can neither return early nor depend on how long a
|
|
* decode takes. Asserting the exact layer, not merely a change, keeps a pane with nothing visible
|
|
* from reading as a flip.
|
|
*/
|
|
async function waitForLayerFlip(page, staleIndex) {
|
|
await expect
|
|
.poll(() => visibleLayerIndex(page), { timeout: 15_000, interval: 25 })
|
|
.toBe(1 - staleIndex)
|
|
}
|
|
|
|
describePane('the browser pane in a page', () => {
|
|
/**
|
|
* Zero, which it was not until the Zod jitless flag moved into the bundler banner.
|
|
*
|
|
* Zod decided whether it could compile by constructing `new Function('')`, which the shell's
|
|
* `script-src 'self'` reports even though Zod catches the throw — once on load and again on
|
|
* first paint. This file filtered those out by `blockedURI === 'eval'` for one round, which
|
|
* would also have hidden a real one, so the filter is gone and the cause is fixed instead.
|
|
*/
|
|
it('files no CSP violation at all, through load and first paint', async () => {
|
|
const view = await openPane({ grants: [faultGrant, BINARY_GRANT] })
|
|
try {
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckSubscribes.length > 0)
|
|
const b64 = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 33 })
|
|
await emitFrame(view.page, { b64, frameSeq: 1, ...FRAME })
|
|
await waitForPaint(view.page, 1)
|
|
|
|
expect(await view.csp()).toEqual([])
|
|
expect(view.consoleErrors).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
|
|
it('subscribes over the binary lane and paints the frame it is handed', async () => {
|
|
const view = await openPane({ grants: [faultGrant, BINARY_GRANT] })
|
|
try {
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckSubscribes.length > 0)
|
|
const subscribes = await view.page.evaluate(() => globalThis.__orcaRenderCheckSubscribes)
|
|
expect(subscribes).toHaveLength(1)
|
|
expect(subscribes[0]).toMatchObject({ method: SCREENCAST, wantsBinary: true })
|
|
|
|
const b64 = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 1 })
|
|
expect(await emitFrame(view.page, { b64, frameSeq: 1, ...FRAME })).toBe('posted')
|
|
await waitForPaint(view.page, 1)
|
|
|
|
const layers = await readPaintedLayers(view.page)
|
|
// Both layers, because the first frame is painted on both, and one visible.
|
|
// This does not prove the decode-then-flip ran: with the probe removed entirely, the first
|
|
// frame still paints and a layer is still visible, because the visible layer starts at 0 and
|
|
// never needed to move. The flip is the next case's to prove.
|
|
expect(layers.length).toBeGreaterThan(0)
|
|
expect(layers.filter((layer) => layer.opacity === '1')).toHaveLength(1)
|
|
expect(view.consoleErrors).toEqual([])
|
|
expect(await view.csp()).toEqual([])
|
|
expect(view.foreignRequests).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
|
|
it('flips the double buffer on the second frame', async () => {
|
|
const view = await openPane({ grants: [faultGrant, BINARY_GRANT] })
|
|
try {
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckSubscribes.length > 0)
|
|
const first = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 7 })
|
|
await emitFrame(view.page, { b64: first, frameSeq: 1, ...FRAME })
|
|
await waitForPaint(view.page, 1)
|
|
const before = await readPaintedLayers(view.page)
|
|
const staleIndex = before.findIndex((layer) => layer.opacity === '1')
|
|
|
|
const second = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 99 })
|
|
expect(second).not.toBe(first)
|
|
await emitFrame(view.page, { b64: second, frameSeq: 2, ...FRAME })
|
|
await waitForLayerFlip(view.page, staleIndex)
|
|
|
|
const after = await readPaintedLayers(view.page)
|
|
const visible = after.filter((layer) => layer.opacity === '1')
|
|
expect(visible).toHaveLength(1)
|
|
expect(visible[0].digest).not.toBe(before.find((l) => l.opacity === '1')?.digest)
|
|
expect(view.consoleErrors).toEqual([])
|
|
expect(await view.csp()).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
|
|
it('drops an over-cap frame, keeps the stream, and paints the next one', async () => {
|
|
const view = await openPane({ grants: [faultGrant, BINARY_GRANT] })
|
|
try {
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckSubscribes.length > 0)
|
|
const small = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 3 })
|
|
await emitFrame(view.page, { b64: small, frameSeq: 1, ...FRAME })
|
|
await waitForPaint(view.page, 1)
|
|
const before = await readPaintedLayers(view.page)
|
|
const staleIndex = before.findIndex((layer) => layer.opacity === '1')
|
|
|
|
// Noise at the largest layout the clamps admit, which §1 measured at 574% of the cap.
|
|
const huge = await encodeNoiseJpeg(view.page, { ...OVER_CAP_FRAME, seed: 5 })
|
|
expect(huge.length).toBeGreaterThan(windowCaps.maxMessageBytes)
|
|
expect(await emitFrame(view.page, { b64: huge, frameSeq: 2, ...OVER_CAP_FRAME })).toBe(
|
|
'dropped'
|
|
)
|
|
|
|
// The stream is still open: the next frame arrives on the same subscription and paints.
|
|
const next = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 11 })
|
|
expect(await emitFrame(view.page, { b64: next, frameSeq: 3, ...FRAME })).toBe('posted')
|
|
await waitForLayerFlip(view.page, staleIndex)
|
|
|
|
expect(await view.page.evaluate(() => globalThis.__orcaRenderCheckDroppedFrames)).toEqual([2])
|
|
expect(await view.page.evaluate(() => globalThis.__orcaRenderCheckSubscribes.length)).toBe(1)
|
|
expect(await view.csp()).toEqual([])
|
|
expect(view.consoleErrors).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
|
|
it('asks for no binary lane at all when the shell withholds the grant', async () => {
|
|
const view = await openPane({ grants: [faultGrant] })
|
|
try {
|
|
await view.page.waitForFunction(() =>
|
|
document.body.innerText.includes('Update the Orca app to stream browser tabs here.')
|
|
)
|
|
expect(await view.page.evaluate(() => globalThis.__orcaRenderCheckSubscribes)).toEqual([])
|
|
expect(view.consoleErrors).toEqual([])
|
|
expect(await view.csp()).toEqual([])
|
|
expect(view.foreignRequests).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
|
|
it('streams past the unacked window because the page acks, and drops nothing', async () => {
|
|
// The two `canCarry` arms the size check hides. Thirty frames of about 200 KB is roughly 6 MB
|
|
// through a 4 MiB window, so a page that did not ack, or a shell double that ignored the acks
|
|
// it sent, starts dropping partway. Nothing here is over the message cap, so a drop can only
|
|
// come from the window.
|
|
const view = await openPane({ grants: [faultGrant, BINARY_GRANT] })
|
|
try {
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckSubscribes.length > 0)
|
|
const b64 = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 55 })
|
|
const cumulative = b64.length * 30
|
|
expect(cumulative).toBeGreaterThan(windowCaps.maxUnackedBytes)
|
|
|
|
const outcomes = []
|
|
for (let frameSeq = 1; frameSeq <= 30; frameSeq += 1) {
|
|
outcomes.push(await emitFrame(view.page, { b64, frameSeq, ...FRAME }))
|
|
}
|
|
|
|
expect(new Set(outcomes)).toEqual(new Set(['posted']))
|
|
expect(await view.page.evaluate(() => globalThis.__orcaRenderCheckDroppedFrames)).toEqual([])
|
|
// And the acks are real rather than the window merely being generous.
|
|
const acks = await view.page.evaluate(() => globalThis.__orcaRenderCheckAcks)
|
|
expect(acks.length).toBeGreaterThan(0)
|
|
expect(await view.csp()).toEqual([])
|
|
expect(view.consoleErrors).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
|
|
it('charges a JSON event to the window too, so one ack does not drive the ledger negative', async () => {
|
|
// The two emitters share one ledger and the `ack` arm subtracts whatever it finds on `unacked`.
|
|
// A JSON event that took a slot without paying for its bytes left `unackedBytes` below zero on
|
|
// the first ack, and the binary arm reads that floor: every later window check admitted frames
|
|
// the real `BridgeHostSubscriptions` would have refused.
|
|
const view = await openPane({ grants: [faultGrant, BINARY_GRANT] })
|
|
try {
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckSubscribes.length > 0)
|
|
const ledger = await view.page.evaluate((protocolVersion) => {
|
|
const id = globalThis.__orcaRenderCheckSubscribes[0].id
|
|
globalThis.__orcaRenderCheckEmitEvent(id, { type: 'probe', payload: 'x'.repeat(4096) })
|
|
const charged = globalThis.__orcaRenderCheckWindow(id)
|
|
globalThis.orcaBridge.postMessage(
|
|
JSON.stringify({ v: protocolVersion, type: 'ack', id, seq: charged.frames })
|
|
)
|
|
return { charged, settled: globalThis.__orcaRenderCheckWindow(id) }
|
|
}, bridgeVersion)
|
|
|
|
// Charged on the way out, which is the precondition: a zero here would make the line below
|
|
// read as balanced when nothing was ever counted.
|
|
expect(ledger.charged.frames).toBe(1)
|
|
expect(ledger.charged.unackedBytes).toBeGreaterThan(4096)
|
|
// And returned whole by the ack, rather than past it.
|
|
expect(ledger.settled).toEqual({ frames: 0, unackedBytes: 0 })
|
|
expect(view.consoleErrors).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
|
|
it('issues one mouseClick with the geometry the native pane would send', async () => {
|
|
const view = await openPane({ grants: [faultGrant, BINARY_GRANT] })
|
|
try {
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckSubscribes.length > 0)
|
|
const b64 = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 21 })
|
|
await emitFrame(view.page, { b64, frameSeq: 1, ...FRAME })
|
|
await waitForPaint(view.page, 1)
|
|
|
|
// The centre of the rendered frame, which maps back to the centre of the source viewport
|
|
// whatever the letterboxing did, so the expectation is exact rather than approximate.
|
|
const box = await view.page.evaluate(() => {
|
|
const painted = [...document.querySelectorAll('*')].find((element) =>
|
|
element.style?.backgroundImage?.startsWith('url("data:image/jpeg')
|
|
)
|
|
const rect = painted.getBoundingClientRect()
|
|
return { x: rect.x + rect.width / 2, y: rect.y + rect.height / 2 }
|
|
})
|
|
await view.page.mouse.click(box.x, box.y)
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckRequests.length > 0)
|
|
|
|
const requests = await view.page.evaluate(() => globalThis.__orcaRenderCheckRequests)
|
|
// One, not four: the double replies, so the pane never takes its move/down/up fallback. With
|
|
// the refusal the double gives every other method, this is four requests instead.
|
|
expect(requests).toHaveLength(1)
|
|
expect(requests[0].method).toBe('browser.mouseClick')
|
|
expect(requests[0].params).toMatchObject({
|
|
worktree: 'id:render-check-worktree',
|
|
page: 'render-check-page',
|
|
button: 'left',
|
|
modifiers: []
|
|
})
|
|
// The centre of the rendered frame is the centre of the source viewport, to within the one
|
|
// device pixel the rendered width's own fraction costs: the frame is 382.33 CSS px wide for
|
|
// 390 source px, so the centre is not on a pixel boundary in either space. Wider than that
|
|
// is a scale, an axis or a letterbox offset being wrong, which is what this is here for.
|
|
expect(Math.abs(requests[0].params.x - SOURCE.deviceWidth / 2)).toBeLessThanOrEqual(1)
|
|
expect(Math.abs(requests[0].params.y - SOURCE.deviceHeight / 2)).toBeLessThanOrEqual(1)
|
|
expect(await view.csp()).toEqual([])
|
|
expect(view.consoleErrors).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
it('maps a tap through the page scale the frame was painted at', async () => {
|
|
const view = await openPane({ grants: [faultGrant, BINARY_GRANT] })
|
|
try {
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckSubscribes.length > 0)
|
|
const b64 = await encodeNoiseJpeg(view.page, { ...FRAME, seed: 22 })
|
|
await emitFrame(view.page, {
|
|
b64,
|
|
frameSeq: 1,
|
|
...FRAME,
|
|
pageScaleFactor: NO_VIEWPORT_META_PAGE_SCALE
|
|
})
|
|
await waitForPaint(view.page, 1)
|
|
|
|
const box = await view.page.evaluate(() => {
|
|
const painted = [...document.querySelectorAll('*')].find((element) =>
|
|
element.style?.backgroundImage?.startsWith('url("data:image/jpeg')
|
|
)
|
|
const rect = painted.getBoundingClientRect()
|
|
return { x: rect.x + rect.width / 2, y: rect.y + rect.height / 2 }
|
|
})
|
|
await view.page.mouse.click(box.x, box.y)
|
|
await view.page.waitForFunction(() => globalThis.__orcaRenderCheckRequests.length > 0)
|
|
|
|
const requests = await view.page.evaluate(() => globalThis.__orcaRenderCheckRequests)
|
|
expect(requests[0].method).toBe('browser.mouseClick')
|
|
// The centre of the frame is the centre of the layout Chromium scaled into it: 980 CSS px
|
|
// wide, and 712 device px tall over the same scale. The tolerance is three CSS px because
|
|
// one device px is 2.5 of them here, and the rendered width's own fraction costs one.
|
|
expect(Math.abs(requests[0].params.x - 980 / 2)).toBeLessThanOrEqual(3)
|
|
expect(
|
|
Math.abs(requests[0].params.y - SOURCE.deviceHeight / 2 / NO_VIEWPORT_META_PAGE_SCALE)
|
|
).toBeLessThanOrEqual(3)
|
|
// Unmapped, this is what the device proof recorded: the frame's own device space, on BODY.
|
|
expect(requests[0].params.x).not.toBe(Math.round(SOURCE.deviceWidth / 2))
|
|
expect(await view.csp()).toEqual([])
|
|
expect(view.consoleErrors).toEqual([])
|
|
} finally {
|
|
await view.context.close()
|
|
}
|
|
}, 120_000)
|
|
})
|