Files
orca/mobile/src/session/MobileNativeChatOverlay.test.ts
T
Brennan Benson 1617ff32ef feat(mobile): show chat visuals inline in the phone's native chat (#26071)
* feat(native-chat): visual directive grammar and host read for a chat's visuals folder

A shared grammar for the ::orca-visual{file="..." title="..."} reply line,
the per-chat visuals folder location on the owning host, and the
agentSession.readVisual runtime method that reads one visual with lexical and
canonical containment, a 512 KiB bounded read and UTF-8 refusal.

* feat(native-chat): shared frame document for chat visuals

One string builder every client wraps a visual's HTML with: the policy
(CDN assets only, no fetch, frames, workers, forms or base rewrites),
the theme variables, and a prelude that reports height, routes links to
the parent and refuses navigation. Also the validated frame-to-parent
message reader and the live theme message.

* feat(mobile): render native-chat visuals inline in the phone's chat

A finished assistant reply's ::orca-visual line now shows the visual
inline, read from the chat's owning host through agentSession.readVisual.
The visual runs in an opaque sandboxed child of a trusted host document
inside the WebView; the app accepts only a token-checked height and an
http(s) link opened under user activation. Navigation away from the
visual's own document is refused, a dead web process reloads once, and
the visual opens full screen. The hybrid shell's page renders it sealed.

* feat(native-chat): render chat visuals inline and in the right sidebar

Native-chat assistant replies render a ::orca-visual{...} line as the chat's
HTML visual in an opaque, scripts-only sandboxed frame: CSP first, the host
frame navigation guard registered before content runs, live theme without a
reload, fitted height, links opened in the viewer's browser only from a real
gesture, lazy mount, and one muted line when the visual cannot be shown.
Open in sidebar shows the same frame in the right sidebar, widened while it
is open and restored after.

* fix(mobile): chat visuals use the shared shell; only the host page may message the app

Review round 1:
- use PR 1's shared visual shell and height governor instead of a second
  builder; the app decides heights and pushes them to the host page
- react-native-webview patch: the message channel accepts only string
  messages from the main frame on iOS and Android, so a visual in its
  sandboxed child cannot reach it (or crash Android with a non-string)
- structured replies grow in place: while a turn works, a row holds back a
  directive still being typed at its tail; finished lines mount
- links need child focus + activation, one per activation window
- a refused read takes the visual down and drops its cached bytes
- in-page anchors load; text spelling a placeholder renders no visuals

* refactor(native-chat): move the visual height governor to src/shared

The phone bundles only src/shared, so the governor the desktop frame uses
moves there unchanged and the mobile frame shares it.

* fix(native-chat): point the desktop and mobile frames at the moved governor

* fix(mobile): review round 2 for chat visuals

- host page relays only messages on the visual's own channel, as its own
  copy, so a visual cannot push oversized fields through it; relay rate
  halved; a link is validated before it uses up the link window
- the frame denies camera, microphone, geolocation, clipboard and display
  capture; iOS media capture requests are denied
- react-native-webview patch: the iOS history-shim handler also accepts
  only main-frame string messages
- only the newest assistant row of a working turn holds back an unfinished
  directive; earlier finished rows show their visuals
- an error reply or older host is not a verdict: it keeps a visual on
  screen and its cache, and retries; only a host refusal takes it down

* fix(mobile): a drag that starts on an inline visual scrolls the chat

iOS gives a scrollable frame its own scroll view, which took the drag;
the inline frame is sized to its content, so it no longer scrolls.
Full screen still does.

* fix(mobile): review round 3 for chat visuals

- hold back only the last block of the newest assistant row, and not
  while a question or approval is open, so a visual followed by a tool
  call or a pending question shows at once
- an older host (method_not_found) reads as unavailable without retries
- drop a stray @pnpm/exe lockfile block; only the patch hash changes

* test(mobile): list the visual frame's web sibling; mock it in the prompt-controller harness

* fix(mobile): an older desktop's mobile allowlist refusal reads as unavailable at once

* fix(native-chat): visual CI fixes, shared height governor, live-turn streaming hold

Registers agentSession.readVisual from the methods index so the structured
method file stays under its line budget, replaces reflective reads with checked
narrowing, moves the pure height governor to src/shared for mobile, and holds a
half-written directive tail while the turn works (structured text rows carry no
running state).

* fix(native-chat): harden the visual read and link opening

Re-checks after the open that the chat's visuals folder is still the real
directory at Orca's path, reports unexpected filesystem faults by code without
host paths, and lets one click in a visual open at most one page.

* fix(native-chat): keep visual lines out of plain-text reply surfaces; review fixes

One shared helper drops visual lines (outside fenced code) from reply text where
it becomes plain text: the structured status summary that feeds the sidebar row,
dashboard, notifications, phone rows and handoffs, and AI Vault reply previews.
Review fixes: height also counts a pinned body's overflow, only the live
frontier row holds a half-written visual line, the runaway-height stop needs the
same step repeated, and any host refusal evicts the cached revision.

* fix(native-chat): resolve the visuals folder without the removed journal-paths helper

Main removed the per-chat journal paths and the journal database's state
directory; the visuals folder keeps the same sha256 layout on its own and the
read method uses the profile state directory the chat host is opened in.

* fix(mobile): keep visual lines out of the worktree row and copied message text

A reply's ::orca-visual line renders only in the transcript. The
worktree list's agent row and the message actions sheet's copy text now
drop it with the shared helper; a reply that is only a visual falls back
to the prompt, as an empty one does.

* fix(native-chat): review round 2 fixes; copy a reply without its visual lines

Reply previews in Agent Session History drop visual lines per text part before
lines are folded; the frame adds a body's overflow only when the body really
overflows; fence tracking follows CommonMark closers and openers; the copy
button copies a reply without visual lines; a coded read fault keeps its cause.

* fix(native-chat): update the frame's theme ref after render; read the visuals folder pair at once

* test(native-chat): declare agentSession.readVisual on the cross-version agent-session surface

* fix(native-chat): copying a reply keeps its code blocks and indentation

Removing visual lines now closes only the gap each removal leaves, instead of
collapsing blank lines across the whole reply and trimming its indentation; the
visuals folder is checked parent first again so a broken path answers the same
way every time.
2026-10-07 14:21:05 -07:00

221 lines
8.0 KiB
TypeScript

import { createElement } from 'react'
import { act, create, type ReactTestRenderer } from 'react-test-renderer'
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { NativeChatMessage } from '../../../src/shared/native-chat-types'
import { MobileNativeChatOverlay } from './MobileNativeChatOverlay'
import type { MobileNativeChatController } from './use-mobile-native-chat-controller'
vi.mock('react-native', () => ({
StyleSheet: { create: (styles: unknown) => styles, absoluteFillObject: {} },
View: 'View'
}))
vi.mock('./MobileNativeChatView', () => ({ MobileNativeChatView: 'ChatView' }))
vi.mock('./MobileNativeChatQueuedMessages', () => ({ MobileNativeChatQueuedMessages: 'Queued' }))
vi.mock('./MobileNativeChatVisual', () => ({ useMobileNativeChatVisualRenderer: () => null }))
function assistantTurn(id: string, text: string): NativeChatMessage {
return { id, role: 'assistant', blocks: [{ type: 'text', text }], timestamp: 0, source: 'hook' }
}
/** One render of the route: chat visible or not, the transcript it currently
* holds, and the agent-status stream behind it. */
type Tick = {
show?: boolean
messages?: NativeChatMessage[]
streamingText?: string
streamLive?: boolean
identity?: string
/** The host says a person's Stop is ending the turn. */
stopping?: boolean
}
function overlayElement(tick: Tick): ReturnType<typeof createElement> {
// oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: the overlay reads only these controller members; the rest of the controller is unreachable from it.
const controller = {
showNativeChat: tick.show ?? true,
nativeChatSession: { messages: tick.messages ?? [], status: 'ready' },
nativeChatAgent: 'claude',
nativeChatAgentWorking: tick.streamLive ?? false,
nativeChatTurnIndicator: {
thinking: false,
activityText: null,
stopping: tick.stopping ?? false
},
nativeChatStreamingText: tick.streamingText,
nativeChatStreamLive: tick.streamLive ?? false,
nativeChatStreamScopeKey: tick.identity ?? 'tab-a',
chatPending: [],
chatImagePreviewsByMessageId: {},
chatComposerText: '',
setChatComposerText: vi.fn(),
nativeChatQueued: {
cards: [],
send: vi.fn(),
delete: vi.fn(),
edit: vi.fn(),
pause: null,
resume: vi.fn(),
sessionKey: 'session-a'
}
} as unknown as MobileNativeChatController
return createElement(MobileNativeChatOverlay, {
controller,
images: {} as never,
onMicPress: vi.fn(),
micActive: false,
dictationMode: 'toggle',
onMicPressIn: vi.fn(),
onMicPressOut: vi.fn(),
inputLockReason: null,
sendErrorMessage: null,
onClearSendError: vi.fn(),
sendSurfaceId: tick.identity ?? 'tab-a',
getSendCompletionGeneration: () => 0,
keyboardInset: 0
})
}
describe('MobileNativeChatOverlay streaming gate', () => {
let renderer: ReactTestRenderer | null = null
afterEach(() => {
act(() => renderer?.unmount())
renderer = null
})
async function render(tick: Tick): Promise<void> {
await act(async () => {
renderer = create(overlayElement(tick))
})
}
async function update(tick: Tick): Promise<void> {
await act(async () => {
renderer?.update(overlayElement(tick))
})
}
/** The bubble text handed to the chat list, or `'hidden'` when chat is off. */
function streaming(): string | null | 'hidden' {
const views = renderer!.root.findAll((node) => node.type === 'ChatView')
return views.length === 0 ? 'hidden' : (views[0].props.streaming as string | null)
}
it('keeps streaming a reply that repeats the previous turn as a prefix', async () => {
const prior = [assistantTurn('a1', 'The tests pass.')]
await render({ messages: prior })
expect(streaming()).toBeNull()
await update({ messages: prior, streamingText: 'The tests', streamLive: true })
expect(streaming()).toBe('The tests')
})
it('drops the streaming bubble once the reply lands as its own turn', async () => {
const prior = [assistantTurn('a1', 'Done.')]
await render({ messages: prior })
await update({ messages: prior, streamingText: 'Done.', streamLive: true })
expect(streaming()).toBe('Done.')
await update({
messages: [...prior, assistantTurn('a2', 'Done.')],
streamingText: 'Done.',
streamLive: true
})
expect(streaming()).toBeNull()
})
it('keeps the bubble across a peek at the terminal view', async () => {
// Toggling to the terminal unmounts the chat list and unsubscribes its
// transcript. The gate lives above that boundary, so the baseline survives
// and the repeated-prefix reply keeps streaming on the way back.
const prior = [assistantTurn('a1', 'Done.')]
await render({ messages: prior })
await update({ messages: prior, streamingText: 'Done.', streamLive: true })
expect(streaming()).toBe('Done.')
await update({ show: false, messages: [], streamLive: true })
expect(streaming()).toBe('hidden')
// Back on chat the session withholds its transcript until a fresh read
// settles, so the throttled stream text returns a round trip ahead of it.
await update({ messages: [], streamLive: true })
await update({ messages: [], streamingText: 'Done.', streamLive: true })
await update({ messages: prior, streamingText: 'Done.', streamLive: true })
expect(streaming()).toBe('Done.')
})
it('keeps the bubble across a peek at the terminal taken between turns', async () => {
// Same toggle, but taken while idle: the transcript empties before the next
// turn starts, so the gate has to reject that empty tail as a baseline.
const prior = [assistantTurn('a1', 'Done.')]
await render({ messages: prior })
await update({ show: false, messages: [] })
await update({ show: false, messages: [], streamLive: true })
await update({ messages: [], streamLive: true })
await update({ messages: [], streamingText: 'Done.', streamLive: true })
await update({ messages: prior, streamingText: 'Done.', streamLive: true })
expect(streaming()).toBe('Done.')
})
it('hides a repeated part whose own turn landed during a mid-turn gap', async () => {
// Between parts the status frame carries no assistant text (a tool call), so
// the stream goes textless while the turn is still live and the part that
// just finished lands in the transcript. Re-anchoring on that tick would
// adopt it as history and render it a second time.
const prior = [assistantTurn('a1', 'Done.')]
await render({ messages: prior })
await update({ messages: prior, streamingText: 'Done.', streamLive: true })
expect(streaming()).toBe('Done.')
const landed = [...prior, assistantTurn('a2', 'Done.')]
await update({ messages: landed, streamLive: true })
await update({ messages: landed, streamingText: 'Done.', streamLive: true })
expect(streaming()).toBeNull()
})
it("does not carry one chat's baseline into another stream identity", async () => {
const prior = [assistantTurn('a1', 'Shared answer text')]
await render({ messages: prior, identity: 'tab-a' })
await update({
messages: prior,
streamingText: 'Shared answer',
streamLive: true,
identity: 'tab-b'
})
expect(streaming()).toBeNull()
})
})
describe("MobileNativeChatOverlay while a person's Stop ends the turn", () => {
let renderer: ReactTestRenderer | null = null
afterEach(() => {
act(() => renderer?.unmount())
renderer = null
})
async function steerHeld(tick: Tick): Promise<unknown> {
await act(async () => {
renderer = create(overlayElement(tick))
})
const view = renderer!.root.find((node) => node.type === 'ChatView')
return view.props.queuedSlot?.cards?.props?.steerHeld
}
it("holds the queued cards' Steer while the agent works and the host says Stopping", async () => {
expect(await steerHeld({ streamLive: true, stopping: true })).toBe(true)
})
it('leaves Steer to the cards otherwise', async () => {
expect(await steerHeld({ streamLive: true, stopping: false })).toBe(false)
})
})