mirror of
https://github.com/stablyai/orca.git
synced 2026-10-09 16:02:46 +00:00
* feat(native-chat): visual directive grammar and host read for a chat's visuals folder
A shared grammar for the ::orca-visual{file="..." title="..."} reply line,
the per-chat visuals folder location on the owning host, and the
agentSession.readVisual runtime method that reads one visual with lexical and
canonical containment, a 512 KiB bounded read and UTF-8 refusal.
* feat(native-chat): shared frame document for chat visuals
One string builder every client wraps a visual's HTML with: the policy
(CDN assets only, no fetch, frames, workers, forms or base rewrites),
the theme variables, and a prelude that reports height, routes links to
the parent and refuses navigation. Also the validated frame-to-parent
message reader and the live theme message.
* feat(mobile): render native-chat visuals inline in the phone's chat
A finished assistant reply's ::orca-visual line now shows the visual
inline, read from the chat's owning host through agentSession.readVisual.
The visual runs in an opaque sandboxed child of a trusted host document
inside the WebView; the app accepts only a token-checked height and an
http(s) link opened under user activation. Navigation away from the
visual's own document is refused, a dead web process reloads once, and
the visual opens full screen. The hybrid shell's page renders it sealed.
* feat(native-chat): render chat visuals inline and in the right sidebar
Native-chat assistant replies render a ::orca-visual{...} line as the chat's
HTML visual in an opaque, scripts-only sandboxed frame: CSP first, the host
frame navigation guard registered before content runs, live theme without a
reload, fitted height, links opened in the viewer's browser only from a real
gesture, lazy mount, and one muted line when the visual cannot be shown.
Open in sidebar shows the same frame in the right sidebar, widened while it
is open and restored after.
* fix(mobile): chat visuals use the shared shell; only the host page may message the app
Review round 1:
- use PR 1's shared visual shell and height governor instead of a second
builder; the app decides heights and pushes them to the host page
- react-native-webview patch: the message channel accepts only string
messages from the main frame on iOS and Android, so a visual in its
sandboxed child cannot reach it (or crash Android with a non-string)
- structured replies grow in place: while a turn works, a row holds back a
directive still being typed at its tail; finished lines mount
- links need child focus + activation, one per activation window
- a refused read takes the visual down and drops its cached bytes
- in-page anchors load; text spelling a placeholder renders no visuals
* refactor(native-chat): move the visual height governor to src/shared
The phone bundles only src/shared, so the governor the desktop frame uses
moves there unchanged and the mobile frame shares it.
* fix(native-chat): point the desktop and mobile frames at the moved governor
* fix(mobile): review round 2 for chat visuals
- host page relays only messages on the visual's own channel, as its own
copy, so a visual cannot push oversized fields through it; relay rate
halved; a link is validated before it uses up the link window
- the frame denies camera, microphone, geolocation, clipboard and display
capture; iOS media capture requests are denied
- react-native-webview patch: the iOS history-shim handler also accepts
only main-frame string messages
- only the newest assistant row of a working turn holds back an unfinished
directive; earlier finished rows show their visuals
- an error reply or older host is not a verdict: it keeps a visual on
screen and its cache, and retries; only a host refusal takes it down
* fix(mobile): a drag that starts on an inline visual scrolls the chat
iOS gives a scrollable frame its own scroll view, which took the drag;
the inline frame is sized to its content, so it no longer scrolls.
Full screen still does.
* fix(mobile): review round 3 for chat visuals
- hold back only the last block of the newest assistant row, and not
while a question or approval is open, so a visual followed by a tool
call or a pending question shows at once
- an older host (method_not_found) reads as unavailable without retries
- drop a stray @pnpm/exe lockfile block; only the patch hash changes
* test(mobile): list the visual frame's web sibling; mock it in the prompt-controller harness
* fix(mobile): an older desktop's mobile allowlist refusal reads as unavailable at once
* fix(native-chat): visual CI fixes, shared height governor, live-turn streaming hold
Registers agentSession.readVisual from the methods index so the structured
method file stays under its line budget, replaces reflective reads with checked
narrowing, moves the pure height governor to src/shared for mobile, and holds a
half-written directive tail while the turn works (structured text rows carry no
running state).
* fix(native-chat): harden the visual read and link opening
Re-checks after the open that the chat's visuals folder is still the real
directory at Orca's path, reports unexpected filesystem faults by code without
host paths, and lets one click in a visual open at most one page.
* fix(native-chat): keep visual lines out of plain-text reply surfaces; review fixes
One shared helper drops visual lines (outside fenced code) from reply text where
it becomes plain text: the structured status summary that feeds the sidebar row,
dashboard, notifications, phone rows and handoffs, and AI Vault reply previews.
Review fixes: height also counts a pinned body's overflow, only the live
frontier row holds a half-written visual line, the runaway-height stop needs the
same step repeated, and any host refusal evicts the cached revision.
* fix(native-chat): resolve the visuals folder without the removed journal-paths helper
Main removed the per-chat journal paths and the journal database's state
directory; the visuals folder keeps the same sha256 layout on its own and the
read method uses the profile state directory the chat host is opened in.
* fix(mobile): keep visual lines out of the worktree row and copied message text
A reply's ::orca-visual line renders only in the transcript. The
worktree list's agent row and the message actions sheet's copy text now
drop it with the shared helper; a reply that is only a visual falls back
to the prompt, as an empty one does.
* fix(native-chat): review round 2 fixes; copy a reply without its visual lines
Reply previews in Agent Session History drop visual lines per text part before
lines are folded; the frame adds a body's overflow only when the body really
overflows; fence tracking follows CommonMark closers and openers; the copy
button copies a reply without visual lines; a coded read fault keeps its cause.
* fix(native-chat): update the frame's theme ref after render; read the visuals folder pair at once
* test(native-chat): declare agentSession.readVisual on the cross-version agent-session surface
* fix(native-chat): copying a reply keeps its code blocks and indentation
Removing visual lines now closes only the gap each removal leaves, instead of
collapsing blank lines across the whole reply and trimming its indentation; the
visuals folder is checked parent first again so a broken path answers the same
way every time.
190 lines
5.8 KiB
TypeScript
190 lines
5.8 KiB
TypeScript
import { memo, useCallback, useMemo, useState } from 'react'
|
|
import { ActivityIndicator, Modal, Pressable, StyleSheet, Text, View } from 'react-native'
|
|
import { useSafeAreaInsets } from 'react-native-safe-area-context'
|
|
import { Maximize2, X } from 'lucide-react-native'
|
|
import type { NativeChatVisualDirective } from '../../../src/shared/native-chat-visual-directive'
|
|
import { colors, radii, spacing, typography } from '../theme/mobile-theme'
|
|
import type { MobileNativeChatVisualSource } from './mobile-native-chat-visual-read'
|
|
import { MOBILE_NATIVE_CHAT_VISUAL_INITIAL_HEIGHT } from './mobile-native-chat-visual-host-document'
|
|
import { MobileNativeChatVisualFrame } from './MobileNativeChatVisualFrame'
|
|
import { useMobileNativeChatVisual } from './use-mobile-native-chat-visual'
|
|
import type { MobileNativeChatVisualRender } from './mobile-native-chat-visual-context'
|
|
|
|
const DEFAULT_TITLE = 'Visualization'
|
|
|
|
/** The quiet reserved space a visual takes while it loads. */
|
|
function MobileNativeChatVisualPlaceholder() {
|
|
return (
|
|
<View style={[styles.frame, styles.placeholder]}>
|
|
<ActivityIndicator size="small" color={colors.textMuted} />
|
|
</View>
|
|
)
|
|
}
|
|
|
|
/** The transcript's visual renderer for a structured chat's source; null without one. */
|
|
export function useMobileNativeChatVisualRenderer(
|
|
source: MobileNativeChatVisualSource | null
|
|
): MobileNativeChatVisualRender | null {
|
|
return useMemo(
|
|
() =>
|
|
source
|
|
? (directive) => <MobileNativeChatVisual directive={directive} source={source} />
|
|
: null,
|
|
[source]
|
|
)
|
|
}
|
|
|
|
/** A `::orca-visual{...}` line in a finished assistant reply, rendered from the chat's host. */
|
|
export const MobileNativeChatVisual = memo(function MobileNativeChatVisual({
|
|
directive,
|
|
source
|
|
}: {
|
|
directive: NativeChatVisualDirective
|
|
source: MobileNativeChatVisualSource
|
|
}) {
|
|
const { state, retry } = useMobileNativeChatVisual(source, directive.file)
|
|
// A frame that navigated away or kept crashing; cleared by the reader's retry.
|
|
const [frameFailed, setFrameFailed] = useState(false)
|
|
const [fullscreen, setFullscreen] = useState(false)
|
|
const title = directive.title ?? DEFAULT_TITLE
|
|
const onFrameFailed = useCallback(() => {
|
|
setFrameFailed(true)
|
|
setFullscreen(false)
|
|
}, [])
|
|
const onRetry = useCallback(() => {
|
|
setFrameFailed(false)
|
|
retry()
|
|
}, [retry])
|
|
const closeFullscreen = useCallback(() => setFullscreen(false), [])
|
|
|
|
if (state.kind === 'loading') {
|
|
return <MobileNativeChatVisualPlaceholder />
|
|
}
|
|
if (state.kind === 'unavailable' || frameFailed) {
|
|
return (
|
|
<Pressable
|
|
onPress={onRetry}
|
|
accessibilityRole="button"
|
|
accessibilityLabel="Visualization unavailable"
|
|
accessibilityHint="Tries to load it again"
|
|
style={styles.unavailable}
|
|
>
|
|
<Text style={styles.unavailableText}>Visualization unavailable</Text>
|
|
</Pressable>
|
|
)
|
|
}
|
|
return (
|
|
<View style={styles.frame}>
|
|
<MobileNativeChatVisualFrame
|
|
html={state.html}
|
|
title={title}
|
|
mode="inline"
|
|
onFailed={onFrameFailed}
|
|
/>
|
|
<Pressable
|
|
onPress={() => setFullscreen(true)}
|
|
accessibilityRole="button"
|
|
accessibilityLabel={`Open ${title} full screen`}
|
|
hitSlop={8}
|
|
style={styles.openButton}
|
|
>
|
|
<Maximize2 size={14} color={colors.textSecondary} strokeWidth={2.2} />
|
|
</Pressable>
|
|
{fullscreen ? (
|
|
<MobileNativeChatVisualFullscreen
|
|
html={state.html}
|
|
title={title}
|
|
onClose={closeFullscreen}
|
|
onFailed={onFrameFailed}
|
|
/>
|
|
) : null}
|
|
</View>
|
|
)
|
|
})
|
|
|
|
function MobileNativeChatVisualFullscreen({
|
|
html,
|
|
title,
|
|
onClose,
|
|
onFailed
|
|
}: {
|
|
html: string
|
|
title: string
|
|
onClose: () => void
|
|
onFailed: () => void
|
|
}) {
|
|
const insets = useSafeAreaInsets()
|
|
return (
|
|
<Modal visible animationType="slide" presentationStyle="fullScreen" onRequestClose={onClose}>
|
|
<View style={[styles.sheet, { paddingTop: insets.top, paddingBottom: insets.bottom }]}>
|
|
<View style={styles.sheetHeader}>
|
|
<Text style={styles.sheetTitle} numberOfLines={1}>
|
|
{title}
|
|
</Text>
|
|
<Pressable
|
|
onPress={onClose}
|
|
accessibilityRole="button"
|
|
accessibilityLabel="Close"
|
|
hitSlop={8}
|
|
style={styles.closeButton}
|
|
>
|
|
<X size={18} color={colors.textSecondary} strokeWidth={2.2} />
|
|
</Pressable>
|
|
</View>
|
|
<MobileNativeChatVisualFrame
|
|
html={html}
|
|
title={title}
|
|
mode="fullscreen"
|
|
onFailed={onFailed}
|
|
/>
|
|
</View>
|
|
</Modal>
|
|
)
|
|
}
|
|
|
|
const styles = StyleSheet.create({
|
|
frame: {
|
|
borderWidth: StyleSheet.hairlineWidth,
|
|
borderColor: colors.borderSubtle,
|
|
borderRadius: radii.row,
|
|
marginBottom: spacing.sm,
|
|
overflow: 'hidden'
|
|
},
|
|
placeholder: {
|
|
height: MOBILE_NATIVE_CHAT_VISUAL_INITIAL_HEIGHT,
|
|
alignItems: 'center',
|
|
justifyContent: 'center'
|
|
},
|
|
unavailable: { marginBottom: spacing.sm, paddingVertical: spacing.xs },
|
|
unavailableText: { color: colors.textMuted, fontSize: typography.metaSize },
|
|
openButton: {
|
|
position: 'absolute',
|
|
top: spacing.xs,
|
|
right: spacing.xs,
|
|
width: 28,
|
|
height: 28,
|
|
borderRadius: 14,
|
|
alignItems: 'center',
|
|
justifyContent: 'center',
|
|
backgroundColor: colors.bgPanel,
|
|
borderWidth: StyleSheet.hairlineWidth,
|
|
borderColor: colors.borderSubtle
|
|
},
|
|
sheet: { flex: 1, backgroundColor: colors.bgBase },
|
|
sheetHeader: {
|
|
flexDirection: 'row',
|
|
alignItems: 'center',
|
|
paddingHorizontal: spacing.md,
|
|
paddingVertical: spacing.sm,
|
|
borderBottomWidth: StyleSheet.hairlineWidth,
|
|
borderBottomColor: colors.borderSubtle
|
|
},
|
|
sheetTitle: {
|
|
flex: 1,
|
|
color: colors.textPrimary,
|
|
fontSize: typography.bodySize,
|
|
fontWeight: '600'
|
|
},
|
|
closeButton: { padding: spacing.xs }
|
|
})
|