mirror of
https://github.com/stablyai/orca.git
synced 2026-10-20 08:03:28 +00:00
* fix(mobile): show attached images in native (rich) chat Attaching an image in the mobile native chat did nothing visible — it reused the terminal attach flow, which pastes a bracketed host path into the hidden terminal, so there was no composer preview and nothing in the transcript. Give native chat the desktop model instead: - pick + upload shows a removable thumbnail chip in the composer (no early paste) - on submit, images ride along: Ctrl+U clear -> bracketed paste(s) -> settle -> text + Enter (idempotent on retry) - the optimistic echo carries the local preview URIs and the message renderer draws image-ref blocks as real thumbnails when the URI is loadable, so the sent photo appears in the conversation immediately - image-only echoes reconcile by ordinal against user turns after their tail (ignores agent replies / paginated history / the 'unknown' ack-loss path) Terminal chat attach is unchanged (both flows consolidated behind useMobileSessionImageAttachments). Adds unit coverage for pick+upload, the ride-along byte order, chip render/remove, and echo reconciliation. * test(mobile): interactive native-chat image proof (real hooks, click-driven) Replace the hand-fed component render with an interactive harness that mounts the real MobileNativeChatComposer/Message + useMobileNativeChatImageAttachments + drafts under react-native-web and drives the actual flow via clicks. Only the two OS boundaries are faked: the photo picker and the paired-host RPC socket. Screenshots (mobile/docs/native-chat-image-attachment/) are produced by real clicks, not props: - attach -> real upload pipeline -> chip appears, nothing pasted yet - send -> real ride-along emits Ctrl+U clear, bracketed image paste, text+Enter (shown in the live byte trace) and the sent bubble renders the photo thumbnail * fix(mobile): scope native-chat image attachments by active tab Images are now scoped to the tab that initiated the pick, so switching tabs during upload cannot ride an image into another terminal. Chips stay with their original tab, and only the active scope's images send with text. Improved error handling with user-facing toast messages for disconnection and send failures. * test(mobile): add image attachment tab-scoping and error tests Add comprehensive test coverage for tab-scoped attachment behavior, error handling when transport fails or lease is gated, and edge cases like attaching images during an in-flight send. Extract baseArgs and update helpers to reduce boilerplate across test cases. * fix(mobile): show attached images in native rich chat Images attached in the mobile native (rich) chat now display as: - Removable composer chips while composing - Thumbnails in the sent user bubble after sending (desktop parity) Implements proper image echo reconciliation by distinguishing image-source marker turns from text echoes, so an image send isn't cleared by an unrelated text echo. Adds scope isolation to prevent chips and drafts from leaking between tabs, and detects tab switches during the image-paste settle window to abort the send. Fixes Android tap-target positioning for the image removal badge and clears stale terminal input after failed pastes to avoid gluing fragments onto the next message. * rm stubs --------- Co-authored-by: Jinjing <6427696+AmethystLiang@users.noreply.github.com>
436 lines
17 KiB
TypeScript
436 lines
17 KiB
TypeScript
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
|
import {
|
|
ActivityIndicator,
|
|
FlatList,
|
|
type NativeScrollEvent,
|
|
type NativeSyntheticEvent,
|
|
Pressable,
|
|
Text,
|
|
View
|
|
} from 'react-native'
|
|
import { useSafeAreaInsets } from 'react-native-safe-area-context'
|
|
import { GestureDetector, GestureHandlerRootView } from 'react-native-gesture-handler'
|
|
import { ArrowDown, ChevronsDownUp, ChevronsUpDown, Square } from 'lucide-react-native'
|
|
import type { NativeChatMessage } from '../../../src/shared/native-chat-types'
|
|
import { colors } from '../theme/mobile-theme'
|
|
import { styles } from './mobile-native-chat-view-styles'
|
|
import {
|
|
buildMobileNativeChatTransientData,
|
|
foldMobileNativeChatMessages,
|
|
mobileNativeChatEmptyState,
|
|
type MobileNativeChatPendingItem
|
|
} from './mobile-native-chat-render-data'
|
|
import { useMobileNativeChatAskDismiss } from './use-mobile-native-chat-ask-dismiss'
|
|
import { useMobileNativeChatPinchGesture } from './use-mobile-native-chat-pinch-gesture'
|
|
import { MobileAgentWorkingIndicator } from './MobileAgentWorkingIndicator'
|
|
import type { PendingNativeChatImage } from './mobile-native-chat-image-attachment'
|
|
import { MobileNativeChatComposer } from './MobileNativeChatComposer'
|
|
import { MobileNativeChatMessage } from './MobileNativeChatMessage'
|
|
import { MobileNativeChatAsk } from './MobileNativeChatAsk'
|
|
import type { AskAnswerSelection, AskPrompt } from './mobile-native-chat-ask'
|
|
import { MobileNativeChatPermission } from './MobileNativeChatPermission'
|
|
import type { MobileChatPermission } from './mobile-native-chat-permission'
|
|
import { MobileNativeChatQuestion } from './MobileNativeChatQuestion'
|
|
import { mobileChatQuestionKey, type MobileChatQuestion } from './mobile-native-chat-question'
|
|
import type { MobileNativeChatStatus } from './use-mobile-native-chat-session'
|
|
|
|
/** Why the composer input is locked: the transport is disconnected, or the
|
|
* terminal subscription has not acknowledged its input lease yet. */
|
|
export type MobileNativeChatInputLockReason = 'disconnected' | 'waiting'
|
|
|
|
type Props = {
|
|
messages: NativeChatMessage[]
|
|
status: MobileNativeChatStatus
|
|
error?: string
|
|
/** Resolved agent for this chat; names the empty-state copy (desktop parity). */
|
|
agent?: string | null
|
|
agentWorking?: boolean
|
|
/** Interrupt the agent mid-turn (shown as a Stop button on the working bar). */
|
|
onStop?: () => void
|
|
/** Live partial assistant text while a turn is still streaming (from the agent
|
|
* status hook). Shown as an in-progress bubble until the transcript catches up. */
|
|
streamingText?: string
|
|
hasMore?: boolean
|
|
loadingEarlier?: boolean
|
|
onLoadEarlier?: () => void
|
|
onSend: (text: string) => Promise<boolean>
|
|
/** Optimistic queued sends (owned by the route so they survive view switches). */
|
|
/** Optimistic user echoes, including any ridden-along image preview URIs. */
|
|
pending: MobileNativeChatPendingItem[]
|
|
/** Controlled composer text (owned by the route so dictation can write to it). */
|
|
composerText: string
|
|
onComposerTextChange: (text: string) => void
|
|
onAttachImage?: () => void
|
|
/** Pending image attachments shown as composer thumbnails until the next send. */
|
|
attachments?: PendingNativeChatImage[]
|
|
onRemoveAttachment?: (id: string) => void
|
|
isAttaching?: boolean
|
|
onMicPress?: () => void
|
|
micActive?: boolean
|
|
dictationMode?: 'toggle' | 'hold'
|
|
onMicPressIn?: () => void
|
|
onMicPressOut?: () => void
|
|
inputLockReason?: MobileNativeChatInputLockReason | null
|
|
filePaths?: string[]
|
|
onNeedFiles?: (query: string) => void
|
|
/** A pending agent question/permission detected from live status, shown as a
|
|
* native card above the composer; answering sends text to the agent. */
|
|
/** Structured AskUserQuestion prompt parsed from the transcript (preferred over
|
|
* the heuristic question card). */
|
|
ask?: AskPrompt | null
|
|
/** Deliver the ask answer as per-question selections; the send hook turns them
|
|
* into selector keystrokes (Claude) or pasted label text (other agents). */
|
|
onAnswerAsk?: (prompt: AskPrompt, selections: AskAnswerSelection[]) => Promise<boolean>
|
|
onCancelAsk?: () => Promise<boolean>
|
|
question?: MobileChatQuestion | null
|
|
onAnswerQuestion?: (text: string) => Promise<boolean>
|
|
permission?: MobileChatPermission | null
|
|
onRespondPermission?: (send: string) => Promise<boolean>
|
|
/** Open a worktree file tapped in agent markdown. */
|
|
onOpenFile?: (relativePath: string) => void
|
|
/** Pixels to lift the composer by when the soft keyboard is open. The route
|
|
* owns keyboard tracking (the app uses manual lift, not KeyboardAvoidingView). */
|
|
keyboardInset?: number
|
|
}
|
|
|
|
export function MobileNativeChatView({
|
|
messages,
|
|
status,
|
|
error,
|
|
agent,
|
|
agentWorking,
|
|
onStop,
|
|
streamingText,
|
|
hasMore,
|
|
loadingEarlier,
|
|
onLoadEarlier,
|
|
onSend,
|
|
pending,
|
|
composerText,
|
|
onComposerTextChange,
|
|
onAttachImage,
|
|
attachments,
|
|
onRemoveAttachment,
|
|
isAttaching,
|
|
onMicPress,
|
|
micActive,
|
|
dictationMode,
|
|
onMicPressIn,
|
|
onMicPressOut,
|
|
inputLockReason,
|
|
filePaths,
|
|
onNeedFiles,
|
|
ask,
|
|
onAnswerAsk,
|
|
onCancelAsk,
|
|
question,
|
|
onAnswerQuestion,
|
|
permission,
|
|
onRespondPermission,
|
|
onOpenFile,
|
|
keyboardInset = 0
|
|
}: Props): React.JSX.Element {
|
|
const insets = useSafeAreaInsets()
|
|
const listRef = useRef<FlatList<NativeChatMessage>>(null)
|
|
const [toolsExpanded, setToolsExpanded] = useState(false)
|
|
// Dismiss the question card as soon as it's answered; the live status lingers
|
|
// briefly (the agent emits a post-tool event with the same prompt), so hide it
|
|
// until a genuinely different question arrives.
|
|
const { askKey, showAsk, dismissAsk } = useMobileNativeChatAskDismiss(ask)
|
|
// Lift the composer clear of the keyboard, plus the bottom safe-area so it
|
|
// never sits under the home indicator / nav bar (mirrors the terminal dock).
|
|
const bottomPad = keyboardInset > 0 ? keyboardInset + insets.bottom : insets.bottom
|
|
const [atBottom, setAtBottom] = useState(true)
|
|
const sendScrollTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null)
|
|
const { fontScale, pinchGesture } = useMobileNativeChatPinchGesture()
|
|
// Surface a rejected send inline above the composer — a bottom toast gets hidden
|
|
// behind the keyboard (the case that prompted this). Auto-dismisses after a beat.
|
|
const [sendFailed, setSendFailed] = useState(false)
|
|
useEffect(() => {
|
|
if (!sendFailed) {
|
|
return
|
|
}
|
|
const t = setTimeout(() => setSendFailed(false), 4000)
|
|
return () => clearTimeout(t)
|
|
}, [sendFailed])
|
|
|
|
useEffect(
|
|
() => () => {
|
|
if (sendScrollTimerRef.current) {
|
|
clearTimeout(sendScrollTimerRef.current)
|
|
}
|
|
},
|
|
[]
|
|
)
|
|
|
|
const pendingIds = useMemo(() => new Set(pending.map((p) => p.id)), [pending])
|
|
// `data` is the list source: folded transcript + synthetic streaming bubble +
|
|
// route-owned optimistic queued messages. Memoize on the same deps so the
|
|
// downstream autoscroll effects/`renderItem` keep referential stability.
|
|
const foldedMessages = useMemo(() => foldMobileNativeChatMessages(messages), [messages])
|
|
const { data } = useMemo(
|
|
() => buildMobileNativeChatTransientData({ folded: foldedMessages, streamingText, pending }),
|
|
[foldedMessages, streamingText, pending]
|
|
)
|
|
|
|
// Follow the tail as the conversation grows and keep the newest message above
|
|
// the keyboard when it opens — but only when already pinned to the bottom, so
|
|
// we don't yank the user away while they read history. (Also fires on keyboard
|
|
// close, which is harmless while atBottom.)
|
|
useEffect(() => {
|
|
if (data.length === 0 || !atBottom) {
|
|
return
|
|
}
|
|
const t = setTimeout(() => listRef.current?.scrollToEnd({ animated: true }), 60)
|
|
return () => clearTimeout(t)
|
|
}, [data.length, atBottom, keyboardInset])
|
|
|
|
const handleSend = useCallback(
|
|
async (text: string): Promise<boolean> => {
|
|
const accepted = await onSend(text)
|
|
if (!accepted) {
|
|
setSendFailed(true)
|
|
return false
|
|
}
|
|
setSendFailed(false)
|
|
// Always jump to the newest message when the user sends.
|
|
setAtBottom(true)
|
|
if (sendScrollTimerRef.current) {
|
|
clearTimeout(sendScrollTimerRef.current)
|
|
}
|
|
sendScrollTimerRef.current = setTimeout(() => {
|
|
sendScrollTimerRef.current = null
|
|
listRef.current?.scrollToEnd({ animated: true })
|
|
}, 60)
|
|
return true
|
|
},
|
|
[onSend]
|
|
)
|
|
|
|
const onScroll = useCallback(
|
|
(e: NativeSyntheticEvent<NativeScrollEvent>) => {
|
|
const { contentOffset, contentSize, layoutMeasurement } = e.nativeEvent
|
|
const distanceFromBottom = contentSize.height - (contentOffset.y + layoutMeasurement.height)
|
|
setAtBottom(distanceFromBottom < 80)
|
|
// Near the top — page in older history.
|
|
if (contentOffset.y < 60 && hasMore && !loadingEarlier) {
|
|
onLoadEarlier?.()
|
|
}
|
|
},
|
|
[hasMore, loadingEarlier, onLoadEarlier]
|
|
)
|
|
|
|
// Align a single message's top to the top of the viewport.
|
|
const onScrollToMessage = useCallback((index: number) => {
|
|
listRef.current?.scrollToIndex({ index, viewPosition: 0, animated: true })
|
|
}, [])
|
|
|
|
const renderItem = useCallback(
|
|
({ item, index }: { item: NativeChatMessage; index: number }) => (
|
|
<MobileNativeChatMessage
|
|
message={item}
|
|
queued={pendingIds.has(item.id)}
|
|
toolsExpanded={toolsExpanded}
|
|
fontScale={fontScale}
|
|
messageIndex={index}
|
|
onScrollToMessage={onScrollToMessage}
|
|
onOpenFile={onOpenFile}
|
|
/>
|
|
),
|
|
[pendingIds, toolsExpanded, fontScale, onScrollToMessage, onOpenFile]
|
|
)
|
|
|
|
const emptyState = mobileNativeChatEmptyState(status, agent ?? null, error)
|
|
const showLoading = status === 'loading' && messages.length === 0
|
|
|
|
// Composer-lock flicker guard: on a remote link, brief connState blips or lease
|
|
// hand-offs would otherwise toggle the lock placeholder on and off. Only surface
|
|
// a lock once it has held ~600ms; drop it instantly so unlocking stays snappy.
|
|
const rawLockReason = inputLockReason ?? null
|
|
const [lockHeld, setLockHeld] = useState(false)
|
|
useEffect(() => {
|
|
if (rawLockReason === null) {
|
|
setLockHeld(false)
|
|
return
|
|
}
|
|
const timer = setTimeout(() => setLockHeld(true), 600)
|
|
return () => clearTimeout(timer)
|
|
}, [rawLockReason])
|
|
const lockReason = lockHeld ? rawLockReason : null
|
|
|
|
return (
|
|
<View style={[styles.root, { paddingBottom: bottomPad }]}>
|
|
{showLoading ? (
|
|
<View style={styles.center}>
|
|
<ActivityIndicator color={colors.textSecondary} />
|
|
</View>
|
|
) : (
|
|
<GestureHandlerRootView style={styles.listWrap}>
|
|
<GestureDetector gesture={pinchGesture}>
|
|
<FlatList
|
|
ref={listRef}
|
|
data={data}
|
|
keyExtractor={(item) => item.id}
|
|
renderItem={renderItem}
|
|
contentContainerStyle={styles.listContent}
|
|
onScroll={onScroll}
|
|
scrollEventThrottle={32}
|
|
onContentSizeChange={() => {
|
|
if (data.length > 0 && atBottom) {
|
|
listRef.current?.scrollToEnd({ animated: false })
|
|
}
|
|
}}
|
|
// scrollToIndex can fail before an off-screen row is measured —
|
|
// fall back to an estimated offset, then retry once it's laid out.
|
|
onScrollToIndexFailed={(info) => {
|
|
listRef.current?.scrollToOffset({
|
|
offset: info.averageItemLength * info.index,
|
|
animated: true
|
|
})
|
|
setTimeout(() => {
|
|
listRef.current?.scrollToIndex({
|
|
index: info.index,
|
|
viewPosition: 0,
|
|
animated: true
|
|
})
|
|
}, 120)
|
|
}}
|
|
ListHeaderComponent={
|
|
hasMore ? (
|
|
<Pressable
|
|
style={styles.loadEarlier}
|
|
onPress={onLoadEarlier}
|
|
disabled={loadingEarlier}
|
|
>
|
|
{loadingEarlier ? (
|
|
<ActivityIndicator size="small" color={colors.textMuted} />
|
|
) : (
|
|
<Text style={styles.loadEarlierText}>Load earlier messages</Text>
|
|
)}
|
|
</Pressable>
|
|
) : null
|
|
}
|
|
ListEmptyComponent={
|
|
emptyState ? (
|
|
<View style={styles.center}>
|
|
<Text style={styles.emptyTitle}>{emptyState.title}</Text>
|
|
<Text style={styles.emptySubtitle}>{emptyState.subtitle}</Text>
|
|
</View>
|
|
) : null
|
|
}
|
|
/>
|
|
</GestureDetector>
|
|
{/* Jump-to-latest control. The scroll-to-top affordance now lives
|
|
per-message (the up-arrow in each agent message's controls). */}
|
|
{!atBottom ? (
|
|
<Pressable
|
|
accessibilityLabel="Scroll to latest"
|
|
style={[styles.fab, styles.fabBottom]}
|
|
onPress={() => listRef.current?.scrollToEnd({ animated: true })}
|
|
>
|
|
<ArrowDown size={18} color={colors.textPrimary} strokeWidth={2.2} />
|
|
</Pressable>
|
|
) : null}
|
|
</GestureHandlerRootView>
|
|
)}
|
|
{/* Pending agent prompt: a structured AskUserQuestion wins, then a
|
|
heuristic permission, then a heuristic question. */}
|
|
{showAsk && ask ? (
|
|
<MobileNativeChatAsk
|
|
key={askKey ?? 'ask'}
|
|
prompt={ask}
|
|
onAnswer={async (selections) => {
|
|
const accepted = (await onAnswerAsk?.(ask, selections)) ?? false
|
|
if (accepted) {
|
|
dismissAsk()
|
|
}
|
|
return accepted
|
|
}}
|
|
onCancel={async () => {
|
|
const accepted = (await onCancelAsk?.()) ?? false
|
|
if (accepted) {
|
|
dismissAsk()
|
|
}
|
|
return accepted
|
|
}}
|
|
/>
|
|
) : permission ? (
|
|
<MobileNativeChatPermission
|
|
key={JSON.stringify(permission)}
|
|
permission={permission}
|
|
onRespond={async (send) => (await onRespondPermission?.(send)) ?? false}
|
|
/>
|
|
) : question ? (
|
|
<MobileNativeChatQuestion
|
|
key={mobileChatQuestionKey(question)}
|
|
question={question}
|
|
onAnswer={async (text) => (await onAnswerQuestion?.(text)) ?? false}
|
|
/>
|
|
) : null}
|
|
{/* Chrome row above the composer: the working indicator and the global
|
|
tool-calls expand/collapse toggle on the left, Stop in the far corner. */}
|
|
<View style={styles.chromeRow}>
|
|
<View style={styles.chromeLeft}>
|
|
{agentWorking ? <MobileAgentWorkingIndicator /> : null}
|
|
<Pressable
|
|
style={({ pressed }) => [styles.chromeToggle, pressed && styles.pressed]}
|
|
onPress={() => setToolsExpanded((v) => !v)}
|
|
hitSlop={8}
|
|
>
|
|
{toolsExpanded ? (
|
|
<ChevronsDownUp size={14} color={colors.textMuted} strokeWidth={2} />
|
|
) : (
|
|
<ChevronsUpDown size={14} color={colors.textMuted} strokeWidth={2} />
|
|
)}
|
|
<Text style={styles.chromeToggleLabel}>{toolsExpanded ? 'Collapse' : 'Tools'}</Text>
|
|
</Pressable>
|
|
</View>
|
|
{agentWorking ? (
|
|
<Pressable
|
|
style={({ pressed }) => [styles.stopButton, pressed && styles.pressed]}
|
|
onPress={onStop}
|
|
hitSlop={8}
|
|
accessibilityLabel="Stop the agent"
|
|
>
|
|
<Square size={13} color={colors.statusRed} strokeWidth={2.4} fill={colors.statusRed} />
|
|
<Text style={styles.stopLabel}>Stop</Text>
|
|
</Pressable>
|
|
) : null}
|
|
</View>
|
|
{sendFailed ? (
|
|
<View style={styles.sendError}>
|
|
<Text style={styles.sendErrorText}>
|
|
{rawLockReason === 'disconnected'
|
|
? 'Message not sent — reconnecting…'
|
|
: 'Message not sent'}
|
|
</Text>
|
|
</View>
|
|
) : null}
|
|
<MobileNativeChatComposer
|
|
value={composerText}
|
|
onChangeText={onComposerTextChange}
|
|
onSend={handleSend}
|
|
onAttachImage={onAttachImage}
|
|
attachments={attachments}
|
|
onRemoveAttachment={onRemoveAttachment}
|
|
isAttaching={isAttaching}
|
|
onMicPress={onMicPress}
|
|
micActive={micActive}
|
|
dictationMode={dictationMode}
|
|
onMicPressIn={onMicPressIn}
|
|
onMicPressOut={onMicPressOut}
|
|
disabled={lockReason !== null}
|
|
placeholder={
|
|
lockReason === 'disconnected'
|
|
? 'Reconnecting…'
|
|
: lockReason === 'waiting'
|
|
? 'Waiting for terminal…'
|
|
: 'Message, @files, /commands'
|
|
}
|
|
filePaths={filePaths}
|
|
onNeedFiles={onNeedFiles}
|
|
/>
|
|
</View>
|
|
)
|
|
}
|