Split browser pane (#14861)

* refactor: split BrowserPane.tsx under 400 lines

* rm plan

* refactor(browser-pane): reorganize into lifecycle folders

Cut/paste + import rewrites only; no intentional behavior change.

- annotate/, assemble-chrome/, host-guest/, navigate/, stream-remote/,
  describe-page/ (foundation sink, zero outgoing edges)
- BrowserPane.tsx is now a pure re-export barrel; its component body moved
  verbatim to assemble-chrome/browser-workspace-pane.tsx so no dest file
  imports the barrel
- browser-runtime.ts -> describe-page/live-browser-url-registry.ts (banned
  name; relocating the contract collapsed the host-guest/navigate mutual pair)
- repath browser-pane test paths in config/reliability-gates.jsonc

* refactor: sync addressBarValueRef with useEffect

Move ref synchronization into useEffect hook with proper dependency
tracking to ensure the ref updates are handled through React's
lifecycle. Consolidate related imports from browser-page-types.

* refactor(browser-pane): fix React lifecycle and external store patterns

- Replace local state + effects with useSyncExternalStore for external subscriptions (draw hint, address bar, slot viewport)
- Fix React StrictMode double-invoke issues in pointer handlers and state updates
- Add keyboard navigation to context menu (arrows, Home, End, Escape) with focus management
- Improve error handling for mobile driver reclaim and grab action IPC failures
- Add test coverage for BrowserFind session flags, keyboard behavior, viewport lifecycle
- Remove react-doctor/no-adjust-state-on-prop-change lint disables (root causes now fixed)

* i18n: extract grab and download UI messages

Move hardcoded toast notifications and error messages to translation
system for both grab annotations and file drop handling. Also apply
lazy initialization to address bar value and remove duplicate event
recording.

* fix(browser-pane): stop mutating refs during render

React Doctor fails static analysis when refs are written in render.
Mirror latest values in useLayoutEffect, and read the current page id
from the latest grab callbacks.

* fix(browser-pane): drop unused grab-mode exit dependency

exit already reads the page id from a ref, so listing browserPageId
trips the changed-code exhaustive-deps gate.

* test(e2e): hide the window when Linux minimize is a no-op

Xvfb has no window manager, so BrowserWindow.minimize() never sets
isMinimized() on the frameless Linux CI window. Hide still occludes
the guest compositor so restore coverage can run.
This commit is contained in:
Jinjing
2026-08-17 14:53:19 -07:00
committed by GitHub
parent 39260d16c7
commit a3a2c44edf
208 changed files with 9472 additions and 6010 deletions
-1
View File
@@ -98,7 +98,6 @@ inline src/renderer/src/components/UpdateCard.tsx
inline src/renderer/src/components/WorktreeJumpPalette.tsx
inline src/renderer/src/components/activity/ActivityPrototypePage.tsx
inline src/renderer/src/components/automations/AutomationsPage.tsx
inline src/renderer/src/components/browser-pane/BrowserPane.tsx
inline src/renderer/src/components/editor/CombinedDiffViewer.tsx
inline src/renderer/src/components/editor/EditorContent.tsx
inline src/renderer/src/components/editor/IpynbViewer.tsx
+12 -12
View File
@@ -2667,19 +2667,19 @@
"invariant": "A current remote browser stream that closes must retry transient failures on one counted, capped backoff chain. Cancellation, pane or runtime supersession, missing pages, and permanent failures stop that chain without stale state mutation. A stream that stays healthy refills the budget; a silent or flapping stream cannot retry forever. Exhaustion and every non-retrying failure leave an actionable Reconnect control, and manual reconnect creates a fresh stream without resurrecting stale subscriptions.",
"oracle": "With one fake clock and exact pane, operation, stream, page, and runtime identities, drop a confirmed stream and require the counted delay sequence, one active chain, stale-token rejection, exact cancellation, full cleanup, and either a new live subscription or one stopped state with Reconnect after exhaustion. Inject transport error without close, close racing rejection, never-ready subscriptions, ready-then-end flapping, viewport supersession, missing pages, and permanent failures. In a separate paired client against an isolated headless Orca host, paint one remote frame, disconnect the owning runtime, require the bounded-retry notice and Reconnect control, reconnect, click once, and require a different frame object URL from the recovered stream.",
"commands": [
"pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/browser-pane/remote-browser-stream-errors.test.ts src/renderer/src/components/browser-pane/remote-browser-stream-lifecycle.test.ts src/renderer/src/components/browser-pane/remote-browser-stream-restart-scheduler.test.ts src/renderer/src/components/browser-pane/remote-browser-stream-status.test.ts",
"pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-errors.test.ts src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-lifecycle.test.ts src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-restart-scheduler.test.ts src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-status.test.ts",
"pnpm exec playwright test tests/e2e/paired-remote-browser-stream-reconnect.spec.ts --config tests/playwright.config.ts --project electron-headless --workers=1"
],
"testFiles": [
"src/renderer/src/components/browser-pane/remote-browser-stream-errors.test.ts",
"src/renderer/src/components/browser-pane/remote-browser-stream-lifecycle.test.ts",
"src/renderer/src/components/browser-pane/remote-browser-stream-restart-scheduler.test.ts",
"src/renderer/src/components/browser-pane/remote-browser-stream-status.test.ts",
"src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-errors.test.ts",
"src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-lifecycle.test.ts",
"src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-restart-scheduler.test.ts",
"src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-status.test.ts",
"tests/e2e/paired-remote-browser-stream-reconnect.spec.ts"
],
"assertionRefs": [
{
"file": "src/renderer/src/components/browser-pane/remote-browser-stream-restart-scheduler.test.ts",
"file": "src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-restart-scheduler.test.ts",
"assertions": [
"transient failures retry until recovery on counted delays",
"budget exhaustion stops exactly once",
@@ -2688,7 +2688,7 @@
]
},
{
"file": "src/renderer/src/components/browser-pane/remote-browser-stream-lifecycle.test.ts",
"file": "src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-lifecycle.test.ts",
"assertions": [
"dropped, silent, and flapping streams converge on a bounded retry or actionable stop",
"stale, superseded, missing-page, and permanent-failure work cannot mutate the current pane",
@@ -2710,7 +2710,7 @@
"date": "2026-08-07",
"runner": "local",
"platform": "macos",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/browser-pane/remote-browser-stream-errors.test.ts src/renderer/src/components/browser-pane/remote-browser-stream-lifecycle.test.ts src/renderer/src/components/browser-pane/remote-browser-stream-restart-scheduler.test.ts src/renderer/src/components/browser-pane/remote-browser-stream-status.test.ts",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-errors.test.ts src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-lifecycle.test.ts src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-restart-scheduler.test.ts src/renderer/src/components/browser-pane/stream-remote/remote-browser-stream-status.test.ts",
"result": "passed",
"durationSeconds": 0.36,
"summary": "Four focused files passed 61 deterministic error, status, scheduler, lifecycle, in-flight ownership, stale-rejection, liveness, and cleanup tests."
@@ -9119,7 +9119,7 @@
"oracle": "A quarter-circle Claude task title authorizes without a provider lookup only on the exact PTY incarnation carrying verified managed-Claude launch identity; the same title in a bare pane, with an unverified launch hint, or after PTY incarnation replacement is refused while current Claude activity remains a working signal. Fresh explicit state plus ordinary PowerShell plus confirmed recognized agent is sendable on the same PTY. Confirmed shell/non-agent, unavailable confirmation, PTY exit, handle rebind, or a callback PTY mismatch returns a refusal or not-writable result and writes zero bytes.",
"commands": [
"pnpm exec vitest run --config config/vitest.config.ts src/main/runtime/quarter-circle-title-send-authorization.test.ts",
"pnpm exec vitest run --config config/vitest.config.ts src/main/runtime/orca-runtime.test.ts src/main/runtime/rpc/terminal-send.test.ts src/main/ipc/pty-runtime-kill-and-exit.test.ts src/main/providers/agent-foreground-process.test.ts src/main/providers/local-pty-provider-foreground-process.test.ts src/main/providers/windows-conpty-process-membership.test.ts src/main/daemon/daemon-foreground-confirmation-protocol.test.ts src/main/daemon/pty-subprocess.test.ts src/renderer/src/lib/active-agent-note-send-explicit-target.test.ts src/renderer/src/components/browser-pane/BrowserAnnotationSendMenuContent.test.tsx"
"pnpm exec vitest run --config config/vitest.config.ts src/main/runtime/orca-runtime.test.ts src/main/runtime/rpc/terminal-send.test.ts src/main/ipc/pty-runtime-kill-and-exit.test.ts src/main/providers/agent-foreground-process.test.ts src/main/providers/local-pty-provider-foreground-process.test.ts src/main/providers/windows-conpty-process-membership.test.ts src/main/daemon/daemon-foreground-confirmation-protocol.test.ts src/main/daemon/pty-subprocess.test.ts src/renderer/src/lib/active-agent-note-send-explicit-target.test.ts src/renderer/src/components/browser-pane/annotate/BrowserAnnotationSendMenuContent.test.tsx"
],
"testFiles": [
"src/main/runtime/quarter-circle-title-send-authorization.test.ts",
@@ -9132,7 +9132,7 @@
"src/main/daemon/daemon-foreground-confirmation-protocol.test.ts",
"src/main/daemon/pty-subprocess.test.ts",
"src/renderer/src/lib/active-agent-note-send-explicit-target.test.ts",
"src/renderer/src/components/browser-pane/BrowserAnnotationSendMenuContent.test.tsx"
"src/renderer/src/components/browser-pane/annotate/BrowserAnnotationSendMenuContent.test.tsx"
],
"assertionRefs": [
{
@@ -9202,7 +9202,7 @@
"assertions": ["selected active-agent note sends retain guarded paste and submit routing"]
},
{
"file": "src/renderer/src/components/browser-pane/BrowserAnnotationSendMenuContent.test.tsx",
"file": "src/renderer/src/components/browser-pane/annotate/BrowserAnnotationSendMenuContent.test.tsx",
"assertions": [
"browser annotation send content routes through review-notes send so existing agent sessions remain selectable",
"both browser annotation send surfaces wire the shared menu content"
@@ -9223,7 +9223,7 @@
"date": "2026-07-11",
"runner": "local",
"platform": "macos",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/main/runtime/orca-runtime.test.ts src/main/runtime/rpc/terminal-send.test.ts src/main/ipc/pty-runtime-kill-and-exit.test.ts src/main/providers/agent-foreground-process.test.ts src/main/providers/local-pty-provider-foreground-process.test.ts src/main/providers/windows-conpty-process-membership.test.ts src/main/daemon/daemon-foreground-confirmation-protocol.test.ts src/main/daemon/pty-subprocess.test.ts src/renderer/src/lib/active-agent-note-send-explicit-target.test.ts src/renderer/src/components/browser-pane/BrowserAnnotationSendMenuContent.test.tsx",
"command": "pnpm exec vitest run --config config/vitest.config.ts src/main/runtime/orca-runtime.test.ts src/main/runtime/rpc/terminal-send.test.ts src/main/ipc/pty-runtime-kill-and-exit.test.ts src/main/providers/agent-foreground-process.test.ts src/main/providers/local-pty-provider-foreground-process.test.ts src/main/providers/windows-conpty-process-membership.test.ts src/main/daemon/daemon-foreground-confirmation-protocol.test.ts src/main/daemon/pty-subprocess.test.ts src/renderer/src/lib/active-agent-note-send-explicit-target.test.ts src/renderer/src/components/browser-pane/annotate/BrowserAnnotationSendMenuContent.test.tsx",
"result": "passed",
"durationSeconds": 9.58,
"summary": "Ten focused test files passed (1183 tests), covering runtime confirmation and PTY revalidation, guarded RPC zero-write behavior, PTY controller routing, local/daemon fresh scans, exact ConPTY membership, and unchanged renderer note routing."
+6 -6
View File
@@ -42,13 +42,13 @@ import type { TerminalTab } from '../../../shared/terminal-tab-types'
import type { TuiAgent } from '../../../shared/tui-agent'
import { hasFeatureInteraction } from '../../../shared/feature-interactions'
import BrowserPane from './browser-pane/BrowserPane'
import { RetainedBrowserPaneOverlayLayer } from './browser-pane/BrowserPaneOverlayLayer'
import { RetainedBrowserPaneOverlayLayer } from './browser-pane/assemble-chrome/BrowserPaneOverlayLayer'
import EmulatorPaneOverlayLayer from './emulator-pane/EmulatorPaneOverlayLayer'
import {
isBrowserAutomationVisible,
onBrowserAutomationVisibilityChange,
useBrowserAutomationVisibilityForAny
} from './browser-pane/browser-automation-visibility'
} from './browser-pane/host-guest/browser-automation-visibility'
import {
isBrowserPageMobileDriven,
onBrowserDriverChange,
@@ -57,7 +57,7 @@ import {
import {
useAnyBrowserGuestNeedsPaint,
useWorktreeBrowserPageIds
} from './browser-pane/browser-guest-paint-retention'
} from './browser-pane/host-guest/browser-guest-paint-retention'
import TerminalPaneOverlayLayer from './terminal-pane/TerminalPaneOverlayLayer'
import {
collectBrowserWebviewIds,
@@ -70,12 +70,12 @@ import {
selectBrowserGuestEvictionWorktreeIds,
touchBrowserGuestWorktreeRecency,
worktreeHoldsLiveBrowserGuests
} from './browser-pane/browser-guest-worktree-retention'
} from './browser-pane/host-guest/browser-guest-worktree-retention'
import {
hasActiveBrowserPageDownload,
installBrowserPageDownloadActivityTracking
} from './browser-pane/browser-page-download-activity'
import { hasLiveBrowserGuest } from './browser-pane/webview-registry'
} from './browser-pane/navigate/browser-page-download-activity'
import { hasLiveBrowserGuest } from './browser-pane/host-guest/webview-registry'
import {
handleSwitchRecentTab,
handleSwitchTab,
@@ -20,7 +20,7 @@ vi.mock('../store', () => ({
// the wiring under test unproven, which is the whole point of this file.
const { setDriverForBrowserPage } = await import('../lib/pane-manager/browser-mobile-driver-state')
const { acquireBrowserAutomationVisibility, releaseBrowserAutomationVisibility } =
await import('./browser-pane/browser-automation-visibility')
await import('./browser-pane/host-guest/browser-automation-visibility')
const { TerminalWorkbenchContainer } = await import('./TerminalWorkbenchContainer')
const PAGE_ID = 'page-1'
@@ -1,5 +1,5 @@
import type React from 'react'
import { useAnyBrowserGuestNeedsPaint } from './browser-pane/browser-guest-paint-retention'
import { useAnyBrowserGuestNeedsPaint } from './browser-pane/host-guest/browser-guest-paint-retention'
// Why: the outermost ancestor of every browser <webview>. Parking it with `hidden` whenever
// the user leaves the workspace view also stops the guest compositing, which silently kills
@@ -130,7 +130,7 @@ import { useShortcutKeyComboDetails } from '@/hooks/useShortcutLabel'
import {
ORCA_BROWSER_FOCUS_REQUEST_EVENT,
queueBrowserFocusRequest
} from '@/components/browser-pane/browser-focus'
} from '@/components/browser-pane/host-guest/browser-focus'
import { RepoBadgeMark } from '@/components/repo/RepoBadgeLabel'
import { buildSidebarHostOptions } from '@/components/sidebar/sidebar-host-options'
import { getPaletteHostBadge, type PaletteHostBadge } from '@/components/cmd-j/palette-host-badge'
@@ -1,7 +1,7 @@
import { useEffect, useRef, useState } from 'react'
import { AlertCircle, Loader2 } from 'lucide-react'
import { ORCA_BROWSER_GUEST_WEB_PREFERENCES_ATTRIBUTE } from '../../../../shared/browser-guest-web-preferences'
import { moveFocusToRendererBeforeWebviewDetach } from '@/components/browser-pane/webview-registry'
import { moveFocusToRendererBeforeWebviewDetach } from '@/components/browser-pane/host-guest/webview-registry'
import { translate } from '@/i18n/i18n'
type PreviewState = 'loading' | 'ready' | 'unavailable'
@@ -20,8 +20,8 @@ vi.mock('../../store', () => ({ useAppStore: appStoreStub() }))
vi.mock('sonner', () => ({ toast: { success: vi.fn(), error: vi.fn() } }))
import { BrowserCookieImportDisclosure } from './BrowserCookieImportDisclosure'
import { BrowserImportHintButton } from './browser-pane/BrowserImportHintButton'
import { BrowserToolbarMenuDropdown } from './browser-pane/browser-toolbar-menu-dropdown'
import { BrowserImportHintButton } from './browser-pane/assemble-chrome/BrowserImportHintButton'
import { BrowserToolbarMenuDropdown } from './browser-pane/assemble-chrome/browser-toolbar-menu-dropdown'
import { BrowserProfileRow } from './settings/BrowserProfileRow'
import { BrowserUseCookieImportStep } from './settings/BrowserUseCookieImportStep'
File diff suppressed because it is too large Load Diff
@@ -94,12 +94,17 @@ describe('BrowserAnnotationSendMenuContent', () => {
})
it('is wired into both browser annotation send surfaces', () => {
const browserPaneSource = readFileSync(
fileURLToPath(new URL('./BrowserPane.tsx', import.meta.url)),
const bannerSource = readFileSync(
fileURLToPath(new URL('../assemble-chrome/browser-page-chrome-banners.tsx', import.meta.url)),
'utf8'
)
const traySource = readFileSync(
fileURLToPath(new URL('./browser-page-annotation-tray.tsx', import.meta.url)),
'utf8'
)
const sendSurfaces = `${bannerSource}\n${traySource}`
expect(browserPaneSource.match(/<BrowserAnnotationSendMenuContent\b/g)).toHaveLength(2)
expect(browserPaneSource).not.toContain('<QuickLaunchAgentMenuItems')
expect(sendSurfaces.match(/<BrowserAnnotationSendMenuContent\b/g)).toHaveLength(2)
expect(sendSurfaces).not.toContain('<QuickLaunchAgentMenuItems')
})
})
@@ -1,6 +1,6 @@
import { describe, expect, it } from 'vitest'
import { formatGrabPayloadAsText } from './GrabConfirmationSheet'
import type { BrowserGrabPayload } from '../../../../shared/browser-grab-types'
import type { BrowserGrabPayload } from '../../../../../shared/browser-grab-types'
function makeTestPayload(overrides?: Partial<BrowserGrabPayload>): BrowserGrabPayload {
return {
@@ -1,7 +1,7 @@
import { Copy, Image, MessageSquarePlus, X } from 'lucide-react'
import { Button } from '@/components/ui/button'
import { ScrollArea } from '@/components/ui/scroll-area'
import type { BrowserGrabPayload } from '../../../../shared/browser-grab-types'
import type { BrowserGrabPayload } from '../../../../../shared/browser-grab-types'
import { translate } from '@/i18n/i18n'
// ---------------------------------------------------------------------------
@@ -103,6 +103,15 @@ function EscapedText({ text, className }: { text: string; className?: string }):
return <span className={className}>{text}</span>
}
function getNearbyTextRows(nearbyText: readonly string[]): { key: string; text: string }[] {
const counts = new Map<string, number>()
return nearbyText.map((text) => {
const count = counts.get(text) ?? 0
counts.set(text, count + 1)
return { key: `${text}:${count}`, text }
})
}
// ---------------------------------------------------------------------------
// Confirmation Sheet Component
// ---------------------------------------------------------------------------
@@ -121,6 +130,7 @@ export default function GrabConfirmationSheet({
onCancel: () => void
}): React.JSX.Element {
const { target, page, nearbyText } = payload
const nearbyTextRows = getNearbyTextRows(nearbyText)
return (
<div className="absolute inset-0 z-20 flex flex-col bg-background/98 backdrop-blur-sm">
@@ -246,9 +256,9 @@ export default function GrabConfirmationSheet({
</h3>
<div className="rounded-lg border border-border/60 bg-muted/20 p-3">
<ul className="list-inside list-disc space-y-0.5 text-sm text-muted-foreground">
{nearbyText.map((text, i) => (
<li key={i}>
<EscapedText text={text} />
{nearbyTextRows.map((row) => (
<li key={row.key}>
<EscapedText text={row.text} />
</li>
))}
</ul>
@@ -1,5 +1,5 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { BrowserPageAnnotation } from '../../../../shared/browser-grab-types'
import type { BrowserPageAnnotation } from '../../../../../shared/browser-grab-types'
import {
BROWSER_ANNOTATION_INLINE_TEXT_MAX_LENGTH,
formatBrowserAnnotationsAsMarkdown
@@ -2,7 +2,7 @@ import type {
BrowserGrabComputedStyles,
BrowserGrabPayload,
BrowserPageAnnotation
} from '../../../../shared/browser-grab-types'
} from '../../../../../shared/browser-grab-types'
function formatPageHeading(payload: BrowserGrabPayload): string {
try {
@@ -0,0 +1,164 @@
import { CircleCheck, Copy, MessageSquarePlus, Send, Trash2 } from 'lucide-react'
import { Button } from '@/components/ui/button'
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuTrigger
} from '@/components/ui/dropdown-menu'
import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip'
import { translate } from '@/i18n/i18n'
import type { BrowserPageAnnotation } from '../../../../../shared/browser-grab-types'
import { BrowserAnnotationSendMenuContent } from './BrowserAnnotationSendMenuContent'
import { preventAgentSendTargetOutsideDismiss } from './prevent-agent-send-target-outside-dismiss'
export function BrowserPageAnnotationTray({
browserAnnotations,
annotationTraySendOpen,
handleAnnotationTraySendOpenChange,
worktreeId,
activeGroupId,
browserAnnotationsPrompt,
handleBrowserAnnotationsSentToAgent,
handleCopyBrowserAnnotations,
browserAnnotationsCopied,
handleClearBrowserAnnotations,
handleDeleteBrowserAnnotation
}: {
browserAnnotations: BrowserPageAnnotation[]
annotationTraySendOpen: boolean
handleAnnotationTraySendOpenChange: (open: boolean) => void
worktreeId: string
activeGroupId: string | undefined
browserAnnotationsPrompt: string
handleBrowserAnnotationsSentToAgent: () => void
handleCopyBrowserAnnotations: () => void
browserAnnotationsCopied: boolean
handleClearBrowserAnnotations: () => void
handleDeleteBrowserAnnotation: (annotationId: string) => void
}): React.JSX.Element {
return (
<div className="absolute right-3 bottom-3 z-30 flex max-h-[45%] w-[min(20rem,calc(100%-1.5rem))] flex-col overflow-hidden rounded-lg border border-border bg-popover text-popover-foreground shadow-[0_10px_24px_rgba(0,0,0,0.18)]">
<div className="flex items-center gap-2 border-b border-border px-3 py-2">
<MessageSquarePlus className="size-4 text-muted-foreground" />
<div className="min-w-0 flex-1 text-sm font-medium">
{browserAnnotations.length === 1
? translate(
'auto.components.browser.pane.BrowserPane.ea6af700da',
'{{value0}} annotation',
{ value0: browserAnnotations.length }
)
: translate(
'auto.components.browser.pane.BrowserPane.c13693fe27',
'{{value0}} annotations',
{ value0: browserAnnotations.length }
)}
</div>
<DropdownMenu
modal={false}
open={annotationTraySendOpen}
onOpenChange={handleAnnotationTraySendOpenChange}
>
<Tooltip>
<TooltipTrigger asChild>
<DropdownMenuTrigger asChild>
<Button size="xs" variant="outline" className="gap-1.5">
<Send className="size-3" />
{translate('auto.components.browser.pane.BrowserPane.ac39b9366b', 'Send')}
</Button>
</DropdownMenuTrigger>
</TooltipTrigger>
<TooltipContent side="bottom" sideOffset={6}>
{translate(
'auto.components.browser.pane.BrowserPane.95af781091',
'Send feedback to an agent'
)}
</TooltipContent>
</Tooltip>
<DropdownMenuContent
align="end"
className="min-w-[180px]"
onInteractOutside={preventAgentSendTargetOutsideDismiss}
onPointerDownOutside={preventAgentSendTargetOutsideDismiss}
>
<BrowserAnnotationSendMenuContent
worktreeId={worktreeId}
groupId={activeGroupId ?? worktreeId}
prompt={browserAnnotationsPrompt}
onPromptDelivered={handleBrowserAnnotationsSentToAgent}
/>
</DropdownMenuContent>
</DropdownMenu>
<Button
size="xs"
variant="outline"
className="gap-1.5"
onClick={handleCopyBrowserAnnotations}
>
{browserAnnotationsCopied ? (
<CircleCheck className="size-3" />
) : (
<Copy className="size-3" />
)}
{browserAnnotationsCopied
? translate('auto.components.browser.pane.BrowserPane.6f4ab3592b', 'Copied')
: translate('auto.components.browser.pane.BrowserPane.d51ef37351', 'Copy')}
</Button>
<Tooltip>
<TooltipTrigger asChild>
<Button
size="icon-xs"
variant="ghost"
className="text-muted-foreground hover:text-foreground"
onClick={handleClearBrowserAnnotations}
aria-label={translate(
'auto.components.browser.pane.BrowserPane.734e4343ec',
'Clear browser annotations'
)}
>
<Trash2 className="size-3" />
</Button>
</TooltipTrigger>
<TooltipContent side="bottom" sideOffset={6}>
{translate('auto.components.browser.pane.BrowserPane.11c5084aa2', 'Clear annotations')}
</TooltipContent>
</Tooltip>
</div>
<div className="scrollbar-sleek min-h-0 flex-1 overflow-auto p-1.5">
{browserAnnotations.map((annotation, index) => (
<div
key={annotation.id}
className="group flex gap-2 rounded-md px-2 py-1.5 text-xs hover:bg-accent focus-within:bg-accent"
>
<div className="mt-0.5 flex size-5 shrink-0 items-center justify-center rounded-full bg-primary text-[10px] font-semibold text-primary-foreground">
{index + 1}
</div>
<div className="min-w-0 flex-1">
<div className="truncate font-medium text-foreground">
{annotation.payload.target.accessibility.accessibleName ||
annotation.payload.target.textSnippet ||
annotation.payload.target.tagName}
</div>
<div className="mt-0.5 line-clamp-2 text-muted-foreground">{annotation.comment}</div>
<div className="mt-1 text-[11px] text-muted-foreground">
<span>{annotation.intent}</span>
</div>
</div>
<Button
size="icon-xs"
variant="ghost"
className="can-hover:opacity-0 transition-opacity group-hover:opacity-100 focus-visible:opacity-100 group-focus-within:opacity-100"
onClick={() => handleDeleteBrowserAnnotation(annotation.id)}
aria-label={translate(
'auto.components.browser.pane.BrowserPane.f2d0c22d67',
'Delete annotation {{value0}}',
{ value0: index + 1 }
)}
>
<Trash2 className="size-3" />
</Button>
</div>
))}
</div>
</div>
)
}
@@ -0,0 +1,103 @@
import type { MutableRefObject } from 'react'
import type {
BrowserGrabPayload,
BrowserGrabScreenshot
} from '../../../../../shared/browser-grab-types'
import { formatGrabPayloadAsText } from './GrabConfirmationSheet'
import type { GrabModeHook } from './useGrabMode'
import type { BrowserPageGrabToastState, GrabIntent } from '../describe-page/browser-page-types'
export function runBrowserGrabActionShortcut({
key,
grabIntent,
grab,
grabPayloadRef,
browserTabIdRef,
recordFeatureInteraction,
showGrabToast
}: {
key: 'c' | 's'
grabIntent: GrabIntent
grab: GrabModeHook
grabPayloadRef: MutableRefObject<BrowserGrabPayload | null>
browserTabIdRef: MutableRefObject<string>
recordFeatureInteraction: (feature: 'browser-grab') => void | Promise<void>
showGrabToast: (
message: string,
type: BrowserPageGrabToastState['type'],
payload?: BrowserGrabPayload | null
) => void
}): void {
if (grabIntent === 'annotate') {
return
}
const copyFromPayload = (payload: BrowserGrabPayload): void => {
if (key === 'c') {
const text = formatGrabPayloadAsText(payload)
void window.api.ui.writeClipboardText(text)
recordFeatureInteraction('browser-grab')
showGrabToast('Copied', 'success', payload)
} else {
const dataUrl = payload.screenshot?.dataUrl
if (dataUrl?.startsWith('data:image/png;base64,')) {
void window.api.ui.writeClipboardImage(dataUrl)
recordFeatureInteraction('browser-grab')
showGrabToast('Screenshotted', 'success', payload)
} else {
showGrabToast('No screenshot available', 'error', payload)
}
}
}
if (grab.state === 'confirming') {
// Why: right-click (contextMenu) skips the left-click auto-copy, so C must still work here.
if (grab.contextMenu && key === 'c') {
const currentPayload = grabPayloadRef.current
if (currentPayload) {
copyFromPayload(currentPayload)
}
grab.rearm()
} else if (key === 's') {
const currentPayload = grabPayloadRef.current
if (currentPayload) {
copyFromPayload(currentPayload)
}
grab.rearm()
}
} else {
// armed/awaiting — extract hovered element via IPC without clicking
void (async () => {
let result: Awaited<ReturnType<typeof window.api.browser.extractHoverPayload>>
try {
result = await window.api.browser.extractHoverPayload({
browserPageId: browserTabIdRef.current
})
} catch {
// Why: the guest can be destroyed or the IPC channel torn down mid-shortcut; surface it like a miss instead of an unhandled rejection.
showGrabToast('Could not read the hovered element', 'error')
return
}
if (!result.ok) {
showGrabToast('No element hovered', 'error')
return
}
const payload = result.payload as BrowserGrabPayload
if (key === 's') {
try {
const ssResult = await window.api.browser.captureSelectionScreenshot({
browserPageId: browserTabIdRef.current,
rect: payload.target.rectViewport
})
if (ssResult.ok) {
payload.screenshot = ssResult.screenshot as BrowserGrabScreenshot
}
} catch {
// Screenshot failure is non-fatal for the copy flow
}
}
copyFromPayload(payload)
})()
}
}
@@ -0,0 +1,105 @@
import type { Dispatch, MutableRefObject, SetStateAction } from 'react'
import { CircleCheck, Image, OctagonX } from 'lucide-react'
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuShortcut,
DropdownMenuTrigger
} from '@/components/ui/dropdown-menu'
import { translate } from '@/i18n/i18n'
import type { BrowserPageGrabToastState } from '../describe-page/browser-page-types'
export function BrowserPageGrabToast({
grabToast,
grabToastTimerRef,
dismissGrabToast,
setGrabToast
}: {
grabToast: BrowserPageGrabToastState
grabToastTimerRef: MutableRefObject<ReturnType<typeof setTimeout> | undefined>
dismissGrabToast: () => void
setGrabToast: Dispatch<SetStateAction<BrowserPageGrabToastState | null>>
}): React.JSX.Element {
return (
<div
className="absolute z-30 flex items-center animate-in fade-in zoom-in-95 duration-150"
style={{
left: grabToast.x,
top: grabToast.y,
transform: grabToast.below
? 'translate(-50%, 8px)'
: 'translate(-50%, -100%) translateY(-8px)',
flexDirection: grabToast.below ? 'column' : 'column-reverse'
}}
>
{/* Caret pointing toward the element */}
<div
className="h-2 w-4 shrink-0"
style={{
clipPath: grabToast.below
? 'polygon(50% 0%, 0% 100%, 100% 100%)'
: 'polygon(0% 0%, 100% 0%, 50% 100%)',
background: 'white'
}}
/>
<div
className={`flex items-center gap-1.5 rounded-full py-1.5 pl-3 pr-1.5 shadow-lg ${
grabToast.type === 'success' ? 'bg-white text-gray-900' : 'bg-white text-red-600'
}`}
>
{grabToast.type === 'success' ? (
<CircleCheck className="size-4 fill-blue-600 text-white" />
) : (
<OctagonX className="size-4 text-red-500" />
)}
<span className="text-sm font-semibold">{grabToast.message}</span>
{grabToast.payload?.screenshot?.dataUrl?.startsWith('data:image/png;base64,') ? (
<DropdownMenu
onOpenChange={(open) => {
if (open) {
clearTimeout(grabToastTimerRef.current)
} else {
grabToastTimerRef.current = setTimeout(() => dismissGrabToast(), 1200)
}
}}
>
<DropdownMenuTrigger asChild>
<button className="flex size-6 items-center justify-center rounded-full text-gray-500 transition-colors hover:bg-black/10 hover:text-gray-700">
<span className="text-sm font-bold leading-none">···</span>
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" sideOffset={4}>
<DropdownMenuItem
onSelect={() => {
const dataUrl = grabToast.payload?.screenshot?.dataUrl
if (dataUrl?.startsWith('data:image/png;base64,')) {
void window.api.ui.writeClipboardImage(dataUrl)
setGrabToast((prev) =>
prev
? {
...prev,
message: translate(
'auto.components.browser.pane.BrowserPane.f30d2d35a7',
'Screenshotted'
)
}
: null
)
}
}}
>
<Image className="size-3.5" />
{translate(
'auto.components.browser.pane.BrowserPane.1ded0d3168',
'Copy Screenshot'
)}
<DropdownMenuShortcut>S</DropdownMenuShortcut>
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
) : null}
</div>
</div>
)
}
@@ -0,0 +1,166 @@
import { useState } from 'react'
import { CornerDownLeft, MessageSquarePlus } from 'lucide-react'
import { Button } from '@/components/ui/button'
import { ToggleGroup, ToggleGroupItem } from '@/components/ui/toggle-group'
import { Label } from '@/components/ui/label'
import { Popover, PopoverAnchor, PopoverContent } from '@/components/ui/popover'
import { getScreenSubmitModifierLabel, isScreenSubmitShortcut } from '@/lib/screen-submit-shortcut'
import {
GRAB_BUDGET,
type BrowserAnnotationIntent,
type BrowserGrabPayload
} from '../../../../../shared/browser-grab-types'
import { translate } from '@/i18n/i18n'
import {
BROWSER_ANNOTATION_INTENT_OPTIONS,
type BrowserOverlayAnchor
} from '../describe-page/browser-annotation-geometry'
export function PendingBrowserAnnotationCard({
payload,
anchor,
portalContainer,
onAdd,
onCancel
}: {
payload: BrowserGrabPayload
anchor: BrowserOverlayAnchor
portalContainer: HTMLElement | null
onAdd: (comment: string, intent: BrowserAnnotationIntent) => void
onCancel: () => void
}): React.JSX.Element {
const [comment, setComment] = useState('')
const [intent, setIntent] = useState<BrowserAnnotationIntent>('change')
const trimmed = comment.trim()
const submitModifierLabel = getScreenSubmitModifierLabel()
return (
<Popover
open
onOpenChange={(open) => {
if (!open) {
onCancel()
}
}}
>
<PopoverAnchor asChild>
<span
className="pointer-events-none absolute size-px"
style={{ left: anchor.x, top: anchor.y }}
/>
</PopoverAnchor>
<PopoverContent
side={anchor.below ? 'bottom' : 'top'}
align="center"
sideOffset={10}
collisionBoundary={portalContainer ?? undefined}
collisionPadding={12}
portalContainer={portalContainer}
className="z-40 w-[22rem] max-w-[calc(var(--radix-popover-content-available-width)-1rem)] p-3 shadow-[0_10px_24px_rgba(0,0,0,0.18)]"
aria-label={translate(
'auto.components.browser.pane.BrowserPane.b472c5fe03',
'Add browser annotation'
)}
onEscapeKeyDown={(event) => {
event.preventDefault()
onCancel()
}}
>
<div className="mb-2 min-w-0">
<div className="truncate text-xs font-medium text-foreground">
{payload.target.accessibility.accessibleName ||
payload.target.textSnippet ||
payload.target.tagName}
</div>
<div className="mt-0.5 truncate font-mono text-[11px] text-muted-foreground">
{payload.target.selector}
</div>
</div>
<Label htmlFor="browser-annotation-comment" className="sr-only">
{translate('auto.components.browser.pane.BrowserPane.d2a7092e6e', 'Annotation comment')}
</Label>
<textarea
id="browser-annotation-comment"
value={comment}
onChange={(event) => setComment(event.target.value)}
placeholder={translate(
'auto.components.browser.pane.BrowserPane.532bac48c5',
'Describe what the agent should change here...'
)}
maxLength={GRAB_BUDGET.annotationCommentMaxLength}
className="h-24 w-full resize-none rounded-md border border-input bg-background px-3 py-2 text-sm outline-none ring-offset-background placeholder:text-muted-foreground focus-visible:ring-2 focus-visible:ring-ring"
autoFocus
onKeyDown={(event) => {
if (event.key === 'Escape') {
event.preventDefault()
event.stopPropagation()
onCancel()
return
}
if (isScreenSubmitShortcut(event)) {
event.preventDefault()
event.stopPropagation()
if (trimmed) {
onAdd(trimmed, intent)
}
}
}}
/>
<div className="mt-2 min-w-0">
<Label className="mb-1 block text-xs text-muted-foreground">
{translate('auto.components.browser.pane.BrowserPane.8f87e6c2e5', 'Intent')}
</Label>
<ToggleGroup
type="single"
size="sm"
variant="outline"
value={intent}
onValueChange={(value) => {
if (value) {
setIntent(value as BrowserAnnotationIntent)
}
}}
className="h-8 w-full [&_[data-slot=toggle-group-item]]:h-8 [&_[data-slot=toggle-group-item]]:flex-1 [&_[data-slot=toggle-group-item]]:px-2"
aria-label={translate(
'auto.components.browser.pane.BrowserPane.0cb3bd6221',
'Annotation intent'
)}
>
{BROWSER_ANNOTATION_INTENT_OPTIONS.map((option) => {
const Icon = option.icon
return (
<ToggleGroupItem
key={option.value}
value={option.value}
aria-label={option.label}
className="gap-1.5 text-xs data-[state=on]:border-foreground/20 data-[state=on]:bg-foreground/10 data-[state=on]:text-foreground data-[state=on]:shadow-xs data-[state=on]:hover:bg-foreground/15 data-[state=on]:hover:text-foreground"
>
<Icon className="size-3.5" />
<span>{option.label}</span>
</ToggleGroupItem>
)
})}
</ToggleGroup>
</div>
<div className="mt-3 flex justify-end gap-2">
<Button size="sm" variant="ghost" className="h-8" onClick={onCancel}>
{translate('auto.components.browser.pane.BrowserPane.fa6ea61de3', 'Cancel')}
</Button>
<Button
size="sm"
className="h-8 gap-1.5"
disabled={!trimmed}
onClick={() => onAdd(trimmed, intent)}
>
<MessageSquarePlus className="size-3.5" />
{translate('auto.components.browser.pane.BrowserPane.90d021f2ad', 'Add')}
<span className="ml-1 inline-flex items-center gap-0.5 rounded border border-white/20 px-1.5 py-0.5 text-[10px] font-medium leading-none text-current/80">
<span>{submitModifierLabel}</span>
<CornerDownLeft className="size-3" />
</span>
</Button>
</div>
</PopoverContent>
</Popover>
)
}
@@ -0,0 +1,13 @@
export function preventAgentSendTargetOutsideDismiss(event: CustomEvent<{ originalEvent: Event }>) {
const target = event.detail.originalEvent.target
if (!(target instanceof Element)) {
return
}
if (
target.closest(
'[data-agent-send-target="eligible"], [data-agent-send-target="disabled"], [data-agent-send-target="sending"]'
)
) {
event.preventDefault()
}
}
@@ -0,0 +1,193 @@
import {
useCallback,
useEffect,
useLayoutEffect,
useMemo,
useRef,
useState,
type Dispatch,
type SetStateAction
} from 'react'
import { useAppStore } from '@/store'
import { translate } from '@/i18n/i18n'
import type { BrowserPageAnnotation } from '../../../../../shared/browser-grab-types'
import { formatBrowserAnnotationsAsMarkdown } from './browser-annotation-output'
import { EMPTY_BROWSER_ANNOTATIONS } from '../describe-page/browser-annotation-geometry'
export function useBrowserPageAnnotationSend({
browserTabId,
worktreeId
}: {
browserTabId: string
worktreeId: string
}): {
browserAnnotations: BrowserPageAnnotation[]
browserAnnotationsPrompt: string
browserAnnotationTrayOpen: boolean
setBrowserAnnotationTrayOpen: Dispatch<SetStateAction<boolean>>
browserAnnotationsCopied: boolean
annotationBannerSendOpen: boolean
annotationTraySendOpen: boolean
handleAnnotationBannerSendOpenChange: (open: boolean) => void
handleAnnotationTraySendOpenChange: (open: boolean) => void
handleCopyBrowserAnnotations: () => void
handleClearBrowserAnnotations: () => void
handleDeleteBrowserAnnotation: (annotationId: string) => void
handleBrowserAnnotationsSentToAgent: () => void
activeGroupId: string | undefined
} {
const browserAnnotations = useAppStore(
(s) => s.browserAnnotationsByPageId[browserTabId] ?? EMPTY_BROWSER_ANNOTATIONS
)
const activeGroupId = useAppStore((s) => s.activeGroupIdByWorktree[worktreeId])
const browserAnnotationsRef = useRef(browserAnnotations)
const [browserAnnotationTrayOpen, setBrowserAnnotationTrayOpen] = useState(true)
const [browserAnnotationsCopied, setBrowserAnnotationsCopied] = useState(false)
const annotationCopyTimerRef = useRef<ReturnType<typeof setTimeout>>(undefined)
const browserAnnotationsPrompt = useMemo(
() => formatBrowserAnnotationsAsMarkdown(browserAnnotations),
[browserAnnotations]
)
const openAgentSendPopoverTargetMode = useAppStore((s) => s.openAgentSendPopoverTargetMode)
const closeAgentSendPopoverTargetMode = useAppStore((s) => s.closeAgentSendPopoverTargetMode)
const activeAgentSendTargetModeId = useAppStore((s) => s.agentSendPopoverTargetMode?.id ?? null)
const annotationBannerSendModeId = `browser-annotations:${browserTabId}:banner`
const annotationTraySendModeId = `browser-annotations:${browserTabId}:tray`
const annotationBannerSendOpen = activeAgentSendTargetModeId === annotationBannerSendModeId
const annotationTraySendOpen = activeAgentSendTargetModeId === annotationTraySendModeId
const deleteBrowserPageAnnotation = useAppStore((s) => s.deleteBrowserPageAnnotation)
const clearBrowserPageAnnotations = useAppStore((s) => s.clearBrowserPageAnnotations)
const recordFeatureInteraction = useAppStore((s) => s.recordFeatureInteraction)
useLayoutEffect(() => {
browserAnnotationsRef.current = browserAnnotations
}, [browserAnnotations])
useEffect(() => {
return () => {
clearTimeout(annotationCopyTimerRef.current)
}
}, [])
const handleAnnotationBannerSendOpenChange = useCallback(
(open: boolean): void => {
if (open) {
openAgentSendPopoverTargetMode({
id: annotationBannerSendModeId,
worktreeId,
source: 'browser-annotations',
prompt: browserAnnotationsPrompt,
label: translate(
'auto.components.browser.pane.BrowserPane.27d863542c',
'Browser annotations'
),
launchSource: 'notes_send'
})
} else {
closeAgentSendPopoverTargetMode(annotationBannerSendModeId)
}
},
[
annotationBannerSendModeId,
browserAnnotationsPrompt,
closeAgentSendPopoverTargetMode,
openAgentSendPopoverTargetMode,
worktreeId
]
)
const handleAnnotationTraySendOpenChange = useCallback(
(open: boolean): void => {
if (open) {
openAgentSendPopoverTargetMode({
id: annotationTraySendModeId,
worktreeId,
source: 'browser-annotations',
prompt: browserAnnotationsPrompt,
label: translate(
'auto.components.browser.pane.BrowserPane.27d863542c',
'Browser annotations'
),
launchSource: 'notes_send'
})
} else {
closeAgentSendPopoverTargetMode(annotationTraySendModeId)
}
},
[
annotationTraySendModeId,
browserAnnotationsPrompt,
closeAgentSendPopoverTargetMode,
openAgentSendPopoverTargetMode,
worktreeId
]
)
useEffect(
() => () => {
closeAgentSendPopoverTargetMode(annotationBannerSendModeId)
closeAgentSendPopoverTargetMode(annotationTraySendModeId)
},
[annotationBannerSendModeId, annotationTraySendModeId, closeAgentSendPopoverTargetMode]
)
const handleCopyBrowserAnnotations = useCallback((): void => {
if (!browserAnnotationsPrompt) {
return
}
void window.api.ui.writeClipboardText(browserAnnotationsPrompt)
recordFeatureInteraction('browser-annotations')
clearTimeout(annotationCopyTimerRef.current)
setBrowserAnnotationsCopied(true)
annotationCopyTimerRef.current = setTimeout(() => setBrowserAnnotationsCopied(false), 1400)
}, [browserAnnotationsPrompt, recordFeatureInteraction])
const handleBrowserAnnotationsSentToAgent = useCallback((): void => {
recordFeatureInteraction('browser-annotations-sent-to-agent')
}, [recordFeatureInteraction])
const handleClearBrowserAnnotations = useCallback((): void => {
if (browserAnnotationsRef.current.length === 0) {
return
}
clearTimeout(annotationCopyTimerRef.current)
setBrowserAnnotationsCopied(false)
recordFeatureInteraction('browser-annotations')
clearBrowserPageAnnotations(browserTabId)
}, [browserTabId, clearBrowserPageAnnotations, recordFeatureInteraction])
const handleDeleteBrowserAnnotation = useCallback(
(annotationId: string): void => {
if (browserAnnotationsRef.current.length === 1) {
clearTimeout(annotationCopyTimerRef.current)
setBrowserAnnotationsCopied(false)
setBrowserAnnotationTrayOpen(true)
}
deleteBrowserPageAnnotation(browserTabId, annotationId)
recordFeatureInteraction('browser-annotations')
},
[
browserTabId,
deleteBrowserPageAnnotation,
recordFeatureInteraction,
setBrowserAnnotationTrayOpen
]
)
return {
browserAnnotations,
browserAnnotationsPrompt,
browserAnnotationTrayOpen,
setBrowserAnnotationTrayOpen,
browserAnnotationsCopied,
annotationBannerSendOpen,
annotationTraySendOpen,
handleAnnotationBannerSendOpenChange,
handleAnnotationTraySendOpenChange,
handleCopyBrowserAnnotations,
handleClearBrowserAnnotations,
handleDeleteBrowserAnnotation,
handleBrowserAnnotationsSentToAgent,
activeGroupId
}
}
@@ -0,0 +1,320 @@
import {
useCallback,
useEffect,
useLayoutEffect,
useRef,
useState,
type Dispatch,
type MutableRefObject,
type SetStateAction
} from 'react'
import { translate } from '@/i18n/i18n'
import { useAppStore } from '@/store'
import type {
BrowserAnnotationIntent,
BrowserGrabPayload
} from '../../../../../shared/browser-grab-types'
import { formatGrabPayloadAsText } from './GrabConfirmationSheet'
import type { GrabModeHook } from './useGrabMode'
import {
createBrowserAnnotationId,
createBrowserAnnotationPayload,
DEFAULT_BROWSER_ANNOTATION_PRIORITY,
type BrowserOverlayViewport
} from '../describe-page/browser-annotation-geometry'
import { runBrowserGrabActionShortcut } from './browser-page-grab-action'
import type { BrowserPageGrabToastState, GrabIntent } from '../describe-page/browser-page-types'
const copiedGrabToastMessage = (): string =>
translate(
'auto.components.browser.pane.annotate.use.browser.page.grab.annotations.0c7b9b2b7a',
'Copied'
)
const screenshottedGrabToastMessage = (): string =>
translate(
'auto.components.browser.pane.annotate.use.browser.page.grab.annotations.c937229f19',
'Screenshotted'
)
const annotationAddedGrabToastMessage = (): string =>
translate(
'auto.components.browser.pane.annotate.use.browser.page.grab.annotations.1f5cb19034',
'Annotation added'
)
export function useBrowserPageGrabAnnotations({
browserTabId,
isActive,
grab,
containerRef,
webviewRef,
setBrowserOverlayViewport,
browserAnnotationsLength,
setBrowserAnnotationTrayOpen
}: {
browserTabId: string
isActive: boolean
grab: GrabModeHook
containerRef: MutableRefObject<HTMLDivElement | null>
webviewRef: MutableRefObject<Electron.WebviewTag | null>
setBrowserOverlayViewport: Dispatch<SetStateAction<BrowserOverlayViewport>>
browserAnnotationsLength: number
setBrowserAnnotationTrayOpen: Dispatch<SetStateAction<boolean>>
}): {
grabIntent: GrabIntent
startGrabIntent: (nextIntent: GrabIntent) => void
pendingAnnotationPayload: BrowserGrabPayload | null
setPendingAnnotationPayload: Dispatch<SetStateAction<BrowserGrabPayload | null>>
grabToast: BrowserPageGrabToastState | null
setGrabToast: Dispatch<SetStateAction<BrowserPageGrabToastState | null>>
grabToastTimerRef: MutableRefObject<ReturnType<typeof setTimeout> | undefined>
dismissGrabToast: () => void
handleGrabCopy: () => void
handleGrabCopyScreenshot: () => void
grabMenuActionTakenRef: MutableRefObject<boolean>
handleAddBrowserAnnotation: (comment: string, intent: BrowserAnnotationIntent) => void
handleCancelPendingBrowserAnnotation: () => void
handleGrabActionShortcut: (key: 'c' | 's') => void
} {
const browserTabIdRef = useRef(browserTabId)
const grabToastTimerRef = useRef<ReturnType<typeof setTimeout>>(undefined)
const [grabIntent, setGrabIntent] = useState<GrabIntent>('copy')
const grabIntentRef = useRef(grabIntent)
const [pendingAnnotationPayload, setPendingAnnotationPayload] =
useState<BrowserGrabPayload | null>(null)
const pendingAnnotationPayloadRef = useRef<BrowserGrabPayload | null>(null)
// Inline toast near the grabbed element (below, or above near the viewport bottom) so it doesn't occlude the selection.
const [grabToast, setGrabToast] = useState<BrowserPageGrabToastState | null>(null)
const grabRef = useRef(grab)
const grabPayloadRef = useRef(grab.payload)
useLayoutEffect(() => {
browserTabIdRef.current = browserTabId
grabIntentRef.current = grabIntent
pendingAnnotationPayloadRef.current = pendingAnnotationPayload
grabRef.current = grab
grabPayloadRef.current = grab.payload
}, [browserTabId, grab, grabIntent, pendingAnnotationPayload])
// Why: Radix fires onOpenChange(false) before onSelect, so this flag lets onOpenChange skip the rearm that would clear the payload first.
const grabMenuActionTakenRef = useRef(false)
const recordFeatureInteraction = useAppStore((s) => s.recordFeatureInteraction)
const addBrowserPageAnnotation = useAppStore((s) => s.addBrowserPageAnnotation)
useEffect(() => {
return () => {
clearTimeout(grabToastTimerRef.current)
}
}, [])
const dismissGrabToast = useCallback(() => {
clearTimeout(grabToastTimerRef.current)
setGrabToast(null)
// Why: only rearm while 'confirming'; if a C/S shortcut already rearmed (state 'armed'), skip to avoid a double-rearm race.
if (
grabRef.current.state === 'confirming' &&
!(grabIntentRef.current === 'annotate' && pendingAnnotationPayloadRef.current)
) {
grabRef.current.rearm()
}
}, [])
const showGrabToast = useCallback(
(message: string, type: 'success' | 'error', payload?: BrowserGrabPayload | null) => {
let x = 0
let y = 0
let below = true
const containerRect = containerRef.current?.getBoundingClientRect()
if (payload) {
const rect = payload.target.rectViewport
const webview = webviewRef.current
const webviewRect = webview?.getBoundingClientRect()
const offsetX = (webviewRect?.left ?? 0) - (containerRect?.left ?? 0)
const offsetY = (webviewRect?.top ?? 0) - (containerRect?.top ?? 0)
x = offsetX + rect.x + rect.width / 2
const elementBottom = offsetY + rect.y + rect.height
const elementTop = offsetY + rect.y
const containerHeight = containerRect?.height ?? 0
// Show below the element unless it's too close to the bottom edge
below = elementBottom + 52 < containerHeight
y = below ? elementBottom : elementTop
} else if (containerRect) {
x = containerRect.width / 2
y = containerRect.height / 2
}
clearTimeout(grabToastTimerRef.current)
setGrabToast({ message, type, x, y, below, payload: payload ?? null })
grabToastTimerRef.current = setTimeout(() => dismissGrabToast(), 2000)
},
[containerRef, dismissGrabToast, webviewRef]
)
// Why: the same in-guest picker powers two flows — Cmd/Ctrl+C copies, the toolbar action creates a pending annotation.
useEffect(() => {
if (grab.state !== 'confirming' || !grab.payload) {
return
}
if (grabIntent === 'annotate') {
setPendingAnnotationPayload(grab.payload)
return
}
if (!grab.contextMenu) {
const text = formatGrabPayloadAsText(grab.payload)
void window.api.ui.writeClipboardText(text)
recordFeatureInteraction('browser-grab')
showGrabToast(copiedGrabToastMessage(), 'success', grab.payload)
}
}, [
grab.state,
grab.payload,
grab.contextMenu,
grabIntent,
recordFeatureInteraction,
showGrabToast
])
useEffect(() => {
if (!isActive || (!pendingAnnotationPayload && browserAnnotationsLength === 0)) {
return
}
const observedContainer = containerRef.current
const resizeObserver =
typeof ResizeObserver === 'undefined' || !observedContainer
? null
: new ResizeObserver(() => {
setBrowserOverlayViewport((current) => ({ ...current, version: current.version + 1 }))
})
if (resizeObserver && observedContainer) {
resizeObserver.observe(observedContainer)
}
return () => {
resizeObserver?.disconnect()
}
}, [
browserAnnotationsLength,
containerRef,
isActive,
pendingAnnotationPayload,
setBrowserOverlayViewport
])
const startGrabIntent = useCallback(
(nextIntent: GrabIntent): void => {
recordFeatureInteraction('browser-grab')
if (nextIntent === 'annotate') {
recordFeatureInteraction('browser-annotations')
}
setGrabIntent(nextIntent)
if (nextIntent === 'copy') {
setPendingAnnotationPayload(null)
} else {
setBrowserAnnotationTrayOpen(true)
}
if (grab.state === 'idle' || grab.state === 'error' || grabIntent === nextIntent) {
grab.toggle()
}
},
[grab, grabIntent, recordFeatureInteraction, setBrowserAnnotationTrayOpen]
)
// C / S copy the hovered element without clicking: extract via IPC while armed/awaiting, else use the captured payload.
const handleGrabActionShortcut = useCallback(
(key: 'c' | 's'): void => {
runBrowserGrabActionShortcut({
key,
grabIntent,
grab,
grabPayloadRef,
browserTabIdRef,
recordFeatureInteraction,
showGrabToast
})
},
[grab, grabIntent, recordFeatureInteraction, showGrabToast]
)
const handleGrabCopy = useCallback(() => {
grabMenuActionTakenRef.current = true
const payload = grabPayloadRef.current
if (!payload) {
return
}
const text = formatGrabPayloadAsText(payload)
void window.api.ui.writeClipboardText(text)
recordFeatureInteraction('browser-grab')
showGrabToast(copiedGrabToastMessage(), 'success', payload)
grab.rearm()
}, [grab, recordFeatureInteraction, showGrabToast])
const handleGrabCopyScreenshot = useCallback(() => {
grabMenuActionTakenRef.current = true
const payload = grabPayloadRef.current
if (!payload) {
return
}
const dataUrl = payload.screenshot?.dataUrl
if (!dataUrl?.startsWith('data:image/png;base64,')) {
return
}
void window.api.ui.writeClipboardImage(dataUrl)
recordFeatureInteraction('browser-grab')
showGrabToast(screenshottedGrabToastMessage(), 'success', payload)
grab.rearm()
}, [grab, recordFeatureInteraction, showGrabToast])
const handleAddBrowserAnnotation = useCallback(
(comment: string, intent: BrowserAnnotationIntent): void => {
const payload = pendingAnnotationPayload
if (!payload) {
return
}
addBrowserPageAnnotation({
id: createBrowserAnnotationId(),
browserPageId: browserTabId,
comment,
intent,
priority: DEFAULT_BROWSER_ANNOTATION_PRIORITY,
createdAt: new Date().toISOString(),
payload: createBrowserAnnotationPayload(payload)
})
recordFeatureInteraction('browser-annotations')
setPendingAnnotationPayload(null)
setBrowserAnnotationTrayOpen(true)
showGrabToast(annotationAddedGrabToastMessage(), 'success', payload)
grab.rearm()
},
[
addBrowserPageAnnotation,
browserTabId,
grab,
pendingAnnotationPayload,
recordFeatureInteraction,
setBrowserAnnotationTrayOpen,
showGrabToast
]
)
const handleCancelPendingBrowserAnnotation = useCallback((): void => {
setPendingAnnotationPayload(null)
if (grabIntent === 'annotate' && grab.state === 'confirming') {
grab.rearm()
}
}, [grab, grabIntent])
return {
grabIntent,
startGrabIntent,
pendingAnnotationPayload,
setPendingAnnotationPayload,
grabToast,
setGrabToast,
grabToastTimerRef,
dismissGrabToast,
handleGrabCopy,
handleGrabCopyScreenshot,
grabMenuActionTakenRef,
handleAddBrowserAnnotation,
handleCancelPendingBrowserAnnotation,
handleGrabActionShortcut
}
}
@@ -0,0 +1,33 @@
import { useCallback, type MutableRefObject } from 'react'
import { deliverMarkupToClipboard } from './markup-clipboard-delivery'
import {
useMarkupMode,
type MarkupCaptureContext,
type MarkupModeController
} from './useMarkupMode'
export function useBrowserPageMarkupCapture(
webviewRef: MutableRefObject<Electron.WebviewTag | null>,
containerRef: MutableRefObject<HTMLDivElement | null>
): MarkupModeController {
return useMarkupMode({
getCaptureContext: useCallback((): MarkupCaptureContext | null => {
const webview = webviewRef.current
const container = containerRef.current
if (!webview || !container) {
return null
}
const rect = container.getBoundingClientRect()
if (rect.width <= 0 || rect.height <= 0) {
return null
}
return {
source: { kind: 'webview', webview },
cssWidth: rect.width,
cssHeight: rect.height,
outputScale: window.devicePixelRatio || 1
}
}, [containerRef, webviewRef]),
onDeliver: deliverMarkupToClipboard
})
}
@@ -1,4 +1,4 @@
import { useCallback, useEffect, useState } from 'react'
import { useCallback, useRef, useSyncExternalStore } from 'react'
import { useAppStore } from '@/store'
// One-time discovery highlight for the screenshot-markup Draw button. Shows once
@@ -30,22 +30,32 @@ export type MarkupDrawHint = { hintOpen: boolean; dismissHint: () => void }
export function useMarkupDrawHint(eligible: boolean): MarkupDrawHint {
const persistedUIReady = useAppStore((state) => state.persistedUIReady)
const [hintOpen, setHintOpen] = useState(false)
useEffect(() => {
// Why: only nudge once the app is ready and the button is usable on a
// visible surface. If eligibility drops mid-hint (tab switch, grab
// started, markup open, blank tab), close it so a forced-open floating
// layer can't stick over a hidden or disabled control at (0,0).
if (!persistedUIReady || !eligible) {
setHintOpen(false)
return
}
if (claimFirstView()) {
setHintOpen(true)
}
}, [eligible, persistedUIReady])
const dismissHint = useCallback(() => setHintOpen(false), [])
const hintOpenRef = useRef(false)
const notifyRef = useRef<() => void>(() => {})
const subscribe = useCallback(
(notify: () => void): (() => void) => {
notifyRef.current = notify
const nextOpen = persistedUIReady && eligible && (hintOpenRef.current || claimFirstView())
if (hintOpenRef.current !== nextOpen) {
hintOpenRef.current = nextOpen
notify()
}
return () => {
if (notifyRef.current === notify) {
notifyRef.current = () => {}
}
}
},
[eligible, persistedUIReady]
)
const getSnapshot = useCallback(
() => hintOpenRef.current && persistedUIReady && eligible,
[eligible, persistedUIReady]
)
const hintOpen = useSyncExternalStore(subscribe, getSnapshot, () => false)
const dismissHint = useCallback(() => {
hintOpenRef.current = false
notifyRef.current()
}, [])
return { hintOpen, dismissHint }
}
@@ -19,6 +19,7 @@ function createReactHookHarness() {
useEffect: (effect: () => void | (() => void), deps?: readonly unknown[]) => {
effects.push({ effect, deps })
},
useLayoutEffect: () => {},
useRef: <T>(initialValue: T): { current: T } => {
const index = refIndex
refIndex += 1
@@ -1,12 +1,12 @@
import { useCallback, useEffect, useRef, useState } from 'react'
import { useCallback, useEffect, useLayoutEffect, useRef, useState } from 'react'
import type {
BrowserGrabPayload,
BrowserGrabRejectReason,
BrowserGrabScreenshot
} from '../../../../shared/browser-grab-types'
} from '../../../../../shared/browser-grab-types'
import { useMountedRef } from '@/hooks/useMountedRef'
import { translate } from '@/i18n/i18n'
import { isEditableKeyboardTarget } from './browser-keyboard'
import { isEditableKeyboardTarget } from '../host-guest/browser-keyboard'
// ---------------------------------------------------------------------------
// Grab mode state machine
@@ -74,15 +74,15 @@ export function useGrabMode(browserPageId: string): GrabModeHook {
const grabTabIdRef = useRef<string | null>(null)
const armGenerationRef = useRef(0)
const browserTabIdRef = useRef(browserPageId)
// Why: toolbar/key handlers from the latest render can fire before passive
// effects run after a page switch, so keep the target page current in render.
browserTabIdRef.current = browserPageId
const mountedRef = useMountedRef()
useLayoutEffect(() => {
browserTabIdRef.current = browserPageId
}, [browserPageId])
// Why: when the browser page changes while grab is active, cancel the
// current grab operation so stale overlays don't survive tab switches.
useEffect(() => {
browserTabIdRef.current = browserPageId
return () => {
const grabTabId = grabTabIdRef.current
if (grabTabId) {
@@ -96,7 +96,7 @@ export function useGrabMode(browserPageId: string): GrabModeHook {
}, [browserPageId])
const armAndAwait = useCallback(async () => {
const tabId = browserTabIdRef.current
const tabId = browserPageId
const armGeneration = (armGenerationRef.current += 1)
grabTabIdRef.current = tabId
setState('armed')
@@ -181,10 +181,10 @@ export function useGrabMode(browserPageId: string): GrabModeHook {
setState('error')
setError(result.reason)
}
}, [mountedRef])
}, [browserPageId, mountedRef])
const cancel = useCallback(() => {
const targetTabId = grabTabIdRef.current ?? browserTabIdRef.current
const targetTabId = grabTabIdRef.current ?? browserPageId
armGenerationRef.current += 1
void window.api.browser.setGrabMode({
browserPageId: targetTabId,
@@ -201,7 +201,7 @@ export function useGrabMode(browserPageId: string): GrabModeHook {
setPayload(null)
setError(null)
setContextMenu(false)
}, [])
}, [browserPageId])
const toggle = useCallback(() => {
if ((state === 'idle' || state === 'error') && grabTabIdRef.current === null) {
@@ -124,6 +124,7 @@ export function useMarkupEditor(busy: boolean, onCancel: () => void) {
color,
width,
pendingText,
inProgress,
canvasRef,
setInProgress,
setPendingText,
@@ -24,6 +24,7 @@ function baseParams(overrides: Partial<MarkupPointerParams> = {}): MarkupPointer
color: '#ef4444',
width: 4,
pendingText: null,
inProgress: null,
canvasRef: {
current: {
getBoundingClientRect: () => ({ left: 0, top: 0 })
@@ -61,3 +62,40 @@ describe('useMarkupPointerHandlers in a non-secure browser context', () => {
expect(shape.id).toMatch(/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[0-9a-f]{4}-[0-9a-f]{12}$/)
})
})
describe('useMarkupPointerHandlers pointer up', () => {
const inProgress: MarkupShape = {
id: 'shape-1',
kind: 'pen',
color: '#ef4444',
width: 4,
points: [{ x: 0, y: 0 }]
} as MarkupShape
it('commits outside the setInProgress updater so a double-invoked updater cannot duplicate the shape', () => {
const setInProgress = vi.fn()
const setDoc = vi.fn()
const { result } = renderHook(() =>
useMarkupPointerHandlers(baseParams({ inProgress, setInProgress, setDoc }))
)
act(() => result.current.onPointerUp())
expect(setDoc).toHaveBeenCalledTimes(1)
// The clear must be a plain value, not an updater that also commits.
expect(setInProgress).toHaveBeenCalledWith(null)
})
it('does not commit when no shape is in progress', () => {
const setInProgress = vi.fn()
const setDoc = vi.fn()
const { result } = renderHook(() =>
useMarkupPointerHandlers(baseParams({ inProgress: null, setInProgress, setDoc }))
)
act(() => result.current.onPointerUp())
expect(setDoc).not.toHaveBeenCalled()
expect(setInProgress).toHaveBeenCalledWith(null)
})
})
@@ -16,6 +16,7 @@ export type MarkupPointerParams = {
color: string
width: number
pendingText: PendingText | null
inProgress: MarkupShape | null
canvasRef: React.RefObject<HTMLCanvasElement | null>
setInProgress: React.Dispatch<React.SetStateAction<MarkupShape | null>>
setPendingText: (value: PendingText | null) => void
@@ -31,6 +32,7 @@ export function useMarkupPointerHandlers(params: MarkupPointerParams) {
color,
width,
pendingText,
inProgress,
canvasRef,
setInProgress,
setPendingText,
@@ -96,14 +98,14 @@ export function useMarkupPointerHandlers(params: MarkupPointerParams) {
[pointFromEvent, setInProgress]
)
// Why: committing inside the setInProgress updater made it impure, so StrictMode's
// double-invoke appended the shape twice (commitShape does not dedupe by id).
const onPointerUp = useCallback(() => {
setInProgress((current) => {
if (current) {
setDoc((document) => commitShape(document, current))
}
return null
})
}, [setDoc, setInProgress])
if (inProgress) {
setDoc((document) => commitShape(document, inProgress))
}
setInProgress(null)
}, [inProgress, setDoc, setInProgress])
return { onPointerDown, onPointerMove, onPointerUp }
}
@@ -3,7 +3,7 @@
import { act, type ReactNode, useRef, useState } from 'react'
import { createRoot, type Root } from 'react-dom/client'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import type { BrowserHistoryEntry } from '../../../../shared/browser-workspace-types'
import type { BrowserHistoryEntry } from '../../../../../shared/browser-workspace-types'
import BrowserAddressBar from './BrowserAddressBar'
const mocks = vi.hoisted(() => ({
@@ -1,11 +1,10 @@
/* oxlint-disable react-doctor/no-adjust-state-on-prop-change -- Why: dropdown visibility depends on DOM focus plus browser-history suggestions, so the close path is an imperative popover sync. */
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { Globe } from 'lucide-react'
import { Input } from '@/components/ui/input'
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover'
import { cn } from '@/lib/utils'
import { useAppStore } from '@/store'
import { DEFAULT_SEARCH_ENGINE, type SearchEngine } from '../../../../shared/browser-url'
import { DEFAULT_SEARCH_ENGINE, type SearchEngine } from '../../../../../shared/browser-url'
import { buildBrowserAddressBarSuggestions } from './browser-address-bar-suggestions'
import { shouldOverlayBrowserAddressBar } from './browser-address-bar-expansion'
import BrowserAddressBarSuggestionList from './BrowserAddressBarSuggestionList'
@@ -29,10 +28,10 @@ export default function BrowserAddressBar({
}: BrowserAddressBarProps): React.ReactElement {
const [open, setOpen] = useState(false)
const [selectedValueOverride, setSelectedValueOverride] = useState<string | null>(null)
const prePreviewValueRef = useRef<string | null>(null)
// Why: while previewing a highlighted suggestion the input shows the full URL,
// but suggestions must keep matching the original typed query.
const [autocompleteQuery, setAutocompleteQuery] = useState(value)
const prePreviewValueRef = useRef<string | null>(null)
const autocompleteQuery = prePreviewValueRef.current ?? value
const browserUrlHistory = useAppStore((s) => s.browserUrlHistory)
const browserDefaultSearchEngine = useAppStore((s) => s.browserDefaultSearchEngine)
const browserKagiSessionLink = useAppStore((s) => s.browserKagiSessionLink)
@@ -93,20 +92,6 @@ export default function BrowserAddressBar({
[browserUrlHistory, autocompleteQuery, searchEngine, browserKagiSessionLink]
)
useEffect(() => {
if (prePreviewValueRef.current === null) {
setAutocompleteQuery(value)
}
}, [value])
useEffect(() => {
if (open) {
return
}
prePreviewValueRef.current = null
setSelectedValueOverride(null)
}, [open])
const clearSuggestionPreview = useCallback((): void => {
prePreviewValueRef.current = null
setSelectedValueOverride(null)
@@ -149,7 +134,6 @@ export default function BrowserAddressBar({
}
prePreviewValueRef.current = null
setSelectedValueOverride(null)
setAutocompleteQuery(typed)
onChange(typed)
}, [onChange])
@@ -240,7 +224,6 @@ export default function BrowserAddressBar({
event.preventDefault()
setOpen(false)
clearSuggestionPreview()
setAutocompleteQuery(value)
onSubmit()
return
}
@@ -287,25 +270,10 @@ export default function BrowserAddressBar({
restoreTypedQuery,
cancelSuggestionPreview,
clearSuggestionPreview,
onSubmit,
value
onSubmit
]
)
// Why: close the dropdown only when the input has lost focus AND there are
// no suggestions. Previously this closed unconditionally on empty suggestions,
// which caused the dropdown to vanish mid-typing when backspacing produced a
// query that didn't match any history entries. Keeping the popover open while
// focused lets the user continue editing and see results reappear.
useEffect(() => {
if (open && suggestions.length === 0) {
if (inputRef.current && document.activeElement === inputRef.current) {
return
}
dismissSuggestions()
}
}, [dismissSuggestions, open, suggestions.length, inputRef])
// Why: Electron <webview> guests run in a separate process, so clicking the
// page never dispatches pointerdown on the renderer document and Radix cannot
// detect an outside dismiss. Window blur and focus moves into the guest (the
@@ -399,7 +367,6 @@ export default function BrowserAddressBar({
event.preventDefault()
setOpen(false)
clearSuggestionPreview()
setAutocompleteQuery(value)
onSubmit()
}}
>
@@ -423,7 +390,6 @@ export default function BrowserAddressBar({
// from repopulating the input after Cmd+A → Delete.
prePreviewValueRef.current = null
setSelectedValueOverride(null)
setAutocompleteQuery(nextValue)
onChange(nextValue)
}}
role="combobox"
@@ -0,0 +1,104 @@
// @vitest-environment happy-dom
import { act, cleanup, fireEvent, render, screen } from '@testing-library/react'
import { afterEach, describe, expect, it, vi } from 'vitest'
import BrowserFind from './BrowserFind'
function createWebviewRef(): {
ref: React.RefObject<Electron.WebviewTag | null>
findInPage: ReturnType<typeof vi.fn>
stopFindInPage: ReturnType<typeof vi.fn>
} {
const findInPage = vi.fn()
const stopFindInPage = vi.fn()
const ref = {
current: {
findInPage,
stopFindInPage,
addEventListener: vi.fn(),
removeEventListener: vi.fn()
} as unknown as Electron.WebviewTag
}
return { ref, findInPage, stopFindInPage }
}
function openFindWithQuery(query: string): ReturnType<typeof createWebviewRef> {
const webview = createWebviewRef()
vi.useFakeTimers()
render(<BrowserFind isOpen onClose={vi.fn()} webviewRef={webview.ref} />)
fireEvent.change(screen.getByPlaceholderText('Find in page...'), { target: { value: query } })
act(() => {
vi.advanceTimersByTime(250)
})
return webview
}
afterEach(() => {
cleanup()
vi.useRealTimers()
})
describe('BrowserFind session flags', () => {
it('starts a new find session when the query changes', () => {
const { findInPage } = openFindWithQuery('needle')
expect(findInPage).toHaveBeenLastCalledWith('needle', { findNext: true })
})
it('advances forward as a follow-up request, not a new session', () => {
const { findInPage } = openFindWithQuery('needle')
findInPage.mockClear()
fireEvent.click(screen.getByTitle('Next match'))
expect(findInPage).toHaveBeenCalledExactlyOnceWith('needle', {
forward: true,
findNext: false
})
})
it('advances backward as a follow-up request, not a new session', () => {
const { findInPage } = openFindWithQuery('needle')
findInPage.mockClear()
fireEvent.click(screen.getByTitle('Previous match'))
expect(findInPage).toHaveBeenCalledExactlyOnceWith('needle', {
forward: false,
findNext: false
})
})
it('treats Enter and Shift+Enter as follow-up requests in both directions', () => {
const { findInPage } = openFindWithQuery('needle')
findInPage.mockClear()
const input = screen.getByPlaceholderText('Find in page...')
fireEvent.keyDown(input, { key: 'Enter' })
fireEvent.keyDown(input, { key: 'Enter', shiftKey: true })
expect(findInPage.mock.calls).toEqual([
['needle', { forward: true, findNext: false }],
['needle', { forward: false, findNext: false }]
])
})
it('starts the new query when Enter is pressed before its debounce settles', () => {
const webview = createWebviewRef()
vi.useFakeTimers()
render(<BrowserFind isOpen onClose={vi.fn()} webviewRef={webview.ref} />)
const input = screen.getByPlaceholderText('Find in page...')
fireEvent.change(input, { target: { value: 'needle' } })
fireEvent.keyDown(input, { key: 'Enter' })
expect(webview.findInPage).toHaveBeenCalledExactlyOnceWith('needle', {
forward: true,
findNext: true
})
act(() => {
vi.advanceTimersByTime(250)
})
expect(webview.findInPage).toHaveBeenCalledTimes(1)
})
})
@@ -17,6 +17,7 @@ export default function BrowserFind({
}: BrowserFindProps): React.JSX.Element | null {
const inputRef = useRef<HTMLInputElement>(null)
const wasOpenRef = useRef(isOpen)
const activeFindQueryRef = useRef<string | null>(null)
const [query, setQuery] = useState('')
const [activeMatch, setActiveMatch] = useState(0)
const [totalMatches, setTotalMatches] = useState(0)
@@ -50,15 +51,21 @@ export default function BrowserFind({
}
}, [webviewRef])
// Why: Electron's findNext means "start a NEW session" — follow-up requests that
// advance the selection must pass false, or every Enter restarts at the first match.
const findNext = useCallback(() => {
if (requestQuery) {
safeFindInPage(requestQuery, { forward: true, findNext: true })
const findNext = activeFindQueryRef.current !== requestQuery
safeFindInPage(requestQuery, { forward: true, findNext })
activeFindQueryRef.current = requestQuery
}
}, [requestQuery, safeFindInPage])
const findPrevious = useCallback(() => {
if (requestQuery) {
safeFindInPage(requestQuery, { forward: false, findNext: true })
const findNext = activeFindQueryRef.current !== requestQuery
safeFindInPage(requestQuery, { forward: false, findNext })
activeFindQueryRef.current = requestQuery
}
}, [requestQuery, safeFindInPage])
@@ -75,14 +82,23 @@ export default function BrowserFind({
const wasOpen = wasOpenRef.current
wasOpenRef.current = isOpen
if (!isOpen) {
activeFindQueryRef.current = null
return
}
if (!requestQuery) {
activeFindQueryRef.current = null
safeStopFindInPage()
return
}
const runFind = (): void => safeFindInPage(requestQuery)
// A reopen or a changed query starts a fresh session, so findNext must be true here.
const runFind = (): void => {
if (activeFindQueryRef.current === requestQuery) {
return
}
safeFindInPage(requestQuery, { findNext: true })
activeFindQueryRef.current = requestQuery
}
if (!wasOpen) {
runFind()
return
@@ -18,7 +18,7 @@ import {
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover'
import { useAppStore } from '@/store'
import { isLinuxUserAgent, isMacUserAgent } from '@/components/terminal-pane/pane-helpers'
import { getBrowserCookieImportSourceLabels } from '../../../../shared/browser-cookie-import-sources'
import { getBrowserCookieImportSourceLabels } from '../../../../../shared/browser-cookie-import-sources'
import { shouldShowBrowserImportHint } from './browser-import-hint-visibility'
import { formatBrowserImportSummary } from './browser-detected-browsers-summary'
import { translate } from '@/i18n/i18n'
@@ -9,8 +9,11 @@ type Props = {
onTakeBack: () => void | Promise<void>
}
const OVERLAY_TITLE_ID = 'browser-mobile-driver-overlay-title'
export function BrowserMobileDriverOverlay({ driver, onTakeBack }: Props): ReactElement | null {
const [pending, setPending] = useState(false)
const [takeBackFailed, setTakeBackFailed] = useState(false)
const mountedRef = useRef(false)
const setOverlayRef = useCallback((node: HTMLDivElement | null): void => {
@@ -19,6 +22,7 @@ export function BrowserMobileDriverOverlay({ driver, onTakeBack }: Props): React
// Why: take-back can resolve after the overlay renders null; a later
// mobile session must not inherit the stale disabled state.
setPending(false)
setTakeBackFailed(false)
}
}, [])
@@ -31,8 +35,15 @@ export function BrowserMobileDriverOverlay({ driver, onTakeBack }: Props): React
return
}
setPending(true)
setTakeBackFailed(false)
try {
await onTakeBack()
} catch {
// Why: the reclaim RPC can fail (runtime gone, phone still holding the lock).
// Surface it and leave the button enabled instead of silently re-locking.
if (mountedRef.current) {
setTakeBackFailed(true)
}
} finally {
if (mountedRef.current) {
setPending(false)
@@ -44,6 +55,8 @@ export function BrowserMobileDriverOverlay({ driver, onTakeBack }: Props): React
<div
ref={setOverlayRef}
role="dialog"
aria-modal="true"
aria-labelledby={OVERLAY_TITLE_ID}
aria-live="assertive"
className={cn(
'pointer-events-none absolute inset-0 z-50 flex items-center justify-center bg-background/80 p-6 backdrop-blur-sm',
@@ -60,7 +73,7 @@ export function BrowserMobileDriverOverlay({ driver, onTakeBack }: Props): React
)}
</span>
</div>
<div className="text-base font-semibold leading-tight">
<div id={OVERLAY_TITLE_ID} className="text-base font-semibold leading-tight">
{translate(
'auto.components.browser.pane.BrowserMobileDriverOverlay.d9768ec642',
'Browser input is paused'
@@ -72,6 +85,14 @@ export function BrowserMobileDriverOverlay({ driver, onTakeBack }: Props): React
'This tab is being controlled from your phone. Take back to use it on desktop.'
)}
</div>
{takeBackFailed ? (
<div role="alert" className="text-sm leading-relaxed text-destructive">
{translate(
'auto.components.browser.pane.BrowserMobileDriverOverlay.7c31b0da94',
"Couldn't take back this tab. Check the phone session, then try again."
)}
</div>
) : null}
<div className="mt-1 flex justify-end">
{/* autoFocus puts keyboard users on the recovery action when the lock appears. */}
<Button type="button" size="sm" onClick={handleTakeBack} disabled={pending} autoFocus>
@@ -5,7 +5,7 @@ import { describe, expect, it } from 'vitest'
describe('remote browser link routing', () => {
it('pins context-menu opens to the runtime that owns the pane', () => {
const source = readFileSync(
fileURLToPath(new URL('./BrowserPane.tsx', import.meta.url)),
fileURLToPath(new URL('../stream-remote/remote-browser-page-pane.tsx', import.meta.url)),
'utf8'
)
const paneStart = source.indexOf('function RemoteBrowserPagePane')
@@ -5,7 +5,7 @@ import { describe, expect, it } from 'vitest'
describe('BrowserPagePane render IPC boundary', () => {
it('derives toolbar URLs without querying the webview', () => {
const source = readFileSync(
fileURLToPath(new URL('./BrowserPane.tsx', import.meta.url)),
fileURLToPath(new URL('./browser-page-pane.tsx', import.meta.url)),
'utf8'
)
const start = source.indexOf('const isBlankTab =', source.indexOf('function BrowserPagePane'))
@@ -1,6 +1,6 @@
// @vitest-environment happy-dom
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { ORCA_BROWSER_GUEST_WEB_PREFERENCES_ATTRIBUTE } from '../../../../shared/browser-guest-web-preferences'
import { ORCA_BROWSER_GUEST_WEB_PREFERENCES_ATTRIBUTE } from '../../../../../shared/browser-guest-web-preferences'
const registryMocks = vi.hoisted(() => ({
destroyPersistentWebview: vi.fn(),
@@ -9,14 +9,14 @@ const registryMocks = vi.hoisted(() => ({
webviewRegistry: new Map<string, Electron.WebviewTag>()
}))
vi.mock('./webview-registry', () => ({
vi.mock('../host-guest/webview-registry', () => ({
destroyPersistentWebview: registryMocks.destroyPersistentWebview,
registerPersistentWebview: registryMocks.registerPersistentWebview,
replacePersistentWebview: registryMocks.replacePersistentWebview,
webviewRegistry: registryMocks.webviewRegistry
}))
import { ensureBrowserPageWebview } from './browser-page-webview'
import { ensureBrowserPageWebview } from '../host-guest/browser-page-webview'
function createContainer(id: string): HTMLDivElement {
const container = document.createElement('div')
@@ -3,8 +3,8 @@ import { cleanup, render } from '@testing-library/react'
import { Suspense } from 'react'
import { renderToStaticMarkup } from 'react-dom/server'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import type { BrowserTab as BrowserTabState } from '../../../../shared/browser-workspace-types'
import type { Tab, TabGroup } from '../../../../shared/tab-types'
import type { BrowserTab as BrowserTabState } from '../../../../../shared/browser-workspace-types'
import type { Tab, TabGroup } from '../../../../../shared/tab-types'
type MockAppState = {
browserTabsByWorktree: Record<string, readonly BrowserTabState[]>
@@ -21,7 +21,7 @@ const mocks = vi.hoisted(() => ({
focusGroup: vi.fn()
}))
vi.mock('../../store', () => ({
vi.mock('../../../store', () => ({
useAppStore: (selector: (state: MockAppState) => unknown) => {
if (!mocks.state) {
throw new Error('mock app state not initialized')
@@ -30,7 +30,7 @@ vi.mock('../../store', () => ({
}
}))
vi.mock('./browser-automation-visibility', () => ({
vi.mock('../host-guest/browser-automation-visibility', () => ({
useBrowserAutomationVisibilityForAny: (pageIds: readonly string[]) =>
pageIds.some((pageId) => mocks.automationVisiblePageIds.has(pageId))
}))
@@ -40,7 +40,7 @@ vi.mock('@/lib/pane-manager/browser-mobile-driver-state', () => ({
pageIds.some((pageId) => mocks.mobileDrivenPageIds.has(pageId))
}))
vi.mock('./BrowserPane', () => ({
vi.mock('./browser-workspace-pane', () => ({
default: ({
browserTab,
isActive,
@@ -1,12 +1,13 @@
import { memo, useCallback, useLayoutEffect, useMemo, useState } from 'react'
import { registerBrowserOverlaySlotViewport } from './browser-page-viewport'
import { registerBrowserOverlaySlotViewport } from '../host-guest/browser-page-viewport'
import { useShallow } from 'zustand/react/shallow'
import { useAppStore } from '../../store'
import type { BrowserTab as BrowserTabState } from '../../../../shared/browser-workspace-types'
import type { Tab, TabGroup } from '../../../../shared/tab-types'
import BrowserPane, { type BrowserFindShortcutScope } from './BrowserPane'
import { tabGroupBodyAnchorName } from '../tab-group/tab-group-body-anchor'
import { useBrowserAutomationVisibilityForAny } from './browser-automation-visibility'
import { useAppStore } from '../../../store'
import type { BrowserTab as BrowserTabState } from '../../../../../shared/browser-workspace-types'
import type { Tab, TabGroup } from '../../../../../shared/tab-types'
import BrowserPane from './browser-workspace-pane'
import type { BrowserFindShortcutScope } from '../describe-page/browser-page-types'
import { tabGroupBodyAnchorName } from '../../tab-group/tab-group-body-anchor'
import { useBrowserAutomationVisibilityForAny } from '../host-guest/browser-automation-visibility'
import { useBrowserMobileDriverForAny } from '@/lib/pane-manager/browser-mobile-driver-state'
// Why: Electron <webview> destroys its guest on DOM reparent, so BrowserPanes render at worktree level and moving a tab between groups only swaps the overlay's CSS position-anchor.
@@ -4,11 +4,11 @@ import { emitBrowserCookieImportToast } from '@/lib/browser-cookie-import-toast'
import { useAppStore } from '@/store'
import { useMountedRef } from '@/hooks/useMountedRef'
import { shouldShowBrowserImportHint } from './browser-import-hint-visibility'
import type { BrowserViewportPresetId } from '../../../../shared/browser-workspace-types'
import type { BrowserViewportPresetId } from '../../../../../shared/browser-workspace-types'
import {
browserViewportPresetToOverride,
getBrowserViewportPreset
} from '../../../../shared/browser-viewport-presets'
} from '../../../../../shared/browser-viewport-presets'
import { BrowserToolbarMenuDropdown } from './browser-toolbar-menu-dropdown'
import { BrowserToolbarProfileDialogs } from './browser-toolbar-profile-dialogs'
import { translate } from '@/i18n/i18n'
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest'
import type { BrowserHistoryEntry } from '../../../../shared/browser-workspace-types'
import type { BrowserHistoryEntry } from '../../../../../shared/browser-workspace-types'
import {
BROWSER_ADDRESS_BAR_QUERY_MAX_BYTES,
buildBrowserAddressBarSuggestions,
@@ -5,9 +5,9 @@ import {
normalizeBrowserNavigationUrl,
SEARCH_ENGINE_LABELS,
type SearchEngine
} from '../../../../shared/browser-url'
import type { BrowserHistoryEntry } from '../../../../shared/browser-workspace-types'
import { isClipboardTextByteLengthOverLimit } from '../../../../shared/clipboard-text'
} from '../../../../../shared/browser-url'
import type { BrowserHistoryEntry } from '../../../../../shared/browser-workspace-types'
import { isClipboardTextByteLengthOverLimit } from '../../../../../shared/clipboard-text'
import { translate } from '@/i18n/i18n'
export const MAX_BROWSER_ADDRESS_BAR_SUGGESTIONS = 8
@@ -0,0 +1,213 @@
import type { Dispatch, SetStateAction } from 'react'
import { cn } from '@/lib/utils'
import { CircleCheck, Copy, Crosshair, Send, Trash2 } from 'lucide-react'
import { Button } from '@/components/ui/button'
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuTrigger
} from '@/components/ui/dropdown-menu'
import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip'
import { translate } from '@/i18n/i18n'
import { BrowserAnnotationSendMenuContent } from '../annotate/BrowserAnnotationSendMenuContent'
import type { BrowserGrabPayload } from '../../../../../shared/browser-grab-types'
import type { GrabModeHook } from '../annotate/useGrabMode'
import { preventAgentSendTargetOutsideDismiss } from '../annotate/prevent-agent-send-target-outside-dismiss'
import type { GrabIntent } from '../describe-page/browser-page-types'
export function BrowserPageChromeBanners({
resourceNotice,
setResourceNotice,
grab,
grabIntent,
pendingAnnotationPayload,
browserAnnotationsLength,
annotationBannerSendOpen,
handleAnnotationBannerSendOpenChange,
worktreeId,
activeGroupId,
browserAnnotationsPrompt,
handleBrowserAnnotationsSentToAgent,
handleCopyBrowserAnnotations,
browserAnnotationsCopied,
handleClearBrowserAnnotations,
setPendingAnnotationPayload
}: {
resourceNotice: string | null
setResourceNotice: Dispatch<SetStateAction<string | null>>
grab: GrabModeHook
grabIntent: GrabIntent
pendingAnnotationPayload: BrowserGrabPayload | null
browserAnnotationsLength: number
annotationBannerSendOpen: boolean
handleAnnotationBannerSendOpenChange: (open: boolean) => void
worktreeId: string
activeGroupId: string | undefined
browserAnnotationsPrompt: string
handleBrowserAnnotationsSentToAgent: () => void
handleCopyBrowserAnnotations: () => void
browserAnnotationsCopied: boolean
handleClearBrowserAnnotations: () => void
setPendingAnnotationPayload: Dispatch<SetStateAction<BrowserGrabPayload | null>>
}): React.JSX.Element | null {
if (!resourceNotice && grab.state === 'idle') {
return null
}
return (
<>
{resourceNotice ? (
<div className="flex items-center justify-between gap-2 border-b border-border/60 bg-background px-3 py-1.5 text-xs text-muted-foreground">
<span>{resourceNotice}</span>
<button
type="button"
onClick={() => setResourceNotice(null)}
className="shrink-0 text-muted-foreground/60 hover:text-foreground"
aria-label={translate('auto.components.browser.pane.BrowserPane.2fdca7df09', 'Dismiss')}
>
</button>
</div>
) : null}
{grab.state !== 'idle' ? (
<div
className={cn(
'flex items-center gap-2 border-b border-border/60 px-3 py-1.5 text-xs text-foreground/90',
grab.state === 'error' ? 'bg-destructive/10' : 'bg-accent'
)}
>
<Crosshair
className={cn(
'size-3 shrink-0',
grab.state === 'error' ? 'text-destructive' : 'text-muted-foreground'
)}
/>
<span className="min-w-0 flex-1 truncate">
{grab.state === 'error'
? translate(
'auto.components.browser.pane.BrowserPane.4328a0a062',
'Grab failed: {{value0}}',
{ value0: grab.error ?? 'Unknown error' }
)
: grabIntent === 'annotate'
? pendingAnnotationPayload
? translate(
'auto.components.browser.pane.BrowserPane.b733a91bd9',
'Add feedback for the selected element.'
)
: browserAnnotationsLength === 1
? translate(
'auto.components.browser.pane.BrowserPane.074f0ed10b',
'{{value0}} annotation ready. Select another element or copy all feedback.',
{ value0: browserAnnotationsLength }
)
: browserAnnotationsLength > 0
? translate(
'auto.components.browser.pane.BrowserPane.a2164a6e5a',
'{{value0}} annotations ready. Select another element or copy all feedback.',
{ value0: browserAnnotationsLength }
)
: translate(
'auto.components.browser.pane.BrowserPane.777b5bc4ec',
'Click an element to add feedback for the agent.'
)
: grab.state === 'confirming'
? translate(
'auto.components.browser.pane.BrowserPane.e852e20cea',
'Copied — press S to screenshot, or select another element'
)
: translate(
'auto.components.browser.pane.BrowserPane.168350ae6a',
'Click or hover an element, then press C to copy or S to screenshot.'
)}
</span>
{grabIntent === 'annotate' && browserAnnotationsLength > 0 ? (
<>
<DropdownMenu
modal={false}
open={annotationBannerSendOpen}
onOpenChange={handleAnnotationBannerSendOpenChange}
>
<Tooltip>
<TooltipTrigger asChild>
<DropdownMenuTrigger asChild>
<Button size="xs" variant="outline" className="h-6 gap-1.5">
<Send className="size-3" />
{translate('auto.components.browser.pane.BrowserPane.ac39b9366b', 'Send')}
</Button>
</DropdownMenuTrigger>
</TooltipTrigger>
<TooltipContent side="bottom" sideOffset={6}>
{translate(
'auto.components.browser.pane.BrowserPane.95af781091',
'Send feedback to an agent'
)}
</TooltipContent>
</Tooltip>
<DropdownMenuContent
align="end"
className="min-w-[180px]"
onInteractOutside={preventAgentSendTargetOutsideDismiss}
onPointerDownOutside={preventAgentSendTargetOutsideDismiss}
>
<BrowserAnnotationSendMenuContent
worktreeId={worktreeId}
groupId={activeGroupId ?? worktreeId}
prompt={browserAnnotationsPrompt}
onPromptDelivered={handleBrowserAnnotationsSentToAgent}
/>
</DropdownMenuContent>
</DropdownMenu>
<Button
size="xs"
variant="outline"
className="h-6 gap-1.5"
onClick={handleCopyBrowserAnnotations}
>
{browserAnnotationsCopied ? (
<CircleCheck className="size-3" />
) : (
<Copy className="size-3" />
)}
{browserAnnotationsCopied
? translate('auto.components.browser.pane.BrowserPane.6f4ab3592b', 'Copied')
: translate('auto.components.browser.pane.BrowserPane.499b31b84e', 'Copy All')}
</Button>
<Tooltip>
<TooltipTrigger asChild>
<Button
size="icon-xs"
variant="ghost"
className="h-6 w-6 text-muted-foreground hover:text-foreground"
onClick={handleClearBrowserAnnotations}
aria-label={translate(
'auto.components.browser.pane.BrowserPane.734e4343ec',
'Clear browser annotations'
)}
>
<Trash2 className="size-3" />
</Button>
</TooltipTrigger>
<TooltipContent side="bottom" sideOffset={6}>
{translate(
'auto.components.browser.pane.BrowserPane.11c5084aa2',
'Clear annotations'
)}
</TooltipContent>
</Tooltip>
</>
) : null}
<button
className="ml-auto shrink-0 rounded px-2 py-0.5 text-muted-foreground transition-colors hover:text-foreground"
onClick={() => {
setPendingAnnotationPayload(null)
grab.cancel()
}}
>
{translate('auto.components.browser.pane.BrowserPane.fa6ea61de3', 'Cancel')}
</button>
</div>
) : null}
</>
)
}
@@ -0,0 +1,128 @@
import type { Dispatch, MutableRefObject, RefObject, SetStateAction } from 'react'
import type { BrowserPage as BrowserPageState } from '../../../../../shared/browser-workspace-types'
import type { ShareableBrowserArtifactFile } from '../describe-page/browser-artifact-upload'
import type { GrabModeHook } from '../annotate/useGrabMode'
import { BrowserPageToolbar } from './browser-page-toolbar'
import { BrowserPageDownloadList } from '../navigate/browser-page-download-list'
import { BrowserPageChromeBanners } from './browser-page-chrome-banners'
import type { useBrowserPageReloadActions } from '../navigate/use-browser-page-reload-actions'
import type { useBrowserPageNavigationDownloads } from '../navigate/use-browser-page-navigation-downloads'
import type { useBrowserPageGrabAnnotations } from '../annotate/use-browser-page-grab-annotations'
import type { useBrowserPageAnnotationSend } from '../annotate/use-browser-page-annotation-send'
export function BrowserPageChromeHeader({
chromeHeaderRef,
browserTab,
workspaceId,
worktreeId,
sessionProfileId,
isActive,
webviewRef,
addressBarInputRef,
dismissAddressBarSuggestionsRef,
reload,
nav,
grab,
grabAnnotations,
annotationSend,
markupIsActive,
markupStart,
markupCancel,
grabElementShortcut,
shareableArtifactFile,
currentBrowserUrl,
externalUrl,
isBlankTab,
resourceNotice,
setResourceNotice
}: {
chromeHeaderRef: RefObject<HTMLDivElement | null>
browserTab: BrowserPageState
workspaceId: string
worktreeId: string
sessionProfileId: string | null
isActive: boolean
webviewRef: MutableRefObject<Electron.WebviewTag | null>
addressBarInputRef: MutableRefObject<HTMLInputElement | null>
dismissAddressBarSuggestionsRef: MutableRefObject<(() => void) | null>
reload: ReturnType<typeof useBrowserPageReloadActions>
nav: ReturnType<typeof useBrowserPageNavigationDownloads>
grab: GrabModeHook
grabAnnotations: ReturnType<typeof useBrowserPageGrabAnnotations>
annotationSend: ReturnType<typeof useBrowserPageAnnotationSend>
markupIsActive: boolean
markupStart: () => Promise<void>
markupCancel: () => void
grabElementShortcut: string
shareableArtifactFile: ShareableBrowserArtifactFile | null
currentBrowserUrl: string
externalUrl: string | null
isBlankTab: boolean
resourceNotice: string | null
setResourceNotice: Dispatch<SetStateAction<string | null>>
}): React.JSX.Element {
return (
<div ref={chromeHeaderRef} className="pointer-events-auto shrink-0">
<BrowserPageToolbar
browserPageId={browserTab.id}
workspaceId={workspaceId}
sessionProfileId={sessionProfileId}
viewportPresetId={browserTab.viewportPresetId ?? null}
isActive={isActive}
canGoBack={browserTab.canGoBack}
canGoForward={browserTab.canGoForward}
loading={browserTab.loading}
webviewRef={webviewRef}
reloadMenuOpen={reload.reloadMenuOpen}
setReloadMenuOpen={reload.setReloadMenuOpen}
reloadButtonLabel={reload.reloadButtonLabel}
reloadButtonLabelKind={reload.reloadButtonLabelKind}
reloadShortcut={reload.reloadShortcut}
hardReloadShortcut={reload.hardReloadShortcut}
runReloadTrigger={reload.runReloadTrigger}
addressBarValue={nav.addressBarValue}
setAddressBarValue={nav.setAddressBarValue}
submitAddressBar={nav.submitAddressBar}
navigateToUrl={nav.navigateToUrl}
addressBarInputRef={addressBarInputRef}
dismissAddressBarSuggestionsRef={dismissAddressBarSuggestionsRef}
grab={grab}
grabIntent={grabAnnotations.grabIntent}
startGrabIntent={grabAnnotations.startGrabIntent}
isBlankTab={isBlankTab}
markupIsActive={markupIsActive}
markupStart={markupStart}
markupCancel={markupCancel}
grabElementShortcut={grabElementShortcut}
browserAnnotationsLength={annotationSend.browserAnnotations.length}
shareableArtifactFile={shareableArtifactFile}
currentBrowserUrl={currentBrowserUrl}
externalUrl={externalUrl}
/>
<BrowserPageDownloadList
visibleDownloads={nav.visibleDownloads}
onOpenDownloadedFile={nav.handleOpenDownloadedFile}
onShowDownloadedFile={nav.handleShowDownloadedFile}
onDismissDownload={nav.dismissBrowserDownload}
/>
<BrowserPageChromeBanners
resourceNotice={resourceNotice}
setResourceNotice={setResourceNotice}
grab={grab}
grabIntent={grabAnnotations.grabIntent}
pendingAnnotationPayload={grabAnnotations.pendingAnnotationPayload}
browserAnnotationsLength={annotationSend.browserAnnotations.length}
annotationBannerSendOpen={annotationSend.annotationBannerSendOpen}
handleAnnotationBannerSendOpenChange={annotationSend.handleAnnotationBannerSendOpenChange}
worktreeId={worktreeId}
activeGroupId={annotationSend.activeGroupId}
browserAnnotationsPrompt={annotationSend.browserAnnotationsPrompt}
handleBrowserAnnotationsSentToAgent={annotationSend.handleBrowserAnnotationsSentToAgent}
handleCopyBrowserAnnotations={annotationSend.handleCopyBrowserAnnotations}
browserAnnotationsCopied={annotationSend.browserAnnotationsCopied}
handleClearBrowserAnnotations={annotationSend.handleClearBrowserAnnotations}
setPendingAnnotationPayload={grabAnnotations.setPendingAnnotationPayload}
/>
</div>
)
}
@@ -0,0 +1,321 @@
import {
useCallback,
useEffect,
useLayoutEffect,
useRef,
useState,
type MutableRefObject
} from 'react'
import { createPortal } from 'react-dom'
import { useAppStore } from '@/store'
import { translate } from '@/i18n/i18n'
import { normalizeExternalBrowserUrl } from '../../../../../shared/browser-url'
import type { BrowserPageContextMenuState } from '../describe-page/browser-page-types'
// `focus:` rather than `focus-visible:` — items are only ever focused programmatically
// while the menu is open, so every focus here is keyboard navigation.
const MENU_ITEM_CLASS =
'relative flex w-full cursor-default items-center gap-2 rounded-[7px] px-2 py-0.5 text-[12px] leading-5 font-medium outline-none select-none hover:bg-black/8 focus:bg-black/8 disabled:pointer-events-none disabled:opacity-50 dark:hover:bg-white/14 dark:focus:bg-white/14'
export function BrowserPageContextMenu({
browserPageId,
worktreeId,
canGoBack,
canGoForward,
webviewRef,
onReload
}: {
browserPageId: string
worktreeId: string
canGoBack: boolean
canGoForward: boolean
webviewRef: MutableRefObject<Electron.WebviewTag | null>
onReload: () => void
}): React.JSX.Element | null {
const createBrowserTab = useAppStore((s) => s.createBrowserTab)
const [contextMenu, setContextMenu] = useState<BrowserPageContextMenuState | null>(null)
const contextMenuRef = useRef<HTMLDivElement>(null)
useEffect(() => {
return window.api.browser.onContextMenuRequested((event) => {
if (event.browserPageId !== browserPageId) {
return
}
// Why: convert OS screen cursor coords to renderer CSS pixels — immune to guest/renderer coordinate-space mismatches from zoom/DPI.
const zoomFactor = 1.2 ** window.api.ui.getZoomLevel()
const x = Math.round((event.screenX - window.screenX) / zoomFactor)
const y = Math.round((event.screenY - window.screenY) / zoomFactor)
setContextMenu({
x,
y,
linkUrl: event.linkUrl,
pageUrl: event.pageUrl,
selectionText: event.selectionText ?? ''
})
})
}, [browserPageId])
useEffect(() => {
return window.api.browser.onContextMenuDismissed((event) => {
if (event.browserPageId !== browserPageId) {
return
}
setContextMenu(null)
})
}, [browserPageId])
// Why: the guest owns focus while the menu is open, so hand it back on close —
// otherwise dismissing leaves the page unable to receive keystrokes.
const closeMenu = useCallback((): void => {
setContextMenu(null)
try {
webviewRef.current?.focus()
} catch {
// The guest can be destroyed while its renderer-owned menu is open.
}
}, [webviewRef])
const menuItems = useCallback((): HTMLButtonElement[] => {
const el = contextMenuRef.current
if (!el) {
return []
}
return [...el.querySelectorAll<HTMLButtonElement>('[role="menuitem"]:not(:disabled)')]
}, [])
useEffect(() => {
if (!contextMenu) {
return
}
const handleKeyDown = (e: KeyboardEvent): void => {
if (e.key === 'Escape') {
e.preventDefault()
closeMenu()
}
}
window.addEventListener('keydown', handleKeyDown, true)
return () => window.removeEventListener('keydown', handleKeyDown, true)
}, [closeMenu, contextMenu])
// Why: role="menu" is unreachable by keyboard unless focus moves in on open.
useEffect(() => {
if (!contextMenu) {
return
}
menuItems()[0]?.focus()
}, [contextMenu, menuItems])
const handleMenuKeyDown = useCallback(
(e: React.KeyboardEvent<HTMLDivElement>): void => {
const items = menuItems()
if (items.length === 0) {
return
}
const currentIndex = items.indexOf(document.activeElement as HTMLButtonElement)
let nextIndex: number
if (e.key === 'ArrowDown') {
nextIndex = (currentIndex + 1) % items.length
} else if (e.key === 'ArrowUp') {
nextIndex = (currentIndex <= 0 ? items.length : currentIndex) - 1
} else if (e.key === 'Home') {
nextIndex = 0
} else if (e.key === 'End') {
nextIndex = items.length - 1
} else {
return
}
e.preventDefault()
e.stopPropagation()
items[nextIndex]?.focus()
},
[menuItems]
)
// Why: ancestor CSS (transform/backdrop-filter) can shift position:fixed even via a body Portal, so measure/correct before paint; also flip on viewport overflow.
useLayoutEffect(() => {
const el = contextMenuRef.current
if (!el || !contextMenu) {
return
}
el.style.left = `${contextMenu.x}px`
el.style.top = `${contextMenu.y}px`
const rect = el.getBoundingClientRect()
// Why: CSS containing blocks can shift "fixed" elements; capture the offset between requested and actual position.
const offsetX = contextMenu.x - rect.left
const offsetY = contextMenu.y - rect.top
let renderX = contextMenu.x
let renderY = contextMenu.y
// Flip so the opposite corner aligns with the cursor when the menu overflows.
if (rect.right > window.innerWidth) {
renderX = contextMenu.x - rect.width
}
if (rect.bottom > window.innerHeight) {
renderY = contextMenu.y - rect.height
}
renderX = Math.max(0, renderX)
renderY = Math.max(0, renderY)
el.style.left = `${renderX + offsetX}px`
el.style.top = `${renderY + offsetY}px`
}, [contextMenu])
if (!contextMenu) {
return null
}
return createPortal(
<>
<div className="fixed inset-0 z-50" onPointerDown={closeMenu} />
<div
ref={contextMenuRef}
role="menu"
aria-orientation="vertical"
tabIndex={-1}
onKeyDown={handleMenuKeyDown}
data-testid="browser-context-menu"
style={{ left: contextMenu.x, top: contextMenu.y }}
className="fixed z-50 min-w-[13rem] overflow-hidden rounded-[11px] border border-black/14 bg-[rgba(255,255,255,0.82)] p-1 text-black shadow-[0_16px_36px_rgba(0,0,0,0.24),inset_0_1px_0_rgba(255,255,255,0.14)] backdrop-blur-2xl dark:border-white/14 dark:bg-[rgba(0,0,0,0.72)] dark:text-white dark:shadow-[0_20px_44px_rgba(0,0,0,0.42),inset_0_1px_0_rgba(255,255,255,0.04)]"
>
{contextMenu.linkUrl ? (
<>
<button
role="menuitem"
className={MENU_ITEM_CLASS}
onClick={() => {
createBrowserTab(worktreeId, contextMenu.linkUrl!, {
title: contextMenu.linkUrl!
})
closeMenu()
}}
>
{translate(
'auto.components.browser.pane.BrowserPane.b5b87d6cbb',
'Open Link In Orca Browser'
)}
</button>
<button
role="menuitem"
className={MENU_ITEM_CLASS}
onClick={() => {
const targetUrl = normalizeExternalBrowserUrl(contextMenu.linkUrl!)
if (targetUrl) {
void window.api.shell.openUrl(targetUrl)
}
closeMenu()
}}
>
{translate(
'auto.components.browser.pane.BrowserPane.8ce4f6b12e',
'Open Link In Default Browser'
)}
</button>
<button
role="menuitem"
className={MENU_ITEM_CLASS}
onClick={() => {
void window.api.ui.writeClipboardText(contextMenu.linkUrl ?? '')
closeMenu()
}}
>
{translate(
'auto.components.browser.pane.BrowserPane.efb0e8f7f3',
'Copy Link Address'
)}
</button>
<div className="my-1 h-px bg-border/70" />
</>
) : null}
{contextMenu.selectionText.trim() ? (
<>
<button
role="menuitem"
className={MENU_ITEM_CLASS}
onClick={() => {
void window.api.ui.writeClipboardText(contextMenu.selectionText)
closeMenu()
}}
>
{translate('auto.components.browser.pane.BrowserPane.2a4c4b8e1f', 'Copy')}
</button>
<div className="my-1 h-px bg-border/70" />
</>
) : null}
<button
role="menuitem"
disabled={!canGoBack}
className={MENU_ITEM_CLASS}
onClick={() => {
webviewRef.current?.goBack()
closeMenu()
}}
>
{translate('auto.components.browser.pane.BrowserPane.40edfa75cb', 'Back')}
</button>
<button
role="menuitem"
disabled={!canGoForward}
className={MENU_ITEM_CLASS}
onClick={() => {
webviewRef.current?.goForward()
closeMenu()
}}
>
{translate('auto.components.browser.pane.BrowserPane.250a9b3e42', 'Forward')}
</button>
<button
role="menuitem"
className={MENU_ITEM_CLASS}
onClick={() => {
onReload()
closeMenu()
}}
>
{translate('auto.components.browser.pane.BrowserPane.0e080d820e', 'Reload')}
</button>
<div className="my-1 h-px bg-border/70" />
<button
role="menuitem"
className={MENU_ITEM_CLASS}
onClick={() => {
const targetUrl = normalizeExternalBrowserUrl(contextMenu.pageUrl)
if (targetUrl) {
void window.api.shell.openUrl(targetUrl)
}
closeMenu()
}}
>
{translate(
'auto.components.browser.pane.BrowserPane.f7ab83f7ed',
'Open Page In Default Browser'
)}
</button>
<button
role="menuitem"
className={MENU_ITEM_CLASS}
onClick={() => {
void window.api.ui.writeClipboardText(contextMenu.pageUrl)
closeMenu()
}}
>
{translate('auto.components.browser.pane.BrowserPane.1b179ab561', 'Copy Page URL')}
</button>
<div className="my-1 h-px bg-border/70" />
<button
role="menuitem"
className={MENU_ITEM_CLASS}
onClick={() => {
void window.api.browser.openDevTools({ browserPageId })
closeMenu()
}}
>
{translate('auto.components.browser.pane.BrowserPane.a8f37f70c3', 'Inspect Page')}
</button>
</div>
</>,
document.body
)
}
@@ -0,0 +1,366 @@
import { useEffect, useLayoutEffect, useRef, useState } from 'react'
import { createPortal } from 'react-dom'
import { cn } from '@/lib/utils'
import { getConnectionIdFromState } from '@/lib/connection-context'
import { useAppStore } from '@/store'
import { useShortcutLabel } from '@/hooks/useShortcutLabel'
import { ORCA_BROWSER_BLANK_URL } from '../../../../../shared/constants'
import type { BrowserPage as BrowserPageState } from '../../../../../shared/browser-workspace-types'
import { normalizeExternalBrowserUrl } from '../../../../../shared/browser-url'
import { getLiveBrowserUrl } from '../describe-page/live-browser-url-registry'
import { ensureBrowserPageViewport } from '../host-guest/browser-page-viewport'
import { isBrowserPagePanePaintable } from '../host-guest/browser-page-paintability'
import { getShareableBrowserArtifactFile } from '../describe-page/browser-artifact-upload'
import { useGrabMode } from '../annotate/useGrabMode'
import { getBrowserPageZoomIndicatorState } from '../host-guest/browser-page-zoom'
import { getOpenableExternalUrl, toDisplayUrl } from '../describe-page/browser-page-url-display'
import type { BrowserOverlayViewport } from '../describe-page/browser-annotation-geometry'
import type {
BrowserFindShortcutScope,
BrowserPageUrlSetter,
BrowserTabPageState
} from '../describe-page/browser-page-types'
import { BrowserPageChromeHeader } from './browser-page-chrome-header'
import { BrowserPageContextMenu } from './browser-page-context-menu'
import { BrowserPageViewportOverlays } from './browser-page-viewport-overlays'
import { useBrowserPageAnnotationSend } from '../annotate/use-browser-page-annotation-send'
import { useBrowserPageChromeFocus } from './use-browser-page-chrome-focus'
import { useBrowserPageFindShortcuts } from './use-browser-page-find-shortcuts'
import { useBrowserPageGrabAnnotations } from '../annotate/use-browser-page-grab-annotations'
import { useBrowserPageKeyboardShortcuts } from '../host-guest/use-browser-page-keyboard-shortcuts'
import { useBrowserPageMarkupCapture } from '../annotate/use-browser-page-markup-capture'
import { useBrowserPageNavigationDownloads } from '../navigate/use-browser-page-navigation-downloads'
import { useBrowserPageReloadActions } from '../navigate/use-browser-page-reload-actions'
import { useBrowserPageResourceNotices } from '../navigate/use-browser-page-resource-notices'
import { useBrowserPageSlotViewport } from '../host-guest/use-browser-page-slot-viewport'
import { useBrowserPageWebviewLifecycle } from '../host-guest/use-browser-page-webview-lifecycle'
import { useBrowserPageWebviewPartition } from '../host-guest/use-browser-page-webview-partition'
import { useBrowserPageWebviewUrlSync } from '../navigate/use-browser-page-webview-url-sync'
import { useBrowserPageZoomFeedback } from '../host-guest/use-browser-page-zoom-feedback'
export function BrowserPagePane({
browserTab,
workspaceId,
worktreeId,
sessionProfileId,
sessionPartition,
isActive,
findShortcutScope,
isAutomationVisible,
isMobileDriven,
inputLocked,
onUpdatePageState,
onSetUrl
}: {
browserTab: BrowserPageState
workspaceId: string
worktreeId: string
sessionProfileId: string | null
sessionPartition: string | null
isActive: boolean
findShortcutScope: BrowserFindShortcutScope
isAutomationVisible: boolean
isMobileDriven: boolean
inputLocked: boolean
onUpdatePageState: (tabId: string, updates: BrowserTabPageState) => void
onSetUrl: BrowserPageUrlSetter
}): React.JSX.Element {
const isPaintable = isBrowserPagePanePaintable({
isActive,
isAutomationVisible,
isMobileDriven
})
const pageViewport = ensureBrowserPageViewport(browserTab.id, workspaceId)
const pageViewportContainer = pageViewport?.container ?? null
const containerRef = useRef<HTMLDivElement | null>(pageViewportContainer)
useLayoutEffect(() => {
containerRef.current = pageViewportContainer
}, [pageViewportContainer])
const chromeHeaderRef = useRef<HTMLDivElement | null>(null)
const webviewRef = useRef<Electron.WebviewTag | null>(null)
const addressBarInputRef = useRef<HTMLInputElement | null>(null)
const dismissAddressBarSuggestionsRef = useRef<(() => void) | null>(null)
const addressBarValueRef = useRef(browserTab.url)
const browserTabUrlRef = useRef(browserTab.url)
const keepAddressBarFocusRef = useRef(false)
// Most-recent observed webview URL; URL sync checks it to avoid force-navigating to an intermediate redirect (which would loop the redirect chain).
const lastKnownWebviewUrlRef = useRef<string | null>(null)
const trackNextLoadingEventRef = useRef(false)
const recoveryNavigationValidationRef = useRef<{
committed: boolean
started: boolean
targetUrl: string
} | null>(null)
const activeLoadFailureRef = useRef(browserTab.loadError)
const retryGuestRecoveryRef = useRef<() => void>(() => {})
const onUpdatePageStateRef = useRef(onUpdatePageState)
const onSetUrlRef = useRef(onSetUrl)
const isActiveRef = useRef(isActive)
useLayoutEffect(() => {
isActiveRef.current = isActive
}, [isActive])
const [findOpen, setFindOpen] = useState(false)
const [browserOverlayViewport, setBrowserOverlayViewport] = useState<BrowserOverlayViewport>({
scrollX: 0,
scrollY: 0,
version: 0
})
const workspaceConnectionId = useAppStore((state) => getConnectionIdFromState(state, worktreeId))
const certificateFailure = useAppStore(
(s) => s.browserCertificateFailuresByPageId[browserTab.id] ?? null
)
const webviewPartition = useBrowserPageWebviewPartition({
sessionProfileId,
sessionPartition
})
const grabElementShortcut = useShortcutLabel('browser.grabElement')
const slotViewport = useBrowserPageSlotViewport(workspaceId)
const zoom = useBrowserPageZoomFeedback(browserTab.id)
const { resourceNotice, setResourceNotice } = useBrowserPageResourceNotices(browserTab.id)
const { focusAddressBarNow, focusWebviewNow } = useBrowserPageChromeFocus({
browserTabId: browserTab.id,
isActive,
addressBarInputRef,
webviewRef,
keepAddressBarFocusRef
})
const annotationSend = useBrowserPageAnnotationSend({
browserTabId: browserTab.id,
worktreeId
})
const grab = useGrabMode(browserTab.id)
const markup = useBrowserPageMarkupCapture(webviewRef, containerRef)
const grabAnnotations = useBrowserPageGrabAnnotations({
browserTabId: browserTab.id,
isActive,
grab,
containerRef,
webviewRef,
setBrowserOverlayViewport,
browserAnnotationsLength: annotationSend.browserAnnotations.length,
setBrowserAnnotationTrayOpen: annotationSend.setBrowserAnnotationTrayOpen
})
const nav = useBrowserPageNavigationDownloads({
browserTabId: browserTab.id,
worktreeId,
webviewRef,
activeLoadFailureRef,
lastKnownWebviewUrlRef,
trackNextLoadingEventRef,
recoveryNavigationValidationRef,
onSetUrlRef,
onUpdatePageStateRef,
keepAddressBarFocusRef,
focusWebviewNow,
setResourceNotice,
addressBarValueRef,
addressBarInputRef,
browserTabUrl: browserTab.url
})
useBrowserPageWebviewLifecycle({
browserTabId: browserTab.id,
browserTabUrl: browserTab.url,
browserTabLoadError: browserTab.loadError,
workspaceId,
worktreeId,
sessionProfileId,
webviewPartition,
isActive,
isPaintable,
slotViewport,
viewportPresetId: browserTab.viewportPresetId ?? null,
addressBarInputRef,
addressBarValueRef,
browserTabUrlRef,
keepAddressBarFocusRef,
handleInternalFileDragOverRef: nav.handleInternalFileDragOverRef,
handleInternalFileDropRef: nav.handleInternalFileDropRef,
dismissAddressBarSuggestionsRef,
onUpdatePageState,
onSetUrl,
setAddressBarValue: nav.setAddressBarValue,
setPendingAnnotationPayload: grabAnnotations.setPendingAnnotationPayload,
setBrowserOverlayViewport,
setFindOpen,
focusAddressBarNow,
focusWebviewNow,
paneZoomLevelRef: zoom.paneZoomLevelRef,
setBrowserZoomPercent: zoom.setBrowserZoomPercent,
pendingAnnotationPayload: grabAnnotations.pendingAnnotationPayload,
browserAnnotationsLength: annotationSend.browserAnnotations.length,
inputLocked,
faviconUrl: browserTab.faviconUrl,
webviewRef,
lastKnownWebviewUrlRef,
trackNextLoadingEventRef,
recoveryNavigationValidationRef,
activeLoadFailureRef,
retryGuestRecoveryRef,
onUpdatePageStateRef,
onSetUrlRef
})
useBrowserPageWebviewUrlSync({
browserTabId: browserTab.id,
browserTabUrl: browserTab.url,
browserTabLoading: browserTab.loading,
isActive,
isPaintable,
slotViewport,
webviewRef,
chromeHeaderRef,
lastKnownWebviewUrlRef,
trackNextLoadingEventRef,
keepAddressBarFocusRef,
addressBarInputRef,
browserTabUrlRef,
addressBarValueRef,
onUpdatePageStateRef,
focusWebviewNow
})
const reload = useBrowserPageReloadActions({
browserTab,
webviewRef,
retryGuestRecoveryRef,
onUpdatePageStateRef
})
useBrowserPageFindShortcuts({
browserTabId: browserTab.id,
workspaceId,
isActive,
findShortcutScope,
setFindOpen
})
useBrowserPageKeyboardShortcuts({
browserTabId: browserTab.id,
isActive,
isActiveRef,
markupIsActive: markup.isActive,
webviewRef,
paneZoomLevelRef: zoom.paneZoomLevelRef,
setBrowserDefaultZoomLevel: zoom.setBrowserDefaultZoomLevel,
showBrowserZoomFeedback: zoom.showBrowserZoomFeedback,
reloadWebviewOrRecoverGuest: reload.reloadWebviewOrRecoverGuest,
startGrabIntent: grabAnnotations.startGrabIntent,
focusAddressBarNow,
handleGrabActionShortcut: grabAnnotations.handleGrabActionShortcut,
grabIsInteractive: grab.state !== 'idle' && grab.state !== 'error'
})
// Why: a blank tab reads as 'about:blank' or the resolved data: URL, so match both to keep the "New Browser Tab" overlay visible.
const isBlankTab = browserTab.url === 'about:blank' || browserTab.url === ORCA_BROWSER_BLANK_URL
// Why: synchronous webview URL access blocks render; navigation handlers update this cache before their store writes can re-render the pane.
const liveBrowserUrl = getLiveBrowserUrl(browserTab.id) ?? browserTab.url
const externalUrl = getOpenableExternalUrl(liveBrowserUrl)
const currentBrowserUrl = toDisplayUrl(liveBrowserUrl)
const shareableArtifactFile =
workspaceConnectionId === null ? getShareableBrowserArtifactFile(currentBrowserUrl) : null
const failedNavigationUrl = browserTab.loadError?.validatedUrl ?? currentBrowserUrl
const failureExternalUrl = normalizeExternalBrowserUrl(failedNavigationUrl)
const showFailureOverlay = Boolean(browserTab.loadError) && !isBlankTab
const browserZoomIndicatorState = getBrowserPageZoomIndicatorState({
feedbackVisible: zoom.browserZoomFeedbackVisible,
isDefaultZoom: zoom.browserZoomPercent === zoom.browserDefaultZoomPercent
})
useEffect(() => {
const webview = webviewRef.current
if (!webview) {
return
}
// Why: Electron webviews keep receiving native input under a React overlay unless their own hit testing is disabled.
webview.style.pointerEvents = inputLocked ? 'none' : 'auto'
}, [inputLocked])
useEffect(() => {
const webview = webviewRef.current
if (!webview) {
return
}
// Why: some Electron builds keep painting a hidden guest layer, so drop it from layout (display:none) instead of just hiding it.
webview.style.display = showFailureOverlay ? 'none' : 'flex'
}, [showFailureOverlay])
return (
<div
data-browser-page-pane-id={browserTab.id}
className={cn(
'absolute inset-0 flex min-h-0 flex-1 flex-col',
isActive
? 'pointer-events-none z-10'
: isPaintable
? 'pointer-events-none z-0 opacity-0'
: 'pointer-events-none hidden'
)}
// Why: hidden panes stay paintable (automation/mobile) but must not stay keyboard-focusable.
inert={!isActive}
aria-hidden={!isActive}
>
{/* IPC-driven context menu in a Portal so position:fixed escapes ancestor transform/backdrop-filter containing blocks. */}
<BrowserPageContextMenu
browserPageId={browserTab.id}
worktreeId={worktreeId}
canGoBack={browserTab.canGoBack}
canGoForward={browserTab.canGoForward}
webviewRef={webviewRef}
onReload={() => reload.reloadWebviewOrRecoverGuest(false)}
/>
<BrowserPageChromeHeader
chromeHeaderRef={chromeHeaderRef}
browserTab={browserTab}
workspaceId={workspaceId}
worktreeId={worktreeId}
sessionProfileId={sessionProfileId}
isActive={isActive}
webviewRef={webviewRef}
addressBarInputRef={addressBarInputRef}
dismissAddressBarSuggestionsRef={dismissAddressBarSuggestionsRef}
reload={reload}
nav={nav}
grab={grab}
grabAnnotations={grabAnnotations}
annotationSend={annotationSend}
markupIsActive={markup.isActive}
markupStart={markup.start}
markupCancel={markup.cancel}
grabElementShortcut={grabElementShortcut}
shareableArtifactFile={shareableArtifactFile}
currentBrowserUrl={currentBrowserUrl}
externalUrl={externalUrl}
isBlankTab={isBlankTab}
resourceNotice={resourceNotice}
setResourceNotice={setResourceNotice}
/>
{pageViewport?.container
? createPortal(
<BrowserPageViewportOverlays
markup={markup}
browserZoomIndicatorState={browserZoomIndicatorState}
browserZoomPercent={zoom.browserZoomPercent}
findOpen={findOpen}
setFindOpen={setFindOpen}
webviewRef={webviewRef}
showFailureOverlay={showFailureOverlay}
browserTab={browserTab}
failureExternalUrl={failureExternalUrl}
failedNavigationUrl={failedNavigationUrl}
onUpdatePageStateRef={onUpdatePageStateRef}
retryGuestRecoveryRef={retryGuestRecoveryRef}
navigateToUrl={nav.navigateToUrl}
setResourceNotice={setResourceNotice}
certificateFailure={certificateFailure}
isBlankTab={isBlankTab}
containerRef={containerRef}
browserOverlayViewport={browserOverlayViewport}
worktreeId={worktreeId}
grab={grab}
annotationSend={annotationSend}
grabAnnotations={grabAnnotations}
/>,
pageViewport.container
)
: null}
</div>
)
}
@@ -0,0 +1,325 @@
import type { Dispatch, RefObject, SetStateAction } from 'react'
import { cn } from '@/lib/utils'
import {
ArrowLeft,
ArrowRight,
Crosshair,
ExternalLink,
Loader2,
MessageSquarePlus,
RefreshCw,
SquareCode
} from 'lucide-react'
import { Button } from '@/components/ui/button'
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuShortcut,
DropdownMenuTrigger
} from '@/components/ui/dropdown-menu'
import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip'
import { ArtifactPublishButton } from '@/components/artifacts/ArtifactPublishButton'
import { translate } from '@/i18n/i18n'
import type { BrowserReloadTrigger } from '../navigate/browser-reload-action'
import BrowserAddressBar from './BrowserAddressBar'
import { BrowserImportHintButton } from './BrowserImportHintButton'
import { BrowserToolbarMenu } from './BrowserToolbarMenu'
import { MarkupDrawButton } from '../annotate/MarkupDrawButton'
import { destroyPersistentWebview } from '../host-guest/webview-registry'
import { readBrowserHtmlArtifactRequest } from '../describe-page/browser-artifact-upload'
import type { GrabModeHook } from '../annotate/useGrabMode'
import type { BrowserViewportPresetId } from '../../../../../shared/browser-workspace-types'
import type { GrabIntent } from '../describe-page/browser-page-types'
export function BrowserPageToolbar({
browserPageId,
workspaceId,
sessionProfileId,
viewportPresetId,
isActive,
canGoBack,
canGoForward,
loading,
webviewRef,
reloadMenuOpen,
setReloadMenuOpen,
reloadButtonLabel,
reloadButtonLabelKind,
reloadShortcut,
hardReloadShortcut,
runReloadTrigger,
addressBarValue,
setAddressBarValue,
submitAddressBar,
navigateToUrl,
addressBarInputRef,
dismissAddressBarSuggestionsRef,
grab,
grabIntent,
startGrabIntent,
isBlankTab,
markupIsActive,
markupStart,
markupCancel,
grabElementShortcut,
browserAnnotationsLength,
shareableArtifactFile,
currentBrowserUrl,
externalUrl
}: {
browserPageId: string
workspaceId: string
sessionProfileId: string | null
viewportPresetId: BrowserViewportPresetId | null
isActive: boolean
canGoBack: boolean
canGoForward: boolean
loading: boolean
webviewRef: RefObject<Electron.WebviewTag | null>
reloadMenuOpen: boolean
setReloadMenuOpen: Dispatch<SetStateAction<boolean>>
reloadButtonLabel: string
reloadButtonLabelKind: 'stop' | 'retry' | 'reload'
reloadShortcut: string
hardReloadShortcut: string
runReloadTrigger: (trigger: BrowserReloadTrigger) => void
addressBarValue: string
setAddressBarValue: Dispatch<SetStateAction<string>>
submitAddressBar: () => void
navigateToUrl: (url: string) => void
addressBarInputRef: RefObject<HTMLInputElement | null>
dismissAddressBarSuggestionsRef: RefObject<(() => void) | null>
grab: GrabModeHook
grabIntent: GrabIntent
startGrabIntent: (intent: GrabIntent) => void
isBlankTab: boolean
markupIsActive: boolean
markupStart: () => Promise<void>
markupCancel: () => void
grabElementShortcut: string
browserAnnotationsLength: number
shareableArtifactFile: { filePath: string } | null
currentBrowserUrl: string
externalUrl: string | null
}): React.JSX.Element {
return (
<div
className="relative z-10 flex items-center gap-2 border-b border-border/70 bg-background/95 px-3 py-1.5"
data-contextual-tour-target="browser-toolbar"
>
<Button
size="icon"
variant="ghost"
className="h-7 w-7"
onClick={() => webviewRef.current?.goBack()}
disabled={!canGoBack}
>
<ArrowLeft className="size-4" />
</Button>
<Button
size="icon"
variant="ghost"
className="h-7 w-7"
onClick={() => webviewRef.current?.goForward()}
disabled={!canGoForward}
>
<ArrowRight className="size-4" />
</Button>
<DropdownMenu modal={false} open={reloadMenuOpen} onOpenChange={setReloadMenuOpen}>
{/* Why: suppress the tooltip while the menu is open — both anchor below the button and would overlap. */}
<Tooltip open={reloadMenuOpen ? false : undefined}>
<TooltipTrigger asChild>
<DropdownMenuTrigger asChild>
<Button
size="icon"
variant="ghost"
className="h-7 w-7"
aria-label={reloadButtonLabel}
// Why: preventDefault suppresses Radix's open-on-left-click (composeEventHandlers skips its
// handler once defaultPrevented), keeping left-click on the primary action and the menu on right-click.
onPointerDown={(e) => {
if (e.button === 0) {
e.preventDefault()
}
}}
// Why: same trick for Radix's open-on-Enter/Space, which would otherwise preventDefault the
// synthesized click and strand keyboard users. ArrowDown still falls through to open the menu.
onKeyDown={(e) => {
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault()
runReloadTrigger('button')
}
}}
onClick={() => runReloadTrigger('button')}
onContextMenu={(e) => {
e.preventDefault()
setReloadMenuOpen(true)
}}
>
{loading ? (
<Loader2 className="size-4 animate-spin" />
) : (
<RefreshCw className="size-4" />
)}
</Button>
</DropdownMenuTrigger>
</TooltipTrigger>
<TooltipContent side="bottom" sideOffset={4}>
{reloadButtonLabel}
{/* Why: the chord maps to plain reload(), which is not what Stop or Retry do — only hint when they match. */}
{reloadShortcut && reloadButtonLabelKind === 'reload' ? ` · ${reloadShortcut}` : ''}
</TooltipContent>
</Tooltip>
<DropdownMenuContent align="start" alignOffset={-4}>
<DropdownMenuItem onClick={() => runReloadTrigger('reload')}>
{translate('auto.components.browser.pane.BrowserPane.0e080d820e', 'Reload')}
<DropdownMenuShortcut>{reloadShortcut}</DropdownMenuShortcut>
</DropdownMenuItem>
<DropdownMenuItem onClick={() => runReloadTrigger('hard-reload')}>
{translate('auto.components.browser.pane.BrowserPane.a1f3c2e4b5', 'Hard Reload')}
<DropdownMenuShortcut>{hardReloadShortcut}</DropdownMenuShortcut>
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
<BrowserAddressBar
value={addressBarValue}
onChange={setAddressBarValue}
onSubmit={submitAddressBar}
onNavigate={navigateToUrl}
inputRef={addressBarInputRef}
dismissSuggestionsRef={dismissAddressBarSuggestionsRef}
/>
<BrowserImportHintButton profileId={sessionProfileId} />
<Tooltip>
<TooltipTrigger asChild>
<span className="inline-flex">
<Button
size="icon"
variant={grab.state !== 'idle' && grabIntent === 'copy' ? 'default' : 'ghost'}
className={cn(
'h-8 w-8',
grab.state !== 'idle' &&
grabIntent === 'copy' &&
'bg-foreground/80 text-background hover:bg-foreground/90'
)}
onClick={() => startGrabIntent('copy')}
disabled={isBlankTab || markupIsActive}
aria-label={translate(
'auto.components.browser.pane.BrowserPane.fdfc7fe0ef',
'Grab page element'
)}
data-contextual-tour-target="browser-grab-control"
>
<Crosshair className="size-4" />
</Button>
</span>
</TooltipTrigger>
<TooltipContent side="bottom" sideOffset={4}>
{translate(
'auto.components.browser.pane.BrowserPane.acbe79fd01',
'Grab page element ({{value0}})',
{ value0: grabElementShortcut }
)}
</TooltipContent>
</Tooltip>
<Tooltip>
<TooltipTrigger asChild>
{/* Why: disabled <button> drops hover events, so wrap in a span so the tooltip trigger still fires. */}
<span className="inline-flex">
<Button
size="icon"
variant={grab.state !== 'idle' && grabIntent === 'annotate' ? 'default' : 'ghost'}
className={cn(
'relative h-8 w-8',
grab.state !== 'idle' &&
grabIntent === 'annotate' &&
'bg-foreground/80 text-background hover:bg-foreground/90'
)}
onClick={() => startGrabIntent('annotate')}
disabled={isBlankTab || markupIsActive}
aria-label={translate(
'auto.components.browser.pane.BrowserPane.fc9be38f6f',
'Annotate page element'
)}
data-contextual-tour-target="browser-annotation-control"
>
<MessageSquarePlus className="size-4" />
{browserAnnotationsLength > 0 ? (
<span className="absolute -top-1 -right-1 flex min-w-4 items-center justify-center rounded-full bg-primary px-1 text-[10px] leading-4 text-primary-foreground">
{browserAnnotationsLength}
</span>
) : null}
</Button>
</span>
</TooltipTrigger>
<TooltipContent side="bottom" sideOffset={4}>
{translate(
'auto.components.browser.pane.BrowserPane.fc9be38f6f',
'Annotate page element'
)}
</TooltipContent>
</Tooltip>
<MarkupDrawButton
onClick={() => (markupIsActive ? markupCancel() : void markupStart())}
disabled={isBlankTab || grab.state !== 'idle'}
active={markupIsActive}
surfaceActive={isActive}
/>
{shareableArtifactFile ? (
<ArtifactPublishButton
sourceKey={shareableArtifactFile.filePath}
className="h-7 w-7"
createRequest={() => readBrowserHtmlArtifactRequest(currentBrowserUrl)}
/>
) : null}
<Button
size="icon"
variant="ghost"
className="h-7 w-7"
onClick={() => void window.api.browser.openDevTools({ browserPageId })}
title={translate(
'auto.components.browser.pane.BrowserPane.ec75d0c412',
'Open browser devtools'
)}
>
<SquareCode className="size-4" />
</Button>
<Button
size="icon"
variant="ghost"
className="h-7 w-7"
onClick={() => {
if (!externalUrl) {
return
}
void window.api.shell.openUrl(externalUrl)
}}
title={translate(
'auto.components.browser.pane.BrowserPane.0f41bf80c7',
'Open in default browser'
)}
disabled={!externalUrl}
>
<ExternalLink className="size-4" />
</Button>
<BrowserToolbarMenu
currentProfileId={sessionProfileId}
workspaceId={workspaceId}
browserPageId={browserPageId}
viewportPresetId={viewportPresetId}
onDestroyWebview={() => destroyPersistentWebview(browserPageId)}
isActive={isActive}
/>
</div>
)
}
@@ -0,0 +1,288 @@
import type { Dispatch, MutableRefObject, RefObject, SetStateAction } from 'react'
import { cn } from '@/lib/utils'
import { Copy, Globe, Image } from 'lucide-react'
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuSeparator,
DropdownMenuShortcut,
DropdownMenuTrigger
} from '@/components/ui/dropdown-menu'
import { translate } from '@/i18n/i18n'
import { toHttpsRecoveryUrl } from '../../../../../shared/browser-url'
import type {
BrowserCertificateFailure,
BrowserPage as BrowserPageState
} from '../../../../../shared/browser-workspace-types'
import { BROWSER_GUEST_RECOVERY_ERROR_CODE } from '../host-guest/browser-page-guest-recovery'
import { BrowserLoadFailureOverlay } from '../navigate/browser-load-failure-overlay'
import BrowserFind from './BrowserFind'
import { MarkupOverlay } from '../annotate/MarkupOverlay'
import type { MarkupModeController } from '../annotate/useMarkupMode'
import type { GrabModeHook } from '../annotate/useGrabMode'
import {
getBrowserOverlayAnchor,
type BrowserOverlayViewport
} from '../describe-page/browser-annotation-geometry'
import { BrowserPageAnnotationTray } from '../annotate/browser-page-annotation-tray'
import { BrowserPageGrabToast } from '../annotate/browser-page-grab-toast'
import { retryBrowserTabLoad, toDisplayUrl } from '../describe-page/browser-page-url-display'
import { PendingBrowserAnnotationCard } from '../annotate/pending-browser-annotation-card'
import type { BrowserTabPageState } from '../describe-page/browser-page-types'
import type { useBrowserPageAnnotationSend } from '../annotate/use-browser-page-annotation-send'
import type { useBrowserPageGrabAnnotations } from '../annotate/use-browser-page-grab-annotations'
export function BrowserPageViewportOverlays({
markup,
browserZoomIndicatorState,
browserZoomPercent,
findOpen,
setFindOpen,
webviewRef,
showFailureOverlay,
browserTab,
failureExternalUrl,
failedNavigationUrl,
onUpdatePageStateRef,
retryGuestRecoveryRef,
navigateToUrl,
setResourceNotice,
certificateFailure,
isBlankTab,
containerRef,
browserOverlayViewport,
worktreeId,
grab,
annotationSend,
grabAnnotations
}: {
markup: MarkupModeController
browserZoomIndicatorState: { ariaHidden: boolean; opacityClassName: string }
browserZoomPercent: number
findOpen: boolean
setFindOpen: Dispatch<SetStateAction<boolean>>
webviewRef: MutableRefObject<Electron.WebviewTag | null>
showFailureOverlay: boolean
browserTab: BrowserPageState
failureExternalUrl: string | null
failedNavigationUrl: string
onUpdatePageStateRef: MutableRefObject<(tabId: string, updates: BrowserTabPageState) => void>
retryGuestRecoveryRef: MutableRefObject<() => void>
navigateToUrl: (url: string) => void
setResourceNotice: Dispatch<SetStateAction<string | null>>
certificateFailure: BrowserCertificateFailure | null
isBlankTab: boolean
containerRef: RefObject<HTMLDivElement | null>
browserOverlayViewport: BrowserOverlayViewport
worktreeId: string
grab: GrabModeHook
annotationSend: ReturnType<typeof useBrowserPageAnnotationSend>
grabAnnotations: ReturnType<typeof useBrowserPageGrabAnnotations>
}): React.JSX.Element {
const {
pendingAnnotationPayload,
handleAddBrowserAnnotation,
handleCancelPendingBrowserAnnotation,
grabIntent,
grabMenuActionTakenRef,
handleGrabCopy,
handleGrabCopyScreenshot,
grabToast,
grabToastTimerRef,
dismissGrabToast,
setGrabToast
} = grabAnnotations
const {
browserAnnotations,
browserAnnotationTrayOpen,
annotationTraySendOpen,
handleAnnotationTraySendOpenChange,
activeGroupId,
browserAnnotationsPrompt,
handleBrowserAnnotationsSentToAgent,
handleCopyBrowserAnnotations,
browserAnnotationsCopied,
handleClearBrowserAnnotations,
handleDeleteBrowserAnnotation
} = annotationSend
return (
<>
{markup.isActive && markup.baseImage ? (
<MarkupOverlay
baseImage={markup.baseImage}
busy={markup.state === 'composing'}
onComplete={(input) => void markup.complete(input)}
onCancel={markup.cancel}
/>
) : null}
<div
role="status"
aria-live="polite"
aria-hidden={browserZoomIndicatorState.ariaHidden}
className={cn(
'pointer-events-none absolute top-3 right-3 z-30 rounded-md border border-border bg-popover/95 px-2.5 py-1 text-xs font-medium text-popover-foreground shadow-xs transition-opacity duration-300 ease-out',
browserZoomIndicatorState.opacityClassName
)}
>
{browserZoomPercent}%
</div>
<BrowserFind isOpen={findOpen} onClose={() => setFindOpen(false)} webviewRef={webviewRef} />
{showFailureOverlay && browserTab.loadError ? (
<BrowserLoadFailureOverlay
loadError={browserTab.loadError}
externalUrl={failureExternalUrl}
currentUrl={toDisplayUrl(failedNavigationUrl)}
httpsRecoveryUrl={toHttpsRecoveryUrl(failedNavigationUrl)}
onRetry={() => {
const webview = webviewRef.current
if (!webview) {
return
}
onUpdatePageStateRef.current(browserTab.id, { loading: true })
if (browserTab.loadError?.code === BROWSER_GUEST_RECOVERY_ERROR_CODE) {
retryGuestRecoveryRef.current()
return
}
retryBrowserTabLoad(webview, browserTab, onUpdatePageStateRef.current)
}}
onTryHttps={navigateToUrl}
onCopy={(url) => {
void window.api.ui.writeClipboardText(url)
setResourceNotice(
translate('browser.loadFailure.addressCopied', 'Copied the current page address.')
)
}}
onOpenExternal={(url) => void window.api.shell.openUrl(url)}
certificateFailure={certificateFailure}
expectedBrowserPageId={browserTab.id}
onProceedCertificate={(challengeId) =>
window.api.browser.proceedCertificate({
browserPageId: browserTab.id,
challengeId
})
}
/>
) : null}
{isBlankTab ? (
<div className="pointer-events-none absolute inset-0 z-10 flex items-center justify-center bg-[radial-gradient(circle_at_center,rgba(255,255,255,0.02),transparent_58%)] px-6">
<div className="flex flex-col items-center px-8 py-8 text-center opacity-70">
<div className="mb-4 rounded-full border border-border/70 bg-muted/30 p-3">
<Globe className="size-5 text-muted-foreground" />
</div>
<div className="text-center">
<p className="text-base font-semibold text-foreground/85">
{translate('auto.components.browser.pane.BrowserPane.366bf5d62c', 'New Tab')}
</p>
<p className="mt-2 text-sm text-muted-foreground">
{translate(
'auto.components.browser.pane.BrowserPane.f796c774a4',
'Type a URL above to start browsing.'
)}
</p>
</div>
</div>
</div>
) : null}
{pendingAnnotationPayload ? (
<PendingBrowserAnnotationCard
payload={pendingAnnotationPayload}
anchor={getBrowserOverlayAnchor(
pendingAnnotationPayload,
containerRef.current,
webviewRef.current,
browserOverlayViewport
)}
portalContainer={containerRef.current}
onAdd={handleAddBrowserAnnotation}
onCancel={handleCancelPendingBrowserAnnotation}
/>
) : null}
{browserAnnotations.length > 0 && browserAnnotationTrayOpen ? (
<BrowserPageAnnotationTray
browserAnnotations={browserAnnotations}
annotationTraySendOpen={annotationTraySendOpen}
handleAnnotationTraySendOpenChange={handleAnnotationTraySendOpenChange}
worktreeId={worktreeId}
activeGroupId={activeGroupId}
browserAnnotationsPrompt={browserAnnotationsPrompt}
handleBrowserAnnotationsSentToAgent={handleBrowserAnnotationsSentToAgent}
handleCopyBrowserAnnotations={handleCopyBrowserAnnotations}
browserAnnotationsCopied={browserAnnotationsCopied}
handleClearBrowserAnnotations={handleClearBrowserAnnotations}
handleDeleteBrowserAnnotation={handleDeleteBrowserAnnotation}
/>
) : null}
{/* Right-click context dropdown, positioned at the grabbed element's center. */}
<DropdownMenu
open={grab.state === 'confirming' && grab.contextMenu && grabIntent === 'copy'}
onOpenChange={(open) => {
if (!open && grab.state === 'confirming') {
// Why: skip rearm if a menu action already handled it — see grabMenuActionTakenRef.
if (grabMenuActionTakenRef.current) {
grabMenuActionTakenRef.current = false
return
}
grab.rearm()
}
}}
>
<DropdownMenuTrigger asChild>
<button
aria-hidden
tabIndex={-1}
className="pointer-events-none absolute size-px opacity-0"
style={(() => {
if (!grab.payload) {
return { left: 0, top: 0 }
}
const rect = grab.payload.target.rectViewport
const webview = webviewRef.current
const webviewRect = webview?.getBoundingClientRect()
const cRect = containerRef.current?.getBoundingClientRect()
const offsetX = (webviewRect?.left ?? 0) - (cRect?.left ?? 0)
const offsetY = (webviewRect?.top ?? 0) - (cRect?.top ?? 0)
return {
left: offsetX + rect.x + rect.width / 2,
top: offsetY + rect.y + rect.height / 2
}
})()}
/>
</DropdownMenuTrigger>
<DropdownMenuContent align="start" sideOffset={4}>
<DropdownMenuItem onSelect={handleGrabCopy}>
<Copy className="size-3.5" />
{translate('auto.components.browser.pane.BrowserPane.c2ef0359b9', 'Copy Contents')}
<DropdownMenuShortcut>C</DropdownMenuShortcut>
</DropdownMenuItem>
{grab.payload?.screenshot?.dataUrl?.startsWith('data:image/png;base64,') ? (
<DropdownMenuItem onSelect={handleGrabCopyScreenshot}>
<Image className="size-3.5" />
{translate('auto.components.browser.pane.BrowserPane.1ded0d3168', 'Copy Screenshot')}
<DropdownMenuShortcut>S</DropdownMenuShortcut>
</DropdownMenuItem>
) : null}
<DropdownMenuSeparator />
<DropdownMenuItem
onSelect={() => {
grabMenuActionTakenRef.current = true
grab.cancel()
}}
>
{translate('auto.components.browser.pane.BrowserPane.fa6ea61de3', 'Cancel')}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
{/* Inline toast bubble; flips above the element when near the viewport bottom so it doesn't occlude it. */}
{grabToast ? (
<BrowserPageGrabToast
grabToast={grabToast}
grabToastTimerRef={grabToastTimerRef}
dismissGrabToast={dismissGrabToast}
setGrabToast={setGrabToast}
/>
) : null}
</>
)
}
@@ -1,6 +1,6 @@
import { describe, expect, it } from 'vitest'
import { getBrowserPagesForWorkspace } from './browser-pane-page-selection'
import type { BrowserPage } from '../../../../shared/browser-workspace-types'
import type { BrowserPage } from '../../../../../shared/browser-workspace-types'
function makeBrowserPage(id: string): BrowserPage {
return {
@@ -1,4 +1,4 @@
import type { BrowserPage } from '../../../../shared/browser-workspace-types'
import type { BrowserPage } from '../../../../../shared/browser-workspace-types'
const EMPTY_BROWSER_PAGES: BrowserPage[] = []
@@ -15,11 +15,11 @@ import {
} from '@/components/ui/dropdown-menu'
import { BrowserCookieImportDisclosure } from '@/components/BrowserCookieImportDisclosure'
import { useAppStore } from '@/store'
import { BROWSER_FAMILY_LABELS } from '../../../../shared/constants'
import { BROWSER_FAMILY_LABELS } from '../../../../../shared/constants'
import type {
BrowserSessionProfile,
BrowserViewportPresetId
} from '../../../../shared/browser-workspace-types'
} from '../../../../../shared/browser-workspace-types'
type DetectedBrowserEntry = {
family: string
@@ -27,7 +27,7 @@ type DetectedBrowserEntry = {
profiles: { name: string; directory: string }[]
selectedProfile: string
}
import { BROWSER_VIEWPORT_PRESETS } from '../../../../shared/browser-viewport-presets'
import { BROWSER_VIEWPORT_PRESETS } from '../../../../../shared/browser-viewport-presets'
import { translate } from '@/i18n/i18n'
type BrowserToolbarMenuDropdownProps = {
@@ -9,7 +9,7 @@ import {
DialogTitle
} from '@/components/ui/dialog'
import { translate } from '@/i18n/i18n'
import { BrowserProfileUserAgentOption } from '../browser-profile-user-agent-option'
import { BrowserProfileUserAgentOption } from '../../browser-profile-user-agent-option'
type BrowserToolbarProfileDialogsProps = {
pendingSwitchProfileId: string | null | undefined
@@ -0,0 +1,130 @@
import { useCallback, useEffect, useMemo } from 'react'
import { useAppStore } from '@/store'
import { getRuntimeEnvironmentIdForWorktree } from '@/lib/worktree-runtime-owner'
import type { BrowserWorkspace as BrowserWorkspaceState } from '../../../../../shared/browser-workspace-types'
import { destroyPersistentWebview } from '../host-guest/webview-registry'
import { useBrowserAutomationVisiblePageIds } from '../host-guest/browser-automation-visibility'
import { getBrowserPagesForWorkspace } from './browser-pane-page-selection'
import { BrowserMobileDriverOverlay } from './BrowserMobileDriverOverlay'
import {
IDLE_BROWSER_DRIVER,
useBrowserDriverForPage,
useBrowserMobileDrivenPageIds
} from '@/lib/pane-manager/browser-mobile-driver-state'
import { useContextualTour } from '@/components/contextual-tours/use-contextual-tour'
import { getBrowserPageRuntimeEnvironmentId } from '../describe-page/browser-page-url-display'
import type { BrowserFindShortcutScope } from '../describe-page/browser-page-types'
import { RemoteBrowserPagePane } from '../stream-remote/remote-browser-page-pane'
import { BrowserPagePane } from './browser-page-pane'
export default function BrowserPane({
browserTab,
isActive,
findShortcutScope
}: {
browserTab: BrowserWorkspaceState
isActive: boolean
findShortcutScope?: BrowserFindShortcutScope
}): React.JSX.Element {
const resolvedFindShortcutScope = findShortcutScope ?? (isActive ? 'focused' : 'inactive')
const activeRuntimeEnvironmentId = useAppStore((s) =>
getRuntimeEnvironmentIdForWorktree(s, browserTab.worktreeId)
)
const browserPages = useAppStore((s) =>
getBrowserPagesForWorkspace(s.browserPagesByWorkspace, browserTab.id)
)
const activeBrowserPage =
browserPages.find((page) => page.id === browserTab.activePageId) ?? browserPages[0] ?? null
const updateBrowserPageState = useAppStore((s) => s.updateBrowserPageState)
const setBrowserPageUrl = useAppStore((s) => s.setBrowserPageUrl)
const activeBrowserRuntimeEnvironmentId = activeBrowserPage
? getBrowserPageRuntimeEnvironmentId(activeBrowserPage, activeRuntimeEnvironmentId)
: null
const runtimeEnvironmentActive = Boolean(activeBrowserRuntimeEnvironmentId)
const activeBrowserPageId = activeBrowserPage?.id ?? null
const browserPageIds = useMemo(() => browserPages.map((page) => page.id), [browserPages])
const automationVisiblePageIds = useBrowserAutomationVisiblePageIds(browserPageIds)
const mobileDrivenPageIds = useBrowserMobileDrivenPageIds(browserPageIds)
// Why: inactive webviews must stay mounted in their original DOM parent; unmounting/reparenting loses form text and SPA state.
const renderedBrowserPages = browserPages.filter(
(page) => !getBrowserPageRuntimeEnvironmentId(page, activeRuntimeEnvironmentId)
)
const pageDriver = useBrowserDriverForPage(activeBrowserPageId)
// Why: a runtime-backed page is streamed, never locally driven, so its driver must read idle.
const activeBrowserDriver = runtimeEnvironmentActive ? IDLE_BROWSER_DRIVER : pageDriver
useEffect(() => {
if (!runtimeEnvironmentActive) {
return
}
for (const page of browserPages) {
if (getBrowserPageRuntimeEnvironmentId(page, activeRuntimeEnvironmentId)) {
destroyPersistentWebview(page.id)
}
}
}, [activeRuntimeEnvironmentId, browserPages, runtimeEnvironmentActive])
useContextualTour(
'browser',
isActive && activeBrowserPage !== null && !runtimeEnvironmentActive,
'browser_visible'
)
const reclaimActiveBrowserForDesktop = useCallback(async (): Promise<void> => {
if (!activeBrowserPageId) {
return
}
const { reclaimed } = await window.api.runtime.reclaimBrowserForDesktop(activeBrowserPageId)
if (!reclaimed) {
throw new Error('Could not reclaim browser control')
}
}, [activeBrowserPageId])
if (activeBrowserRuntimeEnvironmentId) {
return activeBrowserPage ? (
<RemoteBrowserPagePane
key={`${activeBrowserRuntimeEnvironmentId ?? ''}:${activeBrowserPage.id}`}
browserTab={activeBrowserPage}
runtimeEnvironmentId={activeBrowserRuntimeEnvironmentId}
worktreeId={browserTab.worktreeId}
isActive={isActive}
onUpdatePageState={updateBrowserPageState}
onSetUrl={setBrowserPageUrl}
/>
) : (
<div className="flex h-full min-h-0 flex-1 bg-background" />
)
}
return (
<div className="relative flex h-full min-h-0 flex-1 flex-col">
{renderedBrowserPages.length > 0 ? (
<div className="relative flex min-h-0 flex-1">
{renderedBrowserPages.map((page) => (
<BrowserPagePane
key={page.id}
browserTab={page}
workspaceId={browserTab.id}
worktreeId={browserTab.worktreeId}
sessionProfileId={browserTab.sessionProfileId ?? null}
sessionPartition={browserTab.sessionPartition ?? null}
isActive={isActive && page.id === activeBrowserPage?.id}
findShortcutScope={
page.id === activeBrowserPage?.id ? resolvedFindShortcutScope : 'inactive'
}
isAutomationVisible={automationVisiblePageIds.has(page.id)}
isMobileDriven={mobileDrivenPageIds.has(page.id)}
inputLocked={activeBrowserDriver.kind === 'mobile'}
onUpdatePageState={updateBrowserPageState}
onSetUrl={setBrowserPageUrl}
/>
))}
<BrowserMobileDriverOverlay
driver={activeBrowserDriver}
onTakeBack={reclaimActiveBrowserForDesktop}
/>
</div>
) : null}
</div>
)
}
@@ -0,0 +1,160 @@
import { useCallback, useEffect, type MutableRefObject, type RefObject } from 'react'
import { useAppStore } from '@/store'
import {
consumeBrowserFocusRequest,
ORCA_BROWSER_FOCUS_REQUEST_EVENT,
type BrowserFocusRequestDetail
} from '../host-guest/browser-focus'
export function useBrowserPageChromeFocus({
browserTabId,
isActive,
addressBarInputRef,
webviewRef,
keepAddressBarFocusRef
}: {
browserTabId: string
isActive: boolean
addressBarInputRef: RefObject<HTMLInputElement | null>
webviewRef: MutableRefObject<Electron.WebviewTag | null>
keepAddressBarFocusRef: MutableRefObject<boolean>
}): {
focusAddressBarNow: () => boolean
focusWebviewNow: () => boolean
} {
const consumeAddressBarFocusRequest = useAppStore((s) => s.consumeAddressBarFocusRequest)
const focusAddressBarNow = useCallback(() => {
const input = addressBarInputRef.current
if (!input) {
return false
}
webviewRef.current?.blur()
input.focus()
input.select()
return document.activeElement === input
}, [addressBarInputRef, webviewRef])
const focusWebviewNow = useCallback(() => {
const webview = webviewRef.current
if (!webview) {
return false
}
addressBarInputRef.current?.blur()
try {
webview.focus()
} catch {
// Why: WebViewElement.focus() reads null internals once the guest is destroyed (STA-3448).
return false
}
return document.activeElement === webview
}, [addressBarInputRef, webviewRef])
useEffect(() => {
if (!isActive) {
return
}
if (!consumeAddressBarFocusRequest(browserTabId)) {
return
}
keepAddressBarFocusRef.current = true
// Why: terminal activation re-grabs focus a frame later; retry a few frames to win the race, but stay one-shot so revisiting the tab doesn't steal focus.
let cancelled = false
let frameId = 0
let attempts = 0
const focusAddressBar = (): void => {
if (cancelled) {
return
}
focusAddressBarNow()
attempts += 1
if (attempts < 6) {
frameId = window.requestAnimationFrame(focusAddressBar)
} else {
keepAddressBarFocusRef.current = false
}
}
frameId = window.requestAnimationFrame(focusAddressBar)
return () => {
cancelled = true
window.cancelAnimationFrame(frameId)
// Why: aborting mid-retry would otherwise latch the flag on and suppress guest focus forever.
keepAddressBarFocusRef.current = false
}
}, [
browserTabId,
consumeAddressBarFocusRequest,
focusAddressBarNow,
isActive,
keepAddressBarFocusRef
])
useEffect(() => {
if (!isActive) {
return
}
return window.api.ui.onFocusBrowserAddressBar(() => {
focusAddressBarNow()
})
}, [focusAddressBarNow, isActive])
useEffect(() => {
if (!isActive) {
return
}
const focusTarget = consumeBrowserFocusRequest(browserTabId)
if (!focusTarget) {
return
}
keepAddressBarFocusRef.current = focusTarget === 'address-bar'
let cancelled = false
let frameId = 0
let attempts = 0
const runFocus = (): void => {
if (cancelled) {
return
}
const didFocus = focusTarget === 'address-bar' ? focusAddressBarNow() : focusWebviewNow()
attempts += 1
if (!didFocus && attempts < 6) {
frameId = window.requestAnimationFrame(runFocus)
}
}
// Why: focus can be queued before the pane mounts; persisting outside React lets it be claimed on mount instead of racing an event.
frameId = window.requestAnimationFrame(runFocus)
return () => {
cancelled = true
window.cancelAnimationFrame(frameId)
}
}, [browserTabId, focusAddressBarNow, focusWebviewNow, isActive, keepAddressBarFocusRef])
useEffect(() => {
if (!isActive) {
return
}
const handleBrowserFocusRequest = (event: Event): void => {
const detail = (event as CustomEvent<BrowserFocusRequestDetail>).detail
if (!detail || detail.pageId !== browserTabId) {
return
}
const focusTarget = consumeBrowserFocusRequest(browserTabId)
if (!focusTarget) {
return
}
if (focusTarget === 'address-bar') {
// Why: palette-triggered address-bar focus must survive the same follow-up load events as the blank-tab path.
keepAddressBarFocusRef.current = true
focusAddressBarNow()
return
}
keepAddressBarFocusRef.current = false
focusWebviewNow()
}
// Why: an already-active page never remounts, so listen for the event to consume the durable focus request immediately.
window.addEventListener(ORCA_BROWSER_FOCUS_REQUEST_EVENT, handleBrowserFocusRequest)
return () =>
window.removeEventListener(ORCA_BROWSER_FOCUS_REQUEST_EVENT, handleBrowserFocusRequest)
}, [browserTabId, focusAddressBarNow, focusWebviewNow, isActive, keepAddressBarFocusRef])
return { focusAddressBarNow, focusWebviewNow }
}
@@ -0,0 +1,68 @@
import { useEffect, type Dispatch, type SetStateAction } from 'react'
import { getShortcutPlatform } from '@/hooks/useShortcutLabel'
import { useAppStore } from '@/store'
import { keybindingMatchesAction } from '../../../../../shared/keybindings'
import { browserOverlayOwnsShortcutTarget } from '../describe-page/browser-overlay-shortcut-target'
import type { BrowserFindShortcutScope } from '../describe-page/browser-page-types'
export function useBrowserPageFindShortcuts({
browserTabId,
workspaceId,
isActive,
findShortcutScope,
setFindOpen
}: {
browserTabId: string
workspaceId: string
isActive: boolean
findShortcutScope: BrowserFindShortcutScope
setFindOpen: Dispatch<SetStateAction<boolean>>
}): void {
const keybindings = useAppStore((state) => state.keybindings)
// Cmd/Ctrl+F — find in page (renderer path: focus on browser chrome)
// Why: unlike bare C/S grab shortcuts, Cmd+F should always open find even from the address bar (matches Chrome/Safari).
useEffect(() => {
if (findShortcutScope === 'inactive') {
return
}
const shortcutPlatform = getShortcutPlatform()
const handleKeyDown = (e: KeyboardEvent): void => {
if (!keybindingMatchesAction('browser.find', e, shortcutPlatform, keybindings)) {
return
}
if (
findShortcutScope === 'owned-target' &&
!browserOverlayOwnsShortcutTarget(e.target, workspaceId)
) {
return
}
e.preventDefault()
e.stopPropagation()
setFindOpen(true)
}
window.addEventListener('keydown', handleKeyDown, true)
return () => window.removeEventListener('keydown', handleKeyDown, true)
}, [findShortcutScope, keybindings, setFindOpen, workspaceId])
// Cmd/Ctrl+F — find in page (IPC path: focus inside webview guest)
// Why: a focused guest is a separate Chromium process, so main forwards the chord back here.
useEffect(() => {
if (!isActive) {
return
}
return window.api.ui.onFindInBrowserPage(
{ browserPageId: browserTabId, browserWorkspaceId: workspaceId },
() => {
setFindOpen(true)
}
)
}, [browserTabId, isActive, setFindOpen, workspaceId])
// Close find bar when tab is deactivated
useEffect(() => {
if (!isActive) {
setFindOpen(false)
}
}, [isActive, setFindOpen])
}
@@ -1,36 +0,0 @@
import { describe, expect, it } from 'vitest'
import { isEditableKeyboardTarget } from './browser-keyboard'
describe('isEditableKeyboardTarget', () => {
it('returns true for input elements', () => {
const input = {
isContentEditable: false,
closest: (selector: string) => (selector.includes('input') ? {} : null)
}
expect(isEditableKeyboardTarget(input)).toBe(true)
})
it('returns true for descendants inside editable hosts', () => {
const child = {
isContentEditable: false,
closest: (selector: string) => (selector.includes('[contenteditable="true"]') ? {} : null)
}
expect(isEditableKeyboardTarget(child)).toBe(true)
})
it('returns true for Monaco editor descendants', () => {
const child = {
isContentEditable: false,
closest: (selector: string) => (selector.includes('.monaco-editor') ? {} : null)
}
expect(isEditableKeyboardTarget(child)).toBe(true)
})
it('returns false for non-editable elements', () => {
const div = {
isContentEditable: false,
closest: () => null
}
expect(isEditableKeyboardTarget(div)).toBe(false)
})
})
@@ -0,0 +1,63 @@
import { describe, expect, it } from 'vitest'
import type { BrowserGrabPayload } from '../../../../../shared/browser-grab-types'
import {
clampNumber,
createBrowserAnnotationPayload,
getLiveBrowserAnnotationRect
} from './browser-annotation-geometry'
function payload(overrides: Partial<BrowserGrabPayload['target']> = {}): BrowserGrabPayload {
return {
target: {
selector: 'button',
tagName: 'BUTTON',
textSnippet: 'Go',
isFixed: false,
rectPage: { x: 100, y: 200, width: 40, height: 20 },
rectViewport: { x: 10, y: 20, width: 40, height: 20 },
accessibility: { accessibleName: 'Go' },
...overrides
},
nearbyText: [],
ancestorPath: [],
page: { url: 'https://example.com', title: 'Example', scrollX: 5, scrollY: 15 },
screenshot: { dataUrl: 'data:image/png;base64,xx', width: 1, height: 1 }
} as unknown as BrowserGrabPayload
}
describe('browser annotation geometry', () => {
it('clamps inclusive of both ends', () => {
expect(clampNumber(5, 0, 10)).toBe(5)
expect(clampNumber(-1, 0, 10)).toBe(0)
expect(clampNumber(11, 0, 10)).toBe(10)
})
it('uses the viewport rect for fixed targets and scroll-adjusts flowing targets', () => {
const fixed = payload({ isFixed: true })
expect(getLiveBrowserAnnotationRect(fixed, { scrollX: 99, scrollY: 99, version: 2 })).toEqual(
fixed.target.rectViewport
)
const flowing = payload({ isFixed: false })
expect(getLiveBrowserAnnotationRect(flowing, { scrollX: 30, scrollY: 40, version: 1 })).toEqual(
{
...flowing.target.rectViewport,
x: 70,
y: 160
}
)
expect(getLiveBrowserAnnotationRect(flowing, { scrollX: 30, scrollY: 40, version: 0 })).toEqual(
{
...flowing.target.rectViewport,
x: 95,
y: 185
}
)
})
it('strips screenshot bytes from persisted annotation payloads', () => {
const source = payload()
expect(createBrowserAnnotationPayload(source).screenshot).toBeNull()
expect(createBrowserAnnotationPayload(source).target.selector).toBe('button')
})
})
@@ -0,0 +1,102 @@
import { MessageCircleQuestionMark, PencilLine } from 'lucide-react'
import type {
BrowserAnnotationPayload,
BrowserAnnotationPriority,
BrowserGrabPayload,
BrowserGrabRect,
BrowserPageAnnotation
} from '../../../../../shared/browser-grab-types'
import { translate } from '@/i18n/i18n'
export type BrowserOverlayAnchor = {
x: number
y: number
below: boolean
}
export const BROWSER_ANNOTATION_INTENT_OPTIONS = [
{
value: 'change',
get label() {
return translate('auto.components.browser.pane.BrowserPane.143204e423', 'Change')
},
icon: PencilLine
},
{
value: 'question',
get label() {
return translate('auto.components.browser.pane.BrowserPane.b5ba6085de', 'Question')
},
icon: MessageCircleQuestionMark
}
] as const
// Why: priority stays in the persisted annotation shape for backwards compat, though the UI no longer exposes urgency choices.
export const DEFAULT_BROWSER_ANNOTATION_PRIORITY: BrowserAnnotationPriority = 'important'
export const BROWSER_PAGE_ZOOM_FEEDBACK_MS = 1400
export type BrowserOverlayViewport = {
scrollX: number
scrollY: number
version: number
}
export const EMPTY_BROWSER_ANNOTATIONS: BrowserPageAnnotation[] = []
export const PENDING_ANNOTATION_CARD_HEIGHT = 330
export function createBrowserAnnotationId(): string {
return `browser-annotation-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`
}
export function createBrowserAnnotationPayload(
payload: BrowserGrabPayload
): BrowserAnnotationPayload {
return {
...payload,
// Why: annotations are persisted; screenshot data is a transient copy payload that can be megabytes per selection.
screenshot: null
}
}
export function getBrowserOverlayAnchor(
payload: BrowserGrabPayload,
container: HTMLElement | null,
webview: Electron.WebviewTag | null,
viewport: BrowserOverlayViewport
): BrowserOverlayAnchor {
const containerRect = container?.getBoundingClientRect()
const webviewRect = webview?.getBoundingClientRect()
const rect = getLiveBrowserAnnotationRect(payload, viewport)
const offsetX = (webviewRect?.left ?? 0) - (containerRect?.left ?? 0)
const offsetY = (webviewRect?.top ?? 0) - (containerRect?.top ?? 0)
const elementBottom = offsetY + rect.y + rect.height
const elementTop = offsetY + rect.y
const containerWidth = containerRect?.width ?? 0
const containerHeight = containerRect?.height ?? 0
const below = elementBottom + PENDING_ANNOTATION_CARD_HEIGHT < containerHeight
return {
x: clampNumber(offsetX + rect.x + rect.width / 2, 12, Math.max(12, containerWidth - 12)),
y: clampNumber(below ? elementBottom : elementTop, 12, Math.max(12, containerHeight - 12)),
below
}
}
export function clampNumber(value: number, min: number, max: number): number {
return Math.min(Math.max(value, min), max)
}
export function getLiveBrowserAnnotationRect(
payload: BrowserGrabPayload,
viewport: BrowserOverlayViewport
): BrowserGrabRect {
if (payload.target.isFixed) {
return payload.target.rectViewport
}
const scrollX = viewport.version === 0 ? payload.page.scrollX : viewport.scrollX
const scrollY = viewport.version === 0 ? payload.page.scrollY : viewport.scrollY
return {
...payload.target.rectViewport,
x: payload.target.rectPage.x - scrollX,
y: payload.target.rectPage.y - scrollY
}
}
@@ -1,7 +1,7 @@
// @vitest-environment happy-dom
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { ARTIFACT_CLI_MAX_RPC_BYTES } from '../../../../shared/artifacts'
import { ARTIFACT_CLI_MAX_RPC_BYTES } from '../../../../../shared/artifacts'
import type { ArtifactPublishPreparationError } from '@/components/artifacts/artifact-publish-flow'
import {
getShareableBrowserArtifactFile,
@@ -1,6 +1,6 @@
import type { ArtifactWriteRequest } from '../../../../shared/artifacts'
import { ARTIFACT_CLI_MAX_RPC_BYTES } from '../../../../shared/artifacts'
import { getRuntimePathBasename } from '../../../../shared/cross-platform-path'
import type { ArtifactWriteRequest } from '../../../../../shared/artifacts'
import { ARTIFACT_CLI_MAX_RPC_BYTES } from '../../../../../shared/artifacts'
import { getRuntimePathBasename } from '../../../../../shared/cross-platform-path'
import { ArtifactPublishPreparationError } from '@/components/artifacts/artifact-publish-flow'
export type ShareableBrowserArtifactFile = {
@@ -0,0 +1,18 @@
// @vitest-environment happy-dom
import { describe, expect, it } from 'vitest'
import { browserOverlayOwnsShortcutTarget } from './browser-overlay-shortcut-target'
describe('browserOverlayOwnsShortcutTarget', () => {
it('matches only overlay descendants tagged with the same tab id', () => {
const overlay = document.createElement('div')
overlay.setAttribute('data-browser-overlay-tab-id', 'tab-1')
const child = document.createElement('button')
overlay.appendChild(child)
document.body.appendChild(overlay)
expect(browserOverlayOwnsShortcutTarget(child, 'tab-1')).toBe(true)
expect(browserOverlayOwnsShortcutTarget(child, 'tab-2')).toBe(false)
expect(browserOverlayOwnsShortcutTarget(document.body, 'tab-1')).toBe(false)
expect(browserOverlayOwnsShortcutTarget(null, 'tab-1')).toBe(false)
})
})
@@ -0,0 +1,12 @@
export function browserOverlayOwnsShortcutTarget(
target: EventTarget | null,
browserTabId: string
): boolean {
if (!(target instanceof Element)) {
return false
}
return (
target.closest('[data-browser-overlay-tab-id]')?.getAttribute('data-browser-overlay-tab-id') ===
browserTabId
)
}
@@ -0,0 +1,23 @@
import { describe, expect, it } from 'vitest'
import { buildLoadError } from './browser-page-load-error'
describe('buildLoadError', () => {
it('fills defaults and redacts the validated URL', () => {
expect(buildLoadError({})).toEqual({
code: -1,
description: 'Unknown load failure',
validatedUrl: 'about:blank'
})
expect(
buildLoadError({
errorCode: -102,
errorDescription: 'ERR_CONNECTION_REFUSED',
validatedURL: 'https://example.com/app'
})
).toEqual({
code: -102,
description: 'ERR_CONNECTION_REFUSED',
validatedUrl: 'https://example.com/app'
})
})
})
@@ -0,0 +1,21 @@
import { useAppStore } from '@/store'
import type { BrowserLoadError } from '../../../../../shared/browser-workspace-types'
import { redactKagiSessionToken } from '../../../../../shared/browser-url'
export function browserPageExists(tabId: string): boolean {
return Object.values(useAppStore.getState().browserPagesByWorkspace).some((pages) =>
pages.some((page) => page.id === tabId)
)
}
export function buildLoadError(event: {
errorCode?: number
errorDescription?: string
validatedURL?: string
}): BrowserLoadError {
return {
code: event.errorCode ?? -1,
description: event.errorDescription ?? 'Unknown load failure',
validatedUrl: redactKagiSessionToken(event.validatedURL ?? 'about:blank')
}
}
@@ -0,0 +1,54 @@
import type { BrowserGrabPayload } from '../../../../../shared/browser-grab-types'
import type { BrowserPage as BrowserPageState } from '../../../../../shared/browser-workspace-types'
export type BrowserTabPageState = Partial<
Pick<
BrowserPageState,
'title' | 'loading' | 'faviconUrl' | 'canGoBack' | 'canGoForward' | 'loadError'
>
>
export type BrowserPageUrlSetter = (
tabId: string,
url: string,
options?: { preserveLoadError?: boolean }
) => void
export type BrowserFindShortcutScope = 'focused' | 'inactive' | 'owned-target'
export type GrabIntent = 'copy' | 'annotate'
export type BrowserPageContextMenuState = {
x: number
y: number
linkUrl: string | null
pageUrl: string
selectionText: string
}
export type BrowserPageGrabToastState = {
message: string
type: 'success' | 'error'
x: number
y: number
below: boolean
payload: BrowserGrabPayload | null
}
export type BrowserPageRecoveryNavigationValidation = {
committed: boolean
started: boolean
targetUrl: string
}
export type BrowserPageNavigateEvent = {
url?: string
isMainFrame?: boolean
}
export type BrowserPageFailLoadEvent = {
errorCode?: number
errorDescription?: string
validatedURL?: string
isMainFrame?: boolean
}
@@ -0,0 +1,90 @@
import { describe, expect, it, vi } from 'vitest'
import { ORCA_BROWSER_BLANK_URL } from '../../../../../shared/constants'
import {
getBrowserDisplayTitle,
getBrowserPageRuntimeEnvironmentId,
getNotebookPathFromBrowserUrl,
getOpenableExternalUrl,
isChromiumErrorPage,
retryBrowserTabLoad,
toDisplayUrl
} from './browser-page-url-display'
import type { BrowserPage as BrowserPageState } from '../../../../../shared/browser-workspace-types'
describe('browser page URL display', () => {
it('maps the blank-tab sentinel to about:blank and redacts Kagi session tokens', () => {
expect(toDisplayUrl(ORCA_BROWSER_BLANK_URL)).toBe('about:blank')
expect(toDisplayUrl('https://kagi.com/search?q=a&token=secret')).not.toContain('secret')
})
it('titles blank tabs New Tab and otherwise uses the provided title', () => {
expect(getBrowserDisplayTitle('Example', 'https://example.com')).toBe('Example')
expect(getBrowserDisplayTitle(null, 'about:blank')).toBe('New Tab')
expect(getBrowserDisplayTitle('about:blank', 'https://example.com')).toBe('New Tab')
expect(getBrowserDisplayTitle('Example', ORCA_BROWSER_BLANK_URL)).toBe('New Tab')
})
it('detects Chromium error pages', () => {
expect(isChromiumErrorPage('chrome-error://chromewebdata/')).toBe(true)
expect(isChromiumErrorPage('https://example.com')).toBe(false)
})
it('extracts notebook paths only from file URLs ending in .ipynb', () => {
expect(getNotebookPathFromBrowserUrl('https://example.com/notebook.ipynb')).toBeNull()
})
it('prefers the page-owned runtime environment id when present', () => {
expect(
getBrowserPageRuntimeEnvironmentId(
{ browserRuntimeEnvironmentId: ' env-1 ' } as BrowserPageState,
'inferred'
)
).toBe('env-1')
expect(
getBrowserPageRuntimeEnvironmentId(
{ browserRuntimeEnvironmentId: undefined } as BrowserPageState,
' inferred '
)
).toBe('inferred')
expect(
getBrowserPageRuntimeEnvironmentId(
{ browserRuntimeEnvironmentId: ' ' } as BrowserPageState,
'inferred'
)
).toBeNull()
})
it('opens only normalizable external URLs', () => {
expect(getOpenableExternalUrl('https://example.com')).toBe('https://example.com/')
expect(getOpenableExternalUrl(ORCA_BROWSER_BLANK_URL)).toBeNull()
})
it('retries a failed load by assigning the attempted URL instead of reload()', () => {
const webview = { src: 'chrome-error://chromewebdata/' }
const onUpdatePageState = vi.fn()
retryBrowserTabLoad(
webview as Electron.WebviewTag,
{
id: 'page-1',
url: 'https://example.com/app',
loadError: { code: -102, description: 'refused', validatedUrl: 'https://example.com/app' }
} as BrowserPageState,
onUpdatePageState
)
expect(onUpdatePageState).toHaveBeenCalledWith('page-1', {
loading: true,
title: 'https://example.com/app'
})
expect(webview.src).toBe('https://example.com/app')
})
it('does nothing when there is no webview', () => {
const onUpdatePageState = vi.fn()
retryBrowserTabLoad(
null,
{ id: 'page-1', url: 'https://example.com' } as BrowserPageState,
onUpdatePageState
)
expect(onUpdatePageState).not.toHaveBeenCalled()
})
})
@@ -0,0 +1,73 @@
import type { BrowserPage as BrowserPageState } from '../../../../../shared/browser-workspace-types'
import { ORCA_BROWSER_BLANK_URL } from '../../../../../shared/constants'
import {
normalizeBrowserNavigationUrl,
normalizeExternalBrowserUrl,
redactKagiSessionToken
} from '../../../../../shared/browser-url'
import { browserFileUrlToAbsolutePath } from './browser-artifact-upload'
import type { BrowserTabPageState } from './browser-page-types'
export function getBrowserPageRuntimeEnvironmentId(
page: BrowserPageState,
inferredRuntimeEnvironmentId: string | null | undefined
): string | null {
if (page.browserRuntimeEnvironmentId !== undefined) {
return page.browserRuntimeEnvironmentId?.trim() || null
}
return inferredRuntimeEnvironmentId?.trim() || null
}
export function toDisplayUrl(url: string): string {
return url === ORCA_BROWSER_BLANK_URL ? 'about:blank' : redactKagiSessionToken(url)
}
export function getBrowserDisplayTitle(title: string | null | undefined, url: string): string {
if (
url === 'about:blank' ||
url === ORCA_BROWSER_BLANK_URL ||
title === 'about:blank' ||
title === ORCA_BROWSER_BLANK_URL ||
!title
) {
return 'New Tab'
}
return title
}
export function isChromiumErrorPage(url: string): boolean {
return url.startsWith('chrome-error://')
}
export function getNotebookPathFromBrowserUrl(url: string): string | null {
const filePath = browserFileUrlToAbsolutePath(url)
return filePath?.toLowerCase().endsWith('.ipynb') ? filePath : null
}
export function getOpenableExternalUrl(currentUrl: string): string | null {
return normalizeExternalBrowserUrl(redactKagiSessionToken(currentUrl))
}
export function retryBrowserTabLoad(
webview: Electron.WebviewTag | null,
browserTab: BrowserPageState,
onUpdatePageState: (tabId: string, updates: BrowserTabPageState) => void
): void {
if (!webview) {
return
}
const retryUrl = normalizeBrowserNavigationUrl(
browserTab.loadError?.validatedUrl ?? browserTab.url
)
if (!retryUrl) {
return
}
// Why: after chrome-error://, reload() only refreshes the error page — force navigation back to the attempted URL; keep the failure visible until success.
onUpdatePageState(browserTab.id, {
loading: true,
title: retryUrl
})
webview.src = retryUrl
}
@@ -4,7 +4,7 @@ import {
getLiveBrowserUrl,
rememberLiveBrowserUrl,
seedLiveBrowserUrl
} from './browser-runtime'
} from './live-browser-url-registry'
describe('browser runtime live URL cache', () => {
afterEach(() => {
@@ -0,0 +1,141 @@
import type { Dispatch, DragEvent, MutableRefObject, RefObject, SetStateAction } from 'react'
import type { BrowserGrabPayload } from '../../../../../shared/browser-grab-types'
import {
normalizeBrowserNavigationUrl,
redactKagiSessionToken
} from '../../../../../shared/browser-url'
import type {
BrowserLoadError,
BrowserViewportPresetId
} from '../../../../../shared/browser-workspace-types'
import { ensureBrowserPageWebview } from './browser-page-webview'
import { applyBrowserPageViewportLayout, ensureBrowserPageViewport } from './browser-page-viewport'
import { seedLiveBrowserUrl } from '../describe-page/live-browser-url-registry'
import type { BrowserOverlayViewport } from '../describe-page/browser-annotation-geometry'
import { bindBrowserPageWebviewListeners } from './bind-browser-page-webview-listeners'
import type {
BrowserPageRecoveryNavigationValidation,
BrowserPageUrlSetter,
BrowserTabPageState
} from '../describe-page/browser-page-types'
export type AttachBrowserPageWebviewArgs = {
browserTabId: string
browserTabUrl: string
workspaceId: string
worktreeId: string
sessionProfileId: string | null
webviewPartition: string
isActive: boolean
isPaintable: boolean
inputLockedRef: MutableRefObject<boolean>
webviewRef: MutableRefObject<Electron.WebviewTag | null>
handleInternalFileDragOverRef: MutableRefObject<(event: DragEvent<HTMLDivElement>) => void>
handleInternalFileDropRef: MutableRefObject<(event: DragEvent<HTMLDivElement>) => void>
dismissAddressBarSuggestionsRef: MutableRefObject<(() => void) | null>
isPaintableRef: MutableRefObject<boolean>
guestRecoveryPendingRef: MutableRefObject<boolean>
browserTabUrlRef: MutableRefObject<string>
addressBarValueRef: MutableRefObject<string>
activeLoadFailureRef: MutableRefObject<BrowserLoadError | null>
recoveryNavigationValidationRef: MutableRefObject<BrowserPageRecoveryNavigationValidation | null>
keepAddressBarFocusRef: MutableRefObject<boolean>
paneZoomLevelRef: MutableRefObject<number>
viewportPresetIdRef: MutableRefObject<BrowserViewportPresetId | null>
onUpdatePageStateRef: MutableRefObject<(tabId: string, updates: BrowserTabPageState) => void>
setGuestRecoveryGeneration: Dispatch<SetStateAction<number>>
setBrowserZoomPercent: Dispatch<SetStateAction<number>>
focusAddressBarNow: () => boolean
syncNavigationState: (webview: Electron.WebviewTag) => void
syncBrowserAnnotationViewportBridge: () => void
faviconUrlRef: MutableRefObject<string | null>
addressBarInputRef: RefObject<HTMLInputElement | null>
lastKnownWebviewUrlRef: MutableRefObject<string | null>
trackNextLoadingEventRef: MutableRefObject<boolean>
clearBrowserPageAnnotationsRef: MutableRefObject<(pageId: string) => void>
onSetUrlRef: MutableRefObject<BrowserPageUrlSetter>
setPendingAnnotationPayload: Dispatch<SetStateAction<BrowserGrabPayload | null>>
setBrowserOverlayViewport: Dispatch<SetStateAction<BrowserOverlayViewport>>
setAddressBarValue: Dispatch<SetStateAction<string>>
addBrowserHistoryEntryRef: MutableRefObject<(url: string, title: string) => void>
annotationViewportBridgeTokenRef: MutableRefObject<string>
initialBrowserUrlRef: MutableRefObject<string>
validateVisibleGuestRegistrationRef: MutableRefObject<() => void>
retryGuestRecoveryRef: MutableRefObject<() => void>
setFindOpen: Dispatch<SetStateAction<boolean>>
}
export function attachBrowserPageWebview(
args: AttachBrowserPageWebviewArgs
): (() => void) | undefined {
const {
browserTabId,
workspaceId,
webviewPartition,
isActive,
isPaintable,
inputLockedRef,
webviewRef,
handleInternalFileDragOverRef,
handleInternalFileDropRef,
dismissAddressBarSuggestionsRef,
browserTabUrlRef,
lastKnownWebviewUrlRef,
syncNavigationState
} = args
let container = ensureBrowserPageViewport(browserTabId, workspaceId)?.container ?? null
if (!container) {
return
}
const ensuredWebview = ensureBrowserPageWebview({
browserTabId,
container,
inputLocked: inputLockedRef.current,
webviewPartition,
resolveContainer: () => ensureBrowserPageViewport(browserTabId, workspaceId)?.container ?? null
})
if (!ensuredWebview) {
return
}
container = ensuredWebview.container
const webview = ensuredWebview.webview
const needsInitialNavigation = ensuredWebview.created
seedLiveBrowserUrl(browserTabId, redactKagiSessionToken(browserTabUrlRef.current))
if (!ensuredWebview.created) {
// pointerEvents already applied inside ensureBrowserPageWebview for the reused-webview path.
syncNavigationState(webview)
// Why: seed from the store URL (getURL() can throw during attach) so URL sync won't force-navigate an already-correct webview.
lastKnownWebviewUrlRef.current = normalizeBrowserNavigationUrl(browserTabUrlRef.current) ?? null
}
webviewRef.current = webview
// Why: un-park the shell before webview.src is assigned or the guest navigates while hidden and stays blank (the visibility layout effect doesn't re-run on first appear).
applyBrowserPageViewportLayout(browserTabId, { paintable: isPaintable, active: isActive })
const onContainerDragOver = (event: globalThis.DragEvent): void => {
handleInternalFileDragOverRef.current(event as unknown as DragEvent<HTMLDivElement>)
}
const onContainerDrop = (event: globalThis.DragEvent): void => {
handleInternalFileDropRef.current(event as unknown as DragEvent<HTMLDivElement>)
}
container.addEventListener('dragover', onContainerDragOver)
container.addEventListener('drop', onContainerDrop)
const dismissAddressBarSuggestions = (): void => {
dismissAddressBarSuggestionsRef.current?.()
}
return bindBrowserPageWebviewListeners({
container,
webview,
needsInitialNavigation,
onContainerDragOver,
onContainerDrop,
dismissAddressBarSuggestions,
args
})
}
@@ -0,0 +1,216 @@
import { normalizeBrowserNavigationUrl } from '../../../../../shared/browser-url'
import { ORCA_BROWSER_BLANK_URL } from '../../../../../shared/constants'
import { parkBrowserPageViewport } from './browser-page-viewport'
import { subscribeBrowserSystemResume } from './browser-system-resume'
import {
isBrowserPageRendererRecoveryPending,
moveFocusToRendererBeforeWebviewDetach
} from './webview-registry'
import type { AttachBrowserPageWebviewArgs } from './attach-browser-page-webview'
import { createBrowserPageWebviewGuestSession } from './browser-page-webview-guest-session'
import { createBrowserPageWebviewLoadingHandlers } from './browser-page-webview-loading-handlers'
import { createBrowserPageWebviewNavigationHandlers } from './browser-page-webview-navigation-handlers'
export function bindBrowserPageWebviewListeners({
container,
webview,
needsInitialNavigation,
onContainerDragOver,
onContainerDrop,
dismissAddressBarSuggestions,
args
}: {
container: HTMLDivElement
webview: Electron.WebviewTag
needsInitialNavigation: boolean
onContainerDragOver: (event: globalThis.DragEvent) => void
onContainerDrop: (event: globalThis.DragEvent) => void
dismissAddressBarSuggestions: () => void
args: AttachBrowserPageWebviewArgs
}): () => void {
const {
browserTabId,
browserTabUrl,
workspaceId,
worktreeId,
sessionProfileId,
webviewRef,
isPaintableRef,
guestRecoveryPendingRef,
browserTabUrlRef,
addressBarValueRef,
activeLoadFailureRef,
recoveryNavigationValidationRef,
keepAddressBarFocusRef,
paneZoomLevelRef,
viewportPresetIdRef,
onUpdatePageStateRef,
setGuestRecoveryGeneration,
setBrowserZoomPercent,
focusAddressBarNow,
syncNavigationState,
syncBrowserAnnotationViewportBridge,
faviconUrlRef,
addressBarInputRef,
lastKnownWebviewUrlRef,
trackNextLoadingEventRef,
clearBrowserPageAnnotationsRef,
onSetUrlRef,
setPendingAnnotationPayload,
setBrowserOverlayViewport,
setAddressBarValue,
addBrowserHistoryEntryRef,
annotationViewportBridgeTokenRef,
initialBrowserUrlRef,
validateVisibleGuestRegistrationRef,
retryGuestRecoveryRef,
setFindOpen
} = args
const { guestRecovery, handleDidAttach, handleDomReady, handleGuestDestroyed } =
createBrowserPageWebviewGuestSession({
webview,
browserTabId,
workspaceId,
worktreeId,
sessionProfileId,
webviewRef,
isPaintableRef,
guestRecoveryPendingRef,
browserTabUrlRef,
addressBarValueRef,
activeLoadFailureRef,
recoveryNavigationValidationRef,
keepAddressBarFocusRef,
paneZoomLevelRef,
viewportPresetIdRef,
onUpdatePageStateRef,
setGuestRecoveryGeneration,
setBrowserZoomPercent,
focusAddressBarNow,
syncNavigationState,
syncBrowserAnnotationViewportBridge
})
const { handleDidStartLoading, handleDidStopLoading, handleFailLoad } =
createBrowserPageWebviewLoadingHandlers({
webview,
browserTabId,
faviconUrlRef,
browserTabUrlRef,
addressBarValueRef,
addressBarInputRef,
activeLoadFailureRef,
lastKnownWebviewUrlRef,
trackNextLoadingEventRef,
keepAddressBarFocusRef,
recoveryNavigationValidationRef,
clearBrowserPageAnnotationsRef,
onUpdatePageStateRef,
onSetUrlRef,
setPendingAnnotationPayload,
setBrowserOverlayViewport,
setAddressBarValue,
focusAddressBarNow
})
const {
handleDidStartNavigation,
handleFullDidNavigate,
handleDidNavigateInPage,
handleTitleUpdate,
handleFaviconUpdate,
handleAnnotationViewportMessage
} = createBrowserPageWebviewNavigationHandlers({
webview,
browserTabId,
browserTabUrl,
recoveryNavigationValidationRef,
activeLoadFailureRef,
lastKnownWebviewUrlRef,
addressBarInputRef,
onSetUrlRef,
onUpdatePageStateRef,
addBrowserHistoryEntryRef,
faviconUrlRef,
setAddressBarValue,
annotationViewportBridgeTokenRef,
setBrowserOverlayViewport
})
const unsubscribeSystemResumed = subscribeBrowserSystemResume(guestRecovery.validateAfterResume)
validateVisibleGuestRegistrationRef.current = guestRecovery.validateAfterResume
retryGuestRecoveryRef.current = guestRecovery.retryRecovery
webview.addEventListener('did-attach', handleDidAttach)
webview.addEventListener('dom-ready', handleDomReady)
webview.addEventListener('render-process-gone', guestRecovery.recoverRenderer)
webview.addEventListener('destroyed', handleGuestDestroyed)
webview.addEventListener('focus', dismissAddressBarSuggestions)
webview.addEventListener('did-start-loading', handleDidStartLoading)
webview.addEventListener('did-start-navigation', handleDidStartNavigation)
webview.addEventListener('did-stop-loading', handleDidStopLoading)
// Why: close find only on full 'did-navigate', not the shared handler, which also fires on SPA in-page hash/pushState changes.
const handleFindCloseOnNavigate = (): void => {
setFindOpen(false)
}
webview.addEventListener('did-navigate', handleFullDidNavigate)
webview.addEventListener('did-navigate', handleFindCloseOnNavigate)
webview.addEventListener('did-navigate-in-page', handleDidNavigateInPage)
webview.addEventListener('page-title-updated', handleTitleUpdate)
webview.addEventListener('page-favicon-updated', handleFaviconUpdate)
webview.addEventListener('did-fail-load', handleFailLoad)
webview.addEventListener('console-message', handleAnnotationViewportMessage)
if (needsInitialNavigation) {
// Why: set src only after listeners attach so a fast localhost failure isn't missed; only non-blank tabs show the loading indicator.
const initialUrl =
normalizeBrowserNavigationUrl(initialBrowserUrlRef.current) ?? ORCA_BROWSER_BLANK_URL
trackNextLoadingEventRef.current = initialUrl !== ORCA_BROWSER_BLANK_URL
lastKnownWebviewUrlRef.current = initialUrl
webview.src = initialUrl
} else if (isPaintableRef.current) {
if (isBrowserPageRendererRecoveryPending(browserTabId)) {
guestRecovery.recoverRenderer()
} else {
guestRecovery.validateAfterResume()
}
}
return () => {
webview.removeEventListener('did-attach', handleDidAttach)
webview.removeEventListener('dom-ready', handleDomReady)
webview.removeEventListener('render-process-gone', guestRecovery.recoverRenderer)
webview.removeEventListener('destroyed', handleGuestDestroyed)
webview.removeEventListener('focus', dismissAddressBarSuggestions)
webview.removeEventListener('did-start-loading', handleDidStartLoading)
webview.removeEventListener('did-start-navigation', handleDidStartNavigation)
webview.removeEventListener('did-stop-loading', handleDidStopLoading)
webview.removeEventListener('did-navigate', handleFullDidNavigate)
webview.removeEventListener('did-navigate', handleFindCloseOnNavigate)
webview.removeEventListener('did-navigate-in-page', handleDidNavigateInPage)
webview.removeEventListener('page-title-updated', handleTitleUpdate)
webview.removeEventListener('page-favicon-updated', handleFaviconUpdate)
webview.removeEventListener('did-fail-load', handleFailLoad)
webview.removeEventListener('console-message', handleAnnotationViewportMessage)
container.removeEventListener('dragover', onContainerDragOver)
container.removeEventListener('drop', onContainerDrop)
unsubscribeSystemResumed()
guestRecovery.dispose()
if (validateVisibleGuestRegistrationRef.current === guestRecovery.validateAfterResume) {
validateVisibleGuestRegistrationRef.current = () => {}
}
if (retryGuestRecoveryRef.current === guestRecovery.retryRecovery) {
retryGuestRecoveryRef.current = () => {}
}
if (webviewRef.current === webview) {
webviewRef.current = null
}
// Why: park the viewport on chrome unmount (worktree switch) to keep the guest alive; destroy only on explicit close.
moveFocusToRendererBeforeWebviewDetach(webview)
parkBrowserPageViewport(browserTabId)
}
}

Some files were not shown because too many files have changed in this diff Show More