Native chat: add an inline visuals switch and simplify the skill (#26358)

* Native chat: let users turn off inline visuals for new chats

* Native chat: keep inline visuals guidance light

* Move inline visuals setting to Chat settings
This commit is contained in:
Brennan Benson
2026-10-08 00:48:05 -07:00
committed by GitHub
parent 952898b8b9
commit a53967fc05
22 changed files with 471 additions and 70 deletions
@@ -1,58 +1,33 @@
---
name: orca-chat-visuals
description: Show a chart, diagram, timeline, comparison or UI mockup inline in this Orca chat by writing one self-contained HTML file and adding a visual line to your reply. Use when seeing something would explain the answer faster than prose or a small Markdown table, and only when the ORCA_CHAT_VISUALS_DIR environment variable is set.
description: Show a chart, diagram, table, report or mockup inline in this Orca chat as an HTML
page. Use when a visual would help the user understand your answer. Only available when the
ORCA_CHAT_VISUALS_DIR environment variable is set.
user-invocable: false
---
# Inline visuals in an Orca chat
Orca can render an HTML page inside your reply. Reach for it when the reader gains from seeing the
answer: trends, many values side by side, flows, architecture, layouts. Answers that read fine as
text, a short list or a small table don't need one.
Orca can show an HTML page inside your reply. Use one whenever a visual would make the answer
clearer.
## When you can't make one
## Writing the page
- This chat's folder is the absolute path in the `ORCA_CHAT_VISUALS_DIR` environment variable. Read
it from your shell (`printenv ORCA_CHAT_VISUALS_DIR`; in PowerShell `$env:ORCA_CHAT_VISUALS_DIR`).
- If the variable is unset or empty, or writing there fails, don't make a visual.
- Stay inside what the user allowed. In plan or read-only mode don't make a visual. If a write
there is refused, don't retry it; never switch modes or ask for wider access for a visual.
- Never put visuals in the user's project and never edit `.gitignore` or other files for them.
Without a visual, use a Markdown table, a Mermaid code block or plain prose instead.
## Writing the file
- One visual is one complete HTML document saved directly in that folder (no subfolders).
- Every visual gets a file name not used before in this chat: a few lowercase words and a short
random suffix, like `latency-by-region-7c1e.html`. Never overwrite or edit an earlier visual;
write a new file. Allowed: letters, digits, `.`, `_` and `-`; starts with a letter or digit;
ends in `.html`; at most 128 characters; no `..`; not a Windows device name like `con` or `nul`.
- Put everything in the file: inline `<style>` and `<script>`, data embedded. Keep it under 512 KB.
- Scripts, styles and fonts may load only from `https://cdn.jsdelivr.net`, `https://unpkg.com`,
`https://cdnjs.cloudflare.com`, `https://esm.sh`, `https://fonts.googleapis.com` and
`https://fonts.gstatic.com`. Pin library versions.
- The page can't make requests: no `fetch`, `XMLHttpRequest`, `WebSocket` or `EventSource`.
Images must be inline SVG or `data:` URLs. No forms, popups, nested frames or navigation.
- Fit the width you're given (fluid layout, `max-width: 100%`). Don't size anything to the viewport
height (`100vh`, `height: 100%` on the page): Orca sizes the frame to your content. Aim for well
under 1000px tall.
- Match the user's theme with Orca's CSS variables: `--background`, `--foreground`, `--muted`,
`--muted-foreground`, `--border`, `--primary`, `--accent`, and `--chart-1` to `--chart-5` for data
series. Leave the page background transparent. When the user switches theme the values change
and `<html>` gains or loses the `dark` class, with no reload and no event: use the variables from
CSS or SVG, and redraw a canvas chart when that class changes (a `MutationObserver` on `<html>`).
- Use real text for labels and give the page a heading so it reads without color alone.
- Save it in the folder named by `ORCA_CHAT_VISUALS_DIR` (`printenv ORCA_CHAT_VISUALS_DIR`; in
PowerShell `$env:ORCA_CHAT_VISUALS_DIR`). If it's unset or you can't write there, skip the visual.
- One complete HTML file per visual, directly in that folder, under 512 KB, with a new file name
each time (e.g. `latency-by-region-7c1e.html`): letters, digits, `.`, `_`, `-`, ending in `.html`.
- Inline your styles, scripts and data. Libraries and fonts may load only from cdn.jsdelivr.net,
unpkg.com, cdnjs.cloudflare.com, esm.sh, fonts.googleapis.com and fonts.gstatic.com. The page
can't make network requests; use inline SVG or `data:` URLs for images.
- Don't size anything to the viewport height (`100vh`); Orca fits the frame to your content.
- Optional: Orca's theme colors are available as CSS variables (`--background`, `--foreground`,
`--muted-foreground`, `--border`, `--primary`, `--chart-1` to `--chart-5`).
## Showing it
Only after the file exists, add this line to your reply on its own line, at the top level (not in
a code block, list, table or quote):
After the file exists, put this on its own line in your reply, outside any code block:
::orca-visual{file="latency-by-region-7c1e.html" title="Latency by region"}
- `file` is the bare file name, never a path. `title` is optional plain text, at most 120
characters, without `"` or `\`.
- At most 8 visuals in one reply.
- Don't announce the visual or walk through it point by point. Do state the takeaway in a sentence,
so the reply still makes sense where the visual can't be shown.
`file` is the bare file name, never a path; `title` is optional. At most 8 per reply.
@@ -39,21 +39,24 @@ export async function resolveClaudeLaunchFlags(
deps.cliFlags?.supports(CLAUDE_THINKING_DISPLAY_FLAG, launch, budgetMs, startProbe) ?? false
// Unlike the thinking display, a missed answer costs the chat its skill for its whole life, so
// the plugin check waits for the probe up to its own kill time: bounded, and instant once known.
const preparedVisuals = deps.prepareVisuals?.(sessionId) ?? Promise.resolve(null)
const [thinksFirst, visuals, loadsPlugins] = await Promise.all([
thinking(),
deps.prepareVisuals?.(sessionId) ?? null,
deps.prepareVisuals
? (deps.cliFlags?.supports(
CLAUDE_PLUGIN_DIR_FLAG,
launch,
CLAUDE_CLI_FLAG_PROBE_KILL_AFTER_MS
) ?? false)
: false
preparedVisuals,
preparedVisuals.then((visuals) =>
visuals
? (deps.cliFlags?.supports(
CLAUDE_PLUGIN_DIR_FLAG,
launch,
CLAUDE_CLI_FLAG_PROBE_KILL_AFTER_MS
) ?? false)
: false
)
])
// The plugin check may have outwaited the thinking budget and learned the version meanwhile; a
// probe that gave none is not asked again here.
const thinks =
thinksFirst || (deps.prepareVisuals ? await thinking(THINKING_RECHECK_BUDGET_MS, false) : false)
thinksFirst || (visuals ? await thinking(THINKING_RECHECK_BUDGET_MS, false) : false)
return {
thinkingDisplayArgs: thinks ? THINKING_DISPLAY_ARGS : {},
visuals: visuals ? { visuals, pluginDir: loadsPlugins ? visuals.skill.pluginDir : null } : null
@@ -0,0 +1,154 @@
import { existsSync, mkdtempSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { LOCAL_EXECUTION_HOST_ID } from '../../shared/execution-host'
import { createClaudeStructuredLaunchResolver } from '../claude/claude-structured-launch-resolution'
import { CLAUDE_PLUGIN_DIR_FLAG, type ClaudeCliFlag } from '../claude/claude-cli-flag-support'
import { createCodexStructuredLaunchResolver } from '../codex/codex-structured-launch-resolution'
import {
adapterFor,
fakeCodex,
identityFor
} from '../codex/codex-structured-session-adapter-fixture'
import { agentSessionRecordFixture } from './agent-session-record-test-fixture'
import {
createNativeChatVisualsDelivery,
NATIVE_CHAT_VISUALS_DIR_ENV
} from './native-chat-visuals-delivery'
import { nativeChatVisualsFolderFor } from './native-chat-visuals-folder'
const SKILL = { pluginDir: '/app/plugin', skillsRoot: '/app/plugin/skills' }
const MANUAL = { approvalPolicy: 'on-request', sandbox: 'workspace-write' } as const
const scratch: string[] = []
afterEach(() => {
for (const dir of scratch.splice(0)) {
rmSync(dir, { recursive: true, force: true })
}
})
function delivery(isEnabled: () => boolean) {
const stateDirectory = mkdtempSync(join(tmpdir(), 'orca-visuals-setting-'))
scratch.push(stateDirectory)
const prepareVisuals = createNativeChatVisualsDelivery({
stateDirectory,
logger: { warn: vi.fn(), error: vi.fn() },
isEnabled,
resolveSkill: async () => SKILL
})
return { stateDirectory, prepareVisuals }
}
describe('inline visuals launch preference', () => {
it('starts Claude without visuals while preserving configured plugins, user folders and attachment access', async () => {
const sessionId = 'claude-disabled'
const { stateDirectory, prepareVisuals } = delivery(() => false)
const supports = vi.fn(async (_flag: ClaudeCliFlag) => true)
const record = agentSessionRecordFixture({ sessionId })
const launch = await createClaudeStructuredLaunchResolver({
store: { getRecord: () => record, pinLaunchDirectory: vi.fn() },
resolveWorkspacePath: async () => '/repo',
resolveCommand: () => '/bin/claude',
resolveAuthPolicy: () => ({ stripAuthEnv: false }),
resolveLaunchArgs: () => ['--add-dir', '/user/notes', '--plugin-dir', '/user/plugin'],
resolveEnv: () => ({ [NATIVE_CHAT_VISUALS_DIR_ENV]: '/inherited', KEEP: 'yes' }),
resolveInheritedEnv: async () => ({ PATH: '/bin' }),
attachmentDirectory: '/state/agent-session-attachments',
hasTranscript: async () => false,
cliFlags: { supports },
prepareVisuals
})({ identity: { ...identityFor(sessionId), agent: 'claude', providerHandle: null } })
expect(launch.options).not.toHaveProperty('plugins')
expect(launch.options.extraArgs).toHaveProperty('plugin-dir', '/user/plugin')
expect(launch.options.additionalDirectories).toEqual([
'/user/notes',
'/state/agent-session-attachments'
])
expect(launch.env).not.toHaveProperty(NATIVE_CHAT_VISUALS_DIR_ENV)
expect(launch.env?.KEEP).toBe('yes')
expect(supports.mock.calls.map(([flag]) => flag)).not.toContain(CLAUDE_PLUGIN_DIR_FLAG)
expect(existsSync(nativeChatVisualsFolderFor(stateDirectory, sessionId))).toBe(false)
})
it('gives new Codex chats the current preference without changing another live chat', async () => {
let enabled = true
const { stateDirectory, prepareVisuals } = delivery(() => enabled)
const codex = fakeCodex({
'config/read': () => ({
config: { sandbox_workspace_write: { writable_roots: ['/user/notes'] } }
})
})
const resolveLaunch = createCodexStructuredLaunchResolver({
store: {
getRecord: (sessionId) =>
agentSessionRecordFixture({
sessionId,
provider: 'codex',
accountHome: { variable: 'CODEX_HOME', path: '/home/codex' }
}),
pinLaunchDirectory: vi.fn()
},
resolveWorkspacePath: async () => '/repo',
resolveCommand: () => '/bin/codex',
resolveLaunchArgs: () => [],
resolvePermissionPolicy: () => MANUAL,
resolveEnvironment: async () => ({
[NATIVE_CHAT_VISUALS_DIR_ENV]: '/inherited',
KEEP: 'yes'
}),
prepareVisuals
})
const adapter = adapterFor(codex, {}, [], { resolveLaunch })
const start = (sessionId: string) =>
adapter.acquire({
identity: {
...identityFor(sessionId),
hostId: LOCAL_EXECUTION_HOST_ID,
providerHandle: null
},
fence: 1,
spawnToken: sessionId
})
try {
await start('first')
const first = codex.connections[0]!
const firstCalls = [...first.calls]
const firstFolder = nativeChatVisualsFolderFor(stateDirectory, 'first')
expect(first.launch.env?.[NATIVE_CHAT_VISUALS_DIR_ENV]).toBe(firstFolder)
expect(first.calls.find(({ method }) => method === 'thread/start')?.params).toMatchObject({
config: { 'sandbox_workspace_write.writable_roots': ['/user/notes', firstFolder] }
})
enabled = false
await start('disabled')
const disabled = codex.connections[1]!
expect(disabled.launch.env).not.toHaveProperty(NATIVE_CHAT_VISUALS_DIR_ENV)
expect(disabled.launch.env?.KEEP).toBe('yes')
expect(disabled.calls.map(({ method }) => method)).not.toContain('skills/extraRoots/set')
const openIndex = disabled.calls.findIndex(({ method }) => method === 'thread/start')
expect(disabled.calls.slice(0, openIndex).map(({ method }) => method)).not.toContain(
'config/read'
)
expect(
disabled.calls.find(({ method }) => method === 'thread/start')?.params
).not.toHaveProperty('config')
expect(existsSync(nativeChatVisualsFolderFor(stateDirectory, 'disabled'))).toBe(false)
expect(first.closed).toBe(false)
expect(first.calls).toEqual(firstCalls)
enabled = true
await start('reenabled')
const reenabled = codex.connections[2]!
expect(
reenabled.calls.find(({ method }) => method === 'skills/extraRoots/set')?.params
).toEqual({ extraRoots: [SKILL.skillsRoot] })
expect(reenabled.launch.env?.[NATIVE_CHAT_VISUALS_DIR_ENV]).toBe(
nativeChatVisualsFolderFor(stateDirectory, 'reenabled')
)
expect(first.closed).toBe(false)
expect(first.calls).toEqual(firstCalls)
} finally {
await adapter.closeAll()
}
})
})
@@ -1,12 +1,10 @@
import { mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs'
import { existsSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it, vi } from 'vitest'
import {
NATIVE_CHAT_VISUAL_FILE_MAX_LENGTH,
NATIVE_CHAT_VISUAL_MAX_BYTES,
NATIVE_CHAT_VISUAL_MAX_PER_MESSAGE,
NATIVE_CHAT_VISUAL_TITLE_MAX_LENGTH,
parseNativeChatVisualDirectiveLine
} from '../../shared/native-chat-visual-directive'
import { NATIVE_CHAT_VISUAL_THEME_TOKENS } from '../../shared/native-chat-visual-shell'
@@ -16,7 +14,7 @@ import {
NATIVE_CHAT_VISUALS_DIR_ENV,
withNativeChatVisualsEnv
} from './native-chat-visuals-delivery'
import { nativeChatVisualsFolderFor } from './native-chat-visuals-folder'
import { nativeChatVisualsFolderFor, nativeChatVisualsRootFor } from './native-chat-visuals-folder'
import { buildClaudeChildProcessEnv } from '../claude/claude-child-process-environment'
import { codexStructuredChildEnvironment } from '../codex/codex-structured-child-environment'
import { resolveProviderChildEnv } from '../provider-process/provider-process-launch'
@@ -50,6 +48,32 @@ function tempDir(): string {
const logger = () => ({ warn: vi.fn(), error: vi.fn() })
describe('preparing a chat for visuals', () => {
it('reads the preference for each launch and prepares nothing while it is off', async () => {
const state = tempDir()
let enabled = false
const resolveSkill = vi.fn(async () => SKILL)
const prepare = createNativeChatVisualsDelivery({
stateDirectory: state,
logger: logger(),
isEnabled: () => enabled,
resolveSkill
})
await expect(prepare('disabled-chat')).resolves.toBeNull()
expect(resolveSkill).not.toHaveBeenCalled()
expect(existsSync(nativeChatVisualsRootFor(state))).toBe(false)
enabled = true
const launched = await prepare('enabled-chat')
expect(launched).toEqual({
folder: nativeChatVisualsFolderFor(state, 'enabled-chat'),
skill: SKILL
})
enabled = false
await expect(prepare('next-chat')).resolves.toBeNull()
expect(resolveSkill).toHaveBeenCalledOnce()
expect(existsSync(nativeChatVisualsFolderFor(state, 'next-chat'))).toBe(false)
expect(existsSync(launched!.folder)).toBe(true)
})
it("creates the chat's own private folder and hands back the skill", async () => {
const state = tempDir()
const prepare = createNativeChatVisualsDelivery({
@@ -159,9 +183,7 @@ describe('the bundled skill', () => {
file: 'latency-by-region-7c1e.html',
title: 'Latency by region'
})
expect(SKILL_TEXT).toContain(`At most ${NATIVE_CHAT_VISUAL_MAX_PER_MESSAGE} visuals`)
expect(SKILL_TEXT).toContain(`at most ${NATIVE_CHAT_VISUAL_FILE_MAX_LENGTH} characters`)
expect(SKILL_TEXT).toContain(`at most ${NATIVE_CHAT_VISUAL_TITLE_MAX_LENGTH}`)
expect(SKILL_TEXT).toContain(`At most ${NATIVE_CHAT_VISUAL_MAX_PER_MESSAGE} per reply`)
expect(SKILL_TEXT).toContain(`under ${NATIVE_CHAT_VISUAL_MAX_BYTES / 1024} KB`)
expect(SKILL_TEXT).toContain(NATIVE_CHAT_VISUALS_DIR_ENV)
expect(SKILL_TEXT).toMatch(new RegExp(`^name: ${NATIVE_CHAT_VISUALS_SKILL_NAME}$`, 'm'))
@@ -28,11 +28,15 @@ export type PrepareNativeChatVisuals = (
export function createNativeChatVisualsDelivery(deps: {
stateDirectory: string
logger: StructuredAgentSessionLogger
isEnabled?: () => boolean
resolveSkill?: () => Promise<NativeChatVisualsSkillLocation | null>
}): PrepareNativeChatVisuals {
const resolveSkill = deps.resolveSkill ?? (() => resolveNativeChatVisualsSkillLocation())
return async (sessionId) => {
try {
if (deps.isEnabled?.() === false) {
return null
}
const skill = await resolveSkill()
if (!skill) {
deps.logger.warn('native-chat visuals skill is missing from this install', {
@@ -229,6 +229,7 @@ export class OrcaRuntimeWithGetWorktreePs extends OrcaRuntimeWithStartTuiIdleVis
// Wired only here, so a test runtime never runs a real `claude --version`.
claudeCliFlags: claudeCliFlagSupport,
nativeChatVisuals: {
isEnabled: () => this.requireStore().getSettings().nativeChatInlineVisuals !== false,
// Chats and their visuals are shared by every profile; each profile keeps its own catalog.
workspaceVerdicts: createNativeChatVisualsWorkspaceVerdicts(() =>
this.store ? readNativeChatVisualsWorkspaceCatalogs(this.store) : null
@@ -130,6 +130,7 @@ export type RuntimeStore = {
hostSettingOverrides?: GlobalSettings['hostSettingOverrides']
agentSkillSharingEnabled?: GlobalSettings['agentSkillSharingEnabled']
nativeChatSessionOptions?: GlobalSettings['nativeChatSessionOptions']
nativeChatInlineVisuals?: GlobalSettings['nativeChatInlineVisuals']
nativeChatInheritShellEnvironment?: GlobalSettings['nativeChatInheritShellEnvironment']
nativeChatShellEnvironmentVariables?: GlobalSettings['nativeChatShellEnvironmentVariables']
aiVaultSearch?: GlobalSettings['aiVaultSearch']
@@ -116,7 +116,8 @@ function nativeChatVisualsFor(deps: StructuredAgentSessionRuntimeDeps): {
? {
prepareVisuals: createNativeChatVisualsDelivery({
stateDirectory: deps.stateDirectory,
logger: deps.logger
logger: deps.logger,
isEnabled: deps.nativeChatVisuals.isEnabled
})
}
: {}
@@ -106,6 +106,8 @@ export type StructuredAgentSessionRuntimeDeps = {
/** Gives each chat a visuals folder and the skill that teaches it, and sweeps folders whose chat
* is gone. Wired by the real hosts only, so a test runtime never loads the bundled skill. */
nativeChatVisuals?: {
/** Read this host's preference when a chat starts a provider process. */
isEnabled: () => boolean
workspaceVerdicts: NonNullable<NativeChatVisualsSweepDeps['workspaceVerdicts']>
}
/** Provider transports are overridden only to drive the runtime against scripted children. */
@@ -19,6 +19,8 @@ import {
} from './native-chat-visual-owner'
import { clearNativeChatVisualCacheForTests } from './native-chat-visual-read-client'
import { TooltipProvider } from '@/components/ui/tooltip'
import { useAppStore } from '@/store'
import { getDefaultSettings } from '../../../../shared/constants'
const owner: NativeChatVisualOwner = {
target: { kind: 'local' },
@@ -61,6 +63,22 @@ afterEach(() => {
})
describe('NativeChatMarkdown visuals', () => {
it('still renders an existing reply after inline visuals are turned off', async () => {
const settings = useAppStore.getState().settings
useAppStore.setState({
settings: { ...getDefaultSettings('/tmp'), nativeChatInlineVisuals: false }
})
try {
const { container } = render(
withOwner(<NativeChatMarkdown content={LINE} visualMessageId="m1" />)
)
await waitFor(() => expect(container.querySelector('iframe')).not.toBeNull())
expect(container).not.toHaveTextContent('::orca-visual')
} finally {
useAppStore.setState({ settings })
}
})
it('holds back an unfinished directive at the end of a streaming reply', () => {
const { container } = render(
withOwner(
@@ -0,0 +1,96 @@
// @vitest-environment happy-dom
import { cleanup, fireEvent, render, screen } from '@testing-library/react'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { getDefaultSettings } from '../../../../shared/constants'
import type { GlobalSettings } from '../../../../shared/global-settings-types'
import { ChatSettingsSection } from './ChatSettingsSection'
import { ActiveSettingsSectionProvider } from './SettingsSection'
import { getChatInlineVisualsSearchEntry } from './chat-inline-visuals-search'
import { matchesSettingsSearch } from './settings-search'
import { getSettingsScrollTarget } from './settings-navigation-foundations'
import { buildSettingsNavigationMetadata } from '@/hooks/useSettingsNavigationMetadata'
import { buildCmdJSettingsResults } from '../cmd-j/palette-results'
const state = vi.hoisted((): { settingsSearchQuery: string; settings: GlobalSettings | null } => ({
settingsSearchQuery: '',
settings: null
}))
vi.mock('../../store', () => ({
useAppStore: Object.assign((selector: (value: typeof state) => unknown) => selector(state), {
getState: () => state
})
}))
afterEach(cleanup)
beforeEach(() => {
state.settingsSearchQuery = ''
state.settings = null
})
function renderChat(nativeChatInlineVisuals: boolean | undefined) {
const settings = {
...getDefaultSettings('/tmp'),
experimentalStructuredNativeChat: true,
nativeChatInlineVisuals
}
state.settings = settings
const updateSettings = vi.fn()
return {
...render(
<ActiveSettingsSectionProvider value="chat">
<ChatSettingsSection
settings={settings}
updateSettings={updateSettings}
writeSourceControlAiSettings={async () => {}}
searchEntries={[getChatInlineVisualsSearchEntry()]}
showDesktopOnlySettings
isMounted
/>
</ActiveSettingsSectionProvider>
),
updateSettings
}
}
describe('Inline visuals on the Chat settings page', () => {
it.each([true, undefined])('defaults on for saved preference %s and turns off', (preference) => {
expect(getDefaultSettings('/tmp').nativeChatInlineVisuals).toBe(true)
const { updateSettings } = renderChat(preference)
const toggle = screen.getByRole('switch', { name: 'Toggle inline visuals' })
expect(toggle.getAttribute('aria-checked')).toBe('true')
fireEvent.click(toggle)
expect(updateSettings).toHaveBeenCalledWith({ nativeChatInlineVisuals: false })
})
it('uses saved off and turns on', () => {
const { updateSettings } = renderChat(false)
expect(screen.getByText(/Applies to newly started chats/)).toBeTruthy()
const toggle = screen.getByRole('switch', { name: 'Toggle inline visuals' })
expect(toggle.getAttribute('aria-checked')).toBe('false')
fireEvent.click(toggle)
expect(updateSettings).toHaveBeenCalledWith({ nativeChatInlineVisuals: true })
})
it.each(['inline visuals', 'charts', 'mockups'])(
'finds %s and resolves its own Chat target',
(query) => {
state.settingsSearchQuery = query
const { container } = renderChat(true)
const toggle = screen.getByRole('switch', { name: 'Toggle inline visuals' })
expect(screen.queryByRole('spinbutton', { name: 'Text size' })).toBeNull()
const sections = buildSettingsNavigationMetadata({
isMac: false,
isWindows: false,
isWebClient: false,
experimentalStructuredNativeChat: true,
repos: []
})
const results = buildCmdJSettingsResults(sections)
expect(matchesSettingsSearch(query, getChatInlineVisualsSearchEntry())).toBe(true)
const result = results.find((entry) => entry.title === 'Inline visuals')
expect(result?.sectionId).toBe('chat')
expect(result?.targetSectionId).toBe('chat-inline-visuals')
expect(getSettingsScrollTarget('chat-inline-visuals', container)?.contains(toggle)).toBe(true)
expect(results.filter((entry) => entry.title === 'Inline visuals')).toHaveLength(1)
}
)
})
@@ -8,6 +8,7 @@ import { ChatSettingsSection } from './ChatSettingsSection'
import { ActiveSettingsSectionProvider } from './SettingsSection'
import { getChatAppearanceSearchEntries } from './chat-appearance-search'
import { getChatNamingSearchEntry } from './chat-naming-search'
import { getChatInlineVisualsSearchEntry } from './chat-inline-visuals-search'
import { useSettingsRepoScrollEffects } from './use-settings-repo-scroll-effects'
import type { SettingsStoreModel } from './use-settings-store-model'
import type { SettingsInteractionController } from './use-settings-interaction-controller'
@@ -96,7 +97,11 @@ function NavigationHarness({
settings={settings}
updateSettings={vi.fn()}
writeSourceControlAiSettings={async () => {}}
searchEntries={[...getChatAppearanceSearchEntries(), getChatNamingSearchEntry()]}
searchEntries={[
...getChatAppearanceSearchEntries(),
getChatNamingSearchEntry(),
getChatInlineVisualsSearchEntry()
]}
showDesktopOnlySettings
isMounted
/>
@@ -105,6 +110,23 @@ function NavigationHarness({
}
describe('Chat settings deep links', () => {
it('activates Chat and scrolls to Inline visuals from another settings page', async () => {
const scrollIntoView = vi.spyOn(HTMLElement.prototype, 'scrollIntoView')
const { container } = render(
<NavigationHarness enabled initialSection="appearance" targetSection="chat-inline-visuals" />
)
await waitFor(() => {
expect(screen.getByRole('status', { name: 'Selected settings page' }).textContent).toBe(
'chat'
)
})
const visuals = container.querySelector('#chat-inline-visuals')
expect(visuals?.querySelector('[role="switch"]')).toBe(
screen.getByRole('switch', { name: 'Toggle inline visuals' })
)
await waitFor(() => expect(scrollIntoView.mock.contexts).toContain(visuals))
})
it('activates Chat and renders the moved row for a deep link', async () => {
render(<NavigationHarness enabled initialSection="appearance" />)
await waitFor(() => {
@@ -7,6 +7,7 @@ import { ChatSettingsSection } from './ChatSettingsSection'
import { ActiveSettingsSectionProvider } from './SettingsSection'
import { getChatAppearanceSearchEntries } from './chat-appearance-search'
import { getChatNamingSearchEntry } from './chat-naming-search'
import { getChatInlineVisualsSearchEntry } from './chat-inline-visuals-search'
import { buildSettingsNavigationMetadata } from '@/hooks/useSettingsNavigationMetadata'
import { buildCmdJSettingsResults } from '../cmd-j/palette-results'
import { isSettingsNavigationTarget } from '@/lib/settings-navigation-types'
@@ -46,7 +47,12 @@ function renderChat(
settings={settings}
updateSettings={updateSettings}
writeSourceControlAiSettings={async () => {}}
searchEntries={[...getChatAppearanceSearchEntries(), getChatNamingSearchEntry()]}
searchEntries={[
...getChatAppearanceSearchEntries(),
...(showDesktopOnlySettings
? [getChatNamingSearchEntry(), getChatInlineVisualsSearchEntry()]
: [])
]}
showDesktopOnlySettings={showDesktopOnlySettings}
isMounted
hasUnsavedChatPromptChanges={hasUnsavedChatPromptChanges}
@@ -62,6 +68,7 @@ describe('Chat settings page', () => {
expect(screen.getByRole('spinbutton', { name: 'Text size' })).toBeTruthy()
expect(container.querySelector('#chat-names')).toBeNull()
expect(screen.queryByRole('switch', { name: 'Name chats automatically' })).toBeNull()
expect(screen.queryByRole('switch', { name: 'Toggle inline visuals' })).toBeNull()
const sections = buildSettingsNavigationMetadata({
isMac: false,
isWindows: false,
@@ -72,6 +79,7 @@ describe('Chat settings page', () => {
const results = buildCmdJSettingsResults(sections).filter((entry) => entry.sectionId === 'chat')
expect(results.some((entry) => entry.targetSectionId === 'chat-text-size')).toBe(true)
expect(results.some((entry) => entry.targetSectionId === 'chat-names')).toBe(false)
expect(results.some((entry) => entry.targetSectionId === 'chat-inline-visuals')).toBe(false)
})
it.each([false, undefined])('is absent with structured chat set to %s', (enabled) => {
@@ -99,14 +107,19 @@ describe('Chat settings page', () => {
)
})
it('renders Appearance and Chat names as peer sections with separate cards', () => {
it('renders Appearance, Inline visuals and Chat names as peers with separate cards', () => {
const { container } = renderChat(true)
const appearance = container.querySelector('#chat-appearance')
const names = container.querySelector('#chat-names')
const visuals = container.querySelector('#chat-inline-visuals')
expect(screen.getByRole('heading', { name: 'Chat names', level: 3 })).toBeTruthy()
expect(appearance).toBeTruthy()
expect(names).toBeTruthy()
expect(appearance?.parentElement).toBe(names?.parentElement)
expect(visuals?.parentElement).toBe(appearance?.parentElement)
expect(visuals?.querySelector('[data-slot="card"]')).toBeTruthy()
expect(visuals?.contains(appearance)).toBe(false)
expect(visuals?.contains(names)).toBe(false)
expect(appearance?.closest('[data-slot="card"]')).toBeNull()
expect(names?.closest('[data-slot="card"]')).toBeNull()
const appearanceCard = screen
@@ -1,6 +1,7 @@
import { translate } from '@/i18n/i18n'
import { AppearanceChatSection } from './AppearanceChatSection'
import { ChatNamingSetting } from './ChatNamingSetting'
import { NativeChatInlineVisualsSetting } from './NativeChatInlineVisualsSetting'
import { SettingsSection } from './SettingsSection'
import { SettingsSubsectionHeader } from './SettingsFormControls'
import { matchesSettingsSearch } from './settings-search'
@@ -42,10 +43,7 @@ export function ChatSettingsSection({
<SettingsSection
id="chat"
title={title}
description={translate(
'settings.chat.description',
'Choose how chats look and get their names.'
)}
description={translate('settings.chat.description', 'Choose how chats look and behave.')}
searchEntries={searchEntries}
forceVisible={hasUnsavedChatPromptChanges}
bodyClassName="rounded-none border-0 bg-transparent p-0 shadow-none"
@@ -69,6 +67,13 @@ export function ChatSettingsSection({
</Card>
</section>
) : null}
{showDesktopOnlySettings ? (
<NativeChatInlineVisualsSetting
settings={settings}
updateSettings={updateSettings}
forceVisible={matchesSettingsSearch(query, [{ title }])}
/>
) : null}
{showDesktopOnlySettings ? (
<ChatNamingSetting
key={chatPromptDiscardSignal}
@@ -258,3 +258,15 @@ describe('NativeChatExperimentalSetting queue follow-ups', () => {
expect(container.querySelector(QUEUE_TOGGLE)).toBeNull()
})
})
describe('NativeChatExperimentalSetting inline visuals', () => {
it('leaves Inline visuals on the Chat page even while structured chat is enabled', () => {
const { queryByRole, getByRole } = renderSetting({
experimentalNativeChat: true,
openAgentTabsInChatByDefault: true,
experimentalStructuredNativeChat: true
})
expect(getByRole('switch', { name: 'Toggle automatic resume after a restart' })).toBeTruthy()
expect(queryByRole('switch', { name: 'Toggle inline visuals' })).toBeNull()
})
})
@@ -0,0 +1,44 @@
import type { GlobalSettings } from '../../../../shared/global-settings-types'
import { translate } from '@/i18n/i18n'
import { SettingsSwitchRow } from './SettingsFormControls'
import { Card, CardContent } from '../ui/card'
import { SearchableSetting } from './SearchableSetting'
import { getChatInlineVisualsSearchEntry } from './chat-inline-visuals-search'
type NativeChatInlineVisualsSettingProps = {
settings: GlobalSettings
updateSettings: (updates: Partial<GlobalSettings>) => void
forceVisible?: boolean
}
export function NativeChatInlineVisualsSetting({
settings,
updateSettings,
forceVisible = false
}: NativeChatInlineVisualsSettingProps): React.JSX.Element {
const enabled = settings.nativeChatInlineVisuals !== false
const entry = getChatInlineVisualsSearchEntry()
return (
<SearchableSetting
{...entry}
id="chat-inline-visuals"
forceVisible={forceVisible}
className="max-w-none"
>
<Card>
<CardContent>
<SettingsSwitchRow
label={entry.title}
description={entry.description}
checked={enabled}
ariaLabel={translate(
'components.settings.nativeChat.inlineVisualsToggleLabel',
'Toggle inline visuals'
)}
onChange={() => updateSettings({ nativeChatInlineVisuals: !enabled })}
/>
</CardContent>
</Card>
</SearchableSetting>
)
}
@@ -0,0 +1,12 @@
import { translate } from '@/i18n/i18n'
import { createLocalizedCatalog } from '@/i18n/localized-catalog'
import type { SettingsSearchEntry } from './settings-search'
export const getChatInlineVisualsSearchEntry = createLocalizedCatalog((): SettingsSearchEntry => ({
targetSectionId: 'chat-inline-visuals',
title: translate('components.settings.nativeChat.inlineVisualsTitle', 'Inline visuals'),
description: translate(
'components.settings.nativeChat.inlineVisualsCopy',
'Let the agent show charts, diagrams and mockups inside its replies. Applies to newly started chats.'
)
}))
@@ -10,4 +10,11 @@ describe('native chat experimental search entry', () => {
it('matches "queue" so the Queue follow-ups switch is findable', () => {
expect(matchesSettingsSearch('queue', getNativeChatExperimentalSearchEntry())).toBe(true)
})
it.each(['inline visuals', 'visuals'])(
'leaves the moved switch out of Experimental for %s',
(query) => {
expect(matchesSettingsSearch(query, getNativeChatExperimentalSearchEntry())).toBe(false)
}
)
})
@@ -1,6 +1,7 @@
import { getAppearancePaneSearchEntries } from '@/components/settings/appearance-search'
import { getChatAppearanceSearchEntries } from '@/components/settings/chat-appearance-search'
import { getChatNamingSearchEntry } from '@/components/settings/chat-naming-search'
import { getChatInlineVisualsSearchEntry } from '@/components/settings/chat-inline-visuals-search'
import { getInputPaneSearchEntries } from '@/components/settings/input-search'
import { getNotificationsPaneSearchEntries } from '@/components/settings/notifications-search'
import { getShortcutsPaneSearchEntries } from '@/components/settings/shortcuts-search'
@@ -42,12 +43,14 @@ export function buildInterfaceSettingsSections({
title: translate('settings.appearance.chat.title', 'Chat'),
description: translate(
'settings.chat.description',
'Choose how chats look and get their names.'
'Choose how chats look and behave.'
),
icon: MessageSquare,
searchEntries: [
...getChatAppearanceSearchEntries(),
...(showDesktopOnlySettings ? [getChatNamingSearchEntry()] : [])
...(showDesktopOnlySettings
? [getChatNamingSearchEntry(), getChatInlineVisualsSearchEntry()]
: [])
],
group: 'interface'
}
+4 -1
View File
@@ -177,7 +177,7 @@
},
"settings": {
"chat": {
"description": "Choose how chats look and get their names.",
"description": "Choose how chats look and behave.",
"names": {
"title": "Chat names",
"description": "Use an agent to name new chats from their first message. Without a generated name, chats stay Claude Chat or Codex Chat.",
@@ -18954,6 +18954,9 @@
},
"nativeChat": {
"queueFollowUpsTitle": "Queue follow-ups",
"inlineVisualsTitle": "Inline visuals",
"inlineVisualsCopy": "Let the agent show charts, diagrams and mockups inside its replies. Applies to newly started chats.",
"inlineVisualsToggleLabel": "Toggle inline visuals",
"queueFollowUpsCopy": "Messages you send while the agent is working wait as cards you can steer, edit, or delete. Messages with images send right away.",
"queueFollowUpsToggleLabel": "Toggle queue follow-ups"
}
+1
View File
@@ -137,6 +137,7 @@ export function buildDefaultSettings(args: {
experimentalStructuredNativeChat: false,
nativeChatResumeWorkOnRestart: false,
nativeChatQueueFollowUps: true,
nativeChatInlineVisuals: true,
nativeChatInheritShellEnvironment: true,
nativeChatShellEnvironmentVariables: [],
nativeChatSessionOptions: {},
+2
View File
@@ -228,6 +228,8 @@ export type GlobalSettings = NativeChatGlobalSettings & {
/** Chat-wide: hold a mid-turn send as an editable queued draft that goes when the turn ends
* (capable hosts only). Absent = on; off keeps mid-turn sends immediate. */
nativeChatQueueFollowUps?: boolean
/** Teach newly started native chats to create inline visuals; absent means on. */
nativeChatInlineVisuals?: boolean
/** Structured chat only: Codex/Claude children inherit the whole login-shell environment.
* Off passes only `nativeChatShellEnvironmentVariables` (plus a PATH/locale baseline). */
nativeChatInheritShellEnvironment?: boolean