mirror of
https://github.com/stablyai/orca.git
synced 2026-10-08 16:02:37 +00:00
Native chat: add an inline visuals switch and simplify the skill (#26358)
* Native chat: let users turn off inline visuals for new chats * Native chat: keep inline visuals guidance light * Move inline visuals setting to Chat settings
This commit is contained in:
@@ -1,58 +1,33 @@
|
||||
---
|
||||
name: orca-chat-visuals
|
||||
description: Show a chart, diagram, timeline, comparison or UI mockup inline in this Orca chat by writing one self-contained HTML file and adding a visual line to your reply. Use when seeing something would explain the answer faster than prose or a small Markdown table, and only when the ORCA_CHAT_VISUALS_DIR environment variable is set.
|
||||
description: Show a chart, diagram, table, report or mockup inline in this Orca chat as an HTML
|
||||
page. Use when a visual would help the user understand your answer. Only available when the
|
||||
ORCA_CHAT_VISUALS_DIR environment variable is set.
|
||||
user-invocable: false
|
||||
---
|
||||
|
||||
# Inline visuals in an Orca chat
|
||||
|
||||
Orca can render an HTML page inside your reply. Reach for it when the reader gains from seeing the
|
||||
answer: trends, many values side by side, flows, architecture, layouts. Answers that read fine as
|
||||
text, a short list or a small table don't need one.
|
||||
Orca can show an HTML page inside your reply. Use one whenever a visual would make the answer
|
||||
clearer.
|
||||
|
||||
## When you can't make one
|
||||
## Writing the page
|
||||
|
||||
- This chat's folder is the absolute path in the `ORCA_CHAT_VISUALS_DIR` environment variable. Read
|
||||
it from your shell (`printenv ORCA_CHAT_VISUALS_DIR`; in PowerShell `$env:ORCA_CHAT_VISUALS_DIR`).
|
||||
- If the variable is unset or empty, or writing there fails, don't make a visual.
|
||||
- Stay inside what the user allowed. In plan or read-only mode don't make a visual. If a write
|
||||
there is refused, don't retry it; never switch modes or ask for wider access for a visual.
|
||||
- Never put visuals in the user's project and never edit `.gitignore` or other files for them.
|
||||
|
||||
Without a visual, use a Markdown table, a Mermaid code block or plain prose instead.
|
||||
|
||||
## Writing the file
|
||||
|
||||
- One visual is one complete HTML document saved directly in that folder (no subfolders).
|
||||
- Every visual gets a file name not used before in this chat: a few lowercase words and a short
|
||||
random suffix, like `latency-by-region-7c1e.html`. Never overwrite or edit an earlier visual;
|
||||
write a new file. Allowed: letters, digits, `.`, `_` and `-`; starts with a letter or digit;
|
||||
ends in `.html`; at most 128 characters; no `..`; not a Windows device name like `con` or `nul`.
|
||||
- Put everything in the file: inline `<style>` and `<script>`, data embedded. Keep it under 512 KB.
|
||||
- Scripts, styles and fonts may load only from `https://cdn.jsdelivr.net`, `https://unpkg.com`,
|
||||
`https://cdnjs.cloudflare.com`, `https://esm.sh`, `https://fonts.googleapis.com` and
|
||||
`https://fonts.gstatic.com`. Pin library versions.
|
||||
- The page can't make requests: no `fetch`, `XMLHttpRequest`, `WebSocket` or `EventSource`.
|
||||
Images must be inline SVG or `data:` URLs. No forms, popups, nested frames or navigation.
|
||||
- Fit the width you're given (fluid layout, `max-width: 100%`). Don't size anything to the viewport
|
||||
height (`100vh`, `height: 100%` on the page): Orca sizes the frame to your content. Aim for well
|
||||
under 1000px tall.
|
||||
- Match the user's theme with Orca's CSS variables: `--background`, `--foreground`, `--muted`,
|
||||
`--muted-foreground`, `--border`, `--primary`, `--accent`, and `--chart-1` to `--chart-5` for data
|
||||
series. Leave the page background transparent. When the user switches theme the values change
|
||||
and `<html>` gains or loses the `dark` class, with no reload and no event: use the variables from
|
||||
CSS or SVG, and redraw a canvas chart when that class changes (a `MutationObserver` on `<html>`).
|
||||
- Use real text for labels and give the page a heading so it reads without color alone.
|
||||
- Save it in the folder named by `ORCA_CHAT_VISUALS_DIR` (`printenv ORCA_CHAT_VISUALS_DIR`; in
|
||||
PowerShell `$env:ORCA_CHAT_VISUALS_DIR`). If it's unset or you can't write there, skip the visual.
|
||||
- One complete HTML file per visual, directly in that folder, under 512 KB, with a new file name
|
||||
each time (e.g. `latency-by-region-7c1e.html`): letters, digits, `.`, `_`, `-`, ending in `.html`.
|
||||
- Inline your styles, scripts and data. Libraries and fonts may load only from cdn.jsdelivr.net,
|
||||
unpkg.com, cdnjs.cloudflare.com, esm.sh, fonts.googleapis.com and fonts.gstatic.com. The page
|
||||
can't make network requests; use inline SVG or `data:` URLs for images.
|
||||
- Don't size anything to the viewport height (`100vh`); Orca fits the frame to your content.
|
||||
- Optional: Orca's theme colors are available as CSS variables (`--background`, `--foreground`,
|
||||
`--muted-foreground`, `--border`, `--primary`, `--chart-1` to `--chart-5`).
|
||||
|
||||
## Showing it
|
||||
|
||||
Only after the file exists, add this line to your reply on its own line, at the top level (not in
|
||||
a code block, list, table or quote):
|
||||
After the file exists, put this on its own line in your reply, outside any code block:
|
||||
|
||||
::orca-visual{file="latency-by-region-7c1e.html" title="Latency by region"}
|
||||
|
||||
- `file` is the bare file name, never a path. `title` is optional plain text, at most 120
|
||||
characters, without `"` or `\`.
|
||||
- At most 8 visuals in one reply.
|
||||
- Don't announce the visual or walk through it point by point. Do state the takeaway in a sentence,
|
||||
so the reply still makes sense where the visual can't be shown.
|
||||
`file` is the bare file name, never a path; `title` is optional. At most 8 per reply.
|
||||
|
||||
@@ -39,21 +39,24 @@ export async function resolveClaudeLaunchFlags(
|
||||
deps.cliFlags?.supports(CLAUDE_THINKING_DISPLAY_FLAG, launch, budgetMs, startProbe) ?? false
|
||||
// Unlike the thinking display, a missed answer costs the chat its skill for its whole life, so
|
||||
// the plugin check waits for the probe up to its own kill time: bounded, and instant once known.
|
||||
const preparedVisuals = deps.prepareVisuals?.(sessionId) ?? Promise.resolve(null)
|
||||
const [thinksFirst, visuals, loadsPlugins] = await Promise.all([
|
||||
thinking(),
|
||||
deps.prepareVisuals?.(sessionId) ?? null,
|
||||
deps.prepareVisuals
|
||||
? (deps.cliFlags?.supports(
|
||||
CLAUDE_PLUGIN_DIR_FLAG,
|
||||
launch,
|
||||
CLAUDE_CLI_FLAG_PROBE_KILL_AFTER_MS
|
||||
) ?? false)
|
||||
: false
|
||||
preparedVisuals,
|
||||
preparedVisuals.then((visuals) =>
|
||||
visuals
|
||||
? (deps.cliFlags?.supports(
|
||||
CLAUDE_PLUGIN_DIR_FLAG,
|
||||
launch,
|
||||
CLAUDE_CLI_FLAG_PROBE_KILL_AFTER_MS
|
||||
) ?? false)
|
||||
: false
|
||||
)
|
||||
])
|
||||
// The plugin check may have outwaited the thinking budget and learned the version meanwhile; a
|
||||
// probe that gave none is not asked again here.
|
||||
const thinks =
|
||||
thinksFirst || (deps.prepareVisuals ? await thinking(THINKING_RECHECK_BUDGET_MS, false) : false)
|
||||
thinksFirst || (visuals ? await thinking(THINKING_RECHECK_BUDGET_MS, false) : false)
|
||||
return {
|
||||
thinkingDisplayArgs: thinks ? THINKING_DISPLAY_ARGS : {},
|
||||
visuals: visuals ? { visuals, pluginDir: loadsPlugins ? visuals.skill.pluginDir : null } : null
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
import { existsSync, mkdtempSync, rmSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { LOCAL_EXECUTION_HOST_ID } from '../../shared/execution-host'
|
||||
import { createClaudeStructuredLaunchResolver } from '../claude/claude-structured-launch-resolution'
|
||||
import { CLAUDE_PLUGIN_DIR_FLAG, type ClaudeCliFlag } from '../claude/claude-cli-flag-support'
|
||||
import { createCodexStructuredLaunchResolver } from '../codex/codex-structured-launch-resolution'
|
||||
import {
|
||||
adapterFor,
|
||||
fakeCodex,
|
||||
identityFor
|
||||
} from '../codex/codex-structured-session-adapter-fixture'
|
||||
import { agentSessionRecordFixture } from './agent-session-record-test-fixture'
|
||||
import {
|
||||
createNativeChatVisualsDelivery,
|
||||
NATIVE_CHAT_VISUALS_DIR_ENV
|
||||
} from './native-chat-visuals-delivery'
|
||||
import { nativeChatVisualsFolderFor } from './native-chat-visuals-folder'
|
||||
|
||||
const SKILL = { pluginDir: '/app/plugin', skillsRoot: '/app/plugin/skills' }
|
||||
const MANUAL = { approvalPolicy: 'on-request', sandbox: 'workspace-write' } as const
|
||||
const scratch: string[] = []
|
||||
afterEach(() => {
|
||||
for (const dir of scratch.splice(0)) {
|
||||
rmSync(dir, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
function delivery(isEnabled: () => boolean) {
|
||||
const stateDirectory = mkdtempSync(join(tmpdir(), 'orca-visuals-setting-'))
|
||||
scratch.push(stateDirectory)
|
||||
const prepareVisuals = createNativeChatVisualsDelivery({
|
||||
stateDirectory,
|
||||
logger: { warn: vi.fn(), error: vi.fn() },
|
||||
isEnabled,
|
||||
resolveSkill: async () => SKILL
|
||||
})
|
||||
return { stateDirectory, prepareVisuals }
|
||||
}
|
||||
|
||||
describe('inline visuals launch preference', () => {
|
||||
it('starts Claude without visuals while preserving configured plugins, user folders and attachment access', async () => {
|
||||
const sessionId = 'claude-disabled'
|
||||
const { stateDirectory, prepareVisuals } = delivery(() => false)
|
||||
const supports = vi.fn(async (_flag: ClaudeCliFlag) => true)
|
||||
const record = agentSessionRecordFixture({ sessionId })
|
||||
const launch = await createClaudeStructuredLaunchResolver({
|
||||
store: { getRecord: () => record, pinLaunchDirectory: vi.fn() },
|
||||
resolveWorkspacePath: async () => '/repo',
|
||||
resolveCommand: () => '/bin/claude',
|
||||
resolveAuthPolicy: () => ({ stripAuthEnv: false }),
|
||||
resolveLaunchArgs: () => ['--add-dir', '/user/notes', '--plugin-dir', '/user/plugin'],
|
||||
resolveEnv: () => ({ [NATIVE_CHAT_VISUALS_DIR_ENV]: '/inherited', KEEP: 'yes' }),
|
||||
resolveInheritedEnv: async () => ({ PATH: '/bin' }),
|
||||
attachmentDirectory: '/state/agent-session-attachments',
|
||||
hasTranscript: async () => false,
|
||||
cliFlags: { supports },
|
||||
prepareVisuals
|
||||
})({ identity: { ...identityFor(sessionId), agent: 'claude', providerHandle: null } })
|
||||
expect(launch.options).not.toHaveProperty('plugins')
|
||||
expect(launch.options.extraArgs).toHaveProperty('plugin-dir', '/user/plugin')
|
||||
expect(launch.options.additionalDirectories).toEqual([
|
||||
'/user/notes',
|
||||
'/state/agent-session-attachments'
|
||||
])
|
||||
expect(launch.env).not.toHaveProperty(NATIVE_CHAT_VISUALS_DIR_ENV)
|
||||
expect(launch.env?.KEEP).toBe('yes')
|
||||
expect(supports.mock.calls.map(([flag]) => flag)).not.toContain(CLAUDE_PLUGIN_DIR_FLAG)
|
||||
expect(existsSync(nativeChatVisualsFolderFor(stateDirectory, sessionId))).toBe(false)
|
||||
})
|
||||
|
||||
it('gives new Codex chats the current preference without changing another live chat', async () => {
|
||||
let enabled = true
|
||||
const { stateDirectory, prepareVisuals } = delivery(() => enabled)
|
||||
const codex = fakeCodex({
|
||||
'config/read': () => ({
|
||||
config: { sandbox_workspace_write: { writable_roots: ['/user/notes'] } }
|
||||
})
|
||||
})
|
||||
const resolveLaunch = createCodexStructuredLaunchResolver({
|
||||
store: {
|
||||
getRecord: (sessionId) =>
|
||||
agentSessionRecordFixture({
|
||||
sessionId,
|
||||
provider: 'codex',
|
||||
accountHome: { variable: 'CODEX_HOME', path: '/home/codex' }
|
||||
}),
|
||||
pinLaunchDirectory: vi.fn()
|
||||
},
|
||||
resolveWorkspacePath: async () => '/repo',
|
||||
resolveCommand: () => '/bin/codex',
|
||||
resolveLaunchArgs: () => [],
|
||||
resolvePermissionPolicy: () => MANUAL,
|
||||
resolveEnvironment: async () => ({
|
||||
[NATIVE_CHAT_VISUALS_DIR_ENV]: '/inherited',
|
||||
KEEP: 'yes'
|
||||
}),
|
||||
prepareVisuals
|
||||
})
|
||||
const adapter = adapterFor(codex, {}, [], { resolveLaunch })
|
||||
const start = (sessionId: string) =>
|
||||
adapter.acquire({
|
||||
identity: {
|
||||
...identityFor(sessionId),
|
||||
hostId: LOCAL_EXECUTION_HOST_ID,
|
||||
providerHandle: null
|
||||
},
|
||||
fence: 1,
|
||||
spawnToken: sessionId
|
||||
})
|
||||
try {
|
||||
await start('first')
|
||||
const first = codex.connections[0]!
|
||||
const firstCalls = [...first.calls]
|
||||
const firstFolder = nativeChatVisualsFolderFor(stateDirectory, 'first')
|
||||
expect(first.launch.env?.[NATIVE_CHAT_VISUALS_DIR_ENV]).toBe(firstFolder)
|
||||
expect(first.calls.find(({ method }) => method === 'thread/start')?.params).toMatchObject({
|
||||
config: { 'sandbox_workspace_write.writable_roots': ['/user/notes', firstFolder] }
|
||||
})
|
||||
|
||||
enabled = false
|
||||
await start('disabled')
|
||||
const disabled = codex.connections[1]!
|
||||
expect(disabled.launch.env).not.toHaveProperty(NATIVE_CHAT_VISUALS_DIR_ENV)
|
||||
expect(disabled.launch.env?.KEEP).toBe('yes')
|
||||
expect(disabled.calls.map(({ method }) => method)).not.toContain('skills/extraRoots/set')
|
||||
const openIndex = disabled.calls.findIndex(({ method }) => method === 'thread/start')
|
||||
expect(disabled.calls.slice(0, openIndex).map(({ method }) => method)).not.toContain(
|
||||
'config/read'
|
||||
)
|
||||
expect(
|
||||
disabled.calls.find(({ method }) => method === 'thread/start')?.params
|
||||
).not.toHaveProperty('config')
|
||||
expect(existsSync(nativeChatVisualsFolderFor(stateDirectory, 'disabled'))).toBe(false)
|
||||
expect(first.closed).toBe(false)
|
||||
expect(first.calls).toEqual(firstCalls)
|
||||
|
||||
enabled = true
|
||||
await start('reenabled')
|
||||
const reenabled = codex.connections[2]!
|
||||
expect(
|
||||
reenabled.calls.find(({ method }) => method === 'skills/extraRoots/set')?.params
|
||||
).toEqual({ extraRoots: [SKILL.skillsRoot] })
|
||||
expect(reenabled.launch.env?.[NATIVE_CHAT_VISUALS_DIR_ENV]).toBe(
|
||||
nativeChatVisualsFolderFor(stateDirectory, 'reenabled')
|
||||
)
|
||||
expect(first.closed).toBe(false)
|
||||
expect(first.calls).toEqual(firstCalls)
|
||||
} finally {
|
||||
await adapter.closeAll()
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -1,12 +1,10 @@
|
||||
import { mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs'
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import {
|
||||
NATIVE_CHAT_VISUAL_FILE_MAX_LENGTH,
|
||||
NATIVE_CHAT_VISUAL_MAX_BYTES,
|
||||
NATIVE_CHAT_VISUAL_MAX_PER_MESSAGE,
|
||||
NATIVE_CHAT_VISUAL_TITLE_MAX_LENGTH,
|
||||
parseNativeChatVisualDirectiveLine
|
||||
} from '../../shared/native-chat-visual-directive'
|
||||
import { NATIVE_CHAT_VISUAL_THEME_TOKENS } from '../../shared/native-chat-visual-shell'
|
||||
@@ -16,7 +14,7 @@ import {
|
||||
NATIVE_CHAT_VISUALS_DIR_ENV,
|
||||
withNativeChatVisualsEnv
|
||||
} from './native-chat-visuals-delivery'
|
||||
import { nativeChatVisualsFolderFor } from './native-chat-visuals-folder'
|
||||
import { nativeChatVisualsFolderFor, nativeChatVisualsRootFor } from './native-chat-visuals-folder'
|
||||
import { buildClaudeChildProcessEnv } from '../claude/claude-child-process-environment'
|
||||
import { codexStructuredChildEnvironment } from '../codex/codex-structured-child-environment'
|
||||
import { resolveProviderChildEnv } from '../provider-process/provider-process-launch'
|
||||
@@ -50,6 +48,32 @@ function tempDir(): string {
|
||||
const logger = () => ({ warn: vi.fn(), error: vi.fn() })
|
||||
|
||||
describe('preparing a chat for visuals', () => {
|
||||
it('reads the preference for each launch and prepares nothing while it is off', async () => {
|
||||
const state = tempDir()
|
||||
let enabled = false
|
||||
const resolveSkill = vi.fn(async () => SKILL)
|
||||
const prepare = createNativeChatVisualsDelivery({
|
||||
stateDirectory: state,
|
||||
logger: logger(),
|
||||
isEnabled: () => enabled,
|
||||
resolveSkill
|
||||
})
|
||||
await expect(prepare('disabled-chat')).resolves.toBeNull()
|
||||
expect(resolveSkill).not.toHaveBeenCalled()
|
||||
expect(existsSync(nativeChatVisualsRootFor(state))).toBe(false)
|
||||
enabled = true
|
||||
const launched = await prepare('enabled-chat')
|
||||
expect(launched).toEqual({
|
||||
folder: nativeChatVisualsFolderFor(state, 'enabled-chat'),
|
||||
skill: SKILL
|
||||
})
|
||||
enabled = false
|
||||
await expect(prepare('next-chat')).resolves.toBeNull()
|
||||
expect(resolveSkill).toHaveBeenCalledOnce()
|
||||
expect(existsSync(nativeChatVisualsFolderFor(state, 'next-chat'))).toBe(false)
|
||||
expect(existsSync(launched!.folder)).toBe(true)
|
||||
})
|
||||
|
||||
it("creates the chat's own private folder and hands back the skill", async () => {
|
||||
const state = tempDir()
|
||||
const prepare = createNativeChatVisualsDelivery({
|
||||
@@ -159,9 +183,7 @@ describe('the bundled skill', () => {
|
||||
file: 'latency-by-region-7c1e.html',
|
||||
title: 'Latency by region'
|
||||
})
|
||||
expect(SKILL_TEXT).toContain(`At most ${NATIVE_CHAT_VISUAL_MAX_PER_MESSAGE} visuals`)
|
||||
expect(SKILL_TEXT).toContain(`at most ${NATIVE_CHAT_VISUAL_FILE_MAX_LENGTH} characters`)
|
||||
expect(SKILL_TEXT).toContain(`at most ${NATIVE_CHAT_VISUAL_TITLE_MAX_LENGTH}`)
|
||||
expect(SKILL_TEXT).toContain(`At most ${NATIVE_CHAT_VISUAL_MAX_PER_MESSAGE} per reply`)
|
||||
expect(SKILL_TEXT).toContain(`under ${NATIVE_CHAT_VISUAL_MAX_BYTES / 1024} KB`)
|
||||
expect(SKILL_TEXT).toContain(NATIVE_CHAT_VISUALS_DIR_ENV)
|
||||
expect(SKILL_TEXT).toMatch(new RegExp(`^name: ${NATIVE_CHAT_VISUALS_SKILL_NAME}$`, 'm'))
|
||||
|
||||
@@ -28,11 +28,15 @@ export type PrepareNativeChatVisuals = (
|
||||
export function createNativeChatVisualsDelivery(deps: {
|
||||
stateDirectory: string
|
||||
logger: StructuredAgentSessionLogger
|
||||
isEnabled?: () => boolean
|
||||
resolveSkill?: () => Promise<NativeChatVisualsSkillLocation | null>
|
||||
}): PrepareNativeChatVisuals {
|
||||
const resolveSkill = deps.resolveSkill ?? (() => resolveNativeChatVisualsSkillLocation())
|
||||
return async (sessionId) => {
|
||||
try {
|
||||
if (deps.isEnabled?.() === false) {
|
||||
return null
|
||||
}
|
||||
const skill = await resolveSkill()
|
||||
if (!skill) {
|
||||
deps.logger.warn('native-chat visuals skill is missing from this install', {
|
||||
|
||||
@@ -229,6 +229,7 @@ export class OrcaRuntimeWithGetWorktreePs extends OrcaRuntimeWithStartTuiIdleVis
|
||||
// Wired only here, so a test runtime never runs a real `claude --version`.
|
||||
claudeCliFlags: claudeCliFlagSupport,
|
||||
nativeChatVisuals: {
|
||||
isEnabled: () => this.requireStore().getSettings().nativeChatInlineVisuals !== false,
|
||||
// Chats and their visuals are shared by every profile; each profile keeps its own catalog.
|
||||
workspaceVerdicts: createNativeChatVisualsWorkspaceVerdicts(() =>
|
||||
this.store ? readNativeChatVisualsWorkspaceCatalogs(this.store) : null
|
||||
|
||||
@@ -130,6 +130,7 @@ export type RuntimeStore = {
|
||||
hostSettingOverrides?: GlobalSettings['hostSettingOverrides']
|
||||
agentSkillSharingEnabled?: GlobalSettings['agentSkillSharingEnabled']
|
||||
nativeChatSessionOptions?: GlobalSettings['nativeChatSessionOptions']
|
||||
nativeChatInlineVisuals?: GlobalSettings['nativeChatInlineVisuals']
|
||||
nativeChatInheritShellEnvironment?: GlobalSettings['nativeChatInheritShellEnvironment']
|
||||
nativeChatShellEnvironmentVariables?: GlobalSettings['nativeChatShellEnvironmentVariables']
|
||||
aiVaultSearch?: GlobalSettings['aiVaultSearch']
|
||||
|
||||
@@ -116,7 +116,8 @@ function nativeChatVisualsFor(deps: StructuredAgentSessionRuntimeDeps): {
|
||||
? {
|
||||
prepareVisuals: createNativeChatVisualsDelivery({
|
||||
stateDirectory: deps.stateDirectory,
|
||||
logger: deps.logger
|
||||
logger: deps.logger,
|
||||
isEnabled: deps.nativeChatVisuals.isEnabled
|
||||
})
|
||||
}
|
||||
: {}
|
||||
|
||||
@@ -106,6 +106,8 @@ export type StructuredAgentSessionRuntimeDeps = {
|
||||
/** Gives each chat a visuals folder and the skill that teaches it, and sweeps folders whose chat
|
||||
* is gone. Wired by the real hosts only, so a test runtime never loads the bundled skill. */
|
||||
nativeChatVisuals?: {
|
||||
/** Read this host's preference when a chat starts a provider process. */
|
||||
isEnabled: () => boolean
|
||||
workspaceVerdicts: NonNullable<NativeChatVisualsSweepDeps['workspaceVerdicts']>
|
||||
}
|
||||
/** Provider transports are overridden only to drive the runtime against scripted children. */
|
||||
|
||||
@@ -19,6 +19,8 @@ import {
|
||||
} from './native-chat-visual-owner'
|
||||
import { clearNativeChatVisualCacheForTests } from './native-chat-visual-read-client'
|
||||
import { TooltipProvider } from '@/components/ui/tooltip'
|
||||
import { useAppStore } from '@/store'
|
||||
import { getDefaultSettings } from '../../../../shared/constants'
|
||||
|
||||
const owner: NativeChatVisualOwner = {
|
||||
target: { kind: 'local' },
|
||||
@@ -61,6 +63,22 @@ afterEach(() => {
|
||||
})
|
||||
|
||||
describe('NativeChatMarkdown visuals', () => {
|
||||
it('still renders an existing reply after inline visuals are turned off', async () => {
|
||||
const settings = useAppStore.getState().settings
|
||||
useAppStore.setState({
|
||||
settings: { ...getDefaultSettings('/tmp'), nativeChatInlineVisuals: false }
|
||||
})
|
||||
try {
|
||||
const { container } = render(
|
||||
withOwner(<NativeChatMarkdown content={LINE} visualMessageId="m1" />)
|
||||
)
|
||||
await waitFor(() => expect(container.querySelector('iframe')).not.toBeNull())
|
||||
expect(container).not.toHaveTextContent('::orca-visual')
|
||||
} finally {
|
||||
useAppStore.setState({ settings })
|
||||
}
|
||||
})
|
||||
|
||||
it('holds back an unfinished directive at the end of a streaming reply', () => {
|
||||
const { container } = render(
|
||||
withOwner(
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
// @vitest-environment happy-dom
|
||||
import { cleanup, fireEvent, render, screen } from '@testing-library/react'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { getDefaultSettings } from '../../../../shared/constants'
|
||||
import type { GlobalSettings } from '../../../../shared/global-settings-types'
|
||||
import { ChatSettingsSection } from './ChatSettingsSection'
|
||||
import { ActiveSettingsSectionProvider } from './SettingsSection'
|
||||
import { getChatInlineVisualsSearchEntry } from './chat-inline-visuals-search'
|
||||
import { matchesSettingsSearch } from './settings-search'
|
||||
import { getSettingsScrollTarget } from './settings-navigation-foundations'
|
||||
import { buildSettingsNavigationMetadata } from '@/hooks/useSettingsNavigationMetadata'
|
||||
import { buildCmdJSettingsResults } from '../cmd-j/palette-results'
|
||||
|
||||
const state = vi.hoisted((): { settingsSearchQuery: string; settings: GlobalSettings | null } => ({
|
||||
settingsSearchQuery: '',
|
||||
settings: null
|
||||
}))
|
||||
vi.mock('../../store', () => ({
|
||||
useAppStore: Object.assign((selector: (value: typeof state) => unknown) => selector(state), {
|
||||
getState: () => state
|
||||
})
|
||||
}))
|
||||
afterEach(cleanup)
|
||||
beforeEach(() => {
|
||||
state.settingsSearchQuery = ''
|
||||
state.settings = null
|
||||
})
|
||||
|
||||
function renderChat(nativeChatInlineVisuals: boolean | undefined) {
|
||||
const settings = {
|
||||
...getDefaultSettings('/tmp'),
|
||||
experimentalStructuredNativeChat: true,
|
||||
nativeChatInlineVisuals
|
||||
}
|
||||
state.settings = settings
|
||||
const updateSettings = vi.fn()
|
||||
return {
|
||||
...render(
|
||||
<ActiveSettingsSectionProvider value="chat">
|
||||
<ChatSettingsSection
|
||||
settings={settings}
|
||||
updateSettings={updateSettings}
|
||||
writeSourceControlAiSettings={async () => {}}
|
||||
searchEntries={[getChatInlineVisualsSearchEntry()]}
|
||||
showDesktopOnlySettings
|
||||
isMounted
|
||||
/>
|
||||
</ActiveSettingsSectionProvider>
|
||||
),
|
||||
updateSettings
|
||||
}
|
||||
}
|
||||
|
||||
describe('Inline visuals on the Chat settings page', () => {
|
||||
it.each([true, undefined])('defaults on for saved preference %s and turns off', (preference) => {
|
||||
expect(getDefaultSettings('/tmp').nativeChatInlineVisuals).toBe(true)
|
||||
const { updateSettings } = renderChat(preference)
|
||||
const toggle = screen.getByRole('switch', { name: 'Toggle inline visuals' })
|
||||
expect(toggle.getAttribute('aria-checked')).toBe('true')
|
||||
fireEvent.click(toggle)
|
||||
expect(updateSettings).toHaveBeenCalledWith({ nativeChatInlineVisuals: false })
|
||||
})
|
||||
|
||||
it('uses saved off and turns on', () => {
|
||||
const { updateSettings } = renderChat(false)
|
||||
expect(screen.getByText(/Applies to newly started chats/)).toBeTruthy()
|
||||
const toggle = screen.getByRole('switch', { name: 'Toggle inline visuals' })
|
||||
expect(toggle.getAttribute('aria-checked')).toBe('false')
|
||||
fireEvent.click(toggle)
|
||||
expect(updateSettings).toHaveBeenCalledWith({ nativeChatInlineVisuals: true })
|
||||
})
|
||||
|
||||
it.each(['inline visuals', 'charts', 'mockups'])(
|
||||
'finds %s and resolves its own Chat target',
|
||||
(query) => {
|
||||
state.settingsSearchQuery = query
|
||||
const { container } = renderChat(true)
|
||||
const toggle = screen.getByRole('switch', { name: 'Toggle inline visuals' })
|
||||
expect(screen.queryByRole('spinbutton', { name: 'Text size' })).toBeNull()
|
||||
const sections = buildSettingsNavigationMetadata({
|
||||
isMac: false,
|
||||
isWindows: false,
|
||||
isWebClient: false,
|
||||
experimentalStructuredNativeChat: true,
|
||||
repos: []
|
||||
})
|
||||
const results = buildCmdJSettingsResults(sections)
|
||||
expect(matchesSettingsSearch(query, getChatInlineVisualsSearchEntry())).toBe(true)
|
||||
const result = results.find((entry) => entry.title === 'Inline visuals')
|
||||
expect(result?.sectionId).toBe('chat')
|
||||
expect(result?.targetSectionId).toBe('chat-inline-visuals')
|
||||
expect(getSettingsScrollTarget('chat-inline-visuals', container)?.contains(toggle)).toBe(true)
|
||||
expect(results.filter((entry) => entry.title === 'Inline visuals')).toHaveLength(1)
|
||||
}
|
||||
)
|
||||
})
|
||||
@@ -8,6 +8,7 @@ import { ChatSettingsSection } from './ChatSettingsSection'
|
||||
import { ActiveSettingsSectionProvider } from './SettingsSection'
|
||||
import { getChatAppearanceSearchEntries } from './chat-appearance-search'
|
||||
import { getChatNamingSearchEntry } from './chat-naming-search'
|
||||
import { getChatInlineVisualsSearchEntry } from './chat-inline-visuals-search'
|
||||
import { useSettingsRepoScrollEffects } from './use-settings-repo-scroll-effects'
|
||||
import type { SettingsStoreModel } from './use-settings-store-model'
|
||||
import type { SettingsInteractionController } from './use-settings-interaction-controller'
|
||||
@@ -96,7 +97,11 @@ function NavigationHarness({
|
||||
settings={settings}
|
||||
updateSettings={vi.fn()}
|
||||
writeSourceControlAiSettings={async () => {}}
|
||||
searchEntries={[...getChatAppearanceSearchEntries(), getChatNamingSearchEntry()]}
|
||||
searchEntries={[
|
||||
...getChatAppearanceSearchEntries(),
|
||||
getChatNamingSearchEntry(),
|
||||
getChatInlineVisualsSearchEntry()
|
||||
]}
|
||||
showDesktopOnlySettings
|
||||
isMounted
|
||||
/>
|
||||
@@ -105,6 +110,23 @@ function NavigationHarness({
|
||||
}
|
||||
|
||||
describe('Chat settings deep links', () => {
|
||||
it('activates Chat and scrolls to Inline visuals from another settings page', async () => {
|
||||
const scrollIntoView = vi.spyOn(HTMLElement.prototype, 'scrollIntoView')
|
||||
const { container } = render(
|
||||
<NavigationHarness enabled initialSection="appearance" targetSection="chat-inline-visuals" />
|
||||
)
|
||||
await waitFor(() => {
|
||||
expect(screen.getByRole('status', { name: 'Selected settings page' }).textContent).toBe(
|
||||
'chat'
|
||||
)
|
||||
})
|
||||
const visuals = container.querySelector('#chat-inline-visuals')
|
||||
expect(visuals?.querySelector('[role="switch"]')).toBe(
|
||||
screen.getByRole('switch', { name: 'Toggle inline visuals' })
|
||||
)
|
||||
await waitFor(() => expect(scrollIntoView.mock.contexts).toContain(visuals))
|
||||
})
|
||||
|
||||
it('activates Chat and renders the moved row for a deep link', async () => {
|
||||
render(<NavigationHarness enabled initialSection="appearance" />)
|
||||
await waitFor(() => {
|
||||
|
||||
@@ -7,6 +7,7 @@ import { ChatSettingsSection } from './ChatSettingsSection'
|
||||
import { ActiveSettingsSectionProvider } from './SettingsSection'
|
||||
import { getChatAppearanceSearchEntries } from './chat-appearance-search'
|
||||
import { getChatNamingSearchEntry } from './chat-naming-search'
|
||||
import { getChatInlineVisualsSearchEntry } from './chat-inline-visuals-search'
|
||||
import { buildSettingsNavigationMetadata } from '@/hooks/useSettingsNavigationMetadata'
|
||||
import { buildCmdJSettingsResults } from '../cmd-j/palette-results'
|
||||
import { isSettingsNavigationTarget } from '@/lib/settings-navigation-types'
|
||||
@@ -46,7 +47,12 @@ function renderChat(
|
||||
settings={settings}
|
||||
updateSettings={updateSettings}
|
||||
writeSourceControlAiSettings={async () => {}}
|
||||
searchEntries={[...getChatAppearanceSearchEntries(), getChatNamingSearchEntry()]}
|
||||
searchEntries={[
|
||||
...getChatAppearanceSearchEntries(),
|
||||
...(showDesktopOnlySettings
|
||||
? [getChatNamingSearchEntry(), getChatInlineVisualsSearchEntry()]
|
||||
: [])
|
||||
]}
|
||||
showDesktopOnlySettings={showDesktopOnlySettings}
|
||||
isMounted
|
||||
hasUnsavedChatPromptChanges={hasUnsavedChatPromptChanges}
|
||||
@@ -62,6 +68,7 @@ describe('Chat settings page', () => {
|
||||
expect(screen.getByRole('spinbutton', { name: 'Text size' })).toBeTruthy()
|
||||
expect(container.querySelector('#chat-names')).toBeNull()
|
||||
expect(screen.queryByRole('switch', { name: 'Name chats automatically' })).toBeNull()
|
||||
expect(screen.queryByRole('switch', { name: 'Toggle inline visuals' })).toBeNull()
|
||||
const sections = buildSettingsNavigationMetadata({
|
||||
isMac: false,
|
||||
isWindows: false,
|
||||
@@ -72,6 +79,7 @@ describe('Chat settings page', () => {
|
||||
const results = buildCmdJSettingsResults(sections).filter((entry) => entry.sectionId === 'chat')
|
||||
expect(results.some((entry) => entry.targetSectionId === 'chat-text-size')).toBe(true)
|
||||
expect(results.some((entry) => entry.targetSectionId === 'chat-names')).toBe(false)
|
||||
expect(results.some((entry) => entry.targetSectionId === 'chat-inline-visuals')).toBe(false)
|
||||
})
|
||||
|
||||
it.each([false, undefined])('is absent with structured chat set to %s', (enabled) => {
|
||||
@@ -99,14 +107,19 @@ describe('Chat settings page', () => {
|
||||
)
|
||||
})
|
||||
|
||||
it('renders Appearance and Chat names as peer sections with separate cards', () => {
|
||||
it('renders Appearance, Inline visuals and Chat names as peers with separate cards', () => {
|
||||
const { container } = renderChat(true)
|
||||
const appearance = container.querySelector('#chat-appearance')
|
||||
const names = container.querySelector('#chat-names')
|
||||
const visuals = container.querySelector('#chat-inline-visuals')
|
||||
expect(screen.getByRole('heading', { name: 'Chat names', level: 3 })).toBeTruthy()
|
||||
expect(appearance).toBeTruthy()
|
||||
expect(names).toBeTruthy()
|
||||
expect(appearance?.parentElement).toBe(names?.parentElement)
|
||||
expect(visuals?.parentElement).toBe(appearance?.parentElement)
|
||||
expect(visuals?.querySelector('[data-slot="card"]')).toBeTruthy()
|
||||
expect(visuals?.contains(appearance)).toBe(false)
|
||||
expect(visuals?.contains(names)).toBe(false)
|
||||
expect(appearance?.closest('[data-slot="card"]')).toBeNull()
|
||||
expect(names?.closest('[data-slot="card"]')).toBeNull()
|
||||
const appearanceCard = screen
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { translate } from '@/i18n/i18n'
|
||||
import { AppearanceChatSection } from './AppearanceChatSection'
|
||||
import { ChatNamingSetting } from './ChatNamingSetting'
|
||||
import { NativeChatInlineVisualsSetting } from './NativeChatInlineVisualsSetting'
|
||||
import { SettingsSection } from './SettingsSection'
|
||||
import { SettingsSubsectionHeader } from './SettingsFormControls'
|
||||
import { matchesSettingsSearch } from './settings-search'
|
||||
@@ -42,10 +43,7 @@ export function ChatSettingsSection({
|
||||
<SettingsSection
|
||||
id="chat"
|
||||
title={title}
|
||||
description={translate(
|
||||
'settings.chat.description',
|
||||
'Choose how chats look and get their names.'
|
||||
)}
|
||||
description={translate('settings.chat.description', 'Choose how chats look and behave.')}
|
||||
searchEntries={searchEntries}
|
||||
forceVisible={hasUnsavedChatPromptChanges}
|
||||
bodyClassName="rounded-none border-0 bg-transparent p-0 shadow-none"
|
||||
@@ -69,6 +67,13 @@ export function ChatSettingsSection({
|
||||
</Card>
|
||||
</section>
|
||||
) : null}
|
||||
{showDesktopOnlySettings ? (
|
||||
<NativeChatInlineVisualsSetting
|
||||
settings={settings}
|
||||
updateSettings={updateSettings}
|
||||
forceVisible={matchesSettingsSearch(query, [{ title }])}
|
||||
/>
|
||||
) : null}
|
||||
{showDesktopOnlySettings ? (
|
||||
<ChatNamingSetting
|
||||
key={chatPromptDiscardSignal}
|
||||
|
||||
@@ -258,3 +258,15 @@ describe('NativeChatExperimentalSetting queue follow-ups', () => {
|
||||
expect(container.querySelector(QUEUE_TOGGLE)).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('NativeChatExperimentalSetting inline visuals', () => {
|
||||
it('leaves Inline visuals on the Chat page even while structured chat is enabled', () => {
|
||||
const { queryByRole, getByRole } = renderSetting({
|
||||
experimentalNativeChat: true,
|
||||
openAgentTabsInChatByDefault: true,
|
||||
experimentalStructuredNativeChat: true
|
||||
})
|
||||
expect(getByRole('switch', { name: 'Toggle automatic resume after a restart' })).toBeTruthy()
|
||||
expect(queryByRole('switch', { name: 'Toggle inline visuals' })).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import type { GlobalSettings } from '../../../../shared/global-settings-types'
|
||||
import { translate } from '@/i18n/i18n'
|
||||
import { SettingsSwitchRow } from './SettingsFormControls'
|
||||
import { Card, CardContent } from '../ui/card'
|
||||
import { SearchableSetting } from './SearchableSetting'
|
||||
import { getChatInlineVisualsSearchEntry } from './chat-inline-visuals-search'
|
||||
|
||||
type NativeChatInlineVisualsSettingProps = {
|
||||
settings: GlobalSettings
|
||||
updateSettings: (updates: Partial<GlobalSettings>) => void
|
||||
forceVisible?: boolean
|
||||
}
|
||||
|
||||
export function NativeChatInlineVisualsSetting({
|
||||
settings,
|
||||
updateSettings,
|
||||
forceVisible = false
|
||||
}: NativeChatInlineVisualsSettingProps): React.JSX.Element {
|
||||
const enabled = settings.nativeChatInlineVisuals !== false
|
||||
const entry = getChatInlineVisualsSearchEntry()
|
||||
return (
|
||||
<SearchableSetting
|
||||
{...entry}
|
||||
id="chat-inline-visuals"
|
||||
forceVisible={forceVisible}
|
||||
className="max-w-none"
|
||||
>
|
||||
<Card>
|
||||
<CardContent>
|
||||
<SettingsSwitchRow
|
||||
label={entry.title}
|
||||
description={entry.description}
|
||||
checked={enabled}
|
||||
ariaLabel={translate(
|
||||
'components.settings.nativeChat.inlineVisualsToggleLabel',
|
||||
'Toggle inline visuals'
|
||||
)}
|
||||
onChange={() => updateSettings({ nativeChatInlineVisuals: !enabled })}
|
||||
/>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</SearchableSetting>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
import { translate } from '@/i18n/i18n'
|
||||
import { createLocalizedCatalog } from '@/i18n/localized-catalog'
|
||||
import type { SettingsSearchEntry } from './settings-search'
|
||||
|
||||
export const getChatInlineVisualsSearchEntry = createLocalizedCatalog((): SettingsSearchEntry => ({
|
||||
targetSectionId: 'chat-inline-visuals',
|
||||
title: translate('components.settings.nativeChat.inlineVisualsTitle', 'Inline visuals'),
|
||||
description: translate(
|
||||
'components.settings.nativeChat.inlineVisualsCopy',
|
||||
'Let the agent show charts, diagrams and mockups inside its replies. Applies to newly started chats.'
|
||||
)
|
||||
}))
|
||||
@@ -10,4 +10,11 @@ describe('native chat experimental search entry', () => {
|
||||
it('matches "queue" so the Queue follow-ups switch is findable', () => {
|
||||
expect(matchesSettingsSearch('queue', getNativeChatExperimentalSearchEntry())).toBe(true)
|
||||
})
|
||||
|
||||
it.each(['inline visuals', 'visuals'])(
|
||||
'leaves the moved switch out of Experimental for %s',
|
||||
(query) => {
|
||||
expect(matchesSettingsSearch(query, getNativeChatExperimentalSearchEntry())).toBe(false)
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { getAppearancePaneSearchEntries } from '@/components/settings/appearance-search'
|
||||
import { getChatAppearanceSearchEntries } from '@/components/settings/chat-appearance-search'
|
||||
import { getChatNamingSearchEntry } from '@/components/settings/chat-naming-search'
|
||||
import { getChatInlineVisualsSearchEntry } from '@/components/settings/chat-inline-visuals-search'
|
||||
import { getInputPaneSearchEntries } from '@/components/settings/input-search'
|
||||
import { getNotificationsPaneSearchEntries } from '@/components/settings/notifications-search'
|
||||
import { getShortcutsPaneSearchEntries } from '@/components/settings/shortcuts-search'
|
||||
@@ -42,12 +43,14 @@ export function buildInterfaceSettingsSections({
|
||||
title: translate('settings.appearance.chat.title', 'Chat'),
|
||||
description: translate(
|
||||
'settings.chat.description',
|
||||
'Choose how chats look and get their names.'
|
||||
'Choose how chats look and behave.'
|
||||
),
|
||||
icon: MessageSquare,
|
||||
searchEntries: [
|
||||
...getChatAppearanceSearchEntries(),
|
||||
...(showDesktopOnlySettings ? [getChatNamingSearchEntry()] : [])
|
||||
...(showDesktopOnlySettings
|
||||
? [getChatNamingSearchEntry(), getChatInlineVisualsSearchEntry()]
|
||||
: [])
|
||||
],
|
||||
group: 'interface'
|
||||
}
|
||||
|
||||
@@ -177,7 +177,7 @@
|
||||
},
|
||||
"settings": {
|
||||
"chat": {
|
||||
"description": "Choose how chats look and get their names.",
|
||||
"description": "Choose how chats look and behave.",
|
||||
"names": {
|
||||
"title": "Chat names",
|
||||
"description": "Use an agent to name new chats from their first message. Without a generated name, chats stay Claude Chat or Codex Chat.",
|
||||
@@ -18954,6 +18954,9 @@
|
||||
},
|
||||
"nativeChat": {
|
||||
"queueFollowUpsTitle": "Queue follow-ups",
|
||||
"inlineVisualsTitle": "Inline visuals",
|
||||
"inlineVisualsCopy": "Let the agent show charts, diagrams and mockups inside its replies. Applies to newly started chats.",
|
||||
"inlineVisualsToggleLabel": "Toggle inline visuals",
|
||||
"queueFollowUpsCopy": "Messages you send while the agent is working wait as cards you can steer, edit, or delete. Messages with images send right away.",
|
||||
"queueFollowUpsToggleLabel": "Toggle queue follow-ups"
|
||||
}
|
||||
|
||||
@@ -137,6 +137,7 @@ export function buildDefaultSettings(args: {
|
||||
experimentalStructuredNativeChat: false,
|
||||
nativeChatResumeWorkOnRestart: false,
|
||||
nativeChatQueueFollowUps: true,
|
||||
nativeChatInlineVisuals: true,
|
||||
nativeChatInheritShellEnvironment: true,
|
||||
nativeChatShellEnvironmentVariables: [],
|
||||
nativeChatSessionOptions: {},
|
||||
|
||||
@@ -228,6 +228,8 @@ export type GlobalSettings = NativeChatGlobalSettings & {
|
||||
/** Chat-wide: hold a mid-turn send as an editable queued draft that goes when the turn ends
|
||||
* (capable hosts only). Absent = on; off keeps mid-turn sends immediate. */
|
||||
nativeChatQueueFollowUps?: boolean
|
||||
/** Teach newly started native chats to create inline visuals; absent means on. */
|
||||
nativeChatInlineVisuals?: boolean
|
||||
/** Structured chat only: Codex/Claude children inherit the whole login-shell environment.
|
||||
* Off passes only `nativeChatShellEnvironmentVariables` (plus a PATH/locale baseline). */
|
||||
nativeChatInheritShellEnvironment?: boolean
|
||||
|
||||
Reference in New Issue
Block a user