Commit Graph
4 Commits
Author SHA1 Message Date
2584ed906e feat: play video and music files in mobile previews (#26148)
Play workspace video and music files on mobile using bounded authenticated downloads and native media controls.

Adversarial review fixes cover rewritten files and Android policy tests. Includes iOS simulator screenshots and a playback recording in PR #26148.

Co-authored-by: ChangJun Park <40492343+ckdwns9121@users.noreply.github.com>
Co-authored-by: Lirone Levy <lirone88@outlook.fr>
Co-authored-by: dupi <david.li.du@gmail.com>
Co-authored-by: John Cusack <5961784+John-Cusack@users.noreply.github.com>
2026-10-07 03:41:42 -07:00
Brennan Benson ca7c14db08 fix(mobile): start + menu, quick command and diff-note agents through agent.launch (#22954)
* fix(mobile): start + menu, quick command and diff-note agents through agent.launch

The session screen's + menu, agent quick commands and diff notes' New agent
session now ask the host to start the agent with agent.launchReplay, so the
host picks chat or terminal from the desktop's default and delivers any
prompt. Hosts without the launch capabilities keep today's paths.

The phone's pending tab choice is one value (a tab, a terminal by handle, or a
launched surface) instead of two refs, and a launched chat is found by its
session id in the next snapshot rather than a predicted tab id. A launched
surface waits a bounded number of snapshots for its tab.

* test(mobile): add the + menu and diff-note launch scenarios to the recording corpus

* test(mobile): repin bridged-parity tallies for the four launch goldens; drop test casts

The corpus grows from 790 to 794 goldens; all four new ones replay identically.

* fix(mobile): show a refused agent launch as a toast beside open tabs

The inline create error renders only in an empty session, so a host refusal
(for example a disabled agent) from the + menu in a session with tabs showed
nothing. Always toast the failure: the caller's own copy when it gave one,
otherwise the host's reason.

* test(mobile): type the launch reply helper with the shared launch outcome types

* fix(mobile): record a launched agent's tab as this device's pick on the host

A launch carries no navigation, so the phone selected the new tab only
locally while the host kept this device on the tab it had before. Leaving
the session and coming back, or a reconnect that reset the screen, reopened
that old tab. The "+" terminal path this replaced asked the host to select
the tab for the caller.

When a launched surface's tab lands in a snapshot, activate it for the
caller exactly as a tap does. The resolver now names the landed tab in
place of the unused `missed` flag. Route parity re-pinned for the new
activation body, identity payload and strings.

* fix(mobile): land on a launched agent's tab without a 500 ms wait or a blank pane

The host publishes a launched tab before it replies, so the tab list the
phone already holds usually has it by the time the reply arrives. The
launch paths still waited for a refetch 500 ms later, leaving the phone on
the old tab for that long after every launch. Read the tab list at once.

On hosts without agent.launch, the chat path also unsubscribed the open
terminal and cleared its handle before the chat's tab landed, while the old
terminal tab stayed selected: a blank pane until the next tab list. Leave the
open tab live until the chat lands, as the launch path does; applying that
tab list tears the old terminal down.

Route parity re-pinned for the two bodies.

* fix(mobile): keep a tab the user picked while a prompted launch was still replying

A quick command or review-notes launch now waits for the host to deliver the prompt, which can take up to a minute. The launched tab shows up in the tab row well before that, so a user who tapped another tab meanwhile was pulled back onto the launched one when the reply arrived, and that pick was recorded on the host.

The launch now remembers which tab the phone was on when it started and only takes focus if the phone is still there when the reply lands. Any move made in between, by a tap or by the computer navigating this phone, wins. Session route parity re-pinned for the handleCreateTerminal body only.

* fix(mobile): name a launched agent's tab before asking, and land on it when it is listed

A "+" menu, quick-command or review-notes launch now reserves its tab before it asks the host: a fresh pane key (tab and leaf UUIDs) and, for an agent the host may start as a chat, a session id. Both are minted once per launch and sent unchanged on every replay, since the host's replay fingerprint covers them.

The phone arms its pending selection with that reservation before sending, so it lands on the terminal (matched by pane halves) or chat (matched by session id) as soon as the tab is listed. For an agent whose prompt is pasted after start, that is long before the reply, which waits for delivery. Landing also frees the "+" lock; the lock holds the create's id, so an older launch's reply cannot free a newer one's. The reply now only adds its own handle or session id (an older host ignores the reservation), starts the fallback countdown, and reports prompt delivery.

A tab the user picks mid-launch replaces the pending selection, so the launch-start tab check is gone. A reservation the host refuses as already taken reads "Couldn't start the agent. Try again." on the first send, and as unconfirmed after a replay. The mobile UUID fallback now yields a v4 UUID, because a pane key's leaf must be one. The host launch path moved to new-tab-agent-host-launch.ts; session route parity re-pinned for that move and the landing's lock release.

* test(mobile): expect the launch reservation in the four launch scenarios

The four launch scenarios now expect the pane key and session id the phone sends (the scripted ids come first, so the operation id moves from ...001 to ...004).

* fix(mobile): don't say an agent may not have started while the user is looking at it

When a launch's reply was lost after its tab had already landed, the phone said "Couldn't confirm the agent started", although the listed tab proves it did. Now a listed tab narrows the doubt to the prompt or notes ("The agent started, but couldn't confirm the notes were sent."), the notes stay unsent, and a bare launch says nothing. Only the nested-function parity pin moves, for handleCreateTerminal passing the tab list to the launch.

* test(mobile): read the launch's sent reservation through the host's params schema

The anti-slop audit rejects Reflect.get; parsing with AgentLaunchReplay also
asserts the host accepts the params the phone sent.

* test(mobile): check the launch reservation against the host without importing its schema

Mobile code may import the params contract only as types. The phone's tests
now read the sent reservation by narrowing, a chat reservation is checked
through the real host dispatcher, and the older-host drop is pinned host-side.

* fix(mobile): don't send the same review notes to a second new agent

The "+" lock is now freed when the launched tab lands, but review notes are
only cleared when the launch's reply confirms delivery, which for a prompted
launch can take up to a minute. In that window "Send review notes to AI" still
offered the same notes, and choosing a new agent session started a second
agent with them.

The notes a new agent session is being started with are now held from the tap
until that launch settles: the Send button no longer counts them, the sheet no
longer offers them, and a stale tap on the old sheet starts nothing. Notes the
host did not deliver become sendable again once the reply arrives.

* test(mobile): record the + menu and diff-note launch goldens

4 added (+ as a terminal, + as a chat, notes delivered, notes not delivered). 10 existing create-terminal goldens move only because the recorded state now shows one pending selection instead of two refs; their requests are unchanged.
2026-09-30 00:42:52 -07:00
Neil 7caf7ab1e1 refactor(mobile): split session and terminal surfaces
(cherry picked from commit f129f2926a)
2026-08-31 23:35:08 -07:00
3d341c132f fix(mobile): dismiss the keyboard after sending to an agent (#17059)
* fix(mobile): dismiss the keyboard after sending to an agent

Sending a message left the software keyboard up, covering the reply the
user was waiting on. Drop it once the send is accepted, on all three send
paths: the terminal live input, the buffered command input, and the chat
composer.

Gated on the tab being an agent session. A plain shell keeps the keyboard
so back-to-back commands stay typeable, a rejected send keeps it so the
handed-back draft stays editable, and the accessory shortcut row is
untouched because dismissing would pull away the row being tapped.

* fix(mobile): gate keyboard dismissal on accepted sends

* fix(mobile): fence keyboard dismissal completions

* fix(mobile): fence stale send completions

* test(mobile): update terminal guard expectations

* fix(mobile): restore rejected buffered drafts by origin

* fix(mobile): preserve intentional buffered draft clears

* fix(mobile): harden send dismissal authority

* test(mobile): preserve Strict Mode send dismissal

* fix(mobile): preserve drafts across terminal remints

* fix(mobile): preserve draft ownership through terminal races

* fix(mobile): harden draft recovery and send freshness

* fix(mobile): fence route reuse and native draft clears

* fix(mobile): preserve native draft edits before clear

* test(mobile): pin the terminal-list sweep that bounds buffered drafts

`bufferedTerminalDraftState.pruneDrafts(retainedHandles)` is the only bound on
two structures that live as long as the session screen — the buffered-draft
record and the pending-restoration map — and nothing failed when it was deleted
or when it was pointed at the raw `terminal.list` handles instead of the
retained set. Both mutations reddened 0 of 3,949 mobile tests.

Adds the wiring pin (both mutations now redden it) plus two behavioural tests
showing why the argument matters: `terminal.list` omits a chat-covered handle
while the desktop graph reloads, so the raw list drops a draft the user is
still holding while the retained set keeps it.

---------

Co-authored-by: Merge Sim <merge@sim.local>
Co-authored-by: Merge Sim <sim@local>
2026-08-30 16:59:55 -07:00