mirror of
https://github.com/stablyai/orca.git
synced 2026-11-03 18:47:40 +00:00
* fix(browser): let agents and users answer page alert, confirm and prompt dialogs on their page * fix(browser): watch a replacement guest for dialogs and ignore the opening keypress * fix(browser): ignore a repeated Enter in a guarded prompt field
6.0 KiB
6.0 KiB
Built-in browser commands
Pin one page, then use a snapshot-interact-re-snapshot loop on it:
ORCA tab current --json
ORCA goto --page <browserPageId> --url https://example.com --json
ORCA snapshot --page <browserPageId> --json
ORCA click --page <browserPageId> --element @e3 --json
ORCA snapshot --page <browserPageId> --json
Common commands:
ORCA goto --page <browserPageId> --url <url> --json
ORCA back --page <browserPageId> --json
ORCA reload --page <browserPageId> --json
ORCA snapshot --page <browserPageId> --json
ORCA screenshot --page <browserPageId> --json
ORCA full-screenshot --page <browserPageId> --json
ORCA pdf --page <browserPageId> --json
ORCA click --page <browserPageId> --element <ref> --json
ORCA fill --page <browserPageId> --element <ref> --value <text> --json
ORCA type --page <browserPageId> --input <text> --json
ORCA select --page <browserPageId> --element <ref> --value <value> --json
ORCA check --page <browserPageId> --element <ref> --json
ORCA scroll --page <browserPageId> --direction down --amount 1000 --json
ORCA hover --page <browserPageId> --element <ref> --json
ORCA focus --page <browserPageId> --element <ref> --json
ORCA keypress --page <browserPageId> --key Enter --json
ORCA upload --page <browserPageId> --element <ref> --files <paths> --json
ORCA wait --page <browserPageId> --text <text> --json
ORCA wait --page <browserPageId> --url <substring> --json
ORCA wait --page <browserPageId> --selector <css> --json
ORCA wait --page <browserPageId> --load networkidle --json
ORCA eval --page <browserPageId> --expression <js> --json
ORCA dialog accept --page <browserPageId> [--text <answer>] --json
ORCA dialog dismiss --page <browserPageId> --json
ORCA set device --page <browserPageId> --name "iPhone 15" --json
ORCA viewport --page <browserPageId> --width 390 --height 844 --scale 3 --mobile --json
ORCA tab list --json
ORCA tab create --url <url> --json
ORCA tab switch --index <n> --json
ORCA tab close --index <n> --json
ORCA cookie get --page <browserPageId> --json
ORCA capture start --page <browserPageId> --json
ORCA console --page <browserPageId> --limit 50 --json
ORCA network --page <browserPageId> --limit 50 --json
ORCA exec --page <browserPageId> --command "help" --json
Browser rules:
- Re-snapshot after navigation, tab switches, clicks that change the page, and any
browser_stale_ref. - Refs like
@e1are assigned bysnapshot, scoped to one tab, and invalidated by navigation or tab switch. - Pass
--page <browserPageId>on every command that acts on one page;ORCA <command> --helplists--pagewhen it does.tab list,tab current,tab createandopen-urltake no--page, andtab switch/tab closepick a tab by--indexor--page, never both, since--pagewins. Take the ID fromresult.tab.browserPageIdinORCA tab current --jsonorresult.browserPageIdinORCA tab create --jsonand keep it for the task;tab currentshows the routed tab, not the one you pin. When the user means a specific tab, find it by URL or title inORCA tab list --json. - Without
--page, a command goes to the worktree's current tab. That stays the agent's tab while it has had a command in the last 20 seconds or so, even after the user picks another (tab currentreports it too), then moves to the user's pick. - After
ORCA tab switch, pass the new page'sbrowserPageId. - Browser commands default to the current worktree.
--pagereaches the page in any worktree unless you also pass--worktree <selector>. Use--worktree allonly intentionally. - Use typed tab commands (
ORCA tab list/create/close/switch), notORCA exec --command "tab ...", so Orca keeps UI state synchronized. - Use
ORCA set deviceorORCA viewport --mobile, notORCA exec --command "set device ...", so the page also gets touch input andpointer: coarse. Each page load (reload,goto, link click) resets it, so run it again after navigating. - Prefer
wait --text,--url,--selector, or--loadafter async page changes instead of bare timeouts. - Anything not listed above goes through
ORCA exec --page <browserPageId> --command "<agent-browser command>". fillsets a field's whole value in one step, at any size.typepresses real keys into the focused element, sominlength/maxlength, key handlers, Enter and Tab act as they would for a person;ORCA focusthe element first. Usetypewhen the page reacts to keystrokes.- On server-hosted pages
fillandtypestill insert text the engine's own way. If either does nothing on a custom input there, tryORCA focus --page <browserPageId> --element @e1 --jsonthenORCA inserttext --page <browserPageId> --text "text" --json. - A client-hosted page renders in the paired desktop's browser engine, so every command against it needs that desktop online and returns
browser_host_unavailablewhile it is closed, asleep, or disconnected. Server-hosted pages run with no desktop attached; prefer them for long or unattended automation.
Common recoveries:
browser_no_tab: open a tab withORCA tab create --url <url> --json.browser_stale_ref: runORCA snapshot --page <browserPageId> --jsonand retry on the same page with fresh refs.browser_element_not_interactable: the element can't take that command, and the message says why (not a text field, disabled, read-only, not a<select>, not a file input). Re-snapshot and pick the right ref; for a custom dropdown, click it and then click the option.browser_tab_not_found: runORCA tab list --jsonbefore switching or closing.browser_dialog_open/browser_dialog_opened: the page has a JavaScript alert, confirm or prompt open, and the error quotes it. Answer it withORCA dialog accept(--textfor a prompt; without it a prompt takes its default) orORCA dialog dismissbefore any other command on that page; a command that opened it may already have taken effect, so check before repeating it.browser_host_unavailable: the desktop hosting the page is offline. Bring it back, or recreate the page with server placement if the work must outlive the desktop session.