1 Commits

Include renames

Author	SHA1	Message	Date
4 people	42b2ecc5c6	feat(emulator): Android emulation via scrcpy (cross-platform, iOS parity) (#6434) ... * docs: add Android emulation design spec Adds the design for first-class Android emulator support as a cross-platform peer of the iOS simulator feature: an extracted EmulatorBackend interface (iOS + Android), full AVD lifecycle management via the Android SDK, a live scrcpy H.264 pane decoded in-renderer with WebCodecs, the full control surface (tap/gesture/type/buttons/rotate), accessibility tree, app install/launch, runtime permissions, logcat, and a dedicated orca-emulator-android skill. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(emulator): add EmulatorBackend interface + backend/codec session tags First step of multi-backend emulator support: introduce the EmulatorBackend type and tag each session with its backend kind + stream codec, defaulting to ios/mjpeg so existing serve-sim behavior is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(emulator): extract IosEmulatorBackend and make the bridge a router Move the serve-sim/simctl device + helper + input mechanics out of EmulatorBridge into IosEmulatorBackend (implementing EmulatorBackend). The bridge now owns the session registry and lifecycle orchestration and routes each command to the backend that owns the target device. iOS behavior is unchanged; the existing bridge tests pass untouched and the backend gains its own input-op coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): add pure Android leaf modules (sdk/adb/avd/scrcpy/input/ax) Dependency-injected building blocks for the Android emulator backend, each unit- tested in isolation: SDK + tool discovery, adb device/output parsing, AVD list + boot arg building, scrcpy control-socket byte encoders, normalized<->pixel + keycode mapping, and a uiautomator XML accessibility-tree parser. Not yet wired; AndroidEmulatorBackend composes these in the next phase. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(emulator): gate availability on the iOS backend + lock it with tests inspectEmulatorAvailability now decides iOS host support via the registered iOS backend instead of a bare platform literal, routing the decision through the multi-backend seam. Output shape and all messages are unchanged (the settings pane still reads simctl/serveSim). Adds the previously-missing regression tests covering the unsupported, ready, no-devices, and tool-failure paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): add Android app/permissions/logcat arg builders Pure adb arg-builders + a logcat line parser for app install/launch, runtime permission grant/revoke/reset, and logcat capture. Unit-tested in isolation; wired into AndroidEmulatorBackend's capability verbs in a later phase. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): AndroidEmulatorBackend device management + unified device list Adds the Android backend (registered alongside iOS in the bridge): SDK-gated host support, device/AVD discovery and merge, AVD boot + boot-completion wait, shutdown, and tap/swipe/type/button/rotate/exec via `adb shell input` so control works without the scrcpy server (the live H.264 stream lands in the streaming phase). Surfaces everything through a new cross-platform `orca emulator devices` command (RPC emulator.listDevices -> bridge.listAllDevices) with a platform column. Device inventory is split into its own module to keep files focused. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): Android capability verbs (install/launch/permissions/ax/logcat) Wires the Android capability operations into AndroidEmulatorBackend and exposes them through a capability-gated bridge router (runCapability), RPC, and CLI: - orca emulator install/launch/permissions/ax/logcat Capabilities are advertised per backend; calling one on a backend that lacks it (e.g. iOS) fails with emulator_unsupported instead of a silent no-op. Input ops and capability ops are split into focused modules to keep files under the line cap; the runtime shares one target-param type. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(skill): add orca-emulator-android skill + cross-ref from iOS skill Documents the cross-platform Android emulator control surface (devices, input, hardware buttons, rotate, install/launch, permissions, ax, logcat) driveable via the orca CLI today, and notes the live visual pane is in development. Points the iOS skill's "when not to use" at the new Android skill. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): Android live-pane streaming scaffolding (scrcpy + WebCodecs) Builds the H.264 video path as scaffolding: scrcpy frame/codec-meta parsing, server-deploy arg builders, control-protocol encoders (committed earlier), the stream session (server + sockets), a video pub/sub registry, the emulator:videoStream* IPC channel, and a renderer WebCodecs->canvas hook. Pure framing/deploy/registry are unit-tested; the socket/WebCodecs/jar integration is clearly flagged UNVERIFIED and the remaining wiring (startSession, preload, pane codec branch, packaging the jar) is documented in docs/android-emulation-streaming.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: fix streaming notes doc path in video-stream hook comment Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(emulator): add diagnostic probes for Android testing Adds an emulator-probe logger (console + temp file at os.tmpdir()/orca-android-emu-probe.log) and wires probes at four layers so errors surface during manual testing: every emulator.* RPC call + error (RPC dispatcher), every adb/emulator command + non-zero exit (command runner), and the scrcpy session + video-stream IPC lifecycle. Temporary diagnostics; remove or gate behind a flag once the Android pane is validated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): show Mobile Emulator settings cross-platform + aggregate Android availability The Mobile Emulator settings section is no longer macOS-gated (Android works on Windows/Linux), and inspectEmulatorAvailability now aggregates the iOS and Android backends: Android devices/AVDs appear in the device list and a host without iOS gets the Android setup message instead of "requires macOS". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): show Mobile Emulator sidebar nav entry on non-mac desktops The settings sidebar nav registered the Mobile Emulator entry behind isMac, so it stayed hidden on Windows/Linux even after the section content was ungated. Widen it to showDesktopOnlySettings to match the section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): wire Android startSession to scrcpy + client-downloaded jar AndroidEmulatorBackend.startSession now boots the device, ensures the scrcpy server jar (downloaded by the client into the per-user cache on first use, not bundled), starts a ScrcpyStreamSession, and feeds its H.264 frames to the video registry; stopHelperForDevice tears it down. Sessions carry their backend kind so worktree-active routing picks the right backend. Boot, host SDK discovery, and the stream starter are split into focused modules to stay under the line cap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): stop the iOS backend from claiming Android devices off-mac iOS ownsDevice now returns false unless the host supports it, so on Windows an Android serial routes to the Android backend instead of erroring with "requires macOS". Backend-for-device fallback prefers a host-supported backend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): scrcpy scid 31-bit + retry video socket until server delivers Two fixes validated against a real emulator: scrcpy parses scid as a signed 32-bit hex int, so mask to 31 bits + pad to 8 digits (8-byte values overflowed and the server exited). And adb accepts the forwarded TCP connection before the server's abstract socket exists then resets it, so retry the video socket until it actually delivers the dummy byte before connecting control. H.264 meta now arrives (576x1280). Adds socket/server-exit diagnostics probes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): render the Android H.264 pane via WebCodecs Wires the live Android pane end-to-end: preload exposes emulator video stream APIs; the pane's device list uses the unified emulator.listDevices (Android + iOS); and emulator-screen-stream-content renders a WebCodecs <canvas> for scrcpy:// sessions (H.264, SPS/PPS prepended to the first keyframe) instead of the MJPEG <img>. The video hook reports the stream size for the device frame. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): buffer the current GOP for late video subscribers The renderer subscribes after attach already started the scrcpy stream, so the registry now caches the current GOP (keyframe + following deltas) alongside the codec meta and config, and replays it on subscribe. A pane opened mid-stream decodes from the keyframe immediately instead of showing black until scrcpy's next periodic keyframe (~10s). Refreshes the now-validated session doc comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): show New Mobile Emulator tab action off macOS The tab create menu and its dropdown item gated the New Mobile Emulator action on isMacOs, hiding it on Windows/Linux where Android emulation is now supported. Gate on mobileEmulatorEnabled + onNewSimulatorTab (already cross-platform) so the action appears wherever a mobile emulator backend is available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): open the Mobile Emulator tab off macOS openMobileEmulatorTab and ensureSimulatorTab both returned null unless the host was macOS, so the New Mobile Emulator action no-opped on Windows/Linux even though the menu entry showed. Drop the isMacOsHost early-returns; the mobileEmulatorEnabled setting and backend availability already gate the feature. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): resolve a default attach device across backends emulatorAttach with no device fell back only to the iOS listSimulators picker (empty on Windows/Linux), so the pane's no-device launch flow errored. Extract resolveDefaultAttachDevice: iOS default first, else the first booted (else first) device across host backends, so Android attaches without an explicit device. Split into its own module to stay under the line cap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): render the pane off macOS instead of an unavailable wall EmulatorPane short-circuited to the "macOS only" EmulatorUnavailablePane on any non-Mac host, blocking the now-working Android pane. Always render the pane content; its device discovery and error surface handle a missing backend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): let attach boot a shut-down AVD with a stale active session getReusableActiveForWorktree called resolveDeviceId on the requested device, which throws for a not-yet-booted Android AVD, aborting the attach. Guard it so a resolve failure means "not the active device" and the attach falls through to a fresh boot — so picking a shut-down AVD in the pane and hitting Connect boots it via ensureBooted instead of erroring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): launch the AVD detached instead of via execFile bootAndroidDevice started the emulator through the command runner (execFile with a timeout + 1MB stdout maxBuffer), which kills the long-running, verbose emulator process — so booting an AVD from the pane never actually came up. Spawn it detached with no stdio and unref it so it outlives the call, mirroring how the scrcpy server is launched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): dedupe concurrent attaches into one scrcpy stream Extract AndroidStreamController to own the per-serial scrcpy lifecycle and dedupe starts: concurrent attaches (e.g. the pane's auto-attach racing the tab launch) now share one in-flight start and reuse the live stream instead of spawning a second scrcpy server that fights for the port and kills the first. Also initialize the registry GOP buffer in register() (latent type error). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): boot the AVD headless without a console window The detached spawn opened a Windows console (showing the emulator's verbose qemu/netsim logs) and a redundant native emulator window. Pass windowsHide and run the emulator with -no-window so it boots headless — the scrcpy pane is the view, matching how iOS hides Simulator.app. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): boot the AVD with a hidden console, not detached detached: true sets DETACHED_PROCESS, which gives the console-subsystem emulator no console — so it and its qemu/netsim children pop their own visible cmd window that windowsHide can't suppress. Drop detached and rely on windowsHide (CREATE_NO_WINDOW = hidden console) + unref; spawn already keeps it alive past the launch call, and managed emulators are shut down on app quit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): keep Android emulators alive when switching devices Attaching a different device shut down the active one (shutdownDevice: true), which for Android meant killing the running emulator and cold-booting the target (~60s) on every switch — and switching back. Add bridge.stopActiveForSwitch: Android emulators stay running for instant switch-back, while iOS simulators are still replaced. Switching to an already-running emulator is now immediate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(emulator): only resize the video canvas when dimensions change The decoder output handler set canvas.width/height on every frame, which reallocates the canvas backing store and forces an object-contain reflow each frame — a needless per-frame cost. Resize only when the frame dimensions actually change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): genericize copy + add Android Studio setup link Replace iOS-only wording (Xcode/Simulator/iPhone) in the pane and settings with backend-neutral copy so Android reads correctly on every platform. When no emulator is available, the Mobile Emulator settings now show a "Download Android Studio" link plus setup guidance (ANDROID_HOME / default install path). Removes the now-unused, macOS-only EmulatorUnavailablePane. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): show emulator SDK status in settings The backend availability now reports the resolved Android SDK path, aggregated into emulator.availability as an `android` block. The Mobile Emulator settings render an "Emulator SDKs" card showing Android SDK (detected at <path> / not found, with a Download Android Studio link) and, on macOS, iOS Simulator (Xcode) status — mirroring the agent-control card. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(emulator): locate a custom Android SDK folder from settings Add an androidSdkPath setting and a "Locate SDK folder…" / Clear action in the emulator SDK status card. The path is applied as the highest-priority discovery candidate (falls back if invalid), and the backend's SDK is re-resolved on use via a new AndroidSdkState — so locating or installing the SDK takes effect on Refresh without restarting Orca. Guards the status card against older runtimes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): settle the scrcpy video socket once to stop retry storms A failed TCP connect emits both 'error' and 'close', so retry ran twice and scheduled openVideoSocket(attempt+1) twice — fanning out into an exponential connection storm while waiting for the server to start listening. A runaway chain could then hit attempt 100 and fail/close a stream that had already connected. Replace the delivered flag with a single settled latch so each socket retries (or delivers) exactly once. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): address CodeRabbit review findings - avd-boot: handle spawn 'error' (an unhandled ChildProcess error crashed the main process); validate the target is a known AVD before launching. - capability-ops: propagate adb non-zero exits for launch/permission/logcat and check the uiautomator dump before reading (avoids stale XML). - scrcpy-video-registry: actually replay the buffered GOP on subscribe so late subscribers decode immediately. - android-sdk-state: re-resolve host discovery every call so a changed SDK path takes effect live (no restart). - android-sdk-discovery: require both adb and the emulator binary. - emulator-bridge: fall back to the platform-primary backend (Android off-mac) so setup errors aren't iOS/CoreSimulator on Windows/Linux. - scrcpy-server-download: dedupe concurrent first-use downloads + add a timeout. - scrcpy-stream-session: idle-socket connect timeout; surface control-socket errors instead of swallowing them. - android-exec: pass the whole command so the device shell parses quotes/pipes. - avd-manager: match emulator log prefixes exactly (keep AVD names like PixelWARNINGTest). - permissions: `pm reset-permissions` is global and takes no package argument. - stream controller/starter: drop stale handles for dead streams; idempotent teardown. use-emulator-video-stream: stopVideoStream returns Promise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(emulator): populate the GOP buffer and reuse live scrcpy streams Self-audit follow-ups in the same class as the CodeRabbit GOP-replay finding: - scrcpy-video-registry: pushFrame never wrote to entry.gop, so the replay loop added for late subscribers iterated an empty array — a no-op. Build the GOP on ingest (start at each keyframe, append following deltas; don't buffer deltas before the first keyframe). Adds tests for population, reset, and the pre-keyframe guard. - android backend: isSessionReusable was stubbed to always return false with a "no persistent stream yet" note, but scrcpy streams are persistent now — so every renderer remount tore down and respawned the server. Reuse a live stream (scrcpyVideoRegistry.has) so remounts reconnect, matching iOS. The device-mismatch check still runs first, so device switching is unaffected. - Refresh stale comments that implied unfinished/unverified work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: Changes before auto-review fixes Co-authored-by: Orca <help@stably.ai> * Refine mobile emulator availability settings Co-authored-by: Orca <help@stably.ai> * Address emulator review follow-ups Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Jinwoo-H <jinwoo0825@gmail.com> Co-authored-by: Orca <help@stably.ai>	2026-06-28 21:01:09 -07:00