starred/moli
1
0
Fork 0
mirror of https://github.com/lexmount/moli.git synced 2026-09-22 16:01:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
counters-parser
moli/moli-webdriver-smoke
T
History

README.md

moli-webdriver-smoke

This is a standalone WebDriver smoke project for moli. It is a peer of moli-cdp-smoke/: the runner starts a local fixture server, starts moli serve, and exercises the public WebDriver wire surface from outside the Rust test harness.

The default suite runs every Moli group: raw WebDriver Classic HTTP, raw BiDi WebSocket messages, Selenium Python's webdriver.Remote client, cross-engine semantics and navigation contracts, Moli's URL policy, and renderer-IO script interruption. This covers both protocol-level wire shape and real Selenium client workflows without requiring a group allowlist.

The semantics group contains isolated cross-engine contracts. It can run against Moli's native WebDriver endpoint or an external ChromeDriver; Moli uses standard W3C capabilities without Chrome vendor options.

The url-policy group is a Moli hosted-product contract. It uses isolated Classic and BiDi sessions to verify the exact local-file navigation error envelopes, unchanged browsing contexts, and absence of BiDi navigation lifecycle events. It is deliberately skipped for desktop Chromium, whose browser-granted local-file capability is a different product policy.

The navigation-errors group is a cross-engine Chromium/WPT contract. It runs the same 37-case public-wire matrix against Moli or ChromeDriver: ten Classic malformed-body/URL cases, 22 BiDi invalid-argument cases, two missing-context cases, and three real address failures. It also checks the W3C HTTP and BiDi error envelopes, unchanged Classic URL, and post-failure session liveness. The baseline was executed on 2026-08-09 with Debian Chromium 145.0.7632.116 and matching ChromeDriver 145.0.7632.117.

The script-timeout boundary has two deliberately separate groups:

  • script-timeout-chromium is the ChromeDriver 147 oracle. A yielding async script times out near its deadline, while a bounded non-yielding loop first returns to the renderer and only then produces script timeout.
  • script-interrupt is a default Moli group for the renderer-IO extension. It runs actual infinite loops through both Classic sync and async execution, requires timeout-driven Runtime.terminateExecution near the configured deadline, and repeats recovery work in the same window after every termination.

The split is intentional: the Moli group proves preemption, while the ChromeDriver group records that ChromeDriver itself does not offer the same non-yielding WebDriver timeout behavior.

Coverage

  • WebDriver Classic status/session/delete value envelopes.
  • Classic navigation, current URL, title, page source, CSS/XPath element lookup and identity, element text/tag/displayed/enabled/rect/attribute/property/computed label/computed role, send keys, clear form controls with input/change events, file upload, click, execute script, key actions whose handler performs a top-level Page replacement while the action request is completing, explicit screenshot unsupported errors, cookies, alerts, unhandled prompt behavior, window-scoped prompt switching, shadow roots, page-side SharedWorker probe without polluting window handles, document.open() replacement stale-element behavior, and headless window state surface.
  • WebDriver BiDi session.status, session.new, browsingContext.create, session.subscribe, browsingContext.navigate, DOMContentLoaded lifecycle events, input.performActions, input.releaseActions, key actions that trigger a top-level Page replacement and remain successful through the replacement lifecycle, element-origin input action routing without asserting layout hit-test effects, input.setFiles, network.getData, browser.setDownloadBehavior, browsingContext.downloadWillBegin/downloadEnd, network.setCacheBehavior, SharedWorker browsingContext.contextCreated, live script.realmCreated(type = "shared-worker"), and script.getRealms(type = "shared-worker"), user context creation/removal, user-context viewport inheritance, explicit browsingContext.captureScreenshot unsupported errors, emulation.setUserAgentOverride, emulation.setLocaleOverride, emulation.setTimezoneOverride, emulation.setNetworkConditions, storage isolation, storage.setCookie, storage.getCookies, and session.end.
  • Selenium Python Remote WebDriver session creation, Selenium BiDi facade webSocketUrl attachment with browsing_context create/navigate/get_tree/ locate_nodes, browsing context fragment/history event handlers, script evaluate/console/javascript-error handlers, add/remove preload scripts with context filtering, channel arguments, sandbox realms, and userContext filtering, getRealms, root-owned script handles, callFunction, disown, special number/BigInt serialization, nested handle/local-value arguments, network.add_request_handler() request interception, network.add_auth_handler() Basic Auth continuation, browser user context/client-window, and emulation user-context UA/locale/timezone/viewport/network-conditions workflows, navigation, title/current URL, page source, element text/tag/attribute/property/css/displayed/enabled/ rect, link text locators, CSS/XPath user-facing locator workflows, accessible name, computed ARIA role, label-associated controls including link/img/select/textarea/input-button naming, send keys, clear, execute sync/async scripts with primitive/null/array/WebElement results, WebElement arguments, timeout/error recovery, and nested WebElement return values, Selenium client-side script pin/list/unpin workflows, Selenium text handling for child/block/inline content, whitespace, NBSP, hidden/script text exclusion, Selenium relative locators via locate_with(), fetch/XHR through async scripts, localStorage, sessionStorage, IndexedDB, Selenium remote downloads via /se/files, frame switching by index/name/id/WebElement with nested parent/default-content recovery and relative lookup errors, deleted-frame recovery, Selenium Select single/multi-select option lookup, select/deselect, disabled/hidden option errors, form submit via Selenium WebElement.submit() and submit-button click, checkbox/radio selection, text input/textarea typing and clear input/change events, all-printable key input, text-control change commit on blur, WebDriver private-key caret navigation/PageUp/PageDown, arrow-key legacy KeyboardEvent.keyCode, selection/delete/backspace, numpad, and function-key send keys, direct stale element reference errors after navigation/DOM removal/document replacement, implicit waits for single and multiple element lookup, WebDriverWait expected conditions for presence/visibility/text/clickability/invisibility/staleness/ selection/value/attribute/frame/alert, W3C keyboard actions, and explicit unsupported-operation errors for pointer/wheel actions including hover/double-click/context-click/drag-drop, Selenium remote file upload via /se/file with FileReader content reads, action key input with Shift modifiers, explicit screenshot/element screenshot and print_page unsupported errors, cookie management including SameSite, expired cookies, delete-all, and exact-name deletion, window handles/new/switch/close/rect/ maximize/minimize, anchor/script popup windows, named window target reuse, window.open(..., "_self"), Selenium ShadowRoot lookup/scoped find/ execute_script equality/no-such-shadow-root errors, alert/confirm/prompt text, and alert accept/dismiss/send keys.
  • Cross-engine Selenium semantics for capability matching, same-document realm and WebElement identity, cross-document stale elements, top-level and popup storage ownership, nested frame recovery, shadow-frame Window named access, dialog return values, and standard W3C error classes. Each scenario uses an independent session and emits a structured contract record.
  • Hosted local-file navigation rejection as Classic HTTP 500 with W3C unknown error, and as a BiDi unknown error; both retain about:blank, and the BiDi path emits no immediate or delayed navigation lifecycle event.
  • Chromium/WPT navigation failure shapes: Classic HTTP 400 invalid argument envelopes for null/missing/non-string/relative/malformed URLs; BiDi invalid argument for context, URL, and wait validation; no such frame for missing context ids; unknown error for unsupported schemes, DNS failure, and unsafe ports; and a usable session after the complete failure matrix.

Known remaining worker-surface gap: the BiDi SharedWorker smoke now requires target context creation, live realm creation, and script.getRealms; richer lifecycle, error, and teardown coverage is still pending.

Usage

Build moli first when no binary exists:

cargo build -p moli

Then run the default smoke:

cd moli-webdriver-smoke
uv sync
uv run moli-webdriver-smoke

With no --group, both local and CI runs execute all seven Moli groups:

classic,bidi,selenium,semantics,url-policy,navigation-errors,script-interrupt

CI adds --continue-on-failure so one failing group does not hide later smoke results. The ChromeDriver-only script-timeout-chromium oracle remains an explicit external-target run.

List available groups:

uv run moli-webdriver-smoke --list-groups

Run focused groups:

uv run moli-webdriver-smoke --group classic
uv run moli-webdriver-smoke --group bidi
uv run moli-webdriver-smoke --group selenium
uv run moli-webdriver-smoke --group semantics --continue-on-failure
uv run moli-webdriver-smoke --group url-policy --continue-on-failure
uv run moli-webdriver-smoke --group navigation-errors --continue-on-failure
uv run moli-webdriver-smoke --group script-interrupt
MOLI_WEBDRIVER_SMOKE_GROUPS=classic,bidi,selenium uv run moli-webdriver-smoke

Collect all subgroup failures in one diagnostic run:

uv run moli-webdriver-smoke \
  --continue-on-failure

Run the same semantics group against a matching local Chrome/ChromeDriver build:

~/chromium/src/out/Default/chromedriver --port=9515

uv run moli-webdriver-smoke \
  --endpoint http://127.0.0.1:9515 \
  --browser-name chrome \
  --browser-binary ~/chromium/src/out/Default/chrome \
  --group semantics --continue-on-failure

Run the bounded ChromeDriver script-timeout oracle against the local Chromium 147 pair:

~/chromium/src/out/Default/chromedriver --port=9515

uv run moli-webdriver-smoke \
  --endpoint http://127.0.0.1:9515 \
  --browser-name chrome \
  --browser-binary ~/chromium/src/out/Default/chrome \
  --group script-timeout-chromium

Run Moli's non-yielding sync/async termination contract against moli serve:

uv run moli-webdriver-smoke --group script-interrupt

Run the Chromium/WPT navigation failure matrix against any matching local Chrome/ChromeDriver pair (the driver and browser major versions must agree):

/path/to/chromedriver-145.0.7632.117 --port=9516

uv run moli-webdriver-smoke \
  --endpoint http://127.0.0.1:9516 \
  --browser-name chrome \
  --browser-binary /usr/bin/chromium \
  --group navigation-errors

Useful environment variables:

  • MOLI_BIN: path to the moli binary under test.
  • MOLI_WEBDRIVER_PORT: WebDriver server port. Defaults to a free local port.
  • MOLI_WEBDRIVER_SMOKE_GROUPS: comma-separated smoke group list. CLI --group takes precedence.
  • MOLI_WEBDRIVER_SMOKE_CONTINUE_ON_FAILURE=1: equivalent to --continue-on-failure.
  • MOLI_WEBDRIVER_SMOKE_TRACE_BG=1: print background moli serve logs.

If both debug and release binaries exist, the runner uses the newest one by file modification time. Set MOLI_BIN when you need an exact binary.

Reference in New Issue View Git Blame Copy Permalink