mirror of
https://github.com/lexmount/moli.git
synced 2026-10-02 16:00:44 +00:00
test(fingerprint): add native Windows identity baseline collector
This commit is contained in:
@@ -0,0 +1,67 @@
|
||||
# Fingerprint Pro:Windows 身份对照的进展与阻塞
|
||||
|
||||
## 本次结论
|
||||
|
||||
本次没有修改产品默认身份,也没有将 Linux UA override 当成 Windows
|
||||
浏览器。**真正的 Windows Chrome 对照仍未完成**:当前会话只有 Linux
|
||||
主机,尚未获得可用的 Windows 机器或已授权端点。没有据此猜测、修改
|
||||
Navigator、字体或商业站点的结果。
|
||||
|
||||
已新增独立采集入口
|
||||
`moli_cdp_smoke.diagnostics.fingerprint_identity`,Windows 上的运行方法见
|
||||
`moli-cdp-smoke/DIAGNOSTICS.md`。`--require-windows` 会拒绝 Linux/macOS;
|
||||
输出保留宿主 OS、浏览器版本、可执行文件 SHA-256、原生 Navigator/UA-CH
|
||||
和服务端结果的白名单字段。没有伪装 OS 或读取用户现有 Chrome profile。
|
||||
|
||||
## 2026-09-12 的同机补充采样
|
||||
|
||||
二进制由 `2264ea6db` 加本次 InputEvent 改动构建,Moli release SHA-256:
|
||||
`e4f3d498a836436f0325bb42779688a805c23726456be8e9c695ea62bd2ba382`。
|
||||
对照为 `/usr/bin/chromium` 145.0.7632.116,xvfb headed,非 headless。
|
||||
两端均新进程、新 context,CDP Runtime/Page/Network 开启,无 UA override,
|
||||
无代理,保留 TLS 校验。DOMContentLoaded 后观察 12 秒,然后才读取 identity。
|
||||
|
||||
| 项目 | Moli 默认 Windows identity,Linux 后端 | Chromium 原生 Linux |
|
||||
| --- | --- | --- |
|
||||
| 有效结果 | HTTP 200,约 4.70 秒收到 | HTTP 200,约 3.00 秒收到 |
|
||||
| bot | not_detected | not_detected |
|
||||
| tampering / suspect_score | true / 22 | false / 12 |
|
||||
| tampering_ml_score | 0.9998 | 0 |
|
||||
| 七项 font_preferences | 149.3125、149.3125、144.015625、133.0625、149.3125、9.34375、162 | 全部相同 |
|
||||
| touch_event / touch_start / max_touch_points | false / false / 0 | 相同 |
|
||||
| fonts | 空列表 | 空列表 |
|
||||
|
||||
字段顺序为 default、serif、sans、mono、apple、min、system。这里的 fonts
|
||||
是本次 Fingerprint Pro 的输出,不是之前 CreepJS 的字体数量。
|
||||
|
||||
仍存在的观察差别包括平台、语言、screen、hardwareConcurrency、audio 和
|
||||
math 的结果;这些不是已经证实的触发原因。尤其两端的 V8 版本也不同,不能
|
||||
将任何一个 hash 的差别直接解释成 Windows 身份矛盾。其他会话当时仍有
|
||||
Cargo 构建,环境文件保留 load average;本轮不是隔离负载的 timing 实验。
|
||||
|
||||
历史 OS-only 对照见 `cdp-fingerprint-identity-status-2026-09-10.md`:
|
||||
改变 OS identity 这一组能够改变判定,但并没有定位商业模型的单一规则。
|
||||
本轮字体偏好与触摸字段已经一致而 verdict 仍不同,再次说明不能继续把
|
||||
“调整这七项宽度”当作已有证据支持的修复。
|
||||
|
||||
## 下一步需要的证据
|
||||
|
||||
1. 在真正 Windows Chrome 上运行采集器,记录 Windows/Chrome 版本和字体环境。
|
||||
2. 比较原生 UA-CH 与站点实际收到的 raw attributes,先列可复现差异。
|
||||
3. 对具体 Web API 用最小离线探针验证;只修确定的兼容性问题。
|
||||
4. 保持 Moli 的 Windows 默认身份,不覆盖站点检测脚本、请求或结果。
|
||||
|
||||
Windows 启动路径尚未做真实执行验证。Linux 下通过的 guard、字段筛选单测
|
||||
不能替代这一步,也不能声称本任务已取得 Windows 基线。
|
||||
|
||||
## 采样与工具验证记录
|
||||
|
||||
- `docs/inner/fp-native-moli-20260912-direct/`:本次 Moli 有效结果。
|
||||
- `docs/inner/fp-native-chromium-20260912/`:本次 Chromium 有效结果。
|
||||
- `docs/inner/fp-native-moli-20260912-input-event/`:首次工具启动受驱动 HTTP
|
||||
discovery 的代理环境影响,连接失败,**尚未导航到站点**;保留为工具失败,
|
||||
不计为网站样本。修正为使用已直连发现的 WebSocket URL 后才执行上述采样。
|
||||
- 采集器白名单、API URL 筛选、真实 Windows guard 的三个专项单元测试通过。
|
||||
|
||||
原始目录仅留本地。提交的文档不包含 visitor ID、IP、cookies、SDK token
|
||||
或未筛选的站点响应。
|
||||
@@ -0,0 +1,51 @@
|
||||
# Optional live-site diagnostics
|
||||
|
||||
These collectors are **not** CI smoke tests or assertions that a commercial
|
||||
detector must accept automation. They keep failures and missing verdicts as
|
||||
evidence instead of retrying until a site returns a desired result.
|
||||
|
||||
Run from `moli-cdp-smoke` with its Python dependencies installed (`uv sync`).
|
||||
Use a fresh output directory for every invocation. Each collector starts its
|
||||
own browser, uses a disposable context, enables CDP Runtime/Page/Network,
|
||||
leaves identity and site scripts untouched, and terminates its browser.
|
||||
Browser traffic is direct, with TLS verification left enabled. On Linux use
|
||||
`xvfb-run -a` for **headed** Chromium, not `--headless`.
|
||||
|
||||
## Fingerprint Pro: a real Windows baseline
|
||||
|
||||
Run this on Windows, with a native Chrome installation:
|
||||
|
||||
```powershell
|
||||
uv run python -m moli_cdp_smoke.diagnostics.fingerprint_identity `
|
||||
--engine chromium `
|
||||
--binary 'C:\Program Files\Google\Chrome\Application\chrome.exe' `
|
||||
--require-windows --output windows-fp-baseline
|
||||
```
|
||||
|
||||
`--require-windows` refuses Linux/macOS hosts. Changing a UA on Linux is not
|
||||
a Windows baseline. The collector records host OS, binary SHA-256, browser
|
||||
version and native Navigator/UA-CH values; it does not set UA overrides.
|
||||
Record the Windows version and installed font environment when comparing
|
||||
results. Browser-version and machine differences are potential confounders,
|
||||
not evidence that a single identity field caused a commercial verdict.
|
||||
|
||||
For Moli, keep the same command but use `--engine moli`, its local binary,
|
||||
and omit `--require-windows` when running on Linux. Moli retains its own
|
||||
default identity; the collector never replaces it with the host identity.
|
||||
|
||||
`result.json` selects the final event API's bot/tampering/score fields and
|
||||
an explicit allowlist of device attributes. No API reply within the fixed
|
||||
12-second post-navigation window means **no valid verdict**. Identity reads
|
||||
happen after that window. Responses are not intercepted or rewritten.
|
||||
|
||||
## Evidence handling
|
||||
|
||||
Keep `environment.json` and `result.json` with the investigation. Do not
|
||||
commit unreviewed raw artifacts: `server.log` can contain URLs and third-party
|
||||
script diagnostics. Even selected fingerprints describe a device and should
|
||||
not be published indiscriminately. The collectors omit visitor IDs, cookies,
|
||||
IP addresses and opaque SDK tokens from their structured reports.
|
||||
|
||||
The native Windows guard, result URL selection and structured-report
|
||||
allowlists have local unit tests. Actual Windows launch and site results must
|
||||
still be verified on Windows; a passing Linux unit test is not that evidence.
|
||||
@@ -0,0 +1 @@
|
||||
"""Opt-in live compatibility investigations; never part of the default smoke suite."""
|
||||
@@ -0,0 +1,118 @@
|
||||
"""Owned, disposable native-browser CDP sessions for live investigations."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime, timezone
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import socket
|
||||
import tempfile
|
||||
import urllib.request
|
||||
|
||||
from playwright.async_api import async_playwright
|
||||
|
||||
from ..process import subprocess_starts_new_session, terminate_process_tree
|
||||
|
||||
|
||||
def save(path: Path, value: object) -> None:
|
||||
path.write_text(json.dumps(value, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def require_native_windows(required: bool) -> None:
|
||||
if required and platform.system() != "Windows":
|
||||
raise ValueError("Windows baseline requires this collector and browser to run on Windows; a UA override is not a Windows baseline")
|
||||
|
||||
|
||||
def discovery(endpoint: str) -> dict:
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
with opener.open(endpoint + "/json/version", timeout=2) as response:
|
||||
return json.load(response)
|
||||
|
||||
|
||||
async def stop_browser(process) -> bool:
|
||||
if os.name == "posix":
|
||||
return await terminate_process_tree(process)
|
||||
# Windows has no SIGKILL/process group API. Browser.close above normally
|
||||
# closes the full browser; the owned root process still has a bounded exit.
|
||||
if process.returncode is not None:
|
||||
return True
|
||||
process.terminate()
|
||||
try:
|
||||
await asyncio.wait_for(process.wait(), 5)
|
||||
except asyncio.TimeoutError:
|
||||
process.kill()
|
||||
await asyncio.wait_for(process.wait(), 2)
|
||||
return True
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def browser_session(binary: Path, engine: str, output: Path, require_windows: bool = False):
|
||||
require_native_windows(require_windows)
|
||||
binary = binary.resolve(strict=True)
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
with socket.socket() as sock:
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
port = sock.getsockname()[1]
|
||||
endpoint = f"http://127.0.0.1:{port}"
|
||||
environment = {k: v for k, v in os.environ.items()
|
||||
if k.lower() not in {"http_proxy", "https_proxy", "all_proxy", "no_proxy"}}
|
||||
with tempfile.TemporaryDirectory(prefix="moli-native-baseline-") as profile:
|
||||
command = ([str(binary), "serve", "-lr", "--port", str(port)] if engine == "moli" else
|
||||
[str(binary), f"--remote-debugging-port={port}", "--remote-debugging-address=127.0.0.1",
|
||||
f"--user-data-dir={profile}", "--no-first-run", "--no-default-browser-check",
|
||||
"--no-proxy-server", "--password-store=basic", "--window-size=1920,1080", "about:blank"])
|
||||
with binary.open("rb") as source:
|
||||
digest = hashlib.file_digest(source, "sha256").hexdigest()
|
||||
metadata = {"started_at": datetime.now(timezone.utc).isoformat(), "engine": engine,
|
||||
"host_os": platform.system(), "host_release": platform.release(),
|
||||
"machine": platform.machine(), "native_windows_required": require_windows,
|
||||
"binary": str(binary), "binary_sha256": digest,
|
||||
"identity_override": False, "tls_verification": True, "proxy": "disabled",
|
||||
"mode": "CDP; Runtime/Page/Network enabled; fresh process and context; no viewport emulation"}
|
||||
if hasattr(os, "getloadavg"):
|
||||
metadata["load_average_before"] = os.getloadavg()
|
||||
with (output / "server.log").open("w", encoding="utf-8") as log:
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*command, env=environment, stdout=log, stderr=asyncio.subprocess.STDOUT,
|
||||
start_new_session=subprocess_starts_new_session())
|
||||
try:
|
||||
for _ in range(60):
|
||||
if process.returncode is not None:
|
||||
raise RuntimeError(f"Browser exited before discovery: {process.returncode}")
|
||||
try:
|
||||
version = await asyncio.to_thread(discovery, endpoint)
|
||||
break
|
||||
except (OSError, ValueError):
|
||||
await asyncio.sleep(0.2)
|
||||
else:
|
||||
raise TimeoutError("CDP discovery did not become ready")
|
||||
metadata["browser"] = {k: version.get(k) for k in ["Browser", "V8-Version", "User-Agent"]}
|
||||
save(output / "environment.json", metadata)
|
||||
async with async_playwright() as pw:
|
||||
# Discovery already bypassed proxies; use its WebSocket
|
||||
# directly instead of having the driver repeat the HTTP
|
||||
# request with the caller's proxy environment.
|
||||
browser = await pw.chromium.connect_over_cdp(version["webSocketDebuggerUrl"], timeout=15000)
|
||||
try:
|
||||
context = await asyncio.wait_for(browser.new_context(no_viewport=True), 15)
|
||||
page = await asyncio.wait_for(context.new_page(), 15)
|
||||
cdp = await asyncio.wait_for(context.new_cdp_session(page), 10)
|
||||
for domain in ["Runtime", "Page", "Network"]:
|
||||
await asyncio.wait_for(cdp.send(domain + ".enable"), 10)
|
||||
yield page, cdp
|
||||
finally:
|
||||
try:
|
||||
await asyncio.wait_for(browser.close(), 10)
|
||||
except Exception as error:
|
||||
metadata["close_error_type"] = type(error).__name__
|
||||
finally:
|
||||
metadata["process_cleaned_up"] = await stop_browser(process)
|
||||
metadata["finished_at"] = datetime.now(timezone.utc).isoformat()
|
||||
metadata["process_exit"] = process.returncode
|
||||
if hasattr(os, "getloadavg"):
|
||||
metadata["load_average_after"] = os.getloadavg()
|
||||
save(output / "environment.json", metadata)
|
||||
@@ -0,0 +1,104 @@
|
||||
"""Compare native FP Pro observations without changing identity or site probes."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
from pathlib import Path
|
||||
import time
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from .browser import browser_session, save
|
||||
|
||||
URL = "https://demo.fingerprint.com/playground"
|
||||
ATTRIBUTE_KEYS = ("fonts", "font_preferences", "touch_support", "platform", "vendor", "languages",
|
||||
"screen_resolution", "color_depth", "hardware_concurrency", "device_memory",
|
||||
"timezone", "os_cpu", "architecture", "audio", "math", "webgl_basics")
|
||||
IDENTITY = """async () => ({
|
||||
userAgent: navigator.userAgent, platform: navigator.platform, languages: navigator.languages,
|
||||
webdriver: navigator.webdriver, hardwareConcurrency: navigator.hardwareConcurrency,
|
||||
deviceMemory: navigator.deviceMemory,
|
||||
metadata: navigator.userAgentData ? await navigator.userAgentData.getHighEntropyValues([
|
||||
'platformVersion','architecture','bitness','fullVersionList','wow64']) : null
|
||||
})"""
|
||||
|
||||
|
||||
def select_result(value: dict) -> dict:
|
||||
# No visitor/request IDs, IPs, request bodies, cookies or opaque SDK tokens.
|
||||
result = {key: value[key] for key in ["bot", "tampering", "suspect_score", "tampering_ml_score"]
|
||||
if key in value}
|
||||
raw = value.get("raw_device_attributes", {})
|
||||
result["attributes"] = {key: raw[key] for key in ATTRIBUTE_KEYS if key in raw}
|
||||
return result
|
||||
|
||||
|
||||
def is_result_url(url: str) -> bool:
|
||||
parsed = urlsplit(url)
|
||||
return parsed.hostname == "demo.fingerprint.com" and parsed.path.startswith("/api/event/")
|
||||
|
||||
|
||||
async def capture(args) -> dict:
|
||||
result = {"url": URL, "wait_seconds": args.wait_seconds, "responses": [],
|
||||
"errors": [], "valid_verdict": False}
|
||||
try:
|
||||
async with browser_session(args.binary, args.engine, args.output, args.require_windows) as (page, cdp):
|
||||
started = time.monotonic()
|
||||
responses = []
|
||||
|
||||
def received(params):
|
||||
response = params.get("response", {})
|
||||
if is_result_url(response.get("url", "")):
|
||||
responses.append({"requestId": params["requestId"], "status": response["status"],
|
||||
"elapsed": time.monotonic() - started})
|
||||
|
||||
cdp.on("Network.responseReceived", received)
|
||||
try:
|
||||
await page.goto(URL, wait_until="domcontentloaded", timeout=45000)
|
||||
except Exception as error:
|
||||
result["errors"].append({"phase": "navigation", "error_type": type(error).__name__})
|
||||
await asyncio.sleep(args.wait_seconds)
|
||||
cutoff = time.monotonic() - started
|
||||
result["observation_cutoff"] = cutoff
|
||||
for response in list(responses):
|
||||
if response["elapsed"] > cutoff:
|
||||
continue
|
||||
observation = {"status": response["status"], "elapsed": response["elapsed"]}
|
||||
try:
|
||||
body = await asyncio.wait_for(cdp.send("Network.getResponseBody", {"requestId": response["requestId"]}), 5)
|
||||
if body.get("base64Encoded"):
|
||||
value = json.loads(base64.b64decode(body["body"]))
|
||||
else:
|
||||
value = json.loads(body["body"])
|
||||
observation["result"] = select_result(value)
|
||||
except Exception as error:
|
||||
observation["error_type"] = type(error).__name__
|
||||
result["responses"].append(observation)
|
||||
# This post-verdict read does not override or instrument any probe.
|
||||
result["identity"] = await asyncio.wait_for(page.evaluate(IDENTITY), 10)
|
||||
result["valid_verdict"] = any(isinstance(r.get("result", {}).get("tampering"), bool)
|
||||
for r in result["responses"])
|
||||
finally:
|
||||
if args.output.exists():
|
||||
save(args.output / "result.json", result)
|
||||
return result
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--binary", type=Path, required=True)
|
||||
parser.add_argument("--engine", choices=["chromium", "moli"], required=True)
|
||||
parser.add_argument("--output", type=Path, required=True)
|
||||
parser.add_argument("--require-windows", action="store_true", help="Reject a Linux/macOS runner even if its UA claims Windows")
|
||||
parser.add_argument("--wait-seconds", type=int, default=12)
|
||||
args = parser.parse_args()
|
||||
if args.wait_seconds < 1 or args.wait_seconds > 120:
|
||||
parser.error("wait-seconds must be between 1 and 120")
|
||||
if args.output.exists():
|
||||
parser.error("output must be a new directory")
|
||||
result = asyncio.run(capture(args))
|
||||
print(json.dumps({"valid_verdict": result["valid_verdict"], "responses": result["responses"]}))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,37 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from moli_cdp_smoke.diagnostics.browser import require_native_windows
|
||||
from moli_cdp_smoke.diagnostics.fingerprint_identity import is_result_url, select_result
|
||||
|
||||
|
||||
class FingerprintIdentityDiagnosticTests(unittest.TestCase):
|
||||
def test_windows_baseline_rejects_non_windows_hosts(self):
|
||||
for system in ["Linux", "Darwin"]:
|
||||
with self.subTest(system=system), patch("platform.system", return_value=system):
|
||||
with self.assertRaisesRegex(ValueError, "not a Windows baseline"):
|
||||
require_native_windows(True)
|
||||
require_native_windows(False)
|
||||
with patch("platform.system", return_value="Windows"):
|
||||
require_native_windows(True)
|
||||
|
||||
def test_result_url_requires_the_demo_host_and_event_api(self):
|
||||
self.assertTrue(is_result_url("https://demo.fingerprint.com/api/event/opaque"))
|
||||
self.assertFalse(is_result_url("https://demo.fingerprint.com/playground"))
|
||||
self.assertFalse(is_result_url("https://example.com/api/event/opaque"))
|
||||
|
||||
def test_result_snapshot_omits_identifiers_and_unselected_raw_attributes(self):
|
||||
value = {"tampering": True, "suspect_score": 22, "visitor_id": "secret",
|
||||
"request_id": "secret", "ip_address": "secret",
|
||||
"raw_device_attributes": {"fonts": {"value": ["Arial"]},
|
||||
"cookies": "secret", "ip": "secret"}}
|
||||
self.assertEqual(select_result(value), {
|
||||
"tampering": True, "suspect_score": 22,
|
||||
"attributes": {"fonts": {"value": ["Arial"]}},
|
||||
})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user