## What
Adds a **P0 data-layer assertion** to the mobile RUM masking suite:
after driving the replay flow, it queries the **`_sessionreplay`**
stream and requires that segments actually landed. A rendered dashboard
is no longer accepted as proof that Session Replay works.
This is the check that catches **#13942** — on iOS the RN Session-Replay
bridge posts segments to Datadog's `/api/v2/replay` instead of
OpenObserve's `/replay`, so they're rejected (401, no error surfaced)
and **zero** segments reach `_sessionreplay`. RN iOS session replay
silently records nothing.
## RC
The masking suite verified Session Replay **only through the dashboard
UI** (does the player render + is PII masked). It had no data-layer
check that segments actually reached OpenObserve. So a broken upload
showed only as "the replay didn't render" — ambiguous (a benign
dashboard-render flake *or* a real non-upload) — and that non-render had
been scoped to a skip. The bug slipped through the ambiguity.
## The fix
- **P0 (data):** `maskingSuite` asserts `_sessionreplay` received
segments for the session (`> 0`). Non-upload → hard fail.
Platform-agnostic; catches this whole class of bug.
- **P1 (privacy):** the dashboard render + PII scan stays, run only when
the player paints (segments-landed-but-player-didn't = dashboard flake,
not an SDK bug).
- Removed the `requireReplay` skip hack. **`ios-native` masking
unskipped** → runs as hard P0 (native iOS SDK uses the correct `/replay`
endpoint).
- **`rn-ios` masking is a plain test that hard-FAILS until #13942 is
fixed** — **no skip, no expected-fail marker.** The P0 assertion sees 0
segments → red; it **passes on its own** the moment the SDK starts
uploading. The fix lives in `openobserve-react-native-rum`.
## Expected result per platform
| Platform | Masking (P0 upload + PII) | Why |
|---|:-:|---|
| rn-android | ✅ pass | RN replay works on Android |
| **rn-ios** | ❌ **fail** | **#13942** — 0 replay segments |
| android-native | ✅ pass | native SDK, correct endpoint |
| ios-native | ✅ pass | native SDK, correct endpoint |
So whenever iOS runs (the `mobile-e2e` label / an SDK dispatch), the iOS
job is **red until the SDK fix lands** — an intentional, self-healing
signal.
## Validation (local, real device/data runs)
- `android-native` + `ios-native` masking **pass** the new P0 (segments
present in `_sessionreplay`).
- A non-uploading session returns **0 segments → P0 correctly fails** —
the exact rn-ios symptom.
Scope: only `tests/mobile-testing/` — 3 files.
## What this PR adds
end-to-end test suite for the **Mobile RUM SDK** under
`tests/mobile-testing/`, plus a **self-contained CI workflow** that runs
it on real emulators/simulators against an OpenObserve **built from this
PR's source**. Test / fixture / CI only — no product-code/existing test
changes
## How each test works (3 layers)
Every scenario is a real end-to-end journey, so a failure points at the
actual broken layer:
1. **Drive** — a Maestro flow taps through the app like a user
(`maestro/<track>/*.yaml`).
2. **API** — query the `_rumdata` stream and assert the telemetry is
correct (`utils/ooClient.js`, poll-and-retry for async ingestion).
3. **UI** — open the RUM dashboard and assert the sessions
## Coverage (42 tests across 4 platforms)
The **native platforms (android-native, ios-native) are now at parity
with React Native** — each new
native test (masking, network, bg/fg, user-identity, no-phone-home) was
built + validated on a real
emulator/simulator before landing.
| Capability | Layer | RN-Android | Android-native | RN-iOS | iOS-native
|
|---|---|:-:|:-:|:-:|:-:|
| Core telemetry (views/actions/handled-error/crash → `_rumdata`) | API
| ✅ | ✅ | ✅ | ✅ |
| Crash session viewable in dashboard | UI | ✅ | ✅ | ✅ | ✅ |
| Attributes: env / service / version tagging | API | ✅ | ✅ | ✅ | ✅ |
| User identity (`setUser`) | API | ✅ | ✅ | ✅ | ✅ |
| Network: fetch → resource + 4xx/5xx | API | ✅ | ✅ | ✅ | ✅ |
| Session-replay PII masking | UI | ✅ | ✅ | ✅ | ✅ |
| Background / foreground view transitions | API | ✅ | ✅ | — | ✅ |
| No-phone-home (no egress to Datadog hosts) | security | ✅ | ✅ | — | ✅
|
- Shared per-capability factories keep the specs DRY
(`utils/rumChecks.js`, `utils/coreRumSpec.js`).
- Negatives are covered: 4xx capture, PII-must-not-leak (masking), no
phone-home, handled-error ≠ crash.
- `o2-enterprise#2289` dashboard bugs are kept as inline skips that flip
green once fixed.
- Full capability × platform detail + on-device status:
`docs/COVERAGE-MATRIX.md`.
## Fixture apps + build-time config
- Four vendored apps under `apps/`: `o2-rum-tester` (React Native —
drives both RN platforms), `o2-native-android` (Kotlin), `o2-native-ios`
(Swift / xcodegen).
- Each reads its RUM target (endpoint / org / token) and SDK version
from a **build-time-overridable** config. The committed default is the
current pin, so local dev is unchanged; CI regenerates it to point at
the runner's OpenObserve and can pin the latest SDK on release.
## CI — `.github/workflows/mobile-sdk-e2e.yml` (self-contained, no
secrets)
- **Build once per OS from source.** `build_oo_linux` / `build_oo_macos`
compile OpenObserve (binary + embedded web UI) from this PR's code and
cache it. Device jobs download and run it **natively**, and the app
posts real RUM to it (Android reaches the host at `10.0.2.2`, the iOS
simulator via `localhost`). No Docker, no shared cluster, no secrets.
- **Device jobs run in parallel.** `android` (Linux + KVM emulator →
rn-android + android-native) and `ios` (macOS simulator, pinned Xcode
16.4 → rn-ios + ios-native).
- **`merge_reports`** combines both platforms into one browsable HTML
report — even when a device job fails.
- **`decide` picks which platform runs, to keep scarce macOS runners in
check:**
- PR touching Android or shared mobile-RUM code → **Android auto-runs**;
iOS is gated behind the `mobile-e2e` label.
- `mobile-e2e` label → both platforms (full pre-merge gate).
- `repository_dispatch(mobile-sdk-release)` from an SDK repo → runs that
SDK's platform (companion PRs wire this up:
openobserve-react-native-rum#3, openobserve-sdk-android#1,
openobserve-sdk-ios#1).
- `workflow_dispatch` → manual, choose platforms.
*Separate workflow :* it needs macOS **and** Android emulators, builds 4
phone apps, and is dispatch-driven — matching the repo's existing
per-suite split
## Status
Green end-to-end on GitHub runners (Android + iOS device jobs + merge) :
https://github.com/openobserve/openobserve/actions/runs/31733717758/job/94565520891
---------
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
dependabot[bot]
Vamsi Karthik R
