openobserve/tests/mobile-testing/README.md

Mobile RUM SDK — End-to-End Test Suite

E2E tests for OpenObserve's Mobile RUM SDK (@openobserve/mobile-react-native and native tracks). Each test drives a real mobile app on an emulator/simulator, then asserts the telemetry both at the data layer (OpenObserve Search API) and at the product layer (the RUM dashboard UI) — because most mobile-RUM defects live in the dashboard, not the pipeline.

The 3-layer pattern

Every scenario is the same shape, threaded by session_id:

1. DRIVE   Maestro flow taps through the app        (maestro/<track>/*.yaml)
2. API     query _rumdata → assert data is correct  (utils/ooClient.js)
3. UI      open the dashboard → assert it renders    (pages/rumDashboardPage.js)

Why both layers? Nearly every bug found in manual testing (Error Tracking, Web Vitals, breadcrumbs, Traces) was a display bug: the data was correct in _rumdata but the dashboard failed to show it. An API-only suite would have been green on a broken product.

Layout

mobile-testing/
├── apps/                     Vendored fixture apps (o2-rum-tester, o2-native-android, o2-native-ios)
├── maestro/                  Maestro flows per track (react-native, android-native, ios-native, ios-react-native)
├── specs/                    Playwright specs (rn-android.*, rn-ios.*, android-native.*, ios-native.*)
├── pages/rumDashboardPage.js Dashboard page object (login, open session, masking guard)
├── utils/
│   ├── ooClient.js           OpenObserve Search API client (poll-and-retry)
│   ├── rumChecks.js          per-capability test factories (attributes/user/network/masking/…)
│   ├── coreRumSpec.js        the crash → dashboard core journey
│   ├── maestro.js            Maestro flow runner
│   ├── adb.js                device-level Android steps
│   └── config.js             env loader
├── docs/                     Coverage matrix, CI notes, test-case matrix
├── playwright.config.js
└── .env / .env.example       instance URL, org, creds, app id

Prerequisites

• Node 20+, JDK 17, Android SDK + an emulator (AVD)
• Maestro CLI: curl -Ls "https://get.maestro.mobile.dev" | bash
• The app under test (com.o2rumtester) — its source lives in this suite at apps/o2-rum-tester/. Build + install it:

  cd apps/o2-rum-tester && npm install    # postinstall auto-applies the SDK build fix
  (cd android && ./gradlew assembleRelease)
  adb install -r android/app/build/outputs/apk/release/app-release.apk
  

• A reachable OpenObserve instance with a RUM token wired into the app (see .env and apps/o2-rum-tester/App.tsx)

Setup

cp .env.example .env     # fill in OO_URL / OO_ORG / OO_USER / OO_PASS  (quote passwords with #)
npm install
npx playwright install chromium

Run

# one platform (Playwright projects)
npx playwright test --project=rn-android
npx playwright test --project=android-native
npx playwright test --project=rn-ios          # needs a booted iOS simulator
npx playwright test --project=ios-native      # needs a booted iOS simulator

# by tag
npx playwright test --grep @crash
npx playwright test --grep @masking
npx playwright test --grep @known-bug         # the skipped #2289 markers

Coverage (all four mobile tracks) — 42 tests

Each platform has a core RUM bundle (utils/coreRumSpec.js: views + actions + handled error + native crash in _rumdata, then the crashed session rendering in the dashboard) plus per-capability specs built from shared factories in utils/rumChecks.js — so the platforms stay at parity without copy-paste:

• attributesSuite — env / service / version tagging
• userIdentitySuite — events attributed to the set user (usr_* fields)
• networkSuite — a successful fetch is a resource (positive) + a 4xx status is captured (negative)
• maskingSuite — no PII leaks into the session replay (MASK_ALL)
• bgFgSuite — a pre-background view and a post-foreground view share one session
• noPhoneHomeAndroidSuite — the installed APK contains zero Datadog hosts (security/negative)

Project	App	Service	Specs
rn-android	apps/o2-rum-tester (Android)	o2-rum-tester	crash, network(+404), handled-error, views, masking, attributes, user-identity, background-foreground, no-phone-home
android-native	apps/o2-native-android	o2-native-android	core bundle, attributes, user-identity, no-phone-home
rn-ios	apps/o2-rum-tester (iOS)	o2-rum-tester	core bundle, network(+404), attributes, user-identity, masking
ios-native	apps/o2-native-ios	o2-native-ios	core bundle, attributes

See docs/COVERAGE-MATRIX.md for the full capability×platform matrix, on-device validation status, and the deferred follow-ups (native/iOS background-foreground, native network/masking, iOS-native user identity — each needs a fixture-app change or a background harness).

Prereqs: Android emulator booted for the Android projects; an iOS simulator booted (udid in .env IOS_SIM_UDID) for the iOS projects; the fixture apps built + installed (see apps/).

Tags: @mobile @rn-android @rn-ios @android-native @ios-native @P0 @P1 @crash @network @negative @errors @views @replay @masking @lifecycle @attributes @user-identity @security @known-bug.

Notes

• The known Error-Tracking outage (o2-enterprise#2289) is a test.fixme skip inline in the relevant spec — no separate file or project. It stays skipped and turns into a real assertion when the bug is fixed.
• Ingestion is async — assertions poll-and-retry; never assert instantly.
• Session isolation — the current specs scope by service + time window. For parallel/CI runs, stamp a unique run-id as a RUM global attribute in the app and filter on it (see docs/CI-NOTES.md).
• Fixture apps — the four apps live in apps/ (each builds against the SDK from npm/Maven/SPM); iOS tracks additionally need Xcode/simulator. Build/run steps are in each app + docs/CI-NOTES.md.