Files
openobserve/tests/mobile-testing/specs/sdk-version-drift.spec.js
Vamsi Karthik RandShrinath Rao 6930a9b724 test(mobile): RUM SDK end-to-end suite across 4 platforms + self-contained CI (#13628)
## What this PR adds

end-to-end test suite for the **Mobile RUM SDK** under
`tests/mobile-testing/`, plus a **self-contained CI workflow** that runs
it on real emulators/simulators against an OpenObserve **built from this
PR's source**. Test / fixture / CI only — no product-code/existing test
changes

## How each test works (3 layers)

Every scenario is a real end-to-end journey, so a failure points at the
actual broken layer:

1. **Drive** — a Maestro flow taps through the app like a user
(`maestro/<track>/*.yaml`).
2. **API** — query the `_rumdata` stream and assert the telemetry is
correct (`utils/ooClient.js`, poll-and-retry for async ingestion).
3. **UI** — open the RUM dashboard and assert the sessions

## Coverage (42 tests across 4 platforms)

The **native platforms (android-native, ios-native) are now at parity
with React Native** — each new
native test (masking, network, bg/fg, user-identity, no-phone-home) was
built + validated on a real
emulator/simulator before landing.

| Capability | Layer | RN-Android | Android-native | RN-iOS | iOS-native
|
|---|---|:-:|:-:|:-:|:-:|
| Core telemetry (views/actions/handled-error/crash → `_rumdata`) | API
| ✅ | ✅ | ✅ | ✅ |
| Crash session viewable in dashboard | UI | ✅ | ✅ | ✅ | ✅ |
| Attributes: env / service / version tagging | API | ✅ | ✅ | ✅ | ✅ |
| User identity (`setUser`) | API | ✅ | ✅ | ✅ | ✅ |
| Network: fetch → resource + 4xx/5xx | API | ✅ | ✅ | ✅ | ✅ |
| Session-replay PII masking | UI | ✅ | ✅ | ✅ | ✅ |
| Background / foreground view transitions | API | ✅ | ✅ | — | ✅ |
| No-phone-home (no egress to Datadog hosts) | security | ✅ | ✅ | — | ✅
|

- Shared per-capability factories keep the specs DRY
(`utils/rumChecks.js`, `utils/coreRumSpec.js`).
- Negatives are covered: 4xx capture, PII-must-not-leak (masking), no
phone-home, handled-error ≠ crash.
- `o2-enterprise#2289` dashboard bugs are kept as inline skips that flip
green once fixed.
- Full capability × platform detail + on-device status:
`docs/COVERAGE-MATRIX.md`.

## Fixture apps + build-time config

- Four vendored apps under `apps/`: `o2-rum-tester` (React Native —
drives both RN platforms), `o2-native-android` (Kotlin), `o2-native-ios`
(Swift / xcodegen).
- Each reads its RUM target (endpoint / org / token) and SDK version
from a **build-time-overridable** config. The committed default is the
current pin, so local dev is unchanged; CI regenerates it to point at
the runner's OpenObserve and can pin the latest SDK on release.

## CI — `.github/workflows/mobile-sdk-e2e.yml` (self-contained, no
secrets)

- **Build once per OS from source.** `build_oo_linux` / `build_oo_macos`
compile OpenObserve (binary + embedded web UI) from this PR's code and
cache it. Device jobs download and run it **natively**, and the app
posts real RUM to it (Android reaches the host at `10.0.2.2`, the iOS
simulator via `localhost`). No Docker, no shared cluster, no secrets.
- **Device jobs run in parallel.** `android` (Linux + KVM emulator →
rn-android + android-native) and `ios` (macOS simulator, pinned Xcode
16.4 → rn-ios + ios-native).
- **`merge_reports`** combines both platforms into one browsable HTML
report — even when a device job fails.
- **`decide` picks which platform runs, to keep scarce macOS runners in
check:**
- PR touching Android or shared mobile-RUM code → **Android auto-runs**;
iOS is gated behind the `mobile-e2e` label.
  - `mobile-e2e` label → both platforms (full pre-merge gate).
- `repository_dispatch(mobile-sdk-release)` from an SDK repo → runs that
SDK's platform (companion PRs wire this up:
openobserve-react-native-rum#3, openobserve-sdk-android#1,
openobserve-sdk-ios#1).
  - `workflow_dispatch` → manual, choose platforms.

*Separate workflow :* it needs macOS **and** Android emulators, builds 4
phone apps, and is dispatch-driven — matching the repo's existing
per-suite split

## Status

Green end-to-end on GitHub runners (Android + iOS device jobs + merge) :
https://github.com/openobserve/openobserve/actions/runs/31733717758/job/94565520891

---------

Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
2026-08-20 08:13:35 +00:00

55 lines
2.5 KiB
JavaScript

// SDK install-doc version drift — does the SDK version the OpenObserve UI documents stay in sync
// with the LATEST published SDK? (Product finding: the RUM install cards pin alpha.4 while the SDKs
// ship alpha.5/alpha.6 — see docs/COVERAGE-MATRIX.md.)
//
// SKIPPED (test.fixme) for now: the mobile-RUM install docs are still under active development, so
// the pinned versions move. Enable once the docs are version-synced (dev to update). The logic below
// is real and ready — it fetches the latest published version per registry and compares it to what
// the docs pin.
const { test, expect } = require('@playwright/test');
// Latest published version, per registry (the source of truth the UI docs should track).
async function latestPublished(kind) {
if (kind === 'react-native') {
const r = await fetch('https://registry.npmjs.org/@openobserve/mobile-react-native/latest');
return (await r.json()).version;
}
if (kind === 'ios') {
const r = await fetch('https://api.github.com/repos/openobserve/openobserve-sdk-ios/releases/latest');
// GitHub release tags are conventionally `v`-prefixed; strip it so the compare isn't a guaranteed
// miss when the versions are actually in sync.
return ((await r.json()).tag_name || '').replace(/^v/, '');
}
if (kind === 'android') {
const r = await fetch('https://repo1.maven.org/maven2/ai/openobserve/o2-sdk-android-rum/maven-metadata.xml');
const xml = await r.text();
return (xml.match(/<release>([^<]+)<\/release>/) || xml.match(/<latest>([^<]+)<\/latest>/) || [])[1];
}
throw new Error(`unknown SDK kind: ${kind}`);
}
// What each platform's install card currently documents (update as the docs move; Android/iOS cards
// are enterprise-surfaced, React Native is OSS `RUM_RN_SDK_VERSION`).
const DOCUMENTED = {
'react-native': '0.1.0-alpha.4',
android: '0.1.0-alpha4',
ios: '0.1.0-alpha.4',
};
test.describe('SDK install-doc version drift (o2-enterprise#2289)', () => {
for (const kind of ['react-native', 'android', 'ios']) {
// Enable (remove .fixme) once the docs are kept version-synced.
test.fixme(
`the ${kind} install card documents the latest published SDK version`,
{ tag: ['@docs', '@sdk-drift', '@known-bug'] },
async () => {
const published = await latestPublished(kind);
expect(
DOCUMENTED[kind],
`the ${kind} install doc pins ${DOCUMENTED[kind]} but the latest published is ${published}`,
).toBe(published);
},
);
}
});