mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-03 16:10:21 +00:00
Vamsi Karthik RandShrinath Rao
6930a9b724
## What this PR adds end-to-end test suite for the **Mobile RUM SDK** under `tests/mobile-testing/`, plus a **self-contained CI workflow** that runs it on real emulators/simulators against an OpenObserve **built from this PR's source**. Test / fixture / CI only — no product-code/existing test changes ## How each test works (3 layers) Every scenario is a real end-to-end journey, so a failure points at the actual broken layer: 1. **Drive** — a Maestro flow taps through the app like a user (`maestro/<track>/*.yaml`). 2. **API** — query the `_rumdata` stream and assert the telemetry is correct (`utils/ooClient.js`, poll-and-retry for async ingestion). 3. **UI** — open the RUM dashboard and assert the sessions ## Coverage (42 tests across 4 platforms) The **native platforms (android-native, ios-native) are now at parity with React Native** — each new native test (masking, network, bg/fg, user-identity, no-phone-home) was built + validated on a real emulator/simulator before landing. | Capability | Layer | RN-Android | Android-native | RN-iOS | iOS-native | |---|---|:-:|:-:|:-:|:-:| | Core telemetry (views/actions/handled-error/crash → `_rumdata`) | API | ✅ | ✅ | ✅ | ✅ | | Crash session viewable in dashboard | UI | ✅ | ✅ | ✅ | ✅ | | Attributes: env / service / version tagging | API | ✅ | ✅ | ✅ | ✅ | | User identity (`setUser`) | API | ✅ | ✅ | ✅ | ✅ | | Network: fetch → resource + 4xx/5xx | API | ✅ | ✅ | ✅ | ✅ | | Session-replay PII masking | UI | ✅ | ✅ | ✅ | ✅ | | Background / foreground view transitions | API | ✅ | ✅ | — | ✅ | | No-phone-home (no egress to Datadog hosts) | security | ✅ | ✅ | — | ✅ | - Shared per-capability factories keep the specs DRY (`utils/rumChecks.js`, `utils/coreRumSpec.js`). - Negatives are covered: 4xx capture, PII-must-not-leak (masking), no phone-home, handled-error ≠ crash. - `o2-enterprise#2289` dashboard bugs are kept as inline skips that flip green once fixed. - Full capability × platform detail + on-device status: `docs/COVERAGE-MATRIX.md`. ## Fixture apps + build-time config - Four vendored apps under `apps/`: `o2-rum-tester` (React Native — drives both RN platforms), `o2-native-android` (Kotlin), `o2-native-ios` (Swift / xcodegen). - Each reads its RUM target (endpoint / org / token) and SDK version from a **build-time-overridable** config. The committed default is the current pin, so local dev is unchanged; CI regenerates it to point at the runner's OpenObserve and can pin the latest SDK on release. ## CI — `.github/workflows/mobile-sdk-e2e.yml` (self-contained, no secrets) - **Build once per OS from source.** `build_oo_linux` / `build_oo_macos` compile OpenObserve (binary + embedded web UI) from this PR's code and cache it. Device jobs download and run it **natively**, and the app posts real RUM to it (Android reaches the host at `10.0.2.2`, the iOS simulator via `localhost`). No Docker, no shared cluster, no secrets. - **Device jobs run in parallel.** `android` (Linux + KVM emulator → rn-android + android-native) and `ios` (macOS simulator, pinned Xcode 16.4 → rn-ios + ios-native). - **`merge_reports`** combines both platforms into one browsable HTML report — even when a device job fails. - **`decide` picks which platform runs, to keep scarce macOS runners in check:** - PR touching Android or shared mobile-RUM code → **Android auto-runs**; iOS is gated behind the `mobile-e2e` label. - `mobile-e2e` label → both platforms (full pre-merge gate). - `repository_dispatch(mobile-sdk-release)` from an SDK repo → runs that SDK's platform (companion PRs wire this up: openobserve-react-native-rum#3, openobserve-sdk-android#1, openobserve-sdk-ios#1). - `workflow_dispatch` → manual, choose platforms. *Separate workflow :* it needs macOS **and** Android emulators, builds 4 phone apps, and is dispatch-driven — matching the repo's existing per-suite split ## Status Green end-to-end on GitHub runners (Android + iOS device jobs + merge) : https://github.com/openobserve/openobserve/actions/runs/31733717758/job/94565520891 --------- Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
93 lines
2.6 KiB
JSON
93 lines
2.6 KiB
JSON
{
|
|
"name": "openobserve-mobile-testing",
|
|
"version": "0.1.0",
|
|
"lockfileVersion": 3,
|
|
"requires": true,
|
|
"packages": {
|
|
"": {
|
|
"name": "openobserve-mobile-testing",
|
|
"version": "0.1.0",
|
|
"devDependencies": {
|
|
"@playwright/test": "^1.47.0",
|
|
"dotenv": "^16.4.5"
|
|
}
|
|
},
|
|
"node_modules/@playwright/test": {
|
|
"version": "1.62.1",
|
|
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.62.1.tgz",
|
|
"integrity": "sha512-DTcUc8qii+cpHvtOwggMtBRMjKZHXYWdw8syRYu2vtzuq4Wxphqq4NfCs5Zt44L6mA8rfDfj+PHnxFc/FeK6mQ==",
|
|
"dev": true,
|
|
"license": "Apache-2.0",
|
|
"dependencies": {
|
|
"playwright": "1.62.1"
|
|
},
|
|
"bin": {
|
|
"playwright": "cli.js"
|
|
},
|
|
"engines": {
|
|
"node": ">=20"
|
|
}
|
|
},
|
|
"node_modules/dotenv": {
|
|
"version": "16.6.1",
|
|
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz",
|
|
"integrity": "sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==",
|
|
"dev": true,
|
|
"license": "BSD-2-Clause",
|
|
"engines": {
|
|
"node": ">=12"
|
|
},
|
|
"funding": {
|
|
"url": "https://dotenvx.com"
|
|
}
|
|
},
|
|
"node_modules/fsevents": {
|
|
"version": "2.3.2",
|
|
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
|
|
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
|
|
"dev": true,
|
|
"hasInstallScript": true,
|
|
"license": "MIT",
|
|
"optional": true,
|
|
"os": [
|
|
"darwin"
|
|
],
|
|
"engines": {
|
|
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
|
}
|
|
},
|
|
"node_modules/playwright": {
|
|
"version": "1.62.1",
|
|
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.1.tgz",
|
|
"integrity": "sha512-0M+L3LAD8/nm554LOla9Ayx0j0tmFZ0FBcoQ7F1VuVHpM/XpiC8RcDzBQB8W5+hA8L22THxELzeF+2WcUzvcLg==",
|
|
"dev": true,
|
|
"license": "Apache-2.0",
|
|
"dependencies": {
|
|
"playwright-core": "1.62.1"
|
|
},
|
|
"bin": {
|
|
"playwright": "cli.js"
|
|
},
|
|
"engines": {
|
|
"node": ">=20"
|
|
},
|
|
"optionalDependencies": {
|
|
"fsevents": "2.3.2"
|
|
}
|
|
},
|
|
"node_modules/playwright-core": {
|
|
"version": "1.62.1",
|
|
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.1.tgz",
|
|
"integrity": "sha512-wPYSwEBJY9GHraISXqyqtx0na0LpO3XEX7jNDhntbex7tzUS7kLnZsOlFruFJB4Hi/rhDMjXGqHewDZ68nYZVw==",
|
|
"dev": true,
|
|
"license": "Apache-2.0",
|
|
"bin": {
|
|
"playwright-core": "cli.js"
|
|
},
|
|
"engines": {
|
|
"node": ">=20"
|
|
}
|
|
}
|
|
}
|
|
}
|