openobserve

8,362 Commits 646 Branches 243 Tags

2 Commits

Include renames

Author	SHA1	Message	Date
Vamsi Karthik R	9a0a272103	test(mobile): P0 session-replay upload assertion (#13945 ) ## What Adds a P0 data-layer assertion to the mobile RUM masking suite: after driving the replay flow, it queries the `_sessionreplay` stream and requires that segments actually landed. A rendered dashboard is no longer accepted as proof that Session Replay works. This is the check that catches #13942 — on iOS the RN Session-Replay bridge posts segments to Datadog's `/api/v2/replay` instead of OpenObserve's `/replay`, so they're rejected (401, no error surfaced) and zero segments reach `_sessionreplay`. RN iOS session replay silently records nothing. ## RC The masking suite verified Session Replay only through the dashboard UI (does the player render + is PII masked). It had no data-layer check that segments actually reached OpenObserve. So a broken upload showed only as "the replay didn't render" — ambiguous (a benign dashboard-render flake or a real non-upload) — and that non-render had been scoped to a skip. The bug slipped through the ambiguity. ## The fix - P0 (data): `maskingSuite` asserts `_sessionreplay` received segments for the session (`> 0`). Non-upload → hard fail. Platform-agnostic; catches this whole class of bug. - P1 (privacy): the dashboard render + PII scan stays, run only when the player paints (segments-landed-but-player-didn't = dashboard flake, not an SDK bug). - Removed the `requireReplay` skip hack. `ios-native` masking unskipped → runs as hard P0 (native iOS SDK uses the correct `/replay` endpoint). - `rn-ios` masking is a plain test that hard-FAILS until #13942 is fixed — no skip, no expected-fail marker. The P0 assertion sees 0 segments → red; it passes on its own the moment the SDK starts uploading. The fix lives in `openobserve-react-native-rum`. ## Expected result per platform \| Platform \| Masking (P0 upload + PII) \| Why \| \|---\|:-:\|---\| \| rn-android \| ✅ pass \| RN replay works on Android \| \| rn-ios \| ❌ fail \| #13942 — 0 replay segments \| \| android-native \| ✅ pass \| native SDK, correct endpoint \| \| ios-native \| ✅ pass \| native SDK, correct endpoint \| So whenever iOS runs (the `mobile-e2e` label / an SDK dispatch), the iOS job is red until the SDK fix lands — an intentional, self-healing signal. ## Validation (local, real device/data runs) - `android-native` + `ios-native` masking pass the new P0 (segments present in `_sessionreplay`). - A non-uploading session returns 0 segments → P0 correctly fails — the exact rn-ios symptom. Scope: only `tests/mobile-testing/` — 3 files.	2026-08-25 04:36:18 +00:00
Vamsi Karthik RandShrinath Rao	6930a9b724	test(mobile): RUM SDK end-to-end suite across 4 platforms + self-contained CI (#13628 ) ## What this PR adds end-to-end test suite for the Mobile RUM SDK under `tests/mobile-testing/`, plus a self-contained CI workflow that runs it on real emulators/simulators against an OpenObserve built from this PR's source. Test / fixture / CI only — no product-code/existing test changes ## How each test works (3 layers) Every scenario is a real end-to-end journey, so a failure points at the actual broken layer: 1. Drive — a Maestro flow taps through the app like a user (`maestro/<track>/.yaml`). 2. API* — query the `_rumdata` stream and assert the telemetry is correct (`utils/ooClient.js`, poll-and-retry for async ingestion). 3. UI — open the RUM dashboard and assert the sessions ## Coverage (42 tests across 4 platforms) The native platforms (android-native, ios-native) are now at parity with React Native — each new native test (masking, network, bg/fg, user-identity, no-phone-home) was built + validated on a real emulator/simulator before landing. \| Capability \| Layer \| RN-Android \| Android-native \| RN-iOS \| iOS-native \| \|---\|---\|:-:\|:-:\|:-:\|:-:\| \| Core telemetry (views/actions/handled-error/crash → `_rumdata`) \| API \| ✅ \| ✅ \| ✅ \| ✅ \| \| Crash session viewable in dashboard \| UI \| ✅ \| ✅ \| ✅ \| ✅ \| \| Attributes: env / service / version tagging \| API \| ✅ \| ✅ \| ✅ \| ✅ \| \| User identity (`setUser`) \| API \| ✅ \| ✅ \| ✅ \| ✅ \| \| Network: fetch → resource + 4xx/5xx \| API \| ✅ \| ✅ \| ✅ \| ✅ \| \| Session-replay PII masking \| UI \| ✅ \| ✅ \| ✅ \| ✅ \| \| Background / foreground view transitions \| API \| ✅ \| ✅ \| — \| ✅ \| \| No-phone-home (no egress to Datadog hosts) \| security \| ✅ \| ✅ \| — \| ✅ \| - Shared per-capability factories keep the specs DRY (`utils/rumChecks.js`, `utils/coreRumSpec.js`). - Negatives are covered: 4xx capture, PII-must-not-leak (masking), no phone-home, handled-error ≠ crash. - `o2-enterprise#2289` dashboard bugs are kept as inline skips that flip green once fixed. - Full capability × platform detail + on-device status: `docs/COVERAGE-MATRIX.md`. ## Fixture apps + build-time config - Four vendored apps under `apps/`: `o2-rum-tester` (React Native — drives both RN platforms), `o2-native-android` (Kotlin), `o2-native-ios` (Swift / xcodegen). - Each reads its RUM target (endpoint / org / token) and SDK version from a build-time-overridable config. The committed default is the current pin, so local dev is unchanged; CI regenerates it to point at the runner's OpenObserve and can pin the latest SDK on release. ## CI — `.github/workflows/mobile-sdk-e2e.yml` (self-contained, no secrets) - Build once per OS from source. `build_oo_linux` / `build_oo_macos` compile OpenObserve (binary + embedded web UI) from this PR's code and cache it. Device jobs download and run it natively, and the app posts real RUM to it (Android reaches the host at `10.0.2.2`, the iOS simulator via `localhost`). No Docker, no shared cluster, no secrets. - Device jobs run in parallel. `android` (Linux + KVM emulator → rn-android + android-native) and `ios` (macOS simulator, pinned Xcode 16.4 → rn-ios + ios-native). - `merge_reports` combines both platforms into one browsable HTML report — even when a device job fails. - `decide` picks which platform runs, to keep scarce macOS runners in check: - PR touching Android or shared mobile-RUM code → Android auto-runs; iOS is gated behind the `mobile-e2e` label. - `mobile-e2e` label → both platforms (full pre-merge gate). - `repository_dispatch(mobile-sdk-release)` from an SDK repo → runs that SDK's platform (companion PRs wire this up: openobserve-react-native-rum#3, openobserve-sdk-android#1, openobserve-sdk-ios#1). - `workflow_dispatch` → manual, choose platforms. Separate workflow : it needs macOS and Android emulators, builds 4 phone apps, and is dispatch-driven — matching the repo's existing per-suite split ## Status Green end-to-end on GitHub runners (Android + iOS device jobs + merge) : https://github.com/openobserve/openobserve/actions/runs/31733717758/job/94565520891 --------- Co-authored-by: Shrinath Rao <shnath@openobserve.ai>	2026-08-20 08:13:35 +00:00

Author

SHA1

Message

Date

Vamsi Karthik R

9a0a272103

test(mobile): P0 session-replay upload assertion (#13945 )

## What

Adds a **P0 data-layer assertion** to the mobile RUM masking suite:
after driving the replay flow, it queries the **`_sessionreplay`**
stream and requires that segments actually landed. A rendered dashboard
is no longer accepted as proof that Session Replay works.

This is the check that catches **#13942** — on iOS the RN Session-Replay
bridge posts segments to Datadog's `/api/v2/replay` instead of
OpenObserve's `/replay`, so they're rejected (401, no error surfaced)
and **zero** segments reach `_sessionreplay`. RN iOS session replay
silently records nothing.

## RC

The masking suite verified Session Replay **only through the dashboard
UI** (does the player render + is PII masked). It had no data-layer
check that segments actually reached OpenObserve. So a broken upload
showed only as "the replay didn't render" — ambiguous (a benign
dashboard-render flake *or* a real non-upload) — and that non-render had
been scoped to a skip. The bug slipped through the ambiguity.

## The fix

- **P0 (data):** `maskingSuite` asserts `_sessionreplay` received
segments for the session (`> 0`). Non-upload → hard fail.
Platform-agnostic; catches this whole class of bug.
- **P1 (privacy):** the dashboard render + PII scan stays, run only when
the player paints (segments-landed-but-player-didn't = dashboard flake,
not an SDK bug).
- Removed the `requireReplay` skip hack. **`ios-native` masking
unskipped** → runs as hard P0 (native iOS SDK uses the correct `/replay`
endpoint).
- **`rn-ios` masking is a plain test that hard-FAILS until #13942 is
fixed** — **no skip, no expected-fail marker.** The P0 assertion sees 0
segments → red; it **passes on its own** the moment the SDK starts
uploading. The fix lives in `openobserve-react-native-rum`.

## Expected result per platform

| Platform | Masking (P0 upload + PII) | Why |
|---|:-:|---|
| rn-android | ✅ pass | RN replay works on Android |
| **rn-ios** | ❌ **fail** | **#13942** — 0 replay segments |
| android-native | ✅ pass | native SDK, correct endpoint |
| ios-native | ✅ pass | native SDK, correct endpoint |

So whenever iOS runs (the `mobile-e2e` label / an SDK dispatch), the iOS
job is **red until the SDK fix lands** — an intentional, self-healing
signal.

## Validation (local, real device/data runs)

- `android-native` + `ios-native` masking **pass** the new P0 (segments
present in `_sessionreplay`).
- A non-uploading session returns **0 segments → P0 correctly fails** —
the exact rn-ios symptom.

Scope: only `tests/mobile-testing/` — 3 files.

2026-08-25 04:36:18 +00:00

Vamsi Karthik RandShrinath Rao

6930a9b724

test(mobile): RUM SDK end-to-end suite across 4 platforms + self-contained CI (#13628 )

## What this PR adds

end-to-end test suite for the **Mobile RUM SDK** under
`tests/mobile-testing/`, plus a **self-contained CI workflow** that runs
it on real emulators/simulators against an OpenObserve **built from this
PR's source**. Test / fixture / CI only — no product-code/existing test
changes

## How each test works (3 layers)

Every scenario is a real end-to-end journey, so a failure points at the
actual broken layer:

1. **Drive** — a Maestro flow taps through the app like a user
(`maestro/<track>/*.yaml`).
2. **API** — query the `_rumdata` stream and assert the telemetry is
correct (`utils/ooClient.js`, poll-and-retry for async ingestion).
3. **UI** — open the RUM dashboard and assert the sessions

## Coverage (42 tests across 4 platforms)

The **native platforms (android-native, ios-native) are now at parity
with React Native** — each new
native test (masking, network, bg/fg, user-identity, no-phone-home) was
built + validated on a real
emulator/simulator before landing.

| Capability | Layer | RN-Android | Android-native | RN-iOS | iOS-native
|
|---|---|:-:|:-:|:-:|:-:|
| Core telemetry (views/actions/handled-error/crash → `_rumdata`) | API
| ✅ | ✅ | ✅ | ✅ |
| Crash session viewable in dashboard | UI | ✅ | ✅ | ✅ | ✅ |
| Attributes: env / service / version tagging | API | ✅ | ✅ | ✅ | ✅ |
| User identity (`setUser`) | API | ✅ | ✅ | ✅ | ✅ |
| Network: fetch → resource + 4xx/5xx | API | ✅ | ✅ | ✅ | ✅ |
| Session-replay PII masking | UI | ✅ | ✅ | ✅ | ✅ |
| Background / foreground view transitions | API | ✅ | ✅ | — | ✅ |
| No-phone-home (no egress to Datadog hosts) | security | ✅ | ✅ | — | ✅
|

- Shared per-capability factories keep the specs DRY
(`utils/rumChecks.js`, `utils/coreRumSpec.js`).
- Negatives are covered: 4xx capture, PII-must-not-leak (masking), no
phone-home, handled-error ≠ crash.
- `o2-enterprise#2289` dashboard bugs are kept as inline skips that flip
green once fixed.
- Full capability × platform detail + on-device status:
`docs/COVERAGE-MATRIX.md`.

## Fixture apps + build-time config

- Four vendored apps under `apps/`: `o2-rum-tester` (React Native —
drives both RN platforms), `o2-native-android` (Kotlin), `o2-native-ios`
(Swift / xcodegen).
- Each reads its RUM target (endpoint / org / token) and SDK version
from a **build-time-overridable** config. The committed default is the
current pin, so local dev is unchanged; CI regenerates it to point at
the runner's OpenObserve and can pin the latest SDK on release.

## CI — `.github/workflows/mobile-sdk-e2e.yml` (self-contained, no
secrets)

- **Build once per OS from source.** `build_oo_linux` / `build_oo_macos`
compile OpenObserve (binary + embedded web UI) from this PR's code and
cache it. Device jobs download and run it **natively**, and the app
posts real RUM to it (Android reaches the host at `10.0.2.2`, the iOS
simulator via `localhost`). No Docker, no shared cluster, no secrets.
- **Device jobs run in parallel.** `android` (Linux + KVM emulator →
rn-android + android-native) and `ios` (macOS simulator, pinned Xcode
16.4 → rn-ios + ios-native).
- **`merge_reports`** combines both platforms into one browsable HTML
report — even when a device job fails.
- **`decide` picks which platform runs, to keep scarce macOS runners in
check:**
- PR touching Android or shared mobile-RUM code → **Android auto-runs**;
iOS is gated behind the `mobile-e2e` label.
  - `mobile-e2e` label → both platforms (full pre-merge gate).
- `repository_dispatch(mobile-sdk-release)` from an SDK repo → runs that
SDK's platform (companion PRs wire this up:
openobserve-react-native-rum#3, openobserve-sdk-android#1,
openobserve-sdk-ios#1).
  - `workflow_dispatch` → manual, choose platforms.

*Separate workflow :* it needs macOS **and** Android emulators, builds 4
phone apps, and is dispatch-driven — matching the repo's existing
per-suite split

## Status

Green end-to-end on GitHub runners (Android + iOS device jobs + merge) :
https://github.com/openobserve/openobserve/actions/runs/31733717758/job/94565520891

---------

Co-authored-by: Shrinath Rao <shnath@openobserve.ai>

2026-08-20 08:13:35 +00:00