openobserve

mirror of https://github.com/openobserve/openobserve.git synced 2026-11-05 00:09:35 +00:00

Files

T

Shrinath Rao 184268ff1b test(ci): loud stdout banner when Playwright retries a test (#13529 )

## What

Adds a small **log-only** Playwright reporter that prints a LOUD,
unmissable banner to stdout **the moment any test starts a retry
attempt**, plus an end-of-shard summary listing every test that was
retried.

## Why

In CI the reporter is `blob`, which writes nothing to stdout. So when a
failed test is retried (`retries: process.env.CI ? 3 : 0` in
`playwright.config.js`) there is **no signal in the GitHub Actions logs
that a retry is even happening** — a test silently re-runs up to 3 more
times and the log just shows the next `Test Started` line. This makes
flaky-vs-real failures hard to spot when scanning CI output.

The retry is internal to `npx playwright test`, so the workflow YAML
can't see it — it has to be emitted from the Playwright reporter layer.
This is that layer.

## What it looks like

```
🔁🔁🔁══════════════════════════════════════════════════════════🔁🔁🔁
🔁  R E T R Y   # 1   ·   attempt 2 of 4  (test previously failed — re-running)
🔁  ▶ chromium › logsqueries.cte.spec.js › should query with simple TotalCols CTE
🔁  ▶ Logs/logsqueries.cte.spec.js
🔁🔁🔁══════════════════════════════════════════════════════════🔁🔁🔁
```

…and once per shard at the end:

```
🔁  RETRY SUMMARY — N test(s) were retried in this shard:
🔁    • [2 retries] chromium › … › should query …  (Logs/logsqueries.cte.spec.js)
```

## Scope / safety

- Added to the **CI** reporter list only, alongside `blob` (multiple
reporters are fine). It only `console.log`s — never sends anything,
never fails the run.
- **ENT gets this for free**: the ENT Playwright workflow checks out and
runs the OSS `tests/ui-testing` tree, so this one OSS change makes
retries loud in both OSS and ENT CI. There is no ENT test tree to
change.
- Verified locally with a forced-failure probe (`CI=true`, `retries:
2`): the per-retry banner fires on attempt 2 and 3, and the summary
lists the retried test.

## Files

- `tests/ui-testing/playwright-tests/utils/retry-banner-reporter.js` —
new reporter
- `tests/ui-testing/playwright.config.js` — wire it into the CI reporter
array