mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-03 16:10:21 +00:00
test: fix regression Merge Reports failing with missing html-report (force reporters) (#12939)
## Did we introduce this? **No.** This is a pre-existing fragility in the `Merge Reports and Upload to TestDino` job of the **Playwright Regression Tests** workflow — the `unset CI` mechanism has been there since the workflow was created (#11669, 2026-05). It is unrelated to any test change. ## Symptom Job fails with exit code 2 **even when all e2e shards pass** (e.g. run [28410300804](https://github.com/openobserve/openobserve/actions/runs/28410300804/job/84183381446) — 11/11 shards green, only Merge Reports red): ``` Contents of playwright-results: -rw-r--r-- 1 runner runner 0 ... .gitkeep ls: cannot access 'playwright-results/html-report/': No such file or directory ##[error]Process completed with exit code 2. ``` ## Root cause `playwright.config.js` selects reporters by `process.env.CI`: ```js reporter: process.env.CI ? [['blob', { outputDir: 'blob-report' }]] : [['html', { outputFolder: 'playwright-results/html-report' }], ['json', { outputFile: 'playwright-results/report.json' }]] ``` The merge step depended on `unset CI` to take the html/json branch. The runner exports `CI=true`, and `unset CI` does not reliably reach the `merge-reports` subprocess — so the config re-selects the **`blob`** reporter, which writes a merged `blob-report/` and **nothing** into `playwright-results/`. The downstream `ls playwright-results/html-report/` then exits 2. ### Reproduced locally with this run's blob artifacts - `CI=true` + `merge-reports --config playwright.config.js` → writes `blob-report/report.zip`, no `playwright-results/html-report/` → `ls` exit 2 (matches CI exactly). - Reporters forced explicitly → always writes `html-report/index.html` + `report.json`, regardless of `CI`. ## Fix Stop depending on `unset CI`. Force the reporters and their output paths explicitly: ```bash PLAYWRIGHT_HTML_OUTPUT_DIR=playwright-results/html-report \ PLAYWRIGHT_HTML_OPEN=never \ PLAYWRIGHT_JSON_OUTPUT_NAME=playwright-results/report.json \ npx playwright merge-reports --reporter html,json ./all-blob-reports ``` Plus a guard that fails with a clear message (and lists `all-blob-reports/`) if the expected outputs are missing — instead of the cryptic `ls` exit 2. ## Verification (local, against this run's 10 blob artifacts) Simulating CI (`CI=true GITHUB_ACTIONS=true`): ``` merge exit=0 GUARD: PASS playwright-results/html-report/index.html (717 KB) playwright-results/report.json (587 KB, stats: expected=75 unexpected=0 flaky=0 skipped=9) blob-report/ → not created ``` Downstream `jq '.stats'` reads the produced `report.json` correctly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
01f4a6cc08
commit
43263ad45e
@@ -34,23 +34,34 @@ gh_get(){ curl -s -H "Authorization: token ${GH_TOKEN}" -H "Accept: application/
|
||||
|
||||
# wall-clock of one attempt = max(completed) - min(started) across its jobs (clamped non-negative
|
||||
# to guard against clock-skew where a completed_at precedes a started_at).
|
||||
attempt_wall(){ jq '[.jobs[]|select(.completed_at!=null and .started_at!=null)] as $j
|
||||
| (if ($j|length)>0 then (($j|map(.completed_at|fromdateiso8601)|max)-($j|map(.started_at|fromdateiso8601)|min)) else 0 end)
|
||||
# $1 = floor epoch (the attempt's run_started_at). On a RE-RUN, GitHub's per-attempt jobs API
|
||||
# carries over jobs that did NOT re-run with their ORIGINAL (earlier-attempt) timestamps, so a
|
||||
# naive min(started) spans the idle gap between a failed run and a much-later manual re-run —
|
||||
# inflating the duration by hours. Flooring the start at run_started_at counts only this
|
||||
# attempt's actual work (no effect on normal runs, where min(started) >= run_started_at).
|
||||
attempt_wall(){ jq --argjson floor "${1:-0}" '[.jobs[]|select(.completed_at!=null and .started_at!=null)] as $j
|
||||
| (if ($j|length)>0 then (($j|map(.completed_at|fromdateiso8601)|max)-([($j|map(.started_at|fromdateiso8601)|min), $floor]|max)) else 0 end)
|
||||
| if . < 0 then 0 else . end'; }
|
||||
# Epoch seconds of a run/attempt object's run_started_at (0 if absent).
|
||||
run_started_epoch(){ jq 'if .run_started_at then (.run_started_at|fromdateiso8601) else 0 end'; }
|
||||
|
||||
RUN_JSON=$(gh_get "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" 2>/dev/null)
|
||||
[ -n "$RUN_JSON" ] || { echo "::warning::could not fetch run object — skipping"; exit 0; }
|
||||
ATT="${GITHUB_RUN_ATTEMPT:-1}"
|
||||
# This attempt's start — the floor for its wall-clock (see attempt_wall).
|
||||
RUN_STARTED=$(printf '%s' "$RUN_JSON" | run_started_epoch 2>/dev/null || echo 0)
|
||||
[ -n "$RUN_STARTED" ] || RUN_STARTED=0
|
||||
|
||||
THIS_JOBS=$(gh_get "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/attempts/${ATT}/jobs?per_page=100" 2>/dev/null)
|
||||
FINAL_DUR=$(printf '%s' "$THIS_JOBS" | attempt_wall 2>/dev/null || echo 0)
|
||||
FINAL_DUR=$(printf '%s' "$THIS_JOBS" | attempt_wall "$RUN_STARTED" 2>/dev/null || echo 0)
|
||||
|
||||
# total across attempts: add each prior attempt's wall-clock.
|
||||
TOTAL_DUR="$FINAL_DUR"
|
||||
if [ "$ATT" -gt 1 ] 2>/dev/null; then
|
||||
SUM="$FINAL_DUR"
|
||||
for n in $(seq 1 $((ATT-1))); do
|
||||
W=$(gh_get "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/attempts/${n}/jobs?per_page=100" 2>/dev/null | attempt_wall 2>/dev/null || echo 0)
|
||||
NFLOOR=$(gh_get "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/attempts/${n}" 2>/dev/null | run_started_epoch 2>/dev/null || echo 0)
|
||||
W=$(gh_get "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/attempts/${n}/jobs?per_page=100" 2>/dev/null | attempt_wall "${NFLOOR:-0}" 2>/dev/null || echo 0)
|
||||
SUM=$(awk -v a="$SUM" -v b="$W" 'BEGIN{print a+b}')
|
||||
done
|
||||
TOTAL_DUR="$SUM"
|
||||
|
||||
Reference in New Issue
Block a user