mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 16:10:43 +00:00
## Problem Playwright CI jobs were hanging for 2–6 hours and then timing out without running a single test. Root cause: `npx playwright install --with-deps chromium` downloads the browser binary successfully (~3s), but the `--with-deps` flag then runs a second `apt-get` call that waits indefinitely for the dpkg lock held by Ubuntu's `unattended-upgrades` background daemon on ubicloud runners. Evidence: in a 2h8m run, Chromium downloaded at `08:51:49`, but the `playwright install` process was still listed as a zombie when the job was killed at `11:00:23`. ## Fix Two-layer fix applied to all four workflow files (`playwright.yml` + `playwright_regression.yml` in both OSS and enterprise): **1. Kill background apt services at job start** (new first step) ```bash sudo systemctl stop apt-daily.service apt-daily-upgrade.service unattended-upgrades.service sudo systemctl stop apt-daily.timer apt-daily-upgrade.timer sudo systemctl disable apt-daily.service apt-daily-upgrade.service unattended-upgrades.service sudo flock --wait 30 /var/lib/dpkg/lock-frontend true ``` This eliminates the root cause — the dpkg lock contender is gone before `playwright install` runs. **2. Cache the browser binary** (`actions/cache@v4` keyed on `package-lock.json` hash) On cache hit, only system dependencies are installed (`playwright install-deps chromium`), avoiding apt entirely. On cache miss, the full `playwright install --with-deps chromium` runs with apt services already stopped. **3. Filter TestDino uploads to failures only** Before uploading to TestDino, the JSON report is filtered to only include tests with `status === 'unexpected'`. This prevents passing tests from polluting the TestDino failure history. ## Files changed - `.github/workflows/playwright.yml` — kill apt, cache browser, cache-aware install, TestDino filter - `.github/workflows/playwright_regression.yml` — kill apt, cache browser, split into separate npm/playwright/test steps --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>