ci: drop redundant post-merge runs, gate regression on main changes (#13024)

## Why

Our OSS Playwright + API CI compute (ubicloud build + eks matrix) is a
major cost driver. Two sources of avoidable spend:

1. **Every merged PR ran the full heavy suite up to 3×** — on the PR
(label-gated), in the merge queue (`merge_group`), and *again* on `push:
main`. The `push: main` run is a duplicate of the `merge_group` run
(both build/test the exact commit landing on main) **and** it bypasses
the `e2e` label gate, clawing back the gate's savings.
2. **Regression runs every 4h regardless of whether `main` moved** —
rebuilding and re-running the full matrix on identical code
overnight/weekends.

## Changes

**`playwright.yml` + `api-testing.yml`**
- Remove the `push: main` trigger. `merge_group` remains the trusted
main-ward validation.
- **Cache fix (important):** the Rust cache was only *saved* on `push:
main` runs (`save-if: github.ref == 'refs/heads/main'`). Deleting `push`
alone would stop the cache ever refreshing → cold builds → *more*
compute. So `save-if` now also fires on the `merge_group` run, which
builds the exact soon-to-be-main commit.

**`playwright_regression.yml`**
- New `check_main_changed` gate: compares `main`'s HEAD to the last
run's `head_sha` and skips build+matrix when nothing merged since.
Manual `workflow_dispatch` always runs. The skip cascades cleanly
(`playwright_summary` already treats skipped build/ui as success → stays
green).

## Impact
- ~1 full build + 146-shard matrix eliminated per merged PR (the
`push:main` duplicate).
- Idle-period regression runs (no merges) skipped instead of running the
full matrix.
- No coverage loss: merge queue still gates every merge; cache stays
warm.

## Not in this PR (deferred per discussion)
- `db-testing.yml` / `unit-tests.yml` `push:main` removal (cheaper,
later).
- ENT merge-queue adoption.
- OSS→ENT dispatch (semgrep / endpoint-auth-probe) gating.

Paired ENT change (regression gate): openobserve/o2-enterprise
`ci/trim-redundant-runs`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

.github/workflows/api-testing.yml

@@ -1,9 +1,10 @@
 name: API Integration Test
 
 on:
-  push:
-    branches:
-      - "main"
+  # No `push: main` trigger: the merge_group (merge-queue) run already builds and
+  # tests the exact commit that lands on main, so a post-merge push run would be a
+  # full-suite duplicate. The merge_group run is now the trusted main-ward build and
+  # also warms the shared Rust cache (see save-if on the rust-cache step below).
   pull_request:
     branches:
       - "**"
@@ -89,7 +90,9 @@ jobs:
         with:
           cache-on-failure: true
           shared-key: backend-debug-mimalloc
-          save-if: ${{ github.ref == 'refs/heads/main' }}
+          # Save the shared cache on merge_group runs (the main-ward build now that
+          # push:main is gone); keep the main-ref check for any non-PR fallback.
+          save-if: ${{ github.event_name == 'merge_group' || github.ref == 'refs/heads/main' }}
 
       - name: Install Protoc
         run: |

.github/workflows/playwright.yml

@@ -1,9 +1,10 @@
 name: Playwright UI Tests
 
 on:
-  push:
-    branches:
-      - "main"
+  # No `push: main` trigger: the merge_group (merge-queue) run already builds and
+  # tests the exact commit that lands on main, so a post-merge push run would be a
+  # full-suite duplicate. The merge_group run is now the trusted main-ward build and
+  # also warms the shared Rust cache (see save-if on the rust-cache step below).
   pull_request:
     branches:
       - "**"
@@ -114,7 +115,9 @@ jobs:
         with:
           cache-on-failure: true
           shared-key: playwright-release-ci
-          save-if: ${{ github.ref == 'refs/heads/main' }}
+          # Save the shared cache on merge_group runs (the main-ward build now that
+          # push:main is gone); keep the main-ref check for any non-PR fallback.
+          save-if: ${{ github.event_name == 'merge_group' || github.ref == 'refs/heads/main' }}
 
       - name: Install Protoc
         run: |

.github/workflows/playwright_regression.yml

@@ -2,7 +2,9 @@ name: Playwright Regression Tests
 
 on:
   schedule:
-    - cron: '30 3,7,11,15,19,23 * * *'  # Every 4 hrs from 9 AM IST (3:30, 7:30, 11:30, 15:30, 19:30, 23:30 UTC)
+    # Daytime IST only (9 AM, 1 PM, 5 PM, 9 PM IST) — the 1 AM and 5 AM IST slots were
+    # dropped: main rarely moves overnight, so those runs mostly re-tested unchanged code.
+    - cron: '30 3,7,11,15 * * *'  # 9 AM, 1 PM, 5 PM, 9 PM IST (3:30, 7:30, 11:30, 15:30 UTC)
   workflow_dispatch:
 
 concurrency:
@@ -45,9 +47,59 @@ env:
   UPLOAD_TO_TESTDINO: ${{ vars.UPLOAD_TO_TESTDINO || 'false' }}
 
 jobs:
+  # Skip the whole suite when main has not moved since the last regression run.
+  # Scheduled runs fire every 4h; if nothing merged in between, rebuilding and
+  # re-running the full matrix on identical code is pure waste. Manual dispatch
+  # always runs.
+  check_main_changed:
+    timeout-minutes: 5
+    name: check_main_changed
+    runs-on: ubuntu-latest
+    permissions:
+      actions: read  # for `gh run list` (last run's head_sha)
+    outputs:
+      should_run: ${{ steps.decide.outputs.should_run }}
+    steps:
+      - id: decide
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          # Manual dispatch always runs.
+          if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
+            echo "should_run=true" >> "$GITHUB_OUTPUT"
+            echo "::notice::Manual dispatch — running regardless of main changes."
+            exit 0
+          fi
+          # On a scheduled run github.sha is main's tip, so no checkout is needed.
+          CURRENT_SHA="${{ github.sha }}"
+          # head_sha of the most recent completed previous run of THIS workflow on
+          # main, excluding the current run. Limit 50 so the current run is always in
+          # the window even under bursty history. On any failure LAST_SHA stays empty
+          # and we fall through to should_run=true (fail-safe: run rather than skip).
+          LAST_SHA=$(gh run list \
+            --workflow "${{ github.workflow }}" \
+            --branch main \
+            --limit 50 \
+            --json databaseId,headSha,status \
+            --jq "[.[] | select(.databaseId != ${{ github.run_id }} and .status == \"completed\")] | .[0].headSha" \
+            2>/dev/null || echo "")
+          if [ -z "$LAST_SHA" ]; then
+            echo "::warning::Could not determine last run SHA (no prior completed run, or gh query failed) — defaulting to run."
+          fi
+          echo "Current main SHA: ${CURRENT_SHA:-<none>}"
+          echo "Last run SHA:     ${LAST_SHA:-<none>}"
+          if [ -n "$LAST_SHA" ] && [ "$CURRENT_SHA" = "$LAST_SHA" ]; then
+            echo "should_run=false" >> "$GITHUB_OUTPUT"
+            echo "::notice::main unchanged since last regression run ($CURRENT_SHA) — skipping."
+          else
+            echo "should_run=true" >> "$GITHUB_OUTPUT"
+          fi
+
   build_binary:
     timeout-minutes: 45
     name: build_binary
+    needs: [check_main_changed]
+    if: ${{ needs.check_main_changed.outputs.should_run == 'true' }}
     runs-on:
       labels: ubicloud-standard-16
     permissions: