mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-06 08:09:24 +00:00
Shrinath RaoandClaude Opus 4.8
ad1c7444c6
Follow-up to #12657. Builds out the **entire generation pipeline** (Jobs 2–4) plus access control and failure handling. Everything still runs only with `dry_run=false`, so dry-run/triage behavior is unchanged. ## The full flow (dry_run=false) `/e2e` or dispatch → **triage** → **generate** → **verify/heal (heavy)** → **open test PR** — the PR only ever contains **working, healed tests**, so review is about quality + coverage, not debugging. ## Access control (allowlist) - **Trigger gate:** only the engineering-team allowlist (mirrors `auto-assign-metadata.yml`) can start a run. - **PR-author gate:** generation only runs when the **PR author** is also on the allowlist. A stranger's PR short-circuits the whole AI path in triage (no DeepSeek call) → just a "skipped, external" comment. ## Job 2 — generate Analyst → Architect → Engineer → Sentinel. Checkout is **SHA-pinned + `persist-credentials:false`**; pipeline files (`.opencode`/`opencode.jsonc`) restored from trusted `main` (anti-hijack). **Hand-off validation gate** confirms each agent wrote its artifact (spec + registration) and fails fast with a clear message otherwise. Sentinel verdict is **fail-closed**. A **deterministic, non-LLM** step (`yq`) registers the spec in `playwright.yml`, creates **`autoe2e/<orig-branch>`** stacked on the dev's branch, commits, and force-pushes (idempotent re-runs). **No PR yet.** ## Job 3 — verify/heal (HEAVY) On `ubicloud-standard-16`, mirrors `playwright.yml`: builds + boots a **real OpenObserve**, runs **only the generated spec** (`--workers=4`), runs the Healer to fix it, then an **authoritative final run decides pass/fail (not the LLM)**, Sentinel re-audits, pushes fixes. **Loop gating:** hard `timeout-minutes: 30` on the heal step + `continue-on-error` (a hung healer is killed and the pipeline proceeds to the final run → fails → no PR), job backstop 90 min. ## Job 4 — open the test PR Opens the stacked test PR **only if healed (passing)**, via **`ORG_ADMIN_TOKEN`** so the repo's existing `playwright.yml` re-validates it (a `GITHUB_TOKEN`-created PR wouldn't trigger CI). Sticky comment links the test PR on the source PR. ## Failure handling (never silent, never junk) Any failure is reported **on the source PR by the job where it happens**, with a root cause + run link: - **generate** → agent error / broken hand-off / Sentinel block. - **verify/heal** → build failed / boot failed / **healer timed out (hung)** / still-failing after retries. No PR is opened on failure — only a comment — so the PR never carries broken tests, and the dev is always notified. ## Security / CodeQL SHA-pinned + no-persisted-creds checkouts, allowlist on trigger + author, anti-hijack config restore, deterministic `playwright.yml` edit, and untrusted paths passed via `env:` (no `${{ }}` in shell). CodeQL on this PR: **4 code-injection FIXED**, **3 untrusted-checkout DISMISSED** (allowlist-author false positive — generation runs only on trusted-teammate code, dispatch-only). Note: each push shifts lines, so CodeQL may re-raise the same toctou under new alert numbers — dismiss with the same justification. ## Status - ✅ All 4 jobs implemented + gating + failure handling. - ⚠️ **Untested end-to-end** — Jobs 2–4 only run on a real `dry_run=false` dispatch; the first live run is the shakedown. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>