mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 08:09:27 +00:00
Shrinath Rao
8a46a836c1
## Problem A **successful** AI review shows up as a **red 25-minute CI timeout**. Two plumbing bugs, both proven from run logs: **1. Trace-export hang** (OSS #13875) ``` 10:19:53 AI Code Review complete ← review posted in 147s 10:19:53 Exporting 14 trace spans to OpenObserve... 10:20:53 Trace export failed: Span processor did not complete within 60000 ms Error: The operation was canceled ← job killed 25 min later ``` After the comment is posted, `TRACE.flush()` exports OTel spans. If OpenObserve is slow/unreachable, `forceFlush()` times out and throws, so `shutdown()` never runs — the `BatchSpanProcessor` timer + exporter sockets stay open and keep Node alive until the job's 25-min timeout. **2. Update-404** (OSS #13871) ``` Updating existing review comment #5336607219 Fatal error: gh api ... PATCH ... gh: Not Found (HTTP 404) ``` If the captured review comment is deleted mid-run, the PATCH 404s and fatal-errors the whole run. ## Fix (plumbing only — review logic untouched) 1. Bound `forceFlush()` **and** `shutdown()` against a 15s deadline (deadline timer `.unref()`'d), always run `shutdown()` in a `finally`, and **`process.exit()` explicitly** after flush so lingering OTel handles can't keep the process alive. 2. On a 404 from the comment PATCH, **fall back to posting a fresh comment**. ## Verified - `node --check` passes. - Runtime repro with an **unreachable trace endpoint**: the process exits cleanly (code 1) instead of hanging to the 25-min job timeout. ## Review feedback (addressed) - 🟡 **Flush deadline no longer hardcoded to 15s.** It's now `Math.max(traceExportTimeoutMs() + 5_000, 15_000)`, so the outer race sits above the configured `forceFlushTimeoutMillis` — a healthy-but-slow OpenObserve gets its full `O2_TRACE_EXPORT_TIMEOUT_MS` window (default 60s) and the precise inner timeout fires first. **Behavior note:** an unreachable endpoint now exits in ~65s at the default config instead of 15s (still well under the 25-min cap); lower `O2_TRACE_EXPORT_TIMEOUT_MS` for a faster exit. - 🔵 **Accurate comment-span attribution.** `updateReviewComment`/`postReviewComment` now return the real `{action, id}`, so the `github.pr.review_comment` span records `create` + the new comment id on a 404 fallback instead of the stale `update` + deleted id. Engine logic in this change is identical across OSS/ENT. (Note: the ENT branch has since merged the OpenObserve Code Review branding from `main`, so the full file is no longer byte-identical to OSS — only the plumbing touched here is.) No change to what the reviewer finds or decides.
AI Code Review (scripts/ai-review)
Automated PR review that posts a single consolidated comment on every pull request. Driven by
.github/workflows/ai-code-review.yml → run-review.mjs, using DeepSeek-V4-Pro via opencode.
The posted comment is branded OpenObserve Code Reviewer. Branding is cosmetic + identity only — it does not change what the reviewer finds or decides.
How it works (one paragraph)
On each PR the workflow fetches the diff, filters noise, picks a risk tier (trivial / lite /
full), runs up to five read-only specialist agents (security, code-quality, performance,
documentation, release) in parallel, and a coordinator consolidates their findings into one
comment with an approve/needs-work decision. The comment PATCHes itself in place on re-review.
Full engine details live in the code; this README covers only setup, activation, and rollback
of the branded identity.
Identity: how the comment gets its author
The comment is posted with whatever token run-review.mjs sees as GH_TOKEN:
- Branded — a GitHub App installation token (comment authored by OpenObserve Code Reviewer with its avatar).
- Fallback —
github.token(comment authored bygithub-actions[bot]).
The workflow mints the App token via actions/create-github-app-token, gated on the
OO_REVIEWER_APP_ID repo/org variable:
- id: app_token
if: ${{ steps.skip.outputs.skip_review == 'false' && vars.OO_REVIEWER_APP_ID != '' }}
continue-on-error: true # never breaks CI if minting fails
uses: actions/create-github-app-token@<pinned-sha> # v2.2.2
with:
app-id: ${{ vars.OO_REVIEWER_APP_ID }}
private-key: ${{ secrets.OO_REVIEWER_APP_PRIVATE_KEY }}
# ...
env:
GH_TOKEN: ${{ steps.app_token.outputs.token || github.token }}
This is fail-safe at every stage:
| Situation | What happens |
|---|---|
| Variable unset | Token step skipped → posts as github-actions[bot] |
| Variable set, App not installed / key wrong | Mint fails, continue-on-error → empty token → || github.token → posts as github-actions[bot] |
| Variable set, App installed | Mint succeeds → posts as OpenObserve Code Reviewer |
Provisioning the GitHub App (one-time)
- Create a GitHub App (org-owned preferred; personal works but must be set to
Any account to install on org repos). Name:
OpenObserve Code Reviewer; upload the brand logo as the avatar; uncheck Webhook → Active (no server, no events). - Permissions (minimum): Pull requests Read & write, Contents Read-only, Metadata Read-only. Do NOT grant Contents write — the review agents are read-only by design and the token must not be write-capable.
- Install the App on
openobserveando2-enterprise. - Generate a private key (
.pem) and note the numeric App ID. - Set on both repos (or at org level), under Settings → Secrets and variables → Actions:
- Variable
OO_REVIEWER_APP_ID= the App ID (this is the on/off gate). - Secret
OO_REVIEWER_APP_PRIVATE_KEY= full contents of the.pem.
- Variable
Via CLI:
gh variable set OO_REVIEWER_APP_ID --body "<APP_ID>" --repo openobserve/openobserve
gh secret set OO_REVIEWER_APP_PRIVATE_KEY < <key.pem> --repo openobserve/openobserve
gh variable set OO_REVIEWER_APP_ID --body "<APP_ID>" --repo openobserve/o2-enterprise
gh secret set OO_REVIEWER_APP_PRIVATE_KEY < <key.pem> --repo openobserve/o2-enterprise
Rollback
Delete the OO_REVIEWER_APP_ID variable on the repo(s). The next review instantly reverts to
github-actions[bot]. No code change, no revert needed.
Skipping a review
Put any of these in the PR body or a comment: [skip ai review], [skip review], [skip ci],
or break glass.
Notes
actions/create-github-app-tokenis pinned to a full commit SHA (it receives the App's private key). Bump the pin deliberately after review — same discipline as theOPENCODE_VERSIONpin.- The App installation's granted permissions bound the token; keep the install at the minimum above so a mis-scoped install can't hand the reviewer a write-capable credential.
- OSS (
openobserve) and ENT (o2-enterprise) run identical engine files — keep them in sync.