## Problem
A **successful** AI review shows up as a **red 25-minute CI timeout**.
Two plumbing bugs, both proven from run logs:
**1. Trace-export hang** (OSS #13875)
```
10:19:53 AI Code Review complete ← review posted in 147s
10:19:53 Exporting 14 trace spans to OpenObserve...
10:20:53 Trace export failed: Span processor did not complete within 60000 ms
Error: The operation was canceled ← job killed 25 min later
```
After the comment is posted, `TRACE.flush()` exports OTel spans. If
OpenObserve is slow/unreachable, `forceFlush()` times out and throws, so
`shutdown()` never runs — the `BatchSpanProcessor` timer + exporter
sockets stay open and keep Node alive until the job's 25-min timeout.
**2. Update-404** (OSS #13871)
```
Updating existing review comment #5336607219
Fatal error: gh api ... PATCH ... gh: Not Found (HTTP 404)
```
If the captured review comment is deleted mid-run, the PATCH 404s and
fatal-errors the whole run.
## Fix (plumbing only — review logic untouched)
1. Bound `forceFlush()` **and** `shutdown()` against a 15s deadline
(deadline timer `.unref()`'d), always run `shutdown()` in a `finally`,
and **`process.exit()` explicitly** after flush so lingering OTel
handles can't keep the process alive.
2. On a 404 from the comment PATCH, **fall back to posting a fresh
comment**.
## Verified
- `node --check` passes.
- Runtime repro with an **unreachable trace endpoint**: the process
exits cleanly (code 1) instead of hanging to the 25-min job timeout.
## Review feedback (addressed)
- 🟡 **Flush deadline no longer hardcoded to 15s.** It's now
`Math.max(traceExportTimeoutMs() + 5_000, 15_000)`, so the outer race
sits above the configured `forceFlushTimeoutMillis` — a healthy-but-slow
OpenObserve gets its full `O2_TRACE_EXPORT_TIMEOUT_MS` window (default
60s) and the precise inner timeout fires first. **Behavior note:** an
unreachable endpoint now exits in ~65s at the default config instead of
15s (still well under the 25-min cap); lower
`O2_TRACE_EXPORT_TIMEOUT_MS` for a faster exit.
- 🔵 **Accurate comment-span attribution.**
`updateReviewComment`/`postReviewComment` now return the real `{action,
id}`, so the `github.pr.review_comment` span records `create` + the new
comment id on a 404 fallback instead of the stale `update` + deleted id.
Engine logic in this change is identical across OSS/ENT. (Note: the ENT
branch has since merged the OpenObserve Code Review branding from
`main`, so the full file is no longer byte-identical to OSS — only the
plumbing touched here is.) No change to what the reviewer finds or
decides.
## What & why
Repackages our existing automated PR reviewer under a branded identity —
**OpenObserve Code Reviewer** — and gives its comment a colorful,
attention-grabbing format. This is **purely aesthetic + repackaging**.
> [!IMPORTANT]
> The review **engine is frozen**. The 5 specialist agents, coordinator
judgment/rubric, risk tiering, XML finding contract, re-review
PATCH-in-place logic, total-outage guard, DeepSeek/opencode wiring,
read-only security model, and OTel tracing are all **unchanged**. Only
*who signs the comment* and *how it looks* change.
## The two things that move
**1. Identity (fail-safe).** The workflow mints a GitHub App
installation token via `actions/create-github-app-token`, **gated on the
`OO_REVIEWER_APP_ID` variable**. If the app isn't configured, it skips
and falls back to `github.token` (posts as `github-actions[bot]`,
exactly as today). So this PR is **safe to merge before the app exists**
and auto-activates once the secrets are provisioned.
**2. Presentation.** The coordinator now emits:
- Decision as a native GitHub **callout box** — 🟢 `[!TIP]` approved · 🔵
`[!NOTE]` approved-with-comments · 🟡 `[!WARNING]` minor · 🔴 `[!CAUTION]`
significant.
- Rebranded heading `## 🔎 OpenObserve Code Review`.
- Per-category glyphs 🔒⚡🧩📝📦 on each finding line.
`sanitizeReviewBody` normalizes the heading (tolerates a missing emoji /
legacy heading, idempotent on re-review) and the deterministic fallback
renderer gets the same cosmetic treatment.
## To activate the brand (one-time, org admin)
Register a GitHub App "OpenObserve Code Reviewer", install on this repo,
then set:
- Variable `OO_REVIEWER_APP_ID` (numeric App ID — this is the gate)
- Secret `OO_REVIEWER_APP_PRIVATE_KEY` (the app's `.pem`)
Remove the variable to instantly roll back to `github-actions[bot]`.
## Verification
- JS parses; both workflows valid YAML.
- Engine files byte-identical with the o2-enterprise counterpart PR.
- 6/6 sanitizer heading cases pass (fresh, no-emoji, legacy, preamble,
idempotent, trailing-words).
- This PR's own review comment will preview the new format.
Paired with the identical o2-enterprise PR (same branch name).
Design at: alerts_2.md
Per-group and per-series alerting, SLO measurement, and the UI for both.
## Alerts
**Multi-alerts (Feature 3).** A grouped alert can opt in to evaluating
and paging **per group** rather than collapsing to one verdict. Opt-in
only (`multi_alert`), so no existing alert changes behaviour: the flag
cannot be present in JSON written before it existed. Per-group state,
transitions, silence fingerprints, disappearance/reaping and a
cardinality cap, with the count-gate rules that make "most severe group"
and the legacy collapsed level provably the same verdict.
**Per-series alerting for PromQL.** The same feature for the metrics
family, where a group is one returned **series** keyed by its full label
set — that is a series' identity in Prometheus, and the expression's own
`by (…)` clause is already where the label set is chosen, so a second
picker could only disagree with it. One additive nullable column; NULL
means off, so stored alerts are untouched.
**Alert detail page**, replacing the row-click side panel: stat strip,
evaluation chart with threshold marklines, per-group table and history.
Simple alerts get a real history too — evaluations (one record per run)
by default, level changes alongside.
## SLOs
Count / time-slice / alert SLIs, error-budget and burn-rate arithmetic,
rolling windows, per-group slices with an exact rollup, coverage gating,
backfill, and SLO-based alerts.
Two capabilities the model could not express, both added tests-first:
- **Metrics-native counting** (`CountSource::PromQl`) — counter metrics
have no rows for a predicate to classify, so "good" exists only as
arithmetic between series, and correct counter arithmetic is
`increase()`.
- **Freshness semantics** (`absent_is_bad`) — for a pipeline SLO,
silence IS the failure, which the default gap rule can never report. A
*failed* query still freezes rather than pages; only a query that PROVED
emptiness counts as bad.
**SLO form**: live good/bad preview so a wrong predicate is caught
before saving rather than days later, real query editors with field
autocomplete, folders shared with alerts, and a move endpoint.
## Notes for review
- The SLO and composite features are **deferred (TODO)**:
`ZO_SLO_ENABLED` stays default **false** and the SLOs menu entry is
hidden until they ship; the SLO backend and UI remain in the tree behind
the flag, and composites stay pure logic + tests.
- One migration (schema 61): a nullable column for the PromQL opt-in. No
backfill.
- Two fixes outside the feature that it surfaced: query-editor
completions were duplicated per mounted editor (Monaco registers
providers per *language*), and the alert detail chart had no PromQL
branch, so it charted row counts against the series-count gate.
- Lands together with the matching o2-enterprise PR — `QueryType::Slo`
is new here and the enterprise dedup matches are exhaustive over it.
- `ratelimit::test_add_batch_missing_rule_id` fails intermittently on
`main` as well; unrelated to this branch.
---------
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
Co-authored-by: hengfeiyang <hengfei.yang@gmail.com>
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
chore: update package.json scripts for improved type-checking
fix: adjust frontmatter regex to handle BOM characters
refactor: streamline error handling in dashboard data conversion
style: clean up comments and formatting in various files
Shrinath Rao
Harsh Mahajan
udev
Prabhat Sharmaandgithub-actions[bot]
Abhay Padamani
Aswani Karteek Yadavilli