mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 16:09:05 +00:00
ea2e2131bdc808b829a1dc3b73abce1cd1737708
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Subhra264andgithub-actions[bot]
|
d71f1943e4 |
feat(alerts): recover a firing once, and let every consumer read the same event (#14837)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
 Ashish Kolhe
|
58c7dafa55 |
fix: notification templates v2 — rich content templates + alert chart images (#13640)
## Summary
Two coupled alerting features:
**1. Notification Templates v2 ("content" templates)**
- New `kind: content` template type: a structured spec (title, body
markdown, fields, links, rows, per-channel title overrides,
severity-filtered visibility) rendered natively per channel — existing
`custom` (raw body) templates are untouched and byte-identical on the
send path.
- Per-channel renderers: Slack Block Kit, Teams (Adaptive Card +
MessageCard), Discord embeds, PagerDuty Events v2, Opsgenie, ServiceNow,
SNS, HTML/text email, and a stable machine `webhook` envelope.
- Escaping is renderer-owned (mrkdwn entities, HTML encoding, markdown
stripping) with a shared URL-scheme allowlist; golden-fixture tests per
channel.
- Template editor UI: guided content form (fields/links row tables,
compact rows config, channel title overrides), live preview panel with
per-channel preview + payload tab, test-send (rate-limited,
`[TEST]`-marked).
- Migration `m20260802_000001_add_template_kind`; prebuilt default
template + org backfill.
**2. Alert chart images (stateless, storage-free)**
- Alerts can attach a metric-history chart PNG rendered from the
triggers self-reporting stream (`actual_value` per evaluation), with
warn/crit threshold lines and the firing point appended.
- Zero storage: the chart URL *is* the data — deflate+base64url payload
signed with truncated HMAC-SHA256 (org-bound, expiry embedded), rendered
on fetch by the HTTP-terminating node (`GET
/api/v2/{org}/alerts/charts/render`). Signature verified before inflate
(decompression-bomb guard), uniform 404s, per-node render semaphore +
per-minute query budget + small LRU.
- Renderer: plotters + embedded DejaVu Sans (Bitstream Vera license
included), gap-broken line segments so sparse evaluation history doesn't
draw false plateaus.
- Channel delivery: Slack image block, email inline CID (no remote
fetch), Discord multipart attachment, Teams/PagerDuty image URLs;
per-destination opt-in via template chart flag.
- Slack layout live-A/B'd on a real channel: content + buttons/chart
split across two same-colored attachments so the severity stripe spans
everything while staying under Slack's "Show more" fold; clickable link
title; "View in OpenObserve" keeps a reserved primary-button slot.
- Slack image-delivery fallback: Slack validates image URLs server-side
and rejects the ENTIRE message with `400 invalid_attachments` when its
proxy can't fetch the chart (VPN-only/private `ZO_WEB_URL`) —
live-verified. On that rejection the send strips the image blocks and
resends once (alert always delivers), sets a process-wide flag so later
sends pre-strip (reachability is deployment-wide, not per-destination),
and re-probes hourly. Zero config; observable via `slack_image_rejected`
/ `slack_image_pre_stripped` metric events.
## Testing
- 147 notification unit tests (resolve/render/preview/test-send incl.
per-channel goldens and hostile-input/injection coverage), 8 chart
payload tests, renderer PNG tests.
- Live-verified end to end: Slack (Slackbot chart fetches ~100ms,
cache-immutable semantics), SES email inline image, multi-severity
firing scenario (warning→critical) with correct threshold lines.
- FE: AddTemplate/ContentTemplateForm/preview specs; UX audit pass with
fixes.
- `cargo clippy -D warnings` clean in both OSS and enterprise
arrangements.
## Enterprise pairing
Must merge together with openobserve/o2-enterprise#2353 (workspace
twin-manifest mirror for the new OSS deps).
|
||
|
Subhra264
|
da5c9a9304 |
fix: pagerduty integration bugs (#12975)
## Fix PagerDuty prebuilt destination
### Problem
PagerDuty Events API v2 requires the integration key as `routing_key`
**in the JSON body**, but it never got there:
- Template used `{integration_key}` — not a real server variable → sent
literally → PagerDuty rejects payload. `severity`/`source` hardcoded.
- UI stored integration key only in an `X-Routing-Key` header, which
PagerDuty ignores.
- Severity stored as `credential_severity`, never referenced by the
template.
- "Test" passed only via client-side substitution, diverging from the
real server send path → Test could pass while real alerts failed.
### Fix
Route everything through the alert engine's existing endpoint-metadata
substitution so real sends and Test/Preview render identical, valid
payloads.
- **Template body**: corrected to PagerDuty v2 shape with substitutable
`{routing_key}`, `{severity}`, `{source}` (`prebuilt-destinations.json`,
Rust fallback, web `pagerduty.ts`).
- **Fields now used**: new `generatePrebuiltMetadata()` writes
`routing_key` (=integration key), `severity`, `source` into destination
metadata on save → server fills them at send time. Dropped useless
`X-Routing-Key` header; edit-restore reads key from metadata (legacy
header kept as fallback).
- **Test uses real template**: Test/Preview sample data now supplies
`routing_key`/`source` → client renders the same body the server sends.
- **Stale templates**: `ensure_system_templates()` previously skipped
already-existing prebuilt templates → existing installs kept the broken
body. Now refreshes system-managed templates whose stored body has
drifted from shipped source.
### Verification
- Rust: `config` + `openobserve` binary compile; prebuilt loader tests
pass (10/10). Added `PartialEq`/`Eq` to `TemplateType` for drift check.
- Updated header unit test + added `generatePrebuiltMetadata` tests. JS
suite not runnable locally (pre-existing `ERR_REQUIRE_ESM` in vitest
setup, unrelated) — run in CI.
### Note
Out of scope: Test always uses `prebuilt_pagerduty` template name, not a
per-destination custom template override.
|
||
|
Subhra264
|
452b7c00db |
fix: service now and opsgenie integration bugs (#12000)
Addresses https://github.com/openobserve/openobserve/issues/11930 (1st and 2nd issues) |
||
 Ashish Kolheandgithub-actions[bot]
|
e52dacb4b2 |
fix: alert destinations (#10301)
### **PR Type**
Enhancement, Tests
___
### **Description**
- Support backend-managed prebuilt templates
- Cache and fetch templates with async preview
- Rename templates prefix to `prebuilt_`, update tests
- Add backend ensure_system_templates & template reuse
___
### Diagram Walkthrough
```mermaid
flowchart LR
BL["Backend load_prebuilt_config & builtins"]
Endpoint["GET /api/{org}/alerts/templates/system/prebuilt"]
FE["Frontend fetchSystemTemplates()"]
Cache["systemTemplatesCache in memory"]
Generate["generatePreview/testDestination"]
UI["UI preview/test buttons"]
BL -- "provides templates" --> Endpoint
Endpoint -- "called by" --> FE
FE -- "stores in" --> Cache
Cache -- "used by" --> Generate
Generate -- "triggers" --> UI
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14
files</summary><table>
<tr>
<td><strong>usePrebuiltDestinations.ts</strong><dd><code>Implement
fetchSystemTemplates and caching logic</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-2c90b706aae7486541bbee80d2c67c9de18504b90e1d9618ba130e017d0a5246">+195/-28</a></td>
</tr>
<tr>
<td><strong>slack.ts</strong><dd><code>Rename slack template to
`prebuilt_slack`</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-f7068d6fc979cc1fbc6e2bc562be53f5453fc874b367c1688adf70c0896aac62">+41/-45</a>
</td>
</tr>
<tr>
<td><strong>email.ts</strong><dd><code>Rename email template and remove
CC/subject</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-3e756aefde39bd7c780066f6ee6e22df1f2846f5e0013fd6ec568a98dd54f64b">+18/-16</a>
</td>
</tr>
<tr>
<td><strong>servicenow.ts</strong><dd><code>Rename ServiceNow template
to `prebuilt_servicenow`</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-3790cdccf9f6221cf55c86664b4cbfb65d20f3da9db719167898b8b76b698a2f">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>alert_templates.ts</strong><dd><code>Add
`get_system_templates` API call</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-fd8c4260abc3df601e4331d36b0c211eca568ee70139ad90c87247bc552fe4e2">+3/-0</a>
</td>
</tr>
<tr>
<td><strong>prebuilt_loader.rs</strong><dd><code>Add
get_prebuilt_template and fallback loader</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-d75e85b239701cb3c9e85c15a8a5cd8d23120a4b5ba0a61003c0df1f4854b74d">+345/-10</a></td>
</tr>
<tr>
<td><strong>destinations.rs</strong><dd><code>Implement
ensure_prebuilt_template & reuse logic</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-c06292c108cc15b585a0e3318fd15d3e86c1f1a0f001ab84fff85facefd3c4d3">+420/-1</a>
</td>
</tr>
<tr>
<td><strong>templates.rs</strong><dd><code>Add ensure_system_templates
initialization</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-975d7665f1439ca49810c28554f85e513ff3a4c8b849f2bb1966536230951464">+86/-0</a>
</td>
</tr>
<tr>
<td><strong>templates.rs</strong><dd><code>Expose GET system prebuilt
templates endpoint</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-b08e274a64cc935b0cec402a04271a872b1586027a77dc0ccb84c597c3960f61">+54/-0</a>
</td>
</tr>
<tr>
<td><strong>AddDestination.vue</strong><dd><code>Update form logic for
prebuilt vs custom</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-61b295458e13049a538f5014124855d58dce664dfbd644b57a964a37a07b7a3f">+123/-66</a></td>
</tr>
<tr>
<td><strong>DestinationTestResult.vue</strong><dd><code>Improve test
error message handling</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-1fa17c0e52becd683dec1bbee12aab6823b28c1bbedca0d5039d688527f826e5">+29/-2</a>
</td>
</tr>
<tr>
<td><strong>AlertsDestinationList.vue</strong><dd><code>Add custom
destination label logic</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-e9d05ba6c071c0e2f5d9ef7acdef757f79910abd6897b29e27b069a10d4ba8de">+15/-2</a>
</td>
</tr>
<tr>
<td><strong>PrebuiltDestinationForm.vue</strong><dd><code>Comment out
unsupported email fields</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-606eb6f7e6e43518cc9b1f3194097c52acf3df0df9ab5945274196d7221035a0">+3/-2</a>
</td>
</tr>
<tr>
<td><strong>pagerduty.ts</strong><dd><code>Rename PagerDuty template to
`prebuilt_pagerduty`</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-a216d76143343db3aec8efe02596be53b438813aa2aa8af2fb1080e17f5540aa">+2/-2</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>5
files</summary><table>
<tr>
<td><strong>templates.spec.ts</strong><dd><code>Update tests to use
`prebuilt_` prefix</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-262b3003a3045192e0aaccddebe76bdb29fcd68655e8bacfd486e50575a14801">+9/-9</a>
</td>
</tr>
<tr>
<td><strong>types.spec.ts</strong><dd><code>Adjust PrebuiltConfig tests
for new prefix</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-67fc167ebe4d6a433579ba9850a6d7ce3a83d97bebe410521f85775a948595fc">+4/-4</a>
</td>
</tr>
<tr>
<td><strong>alertDestinationsPage.js</strong><dd><code>Update selector
for test result element</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-57e5c53b4ef998b3d8708149620ff94dcc46807927f1a3a9e9f28ed3ebe25b0a">+3/-1</a>
</td>
</tr>
<tr>
<td><strong>alerts-destinations-prebuilt.spec.js</strong><dd><code>Add
random name for Slack destination test</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-0295825c868c1d1ed5cc194ba04843a7d5f4e58decff2593c7d9bd5b29ec33a4">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>discord.spec.ts</strong><dd><code>Update Discord tests for
new prefix</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-21a02fd01b42452d1f13140715e5f81146b7a62d61d4b899deea4f30be06284e">+2/-2</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Route for system prebuilt templates
API</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>en.json</strong><dd><code>Add custom destination translation
keys</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+5/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>25 files</summary><table>
<tr>
<td><strong>auth.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+20/-0</a>
</td>
</tr>
<tr>
<td><strong>openapi.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+8/-2</a>
</td>
</tr>
<tr>
<td><strong>main.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+5/-5</a>
</td>
</tr>
<tr>
<td><strong>destinations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-4652ac6be3f7cf720faf5b9c4cb1d5dbab9e2fed608f3abbe295fb270dde54b5">+6/-0</a>
</td>
</tr>
<tr>
<td><strong>de.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-78f24ecb2dbcfc38fdf3ec22b2559a5b40d236ea689d1c55f5d3e87168310264">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>es.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-ac2a2167b8b973e3f776f9fe3b43bef4576470d8ad33f3d7cfaff6edff0e5778">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>fr.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-db80e762b88ddf183820c050cdc6b215e6b02b365b5e73f5152af46b0d26f1a2">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>it.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-2de78074a5f887afddce60d694ead9322e6ab223eb70251c13819f3028bdc70d">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>ja.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-092e49c5f9fbc1b539ab9c2866d108d0ce1d314022b4c294eb4611b285e7bbd3">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>ko.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-50e27ca1b4825a5026f3a251bfcf79c97cab07cfade2ebd512d332453d707c48">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>nl.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-a1706021687ae5d8a1c006c49eab3946174fecd895f9b70a89c2e76d97250906">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>pt.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-38e804f64cb5e0e451849fc84e9b46e53c954a4a4263d51c18665ad45f4b4815">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>tr.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-6f0f131e9e2591480c4c5d1eaffc72d02dc31cbd8d3f90d8f012f08cabac8818">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>zh.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-8818055db58f83230aeea4ebfd4cec31d924e8c47b6f91298e356959ab25339e">+5/-1</a>
</td>
</tr>
<tr>
<td><strong>discord.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-e4703f8270d000f785a48aa220752f303d2330b6e123817310ecb8a60815fb0b">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>email.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-744915b43017fdc102b0e00d157d7deef4b1f23e0e8e5cb483c2f2c9d6af655c">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>index.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-9b9a6bc1e17f1874fda50428263ff607dca3ba94b01f95933f179ba544f6e11a">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>msteams.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-6815fa747fe74c5cd6835f9fce8f037c15489fab95e399e15199a104c0c08061">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>msteams.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-efeec26ed5ec589000448466539dcdb0d7bb918e8a4f80c9df565eaa1e522b5c">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>opsgenie.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-2560842b70b2f74eb3ef2c940e0a55500b3bd5b89a9b5b34e3c0039c57fc1361">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>opsgenie.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-9412abdf9baf6b8c0cc87fa3ea0efec50928399c0112ad695c021040d24d7c3a">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>pagerduty.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-f62fcc0b648144816cef61b1f5512ba5e2bc573237158efbd896d33e779282f2">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>servicenow.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-ada67bb0bbeaf97a2e7c6a21a741d53df941306e6c694d1a1d89cf965403ce85">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>slack.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10301/files#diff-3fa8b2b7534ebec5c2cea1c76c54741c105b64819bb87e6a7f6d3b5a85969740">+2/-2</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
||
  
|
e419449ddb |
feat: implement prebuilt alert destination (Backend) (#10074)
Design at : observability/alert-prebuilt-destinations/alert-prebuilt-destinations-plan.md ## 🤖 Auto-Implementation (Backend) **Feature:** implement prebuilt alert destination **Source Issue:** openobserve#10055 ## Changes - ✅ Backend code (Rust) - ✅ Unit + integration tests - ✅ All quality gates passed ## Related PRs Frontend PR: #FRONTEND_PR_NUMBER (to be linked) Closes openobserve#10055 --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> --------- Co-authored-by: claude-code-bot <claude-code-bot@openobserve.ai> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |