The on-call engine reaches notify() through a registered sender slot. Pages,
escalations, manual escalates and handoffs are Firing with the response id
as the platform key; coverage gap, verdict and test pages are
Informational, so platforms skip them. Email and SNS destinations now get
pages (D9), and a generic webhook keeps today's page JSON (D1). Nothing is
sent on acknowledge or resolve (strict D8, D4). oncall_policies gains a
template column (m20261210_000001) with validation and replication, and a
startup line names the policies whose email or SNS destinations now get
pages.
A location down is sent as Informational with no platform key, so
PagerDuty, Opsgenie and ServiceNow skip it with a logged reason (strict
D8). Every other destination gets the location card, and a generic webhook
keeps today's text body.
D8 is strict: a platform resolve goes out only where a module sends a
recovery today. Incident and on-call Resolved are now a platform skip, and
the incident resolve hook, resolve_platform_destinations and its guard are
removed. Incidents keep the keyed trigger.
Review pass 1: incidents pass "[{event}] {title}" as the subject again,
and the subject rule is a pure function with a matrix test. Alerts set
cluster_name. The rename migration exempts only the exact seeded names, so
a root user's o2_default_foo in the default org is renamed. PlatformNoAck
is removed. The resolve answer mapping and the default_templates miss cache
have tests. A non-alert destination returns Ok(false) from the anomaly
sender again, as before this branch.
Synthetics checks gain a template column (m20261201_000001) with save
validation and replication. CheckNotification carries the template and the
event, filled from the alerting decision: firing for failed and error,
resolved for recovered, informational for degraded, flaky and certificate
notices. Each run sends through notify() with the compiled synthetics card,
today's subject line as its title and the error in a code fence. A generic
webhook keeps today's {"text"} body. Platforms get a trigger keyed by the
check id and one protocol resolve on the recovered branch; notices skip
platforms. Private location down sends through the same path keyed by the
location id; nothing is sent when it returns.
Downtimes send through notify(): the seeded prebuilt_downtime pair is the
compiled default per transport, any user template applies, and platform
destinations are skipped with the skip named in the result. The D3 sniff,
the private renderer, build_email_html, build_plain_text and
dispatch_notification are deleted. The cooldown, mute and claim rules are
unchanged.
Every incident send carries kind incident, event firing and the incident
id as the platform key. The template order is default_templates.incident,
then the alert's template, then the destination template, so an incident
override that does not fit a destination's transport falls through. Merged
destinations and external subjects use the incident template too.
When update_status_as resolves an incident, each PagerDuty, Opsgenie or
ServiceNow destination in the incident's destination merge gets one
protocol resolve, once, with errors logged and never retried (D8). No other
destination receives a message (D4). A second resolve of an incident that
is already resolved sends nothing. send_templated_to_destination had no
caller left and is deleted.
The anomaly sender calls notify(). An anomaly renders the compiled anomaly
card, or the config's template override, and now reaches email and SNS
destinations (D9). A generic webhook keeps today's JSON until a user
template applies or payload_format says rendered (D1). PagerDuty, Opsgenie
and ServiceNow get a trigger keyed by the anomaly id; the gated recovery
sends them one protocol resolve and sends the resolved card elsewhere (D8).
The scheduler's cooldown, budget, cursor hold and recovery gate are
unchanged.
Templates gain applies_to, stored in its own column. Saving checks the
values and refuses a narrowing that drops a kind an anomaly config, an
alert or the org default still binds. The o2_default_ prefix is reserved;
a migration renames user rows with a reserved name to user_{name} and the
alerts that named them follow.
The default_templates org key is settable through settings v2 and checks
each name. The template list filters by applies_to and type, GET shows
used_by, and delete refuses a template that is still used. Preview and test
send take a kind and return variables_for(kind). anomaly_detection_config
gains a template column.
Add NotificationKind, NotificationEvent and the platform rule table.
The notification context carries the kind, the event, the cluster name
and an optional legacy payload. Template precedence takes ordered level 1
candidates, the D2 reserved rule and the transport rule. The
default_templates org key is written at org creation and by the startup
backfill.
notify() resolves the destination and the template, applies the platform
rule (D8), the Jira rule and the legacy rule (D1), sends once and writes
one log line. It never retries and never records. The alert render and
send function moved to notifications/send.rs; the alert loop and the new
kinds both call it, and alert output is unchanged.
A ServiceNow resolve that finds no open record now returns success.
Preview and test send accept a kind.
Loakesh Indiran
Loakesh Indiran
Huaijin
Vamsi Karthik R
Shrinath Rao
Harsh Mahajan
Abhay Padamani
uddhavdave
Manul PatelandManul
ktx-vaidehiandgithub-actions[bot]
Neha Pandgithub-actions[bot]
Sai Nikhil
Mina KaramandClaude Opus 5.5
ktx-akshayandAbhay Padamani
Hengfei Yang
Omkar Kesarkhaneandgithub-actions[bot]
Ashish Kolheandgithub-actions[bot]