mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 16:09:05 +00:00
feat: add organization-level alert deduplication with semantic field groups (#9209)
## Summary
Implements org-level alert deduplication with semantic field groups for
intelligent alert suppression and batched notifications. This feature
prevents alert storms by deduplicating alerts based on configurable
fingerprints and grouping related alerts together.
### Core Capabilities
- **Semantic Field Groups**: Map field name variations (`hostname`,
`host`, `node`) to canonical dimensions for consistent deduplication
across different data sources
- **Organization-Level Configuration**: Global deduplication settings
with cross-alert support to suppress alerts from different rules sharing
semantic dimensions
- **Alert Grouping & Batching**: Wait-and-collect logic with three send
strategies (`FirstWithCount`, `Summary`, `All`) for batched
notifications
- **Per-Alert Control**: Individual alert deduplication settings with
optional fingerprint field customization
### Features
**Deduplication Modes:**
- Per-alert exact matching using fingerprint-based TTL expiration
- Cross-alert semantic matching using org-defined dimension groups
- Configurable time windows (per-alert override or org-level default)
**Alert Grouping:**
- Batch alerts with same fingerprint before sending notifications
- Three strategies: first alert with count, summary with names, or all
details
- Configurable wait time (1-300 seconds) and max batch size (1-100
alerts)
- Background processor runs every 1 second for expired batches
**Semantic Field Groups:**
- Built-in presets for common fields (host, IP address, service)
- Kubernetes resource groups (cluster, namespace, pod, node, container)
- Field normalization support (lowercase + trim)
- Overlap handling with first-defined precedence
### Configuration Types
- `SemanticFieldGroup`: Field name equivalence definitions with
normalization
- `GlobalDeduplicationConfig`: Org-level settings with semantic groups
and cross-alert dedup
- `DeduplicationConfig`: Per-alert fingerprint fields and grouping
options
### API Endpoints
- `GET /{org_id}/alerts/deduplication/config` - Retrieve org-level dedup
config
- `POST /{org_id}/alerts/deduplication/config` - Set org-level dedup
config
- `DELETE /{org_id}/alerts/deduplication/config` - Delete org-level
dedup config
### Implementation Details
- Pure business logic algorithms in enterprise layer (fingerprinting,
matching, calculations)
- OSS service layer orchestrates database operations with enterprise
delegation
- Feature-gated with `#[cfg(feature = "enterprise")]` for
enterprise-only functionality
- Background job architecture for async batch processing
- Storage: Org config in KV DB at `/alert_config/{org_id}/deduplication`
### Architecture
```
Alert Triggered
↓
Query Evaluation
↓
Deduplication Check (fingerprint + TTL)
↓
Grouping Enabled? → Add to Batch → Wait/Collect → Send Grouped
↓ No
Send Individual Notification
```
---------
Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
This commit is contained in:
co-authored by
Ashish Kolhe
Shrinath Rao
parent
b3c63ed35b
commit
44011fbebc
@@ -327,8 +327,8 @@ jobs:
|
||||
# Try to get last failed tests from TestDino
|
||||
# Redirect stderr to /dev/null to avoid capturing npm warnings
|
||||
if LAST_FAILED_FLAGS="$(npx tdpw last-failed --cache-id "$CACHE_ID" 2>/dev/null)"; then
|
||||
# Trim whitespace (original working logic - no filtering)
|
||||
LAST_FAILED_FLAGS="$(echo "$LAST_FAILED_FLAGS" | xargs)"
|
||||
# Trim whitespace using parameter expansion (safer than xargs for quoted strings)
|
||||
LAST_FAILED_FLAGS="$(echo "$LAST_FAILED_FLAGS" | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
|
||||
|
||||
if [ -n "$LAST_FAILED_FLAGS" ] && [ "$LAST_FAILED_FLAGS" != "No failed tests found" ]; then
|
||||
echo "::notice::✅ Found failed tests from previous run."
|
||||
|
||||
Reference in New Issue
Block a user