How I run my query
1. nov 26 10:30 - 11:30 - interval 30 mins
2. nov 26 10:30 - 16:30 - interval 60 mins
3. nov 26 10:30 - 16:30 - interval 60 mins -- refresh cache
Cache dir
1. on first run
- has 2 caches 30 mins
2. on 2nd run
- will havve mixed 30mins cache and 60 mins cache
TODO:
- [x] Problem 1:
on step 3
- the interval for that duration is 60 mins
- but the discovery of cache will also include the 30 mins cache
[@mod.rs](file:///Users/lin/o2/openobserve/src/service/search/mod.rs)
`discover_cache_for_query`
- the overwrite will over write the 60 mins cache only. How to address
this problem?
- [x] Problem 2:
- fix: add rbac control only user with permussion to edit stream can
`refresh cache`
Proposed Solution
- On `refresh_cache` run streaming aggs without using cache
- at `streaming/execution.rs` , at the end of steaming aggs delete all
the cache files for the given duration when there are miss matched cache
interval files
- then remove the `streaming_id` from the `GLOBAL_CACHE`
- eg: nov 26 10:30 - 16:30 - interval 60 mins -- refresh cache
- Cache consists of 30mins or 4hr interval files all of them will be
deleted
- 60 mins cache interval files will be overwritten or created a new.
- This way on the next run there will be no query results mismatch when
the query runs again with the newly updated cache, especially delayed
data for the same duration.
- add rbac control to only allow users with edit permissions to stream
to use this feature
## Summary
Implements org-level alert deduplication with semantic field groups for
intelligent alert suppression and batched notifications. This feature
prevents alert storms by deduplicating alerts based on configurable
fingerprints and grouping related alerts together.
### Core Capabilities
- **Semantic Field Groups**: Map field name variations (`hostname`,
`host`, `node`) to canonical dimensions for consistent deduplication
across different data sources
- **Organization-Level Configuration**: Global deduplication settings
with cross-alert support to suppress alerts from different rules sharing
semantic dimensions
- **Alert Grouping & Batching**: Wait-and-collect logic with three send
strategies (`FirstWithCount`, `Summary`, `All`) for batched
notifications
- **Per-Alert Control**: Individual alert deduplication settings with
optional fingerprint field customization
### Features
**Deduplication Modes:**
- Per-alert exact matching using fingerprint-based TTL expiration
- Cross-alert semantic matching using org-defined dimension groups
- Configurable time windows (per-alert override or org-level default)
**Alert Grouping:**
- Batch alerts with same fingerprint before sending notifications
- Three strategies: first alert with count, summary with names, or all
details
- Configurable wait time (1-300 seconds) and max batch size (1-100
alerts)
- Background processor runs every 1 second for expired batches
**Semantic Field Groups:**
- Built-in presets for common fields (host, IP address, service)
- Kubernetes resource groups (cluster, namespace, pod, node, container)
- Field normalization support (lowercase + trim)
- Overlap handling with first-defined precedence
### Configuration Types
- `SemanticFieldGroup`: Field name equivalence definitions with
normalization
- `GlobalDeduplicationConfig`: Org-level settings with semantic groups
and cross-alert dedup
- `DeduplicationConfig`: Per-alert fingerprint fields and grouping
options
### API Endpoints
- `GET /{org_id}/alerts/deduplication/config` - Retrieve org-level dedup
config
- `POST /{org_id}/alerts/deduplication/config` - Set org-level dedup
config
- `DELETE /{org_id}/alerts/deduplication/config` - Delete org-level
dedup config
### Implementation Details
- Pure business logic algorithms in enterprise layer (fingerprinting,
matching, calculations)
- OSS service layer orchestrates database operations with enterprise
delegation
- Feature-gated with `#[cfg(feature = "enterprise")]` for
enterprise-only functionality
- Background job architecture for async batch processing
- Storage: Org config in KV DB at `/alert_config/{org_id}/deduplication`
### Architecture
```
Alert Triggered
↓
Query Evaluation
↓
Deduplication Check (fingerprint + TTL)
↓
Grouping Enabled? → Add to Batch → Wait/Collect → Send Grouped
↓ No
Send Individual Notification
```
---------
Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
# Add Row Template Type Support for Alerts
## Overview
This PR adds support for configurable row template types in alerts,
allowing users to choose between **String** and **JSON** formats for row
templates. This provides flexibility in how alert data is formatted and
sent to destinations.
## Features
### Row Template Type Selection
- Added a new `row_template_type` field to alerts with two options:
- **String**: Traditional string-based templates with variable
interpolation
- **JSON**: Structured JSON templates for programmatic consumption
### Database Layer
- Uses efficient `smallint` (i16) storage: `0` for String, `1` for JSON
- Added migration `m20251118_000001_add_alert_row_template_type` with
backward compatibility (defaults to String)
- Includes support for PostgreSQL, MySQL, and SQLite
### API Layer
- Returns enum string representation (`"String"` or `"Json"`) in API
responses
- Full support in GET, CREATE, and UPDATE alert endpoints
### UI Changes
- Added toggle button in alert creation/editing form to switch between
String and JSON template types
- Dynamic placeholder text based on selected template type
- Persists user's template type selection in the database
## Usage Example
### String Template (Default)
When creating an alert with row template type set to **String**:
```
Template Type: [String] JSON
Row Template: Alert triggered: {alert_name} at {timestamp} Affected user: {user_id} Error count: {error_count}
```
### JSON Template
When creating an alert with row template type set to **JSON**:
```
Template Type: String [JSON]
Row Template: { "alert": "{alert_name}", "timestamp": "{timestamp}", "user": { "id": "{user_id}", "name": "{user_name}" }, "metrics": { "error_count": {error_count}, "severity": "{severity}" } }
```
The JSON template type automatically parses and validates JSON
structure, making it ideal for:
- Webhook integrations that expect JSON payloads
- Structured logging systems
- API endpoints requiring specific JSON schemas
- Downstream systems that need programmatic access to alert data
## Technical Details
### Architecture
- **Database**: Stores as `smallint` (0 or 1) for efficiency
- **Service Layer**: Uses `RowTemplateType` enum throughout application
logic
- **API**: Serializes enum as strings (`"String"`, `"Json"`)
- **Frontend**: TypeScript interface uses string type with toggle UI
component
### Backward Compatibility
- Existing alerts without `row_template_type` default to `String`
- Migration adds column with `DEFAULT 0` ensuring zero downtime
- API handles both old and new alert formats seamlessly
## Testing
- Added unit tests for database migration (PostgreSQL, MySQL, SQLite)
- Tested enum serialization/deserialization
- Verified UI toggle behavior and data persistence
- Confirmed backward compatibility with existing alerts
<img width="1070" height="770" alt="Screenshot 2025-11-19 at 5 46 34 PM"
src="https://github.com/user-attachments/assets/900edb1c-815e-4240-91e2-50a1a582a9b0"
/>
1. we have created header component which will seperate out the header
part from mainlayout
2. introduced custom dark logo feature , it will fallback to light mode
logo if dark mode logo is not there
When `ZO_USAGE_REPORT_TO_OWN_ORG` is enabled (default: true), triggers
are written to both _meta and originating orgs. This caused schema
divergence where some orgs missing fields like 'error', breaking
dashboard queries.
Add ensure_triggers_stream_schema() to auto-sync missing TriggerData
fields before ingestion. Runs once per stream, logs missing fields, and
updates defined_schema_fields gracefully without blocking ingestion.
Fixes queries like: SELECT error FROM triggers WHERE status='failed'
---------
Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com>
https://github.com/openobserve/openobserve/pull/8876 had made
this variable private, resulting in failed enterprise builds.
It also removed one of the imports causing build failure.
This commit reverts those specific changes.
- Fix panic on path canonicalization in check_has_pending_files()
- Implement proper recursive directory checking for parquet files
- Add exponential backoff to draining loop (50ms -> 1s)
- Validate query parameter strictly in enable endpoint to prevent accidental node disablement
These changes address critical robustness issues identified in the code review:
1. Prevents potential panics during shutdown when WAL path is missing
2. Ensures correct pending file detection instead of assuming all subdirectories have files
3. Reduces CPU usage during drain with exponential backoff
4. Prevents unintentional node disablement from invalid/missing parameters
Implements cloud-agnostic graceful shutdown mechanism to support
Kubernetes preStop hooks and other orchestration platforms.
Changes:
- Add IS_INGESTER_DRAINING global flag with atomic operations
- Enhance /enable?value=false to trigger flush and set draining mode
- Add /drain_status endpoint to monitor shutdown readiness
- Modify parquet upload job to prioritize draining mode
- Exit upload loop when all files are uploaded to S3
API Usage:
- PUT /node/enable?value=false - Initiates drain (flushes memory, sets flag)
- GET /node/drain_status - Returns readyForShutdown status
Benefits:
- Eliminates fixed 900s wait in Helm charts
- Cloud-agnostic (works with K8s, ECS, Docker Swarm, systemd)
- Data-safe: ensures all WAL files uploaded before shutdown
# PR Type
Enhancement
# Description
The columns in the Alerts and Pipelines History are augmented to be
sortable. The backend endpoint is updated accordingly too.
---------
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
# PR Type
Feature/Fix
# Description
The pipelines history and alerts history endpoints are now covered by
RBAC. Added basic e2e api test for the same endpoints.
---------
Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
There were few issues with the middleware, this attempts to fix it.
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Changes alert trigger reporting behavior to write to both the originating
organization and `_meta` org by default, with new UI for viewing history.
**Backend:**
- Change `ZO_USAGE_REPORT_TO_OWN_ORG` config default from `false` to `true`
- Update alert history API to query organization's own `triggers` stream
- Add comprehensive backend tests for history API and config
**Frontend:**
- Integrate history metrics into main alerts table (`Total Evaluations`, `Firing Count`)
- Add alert details drawer showing properties and evaluation timeline
- Display formatted timestamps, status badges, and performance metrics
- Support click-outside and ESC key to close drawer
- Add i18n translations for history-related labels
This eliminates cross-org API calls for alert history while maintaining
global monitoring capability in `_meta` org.
This adds a midddleware for blocking the requests for invalid orgs
before they reach the handler level.
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
- Add `x-o2-mcp` header support for MCP request identification in token validator
- Wrap OpenAPI schema types with `inline()` for better documentation generation
- Update AI module imports to use new agent namespace
- fix compilation for `cloud` feature, also return early if user isn't in trial
- Enable service graph by default
# PR type
fix/enchancement
# Description
The pagination issue with alerts and pipeline history is fixed - now
works as normally expected. A few elements of the views were improved
with better choices of representation. A previous PR about styling
changes has removed the Pipelines History related changes partially.
This was recovered and restored to functional state while also taking in
the styling changes.
---------
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
PR introduces below features:
- [x] Reporting triggers to source org
- [x] Alerts Insights for detailed visibility in alerts
Enterprise features:
- [x] Patterns extraction during query - UI and API
- [x] Patterns extraction as part of ingestion process (No Sampling
today)
- [x] Settings to enable pattern extraction
---------
Co-authored-by: omkark06 <omkar@zinclabs.io>
Hengfei Yang
Ashish Kolheandgithub-actions[bot]
ByteBaker
Loakesh Indiran
Huaijin
Harsh Mahajan
Subhra264
Simha G
Abhay PadamaniandKirtan Thakkar
