# Nested Functions and Joins - User Guide
## Overview
This guide covers two powerful features that enable sophisticated data
analysis in OpenObserve:
- **Nested Functions**: Combine multiple functions together to create
complex calculations
- **Joins**: Merge data from multiple streams (data sources) based on
common fields
These features work in the **Auto SQL** mode of dashboard panels,
allowing you to write advanced queries without deep SQL knowledge.
---
## What's New
### Why These Features?
**Before:**
- Limited function composition - you could only use basic, single-level
functions
- No way to combine data from multiple streams in one query
- Complex analytics required exporting data and manual processing
**Now:**
- Compose unlimited nested functions with full DataFusion support
- Join multiple streams directly in the dashboard panel editor
- Create sophisticated analytics queries with a visual interface
### Key Benefits
✓ Calculate complex metrics (error rates, percentiles, conditional
aggregations)
✓ Build advanced dashboards without custom code
✓ Support for all DataFusion aggregate and scalar functions
---
## Nested Functions
### What Are Nested Functions?
Nested functions let you combine multiple functions together - using the
output of one function as input to another. Think of it like a cooking
recipe where each step builds on the previous one.
**Simple Example:**
- Basic: `AVG(response_time)` - calculates average response time
- Nested: `ROUND(AVG(response_time), 2)` - calculates average AND rounds
it to 2 decimal places
### Supported Functions
OpenObserve supports **all DataFusion functions**, giving you powerful
analytics capabilities:
**Aggregate Functions** (work across multiple rows):
- `count`, `sum`, `avg`, `min`, `max` - basic aggregations
- `median`, `stddev`, `variance` - statistical functions
- `percentile_cont` - calculate percentiles (e.g., p95, p99)
- `string_agg` - combine text values
**Mathematical Functions**:
- `round`, `ceil`, `floor` - rounding operations
- `abs`, `power`, `sqrt` - mathematical operations
- `log`, `log10`, `log2` - logarithmic functions
- `sin`, `cos`, `tan` - trigonometric functions
**String Functions**:
- `concat`, `upper`, `lower`, `trim`
- `substring`, `length`, `replace`
**Conditional Functions**:
- `CASE WHEN ... THEN ... ELSE ... END` - conditional logic
- `COALESCE` - return first non-null value
📖 **Full Reference**: [DataFusion Aggregate
Functions](https://datafusion.apache.org/user-guide/sql/aggregate_functions.html)
| [DataFusion Scalar
Functions](https://datafusion.apache.org/user-guide/sql/scalar_functions.html)
---
### Using Nested Functions (Step-by-Step)
#### Step 1: Create or Edit a Dashboard Panel
Open your dashboard and create a new panel or edit an existing one. Make
sure you're in **Auto SQL** mode (not Custom SQL).
#### Step 2: Select a Function
In the Y-Axis or column configuration, click **Select Function** to
choose your base function.
<img width="795" alt="Nested Function - Adding Layers"
src="https://github.com/user-attachments/assets/0b6b134e-59b8-48b1-a4ae-52fb4800d772"
/>
In this example, we're using the `Concat` function to combine multiple
fields (namespace and pod name).
#### Step 3: Add Nested Layers
Click the function icon to add another function layer. You can nest
functions multiple times.
<img width="793" alt="Nested Function - Select Base Function"
src="https://github.com/user-attachments/assets/4bd9e03e-1104-4e82-b2ec-a34e6bb64870"
/>
Here we're building a nested function:
- Inner layer: `Avg(severity)` - calculates average severity
- Outer layer: `Ceil()` - rounds up the average to the nearest integer
#### Step 4: Build Complex Expressions
For advanced analytics, you can combine nested functions with
conditional logic using `CASE WHEN` statements.
<img width="1340" alt="Complex Nested Function with CASE WHEN"
src="https://github.com/user-attachments/assets/4c3e3623-bfad-44c4-a760-e10cc94d4a95"
/>
This example shows a sophisticated calculation:
```sql
ROUND(100.0 * SUM(CASE WHEN COALESCE(severity, 0) >= 4 THEN 1 ELSE 0 END) / COUNT(_timestamp), 2)
```
**What this does:**
1. `COALESCE(severity, 0)` - replaces null severity with 0
2. `CASE WHEN severity >= 4 THEN 1 ELSE 0` - returns 1 for high
severity, 0 otherwise
3. `SUM(...)` - counts high severity events
4. Divide by `COUNT(_timestamp)` - calculates percentage
5. Multiply by `100.0` - converts to percentage
6. `ROUND(..., 2)` - rounds to 2 decimal places
**Result**: Percentage of high-severity events, rounded to 2 decimals
#### Step 5: View the Generated SQL
Switch to the **Raw** tab to see the complete SQL query that was
generated from your nested function configuration.
<img width="1388" alt="View Generated SQL"
src="https://github.com/user-attachments/assets/d7f6111a-26c6-4dd0-9e9f-e26092fe6368"
/>
You can review the auto-generated SQL in the Query section at the bottom
of the panel editor. The chart visualization updates in real-time as you
build your nested functions.
---
## Joins
### What Are Joins?
Joins allow you to combine data from two or more different streams (data
sources) based on common fields between them.
**Example:**
**employee_data_join** stream:
```json
[
{"employee_id": 1, "name": "Alice", "department_id": 1},
{"employee_id": 2, "name": "Bob", "department_id": 2},
{"employee_id": 3, "name": "Charlie", "department_id": null},
{"employee_id": 4, "name": "David", "department_id": 1},
{"employee_id": 5, "name": "Eve", "department_id": 3}
]
```
**dept_data_join** stream:
```json
[
{"department_id": 1, "department_name": "HR"},
{"department_id": 2, "department_name": "Engineering"},
{"department_id": 3, "department_name": "Sales"},
{"department_id": 4, "department_name": "Marketing"}
]
```
A join lets you combine these streams to see the department name
alongside each employee, matched by `department_id`.
### Types of Joins
OpenObserve supports three types of joins:
| Join Type | Description | When to Use |
|-----------|-------------|-------------|
| **INNER JOIN** | Returns only records with matches in both streams |
When you need data that exists in both sources |
| **LEFT JOIN** | Returns all records from left stream + matching
records from right | When you want all data from your primary source,
with supplemental info where available |
| **RIGHT JOIN** | Returns all records from right stream + matching
records from left | When you want all data from your secondary source,
with supplemental info where available |
> **Note:** UNION and CROSS joins are not currently supported.
---
### Using Joins (Step-by-Step)
#### Step 1: Access the Joins Section
In your dashboard panel editor, locate the **Joins** section. Click the
**+** button to add a join.
<img width="1725" alt="Access Joins Interface"
src="https://github.com/user-attachments/assets/35db4f28-c526-43a2-bdd1-470e188f53d7"
/>
In this example:
- **Primary stream**: `employee_data_join` (contains employee names)
- We'll join it with `dept_data_join` (contains department information)
- The interface shows available fields from both streams in the left
panel
#### Step 2: Configure the Join
Click on the **Joins** dropdown and select the stream you want to join.
Then configure:
1. **Joining Stream**: Select the stream to join (e.g.,
`employee_data_join`)
2. **Join Type**: Choose from LEFT, INNER, or RIGHT using the visual
toggle
3. **On Stream**: Select the target stream (e.g., `dept_data_join`)
<img width="1725" alt="Configure Join Type"
src="https://github.com/user-attachments/assets/35db4f28-c526-43a2-bdd1-470e188f53d7"
/>
The interface displays: **"Performing a Left Join between
employee_data_join and dept_data_join"**
Notice the visual toggle showing three join types:
- **Left** (selected in this example)
- **Inner**
- **Right**
#### Step 3: Define the Joining Clause
Set up the join condition by specifying which fields to match between
the two streams:
**Clause 1:**
- **Select Field** (Left): Choose a field from the first stream (e.g.,
`department_id`)
- **Select Operator**: Choose comparison operator (usually `=`)
- **Select Field** (Right): Choose the matching field from the second
stream (e.g., `department_id`)
You can add multiple clauses using the **+** button for complex join
conditions (e.g., joining on multiple fields or adding additional
filters).
#### Step 4: View the Results
After configuring the join, the data table will show combined results
from both streams.
<img width="1724" alt="Join Results"
src="https://github.com/user-attachments/assets/94ca9f33-6cfa-4ad1-9b84-bee9db34f912"
/>
In this example:
- **Name** column from `employee_data_join` (Alice, Bob, Charlie, David,
Eve)
- **Department Name** column from `dept_data_join` (HR, Engineering, HR,
Sales)
- Data successfully joined on `department_id`
#### Step 5: Review the Generated SQL
Check the **Query** section at the bottom to see the generated SQL:
```sql
SELECT employee_data_join.name as "x_axis_1",
stream_0.department_name as "x_axis_2"
FROM "employee_data_join"
LEFT JOIN "dept_data_join" AS stream_0
ON employee_data_join.department_id = stream_0.department_id
```
The SQL query shows exactly how OpenObserve is combining your data
streams.
---
---
## Examples
### Example 1: String Concatenation with Nested Functions
**Use Case**: Combine multiple fields into a single display value
```sql
Concat(k8s_namespace_name, '-', k8s_pod_name)
```
This joins namespace and pod names with a hyphen separator.
### Example 2: Statistical Analysis with Nested Math Functions
**Use Case**: Calculate rounded average of a metric
```sql
Ceil(Avg(severity))
```
This calculates the average severity level and rounds up to the nearest
whole number.
### Example 3: Conditional Percentage Calculation
**Use Case**: Calculate percentage of high-severity events
```sql
ROUND(100.0 * SUM(CASE WHEN COALESCE(severity, 0) >= 4 THEN 1 ELSE 0 END) / COUNT(_timestamp), 2)
```
**Breakdown:**
- Handles null values with `COALESCE`
- Counts events where severity >= 4
- Calculates percentage of total events
- Rounds to 2 decimal places
### Example 4: Employee-Department Join
**Use Case**: Show employee names with their department information
```sql
SELECT employee_data_join.name as "Name",
dept_data_join.department_name as "Department Name"
FROM employee_data_join
LEFT JOIN dept_data_join
ON employee_data_join.department_id = dept_data_join.department_id
```
This displays all employees with their department names, showing null
for employees without a department assignment.
### Example 5: Complex Analytics - Joins + Nested Functions
**Use Case**: Calculate error rate per service with deployment
information
```sql
SELECT
logs.service_name,
deployments.version,
COUNT(*) as total_requests,
ROUND((COUNT(CASE WHEN logs.status_code >= 500 THEN 1 END) * 100.0 / COUNT(*)), 2) as error_rate_percent
FROM service_logs as logs
LEFT JOIN deployments ON logs.service_name = deployments.service_name
GROUP BY logs.service_name, deployments.version
ORDER BY error_rate_percent DESC
```
This combines:
- Join between service logs and deployment data
- Nested functions for conditional counting and percentage calculation
- Grouping and sorting for analysis
---
## Tips and Best Practices
### Nested Functions Best Practices
✅ **Start Simple**: Build your function one layer at a time
- Begin with inner function: `AVG(response_time)`
- Add outer layers: `ROUND(AVG(response_time), 2)`
✅ **Test Each Layer**: Verify results before adding complexity
- Test the inner function first
- Add one layer at a time and check output
✅ **Handle Null Values**: Use `COALESCE` to avoid unexpected nulls
```sql
COALESCE(AVG(response_time), 0)
```
✅ **Use Clear Aliases**: Name your calculated fields descriptively
```sql
ROUND(AVG(response_time), 2) as avg_response_ms
```
✅ **Check for Division by Zero**: Protect against zero denominators
```sql
CASE
WHEN COUNT(*) > 0 THEN (SUM(errors) * 100.0 / COUNT(*))
ELSE 0
END as error_rate
```
### Joins Best Practices
✅ **Choose the Right Join Type**:
- **INNER JOIN**: Both streams must have matching data
- **LEFT JOIN**: Keep all records from primary stream
- **RIGHT JOIN**: Keep all records from secondary stream
✅ **Verify Field Compatibility**:
- Match data types (don't join string to number)
- Ensure field names are correct
- Check that values actually match between streams
✅ **Consider Performance**:
- Join on indexed fields when possible
- Test with limited data first
---
## FAQ
### Q: Can I nest more than 2 functions?
**A**: Yes! You can nest as many functions as needed. Example:
```sql
ROUND(CEIL(AVG(COALESCE(response_time, 0))), 2)
```
### Q: What's the difference between LEFT and RIGHT joins?
**A**:
- **LEFT JOIN**: Keeps all records from the first (left) stream
- **RIGHT JOIN**: Keeps all records from the second (right) stream
- Choose based on which dataset you want to preserve completely
### Q: Can I join more than 2 streams?
**A**: Yes! Click the **+** button in the Joins section to add
additional joins. You can chain multiple joins together.
### Q: Do nested functions impact performance?
**A**: Complex nested functions may take longer to compute, especially
on large datasets. Test with small time ranges first and optimize your
expressions.
### Q: Can I use Custom SQL instead of Auto SQL?
**A**: Yes! If you prefer writing raw SQL, switch to **Custom SQL**
mode. However, Auto SQL provides a visual interface that's easier for
building queries.
---
## Additional Resources
📚 **DataFusion Documentation**:
- [Aggregate
Functions](https://datafusion.apache.org/user-guide/sql/aggregate_functions.html)
- COUNT, SUM, AVG, etc.
- [Scalar
Functions](https://datafusion.apache.org/user-guide/sql/scalar_functions.html)
- ROUND, CONCAT, CASE, etc.
🔗 **OpenObserve Resources**:
- [GitHub Repository](https://github.com/openobserve/openobserve)
- [PR #9343 - Feature
Implementation](https://github.com/openobserve/openobserve/pull/9343)
- [OpenObserve Documentation](https://openobserve.ai/docs/)
💬 **Get Help**:
- [GitHub Issues](https://github.com/openobserve/openobserve/issues) -
Report bugs or request features
- [Community
Discussions](https://github.com/openobserve/openobserve/discussions) -
Ask questions and share ideas
---
Design at: https://github.com/openobserve/designs/tree/main/joins
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
## Summary
- Add comprehensive Playwright tests for translation coverage across 10
supported languages
- Tests verify translation coverage across all 12 sidebar modules (Home,
Logs, Metrics, Traces, RUM, Pipelines, Dashboards, Streams, Reports,
Alerts, Data Sources, IAM) and their sub-pages
- Implements tiered pass criteria: ≥80% (ideal), 70-79% (acceptable with
warning), <70% (fail)
## Languages Tested
- German (Deutsch)
- Spanish (Español)
- French (Français)
- Italian (Italiano)
- Dutch (Nederlands)
- Portuguese (Português)
- Turkish (Türkçe)
- Japanese (日本語)
- Korean (한국어)
- Chinese Simplified (简体中文)
## Files Added
- `pages/generalPages/languagePage.js` - Page object with all
translation testing logic
- `playwright-tests/LanguageTests/*.spec.js` - Individual test files per
language (10 files)
## Test Plan
- [x] All 10 language tests pass locally (parallel execution ~1.6 min
with 5 workers)
- [x] German at 78% passes as "acceptable" with warning
- [x] Other languages at 88% pass as "ideal"
**Note**: These tests are designed for scheduled/nightly runs, not PR
validation (each test takes ~45s).
## Summary
- Fixed join test failure caused by "stream is being deleted" error
- Uses unique stream names with `testRunId` to avoid conflicts (SDR
pattern)
- `ingestionJoinUnion()` now generates unique stream names like
`e2e_join_a_<runId>`
- `selectIndexAndStreamJoinUnion()` accepts stream names as parameters
## Root Cause
Fixed stream names (`e2e_join_a`, `e2e_join_b`) caused conflicts when
streams were being deleted from previous test runs. The ingestion would
fail with:
```
{"code":400,"message":"Error# stream [e2e_join_a] is being deleted"}
```
## Test plan
- [x] Run `npx playwright test playwright-tests/Logs/join.spec.js --grep
"selecting field and SQL Mode"` - passes
- [x] Cleanup pattern `/^e2e_join_/` already exists in cleanup.spec.js
## Summary
- Add unique testRunId to pattern and stream names for test isolation
during parallel execution
- Add retry logic to `checkPatternExists` for backend indexing delays (3
attempts with 2s wait)
- Add search box filtering in `selectStream` for reliable stream
selection in long lists
- Add retry logic for Stream Detail button in stream association page
- Fix ingestion retry with exponential backoff for "stream being
deleted" errors
- Update regexPatternManagement test with unique pattern names to avoid
conflicts
## Test plan
- [x] All 23 SDR tests pass reliably in parallel execution
- [x] Verified tests pass when run individually and as a suite
- [x] No regressions in other test suites
This commit improves the enrichment table test suite with significant
performance gains and code quality improvements.
## Performance Improvements
- **58% faster execution**: 3.3m → 1.4m
- Moved logs data ingestion from beforeEach to only tests that need it
(2 of 9 tests)
- Removed unnecessary navigation calls after file uploads
- Cleaned up excessive debug logging
## Test Fixes
- Fixed button selector strategy using position-based nth() approach
- Fixed search method - standardized on searchEnrichmentTableInList()
- Fixed disabled field verification using Playwright's toBeDisabled()
- Fixed delete confirmation dialog selector specificity
- Increased authentication timeout from 5s to 15s for first test in
suite
## Code Quality Improvements
- Added helper function setupLogsData() for tests requiring logs
- Added missing tags to 6 tests for better filtering
- Standardized cleanup patterns across all tests
- Improved code comments explaining test flow
- Consistent use of "already on list" pattern
## Test Coverage
- 9 total tests covering:
1. Upload with exploration
2. Upload with VRL processing
3. CSV validation error
4. Name field validation error
5. Append functionality
6. Cancel workflow
7. Search/filter
8. Edit mode
9. Delete confirmation
All 9 tests passing with improved reliability and maintainability.
Files modified:
- tests/ui-testing/pages/generalPages/enrichmentPage.js
- tests/ui-testing/playwright-tests/GeneralTests/enrichment.spec.js
- tests/ui-testing/playwright-tests/utils/enhanced-baseFixtures.js
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
- Fix TimeoutError in `logsqueries.spec.js` histogram bar chart test
caused by ECharts canvas DOM re-rendering
- Add retry mechanism to `clickBarChartCanvas()` to handle canvas
element detachment
- Add proper assertions using page object pattern
## Changes
- **logsPage.js**: Added retry mechanism (3 attempts) with network idle
wait and stabilization delay
- **logsPage.js**: Added `expectBarChartCanvasVisible()` assertion
method
- **logsqueries.spec.js**: Ensure histogram toggle is ON before test
- **logsqueries.spec.js**: Use `waitForLoadState('networkidle')` for
reliability
- **logsqueries.spec.js**: Add visibility assertions before clicking
canvas
## Test plan
- [x] Test passes locally
- [x] Test passes on repeat runs (verified with `--repeat-each=2`)
- [x] Sentinel code quality audit passed
Added 4 new test cases to verify the join optimization fix:
- INTERSECT query: Common user_ids in both streams
- EXCEPT query: Users without orders
- INTERSECT with WHERE clause: Active users with orders
- EXCEPT with multiple columns: Users details without orders
Also improved test_joinqueries.py by adding a 3-second sleep after data
ingestion to allow proper indexing before querying.
Co-authored-by: Claude <noreply@anthropic.com>
- Add 2 second wait after alert deletion before deleting destination to
allow backend cleanup to propagate
- Use timestamp-based unique names for template, destination, and alert
to avoid conflicts between parallel test runs
- Fixes 409 error: "Destination is currently used by alert"
---------
Co-authored-by: Claude <noreply@anthropic.com>
Tests #9439
- Added comprehensive tests for all 8 severity levels (0-7)
- Implemented POM methods in logsPage.js for severity color validation
- Created test data file with 16 test records (integer and string
formats)
- Tests verify correct color mapping for each severity level
- Includes automatic test stream creation and cleanup
- All tests pass successfully with no impact on existing tests
---------
Co-authored-by: Claude <noreply@anthropic.com>
- Verify full camel case token search: DbException should be found when
searching for complete camel case identifiers
- Verify atomic token search: UserAccountService should be searchable
and return matching results for full service class names
- Verify XML acronym tokenization: XMLHttpRequest should be properly
tokenized and discoverable through match_all queries
- Verify camel case with numbers: OAuth2TokenHandlershould handle mixed
alphanumeric camel case tokens correctly
- Verify multi-word service search:UserManagementService should support
complex multi-word camel case service names
1. Verify search filter functionality with multiple streams selected
2. Should add a function and display it in streams with multiple streams
3. Should display combined results when both streams have data
4. Should create and delete saved view with multiple streams selected
## Summary
Implements org-level alert deduplication with semantic field groups for
intelligent alert suppression and batched notifications. This feature
prevents alert storms by deduplicating alerts based on configurable
fingerprints and grouping related alerts together.
### Core Capabilities
- **Semantic Field Groups**: Map field name variations (`hostname`,
`host`, `node`) to canonical dimensions for consistent deduplication
across different data sources
- **Organization-Level Configuration**: Global deduplication settings
with cross-alert support to suppress alerts from different rules sharing
semantic dimensions
- **Alert Grouping & Batching**: Wait-and-collect logic with three send
strategies (`FirstWithCount`, `Summary`, `All`) for batched
notifications
- **Per-Alert Control**: Individual alert deduplication settings with
optional fingerprint field customization
### Features
**Deduplication Modes:**
- Per-alert exact matching using fingerprint-based TTL expiration
- Cross-alert semantic matching using org-defined dimension groups
- Configurable time windows (per-alert override or org-level default)
**Alert Grouping:**
- Batch alerts with same fingerprint before sending notifications
- Three strategies: first alert with count, summary with names, or all
details
- Configurable wait time (1-300 seconds) and max batch size (1-100
alerts)
- Background processor runs every 1 second for expired batches
**Semantic Field Groups:**
- Built-in presets for common fields (host, IP address, service)
- Kubernetes resource groups (cluster, namespace, pod, node, container)
- Field normalization support (lowercase + trim)
- Overlap handling with first-defined precedence
### Configuration Types
- `SemanticFieldGroup`: Field name equivalence definitions with
normalization
- `GlobalDeduplicationConfig`: Org-level settings with semantic groups
and cross-alert dedup
- `DeduplicationConfig`: Per-alert fingerprint fields and grouping
options
### API Endpoints
- `GET /{org_id}/alerts/deduplication/config` - Retrieve org-level dedup
config
- `POST /{org_id}/alerts/deduplication/config` - Set org-level dedup
config
- `DELETE /{org_id}/alerts/deduplication/config` - Delete org-level
dedup config
### Implementation Details
- Pure business logic algorithms in enterprise layer (fingerprinting,
matching, calculations)
- OSS service layer orchestrates database operations with enterprise
delegation
- Feature-gated with `#[cfg(feature = "enterprise")]` for
enterprise-only functionality
- Background job architecture for async batch processing
- Storage: Org config in KV DB at `/alert_config/{org_id}/deduplication`
### Architecture
```
Alert Triggered
↓
Query Evaluation
↓
Deduplication Check (fingerprint + TTL)
↓
Grouping Enabled? → Add to Batch → Wait/Collect → Send Grouped
↓ No
Send Individual Notification
```
---------
Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
Neha PandClaude
Shrinath RaoandClaude Opus 4.5
Sai NikhilandShrinath Rao
dependabot[bot]
priyabrata-stack
Subhra264
