mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-06 08:09:24 +00:00
bdefdde66cb92f18f2aa6a17ad5477f14f7ef061
894
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Hengfei Yang
|
bdefdde66c | fix: migration router (#10058) | ||
 Krishna Chaitanya
|
7d94362d02 | feat: bind templates to alert instead of alert-destination (#10023) | ||
 Huaijin
|
d98d710613 | feat: improve alert create for mcp (#10052) | ||
 
|
cb0855360b |
test: add E2E automation for Bug #9967 - PromQL alert creation (#9995)
### **User description** ## Summary Adds comprehensive E2E automation tests for Bug #9967 (cannot save alert when selecting PromQL mode for metrics streams). ### Test Coverage **Test 1: Comprehensive PromQL alert creation validation (P0/P1)** - Verifies PromQL tab visibility for metrics streams - Validates "Trigger if the value is" condition row appears in Step 4 - Tests operator dropdown and value input visibility - Verifies mode switching clears PromQL condition when switching to Custom - Creates and saves alerts with different operators (`>=`, `<=`) **Test 2: Loading existing PromQL alert (P2)** - Creates a PromQL alert via API - Opens alert for editing - Verifies PromQL condition fields load correctly in edit mode ### Files Changed | File | Description | |------|-------------| | `tests/ui-testing/playwright-tests/RegressionSet/alerts-regression.spec.js` | New test file with 2 comprehensive tests | | `tests/ui-testing/pages/alertsPages/alertsPage.js` | Added 20+ POM methods for PromQL alert testing | | `tests/ui-testing/pages/alertsPages/alertCreationWizard.js` | Fixed stream selection retry logic | | `tests/ui-testing/playwright-tests/cleanup.spec.js` | Added `e2e_promql_` cleanup patterns | ### New Page Object Methods - `clickAddAlertButton()`, `fillAlertName()`, `selectStreamType()`, `selectMetricsStream()` - `selectScheduledAlertType()`, `clickContinueButton()`, `clickBackButton()` - `expectPromqlTabVisible()`, `clickPromqlTab()`, `expectCustomTabVisible()`, `expectSqlTabVisible()` - `expectPromqlConditionRowVisible()`, `expectPromqlConditionRowNotVisible()` - `expectOperatorDropdownVisible()`, `expectValueInputVisible()`, `getPromqlConditionValue()` - `clickStepIndicator()`, `expectAlertRowVisible()`, `clickAlertUpdateButton()` ## Test plan - [x] Tests pass locally with `npx playwright test playwright-tests/RegressionSet/alerts-regression.spec.js` - [x] Code passes Sentinel audit (Page Object Model compliance) - [ ] CI pipeline passes ## Related - Fixes #9967 - Related PR #9970 (fix implementation) ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Add PromQL trigger condition support - Initialize and clear `promql_condition` logic - Include `promql_condition` in payload builder - Add unit and E2E tests for PromQL ___ ### Diagram Walkthrough ```mermaid flowchart LR Settings["AlertSettings.vue: PromQL input"] Component["AddAlert.vue: init and clear promql_condition"] Payload["AddAlert.vue: include/clear promql_condition"] UnitTests["AddAlert.spec.ts: PromQL unit tests"] E2ETests["alerts-regression.spec.js: PromQL E2E tests"] Settings --> Component Component --> Payload Payload --> UnitTests Payload --> E2ETests ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><details><summary>5 files</summary><table> <tr> <td><strong>AddAlert.spec.ts</strong><dd><code>Add unit tests for PromQL conditions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9995/files#diff-84362398fea8abd9acf30f4f40ba8d4cf27348677454017bd4e6f568d5176ac6">+397/-0</a> </td> </tr> <tr> <td><strong>alertCreationWizard.js</strong><dd><code>E2E method for PromQL alert creation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9995/files#diff-3b0a08b55d132502596d17c9d5f197de6d4779684de96ce570282e2402942e2e">+239/-0</a> </td> </tr> <tr> <td><strong>alertsPage.js</strong><dd><code>Add POM methods for PromQL tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9995/files#diff-79ee01d6d7f145f19207b93b87143b545a1188cfc6a82bc8c45fecfbf68f8a1d">+243/-0</a> </td> </tr> <tr> <td><strong>alerts-regression.spec.js</strong><dd><code>Add comprehensive PromQL E2E tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9995/files#diff-123b8d960345fd76542aa75316be70b7d8fe088b4ba59127f14ddb53f5b52d8a">+562/-0</a> </td> </tr> <tr> <td><strong>cleanup.spec.js</strong><dd><code>Extend cleanup for PromQL test prefixes</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9995/files#diff-c214275a512fa285a7e4f2ab61dfcb42d71a0436b0e4241f94d7b45693587983">+4/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>AddAlert.vue</strong><dd><code>Initialize and handle `promql_condition` logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9995/files#diff-7b426bb23e9ed0250c3877d6d97b099c42b2d20e22ddf4459915381b2233a0a8">+27/-0</a> </td> </tr> <tr> <td><strong>AlertSettings.vue</strong><dd><code>Add PromQL condition input and validation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9995/files#diff-1d5bfd6a1ea672884193649b1271d751d6f3b8ec84af759071c9bfd169a8ab28">+111/-7</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
 Ashish Kolhe
|
a6a356c91c | fix: 10024 rbac issues with incidents and service corelation (#10030) (#10041) | ||
|
Hengfei Yang
|
3589c78667 | fix: migration cors issue (#10048) | ||
|
Hengfei Yang
|
62f5def1be | feat: add MCP category annotations to API endpoints (#9937) | ||
|
Hengfei Yang
|
fdcb375256 | fix: unit tests (#10032) | ||
 Yashodhan Joshi
|
2650869568 | fix: path issues and v50 cherry picks (#10033) | ||
|
Hengfei Yang
|
1fc4270f54 |
fix: payload limit (#10021)
1. We only need `ZO_PAYLOAD_LIMIT` now, removed `ZO_JSON_LIMIT` 2. Config axum log level `RUST_LOG = "info,config::axum=warn"` |
||
|
Hengfei Yang
|
49ed7a07e5 | fix: migration router (#10018) | ||
|
Hengfei Yang
|
11d855a149 | fix: migration of router (#10008) | ||
|
Hengfei Yang
|
89e067778b | refactor: simplify header extraction in search handlers (#10006) | ||
|
Hengfei Yang
|
33eecaefe3 | refactor: migrate actix-web to axum (#9912) | ||
 Subhra264
|
b25b7eb772 |
fix: dashboard should not be created using PUT dashboard update api (#9999)
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Add `folder` query param for all dashboard endpoints
- Return 404, 403, and 409 HTTP errors
- Enforce dashboard-folder ownership in service
___
### Diagram Walkthrough
```mermaid
flowchart LR
Client["Client request with folder param"] -- "calls" --> Handler["HTTP handler"]
Handler -- "validates folder & params" --> Service["update_dashboard"]
Service -- "checks dashboard in folder" --> DB["dashboards table"]
DB -- "not found" --> Service
Service -- "returns DashboardNotFound" --> Handler
Handler -- "sends 404/403/409 response" --> Client
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Add folder param and error
responses</code>
</dd></summary>
<hr>
src/handler/http/request/dashboards/mod.rs
<ul><li>Added <code>folder</code> Option<String> query parameter to
endpoints<br> <li> Introduced 404, 403, and 409 response statuses<br>
<li> Updated API docs with RBAC requirement details</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9999/files#diff-1b3c257e1d174c826718499faa4a1d4e93465dd56cb2b7b273b690ff7e3223fd">+13/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Enforce folder existence in
update</code>
</dd></summary>
<hr>
src/service/dashboards/mod.rs
<ul><li>Check dashboard exists within specified folder<br> <li> Return
<code>DashboardNotFound</code> if none found<br> <li> Added explanatory
comments for foreign key logic</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9999/files#diff-6ae9e22ced40c50675dde5348b8d696f6559655bbbded915d7e6e88bf64b71a6">+9/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Subhra264
|
36860c7a59 |
fix: allow replacing the failed enrichment table url (#9958)
### **User description**
If there are multiple successful urls added for an enrichment table, but
the last append was failed, user now can replace the failed url with a
new one to rectify if any url issue is there.
___
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Add `replace_failed` flag to enrichment table API
- Backend: allow replacing only failed job URL
- UI: new Replace Failed URL update mode
- Adjust client and handler logic for replace_failed
___
### Diagram Walkthrough
```mermaid
flowchart LR
UI["Vue UI"] -->|select replace_failed| Client["jstransform Service"]
Client -->|POST replace_failed| API["Request Handler"]
API -->|replace_failed| ProcessFailed["Replace Failed Job"]
API -->|append & has_failed| RejectAppend["Reject Append Request"]
API -->|retry| RetryAll["Retry All Jobs"]
API -->|else| ReplaceAll["Replace All Jobs"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>jstransform.ts</strong><dd><code>Add replace_failed to
API client method</code>
</dd></summary>
<hr>
web/src/services/jstransform.ts
<ul><li>Added <code>replace_failed</code> parameter to method
signature<br> <li> Included <code>replace_failed</code> in POST request
body</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9958/files#diff-301add2528e0f32323ca4c503bd7d460ab15ca6b9a9537192a02c5e52fd82926">+2/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Support replace_failed logic
in handler</code>
</dd></summary>
<hr>
src/handler/http/request/enrichment_table/mod.rs
<ul><li>Added <code>replace_failed</code> field with serde default<br>
<li> Extracted <code>replace_failed</code> from request body<br> <li>
Updated append validation to allow replace_failed<br> <li> Added branch
for replacing only failed job URL</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9958/files#diff-7b5b203002d1c3e5f671392a2a8ed84f45225896b652c2dac94ac26e72a43124">+49/-4</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>AddEnrichmentTable.vue</strong><dd><code>UI for Replace
Failed URL mode</code>
</dd></summary>
<hr>
web/src/components/functions/AddEnrichmentTable.vue
<ul><li>Added replace_failed updateMode option and styling<br> <li>
Conditional URL input for replace_failed mode<br> <li> Computed
<code>hasFailedJob</code> and <code>updateModeOptions</code><br> <li>
Forwarded <code>replaceFailedFlag</code> in service call</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9958/files#diff-9aad551dbc051edfdb3994a664506ed6d5e27dcb64f9c2fa917b02e51a35d86e">+42/-10</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Huaijin
|
789ff8915d | fix: streaming partition generate partition start equal end (#9951) | ||
|
Subhra264
|
02f93a729f | fix: rbac for backfill jobs (#9940) | ||
|
d81752d48f |
feat: support multiple jobs for enrichment table urls (#9895)
Design at: pipelines/enrichment-file-url --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
 Ashish KolheandClaude Opus 4.5
|
ce09cd0a4d |
feat: add js for claim parser (#9873)
### **User description** Design at : security/sso-claim-parser/configuration.md ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Introduce rquickjs-based JS runtime for functions - Add compile_js_function and apply_js_fn utilities - Extend pipeline to support JS via CompiledFunctionRuntime - Update web UI: select JS/VRL transforms, show JS errors ___ ### Diagram Walkthrough ```mermaid flowchart LR A["FunctionNode"] -->|JS| B["compile_js_function"] A -->|VRL| C["compile_vrl_function"] B --> D["apply_js_fn"] C --> E["apply_vrl_fn"] D --> F["send to children"] E --> F ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>13 files</summary><table> <tr> <td><strong>function.ts</strong><dd><code>Add optional `trans_type` to TestFunctionPayload</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-c6766f6c671adce05f9eef5d4a57a32371fa7da35628ef5c142fa503d191cb16">+16/-0</a> </td> </tr> <tr> <td><strong>js.rs</strong><dd><code>Introduce JS runtime, compile and apply functions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-88e490bed5fa6c9c8e0aa2738b95c5284bb97899298f54b9cd587713e3ef0109">+756/-0</a> </td> </tr> <tr> <td><strong>batch_execution.rs</strong><dd><code>Extend pipeline execution to handle JS functions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-3f6824a6eddaf1603e5cb0f1a4311ac6a45e1ffb967266ccc1f0bba944625e69">+465/-96</a></td> </tr> <tr> <td><strong>functions.rs</strong><dd><code>Validate and test-run JS transforms, restrict to _meta</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-ed5611a9aa6b5c033e70d7631ba72fafad3cc842672077a7fcf965d11a4fef23">+319/-35</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Forbid JavaScript functions in pipelines globally</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-c186c796a7bdfd9d1301b1429df11456430f1d0d4758783810b9e2e44384e3b2">+39/-1</a> </td> </tr> <tr> <td><strong>jwt.rs</strong><dd><code>Add JS claim parser execution function</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+25/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wrap JS compile/apply in ingestion API</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+24/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Include `trans_type` in test_function API payload</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-a5e1b3644cef9feb689dfe66382e44e5434ecadbff3469127346a333d4733102">+8/-3</a> </td> </tr> <tr> <td><strong>AddFunction.vue</strong><dd><code>UI: add JS/VRL transform type selector</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-ebfd6c538f3cd3ecb9ab17550fd76e2e89e7a1abccc83a7505077e4af3eee110">+42/-25</a> </td> </tr> <tr> <td><strong>FunctionsToolbar.vue</strong><dd><code>Toolbar: transform type radio buttons</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-8283d2ff9c4a66b4b59eac306672aca2d54655a462e2ddb88e14e039d79dcdaf">+51/-3</a> </td> </tr> <tr> <td><strong>DomainManagement.vue</strong><dd><code>Track claim-parser function changes for save button</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-5e47cd01fcb63f7ef27dc64d2ecf66c39e17097e29944951c3ae1bfe715bb97c">+17/-9</a> </td> </tr> <tr> <td><strong>TestFunction.vue</strong><dd><code>Support JS error messaging in test UI</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-b9af9a2d1bf04d23422c26a13e6a763c6953c8713357a2f64ed8073ce6c926a3">+15/-5</a> </td> </tr> <tr> <td><strong>PipelineEditor.vue</strong><dd><code>Filter out JS functions in pipeline editor UI</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-68c21f647336067eb79737ce9aca2d49632f2fa1c0878bcfdae55607e3473a15">+12/-4</a> </td> </tr> </table></details></td></tr><tr><td><strong>Configuration changes</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>function.rs</strong><dd><code>Extend `Transform` model for JS support</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-27c9f3d93723397a0e234bc45c9f543b43eff5fd518d76e735f78d95a58f3f1c">+16/-6</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register new `js` utils module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-2aa59ce4464e7dbabdb9367fc8bb8c1b489c914b8690577d4916870463ab12db">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>errors.rs</strong><dd><code>Rename `MemoryTableOverflowError` display message</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-bf45932466265b732f13d0cd9b9cffbaefbe37c5861ace9315449eeca9250713">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>3 files</summary><table> <tr> <td><strong>result-array-functions.spec.js</strong><dd><code>Add E2E tests for `#ResultArray#` functions in UI</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-e9911625ad2bab76f2b760f0ad254791331993a6a874b96f6a11fc6d88038b10">+470/-0</a> </td> </tr> <tr> <td><strong>cleanup.spec.js</strong><dd><code>Clean up JS function test data in UI tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-c214275a512fa285a7e4f2ab61dfcb42d71a0436b0e4241f94d7b45693587983">+31/-0</a> </td> </tr> <tr> <td><strong>page-manager.js</strong><dd><code>Register `FunctionsPage` in `PageManager`</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-565eab8b3c767ae50c9d05e591621d14f66b81af84d6b5ea6efe136c1c3933df">+6/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Dependencies</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>Cargo.toml</strong><dd><code>Add `rquickjs` dependency for JS runtime</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>18 files</summary><table> <tr> <td><strong>test_functions_meta_org_restriction.py</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-99ef1207d6ae6d7eeaed37d70c8cf0c4770ba78f6f51fbdc067d35492d2fe9ea">+401/-0</a> </td> </tr> <tr> <td><strong>test_js_enrichment.py</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-c5547d46bec60b888d947d1e9b7c5445aee72cf8e837182f7138e0fc903eedbb">+356/-0</a> </td> </tr> <tr> <td><strong>test_js_result_array.py</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-9ed1363a1f58c94e018bb2b0aaa9d08fa71f89e7e55a6ff6be4a82bbeed8985c">+390/-0</a> </td> </tr> <tr> <td><strong>functionsPage.js</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-7bab9cc2eb8f36d3ee5d0aab453c353ec6d6dfecbc401cf401322ff57faa8440">+326/-0</a> </td> </tr> <tr> <td><strong>javascript-functions.spec.js</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-bdae6529346a7222bdec19f75f490cb9d0f3578a260ba835225efcd27db3f64e">+314/-0</a> </td> </tr> <tr> <td><strong>meta-org-js-restriction.spec.js</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-16cc2373f6f1955f8344a59d5f2edcd6ce121c4fdee1d432b1f0ec8e27f25881">+483/-0</a> </td> </tr> <tr> <td><strong>FunctionList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-6ff9af50a1597504b7f0dca91e61cf30c800b4a17b4be2ed2c3a4f7a7b2d7068">+1/-0</a> </td> </tr> <tr> <td><strong>de.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-78f24ecb2dbcfc38fdf3ec22b2559a5b40d236ea689d1c55f5d3e87168310264">+30/-18</a> </td> </tr> <tr> <td><strong>en.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+15/-7</a> </td> </tr> <tr> <td><strong>es.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-ac2a2167b8b973e3f776f9fe3b43bef4576470d8ad33f3d7cfaff6edff0e5778">+30/-18</a> </td> </tr> <tr> <td><strong>fr.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-db80e762b88ddf183820c050cdc6b215e6b02b365b5e73f5152af46b0d26f1a2">+30/-18</a> </td> </tr> <tr> <td><strong>it.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-2de78074a5f887afddce60d694ead9322e6ab223eb70251c13819f3028bdc70d">+30/-18</a> </td> </tr> <tr> <td><strong>ja.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-092e49c5f9fbc1b539ab9c2866d108d0ce1d314022b4c294eb4611b285e7bbd3">+30/-18</a> </td> </tr> <tr> <td><strong>ko.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-50e27ca1b4825a5026f3a251bfcf79c97cab07cfade2ebd512d332453d707c48">+30/-18</a> </td> </tr> <tr> <td><strong>nl.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-a1706021687ae5d8a1c006c49eab3946174fecd895f9b70a89c2e76d97250906">+30/-18</a> </td> </tr> <tr> <td><strong>pt.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-38e804f64cb5e0e451849fc84e9b46e53c954a4a4263d51c18665ad45f4b4815">+30/-18</a> </td> </tr> <tr> <td><strong>tr.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-6f0f131e9e2591480c4c5d1eaffc72d02dc31cbd8d3f90d8f012f08cabac8818">+30/-18</a> </td> </tr> <tr> <td><strong>zh.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9873/files#diff-8818055db58f83230aeea4ebfd4cec31d924e8c47b6f91298e356959ab25339e">+254/-242</a></td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ashish Kolhe
|
0f4dcbe971 | fix: 9882 add option to limit max series return for promql (#9896) | ||
 Yashodhan JoshiandYashodhan Joshi
|
68759b4354 |
fix: sync distinct stream retention to original stream (#9871)
### **User description**
This adds functionality to always sync original stream retention to the
distinct stream created for that stream.
This also includes a once-off migration to sync the retention for
existing streams
___
### **PR Type**
Enhancement
___
### **Description**
- Bump DB schema version to 24
- Add migration syncing existing distinct stream retention
- Sync retention on settings update for distinct streams
- Refactor distinct stream naming logic
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Original Stream Settings"] --> B["Generate Distinct Stream Name"]
C["Migration: Sync Retention"] --> B
D["Settings Update Handler"] --> B
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Bump database schema
version</code>
</dd></summary>
<hr>
src/config/src/config.rs
- bumped `DB_SCHEMA_VERSION` from 23 to 24
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9871/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Register new
migration</code>
</dd></summary>
<hr>
src/infra/src/table/migration/mod.rs
- register new migration in migrator list
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9871/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+2/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>m20260107_000001_sync_distinct_stream_retention.rs</strong><dd><code>Add
migration for distinct stream retention</code>
</dd></summary>
<hr>
src/infra/src/table/migration/m20260107_000001_sync_distinct_stream_retention.rs
<ul><li>add new migration file for retention sync<br> <li> iterate all
streams and update distinct retention<br> <li> support enterprise
super-cluster propagation</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9871/files#diff-e077e2b4452a91f484015bd939eae29bec994be01a67eb4e71e5f111c761c755">+115/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>distinct_values.rs</strong><dd><code>Refactor naming
and retention logic</code>
</dd></summary>
<hr>
src/service/metadata/distinct_values.rs
<ul><li>extract <code>get_distinct_stream_name</code> function<br> <li>
use helper instead of inline formatting<br> <li> update distinct stream
retention during ingest</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9871/files#diff-3fb2320197c6e8e900658f5a24158e922610eadcbf081ab0df230f770ef27e07">+35/-6</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>stream.rs</strong><dd><code>Sync retention on settings
save</code>
</dd></summary>
<hr>
src/service/stream.rs
<ul><li>on <code>save_stream_settings</code>, sync distinct stream
retention<br> <li> add <code>get_stream_retention</code> helper
function<br> <li> skip metadata streams when syncing</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9871/files#diff-41452a7fb29b4634437174fd6aecf6a9050dbfdef41b9077a21d59af3d79e9ed">+68/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
|
||
|
Ashish Kolheandgithub-actions[bot]
|
0284e5ab5a |
feat: tenant management using service accounts (#9838)
### **User description** Design at : service_account_enhancements/FEATURE_TENANT_MANAGEMENT_SERVICE_ACCOUNTS ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Add session expiry cache with cleanup - Implement `assume_service_account` endpoint - Enforce `allow_static_token` policy for service accounts - Automate service account creation and tenant admin role ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Create Organization"] -- "adds service account" --> B["DB org_users (allow_static_token=false)"] B -- "assign tenant admin role" --> C["OpenFGA"] A -- "respond with ServiceAccountTokenInfo" --> D["Client"] E["AssumeServiceAccount API"] -- "create session" --> F["DB sessions with expiry"] F -- "cache token & expiry" --> G["USER_SESSIONS & USER_SESSIONS_EXPIRY"] H["AuthExtractor"] -- "validate session token + expiry" --> I["Request Auth"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Configuration changes</strong></td><td><details><summary>5 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Add session expiry cache mapping</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+1/-0</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Bump DB schema version to 23</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Import new session meta module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-bab79e8da2d0d9f903cb1a6187d922c6de63d3e4561e8ce633660b664b4e859f">+1/-0</a> </td> </tr> <tr> <td><strong>m20251230_000001_add_allow_static_token_to_org_users.rs</strong><dd><code>Migration: add allow_static_token to org_users</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-2c713c6a1665f7cc5b19eda98ed6663e4908201052e31bd3c6643aa48c03130b">+55/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register new migration for allow_static_token</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>20 files</summary><table> <tr> <td><strong>organization.rs</strong><dd><code>Add service_account fields and response types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-4f089715a7ac4f78b45039d773f2fccbb4fbc6a409e0ab9ceee1fcf7911bc44b">+24/-0</a> </td> </tr> <tr> <td><strong>user.rs</strong><dd><code>Include `org_name` in `UserOrg` defaults</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+3/-1</a> </td> </tr> <tr> <td><strong>auth.rs</strong><dd><code>Enhance session auth extraction and expiry logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+101/-8</a> </td> </tr> <tr> <td><strong>session.rs</strong><dd><code>Define assume_service_account request/response schemas</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-cc73e168e23e2cd5d98749492c12493cff611bc9cecfa3d45e8c6a0e9a3ba135">+50/-0</a> </td> </tr> <tr> <td><strong>user.rs</strong><dd><code>Add `org_name` field to `UserOrg` meta</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-fc77f1b0e696889335f118cd5c1d69b7e3b385f48dd8e94c02a69912242e689b">+2/-0</a> </td> </tr> <tr> <td><strong>jwt.rs</strong><dd><code>Populate `org_name` in JWT auth mapping</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+8/-1</a> </td> </tr> <tr> <td><strong>validator.rs</strong><dd><code>Enforce service account static token policy</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+58/-11</a> </td> </tr> <tr> <td><strong>assume_service_account.rs</strong><dd><code>Implement `assume_service_account` HTTP endpoint</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-3cdbc33e83fda4c90aae59b6213861646ce7031b092e1f052c4c3c8e94ac468b">+208/-0</a> </td> </tr> <tr> <td><strong>org.rs</strong><dd><code>Return service_account info on org creation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-b47c0c1a8a78524196176430268eecdbfaf6f46d6b95873d3cbe70cfd67182b0">+12/-5</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register assume_service_account route</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a> </td> </tr> <tr> <td><strong>org_users.rs</strong><dd><code>Add `allow_static_token` to OrgUsers entity</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-aec40965f56c43f49b6c162e7f7cfd549af23e0dcd9980f1e863ddb30eba357b">+1/-0</a> </td> </tr> <tr> <td><strong>org_users.rs</strong><dd><code>Support `allow_static_token` in ORM models</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-8e62ecba19308fdaaecfdb3d1f23021c926e244a510d9452dfbd71ac8f35ee9f">+33/-0</a> </td> </tr> <tr> <td><strong>sessions.rs</strong><dd><code>Add session expiry persistence and cleanup</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-945b60afd1f074c5f3bf875317896181829b19153938184bad9098c75ab44b48">+11/-12</a> </td> </tr> <tr> <td><strong>org_users.rs</strong><dd><code>Add `add_with_flags` for org_users</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-3cbb516ea021b4c03cf4b5f9de2ebcd4a8afee0722a4bcc5c39cb78f485900b6">+73/-0</a> </td> </tr> <tr> <td><strong>organization.rs</strong><dd><code>Default `service_account` in org DB functions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-ce65cea926a09721a8fea14d07fe188bc2b887218e95d74a8bffefb54001da50">+3/-0</a> </td> </tr> <tr> <td><strong>session.rs</strong><dd><code>Enhance session caching with expiry checks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-64311ff8ef53118e7611a2a2a28c5380a0e8b8213139b2c2108c728b1d460a2b">+152/-17</a></td> </tr> <tr> <td><strong>user.rs</strong><dd><code>Map `org_name` in DB user mapping</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-6ee6b0fc9ed03224d4ddf0b7b631c6f81ff7d1d579bdb98d2c5d4453852eb5d6">+4/-2</a> </td> </tr> <tr> <td><strong>organization.rs</strong><dd><code>Add service account creation & tenant role logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-6b3c3c2b027c1158c0d41302cc69dfcd20ab68a6a4fa365cda403f838940bbb4">+270/-7</a> </td> </tr> <tr> <td><strong>session.rs</strong><dd><code>Use `set_with_expiry` for session creation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-2d2f0c7a3be0a3d7d953c01974be5e093c8d42953952e435bc7b3a0faed6bf38">+3/-1</a> </td> </tr> <tr> <td><strong>users.rs</strong><dd><code>Add `create_service_account_if_not_exists` helper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+56/-7</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Include `allow_static_token` in test fixtures</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-2ba3bceb0d80c8008da06e45374c1f744441375e2c28b55e6a55a1be3337e79b">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Export assume_service_account module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-69dda3b2107f353d44361f3f47fad6f504470d26f888dec797664fb7de35ab08">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Mask static tokens in API docs and logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-095d70f65b6e69bf7de0180fcbcddcd7c89c4cf00095d7b06f039a07e73916bb">+13/-3</a> </td> </tr> <tr> <td><strong>openapi.rs</strong><dd><code>Document assume_service_account in OpenAPI</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9838/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+3/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Hengfei Yang
|
c96605a093 | fix: metrics streaming partition (#9839) | ||
 ByteBaker
|
20438076dd |
refactor: deduplicate ingestion handler code
Extract process time tracking and ingestion check logic into helper functions: - `get_process_time()` - consolidates slow log threshold check - `insert_process_time_header()` - adds o2_process_time header - Replace `match` with `if let Err` for cleaner error handling - Remove unused `_in_req` parameters from handler functions Reduces code duplication across `bulk`, `multi`, `json`, `hec`, `otlp_logs_write` handlers. |
||
|
Hengfei Yang
|
ec021ac29e | perf: optimize MCP tools - reduce size from 643KB to 123KB (#9825) | ||
|
Sai Nikhilandgithub-actions[bot]
|
8beb75de63 |
fix: share url disabled when web url is not configured (#9826)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
ByteBakerandsai nikhil kethe
|
09ef3a8f88 |
feat: incident detail drawer enhancements - telemetry tabs and alert graph (#9719)
### **User description** ## Summary Enhances the incident detail drawer with two new features to improve incident investigation: 1. **Telemetry Correlation Tabs** - Logs, Metrics, and Traces tabs showing related telemetry 2. **Alert Graph Visualization** - Interactive graph of services and dependencies Design at : https://github.com/openobserve/designs/tree/main/alert-correlation --- ## Feature 1: Telemetry Correlation Tabs Adds three new tabs (Logs, Metrics, Traces) that display correlated telemetry streams based on the incident's `stable_dimensions`. **Frontend Changes:** - New service method `getCorrelatedStreams()` in `web/src/services/incidents.ts` - Integrates `TelemetryCorrelationDashboard` component in embedded mode - Lazy loading: data fetched only on first tab click - Data caching: subsequent tab switches reuse loaded data - Refresh button for manual data reload - Auto time range from `first_alert_at` to `last_alert_at` - Empty state with manual search fallback - Memory cleanup on drawer close **No backend changes** - uses existing `POST /api/{org}/service_streams/_correlate` endpoint --- ## Feature 2: Alert Graph Visualization Adds "Alert Graph" tab showing interactive visualization of all services involved in an incident. **Backend Changes:** - New endpoint: `GET /api/v2/{org}/alerts/incidents/{id}/service_graph` - Returns graph nodes (services with alert counts) and edges (dependencies) - Builds from existing `topology_context` field - Gated by `O2_INCIDENTS_ALERT_GRAPH_ENABLED` env variable (default: false) - Enterprise-only with 403 handling for non-enterprise builds **Frontend Changes:** - New `IncidentServiceGraph.vue` component using ECharts - Force-directed and circular layouts - Visual encoding: node size = alert count, color = severity/root cause - Purple border for primary incident service - Stats banner and hover tooltips - Empty state when topology unavailable **Configuration:** ```bash export O2_INCIDENTS_ALERT_GRAPH_ENABLED=true # Required to enable ``` ___ ### **PR Type** Enhancement ___ ### **Description** - Add `IncidentServiceGraph` Rust models and API - Implement `get_service_graph` service logic - Extend `incidentsService` with correlation methods - Enhance `IncidentDetailDrawer` with telemetry and graph tabs --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Subhra264
|
509bf7b6fb |
feat: pipeline backfill (#9462)
Design at : pipeline-gap-filler
# feat: Add Pipeline Backfill Jobs Feature
This PR introduces a complete pipeline backfill system that allows users
to reprocess historical data through existing pipelines.
## Key Features
### Backend
- **Dual-table architecture**: Separate `backfill_jobs` table for static
config and `scheduled_jobs` for scheduling state
- **RESTful API endpoints** under
`/api/{org_id}/pipelines/{pipeline_id}/backfill`:
- Create, list, pause, resume, update, and delete backfill jobs
- **Robust error handling**: Retry mechanism matching derived stream
pattern (immediate retries → scheduled retries, never delete jobs)
- **Progress tracking**: Chunk-based processing with real-time position
updates
- **Optional deletion phase**: Delete existing data before backfilling
with separate status tracking
- **Super cluster support**: Proper trigger synchronization across
regions
### Frontend
- **Backfill Jobs List Page**: Table view with status/pipeline filters,
action buttons (pause/resume/edit/delete)
- **Create Dialog**: Time range picker (absolute dates only),
configurable chunk periods and delays
- **Edit Dialog**: Update paused/completed jobs with reset to fresh
state
- **Job Details Dialog**: Real-time progress, timeline visualization,
deletion status tracking
- **Dark mode support**: Full theme compatibility across all components
- **DateTime integration**: Reuses existing DateTime component with
microsecond precision
## Technical Highlights
- Cross-database support (PostgreSQL, MySQL, SQLite)
- Frontend-only filtering for performance
- Status mapping (waiting/pending → "running" display)
- Timestamp formatting: `YYYY-MM-DDTHH:mm:ssZ`
- Default chunk period matches pipeline schedule frequency
- Navigation integration from PipelinesList component
## Files Changed
### Backend
- **Migration**:
`src/infra/src/table/migration/m20250109_000001_create_backfill_jobs.rs`
- **Table CRUD**: `src/infra/src/table/backfill_jobs.rs`
- **Service logic**: `src/service/alerts/backfill.rs`
- **Scheduler handlers**: `src/service/alerts/scheduler/handlers.rs`
- **API routes**: `src/handler/http/request/pipelines/backfill.rs`
- **Router integration**: `src/handler/http/router/mod.rs`,
`src/handler/http/router/openapi.rs`
- **Super cluster sync**: `src/super_cluster_queue/scheduler.rs`
### Frontend
- **Components**:
- `web/src/components/pipelines/BackfillJobsList.vue`
- `web/src/components/pipelines/CreateBackfillJobDialog.vue`
- `web/src/components/pipelines/EditBackfillJobDialog.vue`
- `web/src/components/pipelines/BackfillJobDetails.vue`
- **API service**: `web/src/services/backfill.ts`
- **DateTime enhancement**: `web/src/components/DateTime.vue`
## Testing
- Tested across PostgreSQL, MySQL, and SQLite databases
- Verified dark mode compatibility
- Tested pause/resume/edit/delete operations
- Validated progress tracking and completion status
- Confirmed super cluster synchronization
|
||
|
Subhra264
|
e17e8bc5e6 |
fix: when retrying allow user to choose whether to resume from last byte or start from scratch (#9792)
### **PR Type**
Enhancement
When the job is failed, and the user wants to retry manually, ask the
user whether resume should be from the last successful position or from
start again.
___
### **Description**
- Support resume option for enrichment import
- Parse `resume` query param server-side
- Custom UI for resume or fresh restart
- Update JS service to include `resume` param
- Add `RetryJobDialog.vue` component
___
### Diagram Walkthrough
```mermaid
flowchart LR
UI["UI: RetryJobDialog"] -- "confirm resume choice" --> API["POST /api/{org}/enrichment_tables/{table}/url?append=&resume="]
API -- "parse `resume` param" --> Handler["save_enrichment_table_from_url"]
Handler -- "resume logic" --> DB["Job record (resumed or new)"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>jstransform.ts</strong><dd><code>Add `resume` param in
request service</code>
</dd></summary>
<hr>
web/src/services/jstransform.ts
<ul><li>Add <code>resume</code> parameter with default false<br> <li>
Append <code>resume</code> to POST request URL</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9792/files#diff-301add2528e0f32323ca4c503bd7d460ab15ca6b9a9537192a02c5e52fd82926">+2/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Support resume logic in import
handler</code>
</dd></summary>
<hr>
src/handler/http/request/enrichment_table/mod.rs
<ul><li>Parse <code>resume</code> query parameter<br> <li> Store
<code>existing_job</code> for resume handling<br> <li> Resume failed
ranged jobs preserving progress<br> <li> Skip range support check when
resuming</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9792/files#diff-7b5b203002d1c3e5f671392a2a8ed84f45225896b652c2dac94ac26e72a43124">+79/-42</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>EnrichmentTableList.vue</strong><dd><code>Integrate
retry dialog into table list</code>
</dd></summary>
<hr>
web/src/components/functions/EnrichmentTableList.vue
<ul><li>Import and register <code>RetryJobDialog</code><br> <li> Manage
<code>showRetryDialog</code> and <code>retryJobData</code><br> <li>
Update <code>retryUrlJob</code> to open dialog<br> <li> Adjust
<code>handleRetryConfirm</code> to pass <code>resume</code></ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9792/files#diff-6a29e253f59405d3c0f756dba36afd7483f10ad86927cbbc3348ede6372c3077">+64/-34</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>RetryJobDialog.vue</strong><dd><code>Add RetryJobDialog
component</code>
</dd></summary>
<hr>
web/src/components/functions/RetryJobDialog.vue
<ul><li>New dialog to choose resume or restart<br> <li> Handles focus
trap and ESC key<br> <li> Displays range support warning<br> <li> Emits
<code>confirm</code> with resume choice</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9792/files#diff-7bc926ef894b5aed7010006e729ede28a2f4931809b154dc81880999b3a7e72f">+638/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
|
78c96c3721 |
fix: support bulk deletes for various resources (#9153)
### **User description** fixes https://github.com/openobserve/openobserve/issues/9116 This adds support for bulk delete operation from API and UI side for various resources. ___ ### **PR Type** Enhancement ___ ### **Description** - Add bulk delete endpoints for multiple resources - Enforce per-item permission checks before delete - Aggregate successful/failed items in responses - Register bulk-delete routes in HTTP router - Extend UI: add bulk-delete buttons, dialogs, service calls ___ ### Diagram Walkthrough ```mermaid flowchart LR UI["\"UI: Bulk Delete Trigger\""] API["\"HTTP DELETE /{resource}/bulk Handler\""] AUTH["\"Permission Checks per Item\""] DEL["\"Perform Delete Operations Loop\""] RESP["\"Aggregate Response: successes/ failures\""] UI_FE["\"FE: Show Notifications\""] UI -- "calls" --> API API -- "checks" --> AUTH AUTH -- "allowed" --> DEL DEL -- "deletes items" --> RESP RESP -- "returns" --> UI_FE ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14 files</summary><table> <tr> <td><strong>fga.rs</strong><dd><code>Add bulk delete roles and groups endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-e7c99f37f3abbb5e0507d2760bbf67df010d9218704b38dce84c5477acecd22e">+218/-6</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Implement delete_alert_bulk handler</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-83431801d27626ef27482e67b62661f900043544a45ddf958bd027cf2fa3bd8d">+87/-6</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add delete_function_bulk and error types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-a5e1b3644cef9feb689dfe66382e44e5434ecadbff3469127346a333d4733102">+97/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add delete_dashboard_bulk endpoint</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-1b3c257e1d174c826718499faa4a1d4e93465dd56cb2b7b273b690ff7e3223fd">+78/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Implement re_patterns delete_bulk logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-27d37a8738de89c81b540daa978e684caf2cf28803e1208d90cb4d42528ed28b">+116/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add users bulk delete handler</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-56897e42f9436a640bc70fddc25686485ba8ae1223635b32d252a0333309f748">+80/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add service_accounts delete_bulk API</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-095d70f65b6e69bf7de0180fcbcddcd7c89c4cf00095d7b06f039a07e73916bb">+86/-1</a> </td> </tr> <tr> <td><strong>action.rs</strong><dd><code>Implement delete_action_bulk endpoint</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-616df3014c553f763ea57e925f1bff7bb47f9467c632940d93a45b7b1e4c3a07">+90/-1</a> </td> </tr> <tr> <td><strong>destinations.rs</strong><dd><code>Add delete_destination_bulk for alerts</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-76b00293c621036de31cf499c1c276efb9704c2bf8cf1b728f49cfdd43f9fd1b">+75/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add delete_bulk for cipher_keys</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-043222e433d1e5825a4b7bfc2c7feedba417f2630f0082fa67aff64976736396">+76/-1</a> </td> </tr> <tr> <td><strong>templates.rs</strong><dd><code>Implement delete_template_bulk handler</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-b08e274a64cc935b0cec402a04271a872b1586027a77dc0ccb84c597c3960f61">+73/-4</a> </td> </tr> <tr> <td><strong>pipeline.rs</strong><dd><code>Add delete_pipeline_bulk API handler</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-c69f4f94eee97ce226caddc497b2adf89e26869f2d72b0d0206dc31ab2714daf">+70/-2</a> </td> </tr> <tr> <td><strong>reports.rs</strong><dd><code>Add delete_report_bulk endpoint</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-00435e0d9301342b51cb57ae313f1413e7ed973642e0badf3f3959125d0677d1">+75/-1</a> </td> </tr> <tr> <td><strong>ActionScripts.vue</strong><dd><code>Integrate bulk delete UI in action scripts</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-770f71fcb92a13b3b8b1041567b885fac39903c814825c878595cafa624ebbd8">+284/-5</a> </td> </tr> </table></details></td></tr><tr><td><strong>Configuration changes</strong></td><td><details><summary>3 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Register bulk delete services in router</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+14/-0</a> </td> </tr> <tr> <td><strong>iam.ts</strong><dd><code>Add bulkDeleteGroups and bulkDeleteRoles calls</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-3712c1bb0eecb664b6691d724c3673b8c1befa41efbd8c5de626c5849f8ace98">+10/-0</a> </td> </tr> <tr> <td><strong>dashboards.ts</strong><dd><code>Add bulkDelete service for dashboards</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-913b3f54367036b1b3960aa3eddd7e17b509b1d6011c5ab68ae0e935653179c5">+7/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Error handling</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>auth.rs</strong><dd><code>Allow bulk-delete paths in auth extractor</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+17/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>user.rs</strong><dd><code>Define bulk delete request/response structs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+36/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add PipelineBulkDelete meta types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-c4bd542e4e07ed7706b0643086f5a7d4edbb3647c7ceedbe4bea516c5274d8bd">+12/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>40 files</summary><table> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-6f9c2e6dbc40f8d40e3d3ecbee23240813a3d4856d49bd5d979f5b98551fdc3d">+12/-0</a> </td> </tr> <tr> <td><strong>service_account.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-878ad520e1c84500cc21ff6fc4088c8da64ff2febd79e60c0d4cc8c695c49798">+11/-0</a> </td> </tr> <tr> <td><strong>action.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-fc87c75a7df9539d7e7ee6184dfab02d4a8932323e81f95d57c05363aa44752d">+15/-0</a> </td> </tr> <tr> <td><strong>reports.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-d2117a3d67d4ec47b0017861845c92143baf917e10e6746df5dd97f925f81eeb">+12/-0</a> </td> </tr> <tr> <td><strong>function.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-27c9f3d93723397a0e234bc45c9f543b43eff5fd518d76e735f78d95a58f3f1c">+12/-0</a> </td> </tr> <tr> <td><strong>requests.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-412f11e06c0e901e2292aab80536cb36edebcc4796dec52923389c3aaa32b25a">+6/-0</a> </td> </tr> <tr> <td><strong>responses.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-224027f90893122eea28e35778467acf1a4b5572ce612fc9bc9ec44ca8ebd9fb">+9/-0</a> </td> </tr> <tr> <td><strong>dashboards.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-250eeebeec9d4ac0ac271466ed83cd33fea3f121080491b340dc95d9af287259">+12/-0</a> </td> </tr> <tr> <td><strong>destinations.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-ecffe114c8200bab742dd2ee4f7f41128aa59438a879634f070edd575a1f917c">+24/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+1/-0</a> </td> </tr> <tr> <td><strong>utils.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-96bb9f4a8f547c821f26c957e8505d0437095b08d2b36687531f71565d287bd6">+2/-1</a> </td> </tr> <tr> <td><strong>functions.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-ed5611a9aa6b5c033e70d7631ba72fafad3cc842672077a7fcf965d11a4fef23">+22/-31</a> </td> </tr> <tr> <td><strong>AppTable.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-efcf66a9e913b5d3a7c2eea800b7f4e54d9e5268307131482fd40f32e2074d8e">+72/-16</a> </td> </tr> <tr> <td><strong>AlertList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-be3e41b823e163b226ffc270d95cc4981e59c600851e4e0d10ee5e4ee994ac8e">+123/-2</a> </td> </tr> <tr> <td><strong>AlertsDestinationList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-e9d05ba6c071c0e2f5d9ef7acdef757f79910abd6897b29e27b069a10d4ba8de">+142/-5</a> </td> </tr> <tr> <td><strong>PipelinesDestinationList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-088c66f5a34e73a5350a794a772e5d905262277337c937dd36f4295072203ebf">+163/-28</a></td> </tr> <tr> <td><strong>TemplateList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-7df66364d7d69a26cfa25cbc555d0e090c2f73e51c8386d9b0435bae4412564c">+116/-27</a></td> </tr> <tr> <td><strong>EnrichmentTableList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-6a29e253f59405d3c0f756dba36afd7483f10ad86927cbbc3348ede6372c3077">+124/-13</a></td> </tr> <tr> <td><strong>FunctionList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-6ff9af50a1597504b7f0dca91e61cf30c800b4a17b4be2ed2c3a4f7a7b2d7068">+143/-6</a> </td> </tr> <tr> <td><strong>AppGroups.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-402fb2a5338e60fdc3da489f4d50b5cf52f06b4e884a6fe7b82441c9fa5ea22c">+86/-1</a> </td> </tr> <tr> <td><strong>AppRoles.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-191634ef09436199f91f442968110a01f54ae46b5ceee67476f52613ad796db4">+86/-1</a> </td> </tr> <tr> <td><strong>EditRole.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-263b6bc7d29fdf8358c8437cd637f1350aa3733cc2706eae8ff00028686cc0f5">+20/-0</a> </td> </tr> <tr> <td><strong>ServiceAccountsList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-3afe835bcfbec3fda2e9550850c29581092ee0717e7dd8de8888e774a0048d24">+114/-1</a> </td> </tr> <tr> <td><strong>User.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-e973c2a665f9b27b94673ba64cc5b23650300836a727a2e07673c8e26305aee5">+133/-3</a> </td> </tr> <tr> <td><strong>PipelinesList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-85e3ef322f3eea0fc6d74d89431ba338a1153cb82ee06bef0f7c5f439c04267d">+108/-2</a> </td> </tr> <tr> <td><strong>ReportList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-f7d700f7e67fbaba3c58ab5d2c614925db743101828214445ce9dae35dc29067">+157/-5</a> </td> </tr> <tr> <td><strong>CipherKeys.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-950b715cff2c964d8542512692030c370733b57d698abf40e1dcb3cce2cfaf7a">+104/-13</a></td> </tr> <tr> <td><strong>RegexPatternList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-524927acd9f5bb07b71bfac489e747470108574e21a6dde63ea7c1f219cea853">+109/-14</a></td> </tr> <tr> <td><strong>action_scripts.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-6c1a011ac23290134801b22feb2b50c2b19cabf2a009f37a62b4dd9ca6e415e3">+3/-0</a> </td> </tr> <tr> <td><strong>alert_destination.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-377d28822e9ad4e9e80327ecc2d09bf1981c7997c5f1c1de120c6179eef19c6e">+3/-0</a> </td> </tr> <tr> <td><strong>alert_templates.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-fd8c4260abc3df601e4331d36b0c211eca568ee70139ad90c87247bc552fe4e2">+3/-0</a> </td> </tr> <tr> <td><strong>alerts.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-7a62782cceacac92b787e058219b3c53ea7406d4be5780def899943a01f500c1">+7/-0</a> </td> </tr> <tr> <td><strong>cipher_keys.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-0a0ef3a63c29b2a74b7f904c54c3fb32bee421608474aa32334b7c55133ab1ef">+3/-0</a> </td> </tr> <tr> <td><strong>jstransform.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-301add2528e0f32323ca4c503bd7d460ab15ca6b9a9537192a02c5e52fd82926">+3/-0</a> </td> </tr> <tr> <td><strong>pipelines.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-e8626670bb4e46e073f9858ade6b3dc77eaadf39d6721d9c0006eccc1b430612">+8/-0</a> </td> </tr> <tr> <td><strong>regex_pattern.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-2a0b0828141d39ab0385e0941314ead1cdd2ec3ec7b3baf5fd0c42f8f010728a">+3/-0</a> </td> </tr> <tr> <td><strong>reports.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-32d3299b2b21dc57c39d01080baba03765d853bd05c82d6158834b54713ae825">+3/-0</a> </td> </tr> <tr> <td><strong>service_accounts.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-4038191b0dc0cad335f3d8f7f703505df905ad843fe950096aefb8abbd139a1e">+4/-1</a> </td> </tr> <tr> <td><strong>users.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-a334d8d570d9bdcaa3d4c8c2a993eff009a1ec8eb499c8a3d1a292d9224159d0">+3/-0</a> </td> </tr> <tr> <td><strong>Dashboards.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9153/files#diff-8da3187db8ce3d795d0f196ede6b2290221ffa3e27b15dcc2b645e44dd196213">+122/-6</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Hengfei Yang
|
e59474aa1b | perf: reduce nats events (#9712) | ||
|
Huaijin
|
6082edf321 | feat: follower querier timeout for UI (#9744) | ||
 Harsh Mahajan
|
2bb67c3b7e |
fix: new kv table for kv store (#9459)
This PR make necessary changing for using new KV table for storing PKCE states and also removes the existing states from meta table |
||
|
384aa5ba62 |
fix: issue 9355 (#9706)
### **User description** This PR fixes the issue in #9355 Alert doesn't get triggered if the field seleceted in a non uds field ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Enforce UDS fields in alerts and pipelines UIs - Filter stream fields by defined_schema_fields - Always include timestamp and _all fields - Add `validate` flag to search API calls - Backend search utils: error on non-UDS fields - New alert UI validation: `validateConditionsAgainstUDS` ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Select stream"] --> B["Fetch schema & settings"] B -- "Apply UDS filter" --> C["Available fields list"] C --> D["Build alert conditions"] D --> E["validateConditionsAgainstUDS"] E -- valid --> F["Submit alert"] E -- invalid --> G["Show error notification"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><details><summary>4 files</summary><table> <tr> <td><strong>utils.rs</strong><dd><code>Return error on non-UDS search fields</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-96bb9f4a8f547c821f26c957e8505d0437095b08d2b36687531f71565d287bd6">+1/-2</a> </td> </tr> <tr> <td><strong>alertValidation.ts</strong><dd><code>Always pass `validate:true` in alert SQL validation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-3164a4a58c1865cfcbbba084ac339664c99564545be4d57504dd406d59d8ad4a">+1/-0</a> </td> </tr> <tr> <td><strong>QueryEditorDialog.vue</strong><dd><code>Include `validate:true` in query trigger search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-6c524f609ed374aa29e075e8d046ac06be0edda7bfa1914b0a2bbc24d255e5eb">+1/-0</a> </td> </tr> <tr> <td><strong>Query.vue</strong><dd><code>Deep copy routing data and validate SQL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-6247cbd506bcf963372f2e429507fc579af95f003df246889b478da11326a657">+20/-10</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>5 files</summary><table> <tr> <td><strong>search.ts</strong><dd><code>Add `validate` flag to search URL builder</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+4/-0</a> </td> </tr> <tr> <td><strong>AddAlert.vue</strong><dd><code>Filter and validate conditions against UDS</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-7b426bb23e9ed0250c3877d6d97b099c42b2d20e22ddf4459915381b2233a0a8">+187/-0</a> </td> </tr> <tr> <td><strong>FilterCondition.vue</strong><dd><code>Add column-exists validation rule per module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-53916e2402d0ee094fcd63f0ec0c99ceffde4fa042f9001203d2e2e1b7709e10">+28/-1</a> </td> </tr> <tr> <td><strong>FilterGroup.vue</strong><dd><code>Propagate `module` prop to nested filter groups</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-8d5a73bea3aa0b5c86d48cda4a61ee3e30f27797ac2a5add1ee8bd66c75bef23">+8/-0</a> </td> </tr> <tr> <td><strong>ScheduledPipeline.vue</strong><dd><code>Adjust submit handler and add UDS filter</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-63d8f838bb880356e7804b4a57fd9223d4f2c189de7aadc7cf33cba723b29aaf">+4/-4</a> </td> </tr> </table></details></td></tr><tr><td><strong>Configuration changes</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>QueryConfig.vue</strong><dd><code>Tag QueryConfig with `module="alerts"`</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-7273c8de49b8ebf706d7bee5b275c75818c0493f41c73f00c55efe667cbd00d9">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>Condition.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/9706/files#diff-4c3e624233c9e0355c7ded3b26dc74971b867be63e24f92a5e639284f60639dd">+37/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: loaki07 <rustyloaki@gmail.com> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
|
|
e40938a08c |
feat: enrichment table file url support (#9697)
### **PR Type**
Enhancement
Design at : enrichment-file-url
___
### **Description**
- Async URL-based enrichment table processing
- Event-driven MPSC, streaming CSV, batching
- Retry with backoff, status updates
- Database and ORM additions
- `enrichment_table_urls` table and migration
- HTTP API endpoints
- POST `/url` import and GET status
- Frontend enhancements
- Tabs, status icons, retry button
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["HTTP POST /{org}/enrichment_tables/{table}/url"] --> B["DB: save_url_job"]
B --> C["trigger_url_job_processing()"]
C --> D["MPSC Channel"]
D --> E["process_url_jobs()"]
E --> F["process_single_url_job()"]
F --> G["validate_url() & fetch_and_process_batches()"]
G --> H["parse_csv_chunk() & save_enrichment_batch()"]
H --> I["save_url_job() update status"]
I --> J["notify_update()"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14
files</summary><table>
<tr>
<td><strong>url_processor.rs</strong><dd><code>Implement async URL
enrichment table processor</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-d39589e34157fe7930563b9bc5740128e168b8d8ac0a90371a4bdf1e52ce058b">+1351/-0</a></td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add URL enrichment table HTTP
endpoints</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-7b5b203002d1c3e5f671392a2a8ed84f45225896b652c2dac94ac26e72a43124">+301/-1</a>
</td>
</tr>
<tr>
<td><strong>enrichment_table.rs</strong><dd><code>Add URL job CRUD
persistence functions</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-6ce27ddf8a0d8b4e790bda90d088ff692a5da31a8d330beea8a9dd843294572b">+159/-2</a>
</td>
</tr>
<tr>
<td><strong>enrichment_table_urls.rs</strong><dd><code>SeaORM logic for
URL job records</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-e9ba1ba6e742e8d7652c3f31765de43bee061e7983bca67ffefecf8590210188">+173/-0</a>
</td>
</tr>
<tr>
<td><strong>enrichment_table.rs</strong><dd><code>Define URL job status
and job struct</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-29818b7d921497260251df82a49ae9ab7a93bbaedc0ca03abdf2b8c0726e4085">+90/-0</a>
</td>
</tr>
<tr>
<td><strong>pipelines.rs</strong><dd><code>Handle super-cluster URL job
events</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-7c8aa678045dd00de89d4c2345cf4a0511ce9bfac5d6f64e7404178c81e541ad">+23/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Expose url_processor module in
service</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-3e2b6d3b8da1f9e73aca279052a97e683184d68256c23215b18ffee885836bbf">+6/-0</a>
</td>
</tr>
<tr>
<td><strong>enrichment_table_urls.rs</strong><dd><code>Define SeaORM
entity for URL jobs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-9c3081476927bd337b2cbed8310578869845a6137a2c5305f677affb7604d39f">+26/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Register URL handlers in
router</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Expose enrichment_table_urls entity
module</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-c539d7557d1425986d5c09849d8dbca42d1b309c2bc33d49db641d6a783098f6">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add enrichment_table_urls to table
module</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-af023301203a27a1ef34f698d5a329ad7bfced65ac65e3829b1c0905a0c74fb2">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Include enrichment_table meta
module</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-bab79e8da2d0d9f903cb1a6187d922c6de63d3e4561e8ce633660b664b4e859f">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>jstransform.ts</strong><dd><code>Add client methods for URL
jobs API</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-301add2528e0f32323ca4c503bd7d460ab15ca6b9a9537192a02c5e52fd82926">+18/-0</a>
</td>
</tr>
<tr>
<td><strong>EnrichmentTableList.vue</strong><dd><code>Enhance UI for
URL-based enrichment tables</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-6a29e253f59405d3c0f756dba36afd7483f10ad86927cbbc3348ede6372c3077">+327/-74</a></td>
</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>3 files</summary><table>
<tr>
<td><strong>m20251221_000001_create_enrichment_table_urls.rs</strong><dd><code>Create
migration for URL jobs table</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-330b4b7ebbda5001f4f28050d50e219b91961d80d8369222d94d058cced57702">+189/-0</a>
</td>
</tr>
<tr>
<td><strong>config.rs</strong><dd><code>Add URL fetch configuration
options</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+24/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Register URL jobs migration in
migrator</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+2/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>openapi.rs</strong><dd><code>Document URL endpoints in
OpenAPI spec</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+5/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>auth.rs</strong><dd><code>Extend auth for enrichment_tables
URLs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+9/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>AddEnrichmentTable.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9697/files#diff-9aad551dbc051edfdb3994a664506ed6d5e27dcb64f9c2fa917b02e51a35d86e">+159/-42</a></td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
|
||
|
Hengfei Yang
|
1714f6c570 | fix: metrics stream can't be deleted (#9710) | ||
 Loakesh Indiran
|
adcc0ca2b6 |
fix: feat add validate sql query param for _search and _multi_search (#9652)
### **PR Type**
Enhancement, Tests
___
### **Description**
- Add `validate` query parameter to HTTP search handlers
- Invoke `validate_query_fields` before executing search
- Provide `get_bool_from_request` boolean extractor utility
- Add unit tests for query validation helpers
### Existing behavior:
1. lets assume sql `select continent from oly`
2. for the above sql initial run the stream oly does not have any uds ->
UI calls _search/ _search_multi to validate the query in the alerts and
pipelines it will return records with the col continent
### Problem:
1. when user now ads uds to oly without the continent col
2. the apis return response with just _timestamp col
3. and alerts and pipelines fail silently without the actual field
### Solution:
- add a validate query param to the api, when true we parse the sql get
the cols check if part of uds or not
with this approach we also needs to handle the case for queries like
joins, ctes, subqueries, union, etc (where table 1 can have the field ,
where as table 2 does not have the field part of uds)
So to handle this we honor what datafusion does, I did the testing with
datafusion, results are shared in the pr description
- Error message
```json
{
"code": 20004,
"message": "Search field not found: continent. Field exists but not in User-Defined Schema (UDS). Available UDS fields: [\"body\", \"id\", \"name\", \"total_medals\", \"unique_id\"]",
"error_detail": "",
"trace_id": "019b35dba10073aa8b9dc5064442b417"
}
```
### Testing Results
# DataFusion CLI Test Results - Field Validation Behavior
**Date:** 2025-12-20
**DataFusion CLI Version:** v51.0.0
**Purpose:** Determine field validation behavior to align OpenObserve
UDS validation with DataFusion standards
---
## Test Setup
```sql
CREATE TABLE oly (id VARCHAR, name VARCHAR, continent VARCHAR, gold_medals INT, total_medals INT);
INSERT INTO oly VALUES
('USA', 'EUA', 'AME', 40, 126),
('CHN', 'China', 'ASI', 40, 91),
('JPN', 'Japão', 'ASI', 20, 45),
('AUS', 'Austrália', 'OCE', 18, 53),
('FRA', 'França', 'EUR', 16, 64),
('NED', 'Países Baixos', 'EUR', 15, 34),
('GBR', 'Grã-Bretanha', 'EUR', 14, 65),
('KOR', 'República da Coréia', 'ASI', 13, 32),
('ITA', 'Itália', 'EUR', 12, 40),
('GER', 'Alemanha', 'EUR', 12, 33);
CREATE TABLE test1 AS SELECT * FROM oly;
```
**Result:** ✅ 10 rows inserted, test1 table created successfully
---
## Test Results
### Test 1: Basic SELECT with Existing Field
**Query:**
```sql
SELECT continent FROM oly LIMIT 5;
```
**Result:** ✅ **SUCCESS**
```
+-----------+
| continent |
+-----------+
| AME |
| ASI |
| ASI |
| OCE |
| EUR |
+-----------+
5 row(s) fetched.
```
**Conclusion:** DataFusion allows querying existing schema fields.
---
### Test 2: Basic SELECT with Non-Existent Field
**Query:**
```sql
SELECT non_existent_field FROM oly;
```
**Result:** ❌ **ERROR**
```
Schema error: No field named non_existent_field. Valid fields are oly.id, oly.name, oly.continent, oly.gold_medals, oly.total_medals.
```
**Conclusion:** DataFusion **strictly validates** that fields exist in
schema at query time.
---
### Test 3: JOIN with Existing Fields
**Query:**
```sql
SELECT a.continent AS "oly_continent", b.continent AS "test1_continent"
FROM oly AS a
JOIN test1 AS b ON a.name = b.name
LIMIT 5;
```
**Result:** ✅ **SUCCESS**
```
+---------------+-----------------+
| oly_continent | test1_continent |
+---------------+-----------------+
| AME | AME |
| ASI | ASI |
| ASI | ASI |
| OCE | OCE |
| EUR | EUR |
+---------------+-----------------+
5 row(s) fetched.
```
**Conclusion:** DataFusion allows JOIN queries when all fields exist in
their respective table schemas.
---
### Test 4: JOIN with Non-Existent Field
**Query:**
```sql
SELECT a.non_existent_field, b.continent
FROM oly AS a
JOIN test1 AS b ON a.name = b.name;
```
**Result:** ❌ **ERROR**
```
Schema error: No field named a.non_existent_field. Valid fields are a.id, a.name, a.continent, a.gold_medals, a.total_medals, b.id, b.name, b.continent, b.gold_medals, b.total_medals.
```
**Conclusion:** DataFusion **validates per-table fields** in JOIN
queries. If a field doesn't exist in table 'a', it fails even if table
'b' has valid fields.
---
### Test 5: Subquery with Existing Field
**Query:**
```sql
SELECT a.name
FROM oly AS a
WHERE a.continent IN (
SELECT b.continent FROM test1 AS b
)
LIMIT 5;
```
**Result:** ✅ **SUCCESS**
```
+-----------+
| name |
+-----------+
| EUA |
| China |
| Japão |
| Austrália |
| França |
+-----------+
5 row(s) fetched.
```
**Conclusion:** DataFusion allows subqueries when all fields exist in
their respective schemas.
---
### Test 6: Subquery with Non-Existent Field
**Query:**
```sql
SELECT a.name
FROM oly AS a
WHERE a.non_existent_field IN (
SELECT b.continent FROM test1 AS b
);
```
**Result:** ❌ **ERROR**
```
Schema error: No field named a.non_existent_field. Valid fields are a.id, a.name, a.continent, a.gold_medals, a.total_medals.
```
**Conclusion:** DataFusion **validates fields in subquery WHERE
clauses**. Non-existent fields cause immediate errors.
---
### Test 7: UNION with Existing Fields
**Query:**
```sql
SELECT continent FROM oly
UNION
SELECT continent FROM test1
LIMIT 5;
```
**Result:** ✅ **SUCCESS**
```
+-----------+
| continent |
+-----------+
| OCE |
| EUR |
| AME |
| ASI |
+-----------+
4 row(s) fetched.
```
**Conclusion:** DataFusion allows UNION queries when fields exist in all
involved tables.
---
### Test 8: CTE with Existing Field
**Query:**
```sql
WITH oly_data AS (
SELECT name, continent FROM oly
)
SELECT continent, COUNT(*) as cnt
FROM oly_data
GROUP BY continent;
```
**Result:** ✅ **SUCCESS**
```
+-----------+-----+
| continent | cnt |
+-----------+-----+
| AME | 1 |
| ASI | 3 |
| EUR | 5 |
| OCE | 1 |
+-----------+-----+
4 row(s) fetched.
```
**Conclusion:** DataFusion allows CTEs when all fields exist in the base
table schema.
---
### Test 9: CTE with Non-Existent Field
**Query:**
```sql
WITH oly_data AS (
SELECT name, non_existent_field FROM oly
)
SELECT non_existent_field
FROM oly_data;
```
**Result:** ❌ **ERROR**
```
Schema error: No field named non_existent_field. Valid fields are oly.id, oly.name, oly.continent, oly.gold_medals, oly.total_medals.
```
**Conclusion:** DataFusion **validates fields in CTE definitions**.
Non-existent fields in CTEs cause immediate errors.
---
## Summary of Findings
| Test Case | Query Type | Field Status | DataFusion Behavior |
|-----------|-----------|--------------|---------------------|
| 1 | Basic SELECT | Exists | ✅ SUCCESS |
| 2 | Basic SELECT | Non-existent | ❌ ERROR |
| 3 | JOIN | Both exist | ✅ SUCCESS |
| 4 | JOIN | Non-existent in one table | ❌ ERROR |
| 5 | Subquery | Exists | ✅ SUCCESS |
| 6 | Subquery | Non-existent | ❌ ERROR |
| 7 | UNION | Exists | ✅ SUCCESS |
| 8 | CTE | Exists | ✅ SUCCESS |
| 9 | CTE | Non-existent | ❌ ERROR |
---
## Key Conclusions
1. **Strict Schema Validation:** DataFusion performs **strict schema
validation** at query planning time.
2. **No Permissive Behavior:** DataFusion does **not allow** queries
with non-existent fields, regardless of query complexity.
3. **Per-Table Validation:** In JOIN queries, DataFusion validates
fields **per table** - each table reference is validated against its own
schema.
4. **Early Validation:** All field validation happens during query
planning, **before execution**.
5. **Clear Error Messages:** DataFusion provides helpful error messages
listing valid fields when validation fails.
---
## Recommendation for OpenObserve
**OpenObserve should adopt DataFusion's strict validation approach:**
- ✅ Validate all queried fields exist in stream schema
- ✅ Validate per-table in JOIN queries (each table against its own
schema/UDS)
- ✅ Reject queries with non-existent fields at query planning time
- ✅ Apply same validation to subqueries, CTEs, and UNION queries
- ✅ Provide clear error messages listing available fields
This aligns OpenObserve behavior with industry-standard SQL engines and
prevents confusing runtime errors.
---
**Test completed successfully. All results documented for validation
strategy decision.**
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Receive search request"] --> B["Extract `validate` flag"]
B -- "true" --> C["validate_query_fields"]
C -- "error" --> D["map_error_to_http_response"]
C -- "ok" --> E["Proceed to search execution"]
B -- "false" --> E
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Add query validation in search
handler</code>
</dd></summary>
<hr>
src/handler/http/request/search/mod.rs
<ul><li>Extract <code>validate_query</code> flag from query
parameters<br> <li> Call <code>utils::validate_query_fields</code> when
flag is true<br> <li> Return HTTP error on validation failure</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9652/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>multi_streams.rs</strong><dd><code>Add query validation
in multi-stream handler</code>
</dd></summary>
<hr>
src/handler/http/request/search/multi_streams.rs
<ul><li>Extract <code>validate_query</code> flag in multi-stream
search<br> <li> Validate SQL fields with
<code>validate_query_fields</code><br> <li> Short-circuit and map errors
if validation fails</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9652/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+15/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>utils.rs</strong><dd><code>Implement query validation
utilities</code>
</dd></summary>
<hr>
src/handler/http/request/search/utils.rs
<ul><li>Implement <code>get_bool_from_request</code> boolean
extractor<br> <li> Add <code>validate_query_fields</code> to check SQL
columns against schema<br> <li> Provide helper functions: range adjust,
stream resolve, system-field <br>check<br> <li> Include unit tests for
query helpers</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9652/files#diff-96bb9f4a8f547c821f26c957e8505d0437095b08d2b36687531f71565d287bd6">+192/-2</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
 
|
27704a0d4c |
fix: add alert revamp v2 (#9694)
### **PR Type**
Enhancement, Tests
___
### **Description**
- Introduce `generate_sql` API endpoint for alerts
- Implement request/response models & service linkage
- Register route & update OpenAPI documentation
- Add extensive front-end tests for alert wizard
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["POST /v2/{org_id}/alerts/generate_sql"] -- "validate & check perms" --> B["build_sql service"]
B -- "return generated SQL & metadata" --> C["GenerateSqlResponseBody"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>4
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Add `generate_sql` alert API
endpoint</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-83431801d27626ef27482e67b62661f900043544a45ddf958bd027cf2fa3bd8d">+150/-6</a>
</td>
</tr>
<tr>
<td><strong>responses.rs</strong><dd><code>Add `GenerateSqlResponseBody`
and metadata</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-224027f90893122eea28e35778467acf1a4b5572ce612fc9bc9ec44ca8ebd9fb">+25/-0</a>
</td>
</tr>
<tr>
<td><strong>requests.rs</strong><dd><code>Add `GenerateSqlRequestBody`
request model</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-412f11e06c0e901e2292aab80536cb36edebcc4796dec52923389c3aaa32b25a">+16/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Make `build_sql` function
public</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-ba9b65130898444f7815fd6068bdb0dc590612f74cc893f79d91b0afc44cc851">+1/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Register `generate_sql` service
route</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>openapi.rs</strong><dd><code>Include `generate_sql` in
OpenAPI spec</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+1/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>3
files</summary><table>
<tr>
<td><strong>AlertSettings.spec.ts</strong><dd><code>Add comprehensive
unit tests for AlertSettings</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-4ca37593c11ef263e0b9a9e1733678e2b81f01c8da5f4a8d729306c176802356">+1363/-0</a></td>
</tr>
<tr>
<td><strong>AddAlert.spec.ts</strong><dd><code>Extend AddAlert tests for
SQL & wizard flows</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-84362398fea8abd9acf30f4f40ba8d4cf27348677454017bd4e6f568d5176ac6">+808/-56</a></td>
</tr>
<tr>
<td><strong>CompareWithPast.spec.ts</strong><dd><code>Add tests for
CompareWithPast component</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-5718f3c114808926e2cced290580cb8ae1610338042eebad001e9a7d5429ee77">+596/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>45 files</summary><table>
<tr>
<td><strong>AddAlert.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-7b426bb23e9ed0250c3877d6d97b099c42b2d20e22ddf4459915381b2233a0a8">+1295/-409</a></td>
</tr>
<tr>
<td><strong>AlertList.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-be3e41b823e163b226ffc270d95cc4981e59c600851e4e0d10ee5e4ee994ac8e">+58/-63</a>
</td>
</tr>
<tr>
<td><strong>AlertSummary.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-c22aa1c5934b763f1d4a5a61a7e810b6bcb1d60b76b64c61cb3ce89dc0a04499">+169/-0</a>
</td>
</tr>
<tr>
<td><strong>AlertWizardRightColumn.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-eb14abefd15bc1d3db947347b49899fe264138b7aea63729feae27ad7abed606">+851/-0</a>
</td>
</tr>
<tr>
<td><strong>AlertWizardRightColumn.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-f599a1960d1ac37e352854ad368d23e080b3cec2b217df22f40d0e15a4db6986">+291/-0</a>
</td>
</tr>
<tr>
<td><strong>CustomConfirmDialog.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-0fa5a1da2050cbab522801af8ade04a0e4c27e87933d21f6c32a82603c77a0a3">+179/-0</a>
</td>
</tr>
<tr>
<td><strong>DeduplicationConfig.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-63eb39111409780e4dc064ea80056dfe096038ee76b389cdd8e9d14c0ae29a32">+3/-2</a>
</td>
</tr>
<tr>
<td><strong>FilterGroup.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-8d5a73bea3aa0b5c86d48cda4a61ee3e30f27797ac2a5add1ee8bd66c75bef23">+39/-39</a>
</td>
</tr>
<tr>
<td><strong>HorizontalStepper.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-ccca09440e61bf97621fe06ca606ade4f7fc7c1df8f8a3742cafa4c352f2c23a">+370/-0</a>
</td>
</tr>
<tr>
<td><strong>OrganizationDeduplicationSettings.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-f526d40b50f1e96f025973c6fc251f7a50f698c641c7749b32e6bb90b9ba032f">+19/-21</a>
</td>
</tr>
<tr>
<td><strong>PreviewAlert.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-6d5bfc82662dd55768e258ab92676c2559c054d235ef0e48beb4bd90f31ca60b">+795/-0</a>
</td>
</tr>
<tr>
<td><strong>PreviewAlert.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-760f5f0a261910f17eb167f81802e8a52477b99a83d2d7fc1aeefc7185801795">+41/-16</a>
</td>
</tr>
<tr>
<td><strong>QueryEditorDialog.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-6c524f609ed374aa29e075e8d046ac06be0edda7bfa1914b0a2bbc24d255e5eb">+981/-0</a>
</td>
</tr>
<tr>
<td><strong>RealTimeAlert.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-c3ae6b47832eb33f9ca877cc7165d079de14850e5c5d43a227102ba40326b9d8">+43/-24</a>
</td>
</tr>
<tr>
<td><strong>ScheduledAlert.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-5e143a17ab0fa5d7817f6d9301667fb5597236f27184248c7a71728e85b8d999">+304/-270</a></td>
</tr>
<tr>
<td><strong>Advanced.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-6dacd8b2947f53e7c39a54e990ee26ff6d63478b3a68897ab3afbc937c65c647">+683/-0</a>
</td>
</tr>
<tr>
<td><strong>Advanced.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-eae849f0bf6c8868f5640ce1d74862284d8e8a5330e5046c0c437e6672f4f676">+376/-0</a>
</td>
</tr>
<tr>
<td><strong>AlertSettings.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-1d5bfd6a1ea672884193649b1271d751d6f3b8ec84af759071c9bfd169a8ab28">+1441/-0</a></td>
</tr>
<tr>
<td><strong>AlertSetup.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-ed5a1c54e5fb03ce1d8121223d40606c74419b998b6c071667037347a2debde9">+878/-0</a>
</td>
</tr>
<tr>
<td><strong>AlertSetup.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-4f89438c74f329de730b0e7451d4a8d7df3bab52cdeaded776e8ae18d94294eb">+290/-0</a>
</td>
</tr>
<tr>
<td><strong>CompareWithPast.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-6606848e30caef2cda13d65cb30d6590205556259e7e6daf39cada825502d55b">+416/-0</a>
</td>
</tr>
<tr>
<td><strong>Deduplication.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-d3281cd7632a0fa7ce5c5054e80efafb2aa8d272bb2c586d723436b644565ba4">+591/-0</a>
</td>
</tr>
<tr>
<td><strong>Deduplication.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-f8d256bcec370851c9799c13ffa2b7571c07ca27af1b5a74c335e7084157c9b1">+204/-0</a>
</td>
</tr>
<tr>
<td><strong>QueryConfig.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-5a3ac3e9d93e87bc95c87a20f08c9972158a5882d66ecfb658d38255fe1d8f71">+1000/-0</a></td>
</tr>
<tr>
<td><strong>QueryConfig.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-7273c8de49b8ebf706d7bee5b275c75818c0493f41c73f00c55efe667cbd00d9">+594/-0</a>
</td>
</tr>
<tr>
<td><strong>de.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-78f24ecb2dbcfc38fdf3ec22b2559a5b40d236ea689d1c55f5d3e87168310264">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>en.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+89/-15</a>
</td>
</tr>
<tr>
<td><strong>es.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-ac2a2167b8b973e3f776f9fe3b43bef4576470d8ad33f3d7cfaff6edff0e5778">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>fr.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-db80e762b88ddf183820c050cdc6b215e6b02b365b5e73f5152af46b0d26f1a2">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>it.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-2de78074a5f887afddce60d694ead9322e6ab223eb70251c13819f3028bdc70d">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>ja.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-092e49c5f9fbc1b539ab9c2866d108d0ce1d314022b4c294eb4611b285e7bbd3">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>ko.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-50e27ca1b4825a5026f3a251bfcf79c97cab07cfade2ebd512d332453d707c48">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>nl.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-a1706021687ae5d8a1c006c49eab3946174fecd895f9b70a89c2e76d97250906">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>pt.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-38e804f64cb5e0e451849fc84e9b46e53c954a4a4263d51c18665ad45f4b4815">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>tr.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-6f0f131e9e2591480c4c5d1eaffc72d02dc31cbd8d3f90d8f012f08cabac8818">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>zh.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-8818055db58f83230aeea4ebfd4cec31d924e8c47b6f91298e356959ab25339e">+70/-0</a>
</td>
</tr>
<tr>
<td><strong>alerts.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-7a62782cceacac92b787e058219b3c53ea7406d4be5780def899943a01f500c1">+9/-0</a>
</td>
</tr>
<tr>
<td><strong>app.scss</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-5703d12504b0395cbfbbfad02a592bc391496026e5f03743102ce06c210ea05f">+7/-0</a>
</td>
</tr>
<tr>
<td><strong>alertQueryBuilder.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-5c829ce5fd3ad2f5ef359df5976e6adfc52eaa6b5f249f594c8b8a588acf8b3e">+10/-49</a>
</td>
</tr>
<tr>
<td><strong>alertSummaryGenerator.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-a4464ad7df3ac3afdca4e6fc583122d701bace10e3a2daf083cb6c902565b20f">+393/-0</a>
</td>
</tr>
<tr>
<td><strong>alertValidation.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-3164a4a58c1865cfcbbba084ac339664c99564545be4d57504dd406d59d8ad4a">+8/-8</a>
</td>
</tr>
<tr>
<td><strong>conditionsFormatter.spec.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-12eb070ce171e81117000fefaf2a1037af4c337423344f280c21c77e1836f5e3">+237/-0</a>
</td>
</tr>
<tr>
<td><strong>conditionsFormatter.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-d5df71a7ab2451925cd220828fc3776039e56c206a07ac7b72d4ce5e282c257c">+460/-0</a>
</td>
</tr>
<tr>
<td><strong>focusManager.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-987334b24e56735ac755d8e1add737ffcd35fe11612e31586c21824cbea88c36">+179/-0</a>
</td>
</tr>
<tr>
<td><strong>zincutils.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9694/files#diff-d3d59688cccdcc6c84ebe88b698c6910030abecc3125f4c0ef81b0df023699ba">+11/-0</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com>
Co-authored-by: Uddhav Dave <uddhav@openobserve.ai>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
|
||
|
ac60e181dd |
feat: dashboard join support for auto sql (#9343)
# Nested Functions and Joins - User Guide ## Overview This guide covers two powerful features that enable sophisticated data analysis in OpenObserve: - **Nested Functions**: Combine multiple functions together to create complex calculations - **Joins**: Merge data from multiple streams (data sources) based on common fields These features work in the **Auto SQL** mode of dashboard panels, allowing you to write advanced queries without deep SQL knowledge. --- ## What's New ### Why These Features? **Before:** - Limited function composition - you could only use basic, single-level functions - No way to combine data from multiple streams in one query - Complex analytics required exporting data and manual processing **Now:** - Compose unlimited nested functions with full DataFusion support - Join multiple streams directly in the dashboard panel editor - Create sophisticated analytics queries with a visual interface ### Key Benefits ✓ Calculate complex metrics (error rates, percentiles, conditional aggregations) ✓ Build advanced dashboards without custom code ✓ Support for all DataFusion aggregate and scalar functions --- ## Nested Functions ### What Are Nested Functions? Nested functions let you combine multiple functions together - using the output of one function as input to another. Think of it like a cooking recipe where each step builds on the previous one. **Simple Example:** - Basic: `AVG(response_time)` - calculates average response time - Nested: `ROUND(AVG(response_time), 2)` - calculates average AND rounds it to 2 decimal places ### Supported Functions OpenObserve supports **all DataFusion functions**, giving you powerful analytics capabilities: **Aggregate Functions** (work across multiple rows): - `count`, `sum`, `avg`, `min`, `max` - basic aggregations - `median`, `stddev`, `variance` - statistical functions - `percentile_cont` - calculate percentiles (e.g., p95, p99) - `string_agg` - combine text values **Mathematical Functions**: - `round`, `ceil`, `floor` - rounding operations - `abs`, `power`, `sqrt` - mathematical operations - `log`, `log10`, `log2` - logarithmic functions - `sin`, `cos`, `tan` - trigonometric functions **String Functions**: - `concat`, `upper`, `lower`, `trim` - `substring`, `length`, `replace` **Conditional Functions**: - `CASE WHEN ... THEN ... ELSE ... END` - conditional logic - `COALESCE` - return first non-null value 📖 **Full Reference**: [DataFusion Aggregate Functions](https://datafusion.apache.org/user-guide/sql/aggregate_functions.html) | [DataFusion Scalar Functions](https://datafusion.apache.org/user-guide/sql/scalar_functions.html) --- ### Using Nested Functions (Step-by-Step) #### Step 1: Create or Edit a Dashboard Panel Open your dashboard and create a new panel or edit an existing one. Make sure you're in **Auto SQL** mode (not Custom SQL). #### Step 2: Select a Function In the Y-Axis or column configuration, click **Select Function** to choose your base function. <img width="795" alt="Nested Function - Adding Layers" src="https://github.com/user-attachments/assets/0b6b134e-59b8-48b1-a4ae-52fb4800d772" /> In this example, we're using the `Concat` function to combine multiple fields (namespace and pod name). #### Step 3: Add Nested Layers Click the function icon to add another function layer. You can nest functions multiple times. <img width="793" alt="Nested Function - Select Base Function" src="https://github.com/user-attachments/assets/4bd9e03e-1104-4e82-b2ec-a34e6bb64870" /> Here we're building a nested function: - Inner layer: `Avg(severity)` - calculates average severity - Outer layer: `Ceil()` - rounds up the average to the nearest integer #### Step 4: Build Complex Expressions For advanced analytics, you can combine nested functions with conditional logic using `CASE WHEN` statements. <img width="1340" alt="Complex Nested Function with CASE WHEN" src="https://github.com/user-attachments/assets/4c3e3623-bfad-44c4-a760-e10cc94d4a95" /> This example shows a sophisticated calculation: ```sql ROUND(100.0 * SUM(CASE WHEN COALESCE(severity, 0) >= 4 THEN 1 ELSE 0 END) / COUNT(_timestamp), 2) ``` **What this does:** 1. `COALESCE(severity, 0)` - replaces null severity with 0 2. `CASE WHEN severity >= 4 THEN 1 ELSE 0` - returns 1 for high severity, 0 otherwise 3. `SUM(...)` - counts high severity events 4. Divide by `COUNT(_timestamp)` - calculates percentage 5. Multiply by `100.0` - converts to percentage 6. `ROUND(..., 2)` - rounds to 2 decimal places **Result**: Percentage of high-severity events, rounded to 2 decimals #### Step 5: View the Generated SQL Switch to the **Raw** tab to see the complete SQL query that was generated from your nested function configuration. <img width="1388" alt="View Generated SQL" src="https://github.com/user-attachments/assets/d7f6111a-26c6-4dd0-9e9f-e26092fe6368" /> You can review the auto-generated SQL in the Query section at the bottom of the panel editor. The chart visualization updates in real-time as you build your nested functions. --- ## Joins ### What Are Joins? Joins allow you to combine data from two or more different streams (data sources) based on common fields between them. **Example:** **employee_data_join** stream: ```json [ {"employee_id": 1, "name": "Alice", "department_id": 1}, {"employee_id": 2, "name": "Bob", "department_id": 2}, {"employee_id": 3, "name": "Charlie", "department_id": null}, {"employee_id": 4, "name": "David", "department_id": 1}, {"employee_id": 5, "name": "Eve", "department_id": 3} ] ``` **dept_data_join** stream: ```json [ {"department_id": 1, "department_name": "HR"}, {"department_id": 2, "department_name": "Engineering"}, {"department_id": 3, "department_name": "Sales"}, {"department_id": 4, "department_name": "Marketing"} ] ``` A join lets you combine these streams to see the department name alongside each employee, matched by `department_id`. ### Types of Joins OpenObserve supports three types of joins: | Join Type | Description | When to Use | |-----------|-------------|-------------| | **INNER JOIN** | Returns only records with matches in both streams | When you need data that exists in both sources | | **LEFT JOIN** | Returns all records from left stream + matching records from right | When you want all data from your primary source, with supplemental info where available | | **RIGHT JOIN** | Returns all records from right stream + matching records from left | When you want all data from your secondary source, with supplemental info where available | > **Note:** UNION and CROSS joins are not currently supported. --- ### Using Joins (Step-by-Step) #### Step 1: Access the Joins Section In your dashboard panel editor, locate the **Joins** section. Click the **+** button to add a join. <img width="1725" alt="Access Joins Interface" src="https://github.com/user-attachments/assets/35db4f28-c526-43a2-bdd1-470e188f53d7" /> In this example: - **Primary stream**: `employee_data_join` (contains employee names) - We'll join it with `dept_data_join` (contains department information) - The interface shows available fields from both streams in the left panel #### Step 2: Configure the Join Click on the **Joins** dropdown and select the stream you want to join. Then configure: 1. **Joining Stream**: Select the stream to join (e.g., `employee_data_join`) 2. **Join Type**: Choose from LEFT, INNER, or RIGHT using the visual toggle 3. **On Stream**: Select the target stream (e.g., `dept_data_join`) <img width="1725" alt="Configure Join Type" src="https://github.com/user-attachments/assets/35db4f28-c526-43a2-bdd1-470e188f53d7" /> The interface displays: **"Performing a Left Join between employee_data_join and dept_data_join"** Notice the visual toggle showing three join types: - **Left** (selected in this example) - **Inner** - **Right** #### Step 3: Define the Joining Clause Set up the join condition by specifying which fields to match between the two streams: **Clause 1:** - **Select Field** (Left): Choose a field from the first stream (e.g., `department_id`) - **Select Operator**: Choose comparison operator (usually `=`) - **Select Field** (Right): Choose the matching field from the second stream (e.g., `department_id`) You can add multiple clauses using the **+** button for complex join conditions (e.g., joining on multiple fields or adding additional filters). #### Step 4: View the Results After configuring the join, the data table will show combined results from both streams. <img width="1724" alt="Join Results" src="https://github.com/user-attachments/assets/94ca9f33-6cfa-4ad1-9b84-bee9db34f912" /> In this example: - **Name** column from `employee_data_join` (Alice, Bob, Charlie, David, Eve) - **Department Name** column from `dept_data_join` (HR, Engineering, HR, Sales) - Data successfully joined on `department_id` #### Step 5: Review the Generated SQL Check the **Query** section at the bottom to see the generated SQL: ```sql SELECT employee_data_join.name as "x_axis_1", stream_0.department_name as "x_axis_2" FROM "employee_data_join" LEFT JOIN "dept_data_join" AS stream_0 ON employee_data_join.department_id = stream_0.department_id ``` The SQL query shows exactly how OpenObserve is combining your data streams. --- --- ## Examples ### Example 1: String Concatenation with Nested Functions **Use Case**: Combine multiple fields into a single display value ```sql Concat(k8s_namespace_name, '-', k8s_pod_name) ``` This joins namespace and pod names with a hyphen separator. ### Example 2: Statistical Analysis with Nested Math Functions **Use Case**: Calculate rounded average of a metric ```sql Ceil(Avg(severity)) ``` This calculates the average severity level and rounds up to the nearest whole number. ### Example 3: Conditional Percentage Calculation **Use Case**: Calculate percentage of high-severity events ```sql ROUND(100.0 * SUM(CASE WHEN COALESCE(severity, 0) >= 4 THEN 1 ELSE 0 END) / COUNT(_timestamp), 2) ``` **Breakdown:** - Handles null values with `COALESCE` - Counts events where severity >= 4 - Calculates percentage of total events - Rounds to 2 decimal places ### Example 4: Employee-Department Join **Use Case**: Show employee names with their department information ```sql SELECT employee_data_join.name as "Name", dept_data_join.department_name as "Department Name" FROM employee_data_join LEFT JOIN dept_data_join ON employee_data_join.department_id = dept_data_join.department_id ``` This displays all employees with their department names, showing null for employees without a department assignment. ### Example 5: Complex Analytics - Joins + Nested Functions **Use Case**: Calculate error rate per service with deployment information ```sql SELECT logs.service_name, deployments.version, COUNT(*) as total_requests, ROUND((COUNT(CASE WHEN logs.status_code >= 500 THEN 1 END) * 100.0 / COUNT(*)), 2) as error_rate_percent FROM service_logs as logs LEFT JOIN deployments ON logs.service_name = deployments.service_name GROUP BY logs.service_name, deployments.version ORDER BY error_rate_percent DESC ``` This combines: - Join between service logs and deployment data - Nested functions for conditional counting and percentage calculation - Grouping and sorting for analysis --- ## Tips and Best Practices ### Nested Functions Best Practices ✅ **Start Simple**: Build your function one layer at a time - Begin with inner function: `AVG(response_time)` - Add outer layers: `ROUND(AVG(response_time), 2)` ✅ **Test Each Layer**: Verify results before adding complexity - Test the inner function first - Add one layer at a time and check output ✅ **Handle Null Values**: Use `COALESCE` to avoid unexpected nulls ```sql COALESCE(AVG(response_time), 0) ``` ✅ **Use Clear Aliases**: Name your calculated fields descriptively ```sql ROUND(AVG(response_time), 2) as avg_response_ms ``` ✅ **Check for Division by Zero**: Protect against zero denominators ```sql CASE WHEN COUNT(*) > 0 THEN (SUM(errors) * 100.0 / COUNT(*)) ELSE 0 END as error_rate ``` ### Joins Best Practices ✅ **Choose the Right Join Type**: - **INNER JOIN**: Both streams must have matching data - **LEFT JOIN**: Keep all records from primary stream - **RIGHT JOIN**: Keep all records from secondary stream ✅ **Verify Field Compatibility**: - Match data types (don't join string to number) - Ensure field names are correct - Check that values actually match between streams ✅ **Consider Performance**: - Join on indexed fields when possible - Test with limited data first --- ## FAQ ### Q: Can I nest more than 2 functions? **A**: Yes! You can nest as many functions as needed. Example: ```sql ROUND(CEIL(AVG(COALESCE(response_time, 0))), 2) ``` ### Q: What's the difference between LEFT and RIGHT joins? **A**: - **LEFT JOIN**: Keeps all records from the first (left) stream - **RIGHT JOIN**: Keeps all records from the second (right) stream - Choose based on which dataset you want to preserve completely ### Q: Can I join more than 2 streams? **A**: Yes! Click the **+** button in the Joins section to add additional joins. You can chain multiple joins together. ### Q: Do nested functions impact performance? **A**: Complex nested functions may take longer to compute, especially on large datasets. Test with small time ranges first and optimize your expressions. ### Q: Can I use Custom SQL instead of Auto SQL? **A**: Yes! If you prefer writing raw SQL, switch to **Custom SQL** mode. However, Auto SQL provides a visual interface that's easier for building queries. --- ## Additional Resources 📚 **DataFusion Documentation**: - [Aggregate Functions](https://datafusion.apache.org/user-guide/sql/aggregate_functions.html) - COUNT, SUM, AVG, etc. - [Scalar Functions](https://datafusion.apache.org/user-guide/sql/scalar_functions.html) - ROUND, CONCAT, CASE, etc. 🔗 **OpenObserve Resources**: - [GitHub Repository](https://github.com/openobserve/openobserve) - [PR #9343 - Feature Implementation](https://github.com/openobserve/openobserve/pull/9343) - [OpenObserve Documentation](https://openobserve.ai/docs/) 💬 **Get Help**: - [GitHub Issues](https://github.com/openobserve/openobserve/issues) - Report bugs or request features - [Community Discussions](https://github.com/openobserve/openobserve/discussions) - Ask questions and share ideas --- Design at: https://github.com/openobserve/designs/tree/main/joins --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
|
Sai NikhilandByteBaker
|
e1e91ad54f |
fix: incident analysis changes (#9673)
### **PR Type**
Enhancement
___
### **Description**
- Backend: refactor agent_chat_stream for pooling & immediate status
- Frontend: rotating “Observing…” UX & stream event handling
- Incident drawer → full-screen dialog with two-column tabs layout
- Minor UI tweaks: dialogs maximized & list wrapper update
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["agent_chat_stream endpoint"] -->|"get_agent_client()" B["Global pooled client"]
B -->|"yield processing status" C["text/event-stream start"]
C -->|"client.query_stream()" D["External RCA Agent Service"]
D -->|"stream bytes" E["Forward chunks to client"]
D -->|"on error" F["Yield error event"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>chat.rs</strong><dd><code>Refactor agent chat streaming
& pooling</code>
</dd></summary>
<hr>
src/handler/http/request/agent/chat.rs
<ul><li>Re-export enterprise AgentChat types under
cfg(feature="enterprise")<br> <li> Provide stub types &
init_agent_client for OSS builds<br> <li> Use global OnceCell client for
connection pooling<br> <li> Rewrite streaming: send immediate
"processing" status and handle <br>errors</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9673/files#diff-a0e13f17674a3657690f9e5829da2f1c4a6a957fe8a95f9c44b854643b5fe9b7">+68/-52</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>main.rs</strong><dd><code>Extend AI init to Agent
client</code>
</dd></summary>
<hr>
src/main.rs
<ul><li>Extend AI init to include agent client startup<br> <li> Update
OpenAPI init log messages</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9673/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+3/-3</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>SREChat.vue</strong><dd><code>Enhance chat loading UX &
stream handler</code>
</dd></summary>
<hr>
web/src/components/SREChat.vue
<ul><li>Replace static loading text with rotating “Observing…”
messages<br> <li> Add start/stop rotation & updateAssistantMessage
helpers<br> <li> Support new event types: status, tool_call,
tool_response, error<br> <li> Improve spacing, formatting and error
handling in stream parser</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9673/files#diff-a24e3f6392181aa741979fe1c0cbc6f0f02337de20e6ad2fd6668dd7abe9bada">+131/-62</a></td>
</tr>
<tr>
<td>
<details>
<summary><strong>IncidentDetailDrawer.vue</strong><dd><code>Refactor
incident drawer to dialog UI</code>
</dd></summary>
<hr>
web/src/components/alerts/IncidentDetailDrawer.vue
<ul><li>Switch from QDrawer to full-screen QDialog wrapper<br> <li>
Implement two-column layout with tabs for analysis & triggers<br> <li>
Add tile components for status, severity, alerts, timeline<br> <li>
Themed styling, responsive scrollbars and updated actions</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9673/files#diff-6973d58a5f49658b07d3533d4db5d32cc6430433b51732481e7497243dadb0b0">+531/-179</a></td>
</tr>
<tr>
<td>
<details>
<summary><strong>IncidentList.vue</strong><dd><code>Wrap incident detail
in QDialog</code>
</dd></summary>
<hr>
web/src/components/alerts/IncidentList.vue
<ul><li>Wrap IncidentDetailDrawer in QDialog instead of drawer<br> <li>
Update v-model and close event handling</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9673/files#diff-953ad015996758a8c86e489499cd3b0a614501b99b0c9c5c242a299e34c24ff2">+11/-4</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>TracesAnalysisDashboard.vue</strong><dd><code>Maximize
traces dashboard dialog</code>
</dd></summary>
<hr>
web/src/plugins/traces/metrics/TracesAnalysisDashboard.vue
- Change dialog maximized prop from false to true
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9673/files#diff-d27dd619532ec278c376c09a05d5106fdb38863c3aa1c56e109fc2145a8a5a3a">+1/-1</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: ByteBaker <42913098+ByteBaker@users.noreply.github.com>
|
||
|
udev
|
61dbc6ff7e |
fix: pipeline/alert history and trigger schema merge (#9665)
### **User description**
We need to ensure that whenever a trigger stream is being created (be it
meta or non-meta org) we have the complete schema of the stream. This
will prevent history queries from failing with `field not present` error
We need to make pipeline history query against own orgs trigger stream
we need to create filter with the alert ID for single alert
___
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- fix alert history filtering to use key suffix
- fix pipeline history query to own org triggers
- enhance triggers schema initialization logic
___
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>history.rs</strong><dd><code>Use key suffix filter for
alert history</code>
</dd></summary>
<hr>
src/handler/http/request/alerts/history.rs
<ul><li>updated alert_id filter to use suffix match<br> <li> removed
LIKE prefix filter escaping logic</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9665/files#diff-1c7483ca7a66b00f13cefdecadd02afbdf4baf93df5629adb8abe5942cfe5357">+3/-4</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>history.rs</strong><dd><code>Query pipeline history
from own org</code>
</dd></summary>
<hr>
src/handler/http/request/pipelines/history.rs
<ul><li>replaced META_ORG_ID with dynamic org_id<br> <li> applied own
org context for count and data queries</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9665/files#diff-aa1f4b2b5539a10dbabffb51724c916aac3d6b60e25ec150b54a32a06a6fc5d4">+2/-3</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>triggers_schema.rs</strong><dd><code>Enhance triggers
schema initialization logic</code>
</dd></summary>
<hr>
src/service/self_reporting/triggers_schema.rs
<ul><li>removed META_ORG_ID conditional exclusion<br> <li> eliminated
pre-existence schema check<br> <li> added expected_schema equality check
before merge</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9665/files#diff-135de2980c6c20b6e5c8294e6bfd2c1c3dedb5995a6671a74726579e77ac983c">+16/-21</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
 
|
06df3fb784 |
feat: Issue 9646 promql streaming (#9648)
### **User description**
Design at: dashboard
___
### **PR Type**
Enhancement
___
### **Description**
- Enable HTTP/2 streaming for PromQL queries
- Extend streaming search util with PromQL support
- Add PromQL streaming endpoints in service layer
- Define TypeScript interfaces for PromQL streaming
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Dashboard Loader"] -- "calls stream" --> B["fetchQueryDataWithHttpStream"]
B -- "chunk data" --> C["handlePromQLResponse"]
B -- "error" --> D["handlePromQLError"]
B -- "complete" --> E["handlePromQLComplete"]
C --> F["state.data update"]
D --> G["state.errorDetail update"]
E --> H["final loading false"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>usePanelDataLoader.ts</strong><dd><code>Implement
PromQL streaming in data loader</code>
</dd></summary>
<hr>
web/src/composables/dashboard/usePanelDataLoader.ts
<ul><li>Remove non-streaming PromQL calls<br> <li> Initialize state for
streaming queries<br> <li> Process each query via
fetchQueryDataWithHttpStream<br> <li> Handle streaming chunks, errors,
completion</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9648/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+193/-73</a></td>
</tr>
<tr>
<td>
<details>
<summary><strong>useStreamingSearch.ts</strong><dd><code>Add PromQL
streaming to HTTP streaming hook</code>
</dd></summary>
<hr>
web/src/composables/useStreamingSearch.ts
<ul><li>Extend StreamResponseType with <code>promql_response</code><br>
<li> Support <code>promql</code> type in fetch/initiate functions<br>
<li> Build GET URL for PromQL streaming endpoint<br> <li> Map
<code>promql_response</code> via convertToPromQLResponse</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9648/files#diff-8fce3e25a161cd458e734bd9b1f98aadefe3482da7b6529a5b1f4fa20ec115e2">+51/-16</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>streaming_search.ts</strong><dd><code>Add PromQL
streaming endpoints to service</code>
</dd></summary>
<hr>
web/src/services/streaming_search.ts
<ul><li>Add <code>promqlQueryRangeStreamUrl</code> endpoint builder<br>
<li> Add <code>promqlQueryRange</code> streaming request method<br> <li>
Include query, time window and metadata params</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9648/files#diff-6a8e19a226bbf9e7e04903d5818c8fb5c9005a48988419f94d3e63a61ade7c57">+78/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Documentation</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>query.ts</strong><dd><code>Define PromQL streaming
TypeScript interfaces</code>
</dd></summary>
<hr>
web/src/ts/interfaces/query.ts
<ul><li>Introduce <code>PromQLStreamingPayload</code> and
<code>PromQLQueryPayload</code><br> <li> Add
<code>PromQLStreamingResponse</code> interface</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9648/files#diff-12dd5b3bb9741bbeaa348c0efbbd745d0a582899678738159c6927d34a35191d">+21/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: Huaijin <haohuaijin@gmail.com>
Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com>
|
||
|
Ashish Kolhe
|
bd647bd341 |
fix: organizations ep permisions issue (#9667) (#9669)
fixes: https://github.com/openobserve/openobserve/issues/9670 ### **PR Type** Bug fix ___ ### **Description** - Prioritize `META_ORG_ID` user in token validator - Prioritize `META_ORG_ID` user in credentials validators - Fallback to first available user when no META membership ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Incoming request"] --> B["get_all_users()"] B --> C{"META_ORG_ID user exists?"} C -- "yes" --> D["Select META org user"] C -- "no" --> E["Select first user"] D --> F["Continue validation"] E --> F ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>token.rs</strong><dd><code>Prioritize META org user in token validation</code> </dd></summary> <hr> src/handler/http/auth/token.rs <ul><li>Added logic to prioritize <code>META_ORG_ID</code> user<br> <li> Fallback to first user if META org missing<br> <li> Inserted explanatory comment for permission rationale</ul> </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/9669/files#diff-3339b4a964c121766ef87fcec0070f27a2b7085cf81281b53154e53d33cb28cf">+6/-1</a> </td> </tr> <tr> <td> <details> <summary><strong>validator.rs</strong><dd><code>Prioritize META org user in credentials validators</code> </dd></summary> <hr> src/handler/http/auth/validator.rs <ul><li>Applied META org prioritization logic<br> <li> Replaced <code>all_users.first()</code> with META find or fallback<br> <li> Updated both <code>validate_credentials</code> and <code>validate_credentials_ext</code></ul> </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/9669/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+14/-2</a> </td> </tr> </table></td></tr></tr></tbody></table> </details> ___ |
||
|
Yashodhan Joshi
|
330a179d1c | refactor: file list dump and stream stats (#9236) | ||
|
udev
|
19446a3d32 |
fix: empty user email on grpc metrics ingest (#9608)
check auth can return req without populating metadata, this pr fixes the case and adds an `IngestUser` type to mark all system initiated ingestion calls and user initiated ingestion calls |
||
|
Huaijin
|
6246b7bb27 | feat: add peak memory usage to usage report (#9609) | ||
|
sktrpathiandsai nikhil kethe
|
e52d000217 |
feat: add incident correlation system (#9507)
## Summary Introduces comprehensive incident management system to reduce alert fatigue through intelligent correlation and AI-powered root cause analysis. **Design at:** https://github.com/openobserve/designs/tree/main/alert-correlation --- ## Incident Correlation **3-Tier Correlation Priority:** 1. **Trace-based**: Groups alerts from same distributed trace (cross-service) 2. **Service Discovery**: Uses semantic service matching with normalization 3. **Dimension-based**: Blake3 hash from stable alert dimensions (fallback) **Features:** - Automatic alert grouping into incidents - Real-time incident status tracking (open/acknowledged/resolved/suppressed) - Topology enrichment via Service Graph integration - Cross-environment correlation prevention via dimension matching ## Incident Analysis (RCA) **Automated Analysis:** - Background job queries open incidents and triggers AI agent - Periodic analysis via configurable interval - Results displayed in incident detail view with markdown support **On-Demand Analysis:** - Manual trigger via API: `POST /api/v2/{org}/alerts/incidents/{id}/rca` - Real-time results with loading indicators - One-click "Analyze" button in UI ## SRE Chat Assistant **Conversational AI for Alerts:** - Generic `/api/{org}/agent/chat` endpoint with history support - Context-aware assistance for alerts and incidents - Sparkle icons on alert/incident rows for quick access - Persistent chat history via IndexedDB - Includes RCA analysis in conversation context --- ## Configuration ```bash # Incident Management O2_INCIDENTS_AUTO_RESOLVE_AFTER_MINUTES=-1 # -1=disabled (default), >0=auto-resolve # RCA Agent Integration O2_INCIDENTS_RCA_ENABLED=true O2_INCIDENTS_RCA_AGENT_URL=http://localhost:8000 O2_INCIDENTS_RCA_INTERVAL_SECS=600 ``` ## API Endpoints - `GET /api/v2/{org}/alerts/incidents` - List incidents with pagination - `GET /api/v2/{org}/alerts/incidents/{id}` - Get incident details - `PATCH /api/v2/{org}/alerts/incidents/{id}` - Update incident status - `GET /api/v2/{org}/alerts/incidents/stats` - Get incident statistics - `POST /api/v2/{org}/alerts/incidents/{id}/rca` - Trigger RCA analysis - `POST /api/{org}/agent/chat` - Generic agent chat endpoint - `POST /api/{org}/agent/chat/stream` - Streaming agent chat ## Database - **New tables**: `alert_incidents`, `alert_incident_alerts` - **Migration**: `m20251204_000001_create_alert_incidents_table` - **ORM**: SeaORM entities with full CRUD support ## UI Components - **IncidentList.vue**: Status filtering, pagination, statistics dashboard - **IncidentDetailDrawer.vue**: Full incident view with triggers/alerts/analysis - **SREChat.vue**: AI assistant panel with markdown rendering - Incidents tab in AlertList with unified view - AI sparkle icons for contextual chat access --- ## Enterprise Feature This is an **enterprise-only** feature with OSS stubs for compilation compatibility. - Non-enterprise builds: Handlers return `403 Forbidden` - Enterprise builds: Full functionality enabled ## Test Plan - [x] Build passes without `--features enterprise` - [x] Build passes with `--features enterprise` - [x] Create alerts that fire and verify incidents are created - [x] Verify incident list/detail APIs work correctly - [x] Verify status updates (open → acknowledged → resolved) - [x] Verify correlation logic (trace-based, service discovery, dimension-based) - [x] Verify RCA analysis triggers and displays correctly - [x] Verify SRE chat integration with alert/incident context --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Loakesh Indiran
|
afc0d18d9b |
fix: return 401 Unauthorized instead of 403 Forbidden for authentication failures (#9564)
## Problem APIs were returning **403 Forbidden** when tokens were invalid or expired, which is incorrect. According to HTTP specifications: - **401 Unauthorized**: Authentication failure (invalid/expired token, user not found) - **403 Forbidden**: Authorization failure (authenticated but lacks permissions) ## Changes - [x] Previously, short URL paths would redirect to `/web/` on auth failure. Now they consistently return 401 like all other endpoints. ### token.rs - Changed `ErrorForbidden` to `ErrorUnauthorized` when token validation fails (`is_valid == false`) - Changed `ErrorForbidden` to `ErrorUnauthorized` when user does not exist in the system - Changed `ErrorForbidden` to `ErrorUnauthorized` for non-enterprise builds (unsupported auth method) - Kept `ErrorForbidden` only for `check_permissions()` failures (true authorization issues) |
||
|
Yashodhan JoshiandYashodhan Joshi
|
4780a9edc6 |
fix: rbac for native user for license call (#9529)
Fixes an issue where a native user in non-default account would get 401 for license api call, whereas we want all users to have get access to license call. --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> |