mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 00:09:35 +00:00
187cdd72f4670cf2a63122fbdef4ce9fd30e53d9
920
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ByteBaker
|
187cdd72f4 |
fix: make WAL_DIR_DEFAULT_PREFIX pub
https://github.com/openobserve/openobserve/pull/8876 had made this variable private, resulting in failed enterprise builds. It also removed one of the imports causing build failure. This commit reverts those specific changes. |
||
|
ByteBaker
|
d8e8e27661 |
fix: address code review feedback on graceful drain API
- Fix panic on path canonicalization in check_has_pending_files() - Implement proper recursive directory checking for parquet files - Add exponential backoff to draining loop (50ms -> 1s) - Validate query parameter strictly in enable endpoint to prevent accidental node disablement These changes address critical robustness issues identified in the code review: 1. Prevents potential panics during shutdown when WAL path is missing 2. Ensures correct pending file detection instead of assuming all subdirectories have files 3. Reduces CPU usage during drain with exponential backoff 4. Prevents unintentional node disablement from invalid/missing parameters |
||
|
ByteBaker
|
2bbb915862 |
feat: add graceful drain API for Kubernetes autoscaling
Implements cloud-agnostic graceful shutdown mechanism to support Kubernetes preStop hooks and other orchestration platforms. Changes: - Add IS_INGESTER_DRAINING global flag with atomic operations - Enhance /enable?value=false to trigger flush and set draining mode - Add /drain_status endpoint to monitor shutdown readiness - Modify parquet upload job to prioritize draining mode - Exit upload loop when all files are uploaded to S3 API Usage: - PUT /node/enable?value=false - Initiates drain (flushes memory, sets flag) - GET /node/drain_status - Returns readyForShutdown status Benefits: - Eliminates fixed 900s wait in Helm charts - Cloud-agnostic (works with K8s, ECS, Docker Swarm, systemd) - Data-safe: ensures all WAL files uploaded before shutdown |
||
 Sai Nikhil
|
803434bd30 | fix: theme customization (#9077) | ||
 Hengfei Yang
|
561ddd3b4d | fix: add some debug logs for query slow (#9072) | ||
 Simha Gandsai nikhil kethe
|
e92f385751 |
feat: Alert and Pipeline History columns are now sortable (#9052)
# PR Type Enhancement # Description The columns in the Alerts and Pipelines History are augmented to be sortable. The backend endpoint is updated accordingly too. --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
ByteBaker
|
d0cf8d3fc3 |
chore: utilize with_capacity in collections + cleanup
|
||
 
|
85a8fc1648 |
feat: license check (#8522)
### **User description** This adds license check job ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add enterprise license HTTP APIs and DB watch - Enforce quota checks on search and ingestion - Always enable usage reporting; simplify modes - Expose license expiry in config and UI ___ ### Diagram Walkthrough ```mermaid flowchart LR Backend["Backend: License APIs + quota checks"] DB["DB: License watch + dist lock"] Config["Config: usage/reporting & expiry"] UI["UI: Settings License page + banner"] Search["Search/PromQL: quota enforcement"] Backend -- "watch updates" --> DB Backend -- "expose expiry" --> Config UI -- "GET/POST /api/license" --> Backend Search -- "check_search_allowed" --> Backend ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>21 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Validate usage mode and publish interval bounds</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+16/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add license GET/POST endpoints with validation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-d569000673e415b222a20a6f4f50a93c278c189a7d6e43231d5ab8807e93021f">+75/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register license routes for enterprise builds</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Enforce search quota in PromQL endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-0ad5729a1fbce00ff53076728c51822400c13af9918e1d8554f41be56d060fa0">+24/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add enterprise search quota checks in endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+51/-1</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Enforce quota in streaming search endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+21/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Expose license expiry; always enable usage flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+8/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire license endpoints into router</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+3/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add license check locking and startup watchers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+57/-0</a> </td> </tr> <tr> <td><strong>license.rs</strong><dd><code>Implement license update broadcast and watch</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-43bb2d7d70d03657337dfabb078079de71fae8a7f58962a10ac68c03594e0f81">+70/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Export license DB module for enterprise</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Enforce ingestion quota for enterprise</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+10/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Provide search quota check helper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+18/-0</a> </td> </tr> <tr> <td><strong>ingestion.rs</strong><dd><code>Simplify reporting mode; always requeue on error</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-3edd5f4ed1eed0f765b7b450ef0fbf587632e6b22302006a5e397049e297057f">+19/-50</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Always enable usage queue and publishing</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-c6255010f7a87d6acdb3ad04954fc9f2c27350ae319c53a1857b120aa06408a3">+2/-25</a> </td> </tr> <tr> <td><strong>useManagementRoutes.ts</strong><dd><code>Add license management route</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-b71a1d39b1fab0749f0668d5f16382536fbda6e2f0c92c56a972fb5d8b5835f5">+8/-0</a> </td> </tr> <tr> <td><strong>license_server.ts</strong><dd><code>Add client for license APIs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-3ac3fe8faca5f5a6aebd9d5e8885837f1511834eb7c78d0830af8f00c70523cb">+16/-0</a> </td> </tr> <tr> <td><strong>License.vue</strong><dd><code>New license management UI with usage</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-614c7b48f6f2ab1e8041d20c7583b6864285eacf9db032c27508b76f6de8467f">+469/-0</a> </td> </tr> <tr> <td><strong>index.vue</strong><dd><code>Add License tab and routing guards</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-699c63979f851c48ad440a078e93282fd797a18fc065e5331b2e41122ddb8b69">+27/-0</a> </td> </tr> <tr> <td><strong>LicensePeriod.vue</strong><dd><code>License expiry banner component</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-14f02914aaa78ef3e82717173fa95455b7f1092faff2d0dbd51e243663850ad4">+111/-0</a> </td> </tr> <tr> <td><strong>HomeView.vue</strong><dd><code>Show license expiry banner and navigate</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-9af441af191d3254ef5f9db603dca75486de7638cda41a3c050e7b3c33d68280">+9/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>schema.rs</strong><dd><code>Remove usage_enabled dependency in org creation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-4696a44abde2df4ac48fed716f930bfdcc01759bdfcfd8d802821c6dd857bff9">+1/-3</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>en.json</strong><dd><code>Add i18n key for License tab label</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8522/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+2/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
 Simha GandSubhradeep Chakraborty
|
24ac8ee0ff |
fix: Add RBAC to the Alerts and Pipelines history endpoints and add tests (#8930)
# PR Type Feature/Fix # Description The pipelines history and alerts history endpoints are now covered by RBAC. Added basic e2e api test for the same endpoints. --------- Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com> |
||
  Abhay PadamaniandKirtan Thakkar
|
098ee9d7d2 |
fix: dashboard panel re-position (#9016)
- #9020 --------- Co-authored-by: Kirtan Thakkar <kirtan@kiara.tech> |
||
|
Yashodhan JoshiandYashodhan Joshi
|
a5ea45bd54 |
fix: fix middleware check (#8986)
There were few issues with the middleware, this attempts to fix it. --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> |
||
|
ByteBaker
|
eedebaeec8 |
feat: enable dual reporting and add alert history UI
Changes alert trigger reporting behavior to write to both the originating organization and `_meta` org by default, with new UI for viewing history. **Backend:** - Change `ZO_USAGE_REPORT_TO_OWN_ORG` config default from `false` to `true` - Update alert history API to query organization's own `triggers` stream - Add comprehensive backend tests for history API and config **Frontend:** - Integrate history metrics into main alerts table (`Total Evaluations`, `Firing Count`) - Add alert details drawer showing properties and evaluation timeline - Display formatted timestamps, status badges, and performance metrics - Support click-outside and ESC key to close drawer - Add i18n translations for history-related labels This eliminates cross-org API calls for alert history while maintaining global monitoring capability in `_meta` org. |
||
 Loakesh Indiran
|
c7ee02a9f1 |
fix: aggs cache ratio reporting (#8984)
Signed-off-by: loaki07 <rustyloaki@gmail.com> |
||
|
Loakesh Indiran
|
6b76fbf785 | feat: clear cache main (#8919) | ||
|
Yashodhan JoshiandYashodhan Joshi
|
9492f23353 |
fix: add a middleware for blocked org checks (#8951)
This adds a midddleware for blocking the requests for invalid orgs before they reach the handler level. --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> |
||
|
ByteBaker
|
1593db0e22 |
feat: skip aud validation for MCP-invoked requests
- Add `x-o2-mcp` header support for MCP request identification in token validator - Wrap OpenAPI schema types with `inline()` for better documentation generation - Update AI module imports to use new agent namespace - fix compilation for `cloud` feature, also return early if user isn't in trial - Enable service graph by default |
||
 Simha GandBhargav
|
8bd7b62672 |
fix: Alert and Pipelines history recovery + Pagination fix + View fixes (#8935)
# PR type fix/enchancement # Description The pagination issue with alerts and pipeline history is fixed - now works as normally expected. A few elements of the views were improved with better choices of representation. A previous PR about styling changes has removed the Pipelines History related changes partially. This was recovered and restored to functional state while also taking in the styling changes. --------- Co-authored-by: Bhargav <BJP232004@GMAIL.COM> |
||
 udev
|
ad74be0b9e | fix: cloud build (#8938) | ||
  Ashish Kolheandomkark06
|
196512057c |
feat: log patterns (#8927)
PR introduces below features: - [x] Reporting triggers to source org - [x] Alerts Insights for detailed visibility in alerts Enterprise features: - [x] Patterns extraction during query - UI and API - [x] Patterns extraction as part of ingestion process (No Sampling today) - [x] Settings to enable pattern extraction --------- Co-authored-by: omkark06 <omkar@zinclabs.io> |
||
 Harsh Mahajanandsai nikhil kethe
|
b267159579 |
feat: adding built-in regex patterns (#8827)
### **PR Type**
Enhancement, Documentation
___
### **Description**
- Add built-in regex patterns fetch API
- Introduce GitHub data service with caching
- Handle duplicate pattern names gracefully
- UI to browse and import built-ins
___
### Diagram Walkthrough
```mermaid
flowchart LR
cfg["Config: regex_patterns_source_url"] -- used by --> ghsvc["GitHubDataService"]
ghsvc -- fetch_json/cache --> adapter["PyWhatAdapter.fetch_built_in_patterns"]
adapter -- returns --> api["GET /api/{org}/re_patterns/built-in"]
api -- consumed by --> websvc["web regexPatterns.getBuiltInPatterns()"]
websvc -- powers --> ui["BuiltInPatternsTab.vue"]
ui -- import selected --> importFlow["ImportRegexPattern.vue"]
importFlow -- create --> dbsvc["db::re_pattern::add (duplicate-safe)"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><details><summary>3 files</summary><table>
<tr>
<td><strong>config.rs</strong><dd><code>Add source URL for built-in
regex patterns</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+6/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Register built-in patterns and
reorder routes</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Export github service
module</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-1b15316505c31acccb26ba66d920fdb0894567d08d2bf1e3457716d128a67b83">+1/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>10
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>New endpoint to fetch built-in
patterns and improved save response</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-27d37a8738de89c81b540daa978e684caf2cf28803e1208d90cb4d42528ed28b">+103/-4</a>
</td>
</tr>
<tr>
<td><strong>re_pattern.rs</strong><dd><code>Add duplicate-safe creation
with auto-suffix names</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-0cb2bf6d550d07b66db7e325c6d44c6a598f2ac8e315bf9e168f4a1d9d6e745d">+47/-17</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add adapters module export for
pattern formats</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-3a24aa67600735383b8b94a435974dc9b2677b6bb3f5fb502fb7d617af8bdec0">+20/-0</a>
</td>
</tr>
<tr>
<td><strong>pywhat.rs</strong><dd><code>Implement generic pattern
adapter and filters</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-9ecd12df46c16fca3984bc46122a26568db208839f904e0b216185df43e13e08">+264/-0</a>
</td>
</tr>
<tr>
<td><strong>cache.rs</strong><dd><code>Introduce in-memory cache with
eviction and stats</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-09bcde8f6abf0cbf88628e1e6e7290e9fde7655c7903569902bf28742e958326">+130/-0</a>
</td>
</tr>
<tr>
<td><strong>client.rs</strong><dd><code>Add GitHub data client with
retry and caching</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-20a335d645493f38003faf7303124a6e85c3c2dfd765e148811aebee1ebfd890">+246/-0</a>
</td>
</tr>
<tr>
<td><strong>types.rs</strong><dd><code>Define service config, errors,
and cache types</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-35becfa9ecfcd7711485f645f09a3fe2429231d0c351f0374f20f121225fbaae">+119/-0</a>
</td>
</tr>
<tr>
<td><strong>regex_pattern.ts</strong><dd><code>Add client method to
fetch built-in patterns</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-2a0b0828141d39ab0385e0941314ead1cdd2ec3ec7b3baf5fd0c42f8f010728a">+18/-0</a>
</td>
</tr>
<tr>
<td><strong>BuiltInPatternsTab.vue</strong><dd><code>New UI to search,
filter, preview, import patterns</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-0747e124429b322a69f6ae0117ec635044059bd48fb03e90b2328aa1f79b8b28">+451/-0</a>
</td>
</tr>
<tr>
<td><strong>ImportRegexPattern.vue</strong><dd><code>Integrate built-in
tab and import flow updates</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-e8e1bcb5a02311504d7d571090eab27a270aaa71cae185ad2703023e2e70a31e">+94/-36</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>3
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Expose GitHub service modules and
types</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-90a3518cd7c8b2a8b2a2093535b64a3dc4e6ffead41e5dca7901521e0c0ba910">+39/-0</a>
</td>
</tr>
<tr>
<td><strong>.env.example</strong><dd><code>Document regex patterns
source URL env var</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-a3046da0d15a27e89f2afe639b25748a7ad4d9290af3e7b1b6c1a5533c8f0a8c">+6/-1</a>
</td>
</tr>
<tr>
<td><strong>en.json</strong><dd><code>Add i18n strings for built-in
patterns UI</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8827/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+17/-1</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
|
||
   
|
dd11cb4dae |
feat: removed http api from ui and add multistream search v2 (#8735)
Removed HTTP API support from the UI. The backend API will remain unchanged, and the UI will now communicate exclusively through the streaming API. --------- Co-authored-by: omkark06 <omkar@zinclabs.io> Co-authored-by: udev <uddhavdave1@gmail.com> Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech> Co-authored-by: hengfeiyang <hengfei.yang@gmail.com> Co-authored-by: ByteBaker <42913098+ByteBaker@users.noreply.github.com> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> Co-authored-by: ktx-akshay <akshay.patil@kiara.tech> |
||
|
Simha Gandsai nikhil kethe
|
1aba9c5169 |
feat: Pipeline history endpoint and views composed on _meta org's triggers stream contents. (#8902)
# PR Type Enhancement # Description Implements a complete Pipeline History feature that allows users to view and analyse execution history of pipelines, mirroring the existing Alert History functionality. --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Yashodhan JoshiandYashodhan Joshi
|
83f2213847 |
fix: check ingestion allowed before parsing the request (#8900)
Currently we first parse the request body, and then check if the ingestion is allowed for given org (in case of cloud, for trial period) . However, if the ingestion is not allowed, we shouldn't even spend cpu and mem for parsing the request, and instead reject it with 429 right away. This will also clean up our logs a bit, as right now there are many req, which have invalid body, and log that error, however even if they had a valid body structure, they would be rejected because trial period expiry. With this PR, the logs would be reduced for such kind of requests. Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> |
||
|
ByteBaker
|
4715205654 |
feat: add OpenAPI documentation and rate limiting to service graph endpoints
- Add Utoipa path documentation for get_service_graph_metrics endpoint - Add Utoipa path documentation for get_store_stats endpoint - Add rate limiting via x-o2-ratelimit extension (module: Traces) - Register both endpoints in OpenAPI configuration for Swagger UI - Include detailed response examples and parameter descriptions |
||
|
ByteBaker
|
e4e7274ec8 |
feat: add service graph as enterprise-only feature
Service graph automatically discovers service topology from distributed traces. OSS includes: - API routing endpoints (gated with enterprise feature flag) - UI components (functional only with enterprise backend) - Configuration options Enterprise includes: - 256-shard concurrent edge store with LRU eviction - Span matching and processing logic - Prometheus metrics and background workers |
||
|
Hengfei Yang
|
f56b2b8f17 | fix: metrics can't search parquet in wal (#8896) | ||
|
Hengfei Yang
|
d360151332 | fix: metrics data duplicate when compact (#8888) | ||
 
|
b00444102c |
feat: Alerts History api and the front end! (#8894)
# PR Type
Enhancement
# Description
This PR implements a comprehensive Alert History view for OpenObserve,
providing users with detailed visibility into alert execution history.
The feature includes a dedicated history page accessible from the alerts
list, complete with advanced filtering, search capabilities, and
detailed execution logs.
### Features Implemented
#### 1. **New Alert History Backend Endpoint**
(`/api/{org_id}/alerts/history`)
- Created new Rust handler at
`/src/handler/http/request/alerts/history.rs`
- Queries the `_meta` organization's triggers stream for alert execution
data
- Supports query parameters:
- `alert_name`: Filter by specific alert
- `start_time` & `end_time`: Time range in microseconds
- `from` & `size`: Pagination support (max 1000 records)
- Returns comprehensive execution details including status, timing,
errors, and performance metrics
#### 2. **Alert History Vue Component**
(`/web/src/components/alerts/AlertHistory.vue`)
- Full-featured table view with server-side pagination
- Column display for: Timestamp, Alert Name, Status, Type, Silenced
state, Duration, Retries, Errors, and Actions
- Smart status indicators with color-coded chips
- Responsive design with dark/light theme support
#### 3. **Advanced Search Functionality**
- **Dropdown with type-ahead search**: Users can select from existing
alerts or type custom names
- **Manual search button**: Reduces backend load by eliminating
automatic API calls
- **Debounced search removed**: Prevents excessive API requests while
typing
- Fetches alert list using the same pattern as AlertList.vue for
consistency
#### 4. **DateTime Picker Integration**
- Properly integrated DateTime component (replaced incorrect
DateTimePicker)
- Supports both relative (15m, 1h, 1d, etc.) and absolute date selection
- Default time range: Last 15 minutes
- Time values correctly handled in microseconds for API compatibility
#### 5. **Alert Details Modal**
- Non-maximized modal dialog (700px width, 80vw max)
- Organized sections with visual separators:
- Basic Information (name, status)
- Time Information (timestamp, duration)
- Alert Configuration (type, silenced state)
- Performance Metrics (evaluation time, query time, retries)
- Error/Success details with syntax highlighting
- Scrollable content with custom styled scrollbars
- Professional UI with icons and proper typography hierarchy
#### 6. **Navigation Integration**
- Added "Alert History" button to AlertList.vue with history icon
- Proper routing configuration at `/alerts/history`
- Back navigation to alerts list
- Organization-scoped data with automatic refresh on org change
<img width="1728" height="1080" alt="Screenshot 2025-10-28 at 10 18
12 AM"
src="https://github.com/user-attachments/assets/c4aa13a5-b6e1-4903-92aa-595b8d1c519c"
/>
<img width="1728" height="1080" alt="Screenshot 2025-10-28 at 10 18
42 AM"
src="https://github.com/user-attachments/assets/94f01621-607d-42a5-a692-c0210172e4e6"
/>
<img width="1728" height="1080" alt="Screenshot 2025-10-28 at 10 19
08 AM"
src="https://github.com/user-attachments/assets/5b6ca7f2-37af-4268-baaa-993dff8ea350"
/>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
||
|
Abhay Padamani
|
0c8438db57 |
fix: increase no. of grid columns for dashboard panel (#8459)
### **User description** - #8460 - Increase no. of columns in the grid from 48 to 192 columns per row. - Decrease cellHeight from `34px` to `17px`. ### **PR Type** Enhancement ___ ### **Description** - Add dashboard schema version v6 - Extend API/DB handling for v6 - Migrate FE grid to 192 columns - Add v5→v6 layout width migration ___ ### Diagram Walkthrough ```mermaid flowchart LR FE["Frontend grid 48→192 cols"] -- "w,x scaled x4" --> MIG["Migration v5→v6"] MIG -- "version=6" --> API["API accepts/returns v6"] API -- "serde + routing" --> META["Meta dashboards v6 structs"] META -- "persist/load v6" --> DB["Infra table serialization"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Wire Dashboard v6 into meta model</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-5bf716db7e1d91dfbd427a2e320ce407c4f0c92d55d68f8e636daed1b02abd56">+24/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Introduce full v6 dashboard schema</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-4390c1715a037aea58be340f8b34c1293fd4ea8911851d17384ed700440077c3">+675/-0</a> </td> </tr> <tr> <td><strong>dashboards.rs</strong><dd><code>API models support and parse v6</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-250eeebeec9d4ac0ac271466ed83cd33fea3f121080491b340dc95d9af287259">+11/-2</a> </td> </tr> <tr> <td><strong>dashboards.rs</strong><dd><code>Persist/load v6 dashboards in DB</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-edd90c6ec720f676311aa31a635f954d559e63582f0b4c39cb9206bfb095737b">+12/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Variables extraction supports v6</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-6ae9e22ced40c50675dde5348b8d696f6559655bbbded915d7e6e88bf64b71a6">+5/-1</a> </td> </tr> <tr> <td><strong>commons.ts</strong><dd><code>Default panel layout width to 96 cols</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-17cb74f8142984fc46bbfecdeaacccc118fd5d4ab7f040a55695caaa309d104f">+2/-2</a> </td> </tr> <tr> <td><strong>convertDashboardSchemaVersion.ts</strong><dd><code>Current schema v6 and v5→v6 migration</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-b7ba5dff91b00365f729648ca24a8dfc70bd74e803300da81e5b89d29ed42c9e">+13/-1</a> </td> </tr> <tr> <td><strong>RenderDashboardCharts.vue</strong><dd><code>GridStack columns increased to 192</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-313f7c69102161242c893c863c50f1f1cd1fc19b68b5c99bf840b4b5c72f0f2e">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>5 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Init super Dashboard with v6 None</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-412e639b6e6e1d2a5571773c03c159462976db4f3b891fe326937779572aeec4">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Init super Dashboard with v6 None</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-997402d8930b95aa77124bc72920e3c7ff746286d605c94bf3fb7fe1aec62aa8">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Init super Dashboard with v6 None</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-84b6c5b1ecc108223434a008e1e382d4cc494668a12b6ac22ab457a72e308dd4">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Init super Dashboard with v6 None</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-63a64145239e03a59debc87c63764d425a36746d00f1bd6563ce2be1847b1213">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Init super Dashboard with v6 None</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-da49585bbcb02bf0f737bb68739ebc248edd15c7ad9427c0e68f7728bb4a8c17">+1/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ ___ ### **PR Type** Enhancement ___ ### **Description** - Add dashboard schema version v6 - Support v6 across API and DB - Migrate grid from 48 to 192 cols - FE migration scales layout x4 ___ ### Diagram Walkthrough ```mermaid flowchart LR V5["Dashboard v5 schema"] -- "migrate (x4 w,x)" --> V6["Dashboard v6 schema"] V6 -- "serde + hashing" --> META["Meta model supports v6"] META -- "persist/load v6" --> DB["DB table adapters"] FE["Frontend grid 48→192"] -- "GridStack column=192" --> UI["Panel layout behavior"] API["HTTP models/parsing"] -- "routes v1..v6" --> META ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>13 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Wire v6 into dashboard meta model</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-5bf716db7e1d91dfbd427a2e320ce407c4f0c92d55d68f8e636daed1b02abd56">+24/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Initialize v6 None in root wrapper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-412e639b6e6e1d2a5571773c03c159462976db4f3b891fe326937779572aeec4">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Initialize v6 None in root wrapper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-997402d8930b95aa77124bc72920e3c7ff746286d605c94bf3fb7fe1aec62aa8">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Initialize v6 None in root wrapper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-84b6c5b1ecc108223434a008e1e382d4cc494668a12b6ac22ab457a72e308dd4">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Initialize v6 None in root wrapper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-63a64145239e03a59debc87c63764d425a36746d00f1bd6563ce2be1847b1213">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Initialize v6 None in root wrapper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-da49585bbcb02bf0f737bb68739ebc248edd15c7ad9427c0e68f7728bb4a8c17">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Introduce dashboard schema v6 structures</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-4390c1715a037aea58be340f8b34c1293fd4ea8911851d17384ed700440077c3">+675/-0</a> </td> </tr> <tr> <td><strong>dashboards.rs</strong><dd><code>API models/routes support v6 dashboards</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-250eeebeec9d4ac0ac271466ed83cd33fea3f121080491b340dc95d9af287259">+11/-2</a> </td> </tr> <tr> <td><strong>dashboards.rs</strong><dd><code>Persist and load v6 in DB layer</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-edd90c6ec720f676311aa31a635f954d559e63582f0b4c39cb9206bfb095737b">+12/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Variable extraction supports v6</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-6ae9e22ced40c50675dde5348b8d696f6559655bbbded915d7e6e88bf64b71a6">+5/-1</a> </td> </tr> <tr> <td><strong>commons.ts</strong><dd><code>Default new panel width and bounds x4</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-17cb74f8142984fc46bbfecdeaacccc118fd5d4ab7f040a55695caaa309d104f">+2/-2</a> </td> </tr> <tr> <td><strong>convertDashboardSchemaVersion.ts</strong><dd><code>FE migration v5→v6 scale layout x4</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-b7ba5dff91b00365f729648ca24a8dfc70bd74e803300da81e5b89d29ed42c9e">+13/-1</a> </td> </tr> <tr> <td><strong>RenderDashboardCharts.vue</strong><dd><code>GridStack columns increased to 192</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8459/files#diff-313f7c69102161242c893c863c50f1f1cd1fc19b68b5c99bf840b4b5c72f0f2e">+1/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
001d444521 |
feat: alert dedup (#8884)
### **PR Type** Enhancement, Tests ___ ### **Description** - Introduce alert deduplication feature - Persist dedup state and migration - API/UI support for dedup config - Scheduled cleanup job added ___ ### Diagram Walkthrough ```mermaid flowchart LR Config["Config: DeduplicationConfig + grouping"] -- "exposed in Alert" --> API["HTTP Models: Alert.deduplication"] API -- "persist/read" --> Infra["Infra: alerts table cols + dedup_state table"] Infra -- "migration + entity" --> DB["DB Schema v11"] Scheduler["Scheduler handlers"] -- "apply_deduplication()" --> Service["Service: alerts/deduplication"] Service -- "fingerprint + state" --> Infra Job["Background job"] -- "cleanup_expired_state" --> Service UI["Web UI: form + i18n"] -- "bind Alert.deduplication" --> API ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Configuration changes</strong></td><td><details><summary>3 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Bump DB schema version to 11</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire enterprise-only dedup migration</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+7/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Gate deduplication module behind enterprise feature</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-ba9b65130898444f7815fd6068bdb0dc590612f74cc893f79d91b0afc44cc851">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>15 files</summary><table> <tr> <td><strong>alert.rs</strong><dd><code>Add optional deduplication to Alert model</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-45c7eaf1a764ab52f3e014fe0555c03dc59d77c5ce974684cdee4ed90463a234">+4/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Export deduplication module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-308ddd8764236c9a75da9ba7742843ee096f34a126306a58a1e695c1ee148fe5">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Expose deduplication in HTTP Alert model mappings</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-7937de550cb8784a6c3beca0689c8a88508c4b93ecf441280f47e08f988d2a5a">+7/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Map dedup fields to DB and serialize config</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-238b8087501b1743c3c4608469596d431869f021e9bb30e9a058d1294229ee67">+29/-0</a> </td> </tr> <tr> <td><strong>alert_dedup_state.rs</strong><dd><code>New entity for deduplication state</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-f6ea59e731cb65769f76f5e3a092331e433d73cfb55589861c7cd4fc415744dc">+85/-0</a> </td> </tr> <tr> <td><strong>alerts.rs</strong><dd><code>Add dedup columns to alerts entity</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-a3e9c7993f95c9f2414e5da12d12f5f60422ac927c105f32141e4854f0e7179a">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register alert_dedup_state entity</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-c539d7557d1425986d5c09849d8dbca42d1b309c2bc33d49db641d6a783098f6">+1/-0</a> </td> </tr> <tr> <td><strong>m20251024_000001_add_alert_deduplication.rs</strong><dd><code>Migration: add dedup columns and state table</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-5146fa66b9b2805cd19d8a0fe363976a5966d70132400e54e576f277c23c8d34">+245/-0</a> </td> </tr> <tr> <td><strong>alert_manager.rs</strong><dd><code>Add scheduled dedup state cleanup job</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-7e30858abad652a860186eb047347305488f52356112b1495dd53ce740316d7c">+55/-0</a> </td> </tr> <tr> <td><strong>deduplication.rs</strong><dd><code>Implement dedup logic, fingerprinting, cleanup</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-6073bba67ae904acb5f5b73104f84a81c13608f31aeb6558e9787da67efa03df">+975/-0</a> </td> </tr> <tr> <td><strong>handlers.rs</strong><dd><code>Apply deduplication before sending alerts</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-5eca8995a88fcecb6d6b8efd3c7c4496f4a96a4b060fb4ed85593574f7cf49eb">+49/-0</a> </td> </tr> <tr> <td><strong>alert.ts</strong><dd><code>Add deduplication to Alert interface</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-95f520d803aecbe4142e486037faf07124a5da245d504f0d11147bcb6a4c8918">+11/-0</a> </td> </tr> <tr> <td><strong>AlertsContainer.vue</strong><dd><code>UX: expand/collapse area clickable with hover</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-39b487d54970cff4a1ccdfbb49a4ccb38f7131bb9d30944ce1c6e86d3f818148">+12/-5</a> </td> </tr> <tr> <td><strong>DeduplicationConfig.vue</strong><dd><code>New UI for configuring alert deduplication</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-63eb39111409780e4dc064ea80056dfe096038ee76b389cdd8e9d14c0ae29a32">+401/-0</a> </td> </tr> <tr> <td><strong>ScheduledAlert.vue</strong><dd><code>Integrate dedup config into scheduled alerts (enterprise)</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-5e143a17ab0fa5d7817f6d9301667fb5597236f27184248c7a71728e85b8d999">+13/-3</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>deduplication.rs</strong><dd><code>New deduplication and grouping config with tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-ee8d9d671130757ae84ef01809003701d36e4cc90a1d1eca286d3af9a6def2ea">+183/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>en.json</strong><dd><code>i18n strings for deduplication UI</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8884/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+23/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
ByteBaker
|
ca1ca8e169 |
chore: fix MCP endpoint routing and add OpenAPI documentation
- Fix .well-known/oauth-authorization-server endpoint to use /mcp prefix - Add MCP endpoints to OpenAPI spec - Add rate limiting configuration for all MCP endpoints with module name 'mcp' |
||
|
Hengfei Yang
|
845dd1506e | perf: improve get_min_ts to reduce db calls (#8861) | ||
|
Hengfei Yang
|
e850ea2f29 | fix: read user session from db if the cache is missing (#8846) | ||
|
ByteBaker
|
bb725e94ee |
feat: add MCP OAuth 2.0 support and auth improvements
- Implement OAuth 2.0 Authorization Server Metadata endpoint (RFC 8414) - Add JWT token validation for MCP endpoints with dynamic clients - Add `WWW-Authenticate` header support for unauthorized responses - Add `AuthTokensExt::has_expired()` method for token validation - Refactor base64 decode to accept generic byte inputs - Update authentication flow to allow MCP users without DB records - Clippy delinting |
||
 udevandShrinath Rao
|
0c1268a230 |
feat: add hash replace feature in SDR (#8803)
Adds a config variable to control the length of hash UI changes to accomodate Hash pattern policy <img width="720" height="410" alt="image" src="https://github.com/user-attachments/assets/b99e28ec-2b7d-431e-9606-7b855df62f8d" /> <img width="2037" height="1112" alt="image (3)" src="https://github.com/user-attachments/assets/67432a01-c598-4b0a-830b-137a4cd04bf9" /> --------- Co-authored-by: Shrinath Rao <shnath@openobserve.ai> |
||
|
Loakesh Indiran
|
499de57b4f |
fix: return partial error for single partition on error and continue to (#8825)
### **User description** TODO: - [x] fix: return partial error for parqet file not found for single partition - [x] fix: result array skip vrl - [x] fix: values streaming api Ref: - https://github.com/openobserve/openobserve/pull/8500 - https://github.com/openobserve/openobserve/pull/8654 - https://github.com/openobserve/openobserve/pull/8760/commits/9f2bb94b695e3abdc051b7b52e5e357ea37b6921 ___ ### **PR Type** Bug fix ___ ### **Description** - Handle parquet not found error codes - Return partial results on partition errors - Improve DataFusion error string matching ___ ### Diagram Walkthrough ```mermaid flowchart LR DFErr["DataFusion error string"] -- detect "parquet not found"/"file not found" --> MapErr["Map to SearchParquetFileNotFound"] GRPCRespErr["gRPC do_get error"] -- parse JSON error code --> IsParquetNF["is_parquet_file_not_found"] IsParquetNF -- true or Cancelled/Deadline --> EmptyStream["Return empty stream with error"] EmptyStream -- continue --> RemainingPartitions["Process remaining partitions"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>grpc.rs</strong><dd><code>Expand parquet not found detection in mapper</code> </dd></summary> <hr> src/infra/src/errors/grpc.rs <ul><li>Treat "parquet file not found" like "parquet not found".<br> <li> Map both to <code>SearchParquetFileNotFound</code> with logging.</ul> </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/8825/files#diff-211c5cb8fa12eac14c3637a43d74cd114034b8a6f6f8c1ead51e56efb54141cd">+1/-1</a> </td> </tr> </table></td></tr><tr><td><strong>Error handling</strong></td><td><table> <tr> <td> <details> <summary><strong>remote_scan_exec.rs</strong><dd><code>Partial error handling for remote scan parquet-miss</code> </dd></summary> <hr> src/service/search/datafusion/distributed_plan/remote_scan_exec.rs <ul><li>Parse gRPC Internal error JSON to detect parquet-not-found.<br> <li> Return empty stream on Cancelled/Deadline or parquet-not-found.<br> <li> Continue processing remaining partitions after partial error.</ul> </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/8825/files#diff-81dd7297fe2cf5a3076270fa888bc2c49c7bd9c2c0bfcec20d7b6eb6d520032c">+9/-1</a> </td> </tr> </table></td></tr></tr></tbody></table> </details> ___ --------- Signed-off-by: loaki07 <rustyloaki@gmail.com> |
||
|
Hengfei Yang
|
5dee87a8d5 | feat: add update fields api (#8826) | ||
|
Loakesh Indiranandktx-abhay
|
826cbbd276 |
feat: partial err warning if results exceeds query limit only for (#8802)
### **User description** - show warning in dashboards if the ZO_QUERY_DEFAULT_LIMIT is reached only dashboards - truncate the results exactly to the query limit on error - remove query result error if search_type is ui ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add capped results warning on default limit - Hide cachable function errors for UI - Truncate results when default limit exceeded - Fix partition function naming in cluster paths ___ ### Diagram Walkthrough ```mermaid flowchart LR DF["DataFusion execution"] -- "row count" --> CL["check_query_default_limit_exceeded"] CL -- "set partial_err with capped msg" --> PR["Partial response propagation"] CL -- "truncate rows" --> HC["HTTP cluster response"] UI["UI search requests"] -- "filter function_error" --> HF["HTTP search handlers"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>9 files</summary><table> <tr> <td><strong>search.rs</strong><dd><code>Add capped results warning constant</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-a36a6683e327641569d8098ee0d4f13e894e1b9d51230dc3569abe42e87ba310">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Hide cachable function errors for UI</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+9/-1</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Filter cachable errors in streaming UI responses</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+15/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Standardize partial error message usage and mark partial on fn errors</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+3/-2</a> </td> </tr> <tr> <td><strong>flight.rs</strong><dd><code>Add default-limit check and rename partition function</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-dbffb1a03b65f6a9163b18c5bb1241ba2966735657be39bbe9f4d0e3260d8c1d">+12/-5</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Over-fetch default limit by 5 to detect cap</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-3b1c9f0a84041235ca3765361e8eadfd7e9309a52319da882dee0540c826eb3d">+2/-1</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Simplify partial error handling logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+0/-2</a> </td> </tr> <tr> <td><strong>leader.rs</strong><dd><code>Inject default-limit partial warning via DF path</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-21a32af318adcd10e62fb0d45944993f623fb3d2d6df4bb70d882dfc8f677fdc">+10/-3</a> </td> </tr> <tr> <td><strong>utils.rs</strong><dd><code>Add limit-exceeded checks and cachable-error helper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-bfdb4ff6a6e6bebf219b885f74dcbd3058ada7d2a7aa4c306b2e604aa1471474">+80/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Truncate results when default limit exceeded</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-fe0a1bbec7ef6418120544b589a531e5144b28cd2650f99135ed8a70ac17ac03">+12/-2</a> </td> </tr> <tr> <td><strong>follower.rs</strong><dd><code>Use corrected partition function name</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8802/files#diff-59e0cc386582cf8baf201977475fd9a7e479fd8ed000bbaa6746a51ecb28632f">+2/-2</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Signed-off-by: loaki07 <rustyloaki@gmail.com> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> |
||
|
ByteBaker
|
50e0b64aa9 |
feat: add MCP protocol endpoints
Integrate Model Context Protocol server endpoints at `/api/{org_id}/mcp`
for both enterprise and non-enterprise builds.
Changes:
- Add POST and GET handlers for MCP requests with streaming support
- Extract and pass through Authorization header for API authentication
- Add non-enterprise stub endpoints that return "not available" error
- Update router to register MCP endpoints conditionally
Endpoints:
- POST `/api/{org_id}/mcp`: Single MCP request/response
- GET `/api/{org_id}/mcp`: Streaming MCP requests
|
||
|
Hengfei Yang
|
40bbbe7f2e | fix: improve slow log detail (#8811) | ||
|
Ashish Kolhe
|
f62d0d60c5 |
feat: traces for agent (#8800)
### **PR Type**
Enhancement, Bug fix
___
### **Description**
- Add distributed tracing for AI chat
- Parse W3C traceparent, fallback to UUID v7
- Enable conditional tracer provider/init
- Improve AI error logs with trace context
___
### Diagram Walkthrough
```mermaid
flowchart LR
FE["Frontend generateTraceContext()"] -- "traceparent header" --> API["/api/{org_id}/ai/chat(_stream)"]
API -- "Span root if enabled" --> Tracer["Tracer provider"]
API -- "trace_id propagated" --> AIService["ai::service::{chat,chat_stream}"]
Tracer -- "OTLP exporters" --> OO["OpenObserve OTLP"]
Tracer -- "Filtered ai.* spans" --> OO["OpenObserve OTLP"]
Tracer -- "Optional eval exporter" --> Eval["AI Evaluation OTLP"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>chat.rs</strong><dd><code>Add tracing headers parsing
and span creation</code>
</dd></summary>
<hr>
src/handler/http/request/ai/chat.rs
<ul><li>Accept headers and org path in chat<br> <li> Create root tracing
spans when enabled<br> <li> Parse W3C traceparent to derive trace_id<br>
<li> Propagate trace_id to AI services and logs</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8800/files#diff-12a78cbb31b313cb3a0ca50f25c177470cf49b02708f179fe138ed3d66a2926b">+77/-12</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>main.rs</strong><dd><code>Conditional tracing setup and
AI exporters</code>
</dd></summary>
<hr>
src/main.rs
<ul><li>Initialize tracer when AI tracing enabled<br> <li> Add
FilteringSpanProcessor for ai.* spans<br> <li> Configure additional OTLP
exporters<br> <li> Use UUID v7 ID generator when available</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8800/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+223/-17</a></td>
</tr>
<tr>
<td>
<details>
<summary><strong>useAiChat.ts</strong><dd><code>Add traceparent to AI
chat requests</code>
</dd></summary>
<hr>
web/src/composables/useAiChat.ts
<ul><li>Generate and send traceparent header<br> <li> Pass abort signal
in fetch options</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8800/files#diff-830ee4f21bdbbd0fa5c5c0f43b0a1d0200831e07ccddc91982cf0371c801b046">+6/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>ai_chat.ts</strong><dd><code>Propagate traceparent in
service requests</code>
</dd></summary>
<hr>
web/src/services/ai_chat.ts
<ul><li>Include traceparent header in chat_stream<br> <li> Utilize
generateTraceContext utility</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8800/files#diff-d5a69d2b2a2a97e0875282b079de6092a5aa1e9cb93a4eb7ff68034b43eac3a1">+3/-1</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Miscellaneous</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>wal.rs</strong><dd><code>Remove tracing attribute from
WAL list</code>
</dd></summary>
<hr>
src/service/search/datafusion/storage/wal.rs
- Remove unused tracing instrument on list
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8800/files#diff-9ca700516d742bbc3c93f2271b27034650124c06a594f5b668144c51d7e28637">+0/-1</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>vite.config.ts</strong><dd><code>Adjust Vite dependency
optimization settings</code>
</dd></summary>
<hr>
web/vite.config.ts
<ul><li>Exclude echarts modules from optimizeDeps<br> <li> Disable
forced dependency optimization</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8800/files#diff-9430906049744cbbfe5d0b382b2466843a42cfebf755ff86fec850b0712258ef">+8/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
ByteBaker
|
162dd34277 | fix: (openapi) deprecate duplicate operationId | ||
|
ByteBaker
|
abc11250fd |
refactor: replace manual JSON deser with web::Json extractors in handlers
|
||
|
ByteBaker
|
c868605736 | chore: add tests, fmt, clippy | ||
|
ByteBaker
|
da96d301ab |
refactor: extract reusable Headers extractor for actix-web handlers
This commit: - Creates a new extractors.rs module with a generic Headers<T> extractor - Implements FromRequest trait for type-safe header deserialization - Centralizes header parsing logic to reduce code duplication across HTTP handlers |
||
|
ByteBaker
|
a288a0604a |
chore: update default dashboard version from 1 to 5
In `CreateDashboard` API, the default dashboard version when `version` key was absent in the payload used to be 1, which was wrong. It is now 5 (for v5). Going forward, this value should default to the latest version supported by OpenObserve. This is tracked by `const LATEST_DASHBOARD_VERSION`, which is automatically populated at compile time by counting the number of variants of `enum DashboardRequestBody`, requiring no human intervention beyond adding the new variant itself to the enum. |
||
|
Ashish Kolhe
|
d6339499f8 |
feat: pipeline errors on ui (#8695)
### **PR Type** Enhancement, Bug fix ___ ### **Description** - Surface last pipeline error in API - Persist pipeline errors in new table - Batch upsert pipeline errors from reporting - Safer error messages in VRL failures ___ ### Diagram Walkthrough ```mermaid flowchart LR Reporting["Self-reporting errors"] -- "batch_upsert" --> DB["Table: pipeline_last_errors"] API["GET /pipelines"] -- "join with errors" --> UIData["PipelineList.last_error"] UIData -- "render" --> ListUI["PipelinesList error icon"] UIData -- "per-node errors" --> NodeUI["CustomNode error badge"] Ingestion["VRL apply_vrl_fn"] -- "clean error message" --> Reporting ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Configuration changes</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Bump DB schema version to 10</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire migration for last errors table</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>11 files</summary><table> <tr> <td><strong>pipelines.rs</strong><dd><code>Add last_error to pipeline model</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-61bb28d7091130297c917a0530d96202fcb4b073b514410531f9fe71acfc814f">+22/-3</a> </td> </tr> <tr> <td><strong>pipeline.rs</strong><dd><code>Join pipelines with last error data</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-c69f4f94eee97ce226caddc497b2adf89e26869f2d72b0d0206dc31ab2714daf">+26/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register pipeline_last_errors entity</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-c539d7557d1425986d5c09849d8dbca42d1b309c2bc33d49db641d6a783098f6">+1/-0</a> </td> </tr> <tr> <td><strong>pipeline_last_errors.rs</strong><dd><code>Define ORM entity for last errors</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-e4ff5af0cb9c6e6343ecaad06fbb03515ace8868be6ee6d192ff0f01ee4d3e17">+22/-0</a> </td> </tr> <tr> <td><strong>prelude.rs</strong><dd><code>Export PipelineLastErrors entity</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-a818656b9fb01945155f28450270dd2ae20b67c8248c851ffa0f25e859d263ed">+1/-0</a> </td> </tr> <tr> <td><strong>m20250930_000001_create_pipeline_last_errors_table.rs</strong><dd><code>Create pipeline_last_errors table and index</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-173fe20fccfdcfa68563a13f9d69c2e6cb0901ac257d43386962bbac90f296de">+169/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Expose pipeline_errors DB module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+1/-0</a> </td> </tr> <tr> <td><strong>pipeline_errors.rs</strong><dd><code>CRUD and batch upsert for pipeline errors</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-627b6bb9ca334984b8f08f1405a67e5110c238788f165e30abda39b03f7ab68f">+331/-0</a> </td> </tr> <tr> <td><strong>queues.rs</strong><dd><code>Batch upsert pipeline errors during ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-ed15a56d4d3becc2dfb3d05af2dac7f92962ec3091199728fa953e6aed7ed767">+45/-5</a> </td> </tr> <tr> <td><strong>PipelinesList.vue</strong><dd><code>Show pipeline-level last error indicator</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-85e3ef322f3eea0fc6d74d89431ba338a1153cb82ee06bef0f7c5f439c04267d">+27/-0</a> </td> </tr> <tr> <td><strong>CustomNode.vue</strong><dd><code>Show node-level error badge and navigation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-b9d83cd503b6d3470b238accaaea0b4decf567add8ad4727c63c9e583725f408">+98/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Return sanitized VRL error messages</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+12/-8</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Clarify queuing of error data</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-c6255010f7a87d6acdb3ad04954fc9f2c27350ae319c53a1857b120aa06408a3">+2/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
sktrpathi
|
b220f55435 |
fix: handle LLM-specific errors (#8690)
### **User description**
Enterprise code now returns error when raised by LLM API. These errors
are handled in this commit.
___
### **PR Type**
Bug fix
___
### **Description**
- Map rate limit errors to 429 responses
- Normalize other AI errors to 500
- Improve error logging messages consistency
- Audit reporting updated for error paths
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["AI service call"] --> B{"Error?"}
B -- "No" --> C["200 OK with response"]
B -- "Yes" --> D{"Rate limit (429)?"}
D -- "Yes" --> E["Return 429 Too Many Requests"]
D -- "No" --> F["Return 500 Internal Error"]
E -- "Audit report" --> G["Audit trail updated"]
F -- "Audit report" --> G
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>chat.rs</strong><dd><code>Standardize AI chat error
handling and 429 pass-through</code> </dd></summary>
<hr>
src/handler/http/request/ai/chat.rs
<ul><li>Log errors using extracted message string<br> <li> Return 429
JSON on rate limit errors<br> <li> Convert other AI errors to 500
internal errors<br> <li> Align stream path with audit reporting and
responses</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8690/files#diff-12a78cbb31b313cb3a0ca50f25c177470cf49b02708f179fe138ed3d66a2926b">+40/-6</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
ByteBaker
|
a5507e9dc0 |
feat (AI): add context support to chat requests and improve type safety
- Add context parameter to PromptRequest for passing additional data - Update AiServerRequest initialization to include context across chat endpoints - Change PromptResponse role field from String to Role enum for type safety - Add Serialize trait to DashboardRequestBody for serialization support |
||
 Huaijin
|
55004257f3 | feat: impl broadcast join for enrichment table (#8672) |