openobserve

starred/openobserve
Fork 0
mirror of https://github.com/openobserve/openobserve.git synced 2026-11-05 16:10:43 +00:00
Files
T
History
Simha GandOpenObserve 78efd236ed feat: Introduce query optimization recommendation service to background jobs (#8791 )
### **User description**
# PR Type
Enhancement

# Description
This PR introduces a configurable query recommendation service / back
ground job that will take org wise usage data from metadata and generate
recommendations for the same and reingests this data into _meta org's
query_recommendation stream.


___

### **PR Type**
Enhancement


___

### **Description**
- Add enterprise query recommendation background job

- Expose config for schedule and limits

- Map meta stream to enterprise type

- Wire job init under enterprise feature


___

### Diagram Walkthrough


```mermaid
flowchart LR
  CFG["Config limits: recommendation params"]
  JOB["Background job: QueryRecommendationService"]
  CTX["OptimizerContext (engine impl)"]
  ORGS["List org IDs"]
  STREAMS["Get stream settings"]
  SEARCH["Run usage search"]
  INGEST["Ingest recommendations to _meta/query_recommendations"]

  CFG -- interval/duration/top_k --> JOB
  JOB -- uses --> CTX
  CTX -- fetch --> ORGS
  CTX -- fetch --> STREAMS
  CTX -- analyze --> SEARCH
  CTX -- publish --> INGEST
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>stream.rs</strong><dd><code>Add enterprise Stream
conversion for recommendations</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

src/common/meta/stream.rs

<ul><li>Gate enterprise import behind feature flag<br> <li> Implement
From<Stream> to enterprise meta Stream</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8791/files#diff-6f5ee19bbcc9cc7fb699fc1c0f43fc96d42698ae1168a2827b5c65f7948d5dbc">+14/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mod.rs</strong><dd><code>Wire query recommendation job
into init</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

src/job/mod.rs

<ul><li>Add enterprise-only module include<br> <li> Spawn recommendation
job at init</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8791/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+5/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>

<summary><strong>query_optimization_recommendation.rs</strong><dd><code>Implement
enterprise query recommendation service job</code>&nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

src/job/query_optimization_recommendation.rs

<ul><li>Add enterprise-only background job implementation<br> <li>
Implement engine trait with org/streams/search<br> <li> Ingest
recommendations into meta stream<br> <li> Run service using configurable
schedule</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8791/files#diff-85c42193130f71a11dc1d54d1a80a5888f1311857d5899b48f8c92973c21ae21">+124/-0</a>&nbsp;
</td>

</tr>
</table></td></tr><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>config.rs</strong><dd><code>Configure query
recommendation scheduling and limits</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

src/config/src/config.rs

<ul><li>Add recommendation duration config<br> <li> Add analysis
interval config<br> <li> Add top-k limit config<br> <li> Set sensible
defaults via env vars</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8791/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+7/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

</details>

___

Co-authored-by: OpenObserve <o2@OpenObserves-MacBook-Pro.local>
2025-10-15 12:20:36 +05:30
infra
fix: node health api (#8744 )
2025-10-08 01:59:03 +05:30