Commit Graph
227 Commits
Author SHA1 Message Date
8af3495611 fix: cloud sync (#8222)
### **User description**
This is for syncing the changes done in cloud to main branch. No changes
for non-cloud code.


___

### **PR Type**
Bug fix, Enhancement


___

### **Description**
- Add invite listing and decline APIs

- Bypass auth for user-scoped invite routes

- Improve trial-expiry handling and logging

- Add org cache and invalidation hooks


___

### Diagram Walkthrough


```mermaid
flowchart LR
  FE["Login & Invitations UI"] -- "list/accept/decline" --> API["/api/invites*, /{org}/member_subscription"]
  API -- "auth bypass for user-scoped invites" --> Auth["AuthExtractor/TokenValidator"]
  Auth -- "pending invites flag" --> LoginRedir["/config/redirect builds URL params"]
  OrgSvc["Organization Service"] -- "generate/list/decline/delete" --> InvitesDB["Org Invites"]
  OrgSvc -- "invalidate cache" --> OrgCache["Org table in-memory cache"]
  Ingest["Ingestion services"] -- "trial expired -> 429" --> Clients["Ingestion Clients"]
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>25
files</summary><table>
<tr>
<td><strong>user.rs</strong><dd><code>Expand invite payload with org and
inviter</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>auth.rs</strong><dd><code>Auth bypass and key handling for
invite routes</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+36/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>jwt.rs</strong><dd><code>Return new user and pending invite
flags; defer org creation</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+22/-4</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>token.rs</strong><dd><code>Allow invite/member routes
without org user; header injection</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-3339b4a964c121766ef87fcec0070f27a2b7085cf81281b53154e53d33cb28cf">+34/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>ingest.rs</strong><dd><code>Suppress logging for trial
expiry errors</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-24f250254ecfdbe004af9dc0ea23f8a2c0105ae0898690ed3e565089e5d8eef9">+30/-9</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>org.rs</strong><dd><code>Pending-only invites; delete invite
API; trial extend cache poke</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-b47c0c1a8a78524196176430268eecdbfaf6f46d6b95873d3cbe70cfd67182b0">+40/-4</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Include pending_invites flag in
login redirect</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>List pending invites and decline
invite endpoint</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-56897e42f9436a640bc70fddc25686485ba8ae1223635b32d252a0333309f748">+36/-1</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Wire new invite endpoints into
router</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Add TrialPeriodExpired error
variant</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>organizations.rs</strong><dd><code>Add in-memory org cache
with invalidation</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-4d87f9fe379a5d161a3e7c3c77f7f320be738e6a70c785aa5ce93cb0830c0147">+77/-7</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Start cloud billing watcher on
cloud builds</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+3/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>organization.rs</strong><dd><code>Invalidate org cache on
watch and mutations</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ce65cea926a09721a8fea14d07fe188bc2b887218e95d74a8bffefb54001da50">+4/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>user.rs</strong><dd><code>Delete invites for user
helper</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-6ee6b0fc9ed03224d4ddf0b7b631c6f81ff7d1d579bdb98d2c5d4453852eb5d6">+4/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Return trial expiry as dedicated
error</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+1/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>loki.rs</strong><dd><code>Silence trial expiry logs for Loki
ingestion</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a970c8c4785d4e4d878218615b17eec0cf4b4976121766b002c2b2f76371bfe6">+4/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>json.rs</strong><dd><code>Map trial expiry to HTTP 429 in
JSON ingest</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-506987e61095a54e47f7f208dad89f9f1bce595fa6bf1ba8a7d746f2b2ff6c4a">+15/-6</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>otlp.rs</strong><dd><code>Return 429 for trial expiry;
reduce noise</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a24391bef75a4678c50724e8b5f305ec0ee8c2c74c68be9d677765de68b5d8dd">+17/-7</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>organization.rs</strong><dd><code>Invite generation
validation; delete/decline invite flows</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-6b3c3c2b027c1158c0d41302cc69dfcd20ab68a6a4fa365cda403f838940bbb4">+48/-1</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Handle trial expiry with 429;
logging tweaks</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+34/-14</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>users.rs</strong><dd><code>Enrich invite list; optional
pending filter; cleanup on removal</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+50/-6</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>organizations.ts</strong><dd><code>Add decline subscription
API client</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-2c3bad4c7a9fcedd7a653599381149b82f9c350b177de9d72431013807d4c36f">+10/-7</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>users.ts</strong><dd><code>Add pending/accept/reject invite
APIs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a334d8d570d9bdcaa3d4c8c2a993eff009a1ec8eb499c8a3d1a292d9224159d0">+12/-4</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>InvitationList.vue</strong><dd><code>New UI to accept or
reject invites</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ee97546ff1aba36cc0b24cb20782502c7c88279850cc3d540ea5685209f10702">+409/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>Login.vue</strong><dd><code>First-login invite handling and
logo tweaks</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a135c379ef6ac8f92199ea5f7ff19a227971a9b0a87ccccf87c0268740522c2c">+178/-58</a></td>

</tr>

</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>en.json</strong><dd><code>Add invitation-related
translations</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+18/-0</a>&nbsp;
&nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

</details>

___

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com>
Co-authored-by: udev <uddhav@openobserve.ai>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: udev <uddhavdave1@gmail.com>
2025-09-02 14:04:34 +05:30
sktrpathi d87d346bba fix: OpenAPI descriptions for endpoints (#8231)
### **User description**
So far the OpenAPI descriptions for endpoint handlers was directly
derived from the doc comments, leaving little control. This also leaked
any dev-specific information into API spec.

Now the summary and description are explicitly populated for all the
endpoints, detaching them from doc comments.

Also update response body annotations wherever necessary

Also bump
- utoipa: 4.0.0 -> 5.0.0
- utoipa-swagger-ui: 4.0.0 -> 9.0.0


___

### **PR Type**
Documentation, Enhancement


___

### **Description**
- Add explicit OpenAPI summaries/descriptions

- Correct response body schemas to Object/()

- Update params/types for accuracy

- Improve endpoint examples and metadata
2025-09-01 23:34:30 +05:30
Subhra264 c49df079c7 test: add unit tests for alerts and self reporting (#8228)
### **PR Type**
Tests


___

### **Description**
- Add comprehensive unit tests for alerts

- Add extensive self-reporting queue tests

- Add serialization and model tests for usage

- Validate batching and timing behaviors


___

### Diagram Walkthrough


```mermaid
flowchart LR
  Alerts["Alert model"] -- "unit tests" --> AlertsTests["alerts::tests"]
  Reporting["Self-reporting queue/runner"] -- "unit tests" --> ReportingTests["self_reporting::tests"]
  Usage["Usage/Trigger types"] -- "serde, traits tests" --> UsageTests["usage::tests"]
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>alert.rs</strong><dd><code>Add comprehensive tests for
Alert and params</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/config/src/meta/alerts/alert.rs

<ul><li>Add tests for <code>Alert</code> defaults and equality<br> <li>
Validate unique key logic and timestamps getters/setters<br> <li> Test
<code>ListAlertsParams</code> builder methods and chaining<br> <li>
Cover ancillary fields: destinations, context, owner/editor, flags</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-45c7eaf1a764ab52f3e014fe0555c03dc59d77c5ce974684cdee4ed90463a234">+415/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mod.rs</strong><dd><code>Add tests for self-reporting
queue and runner</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/config/src/meta/self_reporting/mod.rs

<ul><li>Add async tests for <code>ReportingQueue</code>
enqueue/start/shutdown<br> <li> Test batching logic in
<code>ReportingRunner</code><br> <li> Validate debug formatting and
mixed data handling<br> <li> Add ignored test for channel-full
behavior</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-f4101fa8423f7a25f311dd82dab6796ed4ac8fef68449b5f018157dc33ca7de3">+674/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>usage.rs</strong><dd><code>Add serialization and model
tests for usage types</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

src/config/src/meta/self_reporting/usage.rs

<ul><li>Add serde tests for <code>UsageData</code>,
<code>TriggerData</code>, enums<br> <li> Test display helpers and
classification methods<br> <li> Validate <code>RequestStats</code> and
<code>Stats</code> defaults/from/serde<br> <li> Test hashing/equality
for grouping keys and enums</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-b15a6adcb163dc27993c83528105a6726c44c86d8a84c5cf5d1d382c6ec8cc29">+684/-0</a>&nbsp;
</td>

</tr>
</table></td></tr></tr></tbody></table>

</details>

___
2025-09-01 14:09:49 +05:30
sktrpathi 75fc81cecf chore: remove bogus unit tests (#8172)
### **User description**
A significant chunk of API unit tests were not testing any actual
business logic, but performing trivial actions such as
struct-field-access, derive trait validation such as `Eq`, `PartialEq`,
`Clone`, `Serialize` etc, and many of the tests were also just comparing
constant values.

They added no value to the test suite and have now been removed from the
codebase, making the tests more focused.

Additionally,
- A lot of flaky tests have been corrected. These used to fail due to
occasional race conditions when ran locally.
- Some mock setup and teardown functions have also been cleaned up,
which makes tests a lot more consistent.


___

### **PR Type**
Tests, Enhancement


___

### **Description**
- Remove trivial, non-value unit tests

- Relax flaky assertions and panics handling

- Improve test isolation via DB/cache resets

- Minor test output formatting improvements


___

### Diagram Walkthrough


```mermaid
flowchart LR
  tests["Remove trivial/boilerplate tests"] -- cleaner suite --> stability["Stabilize assertions and panics"]
  stability -- reduce flakiness --> isolation["DB/Cache resets in tests"]
  isolation -- consistent runs --> outputs["Improve test log formatting"]
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><details><summary>11
files</summary><table>
<tr>
<td><strong>organization.rs</strong><dd><code>Remove struct-field and
default-value trivial tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-4f089715a7ac4f78b45039d773f2fccbb4fbc6a409e0ab9ceee1fcf7911bc44b">+4/-325</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Drop JSON roundtrip serialization
tests; tidy imports</code>&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-c4bd542e4e07ed7706b0643086f5a7d4edbb3647c7ceedbe4bea516c5274d8bd">+1/-183</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>auth.rs</strong><dd><code>Remove Debug/PartialEq trivial
tests; format strings</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+2/-104</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>main.rs</strong><dd><code>Make tracing test tolerant; ignore
enterprise tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+23/-60</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>event.rs</strong><dd><code>Remove constants and simple
struct creation tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-689769712cbd6bd02beba078c1e1dfc188e2be39ab19a0e232b8bd12bc7f1e78">+1/-95</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>validator.rs</strong><dd><code>Ensure test isolation with
ORM init and cache clears</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+28/-22</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Make base URI tests robust to env
errors</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+25/-4</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>users.rs</strong><dd><code>Use best-effort cleanup; clear
caches; improve logs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+12/-9</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>integration_test.rs</strong><dd><code>Compact println
formatting and panic messages</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-7ff815b68943f08bef79f083c09e55ca8c18db8e25033034b12fa31f1d43dab4">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>config.rs</strong><dd><code>Remove strict zero-length
assertions on globals</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+1/-13</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>schema.rs</strong><dd><code>Remove unused constant import in
tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-a5ea87a7766cb24619f64a6100f385f4b42c47af7d33b8e2f060229c536d7418">+1/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>26 files</summary><table>
<tr>
  <td><strong>authz.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-28a815d1c4728aff854a9996de0ba3d3dafe1bea6bfb0cf2a9f4186c40f985b6">+0/-52</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>loki.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-121c166af5eab8b9cb83716afa7b65fa7856c006c809b3d910ced184c52ec587">+0/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>proxy.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-7f12a4f2483e737671ac41f9c35b411e7323e0ff2731ad92a3b309e144d1316a">+0/-46</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>service.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-61d456e7217d19bcec3920a4d1c17fd396a257c81e5394655db3632a057bdd9e">+0/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>service_account.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-878ad520e1c84500cc21ff6fc4088c8da64ff2febd79e60c0d4cc8c695c49798">+0/-80</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>traces.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-bda8182c1f17a8360337c769edade0cb9478ff1443dd6aedce7f526afbe97afe">+0/-106</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>user.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+0/-101</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>auth_tests.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-2961</a></td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2aa59ce4464e7dbabdb9367fc8bb8c1b489c914b8690577d4916870463ab12db">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>action.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-fc87c75a7df9539d7e7ee6184dfab02d4a8932323e81f95d57c05363aa44752d">+0/-18</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-5bf716db7e1d91dfbd427a2e320ce407c4f0c92d55d68f8e636daed1b02abd56">+0/-13</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>reports.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-d2117a3d67d4ec47b0017861845c92143baf917e10e6746df5dd97f925f81eeb">+0/-10</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>destinations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-e0ba33d95ef7f0eac13bdac7e750462907a0308f01c027ea206d058e282c977b">+0/-103</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>inverted_index.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-8bf9d9403f05f53f0c6fd5e6a6dbf0204ea80465b233567552cb7395d08d1c3c">+0/-22</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>meta_store.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-45018c838da377fa9ded9305a1ee752adde06acdb607addf26c95f5747537615">+0/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>ratelimit.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-fa7dc03fb526c09d5eb58021872d821fcd72eeb4d6a670224f5286012485bc3d">+0/-10</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>timed_annotations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f94b4d6d12d68e7054ad33601e666b08e4594586013dc34384fa21b1df5fe6e8">+0/-12</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+0/-34</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>m20250611_000002_populate_reports_table.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-8247af6d6f8ccb6a5a51daff4cb1e352585be2d1c785d86ae5ebfea1dbc1099c">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>folders.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-402482c22fda5dff8f33382ac11d0908f693975dec7c08b37b1c5fe24a20486a">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>bulk.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2017dbe9076c78a7e7919ed5c8738729123d630038803cef0ce2c356d6fa2c48">+0/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>ingestion.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-3edd5f4ed1eed0f765b7b450ef0fbf587632e6b22302006a5e397049e297057f">+0/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2f7f0bac01afc6e37908521d1d33d05fcd650b7a08792062976e1e7465b4a1f9">+0/-42</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>footer_cache.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f9278e53ed7131c953fc30f8b15cda801b74b81fb2d570fe7f80a3b7fc1de872">+0/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-1602f85399080c1e2838d1ef1d62af23d96f353379e49a7770026e892c954018">+0/-20</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8172/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+0/-31</a>&nbsp;
&nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

</details>

___
2025-08-28 09:14:33 +05:30
udev b4eac8f607 fix: Inc UT coverage (#8128)
increase UT coverage
improve the email regex to catch `doubledotsequence` in email
2025-08-27 12:41:11 +05:30
sktrpathi 9a11a680ae chore: add test cases (#8095)
### **User description**
- also clear over 400 clippy issues


___

### **PR Type**
Tests, Enhancement


___

### **Description**
Add extensive unit tests for enrichment storage
Expose helpers for testability in storage module
Modernize tests: clippy-friendly string formatting
Normalize constants in time-based tests
2025-08-22 21:05:46 +05:30
1d0490875b fix: is_multi_stream_search and enable_align_histogram (#8003)
### **User description**
Fixes:
- https://github.com/openobserve/openobserve/issues/7986


### **PR Type**
Enhancement, Bug fix


___

### **Description**
- Add multi-stream search histogram support

- Gate histogram alignment via query flag

- Remove unused search_type plumbing

- Propagate multi-stream flag through services


___

### Diagram Walkthrough


```mermaid
flowchart LR
  HTTP["HTTP handlers"]
  GRPC["gRPC handlers"]
  Utils["HTTP query utils"]
  SQL["SQL utils: histogram eligibility"]
  Hist["SQL: histogram query builder"]
  Svc["Search service"]
  Cache["Search cache"]
  Stream["Streaming pipeline"]

  Utils -- "parse is_multi_stream_search / enable_align_histogram" --> HTTP
  Utils -- "parse flags" --> GRPC
  HTTP -- "pass flags" --> Svc
  GRPC -- "pass flags" --> Svc
  Svc -- "eligibility check (multi/unified)" --> SQL
  Svc -- "partitioning uses align flag" --> Svc
  Svc -- "search() with multi-stream flag" --> Cache
  Cache -- "eligibility(query, multi)" --> SQL
  HTTP -- "convert_to_histogram_query(..., multi)" --> Hist
  Stream -- "do_search(..., multi)" --> Cache
  Hist -- "build single/multi-stream histogram query" --> Svc
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>15
files</summary><table>
<tr>
<td><strong>http.rs</strong><dd><code>Parse multi-stream and align
histogram flags</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>search.rs</strong><dd><code>Remove
SearchPartitionRequest.search_type field</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>sql.rs</strong><dd><code>Add multi-stream histogram
eligibility (UNION ALL)</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Pass multi-stream flag into search
service</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Wire flags; histogram conversion
uses multi-stream</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>multi_streams.rs</strong><dd><code>Partition multi: add
align flag; remove search_type</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>search_inspector.rs</strong><dd><code>Pass multi-stream flag
in search profile</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>search_stream.rs</strong><dd><code>Stream API supports
multi-stream histogram and flag</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Propagate multi-stream flag in
traces search</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Cache search considers multi-stream
eligibility</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Align partitions via flag; multi
partition API updates</code>&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>histogram.rs</strong><dd><code>Build histogram for
multi-stream queries (UNION ALL)</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>cache.rs</strong><dd><code>Streaming cache handles
multi-stream flag</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>execution.rs</strong><dd><code>Execution path passes
multi-stream flag to searches</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Stream request processing threads
multi-stream flag</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

</details>

___


___

### **PR Type**
Enhancement, Bug fix


___

### **Description**
- Add multi-stream histogram query support

- Align histogram via query flag

- Remove search_type plumbing in partitions

- Propagate multi-stream flag end-to-end


___

### Diagram Walkthrough


```mermaid
flowchart LR
  Http["HTTP handlers"]
  Grpc["gRPC handlers"]
  Utils["HTTP query utils"]
  Front["Web client"]
  Service["Search service"]
  Cache["Search cache"]
  SQLU["SQL utils: histogram eligibility"]
  Hist["Histogram query builder"]

  Front -- "is_multi_stream, align flag" --> Http
  Utils -- "parse is_multi_stream/align" --> Http
  Http -- "enable_align_histogram" --> Service
  Http -- "is_multi_stream_search" --> Cache
  Grpc -- "flags default false" --> Service
  Cache -- "eligibility(sql, multi)" --> SQLU
  Http -- "convert_to_histogram_query(..., multi)" --> Hist
  Service -- "partition(gen interval if enabled)" --> Service
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>19
files</summary><table>
<tr>
<td><strong>http.rs</strong><dd><code>Parse multi-stream and
align-histogram flags</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>search.rs</strong><dd><code>Remove search_type from
partition request</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>sql.rs</strong><dd><code>Histogram eligibility with
multi-stream UNION ALL</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Pass default multi-stream flag to
service</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Add flags to HTTP search; align
histogram handling</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>multi_streams.rs</strong><dd><code>Partition multi endpoint
supports align flag</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>search_inspector.rs</strong><dd><code>Forward multi-stream
flag in inspector search</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>search_stream.rs</strong><dd><code>Streaming search supports
multi-stream histogram</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Forward multi-stream flag in traces
helpers</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Eligibility check aware of
multi-stream</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Align histogram in partition; multi
aggregate</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>histogram.rs</strong><dd><code>Build single/multi-stream
histogram SQL</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>cache.rs</strong><dd><code>Propagate multi-stream flag in
streaming cache</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>execution.rs</strong><dd><code>Pass multi flag through
streaming execution</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Wire multi-stream flag across
streaming path</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>IndexList.vue</strong><dd><code>Use align flag in partition
requests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>usePanelDataLoader.ts</strong><dd><code>Toggle align
histogram for dashboards</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+3/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>useLogs.ts</strong><dd><code>Multi-stream histogram flow and
params</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+3/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>search.ts</strong><dd><code>API params for is_multi_stream
and align flag</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8003/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+7/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

</details>

___

---------

Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
2025-08-14 17:39:48 +05:30
748cca3ecb feat: visualization enhancement (#7456)
### **User description**
### Which query will be used?
- If query is empty will use histogram query
- Else query will be available:
    - If query has group by clause :- will use same query
    - Else will use histogram query


___

### **PR Type**
Enhancement, Tests, Bug fix


___

### **Description**
- Add server API to extract result schema

- Implement DataFusion plan visitor for projections

- Integrate visualization flow with schema and histogram

- Numerous UI fixes and UX improvements


___

### Diagram Walkthrough


```mermaid
flowchart LR
  DF["DataFusion Plan Visitor (Rust)"]
  API["HTTP API /result_schema (Rust)"]
  FE["Logs Visualize UI (Vue)"]
  Loader["Panel Data Loader (TS)"]
  Cache["Histogram Handler & Cache (Rust)"]

  DF -- "extract projections/group_by/timeseries" --> API
  API -- "JSON ResultSchemaResponse" --> FE
  FE -- "build config, set fields, decide histogram" --> Loader
  Loader -- "inject searchResponse / is_ui_histogram" --> FE
  Cache -- "normalize histogram()" --> DF
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14
files</summary><table>
<tr>
<td><strong>projections.rs</strong><dd><code>Add DataFusion logical plan
visitor to extract schema</code>&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-7a0420d3649da44b80be36d7f1043d6ff81d88395c961266882c3c700f7edff7">+651/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Add /result_schema endpoint and
wire parsing/security</code>&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+224/-2</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>cacher.rs</strong><dd><code>Improve histogram() handling and
make public</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-8c387b22b2c0a282600ee41ca702b3db60f5a284d507c0feb7418608ff062c6c">+22/-11</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Export projections
module</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-70bb941c2eef1365e5ead1a59b65c5397457b9197a4e22596e18d17db7eaf922">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>Index.vue</strong><dd><code>Integrate visualization with
schema API and histogram</code>&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-0323c3da69b369843c5d72bf4df5b00ca6b5147192a213baf331659ecf92af62">+564/-154</a></td>

</tr>

<tr>
<td><strong>VisualizeLogsQuery.vue</strong><dd><code>Field list UX,
props for searchResponse/histogram</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-979b01d943d58c5d70faeeac5948b7f02f0b2392a12b84ef07ba7f7393ab0261">+236/-174</a></td>

</tr>

<tr>
<td><strong>SearchBar.vue</strong><dd><code>Visualize toggle gating,
SQL/VRL disabled states</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-81893de4edf186b0556f610acb280c87baa2ffe67d3b48a60a866094c7635baf">+102/-52</a></td>

</tr>

<tr>
<td><strong>FieldList.vue</strong><dd><code>Readonly/visibility options
for logs visualize</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-a071c06e3e26629cb4ae4751450acc39c61fa49b07a1cc4be3711697829668d4">+13/-1</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>TransformSelector.vue</strong><dd><code>Disable transform
controls in visualize mode</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-0037d8dd9cfbacc26309b26e98ae953e7ccbd0891ac2fd01c6314ebd7eed9d2c">+11/-4</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>ChartSelection.vue</strong><dd><code>Allow restricting
selectable chart types</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-384ce178717731e2c85f1312f12a54b42ceabd0b86f7e2befb0e421ef568a6a3">+14/-11</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>PanelSchemaRenderer.vue</strong><dd><code>Accept
searchResponse and is_ui_histogram props</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-e5376e7e77bf3aaf6590c2522d8c2dbffcc95a8006ae0d6dc0040a37825367d6">+13/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>usePanelDataLoader.ts</strong><dd><code>Support injected
searchResponse and ui histogram</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+30/-5</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>visualizationUtils.ts</strong><dd><code>Add SQL alias
validator for visualization</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-c2c391afcaa0b44c33d3f4fa531a4f31f044ae036f0baa182c8c88e83de066c5">+80/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>query.ts</strong><dd><code>Extend response metadata for
visualization</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-12dd5b3bb9741bbeaa348c0efbbd745d0a582899678738159c6927d34a35191d">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Add test for compact delete value
check</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+17/-0</a>&nbsp;
&nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>auth.rs</strong><dd><code>Allow auth bypass for
result_schema route</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+3/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Register result_schema HTTP
route</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>search.rs</strong><dd><code>Add ResultSchemaResponse
contract</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+7/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>TableRenderer.vue</strong><dd><code>Render null/undefined as
empty string</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-794706ea9da3a70012a0327c0464f9fd16802f78ece26055fe951716c2d73d59">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>sqlUtils.ts</strong><dd><code>Minor formatting
fix</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-dde40e8fcb5c37b06c011ede08922c44a3c025682fbc341aee97ba046b73a0e3">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>11 files</summary><table>
<tr>
  <td><strong>playwright.yml</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-7afcd2d8f7b49bda74843f209eefb7b2da45f7e7803bf2e4bd636699b76aa2d3">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>visualise.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-c14ac1db3e4aa86190d7525e8e38c410bcbbae7f46b01890e74937cc11be67e6">+1/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>logsPage.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-8045b343491d8943869c1af74cdafab204dc49c02a70294417e73f47d35e0b59">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>visualize.spec.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-d144f23552ec150db375c89e52c3a53ede9ff3fa5dcd6092694fbe0a308d22ec">+225/-183</a></td>

</tr>

<tr>
  <td><strong>useDashboardPanel.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-fbba1700750053b9eaf38ddd30dab84531bc44131c795ebb713811e4930cd7cb">+420/-6</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>useLogs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+151/-19</a></td>

</tr>

<tr>
  <td><strong>useNotifications.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-32ff5072df9badfebc0a72c401fe337de3324f76b186a52d202eb6cc2fa71e3a">+25/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>en.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>search.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+43/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>logs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-637857fa918f47ffb02ebfc8a35b124001ef4f65fd180c467f0f153382d2dc76">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>convertDataIntoUnitValue.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-14c6fe442ef231566ec411f9e0262cdd24ec912fc67b9154eec743282ab7211a">+120/-29</a></td>

</tr>
</table></details></td></tr></tr></tbody></table>

</details>

___

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: loaki07 <rustyloaki@gmail.com>
Co-authored-by: Loakesh Indiran <66478092+Loaki07@users.noreply.github.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Co-authored-by: Yashodhan Joshi <yashodhan@openobserve.ai>
2025-08-13 13:11:44 +05:30
Loakesh Indiranandsai nikhil kethe e46f6d5b82 feat: custom histogram interval & aligned partitions (#7909)
- `is_ui_histogram` query param for search api, the backend will convert
the sql to histogram query and respond accordingly
- `search_type` for search partition query, and if search type is `ui`
only then we generate partitions with mini partition that is aligned
with histogram interval


### **PR Type**
Enhancement, Bug fix, Tests, Documentation


___

### **Description**
- Add UI histogram query conversion and flag

- Align partitions to histogram with mini

- Stream empty metadata+hints consistently

- Expose histogram eligibility and interval


___

### Diagram Walkthrough


```mermaid
flowchart LR
  FE["Web UI logs & services"] -- "is_ui_histogram, searchType" --> HTTP["HTTP search endpoints"]
  HTTP -- "convert to histogram SQL\n(resolve streams, eligibility)" --> SQL["Histogram SQL converter"]
  HTTP -- "pass search_type, flags" --> SVC["Search service"]
  SVC -- "is_eligible, gen interval" --> PART["PartitionGenerator"]
  PART -- "aligned partitions\n(+mini optional)" --> STREAM["Streaming execution"]
  STREAM -- "SearchResponse + metadata\n(empty hits emitted)" --> FE
  SVC -- "Response: histogram_interval, is_histogram_eligible, converted_query" --> FE
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>16
files</summary><table>
<tr>
<td><strong>partition.rs</strong><dd><code>Histogram-aligned partitions
with optional mini support</code>&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-348fa23375d0cea3e5fb039cc6b381d79aa1d5121e6b9792acc048d3fd02e37c">+798/-269</a></td>

</tr>

<tr>
<td><strong>execution.rs</strong><dd><code>Always emit responses;
accumulate and metadata tweaks</code>&nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+148/-152</a></td>

</tr>

<tr>
<td><strong>search_stream.rs</strong><dd><code>HTTP2 stream: UI
histogram conversion and interval handling</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+149/-69</a></td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Partition API: histogram
eligibility, interval, UI paths</code>&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+71/-5</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>histogram.rs</strong><dd><code>Convert original SQL to
histogram-safe query</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+140/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>HTTP search: UI histogram
conversion and response enrichment</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+38/-3</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>search.rs</strong><dd><code>Response/request fields for
histogram meta and streaming chunks</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+45/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>New error:
SearchHistogramNotAvailable</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36">+7/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>multi_streams.rs</strong><dd><code>Multi-stream partition
search accepts search_type</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+16/-4</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>sql.rs</strong><dd><code>Detect histogram-eligible queries
and subqueries</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+27/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>http.rs</strong><dd><code>Add query flag parser for
is_ui_histogram</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+8/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>Index.vue</strong><dd><code>UI: histogram streaming flow,
timeouts cleanup</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-0323c3da69b369843c5d72bf4df5b00ca6b5147192a213baf331659ecf92af62">+45/-12</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>IndexList.vue</strong><dd><code>Pass searchType=ui to
partition API</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>search.ts</strong><dd><code>API: support is_ui_histogram,
searchType plumbed</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+7/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>useStreamingSearch.ts</strong><dd><code>Include
is_ui_histogram in streaming URL</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8fce3e25a161cd458e734bd9b1f98aadefe3482da7b6529a5b1f4fa20ec115e2">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>usePanelDataLoader.ts</strong><dd><code>Dashboard partition
calls pass searchType</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>3
files</summary><table>
<tr>
<td><strong>cacher.rs</strong><dd><code>Adjust tests for new response
fields</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8c387b22b2c0a282600ee41ca702b3db60f5a284d507c0feb7418608ff062c6c">+6/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>SearchResult.spec.ts</strong><dd><code>Skip flaky
resetPlotChart unit test</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8ff7d14fc3cdac80eeec5550ecd89a77f015b4ebe928da9d4a76a9585d64d788">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>useLogs.spec.ts</strong><dd><code>Skip failing query
partitions test</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-842307ae8db03aa51891ddde98cfdbbabc712db1e1614c9eee99a59add395f70">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Error
handling</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>error_utils.rs</strong><dd><code>Map histogram
unavailability to 400 with code</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-d4de63bc9275a19c868f6967de0f7835bd2d7bda74ff55ce97b8fb3f9af54da9">+3/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Miscellaneous</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Export histogram SQL
module</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-dc0ab53d0ff551cf3cd12027c73d0da5e66680c0bf2d1830778639a33bac3464">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>SearchResult.vue</strong><dd><code>UI: robust
empty-histogram handling and reset watcher</code>&nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-5e65533aa4754bb6d0074fda18ffc1ecacded3b9ca6bc28a161c3f6b12f20195">+17/-15</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>SearchBar.vue</strong><dd><code>Cancel running query before
applying saved view</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-81893de4edf186b0556f610acb280c87baa2ffe67d3b48a60a866094c7635baf">+2/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>query.ts</strong><dd><code>Types: histogram interval and
eligibility fields</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-12dd5b3bb9741bbeaa348c0efbbd745d0a582899678738159c6927d34a35191d">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>2 files</summary><table>
<tr>
  <td><strong>fga.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-e7c99f37f3abbb5e0507d2760bbf67df010d9218704b38dce84c5477acecd22e">+0/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>useLogs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7909/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+154/-168</a></td>

</tr>
</table></details></td></tr></tr></tbody></table>

</details>

___

---------

Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
2025-08-11 13:36:03 +05:30
Loakesh Indiran 1ebd0679db feat: api to delete stream by time period (#7600) 2025-08-07 14:47:14 +05:30
ktx-vaidehiandAshish Kolhe 45bac67dec feat: add run ID management and panel ID for dashboard refresh (#7760)
### **PR Type**
Enhancement


___

### **Description**
- Introduce DashboardInfo struct with run/panel/tab IDs

- Extract dashboard_info from HTTP requests

- Propagate dashboard_info in backend search & stats

- Frontend: manage runId state and include IDs in requests


___

### Diagram Walkthrough


```mermaid
flowchart LR
  FE["Frontend components"] 
  HTTPRequest["HTTP Request"]
  ExtractInfo["get_dashboard_info_from_request"]
  SearchService["SearchService::search"]
  UsageStats["report_request_usage_stats"]

  FE -- "include run/panel/tab IDs" --> HTTPRequest
  HTTPRequest -- "extract dashboard info" --> ExtractInfo
  ExtractInfo -- "pass DashboardInfo" --> SearchService
  SearchService -- "attach stats with info" --> UsageStats
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody></tr></tbody></table>

</details>

___

---------

Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com>
2025-08-05 11:11:15 +05:30
sktrpathi 7e08f5ac7a chore: add UTs, run fmt and clipyy (#7753) 2025-07-30 15:55:10 +08:00
Huaijin b41bb71779 feat: upgrade datafusion to v49 (#7649) 2025-07-30 10:57:25 +08:00
sktrpathiandHengfei Yang 2da35485f5 chore: add UTs, run fmt and clippy (#7720)
### **Description**
- Add unit tests for cipher, registry, saved_view, hash utils

- Refactor `verify_decode_token` logic with algorithm conversion

- Remove async from JWT calls in HTTP handlers

- Simplify code in registry, job/cipher, wal modules

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2025-07-28 09:18:56 +05:30
ByteBaker 90fbcca801 chore: cleanup auth tests 2025-07-27 10:02:57 +05:30
ByteBaker b7e3b331b1 chore: add UTs, run fmt and clipyy 2025-07-27 10:02:57 +05:30
Hengfei Yang 65540f8e79 perf: improve concurrency for index search (#7695)
### **User description**
Add new env 
```
ZO_QUERY_INDEX_THREAD_NUM = 0
```
Default value is `CPU * 4`.


___

### **PR Type**
Enhancement, Tests


___

### **Description**
- Add env var for index search concurrency

- Configure default query_index thread counts

- Use new thread count in search filter

- Refactor file grouping and update tests


___

### Diagram Walkthrough


```mermaid
flowchart LR
  A["Load ZO_QUERY_INDEX_THREAD_NUM env"] --> B["Set query_index_thread_num default"]
  B --> C["filter_file_list_by_tantivy_index uses new count"]
  C --> D["Refactor group & repartition logic"]
  D --> E["Update grouping tests"]
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>config.rs</strong><dd><code>Add query index thread
configuration</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/config/src/config.rs

<ul><li>Introduce <code>ZO_QUERY_INDEX_THREAD_NUM</code> env var<br>
<li> Set default based on local_mode vs production<br> <li> Apply new
var in <code>check_limit_config</code></ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7695/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+9/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Formatting</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>record_batch_ext.rs</strong><dd><code>Cleanup imports
and whitespace in tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

src/config/src/utils/record_batch_ext.rs

<ul><li>Reformat import statements for arrow util<br> <li> Remove
trailing whitespace in tests</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7695/files#diff-f84aa2842a164d6e85aa05a718159e87a325cac139c59e561f6f59313c6d7c5d">+7/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>storage.rs</strong><dd><code>Enhance storage search
partitioning logic</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

src/service/search/grpc/storage.rs

<ul><li>Use <code>query_index_thread_num</code> for
target_partitions<br> <li> Log target_partitions, group_num,
max_group_len<br> <li> Compute <code>expect_group_elements</code> in
grouping<br> <li> Simplify <code>repartition_sorted_groups</code>
logic<br> <li> Remove unused <code>find_max_group_index</code><br> <li>
Update tests to use <code>expect_group_elements</code></ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7695/files#diff-ab2d867e1e2e3dd8307505c2e130039628a84f8131a9b38cd2e7b4e1c141e094">+27/-62</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mod.rs</strong><dd><code>Simplify search stream
logging</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

src/service/search/streaming/mod.rs

<ul><li>Simplify log message placeholders<br> <li> Remove unused
variables in cache log</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7695/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+1/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

</details>

___
2025-07-24 17:35:09 +08:00
41f330af67 feat: re_patterns at ingestion (#7323)
### **User description**

This PR adds support for adding and applying regexp patterns at
ingestion time to redact or drop the values from fields if matched. This
is an ent-only feature, and this PR contains the required table
migrations, along with nats code for super cluster.

This also has some additions for musl build for ent repo.

There are basic crud APIs for the pattersn as well as an test api which
can be used to test a specific pattern on given input.

___

### **PR Type**
Enhancement


___

### **Description**
- Add HTTP CRUD and test endpoints for regex patterns

- Implement DB models, migrations for patterns associations

- Integrate pattern manager into ingestion and streams

- Spawn watchers and super-cluster queue for patterns


___

### **Changes diagram**

```mermaid
flowchart LR
  HTTP["CRUD & test HTTP handlers"]
  DBService["Service DB layer"]
  DB["re_patterns & associations tables"]
  Cluster["Cluster Coordinator"]
  PatternMgr["Pattern Manager"]
  Ingest["Ingestion pipeline"]
  Stream["Stream settings update"]
  HTTP -- "invoke add/update/delete/test" --> DBService
  DBService -- "persist & notify" --> DB
  DBService -- "trigger" --> Cluster
  Cluster -- "watch events" --> PatternMgr
  Ingest -- "redact at ingestion" --> PatternMgr
  Stream -- "update associations" --> PatternMgr
```


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Add HTTP handlers for re_patterns
endpoints</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-27d37a8738de89c81b540daa978e684caf2cf28803e1208d90cb4d42528ed28b">+459/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>re_pattern.rs</strong><dd><code>Implement DB service for
re_patterns</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-0cb2bf6d550d07b66db7e325c6d44c6a598f2ac8e315bf9e168f4a1d9d6e745d">+384/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>re_pattern_stream_map.rs</strong><dd><code>Add pattern
association table ORM</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-13798240832b9110426267b3e0ebb862bd6c2ac03a3462721f5e85b5bfa88f51">+257/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>re_pattern.rs</strong><dd><code>Add re_patterns table ORM
definitions</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-f113ed92667f5e352ac7a0c1c8b5e48ea858992b92be5817c9b38f65843ef4a2">+185/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>stream.rs</strong><dd><code>Integrate pattern associations
in streams</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-41452a7fb29b4634437174fd6aecf6a9050dbfdef41b9077a21d59af3d79e9ed">+62/-2</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>re_pattern.rs</strong><dd><code>Handle super-cluster queue
for patterns</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9e64341dc2d7d50cc5acffd0340b983f37b46a9aa34c86fb5a25978f9eb761dc">+166/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>ingest.rs</strong><dd><code>Redact data via pattern manager
at ingestion</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-e62118e45be183266d39ffa08491f255017954235ff0f3d6937683e314da406f">+23/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>stream.rs</strong><dd><code>Add pattern associations to
Stream metadata</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-6f5ee19bbcc9cc7fb699fc1c0f43fc96d42698ae1168a2827b5c65f7948d5dbc">+4/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>stream.rs</strong><dd><code>Define PatternAssociation and
settings wrapper</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+14/-1</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Spawn pattern manager watchers on
init</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+13/-3</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>re_pattern_stream_map.rs</strong><dd><code>Define ORM entity
for stream pattern map</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-3c32b9e32bea40b63ed43542aa136cc9000132c4049ed7b6310baabadc55cd79">+38/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>re_patterns.rs</strong><dd><code>Define ORM entity for
re_patterns table</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9c4ac843741290c1b999f3101fae6f46d71d1893d7740de50c71e2c29fe1636c">+33/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Register re_pattern processor in
super queue</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-6ed0f1dd36b7216fffde573232a49869f247330ecb03b50f94b2ddd723742e46">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>prelude.rs</strong><dd><code>Expose re_patterns entities in
prelude</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a818656b9fb01945155f28450270dd2ae20b67c8248c851ffa0f25e859d263ed">+2/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>5 files</summary><table>
<tr>

<td><strong>m20250612_000001_create_re_pattern_table.rs</strong><dd><code>Create
migration for re_patterns table</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-db969ee5510136d5b3c712cdea999c151ab04efe55f1b090670c1eed2b36bcfc">+173/-0</a>&nbsp;
</td>

</tr>

<tr>

<td><strong>m20250612_000002_create_re_pattern_stream_map_table.rs</strong><dd><code>Create
migration for association stream map table</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-f1ec25aa2150a4fc857fdb17e45b8e43e7c31e884df77e8c91fd07636a6931ca">+168/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Register new pattern
migrations</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+5/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Add re_patterns services to HTTP
router</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+7/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>config.rs</strong><dd><code>Bump DB schema version to
support patterns</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Error
handling</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>auth.rs</strong><dd><code>Add auth rules for re_patterns
test endpoint</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+11/-0</a>&nbsp;
&nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>23 files</summary><table>
<tr>
  <td><strong>Cross.toml</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-3abca7e5bab5b140e44643dade038033b8a6f6e50771f76c7d48e7bd0b338815">+5/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>Dockerfile.musl</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-f47eb18f838cbf757cd4e34ed5357c767b969d4551dcea6a4ab3b64d384a5614">+16/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-c539d7557d1425986d5c09849d8dbca42d1b309c2bc33d49db641d6a783098f6">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-af023301203a27a1ef34f698d5a329ad7bfced65ac65e3829b1c0905a0c74fb2">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>O2AIChat.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a6a125719a7b32b31b53fcfe05c12081d82c139a401699affdda0db8e73887e3">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>TemplateList.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-7df66364d7d69a26cfa25cbc555d0e090c2f73e51c8386d9b0435bae4412564c">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>FullViewContainer.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-c0722446274c88c250042e38b051201b81450cb5da1ee827cd5a4a927424a357">+1/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>AssociatedRegexPatterns.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-c6fce5e1c83798991b6ff19effb23c67af59f38fac0b9e56dcab7a09d139f550">+940/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>schema.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9196be2ccae4054a00b0b1aa9d1e87b366c10920d6eaf9ef8c6e6293d50bcf46">+134/-4</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>AddRegexPattern.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a3783cfbdd32609bfff45ecfa23542854cad7ae60c3d29fedb832312a4de03dc">+689/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>ImportRegexPattern.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-e8e1bcb5a02311504d7d571090eab27a270aaa71cae185ad2703023e2e70a31e">+771/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>NoRegexPatterns.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-279b923f20042948c87e907d2453e9d5a4c8c73672b23df951d7d7d823ff3a88">+103/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>RegexPatternList.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-524927acd9f5bb07b71bfac489e747470108574e21a6dde63ea7c1f219cea853">+506/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>index.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-699c63979f851c48ad440a078e93282fd797a18fc065e5331b2e41122ddb8b69">+27/-2</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>useManagementRoutes.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-b71a1d39b1fab0749f0668d5f16382536fbda6e2f0c92c56a972fb5d8b5835f5">+12/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>useStreams.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a083369344f022cdf8d4f536719179b1d920369e33e2e26234e70dd36d151a7c">+42/-1</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>MainLayout.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-91e4134141264c4624d4eb4cfa987b33ed31eb6f551e089c0b26f2610dd54362">+3/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>en.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+24/-1</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>JsonPreview.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-3985073134d35ff074742944b114d0905524038b42e6d2ac887fef2c8436fb16">+272/-3</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>regex_pattern.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-2a0b0828141d39ab0385e0941314ead1cdd2ec3ec7b3baf5fd0c42f8f010728a">+63/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>index.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7323/files#diff-955a15c7ee060f1545e82be22911dd0c14307dee5a70071a394e9fd583bd2c55">+9/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
2025-07-23 14:55:03 +05:30
hengfei yang 6083d50cd9 fix: cargo clippy 2025-07-20 15:03:43 +05:30
sktrpathi 784779348d chore: add UTs, run fmt and clippy (#7630) 2025-07-20 11:18:43 +08:00
hengfei yang 31d5e6de07 fix: optimize topk 2025-07-13 06:41:47 -07:00
Hengfei Yang c5be35f071 fix: single node should push down the whole plan (#7528) 2025-07-11 11:47:03 +08:00
9444447498 fix: Free trial rebase (#6922)
Cherry pick of https://github.com/openobserve/openobserve/pull/6807
after cloud branch was merged into main

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com>
Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com>
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
2025-07-10 11:23:41 -07:00
udev c66b273ee6 fix: remove ws (#7486)
### **PR Type**
Enhancement


___

### **Description**
- Introduce HTTP/2 streaming search pipeline

- Implement caching and partitioned query execution

- Add delta processing and progress updates

- Remove WebSocket support and update configs


___

### **Changes diagram**

```mermaid
flowchart LR
  A["process_search_stream_request"] -->|check cache| B["handle_cache_responses_and_deltas"]
  A -->|no cache| C["do_partitioned_search"]
  B --> D["StreamResponses to client"]
  C --> D
```


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8
files</summary><table>
<tr>
<td><strong>execution.rs</strong><dd><code>Add partitioned streaming
search execution</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+705/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Initialize streaming search
module</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+535/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>cache.rs</strong><dd><code>Implement caching for streaming
search</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+485/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>utils.rs</strong><dd><code>Add utilities for streaming
search</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-20915cbb55c73e652f71128beb9898a02d9333115d15ecbf50b5df775923e0d4">+140/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>sorting.rs</strong><dd><code>Refactor sorting strategy for
streaming</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-138cca545e29b77f27da957a19d04bae58ee0c9f84b75509f48c2f0efe5000bb">+46/-48</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>search.rs</strong><dd><code>Add SearchResultType and
SortStrategy enums</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-a36a6683e327641569d8098ee0d4f13e894e1b9d51230dc3569abe42e87ba310">+26/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Move search_stream into streaming
module</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>search_stream.rs</strong><dd><code>Use new streaming request
handler</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+3/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>6 files</summary><table>
<tr>
<td><strong>config.rs</strong><dd><code>Rename WebSocket to
HttpStreaming config</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+15/-30</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>router.rs</strong><dd><code>Remove ws route and adjust
querier routes</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-5227b263a6c37a040ee1e7482b6903ead2186dd71090ee70bdaf1c3b894487ad">+3/-28</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Drop websocket route
handling</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+1/-56</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>search.rs</strong><dd><code>Use HttpStreaming config chunk
size</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Switch to HttpStreaming in status
endpoint</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Remove ws service
endpoint</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>21 files</summary><table>
<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+0/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>auth_tests.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-31</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-2aa59ce4464e7dbabdb9367fc8bb8c1b489c914b8690577d4916870463ab12db">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>websocket.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-6ff41099eded8eca4a0e579142d7af4429efb83350917780e0d3ae7dac13847a">+0/-225</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-bab79e8da2d0d9f903cb1a6187d922c6de63d3e4561e8ce633660b664b4e859f">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>validator.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+0/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-dd04c9099b3f291c7822f6e2adcd4197246c67b1610dbd70b09d225eb3b51e40">+0/-283</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>session.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-6a1a862158668be26ad1575d7050eb9139bca3e469db56b32641b4c46d0f8144">+0/-641</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>connection.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-33c5e604300998724046d4a095c0e45a517b3d68ee279c1f49e5ef827bc6a65e">+0/-631</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>error.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-23a4826e6d2d8afd1df90335d54d1bd31c917e8b5689c0aa0edba957f5308ee1">+0/-150</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>handler.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-eadc17f110edfa7f870ce3b66b2e0d0bf301624c76f3b7ebeede043b073dd020">+0/-753</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-50311955789888994478438442ded5daf30873021cebf4f55d478d03feba116e">+0/-104</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>pool.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-2b0e614010eda77b040ec03f7c9fc5e5feede7e846ae3e21b644e311b1b32956">+0/-118</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>session.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-cdd86aa2ec02449ae32de4b331178c6079dbbc48f4220b8feecf2d1f8e878b00">+0/-205</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-1b15316505c31acccb26ba66d920fdb0894567d08d2bf1e3457716d128a67b83">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>search_stream.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-b952a89375f2745a0daa273f1bdefc80c22997759a9b17a8fccbc70ae294e8e4">+0/-1667</a></td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-21232733827570717e2ef8988a4c9753e5006665cfe9a46c7b88261e44f4947c">+0/-23</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>search.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-13476631e2e604b75e2dfdd8fb7b7a91c46dcd85faa50813afa0b87583bcdb26">+0/-1271</a></td>

</tr>

<tr>
  <td><strong>utils.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-36caf9ef22ae6e1a217d51e023403ce24a1bd3d10b1bc6d5b86c982d6269bdbe">+0/-333</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>values.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7486/files#diff-d5c53f3f28432f096cb4788fa89c0e9bc8551aefe75c4a62be084a08d9025f97">+0/-326</a>&nbsp;
</td>

</tr>
</table></details></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>
2025-07-10 13:43:32 +05:30
Ashish Kolhe 40aa8edd31 chore: clippy fixes (#7513)
### **PR Type**
Enhancement


___

### **Description**
- Replace `{}` with named `{var}` in log messages

- Standardize log formatting across modules

- Improve readability and consistency

- No functional changes


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Formatting</strong></td><td><details><summary>20
files</summary><table>
<tr>
<td><strong>handler.rs</strong><dd><code>Switch log macros to named
interpolation</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-eadc17f110edfa7f870ce3b66b2e0d0bf301624c76f3b7ebeede043b073dd020">+32/-67</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mysql.rs</strong><dd><code>Update MySQL log formatting
placeholders</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-06d02c738c6a4dc22ff022c16befef8d46aece6cfaa88bb4a5d3d5f92c2e50a1">+31/-36</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>nats.rs</strong><dd><code>Refactor NATS log interpolation
syntax</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-2d6809ea34ea04477afb9a6f8287c93cef488b45ef8f0ebeec2241571b93994c">+16/-32</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>schema.rs</strong><dd><code>Standardize migration log
messages</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-503e5382af7a29ec9c2819e32d468f6bbf9499da91cfdf43663def51c77d422b">+19/-34</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Refine OTLP trace handler
logging</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+17/-36</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>parquet.rs</strong><dd><code>Update parquet job log
interpolation</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-27f710981cfe1b938aeeb53abaed05961596c32e2edf3b8a2a59d3e72b2f542b">+18/-49</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>search.rs</strong><dd><code>Refactor search event log
syntax</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-13476631e2e604b75e2dfdd8fb7b7a91c46dcd85faa50813afa0b87583bcdb26">+15/-47</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>merge.rs</strong><dd><code>Improve compactor merge log
formatting</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-0ab016dbd5129c9c285c9c459bf78928147bce97dc92a03aa754da3da71168d7">+14/-54</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Standardize compactor module
logs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-dce43be4d49b0bf99b61099d5b348bc539433d6e9b1ce6b7025d65e907f00b95">+10/-35</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>disk.rs</strong><dd><code>Refactor disk cache log
messages</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-af63f2c5d85313779df0204c9689e50a6c3916e3aadc24e54e23ec640a584fcb">+9/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Update request WS handler
logs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-dd04c9099b3f291c7822f6e2adcd4197246c67b1610dbd70b09d225eb3b51e40">+17/-27</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>postgres.rs</strong><dd><code>Standardize Postgres log
interpolation</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-bb24a30c9c3744eb31fb49e4271a9651d1f9a94a87455ac0074364b879af9651">+17/-26</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mysql.rs</strong><dd><code>Refine scheduler MySQL
logs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a275fb17137e6e7cd3dbc6432164136b20d1c2d4757eadaadd0b4a983473865a">+17/-23</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>wal.rs</strong><dd><code>Update WAL redo logging
syntax</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8ba57e3a90cce57ceb88ea5215024c6d54473732bebdc7bf33847462379ec03e">+13/-22</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>users.rs</strong><dd><code>Refactor user service log
messages</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+13/-18</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>jwt.rs</strong><dd><code>Improve JWT auth log
interpolation</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+9/-24</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>batch_execution.rs</strong><dd><code>Standardize pipeline
debug logs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-3f6824a6eddaf1603e5cb0f1a4311ac6a45e1ffb967266ccc1f0bba944625e69">+16/-34</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>alert.rs</strong><dd><code>Refine alert scheduler log
syntax</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-58f77a5eb00c01f123b13173c99cf9499af7458115e50811d07def0cf6e2d3a7">+12/-13</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Update OFGA migration
logging</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-1b8694fad31c8249afeb419d92a9facbd170289db2c89100552883df876bb275">+5/-10</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>schema.rs</strong><dd><code>Standardize schema service
logs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4696a44abde2df4ac48fed716f930bfdcc01759bdfcfd8d802821c6dd857bff9">+12/-14</a>&nbsp;
</td>

</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>101 files</summary><table>
<tr>
  <td><strong>cli.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>etcd.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-37c2c2803d1c12ebb4ef9299bc79c9e6740fc5f19f7f4d84e7cfbb610b9561a9">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+6/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>nats.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-901773980aa255942e7865b977fe63811f31511ac40bc512d6bbb14dbb54bece">+8/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>config.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+12/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>alert_folders.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6ed0a38075313ced0916343ee6e6b86ab639f8ed59e4968df395aa30d72bd300">+10/-10</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>wal.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4928ae81597b28facdc3644f786ac14c60b0a406e1c7b8eac7232fbafbcbb6e3">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>telemetry.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-3ff026b46d9431e0657828a3bde4ab118fec006b353c2abc80be4b74633c98b4">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>auth.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>ider.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6ea42b8eab8ba0708f93f9a8b44056d9a9e6ac42517ffba12f81420d6b0bd2b0">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>search.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+2/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-2ba3bceb0d80c8008da06e45374c1f744441375e2c28b55e6a55a1be3337e79b">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-9a4dddd253d61f35b8d61d372477e24a4523f419aad2a63b226b19a584b8624f">+4/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>event.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-689769712cbd6bd02beba078c1e1dfc188e2be39ab19a0e232b8bd12bc7f1e78">+6/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>traces.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-38e83bff9d1b99d37475a02c28cf9b2b7fd4d84d366a1eade3928a5d10b471f8">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>token.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-3339b4a964c121766ef87fcec0070f27a2b7085cf81281b53154e53d33cb28cf">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>validator.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+6/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>ai.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-c7e88d14798788e6186cadbb1cef3386dad2e585af6c36fd7b9cab3dbef2e46b">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>reports.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-00435e0d9301342b51cb57ae313f1413e7ed973642e0badf3f3959125d0677d1">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>timed_annotations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-b6a4ef9157a0dbf25b875111f7cdb230d3d0f2d7a430d335de6c6af82f95b365">+5/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-72fb8ce13ff14a37ab38d48e5b452205b30c8a8b544badf42579fec4fe5f90fd">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>ingest.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-24f250254ecfdbe004af9dc0ea23f8a2c0105ae0898690ed3e565089e5d8eef9">+4/-10</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>loki.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-34691872632b619be8aadf255104f0e004fcf701dba98f88e231de5966eed099">+4/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>org.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-b47c0c1a8a78524196176430268eecdbfaf6f46d6b95873d3cbe70cfd67182b0">+2/-10</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-0ad5729a1fbce00ff53076728c51822400c13af9918e1d8554f41be56d060fa0">+6/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8ce79cea5617c3a661e5b64058ad3846abf8652bb44fbb88dfb9117ffcc577fd">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+9/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>multi_streams.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+5/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>search_inspector.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>search_job.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d848b311eae318d198558c29a007c8973afb5f8867a31ec056f4e037dcd1b854">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-619eafba0f0509d905b021f4a16659010aca23c62cffd9cd3e6c2bbe204ce200">+2/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-56897e42f9436a640bc70fddc25686485ba8ae1223635b32d252a0333309f748">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>service_accounts.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8d58793d63492cb2b4ec77288a91b0cf995a56cac7ea6e0faa9f006aa105b368">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>session.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6a1a862158668be26ad1575d7050eb9139bca3e469db56b32641b4c46d0f8144">+6/-16</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>slow_log.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-45522a89e42bfd2029972622703bbf58b40d16fc7a7c4346ac270082728661ec">+1/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-66207bd699343342402f88cdd331f4c549783754e4921d0c8d43e026e2d8e575">+11/-11</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>delete.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-9f484d99b002885e92d01b21c11686f8e99077cf5dd422b673c6877db15f2f8d">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>memory.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-ff00d697a9b223ef2858c5e41111a8918e7cc347dc22891c02bda6982682eee1">+4/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f8e74805561fa288a434e6e5ca8fb2c9068fb7f54a5a21f0a9bbba5f73249a5f">+2/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>alerts.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-9a2bb454050f39797fc752caa4a72da28f398c76f2a7f54f74d1c293d3b27698">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>etcd.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-33bb9257c1a1b9162c8679191b8d8b75c0b7395f72291ce73d77247ff924a66e">+6/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mysql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-412dc407af348f4b54b7d98de6a928876b0333ea501792498a0568850f83d7b7">+10/-10</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-1554e69f6d6c29812af88f0dcf093240b24ff85f9e292322c71cb19b0dda808d">+13/-13</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-85f17b22460902614e5c28df8966ea3e7073325bd8dd43fa396ed9268b6b4b06">+5/-11</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>grpc.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-211c5cb8fa12eac14c3637a43d74cd114034b8a6f6f8c1ead51e56efb54141cd">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-85ff00530831fdaa4e13c05cb649f2e20479cce04a4b65ef78d131cdb89b6c71">+4/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mysql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-836ef77752dcfeba5e1161c2c42e92ec651d9a9131f12d48fb390fe3d861ab53">+8/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-122e67af3f562870a16eff139f68d5c126841cabe5ebefe2c729530d676e883f">+8/-11</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-e81d5c9c0ac9af5dc99c66eca0cfa1523f34063a33f75f6b7c1e4fc0d1802ed6">+8/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a281dfd3124defa76bf1100c7e4993def0cb3049cf1ad0b1c513ea0afe522d08">+9/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-07751d2c8c454417335fa9dd06507099d8b1eaea9b04af99857db6d9c619e69f">+5/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-386fd4fad1e48ab0a90516cab1947b1888b270b2fe912f30d2255cc38e6c519b">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>local.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8e49c3ffac3d43c06780a18937b7ecd7943a8872ec81dca39d4497d9f94e33a7">+4/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-71f24f3312c19e01c4eedcd6a720893e01e64d176d2afbe1e7c6171a23059716">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>remote.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-fea1d32f99821f1913607362236bd9e5b65cb2a5f8ce2c11a9a4956ac646b3d1">+6/-13</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-238b8087501b1743c3c4608469596d431869f021e9bb30e9a058d1294229ee67">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>

<td><strong>m20241227_000100_populate_organizations_table.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-53abf79ce8f9c17107749f25c5a544591903422b918e322643068a38eaee6050">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>m20241227_000400_populate_users_table.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-fd66003231079d3fcf3681ab7a57d64b61dea22e55a3c22ebe5527d095a52b5e">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>

<td><strong>m20250320_000001_remove_alert_name_unique_constraint.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f4094b2945a877db5709045deafdd2883ff31fd671d0782da058af40e522d559">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>m20250611_000002_populate_reports_table.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8247af6d6f8ccb6a5a51daff4cb1e352585be2d1c785d86ae5ebfea1dbc1099c">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>

<td><strong>m20250611_000003_populate_reports_scheduled_jobs.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6567d3d0361c1a184ba7877795a8c3c4f73ea9807b3d4c13efe621449dd3f80f">+3/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-af023301203a27a1ef34f698d5a329ad7bfced65ac65e3829b1c0905a0c74fb2">+1/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>org_users.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8e62ecba19308fdaaecfdb3d1f23021c926e244a510d9452dfbd71ac8f35ee9f">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>organizations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4d87f9fe379a5d161a3e7c3c77f7f320be738e6a70c785aa5ce93cb0830c0147">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>ratelimit.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-231f9b18e0d61409b6e3348196dd4d4456847ebb25e1822eec52e7e09b00b5ef">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>queries.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-144c4572241c5b3f13abe628832674ad8cebb72e51acb1d1b14fae2a8307208a">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>search_job_partitions.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a72ad2e06cfec317bd6901681b9b7c8a27b4b5736e6340f58d711296222ffee1">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>short_urls.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-5e9b140c2fe41290633080bed471cd7473165119bc1fe84bd69853fec91d7a3a">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>timed_annotations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4bc910d02f5ddc598e394d06172debe456c7db655cc16fef53eab55005880378">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>users.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a8183f71611dc4c9329231bb5083ceb43a19f43011f86be5c5c9e47240d2073f">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>lib.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-603267e2f712eeb4bc46fdff981f1fad1f3a9d3b989a7db4b3c0984fc5b1ed43">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>partition.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-441e26dda30cadfa3cb1efbd9569aec96af8fdc779e4916814fb5f9477db3f6c">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>writer.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f">+7/-10</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>alert_manager.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-7e30858abad652a860186eb047347305488f52356112b1495dd53ce740316d7c">+5/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>file_downloader.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-dcd9d92e9718a63ee7341b7239f03d9f55746a0f848537e4982661e9219ea05c">+7/-25</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>broadcast.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-930f8671c8be6478fe4739b46af3ff38994f14a558a2003cc136060a7c790687">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-30f42c3685037c219f0033617f032b23a7f74624ba4ae6c4e3687af401304591">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>metrics.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-ca98d90d1293682c98c33a777a653cf3c1466f6297969ae94a359bb81abca4b1">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mmdb_downloader.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d36dee2ad5f79654acc9d5d5c3b95a552a9b397f97c25c2e23a19195ddc7fe86">+4/-12</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>promql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-15414a41d64eb87bc5c0cd00a07213b3f99c31b346e8e2c36178f7276a1f058c">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>stats.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f3ca67d9d2c68beeafd6e5de947dbc9a648ac2d2699c5dc87458c369887d09b9">+2/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>syslog_server.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-cb7610142582b44885f673583ab16262c3b9ca6de1f59a9def65e503360665a0">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>main.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+8/-23</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-eb855be0fa22d1a540379f6142cc4a473d49f89c3e0fe16cfd5078d5a69f9a3b">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>pipeline_func.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-701a2afe5253e4ef729a1b93372e69d79ada9cf9d1f0f5a8556bc4841f46f301">+1/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>report.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a8d5d0e0fc56e67b38a20a46b91cc2e6bf76ade360a37ad29fce51cc4661a9a6">+1/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>server.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-804074c17dbf0e9757a57ade0cd81a00518a14204762cbbe3fa44bfb43936dc8">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>connection.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-33c5e604300998724046d4a095c0e45a517b3d68ee279c1f49e5ef827bc6a65e">+5/-17</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>resource_extractor.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6bf05d4ebced31bf58505b67b2362abd833c00422743650218998ed6bc4b45af">+2/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>alert.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-574be3504bf4d855c65fd3d7a10797a970237ff552edac4962b474b3983e094e">+1/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-ba9b65130898444f7815fd6068bdb0dc590612f74cc893f79d91b0afc44cc851">+3/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>handlers.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-5eca8995a88fcecb6d6b8efd3c7c4496f4a96a4b060fb4ed85593574f7cf49eb">+8/-17</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>worker.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-3d37991258d3f26b16ad99bba59b63adfa614bfbc4866c4088544ca9308ae49d">+10/-22</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>deleted.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4900b3bce3c590ac846c7f12e82e1d1cf7c2de3de9fc385fdbe53deb8a8e50ab">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>flatten.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-93a66ecfe3a7fdfe5509dda69e4d444f02b18c841c34ab3078f1fb314af8e3ca">+1/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>retention.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6abd048f5ef4e7163cffbb51cb4bee10c07fb88b871469d5b3724f7cd04743dc">+9/-29</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>Additional files not shown</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-2f328e4cd8dbe3ad193e49d92bcf045f47a6b72b1e9487d366f6b8288589b4ca"></a></td>

</tr>
</table></details></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>
2025-07-09 19:16:13 +05:30
Yashodhan JoshiandYashodhan Joshi babe3b08fb feat: support path based value extraction in cipher functions (#7483)
### **User description**
cherry-pick of https://github.com/openobserve/openobserve/pull/7482 for
main


___

### **PR Type**
Enhancement, Tests


___

### **Description**
- Add JSON path extraction utility

- Extend cipher UDFs with path argument

- Rewrite optimizer to handle paths

- Add tests for path-based ciphers


___

### **Changes diagram**

```mermaid
flowchart LR
  J["get_value_from_path"] --> U["cipher UDFs"]
  U --> O["optimizer rewrite"]
  O --> T["path-based cipher tests"]
```


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>auth_tests.rs</strong><dd><code>Remove `batch_size`
default in tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/common/utils/auth_tests.rs

- Remove deprecated `batch_size` default


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7483/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>json.rs</strong><dd><code>Implement JSON path
extraction utility</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/config/src/utils/json.rs

<li>Import <code>itertools</code> for flattening<br> <li> Add
<code>get_value_from_path</code> implementation<br> <li> Handle nested
JSON and wildcard paths<br> <li> Add unit tests for JSON path extraction


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7483/files#diff-f638bd6d218ad53a671681c2fa8e420cfbf8369e6e93269c9559e9711338d52a">+131/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>cipher.rs</strong><dd><code>Support path argument in
optimizer cipher rewrite</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

src/service/search/datafusion/optimizer/cipher.rs

<li>Require only two args for rewrite checks<br> <li> Update sanity
checks to allow optional path<br> <li> Preserve or default path in
<code>CipherKeyRewrite</code><br> <li> Add tests for path-based cipher
calls


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7483/files#diff-1dfdd6163a17d67d426ab189446fa4c43551094aea50ffd82d334abd3ffe4f81">+40/-11</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>cipher_udf.rs</strong><dd><code>Extend cipher UDFs with
JSON path support</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

src/service/search/datafusion/udf/cipher_udf.rs

<li>Extend UDFs to accept three parameters<br> <li> Validate third
<code>path</code> argument in UDFs<br> <li> Parse JSON and extract value
at <code>path</code><br> <li> Update encryption/decryption logic for
arrays


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7483/files#diff-aca30595e1e899bf83b213e84d91063cc621a8972ce22e67bcf4705af817188f">+175/-19</a></td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>sql.rs</strong><dd><code>Require three arguments in SQL
cipher parser</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/service/search/sql.rs

<li>Update cipher function argument count to three<br> <li> Adjust SQL
parser validation for paths


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7483/files#diff-d9ccf393c53794b0ef2816eef12742c26e23d2788cde27780af7570662aea446">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
2025-07-09 15:51:32 +05:30
ByteBaker 25ce21cc3e chore: clippy delints (#7446) 2025-07-06 16:45:54 +08:00
ByteBaker bdb4338903 chore: add UT for CI (#7464)
### **PR Type**
Tests


___

### **Description**
- Add unit tests for email validation

- Add unit tests for OFGA unsupported check

- Add tests for OFGA format conversion

- Add tests to validate if a user is part of organization

- Remove unused invite token stub


___

### **Changes diagram**

```mermaid
flowchart LR
  validTest["Test valid emails"] -- "calls" --> isValidEmail["is_valid_email"]
  ofgaTest["Test OFGA unsupported"] -- "calls" --> isOfgaUnsupported["is_ofga_unsupported"]
  formatTest["Test OFGA format"] -- "calls" --> intoFormat["into_ofga_supported_format"]
  userOrgTest["Test user from org"] -- "calls" --> isUserFromOrg["is_user_from_org"]
```


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>auth.rs</strong><dd><code>Add tests and cleanup in auth
utils</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/common/utils/auth.rs

<li>Removed stub <code>generate_invite_token</code><br> <li> Added
<code>test_valid_emails</code> unit tests<br> <li> Added
<code>test_is_ofga_unsupported</code> unit tests<br> <li> Added
<code>test_into_ofga_supported_format</code> unit tests<br> <li> Updated
<code>println!</code> macros to use inline variables


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7464/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+36/-9</a>&nbsp;
&nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>
2025-07-05 10:59:39 +05:30
Hengfei Yang f88fcb6302 feat: sync rc9 to main(BE) (#7447) 2025-07-04 11:34:23 +08:00
Subhra264 ab5c296f50 fix: openfga migration for ai (#7434)
### **User description**
This pr adds the migration needed for ai RBAC controls.


___

### **PR Type**
Enhancement


___

### **Description**
- Add AI chat permissions migration flow

- Introduce version v0.0.17→v0.0.18 check

- Populate AI ownership tuples per org

- Require authentication for `/ai` endpoints


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>mod.rs</strong><dd><code>Add AI chat permissions
migration logic</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

src/common/infra/ofga/mod.rs

<li>Added <code>need_ai_chat_permissions_migration</code> flag<br> <li>
Defined versions v0.0.17 and v0.0.18<br> <li> Checked meta vs existing
model for AI migration<br> <li> Included AI ownership tuple during
migration


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7434/files#diff-1b8694fad31c8249afeb419d92a9facbd170289db2c89100552883df876bb275">+10/-0</a>&nbsp;
&nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>auth.rs</strong><dd><code>Enforce authentication on AI
endpoints</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

src/common/utils/auth.rs

<li>Removed skip-auth condition for AI paths<br> <li> Enforced
authentication on <code>/ai</code> routes


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7434/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>
2025-07-03 13:20:03 +05:30
Huaijin 5883edc5d3 feat: upgrade datafusion to v47.0.0 (#7421) 2025-07-02 18:48:58 +08:00
Huaijin b141504cf8 chore: upgrade rust to 2025-05-20 (#7316)
### **PR Type**
Enhancement


___

### **Description**
- Use `if let ... && let ...` patterns

- Inline named args in `format!` calls

- Simplify visitor `visit` calls with `_ =`

- Replace string interpolation in logging macros


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>5
files</summary><table>
<tr>
<td><strong>sql.rs</strong><dd><code>Refactor SQL visitor and limit
logic</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-d9ccf393c53794b0ef2816eef12742c26e23d2788cde27780af7570662aea446">+249/-255</a></td>

</tr>

<tr>
<td><strong>mysql.rs</strong><dd><code>Refactor time_range early-return
patterns</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-06d02c738c6a4dc22ff022c16befef8d46aece6cfaa88bb4a5d3d5f92c2e50a1">+51/-51</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>sql.rs</strong><dd><code>Expand UDF list and simplify
visitors</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+43/-39</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Refactor error mapping with
`std::io::Error::other`</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-8ce79cea5617c3a661e5b64058ad3846abf8652bb44fbb88dfb9117ffcc577fd">+20/-36</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>engine.rs</strong><dd><code>Simplify unary ops and format
messages</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-a98ac5b55a4c9440c09c4e2dfa3b6f7f24f9e58fcb6bf8b10577a0b4f123ed41">+49/-63</a>&nbsp;
</td>

</tr>

</table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>4
files</summary><table>
<tr>
<td><strong>mysql.rs</strong><dd><code>Inline format! and logging
braces</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-412dc407af348f4b54b7d98de6a928876b0333ea501792498a0568850f83d7b7">+59/-76</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>nats.rs</strong><dd><code>Simplify error mapping and
formatting</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2d6809ea34ea04477afb9a6f8287c93cef488b45ef8f0ebeec2241571b93994c">+49/-54</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>sqlite.rs</strong><dd><code>Use brace shorthand in SQL
strings</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-85f17b22460902614e5c28df8966ea3e7073325bd8dd43fa396ed9268b6b4b06">+45/-56</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>action.rs</strong><dd><code>Use brace shorthand in Display
impls</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-fc87c75a7df9539d7e7ee6184dfab02d4a8932323e81f95d57c05363aa44752d">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>101 files</summary><table>
<tr>
  <td><strong>Cargo.toml</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>clippy.toml</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f3dbfb2563c3490394f44c26b51837018e79a79e75fd31e89b19e943a38317ea">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>rust-toolchain.toml</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2b1bde2cf3a858b7bf7424cb8bcbf01f35b94dc80b925d9432cbab3319ca9b4e">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>cli.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+8/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>http.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-790b7780a51e8eecd4754b623af8d15ad87baaaf29455ba646e2e410f83f5498">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>load.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-bfc7e9b23a8fa100bcad8490d697a3090f139440fcd2ce02fb539c1791100d50">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>export.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-b9ca11eac019064e747b0c75157c280c8e8fa6d75edb83ca9ae46777e9668cc0">+2/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>import.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-e4415a147f0fe6e557c9fee2a5cad9058a46c30cfb79fe131d13802c0c8bb804">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>etcd.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-37c2c2803d1c12ebb4ef9299bc79c9e6740fc5f19f7f4d84e7cfbb610b9561a9">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>nats.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-901773980aa255942e7865b977fe63811f31511ac40bc512d6bbb14dbb54bece">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>alert_folders.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-6ed0a38075313ced0916343ee6e6b86ab639f8ed59e4968df395aa30d72bd300">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-1b8694fad31c8249afeb419d92a9facbd170289db2c89100552883df876bb275">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>wal.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-4928ae81597b28facdc3644f786ac14c60b0a406e1c7b8eac7232fbafbcbb6e3">+5/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>loki.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-121c166af5eab8b9cb83716afa7b65fa7856c006c809b3d910ced184c52ec587">+8/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>telemetry.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-3ff026b46d9431e0657828a3bde4ab118fec006b353c2abc80be4b74633c98b4">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>auth.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+39/-43</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>auth_tests.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>functions.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f4df68dbcf485d6b98c0e673c12a19be2edab416e5fc22c7d08396d375cd9126">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>http.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+2/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>jwt.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-ad8a7b809caff90b4c49885a9b5d49ae03ca3fe00bef37a72d61b08b59df449c">+5/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>redirect_response.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f0415572330f30fceb16f230517e07f2c186e834cb8ca634795e5f92c1c57717">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>stream.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2a60b2ae71e0a5c9dcceb588f262738f99df65fc2cb5b7002f2c37f87639e3b6">+2/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>websocket.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-6ff41099eded8eca4a0e579142d7af4429efb83350917780e0d3ae7dac13847a">+16/-17</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>ider.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-6ea42b8eab8ba0708f93f9a8b44056d9a9e6ac42517ffba12f81420d6b0bd2b0">+1/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>alert.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-45c7eaf1a764ab52f3e014fe0555c03dc59d77c5ce974684cdee4ed90463a234">+2/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>destinations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-e0ba33d95ef7f0eac13bdac7e750462907a0308f01c027ea206d058e282c977b">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>inverted_index.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-8bf9d9403f05f53f0c6fd5e6a6dbf0204ea80465b233567552cb7395d08d1c3c">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>components.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-e2d647c422ae05ac91abc683c7b11a3fb7da61ee30968ca6e84cdba40221bbb9">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>promql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-0893731b3f0345378627894e3ea4a9f8e8735b38622b2b96ab55e81760d188db">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>ratelimit.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-fa7dc03fb526c09d5eb58021872d821fcd72eeb4d6a670224f5286012485bc3d">+1/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>search.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+22/-22</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>sql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-845dcf22be9d370a278c081df83a72bb8dc4b729f95c201839787088c82b57e7">+39/-39</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>stream.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>timed_annotations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f94b4d6d12d68e7054ad33601e666b08e4594586013dc34384fa21b1df5fe6e8">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>async_file.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-8275f369cab85b5e9a97ed11d236de3364850b6989698936f19e70a0144e6a0d">+19/-23</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>download_utils.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-990aff76afde0d775c850302bf6399998aee73da05c76e719b5fbad6e7ed4b27">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>file.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-405c4fb06d186c2423f78d42f3d195b12961877b091feff032f7df500450a72a">+6/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>flatten.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-d13aed536f8b484d0223d7158e45e3a51ea181846965f73be42dedae975afcc7">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>md5.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f4d317c045b3df075e418dd6a7cd51f275befd1f2d15622afcb41c20024a8ee5">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>prom_json_encoder.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-5c6b4c5bb9f3859855269fe2b06a05167469ebc2ac04b84fb45d8d92aba6b825">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>query_select_utils.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2396baca05d001b681c6cc302c2acc901d4dc1e7acf32dd1be6fa9a759578a46">+30/-30</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>record_batch_ext.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f84aa2842a164d6e85aa05a718159e87a325cac139c59e561f6f59313c6d7c5d">+1/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>schema.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2431bc157b9e51be2f16836a995b5ed52f41dded921a404bf2eebd506b49fdfb">+1/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>cgroup.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-167dbe688715e2a299b86aa662fd5170b68e54ec0f6a04e0ebfa0492b3ec6b00">+11/-9</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>net.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f95051aabe608a9348aecbcc7f6866576f683f0e72cb4c84f7ed3e06c7e78b67">+7/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>time.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-7b7e5972c321c99c5147af56a71afc1136a7c75844b8442e207de60bc86b7ddf">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-9a4dddd253d61f35b8d61d372477e24a4523f419aad2a63b226b19a584b8624f">+2/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>event.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-689769712cbd6bd02beba078c1e1dfc188e2be39ab19a0e232b8bd12bc7f1e78">+11/-11</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>ingest.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-27ffe683f384ff836c6aac746240d755bdf17f3c6e0a65d8ec1a1ff82aaf7d5c">+7/-12</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>jwt.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+61/-62</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>validator.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+17/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>dashboards.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-250eeebeec9d4ac0ac271466ed83cd33fea3f121080491b340dc95d9af287259">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>action.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-616df3014c553f763ea57e925f1bff7bb47f9467c632940d93a45b7b1e4c3a07">+5/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>deprecated.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-3a5d80eac45bff881716e2cf099f90130659576eca017ad238a61781d3341b7e">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-e67f49865e1cd36775282c55ba0c0579a52ce7542f9d8dde450033f0c66a9f4c">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>ingest.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-24f250254ecfdbe004af9dc0ea23f8a2c0105ae0898690ed3e565089e5d8eef9">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>loki.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-34691872632b619be8aadf255104f0e004fcf701dba98f88e231de5966eed099">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>settings.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-1e807254f2faf8fd09b802f71b23412f203eed289033f96bfc62b15dd997db8c">+5/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>around.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-93b3688de26f8f7c9dc0389763e9999a254bcded15b6ae8c8349f18aec83bd4f">+9/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+19/-20</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>multi_streams.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+17/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>search_inspector.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+27/-29</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>search_job.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-d848b311eae318d198558c29a007c8973afb5f8867a31ec056f4e037dcd1b854">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-095d70f65b6e69bf7de0180fcbcddcd7c89c4cf00095d7b06f039a07e73916bb">+2/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-c10dd7b4ea20ffe5840c93a6ae4298aec0c32c8ba1b114192fda6c2bac1c41d1">+7/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+10/-12</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-619eafba0f0509d905b021f4a16659010aca23c62cffd9cd3e6c2bbe204ce200">+14/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+4/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-56897e42f9436a640bc70fddc25686485ba8ae1223635b32d252a0333309f748">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>session.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-6a1a862158668be26ad1575d7050eb9139bca3e469db56b32641b4c46d0f8144">+6/-7</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>encoder.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-1825b00f221539b371b9fe8aa89f9cbc1cfb94f3b731f053e64c51fb5c5934cd">+1/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+2/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-66207bd699343342402f88cdd331f4c549783754e4921d0c8d43e026e2d8e575">+12/-12</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f8e74805561fa288a434e6e5ca8fb2c9068fb7f54a5a21f0a9bbba5f73249a5f">+21/-23</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>etcd.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-33bb9257c1a1b9162c8679191b8d8b75c0b7395f72291ce73d77247ff924a66e">+20/-22</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-1554e69f6d6c29812af88f0dcf093240b24ff85f9e292322c71cb19b0dda808d">+30/-45</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-3df833bc826eb12e0819620d1fe316ef1ac352d293c11e57cc3df0da3c031e93">+10/-10</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-bb24a30c9c3744eb31fb49e4271a9651d1f9a94a87455ac0074364b879af9651">+27/-24</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-85ff00530831fdaa4e13c05cb649f2e20479cce04a4b65ef78d131cdb89b6c71">+36/-35</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-bff7752b2aae0e9cef3ece1b9ddf17b9aa2f52f81fa777e6249039580aefbd69">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>nats.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-b05729b03dff9a5e29eaf536d0f90e94a0c27a27d1a3a41b52f25dc6dc380138">+8/-23</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>mysql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-a275fb17137e6e7cd3dbc6432164136b20d1c2d4757eadaadd0b4a983473865a">+14/-35</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-a281dfd3124defa76bf1100c7e4993def0cb3049cf1ad0b1c513ea0afe522d08">+2/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-07751d2c8c454417335fa9dd06507099d8b1eaea9b04af99857db6d9c619e69f">+2/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-386fd4fad1e48ab0a90516cab1947b1888b270b2fe912f30d2255cc38e6c519b">+15/-12</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>accounts.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-7ff63b9d4046282262c347d3458395eae5b6853cb004e7349a37ec2274465fe0">+4/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-71f24f3312c19e01c4eedcd6a720893e01e64d176d2afbe1e7c6171a23059716">+4/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>remote.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-fea1d32f99821f1913607362236bd9e5b65cb2a5f8ce2c11a9a4956ac646b3d1">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>dashboards.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-edd90c6ec720f676311aa31a635f954d559e63582f0b4c39cb9206bfb095737b">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>

<td><strong>m20250109_092400_recreate_tables_with_ksuids.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-c16140f94722b17bff8753f09fa80ee0bbabc2b2f2f24b8800b459ced12fe0e4">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>

<td><strong>m20250113_144600_create_unique_folder_name_idx.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-1f6dc440bcd651434f5bd6a06a4ec7d3bd88760754edeeb8d38e861208a56cbd">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>m20250125_132500_populate_templates_table.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-fa1e504b43669656f6fd652e221e954f3f3da7fd2892d524a0e8772b6d738107">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>

<td><strong>m20250320_000001_remove_alert_name_unique_constraint.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-f4094b2945a877db5709045deafdd2883ff31fd671d0782da058af40e522d559">+10/-17</a>&nbsp;
</td>

</tr>

<tr>

<td><strong>m20250611_000003_populate_reports_scheduled_jobs.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-6567d3d0361c1a184ba7877795a8c3c4f73ea9807b3d4c13efe621449dd3f80f">+8/-12</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>search_jobs.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-b78a126c6e978d43e798fa2bcb56f7bb0132c107c716d4a9891c58b3c5a33ff9">+7/-8</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>timed_annotations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-4bc910d02f5ddc598e394d06172debe456c7db655cc16fef53eab55005880378">+14/-21</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>partition.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-441e26dda30cadfa3cb1efbd9569aec96af8fdc779e4916814fb5f9477db3f6c">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>writer.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f">+5/-5</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>Additional files not shown</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7316/files#diff-2f328e4cd8dbe3ad193e49d92bcf045f47a6b72b1e9487d366f6b8288589b4ca"></a></td>

</tr>
</table></details></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>
2025-07-01 21:25:45 +08:00
ByteBaker 3aad96721d chore: maintenance and cleanup (#7289) 2025-06-30 16:26:09 +08:00
Loakesh IndiranandHengfei Yang 600c0d3b7e feat: aggregation cache (#6923)
Cache Intermediate record batches for aggregate sql queries

## TODO

- [ ] Add in memory cache file index
- [ ] Modify exiting delete stream results api to also delete
`aggregations` cache
- [ ] use bounded Chanel for `CACHE_QUEUE` and and an env for queue size

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2025-06-17 21:08:48 +05:30
Hengfei Yang 42541ad53a fix: add memory circuit breaker for ingester (#7171)
### **User description**
This PR add circuit breaker for ingester to protect OOM. To enable this
feature we need to config:

```
ZO_MEMORY_CIRCUIT_BREAKER_ENABLED=true
ZO_MEMORY_CIRCUIT_BREAKER_RATIO=90
```

If the system memory usage over `90%` , we will return `503` with error
`MemoryCircuitBreakerError`.

Default is `false`, disabled.


___

### **PR Type**
Enhancement


___

### **Description**
- Add memory circuit breaker config and defaults

- Implement breaker check in ingester writer logic

- Integrate ingestion resource check across services

- Introduce new error variants for circuit breaker


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>config.rs</strong><dd><code>Define memory circuit breaker
configuration defaults</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>loki.proto</strong><dd><code>Add Java proto generation
options</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-c6860d35cb793f55cbf91f6dc9e9366bf45a42c3b6385c6c07e813f22d18ccfb">+10/-14</a>&nbsp;
</td>

</tr>
</table></details></td></tr><tr><td><strong>Error
handling</strong></td><td><details><summary>3 files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Add IngestionError error
variant</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>errors.rs</strong><dd><code>Introduce
MemoryCircuitBreakerError variant</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-bf45932466265b732f13d0cd9b9cffbaefbe37c5861ace9315449eeca9250713">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>syslog.rs</strong><dd><code>Handle circuit breaker error
logging</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa">+2/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Miscellaneous</strong></td><td><details><summary>2
files</summary><table>
<tr>
<td><strong>lib.rs</strong><dd><code>Export check_memory_circuit_breaker
function</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-603267e2f712eeb4bc46fdff981f1fad1f3a9d3b989a7db4b3c0984fc5b1ed43">+4/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>hec.rs</strong><dd><code>Remove pre-ingest resource
check</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-5434f454898e66d4a982801761a349ac7e6862836e15eefaf643b31d0d2933dd">+1/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>12
files</summary><table>
<tr>
<td><strong>writer.rs</strong><dd><code>Implement memory circuit breaker
logic</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f">+16/-2</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Extend check_ingestion_allowed
signature and call breaker</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+9/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>bulk.rs</strong><dd><code>Pass stream type to ingestion
check</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-2017dbe9076c78a7e7919ed5c8738729123d630038803cef0ce2c356d6fa2c48">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>ingest.rs</strong><dd><code>Include circuit breaker in log
ingest checks</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-e62118e45be183266d39ffa08491f255017954235ff0f3d6937683e314da406f">+3/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>otlp.rs</strong><dd><code>Add breaker check for OTLP
logs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-183498d98b17ddf344cbab4da6b2d9fe89ac81112db51655c142a89023b46ef3">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>json.rs</strong><dd><code>Use unified ingestion check for
metrics</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-506987e61095a54e47f7f208dad89f9f1bce595fa6bf1ba8a7d746f2b2ff6c4a">+10/-20</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>otlp.rs</strong><dd><code>Add resource check for metrics
OTLP</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-a24391bef75a4678c50724e8b5f305ec0ee8c2c74c68be9d677765de68b5d8dd">+4/-23</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>prom.rs</strong><dd><code>Invoke ingestion check in Prom
remote_write</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-c4e9e1d49a2bd0fefe67684b3768c4792be63cebcf727d71a48141f5b5957b8c">+4/-18</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Use new error variant for circuit
breaker</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-124b08e44cb8740f8cb0dde3342ccd8c1b3b4e3c6a0edb12c151e060c54a0bd0">+5/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>storage.rs</strong><dd><code>Always apply memory circuit
breaker check</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-ab2d867e1e2e3dd8307505c2e130039628a84f8131a9b38cd2e7b4e1c141e094">+2/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>wal.rs</strong><dd><code>Enforce breaker and release files
on error</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-48aea52fd870678e577e03f29fc9769920b7f74761b1447242835dbf1b8ac8df">+5/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Add breaker check for traces
ingestion</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+14/-58</a>&nbsp;
</td>

</tr>

</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>loki.rs</strong><dd><code>Remove obsolete comment in
StreamAdapter</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-519e5b0c03a8916b62712d1c892dab46282acb6c3f9942c2001169a0e2ea2f3c">+0/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>search_stream.rs</strong><dd><code>Reformat long comment
lines for clarity</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-b952a89375f2745a0daa273f1bdefc80c22997759a9b17a8fccbc70ae294e8e4">+9/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>write.rs</strong><dd><code>Adjust Writer initializer tuple
unpack</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7171/files#diff-791f2d9a83e5cd9620a9649156f12988a8cf1846337d6953bfc07bea9a46b71a">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>
2025-06-14 16:25:07 +08:00
Loakesh Indiranandudev c29bc97f0a fix: histogram interval update subquery for HTTP2 (#7170) (#7174)
### **User description**
fixes #7169 for HTTP2 streaming

cherry-picked from: https://github.com/openobserve/openobserve/pull/7170


___

### **PR Type**
Bug fix, Enhancement


___

### **Description**
- Replace manual AST walk with VisitorMut for histogram

- Inject missing interval arg into `histogram` calls

- Remove deprecated `visit_statements_mut` logic

- Reflow comments in search_stream.rs


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>websocket.rs</strong><dd><code>Use VisitorMut for
histogram interval update</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/common/utils/websocket.rs

<li>Introduce <code>HistogramIntervalVisitorMut</code> struct<br> <li>
Implement <code>VisitorMut</code> to append missing interval args<br>
<li> Update <code>update_histogram_interval_in_query</code> to use
visitor<br> <li> Remove old <code>visit_statements_mut</code> & helper
function


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7174/files#diff-6ff41099eded8eca4a0e579142d7af4429efb83350917780e0d3ae7dac13847a">+39/-44</a>&nbsp;
</td>

</tr>
</table></td></tr><tr><td><strong>Formatting</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>search_stream.rs</strong><dd><code>Reflow comments in
partitioned search</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/service/search/search_stream.rs

<li>Reflow and wrap multi-line comments<br> <li> Adjust comment
formatting around skip logic


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7174/files#diff-b952a89375f2745a0daa273f1bdefc80c22997759a9b17a8fccbc70ae294e8e4">+9/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>

Co-authored-by: udev <uddhav@openobserve.ai>
2025-06-13 23:19:04 +05:30
Subhra264 c4e1c6e363 fix: _around api rbac returns 403 error (#7042)
For rbac for `_around` api, the method needs to be converted into `GET`
method instead of `POST`.
2025-06-09 18:31:14 +05:30
Hengfei Yang 94a0241e3f ci: improve unit test coverage (#7068)
Now the coverage is:

✅ Functions coverage: 46.83%
✅ Lines coverage: 46.35%
✅ Regions coverage: 34.09%
2025-06-06 20:37:41 +08:00
udevandomkark06 2bcae056ac fix: redirect short url (#7026) (#7033)
with this POC we will let FE handle all short url redirections. We keep
the old routes intact for backwards compatibility

---------

---------

Co-authored-by: omkark06 <omkar@zinclabs.io>
2025-06-06 17:33:16 +05:30
udevandloaki07 3f7582e0db fix: increase test case coverage (#7054)
Co-authored-by: loaki07 <rustyloaki@gmail.com>
2025-06-06 13:19:24 +05:30
Taiming Liu 961976f8df fix: ent build (#6957) 2025-05-28 09:43:42 -07:00
Hengfei Yang fa6454267c feat: add search scheduler (#6950)
impl #6948

- [x] When we run query, we get node list and check the node CPU, drop
at most 20% querier nodes.
- [x] When download file, we first try to download from original node(
by consistent hashing), then download from object store.
- [ ] ? should also apply this logic for ingester node ?
- [x] Fixed metrics keep increase issue of DownloadManager
2025-05-28 23:31:57 +08:00
Yashodhan JoshiandYashodhan Joshi 7540824633 fix: use _all_ in org related api call perm checks (#6920)
Fixes permission check for organization related api. In ofga when we set
a role with org perms, it sets to `org:_all_{org}` but in the check , we
check for permission on `org:{org}` which causes users with role having
permission on org, not be able to rename org. This fixes it by updating
it to correct check.

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
2025-05-28 11:02:01 +05:30
31703cc9d2 feat: o2 AI Agent (#6899)
this PR introduces o2 ai agent with this user able to ask o2 ai about
sql query , vrl query and logs

---------

Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
2025-05-27 15:09:19 +05:30
+2 0d9f47966b feat: Organization revamp (#5579)
Fixes #5131 
Addresses #5408 

### Revamp
- Created a new table to store organizations in the `organizations`
table. Earlier, we never stored organizations in any table, rather we
used to store the organizations in the meta table as part of the `user`
json.
- The users and organization membership relations are now moved to its
own separate tables - `users` and `org_users`.

### New changes

**API changes** -
- `/organizations` - `POST` api to create an org and save it in the meta
table. In non-enterprise version, currently only the root user can
create the org. For enterprise version with rbac enabled, anyone with
`CREATE` access for organizations can create the organization and
becomes an admin. Root user can always create an organization.
- `/{org_id}/rename` - `PUT` api to rename the existing org. Only root
user and the admins of the org can access this API.

The payload for following APIs are changed -
- `POST api/{org_id}/roles` - `{"name":"new_role"}` is changed to
`{"role":"new_role"}
- `POST api/{org_id}/users/{user}` -
`{"role":"viewer","custom_role":["role_name"]}` is the new payload where
`custom_role` is optional and is array of custom roles (only works in
enterprise with rbac enabled) which will be assigned to the user. The
value of `role` can be standard role or as well as any custom role (when
it is custom role, the standard role assigned to the user is decided
based on the enterprise env variable - `O2_DEX_DEFAULT_ROLE`).
- `POST api/{org_id}/users`- the payload -
```
{"email":"user@user.com","role":"viewer","custom_role":["create_org_permission"],"password":"12345678", "first_name": "", "last_name":""}
```

- `PUT api//{org_id}/users/{email_id}` the payload -
```
{"first_name":"Hello","last_name":"j","role":"viewer","isExternal":false,"change_password":false,"password":"","custom_role":["create_org_permission"]}
```


A new ENV variable is introduced - `ZO_CREATE_ORG_THROUGH_INGESTION`
(default `false`) which if true, an org will automatically created when
ingesting data to a new org. If false, user can not create org by
ingesting data to new org, they need to first call the `POST
/organizations` API.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

**New Features**
- Added a `cloud` feature for enhanced organizational management
capabilities.
- Introduced new methods for managing organizational users and
asynchronous message processing.
- Enhanced user invitation and membership features, including improved
role handling.
- Implemented configurable ingestion URL support across various
components.
- Added new functionality for user management and organization handling
in the super cluster architecture.

**Improvements**
- Refined user authentication and authorization processes, including
error handling.
- Updated database schema to support multi-organization architecture.
- Enhanced pagination settings and UI components for better user
experience.
- Improved handling of organization names and user roles in
notifications and alerts.
- Streamlined service account token management with visibility
enhancements.
- Enhanced the `AddUpdateOrganization` component with improved user
input handling and validation.
- Updated the `AppOrganizations` component to dynamically load the
appropriate organization management interface.
- Improved the `ServiceAccountsList` component with updated pagination
options and token handling logic.

**Bug Fixes**
- Resolved issues with user role assignments and organization management
inconsistencies.
- Corrected routing and navigation logic for login and logout processes.
- Addressed various error handling scenarios in user and organization
processes.
- Fixed issues with the visibility of service account options in the
`EditGroup` and `EditRole` components based on the cloud configuration.

**Breaking Changes**
- Significant restructuring of user and organization data models.
- Changes to authentication workflows and configuration handling.
- Removal of certain components related to user and organization
management.
- Updated the logic for determining the ingestion URL across various
components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
Co-authored-by: hengfei yang <hengfei.yang@gmail.com>
Co-authored-by: harshiv-7 <harshiv@openobserve.ai>
Co-authored-by: Huaijin <haohuaijin@gmail.com>
Co-authored-by: Zhao.wei <weizhao029@gmail.com>
Co-authored-by: ShyamOOAI <shyam@openobserve.ai>
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
Co-authored-by: Yashodhan Joshi <yashodhan@openobserve.ai>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
2025-05-25 21:53:55 -07:00
Taiming Liu f680073307 feat: add _timestamp to tantivy index to optimize histogram queries (#6801) 2025-05-24 11:15:46 +08:00
Loakesh Indiran 1ad3f8368e fix: chery-pick fixes (#6853)
- fix: auth for _values_stream (#6849)
- fix: simple distinct sql for http api (#6836)
- feat: add env to enable use_stream_setting for partitions (#6866)

# closes
- https://github.com/openobserve/openobserve/issues/6863

New Env
```
ZO_USE_STREAM_SETTINGS_FOR_PARTITIONS_ENABLED # default false
```
2025-05-21 17:12:52 +05:30
udev da2a769762 feat: Add HTTP2 Streaming support (#6721) (#6827) 2025-05-20 00:06:05 +08:00
Subhra264andsai nikhil kethe 330cf14ec0 fix: ZO_DEFAULT_MAX_QUERY_RANGE_DAYS env should be returned by streams list api (#6762)
Fixes #6753

---------

Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
2025-05-09 14:22:58 +05:30
Ashish Kolhe 1f47ac2614 fix: base64 encode cookies (#6596)
fixes : [6564](https://github.com/openobserve/openobserve/issues/6564)
2025-04-28 18:08:48 +05:30