mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-06 00:09:22 +00:00
55a0c62d3fcce7b08053effd2cbd435f862e8ed6
680
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Hengfei Yang
|
55a0c62d3f | fix: root user password reset (#8348) | ||
 sktrpathi
|
f8cc585408 |
feat: store ratelimit info in vendor extension (#8259)
### **PR Type** Enhancement, Documentation ___ ### **Description** - Replace ratelimit comments with OpenAPI extensions - Add `x-o2-ratelimit` metadata to endpoints - Import `serde_json::json` where needed - Clean up obsolete inline ratelimit markers |
||
     
|
8af3495611 |
fix: cloud sync (#8222)
### **User description**
This is for syncing the changes done in cloud to main branch. No changes
for non-cloud code.
___
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Add invite listing and decline APIs
- Bypass auth for user-scoped invite routes
- Improve trial-expiry handling and logging
- Add org cache and invalidation hooks
___
### Diagram Walkthrough
```mermaid
flowchart LR
FE["Login & Invitations UI"] -- "list/accept/decline" --> API["/api/invites*, /{org}/member_subscription"]
API -- "auth bypass for user-scoped invites" --> Auth["AuthExtractor/TokenValidator"]
Auth -- "pending invites flag" --> LoginRedir["/config/redirect builds URL params"]
OrgSvc["Organization Service"] -- "generate/list/decline/delete" --> InvitesDB["Org Invites"]
OrgSvc -- "invalidate cache" --> OrgCache["Org table in-memory cache"]
Ingest["Ingestion services"] -- "trial expired -> 429" --> Clients["Ingestion Clients"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>25
files</summary><table>
<tr>
<td><strong>user.rs</strong><dd><code>Expand invite payload with org and
inviter</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>auth.rs</strong><dd><code>Auth bypass and key handling for
invite routes</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+36/-0</a>
</td>
</tr>
<tr>
<td><strong>jwt.rs</strong><dd><code>Return new user and pending invite
flags; defer org creation</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+22/-4</a>
</td>
</tr>
<tr>
<td><strong>token.rs</strong><dd><code>Allow invite/member routes
without org user; header injection</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-3339b4a964c121766ef87fcec0070f27a2b7085cf81281b53154e53d33cb28cf">+34/-0</a>
</td>
</tr>
<tr>
<td><strong>ingest.rs</strong><dd><code>Suppress logging for trial
expiry errors</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-24f250254ecfdbe004af9dc0ea23f8a2c0105ae0898690ed3e565089e5d8eef9">+30/-9</a>
</td>
</tr>
<tr>
<td><strong>org.rs</strong><dd><code>Pending-only invites; delete invite
API; trial extend cache poke</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-b47c0c1a8a78524196176430268eecdbfaf6f46d6b95873d3cbe70cfd67182b0">+40/-4</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Include pending_invites flag in
login redirect</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+4/-4</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>List pending invites and decline
invite endpoint</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-56897e42f9436a640bc70fddc25686485ba8ae1223635b32d252a0333309f748">+36/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Wire new invite endpoints into
router</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add TrialPeriodExpired error
variant</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>organizations.rs</strong><dd><code>Add in-memory org cache
with invalidation</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-4d87f9fe379a5d161a3e7c3c77f7f320be738e6a70c785aa5ce93cb0830c0147">+77/-7</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Start cloud billing watcher on
cloud builds</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+3/-0</a>
</td>
</tr>
<tr>
<td><strong>organization.rs</strong><dd><code>Invalidate org cache on
watch and mutations</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ce65cea926a09721a8fea14d07fe188bc2b887218e95d74a8bffefb54001da50">+4/-0</a>
</td>
</tr>
<tr>
<td><strong>user.rs</strong><dd><code>Delete invites for user
helper</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-6ee6b0fc9ed03224d4ddf0b7b631c6f81ff7d1d579bdb98d2c5d4453852eb5d6">+4/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Return trial expiry as dedicated
error</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+1/-3</a>
</td>
</tr>
<tr>
<td><strong>loki.rs</strong><dd><code>Silence trial expiry logs for Loki
ingestion</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a970c8c4785d4e4d878218615b17eec0cf4b4976121766b002c2b2f76371bfe6">+4/-1</a>
</td>
</tr>
<tr>
<td><strong>json.rs</strong><dd><code>Map trial expiry to HTTP 429 in
JSON ingest</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-506987e61095a54e47f7f208dad89f9f1bce595fa6bf1ba8a7d746f2b2ff6c4a">+15/-6</a>
</td>
</tr>
<tr>
<td><strong>otlp.rs</strong><dd><code>Return 429 for trial expiry;
reduce noise</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a24391bef75a4678c50724e8b5f305ec0ee8c2c74c68be9d677765de68b5d8dd">+17/-7</a>
</td>
</tr>
<tr>
<td><strong>organization.rs</strong><dd><code>Invite generation
validation; delete/decline invite flows</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-6b3c3c2b027c1158c0d41302cc69dfcd20ab68a6a4fa365cda403f838940bbb4">+48/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Handle trial expiry with 429;
logging tweaks</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+34/-14</a>
</td>
</tr>
<tr>
<td><strong>users.rs</strong><dd><code>Enrich invite list; optional
pending filter; cleanup on removal</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+50/-6</a>
</td>
</tr>
<tr>
<td><strong>organizations.ts</strong><dd><code>Add decline subscription
API client</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-2c3bad4c7a9fcedd7a653599381149b82f9c350b177de9d72431013807d4c36f">+10/-7</a>
</td>
</tr>
<tr>
<td><strong>users.ts</strong><dd><code>Add pending/accept/reject invite
APIs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a334d8d570d9bdcaa3d4c8c2a993eff009a1ec8eb499c8a3d1a292d9224159d0">+12/-4</a>
</td>
</tr>
<tr>
<td><strong>InvitationList.vue</strong><dd><code>New UI to accept or
reject invites</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ee97546ff1aba36cc0b24cb20782502c7c88279850cc3d540ea5685209f10702">+409/-0</a>
</td>
</tr>
<tr>
<td><strong>Login.vue</strong><dd><code>First-login invite handling and
logo tweaks</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a135c379ef6ac8f92199ea5f7ff19a227971a9b0a87ccccf87c0268740522c2c">+178/-58</a></td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>en.json</strong><dd><code>Add invitation-related
translations</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+18/-0</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com>
Co-authored-by: udev <uddhav@openobserve.ai>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: udev <uddhavdave1@gmail.com>
|
||
|
sktrpathi
|
d87d346bba |
fix: OpenAPI descriptions for endpoints (#8231)
### **User description** So far the OpenAPI descriptions for endpoint handlers was directly derived from the doc comments, leaving little control. This also leaked any dev-specific information into API spec. Now the summary and description are explicitly populated for all the endpoints, detaching them from doc comments. Also update response body annotations wherever necessary Also bump - utoipa: 4.0.0 -> 5.0.0 - utoipa-swagger-ui: 4.0.0 -> 9.0.0 ___ ### **PR Type** Documentation, Enhancement ___ ### **Description** - Add explicit OpenAPI summaries/descriptions - Correct response body schemas to Object/() - Update params/types for accuracy - Improve endpoint examples and metadata |
||
|
Subhra264
|
c49df079c7 |
test: add unit tests for alerts and self reporting (#8228)
### **PR Type**
Tests
___
### **Description**
- Add comprehensive unit tests for alerts
- Add extensive self-reporting queue tests
- Add serialization and model tests for usage
- Validate batching and timing behaviors
___
### Diagram Walkthrough
```mermaid
flowchart LR
Alerts["Alert model"] -- "unit tests" --> AlertsTests["alerts::tests"]
Reporting["Self-reporting queue/runner"] -- "unit tests" --> ReportingTests["self_reporting::tests"]
Usage["Usage/Trigger types"] -- "serde, traits tests" --> UsageTests["usage::tests"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>alert.rs</strong><dd><code>Add comprehensive tests for
Alert and params</code>
</dd></summary>
<hr>
src/config/src/meta/alerts/alert.rs
<ul><li>Add tests for <code>Alert</code> defaults and equality<br> <li>
Validate unique key logic and timestamps getters/setters<br> <li> Test
<code>ListAlertsParams</code> builder methods and chaining<br> <li>
Cover ancillary fields: destinations, context, owner/editor, flags</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-45c7eaf1a764ab52f3e014fe0555c03dc59d77c5ce974684cdee4ed90463a234">+415/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Add tests for self-reporting
queue and runner</code>
</dd></summary>
<hr>
src/config/src/meta/self_reporting/mod.rs
<ul><li>Add async tests for <code>ReportingQueue</code>
enqueue/start/shutdown<br> <li> Test batching logic in
<code>ReportingRunner</code><br> <li> Validate debug formatting and
mixed data handling<br> <li> Add ignored test for channel-full
behavior</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-f4101fa8423f7a25f311dd82dab6796ed4ac8fef68449b5f018157dc33ca7de3">+674/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>usage.rs</strong><dd><code>Add serialization and model
tests for usage types</code>
</dd></summary>
<hr>
src/config/src/meta/self_reporting/usage.rs
<ul><li>Add serde tests for <code>UsageData</code>,
<code>TriggerData</code>, enums<br> <li> Test display helpers and
classification methods<br> <li> Validate <code>RequestStats</code> and
<code>Stats</code> defaults/from/serde<br> <li> Test hashing/equality
for grouping keys and enums</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-b15a6adcb163dc27993c83528105a6726c44c86d8a84c5cf5d1d382c6ec8cc29">+684/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Hengfei Yang
|
1989432ba7 |
feat: remove syslog (#8227)
### **User description** close #8186 ___ ### **PR Type** Enhancement, Documentation ___ ### **Description** - Remove syslog feature across codebase - Clean config and routing references - Update metadata prefixes and SBOM files - Refresh i18n and OpenAPI assets ___ ### Diagram Walkthrough ```mermaid flowchart LR SYS["Syslog feature"] -- "deprecated & removed" --> CODE["Code paths"] CODE -- "delete flags/routes" --> CFG["Configuration"] CODE -- "remove storage keys" --> META["Metadata prefixes"] CODE -- "update specs" --> SBOM["SBOM/XML assets"] CODE -- "update docs/i18n" --> WEB["Web locales & OpenAPI"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>13 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Remove syslog config import from job module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+3/-26</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Stop glob importing config; drop syslog usage</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+1/-3</a> </td> </tr> <tr> <td><strong>meta.rs</strong><dd><code>Drop `/syslog` from metadata item prefixes</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-1797798cd07803457a6273ef793d56568d8c91e743f6c4649c6db733325fb0af">+1/-2</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Remove syslog-related configuration options</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+0/-34</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Purge syslog feature flag and defaults</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+0/-25</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Clean references tied to syslog ingestion</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-5061e662a8899b7e00e5744bd72f808b28ac479a8921ac519f7409bffdc4438a">+0/-60</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Delete syslog router endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+0/-5</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Detach syslog handler wiring</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-99620cff6123acf0598ed2ec9d1c06f02cb0ddece1bef898047eb303d0b71fc2">+0/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Remove syslog-specific log ingestion paths</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-ce72aea3fa4c44904cbd9ec07bd844bec71b49c9676d667b14056aa5798f96b2">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Clean DB interactions for syslog keys</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Adjust request handling without syslog</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Unexport syslog services</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-1b15316505c31acccb26ba66d920fdb0894567d08d2bf1e3457716d128a67b83">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Remove syslog metadata definitions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-fbf797574839e69491ea0429141c038c0ff96297bdf3d8c60d60cdc4acec74b1">+0/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>integration_test.rs</strong><dd><code>Update tests to exclude syslog paths</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-7ff815b68943f08bef79f083c09e55ca8c18db8e25033034b12fa31f1d43dab4">+0/-57</a> </td> </tr> <tr> <td><strong>useIngestionRoutes.spec.ts</strong><dd><code>Update UI tests to remove syslog routes</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-3b7ebacf9481f4643a2b56ffe53ab161fc89d416e280d45b48568866a4dee40c">+0/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>20 files</summary><table> <tr> <td><strong>openapi.rs</strong><dd><code>Remove syslog routes from OpenAPI</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+0/-5</a> </td> </tr> <tr> <td><strong>config.cdx.xml</strong><dd><code>Update config component index (remove syslog)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-41ee11320f37e20fc96f754d35a890df5068af5072fd5388169d1cf305a1ed3b">+15011/-4611</a></td> </tr> <tr> <td><strong>wal.cdx.xml</strong><dd><code>Update WAL component index references</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-3d72e8b949a91b0bd78e97aa3ebf69e2887e2851223ea3d10fedeb0125c35a19">+19476/-358</a></td> </tr> <tr> <td><strong>openobserve.cdx.xml</strong><dd><code>Refresh SBOM root; remove syslog entries</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-454f38f03cf69f6b5358f5a528be4214ec2d33081b9e98ebe5d92c8bef47dc48">+14015/-7932</a></td> </tr> <tr> <td><strong>infra.cdx.xml</strong><dd><code>Update infra component index, no syslog</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-e51bf18198bc65a604b53ee18152ad9f2453ab7870d5be949022058e054ada73">+14127/-6086</a></td> </tr> <tr> <td><strong>proto.cdx.xml</strong><dd><code>Update proto index; purge syslog mentions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-a15fa99b53917ea85b4e268bb7612d43ee881dbe4ecf70a6682bb919b1f2f219">+8000/-1513</a></td> </tr> <tr> <td><strong>ingester.cdx.xml</strong><dd><code>Adjust ingester index excluding syslog</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-e72a9be4ba2aaff1a6b837a7f7149489603f8fe720766757c4291082e98b5a65">+15845/-4615</a></td> </tr> <tr> <td><strong>flight.cdx.xml</strong><dd><code>Add/refresh flight component index</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-f41393a0191b0560da893a5e24b2e4529f7c9bb34fc71231a532b8790630cba2">+19900/-0</a></td> </tr> <tr> <td><strong>hi.json</strong><dd><code>Remove syslog strings from Hindi locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-1e7b52b5c4e762db2c923d0d3efb6d693702825ed3a1a112d3069f26b0d5f605">+0/-13</a> </td> </tr> <tr> <td><strong>fr.json</strong><dd><code>Remove syslog strings from French locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-db80e762b88ddf183820c050cdc6b215e6b02b365b5e73f5152af46b0d26f1a2">+0/-13</a> </td> </tr> <tr> <td><strong>tr.json</strong><dd><code>Remove syslog strings from Turkish locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-6f0f131e9e2591480c4c5d1eaffc72d02dc31cbd8d3f90d8f012f08cabac8818">+0/-13</a> </td> </tr> <tr> <td><strong>de.json</strong><dd><code>Remove syslog strings from German locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-78f24ecb2dbcfc38fdf3ec22b2559a5b40d236ea689d1c55f5d3e87168310264">+0/-13</a> </td> </tr> <tr> <td><strong>ko.json</strong><dd><code>Remove syslog strings from Korean locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-50e27ca1b4825a5026f3a251bfcf79c97cab07cfade2ebd512d332453d707c48">+0/-13</a> </td> </tr> <tr> <td><strong>it.json</strong><dd><code>Remove syslog strings from Italian locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-2de78074a5f887afddce60d694ead9322e6ab223eb70251c13819f3028bdc70d">+0/-13</a> </td> </tr> <tr> <td><strong>nl.json</strong><dd><code>Remove syslog strings from Dutch locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-a1706021687ae5d8a1c006c49eab3946174fecd895f9b70a89c2e76d97250906">+0/-13</a> </td> </tr> <tr> <td><strong>es.json</strong><dd><code>Remove syslog strings from Spanish locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-ac2a2167b8b973e3f776f9fe3b43bef4576470d8ad33f3d7cfaff6edff0e5778">+0/-13</a> </td> </tr> <tr> <td><strong>pt.json</strong><dd><code>Remove syslog strings from Portuguese locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-38e804f64cb5e0e451849fc84e9b46e53c954a4a4263d51c18665ad45f4b4815">+0/-13</a> </td> </tr> <tr> <td><strong>zh.json</strong><dd><code>Remove syslog strings from Chinese locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-8818055db58f83230aeea4ebfd4cec31d924e8c47b6f91298e356959ab25339e">+0/-13</a> </td> </tr> <tr> <td><strong>ja.json</strong><dd><code>Remove syslog strings from Japanese locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-092e49c5f9fbc1b539ab9c2866d108d0ce1d314022b4c294eb4611b285e7bbd3">+0/-13</a> </td> </tr> <tr> <td><strong>en.json</strong><dd><code>Remove syslog strings from English locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+0/-13</a> </td> </tr> </table></details></td></tr><tr><td><strong>Dependencies</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>Cargo.toml</strong><dd><code>Update dependencies/features after syslog removal</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542">+0/-3</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>10 files</summary><table> <tr> <td><strong>syslog.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-dc6813e31e32c23aae5760091460d2ec4023d37ae73e49d29052d9f4e9f2e1fa">+0/-43</a> </td> </tr> <tr> <td><strong>websocket.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-48f05dcd4b6f1d8345cb5707e1a3c34eedd76d94a3c3904eeabd799c7820789d">+0/-90</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-58010c6e78f69dc57c353890a807371cd43c2b4cceeeb1e9378130f26d898397">+0/-143</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-66207bd699343342402f88cdd331f4c549783754e4921d0c8d43e026e2d8e575">+0/-135</a> </td> </tr> <tr> <td><strong>syslog_server.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-cb7610142582b44885f673583ab16262c3b9ca6de1f59a9def65e503360665a0">+0/-137</a> </td> </tr> <tr> <td><strong>report_server.cdx.xml</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-d7a2fd1b635919119af9187ef44188c384bd01e444e74a9d15eaa21b0840b7ed">+20472/-0</a></td> </tr> <tr> <td><strong>syslog.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-4c0e9282e3d2352f9a382099901fe034bae2961eed01d764f16ad52f27f55db4">+0/-180</a> </td> </tr> <tr> <td><strong>syslog.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa">+0/-559</a> </td> </tr> <tr> <td><strong>syslogs_route.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-325df35568c993fc0727bfb0499b682e6bd30af781687706306b3bdb429c49e2">+0/-191</a> </td> </tr> <tr> <td><strong>syslog.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-69ec0272ddac65f2cc96c79d84ca3933fc335a8d21ab5dd6909d71ea02a46486">+0/-37</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
 Hengfei YangandBhargav
|
2c0395124f |
feat: deprecated syslog (#8187)
### **User description** impl #8186 ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Mark syslog modules as deprecated - Add allow(deprecated) to callers - Block enabling syslog setting - Remove syslog-related tests and OpenAPI types ___ ### Diagram Walkthrough ```mermaid flowchart LR meta["meta::syslog module"] -- "mark deprecated" --> callers["call sites"] callers -- "add allow(deprecated)" --> compile_ok["compiles with warnings suppressed"] db_set["db::syslog::toggle_syslog_setting"] -- "reject enable=true" --> error["returns deprecation error"] openapi["OpenAPI schema"] -- "remove syslog types" --> spec["updated API spec"] tests["syslog tests"] -- "removed" --> leaner["reduced test surface"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Suppress deprecation warnings for syslog globals</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog handlers; annotate endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-58010c6e78f69dc57c353890a807371cd43c2b4cceeeb1e9378130f26d898397">+6/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Suppress deprecation in API scope setup</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog server wiring</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-66207bd699343342402f88cdd331f4c549783754e4921d0c8d43e026e2d8e575">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog cache/watch tasks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+4/-0</a> </td> </tr> <tr> <td><strong>syslog_server.rs</strong><dd><code>Allow deprecated syslog server run linkage</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-cb7610142582b44885f673583ab16262c3b9ca6de1f59a9def65e503360665a0">+2/-0</a> </td> </tr> <tr> <td><strong>syslog.rs</strong><dd><code>Allow deprecated syslog ingest; remove tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa">+3/-14</a> </td> </tr> <tr> <td><strong>syslogs_route.rs</strong><dd><code>Allow deprecated route endpoints; remove tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-325df35568c993fc0727bfb0499b682e6bd30af781687706306b3bdb429c49e2">+7/-52</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>6 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Mark syslog meta module as deprecated</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-fbf797574839e69491ea0429141c038c0ff96297bdf3d8c60d60cdc4acec74b1">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Mark syslog HTTP request module deprecated</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+1/-0</a> </td> </tr> <tr> <td><strong>openapi.rs</strong><dd><code>Remove syslog types from OpenAPI imports</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslog DB module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslog logs module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-ce72aea3fa4c44904cbd9ec07bd844bec71b49c9676d667b14056aa5798f96b2">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslogs_route service module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-1b15316505c31acccb26ba66d920fdb0894567d08d2bf1e3457716d128a67b83">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>syslog.rs</strong><dd><code>Remove unit tests for syslog meta types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-dc6813e31e32c23aae5760091460d2ec4023d37ae73e49d29052d9f4e9f2e1fa">+0/-127</a> </td> </tr> </table></details></td></tr><tr><td><strong>Error handling</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>syslog.rs</strong><dd><code>Block enabling syslog; allow deprecated routes/cache</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-4c0e9282e3d2352f9a382099901fe034bae2961eed01d764f16ad52f27f55db4">+11/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: Bhargav <BJP232004@GMAIL.COM> |
||
|
sktrpathi
|
75fc81cecf |
chore: remove bogus unit tests (#8172)
### **User description** A significant chunk of API unit tests were not testing any actual business logic, but performing trivial actions such as struct-field-access, derive trait validation such as `Eq`, `PartialEq`, `Clone`, `Serialize` etc, and many of the tests were also just comparing constant values. They added no value to the test suite and have now been removed from the codebase, making the tests more focused. Additionally, - A lot of flaky tests have been corrected. These used to fail due to occasional race conditions when ran locally. - Some mock setup and teardown functions have also been cleaned up, which makes tests a lot more consistent. ___ ### **PR Type** Tests, Enhancement ___ ### **Description** - Remove trivial, non-value unit tests - Relax flaky assertions and panics handling - Improve test isolation via DB/cache resets - Minor test output formatting improvements ___ ### Diagram Walkthrough ```mermaid flowchart LR tests["Remove trivial/boilerplate tests"] -- cleaner suite --> stability["Stabilize assertions and panics"] stability -- reduce flakiness --> isolation["DB/Cache resets in tests"] isolation -- consistent runs --> outputs["Improve test log formatting"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><details><summary>11 files</summary><table> <tr> <td><strong>organization.rs</strong><dd><code>Remove struct-field and default-value trivial tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-4f089715a7ac4f78b45039d773f2fccbb4fbc6a409e0ab9ceee1fcf7911bc44b">+4/-325</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Drop JSON roundtrip serialization tests; tidy imports</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-c4bd542e4e07ed7706b0643086f5a7d4edbb3647c7ceedbe4bea516c5274d8bd">+1/-183</a> </td> </tr> <tr> <td><strong>auth.rs</strong><dd><code>Remove Debug/PartialEq trivial tests; format strings</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+2/-104</a> </td> </tr> <tr> <td><strong>main.rs</strong><dd><code>Make tracing test tolerant; ignore enterprise tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+23/-60</a> </td> </tr> <tr> <td><strong>event.rs</strong><dd><code>Remove constants and simple struct creation tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-689769712cbd6bd02beba078c1e1dfc188e2be39ab19a0e232b8bd12bc7f1e78">+1/-95</a> </td> </tr> <tr> <td><strong>validator.rs</strong><dd><code>Ensure test isolation with ORM init and cache clears</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+28/-22</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Make base URI tests robust to env errors</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+25/-4</a> </td> </tr> <tr> <td><strong>users.rs</strong><dd><code>Use best-effort cleanup; clear caches; improve logs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+12/-9</a> </td> </tr> <tr> <td><strong>integration_test.rs</strong><dd><code>Compact println formatting and panic messages</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-7ff815b68943f08bef79f083c09e55ca8c18db8e25033034b12fa31f1d43dab4">+3/-3</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Remove strict zero-length assertions on globals</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+1/-13</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Remove unused constant import in tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-a5ea87a7766cb24619f64a6100f385f4b42c47af7d33b8e2f060229c536d7418">+1/-8</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>26 files</summary><table> <tr> <td><strong>authz.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-28a815d1c4728aff854a9996de0ba3d3dafe1bea6bfb0cf2a9f4186c40f985b6">+0/-52</a> </td> </tr> <tr> <td><strong>loki.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-121c166af5eab8b9cb83716afa7b65fa7856c006c809b3d910ced184c52ec587">+0/-7</a> </td> </tr> <tr> <td><strong>proxy.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-7f12a4f2483e737671ac41f9c35b411e7323e0ff2731ad92a3b309e144d1316a">+0/-46</a> </td> </tr> <tr> <td><strong>service.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-61d456e7217d19bcec3920a4d1c17fd396a257c81e5394655db3632a057bdd9e">+0/-7</a> </td> </tr> <tr> <td><strong>service_account.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-878ad520e1c84500cc21ff6fc4088c8da64ff2febd79e60c0d4cc8c695c49798">+0/-80</a> </td> </tr> <tr> <td><strong>traces.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-bda8182c1f17a8360337c769edade0cb9478ff1443dd6aedce7f526afbe97afe">+0/-106</a> </td> </tr> <tr> <td><strong>user.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+0/-101</a> </td> </tr> <tr> <td><strong>auth_tests.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-2961</a></td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2aa59ce4464e7dbabdb9367fc8bb8c1b489c914b8690577d4916870463ab12db">+0/-1</a> </td> </tr> <tr> <td><strong>action.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-fc87c75a7df9539d7e7ee6184dfab02d4a8932323e81f95d57c05363aa44752d">+0/-18</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-5bf716db7e1d91dfbd427a2e320ce407c4f0c92d55d68f8e636daed1b02abd56">+0/-13</a> </td> </tr> <tr> <td><strong>reports.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-d2117a3d67d4ec47b0017861845c92143baf917e10e6746df5dd97f925f81eeb">+0/-10</a> </td> </tr> <tr> <td><strong>destinations.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-e0ba33d95ef7f0eac13bdac7e750462907a0308f01c027ea206d058e282c977b">+0/-103</a> </td> </tr> <tr> <td><strong>inverted_index.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-8bf9d9403f05f53f0c6fd5e6a6dbf0204ea80465b233567552cb7395d08d1c3c">+0/-22</a> </td> </tr> <tr> <td><strong>meta_store.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-45018c838da377fa9ded9305a1ee752adde06acdb607addf26c95f5747537615">+0/-6</a> </td> </tr> <tr> <td><strong>ratelimit.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-fa7dc03fb526c09d5eb58021872d821fcd72eeb4d6a670224f5286012485bc3d">+0/-10</a> </td> </tr> <tr> <td><strong>timed_annotations.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f94b4d6d12d68e7054ad33601e666b08e4594586013dc34384fa21b1df5fe6e8">+0/-12</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+0/-34</a> </td> </tr> <tr> <td><strong>m20250611_000002_populate_reports_table.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-8247af6d6f8ccb6a5a51daff4cb1e352585be2d1c785d86ae5ebfea1dbc1099c">+0/-1</a> </td> </tr> <tr> <td><strong>folders.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-402482c22fda5dff8f33382ac11d0908f693975dec7c08b37b1c5fe24a20486a">+0/-1</a> </td> </tr> <tr> <td><strong>bulk.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2017dbe9076c78a7e7919ed5c8738729123d630038803cef0ce2c356d6fa2c48">+0/-9</a> </td> </tr> <tr> <td><strong>ingestion.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-3edd5f4ed1eed0f765b7b450ef0fbf587632e6b22302006a5e397049e297057f">+0/-7</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2f7f0bac01afc6e37908521d1d33d05fcd650b7a08792062976e1e7465b4a1f9">+0/-42</a> </td> </tr> <tr> <td><strong>footer_cache.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f9278e53ed7131c953fc30f8b15cda801b74b81fb2d570fe7f80a3b7fc1de872">+0/-7</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-1602f85399080c1e2838d1ef1d62af23d96f353379e49a7770026e892c954018">+0/-20</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+0/-31</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
udev
|
b4eac8f607 |
fix: Inc UT coverage (#8128)
increase UT coverage improve the email regex to catch `doubledotsequence` in email |
||
|
Hengfei Yang
|
48115e1ab9 |
fix: enhance machine id check (#8160)
### **User description** - Add duplicated machine id check - Add 4 random chars for generate parquet filename - Use Atomic replace unsafe code ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Centralize node ID generation logic - Replace unsafe globals with atomics - Detect and resolve duplicate node IDs - Truncate long Snowflake IDs when parsing ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Cluster register (etcd/nats)"] -- "collect node ids" --> B["generate_node_id()"] B -- "new id" --> C["LOCAL_NODE_ID (AtomicI32)"] C -- "reload if changed" --> D["Snowflake ID generator"] E["set_status()"] -- "detect duplicate id" --> B F["CLI parse-id"] -- "truncate to 19 digits" --> G["parse i64"] H["Parquet filename"] -- "append 4-hex random" --> I["unique file path"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>cli.rs</strong><dd><code>Truncate IDs longer than 19 digits before parse</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+8/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>10 files</summary><table> <tr> <td><strong>etcd.rs</strong><dd><code>Use centralized node-id generation and atomics</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-37c2c2803d1c12ebb4ef9299bc79c9e6740fc5f19f7f4d84e7cfbb610b9561a9">+3/-17</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add reusable generate_node_id utility</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+17/-0</a> </td> </tr> <tr> <td><strong>nats.rs</strong><dd><code>Centralize id generation; detect duplicates; reload</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-901773980aa255942e7865b977fe63811f31511ac40bc512d6bbb14dbb54bece">+20/-17</a> </td> </tr> <tr> <td><strong>cluster.rs</strong><dd><code>Replace unsafe globals with AtomicI32</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-68d7195ab7cccf22579c4944a37c8a2f592392c30371ce907fadf62fca6f97dd">+3/-3</a> </td> </tr> <tr> <td><strong>ider.rs</strong><dd><code>Read machine id atomically; support reload</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-6ea42b8eab8ba0708f93f9a8b44056d9a9e6ac42517ffba12f81420d6b0bd2b0">+10/-2</a> </td> </tr> <tr> <td><strong>cluster.rs</strong><dd><code>Node equality includes id; NodeStatus i32</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-a2667c1be522ce095796a7066e02631a7e01acc85a46573e1e3a157d2aefb7c3">+4/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add 4-hex random suffix to filenames</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-30f42c3685037c219f0033617f032b23a7f74624ba4ae6c4e3687af401304591">+2/-1</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Initialize generators using atomic node id</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-4696a44abde2df4ac48fed716f930bfdcc01759bdfcfd8d802821c6dd857bff9">+7/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Generate record ids using atomic node id</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+2/-2</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Use atomic node id for stream generators</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-d9596182a0c7a46a6a7ff5cee822554cdae05ceec312f86c93b538db9f73bbfd">+7/-2</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
udev
|
77e50caea5 |
fix: make config reactive to env changes (#7996) (#8092)
### **User description** fix #7996 sync to main ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Make periodic jobs react to config changes - Introduce reusable `spawn_pausable_job!` macro - Fix env var name `ZO_SEARCH_JOB_RUN_TIMEOUT` - Replace fixed intervals with dynamic getters ___ ### Diagram Walkthrough ```mermaid flowchart LR CFG["get_config() (dynamic)"] MACRO["spawn_pausable_job! macro"] JOBS["Background jobs (GC, compactor, alerts, stats, etc.)"] ENV["Env vars (e.g., ZO_SEARCH_JOB_RUN_TIMEOUT)"] ENV -- "drives" --> CFG CFG -- "intervals/pause_if" --> MACRO MACRO -- "spawns/controls" --> JOBS ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Heartbeat interval now read each loop</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+2/-1</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Fix search job timeout env var name</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>16 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Export pausable_job utilities module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f5df40ca59915d9d58ad04ea2440e6ebf29388afaeb8cb0d6c63d2a979df0c17">+1/-0</a> </td> </tr> <tr> <td><strong>pausable_job.rs</strong><dd><code>Add pausable job spawning macro and helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-7e2dce7a3942789804bae659939c214c4eeb68d89315e23a08139214c339ea24">+143/-0</a> </td> </tr> <tr> <td><strong>disk.rs</strong><dd><code>Use pausable job for disk cache GC</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-af63f2c5d85313779df0204c9689e50a6c3916e3aadc24e54e23ec640a584fcb">+4/-10</a> </td> </tr> <tr> <td><strong>memory.rs</strong><dd><code>Use pausable job for memory cache GC</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-ff00d697a9b223ef2858c5e41111a8918e7cc347dc22891c02bda6982682eee1">+4/-16</a> </td> </tr> <tr> <td><strong>lib.rs</strong><dd><code>Mem persist uses dynamic sleep interval</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-603267e2f712eeb4bc46fdff981f1fad1f3a9d3b989a7db4b3c0984fc5b1ed43">+4/-5</a> </td> </tr> <tr> <td><strong>alert_manager.rs</strong><dd><code>Make alert/search jobs pausable and reactive</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-7e30858abad652a860186eb047347305488f52356112b1495dd53ce740316d7c">+61/-104</a></td> </tr> <tr> <td><strong>compactor.rs</strong><dd><code>Refactor compactor jobs to pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-550629b7ebf6460b36fdd46cdfea466c3d3f1e30023c71ee54b87e7998efc500">+134/-188</a></td> </tr> <tr> <td><strong>parquet.rs</strong><dd><code>Use dynamic interval for file push loop</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-27f710981cfe1b938aeeb53abaed05961596c32e2edf3b8a2a59d3e72b2f542b">+1/-1</a> </td> </tr> <tr> <td><strong>flatten_compactor.rs</strong><dd><code>Flatten generator via pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-dd30b917380ba55bd17aadeaee6aa2c87a67e861257a6a14ba2113a9b3015030">+15/-20</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire jobs with reactive spawning; telemetry</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+21/-13</a> </td> </tr> <tr> <td><strong>promql.rs</strong><dd><code>Make metrics leader push pausable/optional</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-15414a41d64eb87bc5c0cd00a07213b3f99c31b346e8e2c36178f7276a1f058c">+26/-31</a> </td> </tr> <tr> <td><strong>stats.rs</strong><dd><code>Stats jobs via pausable macro; conditional start</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f3ca67d9d2c68beeafd6e5de947dbc9a648ac2d2699c5dc87458c369887d09b9">+51/-42</a> </td> </tr> <tr> <td><strong>telemetry.rs</strong><dd><code>Remove legacy telemetry loop (replaced)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-e82cf6d4c890bfaa5e711293b599d7f7a742bb96180639815ede33958b1bc1d9">+0/-36</a> </td> </tr> <tr> <td><strong>storage.rs</strong><dd><code>Make enrichment merge job pausable</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-335bb1eea9193dc398da6838c6a6a11dd6f09e984a1ed4a2e611618a427754f0">+42/-35</a> </td> </tr> <tr> <td><strong>distinct_values.rs</strong><dd><code>Periodic flush via pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-3fb2320197c6e8e900658f5a24158e922610eadcbf081ab0df230f770ef27e07">+11/-18</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Audit publish via pausable macro, optional run</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-c6255010f7a87d6acdb3ad04954fc9f2c27350ae319c53a1857b120aa06408a3">+17/-15</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
sktrpathi
|
9a11a680ae |
chore: add test cases (#8095)
### **User description** - also clear over 400 clippy issues ___ ### **PR Type** Tests, Enhancement ___ ### **Description** Add extensive unit tests for enrichment storage Expose helpers for testability in storage module Modernize tests: clippy-friendly string formatting Normalize constants in time-based tests |
||
|
75f2235d04 |
feat: create stream with field and data type (#7851)
### **User description** closes #6660 ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add field data types and UDS support - Validate field names with strict regex - Store defined schema as objects, not strings - Backward-compatible parsing and UI updates ___ ### Diagram Walkthrough ```mermaid flowchart LR API["API: Stream settings/schema"] --> Parse["Parse defined_schema_fields (strings|objects)"] Parse -- "normalize to DataField{name,type}" --> Settings["StreamSettings.defined_schema_fields: DataField[]"] Settings -- "new stream" --> SchemaAdd["Add fields to Arrow schema with types"] Settings -- "existing + UDS" --> UDSUpdate["Add new fields to UDS + persist"] Types["DataType enum (Utf8, Int64, Uint64, Float64, Boolean)"] --> Arrow["Arrow DataType mapping"] Validate["Regex field name validation"] --> Save["save/update settings endpoints"] UI["UI: Add/Edit stream fields"] --> ObjectFmt["Send/handle {name,type} objects"] ObjectFmt -- "compat" --> Compat["Store original + derive string list for display"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>10 files</summary><table> <tr> <td><strong>stream.rs</strong><dd><code>Rename stream property field to raw type key</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-6f5ee19bbcc9cc7fb699fc1c0f43fc96d42698ae1168a2827b5c65f7948d5dbc">+5/-6</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Introduce DataType and DataField; settings parsing</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+99/-13</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Optimize settings fetch; adapt defined fields helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-386fd4fad1e48ab0a90516cab1947b1888b270b2fe912f30d2255cc38e6c519b">+13/-6</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Use DataField names when building UDS sets</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+6/-1</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Add DataField flows; new/UDS logic and conversions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-d9596182a0c7a46a6a7ff5cee822554cdae05ceec312f86c93b538db9f73bbfd">+270/-18</a></td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Validate field names; resolve defined field types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-41452a7fb29b4634437174fd6aecf6a9050dbfdef41b9077a21d59af3d79e9ed">+126/-7</a> </td> </tr> <tr> <td><strong>AddStream.vue</strong><dd><code>Send defined fields as objects with type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-78b8c84a558e60d942bacffee365633d074042e464be465eb6984b0ee5df82b9">+6/-2</a> </td> </tr> <tr> <td><strong>StreamFieldInputs.vue</strong><dd><code>UI for field data type and UX tweaks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-c8173413939f320f197b0f8790e619da18666f79009345f72b3325e89bbb7566">+90/-27</a> </td> </tr> <tr> <td><strong>schema.vue</strong><dd><code>Support object-based defined fields; sync logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-9196be2ccae4054a00b0b1aa9d1e87b366c10920d6eaf9ef8c6e6293d50bcf46">+119/-25</a></td> </tr> <tr> <td><strong>useStreams.ts</strong><dd><code>Compare/sync object-based defined fields</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-a083369344f022cdf8d4f536719179b1d920369e33e2e26234e70dd36d151a7c">+50/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Map UDS fields by object name not string</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-619eafba0f0509d905b021f4a16659010aca23c62cffd9cd3e6c2bbe204ce200">+1/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
Subhra264andudev
|
c46fe2bf8c |
fix: new sso admin/viewer/editor should be able to enjoy given permissions (#8002)
Currently, openfga returns error when new sso user (admin/editor/viewer) tries to login into openobserve. This pr fixes that. --------- Co-authored-by: udev <uddhav@openobserve.ai> |
||
 Huaijin
|
3eb271620a | refactor: try move remotescan, stream aggregate, topk from rewrite to optimize rule (#7998) | ||
   
|
1d0490875b |
fix: is_multi_stream_search and enable_align_histogram (#8003)
### **User description** Fixes: - https://github.com/openobserve/openobserve/issues/7986 ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add multi-stream search histogram support - Gate histogram alignment via query flag - Remove unused search_type plumbing - Propagate multi-stream flag through services ___ ### Diagram Walkthrough ```mermaid flowchart LR HTTP["HTTP handlers"] GRPC["gRPC handlers"] Utils["HTTP query utils"] SQL["SQL utils: histogram eligibility"] Hist["SQL: histogram query builder"] Svc["Search service"] Cache["Search cache"] Stream["Streaming pipeline"] Utils -- "parse is_multi_stream_search / enable_align_histogram" --> HTTP Utils -- "parse flags" --> GRPC HTTP -- "pass flags" --> Svc GRPC -- "pass flags" --> Svc Svc -- "eligibility check (multi/unified)" --> SQL Svc -- "partitioning uses align flag" --> Svc Svc -- "search() with multi-stream flag" --> Cache Cache -- "eligibility(query, multi)" --> SQL HTTP -- "convert_to_histogram_query(..., multi)" --> Hist Stream -- "do_search(..., multi)" --> Cache Hist -- "build single/multi-stream histogram query" --> Svc ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>15 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Parse multi-stream and align histogram flags</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Remove SearchPartitionRequest.search_type field</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-3</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Add multi-stream histogram eligibility (UNION ALL)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass multi-stream flag into search service</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire flags; histogram conversion uses multi-stream</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Partition multi: add align flag; remove search_type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a> </td> </tr> <tr> <td><strong>search_inspector.rs</strong><dd><code>Pass multi-stream flag in search profile</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Stream API supports multi-stream histogram and flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Propagate multi-stream flag in traces search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Cache search considers multi-stream eligibility</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Align partitions via flag; multi partition API updates</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Build histogram for multi-stream queries (UNION ALL)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Streaming cache handles multi-stream flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Execution path passes multi-stream flag to searches</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Stream request processing threads multi-stream flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add multi-stream histogram query support - Align histogram via query flag - Remove search_type plumbing in partitions - Propagate multi-stream flag end-to-end ___ ### Diagram Walkthrough ```mermaid flowchart LR Http["HTTP handlers"] Grpc["gRPC handlers"] Utils["HTTP query utils"] Front["Web client"] Service["Search service"] Cache["Search cache"] SQLU["SQL utils: histogram eligibility"] Hist["Histogram query builder"] Front -- "is_multi_stream, align flag" --> Http Utils -- "parse is_multi_stream/align" --> Http Http -- "enable_align_histogram" --> Service Http -- "is_multi_stream_search" --> Cache Grpc -- "flags default false" --> Service Cache -- "eligibility(sql, multi)" --> SQLU Http -- "convert_to_histogram_query(..., multi)" --> Hist Service -- "partition(gen interval if enabled)" --> Service ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>19 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Parse multi-stream and align-histogram flags</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Remove search_type from partition request</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-4</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Histogram eligibility with multi-stream UNION ALL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass default multi-stream flag to service</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add flags to HTTP search; align histogram handling</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Partition multi endpoint supports align flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a> </td> </tr> <tr> <td><strong>search_inspector.rs</strong><dd><code>Forward multi-stream flag in inspector search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Streaming search supports multi-stream histogram</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Forward multi-stream flag in traces helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Eligibility check aware of multi-stream</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Align histogram in partition; multi aggregate</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Build single/multi-stream histogram SQL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Propagate multi-stream flag in streaming cache</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Pass multi flag through streaming execution</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire multi-stream flag across streaming path</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a> </td> </tr> <tr> <td><strong>IndexList.vue</strong><dd><code>Use align flag in partition requests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+1/-1</a> </td> </tr> <tr> <td><strong>usePanelDataLoader.ts</strong><dd><code>Toggle align histogram for dashboards</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+3/-1</a> </td> </tr> <tr> <td><strong>useLogs.ts</strong><dd><code>Multi-stream histogram flow and params</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+3/-2</a> </td> </tr> <tr> <td><strong>search.ts</strong><dd><code>API params for is_multi_stream and align flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+7/-5</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: Bhargav <BJP232004@GMAIL.COM> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> Co-authored-by: omkark06 <omkar@zinclabs.io> Co-authored-by: ktx-akshay <akshay.patil@kiara.tech> |
||
|
Hengfei Yang
|
1f15b7ed58 | ci: bump version (#7992) | ||
   
|
748cca3ecb |
feat: visualization enhancement (#7456)
### **User description**
### Which query will be used?
- If query is empty will use histogram query
- Else query will be available:
- If query has group by clause :- will use same query
- Else will use histogram query
___
### **PR Type**
Enhancement, Tests, Bug fix
___
### **Description**
- Add server API to extract result schema
- Implement DataFusion plan visitor for projections
- Integrate visualization flow with schema and histogram
- Numerous UI fixes and UX improvements
___
### Diagram Walkthrough
```mermaid
flowchart LR
DF["DataFusion Plan Visitor (Rust)"]
API["HTTP API /result_schema (Rust)"]
FE["Logs Visualize UI (Vue)"]
Loader["Panel Data Loader (TS)"]
Cache["Histogram Handler & Cache (Rust)"]
DF -- "extract projections/group_by/timeseries" --> API
API -- "JSON ResultSchemaResponse" --> FE
FE -- "build config, set fields, decide histogram" --> Loader
Loader -- "inject searchResponse / is_ui_histogram" --> FE
Cache -- "normalize histogram()" --> DF
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14
files</summary><table>
<tr>
<td><strong>projections.rs</strong><dd><code>Add DataFusion logical plan
visitor to extract schema</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-7a0420d3649da44b80be36d7f1043d6ff81d88395c961266882c3c700f7edff7">+651/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add /result_schema endpoint and
wire parsing/security</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+224/-2</a>
</td>
</tr>
<tr>
<td><strong>cacher.rs</strong><dd><code>Improve histogram() handling and
make public</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-8c387b22b2c0a282600ee41ca702b3db60f5a284d507c0feb7418608ff062c6c">+22/-11</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Export projections
module</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-70bb941c2eef1365e5ead1a59b65c5397457b9197a4e22596e18d17db7eaf922">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>Index.vue</strong><dd><code>Integrate visualization with
schema API and histogram</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-0323c3da69b369843c5d72bf4df5b00ca6b5147192a213baf331659ecf92af62">+564/-154</a></td>
</tr>
<tr>
<td><strong>VisualizeLogsQuery.vue</strong><dd><code>Field list UX,
props for searchResponse/histogram</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-979b01d943d58c5d70faeeac5948b7f02f0b2392a12b84ef07ba7f7393ab0261">+236/-174</a></td>
</tr>
<tr>
<td><strong>SearchBar.vue</strong><dd><code>Visualize toggle gating,
SQL/VRL disabled states</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-81893de4edf186b0556f610acb280c87baa2ffe67d3b48a60a866094c7635baf">+102/-52</a></td>
</tr>
<tr>
<td><strong>FieldList.vue</strong><dd><code>Readonly/visibility options
for logs visualize</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-a071c06e3e26629cb4ae4751450acc39c61fa49b07a1cc4be3711697829668d4">+13/-1</a>
</td>
</tr>
<tr>
<td><strong>TransformSelector.vue</strong><dd><code>Disable transform
controls in visualize mode</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-0037d8dd9cfbacc26309b26e98ae953e7ccbd0891ac2fd01c6314ebd7eed9d2c">+11/-4</a>
</td>
</tr>
<tr>
<td><strong>ChartSelection.vue</strong><dd><code>Allow restricting
selectable chart types</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-384ce178717731e2c85f1312f12a54b42ceabd0b86f7e2befb0e421ef568a6a3">+14/-11</a>
</td>
</tr>
<tr>
<td><strong>PanelSchemaRenderer.vue</strong><dd><code>Accept
searchResponse and is_ui_histogram props</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-e5376e7e77bf3aaf6590c2522d8c2dbffcc95a8006ae0d6dc0040a37825367d6">+13/-0</a>
</td>
</tr>
<tr>
<td><strong>usePanelDataLoader.ts</strong><dd><code>Support injected
searchResponse and ui histogram</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+30/-5</a>
</td>
</tr>
<tr>
<td><strong>visualizationUtils.ts</strong><dd><code>Add SQL alias
validator for visualization</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-c2c391afcaa0b44c33d3f4fa531a4f31f044ae036f0baa182c8c88e83de066c5">+80/-0</a>
</td>
</tr>
<tr>
<td><strong>query.ts</strong><dd><code>Extend response metadata for
visualization</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-12dd5b3bb9741bbeaa348c0efbbd745d0a582899678738159c6927d34a35191d">+2/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Add test for compact delete value
check</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+17/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>auth.rs</strong><dd><code>Allow auth bypass for
result_schema route</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+3/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Register result_schema HTTP
route</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>search.rs</strong><dd><code>Add ResultSchemaResponse
contract</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+7/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>TableRenderer.vue</strong><dd><code>Render null/undefined as
empty string</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-794706ea9da3a70012a0327c0464f9fd16802f78ece26055fe951716c2d73d59">+1/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>sqlUtils.ts</strong><dd><code>Minor formatting
fix</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-dde40e8fcb5c37b06c011ede08922c44a3c025682fbc341aee97ba046b73a0e3">+1/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>11 files</summary><table>
<tr>
<td><strong>playwright.yml</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-7afcd2d8f7b49bda74843f209eefb7b2da45f7e7803bf2e4bd636699b76aa2d3">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>visualise.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-c14ac1db3e4aa86190d7525e8e38c410bcbbae7f46b01890e74937cc11be67e6">+1/-5</a>
</td>
</tr>
<tr>
<td><strong>logsPage.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-8045b343491d8943869c1af74cdafab204dc49c02a70294417e73f47d35e0b59">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>visualize.spec.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-d144f23552ec150db375c89e52c3a53ede9ff3fa5dcd6092694fbe0a308d22ec">+225/-183</a></td>
</tr>
<tr>
<td><strong>useDashboardPanel.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-fbba1700750053b9eaf38ddd30dab84531bc44131c795ebb713811e4930cd7cb">+420/-6</a>
</td>
</tr>
<tr>
<td><strong>useLogs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+151/-19</a></td>
</tr>
<tr>
<td><strong>useNotifications.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-32ff5072df9badfebc0a72c401fe337de3324f76b186a52d202eb6cc2fa71e3a">+25/-0</a>
</td>
</tr>
<tr>
<td><strong>en.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>search.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+43/-0</a>
</td>
</tr>
<tr>
<td><strong>logs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-637857fa918f47ffb02ebfc8a35b124001ef4f65fd180c467f0f153382d2dc76">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>convertDataIntoUnitValue.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-14c6fe442ef231566ec411f9e0262cdd24ec912fc67b9154eec743282ab7211a">+120/-29</a></td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: loaki07 <rustyloaki@gmail.com>
Co-authored-by: Loakesh Indiran <66478092+Loaki07@users.noreply.github.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Co-authored-by: Yashodhan Joshi <yashodhan@openobserve.ai>
|
||
|
Loakesh Indiranandsai nikhil kethe
|
e46f6d5b82 |
feat: custom histogram interval & aligned partitions (#7909)
- `is_ui_histogram` query param for search api, the backend will convert the sql to histogram query and respond accordingly - `search_type` for search partition query, and if search type is `ui` only then we generate partitions with mini partition that is aligned with histogram interval ### **PR Type** Enhancement, Bug fix, Tests, Documentation ___ ### **Description** - Add UI histogram query conversion and flag - Align partitions to histogram with mini - Stream empty metadata+hints consistently - Expose histogram eligibility and interval ___ ### Diagram Walkthrough ```mermaid flowchart LR FE["Web UI logs & services"] -- "is_ui_histogram, searchType" --> HTTP["HTTP search endpoints"] HTTP -- "convert to histogram SQL\n(resolve streams, eligibility)" --> SQL["Histogram SQL converter"] HTTP -- "pass search_type, flags" --> SVC["Search service"] SVC -- "is_eligible, gen interval" --> PART["PartitionGenerator"] PART -- "aligned partitions\n(+mini optional)" --> STREAM["Streaming execution"] STREAM -- "SearchResponse + metadata\n(empty hits emitted)" --> FE SVC -- "Response: histogram_interval, is_histogram_eligible, converted_query" --> FE ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>16 files</summary><table> <tr> <td><strong>partition.rs</strong><dd><code>Histogram-aligned partitions with optional mini support</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-348fa23375d0cea3e5fb039cc6b381d79aa1d5121e6b9792acc048d3fd02e37c">+798/-269</a></td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Always emit responses; accumulate and metadata tweaks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+148/-152</a></td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>HTTP2 stream: UI histogram conversion and interval handling</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+149/-69</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Partition API: histogram eligibility, interval, UI paths</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+71/-5</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Convert original SQL to histogram-safe query</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+140/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>HTTP search: UI histogram conversion and response enrichment</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+38/-3</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Response/request fields for histogram meta and streaming chunks</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+45/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>New error: SearchHistogramNotAvailable</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36">+7/-0</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Multi-stream partition search accepts search_type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+16/-4</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Detect histogram-eligible queries and subqueries</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+27/-0</a> </td> </tr> <tr> <td><strong>http.rs</strong><dd><code>Add query flag parser for is_ui_histogram</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+8/-0</a> </td> </tr> <tr> <td><strong>Index.vue</strong><dd><code>UI: histogram streaming flow, timeouts cleanup</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-0323c3da69b369843c5d72bf4df5b00ca6b5147192a213baf331659ecf92af62">+45/-12</a> </td> </tr> <tr> <td><strong>IndexList.vue</strong><dd><code>Pass searchType=ui to partition API</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+1/-0</a> </td> </tr> <tr> <td><strong>search.ts</strong><dd><code>API: support is_ui_histogram, searchType plumbed</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+7/-2</a> </td> </tr> <tr> <td><strong>useStreamingSearch.ts</strong><dd><code>Include is_ui_histogram in streaming URL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8fce3e25a161cd458e734bd9b1f98aadefe3482da7b6529a5b1f4fa20ec115e2">+1/-0</a> </td> </tr> <tr> <td><strong>usePanelDataLoader.ts</strong><dd><code>Dashboard partition calls pass searchType</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>3 files</summary><table> <tr> <td><strong>cacher.rs</strong><dd><code>Adjust tests for new response fields</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8c387b22b2c0a282600ee41ca702b3db60f5a284d507c0feb7418608ff062c6c">+6/-0</a> </td> </tr> <tr> <td><strong>SearchResult.spec.ts</strong><dd><code>Skip flaky resetPlotChart unit test</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8ff7d14fc3cdac80eeec5550ecd89a77f015b4ebe928da9d4a76a9585d64d788">+1/-1</a> </td> </tr> <tr> <td><strong>useLogs.spec.ts</strong><dd><code>Skip failing query partitions test</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-842307ae8db03aa51891ddde98cfdbbabc712db1e1614c9eee99a59add395f70">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Error handling</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>error_utils.rs</strong><dd><code>Map histogram unavailability to 400 with code</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-d4de63bc9275a19c868f6967de0f7835bd2d7bda74ff55ce97b8fb3f9af54da9">+3/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Miscellaneous</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Export histogram SQL module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-dc0ab53d0ff551cf3cd12027c73d0da5e66680c0bf2d1830778639a33bac3464">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>SearchResult.vue</strong><dd><code>UI: robust empty-histogram handling and reset watcher</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-5e65533aa4754bb6d0074fda18ffc1ecacded3b9ca6bc28a161c3f6b12f20195">+17/-15</a> </td> </tr> <tr> <td><strong>SearchBar.vue</strong><dd><code>Cancel running query before applying saved view</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-81893de4edf186b0556f610acb280c87baa2ffe67d3b48a60a866094c7635baf">+2/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>query.ts</strong><dd><code>Types: histogram interval and eligibility fields</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-12dd5b3bb9741bbeaa348c0efbbd745d0a582899678738159c6927d34a35191d">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>fga.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-e7c99f37f3abbb5e0507d2760bbf67df010d9218704b38dce84c5477acecd22e">+0/-2</a> </td> </tr> <tr> <td><strong>useLogs.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+154/-168</a></td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
Hengfei Yang
|
41d0d4acb6 |
fix: add more logs for flower (#7944)
### **User description** - [x] Add more logs for flower - [x] Optimize parquet footer cache calculate ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Switch query partition strategy to enum - Improve file stats cache size accuracy - Add follower leader work-group and partition logs - Minor consistent-hash refactor for clarity ___ ### Diagram Walkthrough ```mermaid flowchart LR CFG["Config Common.feature_query_partition_strategy (enum)"] STREAM["meta/stream QueryPartitionStrategy FromStr"] FLIGHT["search/cluster/flight uses enum directly"] FOLL["super_cluster/follower adds detailed logs"] HASH["cluster get_node_from_consistent_hash minor reorder"] CACHE["file_statistics_cache counts min/max/sum sizes"] STREAM -- "provides parser" --> CFG CFG -- "typed value" --> FLIGHT FLIGHT -- "strategy choice" --> FOLL CACHE -- "more accurate memory estimate" --> FLIGHT HASH -- "clarify flow" --> FLIGHT ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>6 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Precompute hash before node-set access</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7944/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+1/-1</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Make query partition strategy a parsed enum</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7944/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+7/-3</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Add FromStr and default for strategy enum</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7944/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+8/-5</a> </td> </tr> <tr> <td><strong>flight.rs</strong><dd><code>Use enum-config strategy directly</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7944/files#diff-dbffb1a03b65f6a9163b18c5bb1241ba2966735657be39bbe9f4d0e3260d8c1d">+1/-2</a> </td> </tr> <tr> <td><strong>file_statistics_cache.rs</strong><dd><code>Include min/max/sum sizes in cache estimate</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7944/files#diff-5b3f948f2ad04a6babe72c15365b95a284d0533720419fa5dc793aa379783a7d">+5/-1</a> </td> </tr> <tr> <td><strong>follower.rs</strong><dd><code>Add work-group wait and partition logs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7944/files#diff-59e0cc386582cf8baf201977475fd9a7e479fd8ed000bbaa6746a51ecb28632f">+8/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Miscellaneous</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>lib.rs</strong><dd><code>Remove unused async_drop feature gate</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7944/files#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759">+0/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Loakesh Indiran
|
1ebd0679db | feat: api to delete stream by time period (#7600) | ||
 ktx-vaidehiandAshish Kolhe
|
45bac67dec |
feat: add run ID management and panel ID for dashboard refresh (#7760)
### **PR Type** Enhancement ___ ### **Description** - Introduce DashboardInfo struct with run/panel/tab IDs - Extract dashboard_info from HTTP requests - Propagate dashboard_info in backend search & stats - Frontend: manage runId state and include IDs in requests ___ ### Diagram Walkthrough ```mermaid flowchart LR FE["Frontend components"] HTTPRequest["HTTP Request"] ExtractInfo["get_dashboard_info_from_request"] SearchService["SearchService::search"] UsageStats["report_request_usage_stats"] FE -- "include run/panel/tab IDs" --> HTTPRequest HTTPRequest -- "extract dashboard info" --> ExtractInfo ExtractInfo -- "pass DashboardInfo" --> SearchService SearchService -- "attach stats with info" --> UsageStats ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody></tr></tbody></table> </details> ___ --------- Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com> |
||
|
Huaijin
|
682033437e | chore: refactor sql.rs (#7846) | ||
|
udev
|
a5af8cea73 |
ci: update cargo deny (#7791)
### **User description**
add libbz2 to exceptions
dependency from datafusion
___
### **PR Type**
Enhancement
___
### **Description**
- Add libbz2-rs-sys license exception
- Update cargo-deny CI configuration
___
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>deny.toml</strong><dd><code>Add libbz2-rs-sys
BSD-2-Clause exception</code>
</dd></summary>
<hr>
deny.toml
<ul><li>Added <code>libbz2-rs-sys</code> exception entry<br> <li>
Allowed <code>BSD-2-Clause</code> license</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7791/files#diff-1040309c64844eb1b6b63d8fd67938adbf9461f1b3c61f12cf738f064a02d3de">+1/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
sktrpathi
|
7e08f5ac7a | chore: add UTs, run fmt and clipyy (#7753) | ||
|
Huaijin
|
b41bb71779 | feat: upgrade datafusion to v49 (#7649) | ||
|
sktrpathiandHengfei Yang
|
2da35485f5 |
chore: add UTs, run fmt and clippy (#7720)
### **Description** - Add unit tests for cipher, registry, saved_view, hash utils - Refactor `verify_decode_token` logic with algorithm conversion - Remove async from JWT calls in HTTP handlers - Simplify code in registry, job/cipher, wal modules Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
ByteBaker
|
90fbcca801 | chore: cleanup auth tests | ||
|
ByteBaker
|
b7e3b331b1 | chore: add UTs, run fmt and clipyy | ||
|
Yashodhan Joshi
|
b10b234079 | fix: rbac migration for re_patterns (#7701) | ||
|
sktrpathi
|
f0851c3c7e | chore: add UTs, run fmt and clipyy (#7691) | ||
|
Hengfei Yang
|
65540f8e79 |
perf: improve concurrency for index search (#7695)
### **User description**
Add new env
```
ZO_QUERY_INDEX_THREAD_NUM = 0
```
Default value is `CPU * 4`.
___
### **PR Type**
Enhancement, Tests
___
### **Description**
- Add env var for index search concurrency
- Configure default query_index thread counts
- Use new thread count in search filter
- Refactor file grouping and update tests
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Load ZO_QUERY_INDEX_THREAD_NUM env"] --> B["Set query_index_thread_num default"]
B --> C["filter_file_list_by_tantivy_index uses new count"]
C --> D["Refactor group & repartition logic"]
D --> E["Update grouping tests"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Add query index thread
configuration</code>
</dd></summary>
<hr>
src/config/src/config.rs
<ul><li>Introduce <code>ZO_QUERY_INDEX_THREAD_NUM</code> env var<br>
<li> Set default based on local_mode vs production<br> <li> Apply new
var in <code>check_limit_config</code></ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7695/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+9/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Formatting</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>record_batch_ext.rs</strong><dd><code>Cleanup imports
and whitespace in tests</code>
</dd></summary>
<hr>
src/config/src/utils/record_batch_ext.rs
<ul><li>Reformat import statements for arrow util<br> <li> Remove
trailing whitespace in tests</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7695/files#diff-f84aa2842a164d6e85aa05a718159e87a325cac139c59e561f6f59313c6d7c5d">+7/-4</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>storage.rs</strong><dd><code>Enhance storage search
partitioning logic</code>
</dd></summary>
<hr>
src/service/search/grpc/storage.rs
<ul><li>Use <code>query_index_thread_num</code> for
target_partitions<br> <li> Log target_partitions, group_num,
max_group_len<br> <li> Compute <code>expect_group_elements</code> in
grouping<br> <li> Simplify <code>repartition_sorted_groups</code>
logic<br> <li> Remove unused <code>find_max_group_index</code><br> <li>
Update tests to use <code>expect_group_elements</code></ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7695/files#diff-ab2d867e1e2e3dd8307505c2e130039628a84f8131a9b38cd2e7b4e1c141e094">+27/-62</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Simplify search stream
logging</code>
</dd></summary>
<hr>
src/service/search/streaming/mod.rs
<ul><li>Simplify log message placeholders<br> <li> Remove unused
variables in cache log</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7695/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+1/-5</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
|
41f330af67 |
feat: re_patterns at ingestion (#7323)
### **User description** This PR adds support for adding and applying regexp patterns at ingestion time to redact or drop the values from fields if matched. This is an ent-only feature, and this PR contains the required table migrations, along with nats code for super cluster. This also has some additions for musl build for ent repo. There are basic crud APIs for the pattersn as well as an test api which can be used to test a specific pattern on given input. ___ ### **PR Type** Enhancement ___ ### **Description** - Add HTTP CRUD and test endpoints for regex patterns - Implement DB models, migrations for patterns associations - Integrate pattern manager into ingestion and streams - Spawn watchers and super-cluster queue for patterns ___ ### **Changes diagram** ```mermaid flowchart LR HTTP["CRUD & test HTTP handlers"] DBService["Service DB layer"] DB["re_patterns & associations tables"] Cluster["Cluster Coordinator"] PatternMgr["Pattern Manager"] Ingest["Ingestion pipeline"] Stream["Stream settings update"] HTTP -- "invoke add/update/delete/test" --> DBService DBService -- "persist & notify" --> DB DBService -- "trigger" --> Cluster Cluster -- "watch events" --> PatternMgr Ingest -- "redact at ingestion" --> PatternMgr Stream -- "update associations" --> PatternMgr ``` ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Add HTTP handlers for re_patterns endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-27d37a8738de89c81b540daa978e684caf2cf28803e1208d90cb4d42528ed28b">+459/-0</a> </td> </tr> <tr> <td><strong>re_pattern.rs</strong><dd><code>Implement DB service for re_patterns</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-0cb2bf6d550d07b66db7e325c6d44c6a598f2ac8e315bf9e168f4a1d9d6e745d">+384/-0</a> </td> </tr> <tr> <td><strong>re_pattern_stream_map.rs</strong><dd><code>Add pattern association table ORM</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-13798240832b9110426267b3e0ebb862bd6c2ac03a3462721f5e85b5bfa88f51">+257/-0</a> </td> </tr> <tr> <td><strong>re_pattern.rs</strong><dd><code>Add re_patterns table ORM definitions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-f113ed92667f5e352ac7a0c1c8b5e48ea858992b92be5817c9b38f65843ef4a2">+185/-0</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Integrate pattern associations in streams</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-41452a7fb29b4634437174fd6aecf6a9050dbfdef41b9077a21d59af3d79e9ed">+62/-2</a> </td> </tr> <tr> <td><strong>re_pattern.rs</strong><dd><code>Handle super-cluster queue for patterns</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9e64341dc2d7d50cc5acffd0340b983f37b46a9aa34c86fb5a25978f9eb761dc">+166/-0</a> </td> </tr> <tr> <td><strong>ingest.rs</strong><dd><code>Redact data via pattern manager at ingestion</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-e62118e45be183266d39ffa08491f255017954235ff0f3d6937683e314da406f">+23/-0</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Add pattern associations to Stream metadata</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-6f5ee19bbcc9cc7fb699fc1c0f43fc96d42698ae1168a2827b5c65f7948d5dbc">+4/-1</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Define PatternAssociation and settings wrapper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+14/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Spawn pattern manager watchers on init</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+13/-3</a> </td> </tr> <tr> <td><strong>re_pattern_stream_map.rs</strong><dd><code>Define ORM entity for stream pattern map</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-3c32b9e32bea40b63ed43542aa136cc9000132c4049ed7b6310baabadc55cd79">+38/-0</a> </td> </tr> <tr> <td><strong>re_patterns.rs</strong><dd><code>Define ORM entity for re_patterns table</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9c4ac843741290c1b999f3101fae6f46d71d1893d7740de50c71e2c29fe1636c">+33/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register re_pattern processor in super queue</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-6ed0f1dd36b7216fffde573232a49869f247330ecb03b50f94b2ddd723742e46">+2/-0</a> </td> </tr> <tr> <td><strong>prelude.rs</strong><dd><code>Expose re_patterns entities in prelude</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a818656b9fb01945155f28450270dd2ae20b67c8248c851ffa0f25e859d263ed">+2/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Configuration changes</strong></td><td><details><summary>5 files</summary><table> <tr> <td><strong>m20250612_000001_create_re_pattern_table.rs</strong><dd><code>Create migration for re_patterns table</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-db969ee5510136d5b3c712cdea999c151ab04efe55f1b090670c1eed2b36bcfc">+173/-0</a> </td> </tr> <tr> <td><strong>m20250612_000002_create_re_pattern_stream_map_table.rs</strong><dd><code>Create migration for association stream map table</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-f1ec25aa2150a4fc857fdb17e45b8e43e7c31e884df77e8c91fd07636a6931ca">+168/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register new pattern migrations</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+5/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add re_patterns services to HTTP router</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+7/-1</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Bump DB schema version to support patterns</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Error handling</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>auth.rs</strong><dd><code>Add auth rules for re_patterns test endpoint</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+11/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>23 files</summary><table> <tr> <td><strong>Cross.toml</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-3abca7e5bab5b140e44643dade038033b8a6f6e50771f76c7d48e7bd0b338815">+5/-0</a> </td> </tr> <tr> <td><strong>Dockerfile.musl</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-f47eb18f838cbf757cd4e34ed5357c767b969d4551dcea6a4ab3b64d384a5614">+16/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-c539d7557d1425986d5c09849d8dbca42d1b309c2bc33d49db641d6a783098f6">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-af023301203a27a1ef34f698d5a329ad7bfced65ac65e3829b1c0905a0c74fb2">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+2/-0</a> </td> </tr> <tr> <td><strong>O2AIChat.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a6a125719a7b32b31b53fcfe05c12081d82c139a401699affdda0db8e73887e3">+2/-0</a> </td> </tr> <tr> <td><strong>TemplateList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-7df66364d7d69a26cfa25cbc555d0e090c2f73e51c8386d9b0435bae4412564c">+1/-1</a> </td> </tr> <tr> <td><strong>FullViewContainer.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-c0722446274c88c250042e38b051201b81450cb5da1ee827cd5a4a927424a357">+1/-6</a> </td> </tr> <tr> <td><strong>AssociatedRegexPatterns.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-c6fce5e1c83798991b6ff19effb23c67af59f38fac0b9e56dcab7a09d139f550">+940/-0</a> </td> </tr> <tr> <td><strong>schema.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-9196be2ccae4054a00b0b1aa9d1e87b366c10920d6eaf9ef8c6e6293d50bcf46">+134/-4</a> </td> </tr> <tr> <td><strong>AddRegexPattern.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a3783cfbdd32609bfff45ecfa23542854cad7ae60c3d29fedb832312a4de03dc">+689/-0</a> </td> </tr> <tr> <td><strong>ImportRegexPattern.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-e8e1bcb5a02311504d7d571090eab27a270aaa71cae185ad2703023e2e70a31e">+771/-0</a> </td> </tr> <tr> <td><strong>NoRegexPatterns.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-279b923f20042948c87e907d2453e9d5a4c8c73672b23df951d7d7d823ff3a88">+103/-0</a> </td> </tr> <tr> <td><strong>RegexPatternList.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-524927acd9f5bb07b71bfac489e747470108574e21a6dde63ea7c1f219cea853">+506/-0</a> </td> </tr> <tr> <td><strong>index.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-699c63979f851c48ad440a078e93282fd797a18fc065e5331b2e41122ddb8b69">+27/-2</a> </td> </tr> <tr> <td><strong>useManagementRoutes.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-b71a1d39b1fab0749f0668d5f16382536fbda6e2f0c92c56a972fb5d8b5835f5">+12/-0</a> </td> </tr> <tr> <td><strong>useStreams.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-a083369344f022cdf8d4f536719179b1d920369e33e2e26234e70dd36d151a7c">+42/-1</a> </td> </tr> <tr> <td><strong>MainLayout.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-91e4134141264c4624d4eb4cfa987b33ed31eb6f551e089c0b26f2610dd54362">+3/-1</a> </td> </tr> <tr> <td><strong>en.json</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+24/-1</a> </td> </tr> <tr> <td><strong>JsonPreview.vue</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-3985073134d35ff074742944b114d0905524038b42e6d2ac887fef2c8436fb16">+272/-3</a> </td> </tr> <tr> <td><strong>regex_pattern.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-2a0b0828141d39ab0385e0941314ead1cdd2ec3ec7b3baf5fd0c42f8f010728a">+63/-0</a> </td> </tr> <tr> <td><strong>index.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7323/files#diff-955a15c7ee060f1545e82be22911dd0c14307dee5a70071a394e9fd583bd2c55">+9/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> ___ > <details> <summary> Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details> --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com> |
||
 Ashish Kolhe
|
d7a56813a6 |
feat: add derived stream to stream meta (#7586)
### **PR Type** Enhancement ___ ### **Description** - Add `is_derived` Stream metadata flag - Extract `is_derived` in ingest pipeline - Pass default false to non-derived handlers - Integrate `is_derived` in schema checks ___ ### **Changes diagram** ```mermaid flowchart LR RM["Request Metadata: extract is_derived"] IS["Ingestion Service: ingest(..., is_derived)"] SC["Schema Check: check_for_schema(is_derived)"] DB["DB Schema Merge: include is_derived"] RM -- "passes flag" --> IS IS -- "invokes" --> SC SC -- "updates" --> DB ``` ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>20 files</summary><table> <tr> <td><strong>stream.rs</strong><dd><code>Add `is_derived` field to Stream struct</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-6f5ee19bbcc9cc7fb699fc1c0f43fc96d42698ae1168a2827b5c65f7948d5dbc">+2/-0</a> </td> </tr> <tr> <td><strong>ingest.rs</strong><dd><code>Extract `is_derived` metadata in gRPC ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-27ffe683f384ff836c6aac746240d755bdf17f3c6e0a65d8ec1a1ff82aaf7d5c">+10/-0</a> </td> </tr> <tr> <td><strong>ingest.rs</strong><dd><code>Pass false `is_derived` in HTTP logs ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-24f250254ecfdbe004af9dc0ea23f8a2c0105ae0898690ed3e565089e5d8eef9">+4/-0</a> </td> </tr> <tr> <td><strong>ingest.rs</strong><dd><code>Pass false `is_derived` in HTTP RUM ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-40fa0a7da7845799e5bed864e8ccc8111330a3d76fcc8e50af0b76251d566b45">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add `unwrap_stream_is_derived` and SchemaCache flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-386fd4fad1e48ab0a90516cab1947b1888b270b2fe912f30d2255cc38e6c519b">+9/-0</a> </td> </tr> <tr> <td><strong>handlers.rs</strong><dd><code>Inject `is_derived` metadata into scheduled ingests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-5eca8995a88fcecb6d6b8efd3c7c4496f4a96a4b060fb4ed85593574f7cf49eb">+13/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass false `is_derived` for enrichment tables</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-3e2b6d3b8da1f9e73aca279052a97e683184d68256c23215b18ffee885836bbf">+1/-0</a> </td> </tr> <tr> <td><strong>bulk.rs</strong><dd><code>Track derived streams in bulk ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-2017dbe9076c78a7e7919ed5c8738729123d630038803cef0ce2c356d6fa2c48">+6/-0</a> </td> </tr> <tr> <td><strong>hec.rs</strong><dd><code>Pass false `is_derived` to HEC ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-5434f454898e66d4a982801761a349ac7e6862836e15eefaf643b31d0d2933dd">+1/-1</a> </td> </tr> <tr> <td><strong>ingest.rs</strong><dd><code>Accept `is_derived` in ingest function signature</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-e62118e45be183266d39ffa08491f255017954235ff0f3d6937683e314da406f">+11/-0</a> </td> </tr> <tr> <td><strong>loki.rs</strong><dd><code>Pass false `is_derived` in Loki handler</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-a970c8c4785d4e4d878218615b17eec0cf4b4976121766b002c2b2f76371bfe6">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Forward `is_derived` flag to write_logs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-ce72aea3fa4c44904cbd9ec07bd844bec71b49c9676d667b14056aa5798f96b2">+12/-1</a> </td> </tr> <tr> <td><strong>otlp.rs</strong><dd><code>Accumulate derived_streams in OTLP ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-183498d98b17ddf344cbab4da6b2d9fe89ac81112db51655c142a89023b46ef3">+5/-1</a> </td> </tr> <tr> <td><strong>syslog.rs</strong><dd><code>Accumulate derived_streams in syslog ingestion</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa">+6/-0</a> </td> </tr> <tr> <td><strong>json.rs</strong><dd><code>Pass false `is_derived` in metrics JSON ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-506987e61095a54e47f7f208dad89f9f1bce595fa6bf1ba8a7d746f2b2ff6c4a">+1/-0</a> </td> </tr> <tr> <td><strong>otlp.rs</strong><dd><code>Pass false `is_derived` in metrics OTLP ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-a24391bef75a4678c50724e8b5f305ec0ee8c2c74c68be9d677765de68b5d8dd">+1/-0</a> </td> </tr> <tr> <td><strong>prom.rs</strong><dd><code>Pass false `is_derived` in metrics Prom ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-c4e9e1d49a2bd0fefe67684b3768c4792be63cebcf727d71a48141f5b5957b8c">+1/-0</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Propagate `is_derived` through schema checks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-d9596182a0c7a46a6a7ff5cee822554cdae05ceec312f86c93b538db9f73bbfd">+17/-1</a> </td> </tr> <tr> <td><strong>ingestion.rs</strong><dd><code>Pass false `is_derived` in self-report ingestion</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-3edd5f4ed1eed0f765b7b450ef0fbf587632e6b22302006a5e397049e297057f">+1/-1</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Include `is_derived` in stream response</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-41452a7fb29b4634437174fd6aecf6a9050dbfdef41b9077a21d59af3d79e9ed">+5/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7586/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+1/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> ___ > <details> <summary> Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details> |
||
|
hengfei yang
|
6083d50cd9 | fix: cargo clippy | ||
|
sktrpathi
|
784779348d | chore: add UTs, run fmt and clippy (#7630) | ||
|
ByteBaker
|
bba04bc492 |
chore: correct tokio::spawn usage
There was code of the form `tokio::spawn(async move { foo().await }`
While this works without issues, it's redundant, as `tokio::spawn`
needs a future as the argument. Therefore the new code now
has only `tokio::spawn(foo())`.
+
perform `cargo fmt`
|
||
|
ByteBaker
|
314ae81872 |
chore: Option<Vec<_>> to Vec<_> (part #7498)
performed the change for `StreamSettings::defined_schema_fields` and `Stream::uds_schema` |
||
|
ByteBaker
|
50f2378cb5 | chore: combine serde annotations on struct fields | ||
|
hengfei yang
|
31d5e6de07 | fix: optimize topk | ||
|
Loakesh Indiran
|
b588b40120 |
fix: f32 deserialization nats (#7543) (#7544)
### **User description** fix #7543 ___ ### **PR Type** Bug fix ___ ### **Description** - Add JSON parse error handling in cluster module - Log and skip invalid node payloads - Prevent infinite or NaN CPU usage - Warn and reset invalid CPU usage to zero ___ ### **Changes diagram** ```mermaid flowchart LR A["cluster::watch_node_list"] -- "on JSON parse error" --> B["log error and continue"] C["sysinfo::get_node_metrics"] -- "on CPU infinite or NaN" --> D["log warning and set zero"] ``` ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Error handling</strong></td><td><table> <tr> <td> <details> <summary><strong>mod.rs</strong><dd><code>Add JSON parse error handling</code> </dd></summary> <hr> src/common/infra/cluster/mod.rs <li>Replace <code>unwrap</code> with <code>match</code> on JSON deserialization<br> <li> Log detailed error and payload on parse failure<br> <li> Continue loop instead of panicking on invalid node </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7544/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+12/-1</a> </td> </tr> <tr> <td> <details> <summary><strong>mod.rs</strong><dd><code>Sanitize CPU usage values</code> </dd></summary> <hr> src/config/src/utils/sysinfo/mod.rs <li>Introduce sanity check for CPU usage value<br> <li> Log warning on infinite or NaN CPU usage<br> <li> Reset <code>cpu_usage</code> to zero when invalid </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7544/files#diff-48f8beee2099460df806583ba87a0aff864fe38a4fe74224f06f98c95b7b0e42">+8/-1</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > <details> <summary> Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details> |
||
|
Hengfei Yang
|
c5be35f071 | fix: single node should push down the whole plan (#7528) | ||
|
9444447498 |
fix: Free trial rebase (#6922)
Cherry pick of https://github.com/openobserve/openobserve/pull/6807 after cloud branch was merged into main --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Bhargav <BJP232004@GMAIL.COM> Co-authored-by: Taiming Liu <liutaiming3@gmail.com> Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com> Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com> Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |
||
|
udev
|
c66b273ee6 |
fix: remove ws (#7486)
### **PR Type** Enhancement ___ ### **Description** - Introduce HTTP/2 streaming search pipeline - Implement caching and partitioned query execution - Add delta processing and progress updates - Remove WebSocket support and update configs ___ ### **Changes diagram** ```mermaid flowchart LR A["process_search_stream_request"] -->|check cache| B["handle_cache_responses_and_deltas"] A -->|no cache| C["do_partitioned_search"] B --> D["StreamResponses to client"] C --> D ``` ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8 files</summary><table> <tr> <td><strong>execution.rs</strong><dd><code>Add partitioned streaming search execution</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+705/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Initialize streaming search module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+535/-0</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Implement caching for streaming search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+485/-0</a> </td> </tr> <tr> <td><strong>utils.rs</strong><dd><code>Add utilities for streaming search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-20915cbb55c73e652f71128beb9898a02d9333115d15ecbf50b5df775923e0d4">+140/-0</a> </td> </tr> <tr> <td><strong>sorting.rs</strong><dd><code>Refactor sorting strategy for streaming</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-138cca545e29b77f27da957a19d04bae58ee0c9f84b75509f48c2f0efe5000bb">+46/-48</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Add SearchResultType and SortStrategy enums</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-a36a6683e327641569d8098ee0d4f13e894e1b9d51230dc3569abe42e87ba310">+26/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Move search_stream into streaming module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+2/-2</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Use new streaming request handler</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+3/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Configuration changes</strong></td><td><details><summary>6 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Rename WebSocket to HttpStreaming config</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+15/-30</a> </td> </tr> <tr> <td><strong>router.rs</strong><dd><code>Remove ws route and adjust querier routes</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-5227b263a6c37a040ee1e7482b6903ead2186dd71090ee70bdaf1c3b894487ad">+3/-28</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Drop websocket route handling</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+1/-56</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Use HttpStreaming config chunk size</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+2/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Switch to HttpStreaming in status endpoint</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+1/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Remove ws service endpoint</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>21 files</summary><table> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+0/-6</a> </td> </tr> <tr> <td><strong>auth_tests.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-31</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-2aa59ce4464e7dbabdb9367fc8bb8c1b489c914b8690577d4916870463ab12db">+0/-1</a> </td> </tr> <tr> <td><strong>websocket.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-6ff41099eded8eca4a0e579142d7af4429efb83350917780e0d3ae7dac13847a">+0/-225</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-bab79e8da2d0d9f903cb1a6187d922c6de63d3e4561e8ce633660b664b4e859f">+0/-1</a> </td> </tr> <tr> <td><strong>validator.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+0/-5</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+0/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-dd04c9099b3f291c7822f6e2adcd4197246c67b1610dbd70b09d225eb3b51e40">+0/-283</a> </td> </tr> <tr> <td><strong>session.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-6a1a862158668be26ad1575d7050eb9139bca3e469db56b32641b4c46d0f8144">+0/-641</a> </td> </tr> <tr> <td><strong>connection.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-33c5e604300998724046d4a095c0e45a517b3d68ee279c1f49e5ef827bc6a65e">+0/-631</a> </td> </tr> <tr> <td><strong>error.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-23a4826e6d2d8afd1df90335d54d1bd31c917e8b5689c0aa0edba957f5308ee1">+0/-150</a> </td> </tr> <tr> <td><strong>handler.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-eadc17f110edfa7f870ce3b66b2e0d0bf301624c76f3b7ebeede043b073dd020">+0/-753</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-50311955789888994478438442ded5daf30873021cebf4f55d478d03feba116e">+0/-104</a> </td> </tr> <tr> <td><strong>pool.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-2b0e614010eda77b040ec03f7c9fc5e5feede7e846ae3e21b644e311b1b32956">+0/-118</a> </td> </tr> <tr> <td><strong>session.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-cdd86aa2ec02449ae32de4b331178c6079dbbc48f4220b8feecf2d1f8e878b00">+0/-205</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-1b15316505c31acccb26ba66d920fdb0894567d08d2bf1e3457716d128a67b83">+0/-1</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-b952a89375f2745a0daa273f1bdefc80c22997759a9b17a8fccbc70ae294e8e4">+0/-1667</a></td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-21232733827570717e2ef8988a4c9753e5006665cfe9a46c7b88261e44f4947c">+0/-23</a> </td> </tr> <tr> <td><strong>search.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-13476631e2e604b75e2dfdd8fb7b7a91c46dcd85faa50813afa0b87583bcdb26">+0/-1271</a></td> </tr> <tr> <td><strong>utils.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-36caf9ef22ae6e1a217d51e023403ce24a1bd3d10b1bc6d5b86c982d6269bdbe">+0/-333</a> </td> </tr> <tr> <td><strong>values.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7486/files#diff-d5c53f3f28432f096cb4788fa89c0e9bc8551aefe75c4a62be084a08d9025f97">+0/-326</a> </td> </tr> </table></details></td></tr></tr></tbody></table> ___ > <details> <summary> Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details> |
||
|
Ashish Kolhe
|
40aa8edd31 |
chore: clippy fixes (#7513)
### **PR Type**
Enhancement
___
### **Description**
- Replace `{}` with named `{var}` in log messages
- Standardize log formatting across modules
- Improve readability and consistency
- No functional changes
___
### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Formatting</strong></td><td><details><summary>20
files</summary><table>
<tr>
<td><strong>handler.rs</strong><dd><code>Switch log macros to named
interpolation</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-eadc17f110edfa7f870ce3b66b2e0d0bf301624c76f3b7ebeede043b073dd020">+32/-67</a>
</td>
</tr>
<tr>
<td><strong>mysql.rs</strong><dd><code>Update MySQL log formatting
placeholders</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-06d02c738c6a4dc22ff022c16befef8d46aece6cfaa88bb4a5d3d5f92c2e50a1">+31/-36</a>
</td>
</tr>
<tr>
<td><strong>nats.rs</strong><dd><code>Refactor NATS log interpolation
syntax</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-2d6809ea34ea04477afb9a6f8287c93cef488b45ef8f0ebeec2241571b93994c">+16/-32</a>
</td>
</tr>
<tr>
<td><strong>schema.rs</strong><dd><code>Standardize migration log
messages</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-503e5382af7a29ec9c2819e32d468f6bbf9499da91cfdf43663def51c77d422b">+19/-34</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Refine OTLP trace handler
logging</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+17/-36</a>
</td>
</tr>
<tr>
<td><strong>parquet.rs</strong><dd><code>Update parquet job log
interpolation</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-27f710981cfe1b938aeeb53abaed05961596c32e2edf3b8a2a59d3e72b2f542b">+18/-49</a>
</td>
</tr>
<tr>
<td><strong>search.rs</strong><dd><code>Refactor search event log
syntax</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-13476631e2e604b75e2dfdd8fb7b7a91c46dcd85faa50813afa0b87583bcdb26">+15/-47</a>
</td>
</tr>
<tr>
<td><strong>merge.rs</strong><dd><code>Improve compactor merge log
formatting</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-0ab016dbd5129c9c285c9c459bf78928147bce97dc92a03aa754da3da71168d7">+14/-54</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Standardize compactor module
logs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-dce43be4d49b0bf99b61099d5b348bc539433d6e9b1ce6b7025d65e907f00b95">+10/-35</a>
</td>
</tr>
<tr>
<td><strong>disk.rs</strong><dd><code>Refactor disk cache log
messages</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-af63f2c5d85313779df0204c9689e50a6c3916e3aadc24e54e23ec640a584fcb">+9/-9</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Update request WS handler
logs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-dd04c9099b3f291c7822f6e2adcd4197246c67b1610dbd70b09d225eb3b51e40">+17/-27</a>
</td>
</tr>
<tr>
<td><strong>postgres.rs</strong><dd><code>Standardize Postgres log
interpolation</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-bb24a30c9c3744eb31fb49e4271a9651d1f9a94a87455ac0074364b879af9651">+17/-26</a>
</td>
</tr>
<tr>
<td><strong>mysql.rs</strong><dd><code>Refine scheduler MySQL
logs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a275fb17137e6e7cd3dbc6432164136b20d1c2d4757eadaadd0b4a983473865a">+17/-23</a>
</td>
</tr>
<tr>
<td><strong>wal.rs</strong><dd><code>Update WAL redo logging
syntax</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8ba57e3a90cce57ceb88ea5215024c6d54473732bebdc7bf33847462379ec03e">+13/-22</a>
</td>
</tr>
<tr>
<td><strong>users.rs</strong><dd><code>Refactor user service log
messages</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+13/-18</a>
</td>
</tr>
<tr>
<td><strong>jwt.rs</strong><dd><code>Improve JWT auth log
interpolation</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+9/-24</a>
</td>
</tr>
<tr>
<td><strong>batch_execution.rs</strong><dd><code>Standardize pipeline
debug logs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-3f6824a6eddaf1603e5cb0f1a4311ac6a45e1ffb967266ccc1f0bba944625e69">+16/-34</a>
</td>
</tr>
<tr>
<td><strong>alert.rs</strong><dd><code>Refine alert scheduler log
syntax</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-58f77a5eb00c01f123b13173c99cf9499af7458115e50811d07def0cf6e2d3a7">+12/-13</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Update OFGA migration
logging</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-1b8694fad31c8249afeb419d92a9facbd170289db2c89100552883df876bb275">+5/-10</a>
</td>
</tr>
<tr>
<td><strong>schema.rs</strong><dd><code>Standardize schema service
logs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4696a44abde2df4ac48fed716f930bfdcc01759bdfcfd8d802821c6dd857bff9">+12/-14</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>101 files</summary><table>
<tr>
<td><strong>cli.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>etcd.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-37c2c2803d1c12ebb4ef9299bc79c9e6740fc5f19f7f4d84e7cfbb610b9561a9">+4/-4</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+6/-6</a>
</td>
</tr>
<tr>
<td><strong>nats.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-901773980aa255942e7865b977fe63811f31511ac40bc512d6bbb14dbb54bece">+8/-8</a>
</td>
</tr>
<tr>
<td><strong>config.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+12/-18</a>
</td>
</tr>
<tr>
<td><strong>alert_folders.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6ed0a38075313ced0916343ee6e6b86ab639f8ed59e4968df395aa30d72bd300">+10/-10</a>
</td>
</tr>
<tr>
<td><strong>wal.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4928ae81597b28facdc3644f786ac14c60b0a406e1c7b8eac7232fbafbcbb6e3">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>telemetry.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-3ff026b46d9431e0657828a3bde4ab118fec006b353c2abc80be4b74633c98b4">+3/-3</a>
</td>
</tr>
<tr>
<td><strong>auth.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>ider.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6ea42b8eab8ba0708f93f9a8b44056d9a9e6ac42517ffba12f81420d6b0bd2b0">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>search.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+2/-3</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-2ba3bceb0d80c8008da06e45374c1f744441375e2c28b55e6a55a1be3337e79b">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-9a4dddd253d61f35b8d61d372477e24a4523f419aad2a63b226b19a584b8624f">+4/-7</a>
</td>
</tr>
<tr>
<td><strong>event.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-689769712cbd6bd02beba078c1e1dfc188e2be39ab19a0e232b8bd12bc7f1e78">+6/-6</a>
</td>
</tr>
<tr>
<td><strong>traces.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-38e83bff9d1b99d37475a02c28cf9b2b7fd4d84d366a1eade3928a5d10b471f8">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>token.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-3339b4a964c121766ef87fcec0070f27a2b7085cf81281b53154e53d33cb28cf">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>validator.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+6/-7</a>
</td>
</tr>
<tr>
<td><strong>ai.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-c7e88d14798788e6186cadbb1cef3386dad2e585af6c36fd7b9cab3dbef2e46b">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>reports.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-00435e0d9301342b51cb57ae313f1413e7ed973642e0badf3f3959125d0677d1">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>timed_annotations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-b6a4ef9157a0dbf25b875111f7cdb230d3d0f2d7a430d335de6c6af82f95b365">+5/-5</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-72fb8ce13ff14a37ab38d48e5b452205b30c8a8b544badf42579fec4fe5f90fd">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>ingest.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-24f250254ecfdbe004af9dc0ea23f8a2c0105ae0898690ed3e565089e5d8eef9">+4/-10</a>
</td>
</tr>
<tr>
<td><strong>loki.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-34691872632b619be8aadf255104f0e004fcf701dba98f88e231de5966eed099">+4/-8</a>
</td>
</tr>
<tr>
<td><strong>org.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-b47c0c1a8a78524196176430268eecdbfaf6f46d6b95873d3cbe70cfd67182b0">+2/-10</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-0ad5729a1fbce00ff53076728c51822400c13af9918e1d8554f41be56d060fa0">+6/-6</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8ce79cea5617c3a661e5b64058ad3846abf8652bb44fbb88dfb9117ffcc577fd">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+9/-9</a>
</td>
</tr>
<tr>
<td><strong>multi_streams.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+5/-5</a>
</td>
</tr>
<tr>
<td><strong>search_inspector.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>search_job.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d848b311eae318d198558c29a007c8973afb5f8867a31ec056f4e037dcd1b854">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-619eafba0f0509d905b021f4a16659010aca23c62cffd9cd3e6c2bbe204ce200">+2/-6</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-56897e42f9436a640bc70fddc25686485ba8ae1223635b32d252a0333309f748">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>service_accounts.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8d58793d63492cb2b4ec77288a91b0cf995a56cac7ea6e0faa9f006aa105b368">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>session.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6a1a862158668be26ad1575d7050eb9139bca3e469db56b32641b4c46d0f8144">+6/-16</a>
</td>
</tr>
<tr>
<td><strong>slow_log.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-45522a89e42bfd2029972622703bbf58b40d16fc7a7c4346ac270082728661ec">+1/-7</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-66207bd699343342402f88cdd331f4c549783754e4921d0c8d43e026e2d8e575">+11/-11</a>
</td>
</tr>
<tr>
<td><strong>delete.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-9f484d99b002885e92d01b21c11686f8e99077cf5dd422b673c6877db15f2f8d">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>memory.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-ff00d697a9b223ef2858c5e41111a8918e7cc347dc22891c02bda6982682eee1">+4/-7</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f8e74805561fa288a434e6e5ca8fb2c9068fb7f54a5a21f0a9bbba5f73249a5f">+2/-8</a>
</td>
</tr>
<tr>
<td><strong>alerts.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-9a2bb454050f39797fc752caa4a72da28f398c76f2a7f54f74d1c293d3b27698">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>etcd.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-33bb9257c1a1b9162c8679191b8d8b75c0b7395f72291ce73d77247ff924a66e">+6/-9</a>
</td>
</tr>
<tr>
<td><strong>mysql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-412dc407af348f4b54b7d98de6a928876b0333ea501792498a0568850f83d7b7">+10/-10</a>
</td>
</tr>
<tr>
<td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-1554e69f6d6c29812af88f0dcf093240b24ff85f9e292322c71cb19b0dda808d">+13/-13</a>
</td>
</tr>
<tr>
<td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-85f17b22460902614e5c28df8966ea3e7073325bd8dd43fa396ed9268b6b4b06">+5/-11</a>
</td>
</tr>
<tr>
<td><strong>grpc.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-211c5cb8fa12eac14c3637a43d74cd114034b8a6f6f8c1ead51e56efb54141cd">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-85ff00530831fdaa4e13c05cb649f2e20479cce04a4b65ef78d131cdb89b6c71">+4/-9</a>
</td>
</tr>
<tr>
<td><strong>mysql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-836ef77752dcfeba5e1161c2c42e92ec651d9a9131f12d48fb390fe3d861ab53">+8/-8</a>
</td>
</tr>
<tr>
<td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-122e67af3f562870a16eff139f68d5c126841cabe5ebefe2c729530d676e883f">+8/-11</a>
</td>
</tr>
<tr>
<td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-e81d5c9c0ac9af5dc99c66eca0cfa1523f34063a33f75f6b7c1e4fc0d1802ed6">+8/-8</a>
</td>
</tr>
<tr>
<td><strong>postgres.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a281dfd3124defa76bf1100c7e4993def0cb3049cf1ad0b1c513ea0afe522d08">+9/-9</a>
</td>
</tr>
<tr>
<td><strong>sqlite.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-07751d2c8c454417335fa9dd06507099d8b1eaea9b04af99857db6d9c619e69f">+5/-5</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-386fd4fad1e48ab0a90516cab1947b1888b270b2fe912f30d2255cc38e6c519b">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>local.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8e49c3ffac3d43c06780a18937b7ecd7943a8872ec81dca39d4497d9f94e33a7">+4/-7</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-71f24f3312c19e01c4eedcd6a720893e01e64d176d2afbe1e7c6171a23059716">+3/-3</a>
</td>
</tr>
<tr>
<td><strong>remote.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-fea1d32f99821f1913607362236bd9e5b65cb2a5f8ce2c11a9a4956ac646b3d1">+6/-13</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-238b8087501b1743c3c4608469596d431869f021e9bb30e9a058d1294229ee67">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>m20241227_000100_populate_organizations_table.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-53abf79ce8f9c17107749f25c5a544591903422b918e322643068a38eaee6050">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>m20241227_000400_populate_users_table.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-fd66003231079d3fcf3681ab7a57d64b61dea22e55a3c22ebe5527d095a52b5e">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>m20250320_000001_remove_alert_name_unique_constraint.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f4094b2945a877db5709045deafdd2883ff31fd671d0782da058af40e522d559">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>m20250611_000002_populate_reports_table.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8247af6d6f8ccb6a5a51daff4cb1e352585be2d1c785d86ae5ebfea1dbc1099c">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>m20250611_000003_populate_reports_scheduled_jobs.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6567d3d0361c1a184ba7877795a8c3c4f73ea9807b3d4c13efe621449dd3f80f">+3/-6</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-af023301203a27a1ef34f698d5a329ad7bfced65ac65e3829b1c0905a0c74fb2">+1/-2</a>
</td>
</tr>
<tr>
<td><strong>org_users.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-8e62ecba19308fdaaecfdb3d1f23021c926e244a510d9452dfbd71ac8f35ee9f">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>organizations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4d87f9fe379a5d161a3e7c3c77f7f320be738e6a70c785aa5ce93cb0830c0147">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>ratelimit.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-231f9b18e0d61409b6e3348196dd4d4456847ebb25e1822eec52e7e09b00b5ef">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>queries.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-144c4572241c5b3f13abe628832674ad8cebb72e51acb1d1b14fae2a8307208a">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>search_job_partitions.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a72ad2e06cfec317bd6901681b9b7c8a27b4b5736e6340f58d711296222ffee1">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>short_urls.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-5e9b140c2fe41290633080bed471cd7473165119bc1fe84bd69853fec91d7a3a">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>timed_annotations.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4bc910d02f5ddc598e394d06172debe456c7db655cc16fef53eab55005880378">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>users.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a8183f71611dc4c9329231bb5083ceb43a19f43011f86be5c5c9e47240d2073f">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>lib.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-603267e2f712eeb4bc46fdff981f1fad1f3a9d3b989a7db4b3c0984fc5b1ed43">+4/-4</a>
</td>
</tr>
<tr>
<td><strong>partition.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-441e26dda30cadfa3cb1efbd9569aec96af8fdc779e4916814fb5f9477db3f6c">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>writer.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f">+7/-10</a>
</td>
</tr>
<tr>
<td><strong>alert_manager.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-7e30858abad652a860186eb047347305488f52356112b1495dd53ce740316d7c">+5/-8</a>
</td>
</tr>
<tr>
<td><strong>file_downloader.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-dcd9d92e9718a63ee7341b7239f03d9f55746a0f848537e4982661e9219ea05c">+7/-25</a>
</td>
</tr>
<tr>
<td><strong>broadcast.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-930f8671c8be6478fe4739b46af3ff38994f14a558a2003cc136060a7c790687">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-30f42c3685037c219f0033617f032b23a7f74624ba4ae6c4e3687af401304591">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>metrics.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-ca98d90d1293682c98c33a777a653cf3c1466f6297969ae94a359bb81abca4b1">+3/-3</a>
</td>
</tr>
<tr>
<td><strong>mmdb_downloader.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d36dee2ad5f79654acc9d5d5c3b95a552a9b397f97c25c2e23a19195ddc7fe86">+4/-12</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>promql.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-15414a41d64eb87bc5c0cd00a07213b3f99c31b346e8e2c36178f7276a1f058c">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>stats.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-f3ca67d9d2c68beeafd6e5de947dbc9a648ac2d2699c5dc87458c369887d09b9">+2/-5</a>
</td>
</tr>
<tr>
<td><strong>syslog_server.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-cb7610142582b44885f673583ab16262c3b9ca6de1f59a9def65e503360665a0">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>main.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+8/-23</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-eb855be0fa22d1a540379f6142cc4a473d49f89c3e0fe16cfd5078d5a69f9a3b">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>pipeline_func.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-701a2afe5253e4ef729a1b93372e69d79ada9cf9d1f0f5a8556bc4841f46f301">+1/-4</a>
</td>
</tr>
<tr>
<td><strong>report.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-a8d5d0e0fc56e67b38a20a46b91cc2e6bf76ade360a37ad29fce51cc4661a9a6">+1/-2</a>
</td>
</tr>
<tr>
<td><strong>server.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-804074c17dbf0e9757a57ade0cd81a00518a14204762cbbe3fa44bfb43936dc8">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>connection.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-33c5e604300998724046d4a095c0e45a517b3d68ee279c1f49e5ef827bc6a65e">+5/-17</a>
</td>
</tr>
<tr>
<td><strong>resource_extractor.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6bf05d4ebced31bf58505b67b2362abd833c00422743650218998ed6bc4b45af">+2/-7</a>
</td>
</tr>
<tr>
<td><strong>alert.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-574be3504bf4d855c65fd3d7a10797a970237ff552edac4962b474b3983e094e">+1/-4</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-ba9b65130898444f7815fd6068bdb0dc590612f74cc893f79d91b0afc44cc851">+3/-7</a>
</td>
</tr>
<tr>
<td><strong>handlers.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-5eca8995a88fcecb6d6b8efd3c7c4496f4a96a4b060fb4ed85593574f7cf49eb">+8/-17</a>
</td>
</tr>
<tr>
<td><strong>worker.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-3d37991258d3f26b16ad99bba59b63adfa614bfbc4866c4088544ca9308ae49d">+10/-22</a>
</td>
</tr>
<tr>
<td><strong>deleted.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-4900b3bce3c590ac846c7f12e82e1d1cf7c2de3de9fc385fdbe53deb8a8e50ab">+4/-4</a>
</td>
</tr>
<tr>
<td><strong>flatten.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-93a66ecfe3a7fdfe5509dda69e4d444f02b18c841c34ab3078f1fb314af8e3ca">+1/-5</a>
</td>
</tr>
<tr>
<td><strong>retention.rs</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-6abd048f5ef4e7163cffbb51cb4bee10c07fb88b871469d5b3724f7cd04743dc">+9/-29</a>
</td>
</tr>
<tr>
<td><strong>Additional files not shown</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7513/files#diff-2f328e4cd8dbe3ad193e49d92bcf045f47a6b72b1e9487d366f6b8288589b4ca"></a></td>
</tr>
</table></details></td></tr></tr></tbody></table>
___
> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>
|
||
|
Yashodhan JoshiandYashodhan Joshi
|
babe3b08fb |
feat: support path based value extraction in cipher functions (#7483)
### **User description** cherry-pick of https://github.com/openobserve/openobserve/pull/7482 for main ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Add JSON path extraction utility - Extend cipher UDFs with path argument - Rewrite optimizer to handle paths - Add tests for path-based ciphers ___ ### **Changes diagram** ```mermaid flowchart LR J["get_value_from_path"] --> U["cipher UDFs"] U --> O["optimizer rewrite"] O --> T["path-based cipher tests"] ``` ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>auth_tests.rs</strong><dd><code>Remove `batch_size` default in tests</code> </dd></summary> <hr> src/common/utils/auth_tests.rs - Remove deprecated `batch_size` default </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-1</a> </td> </tr> </table></td></tr><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>json.rs</strong><dd><code>Implement JSON path extraction utility</code> </dd></summary> <hr> src/config/src/utils/json.rs <li>Import <code>itertools</code> for flattening<br> <li> Add <code>get_value_from_path</code> implementation<br> <li> Handle nested JSON and wildcard paths<br> <li> Add unit tests for JSON path extraction </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-f638bd6d218ad53a671681c2fa8e420cfbf8369e6e93269c9559e9711338d52a">+131/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>cipher.rs</strong><dd><code>Support path argument in optimizer cipher rewrite</code> </dd></summary> <hr> src/service/search/datafusion/optimizer/cipher.rs <li>Require only two args for rewrite checks<br> <li> Update sanity checks to allow optional path<br> <li> Preserve or default path in <code>CipherKeyRewrite</code><br> <li> Add tests for path-based cipher calls </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-1dfdd6163a17d67d426ab189446fa4c43551094aea50ffd82d334abd3ffe4f81">+40/-11</a> </td> </tr> <tr> <td> <details> <summary><strong>cipher_udf.rs</strong><dd><code>Extend cipher UDFs with JSON path support</code> </dd></summary> <hr> src/service/search/datafusion/udf/cipher_udf.rs <li>Extend UDFs to accept three parameters<br> <li> Validate third <code>path</code> argument in UDFs<br> <li> Parse JSON and extract value at <code>path</code><br> <li> Update encryption/decryption logic for arrays </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-aca30595e1e899bf83b213e84d91063cc621a8972ce22e67bcf4705af817188f">+175/-19</a></td> </tr> <tr> <td> <details> <summary><strong>sql.rs</strong><dd><code>Require three arguments in SQL cipher parser</code> </dd></summary> <hr> src/service/search/sql.rs <li>Update cipher function argument count to three<br> <li> Adjust SQL parser validation for paths </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-d9ccf393c53794b0ef2816eef12742c26e23d2788cde27780af7570662aea446">+1/-1</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > <details> <summary> Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details> --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> |
||
|
ByteBaker
|
9f40388f45 |
chore: add unit tests (#7487)
### **PR Type** Tests, Enhancement ___ ### **Description** - Add extensive unit tests across common meta modules - Refactor LokiError mapping and tag filtering logic - Simplify record_batch usage and string interpolation - Fix histogram interval functions and cache invalidation |
||
|
Hengfei Yang
|
cd3bf2f160 | fix: improve stats from file_list_dump (#7471) | ||
|
ByteBaker
|
25ce21cc3e | chore: clippy delints (#7446) | ||
|
ByteBaker
|
bdb4338903 |
chore: add UT for CI (#7464)
### **PR Type** Tests ___ ### **Description** - Add unit tests for email validation - Add unit tests for OFGA unsupported check - Add tests for OFGA format conversion - Add tests to validate if a user is part of organization - Remove unused invite token stub ___ ### **Changes diagram** ```mermaid flowchart LR validTest["Test valid emails"] -- "calls" --> isValidEmail["is_valid_email"] ofgaTest["Test OFGA unsupported"] -- "calls" --> isOfgaUnsupported["is_ofga_unsupported"] formatTest["Test OFGA format"] -- "calls" --> intoFormat["into_ofga_supported_format"] userOrgTest["Test user from org"] -- "calls" --> isUserFromOrg["is_user_from_org"] ``` ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>auth.rs</strong><dd><code>Add tests and cleanup in auth utils</code> </dd></summary> <hr> src/common/utils/auth.rs <li>Removed stub <code>generate_invite_token</code><br> <li> Added <code>test_valid_emails</code> unit tests<br> <li> Added <code>test_is_ofga_unsupported</code> unit tests<br> <li> Added <code>test_into_ofga_supported_format</code> unit tests<br> <li> Updated <code>println!</code> macros to use inline variables </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7464/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+36/-9</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > <details> <summary> Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details> |